Why is file transfer speed decreased when copying multiple smaller files?
up vote
3
down vote
favorite
Let's say I'm copying a portable apps directory from PC to an external HDD.
An application.exe file is around 4 megabytes in size, but most other files are only tens of kilobytes in size, and most of those files take longer to copy/move than that bigger file having 4 megabytes.
Why is this?
file-transfer speed
add a comment |
up vote
3
down vote
favorite
Let's say I'm copying a portable apps directory from PC to an external HDD.
An application.exe file is around 4 megabytes in size, but most other files are only tens of kilobytes in size, and most of those files take longer to copy/move than that bigger file having 4 megabytes.
Why is this?
file-transfer speed
add a comment |
up vote
3
down vote
favorite
up vote
3
down vote
favorite
Let's say I'm copying a portable apps directory from PC to an external HDD.
An application.exe file is around 4 megabytes in size, but most other files are only tens of kilobytes in size, and most of those files take longer to copy/move than that bigger file having 4 megabytes.
Why is this?
file-transfer speed
Let's say I'm copying a portable apps directory from PC to an external HDD.
An application.exe file is around 4 megabytes in size, but most other files are only tens of kilobytes in size, and most of those files take longer to copy/move than that bigger file having 4 megabytes.
Why is this?
file-transfer speed
file-transfer speed
asked Dec 27 '14 at 1:06
Ivan Ivković
1331113
1331113
add a comment |
add a comment |
4 Answers
4
active
oldest
votes
up vote
6
down vote
When transferring data, if you move one large file, you only have to write the directory entry once and then stream the data and write more or less continuously. When you have many files, you write the directory info, move to the file area and write the data, then go back and write the directory entry for the next file and then move back to the data area and write the data and back and forth. The overhead of moving back and forth adds up.
add a comment |
up vote
5
down vote
If you transfer a file, theres an open+read+close at the source, and an open+write+close at the destination, with all the associated attributes (ctime/atime/mtime etc) being created at the destination. Fragmentation (higher seek times) will also affect the datarate. Congestion of e.g. a sata-channel may also be an issue (use a separate ports for each disk, buy a sata card if you run out). Bigger files have longer consecutive reads or writes, and will transfer faster. Im sure there's much more to it. Does this help you?
add a comment |
up vote
2
down vote
That's the way windows works, and that's the part HDD are bad at. Usually SSDs are good at copying small files. You can check out benchmarking of HDDs and SSDs about file transfer speeds. But you have to know that other things are in place, like windows is reading first the files, then your antivirus checks them, and then they get written to the new destination folder. Not to mention that when you cut a file, the deletion process is added. So, more files, more "real" time added to copy a bigger number of files.
add a comment |
up vote
-1
down vote
Think of it like this,
The computer is assigned a task to take 10 books (10 files) from point A to point B, If they are individual then the computer needs to get one take it to point B and then come back to point A to pick up the next book and so on. Whereas if it needed to take one big book that was the equivalent weight of the 10 books combined it would take longer to move the book but it will be a quicker process because the computer only makes one trip.
1
Nice metaphor, but I can't see why the two processes are similar.
– Máté Juhász
Oct 5 at 4:01
add a comment |
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
6
down vote
When transferring data, if you move one large file, you only have to write the directory entry once and then stream the data and write more or less continuously. When you have many files, you write the directory info, move to the file area and write the data, then go back and write the directory entry for the next file and then move back to the data area and write the data and back and forth. The overhead of moving back and forth adds up.
add a comment |
up vote
6
down vote
When transferring data, if you move one large file, you only have to write the directory entry once and then stream the data and write more or less continuously. When you have many files, you write the directory info, move to the file area and write the data, then go back and write the directory entry for the next file and then move back to the data area and write the data and back and forth. The overhead of moving back and forth adds up.
add a comment |
up vote
6
down vote
up vote
6
down vote
When transferring data, if you move one large file, you only have to write the directory entry once and then stream the data and write more or less continuously. When you have many files, you write the directory info, move to the file area and write the data, then go back and write the directory entry for the next file and then move back to the data area and write the data and back and forth. The overhead of moving back and forth adds up.
When transferring data, if you move one large file, you only have to write the directory entry once and then stream the data and write more or less continuously. When you have many files, you write the directory info, move to the file area and write the data, then go back and write the directory entry for the next file and then move back to the data area and write the data and back and forth. The overhead of moving back and forth adds up.
answered Dec 27 '14 at 1:21
Blackbeagle
6,3611623
6,3611623
add a comment |
add a comment |
up vote
5
down vote
If you transfer a file, theres an open+read+close at the source, and an open+write+close at the destination, with all the associated attributes (ctime/atime/mtime etc) being created at the destination. Fragmentation (higher seek times) will also affect the datarate. Congestion of e.g. a sata-channel may also be an issue (use a separate ports for each disk, buy a sata card if you run out). Bigger files have longer consecutive reads or writes, and will transfer faster. Im sure there's much more to it. Does this help you?
add a comment |
up vote
5
down vote
If you transfer a file, theres an open+read+close at the source, and an open+write+close at the destination, with all the associated attributes (ctime/atime/mtime etc) being created at the destination. Fragmentation (higher seek times) will also affect the datarate. Congestion of e.g. a sata-channel may also be an issue (use a separate ports for each disk, buy a sata card if you run out). Bigger files have longer consecutive reads or writes, and will transfer faster. Im sure there's much more to it. Does this help you?
add a comment |
up vote
5
down vote
up vote
5
down vote
If you transfer a file, theres an open+read+close at the source, and an open+write+close at the destination, with all the associated attributes (ctime/atime/mtime etc) being created at the destination. Fragmentation (higher seek times) will also affect the datarate. Congestion of e.g. a sata-channel may also be an issue (use a separate ports for each disk, buy a sata card if you run out). Bigger files have longer consecutive reads or writes, and will transfer faster. Im sure there's much more to it. Does this help you?
If you transfer a file, theres an open+read+close at the source, and an open+write+close at the destination, with all the associated attributes (ctime/atime/mtime etc) being created at the destination. Fragmentation (higher seek times) will also affect the datarate. Congestion of e.g. a sata-channel may also be an issue (use a separate ports for each disk, buy a sata card if you run out). Bigger files have longer consecutive reads or writes, and will transfer faster. Im sure there's much more to it. Does this help you?
answered Dec 27 '14 at 1:19
user400344
31614
31614
add a comment |
add a comment |
up vote
2
down vote
That's the way windows works, and that's the part HDD are bad at. Usually SSDs are good at copying small files. You can check out benchmarking of HDDs and SSDs about file transfer speeds. But you have to know that other things are in place, like windows is reading first the files, then your antivirus checks them, and then they get written to the new destination folder. Not to mention that when you cut a file, the deletion process is added. So, more files, more "real" time added to copy a bigger number of files.
add a comment |
up vote
2
down vote
That's the way windows works, and that's the part HDD are bad at. Usually SSDs are good at copying small files. You can check out benchmarking of HDDs and SSDs about file transfer speeds. But you have to know that other things are in place, like windows is reading first the files, then your antivirus checks them, and then they get written to the new destination folder. Not to mention that when you cut a file, the deletion process is added. So, more files, more "real" time added to copy a bigger number of files.
add a comment |
up vote
2
down vote
up vote
2
down vote
That's the way windows works, and that's the part HDD are bad at. Usually SSDs are good at copying small files. You can check out benchmarking of HDDs and SSDs about file transfer speeds. But you have to know that other things are in place, like windows is reading first the files, then your antivirus checks them, and then they get written to the new destination folder. Not to mention that when you cut a file, the deletion process is added. So, more files, more "real" time added to copy a bigger number of files.
That's the way windows works, and that's the part HDD are bad at. Usually SSDs are good at copying small files. You can check out benchmarking of HDDs and SSDs about file transfer speeds. But you have to know that other things are in place, like windows is reading first the files, then your antivirus checks them, and then they get written to the new destination folder. Not to mention that when you cut a file, the deletion process is added. So, more files, more "real" time added to copy a bigger number of files.
answered Dec 27 '14 at 1:27
Erlis D.
708413
708413
add a comment |
add a comment |
up vote
-1
down vote
Think of it like this,
The computer is assigned a task to take 10 books (10 files) from point A to point B, If they are individual then the computer needs to get one take it to point B and then come back to point A to pick up the next book and so on. Whereas if it needed to take one big book that was the equivalent weight of the 10 books combined it would take longer to move the book but it will be a quicker process because the computer only makes one trip.
1
Nice metaphor, but I can't see why the two processes are similar.
– Máté Juhász
Oct 5 at 4:01
add a comment |
up vote
-1
down vote
Think of it like this,
The computer is assigned a task to take 10 books (10 files) from point A to point B, If they are individual then the computer needs to get one take it to point B and then come back to point A to pick up the next book and so on. Whereas if it needed to take one big book that was the equivalent weight of the 10 books combined it would take longer to move the book but it will be a quicker process because the computer only makes one trip.
1
Nice metaphor, but I can't see why the two processes are similar.
– Máté Juhász
Oct 5 at 4:01
add a comment |
up vote
-1
down vote
up vote
-1
down vote
Think of it like this,
The computer is assigned a task to take 10 books (10 files) from point A to point B, If they are individual then the computer needs to get one take it to point B and then come back to point A to pick up the next book and so on. Whereas if it needed to take one big book that was the equivalent weight of the 10 books combined it would take longer to move the book but it will be a quicker process because the computer only makes one trip.
Think of it like this,
The computer is assigned a task to take 10 books (10 files) from point A to point B, If they are individual then the computer needs to get one take it to point B and then come back to point A to pick up the next book and so on. Whereas if it needed to take one big book that was the equivalent weight of the 10 books combined it would take longer to move the book but it will be a quicker process because the computer only makes one trip.
edited Oct 5 at 19:50
answered Oct 5 at 3:50
TheLifeHacker
11
11
1
Nice metaphor, but I can't see why the two processes are similar.
– Máté Juhász
Oct 5 at 4:01
add a comment |
1
Nice metaphor, but I can't see why the two processes are similar.
– Máté Juhász
Oct 5 at 4:01
1
1
Nice metaphor, but I can't see why the two processes are similar.
– Máté Juhász
Oct 5 at 4:01
Nice metaphor, but I can't see why the two processes are similar.
– Máté Juhász
Oct 5 at 4:01
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f857243%2fwhy-is-file-transfer-speed-decreased-when-copying-multiple-smaller-files%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown