Video conversion bottleneck with ffmpeg
I'm curious to know which part of my setup is the main bottleneck in converting my video files to H.265 codecs. I'd like to avoid the debate about whether it is worth it or not, and let's assume I've done enough thought to decide it is worth my CPU time and effort to do this. I'd like to diagnose the bottleneck areas that are the most cost-effective or worthwhile to consider upgrading.
The video files reside on a 4-drive external array connected to the computer via USB2. I'm running ffmpeg via a script which is basically batch processing the entire library. I do not transfer any files off the external drive during processing. ffmpeg is called from the main OS drive. If there's other relevant info I've left out, please let me know and I can provide it
Here are my specs:
Operating System: Windows 8.1 64-bit (6.3, Build 9600) (9600.winblue_r4.141028-1500)
Computer: Dell XPS 8700
Processor: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (8 CPUs), ~3.4GHz
Memory: 8192MB RAM
Card name: NVIDIA GeForce GT 635
Main OS HDD: WD Red 1TB NAS Hard Drive - 5400 RPM Class, SATA 6 Gb/s, 64 MB Cache, 3.5"
hard-drive memory ffmpeg cpu
asked Jan 29 at 17:11
J MJ M
1183
|
show 1 more comment
I'm curious to know which part of my setup is the main bottleneck in converting my video files to H.265 codecs. I'd like to avoid the debate about whether it is worth it or not, and let's assume I've done enough thought to decide it is worth my CPU time and effort to do this. I'd like to diagnose the bottleneck areas that are the most cost-effective or worthwhile to consider upgrading.
The video files reside on a 4-drive external array connected to the computer via USB2. I'm running ffmpeg via a script which is basically batch processing the entire library. I do not transfer any files off the external drive during processing. ffmpeg is called from the main OS drive. If there's other relevant info I've left out, please let me know and I can provide it
Here are my specs:
Operating System: Windows 8.1 64-bit (6.3, Build 9600) (9600.winblue_r4.141028-1500)
Computer: Dell XPS 8700
Processor: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (8 CPUs), ~3.4GHz
Memory: 8192MB RAM
Card name: NVIDIA GeForce GT 635
Main OS HDD: WD Red 1TB NAS Hard Drive - 5400 RPM Class, SATA 6 Gb/s, 64 MB Cache, 3.5"
hard-drive memory ffmpeg cpu
asked Jan 29 at 17:11
J MJ M
1183
Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.
– shawn
Jan 29 at 20:36
@shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?
– J M
Jan 29 at 20:48
Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.
– shawn
Jan 29 at 21:09
@shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?
– J M
Jan 30 at 0:11
1
Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.
– shawn
Jan 30 at 0:53
|
show 1 more comment
I'm curious to know which part of my setup is the main bottleneck in converting my video files to H.265 codecs. I'd like to avoid the debate about whether it is worth it or not, and let's assume I've done enough thought to decide it is worth my CPU time and effort to do this. I'd like to diagnose the bottleneck areas that are the most cost-effective or worthwhile to consider upgrading.
The video files reside on a 4-drive external array connected to the computer via USB2. I'm running ffmpeg via a script which is basically batch processing the entire library. I do not transfer any files off the external drive during processing. ffmpeg is called from the main OS drive. If there's other relevant info I've left out, please let me know and I can provide it
Here are my specs:
Operating System: Windows 8.1 64-bit (6.3, Build 9600) (9600.winblue_r4.141028-1500)
Computer: Dell XPS 8700
Processor: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (8 CPUs), ~3.4GHz
Memory: 8192MB RAM
Card name: NVIDIA GeForce GT 635
Main OS HDD: WD Red 1TB NAS Hard Drive - 5400 RPM Class, SATA 6 Gb/s, 64 MB Cache, 3.5"
hard-drive memory ffmpeg cpu
asked Jan 29 at 17:11
J MJ M
1183
I'm curious to know which part of my setup is the main bottleneck in converting my video files to H.265 codecs. I'd like to avoid the debate about whether it is worth it or not, and let's assume I've done enough thought to decide it is worth my CPU time and effort to do this. I'd like to diagnose the bottleneck areas that are the most cost-effective or worthwhile to consider upgrading.
The video files reside on a 4-drive external array connected to the computer via USB2. I'm running ffmpeg via a script which is basically batch processing the entire library. I do not transfer any files off the external drive during processing. ffmpeg is called from the main OS drive. If there's other relevant info I've left out, please let me know and I can provide it
Here are my specs:
Operating System: Windows 8.1 64-bit (6.3, Build 9600) (9600.winblue_r4.141028-1500)
Computer: Dell XPS 8700
Processor: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (8 CPUs), ~3.4GHz
Memory: 8192MB RAM
Card name: NVIDIA GeForce GT 635
Main OS HDD: WD Red 1TB NAS Hard Drive - 5400 RPM Class, SATA 6 Gb/s, 64 MB Cache, 3.5"
hard-drive memory ffmpeg cpu
hard-drive memory ffmpeg cpu
asked Jan 29 at 17:11
J MJ M
1183
asked Jan 29 at 17:11
J MJ M
1183
asked Jan 29 at 17:11
J MJ M
1183
asked Jan 29 at 17:11
J MJ M
1183
asked Jan 29 at 17:11
J MJ M
1183
1183
Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.
– shawn
Jan 29 at 20:36
@shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?
– J M
Jan 29 at 20:48
Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.
– shawn
Jan 29 at 21:09
@shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?
– J M
Jan 30 at 0:11
1
Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.
– shawn
Jan 30 at 0:53
|
show 1 more comment
Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.
– shawn
Jan 29 at 20:36
@shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?
– J M
Jan 29 at 20:48
Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.
– shawn
Jan 29 at 21:09
@shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?
– J M
Jan 30 at 0:11
1
Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.
– shawn
Jan 30 at 0:53
Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.
– shawn
Jan 29 at 20:36
Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.
– shawn
Jan 29 at 20:36
@shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?
– J M
Jan 29 at 20:48
@shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?
– J M
Jan 29 at 20:48
Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.
– shawn
Jan 29 at 21:09
Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.
– shawn
Jan 29 at 21:09
@shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?
– J M
Jan 30 at 0:11
@shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?
– J M
Jan 30 at 0:11
1
1
Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.
– shawn
Jan 30 at 0:53
Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.
– shawn
Jan 30 at 0:53
|
show 1 more comment
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1399735%2fvideo-conversion-bottleneck-with-ffmpeg%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1399735%2fvideo-conversion-bottleneck-with-ffmpeg%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.
– shawn
Jan 29 at 20:36
@shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?
– J M
Jan 29 at 20:48
Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.
– shawn
Jan 29 at 21:09
@shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?
– J M
Jan 30 at 0:11
1
Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.
– shawn
Jan 30 at 0:53