Setting up zfs for multiple drives with redundancy on Ubuntu
up vote
1
down vote
favorite
Currently I have a Ubuntu server setup with 2 4tb disks each with ext4 in a RAID 1 (not using zfs). I want to have complete redundancy in case of a drive failure.
I am adding another 2 4tb drives to the machine. I'll probably do this again in the future so I'm thinking I should switch to zfs to support this.
I read on some guides that you can do both striping and mirroring with zfs, which I think is the way I want to go.
Normally I would stripe the drives into two pairs, and then raid 1 the two pairs to get a fast redundant "8tb" drive.
How would I accomplish this in zfs? And would that allow for an easy addition of more drives down the line?
ubuntu zfs
add a comment |
up vote
1
down vote
favorite
Currently I have a Ubuntu server setup with 2 4tb disks each with ext4 in a RAID 1 (not using zfs). I want to have complete redundancy in case of a drive failure.
I am adding another 2 4tb drives to the machine. I'll probably do this again in the future so I'm thinking I should switch to zfs to support this.
I read on some guides that you can do both striping and mirroring with zfs, which I think is the way I want to go.
Normally I would stripe the drives into two pairs, and then raid 1 the two pairs to get a fast redundant "8tb" drive.
How would I accomplish this in zfs? And would that allow for an easy addition of more drives down the line?
ubuntu zfs
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
Currently I have a Ubuntu server setup with 2 4tb disks each with ext4 in a RAID 1 (not using zfs). I want to have complete redundancy in case of a drive failure.
I am adding another 2 4tb drives to the machine. I'll probably do this again in the future so I'm thinking I should switch to zfs to support this.
I read on some guides that you can do both striping and mirroring with zfs, which I think is the way I want to go.
Normally I would stripe the drives into two pairs, and then raid 1 the two pairs to get a fast redundant "8tb" drive.
How would I accomplish this in zfs? And would that allow for an easy addition of more drives down the line?
ubuntu zfs
Currently I have a Ubuntu server setup with 2 4tb disks each with ext4 in a RAID 1 (not using zfs). I want to have complete redundancy in case of a drive failure.
I am adding another 2 4tb drives to the machine. I'll probably do this again in the future so I'm thinking I should switch to zfs to support this.
I read on some guides that you can do both striping and mirroring with zfs, which I think is the way I want to go.
Normally I would stripe the drives into two pairs, and then raid 1 the two pairs to get a fast redundant "8tb" drive.
How would I accomplish this in zfs? And would that allow for an easy addition of more drives down the line?
ubuntu zfs
ubuntu zfs
asked Nov 27 at 18:42
sharf
1134
1134
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
It's super easy to configure that (line-splitting to make the structure easier to see):
# zpool create <poolname>
mirror <disk1> <disk2>
mirror <disk3> <disk4>
This will create a pool with two stripes, each of which is a mirror between two disks. This is a nice flexible setup (easy to add new drives, easy to understand). To add another mirror in the future, you simply do:
# zpool add -n <poolname>
mirror <disk5> <disk6>
would update '<poolname>' to the following configuration:
<poolname>
mirror
<disk1>
<disk2>
mirror
<disk3>
<disk4>
mirror
<disk5>
<disk6>
(Using option -n does a dry run first and prints the pool configuration for you to inspect, which I would always recommend to avoid misconfigurations that are hard / impossible to back out of after the fact!)
However, note that this setup is "inefficient" in that it only guarantees data safety for one disk failure: you're only safe for two failures when they happen on different stripes, which happens in ~67% of two-disk failure scenarios. Also, the redundant data still uses 50% of your pool.
There is a less-flexible ZFS-specific RAID format called RAID-Z that allows you to pool all your disks and get efficient data protection. (I say less flexible because it is difficult to add more disks to the pool in the future, although that feature is being worked on currently.) There are raidz1 and raidz2 variants, which allow you to lose up to 1 or up to 2 disks respectively, no matter which disks fail. For 4 equal-size disks, the raidz2 variant will still use 50% of your storage for redundancy but will guarantee you survive two disk failures; the raidz1 variant will reduce your redundancy overhead from 50% to only 25%, but only guarantee you survive one disk failure.
for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
– sharf
Nov 28 at 20:57
“Yes” to all of the questions you asked :).
– Dan
Nov 28 at 21:01
Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
– sharf
Nov 28 at 21:07
One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
– Dan
Nov 28 at 21:14
Awesome! Thanks
– sharf
Nov 28 at 21:15
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
It's super easy to configure that (line-splitting to make the structure easier to see):
# zpool create <poolname>
mirror <disk1> <disk2>
mirror <disk3> <disk4>
This will create a pool with two stripes, each of which is a mirror between two disks. This is a nice flexible setup (easy to add new drives, easy to understand). To add another mirror in the future, you simply do:
# zpool add -n <poolname>
mirror <disk5> <disk6>
would update '<poolname>' to the following configuration:
<poolname>
mirror
<disk1>
<disk2>
mirror
<disk3>
<disk4>
mirror
<disk5>
<disk6>
(Using option -n does a dry run first and prints the pool configuration for you to inspect, which I would always recommend to avoid misconfigurations that are hard / impossible to back out of after the fact!)
However, note that this setup is "inefficient" in that it only guarantees data safety for one disk failure: you're only safe for two failures when they happen on different stripes, which happens in ~67% of two-disk failure scenarios. Also, the redundant data still uses 50% of your pool.
There is a less-flexible ZFS-specific RAID format called RAID-Z that allows you to pool all your disks and get efficient data protection. (I say less flexible because it is difficult to add more disks to the pool in the future, although that feature is being worked on currently.) There are raidz1 and raidz2 variants, which allow you to lose up to 1 or up to 2 disks respectively, no matter which disks fail. For 4 equal-size disks, the raidz2 variant will still use 50% of your storage for redundancy but will guarantee you survive two disk failures; the raidz1 variant will reduce your redundancy overhead from 50% to only 25%, but only guarantee you survive one disk failure.
for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
– sharf
Nov 28 at 20:57
“Yes” to all of the questions you asked :).
– Dan
Nov 28 at 21:01
Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
– sharf
Nov 28 at 21:07
One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
– Dan
Nov 28 at 21:14
Awesome! Thanks
– sharf
Nov 28 at 21:15
add a comment |
up vote
1
down vote
accepted
It's super easy to configure that (line-splitting to make the structure easier to see):
# zpool create <poolname>
mirror <disk1> <disk2>
mirror <disk3> <disk4>
This will create a pool with two stripes, each of which is a mirror between two disks. This is a nice flexible setup (easy to add new drives, easy to understand). To add another mirror in the future, you simply do:
# zpool add -n <poolname>
mirror <disk5> <disk6>
would update '<poolname>' to the following configuration:
<poolname>
mirror
<disk1>
<disk2>
mirror
<disk3>
<disk4>
mirror
<disk5>
<disk6>
(Using option -n does a dry run first and prints the pool configuration for you to inspect, which I would always recommend to avoid misconfigurations that are hard / impossible to back out of after the fact!)
However, note that this setup is "inefficient" in that it only guarantees data safety for one disk failure: you're only safe for two failures when they happen on different stripes, which happens in ~67% of two-disk failure scenarios. Also, the redundant data still uses 50% of your pool.
There is a less-flexible ZFS-specific RAID format called RAID-Z that allows you to pool all your disks and get efficient data protection. (I say less flexible because it is difficult to add more disks to the pool in the future, although that feature is being worked on currently.) There are raidz1 and raidz2 variants, which allow you to lose up to 1 or up to 2 disks respectively, no matter which disks fail. For 4 equal-size disks, the raidz2 variant will still use 50% of your storage for redundancy but will guarantee you survive two disk failures; the raidz1 variant will reduce your redundancy overhead from 50% to only 25%, but only guarantee you survive one disk failure.
for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
– sharf
Nov 28 at 20:57
“Yes” to all of the questions you asked :).
– Dan
Nov 28 at 21:01
Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
– sharf
Nov 28 at 21:07
One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
– Dan
Nov 28 at 21:14
Awesome! Thanks
– sharf
Nov 28 at 21:15
add a comment |
up vote
1
down vote
accepted
up vote
1
down vote
accepted
It's super easy to configure that (line-splitting to make the structure easier to see):
# zpool create <poolname>
mirror <disk1> <disk2>
mirror <disk3> <disk4>
This will create a pool with two stripes, each of which is a mirror between two disks. This is a nice flexible setup (easy to add new drives, easy to understand). To add another mirror in the future, you simply do:
# zpool add -n <poolname>
mirror <disk5> <disk6>
would update '<poolname>' to the following configuration:
<poolname>
mirror
<disk1>
<disk2>
mirror
<disk3>
<disk4>
mirror
<disk5>
<disk6>
(Using option -n does a dry run first and prints the pool configuration for you to inspect, which I would always recommend to avoid misconfigurations that are hard / impossible to back out of after the fact!)
However, note that this setup is "inefficient" in that it only guarantees data safety for one disk failure: you're only safe for two failures when they happen on different stripes, which happens in ~67% of two-disk failure scenarios. Also, the redundant data still uses 50% of your pool.
There is a less-flexible ZFS-specific RAID format called RAID-Z that allows you to pool all your disks and get efficient data protection. (I say less flexible because it is difficult to add more disks to the pool in the future, although that feature is being worked on currently.) There are raidz1 and raidz2 variants, which allow you to lose up to 1 or up to 2 disks respectively, no matter which disks fail. For 4 equal-size disks, the raidz2 variant will still use 50% of your storage for redundancy but will guarantee you survive two disk failures; the raidz1 variant will reduce your redundancy overhead from 50% to only 25%, but only guarantee you survive one disk failure.
It's super easy to configure that (line-splitting to make the structure easier to see):
# zpool create <poolname>
mirror <disk1> <disk2>
mirror <disk3> <disk4>
This will create a pool with two stripes, each of which is a mirror between two disks. This is a nice flexible setup (easy to add new drives, easy to understand). To add another mirror in the future, you simply do:
# zpool add -n <poolname>
mirror <disk5> <disk6>
would update '<poolname>' to the following configuration:
<poolname>
mirror
<disk1>
<disk2>
mirror
<disk3>
<disk4>
mirror
<disk5>
<disk6>
(Using option -n does a dry run first and prints the pool configuration for you to inspect, which I would always recommend to avoid misconfigurations that are hard / impossible to back out of after the fact!)
However, note that this setup is "inefficient" in that it only guarantees data safety for one disk failure: you're only safe for two failures when they happen on different stripes, which happens in ~67% of two-disk failure scenarios. Also, the redundant data still uses 50% of your pool.
There is a less-flexible ZFS-specific RAID format called RAID-Z that allows you to pool all your disks and get efficient data protection. (I say less flexible because it is difficult to add more disks to the pool in the future, although that feature is being worked on currently.) There are raidz1 and raidz2 variants, which allow you to lose up to 1 or up to 2 disks respectively, no matter which disks fail. For 4 equal-size disks, the raidz2 variant will still use 50% of your storage for redundancy but will guarantee you survive two disk failures; the raidz1 variant will reduce your redundancy overhead from 50% to only 25%, but only guarantee you survive one disk failure.
edited Nov 29 at 1:52
answered Nov 28 at 2:07
Dan
638314
638314
for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
– sharf
Nov 28 at 20:57
“Yes” to all of the questions you asked :).
– Dan
Nov 28 at 21:01
Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
– sharf
Nov 28 at 21:07
One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
– Dan
Nov 28 at 21:14
Awesome! Thanks
– sharf
Nov 28 at 21:15
add a comment |
for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
– sharf
Nov 28 at 20:57
“Yes” to all of the questions you asked :).
– Dan
Nov 28 at 21:01
Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
– sharf
Nov 28 at 21:07
One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
– Dan
Nov 28 at 21:14
Awesome! Thanks
– sharf
Nov 28 at 21:15
for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
– sharf
Nov 28 at 20:57
for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
– sharf
Nov 28 at 20:57
“Yes” to all of the questions you asked :).
– Dan
Nov 28 at 21:01
“Yes” to all of the questions you asked :).
– Dan
Nov 28 at 21:01
Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
– sharf
Nov 28 at 21:07
Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
– sharf
Nov 28 at 21:07
One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
– Dan
Nov 28 at 21:14
One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
– Dan
Nov 28 at 21:14
Awesome! Thanks
– sharf
Nov 28 at 21:15
Awesome! Thanks
– sharf
Nov 28 at 21:15
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1378855%2fsetting-up-zfs-for-multiple-drives-with-redundancy-on-ubuntu%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown