Setting up zfs for multiple drives with redundancy on Ubuntu











up vote
1
down vote

favorite












Currently I have a Ubuntu server setup with 2 4tb disks each with ext4 in a RAID 1 (not using zfs). I want to have complete redundancy in case of a drive failure.



I am adding another 2 4tb drives to the machine. I'll probably do this again in the future so I'm thinking I should switch to zfs to support this.



I read on some guides that you can do both striping and mirroring with zfs, which I think is the way I want to go.



Normally I would stripe the drives into two pairs, and then raid 1 the two pairs to get a fast redundant "8tb" drive.



How would I accomplish this in zfs? And would that allow for an easy addition of more drives down the line?










share|improve this question


























    up vote
    1
    down vote

    favorite












    Currently I have a Ubuntu server setup with 2 4tb disks each with ext4 in a RAID 1 (not using zfs). I want to have complete redundancy in case of a drive failure.



    I am adding another 2 4tb drives to the machine. I'll probably do this again in the future so I'm thinking I should switch to zfs to support this.



    I read on some guides that you can do both striping and mirroring with zfs, which I think is the way I want to go.



    Normally I would stripe the drives into two pairs, and then raid 1 the two pairs to get a fast redundant "8tb" drive.



    How would I accomplish this in zfs? And would that allow for an easy addition of more drives down the line?










    share|improve this question
























      up vote
      1
      down vote

      favorite









      up vote
      1
      down vote

      favorite











      Currently I have a Ubuntu server setup with 2 4tb disks each with ext4 in a RAID 1 (not using zfs). I want to have complete redundancy in case of a drive failure.



      I am adding another 2 4tb drives to the machine. I'll probably do this again in the future so I'm thinking I should switch to zfs to support this.



      I read on some guides that you can do both striping and mirroring with zfs, which I think is the way I want to go.



      Normally I would stripe the drives into two pairs, and then raid 1 the two pairs to get a fast redundant "8tb" drive.



      How would I accomplish this in zfs? And would that allow for an easy addition of more drives down the line?










      share|improve this question













      Currently I have a Ubuntu server setup with 2 4tb disks each with ext4 in a RAID 1 (not using zfs). I want to have complete redundancy in case of a drive failure.



      I am adding another 2 4tb drives to the machine. I'll probably do this again in the future so I'm thinking I should switch to zfs to support this.



      I read on some guides that you can do both striping and mirroring with zfs, which I think is the way I want to go.



      Normally I would stripe the drives into two pairs, and then raid 1 the two pairs to get a fast redundant "8tb" drive.



      How would I accomplish this in zfs? And would that allow for an easy addition of more drives down the line?







      ubuntu zfs






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 27 at 18:42









      sharf

      1134




      1134






















          1 Answer
          1






          active

          oldest

          votes

















          up vote
          1
          down vote



          accepted










          It's super easy to configure that (line-splitting to make the structure easier to see):



          # zpool create <poolname> 
          mirror <disk1> <disk2>
          mirror <disk3> <disk4>


          This will create a pool with two stripes, each of which is a mirror between two disks. This is a nice flexible setup (easy to add new drives, easy to understand). To add another mirror in the future, you simply do:



          # zpool add -n <poolname> 
          mirror <disk5> <disk6>
          would update '<poolname>' to the following configuration:
          <poolname>
          mirror
          <disk1>
          <disk2>
          mirror
          <disk3>
          <disk4>
          mirror
          <disk5>
          <disk6>


          (Using option -n does a dry run first and prints the pool configuration for you to inspect, which I would always recommend to avoid misconfigurations that are hard / impossible to back out of after the fact!)



          However, note that this setup is "inefficient" in that it only guarantees data safety for one disk failure: you're only safe for two failures when they happen on different stripes, which happens in ~67% of two-disk failure scenarios. Also, the redundant data still uses 50% of your pool.



          There is a less-flexible ZFS-specific RAID format called RAID-Z that allows you to pool all your disks and get efficient data protection. (I say less flexible because it is difficult to add more disks to the pool in the future, although that feature is being worked on currently.) There are raidz1 and raidz2 variants, which allow you to lose up to 1 or up to 2 disks respectively, no matter which disks fail. For 4 equal-size disks, the raidz2 variant will still use 50% of your storage for redundancy but will guarantee you survive two disk failures; the raidz1 variant will reduce your redundancy overhead from 50% to only 25%, but only guarantee you survive one disk failure.






          share|improve this answer























          • for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
            – sharf
            Nov 28 at 20:57










          • “Yes” to all of the questions you asked :).
            – Dan
            Nov 28 at 21:01










          • Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
            – sharf
            Nov 28 at 21:07










          • One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
            – Dan
            Nov 28 at 21:14










          • Awesome! Thanks
            – sharf
            Nov 28 at 21:15











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "3"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1378855%2fsetting-up-zfs-for-multiple-drives-with-redundancy-on-ubuntu%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          1
          down vote



          accepted










          It's super easy to configure that (line-splitting to make the structure easier to see):



          # zpool create <poolname> 
          mirror <disk1> <disk2>
          mirror <disk3> <disk4>


          This will create a pool with two stripes, each of which is a mirror between two disks. This is a nice flexible setup (easy to add new drives, easy to understand). To add another mirror in the future, you simply do:



          # zpool add -n <poolname> 
          mirror <disk5> <disk6>
          would update '<poolname>' to the following configuration:
          <poolname>
          mirror
          <disk1>
          <disk2>
          mirror
          <disk3>
          <disk4>
          mirror
          <disk5>
          <disk6>


          (Using option -n does a dry run first and prints the pool configuration for you to inspect, which I would always recommend to avoid misconfigurations that are hard / impossible to back out of after the fact!)



          However, note that this setup is "inefficient" in that it only guarantees data safety for one disk failure: you're only safe for two failures when they happen on different stripes, which happens in ~67% of two-disk failure scenarios. Also, the redundant data still uses 50% of your pool.



          There is a less-flexible ZFS-specific RAID format called RAID-Z that allows you to pool all your disks and get efficient data protection. (I say less flexible because it is difficult to add more disks to the pool in the future, although that feature is being worked on currently.) There are raidz1 and raidz2 variants, which allow you to lose up to 1 or up to 2 disks respectively, no matter which disks fail. For 4 equal-size disks, the raidz2 variant will still use 50% of your storage for redundancy but will guarantee you survive two disk failures; the raidz1 variant will reduce your redundancy overhead from 50% to only 25%, but only guarantee you survive one disk failure.






          share|improve this answer























          • for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
            – sharf
            Nov 28 at 20:57










          • “Yes” to all of the questions you asked :).
            – Dan
            Nov 28 at 21:01










          • Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
            – sharf
            Nov 28 at 21:07










          • One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
            – Dan
            Nov 28 at 21:14










          • Awesome! Thanks
            – sharf
            Nov 28 at 21:15















          up vote
          1
          down vote



          accepted










          It's super easy to configure that (line-splitting to make the structure easier to see):



          # zpool create <poolname> 
          mirror <disk1> <disk2>
          mirror <disk3> <disk4>


          This will create a pool with two stripes, each of which is a mirror between two disks. This is a nice flexible setup (easy to add new drives, easy to understand). To add another mirror in the future, you simply do:



          # zpool add -n <poolname> 
          mirror <disk5> <disk6>
          would update '<poolname>' to the following configuration:
          <poolname>
          mirror
          <disk1>
          <disk2>
          mirror
          <disk3>
          <disk4>
          mirror
          <disk5>
          <disk6>


          (Using option -n does a dry run first and prints the pool configuration for you to inspect, which I would always recommend to avoid misconfigurations that are hard / impossible to back out of after the fact!)



          However, note that this setup is "inefficient" in that it only guarantees data safety for one disk failure: you're only safe for two failures when they happen on different stripes, which happens in ~67% of two-disk failure scenarios. Also, the redundant data still uses 50% of your pool.



          There is a less-flexible ZFS-specific RAID format called RAID-Z that allows you to pool all your disks and get efficient data protection. (I say less flexible because it is difficult to add more disks to the pool in the future, although that feature is being worked on currently.) There are raidz1 and raidz2 variants, which allow you to lose up to 1 or up to 2 disks respectively, no matter which disks fail. For 4 equal-size disks, the raidz2 variant will still use 50% of your storage for redundancy but will guarantee you survive two disk failures; the raidz1 variant will reduce your redundancy overhead from 50% to only 25%, but only guarantee you survive one disk failure.






          share|improve this answer























          • for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
            – sharf
            Nov 28 at 20:57










          • “Yes” to all of the questions you asked :).
            – Dan
            Nov 28 at 21:01










          • Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
            – sharf
            Nov 28 at 21:07










          • One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
            – Dan
            Nov 28 at 21:14










          • Awesome! Thanks
            – sharf
            Nov 28 at 21:15













          up vote
          1
          down vote



          accepted







          up vote
          1
          down vote



          accepted






          It's super easy to configure that (line-splitting to make the structure easier to see):



          # zpool create <poolname> 
          mirror <disk1> <disk2>
          mirror <disk3> <disk4>


          This will create a pool with two stripes, each of which is a mirror between two disks. This is a nice flexible setup (easy to add new drives, easy to understand). To add another mirror in the future, you simply do:



          # zpool add -n <poolname> 
          mirror <disk5> <disk6>
          would update '<poolname>' to the following configuration:
          <poolname>
          mirror
          <disk1>
          <disk2>
          mirror
          <disk3>
          <disk4>
          mirror
          <disk5>
          <disk6>


          (Using option -n does a dry run first and prints the pool configuration for you to inspect, which I would always recommend to avoid misconfigurations that are hard / impossible to back out of after the fact!)



          However, note that this setup is "inefficient" in that it only guarantees data safety for one disk failure: you're only safe for two failures when they happen on different stripes, which happens in ~67% of two-disk failure scenarios. Also, the redundant data still uses 50% of your pool.



          There is a less-flexible ZFS-specific RAID format called RAID-Z that allows you to pool all your disks and get efficient data protection. (I say less flexible because it is difficult to add more disks to the pool in the future, although that feature is being worked on currently.) There are raidz1 and raidz2 variants, which allow you to lose up to 1 or up to 2 disks respectively, no matter which disks fail. For 4 equal-size disks, the raidz2 variant will still use 50% of your storage for redundancy but will guarantee you survive two disk failures; the raidz1 variant will reduce your redundancy overhead from 50% to only 25%, but only guarantee you survive one disk failure.






          share|improve this answer














          It's super easy to configure that (line-splitting to make the structure easier to see):



          # zpool create <poolname> 
          mirror <disk1> <disk2>
          mirror <disk3> <disk4>


          This will create a pool with two stripes, each of which is a mirror between two disks. This is a nice flexible setup (easy to add new drives, easy to understand). To add another mirror in the future, you simply do:



          # zpool add -n <poolname> 
          mirror <disk5> <disk6>
          would update '<poolname>' to the following configuration:
          <poolname>
          mirror
          <disk1>
          <disk2>
          mirror
          <disk3>
          <disk4>
          mirror
          <disk5>
          <disk6>


          (Using option -n does a dry run first and prints the pool configuration for you to inspect, which I would always recommend to avoid misconfigurations that are hard / impossible to back out of after the fact!)



          However, note that this setup is "inefficient" in that it only guarantees data safety for one disk failure: you're only safe for two failures when they happen on different stripes, which happens in ~67% of two-disk failure scenarios. Also, the redundant data still uses 50% of your pool.



          There is a less-flexible ZFS-specific RAID format called RAID-Z that allows you to pool all your disks and get efficient data protection. (I say less flexible because it is difficult to add more disks to the pool in the future, although that feature is being worked on currently.) There are raidz1 and raidz2 variants, which allow you to lose up to 1 or up to 2 disks respectively, no matter which disks fail. For 4 equal-size disks, the raidz2 variant will still use 50% of your storage for redundancy but will guarantee you survive two disk failures; the raidz1 variant will reduce your redundancy overhead from 50% to only 25%, but only guarantee you survive one disk failure.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Nov 29 at 1:52

























          answered Nov 28 at 2:07









          Dan

          638314




          638314












          • for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
            – sharf
            Nov 28 at 20:57










          • “Yes” to all of the questions you asked :).
            – Dan
            Nov 28 at 21:01










          • Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
            – sharf
            Nov 28 at 21:07










          • One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
            – Dan
            Nov 28 at 21:14










          • Awesome! Thanks
            – sharf
            Nov 28 at 21:15


















          • for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
            – sharf
            Nov 28 at 20:57










          • “Yes” to all of the questions you asked :).
            – Dan
            Nov 28 at 21:01










          • Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
            – sharf
            Nov 28 at 21:07










          • One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
            – Dan
            Nov 28 at 21:14










          • Awesome! Thanks
            – sharf
            Nov 28 at 21:15
















          for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
          – sharf
          Nov 28 at 20:57




          for clarification, would following your answer in the code snippets using 4 4tb drives end up with a virtual 8tb drive, but redundant? Currently I have 2 of the 4TB drives in a raid 1 with data on them. Would it be possible following your answer here, to set up 2 new 4tb in ZFS with a mirror, copy the data over, then format those drives and add them to the pool in another mirror? would that start with a virtual 4tb (redundant) drive than upgrade to 8tb?
          – sharf
          Nov 28 at 20:57












          “Yes” to all of the questions you asked :).
          – Dan
          Nov 28 at 21:01




          “Yes” to all of the questions you asked :).
          – Dan
          Nov 28 at 21:01












          Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
          – sharf
          Nov 28 at 21:07




          Great! Last question, this is for non-critical media (plex server) nothing is irreplaceable, just time consuming. Would the basic mirroring for just one drive be enough for most cases? I seem to think so. And would zfs report a failing drive in anyway?
          – sharf
          Nov 28 at 21:07












          One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
          – Dan
          Nov 28 at 21:14




          One disk of redundancy seems ok if you can recreate the data. ZFS does periodic scrubs that will fix corrupted blocks, and it can notice failed disks on any IO (either normal reads / writes or during scrub) and displays failures in the output of “zpool status”. (You should configure some kind of alerting on top of that though, so you don’t have to check the pool manually.)
          – Dan
          Nov 28 at 21:14












          Awesome! Thanks
          – sharf
          Nov 28 at 21:15




          Awesome! Thanks
          – sharf
          Nov 28 at 21:15


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Super User!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1378855%2fsetting-up-zfs-for-multiple-drives-with-redundancy-on-ubuntu%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Mouse cursor on multiple screens with different PPI

          Agildo Ribeiro

          Sometime when accessing a menu: “Ubuntu 16.04 has experienced an internal error”