rsync and symbolic links












52















I want to backup my home directory to an external drive nightly using a cron job to execute rsync. I am unsure of the exact behavior of rsync's symbolic link flags.





  1. rsync's -a flag includes the -l flag (i.e. "copy symlinks as symlinks"). Does this just mean it will copy the link or that it will follow the link and copy everything in the link-to directory? I want to avoid that because I have links to directories full of media files that would involve copying hundreds of gigabytes I don't need to back up.

  2. Fearing (but not certain) that rsync -a would copy all those media files I instead added the --no-links flag. This does not seem to be behavior I want. It just ignores copying any link which is problematic because I do have links I want copied (e.g. links to common header files from different project directories).

  3. Assuming #1 above (without the --no-links flag) is what I really want and it just copies the link without copying the linked-to files, will the links break when they are backed up? For example I may rsync source directory /home/me/projects/misc to /media/extdrive/backup/home/me/projects/misc. In this case I assume rsync is not smart enough nor does it try to correct the contents of symlinks for the relative directory changes. Is this correct? This is okay, it doesn't matter if the links are broken in the backup directories so long as they would be fixed and working if such time comes when they need to be restored.










share|improve this question




















  • 2





    The -L flag is probably what you want. From the rsync man page on Centos Linux: "-L, --copy-links transform symlink into referent file/dir." I recommend testing this with some sample directories and links to make sure you don't overwrite any data that you don't intend to. Edited to included warning about possible data loss. ;)

    – Mick T
    Oct 23 '15 at 19:22


















52















I want to backup my home directory to an external drive nightly using a cron job to execute rsync. I am unsure of the exact behavior of rsync's symbolic link flags.





  1. rsync's -a flag includes the -l flag (i.e. "copy symlinks as symlinks"). Does this just mean it will copy the link or that it will follow the link and copy everything in the link-to directory? I want to avoid that because I have links to directories full of media files that would involve copying hundreds of gigabytes I don't need to back up.

  2. Fearing (but not certain) that rsync -a would copy all those media files I instead added the --no-links flag. This does not seem to be behavior I want. It just ignores copying any link which is problematic because I do have links I want copied (e.g. links to common header files from different project directories).

  3. Assuming #1 above (without the --no-links flag) is what I really want and it just copies the link without copying the linked-to files, will the links break when they are backed up? For example I may rsync source directory /home/me/projects/misc to /media/extdrive/backup/home/me/projects/misc. In this case I assume rsync is not smart enough nor does it try to correct the contents of symlinks for the relative directory changes. Is this correct? This is okay, it doesn't matter if the links are broken in the backup directories so long as they would be fixed and working if such time comes when they need to be restored.










share|improve this question




















  • 2





    The -L flag is probably what you want. From the rsync man page on Centos Linux: "-L, --copy-links transform symlink into referent file/dir." I recommend testing this with some sample directories and links to make sure you don't overwrite any data that you don't intend to. Edited to included warning about possible data loss. ;)

    – Mick T
    Oct 23 '15 at 19:22
















52












52








52


13






I want to backup my home directory to an external drive nightly using a cron job to execute rsync. I am unsure of the exact behavior of rsync's symbolic link flags.





  1. rsync's -a flag includes the -l flag (i.e. "copy symlinks as symlinks"). Does this just mean it will copy the link or that it will follow the link and copy everything in the link-to directory? I want to avoid that because I have links to directories full of media files that would involve copying hundreds of gigabytes I don't need to back up.

  2. Fearing (but not certain) that rsync -a would copy all those media files I instead added the --no-links flag. This does not seem to be behavior I want. It just ignores copying any link which is problematic because I do have links I want copied (e.g. links to common header files from different project directories).

  3. Assuming #1 above (without the --no-links flag) is what I really want and it just copies the link without copying the linked-to files, will the links break when they are backed up? For example I may rsync source directory /home/me/projects/misc to /media/extdrive/backup/home/me/projects/misc. In this case I assume rsync is not smart enough nor does it try to correct the contents of symlinks for the relative directory changes. Is this correct? This is okay, it doesn't matter if the links are broken in the backup directories so long as they would be fixed and working if such time comes when they need to be restored.










share|improve this question
















I want to backup my home directory to an external drive nightly using a cron job to execute rsync. I am unsure of the exact behavior of rsync's symbolic link flags.





  1. rsync's -a flag includes the -l flag (i.e. "copy symlinks as symlinks"). Does this just mean it will copy the link or that it will follow the link and copy everything in the link-to directory? I want to avoid that because I have links to directories full of media files that would involve copying hundreds of gigabytes I don't need to back up.

  2. Fearing (but not certain) that rsync -a would copy all those media files I instead added the --no-links flag. This does not seem to be behavior I want. It just ignores copying any link which is problematic because I do have links I want copied (e.g. links to common header files from different project directories).

  3. Assuming #1 above (without the --no-links flag) is what I really want and it just copies the link without copying the linked-to files, will the links break when they are backed up? For example I may rsync source directory /home/me/projects/misc to /media/extdrive/backup/home/me/projects/misc. In this case I assume rsync is not smart enough nor does it try to correct the contents of symlinks for the relative directory changes. Is this correct? This is okay, it doesn't matter if the links are broken in the backup directories so long as they would be fixed and working if such time comes when they need to be restored.







linux rsync symbolic-link






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Apr 29 '15 at 18:30









Kevin Panko

5,869113648




5,869113648










asked Aug 18 '14 at 4:35









seansean

390135




390135








  • 2





    The -L flag is probably what you want. From the rsync man page on Centos Linux: "-L, --copy-links transform symlink into referent file/dir." I recommend testing this with some sample directories and links to make sure you don't overwrite any data that you don't intend to. Edited to included warning about possible data loss. ;)

    – Mick T
    Oct 23 '15 at 19:22
















  • 2





    The -L flag is probably what you want. From the rsync man page on Centos Linux: "-L, --copy-links transform symlink into referent file/dir." I recommend testing this with some sample directories and links to make sure you don't overwrite any data that you don't intend to. Edited to included warning about possible data loss. ;)

    – Mick T
    Oct 23 '15 at 19:22










2




2





The -L flag is probably what you want. From the rsync man page on Centos Linux: "-L, --copy-links transform symlink into referent file/dir." I recommend testing this with some sample directories and links to make sure you don't overwrite any data that you don't intend to. Edited to included warning about possible data loss. ;)

– Mick T
Oct 23 '15 at 19:22







The -L flag is probably what you want. From the rsync man page on Centos Linux: "-L, --copy-links transform symlink into referent file/dir." I recommend testing this with some sample directories and links to make sure you don't overwrite any data that you don't intend to. Edited to included warning about possible data loss. ;)

– Mick T
Oct 23 '15 at 19:22












3 Answers
3






active

oldest

votes


















67
















  1. "Copy symlinks as symlinks" means exactly what it says: If rsync sees a symlink in the source directory, it will create an identical symlink in the destination. Nothing more.



    (A few lines down in the manual page, a different option, --copy-links, describes the opposite behavior (always copying the data) which you described as undesirable.)



    See also the section "Symbolic links":




    If --links is specified, then symlinks are recreated with the same target on the destination.




  2. It's not the behavior you want because you're misinterpreting what --links does and therefore asking for the wrong behavior (see answer #1).



  3. By default, it copies the destination exactly.



    That is, if the link pointed to an absolute path (e.g. /home/me/projects), it'll continue pointing to the same path; it won't break, it'll just continue pointing to a file in your home directory rather than the one in your backup.



    Meanwhile, if the link pointed to a relative path (e.g. ../../projects), it'll also continue pointing to the same path, but since it's relative to the symlink's location, the symlink in your backup will also be pointing to a file in your backup.



    Unfortunately there doesn't seem to be any option to translate absolute symlinks for their new base (only an option to break them entirely). To avoid problems, you should change existing symlinks to relative ones (which is a good idea generally, for links inside $HOME).








share|improve this answer



















  • 1





    ummm so what's the difference between --copy-links and --links?

    – Alexander Mills
    Dec 18 '18 at 6:57











  • --copy-links = -L, and --links = -l. See description from the documentation in my answer here: superuser.com/a/1388910/425838

    – Gabriel Staples
    Dec 30 '18 at 7:49





















1














There are several very good pages which describe rsync in great detail.



http://www.jveweb.net/en/archives/2010/11/synchronizing-folders-with-rsync.html



http://www.mikerubel.org/computers/rsync_snapshots/index.html



I would also recommend utilities such as backintime or rsnapshot.






share|improve this answer



















  • 22





    Please include some relevant snippets into you currently link-only answer.

    – Cristian Ciupitu
    Sep 25 '15 at 19:08



















0














@grawity has a great answer, but here's a screenshot of some relevant info from the documentation. Here's the exact wording or the -l and -L options, for instance, which seem to be the most relevant ones in question:



enter image description here



Source: https://linux.die.net/man/1/rsync






share|improve this answer























    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "3"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f799354%2frsync-and-symbolic-links%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    3 Answers
    3






    active

    oldest

    votes








    3 Answers
    3






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    67
















    1. "Copy symlinks as symlinks" means exactly what it says: If rsync sees a symlink in the source directory, it will create an identical symlink in the destination. Nothing more.



      (A few lines down in the manual page, a different option, --copy-links, describes the opposite behavior (always copying the data) which you described as undesirable.)



      See also the section "Symbolic links":




      If --links is specified, then symlinks are recreated with the same target on the destination.




    2. It's not the behavior you want because you're misinterpreting what --links does and therefore asking for the wrong behavior (see answer #1).



    3. By default, it copies the destination exactly.



      That is, if the link pointed to an absolute path (e.g. /home/me/projects), it'll continue pointing to the same path; it won't break, it'll just continue pointing to a file in your home directory rather than the one in your backup.



      Meanwhile, if the link pointed to a relative path (e.g. ../../projects), it'll also continue pointing to the same path, but since it's relative to the symlink's location, the symlink in your backup will also be pointing to a file in your backup.



      Unfortunately there doesn't seem to be any option to translate absolute symlinks for their new base (only an option to break them entirely). To avoid problems, you should change existing symlinks to relative ones (which is a good idea generally, for links inside $HOME).








    share|improve this answer



















    • 1





      ummm so what's the difference between --copy-links and --links?

      – Alexander Mills
      Dec 18 '18 at 6:57











    • --copy-links = -L, and --links = -l. See description from the documentation in my answer here: superuser.com/a/1388910/425838

      – Gabriel Staples
      Dec 30 '18 at 7:49


















    67
















    1. "Copy symlinks as symlinks" means exactly what it says: If rsync sees a symlink in the source directory, it will create an identical symlink in the destination. Nothing more.



      (A few lines down in the manual page, a different option, --copy-links, describes the opposite behavior (always copying the data) which you described as undesirable.)



      See also the section "Symbolic links":




      If --links is specified, then symlinks are recreated with the same target on the destination.




    2. It's not the behavior you want because you're misinterpreting what --links does and therefore asking for the wrong behavior (see answer #1).



    3. By default, it copies the destination exactly.



      That is, if the link pointed to an absolute path (e.g. /home/me/projects), it'll continue pointing to the same path; it won't break, it'll just continue pointing to a file in your home directory rather than the one in your backup.



      Meanwhile, if the link pointed to a relative path (e.g. ../../projects), it'll also continue pointing to the same path, but since it's relative to the symlink's location, the symlink in your backup will also be pointing to a file in your backup.



      Unfortunately there doesn't seem to be any option to translate absolute symlinks for their new base (only an option to break them entirely). To avoid problems, you should change existing symlinks to relative ones (which is a good idea generally, for links inside $HOME).








    share|improve this answer



















    • 1





      ummm so what's the difference between --copy-links and --links?

      – Alexander Mills
      Dec 18 '18 at 6:57











    • --copy-links = -L, and --links = -l. See description from the documentation in my answer here: superuser.com/a/1388910/425838

      – Gabriel Staples
      Dec 30 '18 at 7:49
















    67












    67








    67









    1. "Copy symlinks as symlinks" means exactly what it says: If rsync sees a symlink in the source directory, it will create an identical symlink in the destination. Nothing more.



      (A few lines down in the manual page, a different option, --copy-links, describes the opposite behavior (always copying the data) which you described as undesirable.)



      See also the section "Symbolic links":




      If --links is specified, then symlinks are recreated with the same target on the destination.




    2. It's not the behavior you want because you're misinterpreting what --links does and therefore asking for the wrong behavior (see answer #1).



    3. By default, it copies the destination exactly.



      That is, if the link pointed to an absolute path (e.g. /home/me/projects), it'll continue pointing to the same path; it won't break, it'll just continue pointing to a file in your home directory rather than the one in your backup.



      Meanwhile, if the link pointed to a relative path (e.g. ../../projects), it'll also continue pointing to the same path, but since it's relative to the symlink's location, the symlink in your backup will also be pointing to a file in your backup.



      Unfortunately there doesn't seem to be any option to translate absolute symlinks for their new base (only an option to break them entirely). To avoid problems, you should change existing symlinks to relative ones (which is a good idea generally, for links inside $HOME).








    share|improve this answer















    1. "Copy symlinks as symlinks" means exactly what it says: If rsync sees a symlink in the source directory, it will create an identical symlink in the destination. Nothing more.



      (A few lines down in the manual page, a different option, --copy-links, describes the opposite behavior (always copying the data) which you described as undesirable.)



      See also the section "Symbolic links":




      If --links is specified, then symlinks are recreated with the same target on the destination.




    2. It's not the behavior you want because you're misinterpreting what --links does and therefore asking for the wrong behavior (see answer #1).



    3. By default, it copies the destination exactly.



      That is, if the link pointed to an absolute path (e.g. /home/me/projects), it'll continue pointing to the same path; it won't break, it'll just continue pointing to a file in your home directory rather than the one in your backup.



      Meanwhile, if the link pointed to a relative path (e.g. ../../projects), it'll also continue pointing to the same path, but since it's relative to the symlink's location, the symlink in your backup will also be pointing to a file in your backup.



      Unfortunately there doesn't seem to be any option to translate absolute symlinks for their new base (only an option to break them entirely). To avoid problems, you should change existing symlinks to relative ones (which is a good idea generally, for links inside $HOME).









    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Aug 18 '14 at 5:26









    grawitygrawity

    234k36494549




    234k36494549








    • 1





      ummm so what's the difference between --copy-links and --links?

      – Alexander Mills
      Dec 18 '18 at 6:57











    • --copy-links = -L, and --links = -l. See description from the documentation in my answer here: superuser.com/a/1388910/425838

      – Gabriel Staples
      Dec 30 '18 at 7:49
















    • 1





      ummm so what's the difference between --copy-links and --links?

      – Alexander Mills
      Dec 18 '18 at 6:57











    • --copy-links = -L, and --links = -l. See description from the documentation in my answer here: superuser.com/a/1388910/425838

      – Gabriel Staples
      Dec 30 '18 at 7:49










    1




    1





    ummm so what's the difference between --copy-links and --links?

    – Alexander Mills
    Dec 18 '18 at 6:57





    ummm so what's the difference between --copy-links and --links?

    – Alexander Mills
    Dec 18 '18 at 6:57













    --copy-links = -L, and --links = -l. See description from the documentation in my answer here: superuser.com/a/1388910/425838

    – Gabriel Staples
    Dec 30 '18 at 7:49







    --copy-links = -L, and --links = -l. See description from the documentation in my answer here: superuser.com/a/1388910/425838

    – Gabriel Staples
    Dec 30 '18 at 7:49















    1














    There are several very good pages which describe rsync in great detail.



    http://www.jveweb.net/en/archives/2010/11/synchronizing-folders-with-rsync.html



    http://www.mikerubel.org/computers/rsync_snapshots/index.html



    I would also recommend utilities such as backintime or rsnapshot.






    share|improve this answer



















    • 22





      Please include some relevant snippets into you currently link-only answer.

      – Cristian Ciupitu
      Sep 25 '15 at 19:08
















    1














    There are several very good pages which describe rsync in great detail.



    http://www.jveweb.net/en/archives/2010/11/synchronizing-folders-with-rsync.html



    http://www.mikerubel.org/computers/rsync_snapshots/index.html



    I would also recommend utilities such as backintime or rsnapshot.






    share|improve this answer



















    • 22





      Please include some relevant snippets into you currently link-only answer.

      – Cristian Ciupitu
      Sep 25 '15 at 19:08














    1












    1








    1







    There are several very good pages which describe rsync in great detail.



    http://www.jveweb.net/en/archives/2010/11/synchronizing-folders-with-rsync.html



    http://www.mikerubel.org/computers/rsync_snapshots/index.html



    I would also recommend utilities such as backintime or rsnapshot.






    share|improve this answer













    There are several very good pages which describe rsync in great detail.



    http://www.jveweb.net/en/archives/2010/11/synchronizing-folders-with-rsync.html



    http://www.mikerubel.org/computers/rsync_snapshots/index.html



    I would also recommend utilities such as backintime or rsnapshot.







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Apr 29 '15 at 18:48









    David PaigeDavid Paige

    33529




    33529








    • 22





      Please include some relevant snippets into you currently link-only answer.

      – Cristian Ciupitu
      Sep 25 '15 at 19:08














    • 22





      Please include some relevant snippets into you currently link-only answer.

      – Cristian Ciupitu
      Sep 25 '15 at 19:08








    22




    22





    Please include some relevant snippets into you currently link-only answer.

    – Cristian Ciupitu
    Sep 25 '15 at 19:08





    Please include some relevant snippets into you currently link-only answer.

    – Cristian Ciupitu
    Sep 25 '15 at 19:08











    0














    @grawity has a great answer, but here's a screenshot of some relevant info from the documentation. Here's the exact wording or the -l and -L options, for instance, which seem to be the most relevant ones in question:



    enter image description here



    Source: https://linux.die.net/man/1/rsync






    share|improve this answer




























      0














      @grawity has a great answer, but here's a screenshot of some relevant info from the documentation. Here's the exact wording or the -l and -L options, for instance, which seem to be the most relevant ones in question:



      enter image description here



      Source: https://linux.die.net/man/1/rsync






      share|improve this answer


























        0












        0








        0







        @grawity has a great answer, but here's a screenshot of some relevant info from the documentation. Here's the exact wording or the -l and -L options, for instance, which seem to be the most relevant ones in question:



        enter image description here



        Source: https://linux.die.net/man/1/rsync






        share|improve this answer













        @grawity has a great answer, but here's a screenshot of some relevant info from the documentation. Here's the exact wording or the -l and -L options, for instance, which seem to be the most relevant ones in question:



        enter image description here



        Source: https://linux.die.net/man/1/rsync







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Dec 30 '18 at 7:47









        Gabriel StaplesGabriel Staples

        17510




        17510






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Super User!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f799354%2frsync-and-symbolic-links%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            flock() on closed filehandle LOCK_FILE at /usr/bin/apt-mirror

            Mangá

            Eduardo VII do Reino Unido