Bdtyktl

You can use pv to achieve this. To report the progress correctly, pvneeds to know how much bytes you are throwing at it. So, the first step is to calculate the size (in kbyte). You can also completely drop the progress bar and just let pv tell you how much bytes it has seen; it would report a 'done that much and that fast'.

% SIZE=`du -sk folder-with-big-files | cut -f 1`

And then:

% tar cvf - folder-with-big-files | pv -p -s ${SIZE}k |  

     bzip2 -c > big-files.tar.bz2

better progress bar..

apt-get install pv dialog



(pv -n file.tgz | tar xzf - -C target_directory ) 

2>&1 | dialog --gauge "Extracting file..." 6 50

Check out the --checkpoint and --checkpoint-action options in the tar info page (as for my distribution, the description for these options is not contained in the man page → RTFI).

See https://www.gnu.org/software/tar/manual/html_section/tar_26.html

With these (and maybe the functionality to write your own checkpoint command), you can calculate a percentage…

Inspired by helper’s answer

Another way is use the native tar options

FROMSIZE=`du -sk ${FROMPATH} | cut -f 1`;

CHECKPOINT=`echo ${FROMSIZE}/50 | bc`;

echo "Estimated: [==================================================]";

echo -n "Progess:   [";

tar -c --record-size=1K --checkpoint="${CHECKPOINT}" --checkpoint-action="ttyout=>" -f - "${FROMPATH}" | bzip2 > "${TOFILE}";

echo "]"

the result is like

Estimated: [==================================================]

Progess:   [>>>>>>>>>>>>>>>>>>>>>>>

a complete example here

Just noticed the comment about MacOS, and while I think the solution from @akira (and pv) is much neater I thought I'd chase a hunch and a quick playaround in my MacOS box with tar and sending it a SIGINFO signal. Funnily enough, it worked :) if you're on a BSD-like system, this should work, but on a Linux box, you might need to send a SIGUSR1, and/or tar might not work the same way.

The down side is that it will only provide you with an output (on stdout) showing you how far through the current file it is since I'm guessing it has no idea about how big the data stream it's getting is.

So yes, an alternative approach would be to fire up tar and periodically send it SIGINFOs anytime you want to know how far it's gotten. How to do this?

The ad-hoc, manual approach

If you want to be able to check status on an ad-hoc basis, you can hit control-T (as Brian Swift mentioned) in the relevant window which will send the SIGINFO signal across. One issue with that is it will send it to your entire chain I believe, so if you are doing:

% tar cvf - folder-with-big-files | bzip2 -c > big-files.tar.bz2

You will also see bzip2 report it's status along with tar:

a folder-with-big-files/big-file.imgload 0.79  cmd: bzip2 13325 running 

      14 0.27u 1.02s 



      adding folder-with-big-files/big-file.imgload (17760256 / 32311520)

This works nicely if you just want to check if that tar you're running is stuck, or just slow. You probably don't need to worry too much about formatting issues in this case, since it's only a quick check..

The sort of automated approach

If you know it's going to take a while, but want something like a progress indicator, an alternative would be to fire off your tar process and in another terminal work out it's PID and then throw it into a script that just repeatedly sends a signal over. For example, if you have the following scriptlet (and invoke it as say script.sh PID-to-signal interval-to-signal-at):

#!/bin/sh



PID=$1

INTERVAL=$2

SIGNAL=29      # excuse the voodoo, bash gets the translation of SIGINFO, 

               # sh won't..



kill -0 $PID   # invoke a quick check to see if the PID is present AND that

               # you can access it..



echo "this process is $$, sending signal $SIGNAL to $PID every $INTERVAL s"

while [ $? -eq 0 ]; do

     sleep $INTERVAL;

     kill -$SIGNAL $PID;    # The kill signalling must be the last statement

                            # or else the $? conditional test won't work

done

echo "PID $PID no longer accessible, tar finished?"

If you invoke it this way, since you're targeting only tar you'll get an output more like this

a folder-with-big-files/tinyfile.1

a folder-with-big-files/tinyfile.2

a folder-with-big-files/tinyfile.3

a folder-with-big-files/bigfile.1

adding folder-with-big-files/bigfile.1 (124612 / 94377241)

adding folder-with-big-files/bigfile.1 (723612 / 94377241)

...

which I admit, is kinda pretty.

Last but not least - my scripting is kinda rusty, so if anyone wants to go in and clean up/fix/improve the code, go for your life :)

Inspired by Noah Spurrier’s answer

function tar {

  local bf so

  so=${*: -1}

  case $(file "$so" | awk '{print$2}') in

  XZ) bf=$(xz -lv "$so" |

    perl -MPOSIX -ane '$.==11 && print ceil $F[5]/50688') ;;

  gzip) bf=$(gzip -l "$so" |

    perl -MPOSIX -ane '$.==2 && print ceil $F[1]/50688') ;;

  directory) bf=$(find "$so" -type f | xargs du -B512 --apparent-size |

    perl -MPOSIX -ane '$bk += $F[0]+1; END {print ceil $bk/100}') ;;

  esac

  command tar "$@" --blocking-factor=$bf 

    --checkpoint-action='ttyout=%u%r' --checkpoint=1

}

Source

Using only tar

tar has the option (since v1.12) to print status information on signals using --totals=$SIGNO, e.g.:

tar --totals=USR1 -czf output.tar input.file

Total bytes written: 6005319680 (5.6GiB, 23MiB/s)

The Total bytes written: [...] information gets printed on every USR1 signal, e.g.:

pkill -SIGUSR1 tar

Source:

• gnu.org - GNU tar - 3.7 Checking tar progress


• gnu.org - tar - NEWS

If you known the file number instead of total size of all of them:

an alternative (less accurate but suitable) is to use the -l option and send in the unix pipe the filenames instead of data content.

Let's have 12345 files into mydir, command is:

[myhost@myuser mydir]$ tar cfvz ~/mytarfile.tgz .|pv -s 12345 -l > /dev/null 

you can know such value in advance (because of your use case) or use some command like find+wc to discover it:

[myhost@myuser mydir]$ find | wc -l

12345

Method based upon tqdm:

tar -v -xf tarfile.tar -C TARGET_DIR | tqdm --total $(tar -tvf tarfile.tar | wc -l) > /dev/null