February 24, 2012

tar, gzip, and ftp with pipes

Using pipes in Unix can save a developer a lot of time with scripting and be more productive. Here's an example where you have compressed your postscript file, Layout.psd and now you want to see it without uncompressing it. You could use:

zcat Layout.ps.Z | gv - gunzip -c Layout.ps.gz | gv -

Of course, you could pipe this output into anything you wanted!

Tar Examples

One of the most useful applications of this is to move a directory as well as all its files and subdirectories to another location. This is done by creating a tar file and piping it into a subshell which runs in a different directory and untar's stdin. That is,

tar cvf - . | (cd ~/newdir ; tar xvf -)

GZip Examples

Finally, you can compress stdin and send it to stdout, or uncompress stdin and send it to stdout, using gzip and gunzip alone in a pipe. That is, these pairs are equivalent.

gunzip -c foo.gz > bar cat foo.gz | gunzip > bar

gzip -c foo > bar.gz cat foo | gzip > bar.gz

Combining Tar GZIP and FTP/sFTP

Now this combination is very useful due to a little know property of ftp. ftp allows you to specify pipes as source or receving files. For instance, you can get and view a gif image from an ftp site with

ftp> get file.gif "| xv -"

or view a file with your favorite page using

ftp> get README "| more"

This is useful, but you can also use this trick to create a tar file onto an ftp site without making that tar file on your local disk. This is invaluable for backup processes. An example of this is

ftp> put "| tar cvf - ." myfile.tar

And to retrieve and untar, use

ftp > get myfile.tar "| tar xvf -"

Or, to send and compress a tar file onto an ftp site, you can use this:

ftp> put "| tar cvf - . | gzip " myfile.tar.gz

That is, ftp makes a transfer transfer file dest by effectively taking the stdout of cat file and piping this into dest.


Add your Comments

Feel free to leave a comment about this post.

- Feel Free to add HTML to your comment!