Adminbuntu

Everything for the Ubuntu Server Administrator

User Tools

Site Tools


Sidebar

Server Administration


Server Applications


At the Command Line


Elsewhere


Copyright 2013 Applied Conscious Technologies, LLC

Terms of Agreement

Contact


submit to reddit

archiving

At the Command Line

Archiving

Tarballs

definition: http://www.wordnik.com/words/tarball

Classic Unix tar files archive multiple files and/or directories into a single file. The name came from “tape archive” shortened Unix style to “tar”.

The Gnu tar command, in addition to creating classic tar files, allows more functionality, including compression (using gzip) and retention of file ownership and permissions.

By convention, these tarred/compressed files use an extension of ”.tgz” and are called “tarballs”.

Make and Melt a Tarball of a Directory

Make the tarball

The following creates a tarball of a directory tree retaining ownership/group permissions.

sudo tar -pczf dirname.tgz dirname/

Pneumonic: Perfectly Compress Z Files

option description
p extract information about file permissions (default for superuser). Or –preserve-permissions. Or –same-permissions.
c create a new archive
z filter the archive through gzip. Or –gzip. Or –gunzip. Or –ungzip.
f use archive file or device ARCHIVE

tar man page: http://manpages.ubuntu.com/manpages/precise/man1/tar.1.html

List Users and Groups in a Directory Recursively

When you are moving a directory via tarball to another system, you will need to first create the same users and groups on the new system. The following will allow you to create lists of users and groups that own files (and directories) on the original system.

This will show the users that own files in the current directory and all subdirectories.

find . -type d | xargs ls -hl |  sed '/^$/d' | grep -v '^total' | grep -v '^\.' | awk '{ print $3 }' | sort | uniq

This will show the groups that own files in the current directory and all subdirectories.

find . -type d | xargs ls -hl |  sed '/^$/d' | grep -v '^total' | grep -v '^\.' | awk '{ print $4 }' | sort | uniq

List the Contents of the Tarball

tar -tvzf dirname.tgz
option description
t list the contents of an archive. Or –list.
v verbosely list files processed
z filter the archive through gzip. Or –gzip. Or –gunzip. Or –ungzip.
f use archive file or device ARCHIVE

tar man page: http://manpages.ubuntu.com/manpages/precise/man1/tar.1.html

List the User/Group Ownerships in the Tarball

If you are duplicating user/group ownership on a new system when melting a tarball from the original system, you will first need to create the users and groups form the original system on the new system.

This one-liner will list the users/groups that owned files on the original system:

tar -tvzf test.tgz | awk '{ print $2 }' | sort | uniq

Melt the Tarball

sudo tar -xzf dirname.tgz
  • Must be entered as super user so various ownership and groups can be created.
  • If you are restoring on a different server, first make sure that the users/groups that owned files and directories on the original server are present so the files and directories of the restored directory have the correct ownership.

Pneumonic: eXtract Z Files

option description
x Extract files from an archive. Or –extract. Or –get.
v Verbosely list files processed. Optional.
z Filter the archive through gzip. Or –gzip. Or –gunzip. Or –ungzip.
f Use archive file or device ARCHIVE

tar man page: http://manpages.ubuntu.com/manpages/precise/man1/tar.1.html

Make a Tarball of a Local dirname/ on a Remote Machine

tar -cz dirname/ | ssh user@remote 'dd of=dirname.tgz'

Copy with Permissions to Remote

( sudo tar -pcz dirname/ ) | ssh -C user@remote 'cd /where/to/ && sudo tar -xz'

List the Contents of all Tarballs in Directory

for F in $(find ./ -name "*.tgz") ; do tar -tvzf $F ; done

XKCD Break

xkcd

File Compression

Compress

If you just need to compress a file, use gzip. For example, before downloading a huge MySQL dump from your server to your laptop, SSH into the server and use gzip to compress the file first.

gzip filename

This replaces the original file with a compressed version with ”.gz” appended to the name.

Decompress

gunzip filename.gz

This decompresses the file, replacing the compressed version with a replica of the original uncompressed file, with the original filename (removes the ”.gz” extension).

archiving.txt · Last modified: 2015/05/31 21:20 (external edit)