Using Amazon Glacier on Linux

Amazon Glacier

Amazon provides a low cost backup service called Amazon Glacier which a payment plan reflecting the amount of data that you store with a price in the range of $ 0.01 per GB. However beware of the other costs for requests and especially the fee related to requesting and downloading data from the remote storage.

It might not be strait forward to calculate the actual cost but with the Amazon calculator it is possible to give a pretty good estimate. It roughly seems like the cost of retrieving your data matched what you pay a month to store it.

The service is available through the Amazon Web Services interface where you can signup and joint the service.

Using Amazon Glacier on Linux

Using Amazon Glacier is pretty easy on Windows through e.g. FastGlacier or on Mac with Arq. However, if you are running Linux, in my case Gentoo, with no desktop then the number of clients to choose from is very limited when it comes to being able to sync folders without having to re-upload it all. I found a few options mainly on github.

I found the project mt-aws-glacier project which seemed to do the job.

Synchronisation can be done with a config file that contains the essentials like:

key=GLACIER_KEY
secret=GLACIER_SECRET_KEY
region=us-east-1
protocol=http
dir=/mnt/disk/Photography
vault=photography
journal=/mnt/disk/journals/photography-journal.log
concurrency=10

and then using it with mtglacier like:

mtglacier sync --delete-removed --replace-modified --new  --config=configs/photography.cfg &>logs/photography.log &

This will sync the /mnt/disk/Photography folder with vault ‘photography’ and store synchronisation data in the photography-journal.log file. This log file can be reconstructed but it may be easier (and cheaper) just to store it in e.g. amazon S3 since it is relatively small. This can be done with ‘s3cmd’ which also is able to sync one or more files. The following command syncs the journals folder to a remote vault in S3:

s3cmd sync /mnt/disk/journals/ s3://journals