Monday, 23 August 2010

Backing Up My Life With Amazon Web Services

Amazon's Simple Storage System or S3 for short, is what I have decided to use from now on (for now until I discover yet something else) to backup my life.

S3 seems to be the solution I have been looking for for ages. A simple, flexible, no-contract, reliable and easily accessible storage facility that resides entirely in the cloud. Its cost is based on exactly what you use; rather than renting a fixed amount of space on some server, you just pay AWS for what you use of their services. So if you upload N amount of data you will be charged for storing N amount of data per month. If you access that data for COPYing GETting or PUTting commands then you will be charge per command. So you can use it as little or as much as you like.

There is a perfect little application called s3sync which is a small ruby program which syncs data you specify to S3. It works very much like rsync. This is ideal as it is a command line program so I can run it on my web server which contains a lot of data I wouldn't like to loose, such as family photos/videos.

I altered my backup script to now use this new method, based on a great tutorial I found on the web, which is dated as 2006, so this has been around for a long time.
#!/bin/bash #Backup script for server

#set variable of date for labelling date=`date +%F`

cd /home/user/backupdata/

#remove oldest mysql backup rm `ls -t *mysql* | tail -n 1`

#Dump mysql databases mysqldump --all-databases > /home/user/backupdata/${date}_mysql_backup

cd /home/user/bin/s3sync/ export AWS_ACCESS_KEY_ID=My Amazon Access Key ID
export AWS_SECRET_ACCESS_KEY=My Amazon Secret Access Key
export SSL_CERT_DIR=/home/user/bin/s3sync/certs
export AWS_CALLING_FORMAT=SUBDOMAIN
ruby s3sync.rb -r -v --ssl --exclude="/home/user/some/dir" --delete /home/user mybucket:
ruby s3sync.rb -r -v --ssl --delete /var/www mybucket:


I have this setup as a cron job once a week.

No comments:

Post a Comment