Thursday, 27 August 2009

My simple Website Backup Method

Since playing around with this new drupal site, plus I have a lot of my personal photos stored in the gallery I really wanted to have some sort of backup strategy, so for now I am using lftp to send the selected data to a NAS device that is connected to my home network.

I am using FTP on the NAS device as it means I don't need to have a computer on to send data to it. I like lftp as it lets you write a script of commands as a queue of commands to perform; so you can launch lftp with the -f option and specify a simple text file with all the commands you wish to perform.

I backup the contents of my /home and /var/www/. Also, before backing this up, I tar up /etc and dump the mysql databases into a directory in home/.

I put all this into a little script which runs once a week using cron.

Here is my script:

#!/bin/bash #Backup script for server #set variable of date for labelling date=`date +%F` cd /home/jonr/backupdata/ #remove oldest etc backup rm `ls -t *etc-backup* | tail -n 1` #Backup latest /etc/ tar -czvf /home/jonr/backupdata/${date}-etc-backup.tar /etc/ #remove oldest mysql backup rm `ls -t *mysql* | tail -n 1` #Dump mysql databases mysqldump --all-databases > /home/jonr/backupdata/${date}_mysql_backup #Backup to home NAS using lftp lftp -f /home/jonr/bin/jcr-lftp-script #Calculate the size of the amount of data sent during the backup from lftp's transfer log. grep $date /home/jonr/.lftp/transfer_log | sed -r s/^.+0-// | sed -r s/\ [0-9].+$// > temp.log i=0 for n in `cat temp.log` do i=`expr $i + $n` done echo $i'b' rm temp.log exit

Hopefully most of the comments explain what I am doing. Some things to note:

tar of /etc is being run as user 'jonr', not root, therefore there are some files/directories that jonr does not have permission to tar... I need to sort this out.

To perform the mysqldump, it requires root permissions, normally on the command line you would enter the -uroot and -p options to act as root, so to avoid using these options in plain text in my script you can create a file called '.my.cnf' in your home directory.

This simply (in my case) contains this info:

[client] user = root password = (password)

So when any mysql command is called without -u -p options, it uses these credentials instead.

Similarly, for lftp, if I create a file called .netrc in my home directory containing:

machine login jonr password (password)

...when lftp is launched it can be requested to connect to '' and will not ask for a user and password as it finds it in .netrc

Here is the contents of the lftp script I call:

open cd PUBLIC/jcrdev/ mirror -Ren /home/jonr/ home-backup mirror -Ren /var/www/ varwww-backup exit

Simply, it connects using the above credentials, changes to the appropriate directory, then uses the mirror command to 'put' data on th FTP server.

The -R option means 'reverse mirror', i.e. 'put' data instead of 'get' data. So put it on the FTP drive, rather than pull it off the FTP drive.

The -e option deletes files on the FTP drive which are not present in the source.

The -n option puts only NEWER data on the FTP drive, so only things that have changed basically get transferred.

This is basically behaving like rsync, only transferring data that it needs to to keep it up to date. I just can't use rsync to an FTP drive.

Things I need to improve on:

Don't use FTP (not very secure - SFTP better) Configure certain commands, like tar'ing up /etc to allow root priviledges without transmitting password)

If anyone spots problems or just wants to feedback some comments/advice, please do, I would appreciate it.

Tuesday, 4 August 2009

Installing Edimax EW-7318Ug Wireless USB Dongle in Ubuntu 9.04

I recently installed Ubuntu 9.04 on an old Dell Latitude laptop and wanted it to be wireless capable. This laptop did not have inbuilt wireless capabilities so I wanted to get a USB dongle for it. I thought most wireless USB dongle would just plug'n'play with Ubuntu but as it turns out this one did not. If you are looking to buy a dongle, do a bit of research first to find one that is known to 'just work' as it should be as simple as plugging it in and it gets detected and just works. As it turned out, either I did not do my research well enough or my hardware was just being funny as this dongle did not just work. One thing I had going for me though was the manufacturer listed it as Linux compatible. Here is a page listing known working devices for Ubuntu. Anyway, my memory is a bit fuzzy, but this is basically how I got it working. Firstly, check it was being recognised ok with:
If it is recognised ok, check:
to see if any errors occured when it was plugged in. Perhaps check dmesg before and after plugging it in. On mine I had some lines like this:
[ 613.008118] usb 1-1: new full speed USB device using uhci_hcd and address 3 [ 613.310683] usb 1-1: configuration #1 chosen from 1 choice [ 613.431044] phy2 -> rt2500usb_init_eeprom: Error - Invalid RT chipset detected. [ 613.431057] phy2 -> rt2x00lib_probe_dev: Error - Failed to allocate device.
This indicated that it was using rt2500 drivers which were invalid. So I blacklisted the rt2500 driver by adding 'rt2500usb' to the end of /etc/modprobe/blacklist.conf Next I went and got the latest rt73 driver from Ralink. I installed this following the readme instructions that came with it. I didn't bother with the included wpasupplicant as this was already installed by Ubuntu. Once the driver had compiled I had to execute:
sudo modprobe rt73
Then I executed:
sudo ifconfig rausb0 inet up
to bring the interface up. Next it got a bit wierd... in the end after making sure everything was updated and upgraded it finally worked. A reboot was necessary and helpful. Finally Network Manager detected my network and allowed me to connect. Hope this is of a bit of use to someone.