Creating a Simple Subversion Server on Ubuntu

For a while I have been trying to learn some programming skills. It is something I would like to do purely for fun and if I can get to a stage where I have decent programming capabilties then I can enjoy writing applications and perhaps start giving something useful back to the Linux community.

To start with the best intentions, I decided to set up a simple Subversion version control system on my server to allow me to keep control of my programming progress in an orderly manner. This blog entry just simply describes how I went about setting it up.

Subversion is a version control system which one can use to keep track of a project and have the ability to 'check out' stuff to work on then 'check in' the changes, keeping old revisions of work in case you need to revert back to an earlier version. This is great, especially when learning, as you don't need to keep saving new copies of your program as it develops every time you want to try something new in case you mess it up and need to revert back to your previous version.

You work on one file (or multiple) and keep checking in your changes, creating many versions which you can always fall back on, but in your working area, you just see the latest version.

I am sure more experienced users may well spot flaws in my set up, but for now, as a beginner, it works well enough for me.

Firstly, I am using an Ubuntu 8.04 LTS server on a VPS. To set up subversion, I followed the following steps:

Install and Setting up Subversion

1. Firstly, install subversion, easily done with: sudo apt-get install subversion
2. Then set up a location where subversion will create and manage your projects. I created a directory in my home directory called 'svn'.
3. Then create another directory within 'svn' to hold the repositories, e.g. 'svn/repos'.
4. Then create a group for subversion, in this case I chose 'subversion'. I used the command (as root) addgroup subversion
5. Then change the ownership of the directory 'svn' and all within to the new group: chgrp -R subversion /path/to/svn
6. Next add write permissions of memebers of the group to be able to write to 'repos' directory: chmod g+w /path/to/svn/repos
7. Next, to make sure everything created within svn/ is owned by the group: chmod g+s /path/to/svn/repos
8. Then add users that you want to allow to use subversion to the group: adduser <user> subversion

Creating Repositories

Now we can add some projects or repositories to subversion to work with. A project can contain a single file or multiple files, so for one of my first projects, I have multiple different programs where I have just been experimenting; they are all under one project or repository.

To create repositories:

1. Go to the repo directory: cd /path/to/svn/repos
2. Before creating a repository, set the umask to '002'. This allows the following created files to have the correct permissions: umask 002
3. To create a repository called 'project1' execute: svnadmin create /path/to/svn/repos/project1
4. Reset umask: umask 022
5. Now the new repo is created, go to some location elsewhere (you don't work in the repo directory, this is where subversion keeps all its data) and checkout the initial version: cd /path/to/somewhere/eg/home/user/wip If working locally, i.e. on the server where subversion server is running: svn co file:///path/to/svn/repos/project1 This will check out the current revision of the project, which at this stage is revision 0. It will create a directory named 'project1' wherever you perform the command.

Now you can create or copy existing work into this directory and do whatever you want until it is time to check your work in.

Working with Repositories

As we just checked out the first version of project1 and added some files into it, we now need to check in the work to save it in is current state.

Firstly we need to 'add' the files to the project. Working within the working directory: svn add file1 file2 etc This should confirm it has added the selected files to the project.

Next we have to 'commit' the changes. This saves the data of the current iteration of the project to the subversion server.

svn commit -m "Add a note here to describe your changes" This should confirm to you it is adding files and uploading them and its current revision, in this case, 'revision 1'.

Now another user (who is in the subversion group) can check out the latest revision somewhere else on the system by performing the same commands and it will create a working directory and copy all the files we initially added for them to work on, at revision 1.

If someone else does work on the project and updates it with changes, we need to be able to get the latest revision to work on. To update your local working copy to the latest revsion, simply go to the working directory, e.g. /path/to/wip/project1 (not in svn/repos/) and: svn update This will pull down and overwrite the local copies with the latest revision.

Checking Out Remotely

More than often, you might want to work on your project on a local machine, i.e. not logged into the server, especially if working on GUI apps. The way I am doing it is via ssh. This means that only users with an account on my server and ssh access can download repos.

I do this with the following command (after setting up and getting into a suitable working directory locally): svn co svn+ssh://servername.com/path/to/svn/repos/projectname This then prompts me for my ssh password twice and then downloads the latest revison of the project to my local directory.

I can then use the same commands to add, commit and update this directory.

Non Standard SSH Port

One note worth adding, I use a non-standard port for ssh access to my server. To allow for this when using the ssh method I needed to edit ~/.subversion/config

Find the section titled '[tunnels]' and uncomment/add/edit the line containing 'sshtunnel' within to look like: ssh = ssh -p <your port number>

This will allow the svn+ssh method to work with your non-standard port number.

Hope this helps someone and also if more experienced users read this and think I am doing things wrong or giving out misleading information, please do let me know so I can update this information and also my own set up!

2012 Panic Article

A small article from a newspaper (not sure which) that my father sent me highlighting how people can easily get themselves worried about mythological events with no evidence to back them up, yet fail to bat an eyelid at 'real' problems happening around them. i.e. crime, global warming, nuclear war etc...

Panoramic Photo Stitching With Hugin

Update 19 September 2013: I have since updated the gallery page. As I am moving away from using Drupal, some or all of the links below may not work any more. I made a simple gallery page which makes use of Brain Maps API to emulate the Zoomify function. I also used the great jQuery Masonry library to layout the gallery page.

Something I discovered a while back was stitching photos together to form panoramas. I have been trying out a few examples, which are shown in the pano gallery. Not much of a great selection, but I keep meaning to do more.

Customising the Drupal Theme of my Site

At last I have finally got round to doing a bit of theme development on my site! I have been meaning to get round to this ever since I first installed Drupal and promoted it to being my default website.

For too long I was stuck with the default Garland theme which everyone knows is just the default theme that comes out of the box with Drupal and looks a bit poor as it means you haven't bothered to change it and make it your own. Not that I am saying 'looks' are what make a good site, obviously content is rather important (and yes, I know mine is still lacking in it, but I am working on it).

I have had a look at it briefly on and off since getting drupal running, and kept getting scared off by the complexity of the seeming maze of css style sheets and templates, but after deciding to get my head around it finally I found some useful methods for easy theme development... in a very basic way of course.

Help from the drupal forum and also people in the irc channels (#drupal and #drupal-support on the freenode network) really went far in getting me started. The best bit of advice was to use a nice basic starting theme, which has a very basic layout, giving you an effectively blank canvas, from a CSS styling point of view.

The theme I started with was the zen theme which has excellent documentation to get you started.

Then I just changed my 'admin theme' (Administer > Site Config > Admin theme) to my new theme, so that I could play around with it whilst just in the admin pages, leaving the public part of the site in tact until my new theme was ready.

Then basically I just made good use of the Firefox addon web developer, which is perfect for interrogating a website to find out what CSS style sheet is driving what bit of page, so you can gradually track down all the bits you need to alter and create your own custom style sheets.

One thing to note was that to control the navigation menu, I had to copy the system-menus.css from /modules/system into my theme directory, rename it and add the entry into my theme.info file to make use of it.

So I am pleased with myself now. I have learnt a bit more, which is never a bad thing.

Edit: I don't like the gradient bar at the top at all any more!

My simple Website Backup Method

Since playing around with this new drupal site, plus I have a lot of my personal photos stored in the gallery I really wanted to have some sort of backup strategy, so for now I am using lftp to send the selected data to a NAS device that is connected to my home network.

I am using FTP on the NAS device as it means I don't need to have a computer on to send data to it. I like lftp as it lets you write a script of commands as a queue of commands to perform; so you can launch lftp with the -f option and specify a simple text file with all the commands you wish to perform.

I backup the contents of my /home and /var/www/. Also, before backing this up, I tar up /etc and dump the mysql databases into a directory in home/.

I put all this into a little script which runs once a week using cron.

Here is my script:

#!/bin/bash #Backup script for server #set variable of date for labelling date=`date +%F` cd /home/jonr/backupdata/ #remove oldest etc backup rm `ls -t *etc-backup* | tail -n 1` #Backup latest /etc/ tar -czvf /home/jonr/backupdata/${date}-etc-backup.tar /etc/ #remove oldest mysql backup rm `ls -t *mysql* | tail -n 1` #Dump mysql databases mysqldump --all-databases > /home/jonr/backupdata/${date}_mysql_backup #Backup to home NAS using lftp lftp -f /home/jonr/bin/jcr-lftp-script #Calculate the size of the amount of data sent during the backup from lftp's transfer log. grep $date /home/jonr/.lftp/transfer_log | sed -r s/^.+0-// | sed -r s/\ [0-9].+$// > temp.log i=0 for n in `cat temp.log` do i=`expr $i + $n` done echo $i'b' rm temp.log exit

Hopefully most of the comments explain what I am doing. Some things to note:

tar of /etc is being run as user 'jonr', not root, therefore there are some files/directories that jonr does not have permission to tar... I need to sort this out.

To perform the mysqldump, it requires root permissions, normally on the command line you would enter the -uroot and -p options to act as root, so to avoid using these options in plain text in my script you can create a file called '.my.cnf' in your home directory.

This simply (in my case) contains this info:

[client] user = root password = (password)

So when any mysql command is called without -u -p options, it uses these credentials instead.

Similarly, for lftp, if I create a file called .netrc in my home directory containing:

machine server.address.com login jonr password (password)

...when lftp is launched it can be requested to connect to 'server.address.com' and will not ask for a user and password as it finds it in .netrc

Here is the contents of the lftp script I call:

open server.address.com cd PUBLIC/jcrdev/ mirror -Ren /home/jonr/ home-backup mirror -Ren /var/www/ varwww-backup exit

Simply, it connects using the above credentials, changes to the appropriate directory, then uses the mirror command to 'put' data on th FTP server.

The -R option means 'reverse mirror', i.e. 'put' data instead of 'get' data. So put it on the FTP drive, rather than pull it off the FTP drive.

The -e option deletes files on the FTP drive which are not present in the source.

The -n option puts only NEWER data on the FTP drive, so only things that have changed basically get transferred.

This is basically behaving like rsync, only transferring data that it needs to to keep it up to date. I just can't use rsync to an FTP drive.

Things I need to improve on:

Don't use FTP (not very secure - SFTP better) Configure certain commands, like tar'ing up /etc to allow root priviledges without transmitting password)

If anyone spots problems or just wants to feedback some comments/advice, please do, I would appreciate it.

Installing Edimax EW-7318Ug Wireless USB Dongle in Ubuntu 9.04

I recently installed Ubuntu 9.04 on an old Dell Latitude laptop and wanted it to be wireless capable. This laptop did not have inbuilt wireless capabilities so I wanted to get a USB dongle for it. I thought most wireless USB dongle would just plug'n'play with Ubuntu but as it turns out this one did not. If you are looking to buy a dongle, do a bit of research first to find one that is known to 'just work' as it should be as simple as plugging it in and it gets detected and just works. As it turned out, either I did not do my research well enough or my hardware was just being funny as this dongle did not just work. One thing I had going for me though was the manufacturer listed it as Linux compatible. Here is a page listing known working devices for Ubuntu. Anyway, my memory is a bit fuzzy, but this is basically how I got it working. Firstly, check it was being recognised ok with:

lsusb

If it is recognised ok, check:

dmesg

to see if any errors occured when it was plugged in. Perhaps check dmesg before and after plugging it in. On mine I had some lines like this:

[ 613.008118] usb 1-1: new full speed USB device using uhci_hcd and address 3 [ 613.310683] usb 1-1: configuration #1 chosen from 1 choice [ 613.431044] phy2 -> rt2500usb_init_eeprom: Error - Invalid RT chipset detected. [ 613.431057] phy2 -> rt2x00lib_probe_dev: Error - Failed to allocate device.

This indicated that it was using rt2500 drivers which were invalid. So I blacklisted the rt2500 driver by adding 'rt2500usb' to the end of /etc/modprobe/blacklist.conf Next I went and got the latest rt73 driver from Ralink. I installed this following the readme instructions that came with it. I didn't bother with the included wpasupplicant as this was already installed by Ubuntu. Once the driver had compiled I had to execute:

sudo modprobe rt73

Then I executed:

sudo ifconfig rausb0 inet up

to bring the interface up. Next it got a bit wierd... in the end after making sure everything was updated and upgraded it finally worked. A reboot was necessary and helpful. Finally Network Manager detected my network and allowed me to connect. Hope this is of a bit of use to someone.

My Second Blog

Second blog... Mainly to see if my blog page is working. I have built this blog page using drupal's 'views' module, whereby creating a 'view' that is filtered on certain key things. I create a blog entry as a 'story' and just ensure that it has 'blog' as a tag. Views then filters all 'published', 'story' pages with 'blog' as a tag. Seems to work rather neat... well I will see when I finish typing this entry. Also got postfix and procmail playing together nicely to send mail via a spamassassin server and then redirect 'spam' into a spam folder. My IMAP mail server is working nicely at the mo. Will post an entry about how I set it up soon.

My First Blog

This is my first ever blog. Blogging seems to have been around forever and I have never felt the need to try it out. Mostly because I don't feel I have much to say that the world is interested in. But I thought I would give it a go! For me mainly I think I will be blogging about the usual; things I do/like, but also I may well use it as a place to document the things I am learning, particularly about computing. Computing is probably my main hobby at the moment. I have been using Linux for just over a year now and am loving it. I wish I had done it so many years ago as I feel I have a lot to learn. But that's what I like about it; there is a never ending source of things to learn. I kinda started years ago with a basic website, which was primarily just an online photo gallery using the Menalto project Gallery, which I still use to this day (testing beta 3 at the mo). But I would like to expand and learn more. I am posting this first blog entry on my own web server. I acquired a VPS and am using Ubuntu 8.04 LTS Hardy Heron on it and am learning how to run my own server. Has been fun so far setting up Apache, and a mail server etc. and now, Drupal. As you can see the site is very much the standard theme at the moment, but hopefully will get round to creating my own. Also will get round to adding the old site content. I am also trying to learn Python, the programming language, so I will probably post about my experiments with that. But bare in mind, I am very new to all this so don't think I am trying to show off any skills, it is more as a record of my progression, so perhaps I will look back and see how poor I was in the beginning. Anyway, that's enough for now.