Setting up a full Ubuntu mirror on Trusty

by Chris Glass
in Ubuntu

tagged ubuntu, packaging

Why?

As some of my readers might know, I have recently moved to live in Tanzania. While the network here is much better than in neighbouring countries (thanks to a big investment to roll out a shiny national fiber optics backbone), there is no official Ubuntu archives in the country at this point, and installing new machines daily thanks to LXC containers can get a little slow.

Since my awesome ISP is running Ubuntu on pretty much anything that is not a router, they already had a deb proxy - and they were happy to provide bandwidth to go one step further!

What you need

Installing a full Ubuntu mirror (full package archive, but also cdimages, the release images, and possibly ports of Ubuntu to other architectures) is pretty simple.

You'll need to check a few things before we get started:

  • A lot of storage space - around 1Tb without the ports, as of June 2014. While speed doesn't hurt, the most important is that it's safe and big.
  • A reasonably powerful machine, but no need for a very beefy monster either since all the machine will be doing is serving static content with apache
  • A lot of bandwidth for the initial sync, and a lot of time. Or a friend with an full mirror on a USB disk :)
  • Ideally, a set of caches between your clients and your mirror.

Using Ubumirror

The easiest way to get a full mirror is to install the ubumirror package. It basically installs a few shell scripts in /usr/bin that wrap rsync, and a config file in /etc.

Since the ubumirror in the trusty archives is unfortunately a little outdated, let's use the latest and greatest instead:

sudo add-apt-repository ppa:ubumirror-devs/ubumirror

The package was backported from utopic, so it also sits in backports, but the PPA is probably a safer bet (I'm the maintainer for ubumirror).

As usual in the Debian/Ubuntu world, it all starts with a:

sudo apt-get install ubumirror

Configuring ubumirror

That's the meat of the problem here, but hopefully the configuration should be pretty straightforward.

The four most important variables to set (rather, uncomment) are the ones controlling the actual sync: UBUARC_DIR, UBUCDI_DIR, UBUREL_DIR, UBUPOR_DIR. If one of these is not set, the mirror scripts won't sync this particular component. Make sure the directories point to a partition with plenty of space (1Tb should be just enough for deb packages and releases, but 2Tb is recommended). Ports and cdimages do take quite some space, too.

A word about components

The four components are:
  • UBUARC: The ubuntu package archive. This means you can then point your sources.list entries to your newly created mirror.
  • UBUCDI: The CD archives. This includes most cd images, including cds for flavors (kubuntu, xubuntu etc...). This includes the daily builds for the in-development release, too, so they change relatively often (daily).
  • UBUREL: The CD images for the released distribution (the main "Ubuntu" flavor).
  • UBUPOR: The ports. These are the packages for alternative architectures, such as ARM, Power, Sparc... These will use a lot of space (roughly half a Tb per architecture).

Running the initial sync

Now that the mirror is configured, it's time to start the sync process itself! This will take a long, long time. Of course, it is much faster to get the contents of /srv/mirror/ from a friend's USB disk, but even this will take a while (a USB 2 disk took more than 14 hours last time I tried):

sudo /usr/bin/ubuarchive

This should take care of the archive sync. In a similar fashion, you can run the other sync components:

sudo /usr/bin/uburelease
sudo /usr/bin/ubucdimage
sudo /usr/bin/ubuports

Setting up a regular sync

Ubumirror ships with an example cron configuration file that syncs all the components at regular, sensible intervals.

The following will copy the shipped crontab file to the relevant place:

sudo cp /usr/share/doc/ubumirror/examples/ubumirror.crontab /etc/cron.d/.

Please take a second and tweak the minutes to avoid overloading the main archives artificially at set minutes. Picking something to your liking or random is just fine.

Serving the content

We'll use Apache for this, and the configuration is pretty simple since basically it will be serving static content. As such, installing apache2 with the "worker" model is a good idea:

sudo apt-get install apache2-mpm-worker

This will pull apache2 and required dependencies.

An example apache2 configuration file that can be used almost as-is can be found in /usr/share/doc/ubumirror/examples/apache2-ubuntu-mirror.conf. You can drop it in sites-available and enable it with:

sudo cp /usr/share/doc/ubumirror/examples/apache2-ubuntu-mirror.conf /etc/apache2/sites-available/mirror.conf

Remove the default website in there and don't forget to enable the "headers" and "expires" modules, then enable the site itself:

sudo a2enmod headers expires
sudo a2ensite mirror

Restarting or reloading apache2 should do the trick.

Checking it all works

At this point, you should have a working mirror! Check by pointing your web browser to the mirror on port 80, and then by pointing your sources.list entries to it!

Don't hesitate to get in touch if you find errors or would like some more information about running your own mirror. I disabled comments on the blog for now.