Building Debian/Ubuntu packages using SCons

Background

At CamVine we’ve been using the scons build utility as a replacement for our old make-based system. Sometimes it can be slightly non-intuitive, but we haven’t had anything like the depths of confusion that can occur in a big complex system using make. If you haven’t tried scons, I’d recommend it. I’d also recommend that you stop reading this and go and learn how to use it first, because this will assume some familiarity.

We make extensive use of Ubuntu, so one thing we need to do on a regular basis is build Debian packages, and we wanted to do this under the control of scons. Initially, we tried customising the ‘install’ target. That seems like a sensible thing to do, because the Install() builder was scattered through our directories and knew where to put things on the local machine. But the process becamse rather messy, especially when we needed to build several different packages from the one tree.

So here’s how we do it now. This is a recent change, and will no doubt be improved and updated a great deal over time, but might help somebody else.

Building .deb packages

There are various ways to build Debian packages - we’ll use the dpkg-deb utility. In its simplest use, you create a tree which includes the files to be packaged in the places you want them in the final filesystem, and also has a DEBIAN/control file giving meta-information about the package. You might, for example, end up with a directory structure like this:

    mypkg / DEBIAN / control
    mypkg / usr / bin / myutility
    mypkg / etc / myutility / myutility.conf

and then, in the parent directory, you’d run something like

  fakeroot dpkg-deb -b mypkg myutility_0.1-1_i386.deb

to create the package. The ‘fakeroot’ lets you put things in the package which, when installed, will be owned by root.

We decided to do all of this within a ‘deb’ subdirectory of our top-level SCons tree. The deb directory would just contain an SConscript which knew how to build subdirectories, pkg1, pkg2 etc (or whatever your package was named), copy the files in from the other bits of the tree, create the control file and run dpkg-deb.

Doing it with SCons

I wanted to be able to build the package simply by typing:

  scons debian

so I added the following to the top-level SConstruct:

  if 'debian' in COMMAND_LINE_TARGETS:
      SConscript("deb/SConscript")

If I don’t specify ‘debian’, the whole deb directory is ignored. You may wish to include it by default.

The deb/SConscript file

OK, so what do we put in deb/SConscript?

import os, shutil, sys
Import('env') # exported by parent SConstruct

# I wanted to base the debian version number partly on the
# revision checked out from our SVN repository. 
# Skip this if it's not relevant to you.
svn_version = os.popen('svnversion ..').read()[:-1]
# This may be as simple as '89' or as complex as '4123:4184M'.
# We'll just use the last bit.
svn_version = svn_version.split(':')[-1]


# Here's the core info for the package

DEBNAME = mypkg
DEBVERSION = "0.01"
DEBMAINT = "Quentin Stafford-Fraser [me@myaddr.org]"
DEBARCH = "i386"
DEBDEPENDS = "other-package1, other-package2" # what are we dependent on?
DEBDESC = "A really cool utility"

DEBFILES = [

    # Now we specify the files to be included in the .deb
    # Where they should go, and where they should be copied from.
    # If you have a lot of files, you may wish to generate this 
    # list in some other way.
    ("usr/bin/myutility",             "#src/myutility/myutility"),
    ("etc/myutility/myutility.conf",  "#misc/myutility.conf"),

]
    
# This is the debian package we're going to create
debpkg = '#%s_%s-%s_%s.deb' % (DEBNAME, DEBVERSION, svn_version, DEBARCH)

# and we want it to be built when we build 'debian'
env.Alias("debian", debpkg)

DEBCONTROLFILE = os.path.join(DEBNAME, "DEBIAN/control")

# This copies the necessary files into place into place.
# Fortunately, SCons creates the necessary directories for us.
for f in DEBFILES:
    # We put things in a directory named after the package
    dest = os.path.join(DEBNAME, f[0])
    # The .deb package will depend on this file
    env.Depends(debpkg, dest)
    # Copy from the the source tree.
    env.Command(dest, f[1], Copy('$TARGET','$SOURCE'))
    # The control file also depends on each source because we'd like
    # to know the total installed size of the package
    env.Depends(DEBCONTROLFILE, dest)

# Now to create the control file:

CONTROL_TEMPLATE = """
Package: %s
Priority: extra
Section: misc
Installed-Size: %s
Maintainer: %s
Architecture: %s
Version: %s-%s
Depends: %s
Description: %s

"""
env.Depends(debpkg,DEBCONTROLFILE )

# The control file should be updated when the SVN version changes
env.Depends(DEBCONTROLFILE, env.Value(svn_version))

# This function creates the control file from the template and info
# specified above, and works out the final size of the package.
def make_control(target=None, source=None, env=None):
    installed_size = 0
    for i in DEBFILES:
        installed_size += os.stat(str(env.File(i[1])))[6]
    control_info = CONTROL_TEMPLATE % (
        DEBNAME, installed_size, DEBMAINT, DEBARCH, DEBVERSION,
        svn_version, DEBDEPENDS, DEBDESC)
    f = open(str(target[0]), 'w')
    f.write(control_info)
    f.close()
    
# We can generate the control file by calling make_control
env.Command(DEBCONTROLFILE, None, make_control)

# And we can generate the .deb file by calling dpkg-deb
env.Command(debpkg, DEBCONTROLFILE,
            "fakeroot dpkg-deb -b %s %s" % ("deb/%s" % DEBNAME, "$TARGET"))

Here, the DEBFILES list of which files should be included in the package and where is defined in this file in the deb directory. Others might argue that this information should be located alongside each file in the main source tree. Personally, with a simple package, I like it here, because you could conceivably be building a lot of different package types on a lot of different platforms, and this method keeps everything to do with a package together and leaves the SConscripts in the source directories nice and tidy. But it would be easy to change if you wanted it the other way around.

There are many loose ends to be tidied up here and many things which can be improved. Recommendations welcome! Our version is more complex, for example, because we build different packages using:

  scons DEB=mypkg2 debian

But this, I hope, should get you started.

*Quentin Stafford-Fraser
August 2007
Updated Feb 09 to mention fakeroot - thanks to Neil Hooey.
*