next up previous

Documentation index

The Nemesis Build System
Users' Guide and Reference Manual
7th Edition1

Dickon Reed

April 19992


This manual takes off where the release notes finish. Please refer to the Nemesis release notes first to discover how to obtain Nemesis.


Nemesis contains a very flexible build system and the flexibility may be overwhelming. However, many features are optional and will be introduced later in this document.

Many different projects are built using variations on the make utility supplied with Unix. make is a tool for managing building a large set of programs and is designed to automatically determine at any stage what files need recompiling and invoke them. Nemesis is built using the GNU version make.

Nemesis is too large to be easily managed if it were all in one directory. Therefore Nemesis is stored in multiple directories, and we use the recursion features of make to handle generating multiple directories. To build Nemesis it is necessary to build hundreds of modules (like shared libraries) using hundreds of interfaces, header libraries and a handful of static libraries. Each module and static library is given its own directory. To make it easier for people to find modules, there is a hierachy of directories, sorting the things that compromise Nemesis by theme. This hiearchy of directories is known as a Nemesis tree, and Nemesis trees that are used for building Nemesis modules and boot images and so forth are known as build trees.

In addition, certain things cannot be handled on a per directory basis. For instance, we write system interfaces in a special language called MIDDL. We also need to link a selection of modules together to form a boot image and we need easy access to complied modules at run time so they can be dynamically loaded. Thus, we end up with several different places in which information about a module is needed. To avoid storing information about modules in many different places, we have a database called the blueprint that stores various pieces of data about various modules in one place. This is used to automatically generate most of the makefiles (ie the simple ones), the file used to describe the boot image to the program that produces the boot image (nembuild) and various other things. The blueprint is a set of pieces of data about each build item. Modules and static libraries are build items, as are directories containing header files or system interfaces.

MIDDL interfaces are an important part of Nemesis development. Special tools are used to turn MIDDL interfaces in to C header files and compiled versions of interfaces for incorporation in the Nemesis run time type system. Each build item in the blueprint contains a list of MIDDL interfaces that can be found in that build item's directory. When a Nemesis tree is built, practically the first thing that happens is that a large number of symlinks are generated in one directory, pointing to (practically) all these interfaces that are stored thematically all over the tree. Then the interface building tools are invoked to produce a bunch of header files. This all happens in the ttrepo (for interface REPOsitory) directory.

Nemesis is highly configurable, both at run time and at build time. Build time configuring mostly works by enabling or disabling some of the build items in the blueprint. Some build items require other build items to be enabled in order for them to be buildable, and this may work transitively. (Some build items exist only so that they can be configured on or off, and these build items may not even correspond to real directories). The configuration of build items is not always a simple yes/no choice. Build items that describe modules may be configured to be not built at all, to only build support for the module, to build the module but not place it in the boot image or to build the module and place it in to the boot image. Generally, building in support for a build item means that the interfaces defined in that item will be built but not the module itself. This means that the module could then be built at a later date. The status of the build items are available through header files to be used by the C preprocessor (stored in tth/autoconf) or as a bunch of make directives (stored in ttmk/

It is possible to change lots of things about the set of modules that are built by editing the blueprint directly. It is stored split in to a number of files, for a reason that will become obvious later, and a directory called ttglue/packages. However, to avoid having to edit these files directly it is also possible to edit a file called ttchoices in the root of the build tree, which is initially empty. So, for instance, commands can be placed in the ttchoices to change the status of some build items, that may in turn affect the status of other build items. See the section below on the choices file language.

Nemesis can be built for multiple targets, on multiple build platforms. To manage this, it is necessary to configure a build tree to build a particular target, and to use configuration suitable for a particular build platform. Each build tree can only handle one target and can only be used by one build platform.

Therefore, in order to build Nemesis from scratch, it is necessary to do the following things:

Get all the Nemesis source code together in a hiearchy of directories called a build tree.
Set the platform, target and architecture (kind of target) of the build tree.
Generate all the makefiles and configuration details necessary for Nemesis to be built.
Invoke gnumake to start the compliation of Nemesis.

If the configuration of Nemesis is changed through the ttchoices file or by editing the blueprint in ttglue/packages it is necessary to cause step 3 to be repeated as well as step 4. However, this is normally handled automatically by the build system.


Here is the simplest way to build Nemesis. I'll assume you already have the Nemesis build tools installed on your path.

Create a new directory. Place the Nemesis source code in to this directory.

Usually the Nemesis source code is available via PRCS as a number of packages; see the release notes for instructions on getting hold of the Nemesis packages under PRCS. Once you've done this and set the environment variable ttPRCS_REPOSITORY for PRCS, you'd invoke:

mkdir -p /tmp/nemesisbuild
cd /tmp/nemesisbuild
prcs checkout -r live.@ ccore
prcs checkout -r live.@ cnet
prcs checkout -r live.@ cws
prcs checkout -r live.@ cfs
prcs checkout -r live.@ catm
prcs checkout -r live.@ tgtx86
prcs checkout -r live.@ caudio

You'll find that the source code of Nemesis itself has been placed in to a subdirectory called ttmaster; this subdirectory contains your build tree.

In your build tree, invoke the file ttglue/ with the platform, target and architecture of your choice as arguments. For the sake of example, we will build a Intel x86 Nemesis tree with default configuration on x86 Linux. For instance:

python glue/ ix86_linux intel ix86

You now need to bootstrap the tree by creating configuration files and the makefiles which need autogenerating. Just type:

make replumb
make makefiles

Now you are ready to start the compilation of Nemesis. Just invoke:


Checking out Nemesis rapidly

The above example involves more typing than is necessary. Furthermore, this typing needs to be repeated each time a build tree is created. Instead, some people find it useful to use a tool called to build their tree for them. You can find it in the ttreleasemisc package in the ttmisc/scripts directory.

Checkoutandgo is driven by a configuration file, specifying a directory and a list of packages to be checkout out from a PRCS archived, copied or symlinked in to the directory. The configuration file also contains a shell script to be invoked after these packages have been checked out, copied or linked.

There are several example checkoutandgo scripts supplied with the tool. Here is the above process as a worked checkoutandgo example.

  'packages' : [
     ('prcs', 'ccore:live.@', ''),
     ('prcs', 'cnet:live.@', ''),
     ('prcs', 'cws:live.@', ''),
     ('prcs', 'caudio:live.@', ''),
     ('prcs', 'cfs:live.@', ''),
     ('prcs', 'catm:live.@', ''),
     ('prcs', 'tgtx86:live.@', ''),
  'basepath' : '/tmp/nemesisbuild',
  'actions' : """
python glue/ ix86_linux intel ix86
make replumb # make the configuration files
make makefiles # make the makefiles
  'prcsrepo' : '/local/scratch/dr10009/nemesis/PRCS',
  'prcspath' : '/homes/dr10009/bin/ix86_linux/prcs'

Here you can see the list of PRCS packages that will comprise the tree, the path where the tree will be built, and the commands used to set up the tree. You can also see the details of the PRCS repository (both where the PRCS binary is located and where the PRCS archive is located), that will be different for your site.

By then invoking: simplebuild.coag

(where simplebuild.coag is the name of the configuration file shown above) ttcheckoutandgo will empty the directory tt/tmp/nemesisbuild if it exists, create the directory if necessary, and then invoke PRCS six times to checkout the six different PRCS projects requested. Then, the actions will be invoked which will set the tree going. On a fast machine, this makes it very easy and convenient to build Nemesis.

The source code for checkoutandgo includes detailed documentation for how the tool can be used. It can, for instance, checkout packages in subdirectories, or check out any specified version of a project. If any of the @ symbols in the configuration file above were replaced with numbers, that would cause older versions of the project to be retrieved. You can also use whatever script commands you wish in the actions field. This is used to build a Nemesis quickstart tar ball, for instance.

Configuring and extending Nemesis

The Nemesis build system is designed to make it easy to configure and extend Nemesis. There are different ways in which you might want to do this:

Changing the build configuration
. You may add lines to the ttchoices file in the root of a build tree to change that tree's configuration. For example, it is often useful to use Nemesis with a filesystem from NFS rather than from local disk. This allows more rapid testing of new Nemesis programs. To switch to NFS, you could add the following lines to a ttchoices file in the root of a build tree:

set('initmount_nfs', 1)
set('initmount_ext2fs_ide', 0)

Then, type make in the build tree. The configuration details of the tree will be changed and the affected parts of the build tree will be rebuilt for you.

Adding new module or programs to a Nemesis build tree
. Suppose you wanted to develop a new Nemesis program called adaptivemodplayer. By convention, Nemesis applications are stored in the ttapp subdirectory of a build tree. You would add the following line to a choices file.


This informs the build system about a program called adaptivemodplayer built in app/adaptivemodpalyer. If you reexecute

make makefiles

then this time a Makefile to build it will be created as ttapp/adaptivemodplayer/Makefile, and directions to recurse in to ttapp/adaptivemodplayer will be added to ttapp/Makefile. Then you can just execute make in any of the root directory, app or app/adaptivemodplayer to build the program.

A command line:


does the same thing for a module called ttmesa in the directory ttmod/ws/mesa.

Adding new interfaces to Nemesis

Every build item that corresponds to a real directory (which is most of them) may have a number of interfaces associated with it. So, the easiest way to cause a new interface to be built in to Nemesis is to add to a ttchoices file a line like:

add_interfaces('device_interfaces', ['ExperimentalAudio.if'])

and place the interface files in to the directory corresponding to the build item. In the example above, the directory would be ttdev/interfaces. The correspondence between build items and directories can be looked up in the blueprint manually by inspecting the files in ttglue/pacakges. Items with paths assocaited with them have a path attribute.

Modifying the blueprint data

It is possible to edit the files in the ttglue/packages directory directly. However, those files change quite rapidly, so you may wish to make small tweaks to the files through your ttchoices file rather than modify the files yourself, so that you can continue to take changes to upstream files. There is tweak command that can be used in choices files for this purpose. For example, to disable custom makefile generation for a particular build item, you could use:

tweak('aumod', 'makefileflags', {'custom' : 1})

The first argument is the build item name, the second argument the build item field name to change and the third argument the new value for that field.

Doing new things with the blueprint metadata

Occasionally, it may be necessary to do new things with the blueprint database. This involves writing a new build system backend (probably in Python). You can look at the following backends that already exist for ideas:

Moving source code out of build trees

You may wish to build multiple versions of Nemesis in different configurations from the same source code. You can do this by copying source code between build trees. You may also wish not to store source code in a build tree, because you wish to backup your source tree but not the products of building your code. You may also only wish to backup the parts of Nemesis you have changed and not the entire half a million lines of code.


We have technology to help you with this. There are two incarnations of this technology; one involved alongside Nemesis and is intergrated in to the build system and a number of examples. A newer incarnation separates the management of the locations of source code out from the mechanics of building Nemesis. This newer incarnation takes the form of a tool called ttgrowtree, which is entirely independent of Nemesis and can be used in many different situations.

ttgrowtree is a tool that is driven by two environment variables that specified a source tree and a target tree. The target tree might be a Nemesis build tree or a Linux kernel build tree or any one of a number of other projects. The source tree takes the same directory tree shape as the target tree, but only contains some directories and files. When ttgrowtree is invoked somewhere within the target tree, it will, for every file in the same position and below in the source tree, create symbolic links in the build tree.

If ttgrowtree is invoked with arguments, they should be filenames valid from the current directory and those files are copied in to the source tree and turned in to symlinks in the target tree. There is also an option to move specified files back to the target tree, removing them from the source tree.

You could use this by setting up the ttGROWTREE_TARGET to point to a Nemesis build tree, and ttGROWTREE_SOURCE to point to an empty directory in backed up filespace. Then, whenever you wish to modify a file in the build tree, you invoke growtree with the name of the files you want to work on.

Then, you can setup multiple build trees, and execute growtree in each one so that they all point to the master tree.

There are tools such as Sun's ClearCASE that do this sort of action automatically. Without support from operating systems, you will need to remember to invoke growtree manually every time you modify the set of files in a ``sparse'' source tree (sparse because it does not contain every file you would find in a build tree). Many users find this easy after a little bit of practise.

The ``make grow'' master/private/build tree system

growtree was inspired by the functionality that already exists in the Nemesis build system. Instead of using ttglue/ as the examples so far has shown, a more sophisticated version called ttglue/ can be used. This can be used to actually create the build tree based on a master tree. A master tree contains all the files that make up the build tree. Build trees created by ttglue/ only contain symbolic links to the source code rather than files themselves. If the master tree changes (for instance if a constituent package is updated) then the command:

make grow

can be invoked in any directory of the build tree to freshen symlinks in the build tree.

In addition, if and only if this technology is used then some extra commands become available in the choices file to insert a number of extra of sparse trees in addition, or override the directories that symlinks will be placed in to.


Nemesis is split into packages. These packages are maintained by different people on different sites. Some of these packages contain things like the tools needed to build Nemesis which are handled separately and some contain other things. Most contain source code and form part of a build tree however. These packages all contain a file ttglue/packages/ named after the package, containing the blueprint items for that package. This is why ttglue/packages contains a number of different files. Packages should contain disjoint filenames; ie there should be no filename used by more than one of the packages that make up the Nemesis source. By storing metadata per build item rather than per task (eg the set of interfaces or the configuration for the kernel builder), each package can then easily contribute metadata to each single point where metadata about all build items is required.

It is possible to use PRCS in a number of ways to help manage different source code packages. Because Nemesis is ultimately stored in a single PRCS archive, to change Nemesis means that one has to checkin to a PRCS archive. In nearly all cases, this is done indirectly, either by requesting a package maintainer update the package in certain way, perhaps by emailing the package maintainer with a patch or some new files, or by using the dpatch system and patchman web based patch management system.

Using PRCS in a freshly checked out package copy


Suppose you wish to contribute some changes to a package. The easiest way would be to, in a fresh directory, check out a current copy of the package. You could then make your changes and then use ttddiff to generate a ttdpatch file to upload to ttpatchman or follow whatever other instructions the package maintainer provides. In this case of ttdpatch/patchman, you would type, for instance:

$ mkdir /tmp/ccore-copy
$ cd /tmp/ccore-copy
$ prcs checkout -r live.@ ccore
$ cp ~/mysparsetree/ntsc/generic/atropos.c master/ntsc/generic/atropos.c # for example
$ prcs populate -d # make PRCS check for new or deleted files
$ prcsdiff > /tmp/mypatch
$ less /tmp/mypatch # to make sure you have got the patch right

The invocation in turn invokes ttprcs diff to perform the work, then parses the output of ttprcs diff and reads any files as necessary in order to encapsulate the change. It will also interactively ask for documentation. You can then upload this file to ttpatchman using the web interface.

Using PRCS directly in a build tree

It is also possible to use PRCS directly in a build tree, if you are building in a set of directly checked out PRCS package copy (ie as described at the start of this manual). This method is slightly more tricky, but can be very efficient. The thing to be careful about is the invocation of the PRCS populate command. It may be confused by any symlinks created by ttgrowtree you have created; you may need to edit PRCS .prj files to remove :symlink attributes. You will also need to instruct PRCS as to which package to use.

Using PRCS in a master tree

If you have your own master tree, you can make changes in there and use PRCS again to manage generating patches.

The quickstart shortcuts

When we released Nemesis, we wanted to make things as easy to use as possible. Therefore, there are a couple of simple scripts in the releasemisc package that end up in the root of quickstart tar ball. One is a Makefile, the other a python script called Here are some things that they can be used for:

Configuring your build tree

Out of the box, as described above, Nemesis should have a fairly sensible configuration. However, Nemesis is very flexible, and the build system may be configured in many interesting ways. In the build directory ( ttnemesis/build_intel for instance) you should find a file called ttchoices. You can edit this to file to reconfigure that build tree. Then, in that same directory, with the Nemesis tools on your path, you can type:

$ make grow
$ make

to configure the tree with your changes. The make grow part will create or change symlinks as a result of what is in your choices file.

The language you should use to write your choices file is described in detail in this document, but here are some common examples.

To add your own source code to the tree without changing the master tree (so you can share the master tree and so you can keep your source code on a backed up disk without having to back up the master tree):


The second checkoutandgo example in the release notes does something like this automatically when establishing a build tree. The checkoutandgo configuration file was:

  'packages' : [
     ('prcs', 'releasemisc:live.@', ''),
     ('prcs', 'ccore:live.@', '/nemesis'),
     ('prcs', 'cnet:live.@', '/nemesis'),
     ('prcs', 'cws:live.@', '/nemesis'),
     ('prcs', 'caudio:live.@', '/nemesis'),
     ('prcs', 'cfs:live.@', '/nemesis'),
     ('prcs', 'tgtx86:live.@', '/nemesis'),
  'basepath' : '/anfs/scratch/hornet/dr10009/autobuild/cuttingedge',
  'actions' : """
python establish-intel
echo "add_source_tree('~/mysparsetree')" >> choices
cd nemesis/build_intel
make grow
  'prcsrepo' : '/local/scratch/dr10009/nemesis/PRCS',

The actions script here does three things:

Create a build tree (and the choices file as part of that) for intel.
Add an add source tree command to the end of the choices file.
Enter the build tree and grow and build it.

You could leave some of these actions and do the same thing manually of course.

You can create files and directories in the directory specified (you should pick your own), and they will be substituted for the standard files in the nemesis/master directory whenever you type make grow.

Build system overview

At the centre of the build system is a database, called the blueprint, that contains a large amount of data about how to build Nemesis. Python scripts process this data in order to perform all the necessary work to set up a tree to be built. Building Nemesis is not performed in the master tree itself as it would do normally, but rather in build trees. gnumake is used to control the build process itself, but a number of python tools are used to set up a tree for gnumake to build in. Build trees may be regarded as volatile and need not backed up; no source code should be stored there.

The blueprint database is split between a number of package files, in the subdirectory glue/packages. Every .py file in there is assumed to be a list of blueprint items, corresponding to one of the packages. There is no ordering on blueprint items. The ttglue/ interprets the choices file to specialize the database for one particular build. The customizer works in two ways; firstly, it specializes the blueprint to be unique to the particular target machine specified in the file ( ttglue/ in a build tree. Secondly, it processes the file ttchoices in the root of the build tree, where the user can place configuration tweaks, overrides, extra build items and instructions to include choices files from other places, and packages. The customizer is invoked quite often, whenever data about the tree is required so that the build tree may be manipulated.

The program ttglue/ contains the code to create a build tree. It will create the build tree directory, the file ttglue/ using the command line arguments specified, the file ttchoices and symlink the files in the source tree(s) over to the build tree. Then, the user may execute gnumake at the root of the build tree. Various python scripts will be invoked to derive further information about the tree, as necessary, automatically by gnumake.

Once a build tree has been established, the command gnumake audit in the root of the build tree should produce on stdout a list of all the configuration options and their current values. Furthermore, the output is in a format where it can be fed back in to the build system as a choices file.

The ttglue/ file contains a Python GTK based user interface for editing the choices files. It is optional, and will require additional Python libraries to work, but enables the choices file to be easily edited. (It is old and may not work anymore without some reworking. It predates packages, for instance).

The blueprint and choices file together can be used by ttglue/ to autogenerate a replacement set of package files that can then be moved back in to a source tree, incorporating the changes mentioned in the ttchoices file.

Important directories in the build tree

The root
of the build tree contains a makefile that can be invoked to build the tree. It also contains the file choices, that should be edited by users in order to customize the behaviour of the build tree. See the section on choices below.

contains all the python scripts used to drive the build system, together with a couple of database files-- glue/ and glue/ which should not normally need to be edited by the user.

contains a directory named after the target where the bootable image will appear.

is responsible for building all the ``system'' interfaces-- those which are available throughout the tree and will be in the Nemesis type system from the start of day.

is where symbolic links to the rest of the build tree appear. If your build tree and source tree are readable from the NFS server serving files to your Nemesis box, then it is sufficient to make this directory the boot directory of the Nemesis box. Otherwise, you will need to copy the contents of this directory on to your NFS server.


There are two common ways that a Nemesis build tree can be modified. Firstly, the user can change the tree configuration in the choices file. Secondly, the user can write or modify source code and arrange for this to be built in place of some of the source code in the master tree.

The choices file is a human readable ASCII text file, located in the root of the build tree. It should be treated as source code; it is sensible to symbolically link it from a backed up partition. Multiple build trees can share a choices file. Without a choices file, the build system sticks to the defaults specified in the file glue/ The Choices GUI allows the choices file to be edited without the user having to understand too much about the choices file syntax and Nemesis. It allows more experienced users to edit the choices file interactively with reference to the blueprint database. See the next section for instructions on using the Choices GUI.

Quad choices

The simplest thing one can do in the choices file is to modify the status of one of the build items. For a program or module, this will change the level of support for a build item. The levels are:

Nothing to do with the item is incorporated in to the system. (Symbolic name; NONE or N)

Interfaces used by the item are built and incorporated in to the system. This makes it easier to add the item later. (Symbolic name; SUPPORT or S).

The item is built, and a symbolic link is created for it in the links directory, but it is not placed in the boot image. This is default for some things that are normally dynamically loaded. (Symbolic name; BUILD or B).

The item is built and placed in to the boot image. No symbolic link will be added to the links directory, since it is in the boot image and does not need to be dynamically loaded. (Symbolic name; NBF or Y).

The symbolic names may be used, inside quotes, instead of the numbers. Here are some example lines that could be added to a choices file that set one of these quad-level options:


This causes the Nemesis demo application ``carnage'' to be built and made available for dynamic loading.


This causes the S3 framebuffer device driver to be placed in to the boot image, to avoid having to dynamically load it.


This causes the Spawn module to be completely ignored by the build tree. This may have a knock on effect on some other parts of the build tree, causing things you might not expect to stop working, since they use Spawn.

Boolean choices

There are a few build items that have boolean choice options associated with them. These may be set in a similar way to quad choices, use one of 0, N, NO, FALSE or F to turn an option off, or 1, Y, YES, TRUE or T to turn an option on. For example, to disable the backdrop loading code in the Nemesis windowing system, one could write:


In general, the set command takes an item name in either single or double quotes as the first argument, and either a numeric or string value as the second argument.

Selection choices

There are a few item that take special second argument names. For example, to select the experimental memory system and binder, one could add:

set('memsys', 'EXPT')
set('binder', 'MUX')

Including other choices files and so packages

To include a new choices file, the full path name of the choices file should be specified in an include command. For example:


causes the build system to read the specified file and interpret it as a choices file.

A package is a collection of source files and a choices file that adds them to a certain version of Nemesis. The example above actually adds an experimental audio package to a Nemesis build tree.

Adding programs and modules

To add a new program or module to the system, merely write something like one of the following lines:


You must type gnumake grow for these things to appear in your build tree; they must be in the source tree or wherever you have relocated them using the reloc command at this point. gnumake grow will cause any directories necessary to be created and makefiles to be created or updated. If you then remove the lines from choices, make grow will put the makefiles modified to mention the new directories back again. You only need to run gnumake grow in the directories that have been affected--in the above cases, app and mod. Note that the parent directory of one you modify must be changed; it would not be sufficient to run gnumake grow in app/tone, because this would not update app/Makefile.

By default, programs are set to level 2 support (i.e., built but not placed in the boot image) whereas modules are set to level 3 support (i.e., built and placed in the boot image). set commands may precede add_ commands, in order to override these defaults. For example:

set('3dfxdriver', 'NBF')

(Note that the item name of an added program or module is the final component of its filename).

Adding and deleting interfaces

It is possible to annotate build items with instructions to build interfaces located in those directories and insert them in the start of day typesystem. The command add_interfaces does so, taking a item name and a list of interfaces to add. Similarly, interfaces may be deleted; the command is called del_interfaces and takes the same arguments.

For example, to move the interfaces Rd and Wr from item central_interfaces to item RdWr:

del_interfaces('central_interfaces', ['Rd.if', 'Wr.if'])
add_interfaces('RdWr', ['Rd.if', 'Wr.if'])

Note that the files Rd.if and Wr.if would also have to be moved as well; this only changes the build system database used to locate the interface files.

Destroying build items

The command destroy removes a build item from the database, along with any build items that require it, transitive requirements included. For example, the customizer executes the following commands automatically for all intel trees, ahead of anything the user puts in the choices file himself:

set('ix86_arch', 1)

set('ix86_linux_plat', 1)

set('intel_target', 1)

set('pci', 1)
set('ns16550_serial_device', 1)
set('vga_text_mode', 1)
set('timer_rdtsc', 1)
set('fp_support', 1)

The effect of this is that on the intel architecture, the resulting build system doesn't even contain NTSCs for other architectures, for instance.

Causing files to be obtained from a different place using reloc

The reloc command causes either an entire item or some files in that item to be linked from a different location than they were before. The first argument of the reloc command is an item name, much like most of the other commands described here. If the second argument is a string, it should be the path to the directory where the source code for a build item can be found. For instance:


would cause the app/timidity directory to be linked from /homes/dr10009/hacked_up_timidity instead of (if the source tree was set to the Cambridge live tree location) /usr/groups/pegasus/nemesis/live/master/app/timidity.

In the second form, particular files may be moved. A Python dictionary object mapping filenames in the item to new filenames should be given as an argument.

reloc('device_interfaces', {

Note that the `@' character is special; it is substituted for the path name of the current choices file being read. Thus, the Au and AuMod interfaces above specified relative to current choices file instead of being given an absolute path. In this form only, if the new filename is given as None (no string quotes), then the file is removed from the build tree completely.

Causing files to be obtained from a different place

The add_source_tree command adds an additional source tree to the build tree, after the current source tree. Files found in the new tree override the corresponding files in the original source tree.

For example:


adds a source tree that is searched for source code. Any code there take precedence over the original source tree.

It is possible to have multiple add_source_tree and reloc commands in effect at once, in which case, for each build item, the last reloc command is honoured. If there are no reloc commands for a build item, the source trees are checked in reverse order to the way they were specified, and the first instance of the build item found is used.

To summarise, relocations take precedence over the tree search path, and later add source tree commands take precedence over the earlier source tree commands.

Modifying the blueprint directly

There are a couple of back doors to customizing the blueprint directly. The add_item command takes a build item constructed in a similar syntax to the blueprint and adds it to the blueprint. It must extended over only one line. The tweak command sets a field in a particular item. These are intended to provide supplementary functionality to the above commands.

The tweak command enables one of these fields to be added or, if it was already present, replaced. For example:

tweak('CSIDCTransport', makefileflags, {'custom': 1})

adds a makefileflags field to CSIDCTransport; the effect is to prevent CSIDCTransport's makefile from being autogenerated. It is in an error if the second argument is not a valid command name. The second argument may be quoted.

To use add_item, it is necessary to invoke one of the item constructors of the file glue/ The fields and classes are described later, but first an example of the use of add_item:

  description: 'Login service (Pcnfs based).',
  path: 'mod/security/pcnfs',
  makeflags: {'libsocket':1, 'libsunrpc':1},
  binobject: 'Pcnfs',
  system_interfaces : ['PcnfsLoginMod.if'],
  type : 'quad',
  value : 2}))

This creates a new item called Pcnfs, describe as a PCnfs based login service, residing at mod/security/pcnfs, linked against socket and sunrpc libraries, producing a binary called Pcnfs and introducing a system interface called PcnfsLoginMod.if. It is type quad, and will be built but not placed in to the boot image by default. The item is class AutoModule; all class constructors take the item name as the first argument, and a dictionary of all the fields they define as the second argument.

The Choices Editor GUI

This section and the code it describes is out of date; it predates packages. If anyone is interested in reworking the choices editor for the new packaging scheme, please make yourself known to us.

If you have python set up with the GTK widget set, you may edit choices files using the Choices Editor. The choices editor reads the blueprint database in much the same way as the build system, and can execute choices files. Instead of building Nemesis, however, you can view and change the Nemesis source database.

The choices editor has three levels of operation--beginner, expert and deity mode. Deity mode is not fully developed yet, so will not be described completely here until it is finished. The editor starts in beginner mode.

To start the editor, enter a source tree glue directory and execute, specifying a path to a choices file as a command line option and specifying a path to a file that the choices file refers to. For example:

cd $NP/glue $NP/ $NP/glue/

A window will appear containing a menu bar, a notepad style selector attached to a list of items, and a large empty rectangle. Click on one of the list items, and the large rectangle will be filled by a form telling you about the item you selected. In beginner mode, you are only shown some of the build items, and are only allowed to set the status of build items. If you click on the drop down menu widget labelled value, you can select a new value for the item you chose. The choices file you gave as a command line argument will be amended immediately to reflect the change you just made, and the user interface will be updated to illustrate the change.

The build system will always use the latest choices file on disk, so there is no need to quit the editor in order to build things. The editor can be left running, and changes made in it will be picked up as if the choices file had been edited directly with a text editor. Note that it is important to read the messages produced on the terminal where the choices editor was run, because the choices editor will not correct an already broken choices file nor can it (yet) be guaranteed only to make legal changes.

You can use the notepad buttons to view different categories of build items, and the (scrollable) list to select items within that category. The list also shows you the resultant value of each option, and what properties of each item have been overridden by your choices file. You can use the File menu to select a different choices file to edit.

In expert mode, you are shown all items that might be configurable, rather than just the ones we have decided it is sensible for beginners to configure. You will only be shown the text of the choices file you are editing, and may modify it directly by hitting the edit button. Then, you may edit the choices file and any changes made through the method described above will be batched up in the choices file until you hit either commit (to save the changes) or revert (to discard the changes).

Deity mode adds the (so far unimplemented) ability to add new kinds of item, shows you even options that may not be modified and allows all fields in the blueprint database to be edited. The new edit buttons on the item editor forms pop up dialogs that allow you to specify a new value with reference to an old value.

Item classes

The parent of all build item classes. Not to be used directly.

Used to describe Nemesis Trusted Supervisor Code routines. There should remain exactly one in the blueprint, otherwise glue/ will complain. These items must specify a path.

Used to describe the primal routine. Again, only one per image. Must specify path.

items describe pieces of code with one entry point that are executed as separate domains. Programs may be placed in the boot image, therefore they require path, qos and env fields.

items are like programs (PosixProgram is a subtype of Program), but in addition arrange for posix compatibility libraries to be linked against the program. At the time of writing, this means that PosixPrograms may have an ANSI C main() function. They require path, qos and env fields (because they inherit from Programs).

items describe Nemesis stateless modules. They require path and exports fields.

items describe Nemesis stateless modules, but they do not require an exports field because it is assumed that AutoModules export into the primal (i.e., main) namespace a symbol with the same name as the AutoModule item itself. They still require a path, however.

items describe directories that build documentation. They require a path.

items exist only to provide a configuration option; they must not have a path.

items are directories containing exclusively interfaces. Interfaces may be present in other items as well, but this item type is used for items that contain only interfaces. Require path.

items produce static libraries, and require path to be specified.

items contribute ``Blobs'' of data to the boot image. They require path and bloblist options.

Item fields

associated cpp name
fields are optional. If present, glue/ will generate header files and entries in mk/ describing the state of the build item. (Note that the field name is associated_cpp_name).

For example, the field:

  associated_cpp_name : 'REGION',

would, if the item had type bool, arrange for CONFIG_REGION to be set to zero or one. Setting to zero results in:


#CONFIG_REGION is undefined

in h/autoconf/region.h and mk/ respectively. Setting to one results in:



If the item had type quad, for values 0, 1, 2 and 3 respectively, glue/ would generate, for value set to 0:


#CONFIG_REGION_NBF is undefined
#CONFIG_REGION is undefined

For value set to 1:


#CONFIG_REGION_NBF is undefined

For value set to 2:


#CONFIG_REGION_NBF is undefined

For value set to 3:



For a multiple choice option, a CPP macro postfixed with _VALUE is defined with the index of the choice selected. Then, each choice's symbolic name is treated the same as boolean set to false or true. For example, for memsys if set to EXPT:


#CONFIG_MEMSYS_STD is undefined

or if set to STD:


#CONFIG_MEMSYS_EXPT is undefined

Refer to the source code of glue/ to see how this is implemented.

fields override module, programs, automodules, ntsc and primal build items, specifying a binary object name that the item produces. By default, the item name is used as the binary object name. For example:

  binobject : 'Lance',

is a field for BlobDirectory items only. They are lists of (blob filename in directory, handle in NBF name, namespace anme) 3-tuples. For exmaple:

bloblist : [('boot.rc', 'blobrc', 'blobrc'), ('init.rc', 'initrc', 'initrc')]

fields are strings describing the item. They are optional for any item class. For example:

description : 'common (ethernet, IP, UDP) protocol code',

fields are strings documenting the contents of thing the item refers to. They are optional for any item class. For example:

doctext : 'The windowing system server handles window allocation,
 clipping and event demuxing. It also interfaces with a window manager.',

Note that this should be a single line.

fields are compulsory for programs and posix programs. They are dictionaries, and must define a number of contexts with key `contexts', keys `defstack', `heap' and `sysheap' as strings suitable for nembuild, a number of frames with key `frames', a boolean called `kernel' set to 1 if the program is to be given kernel privileges, and optionally a key `autostart' set to 1 if the program is to be automatically started at boot time.

env : {'autostart':1,'contexts':32,'defstack':'8k','endpoints':128,

Note that this should be a single line.

fields are strings specifying what the effect of configuring an option might be. They are optional for any item class. For example:

helptext : 'Enable this to cause a backdrop to be displayed in the
windowing system',

Note that this should be a single line.

fields are booleans, intended to be set for directory items. They cause the build system to not grow directories recursively. Usually, the whole contents of the source tree at the path specifed and below is linked or copied in to the build tree.

For example, the tth directory build item specifies:

  growrecursive : 0,

otherwise the entire contents of al tth subdirectories would always be present in all build trees, including including those specific to architectures other than the one being built.

fields are instructions to the makefile generator. They are dictionaries, which may define any of the following:

set to 1 to prevent the makefile from being autogenerated.

set to 1 to link libraries defining IO functions with the build item.

set to a list of filenames of files stored in the directory of the build items.

set to 1 to cause to be included in make for that makefile.

set to 1 to request the static window system rendering library to be linked.

set to 1 to request BSD sockets code to be linked.

set to 1 to prevent the binary produced from being installed when gnumake installmost is invoked.

set to 1 to cause the SUN RPC static library to be linked with the binary.

set to 1 to cause the MESA rendering static library to be linked with the binary.

fields are compulsory for most build item classes. They are paths relative to the root of the build tree, indicating where the build item is located. For instance:

  path : 'mod/nemesis/csidc',

fields are lists of items that must be active for the item with the requires field to be active.

For example, the WTime automodule item has a requires line at the moment of:

  requires : ['RTC', 'ix86_arch']

This means that it will be disabled if ix86_arch or RTC are disabled. Disabled is defined as set to 0/No for bools and quads, or for multiple choice type items to a choice where the first option is set to 0. If one of the required items of an item is destroyed, that item is destroyed as well.

There is a special form for requesting one particular choice out of a multiple choice item. This is presently used to depend on one particular memory system. For examples:

requires : [('memsys', 0), 'ix86_arch']

requires : [('memsys', 1), 'eb164_target']

The first example requires memory system 0, the first choice in the memory system (i.e., memory system standard). It also requires ix86 architecture. Similarly, the second example requires the second choice memory system `experimental', and an eb164 tree.

fields are compulsory fields used to divides items in to sections. The sections are purely for readability of the blueprint and use of a GUI. The section ``unclassified'' exists as a catchall. For example:

section : 'WS'

is optional for all items with paths except InterfaceCollections where it is compulsory. It is a list of strings that are filenames of interfaces, in the directory specified by the path of the item. For example:

  system_interfaces : ['CSIDCTransport.if', 'CSClientStubMod.if'],

fields specify the configuration type of the build item. There are three different kinds; boolean (`bool'), quad (`quad') and multiple-choice types. Examples of each are:

type : 'bool'

type : 'quad'

type : [(1, 'standard', 'STD'), (1, 'experimental', 'EXPT')]

Booleans are yes/no choices (surprisingly), generally used for configuration data. For instance, all the architecture, target and platform PureConfiguration build items have type boolean.

Quads are used for compiled pieces of code, indicating whether nothing should be done for that code, that code should be supported, that code should be built or that code should be placed in to the bootable image. An item of type quad is regarded as active if it is set to anything apart from off.

Multiple choices are specified by a list of (activity flag, friendly name, symbolic name) 3-tuples. An item with a multiple choice type may be set to exactly one of these choices at once. The item is active if the chosen value's activity flag is set to 1. The friendly name is for the use of graphical front end, and the symbolic name is for use as an alias to an index into the list for the choices file.

All items have a type.

fields specify the current status of the build item. In glue/, this is the default value for the option. The customizer modifies these fields according to the current target and the choices file. All items have a value field.

If the type field is `bool', the value is 0 or 1.

If the type field is `quad', the value is 0, 1, 2 or 3.

If the type field is a multiple choice, the value is an index starting at 0 in to that list.

fields are for Program and PosixProgram items. For example:

  qos : {'extra':1,'latency':'20ms','period':'20ms','slice':'0ns',},

The are dictionaries that must define the keys 'extra' (as one for extra time for that program or zero for no extra time), 'latency', 'period' and 'slice' as time parameters. The time strings are passed straight to nembuild; see the documentation for that command for their meanings.

At this point, the careful reader should have information to understand the file glue/ with. That file is a useful source of examples. glue/ is written in python, and can be automatically generated by glue/

Choices language reference

Lines containing entirely whitespace are ignored.

Any characters in between a # and a newline are ignored, including the #.

The character @ is replaced with the path name of the current choices file.

All other lines are Python method invocations, from the following:

set(itemname, value)
add_interfaces(itemname, interfacelist)
del_interfaces(itemname, interfacelist)
tweak(itemname, fieldname, value)
reloc(itemname, override)


is the name of a build item in single quotes.

is the path of a binary from the root of the build tree, in single quotes.

is a list in squares brackets of interface names, in single quotes.

is the name of a field, optionally in single quotes.

is a constructed item object, built using the constructors defined in glue/

is either a string or a dictionary mapping filenames to filenames.

is a full path to a directory.

Each line must be self contained; the above method invocations must not span multiple lines.

layfoundations and changefoundations

layfoundations is used to first create a tree. It is recommended that you keep environment variables, called NM and NP around, with paths to your master and build trees around. The build system does not use them, however, but various small scripts people tend to write may. If you have a local copy of the master tree, your build will be quicker if you use it and you will save load on the network. (In Cambridge, local copies of the master tree are stored in some machines in /usr/opt/nemesis/live).

If you have defined them, then:

cd $NM/glue
./ $NB $NM intel ~/

may be a sensible way to set up your build tree. This specifies a tree targeted to build images for Intel machines, with a choices file residing in the root of your home directory called If it is omitted, then no symlink will be created. If the file specified does not exist, then the symlink will be created anyway and the build system will create a null file there (using touch). To build direct from the master tree, use the location of the master tree instead.

Some targets have more than one build platform. To override the selection in such cases, try something like:

cd $NP/glue
./ -platform=linux_for_arm $NB $NP shark $NP/

Note that options may start either with either a minus or plus, but some python interpreters have been miscompiled and will only accept a plus.

The script ttglue/ is in fact a symbolic link to ttglue/ that exists in build trees. ttglue/ is specifically executed by gnumake itso in the root of a build tree, to tidy up a tree as necessary.

Historical note: the tool mkbuildtree is an unsupported wrapper around glue/, which is less flexible and often out of date.

Creating a new set of blueprint files

The tool glue/ can be used to generate a replacement set of glue/packages files, which it saves in a directory called new_packages in the current directory. The idea is that contributors to a Nemesis tree develop their changes using a choices file, then run regenerate_packages to generate a new generic blueprint. This new generic blueprint can then be checked in to replace the old generic blueprint in the Nemesis distribution.

Note that regenerate_package does not perform the sanity check that is usually done by the customizer, as it would disable all the architecture-specific items. Therefore, people using regenerate_package should be careful that they specify a choices file that gives sensible defaults, as these will become the defaults for all users of the new generic blueprint.

It is of course also possible to write and edit the package files manually. regenerate packages is intended to ease major changes such as splitting packages.

Constructing expansion packages

We have two kinds of packages; the ones that make up the source tree and expansion packages. This section introduces expansion packages, which don't involve the source code management scheme at all. The release notes describe how to construct the other kind of package, which will go through the source code management scheme.

Expansion packages are nothing more than sparse trees with extra choices language code to configure the build system in. As mentioned above, a package consists of a specially written choices file, and a collection of code. The choices file should use the `@' feature of the choices language to avoid using absolute path names. Here is an example package choices file, to serve as an extended example of a choices file and a prototypical package.

# Dickon's experimental audio package, v0.1, against the live tree

  'device_interfaces', # so is this
  ['Au.if', 'AuMod.if', 'AuPlaybackWriter.if']
reloc('device_interfaces', {

set('aumod', 2)
reloc('aumod', '@/aumod')

tweak('lksound', 'makefileflags', {'custom':1})
reloc('lksound', '@/lksound')

tweak('amp', 'makefileflags', {'custom':1}) 
reloc('amp', '@/amp')

reloc('auview', '@/auview')

tweak('tone', 'makefileflags', {'libio':1})
reloc('tone', '@/tone')

tweak('timidity', 'makefileflags', {'custom':1})
reloc('timidity', '@/timidity')

tweak('ndoom', 'makefileflags', {'custom':1})
reloc('ndoom', '@/doom')

reloc('udpaudio', '@/udpaudio')


# Intel NTSC Memory management hack
reloc('ntsc_ix86', {'intel/mmgmt.c' : '@/mmgmt.c'})


Beginners can skip the rest of this manual.

Figure 1: Structure of the build system

changefoundations is used to cause bits of the procedure executed by layfoundations to be repeated. The procedure is divided in to a number of steps, each one with a symbolic name:

takes the name of the build tree as an argument. This is the first step, which is why the build tree location goes first in the above examples.
creates the tree directory and a minimum skeleton of directories inside it. It is not destructive.
creates the file glue/ that stores the location of the build tree, source tree, target and so forth. It also creates the choices symlink. gentreeinfo takes two or three arguments, which are the name of the source tree, the target name and optionally the path to choices. The -copy options causes the tree to be set up to copy files rather than link them, and is considered when gentreeinfo happens. The -platform option mentioned above is similarly considered at this point.
generates all the autogenerated makefiles. (Invoked by gnumake grow).
fills all the directories out with symlinks. (Invoked by make grow).
creates a few important symlinks that aren't created any other place; the symlinks from platform and target makefiles to and, the symlink from layfoundations to changefoundations and the root makefile. It also creates mk/


cd $NB/glue
./ -geninfra

recreates all the infrastructure makefiles.

Under the hood

Only read this section if you need to modify the internals of the build system.

Firstly, it is worth pointing out that there may be some residue from the TCL based configuration scheme. Please point any out to the author if you notice anything.

Secondly, below is a description of the function of each file in the glue directory, to remove some of the mysticism surrounding that directory.
is a python module that is imported by any python code that wishes to access the build system database. It uses generic as the basis of the database, but invokes in order to specialize it using the target, platform and architecture, some knowledge in the customizer and the contents of the choices file.
is a python module defining a few useful functions; for instance, most symlinks are created using the symlink routine in here rather than the Pyhton OS module. There should be nothing Nemesis specific in here.
contains a routine to figure out the active subset of the build tree. (Previously it contained a lot more, but now it barely justifies remaining a separate file).
defines the class that provides, as methods, all the commands used in choices. An object of class Customizer is created, and choices commands are really just invocations on that object. includes some knowledge of what the different targets mean.
is a stand alone python program that reads all dependency file and calculates the inverse mapping. It is invoked by the uses.txt rule of the root Makefile.
generates a string used to put identification in to autogenerated files.
creates the directories comprising a build tree.

contains generic data about the nemesis tree. specialises this for particular build tree. In, all targets are off by default; the Customizer will turn them on for particular machines.
figures out the set of makefiles to generate.
is a standalone program invoked by the links directory makefile to create all the installation links.
is used by to actually generate makefiles.
generates input for nembuild; it is invoked by the boot/images subdirectory makefiles.
is the entry point for make grow, invoking geninfra and filldirs as necessary.
is describe above. It is a standalone program.
contains the class definitions for all build items.
is invoked whenever the choices file changed, as a standalone program, in order to create the autoconf header files and makefile.
converts CPU time fractions to QoS parameters suitable for gennbf.
manages obtain source code, using the data in treeinfo.
is autogenerated by layfoundations to store some data about the build tree.

About this document ...

The Nemesis Build System
Users' Guide and Reference Manual
7th Edition1

This document was generated using the LaTeX2HTML translator Version 98.1p1 release (March 2nd, 1998)

Copyright © 1993, 1994, 1995, 1996, 1997, Nikos Drakos, Computer Based Learning Unit, University of Leeds.

The command line arguments were:
latex2html -split 0 buildguide.tex.

The translation was initiated by Dickon Reed on 1999-05-21


... Edition1
Revision date 1999/04/08, last checkin r, by
... 19992
Processed May 21, 1999

next up previous
Dickon Reed