Dickon Reed
January 19992
Welcome to Nemesis. This document describes the licensing of Nemesis and the procedures for obtaining it, working with it and contributing back to it that have been developed for the Nemesis release.
Look at the Nemesis home page for news, new versions and documentation.
The intent of this document is to state the conditions under which a Package may be copied, such that the Copyright Holder maintains some semblance of artistic control over the development of the package, while giving the users of the package the right to use and distribute the Package in a more-or-less customary fashion, plus the right to make reasonable modifications.
Nemesis is managed as a set of packages. Each package consists of a set of branches. Some branches of some packages are publicly available and some have more restricted availabilities for a variety of reasons. Each package, by convention, has a branch called live, where the main thrust of active development takes place. Other branches may exist as bug fixes to stable points in the live development, or when the direction a package is taking branches.
Several versions of Nemesis are built daily and (if they compile!), then tarballs are made available. The most common of these is available as the quickstart tar ball; see the section below on Quickstart to see how to obtain and use it.
We use a custom designed system called dpatch to make changes to Nemesis. See the section on Contributing your work back to us for details. We use PRCS as a back end for dpatch. You may wish to set yourself up with a Nemesis source code repository; this will make it possible for you to submit patches back to us easily, and will make it easier for you to upgrade to new versions of the Nemesis packages when you wish. Some sites may already have a Nemesis source code repository set up, and if so then you may be able to use it saving the time of setting up your own. See the instructions below on obtaining a source code repository.
The standard, publicly available Nemesis packages are currently:
The following packages may become available in the future:
We've put together a fast track mechanism for you to build and evaluate Nemesis. It just involves downloading one file, using tar to extract it and then building it. It includes everything you need to get started.
nemesis/build_intel/links
you should find a Nemesis image and some support files.
PATH=/local/scratch/dr10009/release/master/tools/install/ix86_linux_rh5.1/bin:$PATH export PATH
nemesis/build_intel/choicesand modify that file.
add_source_tree('/usr/groups/pegasus/users/dr10009/experimentalsparsetree')
then type make grow in nemesis/build_intel, all files in /usr/groups/pegasus/users/dr10009/experimentalsparsetree will override the master tree files. You can then copy files there, modify them, and just back up the sparsetree directory instead of the whole of the quickstart tar ball.
Frequently asked questions about the quickstart process:
Nemesis builds on multiple Unix systems that have GNU make installed. However, GNU make is sometimes installed as gnumake (for example on OSF) and sometimes as make (for example on Linux). Most of the time, we handle such issues with platform specific make files, but you need to arrange that invoking gnumake runs a proper GNU make. Any recent version of GNU make should do. We use version 3.76.1.
The quickstart tar ball is usually updated every day, so you can just download new versions of that. However, it may be much more convenient to set yourself up with a local Nemesis source code repository. Then, you will be able to work with whichever version of Nemesis you like, and with whichever combination of packages you like. Furthermore, you will be able to use PRCS to help manage your source code.
You may already have access to a Nemesis source code repository on your site. In the Computer Laboratory of the University of Cambridge, where Nemesis was developed, a PRCS archive is currently available at /local/scratch/dr10009/nemesis/PRCS on all normal build machines. If you already have access to such a PRCS, skip this section and proceed to the next section on working with this PRCS archive.
Obtaining a source code repository will let you work with custom versions of Nemesis, and will save on download times. We recommend you keep source code repositories on each machine you will build on; this really speeds up creating new build trees.
A Nemesis source code repository consists of two parts; a set of patches in a special format called the patch archive, and a PRCS archive to make it easy and efficient to check out particular versions of Nemesis packages. The PRCS repository just provides the data for an easy way to check out particular version of Nemesis; it is generates from the patch archive. The repository you will end up with on your filesystem will of course just be copies of the offical patch archives in Cambridge and other places. But it will enable you to retrieve any combination of any version of any Nemesis package.
{
'patch tree' : '/home/fred/nemesis/patches',
'prcs repository' : '/home/fred/PRCS',
'description map' : {
'project' : 'The Nemesis Project at OurSiteName'
}
}
Change the directory names tagged patch tree and prcs repository to suit your needs. The prcs repository does not need to be backed up. The patch tree only needs to be backed up if you start exporting your own packages.
dcheckin.py webimport http://www.cl.cam.ac.uk/Research/SRG/netos/nemesis/patches.html
(If you believe you have access to some of the non-public branches, you will need to find out the alternative command for you to obtain the extra material).
You may like to run this command from a CRON job. It works incrementally; it will just download new patches.
(At any stage, you may destroy your patch tree and PRCS repository, and start again from scratch. That is unless you are exporting a package in the way suggested below).
{
'patch tree' : '/home/fred/nemesis/patches',
'prcs repository' : '/home/fred/PRCS',
'description map' : {
'project' : 'The Nemesis Project at OurSiteName'
}
}
Change the directory names tagged patch tree and prcs repository to suit your needs. The prcs repository does not need to be backed up. The patch tree only needs to be backed up if you start exporting your own packages.
Download the patch repository from Cambridge by executing:
dcheckin.py webimport http://www.cl.cam.ac.uk/Research/SRG/netos/nemesis/patches.html
(If you believe you have access to some of the non-public branches, you will need to find out the alternative command for you to obtain the extra material).
You may like to run this command from a CRON job. It works incrementally; it will just download new patches.
(At any stage, you may destroy your patch tree and PRCS repository, and start again from scratch. That is unless you are exporting a package in the way suggested below).
You should by now have access to Nemesis in a PRCS archive. We only use prcs for checking versions of Nemesis out. To contribute back to Nemesis, see the section below.
A tool called checkoutandgo.py provides a useful way to checkout multiple PRCS packages and build them. It is located in the quickstart tar ball as master/misc/scripts/checkoutandgo.py but we suggest you copy it somewhere else and keep it on your path. It used by us to generate the quickstart tar ball, for instance. See the build system users guide for more details, or the start of the source code of checkoutandgo.py where you'll find an explanation of what it does and how it works.
For example, suppose you want to build Nemesis in a directory called /anfs/scratch/hornet/dr10009/autobuild/cuttingedge. You've got a PRCS repository in /local/scratch/dr10009/nemesis/PRCS and prcs is on your path, as are built Nemesis tools. You also don't need ATM support. You'd write a configuration file called nemesis_test.coag, in your home directory for instance, containing:
{
'packages' : [
('prcs', 'releasemisc:live.@', '/..'),
('prcs', 'ccore:live.@', '/nemesis'),
('prcs', 'cnet:live.@', '/nemesis'),
('prcs', 'cws:live.@', '/nemesis'),
('prcs', 'caudio:live.@', '/nemesis'),
('prcs', 'cfs:live.@', '/nemesis'),
('prcs', 'tgtx86:live.@', '/nemesis'),
],
'basepath' : '/anfs/scratch/hornet/dr10009/autobuild/cuttingedge',
'actions' : """
python quickbuild.py establish-intel ix86_linux
echo "include('/homes/dr10009/u/mychoicesfile')" >> nemesis/build_intel/choices
cd nemesis/build_intel
make
""",
'prcsrepo' : '/local/scratch/dr10009/nemesis/PRCS',
'postfix' : 'master'
}
Then, invoke (from any directory):
checkoutandgo.py ~/coag/nemesis_test.coag
(specifying the path to your checkoutandgo script as the argument to checkoutandgo.py, of course, and the path to your local PRCS repository instead of the example given above in the line defining prcsrepo).
checkoutandgo.py will empty your build tree (specified by the basepath line if it exists, create the directory if it does not already exist, and then check out the versions you asked for of the packages you want, and invoke the actions script. releasemisc forms the framework for your build tree so you will always need that. core, cfs and a target package such as tgtx86 are usually required. All other packages are optional bolt on extras. In the example above, this then sets up a intel build tree, makes it use the choices file in /homes/dr10009/u/mychoicesfile and then builds the tree. See the build system users guide for an explanation of choices files. The idea is that you keep the choices file, the checkoutandgo script and any of your own source code safe on backed up filespace. At any stage, you can repeat the checkoutandgo.py command to destroy the build tree and start again, should the non backed up filespace become damaged or the build tree become confused. The actions line contains the commands necessary to create the Nemesis build tree. We work by picking configuration files close to what we need, and modifying them for new build trees.
You may use PRCS directly to checkout, merge or diff packages of course. See the PRCS documentation for details. If you want to see exactly what versions of files you need, then look at the CONTENTS file in the directory created by checkoutandgo. It gives the PRCS versions of each file. You may want to inspect the contents of the patch repository to see what has changed; see the dpatch manual for details.
When you wish to upgrade one of the packages in a checked out tree, you can use prcs merge or prcs checkout. prcs checkout is simpler; it will merely write the new package on top of what you have at that time. prcs merge will interactively reconcile your changes with what have gone before. Alternatively, you can just reinvoke checkoutandgo.py which is event easier but means the entire tree will be recreated which may take some time.
All changes to Nemesis must be submitted to us in dpatch format, using the patchman web interface (see the Nemesis home page). There are many ways to generate patch files. First of all, however, make sure you are working with the latest versions of the Nemesis packages you want changed. The more out of date your versions, the less likely it is your patch will still be valid and will be accepted. Alternatively, you may try to write your patches by hand. See the dpatch manual for details. If you are modifying the contents of a Nemesis quickstart tar ball master directory directly:
ddiff.py prcsdiff packagename > mypatch_to_packagename
in the nemesis directory, where packagename is the name of one of the .prj files in that directory.
If you modify the nemtools or dpatch, you need to run it in the directory containing the relevant .prj file.
If you are working with a private tree (sparse or with symlinks), for each package you want to change:
prcs populateto let PRCS update your .prj file.
ddiff.py prcsdiff > mypatch_to_packagename
See the dpatch manual for more ways to use ddiff.
Now, review your patch files and upload them to us using the web interface.
If you have some code which is new, rather than an enhancement of an existing package, or you wish to maintain it yourself, or you cannot persuade us to take on your changes, then you should make a new package. First perform step 1 of ``Contributing your work back to us'' to set up your patch tree. Your package will go in the same patch tree as the patches you import from us. Then:
ddiff.py create > ~/initial_gnemx_patch
dcheckin.py create gnemx live
You'll need a .dcheckinrc file in your home directory to let dcheckin know where to write to; an example is given above. Now you've got a new package, you should add a description of your package to your .dcheckinrc. Mine looks like:
{
'patch tree' : '/usr/groups/pegasus/nemesis/patches',
'prcs repository': '/usr/groups/pegasus/nemesis/PRCS',
'description map': {
'project' : 'The Nemesis Project\n',
'ccore' : 'The Cambridge Nemesis Core\n',
'dpatch' : 'The dpatch Project\n',
'catm' : 'The Cambridge Nemesis ATM subsystem\n',
'caudio' : 'The Cambridge Nemesis Audio subsystem\n',
'cnet' : 'The Cambridge Nemesis Network subsystem\n',
'cfs' : 'The Cambridge Nemesis filing subsystem\n',
'cws' : 'The Cambridge Nemesis windowing subsystem\n',
'nemtools' : 'The Nemesis build tools\n'
}
}
At this point, you have created your package and branch, but there isn't any code in the package yet.
dcheckin.py commit gnemx live ~/initial_gnemx_patch
Make sure you give an absolute path to the name of your patch.
At this point, you have a package with one branch and one patch. You can check that everything is working by inspecting the files metaupdates and allupdates in the patch tree, or you can type:
prcs info gnemx
You should see that there has been one patch in the updates file, and there are now two PRCS versions of your packages (the first one is empty).
dcheckin.py publish ~/public_html/nemesis/patches.html
In order for the patch index to work, in the directory that you have built the HTML file you will need to create a symlink to your patch tree called patches. You can test this by making sure that you can follow the link on your patch index web page to your patch, using HTTP.
If you wish, mail me Dickon.Reed@cl.cam.ac.uk with the URL of your patch archive web index. I can then arrange for it to be indexed off the master patch index in Cambridge.
dcheckin.py commit gnemx live ~/another_patch
The patch will first of all be tested, and if it cannot be applied to the package and branch you specify then you will be told and you will need to edit the patch or ask whoever wrote it to regenerate it against the latest release. If the patch has been accepted then remember to reexecute:
dcheckin.py publish ~/public_html/nemesis/patches.html
in order to expose your patch to the world.
See the dpatch manual for more details on how dpatch works.
Your patch tree contains the canonical copy of your package and its branches. You should make sure that at least the subdirectory containing your package in your patch tree is regularly backed up. The other directories in your tree will continue to be filled from Cambridge every time you run dcheckin webimport.
This document was generated using the LaTeX2HTML translator Version 98.1p1 release (March 2nd, 1998)
Copyright © 1993, 1994, 1995, 1996, 1997, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
The command line arguments were:
latex2html -split 0 -toc_depth 3 releaseguide.tex.
The translation was initiated by Dickon Reed on 1999-04-26