Computer Laboratory

Migrating MH mail to IMAP

The MH mail system has served many people well for decades, but it lacks many modern features, in particular the ability to manage your mail from mobile devices. Migrating email delivery to an IMAP server, such as Hermes, makes good sense. This page collects some advice on how to make this transition relatively painless.

New mail client and configuration

The old MH/exmh system is incompatible with IMAP, so it is unavoidable to get used to a new mail client. Mozilla Thunderbird is currently one popular choice on departmental desktop machines, but there are many others.

The instructions for configuring your mail client to use Hermes explain not only how to configure IMAP to read your stored messages, but also how to configure SMTP to send out your messages via the Computing Service. On your desktop machine, you may want to use the Computer Laboratory SMTP server instead, which is what you used under MH, to keep features such as instant check and automatic expansion of local addresses.

Changing mail delivery

Changing the delivery for new mail to Hermes is easy: simply put your Hermes Email address into your .forward file. Before doing this, please make absolutely certain that there is not already a forwarding from Hermes back to CL. Mail forwarding loops will not be detected until a message exceeds a forwarding limit and ends up with a postmaster, who will be unable to mail you about the problem.

After changing the CL forwarding, it would be a good idea to change your cam.ac.uk forwarding to go direct to the new address as well. This removes a dependency and avoids unnecessary forwarding. You make this change in Lookup by following the Computing Service documentation.

Some people have a more complex local .forward file that splits incoming messages into different folders, for example to deal with high-volume mailing lists. In such files, you could initially replace only those commands that deliver your personal messages (e.g., pipe "rcvstore +inbox" or save .mail) with deliver your-crsid@hermes.cam.ac.uk, to sent all your personal messages to Hermes. Such an interim step would allow you to continue filtering mailing list messages and spam with exim and deliver those to MH. You can then, gradually and at a later time, redirect your mailing-list traffic as well and ultimately migrate your filter logic to Hermes.

Moving legacy folders

You are very likely to want to transfer existing mail folders into the new server. To do this you need to export each folder as a file from MH, and then import that file into Hermes using an MUA. Although this is slightly tedious, it only needs to be done once, and it gives much more satisfactory results than trying to forward the old mail to yourself.

There are several tools that can be used for this.

Exporting mmdf folders from MH into IMAP via mailutil

  • Invoke the following script by Markus Kuhn to export your MH folder into a file in MMDF mailbox format:
    /homes/mgk25/w/scripts/mh-export ~/Mail/path/to/mh/folder >~/folder.mmdf
    
  • Install the mailutil tool found in the University of Washinton IMAP distribution. On Ubuntu Linux:
    cl-asuser apt-get install uw-mailutils
  • Use mailutil to copy your MMDF file via IMAP to Hermes, e.g.
    mailutil copy folder.mmdf {imap.hermes.cam.ac.uk/tls/user=crsid}new-folder
    

Note:

  • mailutil interprets the pathname of your MMDF file relative to your home directory, rather than your current working directory.
  • In Thunderbird, you may have to hide and reexpand your folder list before the new-folder that you have just created is displayed.

Exporting mbox folders from MH into Thunderbird

An alternative (no longer recommended) method is to use the MH command packf to convert an entire folder into mbox format, which can then be imported into a local Thunderbird folder using a Thunderbird add-on ImportExportTools:

  • Export your folder from MH using the packf command:
    packf +folder -file /tmp/folder
    

    Beware that if the output file already exists, the command will append to it, which is only useful in the unlikely event that you want to combine multiple folders into one. Obviously you can write a loop to process multiple folders, and for reasons which will become clear later, it is helpful to use the folder name as the file name.

  • The packed file requires a small adjustment to the message separators before it will work properly in Thunderbird:
    perl -p -i -e 's/^From nobody\@nowhere  /From -  /' /tmp/folder
    
  • Configure Thunderbird to access Hermes via IMAP, if you have not already done so.
  • Install the ImportExportTools add-on from the Thunderbird Add-ons Manager and restart Thunderbird.
  • Go to Local Folders in the folder navigation (or a subfolder of it if you prefer), bring up the right-click menu and select Import/Export and then Import mbox file. Choose Import directly one or more mbox files and click OK. This then brings up a standard browser window from which you can select the file or files that you created earlier, and click Open. Each file is then imported into a folder with the same name as the file, which is why you chose the file names carefully.
  • Move the folders to Hermes: at this point you have the folders in Thunderbird, but they are local folders. If you tried to shortcut this by importing directly into an IMAP folder you will have discovered that it does not work. However copying or moving from local folders to IMAP folders does work. The easiest way to do it in bulk is to select all of the local folders and "drag and drop" them onto the IMAP account (in which case they become top level folders) or an existing folder (in which case they become subfolders).

    If the folders will not move, suspect that the conversion of the separators has not been done correctly. The symptom of incorrect separators is that the local folders appear normal but cannot be copied to an IMAP folder.

One problem with the above packf/Thunderbird route is that neither of these tools implements the mbox format correctly, and therefore it is possible that message text will be modified accidentally (in particular lines matching the regular expression /^>*From /). It also can get confused by "Resent:" and "Replied:" headers that exmh might have prefixed to messages after they have been delivered. Finally, it does not preserve the delivery date of the message, which some IMAP mail clients (e.g. the Apple iOS one) use to define the display order of messages. This is why we recommend the above mh-export/mailutil route instead.

Mail filtering

If you had a non-trivial .forward file which was splitting mail into multiple folders on delivery, you are likely to want to do the same on Hermes. For simple filtering requirements the Hermes Webmail Service provides a convenient menu-based interface.

If you have more elaborate requirements, you need to write a Sieve filter file and upload it.

The Sieve base language and various expansions are fully explained in a number of RFCs. There are also various Sieve tutorials and examples.

The simple filtering facilities work by generating a Sieve file which you can download and use as a starting point for a more elaborate one of your own.

Hermes does not allow you to install a syntatically invalid Sieve file, and whilst it cannot check that the logic is correct, the language makes it hard to lose mail accidentally. If you avoid using the discard command you are unlikely to lose mail, though it may end up in the wrong place.