Department of Computer Science and Technology

Dr Markus Kuhn

NFS filer access from laptops and home computers

This page describes how to arrange access to Lab NFS servers from self-managed Unix/Linux laptops and home computers.

Why use NFS? Although slightly more complex to set up, NFS is preferable to other remote file access protocols (sshfs, webdav, smb) because it provides the best approximation to Unix file system semantics, and in many benchmarks it is vastly faster. (Examples: a repeat find command over hundreds of files took 0.03 seconds over NFS versus 38 seconds over sshfs, both via a 30 Mbit/s cable modem connection. Many version-control tools (e.g. git) and access-control tools (e.g., chmod) work much better via NFS than the alternatives, because they rely on correct implementation of the rename() or stat() system calls. Symbolic links do not work well with the others either.)

These instructions differ from those on NFS access from sm.cl.cam.ac.uk in that they do not install an LDAP client. This is because LDAP clients have a tendency to destabilize a system if connection with the LDAP server is lost, which is not suitable for machines that are not permanently on the departmental LAN.

Instructions

These instructions were tested on Ubuntu Linux 14.04 and 16.04.

  • Set up the Computer Laboratory VPN service (requires packages network-manager-strongswan and strongswan-plugin-eap-mschapv2)
  • Install required packages:
    $ sudo apt-get install krb5-user nfs-common autofs libnss-extrausers
    
  • If the krb5-user installation script asks for your Kerberos realm, enter
    AD.CL.CAM.AC.UK
    
  • Check /etc/krb5.conf has
    [libdefaults]
    default_realm = AD.CL.CAM.AC.UK
    
  • To start and configure two helper processes needed by Kerberos-authenticated NFS (gssd) and NFSv4 (idmapd), make sure that
    • /etc/default/nfs-common has
      NEED_GSSD=yes
      
    • /etc/idmapd.conf has
      Domain = cl.cam.ac.uk
      
  • To allow glibc to find users not only in /etc/{passwd,group}, but also in /var/lib/extrausers/{passwd,group}, edit /etc/nsswitch.conf such that it has
    passwd: files extrausers
    group:  files extrausers
    
  • Install static offline versions of the data files that Lab machines obtain via LDAP:
    $ curl https://www.cl.cam.ac.uk/~mgk25/offline-ldap/ldap-autofs.tar.gz | \
      sudo tar xvzf - -C /etc --wildcards 'auto*'
    $ sudo mkdir -p /etc/auto.master.d
    $ sudo ln -sf /etc/auto_master /etc/auto.master.d/cl.autofs
    $ mkdir -p /var/lib/extrausers
    $ curl https://www.cl.cam.ac.uk/~mgk25/offline-ldap/ldap-users-noshell.tar.gz | \
      sudo tar xvzf - -C /var/lib/extrausers passwd group
    

    Repeat the above curl|tar commands every now and then, as needed, so your system learns about new users, groups, and automount mappings.

  • Reboot to get gssd, idmapd and automount running
  • Start the CL VPN connection
  • Get a Kerberos ticket for yourself:
    $ kinit $CRSID
    

    Your ticket will be stored in /tmp/krb5cc_$UID and you can display it (especially its expiry time) with klist. You will have to get a new ticket each time you reboot your computer (if /tmp is a RAM disk), and each time your ticket expires. You can extend the expiry time in /etc/krb5.conf.

    If your login name equals your CRSID, kinit will not need any argument. If you haven't set default_realm in /etc/krb5.conf, you'll have to type kinit $CRSID@AD.CL.CAM.AC.UK each time.

  • Get a Kerberos ticket for your kernel and root user (machine credential, required by the mount command). There are several ways to do this:
    • After you obtained/refreshed your own ticket with kinit, you can simply copy that each time for root to use as the machine credential:
      $ sudo cp /tmp/krb5cc_$UID /tmp/krb5ccmachine_AD.CL.CAM.AC.UK
      
    • If you have root access to a Lab-managed Linux PC, you can copy from there the file /etc/krb5.keytab onto your own computer, such that root can fetch its own machine credentials. This gives your root user a Kerberos identity separate from your own, such that root processes do not have by default your access privileges on the filer.
    • You can also ask sys-admin to issue a new /etc/krb5.keytab specifically for your machine. [Do this if your machine is a server that you want to offer Kerberos-authenticated services (sshd, nfsd, etc.). Your server should then also have a fixed global IP address, as well as DNS and reverse-DNS entries.]
  • You should now be able to access e.g. /auto/anfs/www or /auto/userfiles/$CRSID/unix_home.
  • Optionally, you may also want to add the CL package repository:
    $ curl https://www.cl.cam.ac.uk/deb/cl.cam.ac.uk-archive-keyring.gpg | sudo apt-key add -
    $ sudo bash -c "echo 'deb http://www-repo-deb.cl.cam.ac.uk/deb CL all' > /etc/apt/sources.list.d/cl.list"
    $ sudo apt-get update
    

    You can then install the CL package that creates symbolic links for /homes, /anfs, /usr/groups, etc.:

    $ sudo apt-get install cl-autofs-dirs
    
  • Optionally, you may want to install a NetworkManager hook script to make the VPN run smoother:
    $ sudo curl -o /etc/NetworkManager/dispatcher.d/cl-vpn 'https://www.cl.cam.ac.uk/~mgk25/offline-ldap/cl-vpn'
    $ sudo chmod go-w,a+x /etc/NetworkManager/dispatcher.d/cl-vpn
    $ sudo ln -sf ../cl-vpn /etc/NetworkManager/dispatcher.d/pre-down.d/
    
    This does two things:
    • Rewrite the routing table such that only traffic to CUDN addresses is routed via the VPN, while all other Internet traffic goes via your own router as usual. (With the default settings, some web sites can become very slow and if you are behind a NAT router, your other local devices on e.g. 192.168.0.0/16 become unreachable. Solution inspired by maj1's draft web page on VPN split tunnels.)
    • Invite the automounter to unmount unused mounts before the VPN goes down, to increase the likelihood of clean unmounts. Without this, some machines can hang during shutdown.
  • Optionally, you may want to install a GUI-tray alternative to kinit, to remind you when your ticket expires:
    $ sudo apt-get install krb5-auth-dialog
    
  • Read the note “Choice of NFS version”.

Enjoy!

Old, low uid/gid numbers

For safety reasons, the passwd/group files provided here excludes some very early users whose uid is still outside the range 1100-9499 and groups whose gid is outside the range 1100-9999 (and 500-599), to reduce the risk of collisions with ones already allocated otherwise on your computer.

If this affects you, please ask sys-admin to change your uid/gid (and chown all your files accordingly) to conform to the current CL uid/gid allocation conventions.

FAQ list

Why amend /etc/{passwd,group}?

The Linux/Unix kernel file system knows users and groups only as uid/gid integer numbers. If you want to see the “ls -l” command to display the owners and groups of files correctly by their human-readable names, these have to be listed in your passwd and group databases. Otherwise you would just see integer numbers.

Will amending passwd allow others to log in?

No, the passwd file provided in ldap-users-noshell.tar.gz has the “shell” field set to /bin/false, to prevent login.

What about macOS?

At present, due to a lack of mutually supported encryption types, Kerberized NFS access to the CL filer is limited to OS X versions before Yosemite (which still permit single-DES). The departmental filer elmer currently runs NetApp’s old “Data ONTAP 8.1 7-mode” operating system, which is out of date and does not support AES, the encryption type required for NFS compatibility with macOS Sierra 10.12 or newer. This could be fixed by upgrading the filer to the newer “Clustered Data ONTAP” operating system, which may happen sometimes in 2018.

Why does this not work on Raspian 8 and Debian 8.0 “jessie”?

The nfs-common package on Debian 8.0 seems broken. It still contains an old /etc/init.d/nfs-common init startup script rather than systemd service files. As a result it does not start gssd or idmapd. Until Debian migrate their nfs-common package properly to systemd, you'll have to start these services manually. (Ubuntu 16.04 shows how an nfs-common ported to systemd looks like.)

To-do list

  • Finish and document merge-pwgr, a tool for merging of passwd/group files on operating systems that lack an equivalent of Debian package libnss-extrausers, which simply loads from /var/lib/extrausers/{passwd,shadow,group} any users and groups with uid/gid ≥ 500.
  • Create an cl-offline-ldap.deb Debian package that does most of these steps automatically.
  • Test on other operating systems (assistance welcome!).
  • At the moment, “ls -l” can be slow (1–2 seconds) on large directories. Investigate whether this is due to TCP congestion control failing (a 10 Gbit/s filer interface firing data at a 30 Mbit/s cable modem), possibly resulting in packet loss and TCP timeouts. (pb22 and maj1 have observed such TCP timeouts affecting 100 Mbit/s PCs on the CL LAN.) See also: Improving TCP network congestion with Appropriate Byte Counting
  • Ubuntu 14.04 can hang during shutdown if there are still NFS directories mounted, but the VPN is already gone. (https://bugs.launchpad.net/ubuntu/+source/autofs5/+bug/614731)
    • The cl-vpn hook script now largely fixes this problem by causing NetworkManager to automatically cleanly unmount the filer via before disconnecting the VPN.
    • Explore alternatives to NetworkManager for connecting and disconnecting the VPN, preferably something that integrates well with systemd’s dependency mechanism, such that the automounter is only started after the VPN and shut down before the VPN goes.
    • Or figure out NFS mount parameters (e.g., soft, retrans) that prevent hanging the shutdown process.
  • Investigate how to improve the VPN setup:
    • On Ubuntu 16.04, the NetworkManager strongswan GUI plugin to configure a new VPN connection appears broken (adding a new connection does not offer strongswan option, and editing an existing connection gives error message: “Could not find VPN plugin service for 'org.freedesktop.NetworkManager.strongswan'.”). However, VPN connections configured under Ubuntu 14.04 still work after an upgrade to Ubuntu 16.04. The configuration is found in the file /etc/NetworkManager/system-connections/VPN connection 1, which probably can also be installed and adjusted manually.
    • Figure out ways to make the VPN available continuously (e.g. to enable Kerberized login or cron jobs).
    • Figure out good ways to make VPN connectivity to inside the Lab available across a home LAN, via a dedicated VPN home gateway (e.g., a Raspberry Pi or an OpenWRT router), such that all home computers have comparable access to machines on VLAN 290, without requiring any VPN software on each home computer.