NFS filer access from laptops and home computers

This page describes how to arrange access to Lab NFS servers from self-managed Unix/Linux laptops and home computers.

Why use NFS? Although slightly more complex to set up, NFS is preferable to other remote file access protocols (sshfs, webdav, smb) because it provides the best approximation to Unix file system semantics, and in many benchmarks it is vastly faster. (Examples: a repeat find command over hundreds of files took 0.03 seconds over NFS versus 38 seconds over sshfs, both via a 30 Mbit/s cable modem connection. Many version-control tools (e.g. git) and access-control tools (e.g., chmod) work much better via NFS than the alternatives, because they rely on correct implementation of the rename() or stat() system calls. Symbolic links do not work well with the others either.)

These instructions differ from those on NFS access from sm.cl.cam.ac.uk in that they do not install an LDAP client. This is because LDAP clients have a tendency to destabilize a system if connection with the LDAP server is lost, which is not suitable for machines that are not permanently on the departmental LAN.

Instructions

These instructions were tested on Ubuntu Linux 14.04, 16.04, 18.04, 20.04, as well as Raspbian 10.

• Set up the UIS Computer Laboratory VPN service (requires packages network-manager-strongswan and strongswan-plugin-eap-mschapv2) or (better) the VPN2 service
• Install required packages:

  $ sudo apt-get install krb5-user nfs-common autofs libnss-extrausers curl
  

• If the krb5-user installation script asks for your Kerberos realm, enter

  DC.CL.CAM.AC.UK
  

• Check /etc/krb5.conf has

  [libdefaults]
  default_realm = DC.CL.CAM.AC.UK
  

• To start and configure two helper processes needed by Kerberos-authenticated NFS (gssd) and NFSv4 (idmapd), make sure that
  • /etc/default/nfs-common has

    NEED_GSSD=yes
    

    Update: Ubuntu 22.04 no longer uses /etc/default/nfs-common. The systemd unit rpc-gssd.service is started if the file /etc/krb5.keytab exists (see below). Other NFS-related configuration options can now be dropped into /etc/nfs.conf.d/*.conf.

  • /etc/idmapd.conf has

    Domain = cl.cam.ac.uk
    

• To allow glibc to find users not only in /etc/{passwd,group}, but also in /var/lib/extrausers/{passwd,group}, edit /etc/nsswitch.conf such that it lists for the passwd and group databases also extrausers as a source to query, e.g. as in

  passwd: files extrausers
  group:  files extrausers
  

• Install static offline versions of the data files that Lab-managed Linux machines obtain via LDAP:

  $ curl https://www.cl.cam.ac.uk/~mgk25/offline-ldap/ldap-autofs.tar.gz | \
    sudo tar xvzf - -C /etc --wildcards 'auto*'
  $ sudo mkdir -p /etc/auto.master.d
  $ sudo ln -sf /etc/auto_master /etc/auto.master.d/cl.autofs
  $ mkdir -p /var/lib/extrausers
  $ curl https://www.cl.cam.ac.uk/~mgk25/offline-ldap/ldap-users-noshell.tar.gz | \
    sudo tar xvzf - -C /var/lib/extrausers passwd group
  

  Repeat the above curl|tar commands every now and then, as needed, so your system learns about new users, groups, and automount mappings.

• Reboot to get gssd, idmapd and automount running
• Start the CL VPN connection
• Get a Kerberos ticket for yourself:

  $ kinit $CRSID
  Password for $CRSID@DC.CL.CAM.AC.UK:
  

  Enter your departmental Kerberos password. This is the password you use to log into Lab-managed Linux or Windows machines. You will likely have received one on a paper slip when you first arrived. (This is independent of your UIS/Raven password.)

  Your Kerberos ticket will be stored in /tmp/krb5cc_$UID and you can display it (especially its expiry time) with klist. You will have to get a new ticket each time you reboot your computer (if /tmp is a RAM disk), and each time your ticket expires. You can extend the expiry time in /etc/krb5.conf (within limits imposed by the departmental Active Directory servers).

  If your login name equals your CRSID, kinit will not need any argument. If you haven't set default_realm in /etc/krb5.conf, you'll have to type kinit $CRSID@DC.CL.CAM.AC.UK each time.

• Get a Kerberos keytab and/or ticket for your kernel and root user (machine credential, required by the mount command). There are several ways to do this:
  • If you have root access to a Lab-managed Linux machine, you can copy from there the file /etc/krb5.keytab onto your own computer, such that root can fetch its own machine credentials. This gives your root user a Kerberos identity separate from your own, such that root processes do not have by default your access privileges on the filer.
  • You can also ask sys-admin to issue a new /etc/krb5.keytab specifically for your machine. But first chose a name for your computer and ask sys-admin to add it to the record of your computer in the departmental inventory table, if there isn't one already. This is because the keytab contains the name of the computer it was issued for (see “sudo klist -k”), even though that doesn't have to match the hostname if you need it only for NFS client access. [You certainly will need an individually issued /etc/krb5.keytab in the name of your machine if you want it to be a server that offers Kerberos-authenticated services (sshd, nfsd, etc.). Your server should then also have a fixed global IP address, as well as DNS and reverse-DNS entries.]
  • [Experimental:] There are also various hacks possible to use your user ticket as a machine credential. For example, after you obtained/refreshed your own ticket with kinit, you can simply copy that each time for root to use as the machine credential:

    $ sudo cp /tmp/krb5cc_$UID /tmp/krb5ccmachine_DC.CL.CAM.AC.UK
    

• You should now be able to access e.g. /auto/anfs/www or /auto/userfiles/$CRSID/unix_home.
• Optionally, you may also want to add the CL package repository:

  $ curl https://www.cl.cam.ac.uk/deb/cl.cam.ac.uk-archive-keyring.gpg | sudo apt-key add -
  $ sudo bash -c "echo 'deb http://www-repo-deb.cl.cam.ac.uk/deb CL all' > /etc/apt/sources.list.d/cl.list"
  $ sudo apt-get update
  $ sudo apt-get install cl-debian
  

  You can then install the CL package that creates symbolic links for /homes, /anfs, /usr/groups, etc.:

  $ sudo apt-get install cl-autofs-dirs
  

  (There may be an error message related to /home, which can be ignored.)

• Optionally (not needed for VPN2), you may want to install a NetworkManager hook script to make the VPN run smoother:

  $ sudo curl -o /etc/NetworkManager/dispatcher.d/cudn-vpn 'https://www.cl.cam.ac.uk/~mgk25/offline-ldap/cudn-vpn'
  $ sudo chmod go-w,a+x /etc/NetworkManager/dispatcher.d/cudn-vpn
  $ sudo ln -sf ../cudn-vpn /etc/NetworkManager/dispatcher.d/pre-down.d/
  

  This does two things:
  • Rewrite the routing table such that only traffic to CUDN addresses is routed via the VPN, while all other Internet traffic goes via your own router as usual. (With the default settings, some web sites can become very slow and if you are behind a NAT router, your other local devices on e.g. 192.168.0.0/16 become unreachable. Solution inspired by maj1's draft web page on VPN split tunnels.)
  • Invite the automounter to unmount unused mounts before the VPN goes down, to increase the likelihood of clean unmounts. Without this, some machines can hang during shutdown.
• Optionally, you may want to install a GUI-tray alternative to kinit, to remind you when your ticket expires:

  $ sudo apt-get install krb5-auth-dialog
  

• Read the note “Choice of NFS version”.

Enjoy!

FAQ list

Why amend /etc/{passwd,group}?

The Linux/Unix kernel file system knows users and groups only as uid/gid integer numbers. If you want to see the “ls -l” command to display the owners and groups of files correctly by their human-readable names, these have to be listed in your passwd and group databases. Otherwise you would just see integer numbers.

Will amending passwd allow others to log in?

No, the passwd file provided in ldap-users-noshell.tar.gz has the “shell” field set to /bin/false, to prevent login.

What about macOS?

Kerberized NFS access to the departmental filer from macOS Sierra or newer has become available with the October 2019 filer upgrade to NetApp's newer “Clustered Data ONTAP” operating system, which finally has a Kerberos NFS implementation that is compatible with macOS. Basic instructions for setting up automounting and LDAP on macOS are already available. However, we are still looking for some macOS equivalent of Debian's extrausers mechanism, to avoid the use of LDAP outside the department.

Does this work on Windows Subsystem for Linux (WSL)?

Yes, kind of, but you need to use a customized version of Microsoft’s WSL2 Linux kernel where in the file Microsoft/config-wsl you've set the options

CONFIG_RPCSEC_GSS_KRB5=y
SUNRPC_DISABLE_INSECURE_ENCTYPES=y

Sadly, as of February 2023, Microsoft still has left CONFIG_RPCSEC_GSS_KRB5 off by default in their own Linux kernel, and therefore there is no Kerberos support for NFS in it.

Try my kernel linux-msft-wsl-5.15.90.1 and provide an absolute path to it in the kernel parameter of your %UserProfile%\.wslconfig file, e.g. as in

[wsl2]
kernel=C:\\Users\\mgk25\\Downloads\\linux-msft-wsl-5.15.90.1

Use a recent version of WSL2 and enable systemd such that Ubuntu can start gssd automatically.

With this I got automounted NFSv4.1 with sec=krb5 to work.

Open questions: why does NFSv3 get “permission denied” and doesn't even request a Kerberos ticket? Why does NFSv4.1 id mapping not work?

Why does this not work on Raspian 8 and Debian 8.0 “jessie”?

The nfs-common package on Debian 8.0 was broken (lacked a systemd unit for rpc-gssd). Upgrade to Debian 10.0 or newer.

Ideas for future work

• Create an cl-offline-ldap.deb Debian package that does most of these steps automatically.
• Test on other operating systems (assistance welcome!) and newer releases.
• At the moment, “ls -l” can be slow (1–2 seconds) on large directories. Investigate whether this is due to TCP congestion control failing (a 10 Gbit/s filer interface firing data at a 30 Mbit/s cable modem), possibly resulting in packet loss and TCP timeouts. (pb22 and maj1 have observed such TCP timeouts affecting 100 Mbit/s PCs on the CL LAN.) See also: Improving TCP network congestion with Appropriate Byte Counting
• Ubuntu 14.04 can hang during shutdown if there are still NFS directories mounted, but the VPN is already gone. (https://bugs.launchpad.net/ubuntu/+source/autofs5/+bug/614731)
  • The cudn-vpn hook script now largely fixes this problem by causing NetworkManager to automatically cleanly unmount the filer via before disconnecting the VPN.
  • Explore alternatives to NetworkManager for connecting and disconnecting the VPN, preferably something that integrates well with systemd’s dependency mechanism, such that the automounter is only started after the VPN and shut down before the VPN goes.
  • Or figure out NFS mount parameters (e.g., soft, retrans) that prevent hanging the shutdown process.
• Investigate how to improve the VPN setup:
  • On Ubuntu 16.04, the NetworkManager strongswan GUI plugin to configure a new VPN connection appears broken (adding a new connection does not offer strongswan option, and editing an existing connection gives error message: “Could not find VPN plugin service for 'org.freedesktop.NetworkManager.strongswan'.”). However, VPN connections configured under Ubuntu 14.04 still work after an upgrade to Ubuntu 16.04. The configuration is found in the file /etc/NetworkManager/system-connections/VPN connection 1, which probably can also be installed and adjusted manually.
  • Figure out ways to make the VPN available continuously (e.g. to enable Kerberized login or cron jobs).
  • Figure out good ways to make VPN connectivity to inside the Lab available across a home LAN, via a dedicated VPN home gateway (e.g., a Raspberry Pi or an OpenWRT router), such that all home computers have comparable access to machines on VLAN 290, without requiring any VPN software on each home computer.
• Finish and document merge-pwgr, a tool for merging of passwd/group files on operating systems that lack an equivalent of Debian package libnss-extrausers, which simply loads from /var/lib/extrausers/{passwd,shadow,group} any users and groups with uid/gid ≥ 500.