We're currently building a 64 node cubic FPGA based system using Terasic DE4 boards which contain Altera Stratix IV parts. Below is a demonstration of an initial four board configuration. The renderer and the inter-board communications hardware was all written in Bluespec. High-level control is via NIOS processors running on each board.
The PCIe to SATA adapter board
We've built an adapter board which breaks out the PCIe channels to SATA links for board-to-board communications. These links happily run at over 6Gb/s per channel.
Many thanks to Altera and Teraisc for their support and to EPSRC for funding some for the equipment.