P35: Tool Set-Up Info

The letter P in P35 denotes this as a hands-on practical course!

The tools described here are provided for you to use but also please feel free to use any other computers or tools that you have to hand (such as a free copy of Modelsim on your own laptop or a version of SystemC that you have compiled for yourself). This material works on 64-bit linux machines and may work on windows and 32-bit systems.

The tools we will use this year (2017/18) will most likely be the following

General Unix Notes
Toy ESL - just to get started and then we will abandon it. SCROLL-TO.
SystemC and gtkwave SCROLL-TO.
Prazor Virtual Platform including TLM Power 3 SCROLL-TO.
Kiwi HLS Compiler SCROLL-TO.
ARM toolchain for Zynq binary generation SCROLL-TO.
Xilinx Vivado for FPGA generation SCROLL-TO.
Verilator - Verilog to C convertor SCROLL-TO.
Zynq Boards (Parcards) SCROLL-TO.
Icarus Verilog - A simple RTL simulator (use Modelsim etc instead if you prefer) SCROLL-TO.

We will probably not use Chisel or Bluespec this year.

General Unix Notes

The material in
```
/usr/groups/han
```
is automounted. You cannot see it via certain GUI file explorers unless it is mounted. It will automount itself if you execute
```
cd /usr/groups/han
```
inside a shell (terminal window).
When using values of environment variables, please note that a different syntax is needed in the shell from inside a makefile. Use
```
$(VARNAME)
```
inside a makefile and
```
$VARNAME
```
in a shell or shellscript.
If you set up environment variables in a shellscript and run that shell script the effects are lost straight afterwards because they will apply only to the fresh, nested shell created by a shell for each command run. Instead you need to use the source shell command, which is just '.' to have the shellscript run in the current shell so its effects persist. For example, if 'mysetup' contains 'export FOO=myfoo' then please use
```
 . mysetup
```

Toy ESL

Toy ESL is a very simple ESL (electronic-system level model) of a processor and memory. It can be copied from the filesystem at /usr/groups/han/clteach/socdam/toyclasses and there is also a zip file (somewhere).

You first need SystemC working. See the SystemC part of this current page SCROLL-TO.

Then follow instructions HERE BUT NOTE THAT THE SETUP FOR SYSTEMC ON THAT LINK IS POSSIBLY A LITTLE OUT OF DATE. Instead use the setup on this current page.

Icarus Verilog or VHDL Modelsim

Icarus Verilog Simulator

Icarus Verilog is a free RTL simulator. It is installed on the ACS workstations and installable on all linux laptops. The following example shoulds its baic use:

  mkdir test1; cd test1
  echo "module TEST(); initial \$display(\"Hello World\"); endmodule"  > test.v
  iverilog test.v
  ./a.out

Modelsim

Modelsim is a commercial product that is licensed to run on many computers in the Computer Laboratory. Students may get a free license from Mentor for Modelsim for their personal computers if they are Windows users but whether this works under wine on linux I sadly do not know.

Setup (using bash shell syntax):

 export PATH=$PATH:/usr/groups/ecad/mentor/modelsim/current/modeltech/bin
 export MGLS_LICENSE_FILE=1717@lmserv-mentor.cl.cam.ac.uk

Documentation : See /usr/groups/ecad/mentor/modelsim/modelsimSE6.5c/modeltech/docs

The following should run something:

        rm -rf simple_lib; vlib simple_lib
        echo "module TEST(); initial \$display(\"Hello World\"); endmodule"  > test.v
        echo "run -all;exit" > run.do
        vmap simple_lib simple_lib
        vlog -work simple_lib test.v
        vsim -c simple_lib.TEST -do "run.do"

The above works by first deleting any old library, then creating a new one, called simple_lib and then creating a component called TEST and putting in the library and then reading it out and running it.

For those interested in using modelsim extensively, you should learn how to open the GUI and view the waveforms, otherwise just use Verilog's $display statement for simple output.

For VHDL: Use vcom instead of vlog if you prefer to use VHDL.

     vcom -work simple_lib ...vhdl

For SystemC

Setup (using bash) (assuming ARCH=linux64):

    export CLTEACH=/usr/groups/han/clteach
    export SYSTEMC=$CLTEACH/systemc/current
    export LD_LIBRARY_PATH=$SYSTEMC/lib-linux64

You can use one of the pre-built SystemC distributions I have provided or download and compile your own from systemc.org. The ARCH variable must match the machine you are running on. If this variable is not already set for you then use the linux 'arch' command to find out how to set it. I have made two or more binary versions of SystemC but we can add others.

Note that 'linux64' is sometimes exchanged with 'x86_64' - these strictly mean different things but often they get interchanged.

A first, get-you-started example is as follows, (but its just as easy to start with toy class number one from $SOCDAM/toyclasses/class1):

/usr/groups/han/clteach/systemc-helloworld

or as the first toyclass in

/usr/groups/han/clteach/socdam/toyclasses

or in this zip file systemc-helloworld.zip.

If you unzip this and type 'make' it should run something, but you may have a minor problem such as the links to where SystemC is installed on the filesystem might be out of date w.r.t. the information on this page.

Note: this filesystem is automounted and it cannot initially be seen with Nautlius or other GUI-based browsers until you 'cd' to it on the command line or open the 'file:/usr/groups/han/clteach' url in firefox, or similar.

Troubleshooting

Note: please understand the operation of the following gcc flags:

    -c             (generate a .o file but do not link)
    -I/some/where  (look for #included files in /some/where)             
    -lxxx          (look for libxxx.a or perhaps libxxx.so and include for linking)
    -L/some/where  (look in /some/where for -l includes)

If you are getting an 'error while loading shared libraries: cannot open shared object file: No such file or directory' error then check your LD_LIBRARY_PATH variable is properly set.

You can check which dynamic library is missing using ldd. You will get output something like as follows if all is set well, but when there is an error the r.h.s. of one of the => operators will say '(not found)'.

ldd ./a.out 
	linux-vdso.so.1 =>  (0x00007fff4d5fd000)
	libsystemc-2.3.0.so => /usr/groups/han/clteach/systemc/systemc-2.3.0/lib-x86_64/libsystemc-2.3.0.so (0x00002b23d26ca000)
	libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003be3e00000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003be0e00000)
	libc.so.6 => /lib64/libc.so.6 (0x0000003bdd600000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003bde600000)
	libm.so.6 => /lib64/libm.so.6 (0x0000003bdda00000)
	/lib64/ld-linux-x86-64.so.2 (0x0000003bdd200000)

gtkwave viewer

Verilog change dump files (.vcd) can be generated using SystemC, Icarus, Kiwi's diosim or Modelsim etc.. Such files can be viewed with the gtkwave program.

This is installed on the ACS machines in /usr/bin/gtkwave.

Kiwi HLS

The KiwiC compiler is still experimental and fragile. The SystemC output mode is just being restored to operation (Feb 2017). If you prefer not to use Kiwi, please either hand craft your RTL or use another tool.

Kiwi Scientific Acceleration using FPGA

The HDL output from KiwiC can be put in the Zynq FPGA for execution. It can also be simulated with Icarus Verilog or modelsim. This year we shall aim to link the SystemC output from KiwiC with Prazor for rapid performance evaluation...

For general use, you need a CSharp compiler, such as mcs, and mono. Both are installed in /usr/bin on the ACS machines.

You can clone Kiwi and build from source from https://bitbucket.org/djg11/bitbucket-hprls2.git. A bitbucket account is needed.

 $ git clone  https://bitbucket.org/djg11/bitbucket-hprls2.git

A copy is installed on the Computer Lab fileserver at

  /usr/groups/han/clteach/hprls/current

You will need to set your HPRLS variable to point at the folder containing kiwipro and hpr.

  export HPRLS=/usr/groups/han/clteach/hprls/current

If you get a fresh copy of KiwiC, via git or otherwise, that is not compiled, you need fsharpc installed and you should type 'make' in the kiwipro/kiwic/src folder. KiwiC is written in FSharp but you only need FSharp installed if you wish to modify KiwiC itself. It is installed in /usr/bin/fsharpc on the ACS machines.

When built, the KiwiC compiler can be found in $(HPRLS)/kiwipro/kiwic/distro/bin/kiwic. This is shellscript. You may wish to put it on your PATH by making a link to it from your own ~/bin folder or however you prefer.

A get started demo should be available in /usr/groups/han/clteach/kiwi-starter so perhaps take a copy of that folder. It also contains a BloomFilter demo at the moment.

KiwiC generates RTL that can be compiled to the a bitstream file for the FPGA using Vivado Logic Synthesis. Vivado HLS is not needed.

Prazor VHLS Virtual Platform

The main virtual platform for this course this year is a SystemC TLM2.0 model of the Zynq FPGA on the ZedBoard, Pynq, Arty or Parallella cards. The virtual platform can run linux but we will mostly run bare metal for simplicity.

The main platform is checked out on the file server or you can build your own live copy. If you are not a C++ and automake expert it can be a struggle to build so seek help. You will need to be able to edit it, so having at least your own makefile for the final arm7smp binary will be needed. To get a live copy from source, I can request you have a user id at xparch so you can

git clone ssh://vcs@phabricator.xparch.com/diffusion/P/vhls.git

A pre-built binary for get-started ACS work should be here: /usr/groups/han/clteach/btlm/current/vhls/src/vhls/pvp10

A pre-built version with TLM_POWER3 turned on should be here: /usr/groups/han/clteach/btlm/current/vhls/src/vhls/pvp11

If you see no energy report files called something like myname.power.txt written to the working directory then you have a POWER3 off version. The sPEEDO interface provides a simulator backdoor to the POWER3 stats so that a running binary can read its own energy use.

You will need to compile your own copy of Prazor from source at some point. Instructions for compiling are split over this page PRAZOR TEMPORARY MANUAL and the README.md file in the Prazor source code. You will probably need to use autoreconf, autoconf, automake, ./configure and make.

The precise environment settings used to build a particular instance of Prazor are by convention kept in a file called setup-notes in the pvpXX folder.

They will be something like this:

export CLTEACH=/usr/groups/han/clteach
export PRAZOR=$CLTEACH/btlm/pvp11
export BOOST=$CLTEACH/boost/boost_1_48_0
export BOOST_ROOT=$BOOST
export SYSTEMC=$CLTEACH/systemc/systemc-current
export TLM_POWER3=$CLTEACH/tlm-power3
export LDFLAGS="-L$SYSTEMC/lib-linux64 -L/usr/local/lib -L$TLM_POWER3/src/.libs"
export CXXFLAGS="-I$SYSTEMC/include/ -I$SYSTEMC/include/tlm_core/tlm_2 -I$BOOST_ROOT  -I$TLM_POWER3/include -g -O2 -DSC_CPLUSPLUS=199701L -DSC_DISABLE_API_VERSION_CHECK=1 "
export TARCH=ARM32
./configure  TLM_POWER3=$TLM_POWER3 --with-tlm-power --with-speedo  --host=x86_64-pc-linux-gnu
# or without TLM3 use:  ./configure   --host=x86_64-pc-linux-gnu
make

You may need to run autoreconf --add-missing first.

The key rune that was holding us up at the start of term is that you needed to add --host=x86_64-pc-linux-gnu to the configure command line.

Zynq Boards (Parcards)

Information about the Computer Laboratory P35 Zynq and Parcards is HERE.

ARM Tools

We will need the ARM tools to make ELF binaries to run on the Zynq cards and Prazor virtual platform. The tools in question are essentially the C compiler and the assembler.

You can use the ARM tools installed natively on the Zynq cards or those installed as cross tools on the ACS workstations. Or you can install them on your laptop since they are available as a pre-built package on most linux distros.

The arm GNU tools should be available on ACS machines via /usr/bin (but only on svr-acs-01 until today (1st Feb 2018)). The precise set up for using them is best copied from Makefile.inc.arm on the vhls/images folder.

The tools are

    arm2hpdl                          arm-linux-gnueabi-dwp             arm-linux-gnueabi-gcov            arm-linux-gnueabi-objcopy
    arm-linux-gnueabi-addr2line       arm-linux-gnueabi-elfedit         arm-linux-gnueabi-gcov-4.7        arm-linux-gnueabi-objdump
    arm-linux-gnueabi-ar              arm-linux-gnueabi-gcc             arm-linux-gnueabi-gprof           arm-linux-gnueabi-ranlib
    arm-linux-gnueabi-as              arm-linux-gnueabi-gcc-4.7         arm-linux-gnueabi-ld              arm-linux-gnueabi-readelf
    arm-linux-gnueabi-c++filt         arm-linux-gnueabi-gcc-ar-4.7      arm-linux-gnueabi-ld.bfd          arm-linux-gnueabi-size
    arm-linux-gnueabi-cpp             arm-linux-gnueabi-gcc-nm-4.7      arm-linux-gnueabi-ld.gold         arm-linux-gnueabi-strings
    arm-linux-gnueabi-cpp-4.7         arm-linux-gnueabi-gcc-ranlib-4.7  arm-linux-gnueabi-nm              arm-linux-gnueabi-strip

If installing on your own machine you will probably need package gcc-arm-linux-gnueabi.

Note that the demos in the vhls/images folder in the Prazor distro include the file Makfile.inc.arm that automatically provided you have set env variable TARCH to ARM32.

export TARCH=ARM32
export LIBGCC=/usr/lib/gcc-cross/arm-linux-gnueabi/4.7

Xilinx Vivado Logic Synthesiser

We shall build some FPGA bit streams to load into the Zynq chips using Vivado. This tool has a GUI that you may like to use, but it can also be driven entirely from a Makefile that invokes a tcl script.

The tool to use is here:

   $ export XILINXD_LICENSE_FILE=27020@lmserv-xilinx
   $ export PATH=$PATH:/usr/groups/ecad/xilinx/Vivado2017/Vivado/2016.4/bin:/usr/groups/ecad/xilinx/Vivado2017/SDK/2016.4/bin
   $ vivado

Older one:

   $ export XILINXD_LICENSE_FILE=27020@lmserv-xilinx
   $ export HPRLS=/usr/groups/han/clteach/hprls2
   $ /usr/groups/ecad/xilinx/vivado_sdk_installed/Vivado/2015.1/bin/vivado

Device selection:

 Pynq wants:       set part xc7z020clg400-1  set vdefine PYNQCARD20=1        
 Parallella wants: set part xc7z010clg400-1  set vdefine PARCARD10=1                                                                          
 Zedboard wants:   set part xc7z020clg484-1  set vdefine ZEDBOARD20=1

Power modelling: you will probably want to download the xpower spreadsheet from Xilinx and see how its results compare with those from Prazor.

Get Started Mouseless Files

Broad explanation of an example mouseless setup. See the ACS-P35 section.

First take a copy of this folder and use it to make a bitstream (topfpga.bit) ksubs3.1.zip. You will have to edit folder names in the .tcl script. Run this on an acs-machine. You will have to edit or rename the link of the DUT (device under test) from primes_offchip.v to refer to a design of your own that has broadly the same signature.

Ksubs3/4 Server

The Kiwi substrate server (binary file name kiwi-ksubs3-server) is C-code to run on ARM to provide O/S services to the loaded FPGA design. You can compile this on the parcard itself (in your own folder, but run it as root). Get it via git clone from https://bitbucket.org/djg11/kiwi-ksubs3.git.

ksubs4 has both the DMA mastering and the PIO and NOC16 slave. It does not have the large amount of Xilinx IP that converts from AXI4 to AXI3, it instead connects directly to the hardened AXI4 connections at the PL boundary. The initial version, that does not have credit-based flow control, is here bitbucket.org/djg11/cbg-hpr-ksubs4a-zynq.git.

Computer Laboratory