Unit testing FPGA blocks

In order to capture block level race conditions and document behaviour at the clock tick level, there is a mechanism for unit testing FPGA blocks. It consists of a number of parts:

  • A Python simulation of the block (simulation/sim_zebra2/<block>.py)
  • A number of unit test sequences (tests/sim_zebra2_sequences/<block>.seq)
  • A Python test runner (tests/test_sim_zebra2.py)
  • The generated FPGA test vectors (tests/fpga_sequences/<block>_*.txt)
  • The block documentation plots (docs/blocks/<block>.rst)

The Python simulation models the expected behaviour of the block by returning the expected outputs from a given set of inputs. Test sequences list specific sets of inputs at specific FPGA clock tick numbers, and list the expected outputs. The test runner scans the sequence files and provides the given inputs to the Python simulation, checking the responses agains the expected outputs. If this succeeds, it produces FPGA test vectors for all sequences merged together. There is also a tool that generates plots for the documentation from named sequences, so that any examples in the documentation have been tested to be correct.

Python block simulation

Each block simulation should inherit from the Block class, implementing the on_event(event) function that returns next_event:

  • event contains the register, bit bus, position bus, and external signal changes that take place at a given FPGA timestamp.
  • next_event contains the changes that take place as a result of event, and an FPGA timestamp when the block next needs to be called

The base class will initialiase attributes on the class for each parameter, and initialise bit_out and pos_out values to be their position on the relevant bus.

Unit test sequences

A test sequence consists of the following grammar:

sequence-list = sequence*
     sequence = header event*
       header = "$" [mark] title
         mark = "!"
        event = ts ":" changes [":" changes]
      changes = "" | assignment ["," assignment]*
   assignment = name "=" value

Where:

  • title is a string describing the sequence
  • ts is the integer FPGA clock tick
  • name is the string name of the register or signal
  • value is the integer value of that register or signal

Empty lines and lines starting with # are ignored.

For example:

#########################
$ Pulse delay and stretch
1       : WIDTH=10
2       : DELAY=10
7       : INP=1         : QUEUE=2
8       : INP=0
17      :               : QUEUE=1, OUT=1
27      :               : QUEUE=0, OUT=0

This says:

  • At FPGA clock tick 1, set reg WIDTH=10, expect no changes (apart from this register set operation)
  • At tick 2, set reg DELAY=10, expect no changes
  • At tick 7, set signal INP=1, expect reg QUEUE to be 2
  • At tick 8, set signal INP=0, expect no changes
  • At tick 17, don’t set anything, expect reg QUEUE to be 1, and signal OUT to be 1
  • At tick 27, don’t set anything, expect reg QUEUE to be 0, and signal OUT to be 0

Running the test

You can invoke the test runner by doing:

python tests/test_sim_zebra2.py

This will then search for all <block>.seq files, and scan them. It will build a sequence for each one found in the file, adding one called “All” that contains all of them one after another, and will be used to generate the FPGA test vectors.

If a test title starts with “$!” instead of just “$”, then it will be marked, and only the marked tests will be run. No FPGA test vectors will be generated. This is used for running just one test while debugging the Python simulation.

The generated FPGA test vectors

When the “All” test has completed successfully, the following files will exist in tests/fpga_sequences/:

  • <block>_bus_in.txt: The bit and position bus inputs at each clock tick.
  • <block>_reg_in.txt: The registers that should be set at each clock tick.
  • <block>_bus_out.txt: The expected bit and position bus outputs at each clock tick. Note that these are 1 tick after the inputs.
  • <block>_reg_out.txt: The expected register values at each clock tick. Again, note that these are 1 tick after the inputs.

Running the FPGA test vectors

There is a new firmware commit. This includes the first block that I want you to simulate. It is the panda_pulse.vhd.

All the test stimulus vector files (generated by Tom’s framework) are already copied in the sim/panda_pulse/do directory.

The testbench is located in sim/panda_pulse/, and it is called panda_pulse_tb.v. Yes, I did use Verilog for the testbench because the file I/O is much easier.

To run the simulations:

  1. First step, you will need to re-run “build_ips.tcl” to generate the required IP for this block.
  2. Run compile.do under sim/panda_pulse/do directory, and observe the windows.

As you will this, the module passes the test and does not report any error between its outputs and expected outputs.

Generating the plots for the block level documentation

In docs/block_plot.py there is a function make_block_plot(block, title) that will generate a plot of a given sequence. You can embed this plot into the block level documentation by writing the following directive:

.. plot::

    from common.python.block_plot import make_block_plot
    make_block_plot("<block>", "<title>")

For instance:

.. plot::

    from common.python.block_plot import make_block_plot
    make_block_plot("pulse", "Pulse stretching with no delay")