Run cocotb timing tests - PandABlocks-FPGA

Any module with a timing.ini file can be tested using the cocotb test runner.

About cocotb¶

Cocotb is an open-source Python library for simulating VHDL or Verilog designs. It lets you write testbenches in Python, driving a simulator such as GHDL or NVC under the hood.

Running the tests¶

make cocotb_tests MODULES=module1,module2,... TESTS="test1,,test2,,..." SIMULATOR=nvc/ghdl

By default all testbenches are run with the NVC simulator. When specifying multiple tests, use a double comma as separator because some test names contain commas.

The runner can also be invoked directly without make:

python3 common/python/cocotb_timing_test_runner.py {module} {test}

Positional arguments:

module: Name of the module to test. Separate multiple modules with commas, or use all to test every module that has a timing.ini file.
test (optional): Name of a specific test to run.

Optional flags:

--sim {nvc,ghdl}: Simulator to use (default: nvc). NVC also collects coverage data.
--skip module1,module2,...: Comma-separated list of modules to skip.
--panda-build-dir PATH: Directory containing autogenerated HDL files (e.g. from make autogen). Defaults to /build.
-c: Save expected and actual output signal values as .csv and .html files.

Results¶

A simulation build directory named sim_build_{module_name} is created for every module tested, containing a subdirectory for each test run.

Timing errors¶

If a condition fails, a timing error is recorded; a test has failed if any timing errors occurred during its run. Timing errors are printed to the terminal and also saved to a file in the test subdirectory.

A table of expected vs actual signal values is produced as both .csv and .html, covering every signal whose value is checked during the test.

Waveforms¶

Each test subdirectory contains a wave.vcd file that can be opened in GTKWave to inspect signal waveforms.

Coverage¶

If NVC is used, a coverage report is generated in the coverage/ subdirectory of the simulation directory.

How it works¶

The runner uses a module’s timing.ini to build a schedule of input signal assignments and output condition checks, then uses cocotb to simulate and verify the module against that schedule.

Most setup information is derived from the module’s block.ini. For more complex modules (e.g. those that depend on additional HDL files), a test_config.py in the module directory can supply:

paths to required HDL files
specific signal details
the name of the top-level entity

Timing is measured in clock rising edges to synchronise assignments and condition checks. The test module is cocotb_simulate_test.py; the decorated function module_timing_test(dut) is called by cocotb, which passes the Design Under Test object for signal access.

Writing tests for new modules¶

Ensure the module directory contains:

{module}.block.ini — describes the signals involved
{module}.timing.ini — schedule of signal assignments and condition checks

Add a test_config.py if the module needs paths to extra HDL files, signal information beyond block.ini, or a non-default top-level entity name.

Modules with a timing.ini are automatically identified as testable.