Testing

The monoprop test suite requires installing the package from source. If you follow the installation guide in Building from source, you should have all the tools needed to run the tests.

Running the testsCopy Anchor Link

Python testsCopy Anchor Link

The Python tests run with pytest. Without MPI:

just test              # build, then the Python and C++ suites
just test-py           # the Python suite against whatever is installed

The recipes run the suites at one thread per rank (monoprop_NUM_THREADS=1 OMP_NUM_THREADS=1 OMP_DYNAMIC=FALSE, on the test command only); tests that need other thread counts launch their own processes.

This collects two suites: monoprop's own (tests/) and the one belonging to the monoprop-bench-tools workspace member (packages/monoprop-bench-tools/tests/). The member has to be installed for the second to import, which is what the workspace-test dependency group is for:

uv sync --all-groups --all-extras   # or: uv sync --group workspace-test --all-extras

The plain test group covers monoprop's suite alone; it is deliberately free of workspace members so it can be installed against a built wheel in CI.

The MPI tests need an MPI-enabled build, which the default source build does not produce. The just recipes build an MPI-enabled extension and launch the suite under mpiexec:

just test-mpi                  # build with MPI, then run every leg
just test-py-mpi "1;2;4" -m mpi # MPI-marked tests across a rank matrix

To do it by hand, build with MPI on, then run under mpiexec with --no-sync so each rank reuses that build. --reinstall-package and --no-cache force a genuine rebuild, since uv does not key its build cache on the environment:

monoprop_ENABLE_MPI=ON \
    uv sync --all-extras --reinstall-package monoprop --no-cache
mpiexec --allow-run-as-root -n 2 uv run --no-sync python -m pytest tests --with-mpi

monoprop_ENABLE_MPI also provisions mpi4py as a build input; see Building from source for why, and for the config-settings equivalent. Installing the mpi extra alone only provides mpi4py and does not enable C++ MPI support. Accordingly, the CI matrix has explicit cache-free MPI build lanes on Linux x86-64, Linux ARM64, and macOS; only those lanes run the suite under mpiexec.

C++ unit testsCopy Anchor Link

The C++ tests run through CTest. The build tree is produced by uv sync (via scikit-build-core); do not invoke cmake --preset to configure directly, as the project requires scikit-build-core's build environment to configure correctly. Once the tree exists, run the tests with:

uv sync --all-extras
ctest --test-dir build/editable/Release --output-on-failure

With MPI, reuse the MPI-enabled uv sync from the Python MPI section above, then run CTest against the same tree:

monoprop_ENABLE_MPI=ON \
    uv sync --all-extras --reinstall-package monoprop --no-cache
ctest --test-dir build/editable/Release --output-on-failure

Or simply just test-mpi.

The full MPI rank matrix (several rank counts) is driven by a helper script:

ctest -S tools/ctest-mpi-matrix.cmake -VV

CI gives every compiler and MPI combination a distinct setup-uv cache suffix, because uv's cache key does not include build environment variables.

A matrix job reports one check per combination, so its check names change whenever a lane is added or removed. Each matrix workflow therefore ends with a small aggregation job — All tests in test.yml, All flake checks in nix.yml — that runs with if: always(), depends on the matrix, and fails unless the matrix succeeded or was skipped. Branch protection requires those stable names, so the matrix dimensions can change without touching the ruleset.

CTest runs each Boost case as its own process, so an MPI build's MPI_Init probes every fabric device per case, whether or not the test sends anything. monoprop_TEST_EXCLUDE_MPI_FABRIC=ON (default) skips that probe for the single-process serial variants only; multi-rank variants keep the full component set, since they exchange real messages. Turn it off by adding SKBUILD_CMAKE_DEFINE="monoprop_TEST_EXCLUDE_MPI_FABRIC=OFF" to the MPI build above.

CTest registers every Boost case individually as a serial variant. When the build has MPI enabled and a launcher is found, it also registers the whole suite once per rank count in monoprop_MPI_TEST_PROCS, labelled mpi and mpi-<n> — one entry per rank count rather than per case, because the ranks have to reach the same collectives. Each MPI entry times out after 600 seconds, preventing a collective deadlock from occupying the runner indefinitely.

The launch-time thread policy is part of the registration, not of the test runner: every discovered case, per case and in the whole-suite variants, runs with monoprop_NUM_THREADS=1 OMP_NUM_THREADS=1 OMP_DYNAMIC=FALSE, so ordinary cases see one store per rank. The runner itself never sets monoprop_NUM_THREADS, so running the binary by hand uses your environment: set those variables yourself to reproduce a CTest case.

A few cases also run again as separate openmp_env_* entries, each a fresh process with its own launch environment. They cover monoprop_NUM_THREADS=1 and =3, the OpenMP default with the variable removed from the environment (the case checks that nothing supplied it), and OMP_PROC_BIND=false for the affinity check, and openmp_env_root_t4 runs the runtime contract at an exact team of four. They are needed because a propagator reads its thread budget once, when it is constructed. Kernel cases rerun under OMP_THREAD_LIMIT=2 and OMP_DYNAMIC=TRUE (openmp_env_kernels_*) to test the worksharing helper's reduced-team contract; no sharded operation runs under those settings, which its launch contract excludes.

The fixed-team phase primitive (detail/sharded/Team.h), with its section limit, has its own fresh-process entries, sharded_team_env_t1, _t2, _t4 and _t4_passive (the last with OMP_WAIT_POLICY=PASSIVE). Each sets monoprop_NUM_THREADS and OMP_NUM_THREADS to T with OMP_DYNAMIC=FALSE, because the primitive assumes the launch supplies exactly T workers. Run them with ctest --test-dir <build> --no-tests=error -R '^sharded_team_env_'; the T=4 entries need four cores.

The owner-initialized shard state (detail/sharded/State.h), the direct-buffer construction and propagation engine (detail/sharded/Construction.h) and the snapshot-safe evaluation (detail/sharded/Evaluation.h) follow the same pattern with sharded_state_env_t1, _t2 and _t4, sharded_construction_env_t1, _t2 and _t4 and sharded_evaluation_env_t1, _t2 and _t4. Select all of them with -R '^sharded_'. evolved_coefficients_env_t2 and _t4 rerun the evolved_operator_coefficients() cases with several shards per rank, against evolved_operator_terms() at the same T; term_expectation_values_env_t2 and _t4 likewise rerun the term_expectation_values() cases against expectation_value(). The unit-test executable replaces the global operator new/delete family (cpp/tests/AllocationProbe.cpp) so these cases can inject real allocation failures; the library and installed consumers are unaffected. Their oracles are independent references and the root at the same launch; bitwise comparisons with a separately built runtime run from separate processes (below).

The sharded runtime's own launchesCopy Anchor Link

sharded_api_tests.cpp checks the public C++ surface at compile time: the removed partition interface, selector and headers are absent, each negative probe paired with a positive control that shows the probe can see what it looks for. sharded_root_tests.cpp drives the root's public operations with private shard inspection, actual worker observation and failure injection, rerun as sharded_root_env_t1, _t2 and _t4 and sharded_root_env_openmp_default_t3 (no monoprop_NUM_THREADS, OMP_NUM_THREADS=3). The fused-sweep cases rerun as fused_env_t2 and _t4. Each fixed-T entry is a fresh process; no case changes T inside a process.

ctest --test-dir build/editable/Release --output-on-failure --no-tests=error -LE mpi
ctest --test-dir build/editable/Release --output-on-failure --no-tests=error -R '^(sharded_root|fused)_env'

Multi-process cases of the sharded runtime have their own launches, one per P ranks x T threads, labelled mpi. Each sets monoprop_NUM_THREADS, OMP_NUM_THREADS and monoprop_TEST_EXPECT_TEAM to T with OMP_DYNAMIC=FALSE, leaves binding to OpenMP (OMPI_MCA_hwloc_base_binding_policy=none), and names the world size in monoprop_TEST_SHARDED_RANKS; a multi-rank case skips without it, so the whole-suite variants never run one at their default budget.

ctest --test-dir build/editable/Release --output-on-failure --no-tests=error -L mpi

The Python integration tests launch a fresh interpreter per T from a parent that is not itself an MPI job (they skip inside one), so run them from an ordinary pytest process:

monoprop_TEST_LEGACY_PYTHON="$LEGACY_ENV/bin/python" uv run --no-sync pytest tests/test_sharded_openmp.py -v

monoprop_TEST_LEGACY_PYTHON names the interpreter of an environment built separately from a checkout from before the partition runtime was removed (with its default, legacy build). Its processes run at monoprop_PARTITIONS=T for a bitwise comparison at the same (1, T) geometry — an independently implemented runtime, never loaded into the same process; without it those cases skip, so a verification run sets it. The same file checks the extension's runtime identity (the attribute and the NUL-terminated literal, and neither pre-removal marker), that the removed constructor arguments raise TypeError at every level (the _core classes, MajoranaPropagator, PauliPropagator and the generated adapters) and that a monoprop_PARTITIONS set before launch changes nothing, bit for bit. In an MPI build with an MPI launcher on PATH, the test_multirank_* cases also run the probe on several processes, each rank writing its own document: at two ranks against a legacy MPI environment rank by rank and bit for bit, across the geometries (1, 4), (2, 2) and (4, 1), and on three processes under splitmix, with the gathered maps, energies and gradients within the map tolerance.

tests/test_mpi_thread_support.py (MPI builds) launches tests/mpi_thread_support_probe.py, which sets mpi4py.rc.thread_level and imports mpi4py.MPI before monoprop, as a host would: FUNNELED, SERIALIZED and MULTIPLE complete real construction, graph, energy, gradient and retained-functional work on one rank and on two ranks of two threads, against a COMM_SELF reference; SINGLE is refused on one rank and aborts two ranks promptly; and a call from another Python thread fails fast at FUNNELED and MULTIPLE. Each case reads back the level MPI actually provided and skips, saying so, when it does not exercise the case's route.

The MPI build also registers mpi_failure_* entries, labelled mpi and mpi_failure, from cpp/tests/mpi_failure_driver.cpp. That driver has its own main(), so the unit-runner glob excludes it. Each entry launches two ranks through cpp/tests/run_mpi_failure_scenario.cmake, with a 30-second timeout. Rank 0 fails in one of these ways while its peer follows the normal phase:

• mpi_failure_sharded_* drive the sharded root at two threads per rank: rank 0's worker 1 throws in a replay finish; the primary throws in a traversal before any physical round, in a cosine pass while a replay round is posted, or while packing queries with the count round posted; rank 1 sends malformed query blocks;
• the low-level engine and the exported replay functions, on the sole shard of a root launched with one thread per rank, with explicit two-worker kernel budgets: a failure (on the caller or on worker 1 of the cosine kernel) while a replay Ticket is posted, or while a PendingAlltoallv is posted; a worker of the threaded scan or self probe throws before the first query exchange; a worker of the incoming decode, frozen probe or scatter throws while the peer enters the response round;
• host-initialized MPI provides only SINGLE (mpi_failure_insufficient_thread_level_single);
• a guarded call arrives from a std::thread under FUNNELED, SERIALIZED or MULTIPLE.

A failure entry passes only on a prompt nonzero exit with the expected diagnostic. The thread-support acceptance entries (mpi_failure_library_initialized_level, mpi_failure_host_initialized_funneled, _serialized, _multiple) instead require a clean exit after real operations at two threads per rank, on the world and on a reordered split communicator, checked against a single-process root; every entry prints the requested and the provided level. A timeout means a rank was left blocked, and it fails the entry. The wrong-thread cases rely on the launcher ending the peers of a process that aborts, which is Open MPI's default. ctest -L mpi_failure runs just these entries. If the MPI library grants a different thread level than a scenario needs (more than SINGLE for the refusal, or not exactly the requested level for an acceptance run), that scenario reports itself as skipped instead of passing.

CI runs the non-MPI and MPI labels separately with --no-tests=error, so an MPI lane fails if its MPI variant was not registered. Standalone MPI-enabled C++ test executables initialize MPI before constructing propagators and destroy the propagators before finalizing MPI. Select the groups locally with just test-cpp and just test-cpp-mpi, which are the same commands CI runs. The just test-mpi '1;2;4' recipe configures and runs the corresponding C++ CTest rank entries as well as the Python rank matrix.

Run just code-coverage to collect coverage in separate serial and MPI-enabled Coverage builds and render monoprop-coverage/index.html. The MPI collection runs the serial and MPI CTest labels plus the MPI-marked Python tests with two ranks. Python ranks write parallel coverage shards, while C++ coverage uses atomic gcov counters. The code-coverage-collect and code-coverage-aggregate recipes also drive the QA workflow, which publishes the combined reports to Codecov and SonarQube. MPI collection requires covered lines in multiple detail/mpi sources so a silently serial build cannot pass as MPI coverage.

Golden baselinesCopy Anchor Link

A refactor that is meant to change no result is checked against a captured baseline rather than by reading the diff. tools/capture-baseline.py propagates every tests/data/*.msgpack fixture at a spread of cutoffs and cutoff types, plus a few hand-written qubit problems, and dumps each run's term count, its full (monomial indices, coefficient) set and its expectation value:

just capture-baseline golden   # seed, once, on an unmodified tree
just diff-baseline             # rebuild, re-capture, diff against the golden

Terms are dumped in the engine's own returned order, not sorted, and diff-baseline is a byte-wise diff. That is deliberate: the order runs from SplitmixHash through probe order, MPI owner routing and insertion order into floating-point accumulation order, so a silent reorder is a regression, not an implementation detail.

For a change that reorders terms on purpose, --compare REF CAND --tol holds the term set and the term count exactly and compares coefficients and energies to a relative tolerance instead. Reach for it only when the reordering is the point; it is not a way to wave through a diff that was supposed to be stable.

The capture is rank-local, so under mpiexec each rank writes its own manifest-rank<r>.json. Output lands in .baseline-capture/, which is gitignored.

Adding testsCopy Anchor Link

Python testsCopy Anchor Link

Place new test files under tests/, named test_<module>.py. We suggest following pytest guidelines.

Simple unit testCopy Anchor Link

Use pytest.mark.parametrize for straightforward parametric tests:

import pytest
from monoprop.pauli import Pauli

class TestPauli:
    def test_default_qubits_are_range(self):
        p = Pauli("XYZ")
        assert p.string == "XYZ"
        assert p.qubits == (0, 1, 2)

    @pytest.mark.parametrize(
        ("string", "expected"),
        [
            ("IZ", Pauli("Z", 1)),
            ("ZI", Pauli("Z", 0)),
        ],
    )
    def test_identity_letters_dropped(self, string, expected):
        assert Pauli(string, (0, 1)) == expected

Data-driven integration testCopy Anchor Link

The problem fixture in tests/conftest.py is parametrized over the three general reference problems: LiH, RX/RZ/RY/RZ rotations, and a random circuit. Each invocation loads a fresh problem from tests/data/ using load_problem() in tests/reference_data.py. Its explicit case IDs make failures easy to identify. The serial_comm and comm fixtures are also provided by conftest.py.

from monoprop import MajoranaPropagator


def test_energy_matches_reference(problem, comm):
    mp = MajoranaPropagator(
        problem.operator,
        problem.monomial_circuit.initial_state,
        cutoff=2 * problem.n_modes,
        comm=comm,
    )
    mp.propagate(problem.monomial_circuit.to_circuit())
    energy = mp.expectation_value()
    assert abs(energy - problem.exact_expval) < 1e-8

For tests that specifically require the LiH molecule and its commutator data, request lih_problem. For orbital-rotation checks using the S0 molecule, request orbital_rotation_problem.

def test_lih_reference(lih_problem, serial_comm): ...

Use @pytest.mark.parametrize with pytest.param(..., id="...") for small in-memory tables. When each scenario needs substantial or optional-library setup, separate test functions keep the setup next to its assertions and construct objects only when the test runs.

Adding a new msgpack fixtureCopy Anchor Link

Add a .msgpack file to tests/data/ following the schema in tests/data/README.md. To exercise it in every test requesting problem, add a pytest.param entry to that fixture's params list in tests/conftest.py:

pytest.param("my_new_problem", id="my_new_problem"),

If the problem is only appropriate for specific tests, define a named fixture instead:

import pytest

from tests.reference_data import load_problem


@pytest.fixture
def my_new_problem(lazy_shared_datadir):
    return load_problem(lazy_shared_datadir / "my_new_problem.msgpack")

C++ testsCopy Anchor Link

C++ tests live in cpp/tests/ and use Boost.Test. They are registered automatically via CMake: new *.cpp files in cpp/tests/ are picked up on the next configure, so no source-list edit is needed.

Simple unit testCopy Anchor Link

#include <boost/test/unit_test.hpp>

BOOST_AUTO_TEST_CASE(my_basic_check) {
    int result = 2 + 2;
    BOOST_TEST(result == 4);
}

Data-driven test using reference fixturesCopy Anchor Link

Use the ExampleDataFix fixture class from TestUtilities.h to load the same msgpack data as Python tests, and BOOST_DATA_TEST_CASE_F to parametrize over it:

#include <boost/test/data/monomorphic.hpp>
#include <boost/test/data/test_case.hpp>
#include <boost/test/unit_test.hpp>

#include "TestUtilities.h"

using namespace test_utils;
namespace utf = boost::unit_test;
namespace bdata = utf::data;

BOOST_DATA_TEST_CASE_F(ExampleDataFix,
                       my_new_test,
                       bdata::make(ds_pare_values) ^ bdata::make(ds_schrodinger_enabled),
                       pare,
                       sch_enabled) {
    const auto schrodinger_cutoff = make_schrodinger_cutoff(sch_enabled, cutoff);
    SimulatorConfig cfg{
        .schrodinger_cutoff = schrodinger_cutoff
                                  ? std::optional<unsigned int>(*schrodinger_cutoff)
                                  : std::nullopt,
        .cutoff_type = cutoff_type,
        .basis_change = basis_change,
    };
    // use data.actual_expval as the reference value
    test_evolve_build_graph<n_modes>(data, cfg, pare, data.actual_expval);
}

See alsoCopy Anchor Link

• Getting Started — installing a prebuilt release from PyPI.
• Parallelism and distribution — running across MPI ranks and shared-memory threads.
• How to Contribute — the full test and documentation workflow.