NutShell Cache Verification with UCAgent

This repository is the competition workspace for automated and human-in-the-loop verification of the NutShell Cache using Picker, Toffee, toffee-test, and UCAgent.

Current Status

• Baseline DUT and Toffee environment imported from XS-MLVP/Example-NutShellCache.
• Docker environment prepared in container UCAgent.
• DeepSeek deepseek-v4-pro thinking mode configured for UCAgent.
• Directed, CRV, fault-injection, and RTL coverage-target tests are included under tests/.
• Verification documentation and human-collaboration records are included under docs/.

Quick Start

Inside the UCAgent container:

cd /workspace/nutshell-cache-verify
make gen_dut
make test
make rtl-cov-sweep
make crv-rtl
make crv-mixed
make bug-hunt
make bug-hunt-adv
make jitter
make perf
make perf-regression

Current finals regression result: 41 pytest/toffee-test cases pass, followed by aggregate functional coverage and RTL selector-sweep evidence under reports/. Latest measured result is 100.0% Python functional coverage (68/68 bins), 1327 observed cross points, and 86.0% merged RTL selector-sweep coverage (1262/1454) across 39 selectors. The single VCache_coverage.dat left by a plain pytest run is only a last-test diagnostic artifact, so official RTL evidence comes from the merged sweep. Latest RTL mutation campaign result is 32/32 killed with zero invalid or timeout-only kills. make crv-rtl adds a 20-seed / 800-transaction DUT-level CRV slice, separate from the 100-seed generator convergence run. make crv-mixed adds a smaller 20-seed / 2000-transaction mixed RTL CRV slice with bounded MMIO, flush, and coherence inserts. make bug-hunt, make bug-hunt-adv, and make jitter are non-default adversarial/protocol-liveness suites; latest results are 8/8 directed bug-hunt selectors passed, 20/20 advanced bug-hunt seeds passed, and 3/3 jitter selectors passed, with 0 confirmed original RTL bugs exposed. The 20-round stability matrix repeats 18 high-risk selectors for 360/360 passes at p95 0.796s. Run coverage commands serially because they use Verilator’s VCache_coverage.dat artifact.

The first command uses Picker to export rtl/Cache.v into a Python-drivable DUT package. The second command runs the Toffee/toffee-test based regression. The third command preserves and merges per-selector Verilator coverage data.

Repository Layout

rtl/        Cache RTL under verification
src/env/    Toffee agents and environment wiring
src/ref/    Reference cache model
tests/      pytest/toffee-test testcases
crv/        standalone RTL-CRV pytest selector used by make crv-rtl
jitter/     non-default SimpleBus ready/valid jitter selectors
bughunt/    non-default adversarial selectors used by make bug-hunt
rtl86/      non-default raw selector promoted only into serial RTL coverage sweep
docs/       Test plan, coverage plan, bug tracking, AI-human collaboration records
reports/    Generated verification reports and coverage artifacts
skills/     UCAgent skill for repeatable Cache verification workflow
scripts/    Local helper scripts

Verification Goal

The high-score target is not only to pass smoke tests, but to build a reusable verification environment that covers:

• load/store hit paths
• miss detection and refill
• dirty victim writeback
• replacement pressure on same set
• MMIO bypass/path separation
• partial write mask behavior
• back-to-back and non-blocking requests
• scoreboard/reference-model consistency
• functional coverage closure
• fault-injection based verification-environment validation

License

Apache License 2.0.

Advanced Verification Commands

make test       # pytest + functional coverage + RTL coverage
make mutation   # temporary RTL mutation, expected to be killed by directed tests
make perf       # finals latency baseline and matrix metrics
make perf-matrix  # dedicated MMIO/refill/dirty-evict/burst performance matrix
make perf-regression  # baseline-plus-margin performance regression gate
make crv-long   # 100-seed constrained-random convergence evidence
make crv-rtl    # 20-seed constrained-random slice through the actual RTL DUT
make crv-mixed  # 20-seed mixed RTL CRV with MMIO/flush/coherence inserts
make bug-hunt   # non-default directed adversarial bug-search evidence
make bug-hunt-adv  # non-default seeded adversarial interleaving search
make jitter     # non-default SimpleBus ready/valid jitter evidence
make rtl-gap    # classify low-hit/uncovered RTL annotated rows
make rtl-gap-deep  # waiver-ready deep classification of remaining RTL gaps
make rtl-cov    # collect Verilator coverage from VCache_coverage.dat
make rtl-cov-sweep  # run selected tests one-by-one and merge RTL coverage