Skip to content

Detector Coverage Ledger

A single auditable view of “bug class → detector → severity → enforcement point” for the static guardrails (TOESTUB detectors, rules.v1.yaml rules, arch-check, CI gates). Derived from the graphify codebase audit (graphify-out/) and a scan of 528 fix() commits — see docs/superpowers/plans/2026-06-14-guardrail-capability-plan.md and AGENTS.md §“Perennial Bug Patterns”.

When you add or change a detector, add/update its row here. Keep GAP rows for classes we have deliberately not yet automated, so they stay visible rather than forgotten.

Severity vs enforcement (how to read the table)

Section titled “Severity vs enforcement (how to read the table)”

Findings only fail a gate per the run mode: enforce-warn (the CI toestub-scoped step) fails on Critical only; enforce-strict (the pre-commit tdd-guard, scoped to skeleton rules) fails on Warning+. So Info detectors are advisory everywhere — they surface in reports and PR review but never block. New, higher-FP detectors land at Info and are promoted only after precision is proven.

Bug classDetector idKindSeverityEnforcementNotes
Cross-crate split-brain (byte-identical body in ≥2 crates)vox/cross-crate/duplicate-logicRust batch (detectors::cross_crate_dup)InfoCI full scanSkips platform-sibling crates (*-cuda/*-metal) + trivial bodies. Exact string equality.
Catch-all-swallow (match _ => returns empty while real arms exist)vox/catch-all-swallowRust AST per-fileInfopre-push complete / CINeutral set = None/empty std containers/Default::default()/0/false/""/()/{}. Skips diverging + guarded arms.
Toolchain-bump lint wave (new clippy/rustdoc lints on rust-toolchain.toml bump)— (CI gate toolchain-lint-wave)CI jobblocking (non-required)CI, only on toolchain changeFresh cargo clean + sccache-defeated clippy+rustdoc.
Stub / hollow / empty bodyskeleton/stub, skeleton/hollow-fn, arch/empty_bodyRust AST + regexWarningpre-commit tdd-guard (skeleton, strict) / CIExisting.
Same-file near-duplicatedry-violationRust ASTWarningCIExisting; 0.98 similarity threshold. Complements cross-crate-dup.
Effect / purity / determinism violationsvox/effect/effect-net-decl, vox/effect/pure-fn-impure, vox/workflow/nondeterministicRust/Vox ASTErrorCIExisting.
Direct LLM-provider call (bypasses facade)vox/llm/direct-provider-callRust/VoxErrorCIExisting. [[project_model_agnostic_llm_boundary]].
Secret-shaped env reads / hardcoded secrets / banned cryptovox/secret/env-secret-shape, security/hardcoded-secret, vox/crypto/banRust/VoxErrorCIExisting.
Retired surfaces (decorator/crate/env/memory-API/Capacitor)vox/retired/*Rust/VoxErrorCI (CR-L6)Existing.
Import cyclesvox/import/cycle (per-file) + detect_import_cycles_in_batchRust/VoxErrorCIPer-file wired; batch path exists but is not engine-wired.
Arch layering / fan-in / LoC / orphan / generated-file driftvox-arch-check (15 rules)dep graphError (layering) / warn (rest)CIExisting.
SSOT / generated-doc driftvox ci ssot-drift (+ ssot-autoregen PR bot)aggregateblockingpre-push fast / CI / merge-queue regenExisting + PR bot from this PR.
SSOT regeneration / hook parity driftssot_regen_parity (in vox ci ssot-drift)Rust unit + CI stageblockingpre-push fast / CIVerifies scripts/ssot-regen.vox and lefthook.yml against ssot_drift_stages().
Overlay contradicts AGENTS.mdoverlay_parityRust stage, YAML claims + line regexblockingpre-push fast + PR CI linux via ssot-driftanchors re-verified each run; line-scoped.
Docs name a nonexistent CLI command/flag or call a shipped command planneddocs_cli_claimsRust stage, pulldown-cmark mentions + clap treeblockingpre-push fast + PR CI linux via ssot-driftshort flags not checked; registry-only paths checked for existence only
Guard passes when its input is missing (silent no-op)check_links / retired_symbol_check / check_docs_ssot input checksRust guardsblockingpre-push fast + PR CIretired-input branches removed; mutation-proved by temp-dir tests
Docs page unreachable by linksdocs_orphan_pagesRust stage over claims::prune link graphblockingpre-push fast + PR CI linux via ssot-driftzero reader-facing orphans, whole-tree reported; mutation-proved
Pattern drift (reqwest bypass, path/timeout/version literals, bearer header, serde-default dup)vox-drift-check (6 rules)regexWarningpre-push fast / CIExisting.

Open gaps (deliberately not yet automated)

Section titled “Open gaps (deliberately not yet automated)”
Bug classWhy still a GAPOwner / plan
Test asserts nothing (reached-but-unproven symbols — ~7,950 in the graph)High value but high FP; “meaningful assertion” is hard to define statically. Actively worked as a coverage ratchet (not a TOESTUB detector) on the semantic-coverage-wave0 branch — building a detector here would collide.semantic-coverage.v1.json ratchet + docs/src/architecture/semantic-coverage-remediation-plan-2026-06-13.md
Semantic-equivalence split-brain (same logic, different body)cross-crate-dup only catches byte-identical bodies; logically-equivalent-but-reworded copies escape it. Needs a normalized-AST or behavioral oracle.guardrail-capability-plan (future)
Per-symbol assertion depthcatch-all-swallow + the coverage ratchet cover slices; a per-symbol “is this return value asserted” detector is a larger effort.guardrail-capability-plan (future)
FP floor on weakest existing rulesThe detect-rules-bench F1 gate is 0.70; raising it requires mining real false-positives per rule and expanding fixtures.guardrail-capability-plan T5
  1. Implement DetectionRule (per-file) in crates/vox-code-audit/src/detectors/<name>.rs, or a detect_<name>_in_batch(&[SourceFile]) free fn for cross-file analysis; register in detectors/mod.rs (all_rules() + rule_count() for per-file, or an engine hook for batch).
  2. Add a pub const to diagnostics/catalog.rs.
  3. New, higher-FP detectors land at Severity::Info. Prove precision on the real corpus before proposing a promotion.
  4. Add #[cfg(test)] unit tests (pos + neg). For simple line-pattern rules, prefer a rules.v1.yaml rule with pos/neg fixtures measured by vox ci detect-rules-bench.
  5. Add a row above.