How we measure accuracy — and what the number doesn't claim.
Most AI plan check tools claim accuracy without telling you what they tested. PlanCheckNow publishes its benchmark, its fixtures, and the exact language of its claim so architects can verify before they hand over a 60-MB drawing set.
Engine v6 regression floor
100%
of patterns in our 50-fixture LADBS correction benchmark caught by engine v4. We don't ship engine changes that drop this number.
Why this number is a floor, not a ceiling
The fixtures and the engine prompt were co-developed — writing the benchmark forced us to articulate every LADBS rejection pattern precisely, and that articulation went into the prompt itself. So 100% is the bar we won't ship below, not a promise that every real-world plan gets approved.
What this does NOT claim
That every PCN-reviewed project gets approved on cycle 1. Real plans are harder than synthetic fixtures, and reviewer discretion always exists. We'll publish a real customer cycle-1 approval rate once we've accumulated enough submissions to be honest about it.
How the benchmark works
- Step 1
Patterns sourced from LADBS-published guidance
Every fixture mirrors a real correction pattern from a LADBS Plan Check Correction Sheet template, an Information Bulletin, or a published code section. No invented criteria — only patterns the city itself documents.
- Step 2
Each fixture contains exactly one known issue
A synthetic plan extract (sheet titles, dimensions, code callouts) is constructed to embed one specific rejection pattern. The expected finding is annotated with the code reference + key phrases an architect would recognize.
- Step 3
The real engine reviews each fixture end-to-end
Every fixture runs through the production review engine — the same prompts, the same calibration anchors, the same jurisdiction rubric your project would hit. No mocks, no cherry-picked configuration.
- Step 4
A finding counts as a catch only with strong evidence
The engine must emit at least one finding whose code reference matches the expected section OR whose title contains two or more of the expected pattern phrases. Partial matches don't count. Findings on the wrong issue don't count.
- Step 5
Re-run on every engine change
Each engine prompt change increments the engine version, and the benchmark re-runs against every fixture. The number on this page is the floor we won't ship below — anything lower is a regression and blocks deploy.
Note on scope
The 100% benchmark above is computed against LADBS-area fixtures — our deepest calibration zone. Engine v5 also covers all 50 US states, 6 Canadian provinces, 19 major metros, and 6 climate overlays at coarser depth. Outside the LA five, expect more "needs verification" notes; calibration depth scales with customer demand. See full coverage map →
Why this number is defensible
The fixtures are public
The full benchmark — every fixture's synthetic plan text, expected code reference, and expected finding phrases — is documented in the engineering repo. Architects, expediters, and other AI plan check vendors can verify each test.
Source rubric is the city's own
Every pattern traces back to a LADBS Plan Check Correction Sheet template, Information Bulletin, or code section. No invented criteria. No selectively chosen patterns we know we catch.
The matcher is strict
A finding counts as a catch only when the engine cites the right code OR uses two or more of the expected pattern phrases. Surface-level keyword overlap doesn't qualify.
Re-published on every change
The number on this page is computed against the current production engine — not against an old or favorable version. Engine improvements only show here if the benchmark agrees they improved.
What this number isn't
Not an approval rate. We measure whether the engine catches the right issues, not whether the architect implements the fix or whether the reviewer approves the resubmittal. Cycle-1 approval depends on the architect closing every finding, the city reviewer's judgment, and a dozen variables outside any software's scope. We'll publish a real cycle-1 approval rate once we've accumulated enough customer outcomes to be honest about it.
Not a substitute for licensed review. PlanCheckNow is a pre-submittal review tool. It does not replace the stamped review of a licensed architect or engineer, and it does not replace the formal plan check by your authority having jurisdiction. The final stamp belongs to the licensed professional and the final approval belongs to the AHJ.
Not the marketing ceiling. The benchmark covers 50 of the most common LADBS rejection patterns; LADBS has hundreds. Real plans are messier than synthetic fixtures. And the fixtures inform the prompt design — so a 100% score is the regression floor we won't ship below, not the proof that PCN will get every project approved. The truth lives in real cycle-1 outcomes, which we'll publish once enough customers have submitted.
Run your plans through the same engine.
First review free. No credit card. See exactly what the engine catches on your plans, with sheet-level corrections you can act on.