Review checks

The Authoring Coach runs a versioned rubric over an authoring project — its sources, discovered fields, tests, generated ruleset, and version history — and returns an objective report: a reproducible score, per-check evidence, and the single highest-leverage improvement to make next. Reach it three ways, all backed by the same server-side rubric:

• CLI — aethis review [project_id] (add --verbose for the full table). See CLI.
• MCP — the aethis_review_project tool. See Tools reference.
• REST — POST /api/v1/public/projects/{project_id}/review. See REST API.

It is advisory only — it never blocks generation, publishing, or a decision. It is deterministic and free: the score and next_skill are computed from objective facts and need no LLM key. Opt into coach: true (with your own Anthropic key) to add a mentoring narrative on top; the deterministic layer ignores the key entirely. Every check carries a stable docs_url that links straight to its entry on this page.

---

​

How scoring works

Each check reports a status — pass, warn, fail, na, or info — with the evidence that produced it. The score (0–100) is the weighted sum of the author-actionable checks, so a project that grounds its fields, tests thoroughly, and iterates to green scores high. A check marked na (for example, version discipline before a second version exists) does not count against the score. When a project is missing a ruleset, tests, or sources, checks degrade to na / fail and the report is marked data_completeness: "thin" — never a 500. On a thin project the score is null and next_skill points at the most foundational missing step. next_skill is the single highest-leverage, author-actionable improvement — the first non-passing check in priority order (G1, P1, P2, P3, P4, L1, L2, G4), framed by its actionable_via lever so you know exactly which tool call moves the needle. Each check names an actionable_via — the author lever that changes its outcome:

---

​

Grounding

Whether the ruleset traces back to the source documents.

​

G1 · Grounded-why cites a source [#g1]

Weight 20. Every field’s why rationale should cite one of your uploaded sources, so the applicant-facing explanation traces back to a document rather than to the model’s general knowledge.

• pass — ≥90% of compiled fields carry a type=="why" note whose text or metadata references a source
• warn — 60–90%
• fail — below 60%
• na — no ruleset generated yet

Lever: source docs + guidance — upload the governing document and re-generate so each field’s rationale cites it.

​

G4 · Key-namespacing consistency [#g4]

Weight 5. Field keys should share one dotted prefix and consistent casing (e.g. child.age, child.school_type), so keys stay collision-free when rulesets compose into a rulebook.

• pass — all field keys share one dotted prefix with consistent casing
• warn — a single outlier
• fail — more than one prefix scheme
• na — no ruleset generated yet

Lever: source docs + guidance.

---

​

Process

Whether the authoring was driven by discovery, sources, tests, and iteration.

​

P1 · Discovery used [#p1]

Weight 10. Field discovery proposes canonical fields from your sources before generation, so the ruleset covers what the documents actually require.

• pass — field discovery has been run (or discovered fields exist)
• fail — discovery has not been run

Lever: run discovery.

​

P2 · Source present [#p2]

Weight 10. Generation grounds rules in your source documents; a substantive source is the difference between authored rules and guessed ones.

• pass — at least one generation source with ≥500 characters of content
• warn — a source exists but is under 500 characters
• fail — no source

Lever: upload sources.

​

P3 · Test coverage [#p3]

Weight 20. Golden test cases are the contract your ruleset is generated against; thin coverage means untested logic ships. The bar scales with the ruleset: at least max(5, ceil(field_count / 2)) cases.

• pass — test count meets the threshold for this ruleset
• warn — between one and one below the threshold
• fail — no test cases
• na — no ruleset generated yet

Lever: add test cases.

​

P4 · Test-driven iteration [#p4]

Weight 15. Iterating generate-and-test until green is how rulesets converge; a single unverified run leaves failures undiagnosed.

• pass — the final run reached a 100% pass rate, with either two or more runs or at least five tests passing one-shot
• warn — the pass rate moved but ended below 100%
• fail — a single run, or the pass rate never moved, or the final rate is below 100%
• na — no test runs yet

Lever: refine loop.

---

​

Lifecycle

Whether the ruleset is bundled and versioned for stable delivery.

​

L1 · Rulebook attached [#l1]

Weight 10. Rulesets bundled into a rulebook get versioned promotion and a stable decision surface; standalone rulesets stay orphaned.

• pass — the active ruleset is attached to a rulebook
• fail — no rulebook attached
• na — no ruleset generated yet

Lever: attach rulebook. See Nomenclature for the rulebook / ruleset model.

​

L2 · Version discipline [#l2]

Weight 10. One active version per slug, with explicit supersedes links, keeps callers on a predictable version history.

• pass — at most one active version per slug, and any superseding version links to what it replaces
• warn — a partial (one condition met)
• fail — multiple active versions for the same slug
• na — fewer than two versions exist

Lever: publish/version.

---

​

Advisory-only checks

Some qualities can’t be scored objectively — source substance beyond a length threshold, edge-case coverage, the nuance of a rationale. With coach: true, the LLM narrates these in the coaching field, but they never affect the score or next_skill. The deterministic checks above are the ground truth; the coaching narrative is mentoring on top.

---

​

See also

• Rule generation — the generate → test → refine loop the process checks measure.
• Field vocabulary — discovery and key-namespacing.
• Section discovery — splitting source legislation into sections.