Forma CLI Docs Bootstrap Evaluation Suite

Purpose

Validate that Forma CLI, embedded product docs, and Agent-facing skill output can guide Human and Agent collaboration from an empty project to a valid first content workflow.

Pressure Tests

• test-cases/docs-backed-agent-bootstrap-pressure
• test-cases/scenario-driven-workspace-bootstrap-pressure
• test-cases/forma-cli-skill-context-budget-pressure
• test-cases/workspace-design-discovery-pressure
• test-cases/example-accelerator-boundary-pressure

Gate Usage

This suite is the Phase 1 no-example bootstrap gate and the Phase 2 example-boundary gate. It should prove that an Agent can start from an empty initialized workspace, design one first content slice from human domain language, and verify it without loading or copying examples by default, while still keeping explicit example acceleration available only on request.

Run this suite before considering changes ready for review when the change affects:

• docs/agents/**;
• docs/workspace/** pages used by empty-workspace setup;
• forma skills output or embedded skill projection;
• forma init output or generated Agent runtime skill content.

Minimum evidence:

• wrong-config baseline reports the expected diagnostic;
• guided kind: term + taxonomy: spaces content group setup passes;
• scenario-driven bootstrap clarifies missing requirements, reuses settled choices, and implements only the first approved slice;
• ordinary workspace operations load the lightweight Skill and core guide without pulling all bootstrap docs into context;
• check, create, list, inspect, and view render pass for the guided content group;
• isolated-page health warnings are reported as relationship feedback and can be cleared by adding explicit links.
• context pressure evidence records the loaded skills/docs and approximate word counts for Agent-facing guidance.
• Phase 2 boundary coverage proves the example accelerator loads only for explicit example-backed requests or accepted-brief fast paths.

Context budget targets:

• project-local skills/forma-cli/SKILL.md stays under 200 words;
• forma-cli-core stays under 500 words and contains no worked examples;
• agents.workspace-design-discovery stays under 900 words unless a split reference doc is introduced;
• agents.workspace-bootstrap stays under 1,100 words unless a split reference doc is introduced;
• workspace.first-slice-config stays under 700 words and is the default first-slice config reference;
• context-pressure optimizations do not hide escalation conditions: load full workspace.configuration for runtime values, named types, entryRef, migration/debug, local identity defaults, or full reference behavior;
• ordinary read or health workflows load the project-local skill, forma-cli-core, and applicable configured guidelines without loading setup references.
• example acceleration remains optional and is never required for empty-workspace bootstrap, read-only health, or default discovery.

Phase 2 entry gate:

• no-example grant applications evidence is recorded;
• workspace design discovery evidence is recorded for at least one non-project-management domain;
• example accelerator boundary evidence is recorded for explicit-request and accepted-brief flows;
• wrong-config baseline reports config.unknownNodeKind;
• ordinary read or health workflows do not load discovery, bootstrap, schema, template, or example accelerator docs;
• cargo test -p forma-core, forma check --json, and forma workspace health --json pass.

Phase 2 completion evidence:

• workspace-design-discovery-pressure has at least one recorded non-example domain run;
• example-accelerator-boundary-pressure has default and explicit-example boundary evidence;
• context budget records forma-cli-core, agents.workspace-design-discovery, agents.workspace-bootstrap, workspace.first-slice-config, and agents.workspace-example-accelerator;
• repository checks pass.

Evaluation Boundary

• Focus on docs-backed bootstrap from empty projects.
• Do not require examples/getting-started-workspace or this repository's project knowledge structure.
• Treat configured content groups as user-defined patterns, not built-in Forma domain objects.
• Treat Skill context as a budget. Keep the project-local Skill and built-in core guide as routers; load detailed docs only for matching workflows.

Evidence Or Execution Notes

2026-06-29 No-Example Grant Applications Run

Temporary workspace: /private/tmp/forma-no-example-phase1.JfWlhJ.

Scenario:

• Human domain: grant application tracking.
• First slice: applications.
• Deferred relationships: funder records, investigators, budget documents, and compliance reviews.
• Examples were not loaded or copied.

Loaded guidance:

• skills/forma-cli/SKILL.md
• forma skills get forma-cli-core
• forma docs get agents.workspace-bootstrap
• forma docs get workspace.configuration
• forma docs get workspace.spaces
• forma docs get workspace.schemas
• forma docs get workspace.templates

Context budget evidence:

122  skills/forma-cli/SKILL.md
477  docs/agents/forma-cli-core.md
1027 docs/agents/workspace-bootstrap.md

Verification results:

• forma init --name "Grant Applications" --json: passed and wrote only .forma.md plus .agents/skills/forma-cli/SKILL.md.
• forma skills list --json: passed and returned only the built-in forma-cli-core skill.
• forma config inspect --json: passed after correct config and reported spaces.applications.
• forma check --json: passed after correct config and sample content.
• forma create applications ... --json: passed for two application entries.
• forma list --space applications --json: passed and returned both entries.
• forma inspect applications/stem-outreach-expansion.md --json: passed and returned the configured metadata.
• forma workspace health --json: initially reported isolated-page warnings, which matched the expected first-slice relationship feedback.
• After adding explicit links between the two applications, forma workspace health --json: passed.

Wrong-config baseline:

• A deliberately wrong imported config node using kind: space and top-level template originally passed silently during the pressure run.
• Runtime diagnostics were updated so the same baseline now reports config.unknownNodeKind with status: warning and expected kinds taxonomy, term, types, or view.

Execution note:

• A shell quoting issue turned --input amountRequested="$75000" into an empty value because $75000 was expanded by the shell. The temporary sample files were corrected manually. Future command examples that include dollar amounts should quote or escape $ carefully.

2026-06-30 First-Slice Reference Budget Split

Default first-slice bootstrap now loads workspace.first-slice-config instead of full workspace.configuration.

Context budget evidence:

122  skills/forma-cli/SKILL.md
486  docs/agents/forma-cli-core.md
413  docs/agents/workspace-design-discovery.md
1028 docs/agents/workspace-bootstrap.md
281  docs/workspace/first-slice-config.md
377  docs/workspace/spaces.md
482  docs/workspace/schemas.md
294  docs/workspace/templates.md

Full workspace.configuration remains available as a 999-word reference for runtime values, named types, migration details, and full configuration behavior, but it is no longer the default first-slice config doc.

2026-08-30 Public Docs And Agent Routing Run

The public docs keep product behavior and reusable examples; repository authoring and deployment guidance remains in internal knowledge. Documentation IDs and builtin skill IDs are unchanged.

• Offline inventory: 26 docs, 36 valid relative documentation links; whitespace word count reduced from 10,476 to 8,818. This is a size proxy, not tokenizer usage.
• Entry skill: 106 words. Default generated core/design/bootstrap skills: 367/210/314 words. All 5 builtin and 10 configured guideline skills were discovered and read successfully, including the two guidelines with full projection.
• Independent read-only health scenario loaded the entry, core, and applicable audit guideline, without design/bootstrap/schema/template/example references; config, check, and health passed with zero diagnostics.
• Independent no-example bootstrap created one captures content group, template, table view, and two approved interlinked records. Preview, create, list, inspect, view render, explain, check, and health passed. Integers and fractions retained numeric types; quoted zero-padded identifiers retained string values.
• Quoted numeric inputs and decimal notation for an integer field failed preview with schema.type.invalid; no invalid records were written.
• Removing the links produced four isolated-page warnings; restoring them returned health to zero diagnostics and restored both files byte-for-byte.
• An imported kind: space node with top-level template produced config.unknownNodeKind with warning status and exit 0. This proves unknown-kind detection, not independent validation of the top-level template field.
• A separate explicit example request loaded the accelerator, inspected only relevant config/template files, and proposed what to reuse or omit without writing files.
• CI=true mise run check passed, covering package checks/lint/builds, 404 frontend tests, 48 script tests, Rust tests, and the Zed target check. No browser, server, deployment, or release acceptance was performed.

Temporary fixtures were retained at /private/tmp/forma-forward-captures.1cBnj3 and /private/tmp/forma-invalid-space.iqdNSQ. The forward run exposed missing term-id naming guidance, now documented; default entry listing was verified with --space, while field values were verified through inspect and the configured table.