Skip to content

Validate Factory files against the server only (REMOTE-2868) - #15219

Merged
advait-m merged 3 commits into
masterfrom
factory/remote-2868-spec-pointer
Aug 17, 2026
Merged

Validate Factory files against the server only (REMOTE-2868)#15219
advait-m merged 3 commits into
masterfrom
factory/remote-2868-spec-pointer

Conversation

@warp-agent-staging

@warp-agent-staging warp-agent-staging Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

The client half of REMOTE-2868. warp-server now owns the Factory file format and validates a tree with the same parser the apply path uses (warp-server#15209). The bundled factory-files skill stops carrying a copy of that format entirely.

It also repoints REMOTE-2727's spec at the superseding one.

Why the copy had to go, rather than become an offline floor

The plan was to keep the bundled schemas as a fallback for when the server is unreachable. That was wrong, and the evidence is concrete: the schemas merged in warp#15039 reject the Linear and Slack trigger aliases — teams, projects, states, issues, channels, users, itemUsers — that the apply path rewrites and accepts. Run against a tree taken from warp-server's own testdata/valid, they produce eleven false rejections on valid input.

A stale copy does not degrade gracefully. It reports a valid field as unknown, and an agent trying to reach a clean run resolves that by deleting working configuration. Reporting that a tree was not checked is recoverable and honest. Reporting the wrong answer is neither.

So there is no local fallback, and no local schema to go stale.

Changes

The validator only relays

scripts/validate_factory_files.py goes from 1,624 lines to 374. Deleted: the six JSON Schema documents, the restricted YAML reader, the JSON Schema evaluator, and the duplicated semantic checks (cron grammar, alias rules, runner shapes, Scorer outcomes, filter conflicts) — all of which live beside the parser in warp-server.

What remains: select resource files by path, refuse symlinks, upload the bytes, print what comes back. It never parses YAML, so it cannot disagree with the parser about what a document means. It does not even read schemaVersion; the endpoint reads that itself and reports an unrecognised one as a diagnostic.

--server-root or WARP_SERVER_ROOT selects a local, staging, or self-hosted root; the default is https://app.warp.dev. No credential is required — WARP_API_KEY is forwarded when the environment already has one, as an Oz sandbox does.

Three outcomes, not two

Exit Meaning
0 The server checked the tree and found no problem.
1 The server checked the tree and reported diagnostics.
2 The tree was not checked.

Exit 2 is neither a pass nor a failure, and SKILL.md tells the agent to say exactly that rather than substituting its own reading of the schema. With --json, an unreached verdict carries validated: false and no valid key at all, so there is nothing to misread.

Documentation

SKILL.md and references/validation.md drop the offline path and the exact-versus-permissive schema guidance, and gain a section explaining why a local copy must not come back. references/scorers.md stops describing forward-compatibility rules that were a property of the bundled schemas.

Tests

The corpus drops its 101 offline cases — their subjects now live beside the parser — and covers what the client still owns, against a stub server: which files it uploads, verbatim relay of the server's verdict, deferred resolutions, every route to a missing verdict, symlink refusal (in-tree and escaping), size caps, and JSON output shape.

Two guards fail the build if the copy comes back: one in Python and one in Rust, both rejecting *.schema.json under the skill or a YAML parser in the validator.

Verification

End to end, the real validator against the real warp-server handler over HTTP, using warp-server's fixtures:

  • testdata/valid — the previously-broken triage tree — passes unauthenticated, with all ten Linear/Slack aliases reported as deferred rather than rejected.
  • testdata/invalid/aliases — eleven real server diagnostics with source positions, exit 1.
  • Server unreachable — "NOT validated", exit 2, no verdict claimed.

Also: corpus 10/10, cargo test -p warp --lib ai::skills::bundled 5/5, ./script/format clean, git diff --check clean.

Blast radius

No production Rust. The only .rs file in this PR is app/src/ai/skills/bundled_tests.rs. Everything else is the bundled skill tree, its test script, and specs.

Sequencing

Merge warp-server#15209 first — the client calls its endpoint.

The two plugin mirrors (claude-code-warp#81, codex-warp#3) are re-pinned to this commit and remain drafts. No plugin release is cut and MINIMUM_PLATFORM_PLUGIN_VERSION is untouched: no published plugin ships this skill yet, so there is no stale version to force anyone off.

Co-Authored-By: Warp <agent@warp.dev>
The bundled schemas and validator were the primary path, so they were also
the drift. They become the offline floor: the validator reads the tree's
schemaVersion, confirms the server publishes it, and submits the tree to the
validation endpoint, falling back only when that cannot be done and always
saying which path ran.

Also corrects the bundled automation schema, which rejected the authoring
aliases the apply path rewrites, and the corpus case that encoded the same
mistake.

Part of REMOTE-2868.

Co-Authored-By: Warp <agent@warp.dev>
@warp-agent-staging warp-agent-staging Bot changed the title Point Factory file skill spec to REMOTE-2868 Fetch-first factory-files skill against the server-owned schema (REMOTE-2868) Aug 17, 2026
@advait-m
advait-m marked this pull request as ready for review August 17, 2026 06:45
The skill shipped a copy of the Factory file format so authoring worked
offline. That copy goes stale inside a Warp release, and a stale copy does not
degrade gracefully: it rejected the Linear and Slack trigger aliases (teams,
projects, states, issues, channels, users, itemUsers) on a tree taken from
warp-server's own testdata/valid, producing eleven false rejections. An agent
clearing those diagnostics deletes working configuration.

Reporting that a tree was not checked is recoverable. Reporting the wrong
answer is not. So the six schemas, the restricted YAML reader, the JSON Schema
evaluator, and the duplicated semantic checks are gone: 1,624 lines of
validator become 374. The client now selects resource files by path, refuses
symlinks, uploads bytes, and relays the server's verdict, so it can no longer
disagree with the parser about what a document means.

Exit 2 is new and distinct: the tree was not checked. It is neither a pass nor
a failure, and the skill tells the agent to say exactly that.

The corpus drops the 101 offline cases, whose subjects now live beside the
parser in warp-server, and covers what the client still owns: which files it
uploads, verbatim relay of the server's verdict, deferred resolutions, every
route to a missing verdict, symlink refusal, and size caps. A guard fails the
build if bundled schemas or a YAML parser reappear.

Co-Authored-By: Warp <agent@warp.dev>
@advait-m advait-m changed the title Fetch-first factory-files skill against the server-owned schema (REMOTE-2868) Validate Factory files against the server only (REMOTE-2868) Aug 17, 2026
@advait-m
advait-m merged commit bbec37f into master Aug 17, 2026
46 of 48 checks passed
@advait-m
advait-m deleted the factory/remote-2868-spec-pointer branch August 17, 2026 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant