Conversation
Signed-off-by: Jakub Dzikowski <jakub.t.dzikowski@gmail.com>
Replace the sequential execSync clone loop in src/cli/commands/install.ts
with a small bounded-concurrency executor (default 4 in flight) using
spawn("git", ["clone", "--depth", "1", ...]) so independent network and
process latency overlap when several libraries are missing. The user
contract is unchanged: warm-path libraries (target directory exists and
--force is absent) still skip without invoking git for both explicit args
and restore-from-lock; failed clones still exit non-zero and do not
produce a lock entry; restore-from-lock still does not invent new lock
entries. runInstall is now async and exposes injectable CloneRunner and
concurrency options for testing. Tests cover concurrent overlap, warm-
path skipping for explicit args and restore, invalid-remote and unknown-
ref failure paths, and mixed success/failure lockfile bookkeeping.
The default local `jaiph run <file.jh>` path no longer parses the
entry module twice and no longer re-parses the full import closure
inside the spawned `node-workflow-runner` child. A new
`prepareCompile` (`src/transpile/compile-prep.ts`) walks the entry
plus its transitive `.jh` imports exactly once and returns a
`CompilePrep` record (`{ entryFile, workspaceRoot, astByFile }`).
`src/cli/commands/run.ts` reuses the entry AST for the banner,
passes the prep into `buildScripts(..., prep)` so emission skips
per-file reads/parses, and writes a deterministic JSON snapshot to
`<outDir>/.jaiph-compile-prep.json`. The spawned runner reads it
via the internal `JAIPH_COMPILE_PREP_FILE` env var and forwards
the deserialized prep to `buildRuntimeGraph`, which consumes the
cached `Map<file, jaiphModule>` instead of re-walking the import
closure on disk. The env var is set only for non-Docker host runs;
`jaiph run --raw`, `jaiph test`, and Docker launches keep their
existing parse paths. User-visible run semantics — banner, hooks,
run artifacts, summaries, return values, exit codes, and
`__JAIPH_EVENT__` streaming — are unchanged. New tests corrupt
every source file on disk after `prepareCompile`, then exercise
`buildScripts` + `buildRuntimeGraph` to prove no second parse
happens, and cover cross-module workflow/rule/script resolution
plus the serialize → deserialize → graph round-trip across the
parent → child process boundary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add design/2026-05-15-parser-compiler-simplification.md documenting five load-bearing refactors (ModuleGraph, Expr-AST collapse, visitor-table validator, unified catch/recover, tokenizer + RD parser) plus five secondary improvements (Trivia/CST split, typed Arg[], single-pass validator walk, Diagnostics collector, validator/runtime decoupling). Append all ten as standalone, #dev-ready tasks to QUEUE.md in the recommended implementation order. Also drop .jaiph/language_redesign_spec.md (superseded). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace three divergent module-discovery strategies (validator callbacks, transpiler re-parse paths, and the file-system walk in buildScripts) with a single ModuleGraph representation. Both validate(graph) and emit(graph, outDir) now operate entirely in-memory; ValidateContext and the optional prep cache are gone. parsejaiph is provably fs-free, enforced by a stub-fs test against the full fixture corpus. LSP edits and full compiles share the pipeline — only the graph root differs.
Around ten formatter-only fields — leadingComments on imports / script
imports / channels / const decls / test blocks, configLeadingComments,
trailingTopLevelComments, configBodySequence, topLevelOrder, bareSource
on return, tripleQuoted flags on literal/return/log/logerr/fail/send/
const, and prompt / script bodyKind / bodyIdentifier discriminators —
are removed from jaiphModule, WorkflowStepDef, ConstRhs, SendRhsDef,
WorkflowMetadata, ImportDef, ScriptImportDef, ChannelDef, ScriptDef, and
TestBlockDef and re-homed in a new parallel Trivia store
(src/parse/trivia.ts) keyed by AST-node identity plus a small
ModuleTrivia record. The parser exposes parsejaiphWithTrivia → {ast,
trivia}; legacy parsejaiph drops trivia for validator / transpiler /
runtime / loadModuleGraph. The formatter (emitModule(ast, trivia, opts?))
is Trivia's only consumer. New tests pin the invariants:
trivia-ast-shape.test.ts (AC1, type-level), trivia-grep.test.ts (AC2),
and roundtrip.test.ts (AC3, parse → format → parse → format bit-for-bit
on every fixture under examples/ and test-fixtures/golden-ast/fixtures/).
Golden AST fixtures regenerated to drop the moved fields. User-visible
contracts (CLI behavior, format round-trip, run artifacts, banner,
hooks, exit codes, __JAIPH_EVENT__ streaming) are unchanged. Implements
design/2026-05-15-parser-compiler-simplification.md § Appendix A.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace every call-bearing node's `args: string` + `bareIdentifierArgs?:
string[]` pair with a single `args?: Arg[]`, where each Arg is either
`{ kind: "literal"; raw }` or `{ kind: "var"; name }`. The parser does
the bare-identifier classification once at parse time, and the
validator and emitter consume the typed list directly without any
downstream re-parse of the raw `args` string. Drops the
`validateBareIdentifierArgs` helper; its scope check now lives in the
per-step validator that already walks the call. Adds grep and
AST-shape regression tests so neither `bareIdentifierArgs` nor an args
re-parse can reappear under src/parse or src/transpile.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the three "managed call that yields a value" encodings — the `run` statement, the `run_capture` / `ensure_capture` / `prompt_capture` / `run_inline_script_capture` / `match_expr` ConstRhs branches, and the `managed:` sidecar on `return` / `log` / `logerr` (with placeholder strings like `"__match__"` and `"run inline_script"`) — with one tagged `Expr` union used everywhere a value can appear. `ConstRhs` and `SendRhsDef` are gone; the placeholder strings are gone; the sidecar field is gone. `WorkflowStepDef` collapses from 14 variants to 8 (`exec`, `const`, `return`, `send`, `say`, `if`, `for_lines`, `trivia`), with `exec` covering the prior `run` / `ensure` / `run_inline_script` / `prompt` / `shell` / standalone `match` statement shapes and `say` covering the prior `log` / `logerr` / `fail`. Parser, validator, formatter, emitter, runtime, and golden AST fixtures are migrated in lockstep; a new `src/types-shape.test.ts` enforces the acceptance criteria (no placeholder strings, exactly 8 step variants, no exported ConstRhs / SendRhsDef). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
src/transpile/validate.ts used to descend each workflow's / rule's step tree four times before the main check loop finished -- collectKnownVars, collectPromptSchemas, validateImmutableBindings, and the per-step validator -- each re-implementing the same recursion over if / for_lines / catch / recover with subtly different rules. The three pre-pass helpers are gone. A new walkStepTree descends the tree once, accumulating knownVars, promptSchemas (gated by withPromptSchemas so rules skip schema collection), and enforcing immutable-binding / script-collision rules inline through a shared bindings map (with a fresh inner map under each for_lines body so loop iterators only shadow inside the body). It emits a flat FlatStepEntry[] of every step in tree order with the enclosing catch / recover failure binding attached; the main per-workflow and per-rule validator loops iterate that flat list non-recursively. walkStepTree's internal descend is now the only recursive helper in the file that takes a WorkflowStepDef[]. All existing E_VALIDATE error messages and locations are preserved bit-for-bit. New tests in validate-single-walk.test.ts pin both invariants (no reappearance of the deleted helpers by name; at most one WorkflowStepDef[] walker). Docs updated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace fail-fast `throw jaiphError(...)` in the validator with a new `Diagnostics` collector (`src/diagnostics.ts`) that accumulates every recoverable error per compile. `collectDiagnostics(graph)` walks the import closure and returns the populated collector; the legacy `validateReferences(graph)` is now a thin wrapper that throws the first sorted diagnostic so existing per-error tests and the script-emit path stay intact. Each top-level validation unit (per-import block, per-rule walk, per-rule step, per-workflow walk, per-workflow step, per-test-block step, per-channel route) is wrapped in `diag.capture(fn)` so a bailout from one error unwinds only that unit; the four leaf helpers (validate-ref-resolution, validate-string, validate-prompt-schema, shell-jaiph-guard) still throw and every caller captures them. `jaiph compile` routes through `collectDiagnostics`, prints the full sorted set (stderr lines or a single JSON array under `--json`), and exits non-zero on any non-empty set. New tests in `src/transpile/diagnostics-collector.test.ts` pin all five acceptance criteria, including a three-error fixture and a source-tree allowlist scan over remaining `throw jaiphError(` sites. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the 1,441-LoC validate.ts monolith — two near-identical inner
walkers (validateRuleStep, validateStep) plus the five-check call-shape
sequence repeated at 12+ sites — with a two-file split. validate.ts
(~430 LoC) keeps the outer layer: import / channel-route / test-block
checks and walkStepTree (the single descent that builds knownVars,
promptSchemas, and the flat step list). validate-step.ts (~1,025 LoC)
holds the per-step visitor: one validateStep(step, ctx) entry, a
VALIDATORS: Record<WorkflowStepDef["type"], StepValidator> table with
one row per variant, a validateExpr(expr, ...) dispatcher over the 8
Expr.kind values, and a single validateCallable(expr, ctx) helper that
runs the five managed-call-shape checks once for both call (run) and
ensure_call (ensure). Rule-vs-workflow differences are captured in a
Scope value (WORKFLOW_SCOPE / RULE_SCOPE) with allowSteps (single
set-lookup gate at the top of validateStep), runRefExpect, and
withPromptSchemas. Every E_VALIDATE message and source location is
preserved bit-for-bit. New tests in validate-visitor.test.ts pin the
invariants: a ≤700-line cap on validate.ts (AC1), a JSON snapshot over
every validate-* txtar fixture asserting each diagnostic's
{ code, line, col, message } bit-for-bit (AC3), and an "unknown step
type" test asserting a synthetic variant produces exactly one
internal: no validator for step type "…" diagnostic in both scopes
(AC4). The diagnostics-collector fatal-allowlist test now sums throw
jaiphError / diag.error counts across both files. Docs updated in
docs/architecture.md, docs/contributing.md, and docs/grammar.md.
Implements design/2026-05-15-parser-compiler-simplification.md §
Refactor 4.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`validate-step.ts` used to import `tripleQuotedRawForRuntime` from `src/runtime/orchestration-text.ts` to compute "what the runtime will see" for triple-quoted match-arm bodies — a one-way compile-time dependency on runtime semantics. Move the helper into the parser layer as `canonicalizeTripleQuotedString` in `src/parse/triple-quote.ts`; both the validator and the runtime now import it from `src/parse/`, and the `src/runtime/orchestration-text.ts` wrapper is deleted. New tests pin the invariants: `no-runtime-imports.test.ts` greps every non-test `*.ts` under `src/transpile/` and fails on any `from "…/runtime/…"` import, and `canonicalize-triple-quoted.test.ts` walks every triple-quoted match-arm body in `test-fixtures/` and `examples/` and asserts bit-for-bit parity against an inlined legacy baseline.
`src/parse/steps.ts` used to contain three near-identical 100+ line
functions — `parseEnsureStep`, `parseRunCatchStep`, `parseRunRecoverStep`
— that parsed the same `<host-step> <keyword> (binding) { body }` shape
and differed only in which host step they decorated and the literal
keyword. Their body parser, `parseCatchStatement` (~280 lines), was a
stripped-down copy of `parseBlockStatement` recognizing only a fixed
subset of statement forms, so the same fix had to land in two places
and divergence wasn't always caught by tests. Collapse all four into
one `parseAttachedBlock(keyword, host)` entry point in `steps.ts` that
parses the bindings and dispatches body statements through the **same**
`parseBlockStatement` used at the top level — no mini parser remains.
The host side moves to a single `parseRunOrEnsure` helper in
`workflow-brace.ts`. `src/parse/steps.ts` drops from 757 → 141 lines;
every existing parse error message and location is preserved bit-for-bit
(verified by snapshot fixtures), and the full parser/validator/emitter
golden corpus passes byte-for-byte. A new `parse-attached-block.test.ts`
pins the invariant that any statement form accepted at top level is
accepted identically inside `catch (e) { … }` and `recover(e) { … }`
without parser changes inside the catch/recover code path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
parseBlockStatement used to dispatch each statement form via an ordered
cascade of startsWith + regex tests (run async before run, prompt before
bare assignment, etc.), so adding a keyword meant finding the right slot
and any reordering risked changing which branch fired. The cascade is
replaced by a STATEMENT: Record<string, BlockHandler> table keyed by the
leading keyword: the dispatcher tokenizes the first identifier on the
trimmed line, looks it up, and invokes the matching handler — which
returns a result, returns null to fall through, or calls fail(...). The
shared per-line context is now a BlockCtx record threaded into every
handler. Assignment-shape guards run in applyAssignmentGuards before
the lookup. Surface syntax and every existing parse-error message /
line / col are preserved. New tests pin the invariants: an error
snapshot test compares { file, line, col, code, message } for every
fixture in parse-errors.txt against parse-errors-snapshot.json, and a
synthetic-keyword test patches STATEMENT at runtime to prove adding a
top-level keyword is a two-place change (STATEMENT row + JAIPH_KEYWORDS
entry). The wider tokenizer rewrite remains future work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.