Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 33 additions & 4 deletions .dagger/modules/packager/main.dang
Original file line number Diff line number Diff line change
Expand Up @@ -51,15 +51,19 @@ type Packager {
with the TypeScript compiler left external, plus the rolled-up type
declarations for it.

The install resolves through the vendored yarn.lock, which is load-bearing:
letting it resolve freely instead pulls newer transitive versions and grows
core.js by roughly a quarter (4.3 MB -> 5.4 MB).
The install is frozen against the committed bun.lock. Both lockfiles matter and
neither is redundant: yarn.lock is what the vendored sources came with, and
dropping it lets resolution pull newer transitives that grow core.js by roughly
a quarter (4.3 MB -> 5.4 MB); but it is a yarn v1 file that does not pin
everything bun resolves, so without bun.lock a rebuild days later can still
drift — observed as a bundle quietly gaining transitive modules on an unchanged
tree. Freezing makes that drift a failed install instead of a silent diff.
"""
libraryBundle(ws: Workspace!): Changeset! @generate {
let built = bun
.withDirectory("/src", ws.directory("/" + libraryPath))
.withWorkdir("/src")
.withExec(["bun", "install"])
.withExec(["bun", "install", "--frozen-lockfile"])
.withExec(["bun", "build", "./src/index.ts", "--external=typescript", "--target=node", "--outfile", "/out/core.js"])
.withExec(["tsc", "--emitDeclarationOnly"])
.withExec(["bun", "x", "rollup", "-c", "rollup.dts.config.mjs", "-o", "/out/core.d.ts"])
Expand Down Expand Up @@ -109,6 +113,31 @@ type Packager {
.withExec(["bun", "install", "-g", "typescript"])
}

"""
Run the vendored library's own test suite.

Mostly the introspector's: it scans the fixture modules under testdata and
compares the result against a recorded expectation. That is the contract
entrypoint generation is built on — the scanner's output is what the
dispatcher is rendered from — and nothing else here would notice it drifting,
since our own golden pins the renderer given a typedef, not the typedef given
a module.

The specs that need a live session (invoke, registry, api, connect) are not
vendored: they would need an engine inside this container, and what they cover
is the library's runtime rather than anything this SDK generates.
"""
libraryTests(ws: Workspace!): Void @check {
bun
.withDirectory("/src", ws.directory("/" + libraryPath))
.withWorkdir("/src")
.withExec(["bun", "install"])
.withExec(["bun", "run", "--bun", "mocha"])
.sync

null
}

"""
Container with the codegen helper compiled and on PATH at
/usr/local/bin/codegen. Read from the workspace rather than this module's own
Expand Down
21 changes: 15 additions & 6 deletions design/module-gen.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,8 +323,15 @@ maintaining it. Its golden test is "reproduce the vendored file byte-for-byte".
- **It must be a dang module.** A TypeScript module here would need the bundle
that the packager produces — a bootstrap loop.
- **Determinism.** `dagger generate` in this repo re-runs it, so a non-reproducible
build means a diff on every run. Pin the bun image by digest (as upstream
does), pin the source ref, and commit the lockfile used for `bun install`.
build means a diff on every run. That needs the bun image pinned by digest,
the source ref pinned, and **both** lockfiles committed — they do different
jobs. `yarn.lock` is what the vendored sources came with; without it,
resolution pulls newer transitives and `core.js` grows by roughly a quarter
(4.3 MB -> 5.4 MB). But it is a yarn v1 file that does not pin everything bun
resolves, so with it alone a rebuild days later still drifted — observed as
the bundle quietly gaining transitive modules on an unchanged tree. `bun.lock`
pins what bun actually installs, and the install is frozen against it so drift
fails the install instead of changing the bundle silently.
- **Staleness check.** A `dagger check` that rebuilds and diffs against the
committed bundle, so a stale artifact fails CI instead of silently shipping.
- **Marked generated.** `.gitattributes` `linguist-generated` for the bundle
Expand Down Expand Up @@ -672,10 +679,12 @@ late):
byte-for-byte**~~ — **yes**, once the generator was synced with the engine
it targets (§7 Phase 3). It did not before: enum members were miscased,
`arguments` went unescaped, and the entrypoint carried no source maps.
3. ~~**Is `bun build` output reproducible enough**~~ — **yes**, with the image
pinned by digest and the vendored lockfile in place: a second packager run
over an unchanged tree reports no changes. Dropping the lockfile is what
breaks it (§4.2).
3. ~~**Is `bun build` output reproducible enough**~~ — **yes, but only once
pinned properly.** With the image pinned by digest and both lockfiles
committed, a second packager run over an unchanged tree reports no changes.
Neither lockfile alone is sufficient: dropping `yarn.lock` changes the
resolved versions outright, and `yarn.lock` on its own still let a rebuild
drift days later (§4.3).

All three are now answered; the work they were guarding is done.
(For the record on the fetch alternative in §4.1: dang does support
Expand Down
6 changes: 6 additions & 0 deletions library/.mocharc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"extension": ["ts"],
"spec": "**/*.spec.ts",
"loader": "ts-node/esm",
"timeout": 60000
}
582 changes: 582 additions & 0 deletions library/bun.lock

Large diffs are not rendered by default.

125 changes: 123 additions & 2 deletions library/bundle/core.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

125 changes: 123 additions & 2 deletions library/bundle/introspector.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading