[pyramation]

Summary

Fixes #169. Before this change, the wrapper could parse less than the query it was given and report no error.

stringToPtr sized the input buffer with Emscripten's lengthBytesUTF8 and then wrote it with stringToUTF8. The two count unpaired surrogates differently:

• lengthBytesUTF8 counts 4 bytes for every two code units.
• stringToUTF8 writes 3 bytes for each code unit.

So the buffer came out too small, the write stopped early, and the end of the query was dropped. Depending on where the cut fell, the second statement disappeared or the parser threw a bogus syntax error. An embedded U+0000 ended the C string early in the same way.

// before
const len = wasmModule.lengthBytesUTF8(str) + 1;   // undercounts lone surrogates
wasmModule.stringToUTF8(str, ptr, len);            // stops at len → tail dropped
// after
if (str.includes('\0')) throw new Error('Query must not contain a NUL (U+0000) character');
const bytes = utf8Encoder.encode(str);             // lone surrogate → U+FFFD (what a pg driver sends)
const ptr = wasmModule._malloc(bytes.length + 1);
wasmModule.HEAPU8.set(bytes, ptr); wasmModule.HEAPU8[ptr + bytes.length] = 0;

The change is made in templates/index.ts and templates/full/index.ts, then copied into versions/*/src/index.ts. HEAPU8 was already in EXPORTED_RUNTIME_METHODS; it is now also declared in the full-API types.

Why replace unpaired surrogates instead of throwing: Node Postgres drivers encode strings with the same WHATWG UTF-8 encoder, which turns an unpaired surrogate into U+FFFD. Postgres therefore receives U+FFFD, and the parse tree now reflects exactly those bytes. UTF-16 offsets don't move, because one code unit becomes one code unit. A NUL is rejected because Postgres can't receive one in query text at all.

CI (v13): the v13 job used to extract the whole wasm/ dir of the prebuilt @libpg-query/v13@13.5.7, wrapper included, so v13 tests never ran this repo's src/. It now extracts only libpg-query.{js,wasm} and runs pnpm run build:js. That is what made v13 fail on the first push: it was testing the old wrapper.

Evidence (reproduce first)

The new test/input-encoding.test.js was added to every versions/* package and wired into each test script.

• Against published libpg-query@18.1.5 (before the fix): 3 pass / 7 fail. 4, 8, 16 and 64 lone surrogates, lone low surrogates, U+FFFD read-back and NUL rejection all fail. The 1/2-surrogate and valid-pair cases pass.
• Local versions/18 build with the fix: full suite 102 / 102 pass, new file 10 / 10.
• v13 (prebuilt 13.5.7 binary + wrapper from this branch): suite passes locally and in CI.
• tsc --noEmit is clean for versions/13–17. CI builds and tests all six versions on three OSes.

Not in this PR: pnpm run copy:templates also rewrites versions/15|17|18/Makefile, because templates/Makefile.template has drifted from them. It would point them back at pganalyze tags and drop the v18 memory knobs. I left the Makefiles unchanged.

Link to Devin session: https://app.devin.ai/sessions/bcd8b2f791164247acb7a659b481efc2
Open in Devin Desktop: https://app.devin.ai/desktop/session/bcd8b2f791164247acb7a659b481efc2?variant=devin
Requested by: @pyramation

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[tenki-reviewer]

Review complete. No issues found — approved ✅.

---

This PR rewrites the string-to-WASM-pointer marshaling in libpg-query-node so query inputs are encoded to UTF-8 via TextEncoder instead of relying on the legacy lengthBytesUTF8/stringToUTF8 path, and rejects embedded NUL bytes with a thrown error rather than silently truncating. The same implementation is duplicated across the shared templates and the per-version packages for PostgreSQL 13 through 18, with typings and new input-encoding tests added to each. The verifier reviewed all scanner candidates and rejected them after checking the repository source, leaving no confirmed defects.

Files	Change
templates/index.ts, templates/full/index.ts, versions/{13..18}/src/index.ts	Replace string marshaling with TextEncoder-based UTF-8 encoding plus NUL-byte rejection in stringToPtr.
templates/full/libpg-query.d.ts, versions/18/src/libpg-query.d.ts	Update typings to match the new input-handling API surface.
versions/{13..18}/test/input-encoding.test.js	Add tests covering UTF-8 encoding, multi-byte characters, and NUL rejection.
versions/{13..18}/package.json	Wire the new input-encoding test script into each version's test setup.

No reviewable defects were confirmed; all candidate issues were rejected on verification against the source.

_{Reviewed commit: 65e440f}