perf: replace regex parser with a charCode scanner - #81
Open
mcollina wants to merge 1 commit into
Open
Conversation
Rewrite the parser as a hand-rolled charCodeAt scanner driven by a single 64KB Uint8Array character-class table, replacing the mediaTypeRE/paramRE regular expressions. parse and safeParse now share one internal parseHeader that returns sentinel objects on failure, so there is no duplicated logic and no try/catch. Behaviour is unchanged: the scanner reproduces the exact grammar accepted by the previous regexes (including their quirks, which are documented in the code), verified by differential fuzzing against the previous implementation over ~19M inputs with zero mismatches. Error messages, result shape, null-prototype parameters and the frozen defaultContentType identity are all preserved. Throughput on Node 24 (ops/sec, before -> after): application/json 13.9M -> 18.0M (1.3x) application/json; charset=utf-8 2.76M -> 4.87M (1.8x) application/json; charset="utf-8" 2.46M -> 4.89M (2.0x) multipart/form-data; boundary=----WebKit... 2.52M -> 3.43M (1.4x) text/html; charset=utf-8; foo=bar; baz="qu\"x" 0.99M -> 2.13M (2.2x) Also adds tests for the previously untested error paths (empty key, missing "=", empty token value, bad/unterminated quoted-pair, Unicode whitespace around the media type) and fixes the README benchmark command. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BHGGooWAqjNjuTqBMiytQU
4 tasks
|
Just a heads up that the "including their quirks" are bugs. E.g. media type should allow backticks, qdtext should not be VT but HTAB, and whitespace include HTAB. Dropping Finally last parameter wins is a choice but does differ from the official node and |
4 tasks
Member
Author
|
@blakeembrey thanks, all addressed in #82 (stacked on this one, semver-major). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rewrites the parser as a hand-rolled
charCodeAtscanner driven by a single 64KBUint8Arraycharacter-class table, replacing themediaTypeRE/paramREregular expressions.parseandsafeParsenow share one internalparseHeaderthat returns sentinel objects on failure — no duplicated logic, notry/catch.Motivation:
content-type@2.x/3.x(jshttp rewrite, 2026) had become 25–100% faster than this package on inputs with parameters. Its speed comes from dropping validation entirely (it acceptsapplication/json foo, empty keys,\r\nin values, etc.). This PR closes the gap without relaxing validation.Behaviour is unchanged
The scanner reproduces the exact grammar the previous regexes accepted, including their quirks (documented in the code):
`while parameter tokens include it0x0b) rather than HTAB;String.prototype.trim()'s full Unicode setError messages, result shape, null-prototype
parameters, the frozendefaultContentTypeidentity and all exports are preserved.Verification: differential fuzzing against the previous implementation — two generators (random structural chars; well-formed headers with mutations), five seeds, ~19M inputs — 0 mismatches in result values, thrown messages, prototype, frozen-ness or object identity. Plus
npm test(100% coverage, tstyche) and lint.Benchmark (Node 24.18, ops/sec)
application/jsonapplication/json; charset=utf-8application/json; charset="utf-8"multipart/form-data; boundary=----WebKit…text/html; charset=utf-8; foo=bar; baz="qu\"x"Implementation note: a first draft was slower than the regex on long inputs because
charCodeAt(len)→NaNwas used as a typed-array index andupper |= undefinedpushed V8's optimized loop onto generic paths. Every loop is now bounded byindex < lenand the table covers the full0x10000range so lookups never go out of bounds.Also
=, empty token value, bad/unterminated quoted-pair, Unicode whitespace around the type).benchmarks/index.js; fixed the command and refreshed the numbers.Checklist
npm run testandnpm run benchmark🤖 Generated with Claude Code
https://claude.ai/code/session_01BHGGooWAqjNjuTqBMiytQU