Skip to content

Fix Windows and Apple transport and add shared data URL decoding - #39

Open
CedricGuillemet wants to merge 4 commits into
mainfrom
cedricguillemet-urllib-transport-fixes
Open

CedricGuillemet wants to merge 4 commits into
mainfrom
cedricguillemet-urllib-transport-fixes

Conversation

@CedricGuillemet

@CedricGuillemet CedricGuillemet commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fix transport problems encountered during Windows NativeDawn testing in BabylonJS/BabylonNative#1781, plus the corresponding Apple POST byte-preservation issue:

  • Guard the optional WinRT response Content-Type before calling ToString(). Responses without that header previously caused an access violation; the header now remains absent.
  • Match Windows POST Content-Type case-insensitively and attach it to content headers. Lowercase/mixed-case names previously went into HttpRequestMessage.Headers() and failed before network I/O with WinRT 0x8000000E (misused header name). Use HttpBufferContent to preserve request bytes, explicit MIME parameters, and an omitted Content-Type without injecting a charset.
  • Preserve Apple POST bodies using length-aware NSData, instead of a NUL-terminated UTF-8 NSString. Embedded NUL and invalid-UTF-8 bytes are transmitted unchanged.
  • Provide platform-independent data: GET decoding through the existing shared scheme resolver, rather than consumer-specific image/fetch wrappers or platform HTTP transports.

Supported behavior

The bounded data URL contract is documented in README.md: standard forgiving-base64 and byte percent decoding, default/explicit MIME types, ASCII token/quoted MIME parameters, case-insensitive scheme/base64 marker, empty and non-ASCII/binary bodies, query bytes, and fragment exclusion. Invalid URL/base64/MIME input and unsupported methods report explicit status-0 errors with no partial successful response. String responses retain raw bytes without charset transcoding.

Custom data resolvers can override the built-in decoder. Shared resolvers now observe cancellation before invocation and retain once-per-open resolution. Existing Windows response semantics are intentionally preserved: successful 2xx body reads normalize to 200; non-success responses retain their status and do not read/publish their body or response headers.

Reuse the pinned base-n helper already used by BabylonNative, validating its input before invoking its permissive decoder, and include its license notice. Avoid decoded-body intermediates and full built-in response-URL copies; ordinary base64 reads directly through a string view into final response storage.

This does not implement browser URL canonicalization, permissive MIME recovery, charset conversion, base64url, streaming, CORS, or all Fetch behavior. Linux POST remains unimplemented and outside this change; common POST wire tests explicitly skip on that backend.

Validation

  • Windows x64 VS18 RelWithDebInfo builds and full CTest pass with both the consumer's cached dependencies and UrlLib's exact standalone dependency pins, including the review follow-up.
  • Current Windows GoogleTest: 60 tests, 54 passing and 6 pre-existing transport-error-detail skips. The original exact-pinned suite passed three shuffled repeats; the portable POST follow-up passed three targeted repeats.
  • Loopback POST wire regressions are shared between Windows and Apple and cover canonical/lowercase/mixed-case headers, MIME parameters, absent MIME, UTF-8 JSON, embedded NUL, invalid UTF-8, and exact binary body bytes. Windows additionally verifies invalid supplied MIME failures, reopening that failed request without retaining invalid MIME, and absent request MIME staying absent. Apple retains the platform's existing header defaults.
  • Windows response regressions cover absent Content-Type, empty/204 responses, existing 404/500/304 semantics, and connection failures. A platform-independent regression verifies that an accepted client sending no bytes cannot indefinitely block loopback-server teardown.
  • Shared data URL regressions cover padded/unpadded/percent-encoded base64, all byte values, malformed input, MIME/defaults, query/fragment/case handling, cancellation, custom overrides, and request reuse across platform/shared transports.
  • The owning JsRuntimeHost session verified end-to-end Fetch JSON POSTs and response consumption against the initial source override on Windows V8 and QuickJS, with record, mixed-case, and Headers-instance initializers.
  • The Apple follow-up revision passed hosted Linux GCC/Clang and Windows CI. Hosted macOS validation is pending runner availability; no local Apple SDK is available on this Windows host. UWP and GPU/image rendering correctness were not validated in this worktree.

Review follow-up

  • Consume request headers before Windows content construction/MIME parsing, retaining Content-Type in an owning optional by moving its value. Failed MIME parsing no longer leaks headers into a reopened request.
  • Apply receive timeouts to all response-mode fixture connections, not only captured requests, and close connections if installing the timeout fails. Regressions cover both review findings.

Related work

Complementary to #35 (HTTP version negotiation) and #38 (Android/Apple response-read failures), with no duplicated production fixes. #38 overlaps the loopback test fixture and README; that fixture should be consolidated when merging both changes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Invalid POST headers can persist across request reuse, and the loopback fixture can hang during teardown.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

Fixes Windows HTTP header and body handling and adds shared, platform-independent data: GET decoding.

Changes:

  • Preserves Windows POST bytes and handles optional Content-Type safely.
  • Adds data URL decoding with validation, cancellation, and custom resolver overrides.
  • Adds regression tests, dependency wiring, and documentation.
File Description
Tests/​UrlRequestErrorReporting.cpp Adds Windows HTTP and transport-reuse regressions.
Tests/​SchemeResolver.cpp Tests cancellation before resolver invocation.
Tests/​Main.cpp Initializes WinRT for tests.
Tests/​DataUrl.cpp Tests decoding, validation, and lifecycle behavior.
Tests/​CMakeLists.txt Includes new tests and custom entry point.
Source/​UrlRequest_Windows_Shared.h Fixes Content-Type handling and preserves POST bytes.
Source/​UrlRequest_Shared.h Routes shared resolution through cancellation-aware tasks.
Source/​UrlRequest_Base.h Integrates built-in data resolution and error handling.
Source/​DataUrl.h Implements data URL decoding and MIME validation.
README.md Documents supported behavior and limitations.
NOTICE.md Adds the base-n license notice.
Include/​UrlLib/​UrlLib.h Documents shared decoding and resolver behavior.
CMakeLists.txt Pins and includes base-n.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Tests/UrlRequestErrorReporting.cpp Outdated
Comment thread Source/UrlRequest_Windows_Shared.h
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@CedricGuillemet CedricGuillemet changed the title Fix Windows HTTP transport and add shared data URL decoding Fix Windows and Apple transport and add shared data URL decoding Oct 6, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Write byte-array chunks directly to OutputStream instead of converting through NewStringUTF. Preserve embedded NULs, UTF-8, and empty bodies without changing dependencies.

Validated unchanged JsRuntimeHost Fetch/XHR regression coverage on Pixel 9a: 331 JavaScript tests pass (3 pending), all 24 native tests pass, and teardown completes without a crash.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 001e7798-8d3b-44be-8a99-5e8c1541297e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants