read_url double-percent-encodes '~' in S3 object keys → 404

[moshap-firebolt]

Summary

vortex.io.read_url() fails to read an S3 object whose key contains a tilde (~). The key is percent-encoded twice on the way to the HTTP request — ~ → %7E → %257E — so the GET/HEAD targets a key that doesn't exist and the store returns 404, even though the object is present and readable by other S3 clients. ~ is an unreserved character in RFC 3986 and should not be percent-encoded at all.

Environment

• vortex-data 0.84.0 (Python), Linux x86_64, Python 3.12
• Object store: MinIO (S3-compatible), http:// endpoint via AWS_ALLOW_HTTP=true
• Reproduces against any S3 backend; MinIO used here for a self-contained repro

Minimal repro

import os, pyarrow as pa, vortex as vx, s3fs

os.environ.update({
    "AWS_ENDPOINT_URL": "http://localhost:9000",
    "AWS_ACCESS_KEY_ID": "minioadmin", "AWS_SECRET_ACCESS_KEY": "minioadmin",
    "AWS_ALLOW_HTTP": "true", "AWS_REGION": "us-east-1",
})

# Write a tiny .vortex file locally
vx.io.write(vx.array(pa.table({"x": [1, 2, 3]})), "/tmp/mini.vortex")

# Upload identical bytes to two keys via a known-good S3 client: one plain, one with '~'
fs = s3fs.S3FileSystem(key="minioadmin", secret="minioadmin",
                       client_kwargs={"endpoint_url": "http://localhost:9000"})
fs.put("/tmp/mini.vortex", "bucket/repro-plain/data.vortex")
fs.put("/tmp/mini.vortex", "bucket/repro~tilde~key/data.vortex")

# Read both via vortex
print(vx.io.read_url("s3://bucket/repro-plain/data.vortex").to_arrow_array())      # OK
print(vx.io.read_url("s3://bucket/repro~tilde~key/data.vortex").to_arrow_array())  # 404

Expected

Both reads return x = [1, 2, 3].

Actual

The plain key reads fine; the ~ key raises:

RuntimeError: Object store error: Object at location repro%7Etilde%7Ekey/data.vortex not found:
Error performing HEAD http://localhost:9000/bucket/repro%257Etilde%257Ekey/data.vortex … 404 Not Found

Note the two encodings in one message: the reported location is …%7E… (already wrong — ~ should be literal), and the actual HTTP request is …%257E… (%7E with its % re-encoded to %25) — i.e. the key was percent-encoded twice.

Root cause (hypothesis)

read_url appears to build the object_store path from the URL's already-percent-encoded path string rather than from decoded path segments — so the literal ~ in the source URL is encoded to %7E when the URL is parsed/normalized, and then object_store percent-encodes that string again (% → %25) when constructing the request. The fix is likely to use object_store::path::Path::from_url_path (which decodes) or to hand read_url the raw, decoded key rather than the encoded URL path. ~ is unreserved per RFC 3986 §2.3 and must never be percent-encoded; other unreserved/sub-delims chars in keys may be affected the same way (only ~ verified here).

Impact

Any S3-hosted .vortex object whose key contains ~ is unreadable via read_url. This is not exotic: systems commonly embed ~ as an id/field separator in object keys (e.g. Firebolt tablet ids like <hash>~<n>~all~0), so entire datasets become unreadable through the vortex object-store path while remaining readable by pyarrow/boto3/s3fs. Local-filesystem reads are unaffected.