#345: narrow HEAD+Range shim so DuckDB-WASM uses range requests (74 MB -> 3 MB cold load)

[rdhyee]

🤖 rbotyee update, 2026-08-27 (Claude, operated by @rdhyee) — Raymond's intent: deploy this, with the numbers stated honestly. What changed today: (1) upstream/main merged in; (2) the deploy path hardened after two Codex rounds — deploy-canary.sh aborts on failed deploy, asserts Content-Length, has timeouts; new deploy-prod.sh (read-only --preflight that records the active version + exact rollback command, canary ok confirmation gate, fail-closed version parsing, indeterminate-state handling on a failed deploy, full post-deploy verification, --rollback <id>); (3) the contract test is now in CI (.github/workflows/data-origin-contract.yml: push/PR/daily); (4) the headline number is corrected.

Corrected claim. "74 MB → 3.8 MB" is bytes until the facets are usable — real, and it is what the 3G 423 s → 78 s comes from. It is not the full cold boot: with range reads working, DuckDB-WASM goes on to read the 63 MB samples_map_lite_v3 in full, twice, in the background (~120 MB), so the 60-second total is ~131 MB vs ~144 MB today. Measured three ways on 2026-08-27 (see #351, which tracks the Explorer-side double read; fixing it makes this a ~70 MB saving). The shim is still the right change: strictly better time-to-usable, neutral on bytes, narrowly scoped, one-command rollback.

Current production version (rollback target): b61d3bbc-2843-4b97-898b-113bf2a57c41 (2026-04-17; Worker source unchanged on main since, so production receives exactly this shim).

Fixes the 74 MB cold load in #345 — but read the framing, because this is a deliberate standards divergence, not a bug fix.

What changes

DuckDB-WASM 1.24.0 probes for range support with exactly HEAD + Range: bytes=0- and requires a 206. We answer 200, so it logs "falling back to full HTTP read" and downloads whole files — 62.9 MB of samples_map_lite_v3.parquet when it needs ~1.5 MB.

This makes only that exact probe return 206. Every other ranged HEAD stays 200.

Why it's framed as a shim

RFC 9110 §14.2: Range is defined only for GET; servers MUST ignore it on other methods. Our 200 was correct. DuckDB's probe is the nonconforming party. So this is a compatibility measure with a named removal path — delete it when the Explorer stops using Quarto's pinned duckdb-wasm and does its own init on a conformant version.

Codex talked me out of calling this a fix, and out of the wider version I first wrote (which would have answered 206 to any ranged HEAD).

Evidence

Controlled A/B — a transparent proxy changing only this one behaviour:

	Control	Treatment
Cold bytes	74,202,598	3,341,812
Full-read fallbacks	8	0
Facet panel on 3G	440.6 s	94.2 s
Warm bytes (relaunch)	0	1,467,731
Warm time-to-facets	4.2 s	4.2 s

So: ~71 MB saved on first visit, ~1.47 MB cost per repeat visit, no warm time regression. The warm number is included because it could have reversed the recommendation — and the original issue omitted it.

Test coverage

test/range_contract.sh — 23 assertions against wrangler dev --local with a seeded local R2:

• Unfixed Worker: 3 failures, all HEAD+Range
• With this change: 23/23

Four assertions exist specifically to prove the shim doesn't widen (bytes=0-99, bytes=100-199, bytes=-100 must all stay 200). Caching, CORS, ETag-stability and ranged GET are all asserted unchanged — this Worker exists for caching, so that contract is guarded.

⚠️ How to test this WITHOUT touching production

There is no staging data host. This Worker serves data.isamples.org on a route, so a plain wrangler deploy goes straight to production for every consumer.

Safe path — deploy to a workers.dev URL instead, then point the staging Explorer at it:

cd workers/data-isamples-org

# 1. Deploy to a NON-production name, with NO route.
#    Comment out the `routes = [...]` block in wrangler.toml first,
#    or use a separate wrangler config with a different `name`.
npx wrangler deploy --name isamples-data-345-canary

# 2. Point the fork's staging Explorer at the canary (no rebuild needed —
#    the Explorer supports a data_base override):
#    https://rdhyee.github.io/isamplesorg.github.io/explorer.html?data_base=https://isamples-data-345-canary.<subdomain>.workers.dev

# 3. Verify the handshake actually changed:
curl -sI -H 'Range: bytes=0-' https://isamples-data-345-canary.<subdomain>.workers.dev/isamples_202608_h3_summary_res4.parquet
#    expect: HTTP/2 206 + content-range

# 4. Measure end-to-end (harness from #346):
python tests/playwright/bandwidth_matrix.py https://rdhyee.github.io/isamplesorg.github.io \
    --profiles unthrottled,3g-fast --budget 600 \
    --query "?data_base=https://isamples-data-345-canary.<subdomain>.workers.dev"

Only promote to the data.isamples.org route once the canary shows 206s and reduced bytes on the real edge.

What is NOT verified

Everything measured here is a local wrangler dev Worker plus a Python proxy over http://localhost — not the real Worker on the real Cloudflare edge. That canary is the gap.

Also relevant: Workers Caching appears disabled today (production returns no CF-Cache-Status, and the Worker demonstrably still sees Range). If it is ever enabled, Cloudflare strips Range, caches a full 200 and slices its own 206s — which would replace this path entirely and require re-testing.

Refs #345, #313

🤖 Generated with Claude Code

https://claude.ai/code/session_01QCCDurpcLzMe7L72y2HDAa

[rdhyee]

🤖 rbotyee (Claude, operated by @rdhyee) — Raymond's intent: get #345 over the finish line, carefully. What I did: three Codex rounds over the deploy path (read-only; no Cloudflare writes), fixed every finding, applied CC+Codex+LGTM. Raymond runs the deploy himself; nothing has been deployed to production by this PR yet.

Codex verdict at df89db3 — LGTM. Rounds: (1) failed-deploy ≠ unchanged → re-query + verify + always print rollback; version-id parsing fail-closed; probe timeouts; contract test wired into CI. (2) --runxfail so the CI gate can actually fail; a duplicated timeout. (3) clean.

Evidence, all 2026-08-27: canary passes every check incl. Content-Length; staging Explorer against the canary: 0 "falling back" warnings, identical counts; production pre-flight via deploy-prod.sh --preflight passes read-only; active production version b61d3bbc-… (Worker source unchanged on main since → production receives exactly the shim); bytes measured three ways (see the corrected claim in the description and #351); background reads shown to terminate (120 s = 60 s).

Sequence from here: ./deploy-prod.sh --preflight → ./deploy-prod.sh (RY) → live Explorer console check + contract tests (CC) → remove the xfail marker (commit to this branch) → merge → ./deploy-canary.sh --teardown.