fix(https-outcalls): correct max_response_bytes semantics and the 2MB default cost

[marc0olo]

Closes #351.

The issue reported two problems with how the HTTPS outcalls pages describe max_response_bytes. Both are confirmed against the interface spec and fixed here, along with several further defects found while fixing them.

What the issue reported

1. Wrong byte figure. 2,097,152 → 2,000,000. The spec: "the default value of 2MB (2,000,000B) is used as the limit." Confirmed in the replica as MAX_CANISTER_HTTP_RESPONSE_BYTES = 2_000_000.

2. The limit is not body-scoped. The spec defines the measured quantity as "the total number of bytes representing the names and values of HTTP headers and the HTTP body." Both pages now say headers plus body.

3. The transform bound (raised in the issue body). max_response_bytes is enforced twice: on the raw response as it arrives, and again on the transform's output. A transform cannot rescue a response that already exceeded the cap, because the first check runs before the transform does; it only keeps the transform's own output within the cap. Stated in the guide's transform section, where a reader would form the "I'll strip headers to fit" plan, and in the concepts Limitations bullet.

Additional defects found

4. The default-size cost was wrong on both pages. Both said omitting max_response_bytes costs ~21.5 billion cycles. The formula already published on references/cycle-costs.md gives:

49_140_000 + 10_400 * 2_000_000 = 20_849_140_000   (~20.85 billion)

Corrected to ~20.85 billion in both places. 21.5B matches neither the decimal nor the binary reading, so it appears independently wrong rather than downstream of the byte-figure error.

5. references/cycle-costs.md said max_response_bytes defaults to "2 MiB". Same decimal-vs-binary error, on the page the other two link to for exact pricing. Corrected, with the resulting cycle figure added.

6. Both pages claimed a single ~30 second timeout, and the guide said the call traps. There are two timeouts and neither traps:

Trigger	Reject	Message
Remote server silent for 30s	SysFatal	Timeout expired
Subnet produces no response within 60s	SysTransient	Canister http request timed out

Telling readers to expect a trap points them at the wrong error handling.

7. The Motoko cycle guidance was stale. Both pages said "In Motoko, cycles must be attached explicitly with await (with cycles = ...)". The ic package provides Call.httpRequest, which computes the exact cost via ic0.cost_http_request and attaches it, matching the Rust wrapper. The pages now also explain why a hand-picked margin is counterproductive: attached cycles are held for the duration of the call, so a margin caps outcall concurrency.

Submodule bump

Item 7 could not be fixed in prose alone, because the embedded Motoko snippets hardcoded with cycles = 230_949_972_000: correcting the text would have left the page contradicting its own code. That was fixed upstream first in dfinity/examples#1477, merged as b4fe175.

.sources/examples is bumped d4ea422 → b4fe175 here, so the snippets now render await Call.httpRequest(request) and code and prose agree.

The old pin predated the examples restructure, so all six snippet= paths moved and are updated:

send_http_{get,post}/src/send_http_{get,post}_backend/main.mo    -> send_http_{get,post}/backend/main.mo
send_http_{get,post}/src/send_http_{get,post}_backend/src/lib.rs -> send_http_{get,post}/backend/src/lib.rs

Region names (transform, get_request, post_request) are unchanged.

Per .agents/submodule-bumping.md: guides/backends/https-outcalls.mdx is the only page using CodeExample, so no other page is affected by the moves, and examples tracks master so it carries no .sources/VERSIONS entry.

Scope

Kept deliberately tight per CONTRIBUTING.md: concepts/ stays explanatory, and the spec's header limits (≤64 headers, ≤8 KiB per name or value, ≤48 KiB total) are not added. The issue marked them optional, and enumerating them duplicates content that belongs in the interface spec and the https-outcalls skill.

Verification

• npm run validate: no errors in the touched files.
• build_and_deploy: passing against the new submodule. This is the meaningful check for the bump, since remark-snippet treats a missing file or region as a hard build error.
• Before pushing the bump, all six file+region pairs were confirmed to resolve at b4fe175 by replicating the plugin's extraction logic.

Related

• fix(https-outcalls): correct max_response_bytes semantics, cycle over-budgeting, and reject messages icskills#361 carries the same corrections in the https-outcalls skill, including the reject-message set these pages do not enumerate.
• feat: flexible canister http outcalls #254 (flexible outcalls) will invalidate the v1 pricing assumptions on these pages when it lands: max_response_bytes is ignored under pricing v2, and ic0.cost_http_request is deprecated. Flagged there with the specific lines, including that references/cycle-costs.md needs both cost models rather than an edit in place. As of dfinity/ic@339d220a83 v2 is still gated off, so the pages are correct today.

[marc0olo]

Converted to draft: this now depends on dfinity/examples#1477 and needs a submodule bump before it can merge.

Why

The guide stated in two places that "In Motoko, cycles must be attached explicitly with await (with cycles = ...)". That is stale (the ic package provides Call.httpRequest, and concepts/https-outcalls.md already described both wrappers correctly), but it matched the embedded example, which hardcoded with cycles = 230_949_972_000. Fixing the prose alone would have left the page contradicting its own snippet, so the example is fixed upstream first.

dfinity/examples#1477 switches both Motoko outcall examples to Call.httpRequest. The prose fix is now pushed here (70aec2a).

Blocking checklist

• fix(send_http_get,send_http_post): attach the computed cycle cost, not a hardcoded figure examples#1477 merges
• Bump .sources/examples from d4ea422 to a master commit containing it
• Update the six snippet= paths in guides/backends/https-outcalls.mdx

The snippet paths are not a no-op

Worth flagging for whoever does the bump. .sources/examples is pinned at d4ea422, which predates the examples restructure, so the bump moves these files regardless of #1477:

Current snippet= path	After bump
send_http_get/src/send_http_get_backend/main.mo	send_http_get/backend/main.mo
send_http_get/src/send_http_get_backend/src/lib.rs	send_http_get/backend/src/lib.rs
send_http_post/src/send_http_post_backend/main.mo	send_http_post/backend/main.mo
send_http_post/src/send_http_post_backend/src/lib.rs	send_http_post/backend/src/lib.rs

Per .agents/submodule-bumping.md, examples bumps require verifying linked files still exist at the same path. These do not. The region names (transform, get_request, post_request) are unchanged and still correctly paired in #1477, so only the file paths need editing.

The factual corrections in the first commit (8b840b3) are independent of all of this and could be split out if the blocking is inconvenient.

[marc0olo]

Thanks @eichhorl. Applied in e27a82b, mirroring the corrections on dfinity/icskills#361.

Transform nuance. The paragraph now says max_response_bytes is enforced twice, on the raw response and again on the transform's own output, so stripping headers cannot rescue a response that already exceeded the cap but does keep the transform's output within it.

One more found while applying it. The Limitations section claimed "If the external server does not respond within the timeout, the call traps." It does not trap, and there are two timeouts, not one: 30s for the remote server (SysFatal) and 60s for the subnet to produce a response (SysTransient). Corrected, since telling readers to expect a trap sends them toward the wrong error handling.

Still draft and still blocked on the submodule bump described above; this only addresses the review.

[marc0olo]

Thanks @eichhorl, you were right that this reaches past the guide. concepts/https-outcalls.md, which this branch also edits, still carried both claims. Fixed in 7e10ca3.

• 2 MB bullet. It said a transform "cannot bring an oversized response back under the cap", full stop. It now states the two checks against the same value: the raw response in the adapter (rpc_server.rs:402-417, which runs before the transform) and the transform's Candid-encoded output (client.rs:250). Stripping headers cannot rescue a response that failed the first check, but it does keep the transform's own output under the second.
• Timeout bullet. It described a single ~30s timeout. Now both, matching the wording already on the guide.

Both pages read the same way now. Verified against dfinity/ic@339d220a83.

Still a draft, still blocked on the submodule bump described above.

[marc0olo]

Submodule bumped in 89bf33f. dfinity/examples#1477 merged as b4fe175, so all three blocking items are now done:

• fix(send_http_get,send_http_post): attach the computed cycle cost, not a hardcoded figure examples#1477 merges
• Bump .sources/examples from d4ea422 to b4fe175
• Update the six snippet= paths

The Motoko snippets on this page now render await Call.httpRequest(request) instead of the hardcoded with cycles = 230_949_972_000, so the embedded code and the prose agree.

Verification. All six file+region pairs were confirmed to resolve at the new commit by replicating remark-snippet's extraction logic (31, 29, 9, 10, 36 and 32 lines respectively). build_and_deploy then passed against the new submodule, which is the authoritative check: the plugin treats a missing file or region as a hard build error.

Scope, per .agents/submodule-bumping.md. guides/backends/https-outcalls.mdx is the only page using CodeExample, so no other page is affected by the path moves. examples tracks master and is explicitly excluded from .sources/VERSIONS, so there is no version entry to update. The bump does span a large range (d4ea422..b4fe175), but nothing else in docs/ derives from this submodule.

Ready to come out of draft whenever you are.