From 8b840b3e13f064c7880250bb3f29f21e822c87a3 Mon Sep 17 00:00:00 2001 From: Marco Walz Date: Tue, 25 Aug 2026 12:13:33 +0200 Subject: [PATCH 1/5] fix(https-outcalls): correct max_response_bytes semantics and the 2MB default cost The 2MB response limit was documented as 2,097,152 bytes and scoped to the response body. Per the interface spec it is 2,000,000 bytes (decimal), it is measured over header names and values plus the body, and it also bounds the transform function's output. Also corrects the cost of omitting max_response_bytes: ~20.85 billion cycles on a 13-node subnet, not ~21.5 billion. The formula already on the cycle-costs page gives 49_140_000 + 10_400 * 2_000_000 = 20_849_140_000. - concepts/https-outcalls.md: byte figure, headers-plus-body scoping, transform bound, default-size cost - guides/backends/https-outcalls.mdx: same, plus a note in the transform section that a transform cannot shrink a response under the cap - references/cycle-costs.md: max_response_bytes defaults to 2 MB, not 2 MiB Closes #351 --- docs/concepts/https-outcalls.md | 4 ++-- docs/guides/backends/https-outcalls.mdx | 6 ++++-- docs/references/cycle-costs.md | 2 +- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/concepts/https-outcalls.md b/docs/concepts/https-outcalls.md index 7a1c61a1..0e1b14f1 100644 --- a/docs/concepts/https-outcalls.md +++ b/docs/concepts/https-outcalls.md @@ -74,14 +74,14 @@ The cost depends on two factors: - **Request size**: the combined byte length of the URL, headers, body, transform function name, and transform context. - **`max_response_bytes`**: the maximum response size you declare. This is what you're charged for, not the actual response size. -If you omit `max_response_bytes`, the system assumes the maximum of 2 MB and charges accordingly: roughly 21.5 billion cycles on a 13-node subnet. Always set this to a reasonable upper bound for your expected response to avoid overpaying. Unused cycles are refunded. +If you omit `max_response_bytes`, the system assumes the maximum of 2 MB and charges accordingly: roughly 20.85 billion cycles on a 13-node subnet. Always set this to a reasonable upper bound for your expected response to avoid overpaying. Unused cycles are refunded. For exact pricing formulas, see the [cycles costs reference](../references/cycle-costs.md). ## Limitations - **HTTPS only.** Plain HTTP is not supported. The target server must have a valid TLS certificate. -- **2 MB response limit.** The maximum response body is 2,097,152 bytes. If the response exceeds `max_response_bytes`, the call fails. +- **2 MB response limit.** The maximum is 2,000,000 bytes (decimal, not 2^21). The limit covers the response's header names and values plus the body, not the body alone, and it also bounds the output of the transform function: a transform cannot bring an oversized response back under the cap. If the response exceeds `max_response_bytes`, the call fails. - **Public endpoints only.** Canisters cannot reach localhost, private IP ranges (10.x.x.x, 192.168.x.x), or other non-routable addresses. - **No streaming or WebSocket.** Outcalls are single request-response pairs. Long-lived connections are not supported. - **~30-second timeout.** If the external server doesn't respond in time, the call fails. diff --git a/docs/guides/backends/https-outcalls.mdx b/docs/guides/backends/https-outcalls.mdx index f7cc2678..d35994b9 100644 --- a/docs/guides/backends/https-outcalls.mdx +++ b/docs/guides/backends/https-outcalls.mdx @@ -19,7 +19,7 @@ For how the consensus mechanism works for outcalls, see [Concepts: HTTPS Outcall By default, every replica node in the subnet independently makes the same HTTP request: called **replicated mode**. All nodes must agree on the response before execution continues. Two constraints apply regardless of mode: - [Cycles](../../concepts/cycles.md) to cover the request cost **must be attached** at call time. In Rust, `ic_cdk::management_canister::http_request` auto-calculates and attaches cycles. In Motoko, cycles must be attached explicitly with `await (with cycles = ...)`. -- The **maximum response body is 2MB** (2,097,152 bytes). Requests exceeding this limit fail. Always set `max_response_bytes` to a tight upper bound: omitting it defaults to 2MB and charges cycles accordingly. +- The **maximum response size is 2MB** (2,000,000 bytes). This covers the response's header names and values plus the body, so size a cap against both: a response can carry 1–2 KB of headers before any body. Requests exceeding this limit fail. Always set `max_response_bytes` to a tight upper bound: omitting it defaults to 2MB and charges cycles accordingly. In replicated mode, a transform function is strongly recommended: without one, responses across nodes will likely differ and consensus will fail. In non-replicated mode (`is_replicated = false`), a transform is unnecessary because only one node makes the request. See [Replicated vs non-replicated mode](#replicated-vs-non-replicated-mode) below. @@ -133,11 +133,13 @@ In replicated mode, a transform function is strongly recommended (without one, r If the response body also contains dynamic fields (timestamps, per-request IDs, the caller's IP), parse and re-serialize the body to extract only the deterministic fields you need. +A transform strips headers for consensus, not to save space: `max_response_bytes` also bounds the transform's own output, so stripping headers there cannot bring an oversized response under the cap. Size the cap for the headers and body as they arrive from the server. + **Debugging "no consensus" errors:** If you see `"No consensus could be reached"`, the transform is not making responses identical. Common culprits: response headers differ, JSON fields arrive in a different order, or the response body contains timestamps. Strip all headers first; if that doesn't resolve it, also normalize or strip the body. ## Cycle costs -HTTPS outcall costs are based on `max_response_bytes`, not the actual response size. If you omit `max_response_bytes`, the system assumes 2MB and charges approximately **21.5 billion cycles**: even for a 1KB response. Always set a tight upper bound. Unused cycles are refunded, but you still pay for the declared maximum. +HTTPS outcall costs are based on `max_response_bytes`, not the actual response size. If you omit `max_response_bytes`, the system assumes 2MB and charges approximately **20.85 billion cycles**: even for a 1KB response. Always set a tight upper bound. Unused cycles are refunded, but you still pay for the declared maximum. In Rust, `ic_cdk::management_canister::http_request` computes and attaches the exact cost automatically using the `ic0.cost_http_request` system API. In Motoko, cycles must be attached explicitly with `await (with cycles = ...)`. diff --git a/docs/references/cycle-costs.md b/docs/references/cycle-costs.md index f30450aa..b89d0fed 100644 --- a/docs/references/cycle-costs.md +++ b/docs/references/cycle-costs.md @@ -124,7 +124,7 @@ base_fee = (3_000_000 + 60_000 * n) * n size_fee = (400 * request_bytes + 800 * max_response_bytes) * n ``` -`request_bytes` is the total serialized request size (URL + headers + body + transform name/context). `max_response_bytes` defaults to 2 MiB if not explicitly set by the canister. +`request_bytes` is the total serialized request size (URL + headers + body + transform name/context). `max_response_bytes` defaults to 2 MB (2,000,000 bytes, decimal) if not explicitly set by the canister, which on a 13-node subnet costs roughly 20.85 billion cycles. | Component | 13-node cycles | ~USD | 34-node cycles | ~USD | |-----------|----------------|------|----------------|------| From 70aec2a6f7f9c7cdf2799e567c737f9da988ed86 Mon Sep 17 00:00:00 2001 From: Marco Walz Date: Tue, 25 Aug 2026 12:22:27 +0200 Subject: [PATCH 2/5] fix(https-outcalls): Motoko has an auto-attaching wrapper too Both places claiming "In Motoko, cycles must be attached explicitly with `await (with cycles = ...)`" are stale. The `ic` package provides `Call.httpRequest`, which computes the exact cost via `ic0.cost_http_request` and attaches it, matching the Rust wrapper. concepts/https-outcalls.md already described both wrappers correctly. Also notes why a hand-picked margin is counterproductive: attached cycles are held for the duration of the call, so a margin caps outcall concurrency. Depends on dfinity/examples#1477, which switches the embedded Motoko examples off the hardcoded `with cycles = 230_949_972_000`. Requires a submodule bump before merge. --- docs/guides/backends/https-outcalls.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/guides/backends/https-outcalls.mdx b/docs/guides/backends/https-outcalls.mdx index d35994b9..bdcd0346 100644 --- a/docs/guides/backends/https-outcalls.mdx +++ b/docs/guides/backends/https-outcalls.mdx @@ -18,7 +18,7 @@ For how the consensus mechanism works for outcalls, see [Concepts: HTTPS Outcall By default, every replica node in the subnet independently makes the same HTTP request: called **replicated mode**. All nodes must agree on the response before execution continues. Two constraints apply regardless of mode: -- [Cycles](../../concepts/cycles.md) to cover the request cost **must be attached** at call time. In Rust, `ic_cdk::management_canister::http_request` auto-calculates and attaches cycles. In Motoko, cycles must be attached explicitly with `await (with cycles = ...)`. +- [Cycles](../../concepts/cycles.md) to cover the request cost **must be attached** at call time. Both languages provide a wrapper that computes the exact amount and attaches it: `ic_cdk::management_canister::http_request` in Rust, and `Call.httpRequest` from the `ic` package in Motoko. Prefer these over a hand-picked figure: attached cycles are held for the duration of the call, so an arbitrary margin caps how many outcalls the canister can have in flight. - The **maximum response size is 2MB** (2,000,000 bytes). This covers the response's header names and values plus the body, so size a cap against both: a response can carry 1–2 KB of headers before any body. Requests exceeding this limit fail. Always set `max_response_bytes` to a tight upper bound: omitting it defaults to 2MB and charges cycles accordingly. In replicated mode, a transform function is strongly recommended: without one, responses across nodes will likely differ and consensus will fail. In non-replicated mode (`is_replicated = false`), a transform is unnecessary because only one node makes the request. See [Replicated vs non-replicated mode](#replicated-vs-non-replicated-mode) below. @@ -141,7 +141,7 @@ A transform strips headers for consensus, not to save space: `max_response_bytes HTTPS outcall costs are based on `max_response_bytes`, not the actual response size. If you omit `max_response_bytes`, the system assumes 2MB and charges approximately **20.85 billion cycles**: even for a 1KB response. Always set a tight upper bound. Unused cycles are refunded, but you still pay for the declared maximum. -In Rust, `ic_cdk::management_canister::http_request` computes and attaches the exact cost automatically using the `ic0.cost_http_request` system API. In Motoko, cycles must be attached explicitly with `await (with cycles = ...)`. +In Rust, `ic_cdk::management_canister::http_request` computes and attaches the exact cost automatically using the `ic0.cost_http_request` system API. In Motoko, `Call.httpRequest` from the `ic` package does the same. Attaching a hand-picked amount instead is counterproductive: the cycles are held for the duration of the call, so a margin caps how many outcalls the canister can have in flight. For reference, on a 13-node subnet: - Base cost: ~49 million cycles From e27a82b377d1cf08c8d0f8e447a399ea1e8f67d9 Mon Sep 17 00:00:00 2001 From: Marco Walz Date: Tue, 25 Aug 2026 15:13:55 +0200 Subject: [PATCH 3/5] =?UTF-8?q?fix(https-outcalls):=20address=20review=20?= =?UTF-8?q?=E2=80=94=20transform=20nuance=20and=20the=20timeout=20claim?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review feedback from @eichhorl, mirroring the corrections on dfinity/icskills#361. The transform claim was too absolute. max_response_bytes is enforced twice: on the raw response as it arrives, and again on the transform's own output. Stripping headers in the transform cannot rescue a response that already exceeded the cap, since that check runs first, but it does keep the transform's own output within the cap. Also corrects the timeout pitfall, which claimed the call traps. There are two timeouts and neither traps: the remote server going silent for 30s rejects with SysFatal, and the subnet failing to produce a response within 60s rejects with SysTransient. --- docs/guides/backends/https-outcalls.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/guides/backends/https-outcalls.mdx b/docs/guides/backends/https-outcalls.mdx index bdcd0346..db45652e 100644 --- a/docs/guides/backends/https-outcalls.mdx +++ b/docs/guides/backends/https-outcalls.mdx @@ -133,7 +133,7 @@ In replicated mode, a transform function is strongly recommended (without one, r If the response body also contains dynamic fields (timestamps, per-request IDs, the caller's IP), parse and re-serialize the body to extract only the deterministic fields you need. -A transform strips headers for consensus, not to save space: `max_response_bytes` also bounds the transform's own output, so stripping headers there cannot bring an oversized response under the cap. Size the cap for the headers and body as they arrive from the server. +`max_response_bytes` is enforced twice: once on the raw response as it arrives from the server, and again on the transform's own output. Stripping headers in the transform therefore cannot rescue a response that already exceeded the cap, because the first check runs before the transform does. It only keeps the transform's own output within the cap. Size `max_response_bytes` for the headers and body as they arrive from the server. **Debugging "no consensus" errors:** If you see `"No consensus could be reached"`, the transform is not making responses identical. Common culprits: response headers differ, JSON fields arrive in a different order, or the response body contains timestamps. Strip all headers first; if that doesn't resolve it, also normalize or strip the body. @@ -154,7 +154,7 @@ See [Cycles costs](../../references/cycle-costs.md#https-outcalls) for the full - **Public endpoints only.** HTTPS outcalls can only reach public internet endpoints. Localhost (`127.0.0.1`), private IP ranges (`10.x.x.x`, `192.168.x.x`), and other non-routable addresses are blocked. - **`Host` header may be required.** Some API endpoints require the `Host` header to be explicitly set. The IC does not automatically set it from the URL: add it to your headers if the server requires it. -- **~30-second timeout.** If the external server does not respond within the timeout, the call traps. Design for failure and handle errors gracefully. +- **Two timeouts.** If the external server does not respond within 30 seconds, or the subnet does not produce a response within 60 seconds, the call is rejected. It does not trap, so handle the error case rather than relying on a trap. ## Testing locally From 7e10ca3059a6ed5a43fa1e05e1f48ae220877326 Mon Sep 17 00:00:00 2001 From: Marco Walz Date: Tue, 25 Aug 2026 16:20:56 +0200 Subject: [PATCH 4/5] fix(https-outcalls): apply the review corrections to the concepts page too The review comment on the guide ("comments of dfinity/icskills#361 also apply here") applies to concepts/https-outcalls.md as well, which this branch also edits. Two bullets there still carried the claims the review corrected: - The 2MB bullet said a transform "cannot bring an oversized response back under the cap" full stop. The cap is enforced twice against the same value: on the raw response in the adapter (rpc_server.rs:402-417, before the transform runs) and on the transform's Candid-encoded output (client.rs:250). Stripping headers cannot rescue a response that failed the first check, but it does keep the transform's own output under the second. - The timeout bullet described a single ~30s timeout. There are two: 30s for the remote server (SysFatal, "Timeout expired") and 60s for the subnet to produce a response (SysTransient, "Canister http request timed out"). Both now match the wording already applied to guides/backends/https-outcalls.mdx. Verified against dfinity/ic@339d220a83. --- docs/concepts/https-outcalls.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/concepts/https-outcalls.md b/docs/concepts/https-outcalls.md index 0e1b14f1..a2e0fbf2 100644 --- a/docs/concepts/https-outcalls.md +++ b/docs/concepts/https-outcalls.md @@ -81,10 +81,10 @@ For exact pricing formulas, see the [cycles costs reference](../references/cycle ## Limitations - **HTTPS only.** Plain HTTP is not supported. The target server must have a valid TLS certificate. -- **2 MB response limit.** The maximum is 2,000,000 bytes (decimal, not 2^21). The limit covers the response's header names and values plus the body, not the body alone, and it also bounds the output of the transform function: a transform cannot bring an oversized response back under the cap. If the response exceeds `max_response_bytes`, the call fails. +- **2 MB response limit.** The maximum is 2,000,000 bytes (decimal, not 2^21). The limit covers the response's header names and values plus the body, not the body alone, and it is enforced twice: on the raw response as it arrives from the server, and again on the output of the transform function. A transform therefore cannot rescue a response that already exceeded the cap, because the first check runs before the transform does. Size `max_response_bytes` for the headers and body as they arrive from the server. - **Public endpoints only.** Canisters cannot reach localhost, private IP ranges (10.x.x.x, 192.168.x.x), or other non-routable addresses. - **No streaming or WebSocket.** Outcalls are single request-response pairs. Long-lived connections are not supported. -- **~30-second timeout.** If the external server doesn't respond in time, the call fails. +- **Two timeouts.** If the external server does not respond within 30 seconds, or the subnet does not produce a response within 60 seconds, the call is rejected. It does not trap, so handle the error case rather than relying on a trap. - **Rate limiting.** All canisters on a subnet share the same IPv6 prefixes. If many canisters on the same subnet call the same server, they share its rate limit quota. Using API keys with per-key quotas mitigates this. - **Shared API keys are visible to all replicas.** An API key stored in canister state is readable by every replica. A compromised replica could use the key to make entirely different, unauthorized requests to the external service: not just replay the canister's intended request. [TEE-enabled subnets](node-infrastructure.md#trusted-execution-environments) mitigate this by running replicas in hardware-enforced enclaves, preventing node operators from reading canister memory. Consider deploying canisters that store sensitive credentials on a TEE-enabled subnet. From 89bf33f71fd2b2c5e37b670dc96f25a46b43ab18 Mon Sep 17 00:00:00 2001 From: Marco Walz Date: Tue, 25 Aug 2026 18:04:15 +0200 Subject: [PATCH 5/5] chore: bump .sources/examples to b4fe175 and fix the outcall snippet paths Bumps the examples submodule from d4ea422 to b4fe175 (master), which contains dfinity/examples#1477: both Motoko HTTPS outcall examples now use Call.httpRequest instead of a hardcoded `with cycles = 230_949_972_000`. The guide's prose on this branch already describes the wrapper, so the embedded snippets and the surrounding text now agree. The pinned commit predated the examples restructure, so all six `snippet=` paths moved and are updated: send_http_{get,post}/src/send_http_{get,post}_backend/main.mo -> send_http_{get,post}/backend/main.mo send_http_{get,post}/src/send_http_{get,post}_backend/src/lib.rs -> send_http_{get,post}/backend/src/lib.rs Region names (transform, get_request, post_request) are unchanged. Verified every file and region resolves at the new commit by replicating remark-snippet's extraction; a missing file or region is a hard build error, so CI covers this too. guides/backends/https-outcalls.mdx is the only page using CodeExample, and examples tracks master so it carries no .sources/VERSIONS entry. --- .sources/examples | 2 +- docs/guides/backends/https-outcalls.mdx | 12 ++++++------ 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/.sources/examples b/.sources/examples index d4ea4220..b4fe175c 160000 --- a/.sources/examples +++ b/.sources/examples @@ -1 +1 @@ -Subproject commit d4ea4220c26b46e676721145d21ca21c8d7dfaa6 +Subproject commit b4fe175c14b3755698d56ece8c4dd353824538bb diff --git a/docs/guides/backends/https-outcalls.mdx b/docs/guides/backends/https-outcalls.mdx index db45652e..3e63ab50 100644 --- a/docs/guides/backends/https-outcalls.mdx +++ b/docs/guides/backends/https-outcalls.mdx @@ -51,7 +51,7 @@ A minimal example that sends a GET request to an echo service. The response body -```motoko snippet="send_http_get/src/send_http_get_backend/main.mo#get_request" +```motoko snippet="send_http_get/backend/main.mo#get_request" ``` @@ -61,7 +61,7 @@ A minimal example that sends a GET request to an echo service. The response body -```rust snippet="send_http_get/src/send_http_get_backend/src/lib.rs#get_request" +```rust snippet="send_http_get/backend/src/lib.rs#get_request" ``` @@ -76,7 +76,7 @@ Because these examples use replicated mode, they include a transform function to -```motoko snippet="send_http_get/src/send_http_get_backend/main.mo#transform" +```motoko snippet="send_http_get/backend/main.mo#transform" ``` @@ -86,7 +86,7 @@ Because these examples use replicated mode, they include a transform function to -```rust snippet="send_http_get/src/send_http_get_backend/src/lib.rs#transform" +```rust snippet="send_http_get/backend/src/lib.rs#transform" ``` @@ -106,7 +106,7 @@ POST requests work the same way, with two additional considerations: -```motoko snippet="send_http_post/src/send_http_post_backend/main.mo#post_request" +```motoko snippet="send_http_post/backend/main.mo#post_request" ``` @@ -116,7 +116,7 @@ POST requests work the same way, with two additional considerations: -```rust snippet="send_http_post/src/send_http_post_backend/src/lib.rs#post_request" +```rust snippet="send_http_post/backend/src/lib.rs#post_request" ```