Keep cache statement names within PostgreSQL's 63-byte limit - #751
Open
aiboupeter wants to merge 1 commit into
Open
Keep cache statement names within PostgreSQL's 63-byte limit#751aiboupeter wants to merge 1 commit into
aiboupeter wants to merge 1 commit into
Conversation
PostgreSQL silently truncates prepared statement names to NAMEDATALEN - 1
(63 bytes). The default names `ecto_<op>_<source>_<n>` and
`ecto_insert_all_<source>` exceed that for long table names, so two sources
that only differ after the 63rd byte (e.g. `..._leave_comp_rest_minutes` and
`..._leave_comp_rest_minutes_event_logs`) map to a single server-side
statement while Postgrex caches them under distinct client names. Once the
second table has been prepared on a connection, the next cached execution of
the first binds against the wrong statement:
ERROR 08P01 (protocol_violation) bind message supplies 8 parameters,
but prepared statement "ecto_insert_..." requires 5
and, since that error does not evict the client cache, the connection keeps
failing until it reconnects.
Cap generated names at 63 bytes: names that already fit are unchanged; longer
ones keep a prefix of the source and add a hash of the full source so they
stay distinct within the limit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
josevalim
reviewed
Aug 22, 2026
| if byte_size(name) <= @max_cache_statement_name_size do | ||
| name | ||
| else | ||
| hash = source |> :erlang.phash2(4_294_967_296) |> Integer.to_string(36) |
Member
There was a problem hiding this comment.
Why the phash2 explicit limit?
josevalim
reviewed
Aug 22, 2026
Comment on lines
+1191
to
+1193
| budget = @max_cache_statement_name_size - byte_size(prefix <> suffix <> hash) - 1 | ||
| kept = source |> truncate_utf8(max(budget, 0)) |> String.trim_trailing("_") | ||
| prefix <> kept <> "_" <> hash <> suffix |
Member
There was a problem hiding this comment.
No need to trim or add _, it is more important to be faster than have pretty names:
Suggested change
| budget = @max_cache_statement_name_size - byte_size(prefix <> suffix <> hash) - 1 | |
| kept = source |> truncate_utf8(max(budget, 0)) |> String.trim_trailing("_") | |
| prefix <> kept <> "_" <> hash <> suffix | |
| budget = @max_cache_statement_name_size - byte_size(prefix <> suffix <> hash) | |
| kept = source |> truncate_utf8(max(budget, 0)) | |
| prefix <> kept <> hash <> suffix |
josevalim
reviewed
Aug 22, 2026
| end | ||
| end | ||
|
|
||
| defp truncate_utf8(string, size) when byte_size(string) <= size, do: string |
Member
There was a problem hiding this comment.
Instead of truncating utf8, you can traverse the string and collect the first n-valid ascii bytes. It should be cheaper to implement.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PostgreSQL silently truncates prepared statement names to
NAMEDATALEN - 1(63 bytes) — the statement hash table is keyed withHASH_STRINGSatNAMEDATALEN, so only the first 63 bytes of a name are compared (pgsql-bugs discussion; the application is expected to avoid it).The default
cache_statementnames (ecto_<op>_<source>_<n>,ecto_insert_all_<source>) exceed that for long table names, so two sources that only differ after the 63rd byte map to one server-side statement while Postgrex caches them under distinct client names. On one connection:insertintobusiness_workplace_attendance_leave_comp_rest_minutes— preparesecto_insert_business_workplace_attendance_leave_comp_rest_minutes_0(server stores it asecto_insert_business_workplace_attendance_leave_comp_rest_minut)insertintobusiness_workplace_attendance_leave_comp_rest_minutes_event_logs— a different client name, so Postgrex closes+parsesecto_insert_…_event_logs_0, which the server also truncates to the same 63 bytes → silently replaces the statement from step 1insertinto the first table again — client cache hit, so it binds straight away:08P01isn't one of the codes that evict the client cache, so every later write to that table on that connection fails the same way until it reconnects.This is what we hit in production; the new integration test (
integration_test/pg/prepare_test.exs) reproduces it onmasterand passes with this change.Change
Ecto.Adapters.SQL.cache_statement_name/3caps the generated name at 63 bytes. Names that already fit are returned byte-for-byte unchanged, so the vast majority of applications see no difference. Longer names keep a (UTF-8 safe) prefix of the source and append:erlang.phash2/2of the full source in base 36, e.g.The cap is applied in the shared SQL layer rather than only in the Postgres adapter: MySQL/TDS don't have the limit, but shorter names are harmless there and it keeps one code path.
Tests
test/ecto/adapters/sql_test.exs— unit tests for the helper (unchanged short names, ≤ 63 bytes, distinct for colliding sources, no split multibyte characters).integration_test/pg/prepare_test.exs— twoTEMPtables sharing a 51-byte prefix, insert A → B → A → B on one sandboxed connection. Fails onmasterwith the08P01above, passes with the fix.mix test: 694 tests, 0 failures.ECTO_ADAPTER=pg mix test: 504 tests, 0 failures (PostgreSQL 18.6 local).🤖 Generated with Claude Code