Skip to content

gh-156310: Make the iter() sequence fallback iterator safe in free-threaded build - #156311

Open
akx wants to merge 1 commit into
python:mainfrom
akx:gh-156310-seqiter-ft
Open

gh-156310: Make the iter() sequence fallback iterator safe in free-threaded build#156311
akx wants to merge 1 commit into
python:mainfrom
akx:gh-156310-seqiter-ft

Conversation

@akx

@akx akx commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Sharing a single PySeqIter between threads could double-DECREF the underlying sequence on the exhaustion path and use the sequence after free while another thread was inside PySequence_GetItem. (See #156310.)

This applies the same approach used for listiter, tupleiter and reversed (gh-120608, #120971).

  • it_index is accessed with FT_ATOMIC_LOAD/STORE_SSIZE_RELAXED and -1 becomes the exhaustion sentinel.
  • In the free-threaded build the iterator keeps its strong reference to the sequence until it is deallocated, so no next() call can free the sequence while another thread is using it.
  • __length_hint__, __reduce__ and __setstate__ now key off the index sentinel so that an exhausted iterator behaves the same in both builds, matching what reversed does since gh-120608: Make reversed iterator work with free-threading #120971).
  • An exhausted iterator keeps the sequence alive until the iterator itself is collected, like listiter/tupleiter/reversed.
  • Concurrent iteration may still see duplicate or missing items; this change only removes the memory unsafety.

New test in Lib/test/test_free_threading/test_iteration.py. It uses a small sequence and many rounds so that many threads reach the racy exhaustion path simultaneously, and asserts the sequence's refcount is intact afterwards — the double-DECREF does not always crash, but it reliably shows up as a sagging refcount. On 53d2e14 (main) the test segfaults a free-threaded build.

Note

callable_iterator (backing iter(callable, sentinel)) has the same defect but no index field to reuse as a sentinel. Should also be fixed, but didn't feel like it's in-scope here.

…ree-threaded build

Sharing a single PySeqIter between threads could double-DECREF the
underlying sequence and use it after free.  Apply the same approach as
listiter/tupleiter/reversed (pythongh-120608): use relaxed atomics for
it_index with -1 as the exhaustion sentinel, and in the free-threaded
build keep the reference to the sequence until the iterator is
deallocated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant