Fix the itertools.tee leak and race, the contextvars and generator races, _asyncio's InvalidStateError lookup, and _imp's frozen data - #8518

Follow-up to #8514, from the same fuzzing + static-review catalogs: the two thread-safety defects that #8514 left open, a lead from the catalog that turns out to reproduce, a leak and a race found beside them, and the _imp follow-ups from that PR's review. One commit per defect.

_imp frozen data

_imp.get_frozen_object(name, data) decoded data with marshal::deserialize_code, which reads a bare code body, while the value importlib passes is a whole marshal value — type byte and all. Every explicit data argument therefore came back as ImportError: Frozen object named '…' is invalid. It is read with marshal.loads now; a non-buffer argument still reports TypeError.

find_frozen then implements withdata=True, which returned None before. The data it hands out is what get_frozen_object takes back, so the stored frozen module — which has its own encoding, not marshal — is re-serialized with marshal.dumps into a memoryview. Its arguments are bound by a #[derive(FromArgs)] struct instead of hand-checked FuncArgs.

data, ispkg, origname = _imp.find_frozen("__hello__", withdata=True)
_imp.get_frozen_object("__hello__", data)   # ImportError before

itertools.tee leaked its buffer

The buffer shared by the iterators of one tee() was a PyRc<PyItertoolsTeeData> — a plain refcount the collector cannot walk — so any cycle through a tee iterator was unreachable to gc and never freed:

container = []; node = Node(); container.append(node)
node.held = itertools.tee(container)[0]     # node is never collected

The buffer is now the _tee_dataobject Python type with a traverse, and tee/_tee split the way they do in CPython: tee() copies an iterator that can copy itself and only wraps one that cannot. test_itertools.test_tee loses its expectedFailure.

A second commit closes the two races the split made visible. _tee::next read index and moved it on afterwards, so two callers on one _tee handed out the same value and advanced past a value that was never cached — leaving index past values.len(), which indexes the buffer out of bounds. _tee_dataobject::get_item released its running claim before caching the value it had fetched, so two callers at the same index each took a value from the source and one of them was dropped without reaching anyone. Both claims now cover the read and the update.

_asyncio InvalidStateError (RPYR-0008)

new_invalid_state_error called whatever asyncio.InvalidStateError names and unwrapped the downcast of the result, so asking a pending future for a result panicked once that attribute was rebound to something that is not an exception:

asyncio.InvalidStateError = lambda *args: 42
_asyncio.Future(loop=object()).result()      # unwrap on Err(PyInt { value: 42 })

The catalog carries this as a lead that was never reproduced; the missing piece is that the lookup imports asyncio.exceptions at level 0, which hands back the package, so it is the name on asyncio that is consulted. The type is now looked up the way get_cancelled_error_type looks its own up, and a lookup that does not produce an exception type falls back to RuntimeError like the other arms of that function.

contextvars is shared between threads (RUSTPY-0019)

A Context's map and each ContextVar's cache are reachable from every thread that touches them, but were held in RefCell/Cells, with three unsafe impl Sync covering the hole. Concurrent set/reset/copy_context panicked on already borrowed: BorrowMutError, and ContextVar::get read the cache through AtomicCell::as_ptr while another thread wrote it.

The map and the cache now sit behind PyMutex, entered is a compare_exchange so two threads cannot both enter one Context, hash/used are atomics, and the three unsafe impl Sync are gone. A value displaced by set/reset/delete is dropped after the lock is released, so a __del__ that calls back into the same Context cannot deadlock.

Generator resume race (RUSTPY-0023)

send(), send_none(), throw() and close() read closed and frame.lasti() before running was compare_exchanged, so the frame they went on to resume could be one another thread had already advanced. A resume that concluded from lasti() == 0 that the generator had not started pushes no value onto the value stack, and the POP_TOP after the yield pops one — tried to pop from empty stack, the fatal in ExecutingFrame::run.

The compare_exchange hands back a guard now, taken before those reads and released after maybe_close(), so the generator is also retired while it is still claimed and a waiting thread cannot resume a frame that has finished.

Tests

New snippets: stdlib_threading_generator.py (four threads resuming the same generator through a barrier; every yielded value has to reach exactly one caller), stdlib_threading_itertools_tee.py, stdlib_threading_contextvars.py, a tee case in stdlib_gc.py, and the rebound InvalidStateError in stdlib_asyncio.py.

test_asyncio, test_threading, test_queue, test_selectors, test_weakref, test_gc, test_context, test_generators, test_coroutines, test_asyncgen, test_yield_from, test_contextlib, test_itertools, test_marshal, test_importlib all pass; the CI clippy line is clean.

Still open

RUSTPY-0007 face 7c — the object-core segfault reported in the selectors and asyncio_queues vehicles — is not fixed here. The report has no minimal reproducer for it and marks it as needing a per-crash-dir gdb pass to separate it from the recursion face 7a, whose guards landed in #8514. Driving both vehicle surfaces (selectors with lying/raising/out-of-range fileno() from several threads, asyncio queues plus the _asyncio task registry with non-task objects), eighteen deep-nesting shapes and ten object-core abuse shapes (resurrection in __del__, weakref callbacks, __class__/__bases__ reassignment, suspended frame.clear()) produced no crash on this branch.

🤖 Generated with Claude Code