Skip to content

Commit c10c7e9

Browse files
authored
gh-151728: Clear the typing caches at interpreter shutdown (GH-155002)
Re-apply GH-154858, which was reverted in GH-154992 because it broke the reference leak buildbots. ``atexit.register(_clear_caches)`` runs on every import of ``typing``, and the registered handler reaches the module dict through ``_clear_caches.__globals__``. Any throwaway copy of ``typing`` therefore stays alive until interpreter shutdown. Two tests create such a copy on each iteration: - ``InternalsTests.test_collect_parameters`` imports a fresh ``typing``. - ``CollectionsAbcTests.test_bytestring`` drops ``typing`` from ``sys.modules`` and re-imports it. Both now unregister the exit handler of the copy they created.
1 parent a4095ee commit c10c7e9

3 files changed

Lines changed: 23 additions & 0 deletions

File tree

Lib/test/test_typing.py

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import annotationlib
2+
import atexit
23
import contextlib
34
import collections
45
import collections.abc
@@ -6550,6 +6551,14 @@ class F:
65506551
class InternalsTests(BaseTestCase):
65516552
def test_collect_parameters(self):
65526553
typing = import_helper.import_fresh_module("typing")
6554+
# Importing typing registers an internal function named _clear_caches
6555+
# with atexit. The throwaway module created here installs its own
6556+
# handler, which holds the module alive and keeps references until
6557+
# interpreter shutdown even after the test finishes. Each repetition
6558+
# of this test under -R would therefore leak another module copy. To
6559+
# avoid this, we unregister the handler once the test is done.
6560+
self.addCleanup(atexit.unregister, typing._clear_caches)
6561+
65536562
with self.assertWarnsRegex(
65546563
DeprecationWarning,
65556564
"The private _collect_parameters function is deprecated"
@@ -7719,6 +7728,10 @@ def test_bytestring(self):
77197728

77207729
with self.assertWarns(DeprecationWarning):
77217730
from typing import ByteString
7731+
# Drop the exit handler of this throwaway copy, see the comment in
7732+
# InternalsTests.test_collect_parameters.
7733+
self.addCleanup(atexit.unregister, sys.modules["typing"]._clear_caches)
7734+
77227735
with self.assertWarns(DeprecationWarning):
77237736
self.assertIsInstance(b'', ByteString)
77247737
with self.assertWarns(DeprecationWarning):

Lib/typing.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
"""
2020

2121
from abc import abstractmethod, ABCMeta
22+
import atexit
2223
import collections
2324
from collections import defaultdict
2425
import collections.abc
@@ -397,6 +398,11 @@ def _clear_caches():
397398
cleanup()
398399

399400

401+
# Release the LRU caches at shutdown, they otherwise redistribute reference
402+
# leaks of one extension to types of unrelated ones. See GH-151728.
403+
atexit.register(_clear_caches)
404+
405+
400406
def _tp_cache(func=None, /, *, typed=False):
401407
"""Internal wrapper caching __getitem__ of generic types.
402408
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Clear the internal :mod:`typing` caches from an exit handler. Previously, an
2+
extension module that leaked a reference to :mod:`typing` would also keep every
3+
subscripted type alive past interpreter shutdown, including types owned by
4+
unrelated extension modules.

0 commit comments

Comments
 (0)