Run the CPython typing test suite against typing-extensions

[JelleZijlstra]

I propose to add a workflow that:

• Grabs test_typing.py from CPython
• Replaces all typing imports with typing_extensions through some hackery (either sed or some cleverness with importlib)
• Runs the test suite

This would help us catch issues where something got changed in CPython and we didn't reapply it to our variant. For example, this would allow us to catch issues in our TypedDict implementation, which is currently different from CPython's on all versions.

We'd have to run this only on matching versions (e.g., run the CPython tests from the 3.14 branch on Python 3.14) because the CPython tests may rely on new syntax or standard library features.

Feel free to pick this up if you see this and are interested! I may do it eventually when I find time.

[AlexWaygood]

Note that we already run test_typing after typing_extensions has been installed, but that's not as comprehensive as what's being proposed here:

typing_extensions/.github/workflows/ci.yml

Lines 74 to 81 in 44de568

	- name: Test CPython typing test suite
	# Test suite fails on PyPy even without typing_extensions
	if: ${{ !startsWith(matrix.python-version, 'pypy') }}
	run: |
	cd src
	# Run the typing test suite from CPython with typing_extensions installed,
	# because we monkeypatch typing under some circumstances.
	python -c 'import typing_extensions; import test.__main__' test_typing -v

[JelleZijlstra]

Right, that just tests that the monkeypatches we do to typing don't break things.

[donBarbos]

I decided to take this issue, and here are the errors that were received as a result of running it (some of them are related to the fact that there are __module__ checks):
https://github.com/donBarbos/typing_extensions/actions/runs/16691671387/
https://github.com/donBarbos/typing_extensions/actions/runs/16691671387/job/47250262252 (example job on 3.14)

[JelleZijlstra]

Thanks, looks like a bunch of them are the result of typing tests testing some private objects in the module. Possibly fixable in the test by monkeypatching typing_extensions to have a module __getattr__ method that delegates to typing.

[donBarbos]

Thanks, I've added the patch you suggested.
Regarding the __module__ issues, I wanted to fix them like this:

_mod = sys.modules[__name__]
for _name in dir(_mod):
    _obj = getattr(_mod, _name)
    if hasattr(_obj, "__module__") and _obj.__module__ == __name__:
        try:
            _obj.__module__ = "typing"
        except (AttributeError, TypeError):
            pass

But then there are errors with pickle serialization. for example, in test_pickling_then_unpickling_tuple_with_typevartuple_equality

This is what output looks like only with __getattr__ patch now:
https://github.com/donBarbos/typing_extensions/actions/runs/16731514906/job/47360435347

[JanEricNitschke]

I am trying to have a look at the errors that come up with @donBarbos currenty test runs over the CPython test (in 3.14 for now) but running into the problem where the typing_extensions tests explicitely expect the opposite.

test_typing.py:

    def test_cannot_make_mutable_key_readonly(self):
        class Base(TypedDict):
            a: int

        with self.assertRaises(TypeError):
            class Child(Base):
                a: ReadOnly[int]

test_typing_extensions.py:

    def test_make_mutable_key_readonly(self):
        class Base(TypedDict):
            a: int

        self.assertEqual(Base.__readonly_keys__, frozenset())
        self.assertEqual(Base.__mutable_keys__, frozenset({'a'}))

        class Child(Base):
            a: ReadOnly[int]  # type checker error, but allowed at runtime

        self.assertEqual(Child.__readonly_keys__, frozenset({'a'}))
        self.assertEqual(Child.__mutable_keys__, frozenset())

Very simple example showing this:

from typing_extensions import TypedDict as TypedDictExt
from typing_extensions import ReadOnly as ReadOnlyExt
class Base(TypedDictExt):
        a: int

class Child(Base):
    a: ReadOnlyExt[int]  # type checker error, but allowed at runtime

from typing import TypedDict, ReadOnly
class Base(TypedDict):
        a: int

class Child(Base):
    a: ReadOnly[int]  # type checker error, but allowed at runtime

$ uv run --python 3.14 my_test.py
Traceback (most recent call last):
  File "D:\Programming\Projects\typing_extensions\src\my_test.py", line 14, in <module>
    class Child(Base):
    ^^^^^^^^^^^^^^^^^^
        a: ReadOnly[int]  # type checker error, but allowed at runtime
        ^^^^^^^^^^^^^^^^
  File "C:\Users\Jan-Eric\AppData\Roaming\uv\python\cpython-3.14.0a4-windows-x86_64-none\Lib\typing.py", line 3206, in __new__
    raise TypeError(
    ...<2 lines>...
    )
TypeError: Cannot override mutable key 'a' with read-only key

I couldnt find anything about the TypeError in the docs, but it has been present in the code since the introduction: https://github.com/python/cpython/blob/3.13/Lib/typing.py#L3208

If i change this to match CPythons i get 3.14 and 3.13 tests to pass:

https://github.com/JanEricNitschke/typing_extensions/actions/runs/17633620675
main...JanEricNitschke:typing_extensions:run-patched-test_typing

I found this commit: d6c50f5 which linked to this dicussion: https://discuss.python.org/t/pep-705-read-only-typeddict-items/37867/40 but the point was not raised on the PR into CPython which included the TypeError: python/cpython#116350

[JelleZijlstra]

We should making typing-extensions match the typing behavior. (I might have preferred not erroring at runtime but it's too late now.) Finding this kind of discrepancy is exactly why I opened this issue.

[JanEricNitschke]

Nice, ok. Actually currently going through these to try to address them. Probably create individual PRs for the first two issues tomorrow.

[donBarbos]

@JelleZijlstra sorry, but I want to clarify that we did decide to revert typing_extensions behavior related to the runtime error in PEP 705 that I mentioned here?

It seems that, although changing the typing does break backward compatibility, this option is still preferable. The drop of runtime error was intentional, and it looks like the same behavior was meant to be introduced into typing as well.
Maybe it would be worth opening a separate issue to discuss the implications of this backward incompatibility (or perhaps this isn’t really something that needs discussion, I'm not sure :).

[JelleZijlstra]

I don't feel too strongly about which of typing and typing-extensions should change, but it's easier to change typing-extensions. Probably best to open a new issue to lay out the options and discuss.

[JanEricNitschke]

Is that something fro here, the CPython repo or discuss?

[JelleZijlstra]

Let's start with here.

[JanEricNitschke]

From what i understand the TypeError in CPython for this behaviour is pretty unique, as i dont think any other such attribute is runtime enforced.

typing.Final and typing.ReadOnly even specify "There is no runtime checking for this property." in the docs. (Although that obviously refers them only being read, but still).

Additionally the typing docs in general state "The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc.". Lastly, neither the docs nor the PEP say anything about such a type error existing.

So i definitely feel like the optimal thing here would be to not have a runtime error.

On top of that i feel like removing a TypeError is probably a less intrusive change than adding one, even though changes in CPython are obviously more impactful than ones in typing_extensions.