surprising new behavior in inspect/annotationlib get_annotations as of 3.14.0b1, not in 3.14.0a7. intended change?

[zzzeek]

Bug report

Bug description:

3.14.0b1 seems to be interpreting the annotations of a metaclass into a new format that is also interfering with the correct annotations of subclasses of that class. The subclass reports the annotations of the parent class, if it does not itself have any annotations. this only occurs if superclass is from a custom metaclass. if the subclass does have annotations, then the superclass annotations are not reported. this behavior is inconsistent with previous python versions including 3.14.0a7 and is also inconsistent with itself. see script below

from __future__ import annotations


import inspect

# a metaclass with an annotation
class MyMetaClass(type):
    metadata: dict[str, str]


# dynamic class based on metaclass
MyNewClass = MyMetaClass("MyNewClass", (object,), {"metadata": {}})

# this seems new, but maybe intentional.   annotations are shown for the
# dynamic class that come from the metaclass.  OK
# 3.13.3: {}
# 3.14.0a7: {}
# 3.14.0b1: {'metadata': 'dict[str, str]'}
print(inspect.get_annotations(MyNewClass))


# declared class based on metaclass.   Add another anno to it.
class MyOtherNewClass(metaclass=MyMetaClass):
    metadata: dict[str, str] = {}
    someint: int

# both pythons look good and this agrees with older pythons too.
# 3.13.3: {'metadata': 'dict[str, str]', 'someint': 'int'}
# 3.14.0a7: {'metadata': 'dict[str, str]', 'someint': 'int'}
# 3.14.0b1: {'metadata': 'dict[str, str]', 'someint': 'int'}
print(inspect.get_annotations(MyOtherNewClass))

# a control, a normal class without special metaclass.
class YetAnotherClass:
    metadata: dict[str, str] = {}
    someint: int



# here's where it goes wrong.   If we make new classes from these
# bases that have *no* annotations, the *superclass annotations leak into the subclass*

class MySubClass(MyNewClass):
    pass


# 3.13.3: {}
# 3.14.0a7: {}
# 3.14.0b1: {'metadata': 'dict[str, str]', 'someint': 'int'}  # this has to be wrong
print(inspect.get_annotations(MySubClass))

# if we add *any* annotations to MySubClass, the above annos *disappear*
class MySubClass(MyNewClass):
    foobar: float

# these look all correct.  but 3.14.0b1's seems to be inconsistent with itself
# 3.13.3: {'foobar': 'float'}
# 3.14.0a7: {'foobar': 'float'}
# 3.14.0b1: {'foobar': 'float'}  # wait what?  What happened to metadata/someint from above?
print(inspect.get_annotations(MySubClass))



class MyOtherSubClass(MyOtherNewClass):
    pass

# similar behaviors for declared class
# 3.13.3: {}
# 3.14.0a7: {}
# 3.14.0b1: {'metadata': 'dict[str, str]', 'someint': 'int'}
print(inspect.get_annotations(MyOtherSubClass))


# behavior does not seem to occur at all without a metaclass.
# YetAnotherClass has annotations but we do not see them from subclasses
# of that class
class MyYetAnotherSubClass(YetAnotherClass):
    pass

# 3.13.3: {}
# 3.14.0a7: {}
# 3.14.0b1: {}
print(inspect.get_annotations(MyYetAnotherSubClass))

outputs:

$ python test4.py    # python 3.13.3
{}
{'metadata': 'dict[str, str]', 'someint': 'int'}
{}
{'foobar': 'float'}
{}
{}

$ ~/.venv314a7/bin/python test4.py   # python 3.14.0a7
{}
{'metadata': 'dict[str, str]', 'someint': 'int'}
{}
{'foobar': 'float'}
{}
{}

$ ~/.venv314/bin/python test4.py   # python 3.14.0b1
{'metadata': 'dict[str, str]'}
{'metadata': 'dict[str, str]', 'someint': 'int'}
{'metadata': 'dict[str, str]'}
{'foobar': 'float'}
{'metadata': 'dict[str, str]', 'someint': 'int'}
{}

CPython versions tested on:

3.14

Operating systems tested on:

No response

Linked PRs

• gh-133684: Fix get_annotations() where PEP 563 is involved #133795
• [3.14] gh-133684: Fix get_annotations() where PEP 563 is involved (GH-133795) #134656

[zzzeek]

cc @JelleZijlstra

[zzzeek]

further issues that seem related to the same thing

make two files, one with from __future__ import annotations, the other not. the one with future acts as a library for the one that does not have them. The annotations of a subclass declared against a base class that comes from the library are also not showing up if the base has a custom metaclass

lib.py

from __future__ import annotations

from typing import TYPE_CHECKING

class SomeMeta(type):
    if TYPE_CHECKING:
        someint: float

def make_base():
    return SomeMeta("Base", (object, ), {})

then make app.py:

from lib import make_base

Base = make_base()

class MyClass(Base):
    foo: int

import inspect

print(inspect.get_annotations(MyClass))

annotations for "foo" are lost in 3.14.0b1

# python 3.14.0b1
$ ~/.venv314/bin/python app.py 
{}

# python 3.14.0a7
$ ~/.venv314a7/bin/python app.py 
{'foo': <class 'int'>}

[zzzeek]

also suffice to say this is a way worse issue for SQLAlchemy than the conditionals thing. we have no way to work around this

[JelleZijlstra]

I think the workaround is to not use from __future__ import annotations? Going to think about this some more; especially the fist report is unexpected. I want behavior under from __future__ import annotations to remain unchanged from 3.13, but clearly I missed something.

[zzzeek]

That's not a workaround for us, this is the SQLAlchemy library. We use modern annotations so we need from __future__ import annotations in our code since we are supporting python 3.9 through current. our users might not use this. it's not an option for the annotations to just be lost when this directive is used in some of the source files

[JelleZijlstra]

This change seems to fix your cases:

diff --git a/Lib/annotationlib.py b/Lib/annotationlib.py
index 32b85534589..5e58e0c2dcd 100644
--- a/Lib/annotationlib.py
+++ b/Lib/annotationlib.py
@@ -1047,7 +1047,10 @@ def _get_dunder_annotations(obj):
 
     Does not return a fresh dictionary.
     """
-    ann = getattr(obj, "__annotations__", None)
+    if isinstance(obj, type):
+        ann = obj.__dict__.get("__annotations__", None)
+    else:
+        ann = getattr(obj, "__annotations__", None)
     if ann is None:
         return None
 

Unfortunately this issue brings us back to a state where cls.__annotations__ is unsafe, though no more so than before 3.14. (You'll see some interesting behavior in some of your examples if you use the __annotations__ attribute directly instead of get_annotations(); discussion in https://peps.python.org/pep-0749/#annotations-and-metaclasses .)

I'll put up a PR with this change later after I've had some time to write tests. Thanks for the early feedback, and sorry I didn't test behavior with the future import enough.

[zzzeek]

OK so I can just vendor that code right now to try to workaround? great

[zzzeek]

that's ironically pretty much the code we used in python versions prior to 3.10

[zzzeek]

this seems to be working though I am getting better results when I am using the exact version of _get_dunder_annotations that's in 3.14.0a7:

    _BASE_GET_ANNOTATIONS = type.__dict__["__annotations__"].__get__

    def _get_dunder_annotations(obj):
        if isinstance(obj, type):
            try:
                ann = _BASE_GET_ANNOTATIONS(obj)
            except AttributeError:
                # For static types, the descriptor raises AttributeError.
                return {}
        else:
            ann = getattr(obj, "__annotations__", None)
            if ann is None:
                return {}

        if not isinstance(ann, dict):
            raise ValueError(
                f"{obj!r}.__annotations__ is neither a dict nor None"
            )
        return dict(ann)

otherwise I'm getting errors on some other classes, TypeError: '>' not supported between instances of 'builtin_function_or_method' and 'int' which it's not showing exactly what it's hitting, haven't dug into it

[zzzeek]

I found another issue with TypedDict, and it seems similar to these, but is different in that the __annotations__ dictionary is simply missing super "class" elements - this is unworkable because the superclasses as declared are not actually in __mro__. Going to make a separate issue, apologies if it's the same thing

[zzzeek]

that new report is at #133701