`t.Optional[t.Union[...]]` leads to incorrect behaviour

[atharva-kelkar]

Bug Report
When using t.Optional with t.Union, mypy checks fail with conditional processing of different types. However, removing t.Optional from the code below leads to a successful check.

To Reproduce

import pandas as pd
import numpy as np
import typing as t

def cast_to_pandas(x: np.ndarray) -> t.Union[pd.Series, pd.DataFrame]:
    if x.ndim == 1:
        return pd.Series(x)
    return pd.DataFrame(x)

def pred(x: t.Optional[t.Union[np.ndarray, pd.Series, pd.DataFrame]] = np.ones(10)) -> None:
    if isinstance(x, np.ndarray):
        x = cast_to_pandas(x)

    print(x.values.ravel)

Expected Behavior
I would have expected this code to pass mypy type-checking because an np.ndarray type is separately cast into pd.Series or pd.DataFrame.

Actual Behavior
Get the following error:

trial.py:14: error: Item "ndarray[Any, Any]" of "ndarray[Any, Any] | Any" has no attribute "values"  [union-attr]
Found 1 error in 1 file

Your Environment
mypy==1.16.0
pandas==2.2.3
numpy==2.2.6

• Mypy version used: 1.16.0
• Mypy command-line flags: -p package
• Mypy configuration options from mypy.ini (and other config files): -
• Python version used: 3.13.3

Edits: Removed redundant comments.

[JelleZijlstra]

I strongly suspect this is because mypy doesn't see types for pandas, and therefore treats pd.Series and pd.DataFrame as Any.

Are you sure the t.Optional part is relevant? Note that Optional is sugar for a union with None.

[atharva-kelkar]

Hey, thanks for the reply.

mypy doesn't see types for pandas, and therefore treats pd.Series and pd.DataFrame as Any.

I am not sure whether this affects the issue I am seeing. The issue is that mypy does not recognize that the np.ndarray type has been eliminated in the conditional statement and still type-checks for it when t.Optional is present.

I am sure the t.Optional part is somehow relevant (I do not know exactly why though) because you can type-check my code without the t.Optional and all mypy checks are passed.

[hauntsaninja]

Try installing pandas-stubs. If mypy does not know what pd.Series and pd.DataFrame are it cannot know that np.ndarray has been eliminated

[atharva-kelkar]

Thanks @hauntsaninja, pandas-stubs did do the trick. However, I still do not get why in the case of no pandas-stubs, it works without the t.Optional but not with it.

[hauntsaninja]

After you assign x = cast(Any, ...), mypy will reset x to the broadest type it has available. If the type is ndarray | Any, you can access .values. If the type is ndarray | Any | None, you cannot access .values