No such file or directory error when installing types with mypy

[jakecyr]

Bug Report

When running the command poetry run mypy --config-file pyproject.toml in GitLab CI, MyPy starts installing types as expected, but then throws an error:

mypy: can't read file '/root/.cache/pypoetry/virtualenvs/our-package-d4xTn211-py3.10/lib/python3.10/site-packages//google': No such file or directory

Expected Behavior

I would expect MyPy to install the required types and run the static analysis check.

Actual Behavior

Installing collected packages: types-PyYAML, types-pytz, types-PyMySQL, urllib3, types-protobuf, pycparser, types-requests, cffi, cryptography, types-pyOpenSSL, types-redis
  Attempting uninstall: urllib3
    Found existing installation: urllib3 1.26.18
    Uninstalling urllib3-1.26.18:
      Successfully uninstalled urllib3-1.26.18
Successfully installed cffi-1.16.0 cryptography-41.0.7 pycparser-2.21 types-PyMySQL-1.1.0.1 types-PyYAML-6.0.12.12 types-protobuf-4.24.0.20240106 types-pyOpenSSL-23.3.0.20240106 types-pytz-2023.3.1.1 types-redis-4.6.0.20240106 types-requests-2.31.0.20240106 urllib3-2.1.0
[notice] A new release of pip is available: 23.3.1 -> 23.3.2
[notice] To update, run: pip install --upgrade pip
mypy: can't read file '/root/.cache/pypoetry/virtualenvs/our-package-d4xTn211-py3.10/lib/python3.10/site-packages//google': No such file or directory
Installing missing stub packages:
/root/.cache/pypoetry/virtualenvs/our-package-d4xTn211-py3.10/bin/python -m pip install types-PyMySQL types-PyYAML types-protobuf types-pytz types-redis types-requests

Your Environment

This error occurs when running the mypy command in GitLab CI.

• Mypy version used: 1.8.0 (also tried with 1.7.0)
• Mypy command-line flags: --config-file pyproject.toml
• Mypy configuration options from mypy.ini (and other config files):

  [tool.mypy]
  disallow_untyped_defs = true
  allow_any_unimported = true
  no_implicit_optional = true
  check_untyped_defs = true
  warn_return_any = true
  warn_unused_ignores = true
  show_error_codes = true
  warn_unused_configs = true
  disallow_incomplete_defs = true
  ignore_missing_imports = true
  files = ["our_package", "examples"]
  exclude = [".venv", ".cache", "build"]
  install_types = false
  non_interactive = true
  pretty = true

• Python version used: 3.10

[jakecyr]

It seemed that my previous pipeline jobs that were succeeding started failing so I look at the versions of the typing packages being installed and noticed these were updated since (working versions are below):

types-protobuf = "4.24.0.4"
types-pyopenssl = "23.3.0.0"
types-redis = "4.6.0.11"

Locking the versions to the versions listed above fixed the error. I'm not sure what the issue is with the new typing packages, but it seems either mypy is pulling in incompatible versions or the there is a bug in latest version of one of the packages.

[odimko]

@jakecyr thanks for your comment! it helped a lot.

waiting until this is fixed now python/typeshed#11254

[getim]

We ran into this as well and I drilled down into mypy since yesterday:

• It looks like the logic here assumes that there's more than 1 component in the import path being resolved. However, when that's not the case just (e.g. just google), dir_chain is empty. From there runtime_path calculated here contains a trailing slash.
• Later on, this path gets used as the prefix in exists_case. The trailing slash breaks the startswith check because something like foo doesn't start with foo/. That means exists_case ends up returning True for something like site-packages//google while it doesn't exist.

I'm just laying this out here because I'm now unsure which component is wrong exactly. The runtime_path assignment can check if dir_chain is empty? Should exists_case be fixed to correctly handle the trailing slash? Both seem to fix the issue from my tests.

Edit: I guess my comment is more about the fact that mypy could handle this better, even if the linked typeshed issue gets fixed.

[hauntsaninja]

@getim thanks for looking into it! While typeshed will need to be fixed, let's remove trailing slash in exists_case — seems less of a sharp edge and matches os.path.exists better. Interested in opening a PR?

[getim]

@hauntsaninja Sounds good, see here