sockets and asyncio on Windows depend on SYSTEMROOT env var

[jariwezerhu]

Bug report

Bug description:

os.environ.clear() # THIS WAS THE CAUSE!!!
import asyncio

This causes the following:

Traceback (most recent call last):
  File "c:\Users\user\Documents\Development\main\main.py", line 3, in <module>
    import asyncio
  File "C:\Program Files\Python311\Lib\asyncio\__init__.py", line 42, in <module>
    from .windows_events import *
  File "C:\Program Files\Python311\Lib\asyncio\windows_events.py", line 8, in <module>
    import _overlapped
OSError: [WinError 10106] The requested service provider could not be loaded or initialized

It took me a whole day to figure out that os.environ.clear() causes this bug to occur. I'm not sure if this is supposed to occur, I just thought I would post this in case other people are struggling with this.

CPython versions tested on:

3.8, 3.9, 3.10, 3.11

Operating systems tested on:

Windows

Linked PRs

• gh-118234: Avoid socket failure with missing SYSTEMROOT env var on Windows #134363
• gh-118234: Document Python subprocess requirement on SystemRoot env, add RuntimeWarning #134435

[gvanrossum]

Presumably there are specific environment variables that are used by the _overlapped module. You can determine which using bisection. :-) Maybe @zooba knows which env vars are needed here? I couldn't find any mentions of "env" in overlapped.c.

[zware]

A bit closer to the source, _overlapped tries to create a socket on import like the following:

>>> import os, socket
>>> os.environ.clear()
>>> socket.socket(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python312\Lib\socket.py", line 233, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [WinError 10106] The requested service provider could not be loaded or initialized

So I think asyncio is just a victim here.

[gvanrossum]

This is not something that's going to be fixed. We may just have to add a note to the docs.

FWIW through trial and error it appears that SYSTEMROOT is the env var that should not be deleted.

[gvanrossum]

If someone feels particularly helpful they could submit a small doc patch.

[eryksun]

FWIW through trial and error it appears that SYSTEMROOT is the env var that should not be deleted.

This is because the path of the provider DLL is stored in the registry as "%SystemRoot%\System32\mswsock.dll", and WinAPI ExpandEnvironmentStringsW() isn't smart enough to call GetSystemDirectoryW() if for some reason the SystemRoot environment variable isn't defined. Since "%SystemRoot%\System32\mswsock.dll" generally doesn't exist, the subsequent LoadLibraryExW() call fails.

However, note that "%" is a valid filename character, and by default the latter LoadLibraryExW() call isn't restricted to exclude searching the working directory and PATH (i.e. Python doesn't call SetDefaultDllDirectories() to restrict the default search since that could cause problems for legacy scripts/applications). I made a dummy DLL that just prints "Fake mswsock loaded!".

>>> del os.environ['SystemRoot']
>>> os.path.exists('%SystemRoot%/System32/mswsock.dll')
True
>>> socket.socket(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP)
Fake mswsock loaded!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python311\Lib\socket.py", line 232, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
OSError: [WinError 10106] The requested service provider could not be loaded or initialized

A more elaborate example could forward the provider API calls to the real "mswsock.dll".

[gvanrossum]

@eryksun Thanks for the explanation. The "exploit" is off-topic though, right?