Skip to content

gh-118234: Document Python subprocess requirement on SystemRoot env, add RuntimeWarning #134435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

gh-118234: Document Python subprocess requirement on SystemRoot env, add RuntimeWarning #134435

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
167aeba
Further document Python subprocess requirement on `SystemRoot` env, a…
jhohm May 21, 2025
b4dfc9c
doc: change will to may (so we don't need to communicate patch versio…
gpshead May 21, 2025
b3285ea
add ! to SystemRoot envvar ref to not link (?)
gpshead May 21, 2025
80afcbc
Merge branch 'main' into gh-118234-2
jhohm May 21, 2025
0cbf8bc
Avoid warning if any capitalization of SystemRoot in env has a nonemp…
jhohm May 21, 2025
9c0ce13
Fix tests to avoid and/or tolerate RuntimeWarning for missing SystemR…
jhohm May 21, 2025
62e8465
Revert unneeded change to `test_launcher.RunPyEnv.fake_venv` context.
jhohm May 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions Doc/library/subprocess.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -648,8 +648,11 @@ functions.
.. note::

If specified, *env* must provide any variables required for the program to
execute. On Windows, in order to run a `side-by-side assembly`_ the
specified *env* **must** include a valid :envvar:`SystemRoot`.
execute. On Windows, in order to run a `side-by-side assembly`_, or a
Python program using the :mod:`socket` module (or another module that
depends on it, such as :mod:`asyncio`), the specified *env* **must**
include a valid :envvar:`!SystemRoot`; omitting it may emit a
:exc:`RuntimeWarning`.

.. _side-by-side assembly: https://en.wikipedia.org/wiki/Side-by-Side_Assembly

Expand Down
4 changes: 4 additions & 0 deletions Lib/subprocess.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1545,6 +1545,10 @@ def _execute_child(self, args, executable, preexec_fn, close_fds,
if cwd is not None:
cwd = os.fsdecode(cwd)

if env is not None and not any(
k.upper() == 'SYSTEMROOT' and v for k, v in env.items()):
warnings.warn("env lacks a valid 'SystemRoot'.", RuntimeWarning)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dislike this RuntimeWarning. It's ok to run programs without SystemRoot env var, or even in an empty environment. It just works fine currently for some programs. Example:

import subprocess, sys
cmd = [sys.executable, '-c', 'print("Hello World")']
subprocess.run(cmd, env={})

This program just works, it displays "Hello World" as expected.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how to synthesize this feedback with #134363 (comment).

How do we prevent the problem, if not warning about it, and not setting it in the subprocess? Should we implicitly override the caller's choices and clone SystemRoot into the subprocess env?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that we have to prevent the problem. We can document it and explain how to avoid it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it works since Windows decided that using PATH to locate runtime dependencies (like an LD_LIBRARY_PATH equivalent) was such a terrible idea that it bypasses it for most OS stuff, but that "fix" upsets people as well.

I'm not particularly strong either way on this fix. It bites people all the time, and it's practically always a pointless thing to do (pass an insufficient environment). At the same time, there are 5-6 other variables that also ought to be included or things will break weirdly (e.g. TEMP), and this change doesn't check for those. But I don't think we have any good way to only warn when things break.

Is there a silent-by-default-enabled-with-X-dev warning that could be used instead? Would that be okay, @vstinner?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we add a warning, it may be better to emit the warning in the socket module, no?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a silent-by-default-enabled-with-X-dev warning that could be used instead? Would that be okay, @vstinner?

A warning only emitted in the Python Developer Mode (-X dev) sounds more acceptable to me, yes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it may be better to emit the warning in the socket module, no?

The user's system could be configured to avoid that error, even without the environment variable, while plenty of other parts could also break if they are reconfigured.

Launching an application with an empty environment is the bad idea, so that's where the warning should be. I've diagnosed that as the root cause of a lot of issues.

Do we have an existing warning? Or do we need to invent a new one? This isn't a deprecation, and that's the only warning I'm aware of.


sys.audit("subprocess.Popen", executable, args, cwd, env)

# Start the process
Expand Down
21 changes: 12 additions & 9 deletions Lib/test/test_subprocess.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -857,13 +857,14 @@ def test_one_environment_variable(self):
'sys.stdout.write("fruit="+os.getenv("fruit"))']
if sys.platform == "win32":
cmd = ["CMD", "/c", "SET", "fruit"]
with subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=newenv) as p:
stdout, stderr = p.communicate()
if p.returncode and support.verbose:
print("STDOUT:", stdout.decode("ascii", "replace"))
print("STDERR:", stderr.decode("ascii", "replace"))
self.assertEqual(p.returncode, 0)
self.assertEqual(stdout.strip(), b"fruit=orange")
with warnings_helper.check_warnings(('env.*SystemRoot', RuntimeWarning), quiet=True):
with subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=newenv) as p:
stdout, stderr = p.communicate()
if p.returncode and support.verbose:
print("STDOUT:", stdout.decode("ascii", "replace"))
print("STDERR:", stderr.decode("ascii", "replace"))
self.assertEqual(p.returncode, 0)
self.assertEqual(stdout.strip(), b"fruit=orange")

def test_invalid_cmd(self):
# null character in the command name
Expand Down Expand Up @@ -1764,7 +1765,8 @@ def test_run_with_an_empty_env(self):
args = [sys.executable, "-c", 'pass']
# Ignore subprocess errors - we only care that the API doesn't
# raise an OSError
subprocess.run(args, env={})
with warnings_helper.check_warnings(('env.*SystemRoot', RuntimeWarning), quiet=True):
subprocess.run(args, env={})

def test_capture_output(self):
cp = self.run_python(("import sys;"
Expand Down Expand Up @@ -3567,7 +3569,8 @@ def test_issue31471(self):
class BadEnv(dict):
keys = None
with self.assertRaises(TypeError):
subprocess.Popen(ZERO_RETURN_CMD, env=BadEnv())
with warnings_helper.check_warnings(('env.*SystemRoot', RuntimeWarning), quiet=True):
subprocess.Popen(ZERO_RETURN_CMD, env=BadEnv())

def test_close_fds(self):
# close file descriptors
Expand Down
4 changes: 4 additions & 0 deletions Misc/NEWS.d/next/Windows/2025-05-21-10-48-38.gh-issue-118234.x1N6xT.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Document that :envvar:`!SystemRoot` is also required for a Python subprocess
that uses :mod:`socket` or :mod:`asyncio` on Windows, and add a
:exc:`RuntimeWarning` if an *env* supplied to :class:`subprocess.Popen`
lacks it. Patch by John Keith Hohm.
Loading