Skip to content

gh-135944: Add a "Runtime Components" Section to the Execution Model Docs #135945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions Doc/reference/executionmodel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,82 @@ See also the description of the :keyword:`try` statement in section :ref:`try`
and :keyword:`raise` statement in section :ref:`raise`.


.. _execcomponents:

Runtime Components
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be a good idea to mention that this is CPython specific. Other implementations are allowed to have different runtime models.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is meant to be implementation agnostic

Oh, oops, didn't see this. Some parts are CPython specific here, though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to avoid any CPython-specific notions here. Do you have an example of another runtime model?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I'm not familiar with how any other implementations work, this is coming solely off a bit of speculation on my part. My concern is that a thread is inherently CPython-specific, because some Python implementations exist in areas that don't have access to OS threads, such as Brython for JS, and probably MicroPython/CircuitPython for some microcontrollers.

I do think that this is a great section to have for subinterpreters, I just think it'd be better to keep some parts of it CPython specific. Does that make sense?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a conceptual standpoint, which is the level of this text, threads are not implementation-specific (or platform-specific): the Python runtime always deals with at least the one thread in which it was started. Not all platforms and implementations support multiple threads, but that's a different matter. I'll add a sentence clarifying that point.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think I can get behind that. Thanks!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

==================

Python's execution model does not operate in a vacuum. It runs on a
computer. When a program runs, the conceptual layers of how it runs
on the computer look something like this::

host computer (or VM or container)
process
OS thread (runs machine code)

While a program always starts with exactly one of each of those, it may
grow to include multiple of each. Hosts and processes are isolated and
independent from one another. However, threads are not.

Not all platforms support threads, though most do. For those that do,
each thread does *run* independently, for the small segments of time it
is scheduled to execute its code on the CPU. Otherwise, all threads
in a process share all the process' resources, including memory.
The initial thread is known as the "main" thread.

.. note::

The way they share resources is exactly what can make threads a pain:
two threads running at the same arbitrary time on different CPU cores
can accidentally interfere with each other's use of some shared data.

The same layers apply to each Python program, with some extra layers
specific to Python::

host
process
Python runtime
interpreter
Python thread (runs bytecode)

When a Python program starts, it looks exactly like that, with one
of each. The process has a single global runtime to manage Python's
process-global resources. Each Python thread has all the state it needs
to run Python code (and use any supported C-API) in its OS thread.
Depending on the implementation, this probably includes the current
exception and the Python call stack.

In between the global runtime and the thread(s) lies the interpreter.
It completely encapsulates all of the non-process-global runtime state
that the interpreter's Python threads share. For example, all its
threads share :data:`sys.modules`. Every Python thread belongs to a
single interpreter and runs using that shared state. The initial
interpreter is known as the "main" interpreter, and the initial thread,
where the runtime was initialized, is known as the "main" thread.

.. note::

The interpreter here is not the same as the "bytecode interpreter",
which is what runs in each thread, executing compiled Python code.

Every Python thread is associated with a single OS thread, which is
where it runs. However, multiple Python threads can be associated with
the same OS thread. For example, an OS thread might run code with a
first interpreter and then with a second, each necessarily with its own
Python thread. Still, regardless of how many are *associated* with
an OS thread, only one Python thread can be actively *running* in
an OS thread at a time. Switching between interpreters means
changing the active Python thread.

Once a program is running, new Python threads can be created using the
:mod:`threading` module (on platforms and Python implementations that
support threads). Additional processes can be created using the
:mod:`os`, :mod:`subprocess`, and :mod:`multiprocessing` modules.
You can run coroutines (async) in the main thread using :mod:`asyncio`.
Interpreters can be created using the :mod:`concurrent.interpreters`
module.


.. rubric:: Footnotes

.. [#] This limitation occurs because the code that is executed by these operations
Expand Down
Loading