cpython/Include
Eric Snow dcd6f226d6
gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters (gh-103084)
Sharing mutable (or non-immortal) objects between interpreters is generally not safe.  We can work around that but not easily. 
 There are two restrictions that are critical for objects that break interpreter isolation.

The first is that the object's state be guarded by a global lock.  For now the GIL meets this requirement, but a granular global lock is needed once we have a per-interpreter GIL.

The second restriction is that the object (and, for a container, its items) be deallocated/resized only when the interpreter in which it was allocated is the current one.  This is because every interpreter has (or will have, see gh-101660) its own object allocator.  Deallocating an object with a different allocator can cause crashes.

The dict for the cache of module defs is completely internal, which simplifies what we have to do to meet those requirements.  To do so, we do the following:

* add a mechanism for re-using a temporary thread state tied to the main interpreter in an arbitrary thread
   * add _PyRuntime.imports.extensions.main_tstate` 
   * add _PyThreadState_InitDetached() and _PyThreadState_ClearDetached() (pystate.c)
   * add _PyThreadState_BindDetached() and _PyThreadState_UnbindDetached() (pystate.c)
* make sure the cache dict (_PyRuntime.imports.extensions.dict) and its items are all owned by the main interpreter)
* add a placeholder using for a granular global lock

Note that the cache is only used for legacy extension modules and not for multi-phase init modules.

https://github.com/python/cpython/issues/100227
2023-03-29 17:15:43 -06:00
..
cpython GH-89987: Shrink the BINARY_SUBSCR caches (GH-103022) 2023-03-29 15:53:30 -07:00
internal gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters (gh-103084) 2023-03-29 17:15:43 -06:00
Python.h bpo-35134: Remove the Include/code.h header file (GH-32385) 2022-04-07 02:29:52 +02:00
README.rst gh-101101: Unstable C API tier (PEP 689) (GH-101102) 2023-02-28 09:31:01 +01:00
abstract.h gh-98586: Add vector call APIs to the Limited API (GH-98587) 2022-10-27 11:45:42 +02:00
bltinmodule.h
boolobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
bytearrayobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
bytesobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
ceval.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
codecs.h
compile.h
complexobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
datetime.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
descrobject.h gh-47146: Soft-deprecate structmember.h, expose its contents via Python.h (GH-99014) 2022-11-22 08:25:43 +01:00
dictobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
dynamic_annotations.h gh-96959: Update HTTP links which are redirected to HTTPS (GH-96961) 2022-09-24 14:38:53 +03:00
enumobject.h
errcode.h bpo-46521: Fix codeop to use a new partial-input mode of the parser (GH-31010) 2022-02-08 11:54:37 +00:00
exports.h gh-88267: Avoid DLL exporting functions from static builds on Windows(GH-99888) 2022-12-09 11:16:15 +00:00
fileobject.h gh-77782: Deprecate global configuration variable (#93943) 2022-06-17 16:12:00 +02:00
fileutils.h
floatobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
frameobject.h
genericaliasobject.h
import.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
intrcheck.h
iterobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
listobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
longobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
marshal.h
memoryobject.h gh-79315: Add Include/cpython/memoryobject.h header (#99723) 2022-11-23 15:44:42 +01:00
methodobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
modsupport.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
moduleobject.h gh-81057: Move the Extension Modules Cache to _PyRuntimeState (gh-99355) 2022-11-11 14:16:28 -07:00
object.h GH-101291: Rearrange the size bits in PyLongObject (GH-102464) 2023-03-22 14:49:51 +00:00
objimpl.h gh-102013: Add PyUnstable_GC_VisitObjects (#102014) 2023-03-14 01:35:54 +00:00
opcode.h GH-100982: Break up COMPARE_AND_BRANCH (GH-102801) 2023-03-23 15:25:09 -07:00
osdefs.h
osmodule.h
patchlevel.h Post 3.12.0a6 2023-03-08 05:04:38 +01:00
py_curses.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
pybuffer.h gh-98410: move getbufferproc and releasebufferproc to buffer.h (#31158) 2022-10-31 15:01:32 +01:00
pycapsule.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
pydtrace.d
pydtrace.h Fix `pydtrace.d` path comment in `Include/pydtrace.h` (#28539) 2022-12-31 22:40:25 +05:30
pyerrors.h GH-101578: Normalize the current exception (GH-101607) 2023-02-08 09:31:12 +00:00
pyexpat.h
pyframe.h gh-93937, C API: Move PyFrame_GetBack() to Python.h (#93938) 2022-06-19 12:02:33 +02:00
pyhash.h
pylifecycle.h bpo-43931: Export Python version as API data (GH-25577) 2021-12-09 17:52:05 -08:00
pymacconfig.h gh-93202: Always use %zd printf formatter (#93201) 2022-05-25 14:21:36 +02:00
pymacro.h gh-91731: Don't define 'static_assert' in C++11 where is a keyword to avoid UB (GH-93700) 2022-06-13 15:49:08 +01:00
pymath.h bpo-46656: Remove Py_NO_NAN macro (GH-31160) 2022-02-25 01:32:57 +01:00
pymem.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
pyport.h gh-101101: Unstable C API tier (PEP 689) (GH-101102) 2023-02-28 09:31:01 +01:00
pystate.h bpo-45459: C API uses type names rather than structure names (GH-31528) 2022-02-24 17:51:59 +01:00
pystats.h Better stats for `LOAD_ATTR` and `STORE_ATTR` (GH-100295) 2022-12-16 15:41:23 +00:00
pystrcmp.h
pystrtod.h bpo-45995: add "z" format specifer to coerce negative 0 to zero (GH-30049) 2022-04-11 15:34:18 +01:00
pythonrun.h gh-102755: PyErr_DisplayException only in ABI >= 3.12. Tests cover PyErr_Display as well (GH-102849) 2023-03-21 10:36:18 +01:00
pythread.h bpo-42047: Add native thread ID for DragonFlyBSD (#22714) 2022-05-18 15:10:10 +02:00
pytypedefs.h bpo-45459: Fix PyModuleDef_Slot type in the limited C API (GH-31668) 2022-03-03 23:06:55 +01:00
rangeobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
setobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
sliceobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
structmember.h gh-47146: Soft-deprecate structmember.h, expose its contents via Python.h (GH-99014) 2022-11-22 08:25:43 +01:00
structseq.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
sysmodule.h gh-88279: Deprecate PySys_SetArgvEx() (#92363) 2022-05-06 05:24:29 +02:00
traceback.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
tracemalloc.h
tupleobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
typeslots.h bpo-45459: Add Py_buffer to limited API (GH-29991) 2022-02-02 07:03:10 -08:00
unicodeobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
warnings.h
weakrefobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00

README.rst

The Python C API
================

The C API is divided into these sections:

1. ``Include/``: Limited API
2. ``Include/cpython/``: CPython implementation details
3. ``Include/cpython/``, names with the ``PyUnstable_`` prefix: API that can
   change between minor releases
4. ``Include/internal/``, and any name with ``_`` prefix: The internal API

Information on changing the C API is available `in the developer guide`_

.. _in the developer guide: https://devguide.python.org/c-api/