Commit Graph

245 Commits

Author SHA1 Message Date
Petr Viktorin d00f7b1b9d
gh-125063: marshal: Add version 5, improve documentation (GH-126829)
* Document that slices can be marshalled
* Deduplicate and organize the list of supported types
  in docs
* Organize the type code list in marshal.c, to make
  it more obvious that this is a versioned format
* Back-fill some historical info

Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2024-11-15 13:48:57 +01:00
Mark Shannon faa3272fb8
GH-125837: Split `LOAD_CONST` into three. (GH-125972)
* Add LOAD_CONST_IMMORTAL opcode

* Add LOAD_SMALL_INT opcode

* Remove RETURN_CONST opcode
2024-10-29 11:15:42 +00:00
sobolevn c8a1818fb0
gh-125517: Fix unreachable code warnings in `_testembed.c` (#125518) 2024-10-15 13:12:32 +00:00
RUANG (Roy James) 546dddca43
gh-125234: Make PyInitConfig_Free(NULL) a no-op (#125266) 2024-10-15 09:21:16 +00:00
Alyssa Coghlan 7c4b6a68f2
bpo-34206: Improve docs and test coverage for pre-init functions (#8023)
- move the Py_Main documentation from the very high level API section
  to the initialization and finalization section
- make it clear that it encapsulates a full Py_Initialize/Finalize
  cycle of its own
- point out that exactly which settings will be read and applied
  correctly when Py_Main is called after a separate runtime
  initialization call is version dependent
- be explicit that Py_IsInitialized can be called prior to
  initialization
- actually test that Py_IsInitialized can be called prior to
  initialization
- flush stdout in the embedding tests that run code so it appears
  in the expected order when running with "-vv"
- make "-vv" on the subinterpreter embedding tests less spammy

---------

Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
2024-10-08 08:34:11 +00:00
Pablo Galindo Salgado 02b49c5150
gh-107954: Fix configuration type for the perf profiler (#124636) 2024-09-28 01:50:16 +02:00
luk1337 7331d0f70b
gh-124160: Pass main_tstate to update_global_state_for_extension() (#124164) 2024-09-19 21:35:20 +05:30
sobolevn c8d1dbef5b
gh-123919: Fix null handling in `_freeze_module.c` (#123920) 2024-09-11 19:07:35 +05:30
Victor Stinner 0d6b6e34a2
gh-107954, PEP 741: Add PyInitConfig_AddModule() function (#123668) 2024-09-04 11:41:57 +00:00
Victor Stinner b423ae6b08
gh-107954, PEP 741: Adjust Python initialization config (#123663)
Setting dev_mode to 1 in an isolated configuration now enables also
faulthandler.

Moreover, setting "module_search_paths" option with
PyInitConfig_SetStrList() now sets "module_search_paths_set" to 1.
2024-09-04 10:58:32 +00:00
Victor Stinner ef9d54703f
gh-107954, PEP 741: Add PyInitConfig C API (#123502)
Add Doc/c-api/config.rst documentation.
2024-09-03 12:33:49 +00:00
Sam Gross d061ffea7b
gh-123022: Fix crash with `Py_Initialize` in background thread (#123052)
Check that the current default heap is initialized in
`_mi_os_get_aligned_hint` and `mi_os_claim_huge_pages`.

The mimalloc function `_mi_os_get_aligned_hint` assumes that there is an
initialized default heap. This is true for our main thread, but not for
background threads. The problematic code path is usually called during
initialization (i.e., `Py_Initialize`), but it may also be called if the
program allocates large amounts of memory in total.

The crash only affected the free-threaded build.
2024-08-17 16:04:08 -04:00
Mark Shannon 7a65439b93
GH-122390: Replace `_Py_GetbaseOpcode` with `_Py_GetBaseCodeUnit` (GH-122942) 2024-08-13 14:22:57 +01:00
Eric Snow 490e0ad83a
gh-117482: Fix the Slot Wrapper Inheritance Tests (gh-122248)
The tests were only checking cases where the slot wrapper was present in the initial case.  They were missing when the slot wrapper was added in the additional initializations.  This fixes that.
2024-07-29 10:23:23 -06:00
Mark Shannon 2e14a52cce
GH-122160: Remove BUILD_CONST_KEY_MAP opcode. (GH-122164) 2024-07-25 16:24:29 +01:00
Mark Shannon 9cefcc0ee7
GH-120507: Lower the `BEFORE_WITH` and `BEFORE_ASYNC_WITH` instructions. (#120640)
* Remove BEFORE_WITH and BEFORE_ASYNC_WITH instructions.

* Add LOAD_SPECIAL instruction

* Reimplement `with` and `async with` statements using LOAD_SPECIAL
2024-06-18 12:17:46 +01:00
Irit Katriel 97b69db167
gh-93691: fix too broad source locations of for statement iterators (#120330) 2024-06-12 12:53:19 +01:00
Jelle Zijlstra 98e855fcc1
gh-119180: Add LOAD_COMMON_CONSTANT opcode (#119321)
The PEP 649 implementation will require a way to load NotImplementedError
from the bytecode. @markshannon suggested implementing this by converting
LOAD_ASSERTION_ERROR into a more general mechanism for loading constants.

This PR adds this new opcode. I will work on the rest of the implementation
of the PEP separately.

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2024-05-22 00:46:39 +00:00
Sam Gross 2ba2c142a6
gh-118527: Intern code name and filename on default build (#118576)
Interned and non-interned strings are treated differently by `marshal`,
so be consistent between the default and free-threaded build.
2024-05-06 17:24:14 -04:00
Guido van Rossum 4c7bfdff90
Remove more remnants of deepfreeze (#118159) 2024-04-22 12:17:57 -07:00
Victor Stinner 340a02b590
gh-117987: Restore several functions removed in Python 3.13 alpha 1 (GH-117993)
Restore these functions removed in Python 3.13 alpha 1:

* Py_SetPythonHome()
* Py_SetProgramName()
* PySys_SetArgvEx()
* PySys_SetArgv()
2024-04-18 15:20:38 +02:00
Mark Shannon 981d172f7f
GH-112354: `END_FOR` instruction to only pop one value. (GH-114247)
* Compiler emits END_FOR; POP_TOP instead of END_FOR. To support tier 2 side exits in loops.
2024-01-24 15:10:17 +00:00
Sam Gross fdee7b7b3e
gh-112532: Require mimalloc in `--disable-gil` builds (gh-112883) 2023-12-12 09:04:48 +09:00
Irit Katriel d49aba5a7a
gh-111354: Simplify _PyGen_yf by moving some of its work to the compiler and frame state (#111648) 2023-11-03 10:01:36 +00:00
Irit Katriel 52cc4af6ae
gh-111354: simplify detection of RESUME after YIELD_VALUE at except-depth 1 (#111459) 2023-11-02 10:18:43 +00:00
Donghee Na 0362cbf908
gh-109595: Add -Xcpu_count=<n> cmdline for container users (#109667)
---------

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2023-10-10 19:00:09 +09:00
Victor Stinner bb057b3370
gh-85283: Add PySys_AuditTuple() function (#108965)
sys.audit() now has assertions to check that the event argument is
not NULL and that the format argument does not use the "N" format.

Add tests on PySys_AuditTuple().
2023-10-05 21:59:35 +00:00
Brandt Bucher 22e65eecaa
GH-105848: Replace KW_NAMES + CALL with LOAD_CONST + CALL_KW (GH-109300) 2023-09-13 10:25:45 -07:00
Irit Katriel 8b55adfa8f
gh-109256: allocate opcode IDs for internal opcodes in their own range (#109269) 2023-09-12 10:36:17 +00:00
Mark Shannon 15d4c9fabc
GH-108716: Turn off deep-freezing of code objects. (GH-108722) 2023-09-08 10:34:40 +01:00
Mark Shannon 0858328ca2
GH-108614: Add `RESUME_CHECK` instruction (GH-108630) 2023-09-07 14:39:03 +01:00
Victor Stinner a0773b89df
gh-108753: Enhance pystats (#108754)
Statistics gathering is now off by default. Use the "-X pystats"
command line option or set the new PYTHONSTATS environment variable
to 1 to turn statistics gathering on at Python startup.

Statistics are no longer dumped at exit if statistics gathering was
off or statistics have been cleared.

Changes:

* Add PYTHONSTATS environment variable.
* sys._stats_dump() now returns False if statistics are not dumped
  because they are all equal to zero.
* Add PyConfig._pystats member.
* Add tests on sys functions and on setting PyConfig._pystats to 1.
* Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h
  header files.
* Rename '_py_stats' variable to '_Py_stats'.
* Exclude Include/cpython/pystats.h from the Py_LIMITED_API.
* Move pystats.h include from object.h to Python.h.
* Add _Py_StatsOn() and _Py_StatsOff() functions. Remove
  '_py_stats_struct' variable from the API: make it static in
  specialize.c.
* Document API in Include/pystats.h and Include/cpython/pystats.h.
* Complete pystats documentation in Doc/using/configure.rst.
* Don't write "all zeros" stats: if _stats_off() and _stats_clear()
  or _stats_dump() were called.
* _PyEval_Fini() now always call _Py_PrintSpecializationStats() which
  does nothing if stats are all zeros.

Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2023-09-06 15:54:59 +00:00
Victor Stinner 594b00057e
gh-108765: Python.h no longer includes <unistd.h> (#108783) 2023-09-02 16:50:18 +02:00
Irit Katriel 665a4391e1
gh-105481: generate op IDs from bytecode.c instead of hard coding them in opcode.py (#107971) 2023-08-16 22:25:18 +00:00
Victor Stinner e35c722d22
gh-106659: Fix test_embed.test_forced_io_encoding() on Windows (#108010)
Use config.legacy_windows_stdio=1 to avoid _io._WindowsConsoleIO.
2023-08-16 11:56:56 +00:00
Brandt Bucher a9caf9cf90
GH-105848: Simplify the arrangement of CALL's stack (GH-107788) 2023-08-09 18:19:39 +00:00
Mark Shannon 1d857da7f0
GH-77273: Better bytecodes for f-strings (GH-6132) 2023-06-14 16:15:08 +01:00
Mark Shannon 4bfa01b9d9
GH-104584: Plugin optimizer API (GH-105100) 2023-06-02 11:46:18 +01:00
Victor Stinner 424049cc11
gh-105145: Remove old functions to config Python init (#105154)
Remove the following old functions to configure the Python
initialization, deprecated in Python 3.11:

* PySys_AddWarnOptionUnicode()
* PySys_AddWarnOption()
* PySys_AddXOption()
* PySys_HasWarnOptions()
* PySys_SetArgvEx()
* PySys_SetArgv()
* PySys_SetPath()
* Py_SetPath()
* Py_SetProgramName()
* Py_SetPythonHome()
* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()

Most of these functions are kept in the stable ABI, except:

* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()

Update Doc/extending/embedding.rst and Doc/extending/extending.rst to
use the new PyConfig API.

_testembed.c:

* check_stdio_details() now sets stdio_encoding and stdio_errors
  of PyConfig.
* Add definitions of functions removed from the API but kept in the
  stable ABI.
* test_init_from_config() and test_init_read_set() now use
  PyConfig_SetString() instead of PyConfig_SetBytesString().

Remove _Py_ClearStandardStreamEncoding() internal function.
2023-06-01 09:14:02 +02:00
Eddie Elizondo ea2c001650
gh-84436: Implement Immortal Objects (gh-19474)
This is the implementation of PEP683

Motivation:

The PR introduces the ability to immortalize instances in CPython which bypasses reference counting. Tagging objects as immortal allows up to skip certain operations when we know that the object will be around for the entire execution of the runtime.

Note that this by itself will bring a performance regression to the runtime due to the extra reference count checks. However, this brings the ability of having truly immutable objects that are useful in other contexts such as immutable data sharing between sub-interpreters.
2023-04-22 13:39:37 -06:00
Pablo Galindo Salgado 1ef61cf71a
gh-102856: Initial implementation of PEP 701 (#102855)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
Co-authored-by: Marta Gómez Macías <mgmacias@google.com>
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-04-19 11:18:16 -05:00
Nikita Sobolev 7f3c106503
gh-103326: Remove `Python/importlib.h` (GH-103331)
Co-authored-by: Brett Cannon <brett@python.org>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-04-12 11:27:14 -07:00
Brandt Bucher b4978ff872
GH-88691: Shrink the CALL caches (GH-103230) 2023-04-05 14:15:49 -07:00
Brandt Bucher 121057aa36
GH-89987: Shrink the BINARY_SUBSCR caches (GH-103022) 2023-03-29 15:53:30 -07:00
chgnrdv 85ba8a3e03
gh-102650: Remove duplicate include directives from multiple source files (#102651)
Remove duplicate include directives from multiple source files
2023-03-13 13:25:17 -06:00
penguin_wwy 753fc8a5d6
gh-101632: Add the new RETURN_CONST opcode (#101633) 2023-02-07 22:32:21 +00:00
Brandt Bucher 3c033a2e6f
GH-99554: Pack location tables more effectively (GH-99556) 2022-12-21 16:41:18 -08:00
Kai Zhang 228c92eb5c
gh-99582: freeze `zipimport` into `_bootstrap_python` (#99583)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-12-10 15:05:56 +05:30
Mark Shannon fb713b2183
GH-98522: Add version number to code objects. (GH-98525)
* Add version number to code object for better versioning of functions.

* Improves specialization for closures and list comprehensions.
2022-12-09 12:18:45 +00:00
SQLPATCH 4e5f2db6f2
gh-98872: Fix a possible resource leak in Python 3.11.0 (GH-99047)
Issue: #98872

Automerge-Triggered-By: GH:kumaraditya303
2022-11-24 01:04:19 -08:00