gh-127295: ctypes: Switch field accessors to fixed-width integers (GH-127297)

This should be a pure refactoring, without user-visible behaviour changes.

Before this change, ctypes uses traditional native C types, usually identified 
by [`struct` format characters][struct-chars] when a short (and 
identifier-friendly) name is needed:
- `signed char` (`b`) / `unsigned char` (`B`)
- `short` (`h`) / `unsigned short` (`h`)
- `int` (`i`) / `unsigned int` (`i`)
- `long` (`l`) / `unsigned long` (`l`)
- `long long` (`q`) / `unsigned long long` (`q`)

These map to C99 fixed-width types, which this PR switches to: - 
- `int8_t`/`uint8_t`
- `int16_t`/`uint16_t`
- `int32_t`/`uint32_t`
- `int64_t`/`uint64_t`

The C standard doesn't guarantee that the “traditional” types must map to the 
fixints. But, [`ctypes` currently requires it][swapdefs], so the assumption won't 
break anything.

By “map” I mean that the *size* of the types matches. The *alignment* 
requirements might not. This needs to be kept in mind but is not an issue in 
`ctypes` accessors, which [explicitly handle unaligned memory][memcpy] for the 
integer types.

Note that there are 5 “traditional” C type sizes, but 4 fixed-width ones. Two of 
the former are functionally identical to one another; which ones they are is 
platform-specific (e.g. `int`==`long`==`int32_t`.) This means that one of the 
[current][current-impls-1] [implementations][current-impls-2] is redundant on 
any given platform.


The fixint types are parametrized by the number of bytes/bits, and one bit for 
signedness. This makes it easier to autogenerate code for them or to write 
generic macros (though generic API like 
[`PyLong_AsNativeBytes`][PyLong_AsNativeBytes] is problematic for performance 
reasons -- especially compared to a `memcpy` with compile-time-constant size).

When one has a *different* integer type, determining the corresponding fixint  
means a `sizeof` and signedness check. This is easier and more robust than the 
current implementations (see [`wchar_t`][sizeof-wchar_t] or 
[`_Bool`][sizeof-bool]).


[swapdefs]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L420-L444
[struct-chars]: https://docs.python.org/3/library/struct.html#format-characters
[current-impls-1]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L470-L653
[current-impls-2]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L703-L944
[memcpy]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L613
[PyLong_AsNativeBytes]: https://docs.python.org/3/c-api/long.html#c.PyLong_AsNativeBytes
[sizeof-wchar_t]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L1547-L1555
[sizeof-bool]: https://github.com/python/cpython/blob/v3.13.0/Modules/_ctypes/cfield.c#L1562-L1572


Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
This commit is contained in:
Petr Viktorin 2024-12-20 14:28:18 +01:00 committed by GitHub
parent ba45e5cdd4
commit 78ffba4221
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 498 additions and 649 deletions

View File

@ -1979,7 +1979,7 @@ c_void_p_from_param_impl(PyObject *type, PyTypeObject *cls, PyObject *value)
return NULL;
parg->pffi_type = &ffi_type_pointer;
parg->tag = 'P';
parg->obj = fd->setfunc(&parg->value, value, 0);
parg->obj = fd->setfunc(&parg->value, value, sizeof(void*));
if (parg->obj == NULL) {
Py_DECREF(parg);
return NULL;
@ -2444,7 +2444,7 @@ PyCSimpleType_from_param_impl(PyObject *type, PyTypeObject *cls,
parg->tag = fmt[0];
parg->pffi_type = fd->pffi_type;
parg->obj = fd->setfunc(&parg->value, value, 0);
parg->obj = fd->setfunc(&parg->value, value, info->size);
if (parg->obj)
return (PyObject *)parg;
PyObject *exc = PyErr_GetRaisedException();

View File

@ -264,7 +264,7 @@ static void _CallPythonObject(ctypes_state *st,
be the result. EXCEPT when restype is py_object - Python
itself knows how to manage the refcount of these objects.
*/
PyObject *keep = setfunc(mem, result, 0);
PyObject *keep = setfunc(mem, result, restype->size);
if (keep == NULL) {
/* Could not convert callback result. */

File diff suppressed because it is too large Load Diff

View File

@ -113,8 +113,17 @@ extern PyType_Spec cthunk_spec;
typedef struct tagPyCArgObject PyCArgObject;
typedef struct tagCDataObject CDataObject;
typedef PyObject *(* GETFUNC)(void *, Py_ssize_t size);
typedef PyObject *(* SETFUNC)(void *, PyObject *value, Py_ssize_t size);
// GETFUNC: convert the C value at *ptr* to Python object, return the object
// SETFUNC: write content of the PyObject *value* to the location at *ptr*;
// return a new reference to either *value*, or None for simple types
// (see _CTYPES_DEBUG_KEEP).
// Note that the *size* arg can have different meanings depending on context:
// for string-like arrays it's the size in bytes
// for int-style fields it's either the type size, or bitfiled info
// that can be unpacked using the LOW_BIT & NUM_BITS macros.
typedef PyObject *(* GETFUNC)(void *ptr, Py_ssize_t size);
typedef PyObject *(* SETFUNC)(void *ptr, PyObject *value, Py_ssize_t size);
typedef PyCArgObject *(* PARAMFUNC)(ctypes_state *st, CDataObject *obj);
/* A default buffer in CDataObject, which can be used for small C types. If
@ -239,9 +248,9 @@ extern CThunkObject *_ctypes_alloc_callback(ctypes_state *st,
/* a table entry describing a predefined ctypes type */
struct fielddesc {
char code;
ffi_type *pffi_type; /* always statically allocated */
SETFUNC setfunc;
GETFUNC getfunc;
ffi_type *pffi_type; /* always statically allocated */
SETFUNC setfunc_swapped;
GETFUNC getfunc_swapped;
};