Rooba
/
cpython
mirror of https://github.com/python/cpython.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
cpython/Include/pymem.h

109 lines
3.5 KiB
C
Raw Normal View History

patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 22:19:30 +00:00
/* Lowest-level memory allocation interface */
#ifndef Py_PYMEM_H
#define Py_PYMEM_H
#include "pyport.h"
#ifdef __cplusplus
extern "C" {
#endif
/* BEWARE:
Each interface exports both functions and macros. Extension modules
should normally use the functions for ensuring binary compatibility
of the user's code across Python versions. Subsequently, if the
Python runtime switches to its own malloc (different from standard
malloc), no recompilation is required for the extensions.
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
The macro versions are free to trade compatibility for speed, although
there's no guarantee they're ever faster. Extensions shouldn't use the
macro versions, as they don't gurantee binary compatibility across
releases.
Do not mix calls to PyMem_xyz with calls to platform
malloc/realloc/calloc/free. */
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 22:19:30 +00:00
/*
* Raw memory interface
* ====================
*/
/* Functions */
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
/* Functions supplying platform-independent semantics for malloc/realloc/
free; useful if you need to be sure you're using the same memory
allocator as Python (this can be especially important on Windows, if
you need to make sure you're using the same MS malloc/free, and out of
the same heap, as the main Python DLL uses).
These functions make sure that allocating 0 bytes returns a distinct
non-NULL pointer (whenever possible -- if we're flat out of memory, NULL
may be returned), even if the platform malloc and realloc don't.
Returned pointers must be checked for NULL explicitly. No action is
performed on failure (no exception is set, no warning is printed, etc).` */
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 22:19:30 +00:00
extern DL_IMPORT(void *) PyMem_Malloc(size_t);
extern DL_IMPORT(void *) PyMem_Realloc(void *, size_t);
extern DL_IMPORT(void) PyMem_Free(void *);
/* Starting from Python 1.6, the wrappers Py_{Malloc,Realloc,Free} are
no longer supported. They used to call PyErr_NoMemory() on failure. */
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
/* Macros. */
Moving pymalloc along. + Redirect PyMem_{Del, DEL} to the object allocator's free() when pymalloc is enabled. Needed so old extensions can continue to mix PyObject_New with PyMem_DEL. + This implies that pgen needs to be able to see the PyObject_XYZ declarations too. pgenheaders.h now includes Python.h. An implication is that I expect obmalloc.o needs to get linked into pgen on non-Windows boxes. + When PYMALLOC_DEBUG is defined, *all* Py memory API functions now funnel through the debug allocator wrapper around pymalloc. This is the default in a debug build. + That caused compile.c to fail: it indirectly mixed PyMem_Malloc with raw platform free() in one place. This is verbotten.
2002-04-22 02:33:27 +00:00
#ifdef PYMALLOC_DEBUG
/* Redirect all memory operations to Python's debugging allocator. */
#define PyMem_MALLOC PyObject_MALLOC
#define PyMem_REALLOC PyObject_REALLOC
#define PyMem_FREE PyObject_FREE
#else /* ! PYMALLOC_DEBUG */
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
#ifdef MALLOC_ZERO_RETURNS_NULL
#define PyMem_MALLOC(n) malloc((n) ? (n) : 1)
#else
#define PyMem_MALLOC malloc
Drop the PyCore_* memory API.
2002-03-18 21:06:21 +00:00
#endif
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
/* Caution: whether MALLOC_ZERO_RETURNS_NULL is #defined has nothing to
do with whether platform realloc(non-NULL, 0) normally frees the memory
or returns NULL. Rather than introduce yet another config variation,
just make a realloc to 0 bytes act as if to 1 instead. */
#define PyMem_REALLOC(p, n) realloc((p), (n) ? (n) : 1)
#define PyMem_FREE free
Moving pymalloc along. + Redirect PyMem_{Del, DEL} to the object allocator's free() when pymalloc is enabled. Needed so old extensions can continue to mix PyObject_New with PyMem_DEL. + This implies that pgen needs to be able to see the PyObject_XYZ declarations too. pgenheaders.h now includes Python.h. An implication is that I expect obmalloc.o needs to get linked into pgen on non-Windows boxes. + When PYMALLOC_DEBUG is defined, *all* Py memory API functions now funnel through the debug allocator wrapper around pymalloc. This is the default in a debug build. + That caused compile.c to fail: it indirectly mixed PyMem_Malloc with raw platform free() in one place. This is verbotten.
2002-04-22 02:33:27 +00:00
#endif /* PYMALLOC_DEBUG */
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 22:19:30 +00:00
/*
* Type-oriented memory interface
* ==============================
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
*
* These are carried along for historical reasons. There's rarely a good
* reason to use them anymore.
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 22:19:30 +00:00
*/
/* Functions */
#define PyMem_New(type, n) \
( (type *) PyMem_Malloc((n) * sizeof(type)) )
#define PyMem_Resize(p, type, n) \
Fix a typo in the PyMem_Resize macro, found by Andrew Kuchling
2000-08-13 11:59:08 +00:00
( (p) = (type *) PyMem_Realloc((p), (n) * sizeof(type)) )
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
/* In order to avoid breaking old code mixing PyObject_{New, NEW} with
PyMem_{Del, DEL} (there was no choice about this in 1.5.2), the latter
have to be redirected to the object allocator. */
#define PyMem_Del PyObject_Free
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 22:19:30 +00:00
/* Macros */
#define PyMem_NEW(type, n) \
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
( (type *) PyMem_MALLOC((n) * sizeof(type)) )
#define PyMem_RESIZE(p, type, n) \
( (p) = (type *) PyMem_REALLOC((p), (n) * sizeof(type)) )
#define PyMem_DEL PyObject_FREE
Give Python a debug-mode pymalloc, much as sketched on Python-Dev. When WITH_PYMALLOC is defined, define PYMALLOC_DEBUG to enable the debug allocator. This can be done independent of build type (release or debug). A debug build automatically defines PYMALLOC_DEBUG when pymalloc is enabled. It's a detected error to define PYMALLOC_DEBUG when pymalloc isn't enabled. Two debugging entry points defined only under PYMALLOC_DEBUG: + _PyMalloc_DebugCheckAddress(const void *p) can be used (e.g., from gdb) to sanity-check a memory block obtained from pymalloc. It sprays info to stderr (see next) and dies via Py_FatalError if the block is detectably damaged. + _PyMalloc_DebugDumpAddress(const void *p) can be used to spray info about a debug memory block to stderr. A tiny start at implementing "API family" checks isn't good for anything yet. _PyMalloc_DebugRealloc() has been optimized to do little when the new size is <= old size. However, if the new size is larger, it really can't call the underlying realloc() routine without either violating its contract, or knowing something non-trivial about how the underlying realloc() works. A memcpy is always done in this case. This was a disaster for (and only) one of the std tests: test_bufio creates single text file lines up to a million characters long. On Windows, fileobject.c's get_line() uses the horridly funky getline_via_fgets(), which keeps growing and growing a string object hoping to find a newline. It grew the string object 1000 bytes each time, so for a million-character string it took approximately forever (I gave up after a few minutes). So, also: fileobject.c, getline_via_fgets(): When a single line is outrageously long, grow the string object at a mildly exponential rate, instead of just 1000 bytes at a time. That's enough so that a debug-build test_bufio finishes in about 5 seconds on my Win98SE box. I'm curious to try this on Win2K, because it has very different memory behavior than Win9X, and test_bufio always took a factor of 10 longer to complete on Win2K. It *could* be that the endless reallocs were simply killing it on Win2K even in the release build.
2002-03-23 10:03:50 +00:00
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 22:19:30 +00:00
#ifdef __cplusplus
}
#endif
#endif /* !Py_PYMEM_H */