Rooba
/
pyodide
mirror of https://github.com/pyodide/pyodide.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
pyodide/packages/scipy/meta.yaml

145 lines
5.4 KiB
YAML
Raw Normal View History

Initial steps toward Scipy support
2018-07-18 13:32:16 +00:00
package:
name: scipy
Use pypa/build (#2272) This resolves #2189. > build isolation would be a bit difficult to use in our case, as for instance > when building scipy we need the patched numpy on the host and not the numpy > version specified in pyproject.toml (which would be unpatched) This is indeed the case, certain packages cannot be isolated. My strategy is to make a list of packages that shouldn't be isolated and add symlinks from the isolated build environment into the `.artifacts` directory to "unisolate" them. Then we remove the unisolated package requirements from the list of packages to install, in case pesky constraints aren't satisfied. In particular, packages that expect to be used with `pypa/build` often feel free to put very specific constraints on their build dependencies (often asking them to be == to a particular version). Specific version constraints is good for build reproducibility and with build isolation doesn't cost anything. So we just ignore the constraints. Hopefully nothing goes wrong. In particular, any package that does stuff both at build time and at runtime and requires synchronization between the build time and run time environments needs the unisolation. This includes cffi with `_cffi_backend.so`, and of course numpy and scipy. pycparser needs to be unisolated because it is a dependency of cffi. Currently I have also unisolated pythran and cython, though these are build time only tools and do not really need to be unisolated. Cython I unisolated specifically because numcodecs needs it but it isn't in the numcodecs build dependencies. Pythran I unisolated because of a problem with the scipy build which I don't fully understand (some problem with long double feature detection).
2022-03-22 05:05:30 +00:00
version: 1.7.3
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
# See extra explanation in info.md
#
# For future reference: if you see the following errors:
# Declaration error: adjustable dimension on non-argument
# or:
# nonconstant array size
# you are trying to compile code that isn't written to the fortran 77 standard.
# The line number in the error points to the last line of the problematic
# subroutine. Try deleting it.
Initial steps toward Scipy support
2018-07-18 13:32:16 +00:00
source:
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
url: https://files.pythonhosted.org/packages/61/67/1a654b96309c991762ee9bc39c363fc618076b155fe52d295211cf2536c7/scipy-1.7.3.tar.gz
sha256: ab5875facfdef77e0a47d5fd39ea178b58e60e454a4c85aa1e52fcb80db7babf
Use scipy 0.17.1
2018-10-05 13:26:21 +00:00
patches:
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
- patches/add-lapack_extras-to-flapack.patch
- patches/convert-xerrwv-message-arg-from-int-to-str.patch
- patches/disable-blas-detection.patch
- patches/fix-fortran-files-minpack.patch
- patches/gemm_-no-const.patch
Make f2c functions return int This patch makes everything in numpy, scipy and CLAPACK that began life as a fortran subroutine return int, whereas before they were a smattering of void plus lots of int, with a bunch of conflicts where the same function was defined as int in some and void in others. This matters because on upstream emscripten, these packages will not build due to linker conflicts, as the wasm linker can't link a function returning int to a function returning void. Annoyingly, this has to be int return not void, because whereas a normal fortran subroutine doesn't return anything, there's an obscure feature of fortran 77 called alternative returns which allows subroutines to say they want to return by jumping to some other place in the code. f2c handles this with integer return values.
2021-01-09 09:44:41 +00:00
- patches/make-int-return-values.patch
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
- patches/remove-cuncsd-dorcsd-sorcsd-zuncsd.patch
- patches/remove-mvnun-not-fortran-77-compliant.patch
- patches/sasum-returns-double-not-float.patch
- patches/skip-fortran-fails-to-link.patch
- patches/rename-_page_trend_test.patch
FIX scipy Templated_PyUFunc (#2109)
2022-01-16 20:30:42 +00:00
- patches/USE_CPP14-branch-doesn-t-work-for-wasm.patch
Fix scipy linking errors (#2289) With newer versions of emscripten, linker errors surface eariler. This makes it easier to find function pointer cast errors without having to execute the bad code path -- the errors happen when the wasm modules are linked (at load time in the browser) Anyways, this fixes more linker errors. Mostly the problems have to do with LAPACK functions that take string arguments. Most LAPACK functions that take string arguments use them as enums and only care about the first character of the string. Because of the way that f2c works, we need to replace these strings with the ascii code of the first character so we should replace: sTRSV( 'UPPER', 'NOTRANS', 'NONUNIT', J, H, LDH, Y, 1 ) ==> CALL sTRSV( 85, 78, 78, J, H, LDH, Y, 1 ) where 85 and 78 are the ascii codes of U and N. Various character variables are subbed into being integer variables. The two functions `ilaenv` and `xerbla` expect actual C strings as an argument, but it is very annoying to produce C strings so instead I added wrapper functions ilaenvf2c and xerblaf2c to clapack and instead of calling ilaenv and xerbla we call the f2c versions.
2022-03-24 06:17:29 +00:00
- patches/0001-Fix-getbreak-dlamch.patch
- patches/0001-Fix-dstevr-in-special-lapack_defs.h.patch
Fix scipy import failure #2338 (#2348)
2022-04-06 15:12:06 +00:00
- patches/0001-loadDynamicLibrary-flapack.patch
Initial steps toward Scipy support
2018-07-18 13:32:16 +00:00
build:
Update to Python 3.9.5 (#1637) Co-authored-by: Roman Yurchak <rth.yurchak@gmail.com>
2021-06-19 22:49:25 +00:00
cflags: |
-I$(PYODIDE_ROOT)/packages/CLAPACK/build/CLAPACK-3.2.1/INCLUDE
Use pypa/build (#2272) This resolves #2189. > build isolation would be a bit difficult to use in our case, as for instance > when building scipy we need the patched numpy on the host and not the numpy > version specified in pyproject.toml (which would be unpatched) This is indeed the case, certain packages cannot be isolated. My strategy is to make a list of packages that shouldn't be isolated and add symlinks from the isolated build environment into the `.artifacts` directory to "unisolate" them. Then we remove the unisolated package requirements from the list of packages to install, in case pesky constraints aren't satisfied. In particular, packages that expect to be used with `pypa/build` often feel free to put very specific constraints on their build dependencies (often asking them to be == to a particular version). Specific version constraints is good for build reproducibility and with build isolation doesn't cost anything. So we just ignore the constraints. Hopefully nothing goes wrong. In particular, any package that does stuff both at build time and at runtime and requires synchronization between the build time and run time environments needs the unisolation. This includes cffi with `_cffi_backend.so`, and of course numpy and scipy. pycparser needs to be unisolated because it is a dependency of cffi. Currently I have also unisolated pythran and cython, though these are build time only tools and do not really need to be unisolated. Cython I unisolated specifically because numcodecs needs it but it isn't in the numcodecs build dependencies. Pythran I unisolated because of a problem with the scipy build which I don't fully understand (some problem with long double feature detection).
2022-03-22 05:05:30 +00:00
-I$(HOSTSITEPACKAGES)/pythran/
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
-Wno-return-type
No replay (#2256) Our package build process currently has a significant flaw: we first run setup.py, recording all compilation commands, then we rewrite these compilation commands to invoke emcc and replay them, and then we pray that the cross compiled executables ended up in the right place to go into the wheel. This is not a good strategy because the build script is allowed to implement arbitrary logic, and if it moves, renames, etc any of the output files then we lose track of them. This has repeatedly caused difficulty for us. However, we also make no particularly significant use of the two pass approach. We can just do the simpler thing: capture the compiler commands as they occur, modify them as needed, and then run the fixed command. I also added a patch to fix the numpy feature detection for wasm so that we don't have to include _npyconfig.h and config.h, numpy can generate them in the way it would for a native build. I opened a numpy PR that would fix the detection for us upstream: numpy/numpy#21154 This clears the way for us to switch to using pypa/build (as @henryiii has suggested) by removing our dependence on specific setuptools behavior. This is on top of #2238.
2022-03-13 18:39:06 +00:00
-DUNDERSCORE_G77
Update to Python 3.9.5 (#1637) Co-authored-by: Roman Yurchak <rth.yurchak@gmail.com>
2021-06-19 22:49:25 +00:00
ldflags: |
-L$(NUMPY_LIB)
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
# IMPORTANT: Other locations important in scipy build process:
# There are two files built in the "capture" pass that need patching:
# _blas_subroutines.h, and _cython
# Scipy has a bunch of custom logic implemented in
# pyodide-build/pyodide_build/_f2c_fixes.py.
Make f2c functions return int This patch makes everything in numpy, scipy and CLAPACK that began life as a fortran subroutine return int, whereas before they were a smattering of void plus lots of int, with a bunch of conflicts where the same function was defined as int in some and void in others. This matters because on upstream emscripten, these packages will not build due to linker conflicts, as the wasm linker can't link a function returning int to a function returning void. Annoyingly, this has to be int return not void, because whereas a normal fortran subroutine doesn't return anything, there's an obscure feature of fortran 77 called alternative returns which allows subroutines to say they want to return by jumping to some other place in the code. f2c handles this with integer return values.
2021-01-09 09:44:41 +00:00
script: |
Use pypa/build (#2272) This resolves #2189. > build isolation would be a bit difficult to use in our case, as for instance > when building scipy we need the patched numpy on the host and not the numpy > version specified in pyproject.toml (which would be unpatched) This is indeed the case, certain packages cannot be isolated. My strategy is to make a list of packages that shouldn't be isolated and add symlinks from the isolated build environment into the `.artifacts` directory to "unisolate" them. Then we remove the unisolated package requirements from the list of packages to install, in case pesky constraints aren't satisfied. In particular, packages that expect to be used with `pypa/build` often feel free to put very specific constraints on their build dependencies (often asking them to be == to a particular version). Specific version constraints is good for build reproducibility and with build isolation doesn't cost anything. So we just ignore the constraints. Hopefully nothing goes wrong. In particular, any package that does stuff both at build time and at runtime and requires synchronization between the build time and run time environments needs the unisolation. This includes cffi with `_cffi_backend.so`, and of course numpy and scipy. pycparser needs to be unisolated because it is a dependency of cffi. Currently I have also unisolated pythran and cython, though these are build time only tools and do not really need to be unisolated. Cython I unisolated specifically because numcodecs needs it but it isn't in the numcodecs build dependencies. Pythran I unisolated because of a problem with the scipy build which I don't fully understand (some problem with long double feature detection).
2022-03-22 05:05:30 +00:00
pip install -t $HOSTSITEPACKAGES pythran scipy==$PKG_VERSION
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
# We get linker errors because the following 36 functions are missing
# Copying them from a more recent LAPACK seems to work fine.
wget https://github.com/Reference-LAPACK/lapack/archive/refs/tags/v3.10.0.tar.gz
tar xzf v3.10.0.tar.gz
cd lapack-3.10.0/SRC
cat \
cgemqrt.f cgeqrfp.f cgeqrt.f clahqr.f csyconv.f csyconvf.f csyconvf_rook.f ctpmqrt.f ctpqrt.f \
dgemqrt.f dgeqrfp.f dgeqrt.f dlahqr.f dsyconv.f dsyconvf.f dsyconvf_rook.f dtpmqrt.f dtpqrt.f \
sgemqrt.f sgeqrfp.f sgeqrt.f slahqr.f ssyconv.f ssyconvf.f ssyconvf_rook.f stpmqrt.f stpqrt.f \
zgemqrt.f zgeqrfp.f zgeqrt.f zlahqr.f zsyconv.f zsyconvf.f zsyconvf_rook.f ztpmqrt.f ztpqrt.f \
>> ../../scipy/linalg/lapack_extras.f
Fix scipy linking errors (#2289) With newer versions of emscripten, linker errors surface eariler. This makes it easier to find function pointer cast errors without having to execute the bad code path -- the errors happen when the wasm modules are linked (at load time in the browser) Anyways, this fixes more linker errors. Mostly the problems have to do with LAPACK functions that take string arguments. Most LAPACK functions that take string arguments use them as enums and only care about the first character of the string. Because of the way that f2c works, we need to replace these strings with the ascii code of the first character so we should replace: sTRSV( 'UPPER', 'NOTRANS', 'NONUNIT', J, H, LDH, Y, 1 ) ==> CALL sTRSV( 85, 78, 78, J, H, LDH, Y, 1 ) where 85 and 78 are the ascii codes of U and N. Various character variables are subbed into being integer variables. The two functions `ilaenv` and `xerbla` expect actual C strings as an argument, but it is very annoying to produce C strings so instead I added wrapper functions ilaenvf2c and xerblaf2c to clapack and instead of calling ilaenv and xerbla we call the f2c versions.
2022-03-24 06:17:29 +00:00
sed -i 's/CHARACTER/INTEGER/g' ../../scipy/linalg/lapack_extras.f
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
cd ../..
# The additional four functions cuncsd, dorcsd, sorcsd, and zuncsd are also
# missing but they use features of Fortran that aren't Fortran 77 compatible
# so f2c can't handle them. We stub them with C definitions that do nothing.
# These stubs come from f2cpatches/wrap_dummy_g77_abi.patch
# Change many functions that return void into functions that return int
MAINT Simplify scipy patches in make-int-return-values.patch (#1291)
2021-02-27 07:44:52 +00:00
find scipy -name "*.c*" | xargs sed -i 's/extern void F_FUNC/extern int F_FUNC/g'
Fix scipy linking errors (#2289) With newer versions of emscripten, linker errors surface eariler. This makes it easier to find function pointer cast errors without having to execute the bad code path -- the errors happen when the wasm modules are linked (at load time in the browser) Anyways, this fixes more linker errors. Mostly the problems have to do with LAPACK functions that take string arguments. Most LAPACK functions that take string arguments use them as enums and only care about the first character of the string. Because of the way that f2c works, we need to replace these strings with the ascii code of the first character so we should replace: sTRSV( 'UPPER', 'NOTRANS', 'NONUNIT', J, H, LDH, Y, 1 ) ==> CALL sTRSV( 85, 78, 78, J, H, LDH, Y, 1 ) where 85 and 78 are the ascii codes of U and N. Various character variables are subbed into being integer variables. The two functions `ilaenv` and `xerbla` expect actual C strings as an argument, but it is very annoying to produce C strings so instead I added wrapper functions ilaenvf2c and xerblaf2c to clapack and instead of calling ilaenv and xerbla we call the f2c versions.
2022-03-24 06:17:29 +00:00
MAINT Simplify scipy patches in make-int-return-values.patch (#1291)
2021-02-27 07:44:52 +00:00
sed -i 's/void F_FUNC/int F_FUNC/g' scipy/odr/__odrpack.c
Fix scipy linking errors (#2289) With newer versions of emscripten, linker errors surface eariler. This makes it easier to find function pointer cast errors without having to execute the bad code path -- the errors happen when the wasm modules are linked (at load time in the browser) Anyways, this fixes more linker errors. Mostly the problems have to do with LAPACK functions that take string arguments. Most LAPACK functions that take string arguments use them as enums and only care about the first character of the string. Because of the way that f2c works, we need to replace these strings with the ascii code of the first character so we should replace: sTRSV( 'UPPER', 'NOTRANS', 'NONUNIT', J, H, LDH, Y, 1 ) ==> CALL sTRSV( 85, 78, 78, J, H, LDH, Y, 1 ) where 85 and 78 are the ascii codes of U and N. Various character variables are subbed into being integer variables. The two functions `ilaenv` and `xerbla` expect actual C strings as an argument, but it is very annoying to produce C strings so instead I added wrapper functions ilaenvf2c and xerblaf2c to clapack and instead of calling ilaenv and xerbla we call the f2c versions.
2022-03-24 06:17:29 +00:00
sed -i 's/^void/int/g' scipy/odr/odrpack.h
sed -i 's/^void/int/g' scipy/odr/__odrpack.c
sed -i 's/void BLAS_FUNC/int BLAS_FUNC/g' scipy/special/lapack_defs.h
# sed -i 's/void F_FUNC/int F_FUNC/g' scipy/linalg/_lapack_subroutines.h
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
sed -i 's/extern void/extern int/g' scipy/optimize/__minpack.h
sed -i 's/void/int/g' scipy/linalg/cython_blas_signatures.txt
Fix scipy linking errors (#2289) With newer versions of emscripten, linker errors surface eariler. This makes it easier to find function pointer cast errors without having to execute the bad code path -- the errors happen when the wasm modules are linked (at load time in the browser) Anyways, this fixes more linker errors. Mostly the problems have to do with LAPACK functions that take string arguments. Most LAPACK functions that take string arguments use them as enums and only care about the first character of the string. Because of the way that f2c works, we need to replace these strings with the ascii code of the first character so we should replace: sTRSV( 'UPPER', 'NOTRANS', 'NONUNIT', J, H, LDH, Y, 1 ) ==> CALL sTRSV( 85, 78, 78, J, H, LDH, Y, 1 ) where 85 and 78 are the ascii codes of U and N. Various character variables are subbed into being integer variables. The two functions `ilaenv` and `xerbla` expect actual C strings as an argument, but it is very annoying to produce C strings so instead I added wrapper functions ilaenvf2c and xerblaf2c to clapack and instead of calling ilaenv and xerbla we call the f2c versions.
2022-03-24 06:17:29 +00:00
sed -i 's/^void/int/g' scipy/interpolate/src/_fitpackmodule.c
sed -i 's/^void/int/g' scipy/optimize/_trlib/trlib_private.h
sed -i 's/^void/int/g' scipy/optimize/_trlib/trlib/trlib_private.h
sed -i 's/, int)/)/g' scipy/optimize/_trlib/trlib_private.h
sed -i 's/, 1)/)/g' scipy/optimize/_trlib/trlib_private.h
sed -i 's/^void/int/g' scipy/spatial/qhull_misc.h
sed -i 's/, size_t)/)/g' scipy/spatial/qhull_misc.h
sed -i 's/,1)/)/g' scipy/spatial/qhull_misc.h
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
# Missing declaration from cython_lapack_signatures.txt
echo "void ilaenv(int *ispec, char *name, char *opts, int *n1, int *n2, int *n3, int *n4)" \
>> scipy/linalg/cython_lapack_signatures.txt
Fix scipy linking errors (#2289) With newer versions of emscripten, linker errors surface eariler. This makes it easier to find function pointer cast errors without having to execute the bad code path -- the errors happen when the wasm modules are linked (at load time in the browser) Anyways, this fixes more linker errors. Mostly the problems have to do with LAPACK functions that take string arguments. Most LAPACK functions that take string arguments use them as enums and only care about the first character of the string. Because of the way that f2c works, we need to replace these strings with the ascii code of the first character so we should replace: sTRSV( 'UPPER', 'NOTRANS', 'NONUNIT', J, H, LDH, Y, 1 ) ==> CALL sTRSV( 85, 78, 78, J, H, LDH, Y, 1 ) where 85 and 78 are the ascii codes of U and N. Various character variables are subbed into being integer variables. The two functions `ilaenv` and `xerbla` expect actual C strings as an argument, but it is very annoying to produce C strings so instead I added wrapper functions ilaenvf2c and xerblaf2c to clapack and instead of calling ilaenv and xerbla we call the f2c versions.
2022-03-24 06:17:29 +00:00
# sed -i 's/^void/int/g' scipy/linalg/cython_lapack_signatures.txt
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
# Input error causes "duplicate symbol" linker errors. Empty out the file.
echo "" > scipy/sparse/linalg/dsolve/SuperLU/SRC/input_error.c
Initial steps toward Scipy support
2018-07-18 13:32:16 +00:00
requirements:
run:
- numpy
Build C libraries via buildall.py (#927) This addresses part of #713, by listing and building C libraries as dependencies. This means we automatically build CLAPACK when needed, and `lxml`'s C dependencies are not built if lxml is not. In particular, building "core" should now be faster. The building itself is still performed by Makefile.
2020-12-23 13:24:32 +00:00
- CLAPACK
More fixes
2018-10-30 12:10:10 +00:00
test:
imports:
- scipy
Refactor scipy tests
2018-10-31 14:06:08 +00:00
- scipy.cluster
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
- scipy.cluster.vq
- scipy.cluster.hierarchy
Refactor scipy tests
2018-10-31 14:06:08 +00:00
- scipy.constants
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
- scipy.fft
Refactor scipy tests
2018-10-31 14:06:08 +00:00
- scipy.fftpack
- scipy.integrate
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
- scipy.interpolate
- scipy.io
- scipy.io.arff
- scipy.io.matlab
- scipy.io.wavfile
More fixes
2018-10-30 12:10:10 +00:00
- scipy.linalg
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
- scipy.linalg.blas
- scipy.linalg.cython_blas
- scipy.linalg.lapack
- scipy.linalg.cython_lapack
- scipy.linalg.interpolative
Refactor scipy tests
2018-10-31 14:06:08 +00:00
- scipy.misc
- scipy.ndimage
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
- scipy.odr
- scipy.optimize
- scipy.signal
- scipy.signal.windows
- scipy.sparse
- scipy.sparse.linalg
- scipy.sparse.csgraph
Refactor scipy tests
2018-10-31 14:06:08 +00:00
- scipy.spatial
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
- scipy.spatial.distance
- scipy.spatial.transform
More fixes
2018-10-30 12:10:10 +00:00
- scipy.special
Update to Scipy 1.7.3 (#2065)
2022-01-03 22:07:13 +00:00
- scipy.stats
- scipy.stats.contingency
- scipy.stats.distributions
- scipy.stats.mstats
- scipy.stats.qmc