* Add opencv-python
* Update comment
* Add JPEG, PNG, WEBP, ZLIB support
* Add tests for image processing
* Add more core modules
* Disable opencl
* Replace lena with baboon and add more tests
* Add file system support
* Add ffmpeg
* Add more tests
* Disable pthread in ffmpeg
* Disable canonical input processing mode in node test
* Update changelog
* Remove import test
* Allow more time in the first test
* Split out libwebp
* Fix node test
* Use a seperate CI job for opencv-python
* Fix generator
* Update changelog
* Remove protobuf package
* Try to fix CI workspace conflict
* Fix CI
* Use another CI job for generating unified packages.json
* Try to fix CI
* Fix CI again
* Disable verbose build
* Prevent from building opencv-python twice
* Persist only build artifacts
* Sepearate Cmake args into a script
* Try to reuse build packages job
* Fix CI
* Fix typo
* Fix merge conflict
* Use large resource class for package build
* Do not upload unwanted artifacts
* Do not upload unwanted artifacts
dist is both more accurate (the 'build' directory is normally where you do the build,
and normally consists of intermediate build artifacts no one cares about). dist also
occurs less frequently in the code base: after this change \bbuild\b has 466 matches,
whereas \bdist\b has 101 matches. build has 1072 matches whereas dist has 362.
We are pushing pyodide_build to PyPI as a Python package, but for now,
installing pyodide_build from PyPI (i.e. pip install pyodide_build`) is almost
useless because:
there are bunch of hard-coded paths (e.g. Path(__file__).parents[2]),
its dependencies are not specified in setup.cfg.
This PR is for mitigating this situation by removing hard-coded paths and
adding tests, and is also a preparation for our new CLI
(https://github.com/pyodide/pyodide/issues/1977).
With newer versions of emscripten, linker errors surface eariler.
This makes it easier to find function pointer cast errors without
having to execute the bad code path -- the errors happen when the
wasm modules are linked (at load time in the browser)
Anyways, this fixes more linker errors. Mostly the problems have
to do with LAPACK functions that take string arguments. Most
LAPACK functions that take string arguments use them as enums and
only care about the first character of the string. Because of the
way that f2c works, we need to replace these strings with the ascii
code of the first character so we should replace:
sTRSV( 'UPPER', 'NOTRANS', 'NONUNIT', J, H, LDH, Y, 1 )
==>
CALL sTRSV( 85, 78, 78, J, H, LDH, Y, 1 )
where 85 and 78 are the ascii codes of U and N. Various character
variables are subbed into being integer variables. The two
functions `ilaenv` and `xerbla` expect actual C strings as an
argument, but it is very annoying to produce C strings so instead
I added wrapper functions ilaenvf2c and xerblaf2c to clapack and
instead of calling ilaenv and xerbla we call the f2c versions.
This resolves#2189.
> build isolation would be a bit difficult to use in our case, as for instance
> when building scipy we need the patched numpy on the host and not the numpy
> version specified in pyproject.toml (which would be unpatched)
This is indeed the case, certain packages cannot be isolated. My strategy is to
make a list of packages that shouldn't be isolated and add symlinks from the
isolated build environment into the `.artifacts` directory to "unisolate" them.
Then we remove the unisolated package requirements from the list of packages to
install, in case pesky constraints aren't satisfied. In particular, packages
that expect to be used with `pypa/build` often feel free to put very specific
constraints on their build dependencies (often asking them to be == to a
particular version). Specific version constraints is good for build
reproducibility and with build isolation doesn't cost anything. So we just
ignore the constraints. Hopefully nothing goes wrong.
In particular, any package that does stuff both at build time and at runtime and
requires synchronization between the build time and run time environments needs
the unisolation. This includes cffi with `_cffi_backend.so`, and of course numpy
and scipy. pycparser needs to be unisolated because it is a dependency of cffi.
Currently I have also unisolated pythran and cython, though these are build time
only tools and do not really need to be unisolated. Cython I unisolated
specifically because numcodecs needs it but it isn't in the numcodecs build
dependencies. Pythran I unisolated because of a problem with the scipy build
which I don't fully understand (some problem with long double feature
detection).
Moves the logic of generating zip archive of a shared library from
`buildall` to `buildpkg` so that thread-unsafe `shutil.make_archive`
won't bother us anymore.
* Fix dangling webdriver
* Embed humor sans font
* Refactor matplotlib testing codes
* Trigger CI
* Trigger CI
* Catch error when import from js fails
* Update changelog
* Apply suggestions from the review
Our package build process currently has a significant flaw: we first run setup.py, recording all compilation commands, then we rewrite these compilation commands to invoke emcc and replay them, and then we pray that the cross compiled executables ended up in the right place to go into the wheel. This is not a good strategy because the build script is allowed to implement arbitrary logic, and if it moves, renames, etc any of the output files then we lose track of them. This has repeatedly caused difficulty for us.
However, we also make no particularly significant use of the two pass approach. We can just do the simpler thing: capture the compiler commands as they occur, modify them as needed, and then run the fixed command.
I also added a patch to fix the numpy feature detection for wasm so that we don't have to include _npyconfig.h and config.h, numpy can generate them in the way it would for a native build. I opened a numpy PR that would fix the detection for us upstream:
numpy/numpy#21154
This clears the way for us to switch to using pypa/build (as @henryiii has suggested) by removing our dependence on specific setuptools behavior.
This is on top of #2238.
Gyeongjae Choi