Upstream PR: numpy/numpy#21733 (files)
Upstream patch was accepted. Final upstream version is slightly different to support windows,
but the difference doesn't matter for us.
Our package build process currently has a significant flaw: we first run setup.py, recording all compilation commands, then we rewrite these compilation commands to invoke emcc and replay them, and then we pray that the cross compiled executables ended up in the right place to go into the wheel. This is not a good strategy because the build script is allowed to implement arbitrary logic, and if it moves, renames, etc any of the output files then we lose track of them. This has repeatedly caused difficulty for us.
However, we also make no particularly significant use of the two pass approach. We can just do the simpler thing: capture the compiler commands as they occur, modify them as needed, and then run the fixed command.
I also added a patch to fix the numpy feature detection for wasm so that we don't have to include _npyconfig.h and config.h, numpy can generate them in the way it would for a native build. I opened a numpy PR that would fix the detection for us upstream:
numpy/numpy#21154
This clears the way for us to switch to using pypa/build (as @henryiii has suggested) by removing our dependence on specific setuptools behavior.
This is on top of #2238.
This patch makes everything in numpy, scipy and CLAPACK that began life as a fortran subroutine return int, whereas before they were a smattering of void plus lots of int, with a bunch of conflicts where the same function was defined as int in some and void in others. This matters because on upstream emscripten, these packages will not build due to linker conflicts, as the wasm linker can't link a function returning int to a function returning void.
Annoyingly, this has to be int return not void, because whereas a normal fortran subroutine doesn't return anything, there's an obscure feature of fortran 77 called alternative returns which allows subroutines to say they want to return by jumping to some other place in the code. f2c handles this with integer return values.