Table of Contents
Supporting Windows computers with > 64 cores
On Windows machines with > 64 cores, the cores are divided into "processor groups". A given process runs in a particular process group, inherited from its parent.
Docs on this:
https://docs.microsoft.com/en-us/windows/win32/procthread/processor-groups
https://docs.microsoft.com/en-us/windows/win32/procthread/what-s-new-in-processes-and-threads
https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-setprocessaffinitymask
Unpleasant, but we're stuck with it.
Currently, the client doesn't do anything related to processor groups, Apparently this leads to underutilization of CPUs; see https://github.com/BOINC/boinc/issues/1357
Proposed development:
Phase 0
On startup, get and print processor group info.
Phase 1
As the client runs apps, it assigns them to a processor group, and keeps track of how many free cores each processor group has. The assignment policy: run the app in the group that has enough free cores, and for which the number of free cores is smallest. This will tend to run multicore apps in their own process groups, and sequential apps in other groups.
Note: the server doesn't know about processor groups. It may send jobs that want to use 256 cores. These will end up running as 256 threads on 64 cores.
Phase 2
Send processor group info to the server, which can use it to set the # of threads for multicore apps.
Notes
-
We're not going to use processor affinity. If a job runs in a processor group, its affinity mask is all the processors in that group.
-
It's possible for applications to create threads in multiple processor groups, e.g. to use all 256 cores. For now, we'll assume that applications don't do this.
It's not clear how (or even whether Windows) lets you specify the processor group for a process, new or existing.
This looks like it might work:
https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-setprocessaffinitymask
One of the above docs says "The operating system initially assigns each process to a single group in a round-robin manner across the groups in the system." Does that mean there's no control in process creation?
If processor group/affinity is inherited, maybe the client can changes its group prior to creating the process?
Can we create and use an "startup info" structure during process creation?
Change processor group size of boot (testing):
This makes it sound difficult or impossible: