mirror of https://github.com/BOINC/boinc.git
edit
David Anderson
parent
b2ecbf6dea
commit
b659143572
123
BUDA-overview.md
123
BUDA-overview.md
|
|
@ -1,116 +1,21 @@
|
|||
BOINC Universal Docker app (BUDA)
|
||||
is a framework for running Docker-based science apps on BOINC.
|
||||
|
||||
It's 'universal' in the sense that one BOINC app
|
||||
handles arbitrary science apps.
|
||||
The science app's Dockerfile and executables
|
||||
are in workunits rather than app versions.
|
||||
The goal of BUDA is to let scientists use a BOINC project to
|
||||
|
||||
On the server, there is a single BOINC app; let's call it 'buda'.
|
||||
This has app versions for the various platforms (Win, Mac, Linux)
|
||||
Each app version contains the Docker wrapper built for that platform.
|
||||
* Develop Docker-based applications (possibly using GPUs)
|
||||
* Run batches of jobs (with different input files and command lines)
|
||||
with these applications
|
||||
* Monitor and download the output files of these jobs
|
||||
|
||||
There are various possible interfaces for job submission to BUDA.
|
||||
We could make a Python-based remote API.
|
||||
We (or others) could use this API to integrate it into batch systems.
|
||||
... entirely through a simple web interface,
|
||||
with no need to understand anything about BOINC
|
||||
or to log into the project's BOINC server.
|
||||
|
||||
But for starters, we implemented a generic (multi-application)
|
||||
web-based job submission system,
|
||||
using the per-user file sandbox system.
|
||||
If you're a scientist using a BUDA-enabled BOINC project,
|
||||
read about [[BUDA-job-submission]].
|
||||
|
||||
## BUDA science apps and versions
|
||||
|
||||
BOINC provides a web interface for managing BUDA science apps
|
||||
and submitting jobs to them.
|
||||
These tools assume the following structure:
|
||||
|
||||
* Each science app has a name, like 'worker' or 'autodock'.
|
||||
* A science app can have multiple variants,
|
||||
each with a different plan class.
|
||||
There might be variants for 1 CPU, for N CPUs, and for various GPU types.
|
||||
(BUDA handles science apps that use GPUs).
|
||||
|
||||
Each science app variant is a collection of files:
|
||||
|
||||
* A Dockerfile
|
||||
* a config file, `job.toml`
|
||||
* input and output templates
|
||||
* A main program or script
|
||||
* Other files
|
||||
* A file 'file_list' listing the other files in template order.
|
||||
|
||||
The set of science apps and variants is represented
|
||||
by a directory hierarchy of the form
|
||||
```
|
||||
project/buda_apps/
|
||||
<sci_app_name>/
|
||||
cpu/
|
||||
... files
|
||||
<plan_class>/
|
||||
... files
|
||||
...
|
||||
...
|
||||
```
|
||||
Note: you can build this hierarchy manually but
|
||||
typically it's maintained using a web interface; see below.
|
||||
|
||||
This is similar to BOINC's hierarchy of apps and app versions, except:
|
||||
|
||||
* It's represented in a directory hierarchy, not in database tables
|
||||
* Science app variants are not associated with platforms
|
||||
(since we're using Docker).
|
||||
* It stores only the current version, not a sequence of versions
|
||||
(that's why we call them 'variants', not 'versions').
|
||||
|
||||
## BUDA is not polymorphic
|
||||
|
||||
Conventional BOINC apps are 'polymorphic':
|
||||
if an app has both CPU and GPU variants,
|
||||
you submit jobs without specifying which one to use;
|
||||
the BOINC scheduler makes the decision.
|
||||
|
||||
It would be possible to make BUDA polymorphic,
|
||||
but this would be complex, requiring significant changes to the scheduler.
|
||||
So - at least for now - BUDA is not polymorphic.
|
||||
|
||||
When you submit jobs you have to specify which plan class to use.
|
||||
This could be a slight nuisance:
|
||||
a plan class could have little computing power,
|
||||
and you might avoid using it, but then you wouldn't get the power.
|
||||
|
||||
## Validators and assimilators
|
||||
|
||||
In the current BOINC architecture,
|
||||
each BOINC app has its own validator and assimilator.
|
||||
If multiple science apps "share" the same BOINC app,
|
||||
we'll need a way to let them have different validators and assimilators.
|
||||
|
||||
This could be built on the script-based framework;
|
||||
each science app could specify the names
|
||||
of validator and assimilator scripts,
|
||||
which would be stored in workunits.
|
||||
|
||||
## Implementation notes
|
||||
|
||||
BUDA will require changes to the scheduler.
|
||||
|
||||
Currently: given a job, it scans app versions,
|
||||
looking for one host can accept based on plan class.
|
||||
That won't work here.
|
||||
The plan class is already fixed.
|
||||
|
||||
Instead:
|
||||
* add plan_class field to workunit (or could put in xml_doc)
|
||||
* if considering sending a WU to a host, and WU has a plan class
|
||||
* skip if no app version with that platform / plan class (e.g. can't send metal job to Win host)
|
||||
* skip if host can't handle the plan class
|
||||
|
||||
## If we wanted to make BUDA polymorphic
|
||||
|
||||
* The scheduler would have to scan the `buda_apps` dir structure (or we could add this info to the DB).
|
||||
|
||||
* Jobs are tagged with BUDA science app name.
|
||||
* The scheduler scans versions of that science app.
|
||||
* If find a plan class the host can accept, build wu.xml_doc based on BUDA app version info.
|
||||
|
||||
The above is possible but would be a lot of work.
|
||||
If you administer a BOINC project and want to BUDA-enable it,
|
||||
read about [[BUDA-setup]].
|
||||
If you want to know how it works, read
|
||||
[[BUDA-implementation]].
|
||||
|
|
|
|||
Loading…
Reference in New Issue