edit

BUDA-overview.md

@@ -1,116 +1,21 @@
 BOINC Universal Docker app (BUDA)
 is a framework for running Docker-based science apps on BOINC.
 
-It's 'universal' in the sense that one BOINC app
+The goal of BUDA is to let scientists use a BOINC project to
-handles arbitrary science apps.
-The science app's Dockerfile and executables
-are in workunits rather than app versions.
 
-On the server, there is a single BOINC app; let's call it 'buda'.
+* Develop Docker-based applications (possibly using GPUs)
-This has app versions for the various platforms (Win, Mac, Linux)
+* Run batches of jobs (with different input files and command lines)
-Each app version contains the Docker wrapper built for that platform.
+with these applications
+* Monitor and download the output files of these jobs
 
-There are various possible interfaces for job submission to BUDA.
+... entirely through a simple web interface,
-We could make a Python-based remote API.
+with no need to understand anything about BOINC
-We (or others) could use this API to integrate it into batch systems.
+or to log into the project's BOINC server.
 
-But for starters, we implemented a generic (multi-application)
+If you're a scientist using a BUDA-enabled BOINC project,
-web-based job submission system,
+read about [[BUDA-job-submission]].
-using the per-user file sandbox system.
 
-## BUDA science apps and versions
+If you administer a BOINC project and want to BUDA-enable it,
-
+read about [[BUDA-setup]].
-BOINC provides a web interface for managing BUDA science apps
+If you want to know how it works, read
-and submitting jobs to them.
+[[BUDA-implementation]].
-These tools assume the following structure:
-
-* Each science app has a name, like 'worker' or 'autodock'.
-* A science app can have multiple variants,
-each with a different plan class.
-There might be variants for 1 CPU, for N CPUs, and for various GPU types.
-(BUDA handles science apps that use GPUs).
-
-Each science app variant is a collection of files:
-
-* A Dockerfile
-* a config file, `job.toml`
-* input and output templates
-* A main program or script
-* Other files
-* A file 'file_list' listing the other files in template order.
-
-The set of science apps and variants is represented
-by a directory hierarchy of the form
-```
-project/buda_apps/
-    <sci_app_name>/
-        cpu/
-            ... files
-        <plan_class>/
-            ... files
-        ...
-    ...
-```
-Note: you can build this hierarchy manually but
-typically it's maintained using a web interface; see below.
-
-This is similar to BOINC's hierarchy of apps and app versions, except:
-
-* It's represented in a directory hierarchy, not in database tables
-* Science app variants are not associated with platforms
-(since we're using Docker).
-* It stores only the current version, not a sequence of versions
-(that's why we call them 'variants', not 'versions').
-
-## BUDA is not polymorphic
-
-Conventional BOINC apps are 'polymorphic':
-if an app has both CPU and GPU variants,
-you submit jobs without specifying which one to use;
-the BOINC scheduler makes the decision.
-
-It would be possible to make BUDA polymorphic,
-but this would be complex, requiring significant changes to the scheduler.
-So - at least for now - BUDA is not polymorphic.
-
-When you submit jobs you have to specify which plan class to use.
-This could be a slight nuisance:
-a plan class could have little computing power,
-and you might avoid using it, but then you wouldn't get the power.
-
-## Validators and assimilators
-
-In the current BOINC architecture,
-each BOINC app has its own validator and assimilator.
-If multiple science apps "share" the same BOINC app,
-we'll need a way to let them have different validators and assimilators.
-
-This could be built on the script-based framework;
-each science app could specify the names
-of validator and assimilator scripts,
-which would be stored in workunits.
-
-## Implementation notes
-
-BUDA will require changes to the scheduler.
-
-Currently: given a job, it scans app versions,
-looking for one host can accept based on plan class.
-That won't work here.
-The plan class is already fixed.
-
-Instead:
-* add plan_class field to workunit (or could put in xml_doc)
-* if considering sending a WU to a host, and WU has a plan class
-    * skip if no app version with that platform / plan class (e.g. can't send metal job to Win host)
-    * skip if host can't handle the plan class
-
-## If we wanted to make BUDA polymorphic
-
-* The scheduler would have to scan the `buda_apps` dir structure (or we could add this info to the DB).
-
-* Jobs are tagged with BUDA science app name.
-* The scheduler scans versions of that science app.
-* If find a plan class the host can accept, build wu.xml_doc based on BUDA app version info.
-
-The above is possible but would be a lot of work.