mirror of https://github.com/BOINC/boinc.git
edits
David Anderson
parent
6c831eab72
commit
5e88264358
|
|
@ -0,0 +1,149 @@
|
|||
Some concepts that are needed to deploy apps with BOINC.
|
||||
|
||||
## Apps and app versions
|
||||
|
||||
In BOINC, 'app' has a special meaning.
|
||||
It's the abstraction of a program:
|
||||
something that takes inputs and produces outputs.
|
||||
Each app has a unique name (like 'autodock').
|
||||
|
||||
An 'app version' is a specific executable file
|
||||
(or set of files) that performs this computation.
|
||||
It's associated with a particular app,
|
||||
and with a plaform (Windows/Intel, Mac/Intel, Linux/ARM, etc.).
|
||||
|
||||
App versions have version numbers (like 3.2).
|
||||
When the BOINC server sends a job to a client,
|
||||
it uses the app version for that platform
|
||||
with the largest version number.
|
||||
|
||||
The [server cookbook](Create-a-BOINC-server-(cookbook))
|
||||
shows how to create apps and app versions.
|
||||
|
||||
## Plan classes
|
||||
|
||||
In some cases info beyond just platform
|
||||
is needed to decide whether an app version can run
|
||||
on a particular host.
|
||||
For example:
|
||||
|
||||
* The executable is compiled to use instructions (like SSE3)
|
||||
that exist on some Intel-type processors but not others.
|
||||
* The program uses a GPU and requires a particular
|
||||
range of GPU models and video driver versions.
|
||||
* The program requires that virtualization software
|
||||
(like VirtualBox) be installed on the host.
|
||||
|
||||
A set of such requirements is called a 'plan class'.
|
||||
An app version can be associated with a plan class,
|
||||
in which case it will be sent only to hosts
|
||||
that meet the requirements.
|
||||
|
||||
There are a number of pre-defined plan classes:
|
||||
* mt: multithreaded programs that use from 2 to 16 cores.
|
||||
* sse3: programs compiled to use SSE3 instructions.
|
||||
* vbox_64: programs that use VirtualBox, 64-bit.
|
||||
* cuda: programs that use NVIDIA GPU, compute capability 1.0+
|
||||
|
||||
You can define your own plan classes in XML.
|
||||
Details are [here](AppPlan).
|
||||
|
||||
## Files
|
||||
|
||||
### File immutability
|
||||
|
||||
We're concerned here with files
|
||||
that are copied between client and server
|
||||
(input and output files, app version files).
|
||||
Such files are 'immutable'.
|
||||
All replicas of a file with a given name, from a given project,
|
||||
are assumed (and required) to be identical.
|
||||
If a file is change, even by a single byte,
|
||||
it becomes a new file and must be given a different name.
|
||||
This can be done, e.g., by including a version number in the file name.
|
||||
|
||||
### Server directory structure
|
||||
|
||||
On a BOINC server, files are stored in 2-level
|
||||
directory hierarchies,
|
||||
one for uploads, one for downloads.
|
||||
Each of these has 1024 subdirectories
|
||||
(named '0' to '3ff').
|
||||
A file is placed in a subdirectory based
|
||||
on the MD5 has of its filename.
|
||||
This was done because projects can have large numbers (millions)
|
||||
of files at a given time,
|
||||
and some filesystems become extremely slow
|
||||
when a single directory contains that many files.
|
||||
|
||||
When a job is submitted, its input files are 'staged'
|
||||
by moving or copying them to the appropriate place in
|
||||
the download hierarchy.
|
||||
When a client completes a job, it uploads the output files
|
||||
to the appropriate place in the upload hierarchy.
|
||||
|
||||
### Client directory structure
|
||||
|
||||
Input and output files have, in addition to their
|
||||
names (or 'physical names'),
|
||||
a 'logical name', which is the name by which the application refers to them.
|
||||
dir structure on client
|
||||
|
||||
The BOINC client, in its data directory, has two subdirectories:
|
||||
* **projects/** has a subdirectory for each project
|
||||
that the client is attached to.
|
||||
* **slots/** has a subdirectory for each currently active job.
|
||||
|
||||
The slot directory contains "link files", with logical names,
|
||||
that refer to the corresponding
|
||||
physical file in the project directory.
|
||||
Link files look like
|
||||
```
|
||||
<soft_link>../../projects/project.url.edu/phys_filename</soft_link>
|
||||
```
|
||||
|
||||
Thus, the client directory structure looks like this:
|
||||
|
||||
<img src="https://github.com/BOINC/boinc/blob/master/doc/client_dirs.jpg" alt="directory structure" width="600"/>
|
||||
|
||||
In some cases (if indicated by a ```<copy/>``` tag the job or app version config file)
|
||||
the client copies the file to the slot directory,
|
||||
giving it the logical name.
|
||||
|
||||
## Process structure
|
||||
|
||||
The BOINC client communicates with running jobs
|
||||
by message-passing through a shared memory segment.
|
||||
The client can tell the app to suspend or resume,
|
||||
or tell it to checkpoint.
|
||||
The app can reports its fraction done and CPU time to the client,
|
||||
or confirm that it's checkpointed.
|
||||
|
||||
The app side of this communication is done by
|
||||
a 'BOINC runtime library' - a C++ library that's
|
||||
linked into the application.
|
||||
The library also provides API functions for
|
||||
converting logical names to physical names
|
||||
(i.e. for parsing link files)
|
||||
and for initialization and finalization.
|
||||
Every BOINC app must use the library.
|
||||
There are several ways to do this:
|
||||
|
||||
## App packaging options
|
||||
|
||||
There are several ways to 'package' an application for use with BOINC:
|
||||
|
||||
* ""Native app**: add API calls to the program source code,
|
||||
then compile it (for a given platform) and link it to the BOINC runtime library.
|
||||
* Wrapper: use a BOINC-supplied program ('wrapper') that interfaces
|
||||
between the BOINC client and an unmodified executable.
|
||||
This removes the need to change your program's source code,
|
||||
but you still need to build it for different platforms (e.g. Windows).
|
||||
* VM apps: your app runs inside a virtual machine (typically Linux).
|
||||
A BOINC-supplied program called vboxwrapper interfaces
|
||||
between the BOINC client, VirtualBox, and the VM.
|
||||
|
||||
Each approach has pros and cons.
|
||||
The VM approach is most convenient - you don't have to
|
||||
develop on Windows or Mac - but it has a performance overhead,
|
||||
and (currently) you can't use GPUs.
|
||||
|
|
@ -1,7 +0,0 @@
|
|||
apps and app versions
|
||||
plan classes
|
||||
files; dir structure
|
||||
process structure
|
||||
BOINC API, communication
|
||||
packaging options
|
||||
native, wrapper, VM
|
||||
|
|
@ -149,7 +149,7 @@ There are several job-submission variants:
|
|||
* Use BC's generic file-management and job-submission web interfaces
|
||||
to submit jobs to Autodock (or other apps on BC).
|
||||
* Install the BOINC job-submission and boinc2docker packages.
|
||||
Use the [command-line job submission system](https://github.com/BOINC/boinc/wiki/BOINC-CLI:-command-line-job-submission)
|
||||
Use the [command-line job submission system](Command-line-job-submission)
|
||||
to submit jobs for arbitrary (Linux/x86 or Python) applications.
|
||||
In this case the user must select science area keywords.
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@
|
|||
* [Scientist interface](Scientist-interface). This is based on and replaces:
|
||||
* [The BOINC out of box experience](The-BOINC-out-of-box-experience)
|
||||
* [The BOINC test drive](The-BOINC-test-drive)
|
||||
* [Command-line job submission](https://github.com/BOINC/boinc/wiki/BOINC-CLI:-command-line-job-submission)
|
||||
* [Command-line job submission](Command-line-job-submission)
|
||||
* [Supporting Windows hosts with > 64 cores](WinMulticore)
|
||||
* [Maintaining upload statistics](UploadStatistics)
|
||||
* [Locality scheduling redesign](LocalityNew)
|
||||
|
|
|
|||
Loading…
Reference in New Issue