mirror of https://github.com/BOINC/boinc.git
updates
David Anderson
parent
c9620babb7
commit
6149b0475f
|
|
@ -1,8 +1,8 @@
|
|||
## Docker and Podman
|
||||
# Docker and Podman
|
||||
|
||||
Docker and Podman provide similar capabilities:
|
||||
they can build images from Dockerfiles, and run containers,
|
||||
through a CLI interface.
|
||||
they offer a CLI that lets you
|
||||
build images from Dockerfiles and run containers,
|
||||
They differ in architecture:
|
||||
|
||||
* Docker uses a daemon, which typically runs as root.
|
||||
|
|
@ -11,17 +11,20 @@ and communicates with the daemon over a pipe.
|
|||
|
||||
* Podman doesn't use a daemon.
|
||||
The CLI program works directly.
|
||||
However, the Linux 'control group' mechanism is available only to root.
|
||||
The container pause/unpause mechanism relies on this.
|
||||
However, the Linux 'control group' mechanism is available only to root,
|
||||
and the container pause/unpause mechanism relies on this.
|
||||
So if you want to pause containers (needed for BOINC)
|
||||
you have to run the CLI program as root.
|
||||
|
||||
## Host requirements
|
||||
# Host requirements
|
||||
|
||||
### Unix
|
||||
What volunteers need to do:
|
||||
|
||||
## Unix
|
||||
|
||||
Docker must be installed and configured so that
|
||||
* the service is always running.
|
||||
|
||||
* the daemon is always running.
|
||||
* the user under which the BOINC client runs
|
||||
can run the DOCKER CLI programs.
|
||||
There are various ways to do this: see
|
||||
|
|
@ -30,7 +33,7 @@ https://docs.docker.com/engine/install/ubuntu/
|
|||
We can't use Podman on Unix because the BOINC client
|
||||
can't run apps as root (see above).
|
||||
|
||||
### Windows
|
||||
## Windows
|
||||
|
||||
WSL and Hyper-V must be enabled.
|
||||
There must be either
|
||||
|
|
@ -46,20 +49,30 @@ sudo apt -y install podman
|
|||
Our plan is to create a WSL with Podman installed,
|
||||
put it on the Windows store, recommend that to volunteers.
|
||||
|
||||
* A WSL distro with Docker installed and configured to always run.
|
||||
It suffices to install Docker Desktop.
|
||||
* A WSL distro with Docker installed and configured so
|
||||
that the daemon always runs.
|
||||
It suffices to install Docker Desktop;
|
||||
this creates a WSL distro called ```docker-desktop```.
|
||||
|
||||
If both are present, a WSL distro with Podman is used.
|
||||
|
||||
We prefer Podman over Docker on Windows because it's simpler
|
||||
(for us and for volunteers who want to do it themselves)
|
||||
to install Podman.
|
||||
We prefer Podman over Docker on Windows because
|
||||
|
||||
## Running a job
|
||||
* Podman is simpler
|
||||
(for us and for volunteers who want to do it themselves)
|
||||
to install and configure.
|
||||
* Podman is free and open source; Docker is not.
|
||||
|
||||
We initially considered requiring volunteers to install
|
||||
Docker Desktop.
|
||||
But this is involved; they have to create an account.
|
||||
Our approach is basically one click.
|
||||
|
||||
# Running a job
|
||||
|
||||
Here's what happens when the BOINC client runs a Docker app:
|
||||
|
||||
### Unix
|
||||
## Unix
|
||||
|
||||
[[images/docker_unix.jpg]]
|
||||
|
||||
|
|
@ -68,42 +81,33 @@ The BOINC client runs ```docker_wrapper```
|
|||
and communicates with it (pause/resume etc.)
|
||||
through shared memory, as with all BOINC apps.
|
||||
|
||||
```docker_wrapper``` interacts with Docker
|
||||
by running the Docker CLI program (using ```popen()```).
|
||||
```docker_wrapper``` interacts with Docker or Podman
|
||||
by running the CLI program (using ```popen()```).
|
||||
It issues commands to list images, build an image,
|
||||
and start a container, and monitor the container.
|
||||
|
||||
```docker_wrapper``` is responsible for getting
|
||||
input files into the container, and output files out of the container.
|
||||
It does this by either
|
||||
```docker_wrapper``` mounts the slot directory at the container's WORKDIR.
|
||||
|
||||
* using the ```docker cp``` command.
|
||||
* mounting the slot and project directories in the container,
|
||||
in which case the logic in the container is responsible for
|
||||
finding the files
|
||||
(e.g. parsing the link files in the slot directory
|
||||
to get the physical names in the project directory).
|
||||
```docker_wrapper``` optionally mounts the project directory in the container,
|
||||
if specified in the job config file, ```job.toml```.
|
||||
|
||||
This is controlled by the job config file, ```job.toml```.
|
||||
|
||||
### Windows
|
||||
## Windows
|
||||
|
||||
[[images/docker_win.jpg]]
|
||||
|
||||
Based on info in ```init_data.xml```,
|
||||
```docker_wrapper``` picks a WSL distro.
|
||||
Using ```CreateProcess()```, it runs ```wsl -d distro -u root```,
|
||||
which runs a shell (as root) in WSL,
|
||||
connected to docker_wrapper by pipes.
|
||||
which runs a shell (as root) in WSL, connected to docker_wrapper by pipes.
|
||||
|
||||
Using these pipes, ```docker_wrapper``` issues Podman commands,
|
||||
Using these pipes, ```docker_wrapper``` issues Docker CLI commands,
|
||||
which run in the WSL container.
|
||||
These commands build images, run Docker containers, etc.
|
||||
|
||||
The WSL container shares the host filesystem.
|
||||
The shell's current directory is that of ```docker_wrapper```,
|
||||
namely the job's slot directory.
|
||||
Moving files in/out of the Docker container is done
|
||||
the same as in the Unix case.
|
||||
|
||||
# Mac OS
|
||||
## Mac OS
|
||||
|
||||
TBD
|
||||
|
|
|
|||
|
|
@ -151,32 +151,33 @@ unless they are marked as ```<sticky/>``` in the job's input template.
|
|||
The files of a Docker app can be divided between
|
||||
app version and workunit in two ways.
|
||||
|
||||
### Single-purpose app
|
||||
### Single-purpose BOINC app
|
||||
|
||||
In this model, the BOINC app version for a platform contains
|
||||
In this model, there is one BOINC app per science app.
|
||||
The BOINC app version for a platform contains
|
||||
|
||||
* Dockerfile
|
||||
* docker_wrapper (compiled for that platform)
|
||||
* job.toml
|
||||
* science executables
|
||||
|
||||
and the workunit contains
|
||||
and each workunit contains
|
||||
|
||||
* input files
|
||||
|
||||
In this model,
|
||||
to deploy a new science application you need to create a new BOINC app,
|
||||
To deploy a new science application you need to create a new BOINC app,
|
||||
and to deploy a new science application version
|
||||
you need to create a new BOINC app version.
|
||||
These both require login access to the BOINC server.
|
||||
|
||||
### Universal app
|
||||
### Universal BOINC app
|
||||
|
||||
In this model, the BOINC app version for a platform contains
|
||||
In this model, a single BOINC handles multiple science apps.
|
||||
The BOINC app version for a platform contains
|
||||
|
||||
* docker_wrapper (compiled for that platform)
|
||||
|
||||
and the workunit contains
|
||||
and each workunit contains
|
||||
|
||||
* Dockerfile
|
||||
* science executables
|
||||
|
|
@ -189,10 +190,10 @@ but this matters less since they run in a container.
|
|||
|
||||
If your project has a mechanism for remote
|
||||
(e.g. web-based) job submission,
|
||||
this means that job submitters can deploy new
|
||||
job submitters can deploy new
|
||||
science app versions or new science apps without login access.
|
||||
|
||||
There are a couple of issues with this model:
|
||||
Issues with this model:
|
||||
|
||||
* In the current BOINC architecture,
|
||||
each BOINC app has its own validator and assimilator.
|
||||
|
|
@ -201,5 +202,5 @@ we'll need a way to let them have different validators and assimilators.
|
|||
This could be built on the script-based framework.
|
||||
|
||||
* Science apps may have different numbers of input files,
|
||||
and they need to control which ones are copy mode.
|
||||
and they need to control which ones use "copy mode".
|
||||
This could be handled using per-batch input templates.
|
||||
|
|
|
|||
Loading…
Reference in New Issue