diff --git a/Docker-app-implementation.md b/Docker-app-implementation.md index 81a60bf..bd544b6 100644 --- a/Docker-app-implementation.md +++ b/Docker-app-implementation.md @@ -1,8 +1,8 @@ -## Docker and Podman +# Docker and Podman Docker and Podman provide similar capabilities: -they can build images from Dockerfiles, and run containers, -through a CLI interface. +they offer a CLI that lets you +build images from Dockerfiles and run containers, They differ in architecture: * Docker uses a daemon, which typically runs as root. @@ -11,17 +11,20 @@ and communicates with the daemon over a pipe. * Podman doesn't use a daemon. The CLI program works directly. -However, the Linux 'control group' mechanism is available only to root. -The container pause/unpause mechanism relies on this. +However, the Linux 'control group' mechanism is available only to root, +and the container pause/unpause mechanism relies on this. So if you want to pause containers (needed for BOINC) you have to run the CLI program as root. -## Host requirements +# Host requirements -### Unix +What volunteers need to do: + +## Unix Docker must be installed and configured so that -* the service is always running. + +* the daemon is always running. * the user under which the BOINC client runs can run the DOCKER CLI programs. There are various ways to do this: see @@ -30,7 +33,7 @@ https://docs.docker.com/engine/install/ubuntu/ We can't use Podman on Unix because the BOINC client can't run apps as root (see above). -### Windows +## Windows WSL and Hyper-V must be enabled. There must be either @@ -46,20 +49,30 @@ sudo apt -y install podman Our plan is to create a WSL with Podman installed, put it on the Windows store, recommend that to volunteers. -* A WSL distro with Docker installed and configured to always run. -It suffices to install Docker Desktop. +* A WSL distro with Docker installed and configured so +that the daemon always runs. +It suffices to install Docker Desktop; +this creates a WSL distro called ```docker-desktop```. If both are present, a WSL distro with Podman is used. -We prefer Podman over Docker on Windows because it's simpler -(for us and for volunteers who want to do it themselves) -to install Podman. +We prefer Podman over Docker on Windows because -## Running a job +* Podman is simpler +(for us and for volunteers who want to do it themselves) +to install and configure. +* Podman is free and open source; Docker is not. + +We initially considered requiring volunteers to install +Docker Desktop. +But this is involved; they have to create an account. +Our approach is basically one click. + +# Running a job Here's what happens when the BOINC client runs a Docker app: -### Unix +## Unix [[images/docker_unix.jpg]] @@ -68,42 +81,33 @@ The BOINC client runs ```docker_wrapper``` and communicates with it (pause/resume etc.) through shared memory, as with all BOINC apps. -```docker_wrapper``` interacts with Docker -by running the Docker CLI program (using ```popen()```). +```docker_wrapper``` interacts with Docker or Podman +by running the CLI program (using ```popen()```). It issues commands to list images, build an image, and start a container, and monitor the container. -```docker_wrapper``` is responsible for getting -input files into the container, and output files out of the container. -It does this by either +```docker_wrapper``` mounts the slot directory at the container's WORKDIR. -* using the ```docker cp``` command. -* mounting the slot and project directories in the container, -in which case the logic in the container is responsible for -finding the files -(e.g. parsing the link files in the slot directory -to get the physical names in the project directory). +```docker_wrapper``` optionally mounts the project directory in the container, +if specified in the job config file, ```job.toml```. -This is controlled by the job config file, ```job.toml```. - -### Windows +## Windows [[images/docker_win.jpg]] Based on info in ```init_data.xml```, ```docker_wrapper``` picks a WSL distro. Using ```CreateProcess()```, it runs ```wsl -d distro -u root```, -which runs a shell (as root) in WSL, -connected to docker_wrapper by pipes. +which runs a shell (as root) in WSL, connected to docker_wrapper by pipes. -Using these pipes, ```docker_wrapper``` issues Podman commands, +Using these pipes, ```docker_wrapper``` issues Docker CLI commands, which run in the WSL container. These commands build images, run Docker containers, etc. The WSL container shares the host filesystem. The shell's current directory is that of ```docker_wrapper```, namely the job's slot directory. -Moving files in/out of the Docker container is done -the same as in the Unix case. -# Mac OS +## Mac OS + +TBD diff --git a/Docker-apps.md b/Docker-apps.md index 89f0ac5..6d24c55 100644 --- a/Docker-apps.md +++ b/Docker-apps.md @@ -151,32 +151,33 @@ unless they are marked as `````` in the job's input template. The files of a Docker app can be divided between app version and workunit in two ways. -### Single-purpose app +### Single-purpose BOINC app -In this model, the BOINC app version for a platform contains +In this model, there is one BOINC app per science app. +The BOINC app version for a platform contains * Dockerfile * docker_wrapper (compiled for that platform) * job.toml * science executables -and the workunit contains +and each workunit contains * input files -In this model, -to deploy a new science application you need to create a new BOINC app, +To deploy a new science application you need to create a new BOINC app, and to deploy a new science application version you need to create a new BOINC app version. These both require login access to the BOINC server. -### Universal app +### Universal BOINC app -In this model, the BOINC app version for a platform contains +In this model, a single BOINC handles multiple science apps. +The BOINC app version for a platform contains * docker_wrapper (compiled for that platform) -and the workunit contains +and each workunit contains * Dockerfile * science executables @@ -189,10 +190,10 @@ but this matters less since they run in a container. If your project has a mechanism for remote (e.g. web-based) job submission, -this means that job submitters can deploy new +job submitters can deploy new science app versions or new science apps without login access. -There are a couple of issues with this model: +Issues with this model: * In the current BOINC architecture, each BOINC app has its own validator and assimilator. @@ -201,5 +202,5 @@ we'll need a way to let them have different validators and assimilators. This could be built on the script-based framework. * Science apps may have different numbers of input files, -and they need to control which ones are copy mode. +and they need to control which ones use "copy mode". This could be handled using per-batch input templates.