Table of Contents

The example app
Download files
Create the app
Set up app version directory and files
Create the app version
Create in/out templates
Add handlers for completed jobs
Process a job
Other platforms

How it works

vm_image_x64_4.vdi
vboxwrapper_26207_windows_x86_64.exe
worker_2_x86_64-pc-linux-gnu
boinc_resolve
run_worker.sh
vbox_job.xml
vm_version.xml
Input/output templates
Debugging VM apps

Testing in isolation
Testing under BOINC

This document shows how to run an existing Linux/Intel application on BOINC. We'll use BOINC's Virtualbox wrapper to allow the application to run on Windows and Mac hosts as well as Linux.

We'll assume that

You've created a BOINC project.
The BOINC client is running on a Windows computer that can access the BOINC project, and that has Virtualbox installed.
The BOINC client is attached to the project.
Virtualbox is installed on your server

One way to do this (as described here) is to create the project in a Virtualbox VM running on a Windows host, with the VM configured for 'Bridged Adapter' networking. In this case the VM will have an IP address like 192.168.42.17. You should be able to access the project web site at http://192.168.42.17/test/ from both the host and guest systems.

The example app

We'll use a C++ program called 'worker' that's in the BOINC software distribution. This program takes a text file, converts it to uppercase, and writes it to an output file. The names of the input and output files are passed as command-line arguments.

You can run 'worker' manually on the BOINC server:

cd ~/boinc/samples/worker

cat > infile
Some Mixed-case Text
^D

./worker_2_x86_64-pc-linux-gnu infile outfile

cat outfile
SOME MIXED-CASE TEXT

Note that the name of the executable includes '2' (a version number) and 'x86_64_linux_gnu' (a platform name). Remember that files are immutable in BOINC; if you change the file, increment the version number.

Download files

BOINC supplies

a VM image for use with VM apps.
a vboxwrapper executable for Windows/x64 (built with an old version of glibc so it should run on most Linux systems).

Download them into ~/boinc/:

cd ~/boinc
wget https://boinc.berkeley.edu/dl/vm_image_x64_4.vdi
wget https://boinc.berkeley.edu/dl/vboxwrapper_26207_windows_x86_64.exe

Change the UUID of the VM image (otherwise, if your project uses the same VDI file as another project, jobs will fail with an error that the VDI file already exists).

VBoxManage internalcommands sethduuid vm_image_x64_4.vdi

Create the app

Go to your project's admin web interface and create an application named worker.

Set up app version directory and files

We'll create an app version for Windows/x64. Create the directory structure:

cd ~/projects/test
mkdir -p apps/worker/1.0/windows_x86_64__vbox_64
cd apps/worker/1.0/windows_x86_64__vbox_64

Copy various files into that directory:

cp ~/boinc/samples/vboxwrapper/boinc_resolve_1 .
cp ~/boinc/samples/worker/worker_2_x86_64-pc-linux-gnu .
cp ~/boinc/samples/worker/run_worker_1.sh .
cp ~/boinc/samples/worker/vbox_job_worker_1.xml .
cp ~/boinc/samples/worker/vm_version.xml version.xml
cp ~/boinc/worker_2_x86_64-pc-linux-gnu .
ln -s ~/boinc/vm_image_x64_4.vdi .
ln -s ~/boinc/vboxwrapper_26207_windows_x86_64.exe .

We'll explain these files below. Note that:

We add a version number to the names of files that will be sent to the client. If we change the files, we must increment the version number.
We create a symbolic links to the VM image (which is 1.9 GB) and to the vboxwrapper executable. This saves disk space if we use them in other app versions.
The '__vbox_64' in the directory name means that the app version has plan class 'vbox_64'. This tells the BOINC server to send it only to client hosts that have VirtualBox installed.

Create the app version

cd ~/projects/test
bin/update_versions

This will display some warnings. Enter y and y.

Create in/out templates

On the BOINC server:

cd ~/projects/test/templates
cp ~/boinc/samples/worker/worker_in .
cp ~/boinc/samples/worker/worker_out .

Add handlers for completed jobs

Add the following to ~/projects/test/config.xml, within the <daemons> section:

    <daemon>
      <cmd>sample_trivial_validator --app worker </cmd>
      <output>validator_worker.out</output>
      <pid_file>validator_worker.pid</pid_file>
    </daemon>
    <daemon>
      <cmd>sample_assimilator --app worker</cmd>
      <output>assimilator_worker.out</output>
      <pid_file>assimilator_worker.pid</pid_file>
    </daemon>

Then restart the project:

cd ~/projects/test
bin/stop
bin/start

Process a job

In the server VM, do

cd ~/projects/test
cat > infile
Some Text
^D

bin/submit_job worker infile

This will print a job name.

In the Windows system, go to the BOINC client, select the test project, and click Update. The client should download a task, run it, and upload the result; this will take about a minute.

Then go back to the BOINC server and do

bin/query_job (jobname)

It should say that the job is completed and print its output:

SOME TEXT

Other platforms

The above creates an app version for Windows/x64. You can create versions for Mac OS and Linux in exactly the same way, but using the Mac and Linux versions of vboxwrapper.

How it works

BOINC's framework for VM apps is a bit complex. You may need to understand some of the details in order to handle more complex apps, or to debug apps.

Recall that the BOINC directory structure on the client looks like this:

The project directory contains the app version files (VM image, executables, XML files) and the job input files.

The slot directory contains "link files" that look like

<soft_link>../../projects/project.url.edu/phys_filename</soft_link>

The process structure on the client looks like this:

To run a VM job, the client runs a program called 'vboxwrapper', which in turn runs a program called 'VBoxManage' (part of the Virtualbox installation), giving it instructions to create, start, and stop a VM using the VM image you've provided.

vm_image_x64_4.vdi

This is a stripped-down Linux image, with no development tools (gcc etc.) or GUI software (X11 etc.). It has the Virtualbox guest extensions installed so that it can access 'shared directories' passed in from the host system.

The VM is set up to do the following when it boots:

Mount the 'shared' directory at /root/shared.
Run /root/shared/boinc_app.
Shut down.

vboxwrapper_26207_windows_x86_64.exe

The vboxwrapper program, compiled for Windows/x64.

worker_2_x86_64-pc-linux-gnu

The 'worker' application, compiled for Linux/x64.

boinc_resolve

This script runs in the VM. It takes a link file and converts it to the path (in the VM) of the corresponding physical file, assuming that the project directory is mounted at root/project. In the example above it would return /root/project/phys_filename.

It uses the Unix 'sed' program to convert the string, and the 'tr' program to remove the trailing newline:

#! /bin/sh

sed 's/<soft_link>..\/..\/projects\///; s/[^\/]*\//\/root\/project\//; s/<\/soft_link>//' $1 | tr -d '\r\n'

run_worker.sh

This is the top-level script run (with the logical name boinc_app) in the VM after it boots. The slot directory has already been mounted at root/shared; this is the current directory. It also contains boinc_resolve.

The script mounts the project directory at /root/project. It uses boinc_resolve to convert the executable filename and the input and output filenames from links to paths. Then it runs the executable, passing the input and output filenames as command-line arguments.

#! /bin/sh

mkdir /root/project
mount -t vboxsf project /root/project
execpath=`./boinc_resolve worker`
inpath=`./boinc_resolve in`
outpath=`./boinc_resolve out`

$execpath $inpath $outpath

vbox_job.xml

This contains instructions for Vboxwrapper:

<vbox_job>
    <memory_size_mb>512</memory_size_mb>
    <os_name>Linux26_64</os_name>
    <share_slot_dir/>
    <share_project_dir/>
    <multiattach_vdi_file>vm_image_x64_4.vdi</multiattach_vdi_file>
</vbox_job>

memory_size_mb: the amount of virtual memory to allocate to the VM. 512 MB is the recommended minimum for Linux. Set it to this plus the max working set size of your application.
os_name: describes the OS in the VM image.
share_slot_dir: share the slot directory as 'shared'.
share_project_dir: share the project directory as 'project'.
multiattach_vdi_file: allow the VM image to be used by multiple simultaneous jobs. This eliminates the need to make a copy of the image for each job.

vm_version.xml

<version>
    <file>
        <physical_name>vboxwrapper_26207_windows_x86_64.exe</physical_name>
        <main_program/>
    </file>
    <file>
        <physical_name>worker_2_x86_64-pc-linux-gnu</physical_name>
        <logical_name>worker</logical_name>
    </file>
    <file>
        <physical_name>run_worker_1.sh</physical_name>
        <logical_name>boinc_app</logical_name>
        <copy_file/>
    </file>
    <file>
        <physical_name>boinc_resolve_1</physical_name>
        <logical_name>boinc_resolve</logical_name>
        <copy_file/>
    </file>
    <file>
        <physical_name>vbox_job_1.xml</physical_name>
        <logical_name>vbox_job.xml</logical_name>
    </file>
    <file>
        <physical_name>vm_image_x64_4.vdi</physical_name>
        <logical_name>vm_image.vdi</logical_name>
    </file>
    <dont_throttle/>
    <is_wrapper/>
</version>

The config file describing the app version.

Notes:

main_program indicates that vboxwrapper is the main program; it's what the BOINC client runs.
Most of the files have physical names with version numbers, and logical names without version numbers.
Two of the files - boinc_resolve_1 and run_work_1.sh - have the copy_file attribute. This means the file itself - not a link - is copied to the slot directory. This is necessary because when run_work_1.sh is run, the project directory is not mounted, so we can't access files via links.
is_wrapper tells the BOINC client to run vboxwrapper at normal process priority.
dont_throttle tells the BOINC client not to apply CPU throttling to the vboxwrapper process (VirtualBox has its own mechanism for CPU throttling).

Input/output templates

worker_in:

<input_template>
    <file_info>
    </file_info>
    <workunit>
        <file_ref>
            <open_name>in</open_name>
        </file_ref>
        <rsc_disk_bound>4e9</rsc_disk_bound>
        <rsc_memory_bound>4e9</rsc_memory_bound>
    </workunit>
</input_template>

The logical name of the input file is in. The disk space used by the job (i.e. the contents of the slot directory) is limited ot 4 GB, as is the memory working-set size of the job.

worker_out:

<output_template>
    <file_info>
        <name><OUTFILE_0/></name>
        <generated_locally/>
        <upload_when_present/>
        <max_nbytes>5000000</max_nbytes>
        <url><UPLOAD_URL/></url>
    </file_info>
    <result>
        <file_ref>
            <file_name><OUTFILE_0/></file_name>
            <open_name>out</open_name>
        </file_ref>
    </result>
</output_template>

The logical name of the output file is out, and its size is limited to 5 MB.

Debugging VM apps

VM apps have a lot of moving parts, and jobs can fail in various ways:

The VM fails to start.
The app in the VM crashes.
The app can't access its input files.
The output files don't end up in the right place.

Here are some techniques for troubleshooting VM apps.

Testing in isolation

You can create a simulated slot directory, populate it with appropriate files, and run the VM manually. This lets you test the app without a BOINC client or server, and it lets you fiddle around inside the VM.

Create the simulated slot directory, and populate it with files. In the case of the above example, these files are:

The main script; name it boinc_app.
The filename resolution script; name it boinc_resolve.
The vbox_job.xml from the above example.
The vbowrapper executable.
The VM image; name it vm_image.vdi.
A file in containing

<soft_link>../../projects/test/infile</soft_link>

A file out containing

<soft_link>../../projects/test/outfile</soft_link>

A file worker containing

<soft_link>../../projects/test/worker</soft_link>

A subdirectory project.
Within project, a file infile containing text (this is the job's input file)
Within project, the worker executable; name it worker.

In the slot directory, run vboxwrapper (e.g. in Windows, double-click on it).

If the VM fails to start, look in stderr.txt, which is the stderr output of vboxwrapper.

If the VM starts up successfully, a VM console window will appear. There will be a 5 second pause. In the console window, type ^C a few times. You should get a shell prompt in the VM. You can then manually run your app:

mount -t vboxsf shared /root/shared
cd /root/shared
./boinc_app

and see what happens. If things work correctly, an output file named outfile should appear in the project directory.

Notes:

You'll need to copy vm_image.vdi to the directory before each test; VirtualBox deletes it when the VM shuts down.
The mouse cursor may be stuck inside the VM console; press the right Ctrl key to release it.

Testing under BOINC

Once your app works in isolation, you can test it under BOINC. In your BOINC project, create the app and app version as described above.

The BOINC client normally deletes the slot directory after each job. When you run a VM job in the BOINC client, it can be useful to see the contents of the slot and project directories immediately before the job starts, and immediately after the job ends. There are two ways to do this:

Run the BOINC client with the ---exit_before_start and/or exit_after_finish command-line options. On Windows, you can do this by making a shortcut to the client.
Build the BOINC client from source, run it under a debugger (e.g. Visual Studio on Windows) and place breakpoints at

app_start.cpp:ACTIVE_TASK::start()

just before the #ifdef _WIN32 around line 700, and/or

app_control.cpp:ACTIVE_TASK::handle_exited_app()

When the client breaks or exits, you can inspect the slot and project directories.

Home