Software Prerequisites. Note that a web server with PHP is required for running a real server, but that requires a lot of path and permissions configuration so we opt not to use them in testing.

Python testing framework

The test/ directory contains a library of Python modules that make end-to-end testing of BOINC easy. See the Python framework.

Quick start

Single test:

cd boinc/test && ./test_uc.py

Full test suite:

cd boinc/test && make check

After two or three minutes you should see 'Passed Test'. During the test you should see various progress status indicators (resize your window so that the status indicator line fits on one line).

Goals of the testing framework

The goal of the framework is to support automated testing of BOINC and BOINC applications. Each test is performed by a Python program that initializes the system to a deterministic state, executes the system, detects termination, and returns success or failure depending on the final state.

Many BOINC features involve multiple projects and/or multiple hosts. It would be prohibitively complex to do automated testing on multiple physical hosts. Fortunately the BOINC architecture makes it possible for multiple projects and (virtual) hosts to reside on a single physical host. So the testing framework uses this approach.

The framework lets you test systems with any of the following attributes:

Multiple projects
Multiple applications per project
Multiple participants per project
Multiple hosts, with arbitrary enrollment in projects

The following system attributes are planned but not implemented yet:

Multiple data servers or scheduling servers per project.
Simulated failures of servers and network connections.
Simulations of various host conditions (out of disk space, crash/reboot, etc.).

TODO: Directory structure

TODO: Modules/ Classes

TODO: Example script

Test applications

The apps directory contains the following test applications:

upper_case: converts a file to upper case.
concat: concatenates its input files.
1sec: uses 1 second of CPU time.

Test cases

The test/ directory contains various python scripts named with prefix test_. Each of these is a test case which runs an end-to-end test of BOINC (creates directory structure for project(s) and host(s), starts a web server, server daemons, host client(s), and checks output is as expected).

`test_uc.py`	The basic test using `uppercase`
`test_concat.py`	tests command-line arguments and filenames
`test_uc_slow.py`	tests checkpoint/restart mechanism
`test_prefs.py`	tests some aspects of preferences.
`test_water.py`	tests some aspects of water marks.
`test_rsc.py`	tests that scheduling server only sends feasible work units.
`test_pers.py`	tests the persistent file transfers for download and upload. It interrupts them in the middle and makes sure that the filesize never decreases along interrupted transfers.
`test_masterurl_failure.py`	tests the exponential backoff mechanism on the client in case of master URL failures. This test is not automated. It has to be run, and then client.out (in the host directory) must be looked at to examine whether everything is working correctly.
`test_sched_failure.py`	tests the exponential backoff mechanism on the client in case of scheduling server failures. This test is not automated. It has to be run, and then client.out (in the host directory) must be looked at to examine whether everything is working correctly.

Appendix: Optional Environment Variables

The following environment variables are optional and apply whatever web server you use:

BOINC_TEST_USER_NAME

User name to use for setting up database name. Defaults to \$USER

BOINC_TEST_VERBOSE

Verbosity level.

0 print nothing

1 [default] print some. If output is a tty, overwrite lines.
2 print all

BOINC_TEST_DELETE

Specifies whether or not to delete testbed after the test finishes. Only relevant when BOINC_TEST_AUTO_SETUP=0. Possible values (case doesn't matter):

No

If-Successful [default]

Always

BOINC_TEST_INSTALL_METHOD

Specifies how to install html/php, CGI from source directories to testbed location.

link hardlink

symlink [default] symbolic link

copy copy

Copying is useful because it preserves what version of the file was used in a particular test run and hardlinking is best because compiled cgi and scheduler programs are not disrupted by parallel builds.

Appendix: Web Server

By default, the test script will use a custom web server that has no security and minimal cgi/php capability. You can also use Apache or some other web server with manually initialized directories by setting these environment variables:

BOINC_TEST_AUTO_SETUP=0   [default=1]
BOINC_TEST_KEY_DIR
BOINC_TEST_PROJECTS_DIR
BOINC_TEST_CGI_URL
BOINC_TEST_HTML_URL
BOINC_TEST_CGI_DIR
BOINC_TEST_HTML_DIR
BOINC_TEST_HOSTS_DIR

Example setup

Bourne shell:

    QHOME=/disks/philmor/a/users/quarl/proj
    TOP=\$QHOME/test-boinc
    URL=http://milhouse.ssl.berkeley.edu/quarl

    export BOINC_TEST_PROJECTS_DIR=\$TOP/projects
    export BOINC_TEST_USER_NAME=quarl
    export BOINC_TEST_SRC_DIR=\$QHOME/boinc
    export BOINC_TEST_CGI_DIR=\$TOP/boinc_cgi
    export BOINC_TEST_CGI_URL=\$URL/boinc_cgi
    export BOINC_TEST_HTML_DIR=\$TOP/boinc_html
    export BOINC_TEST_HTML_URL=\$URL/boinc_html
    export BOINC_TEST_SHMEM_KEY=0x1717f00f
    export BOINC_TEST_KEY_DIR=\$TOP/keys
    export BOINC_TEST_HOSTS_DIR=\$TOP/host

Apache configuration:

    <Directory /disks/philmor/a/users/quarl/proj/test-boinc>
            AllowOverride FileInfo AuthConfig Limit
            Options Indexes SymLinksIfOwnerMatch IncludesNoExec ExecCGI
        <Limit GET POST OPTIONS PROPFIND>
            Order allow,deny
            Allow from all
        </Limit>
    </Directory>

    ScriptAlias /quarl/boinc_cgi/ \"/disks/philmor/a/users/quarl/proj/test-boinc/boinc_cgi/\"
    Alias /quarl/ \"/disks/philmor/a/users/quarl/proj/test-boinc/\"

"; page_tail(); ?>

`0`	print nothing
`1` [default]	print some. If output is a tty, overwrite lines.
`2`	print all