Managing the Alpha test system

We need to test the client software on a wide range of platforms, OS versions, and hardware and software configurations. To do this we use volunteers called "Alpha testers", who run test versions of the client software and report results through a web interface. The web site, https://boinc.berkeley.edu/alpha/, also lets you view test results.

The file boinc_alpha/test.inc (in isaac) lets you configure testing. At the top there is a list of platform (HW/OS version combos) with a "target number" of test results for each. More popular platforms have higher targets; adjust these numbers based on the number of results you're getting.

There follows a list of tests

$versions is a list of versions to test. To add a new version, put it at the top of the list. Following the version number is the set of tests to do, and the set of platforms to cover.

When you start a testing cycle, use all tests and all platforms. As bugs get fixed and you do sub-release testing, you can limit the tests and platforms as needed.

Some tests (like transfer_restart) need to be done only when libcurl is updated, and are disabled by default. Enable them as needed.

Communicating with Alpha testers

Communicate with Alpha testers via the boinc_alpha email list: https://groups.google.com/a/ssl.berkeley.edu/g/boinc_alpha

Send emails when:

a test cycle or subcycle starts or ends. Tell them exactly what's new and different about each test release. When a cycle is done, thank them for their effort.
more results are needed for a particular platform.
Major bugs are identified or fixed.

Some alpha testers aren't signed up for boinc_alpha. You may want to send mass emails instead, using the scripts

reminder.php (use during the testing process)
test_mailer.php (more general)

Test coverage

The Alpha web site shows you the coverage, overall and per-platform. You may need to prod testers to get platform coverage. A version is considered ready to release when there are no critical bugs and overall coverage is > 95%.

Recruiting Alpha testers

If you're not getting enough test results, recruit testers on the BOINC or SETI@home message boards.

Testers register by sending email to an admin (currently David A.). The admin creates an account on the Alpha project on isaac, and sends them the account password and link to testing instructions.

Home