Files and data servers

The BOINC storage model is based on files. Examples of files:

The inputs and/or outputs of computation;
Components of application: executables, libraries, etc.
Data for its own sake, e.g. to implement a distributed storage system.

The BOINC core client transfers files to and from project-operated data servers using HTTP.

Once a file is created (on a data server or a participant host) it is immutable. This means that all replicas of that file are assumed (and required) to be identical.

File properties

Files have various properties, including:

Sticky: don't delete file on client (see below).
Report on RPC: include a description of this file in scheduler requests.
Maximum size: if an output file exceeds its maximum size, the computation is aborted.

File properties are specified in XML. For example, the properties of input files are described in the workunit template, and XML document describing the task. The XML format is described here.

File management

BOINC's default behavior is to delete files when they aren't needed any more. Specifically:

On the client, input files are deleted when no workunit refers to them, and output files are deleted when no result refers to them. Application-version files are deleted when they are referenced only from superceded application versions.
On the client, the 'sticky' flag overrides the above mechanisms and suppresses the deletion of the file. The file may deleted by an explicit server request. The file may also be deleted at any time by the core client in order to honor limits on disk-space usage.
On the server, the file deleter daemon deletes input and output files that are no longer needed. This can be suppressed using the 'no_delete' flag, or using command-line options to the file deleter.

Compression of input files

Starting with version 5.4, the BOINC client is able to handle HTTP Content-Encoding types 'deflate' (zlib algorithm) and 'gzip' (gzip algorithm). The client decompresses these files 'on the fly' and stores them on disk in uncompressed form.

You can use this in two ways:

Use the Apache 2.0 mod_deflate module to automatically compress files on the fly. This method will work with all BOINC clients, but it will do compression only for 5.4+ clients.
Compress files and give them a filename suffix such as '.gz'. The name used in your <file_info> elements, however, is the original filename without '.gz'.
Include the following line in httpd.conf:
```
AddEncoding x-gzip .gz
```
This will add the content encoding to the header so that the client will decompress the file automatically. This method has the advantage of reducing server disk usage and server CPU load, but it will only work with 5.4+ clients. Use the 'min_core_version' field of the app_version table to enforce this. You can use this in conjunction because the mod_deflate module allows you to exempt certain filetypes from on-the-fly compression.

Both methods store files uncompressed on the client. If you need compression on the client, you must do it at the application level. The BOINC source distribution includes a version of the zip library designed for use by BOINC applications on any platform.

Compression of output files

If you include the <gzip_when_done> tag in an output file description, the file will be gzip-compressed after it has been generated. "; page_tail(); ?>