Compute Model: Storage

Files

The BOINC storage model is based on files. The inputs and outputs of applications, and the applications executables, are files.

A file is described by an XML element of the form

<file_info>
    <name>foobar</name>
    <url>http://a.b.c/foobar</url>
    <url>http://x.y.z/foobar</url>
    ...
    <md5_cksum>123123123123</md5_cksum>
    <nbytes>134423</nbytes>
    <max_nbytes>200000</max_nbytes>
    [ <executable/> ]
    [ <upload_when_present/> ]
    [ <sticky/> ]
</file_info>

The components are as follows:

The <name> element gives the file's name, which must be unique within the project.
Each <url> element gives a URL where the file is (or will be) located on a data server.
The <md5_cksum> element is the MD5 checksum of the file, providing protection against corruption and spoofing.
The <nbytes> element is the size of the file in bytes (may be greater than 2^32).
The <max_nbytes> element is the maximum allowable size of the file in bytes (may be greater than 2^32). This is used to prevent flooding data servers with bogus data.
The <sticky> element, if present, indicates that the file should be retained on the client after its initial use.
The <upload_when_present> element, if present, indicates that the file should be uploaded after it is created.
The <executable> element, if present, indicates that the file protections should be set to allow execution.

These attributes allow the specification of various types of files: for example, input or output files that are retained for use as input to later computations.

Once a file is created (on a data server or a participant host) it is immutable.

File references

Files may be associated with workunits, results and application versions. Each such association is represented by an XML element of the form

<file_ref>
    <file_name>foobar</file_name>
    [ <open_name>input</open_name> ]
    [ <fd>2</fd> ]
    [ <main_program/> ]
</file_ref>

The components are as follows:

The <file_name> element specifies a file.
The <open_name> element, if present, is the name by which the application will refer to the file (in a call to fopen(), for example). Note that this file will usually be a BOINC soft link rather than the actual file.
The <fd> element, if present, indicates the file descriptor number by which the application will access the file. This file descriptor will only be set up if the application calls the API function boinc_init.
The <main_program> element is used for files associated with application versions. It indicates that this file is the application's main program.