From 3eb85774bfeb2f4e44129307c5023c541c22b244 Mon Sep 17 00:00:00 2001 From: David Anderson Date: Mon, 19 Aug 2002 18:43:10 +0000 Subject: [PATCH] cleaned up docs svn path=/trunk/boinc/; revision=351 --- client/client_state.C | 2 +- doc/app.html | 26 ++--- doc/back_end.html | 41 +++----- doc/batch.html | 21 +--- doc/client_data.html | 24 ++--- doc/client_debug.html | 125 ++++++++---------------- doc/client_files.html | 38 +++----- doc/client_fsm.html | 73 +++++--------- doc/client_logic.html | 12 +-- doc/client_network.html | 12 +-- doc/comp_examples.html | 12 +-- doc/database.html | 92 +++++------------- doc/dev.html | 12 +-- doc/file_access.html | 28 ++---- doc/file_xfer_policy.html | 12 +-- doc/files.html | 102 +++++++------------- doc/flow.html | 42 +++----- doc/host_measure.html | 86 ++++++----------- doc/index.html | 3 +- doc/master_url.html | 38 +++----- doc/prefs.html | 112 +++++++--------------- doc/prefs_mod.html | 12 +-- doc/project.html | 71 +++++--------- doc/project_startup.html | 47 ++------- doc/protocol.html | 27 ++---- doc/result.html | 154 ++++++++++-------------------- doc/rpc_policy.html | 113 +++++++++------------- doc/sched_debug.html | 15 +-- doc/sched_impl.html | 12 +-- doc/sched_policy.html | 31 +++--- doc/security.html | 195 +++++++++++++++----------------------- doc/test.html | 78 ++++++--------- doc/tools_other.html | 151 ++++++++++++----------------- doc/tools_security.html | 50 ++++------ doc/tools_work.html | 87 ++++++----------- doc/validation.html | 59 +++++------- doc/version.html | 21 ++-- doc/work.html | 81 +++++----------- 38 files changed, 702 insertions(+), 1415 deletions(-) diff --git a/client/client_state.C b/client/client_state.C index 42b029c1a9..6c56fe9ff7 100644 --- a/client/client_state.C +++ b/client/client_state.C @@ -385,7 +385,7 @@ int CLIENT_STATE::exit_tasks() { // int CLIENT_STATE::write_state_file() { unsigned int i, j; - FILE* f = fopen(STATE_FILE_TEMP, "wb"); + FILE* f = fopen(STATE_FILE_TEMP, "w"); int retval; if (log_flags.state_debug) { diff --git a/doc/app.html b/doc/app.html index 13f69fc821..a1f17a0eff 100644 --- a/doc/app.html +++ b/doc/app.html @@ -1,21 +1,14 @@ - - - - Platforms, Applications, and Versions - - - +Platforms, Applications, and Versions

Platforms, Applications, and Versions

- A platform is a compilation target (Windows/Intel, +A platform is a compilation target (Windows/Intel, Linux/Intel, MacOS/PPC, etc.). Platforms are maintained in the platform table in the BOINC DB, and can be created using the add utility program.

- An application is a computational task; in concrete terms it +An application is a computational task; in concrete terms it consists of an executable program (perhaps in versions for various platforms) and a set of input files to be processed by the program. A project can operate many applications. @@ -24,15 +17,15 @@ BOINC DB, and can be created using the add utility.

- An application program may go through a sequence of versions. +An application program may go through a sequence of +versions. A particular version, compiled for a particular platform, is called an application version. An application version can consist of multiple files: for example, a controller script, pre- and post-processing programs, and a primary.

- Each application version has an integer version number. +Each application version has an integer version number. Projects can assign version numbers however they like; for example, version 304 might represent major version 3 and minor version 4. Version numbers should be used consistently across platforms; @@ -40,13 +33,13 @@ Windows version 304 should be computationally identical to Mac version 304.

- Each application has a minimum version. +Each application has a minimum version. When a client is sent work for an application, it is also sent the latest application version for its platform. It is sent work only if this version is the minimum or greater.

- Application version are maintained in the app_version table +Application version are maintained in the app_version table in the BOINC DB, Each entry includes an XML document describing the files that make up the application version: 

@@ -67,6 +60,3 @@ files that make up the application version:
 </app_version>
The add utility program provides a simplified interface for creating application versions. - - - diff --git a/doc/back_end.html b/doc/back_end.html index f243d6a057..2dc68230c7 100644 --- a/doc/back_end.html +++ b/doc/back_end.html @@ -1,46 +1,29 @@ - - - - Back End Examples - - - +Back End Examples

Back End Examples

- Here are some examples of the "back ends" that could be used by +Here are some examples of the "back ends" that could be used by various types of project. -

- SETI@home with local data servers -

+SETI@home with local data servers

-

SETI@home with remote data servers

- As above, but the splitter program writes data files to DLT tape. +As above, but the splitter program writes data files to DLT tape. The result records are generated, using the URL of a selected data -server. The tape is mailed to the data server. An operator at the data +server. +The tape is mailed to the data server. +An operator at the data server transfers the files to disk, than uses a web control panel to mark the batch as ready, which marks the result records as ready. -

- - diff --git a/doc/batch.html b/doc/batch.html index 00fc82d825..64c1bc12d1 100644 --- a/doc/batch.html +++ b/doc/batch.html @@ -1,21 +1,10 @@ - - - - Batches - - - +Batches

Batches

- Workunits and results can be grouped together into batches. -Each batch is represented by an integer. Results must belong to the same -batch as their workunit. -

+Workunits and results can be grouped together into batches. +Each batch is represented by an integer. +Results must belong to the same batch as their workunit.

- BOINC provides tools for manipulating workunits and results by +BOINC provides tools for manipulating workunits and results by batch: e.g., for changing the status of all results in a batch from "inactive" to "unsent". -

- - diff --git a/doc/client_data.html b/doc/client_data.html index 4d1a575ab0..a60bf7c787 100644 --- a/doc/client_data.html +++ b/doc/client_data.html @@ -1,18 +1,9 @@ - - - - Core Client: Data Structures - - - +Core Client: Data Structures

Core Client: Data Structures

- The key data structures in the core client are as follows. -

+The key data structures in the core client are as follows.

- Basic types: -

+Basic types: 
 PROJECT
 APP
@@ -21,16 +12,13 @@ APP_VERSION
 IO_FILE_DESC
 WORKUNIT
 RESULT
-
ACCOUNTS
Contains a vector of PROJECTs, summarizing + +
ACCOUNTS
Contains a vector of PROJECTs, summarizing the accounts file.

-

CLIENT_STATE

- Contains vectors of all the basic types. These are crossed-linked +Contains vectors of all the basic types. These are crossed-linked with pointers; for example, APP_VERSION has a pointer to FILE_INFO. Also contains FSM sets for network connections, HTTP operations, file transfers, and active tasks. -

- - diff --git a/doc/client_debug.html b/doc/client_debug.html index ffe5f3eea7..b5f5e33986 100644 --- a/doc/client_debug.html +++ b/doc/client_debug.html @@ -1,102 +1,55 @@ - - - - Core Client: Debugging - - - -

Core Client: Debugging

Core client logging and error -output

+Core Client: Debugging +

Core Client: Debugging

Core client logging and error output

- The core client writes error messages to stderr. This mechanism is -reserved for serious problems, i.e. those that reflect bugs in the core +The core client writes error messages to stderr. +This mechanism is reserved for serious problems, +i.e. those that reflect bugs in the core client program or conditions that require user intervention. Specifically, The client verifies that arguments to functions have valid -values, and will print a message to stderr on an invalid argument. These -error messages are formatted as error: CLASS_NAME.function: -error_statement argument. Common errors of this type include -unexpected NULL pointer arguments and negative lengths. Error numbers +values, and will print a message to stderr on an invalid argument. +These error messages are formatted as error: CLASS_NAME.function: +error_statement argument. +Common errors of this type include +unexpected NULL pointer arguments and negative lengths. +Error numbers are defined in lib/error_numbers.h. -

- In addition, the core client can write a variety of "logging" -information to stdout. The logging options are read from a file -log_flags.xml. This file has the following format: -

+In addition, the core client can write a variety of "logging" +information to stdout. +The logging options are read from a file +log_flags.xml. +This file has the following format: 
 <log_flags>
     [ flags ]
 </log_flags>
-
The flags are as follows: + +The flags are as follows:
-
- <task/> -
-
- Log the start, restart and completion of computational tasks. -
-
- <file_xfer/> -
-
- Log the start, restart and completion of file transfers. -
-
- <sched_ops/> -
-
- Log connections with scheduling servers. -
-
- <state_debug/> -
-
- Log changes to the "client state" data structures. -
-
- <task_debug/> -
-
- Log debugging information about task execution. -
-
- <file_xfer_debug/> -
-
- Log debugging information about file transfers. -
-
- <sched_op_debug/> -
-
- Log the request and reply messages of exchanges with scheduling +
<task/> +
Log the start, restart and completion of computational tasks. +
<file_xfer/> +
Log the start, restart and completion of file transfers. +
<sched_ops/> +
Log connections with scheduling servers. +
<state_debug/> +
Log changes to the "client state" data structures. +
<task_debug/> +
Log debugging information about task execution. +
<file_xfer_debug/> +
Log debugging information about file transfers. +
<sched_op_debug/> +
Log the request and reply messages of exchanges with scheduling servers. -
-
- <http_debug/> -
-
- Log debugging information about HTTP operations. -
-
- <time_debug/> -
-
- Log the passage of time. -
-
- <net_xfer_debug/> -
-
- Log debugging information about network communication. -
+
<http_debug/> +
Log debugging information about HTTP operations. +
<time_debug/> +
Log the passage of time. +
<net_xfer_debug/> +
Log debugging information about network communication.

Application error output

- The stderr of applications is automatically directed to a file. The +The stderr of applications is automatically directed to a file. The contents of this file are returned to the scheduling server and stored in the database. -

- - diff --git a/doc/client_files.html b/doc/client_files.html index 9f680ff863..2156b6a04f 100644 --- a/doc/client_files.html +++ b/doc/client_files.html @@ -1,41 +1,29 @@ - - - - Core Client: File Structure - - - +Core Client: File Structure

Core Client: File Structure

- The core client runs in a BOINC home directory. An account file -(accounts.xml) must be present in this directory. The core client +The core client runs in a BOINC home directory. +An account file (accounts.xml) must be present in this directory. +The core client creates and uses the following files and directories within the BOINC home directory: -

- Each result executes in particular slot. The core client creates +Each result executes in particular slot. The core client creates symbolic links (UNIX) or .lnk files (Windows) in the slot directory, linking the input, output and executable files to the appropriate files in the project directory. On UNIX, the core client uses chroot() so that the application sees the slot directory as the root of its filesystem, and cannot create or access files outside of it. -

- - diff --git a/doc/client_fsm.html b/doc/client_fsm.html index 7d32f33e09..ae2a3199c0 100644 --- a/doc/client_fsm.html +++ b/doc/client_fsm.html @@ -1,72 +1,45 @@ - - - - Core Client: Finite-State Machine (FSM) Structure - - - +Core Client: Finite-State Machine (FSM) Structure

Core Client: Finite-State Machine (FSM) Structure

- The core client can perform many activities (file transfers, -computations, RPCs to scheduling servers) in parallel. To manage this -parallelism, the core client is structures as a number of -finite-state machines (FSM). For example, an HTTP transaction is +The core client can perform many activities (file transfers, +computations, RPCs to scheduling servers) in parallel. +To manage this parallelism, the core client is structures as a number of +finite-state machines (FSM). +For example, an HTTP transaction is represented by an FSM whose states might include:

- FSMs of a particular type are managed by an FSM container. +FSMs of a particular type are managed by an FSM container. Each FSM container manages a set of FSMs, and provides a poll() function for detecting and performing state transitions. These functions are nonblocking. -

- The core client uses the following FSM types: -

+The core client uses the following FSM types:

- An FSM may be implemented using other FSMs; for example, FILE_XFER +An FSM may be implemented using other FSMs; for example, FILE_XFER is implemented using HTTP_OP, which in turn is implemented using NET_XFER. -

- - diff --git a/doc/client_logic.html b/doc/client_logic.html index 5315f35816..af18024607 100644 --- a/doc/client_logic.html +++ b/doc/client_logic.html @@ -1,13 +1,3 @@ - - - - Core Client: Logic - - - +Core Client: Logic

Core Client: Logic

-

- - diff --git a/doc/client_network.html b/doc/client_network.html index 500a3e7f4b..aebe7a75a5 100644 --- a/doc/client_network.html +++ b/doc/client_network.html @@ -1,13 +1,3 @@ - - - - Core Client: Network Scheduling and Retry Policy - - - +Core Client: Network Scheduling and Retry Policy

Core Client: Network Scheduling and Retry Policy

-

- - diff --git a/doc/comp_examples.html b/doc/comp_examples.html index 4e932c5d08..433a7135ae 100644 --- a/doc/comp_examples.html +++ b/doc/comp_examples.html @@ -1,13 +1,3 @@ - - - - Compute Model: Examples - - - +Compute Model: Examples

Compute Model: Examples

-

- - diff --git a/doc/database.html b/doc/database.html index d52a5efbd8..fd77aee7c4 100644 --- a/doc/database.html +++ b/doc/database.html @@ -1,82 +1,40 @@ - - - - The BOINC Database - - - +The BOINC Database

The BOINC Database

- BOINC uses a relational database to keep track of participants, work -and result. The reference implementation uses MySQL, but any SQL -database can be used. -

+BOINC uses a relational database to keep track of participants, work +and result. +The reference implementation uses MySQL, but any SQL database can be used.

- In boinc/html_user/ there is a file called db_name containing only the -name of the database for that project. Each BOINC project will have a different database -name. This allows for access to different databases through the webpages of projects. +In boinc/html_user/ there is a file called db_name containing only the +name of the database for that project. +Each BOINC project will have a different database name. +This allows for access to different databases through the webpages of projects.

- The database tables are as follows: -

+The database tables are as follows:
-
- platform -
-
- Compilation targets of the core client and/or applications. -
-
- app -
-
- Applications. The core client is treated as an application; its +
platform +
Compilation targets of the core client and/or applications. +
app +
Applications. The core client is treated as an application; its name is "core_client". -
-
- app_version -
-
- Versions of applications. Each record includes a URL for +
app_version +
Versions of applications. Each record includes a URL for downloading the executable, and the MD5 checksum of the executable. -
-
- user -
-
- Describes users, including their email address, name, web +
user +
Describes users, including their email address, name, web password, and authenticator. -
-
- preferences -
-
- Describes preferences. The actual preference information is +
preferences +
Describes preferences. The actual preference information is stored in an XML document in a "blob" field of this table. -
-
- host -
-
- Describes hosts. -
-
- workunit -
-
- Describes workunits. The input file descriptions are stored in +
host +
Describes hosts. +
workunit +
Describes workunits. The input file descriptions are stored in an XML document in a blob field. Includes counts of the number of results linked to this workunit, and the numbers that have been sent, that have succeeded, and that have failed. -
-
- result -
-
- Describes results. Includes a "state" (whether the result has +
result +
Describes results. Includes a "state" (whether the result has been dispatched). Stores a number of items relevant only after the result has been returned: CPU time, exit status, and validation status. -
- - diff --git a/doc/dev.html b/doc/dev.html index e5f8a8a3e1..b4f001818e 100644 --- a/doc/dev.html +++ b/doc/dev.html @@ -1,13 +1,3 @@ - - - - Application Development - - - +Application Development

Application Development

-

- - diff --git a/doc/file_access.html b/doc/file_access.html index c643afa2e2..25a793217d 100644 --- a/doc/file_access.html +++ b/doc/file_access.html @@ -1,26 +1,18 @@ - - - - Compute Model: Remote File Access - - - +Compute Model: Remote File Access

Compute Model: Remote File Access

- Output files can be designated as sticky but not -upload-when-present. Sometimes this is done because the file will be +Output files can be designated as sticky but not +upload-when-present. +Sometimes this is done because the file will be used as an input for a later workunit; it may also be done to spread large output data across participant disks, in cases where the data may be too large to store on the project back-end. -

- To access such a file, the back end inserts a file access -request in the database. This specifies a file and a data-server -URL. When a host storing the file contacts the scheduling server (which +To access such a file, the back end inserts a file access +request in the database. +This specifies a file and a data-server URL. +When a host storing the file contacts the scheduling server (which may not happen for a while) the scheduling server forwards the file -access request. The host confirms the file transfer when it is +access request. +The host confirms the file transfer when it is completed; the back-end must poll for this event. -

- - diff --git a/doc/file_xfer_policy.html b/doc/file_xfer_policy.html index 500a3e7f4b..aebe7a75a5 100644 --- a/doc/file_xfer_policy.html +++ b/doc/file_xfer_policy.html @@ -1,13 +1,3 @@ - - - - Core Client: Network Scheduling and Retry Policy - - - +Core Client: Network Scheduling and Retry Policy

Core Client: Network Scheduling and Retry Policy

-

- - diff --git a/doc/files.html b/doc/files.html index e2ae80380c..8f189c786f 100644 --- a/doc/files.html +++ b/doc/files.html @@ -1,19 +1,12 @@ - - - - Compute Model: Storage - - - -

Compute Model: Storage

Files

+Compute Model: Storage +

Compute Model: Storage

+

Files

- The BOINC storage model is based on files. The inputs and +The BOINC storage model is based on files. +The inputs and outputs of applications, and the applications executables, are files. -

- A file is described by an XML element of the form -

+A file is described by an XML element of the form 
 <file_info>
     <name>foobar</name>
@@ -27,56 +20,39 @@ outputs of applications, and the applications executables, are files.
     [ <upload_when_present/> ]
     [ <sticky/> ]
 </file_info>
-
The components are as follows: + +The components are as follows: These attributes allow the specification of various types of files: for example, input or output files that are retained for use as input to later computations.

- Once a file is created (on a data server or a participant host) it +Once a file is created (on a data server or a participant host) it is immutable. -

File references

- Files may be associated with workunits, results and application -versions. Each such association is represented by an XML element of -the form -

+Files may be associated with workunits, +results and +application versions. +Each such association is represented by an XML element of the form 
 <file_ref>
     <file_name>foobar</file_name>
@@ -84,28 +60,20 @@ the form
     [ <fd>2</fd> ]
     [ <main_program/> ]
 </file_ref>
-
The components are as follows: + +The components are as follows: - - diff --git a/doc/flow.html b/doc/flow.html index 9a463f9658..646aea289d 100644 --- a/doc/flow.html +++ b/doc/flow.html @@ -1,41 +1,29 @@ - - - - Compute Model: Work Distribution - - - +Compute Model: Work Distribution

Compute Model: Work Distribution

- Results are not necessarily dispatched one at a time. Instead, each -participant host maintains an estimate of the amount of work remaining -(i.e. the time until one of its processors will be idle). Each host also +Results are not necessarily dispatched one at a time. +Instead, each participant host +maintains an estimate of the amount of work remaining +(i.e. the time until one of its processors will be idle). +Each host also has two scheduling parameters, the high-water mark and the low-water mark (these are part of the host's "preferences", discussed later). -

- Normally the work remaining is between the high- and low-water -marks. When the work remaining reaches the low-water mark, the core +Normally the work remaining is between the high- and low-water marks. +When the work remaining reaches the low-water mark, the core client contacts one or more scheduling servers, and attempts to get -enough work to exceed the high-water mark. The scheduling server sends a -maximum of four weeks of work. If a work unit is not feasible for a host +enough work to exceed the high-water mark. +The scheduling server sends a maximum of four weeks of work. +If a work unit is not feasible for a host because it consumes too many system resources, the scheduling server will not send that work unit. -

- The amount of work (measured in FP/int ops) sent in a single -scheduling RPC can be limited by the project, on a per-application -basis. -

+The amount of work (measured in FP/int ops) sent in a single +scheduling RPC can be limited by the project, on a per-application basis.

- This scheme allows hosts that are sporadically connected (because +This scheme allows hosts that are sporadically connected (because they're portable or have modem-based connections) to avoid becoming idle due to lack of work. -

- To do: per-workunit flag saying to return the result ASAP? -

- - +To do: per-workunit flag saying to return the result ASAP? diff --git a/doc/host_measure.html b/doc/host_measure.html index 29e332213e..3f83e174b9 100644 --- a/doc/host_measure.html +++ b/doc/host_measure.html @@ -1,86 +1,58 @@ - - - - Measuring and Distinguishing Hosts - - - +Measuring and Distinguishing Hosts

Measuring and Distinguishing Hosts

- The core client measures the following aspects of each host: -

+The core client measures the following aspects of each host: + These quantities are reported in every scheduling RPC, and their latest -values are stored in the BOINC database.

Distinguishing hosts

+values are stored in the BOINC database. + +

Distinguishing hosts

- Not all systems have an application-readable globally unique ID, so -we don't take this approach. When a host first contacts the scheduling +Not all systems have an application-readable globally unique ID, so +we don't take this approach. +When a host first contacts the scheduling server it is assigned a host ID, which it stores in its client_state.xml file. -

+

- The BOINC database maintains the RPC sequence number for each host, +The BOINC database maintains the RPC sequence number for each host, and the host maintains the sequence number in its client_state.xml file. -

+

- To install the BOINC client on a large number of computers, users +To install the BOINC client on a large number of computers, users are instructed to copy only the executable and the prefs.xml -file. If the user copies the client_state.xml file as well, the +file. +If the user copies the client_state.xml file as well, the first RPCs from the other hosts will have a repeat sequence number; in this case the scheduling server creates a new host record and returns its ID. -

- - diff --git a/doc/index.html b/doc/index.html index 2b85f2e99b..a9c6dbf325 100644 --- a/doc/index.html +++ b/doc/index.html @@ -2,7 +2,8 @@

Berkeley Open Infrastructure for Network Computing (BOINC)

-Berkeley Open Infrastructure for Network Computing (BOINC) is a software platform for public-participation distributed computing projects like +Berkeley Open Infrastructure for Network Computing (BOINC) +is a software platform for distributed computing projects like distributed.net and SETI@home.