2002-08-23 19:22:17 +00:00
|
|
|
<title>Persistent File Transfer</title>
|
|
|
|
|
<body bgcolor=ffffff>
|
|
|
|
|
<h2>Persistent File Transfer</h2>
|
|
|
|
|
<p>
|
|
|
|
|
Since BOINC may deal with uploading or downloading very large
|
|
|
|
|
files, or servers which are not always available, the notion
|
|
|
|
|
of a persistent file transfer is necessary.
|
2002-09-03 21:27:21 +00:00
|
|
|
<p>
|
|
|
|
|
The logic to handle resumption of downloads or uploads is
|
|
|
|
|
embedded in the HTTP and file transfer layers. The HTTP
|
|
|
|
|
1.1 byte range option is used to support download
|
|
|
|
|
resumption. Therefore, data servers must support HTTP
|
|
|
|
|
1.1 in order for download resumption to work.
|
|
|
|
|
The upload resumption feature is handled in the
|
|
|
|
|
file_upload_handler program.
|
|
|
|
|
<p>
|
|
|
|
|
A persistent file transfer is described by an XML element
|
|
|
|
|
of the form
|
|
|
|
|
<pre>
|
|
|
|
|
<persistent_file_xfer>
|
|
|
|
|
<num_retries>2</num_retries>
|
|
|
|
|
<first_request_time>1030665600</first_request_time>
|
|
|
|
|
<next_request_time>1030665725</next_request_time>
|
|
|
|
|
</persistent_file_xfer>
|
|
|
|
|
</pre>
|
|
|
|
|
<ul>
|
|
|
|
|
<li> The <b>num_retries</b> element is the number of times the client
|
|
|
|
|
has attempted to download/upload the file.
|
|
|
|
|
<li> The <b>first_request_time</b> element is the first time (in UNIX
|
|
|
|
|
time) that the client attempted to download/upload the file.
|
|
|
|
|
<li> The <b>next_request_time</b> element is the earliest time
|
|
|
|
|
(in UNIX time) that the client will retry the download/upload.
|
|
|
|
|
</ul>
|
|
|
|
|
<p>
|
|
|
|
|
When there is a failure to download or upload a file, the client
|
|
|
|
|
will increment num_retries by 1 and calculate a new next_request_time
|
|
|
|
|
based on randomized exponential backoff. The equation for this is:
|
|
|
|
|
<pre>next_request_time = current_time+max(MIN_DELAY,min(MAX_DELAY,exp(rand(0,1)*num_retries)))</pre>
|
|
|
|
|
Where MIN_DELAY is 1 minute and MAX_DELAY is 4 hours.
|
|
|
|
|
<p>
|
|
|
|
|
The client will consider the download/upload a complete failure if the
|
|
|
|
|
current time becomes much later than this (default is two weeks).
|
|
|
|
|
In this event, the file will be deleted and the failure reported
|
|
|
|
|
to the scheduling server.
|
