In some cases of file staging (both remote and via stage_file)
we'd do the following:
1) create the .md5 file (in check_download_file())
2) move or copy the file into the download dir
This can result in the file having a later mod time than the .md5 file,
which causes process_input_template() to reject the .md5 file.
Solution: touch the .md5 file after the move or copy
Originally, the idea for remote file management was that
the physical name of a file was its md5, or jf_md5.
This was changed to let the submitter choose the name.
But in the DB (job_file.md5) we only kept 64 chars.
Change the field name to job_file.name, and change it to varchar(255)
Files in the download dir can have accompanying ".md5" files
containing their MD5 and size.
This eliminates the need to calculate these when creating a job using the file.
The .md5 files were being created by stage_file (local staging)
but not by remote file management.
In fact, the latter wasn't checking for file immutability violations.
I changed remote file management to add this check,
and to create the .md5 file.
The latter is done in a new function shared with stage_file.
- The RPC handler was expecting the "BOINC names" to be the file MD5.
This is no longer true; the BOINC name can be anything as long as it's unique.
- To reflect this, use <phys_name> instead of <md5> in request messages.
This means that you'll need to update both RPC client and server software.
- BoincJobFile::insert() needed to return the insert ID
The upload_files function did not do a good job on error reporting. It now can handle and report several more problems.
- Loop over the uploaded files not the supplied md5 values so we can detect a discrepancy between the two
- Check upload error value provided by PHP (catches filesize limit problems, permission problems and partial uploads)
- Calculate md5 of uploaded file and check with supplied values (The size of uploaded files is typical small enough so this is not a problem)
- Report any error condition in the response and delete the uploaded files (especially if they are in the PHP temp directory)
- Added script to debug upload problems
The upload_files function did not do a good job on error reporting. It now can handle and report several more problems.
* Loop over the uploaded files not the supplied md5 values so we can detect a discrepancy between the two
* Check upload error value provided by PHP (catches filesize limit problems, permission problems and partial uploads)
* Calculate md5 of uploaded file and check with supplied values (The size of uploaded files is typical small enough so this is not a problem)
* Report any error condition in the response and delete the uploaded files (especially if they are in the PHP temp directory)
The way errors were being handled in submit_rpc_handler.php,
the replies could end up looking like:
<error>
...
</error>
<actual_reply>
...
</actual_reply>
which doesn't parse with PHP's simplexml because there's no outer element.
So the PHP interface to job submission didn't work in some cases.
To fix this, I changed the RPC replies to have an enclosing element,
which is the name of the RPC, e.g.
<submit_batch>
<error>
...
</error>
... other stuff
</submit_batch>
RPC replies should now always be valid XML, errors or not.
- If handling an XML RPC, use set_error_handler() to output PHP warnings as XML.
Otherwise they appear as strings in the XML reply, making them not parse.
- suppress warnings from PHP function calls where we're already checking errors
David Anderson