In some cases of file staging (both remote and via stage_file)
we'd do the following:
1) create the .md5 file (in check_download_file())
2) move or copy the file into the download dir
This can result in the file having a later mod time than the .md5 file,
which causes process_input_template() to reject the .md5 file.
Solution: touch the .md5 file after the move or copy
If no open_name is specified in a file_ref element of the input template but copy_file is set for this file use the physical filename as logical filename.
This is intentional and only an issue if multiple work generators are creating input files that have the same name but different content.
Ignores CID 27922, 27993 found by Coverity
A "remote input file" is located on a data server other than the project server.
Previously these could be specified only in the input template,
which was of limited utility.
Add new ways of specifying remote input files:
1) in the create_work program, a remote input file can be specified
with command-line args
--remote_file name URL nbytes MD5
or by the same syntax in stdin when creating multiple jobs
2) add a variant of create_work() called create_work2(),
which takes a vector of INFILE_DESC structures that can specify
either local or remote files
Previously if you wanted to create lots of jobs from a script (e.g. PHP)
you had to run create_work once per job.
With the --stdin option you run it once,
passing it a file (view stdin) with one line per job.
Each line can specify a command line and/or a set of input files.
On my server this gives a performance of about 1000 jobs per minute,
which is less than I would have expected,
but all the time is spent in doing MySQL inserts
so that's as good as we can do for now.
Also fix a bug in stage_file.
The docs said that putting <gzip/> for a file in your input template
would cause it to be transferred in gzip form.
But most of the server-side implementation was missing.
- in process_input_template(), parse <gzip/>,
and add <gzipped_url> elements to the output.
- stage_file was generating MD5 cache files containing only the MD5,
but process_input_template() expected them to contain file size as well.
Change stage_file to write both,
and change process_input_template() to write an error message
if it finds a bad MD5 file.
- remove stuff from process_input_template() related to
"generated_locally", a feature that doesn't exist anymore.
has an invalid URL, type, or app
- server, create_work() function: if a <file_info> in input template
lists URLs, they're directories; append filename to each one
- Fix various #include issues.
CODING STYLE LAW (minimal inclusion principle):
If foo.cpp requires <blah.h>,
#include <blah.h> in foo.cpp, NOT foo.h
svn path=/trunk/boinc/; revision=25837
- prompt user for DB user/passwd if DB updates needed
- error out immediately if any DB update fails
Fixes#1169, #1170
- create_work: check for duplicate <file_info>s in input template
NOTE: the input template syntax is pretty clunky
svn path=/trunk/boinc/; revision=24969
David Anderson