Originally, the idea for remote file management was that
the physical name of a file was its md5, or jf_md5.
This was changed to let the submitter choose the name.
But in the DB (job_file.md5) we only kept 64 chars.
Change the field name to job_file.name, and change it to varchar(255)
We need to let users have https:// URLs.
Originally http:// was assumed, and the URLs were stored without the prefix.
For compatibility, allow either form to exist in the DB
Also update the "other account info" form to user bootstrap.
On IE11, pages using recaptcha (like profile edit)
weren't showing the recaptcha.
On reloading, it would appear.
Mysterious.
Anyway, removing the
'<meta http-equiv="X-UA-Compatible" content="IE=EmulateIE8" >
fixed the problem.
Files in the download dir can have accompanying ".md5" files
containing their MD5 and size.
This eliminates the need to calculate these when creating a job using the file.
The .md5 files were being created by stage_file (local staging)
but not by remote file management.
In fact, the latter wasn't checking for file immutability violations.
I changed remote file management to add this check,
and to create the .md5 file.
The latter is done in a new function shared with stage_file.
- The RPC handler was expecting the "BOINC names" to be the file MD5.
This is no longer true; the BOINC name can be anything as long as it's unique.
- To reflect this, use <phys_name> instead of <md5> in request messages.
This means that you'll need to update both RPC client and server software.
- BoincJobFile::insert() needed to return the insert ID
This supports the TACC use case,
in the jobs in a batch can use different Docker images
and different input and output file signatures,
none of which are known in advance.
Python API binding:
- A JOB_DESC object can optionally contain wu_template and result_template
elements, which are the templates (the actual XML) to use for that job.
Add these to the XML request message if present.
- Added the same capability to the PHP binding, but not C++.
- Added and debugged test cases for both languages.
Also, submit_batch() can take either a batch name (in which case
the batch is created) or a batch ID
(in which the batch was created prior to remotely staging files).
RPC handler:
- in submit_batch(), check for jobs with templates specified
and store them in files.
For input templates (which are deleted after creating jobs)
we put them in /tmp,
and use a map so that if two templates are the same we use 1 file.
For output templates (which have to last until all jobs are done)
we put them in templates/tmp, with content-based filenames
to economize.
- When creating jobs, or generating SQL strings for multiple jobs,
use these names as --wu_template_filename
and --result_template_filename args to create_work
(either cmdline args or stdin args)
- Delete WU templates when done
create_work.cpp:
handle per-job --wu_template and --result_template args in stdin job lines
(the names of per-job WU and result templates).
Maintain a map mapping WU template name to contents,
to avoid repeatedly reading them.
For jobs that don't specify templates, use the ones specified
at the batch level, or the defaults.
David Anderson