The job submission RPC handler (PHP) originally ran the
create_work program once per job.
This took about 1.5 minutes to create 1000 jobs.
Recently I changed this so that create_work only is run once;
it does one SQL insert per job.
Disappointingly, this was only slightly faster: 1 min per 1000 jobs.
This commit changes create_work to create multiple jobs per SQL insert
(as many as will fit in a 1 MB query, which is the default limit).
This speeds things up by a factor of 100: 1000 jobs in 0.5 sec.
VBOX: Only change the elapsed time is the stop watch elapsed time is a positive value.
This is just weird (http://lhcathome2.cern.ch/test4theory/result.php?resultid=4289098):
2014-04-08 15:52:43 (6732): Status Report: Job Duration: '86400.000000'
2014-04-08 15:52:43 (6732): Status Report: Elapsed Time: '54506.443735'
2014-04-08 16:06:16 (6732): Status Report: Job Duration: '86400.000000'
2014-04-08 16:06:16 (6732): Status Report: Elapsed Time: '450358026.129327'
Previously if you wanted to create lots of jobs from a script (e.g. PHP)
you had to run create_work once per job.
With the --stdin option you run it once,
passing it a file (view stdin) with one line per job.
Each line can specify a command line and/or a set of input files.
On my server this gives a performance of about 1000 jobs per minute,
which is less than I would have expected,
but all the time is spent in doing MySQL inserts
so that's as good as we can do for now.
Also fix a bug in stage_file.
It doesn't make sense to show the project banner in admin pages;
the banner typically has (broken) links to non-admin pages.
Also, include basic stylesheets in admin pages.
The latest client reports the peak working set size, swap size,
and disk usage for completed jobs.
Add fields to the results table to store these.
Parse them in scheduler request messages, and write to the DB.
Display them in the result web page.
This data can be used to improve (or even automate)
the job estimates for memory and disk usage.
Also fixed a bug where, if a job was aborted while not running,
its final CPU and elapsed time weren't copied from ACTIVE_TASK to RESULT,
hence not sent to scheduler
On client startup, decide whether we need to do CPU benchmarks
(cmdline option was set, or we haven't done them for 30 days).
If so, do them when possible.
David Anderson