(It turns out that the compression schemes supported by
Apache and libcurl, suprisingly, aren't restartable.)
if a <file_info> from the server contains <gzipped_url> tags,
use those instead of the <url> tags,
and flag the file as "download_gzipped".
If this is the case, download NAME.gz and save it as NAME.gzt.
When the download is complete, rename NAME.gzt to NAME.gz,
and uncompress it to NAME.
(this ensures that if NAME.gz is present, it's complete).
Also do the uncompression, if needed, in verify_file().
This ensures that the uncompression will eventually get done
even if the client quits are crashes in the middle.
- update_versions: if <gzip> is present in a <file_info>,
add a gzipped copy in the download directory
and add a <gzipped_url> elements to the app version's xml_doc.
svn path=/trunk/boinc/; revision=25112
Report it (along with disk usage) in scheduler request messages.
This will allow the scheduler to send file-delete commands
if the project is using more than its share.
- client: add <disk_usage_debug> log flag
- create_work: add --help, show --command_line option
svn path=/trunk/boinc/; revision=24968
If set, don't run jobs for that app while network is suspended.
- client: parse this flag and maintain in state file;
do a job reschedule when network suspend state changes
- GUI RPC: add RESULT::network_wait flag;
if set, this job is waiting for network access to be allowed
- Manager: display the above in task info
- add support for "web graphics URL" (see above)
- client: parse message containing URL on graphics_reply channel
and store in ACTIVE_TASK::web_graphics_url
- GUI RPC: add RESULT::web_graphics_url
- Manager: if web graphics URL is present, Show Graphics opens a browser
- remove some vestigial code for pre-V6 graphics
svn path=/trunk/boinc/; revision=24899
clear project-level upload and download backoffs,
as well as RPC and individual xfer backoffs.
This was an oversight.
svn path=/trunk/boinc/; revision=24621
If we're contacting a project to report results,
only piggyback work requests for resources for which
that project is the highest priority that may have work.
- client: compute result.not_started more efficiently
TODO: continue efficiency work. There's still some quadratic stuff
svn path=/trunk/boinc/; revision=24523
reduce its runtime from O(N^2) to O(N),
where N is the number of runnable jobs
(which can be in the thousands).
This will make the client emulator run a lot faster,
and will reduce the client CPU overhead a bit.
- API: change boinc_get_opencl_ids() so that it returns
a BOINC error code (< -100) if the app_init.xml is
missing or bad (i.e. we're running standalone),
and an OpenCL error code (> -100) if an OpenCL call failed.
svn path=/trunk/boinc/; revision=24469
so that they do what they're supposed to
(i.e. enforce resource shares)
- client: change log flag <debt_debug> to <priority_debug>
- client simulator: update REC even with large delta-t.
- client simulator: handle "no new work" apps correctly
svn path=/trunk/boinc/; revision=24429
for when the job completed successfully but
one or more output files had permanent upload failures.
Show this state in web interfaces.
- sample_work_generator: check return value of count_unsent_results(),
so that we don't generate infinite work if there's a DB problem
- web: RSS feed shows news items from last 90 days, rather than 14
svn path=/trunk/boinc/; revision=24377
- client: if an app version can't be used because the GPUs it needs
are all excluded, mark it and all its results as "coproc missing"
so that they won't be looked at in scheduling logic.
svn path=/trunk/boinc/; revision=24317
where work fetch didn't work right in the presence of
multiple GPUs and <exclude_gpu> config options.
For example: suppose:
- you have 2 GPUs and 2 projects
- Project A is excluded from GPU 1
- you have lots of jobs for project A
Then the client won't try to fetch jobs from project B.
The problem had 2 parts:
a) round-robin simulation wasn't taking GPU exclusions into account.
In the above example, it would think that both GPUs had jobs.
I fixed this by computing the # of GPUs from each project
is excluded, and using this in the RR simulation.
b) Once this was done, I needed to make the client
request GPU jobs from project B rather than project A.
I did this with following policy:
If a project has excluded GPUs of a given type,
and has a runnable job of that type,
don't ask it for more work of that type.
Notes:
- the policy in b) is crude, and it means that work-buffer
preferences are ignored in some cases.
- neither a) nor b) takes into account app-level exclusions.
I could fix both of these with a lot of work,
but I'd rather move to a model in which dissimilar GPUs
are modeled as different resources,
which would remove the need for the <exclude_gpu> mechanism
in the first place.
- web: remove extraneous ) at end of button tooltips
svn path=/trunk/boinc/; revision=24312
by simulating time-slicing explicitly.
Also simulate changes in project REC
and hence in scheduling priority.
- client: add a log flag "rrsim_detail" that prints
time-slice-level info.
svn path=/trunk/boinc/; revision=24161
If present, "file_prefix/" is prepended to the logical names
of input and output files of jobs using that app version.
I.e. for Vbox wrapper based app versions, file_prefix is "share",
so that I/O files are put in a "share" subdirectory of the slot dir.
- update_versions: add support for
<dont_throttle>
<file_prefix>x</file_prefix>
in version.xml
svn path=/trunk/boinc/; revision=23924
- client: cc_config.xml: if <devnum> is omitted from a <exclude_gpu>,
it means exclude all instances of that GPU type
- client: if all instances of a GPU type are excluded for a project,
don't ask the project for jobs of that type
svn path=/trunk/boinc/; revision=23898
as described here: http://boinc.berkeley.edu/trac/wiki/ClientDataModel
Compatibility: if your project is using upload signatures:
- set ignore_upload_certificates
- disable job creation
- let your job queue drain
- upgrade to new server software
- clear ignore_upload_certificates
- enable job creation
svn path=/trunk/boinc/; revision=23863
as described here: http://boinc.berkeley.edu/trac/wiki/ClientDataModel
Compatibility:
clients that upgrade to this version should see nothing unusual.
Clients that downgrade from this version to a previous version
should see all projects reset
(i.e. tasks disappear and then get re-downloaded).
- manager: always show whether a file transfer is upload or download
- client: don't scale work requests by resource share
svn path=/trunk/boinc/; revision=23862
If you put an element of the form
<exclude_gpu>
<url>http://project_url.com/</url>
<device_num>1</device_num>
</exclude_gpu>
in your cc_config.xml, that GPU won't be used for that project
svn path=/trunk/boinc/; revision=23774
- client: allow "non_cpu_intensive" to be specified independently
for different apps in a project.
This is intended to support projects that use the
Attic file distribution system,
which needs to have a daemon running.
svn path=/trunk/boinc/; revision=23610
individual jobs rather than globally.
To use this, projects must add <report_immediately/>
to the <result> elements in job templates
svn path=/trunk/boinc/; revision=23515
- All sticky files are reported on each scheduler RPC
- If a scheduler reply says to delete a file, clear its sticky flag
In particular:
- remove the "send file list" tag in scheduler RPC replies
- remove FILE_INFO::marked_for_delete
- remove FILE_INFO::report_on_rpc
- remove the request_file_list program
svn path=/trunk/boinc/; revision=23431
- new GPU types can be added easily
- users can specify GPUs in cc_config.xml,
referred to by app_info.xml,
and they will be scheduled by BOINC
and passed --device N options
Note: the parsing of cc_config.xml is not done yet.
- RPC protocols (account manager and scheduler)
can now specify GPU types in separate elements
rather than embedding them in tag names
e.g. <no_rsc>NVIDIA</no_rsc> rather than <no_cuda/>
- client: in account manager replies, parse elements of the form
<no_rsc>NAME</no_rsc>
indicating the GPUs of type NAME should not be used.
This allows account managers to control GPU types
not hardwired into the client.
Note: <no_cuda/> and <no_ati/> will continue to be supported.
- scheduler RPC reply: add
<no_rsc_apps>NAME</no_rsc_apps>
(NAME = GPU name)
to indicate that the project has no jobs for the indicated GPU type.
<no_cuda_apps> etc. are still supported
- client/lib: remove set_debts() GUI RPC
- client/scheduler RPC
remove <cuda_backoff> etc. (superceded by no_app)
Exception: <ip_result> elements in sched request
still have <ncudas> and <natis>.
Fix this later.
Implementation notes:
- client/lib: change "CUDA" to "NVIDIA" in type/variable names, and in XML
Continue to recognize "CUDA" for compatibility
- host_info.coprocs no longer used within the client;
use a global var (COPROCS coprocs) instead.
COPROCS now has an array of COPROCs;
GPUs types are identified by the array index.
Index zero means CPU.
- a bunch of other resource-specific structs (like RSC_WORK_FETCH)
are now stored in arrays, with same indices as COPROCS
(i.e. index 0 is CPU)
- COPROCS still has COPROC_NVIDIA and COPROC_ATI structs to hold vendor-specific info
- APP_VERSION now has a struct GPU_USAGE to describe its GPU usage
svn path=/trunk/boinc/; revision=23253
The problem arises when there are jobs of projects
with widely differing resource shares,
and results in an overestimation of saturated time.
Old: at the start of simulation, call WORK_FETCH::compute_shares()
to get resources of runnable projects.
Use these throughout the simulation.
Problem: suppose you have 2 runnable projects;
P1 has large RS, P2 has small RS.
P1's jobs finish quickly.
P2's jobs then are running alone,
but their FLOPS is scaled (incorrectly) by P2's small RS.
Solution: recompute relative CPU resource share within the
simulation loop,
and compute it over the projects that have actives jobs
in the simulation.
svn path=/trunk/boinc/; revision=23162
(either at startup or during execution)
reset a number of "wait until X" variables;
otherwise we might wait years to contact a project, restart a file xfer, etc.
Notes:
- there is no problem setting clocks forward; things just happen prematurely
- some variables (e.g. task deadlines) are not reset,
because it's not clear what to set them to
- sched: remove ati_opencl plan class until we understand what it is
svn path=/trunk/boinc/; revision=22842
and an upload started in the last 5 min, don't fetch work from it.
The goal is to merge the 2 scheduler RPCs
(fetch work, report completed taskS) into a single RPC.
Note: this may result in idleness in some cases.
- scheduler: if client doesn't handle plan class (pre-5.10),
check plan-class app versions anyway,
but only use if it's a single-CPU app.
This allows single-CPU app versions with specific requirements
(like SSE) to be issued to old clients.
From Bernd Machenschalk
svn path=/trunk/boinc/; revision=22841
Old: enforce_schedule() won't run an active job if its
working set size exceeds remaining available RAM.
Problem: there may be a lot of similar jobs.
The client starts one, finds that its working set is too large,
starts the second, and so on.
Solution: if J is an unstarted job,
and there are started jobs using the same app version,
consider J's working set size to be the largest of
the working sets of those jobs.
- client: fix an apparent bug that could oversaturate
the CPUs with single-thread jobsk
svn path=/trunk/boinc/; revision=22840
recent estimated credit (REC) instead of debt.
These changes are enabled by
#define USE_REC
in work_fetch.h.
If this is commented out (the default) the client uses
debt-based scheduling, same as before.
TODO: work-fetch policy changes
- client simulator: various fixes:
- compute idle and wasted fraction based on all processing resources,
not just CPU
- compute job completion times based on FLOPS, not CPU seconds
- compute and use project->no_X_apps
etc.
svn path=/trunk/boinc/; revision=22741
Additions to request message:
<not_started_dur>X</not_started_dur>
<in_progress_dur>X</in_progress_dur>
The estimated remaining duration of unstarted
and in-progress tasks
Additions to reply message, within <project>, optional:
<suspend>0|1</suspend>
suspend or resume project (overrides local state)
<abort_not_started>0|1</abort_not_started>
if set, abort unstarted jobs
svn path=/trunk/boinc/; revision=22698
if project P is anonymous platform
don't request work for resource R from P
if there is no app version using R in P/app_info.xml
else
don't request work for resource R from P
if P tells us it has no app versions using R
svn path=/trunk/boinc/; revision=22675
- scheduler: improve the deadline check mechanism slightly.
When updating "estimated delay" (a rough measure of how long
a resource is saturated with high-priority work)
take into account the # of instances used by the job,
and the # of total instances
svn path=/trunk/boinc/; revision=22612
They can be determined implicitly by WUs/results,
or explicitly in the <app> record.
If you do neither, the app is ignored.
svn path=/trunk/boinc/; revision=22591
as the major criterion in choosing non-EDF GPU jobs.
GPU scheduling now respects resource share,
and as a result STD should no longer diverge.
- client simulator: various improvements, most notably
that we now generate gnuplot graphs of all debt types
NOTE: the client problem was found and fixed using the simulator!
svn path=/trunk/boinc/; revision=22536
David Anderson