Commit Graph

206 Commits

Author SHA1 Message Date
David Anderson c28a4ef659 - make "needs network" a property of APP_VERSION rather than APP
svn path=/trunk/boinc/; revision=24900
2011-12-26 04:56:36 +00:00
David Anderson b003b8e290 - add support for APP::needs_network flag.
If set, don't run jobs for that app while network is suspended.
		- client: parse this flag and maintain in state file;
			do a job reschedule when network suspend state changes
		- GUI RPC: add RESULT::network_wait flag;
			if set, this job is waiting for network access to be allowed
		- Manager: display the above in task info
	- add support for "web graphics URL" (see above)
		- client: parse message containing URL on graphics_reply channel
			and store in ACTIVE_TASK::web_graphics_url
		- GUI RPC: add RESULT::web_graphics_url
		- Manager: if web graphics URL is present, Show Graphics opens a browser
	- remove some vestigial code for pre-V6 graphics

svn path=/trunk/boinc/; revision=24899
2011-12-26 03:30:32 +00:00
David Anderson 69834e0c01 - client: compile fix; remove redundant total_peak_flops()
svn path=/trunk/boinc/; revision=24738
2011-12-06 09:20:30 +00:00
David Anderson bc35060726 - client: when contacting a project for reasons other than
work fetch (e.g. to report completed jobs)
    only request work if it's the project we would have chosen
    if we were fetching work.
- client: the way in which project priorities were adjusted
    in work fetch to reflected currently queued work was wrong.
- client: fix bug in the way project priorities are adjusted
    in RR simulator
- client emulator: if there are results in the state file
    with states DOWNLOADING or UPLOADING,
    change them to DOWNLOADED or UPLOADED.
    Otherwise they're stuck.


svn path=/trunk/boinc/; revision=24737
2011-12-06 04:21:27 +00:00
David Anderson 07e54fc86b - client: fix work fetch bug.
If we're contacting a project to report results,
    only piggyback work requests for resources for which
    that project is the highest priority that may have work.
- client: compute result.not_started more efficiently

TODO: continue efficiency work.  There's still some quadratic stuff


svn path=/trunk/boinc/; revision=24523
2011-11-04 08:15:04 +00:00
David Anderson 7b28215032 - client: reimplement the round-robin simulator to
reduce its runtime from O(N^2) to O(N),
    where N is the number of runnable jobs
    (which can be in the thousands).
    This will make the client emulator run a lot faster,
    and will reduce the client CPU overhead a bit.
- API: change boinc_get_opencl_ids() so that it returns
    a BOINC error code (< -100) if the app_init.xml is
    missing or bad (i.e. we're running standalone),
    and an OpenCL error code (> -100) if an OpenCL call failed.


svn path=/trunk/boinc/; revision=24469
2011-10-24 17:53:09 +00:00
David Anderson b95ac02c5b - client: change the way project priorities are computed,
so that they do what they're supposed to
    (i.e. enforce resource shares)
- client: change log flag <debt_debug> to <priority_debug>
- client simulator: update REC even with large delta-t.
- client simulator: handle "no new work" apps correctly


svn path=/trunk/boinc/; revision=24429
2011-10-19 06:37:03 +00:00
David Anderson 54311606e3 - client: associate a PROJECT with HTTP_OP where applicable,
so that if you use <http_debug> and filter by project
    you don't see other projects' HTTP stuff
- client simulator: cc_config.xml is part of the scenario;
    log flags are part of the simulation


svn path=/trunk/boinc/; revision=24410
2011-10-18 04:23:03 +00:00
David Anderson 1d38837788 - client: call xp.skip_unexpected() if get unexpected tag,
to avoid showing multiple error messages
- client simulator: bug fixes and tweaks


svn path=/trunk/boinc/; revision=24408
2011-10-17 20:46:06 +00:00
David Anderson 0a89805a70 - client: bug fix for the above
svn path=/trunk/boinc/; revision=24404
2011-10-16 01:29:57 +00:00
David Anderson f8e7662e1f - client: improvements to job scheduling and work fetch policies.
- Job scheduling: the baseline policy is to schedule based on "project priority",
            which is how much processing P should receive based on resource share
            minus how much it actually has received recently.
            This policy tends to run jobs from the same project together,
            so we modified it by adding a priority adjustment as jobs are scheduled.
            The idea is that if 2 projects have about the same priority
            they should split the processors.

            The problem: the adjustment was too large on hosts that are on
            only a small fraction of the time,
            thus tending to run 1 job from each project, regardless of priority.

            Solution: make an adjustment that reflects the host's actual throughput.
            See adjust_rec_sched() for details.

        - Work fetch: similar situation.
            We were making an adjustment based on how much work the project currently has queued,
            but the adjustment drowned out the project priority,
            so we'd tend to always get work from the project that has least work queued.
            Solution: make a smaller adjustment (-.3 ... .3)

    - client: in message announcing app start, show the plan class

    - client: don't show "unrecognized XML" messages for account files.
        It's typically project-specific prefs that the client doesn't know about.

svn path=/trunk/boinc/; revision=24403
2011-10-15 20:28:26 +00:00
David Anderson 57c04979b3 - client: don't do scheduler-requested RPCs if
- project is set to No New Work, and
    - project has no jobs on the client


svn path=/trunk/boinc/; revision=24348
2011-10-07 21:20:42 +00:00
David Anderson 5c0d5d371e - client: compute project scheduling priority more efficiently
- client: if an app version can't be used because the GPUs it needs
    are all excluded, mark it and all its results as "coproc missing"
    so that they won't be looked at in scheduling logic.


svn path=/trunk/boinc/; revision=24317
2011-10-03 06:18:58 +00:00
David Anderson b7f1aa0226 - client: fix a bug reported by Jacob Klein,
where work fetch didn't work right in the presence of
    multiple GPUs and <exclude_gpu> config options.
    For example: suppose:
        - you have 2 GPUs and 2 projects
        - Project A is excluded from GPU 1
        - you have lots of jobs for project A
    Then the client won't try to fetch jobs from project B.

    The problem had 2 parts:
    a) round-robin simulation wasn't taking GPU exclusions into account.
        In the above example, it would think that both GPUs had jobs.
        I fixed this by computing the # of GPUs from each project
        is excluded, and using this in the RR simulation.
    b) Once this was done, I needed to make the client
        request GPU jobs from project B rather than project A.
        I did this with following policy:
        If a project has excluded GPUs of a given type,
        and has a runnable job of that type,
        don't ask it for more work of that type.

    Notes:
    - the policy in b) is crude, and it means that work-buffer
        preferences are ignored in some cases.
    - neither a) nor b) takes into account app-level exclusions.

    I could fix both of these with a lot of work,
    but I'd rather move to a model in which dissimilar GPUs
    are modeled as different resources,
    which would remove the need for the <exclude_gpu> mechanism
    in the first place.

- web: remove extraneous ) at end of button tooltips


svn path=/trunk/boinc/; revision=24312
2011-10-01 16:23:28 +00:00
David Anderson 7f2a3c0ce1 - client: get GPU available RAM at startup (only)
- client: fix compile warning


svn path=/trunk/boinc/; revision=24188
2011-09-13 22:58:39 +00:00
David Anderson 9856f795ed - client: remove code related to debt-based scheduling
svn path=/trunk/boinc/; revision=24163
2011-09-12 17:57:31 +00:00
David Anderson f81cb82b8e - client: make RR simulation more accurate
by simulating time-slicing explicitly.
    Also simulate changes in project REC
    and hence in scheduling priority.
- client: add a log flag "rrsim_detail" that prints
    time-slice-level info.


svn path=/trunk/boinc/; revision=24161
2011-09-12 17:01:54 +00:00
David Anderson e033c50df6 - client: projects with zero resource share are always lower priority,
for both job sched and work fetch,
    than projects with positive resource share.

svn path=/trunk/boinc/; revision=23946
2011-08-08 00:49:16 +00:00
David Anderson 3f4c207636 - client: fix bug in <exclude_gpu> config option
- client: extend <exclude_gpu> option so that if <device_num> is omitted,
		all GPUs of the given type are excluded.

svn path=/trunk/boinc/; revision=23902
2011-07-29 17:54:49 +00:00
David Anderson 5b159c6735 - remote job submission: bug fix and tweaks
- client: cc_config.xml: if <devnum> is omitted from a <exclude_gpu>,
    it means exclude all instances of that GPU type
- client: if all instances of a GPU type are excluded for a project,
    don't ask the project for jobs of that type


svn path=/trunk/boinc/; revision=23898
2011-07-29 00:07:20 +00:00
David Anderson 8296968cbb - client: changes to the client file model to support distributed storage,
as described here: http://boinc.berkeley.edu/trac/wiki/ClientDataModel
        Compatibility:
            clients that upgrade to this version should see nothing unusual.
            Clients that downgrade from this version to a previous version
            should see all projects reset
            (i.e. tasks disappear and then get re-downloaded).
    - manager: always show whether a file transfer is upload or download
    - client: don't scale work requests by resource share

svn path=/trunk/boinc/; revision=23862
2011-07-20 19:12:10 +00:00
David Anderson eb9d387825 - client: if a project has zero resource share,
don't piggyback a work request onto a non-work-request RPC


svn path=/trunk/boinc/; revision=23844
2011-07-14 16:30:53 +00:00
David Anderson 8ca24cbbab - client, work fetch policy:
adjust project REC by the amount of work queued, to increase variety
    NOTE: at some point I think I had a reason to not do this,
    but I can't remember what it is.
- client, job scheduling policy: fix how project REC is adjusted


svn path=/trunk/boinc/; revision=23838
2011-07-13 19:46:03 +00:00
David Anderson 13cfc73b3c - client: fix bug related to deselecting resource types in project prefs.
Some logic was missing.

svn path=/trunk/boinc/; revision=23825
2011-07-08 19:22:22 +00:00
David Anderson 94e8c48220 - client: change --detach_phase_two (??) to --detach_console
- eliminate compiler warnings (e.g. shadowed vars)
    in various places, mostly in client


svn path=/trunk/boinc/; revision=23710
2011-06-12 20:58:43 +00:00
David Anderson 382905661f svn path=/trunk/boinc/; revision=23478 2011-04-28 21:54:42 +00:00
David Anderson 6b0eba4641 - create_work and other tools: verify that the current dir,
parent dir, or BOINC_PROJECT_DIR actually is a project dir.
- client simulator: improvements


svn path=/trunk/boinc/; revision=23415
2011-04-21 17:04:42 +00:00
David Anderson b89ea98838 - client: when estimating job runtime based on fraction done,
use the elapsed time when fraction done was last reported,
    not current elapsed time.
    Fix problem where est time remaining increases linearly,
    then abruptly decreases when new frac done is reported.
    From Bruce Allen.


svn path=/trunk/boinc/; revision=23373
2011-04-18 16:32:57 +00:00
David Anderson 3b906a191c - client: generalize the GPU framework so that
- new GPU types can be added easily
		- users can specify GPUs in cc_config.xml,
			referred to by app_info.xml,
			and they will be scheduled by BOINC
			and passed --device N options
			Note: the parsing of cc_config.xml is not done yet.
		- RPC protocols (account manager and scheduler)
			can now specify GPU types in separate elements
			rather than embedding them in tag names
			e.g. <no_rsc>NVIDIA</no_rsc> rather than <no_cuda/>
	- client: in account manager replies, parse elements of the form
		<no_rsc>NAME</no_rsc>
		indicating the GPUs of type NAME should not be used.
		This allows account managers to control GPU types
		not hardwired into the client.
		Note: <no_cuda/> and <no_ati/> will continue to be supported.
	- scheduler RPC reply: add
		<no_rsc_apps>NAME</no_rsc_apps>
		(NAME = GPU name)
		to indicate that the project has no jobs for the indicated GPU type.
		<no_cuda_apps> etc. are still supported 
	- client/lib: remove set_debts() GUI RPC
	- client/scheduler RPC
		remove <cuda_backoff> etc. (superceded by no_app)
		Exception: <ip_result> elements in sched request
		still have <ncudas> and <natis>.
		Fix this later.

	Implementation notes:
	- client/lib: change "CUDA" to "NVIDIA" in type/variable names, and in XML
		Continue to recognize "CUDA" for compatibility
	- host_info.coprocs no longer used within the client;
		use a global var (COPROCS coprocs) instead.
		COPROCS now has an array of COPROCs;
		GPUs types are identified by the array index.
		Index zero means CPU.
	- a bunch of other resource-specific structs (like RSC_WORK_FETCH)
		are now stored in arrays, with same indices as COPROCS
		(i.e. index 0 is CPU)
	- COPROCS still has COPROC_NVIDIA and COPROC_ATI structs to hold vendor-specific info
	- APP_VERSION now has a struct GPU_USAGE to describe its GPU usage

svn path=/trunk/boinc/; revision=23253
2011-03-25 03:44:09 +00:00
David Anderson a39a0969d9 svn path=/trunk/boinc/; revision=23207 2011-03-08 02:28:46 +00:00
David Anderson 00f7a7778b - client: on finish upload, trigger work fetch
(since we may have been waiting for upload finish)
	- client: fix crash: if remove an RSS feed, remove its active HTTP op if any.

svn path=/trunk/boinc/; revision=23097
2011-02-23 23:11:59 +00:00
David Anderson cd135ea273 - file upload handler: fix faulty bug fix of 2/15
- client: make REC-based scheduling and hysteresis the defaults

svn path=/trunk/boinc/; revision=23062
2011-02-18 01:06:00 +00:00
David Anderson 717c45a2db - client: use std::deque instead of std::vector
for RR sim's pending-job lists.
    Erasing head of vector is slow.
- lib: allow GPU peak FLOPS to be specified in XML (for simulator)
- simulator work
- client: old work fetch policy: projects may need enough jobs
    for all device instances, not just resource_share*ninst.
    E.g. a project that has only CPU jobs in a CPU/GPU client
- client: with REC scheduling, don't ask for work for
    secondary resources if project has negative priority.
- client: in RR sim, make sure we saturate devices if possible.
    Otherwise we may report a shortfall incorrectly


svn path=/trunk/boinc/; revision=22894
2011-01-12 00:47:51 +00:00
David Anderson eeab2aee92 - simulator work
- fix some indentation

svn path=/trunk/boinc/; revision=22891
2011-01-07 20:23:22 +00:00
David Anderson c5462e4917 - client: more hysteresis work fetch policy stuff
- client simulator work

svn path=/trunk/boinc/; revision=22858
2010-12-30 22:41:50 +00:00
David Anderson 7aeef3070a - client: enabled REC-based scheduling with a cmdline option
rather than a compile flag

svn path=/trunk/boinc/; revision=22855
2010-12-25 19:05:57 +00:00
David Anderson f3169fb77a - client: initial, partial checkin for hysteresis work-fetch
svn path=/trunk/boinc/; revision=22853
2010-12-23 23:39:30 +00:00
David Anderson 18f2e90929 - client: work fetch: if the chosen project is currently uploading a file,
and an upload started in the last 5 min, don't fetch work from it.
    The goal is to merge the 2 scheduler RPCs
    (fetch work, report completed taskS) into a single RPC.
    Note: this may result in idleness in some cases.
- scheduler: if client doesn't handle plan class (pre-5.10),
    check plan-class app versions anyway,
    but only use if it's a single-CPU app.
    This allows single-CPU app versions with specific requirements
    (like SSE) to be issued to old clients.
    From Bernd Machenschalk


svn path=/trunk/boinc/; revision=22841
2010-12-13 22:58:15 +00:00
David Anderson 50c5ce8e81 - client: fix scheduling bug when a job has fraction_done = 1
but it's not finished yet.
    ACTIVE_TASK::est_dur() was returning 0
    when it should have returned elapsed_time.

svn path=/trunk/boinc/; revision=22834
2010-12-08 16:59:42 +00:00
David Anderson b39615d461 - client: work fetch fix: try to maintain GPU work all projects,
since we now do round-robin for GPUs as well as CPU.
    NOTE: this bug was found using the client simulator!
- client simulator: generate REC graph

svn path=/trunk/boinc/; revision=22746
2010-11-24 20:51:25 +00:00
David Anderson 6478b3e05d - client: implement more scheduler changes that use
recent estimated credit (REC) instead of debt.
    These changes are enabled by
        #define USE_REC
    in work_fetch.h.
    If this is commented out (the default) the client uses
    debt-based scheduling, same as before.
    TODO: work-fetch policy changes
- client simulator: various fixes:
    - compute idle and wasted fraction based on all processing resources,
        not just CPU
    - compute job completion times based on FLOPS, not CPU seconds
    - compute and use project->no_X_apps
    etc.


svn path=/trunk/boinc/; revision=22741
2010-11-23 19:39:47 +00:00
David Anderson cc7b53b3e3 - client: fix bug where client would ask project for work for a resource
even when the project doesn't have app versions that use the resource.
	TODO: there are 2 functions,
	compute_may_have_work() and dont_fetch(),
	that do the same thing and both have misleading names.
	Clean this up.
	Rom: please back-port to 6.10

svn path=/trunk/boinc/; revision=22733
2010-11-22 21:13:13 +00:00
David Anderson 8d9cf013c5 - client: account manager RPC:
Additions to request message:
        <not_started_dur>X</not_started_dur>
        <in_progress_dur>X</in_progress_dur>
        The estimated remaining duration of unstarted
        and in-progress tasks
    Additions to reply message, within <project>, optional:
        <suspend>0|1</suspend>
            suspend or resume project (overrides local state)
        <abort_not_started>0|1</abort_not_started>
            if set, abort unstarted jobs


svn path=/trunk/boinc/; revision=22698
2010-11-17 20:04:58 +00:00
David Anderson 3648818499 - client, scheduler, RPC protocol:
- If the scheduler doesn't have any app versions for resource type X,
        it includes an element <no_X_apps>1</no_X_apps> in the reply msg
        (e.g., <no_cpu_apps>1</no_cpu_apps>)
    - The client parses and stores these flags,
        and doesn't ask a project for work for a resource
        if the project doesn't have app versions for it.
    Apparently I started this change in [19375] (October 2009)
    and forgot to finish it.


svn path=/trunk/boinc/; revision=22661
2010-11-09 19:04:24 +00:00
David Anderson 805a763e93 - client: comment out a debug msg
svn path=/trunk/boinc/; revision=22632
2010-11-05 19:17:07 +00:00
David Anderson 8a23379003 - client: update STD of ineligible projects by decay only.
Not sure why, but this eliminates gradual negative drift.

svn path=/trunk/boinc/; revision=22594
2010-10-23 22:19:48 +00:00
David Anderson c9be64765c - client: use project STD, rather than arrival time,
as the major criterion in choosing non-EDF GPU jobs.
    GPU scheduling now respects resource share,
    and as a result STD should no longer diverge.
- client simulator: various improvements, most notably
    that we now generate gnuplot graphs of all debt types

NOTE: the client problem was found and fixed using the simulator!


svn path=/trunk/boinc/; revision=22536
2010-10-15 20:16:00 +00:00
David Anderson a20e7d5837 - client simulator: updates. Fix web interface.
svn path=/trunk/boinc/; revision=22476
2010-10-07 21:41:31 +00:00
David Anderson 1c4422985f - client: add <no_info_fetch> config option and --no_info_fetch
cmdline arg.
    Suppresses the fetch of project list and of current client version #.
    Use when running on grid nodes.
- debugging on client simulator.  Not done yet.

svn path=/trunk/boinc/; revision=22414
2010-09-27 20:34:47 +00:00
David Anderson 31db3207e4 - client: fix bug that cause wasted scheduler RPC
Old: when a job finished, we cleared the backoffs for the
        resources it used.  The idea was to get more jobs
        immediately in the case where the client was at
        a jobs-in-progress limit.
    Problem: this resulted in an RPC immediately,
        typically before the output files were uploaded.
        So the client is still at the limit, and doesn't get jobs.
    New: clear the backoffs at the point when output files
        have been uploaded and the job is ready to report.
- client: change range in resource backoff from (0,x) to (.5, 1.5*x)


svn path=/trunk/boinc/; revision=22411
2010-09-24 21:24:02 +00:00