Commit Graph

89 Commits

Author SHA1 Message Date
David Anderson 8c71f6d59a - scheduler: add support for Intel GPUs, and restructure things
to make it easier to add other GPU types in the future


svn path=/trunk/boinc/; revision=25792
2012-06-25 23:09:45 +00:00
David Anderson fd0983b991 - web: server status page should show elapsed time, not CPU time
svn path=/trunk/boinc/; revision=25785
2012-06-22 07:35:54 +00:00
David Anderson 9d25481174 - scheduler: fix bug that tried to open plan class spec file
on each request.
- client: when showing how much work a scheduler request returned,
    scale by availability (as is done to show the amount of the request)
- client in account manager request, <not_started_dur> and
    <in_progress_dur> are in wall time, not run time
    (i.e. scale them by availability)

Note: there's some confusion in the code between runtime and wall time,
    where in general wall time = runtime / availability.
    New convention: let's use "runtime" for the former,
    and "duration" for the latter.

svn path=/trunk/boinc/; revision=25597
2012-04-25 04:10:29 +00:00
Bernd Machenschalk 75a2116f2e - scheduler: fix hu.peak_flops and hu.projected_flops caclculation
in plan_class_spec by using coproc_pref() and capped_host_fpops()
        (moved coproc_perf() to sched_customize.h to make it available
        in plan_class_spec.cpp, and cleaned up includes)


svn path=/trunk/boinc/; revision=25467
2012-03-21 12:40:18 +00:00
David Anderson a16032dee5 - scheduler: add an alternative way of defining app plan functions
that uses an XML configuration file;
    see http://boinc.berkeley.edu/trac/wiki/AppPlanConfig
    From Bernd.


svn path=/trunk/boinc/; revision=25426
2012-03-14 19:53:16 +00:00
David Anderson f18ffd6fe7 - VDA: add some log messages
- scheduler: add VDA
- client, web: change default prefs to min_buf=.1 days, max_buf=.5 days
- scheduler: app plan function for vbox requires 7.0+ client


svn path=/trunk/boinc/; revision=25351
2012-02-28 06:57:28 +00:00
David Anderson c2ff0d9719 - scheduler: tell the user if we're not sending work because
VirtualBox is not installed


svn path=/trunk/boinc/; revision=25313
2012-02-22 21:21:15 +00:00
David Anderson 64608800cb - scheduler: don't send "update GPU driver" messages if
the client didn't report a driver version #


svn path=/trunk/boinc/; revision=25232
2012-02-09 20:34:29 +00:00
David Anderson 480e28b54c - web: fix the user search feature
- scheduler: parse d_project_share
- scheduler: if vbox and vbox_mt are both available,
    use vbox for a 1-CPU machine


svn path=/trunk/boinc/; revision=25176
2012-02-01 03:30:14 +00:00
Rom Walton be9e807e31 - sched: adjust the vbox??_mt plan classes to use 1.5 CPUs instead
of the full 2 CPUs. Vboxwrapper uses ceil() to allocate enough
        whole CPUs for Virtualbox.  Ideally this will cause the BOINC
        client-side scheduler to use the remaining fraction of the CPU
        for GPU data transfer which will then free up one whole CPU for
        another job.  All without over-commiting anything.
        
    sched/
        sched_customize.cpp

svn path=/trunk/boinc/; revision=25120
2012-01-21 18:51:37 +00:00
David Anderson dd16170fc1 - scheduler: the p_fpops value reported by clients can't be trusted.
Some credit cheats (e.g. with credit_by_runtime) can be done
    by reporting a huge value.
    Fix this by capping the value at 1.1 times the 95th percentile
    of host.p_fpops, taken over active hosts.


svn path=/trunk/boinc/; revision=25017
2012-01-09 17:35:48 +00:00
David Anderson 436d56e70b - scheduler: change vbox_mt plan function to use at most 2 cores, not 3
(CERN doesn't actually need 3)


svn path=/trunk/boinc/; revision=25013
2012-01-09 02:48:51 +00:00
David Anderson e8657adfd2 - scheduler: change vbox_mt app plan function to use 1, 2 or 3 CPUs
depending on how many the host has,
    and whether CPU VM extensions are present
    (this reflects the requirements of CernVM).


svn path=/trunk/boinc/; revision=25009
2012-01-08 01:28:39 +00:00
David Anderson 95ebb112c2 - client: for VBox apps, check stderr for "ERR_CPU_VM_EXTENSIONS_DISABLED".
If found, set HOST_INFO::p_vm_extensions_disabled,
    and pass this to the scheduler.
- scheduler (VBox app plan function) if a host has p_vm_extensions_disabled
    set, don't sent it multicore VBox jobs.

Note: if you have a host with VM extensions, and they're disabled
    in the BIOS, and you enable them, you can remove the
    <p_vm_extensions_disabled> line from client_state.xml
    and you'll be eligible to get multicore VM jobs again.


svn path=/trunk/boinc/; revision=24944
2011-12-30 09:43:58 +00:00
David Anderson df5d595c3a - scheduler, vbox plan class function:
send only 32/64 bit version to 32/64 bit host


svn path=/trunk/boinc/; revision=24916
2011-12-27 02:29:51 +00:00
Rom Walton 94c7b82d3b svn path=/trunk/boinc/; revision=24903 2011-12-26 13:42:34 +00:00
David Anderson fe16024982 - scheduler: in vbox plan class, require that host have
VM acceleration hardware feature
- remove job submission: typo fix


svn path=/trunk/boinc/; revision=24902
2011-12-26 08:27:40 +00:00
David Anderson 4774eeda52 - make_project: don't try to copy nonexistent file; fixes #1166
- scheduler: change Vbox app plan function to accommodate
    single and multithreaded variants


svn path=/trunk/boinc/; revision=24884
2011-12-24 05:07:20 +00:00
David Anderson 819360dfe8 - scheduler: encode CAL version numbers in a way that handles
release #s > 1000


svn path=/trunk/boinc/; revision=24746
2011-12-06 19:41:14 +00:00
David Anderson 2d7573b8f7 - scheduler: in app_plan(), check for "opencl" before "ati".
Otherwise "opencl_ati" won't get handled right.
    Fixes #1158


svn path=/trunk/boinc/; revision=24716
2011-12-02 14:37:47 +00:00
David Anderson dd93780787 - API and client: add "ncpus" field to APP_INIT_DATA.
Tells multicore apps how many cores to use.
    The --nthreads command line arg to the app is now deprecated
    though we'll keep it around for the time being.


svn path=/trunk/boinc/; revision=24708
2011-12-01 18:44:19 +00:00
David Anderson 690e18bbe3 - server: plan class names containing 'nvidia' or 'cuda'
are assumed to be for NVIDIA GPU apps;
    plan class names containing 'ati' are assumed to be for AMD GPU apps.
    Clauses for 'nvidia' were missing in a couple of places.


svn path=/trunk/boinc/; revision=24512
2011-11-03 05:26:19 +00:00
David Anderson 836e8aacf7 - scheduler: in cuda_check(), ati_check() and opencl_check()
(in sched_customize.cpp)
    the flops_scale argument is intended to express the
    GPU efficiency (actual/peak).
    Pass appropriate values.


svn path=/trunk/boinc/; revision=24405
2011-10-16 06:04:13 +00:00
David Anderson b1456cfc0b - scheduler: fix a bug that would choose app versions erroneously.
The problem: the choice of app version was based on
    the "projected FLOPS" return by estimate_flops(av).
    If usage stats exist for the host / app version,
    this returns a number X such that
    WU.rsc_fpops_est/X approximates the runtime of a job
    using the given app version..
    (If WU.rsc_fpops_est is way off, this will be correspondingly way off
    from the actual FLOPS the app version will get.)
    However, if there are no usage stats,
    it return an estimate based on host hardware speed,
    which might be 100X less.
    Hence, in some cases a new app version would never get used.

    Solution: choose app versions based on the values
    returned by the app plan functions.
    Use estimate_flops() AFTER choosing the version.
- scheduler: improve the accuracy of FLOPS estimation for GPU apps.
    The "flops_scale" argument to coproc_perf
    (which expresses the difference between peak GPU FLOPS
    and actual FLOPS) should be used to scale GPU FLOPS
    prior to calling coproc_perf(),
    rather than scaling the estimate returned by coproc_perf().
- show_shmem: show have_X_apps flags


svn path=/trunk/boinc/; revision=24385
2011-10-12 23:59:38 +00:00
David Anderson dd3b628748 - client: compare OpenCL-only devices the same as other devices
- code cleanup


svn path=/trunk/boinc/; revision=24354
2011-10-08 06:33:39 +00:00
David Anderson 6da80764fc - scheduler add app_plan() support for plan classes
opencl_nvidia_101 and opencl_ati_101


svn path=/trunk/boinc/; revision=24345
2011-10-07 19:23:37 +00:00
David Anderson 3acc4fa77d - client/server: change field names in OPENCL_DEVICE_PROP
to match those in the clGetDeviceInfo() calls.
    Principles:
    - if there's already a name for something, use it.
    - follow case conventions


svn path=/trunk/boinc/; revision=24344
2011-10-07 18:11:52 +00:00
David Anderson ae3d5a85dd - scheduler: fix bug that caused scheduler to not send GPU jobs
svn path=/trunk/boinc/; revision=24338
2011-10-06 02:56:30 +00:00
David Anderson 49a7287949 - scheduler: in app_plan functions for GPUs,
use available RAM if reported by client;
    otherwise use physical RAM.


svn path=/trunk/boinc/; revision=24216
2011-09-15 04:28:21 +00:00
David Anderson 533b4b88e1 - scheduler: turns out we need separate vbox32 and vbox64
plan classes after all.
    Otherwise (since app_plan() is not passed an app version)
    there's no way to enforce that 64 bit hosts are sent
    only the 64 bit version (which is necessary because
    of the split-registry scheme).


svn path=/trunk/boinc/; revision=23935
2011-08-04 18:06:14 +00:00
David Anderson f3b9286d78 - client: fix bug in get_descendants(),
which prevented the client from cleaning up
    subprocesses of misbehaving multiprocess apps.
- remote job submission system:
    assign physical names to input files (based on their MD5)
    rather than having the user provide physical names
- VM apps: eliminate vbox64 plan class.  Only vbox.


svn path=/trunk/boinc/; revision=23923
2011-08-03 16:45:00 +00:00
David Anderson 3c70004c5c - server: debug distributed storage functions
- generate upload signatures if needed
    - assign deadlines to file xfer jobs (default 1 week)
    - scheduler: ack completed file xfer results


svn path=/trunk/boinc/; revision=23866
2011-07-20 22:27:01 +00:00
David Anderson 27e05a3da9 - server: some stuff to prepare for distributed storage
- don't create result records for uploads and downloads.
        Just create a msg_to_client record.
    - the scheduler handles file-transfer results specially;
        it makes a vector of them, then calls a project-supplied function
        handle_file_xfer_results()
    - change the interface and implementation of put_file and get_file
- client write project sched priority in GUI RPC replies,
    but not to the state file


svn path=/trunk/boinc/; revision=23857
2011-07-19 20:52:41 +00:00
David Anderson 597320db39 - scheduler: compile fixes
svn path=/trunk/boinc/; revision=23281
2011-03-25 22:47:49 +00:00
David Anderson a0fd3445fc - wrapper: improve and bullet-proof the way we kill apps.
Win: enumerate all descendants, and kill them all TerminateProcess().
    Unix:
        send the main process a SIGTERM.
        Check once a sec for existence of descendants.
        if none, done
        If any still exist after 10 sec, kill all descendants
- wrapper fix bug in Win env var stuff
- scheduler: check for VBox version 3.2+ in app_plan()
    

svn path=/trunk/boinc/; revision=23085
2011-02-22 23:11:34 +00:00
David Anderson 396dc5a8b4 svn path=/trunk/boinc/; revision=22895 2011-01-12 01:17:07 +00:00
David Anderson 4f3d628e29 - client: if detect that system clock has been set backwards
(either at startup or during execution)
    reset a number of "wait until X" variables;
    otherwise we might wait years to contact a project, restart a file xfer, etc.
    Notes:
    - there is no problem setting clocks forward; things just happen prematurely
    - some variables (e.g. task deadlines) are not reset,
        because it's not clear what to set them to
- sched: remove ati_opencl plan class until we understand what it is


svn path=/trunk/boinc/; revision=22842
2010-12-17 01:30:01 +00:00
David Anderson 3de1406265 - scheduler: in app_plan_sse3(): check for CPU features in
p_model as well as p_features;
    pre-6.x clients report them in p_model.
- client: fix bug where "reread config file" didn't update
    the max log file sizes

svn path=/trunk/boinc/; revision=22838
2010-12-09 23:27:11 +00:00
David Anderson f8e2d07cf9 - scheduler: add vbox32 and vbox64 plan classes for VirtualBox apps.
svn path=/trunk/boinc/; revision=22778
2010-11-30 19:36:07 +00:00
David Anderson 40c50852f5 - scheduler: fix logic that deals with jobs that need > 2GB RAM.
My change of 1 Oct ([22440]) required that such jobs
    be processed with 64-bit apps,
    on the assumption that 32-bit apps have a 2 GB user address space limit.
    However, it turns out this limit applies only to Windows
    (kernel and user mode share the 4GB address space; each gets half).
    On Linux, the split is 3GB user / 1 GB kernel.
    On Mac OS X, user mode and kernel mode have separate address spaces,
    each of them 4 GB.


svn path=/trunk/boinc/; revision=22599
2010-10-27 22:58:16 +00:00
David Anderson 473c12c9d0 - scheduler: typo in app_plan() for ATI GPUs
svn path=/trunk/boinc/; revision=22433
2010-09-30 21:52:13 +00:00
David Anderson e53e9710e8 - scheduler: make some "notice"-priority messages translatable
- scheduler: add a clause to wu_is_infeasible_custom() for SETI@home:
    don't process VLAR jobs using CUDA apps.
    Note: this is implemented in a slightly non-optimal way.
    If the request asks for both GPU and CPU jobs,
    the scheduler will first decide to use the GPU version.
    It will scan jobs, skipping over VLAR jobs.
    When the GPU request is satisfied, it will switch to the CPU version
    and continue scanning, accepting VLAR jobs.
    But the jobs that were skipped initially won't be rescanned.
    Also, it would be slightly nice to preferentially send
    VLAR jobs to hosts asking for CPU work.
    (This could be done in the scoring function).

svn path=/trunk/boinc/; revision=21895
2010-07-12 22:43:53 +00:00
David Anderson 7c51512cbf - transitioner: the format string for a DB query had %.15d instead of %.15e.
That produced a messed-up query that assigned garbage values to:
        host_app_version.turnaround_var
        host_app_version.turnaround_q
        host_app_version.max_jobs_per_day
        host_app_version.consecutive_valid
    To repair these:
        - set turnaround_var and turnaround_q to zero
        - if max_jobs_per_day is outside of
            (0..config.daily_result_quota)
            set it to config.daily_result_quota
        - if consecutive_valid is outside (0..1000), set it to zero
    I added a script, html/ops/repair_21812.php, that does this;
    if you ran server code between [21181] and [21812], run this script.
- scheduler/transitioner: add <debug_quota> log flag
- changed the build system to always use -Wall
    (if we'd done this before, this bug wouldn't have happened)
- fixed a bunch of other compile warnings


svn path=/trunk/boinc/; revision=21812
2010-06-25 18:54:37 +00:00
David Anderson 81973a9fff - scheduler: fix structural problems with sending user messages.
Old: various redundant and/or misleading messages were sent.
    New:
        - if host w/ no GPU contacts a GPU-only project,
            send high-pri message saying they need a GPU
        - if host w/ GPU has driver too old for all versions,
            send high-pri message saying to update driver
        - if host w/ GPU has driver too old for some versions,
            send low-pri message saying to update driver
        - if host has GPU but too little RAM for any app,
            send low-pri message saying so
- scheduler: revamp GPU plan class functions

svn path=/trunk/boinc/; revision=21760
2010-06-16 22:07:19 +00:00
David Anderson 9194af6722 - scheduler: tweaks to user messages
svn path=/trunk/boinc/; revision=21749
2010-06-15 17:56:30 +00:00
David Anderson 5a28b5672e - client: user-visible text (message body, notice title and description)
are now translatable,
    using the convention that any substring enclosed in _(" ... ")
    should be passed throughh wxGetTranslation() or the equivalent.
- client: when writing messages to stdout, strip out _(...)
- manager: translate strings from client
- scheduler: message tweaks

svn path=/trunk/boinc/; revision=21706
2010-06-08 18:56:53 +00:00
David Anderson fa66519441 - scheduler: SETI@home's CUDA and CUDA 2.3 apps apparently don't
run on Fermi (compute capability 2) hardware.
    Temporary solution: change app_plan() accordingly
- scheduler: message tweaks

svn path=/trunk/boinc/; revision=21595
2010-05-20 22:49:00 +00:00
David Anderson 8066fa7fb2 - scheduler: message tweaks
svn path=/trunk/boinc/; revision=21590
2010-05-20 16:43:03 +00:00
David Anderson 40eebe00af - client/scheduler: in COPROCS, instead of having a vector of
pointers to dynamically allocated COPROC-derived objects,
    just have the objects themselves.
    Dynamic allocation should be avoided at all costs.

svn path=/trunk/boinc/; revision=21564
2010-05-18 19:22:34 +00:00
David Anderson 256c694c96 - client: make GPU available RAM measurement #ifdef-selectable,
and default it to off
- client: if we print available GPU RAM (which we now don't)
    have a separate timer per GPU type
- scheduler: add new plan classes cuda_opencl (sic) and ati_opencl

svn path=/trunk/boinc/; revision=21498
2010-05-13 03:07:33 +00:00