Commit Graph

34 Commits

Author SHA1 Message Date
David Anderson b356552c9c - scheduler/feeder: add a project config option <dont_send_jobs>.
If set, the feeder doesn't read jobs into shmem,
    and the scheduler doesn't send jobs.
    Intended for use when a project wants to process
    a backlog of completed jobs and not issue more.

svn path=/trunk/boinc/; revision=22601
2010-10-28 19:02:19 +00:00
David Anderson 84679f482a - scheduler: change the "primary_platform_only" config option
to "prefer_primary_platform".
    If an app has only only 32-bit versions, use the for 64-bit clients.


svn path=/trunk/boinc/; revision=22282
2010-08-22 19:13:25 +00:00
David Anderson d79ca6a9f2 - scheduler: add <primary_platform_only> config option:
send only 64-bit app versions to 64-bit hosts 
    (the default is to send whatever app version is fastest)

svn path=/trunk/boinc/; revision=22183
2010-08-10 22:17:59 +00:00
David Anderson e0cea31781 - API: add result name to APP_INFO_DATA structure (for Volpex)
- scheduler: add max_download_urls_per_file config option
    (to limit the length of workunit.xml_doc,
    which is currently capped at 64KB).
    From Bernd.

svn path=/trunk/boinc/; revision=22082
2010-07-30 21:43:23 +00:00
David Anderson c0776ea188 - user web: put RSS item titles in CDATA
- sched: get rid of unused config items
- manager: msg tweak

svn path=/trunk/boinc/; revision=22045
2010-07-22 22:57:15 +00:00
David Anderson 0f613d61d8 - scheduler and client: fix the "allow multiple clients" feature.
This feature lets you run the BOINC client as a job on grid systems
    that handle only 1-CPU jobs;
    it disables various mechanisms that prevent multiple clients per host
    (which is normally a bad thing).
    Old:
        - Run the client with a --allow_multiple_clients flag.
            This tells it not to use a mutex that prevents
            multiple clients per host.
        - Run the project with the <multiple_clients_per_host> config flag.
            This suppresses two mechanisms:
            - (avoid duplicate host records)
                on a scheduler request with no host ID,
                looks for a host with same domain name, OS type,
                and mem size, and assumes the request is from that host
            - (job retry)
                If we get a request that doesn't have a host ID
                but does have a host CPID,
                mark its in-progress results as over
                NOTE: I CAN'T REMEMBER WHY WE SUPPRESS THIS;
                MARK S, DO YOU REMEMBER?

    Problem:
        if the grid clients attach to a project that
        doesn't use <multiple_clients_per_host>, bad things happen.
        E.g., if there are several requests at about the same time,
        most of them will fail with
        "another RPC already in progress" errors.
        If a project does include this flag,
        it loses protection from duplicate host records.

    New:
        - If the client is run with --allow_multiple_clients flag,
            it passes a <allow_multiple_clients> element
            in scheduler requests.
        - The scheduler skips the duplicate-host check on
            requests that include this flag.
        - There is no more <multiple_clients_per_host> scheduler option.

    Note: if a project using the old mechanism upgrades to this change,
    it will need to use new clients for its grid deployment.


svn path=/trunk/boinc/; revision=21839
2010-06-29 16:37:28 +00:00
David Anderson 7c51512cbf - transitioner: the format string for a DB query had %.15d instead of %.15e.
That produced a messed-up query that assigned garbage values to:
        host_app_version.turnaround_var
        host_app_version.turnaround_q
        host_app_version.max_jobs_per_day
        host_app_version.consecutive_valid
    To repair these:
        - set turnaround_var and turnaround_q to zero
        - if max_jobs_per_day is outside of
            (0..config.daily_result_quota)
            set it to config.daily_result_quota
        - if consecutive_valid is outside (0..1000), set it to zero
    I added a script, html/ops/repair_21812.php, that does this;
    if you ran server code between [21181] and [21812], run this script.
- scheduler/transitioner: add <debug_quota> log flag
- changed the build system to always use -Wall
    (if we'd done this before, this bug wouldn't have happened)
- fixed a bunch of other compile warnings


svn path=/trunk/boinc/; revision=21812
2010-06-25 18:54:37 +00:00
David Anderson 392f02f8b8 - API: add BOINC copyright notice to graphics2_win.cpp.
This file originally used code from the following tutorial,
    which shows how to open a window using GLUT:
    http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=01
    The code has now been completely rewritten;
    in particular, it doesn't use GLUT anymore.
- scheduler: change default limit on #CPUs from 16 to 64

svn path=/trunk/boinc/; revision=21784
2010-06-21 21:14:34 +00:00
David Anderson 4147249de2 - server: delete old credit stuff
- user web: show host link in user result list.  Fixes #999


svn path=/trunk/boinc/; revision=21735
2010-06-12 22:08:15 +00:00
David Anderson f849faea5e - scheduler: bug fixes for jobs-in-progress limits
- client: msg tweak

svn path=/trunk/boinc/; revision=21692
2010-06-04 16:57:33 +00:00
David Anderson 356327d88c - scheduler: change backoff policy if a host has reached daily job quota.
Old: back off until random time in 1st hour of next day
    New: no server-dictated backoff; rely on client backoff
    This is needed to let hosts recover in a reasonable amount of time
    after a burst of errors.
- scheduler config: it turns out we can't put arbitrary XML in config.xml;
    The Python code is set up to parse only 1 level of tags (??),
    and I'm not up to the task of changing this.
    So the fine-grained job limit feature [21674] needs to use
    a different file, namely config_aux.xml

svn path=/trunk/boinc/; revision=21686
2010-06-03 04:59:27 +00:00
David Anderson cf7fb29227 - scheduler: add fine-grained "max jobs in progress" control.
You can now specify limits for specific apps,
    and/or for the project as a whole.
    Within each of these, you can specify limits on
    CPU jobs, GPU jobs, or total jobs.
    In the case of CPU and GPU limits, you can specify
    whether the limit should be scaled by the number of devices.

    Note: the enforcement of this is done in get_app_version(),
    since per-resource-type limits may dictate what app versions
    we can use for a particular job.

svn path=/trunk/boinc/; revision=21674
2010-06-01 23:41:07 +00:00
David Anderson d45d3b488f - server: code cleanup
svn path=/trunk/boinc/; revision=21664
2010-06-01 03:45:49 +00:00
David Anderson 021edb02c2 - back end programs: improve log msgs
svn path=/trunk/boinc/; revision=21193
2010-04-16 18:07:08 +00:00
David Anderson fb851311e0 - server: various changes;
see http://boinc.berkeley.edu/trac/wiki/CreditNew

    Projects will need to update DB and recompile all back-end programs.

    Summary:
    - new way of computing credit
    - "reliable host" mechanism is per app version
    - "host punishment" mechanism is per app version
    - adjustment of wu.rsc_fpops_est provides the
        equivalent of per app version DCF
    - max jobs in progress is now per app
    - max jobs per RPC is now per app

    TODO:
    - reliable mechanism:
        - populate and use host_app_version.error_rate
        - populate host_app_version.turnaround
    - host punishment:
        - populate host_app_version.max_jobs_per_day
        - populate host_app_version.n_jobs_today
        - use app.max_jobs_per_day_init
    - job limits:
        - use app.max_jobs_in_progress, max_gpu_jobs_in_progress
        - use app.max_jobs_per_rpc
    - adjust wu.rsc_fpops_est
    - remove old credit stuff
        fpops_cumulative, credit_multiplier
        credit computation in scheduler

- AVERAGE class: use the Knuth algorithm (Wikipedia)


svn path=/trunk/boinc/; revision=21021
2010-03-29 22:28:20 +00:00
David Anderson 0c1a1421f8 - scheduler/feeder: if any client version number field
(min_core_version etc.) is < 10000,
    multiply it by 100 and print a warning.

svn path=/trunk/boinc/; revision=20187
2010-01-18 04:52:58 +00:00
David Anderson c2c52eba79 - Unix build fix
svn path=/trunk/boinc/; revision=19114
2009-09-21 16:15:28 +00:00
David Anderson bd8f1f20d1 svn path=/trunk/boinc/; revision=19012 2009-09-07 05:26:23 +00:00
Rytis Slatkevičius f239587bdb Sched: config option not to store stderr_out if exit_status==0 (to save on DB size). With help from Nicolas Alvarez.
svn path=/trunk/boinc/; revision=18528
2009-06-30 18:00:58 +00:00
David Anderson 10f9e11ee6 - lib: created a new file for declaring "replacements"
for functions like strlcpy() etc.
    config.h is included here rather than in str_util.h


svn path=/trunk/boinc/; revision=18437
2009-06-16 20:54:44 +00:00
David Anderson 8a765c0f4a - file deleter: detect cases where the upload/download dir doesn't exist,
and treat it as a recoverable error (i.e., retry).
    The file deleter may run on a host that NSF-mounts
    the upload/download dirs, and NSF mounts can file.
- scheduler: include WU#ID in log msgs for handled results

svn path=/trunk/boinc/; revision=18351
2009-06-10 17:42:18 +00:00
David Anderson 2e5d9bd778 - scheduler: add new config option <max_wus_in_progress_gpus>.
The limit on jobs in progress is now
        max_wus_in_progress * NCPUS
        + max_wus_in_progress * NGPUS
    where NCPUS and NGPUS reflect prefs and are capped.
    Furthermore: if the client reports plan class for in-progress jobs
    (see checkin of 31 May 2009)
    then these limits are enforced separately;
    i.e. the # of in-progress CPU jobs is <= max_wus_in_progress*NCPUS,
    and the # of in-progress GPU jobs is <= max_wus_in_progress_gpu*NGPUS
- scheduler config: rename <cuda_multiplier> to <gpu_multiplier>
- scheduler: <max_wus_to_send> is now scaled by
    (NCPUS + gpu_multiplier*NGPUS)
- scheduler: don't keep scanning array if !work_needed()
- scheduler: moved array-scan logic from sched_send.cpp to sched_array.cpp
- scheduler: don't say "no work available" if jobs are available
    but work_needed() is initially false


svn path=/trunk/boinc/; revision=18255
2009-06-01 22:15:14 +00:00
David Anderson c2fda4db09 - scheduler: add <report_max> config parameter;
limits the # of completed results handled per scheduler RPC.
    This may be needed to avoid crashes due to memory allocation
    failure (each reported result uses about 128KB memory).
- web: In showing result lists,
    include "Validate error" results in the "Invalid" category.
    (Previously they didn't appear in any category)

svn path=/trunk/boinc/; revision=18104
2009-05-14 19:01:40 +00:00
David Anderson 12eb6057e5 - client, Mac: don't do res_init(). It causes a crash.
- client (Unix): if client crashes while benchmark processes are going,
    make sure they detect this and exit.
- back-end programs: remove hardwired assumptions about
    what directory they run in, and hence where config.xml is.
    E.g., daemons look for it in "..", others expect it in current dir.
    New approach: all the programs look for the project dir as follows:
    1) the environment var BOINC_PROJECT_DIR, if defined
    2) the current dir, if config.xml is there.
    3) else ".."
    This means you can run programs in either proj/bin/ or proj/,
    or (using BOINC_PROJECT_DIR) you can keep executables
    outside of the project dir.


svn path=/trunk/boinc/; revision=18042
2009-05-07 13:54:51 +00:00
David Anderson 41ed82f791 - scheduler: fix bugs that caused only 1 job to be sent
svn path=/trunk/boinc/; revision=17555
2009-03-07 01:00:05 +00:00
David Anderson 66ec889431 - scheduler: add <locality_scheduling_sticky_file>
and <locality_scheduling_workunit_file> options
    From Bernd M.

svn path=/trunk/boinc/; revision=17431
2009-03-03 00:25:41 +00:00
David Anderson aadf813336 - scheduler/feeder: add <locality_scheduler_fraction> option;
lets you intermix locality and job-cache scheduling
    From Bernd M.

svn path=/trunk/boinc/; revision=17429
2009-03-03 00:12:55 +00:00
David Anderson 2d707927ab - scheduler: replace choose_download_url_by_timezone with
replace_download_url_by_timezone.


svn path=/trunk/boinc/; revision=17427
2009-03-02 23:47:11 +00:00
David Anderson 85a8e6a772 - scheduler: remove the config flag <have_cuda_apps>,
and add <cuda_multiplier>.
    The latter is used in calculating max jobs/day for a host;
    namely, it's host.max_results_day * (NCPUS + NCUDA*cuda_multiplier).
    Set it to 10 or so if you have CUDA apps.
- scheduler: don't overload effective_ncpus();
    instead, add two new functions,
    max_results_day_multiplier() and max_wus_in_progress_multiplier()
- scheduler: don't reduce max_results_day if we get an aborted job
    (it might have been aborted by the project;
    not appopriate to punish host in this case)

svn path=/trunk/boinc/; revision=16959
2009-01-20 00:54:16 +00:00
David Anderson 91e120b3f4 - scheduler: improve message formatting; add <debug_locality> flag
for locality scheduling messages

svn path=/trunk/boinc/; revision=16921
2009-01-15 20:23:20 +00:00
David Anderson 14f6f9a386 - scheduler: add <ignore_dcf> option;
use this temporarily when you've fixed FLOPS estimate


svn path=/trunk/boinc/; revision=16672
2008-12-12 17:03:54 +00:00
David Anderson 5039207e2c - scheduler: add <have_cuda_apps> config flag.
If set the "effective NCPUS" (which is used to scale
    daily_result_quota and max_wus_in_progress)
    is max'd with the # of CUDA GPUs.

svn path=/trunk/boinc/; revision=16246
2008-10-21 23:16:07 +00:00
David Anderson bb9d546a02 - scheduler: add <no_vista_sandbox> option.
If set, don't send work to sandboxed Vista clients
    (e.g., because of CUDA issue)

svn path=/trunk/boinc/; revision=16105
2008-10-01 19:48:52 +00:00
David Anderson 98cfb8d3b0 - rename .C files to .cpp so that Doxygen will work
svn path=/trunk/boinc/; revision=16069
2008-09-26 18:20:24 +00:00