Commit Graph

79 Commits

Author SHA1 Message Date
David Anderson da7e82fe15 - scheduler and back end: add new fields to result table:
elapsed_time: the elapsed time (runtime) as reported by client
    flops_estimate: the app's estimated FLOPS as reported by app_plan()
    app_version_id: the DB ID of the app_version used
        (or -1 if anonymous platform)
    TODO: show these in the web interfaces,
    and use them where appropriate

svn path=/trunk/boinc/; revision=19002
2009-09-03 20:26:31 +00:00
David Anderson 8b701fc73f - scheduler: fix messed-up deadline check logic.
Old:
        1) check deadline based on wu.delay_bound
        2) in add_result_to_reply(), potentially modify wu.delay_bound,
            e.g. because of retry acceleration
        problem: reducing delay bound may cause deadline miss
    New:
        1) new function get_delay_bound_range()
            (called from wu_is_infeasible_fast())
            returns optimistic and pessimistic delay bounds.
            Retry acceleration logic is here.
        2) check deadline based on optimistic bound;
            if that fails, check based on pessimistic bound.
            Set wu.delay_bound to the one that worked.
    Notes:
    - get_delay_bound_range() needs result priority and report deadline,
        and it's called before we read the full result.
        So add these items to WORK_ITEM and WU_RESULT.
    - get_delay_bound_range() could be customized for
        project-specific deadline policy.
    - add_result_to_reply() was becoming a toxic waste dump.
        Deadline-related stuff should have been factored out in any case.

svn path=/trunk/boinc/; revision=18946
2009-08-31 19:35:46 +00:00
David Anderson a49ba8c2e9 - scheduler: if request is anon platform,
write list of client's app versions to log

svn path=/trunk/boinc/; revision=18923
2009-08-26 18:21:36 +00:00
David Anderson eafb410cf8 - scheduler: simplify and fix the way that app_plan() conveys messages
to the user.  app_plan() now generates the messages directly
    rather than returning integer error codes.

svn path=/trunk/boinc/; revision=18899
2009-08-21 20:38:39 +00:00
David Anderson 9e9f2a9878 - scheduler: code cleanup
svn path=/trunk/boinc/; revision=18896
2009-08-21 19:14:15 +00:00
David Anderson 7278ab1787 - scheduler: add support for ATI GPUs
svn path=/trunk/boinc/; revision=18851
2009-08-17 17:07:38 +00:00
David Anderson b300519444 svn path=/trunk/boinc/; revision=18825 2009-08-10 04:49:02 +00:00
David Anderson e3363c7eb8 - scheduler: on second thought, it would be better to add the above
feature without requiring use of score-based scheduling.
    So add a new customizable function, wu_is_infeasible_custom(),
    where projects can put job-specific checks.

    Also, move customizable functions (of which there are now 4)
    to a new file, sched_customize.cpp.

svn path=/trunk/boinc/; revision=18767
2009-07-29 18:55:50 +00:00
David Anderson 77055d17e7 svn path=/trunk/boinc/; revision=18765 2009-07-29 18:34:27 +00:00
David Anderson 37638081fa - scheduler: improved messages for debug_send
svn path=/trunk/boinc/; revision=18761
2009-07-28 23:07:11 +00:00
David Anderson a71a03f698 - Mac install: fix things a better way
- install: don't install internal .h files
- scheduler: fix spurious "reached limit of 0 GPU tasks" message,
    slight code cleanup

svn path=/trunk/boinc/; revision=18480
2009-06-22 21:11:19 +00:00
David Anderson 3824d359a3 - scheduler: fix <max_wus_in_progress> code
svn path=/trunk/boinc/; revision=18472
2009-06-19 18:34:00 +00:00
David Anderson 9527cb6534 - install: don't install std_fixes.h or msg_queue.cpp
- remove some unused code
- scheduler: small code cleanup

svn path=/trunk/boinc/; revision=18465
2009-06-19 17:21:34 +00:00
David Anderson cc62cce8f7 - client: if scheduler request didn't request work, don't report 0 tasks
- scheduler: fix crash if anonymous platform

svn path=/trunk/boinc/; revision=18259
2009-06-02 05:12:06 +00:00
David Anderson 2e5d9bd778 - scheduler: add new config option <max_wus_in_progress_gpus>.
The limit on jobs in progress is now
        max_wus_in_progress * NCPUS
        + max_wus_in_progress * NGPUS
    where NCPUS and NGPUS reflect prefs and are capped.
    Furthermore: if the client reports plan class for in-progress jobs
    (see checkin of 31 May 2009)
    then these limits are enforced separately;
    i.e. the # of in-progress CPU jobs is <= max_wus_in_progress*NCPUS,
    and the # of in-progress GPU jobs is <= max_wus_in_progress_gpu*NGPUS
- scheduler config: rename <cuda_multiplier> to <gpu_multiplier>
- scheduler: <max_wus_to_send> is now scaled by
    (NCPUS + gpu_multiplier*NGPUS)
- scheduler: don't keep scanning array if !work_needed()
- scheduler: moved array-scan logic from sched_send.cpp to sched_array.cpp
- scheduler: don't say "no work available" if jobs are available
    but work_needed() is initially false


svn path=/trunk/boinc/; revision=18255
2009-06-01 22:15:14 +00:00
David Anderson e777d62672 - scheduler: the multiplier for both #jobs/day and #jobs in progress
is #CPUs + config.cuda_multiplier * #GPUs


svn path=/trunk/boinc/; revision=18085
2009-05-12 23:23:47 +00:00
David Anderson 6262401394 - feeder: add -appids option: lets you specify which apps to
get jobs for (default it all).
    Useful if you're mixing locality and regular scheduling.
- a little E@h-specific stuff
From Bernd Machenschalk.


svn path=/trunk/boinc/; revision=18039
2009-05-06 21:52:50 +00:00
David Anderson 84afd18450 - scheduler: move app-version selection and score-based scheduling
to new files.

svn path=/trunk/boinc/; revision=17630
2009-03-19 16:35:35 +00:00
David Anderson 04cdfe9cab - scheduler and web: add a project preference for whether to use the CPU.
This complements the "use GPU?" pref.
    Neither should be necessary, but what the heck.

svn path=/trunk/boinc/; revision=17628
2009-03-18 21:14:44 +00:00
David Anderson 31ea2eee98 - scheduler: app_plan() is not the right place to check for
"don't use GPU" pref; do it at a higher level


svn path=/trunk/boinc/; revision=17602
2009-03-16 23:22:54 +00:00
David Anderson 2eceda0180 - scheduler: in get_app_version() we check if our current best
app version is a CUDA app and we don't need more CUDA work.
    Need to do this for CPU as well.

svn path=/trunk/boinc/; revision=17598
2009-03-16 15:53:35 +00:00
David Anderson 41ed82f791 - scheduler: fix bugs that caused only 1 job to be sent
svn path=/trunk/boinc/; revision=17555
2009-03-07 01:00:05 +00:00
David Anderson c22b62f25b - scheduler: fix bugs in support for anonymous platform + coprocs
(app versions don't have a <coprocs> around coproc elements,
    may an oversight but let's stick with it).
    Anyway, I think it's working now.
- lib: remove "owner" array from COPROC.
    This was used in client to keep track of assignment of
    coprocessors to tasks, but we got rid of the reserve/free scheme.
    NOTE: this breaks the mechanism for passing --device N to apps;
    I'll have to do this another way.  Stay tuned.

svn path=/trunk/boinc/; revision=17543
2009-03-06 22:21:47 +00:00
David Anderson 60789a105f - scheduler: message tweaks
svn path=/trunk/boinc/; revision=17520
2009-03-06 00:20:27 +00:00
David Anderson efbe5f616f - scheduler: move all send-work setup stuff (including messages)
into a function that's called before resend_lost_results()

svn path=/trunk/boinc/; revision=17515
2009-03-05 23:08:53 +00:00
David Anderson 012bf4c696 - scheduler: get work request parameters before resend_lost_jobs();
otherwise get NaNs for CPU fraction, etc.
- scheduler: show reasons in English when send job aborts


svn path=/trunk/boinc/; revision=17514
2009-03-05 22:12:21 +00:00
David Anderson e6f3027567 - scheduler: add support for anonymous-platform coproc apps.
Old: although the request message contained all info
        about the app version (flops, coproc usage etc.)
        the server ignored this info,
        and assumed that all anonymous platform apps where CPU.
        With 6.6 client, this could produce infinite work fetch:
        - client uses anon platform, has coproc app
        - client has idle CPU, requests CPU work
        - scheduler sends it jobs, thinking they will be done by CPU app
        - client asks for more work etc.
    New: scheduler parses full info on anon platform app versions:
        plan class, FLOPS, coprocs.
        It uses this info to make scheduling decisions;
        in particular, if the request is for CUDA work,
        if will only send jobs that use a CUDA app version.
        The <result> records it returns contain info
        (plan_class) that tells the client which app_version to use.
    This will work correctly even if the client has multiple app versions
    for the same app (e.g., a CPU version and a GPU version)


svn path=/trunk/boinc/; revision=17506
2009-03-05 17:30:10 +00:00
David Anderson 8544b20886 - client: reorganize and improve the logic for deciding
when to do a scheduler RPC:
    if user request or acct mgr request, ignore backoff and suspend via GUI;
    in all other cases honor both of these.

svn path=/trunk/boinc/; revision=17504
2009-03-05 00:10:16 +00:00
David Anderson d8a60fbf37 - scheduler: don't need to set wreq->infeasible_only in send_work()
svn path=/trunk/boinc/; revision=17446
2009-03-03 16:46:47 +00:00
David Anderson 33d5a81cf6 - scheduler: add locality_scheduling arg to add_result_to_reply();
eliminate the need to diddle around with config.locality_scheduling.

svn path=/trunk/boinc/; revision=17445
2009-03-03 16:38:54 +00:00
David Anderson aadf813336 - scheduler/feeder: add <locality_scheduler_fraction> option;
lets you intermix locality and job-cache scheduling
    From Bernd M.

svn path=/trunk/boinc/; revision=17429
2009-03-03 00:12:55 +00:00
David Anderson 2d707927ab - scheduler: replace choose_download_url_by_timezone with
replace_download_url_by_timezone.


svn path=/trunk/boinc/; revision=17427
2009-03-02 23:47:11 +00:00
David Anderson dcc3bbe36f - scheduler: slight code cleanup
svn path=/trunk/boinc/; revision=17395
2009-02-26 03:03:35 +00:00
Eric J. Korpela 8f3abcc835 - Added checks for net/*.h, arpa/*.h, netinet/*.h and code to figure out
which of those files to include
    - Modified MAC address check to work on some non-Linux unixes.
      (mac_address.cpp)
    - Added suggested change to "already attached to project" checking.
      (ProjectInfoPage.cpp)
    - changed includes of standard c header files to their c++ equivalents
      (i.e. replaced <stdio.h> with <cstdio>) for namespace protection.
    - replaced "using namespace std;" with more explicit "using std::function" in
      several files.
    - Fixed bug in checking whether the os is OS/2 and added conditional OS_OS2
      to the build environment. (boinc_platform.m4,configure.ac)
    - Changed build environment to not use -nostandardlibs unless we are using
      G++ and static linkage is specified. (configure.ac)
    - Added makefiles and package building files for solaris CSW package manager.
    - Fixed bug with attempting to find login name using logname. (configure.ac)
    - Added ifdef HAVE_* protection around some include files commonly found in
      sys.
    - Added support for unified binary for x86_64/i686-pc-solaris.
      (cs_platforms.cpp)
    - generate_host_cpid() now uses MAC address on non-linux unix.
      (hostinfo_network.cpp)
    - Macro BOINC_SET_COMPILE_FLAGS now doesn't check gcc only flags on non-gcc
      compilers. (boinc_set_compile_flags.m4)
    - Library compiles no longer depend upon the library extension or require
      the library to be prefixed with lib.
    - More fixes for fcgi builds.
    - Added declaration of "struct ether_addr" and ether_ntoa().  Have not yet
      implemented ether_ntoa() for machines that don't have it, or where it is
      buggy.  (unix_util.h)
    - Added FCGI::perror() which calls FCGI_perror(). (boinc_fcgi.{h,cpp})
    - Fixed library Makefiles so that all required headers get installed.


svn path=/trunk/boinc/; revision=17388
2009-02-26 00:23:23 +00:00
David Anderson 20feef33f2 - fix possible crash on state file parse failures
svn path=/trunk/boinc/; revision=17371
2009-02-25 15:34:51 +00:00
David Anderson 15e9a87315 - client: add a debug flag <dcf_debug> for seeing changes in DCF
- scheduler: add a limit on NVIDIA driver version for CUDA apps
    (default 17500)

svn path=/trunk/boinc/; revision=17277
2009-02-17 03:16:25 +00:00
David Anderson 3452a0e422 - scheduler: show message whether miss or meet deadline
svn path=/trunk/boinc/; revision=17088
2009-01-30 22:30:35 +00:00
David Anderson b7a2c227ca - Work fetch / scheduler:
There are two mechanisms to prevent the scheduler from
    sending jobs that won't finish by their deadline.
    Simple mechanism:
        The client sends the interval x for which CPUs are projected
        to be saturated.
        Given a job with estimated duration y,
        the scheduler doesn't send it if x + y exceeds the delay bound.
        If it does send it, x is incremented by y.
    Complex mechanism:
        Client sends workload description.
        Scheduler does EDF simulation, sees if deadlines are missed.
        The only project using this AFAIK is BOINC alpha test.
    Neither of these mechanisms takes coprocessors into account,
    and as a result jobs could be sent that are doomed to
    miss their deadline.
    This checkin adds coprocessor awareness to the Simple mechanism.

    Changes:
    Client:
        compute estimated delay (i.e. time until non-saturation)
        for coprocessors as well as CPU.
        Send them in scheduler request as part of coproc descriptor.
    Scheduler:
        Keep track of estimated delays separately for different resources
- client: fixed bug that computed CPU estimated delay incorrectly
- client: the work request (req_secs) for a resource is the min
    of the project's share and the shortfall.

svn path=/trunk/boinc/; revision=17086
2009-01-30 21:25:24 +00:00
David Anderson 592bf5c17b - scheduler: get effective RAM sizes and running fraction just once
svn path=/trunk/boinc/; revision=17072
2009-01-29 20:42:45 +00:00
David Anderson 195c0403b7 - scheduler: don't count host as reliable if avg_turnaround is zero
- client: restore notion of overworked;
    if a project is overworked for a resource R,
    don't fetch work for R unless there are idle instances

svn path=/trunk/boinc/; revision=17057
2009-01-28 04:58:01 +00:00
David Anderson dfa1996975 - scheduler: in get_app_version(), if we previously sent a CUDA app,
but we don't need to send any more CUDA jobs,
    delete the BEST_APP_VERSION record and look for another app version.
    This lets the scheduler send both CUDA and CPU app versions
    for a given app in a single RPC.


svn path=/trunk/boinc/; revision=17051
2009-01-27 21:18:06 +00:00
David Anderson d23d481c63 - client: change the LTD policy so that
1) net adjustment for eligible projects is zero;
    2) max LTD is zero
- scheduler: fix msgs so disk size is shown in GB

svn path=/trunk/boinc/; revision=17031
2009-01-27 18:57:27 +00:00
David Anderson 574d1fe087 - client: don't request work for a resource if it has no shortfall.
- client and server: get rid of coproc_cuda global.

svn path=/trunk/boinc/; revision=17019
2009-01-26 05:00:49 +00:00
David Anderson db8f15e396 - scheduler: if anonymous platform, ignore coprocessor requests
(since anonymous platforms apps are treated as CPU)

svn path=/trunk/boinc/; revision=17011
2009-01-24 21:51:19 +00:00
David Anderson 44571ef4bd - scheduler: if client can't do plan class, don't complain about old client
svn path=/trunk/boinc/; revision=17005
2009-01-23 23:32:52 +00:00
David Anderson 4cd3c530b0 - scheduler: reduce frequency of calls to work_needed()
svn path=/trunk/boinc/; revision=17003
2009-01-23 22:52:35 +00:00
David Anderson fc8c9c7ce4 - sched: remove spurious error msg when using NCI class
svn path=/trunk/boinc/; revision=17000
2009-01-23 19:36:17 +00:00
David Anderson 50405c89e3 - scheduler: improve no-work messages
- web: don't use DB conn in mysql_real_escape_string()
    (otherwise won't work if DB is down)

svn path=/trunk/boinc/; revision=16961
2009-01-20 21:31:13 +00:00
David Anderson 85a8e6a772 - scheduler: remove the config flag <have_cuda_apps>,
and add <cuda_multiplier>.
    The latter is used in calculating max jobs/day for a host;
    namely, it's host.max_results_day * (NCPUS + NCUDA*cuda_multiplier).
    Set it to 10 or so if you have CUDA apps.
- scheduler: don't overload effective_ncpus();
    instead, add two new functions,
    max_results_day_multiplier() and max_wus_in_progress_multiplier()
- scheduler: don't reduce max_results_day if we get an aborted job
    (it might have been aborted by the project;
    not appopriate to punish host in this case)

svn path=/trunk/boinc/; revision=16959
2009-01-20 00:54:16 +00:00
David Anderson 91e120b3f4 - scheduler: improve message formatting; add <debug_locality> flag
for locality scheduling messages

svn path=/trunk/boinc/; revision=16921
2009-01-15 20:23:20 +00:00