Commit Graph

93 Commits

Author SHA1 Message Date
David Anderson 1d765245ed - scheduler: sweeping changes to the way job runtimes are estimated:
see http://boinc.berkeley.edu/trac/wiki/RuntimeEstimation


svn path=/trunk/boinc/; revision=21153
2010-04-08 23:14:47 +00:00
David Anderson 85e06afe4b - scheduler: app_plan() no longer has to guess how efficiently
an app version will run on a particular host.
- scheduler: fix memory leak: BEST_APP_VERSIONs weren't being freed


svn path=/trunk/boinc/; revision=21148
2010-04-08 18:27:27 +00:00
David Anderson 4462fe534b - client: don't do RSS fetch if network suspended
svn path=/trunk/boinc/; revision=21123
2010-04-06 20:32:02 +00:00
David Anderson a2a661993b - validator: -d 4 means -d 3 plus print all DB queries
(todo: do this for all daemons)
- validator: change cmdline args from -foo to --foo
    (todo: do this for all daemons)
- validator: pass max_granted_credit to assign_credit_set()

svn path=/trunk/boinc/; revision=21093
2010-04-05 18:59:16 +00:00
David Anderson fb851311e0 - server: various changes;
see http://boinc.berkeley.edu/trac/wiki/CreditNew

    Projects will need to update DB and recompile all back-end programs.

    Summary:
    - new way of computing credit
    - "reliable host" mechanism is per app version
    - "host punishment" mechanism is per app version
    - adjustment of wu.rsc_fpops_est provides the
        equivalent of per app version DCF
    - max jobs in progress is now per app
    - max jobs per RPC is now per app

    TODO:
    - reliable mechanism:
        - populate and use host_app_version.error_rate
        - populate host_app_version.turnaround
    - host punishment:
        - populate host_app_version.max_jobs_per_day
        - populate host_app_version.n_jobs_today
        - use app.max_jobs_per_day_init
    - job limits:
        - use app.max_jobs_in_progress, max_gpu_jobs_in_progress
        - use app.max_jobs_per_rpc
    - adjust wu.rsc_fpops_est
    - remove old credit stuff
        fpops_cumulative, credit_multiplier
        credit computation in scheduler

- AVERAGE class: use the Knuth algorithm (Wikipedia)


svn path=/trunk/boinc/; revision=21021
2010-03-29 22:28:20 +00:00
David Anderson 0c1a1421f8 - scheduler/feeder: if any client version number field
(min_core_version etc.) is < 10000,
    multiply it by 100 and print a warning.

svn path=/trunk/boinc/; revision=20187
2010-01-18 04:52:58 +00:00
David Anderson 737952dbb5 - server: client version numbers are represented as
10000*major + 100*minor + release,
    rather than 100*major + minor.
    Sometimes you need release-level resolution.
    This affects:
    - app_version.min_core_version
    - config: min_core_client_version_announced
    - config: min_core_client_version
    Projects using these must multiply them by 100.

svn path=/trunk/boinc/; revision=20149
2010-01-13 17:28:59 +00:00
Bernd Machenschalk fbf3ca1450 a couple of locality scheduling fixes:
- remove obsolete and buggy code from transitioner (create_result() in backend_lib)
    - account for 'mixed' scheduling in explain_to_user() in sched_send.cpp
    - finish transition to configurable patterns for distinguishing files reported by the client
      in the Einstein@home-specific part of send_work_locality in sched_locality
      (removed previous hardcoded strcmps)

svn path=/trunk/boinc/; revision=20074
2010-01-05 11:03:10 +00:00
David Anderson 56a8296b5b - scheduler: compute no_jobs_available correctly
in the presence of multiple scheduling types
    (e.g., locality and job array)
    From Nils Brause

svn path=/trunk/boinc/; revision=19559
2009-11-12 21:30:33 +00:00
David Anderson 3f084596e0 - scheduler: fix bug that caused no ATI jobs to be sent
if max_jobs_in_progress defined.

svn path=/trunk/boinc/; revision=19529
2009-11-10 22:19:36 +00:00
David Anderson 0e38392885 - scheduler: skip deadline check for non-CPU-intensive projects
svn path=/trunk/boinc/; revision=19273
2009-10-06 20:45:09 +00:00
David Anderson 71c7e7a74b - client/scheduler/web: add per-project preferences for whether
to accept CPU, NVIDIA and ATI jobs.
    These prefs are shown only where relevant:
    e.g., only for processor types for which the project has app versions,
    and if it has versions for only one type, no pref is shown.

    These prefs affect both client and scheduler.
    The client won't ask for work for a device blocked by prefs,
    and the scheduler won't send it.

    This replaces earlier optional project-specific prefs for
    "no CPU jobs" and "no GPU jobs".
    (However, these prefs continue to be honored on the server side).

- client: if NVIDIA driver is unknown, say that rather than 0


svn path=/trunk/boinc/; revision=19194
2009-09-28 04:24:18 +00:00
David Anderson 6582b63cbc - client: extra debug msgs (remove when done)
svn path=/trunk/boinc/; revision=19130
2009-09-22 23:44:58 +00:00
David Anderson a0e657ff54 - scheduler: don't scale by #CPUs for non-CPU-intensive projects
svn path=/trunk/boinc/; revision=19122
2009-09-21 19:58:21 +00:00
David Anderson da7e82fe15 - scheduler and back end: add new fields to result table:
elapsed_time: the elapsed time (runtime) as reported by client
    flops_estimate: the app's estimated FLOPS as reported by app_plan()
    app_version_id: the DB ID of the app_version used
        (or -1 if anonymous platform)
    TODO: show these in the web interfaces,
    and use them where appropriate

svn path=/trunk/boinc/; revision=19002
2009-09-03 20:26:31 +00:00
David Anderson 8b701fc73f - scheduler: fix messed-up deadline check logic.
Old:
        1) check deadline based on wu.delay_bound
        2) in add_result_to_reply(), potentially modify wu.delay_bound,
            e.g. because of retry acceleration
        problem: reducing delay bound may cause deadline miss
    New:
        1) new function get_delay_bound_range()
            (called from wu_is_infeasible_fast())
            returns optimistic and pessimistic delay bounds.
            Retry acceleration logic is here.
        2) check deadline based on optimistic bound;
            if that fails, check based on pessimistic bound.
            Set wu.delay_bound to the one that worked.
    Notes:
    - get_delay_bound_range() needs result priority and report deadline,
        and it's called before we read the full result.
        So add these items to WORK_ITEM and WU_RESULT.
    - get_delay_bound_range() could be customized for
        project-specific deadline policy.
    - add_result_to_reply() was becoming a toxic waste dump.
        Deadline-related stuff should have been factored out in any case.

svn path=/trunk/boinc/; revision=18946
2009-08-31 19:35:46 +00:00
David Anderson a49ba8c2e9 - scheduler: if request is anon platform,
write list of client's app versions to log

svn path=/trunk/boinc/; revision=18923
2009-08-26 18:21:36 +00:00
David Anderson eafb410cf8 - scheduler: simplify and fix the way that app_plan() conveys messages
to the user.  app_plan() now generates the messages directly
    rather than returning integer error codes.

svn path=/trunk/boinc/; revision=18899
2009-08-21 20:38:39 +00:00
David Anderson 9e9f2a9878 - scheduler: code cleanup
svn path=/trunk/boinc/; revision=18896
2009-08-21 19:14:15 +00:00
David Anderson 7278ab1787 - scheduler: add support for ATI GPUs
svn path=/trunk/boinc/; revision=18851
2009-08-17 17:07:38 +00:00
David Anderson b300519444 svn path=/trunk/boinc/; revision=18825 2009-08-10 04:49:02 +00:00
David Anderson e3363c7eb8 - scheduler: on second thought, it would be better to add the above
feature without requiring use of score-based scheduling.
    So add a new customizable function, wu_is_infeasible_custom(),
    where projects can put job-specific checks.

    Also, move customizable functions (of which there are now 4)
    to a new file, sched_customize.cpp.

svn path=/trunk/boinc/; revision=18767
2009-07-29 18:55:50 +00:00
David Anderson 77055d17e7 svn path=/trunk/boinc/; revision=18765 2009-07-29 18:34:27 +00:00
David Anderson 37638081fa - scheduler: improved messages for debug_send
svn path=/trunk/boinc/; revision=18761
2009-07-28 23:07:11 +00:00
David Anderson a71a03f698 - Mac install: fix things a better way
- install: don't install internal .h files
- scheduler: fix spurious "reached limit of 0 GPU tasks" message,
    slight code cleanup

svn path=/trunk/boinc/; revision=18480
2009-06-22 21:11:19 +00:00
David Anderson 3824d359a3 - scheduler: fix <max_wus_in_progress> code
svn path=/trunk/boinc/; revision=18472
2009-06-19 18:34:00 +00:00
David Anderson 9527cb6534 - install: don't install std_fixes.h or msg_queue.cpp
- remove some unused code
- scheduler: small code cleanup

svn path=/trunk/boinc/; revision=18465
2009-06-19 17:21:34 +00:00
David Anderson cc62cce8f7 - client: if scheduler request didn't request work, don't report 0 tasks
- scheduler: fix crash if anonymous platform

svn path=/trunk/boinc/; revision=18259
2009-06-02 05:12:06 +00:00
David Anderson 2e5d9bd778 - scheduler: add new config option <max_wus_in_progress_gpus>.
The limit on jobs in progress is now
        max_wus_in_progress * NCPUS
        + max_wus_in_progress * NGPUS
    where NCPUS and NGPUS reflect prefs and are capped.
    Furthermore: if the client reports plan class for in-progress jobs
    (see checkin of 31 May 2009)
    then these limits are enforced separately;
    i.e. the # of in-progress CPU jobs is <= max_wus_in_progress*NCPUS,
    and the # of in-progress GPU jobs is <= max_wus_in_progress_gpu*NGPUS
- scheduler config: rename <cuda_multiplier> to <gpu_multiplier>
- scheduler: <max_wus_to_send> is now scaled by
    (NCPUS + gpu_multiplier*NGPUS)
- scheduler: don't keep scanning array if !work_needed()
- scheduler: moved array-scan logic from sched_send.cpp to sched_array.cpp
- scheduler: don't say "no work available" if jobs are available
    but work_needed() is initially false


svn path=/trunk/boinc/; revision=18255
2009-06-01 22:15:14 +00:00
David Anderson e777d62672 - scheduler: the multiplier for both #jobs/day and #jobs in progress
is #CPUs + config.cuda_multiplier * #GPUs


svn path=/trunk/boinc/; revision=18085
2009-05-12 23:23:47 +00:00
David Anderson 6262401394 - feeder: add -appids option: lets you specify which apps to
get jobs for (default it all).
    Useful if you're mixing locality and regular scheduling.
- a little E@h-specific stuff
From Bernd Machenschalk.


svn path=/trunk/boinc/; revision=18039
2009-05-06 21:52:50 +00:00
David Anderson 84afd18450 - scheduler: move app-version selection and score-based scheduling
to new files.

svn path=/trunk/boinc/; revision=17630
2009-03-19 16:35:35 +00:00
David Anderson 04cdfe9cab - scheduler and web: add a project preference for whether to use the CPU.
This complements the "use GPU?" pref.
    Neither should be necessary, but what the heck.

svn path=/trunk/boinc/; revision=17628
2009-03-18 21:14:44 +00:00
David Anderson 31ea2eee98 - scheduler: app_plan() is not the right place to check for
"don't use GPU" pref; do it at a higher level


svn path=/trunk/boinc/; revision=17602
2009-03-16 23:22:54 +00:00
David Anderson 2eceda0180 - scheduler: in get_app_version() we check if our current best
app version is a CUDA app and we don't need more CUDA work.
    Need to do this for CPU as well.

svn path=/trunk/boinc/; revision=17598
2009-03-16 15:53:35 +00:00
David Anderson 41ed82f791 - scheduler: fix bugs that caused only 1 job to be sent
svn path=/trunk/boinc/; revision=17555
2009-03-07 01:00:05 +00:00
David Anderson c22b62f25b - scheduler: fix bugs in support for anonymous platform + coprocs
(app versions don't have a <coprocs> around coproc elements,
    may an oversight but let's stick with it).
    Anyway, I think it's working now.
- lib: remove "owner" array from COPROC.
    This was used in client to keep track of assignment of
    coprocessors to tasks, but we got rid of the reserve/free scheme.
    NOTE: this breaks the mechanism for passing --device N to apps;
    I'll have to do this another way.  Stay tuned.

svn path=/trunk/boinc/; revision=17543
2009-03-06 22:21:47 +00:00
David Anderson 60789a105f - scheduler: message tweaks
svn path=/trunk/boinc/; revision=17520
2009-03-06 00:20:27 +00:00
David Anderson efbe5f616f - scheduler: move all send-work setup stuff (including messages)
into a function that's called before resend_lost_results()

svn path=/trunk/boinc/; revision=17515
2009-03-05 23:08:53 +00:00
David Anderson 012bf4c696 - scheduler: get work request parameters before resend_lost_jobs();
otherwise get NaNs for CPU fraction, etc.
- scheduler: show reasons in English when send job aborts


svn path=/trunk/boinc/; revision=17514
2009-03-05 22:12:21 +00:00
David Anderson e6f3027567 - scheduler: add support for anonymous-platform coproc apps.
Old: although the request message contained all info
        about the app version (flops, coproc usage etc.)
        the server ignored this info,
        and assumed that all anonymous platform apps where CPU.
        With 6.6 client, this could produce infinite work fetch:
        - client uses anon platform, has coproc app
        - client has idle CPU, requests CPU work
        - scheduler sends it jobs, thinking they will be done by CPU app
        - client asks for more work etc.
    New: scheduler parses full info on anon platform app versions:
        plan class, FLOPS, coprocs.
        It uses this info to make scheduling decisions;
        in particular, if the request is for CUDA work,
        if will only send jobs that use a CUDA app version.
        The <result> records it returns contain info
        (plan_class) that tells the client which app_version to use.
    This will work correctly even if the client has multiple app versions
    for the same app (e.g., a CPU version and a GPU version)


svn path=/trunk/boinc/; revision=17506
2009-03-05 17:30:10 +00:00
David Anderson 8544b20886 - client: reorganize and improve the logic for deciding
when to do a scheduler RPC:
    if user request or acct mgr request, ignore backoff and suspend via GUI;
    in all other cases honor both of these.

svn path=/trunk/boinc/; revision=17504
2009-03-05 00:10:16 +00:00
David Anderson d8a60fbf37 - scheduler: don't need to set wreq->infeasible_only in send_work()
svn path=/trunk/boinc/; revision=17446
2009-03-03 16:46:47 +00:00
David Anderson 33d5a81cf6 - scheduler: add locality_scheduling arg to add_result_to_reply();
eliminate the need to diddle around with config.locality_scheduling.

svn path=/trunk/boinc/; revision=17445
2009-03-03 16:38:54 +00:00
David Anderson aadf813336 - scheduler/feeder: add <locality_scheduler_fraction> option;
lets you intermix locality and job-cache scheduling
    From Bernd M.

svn path=/trunk/boinc/; revision=17429
2009-03-03 00:12:55 +00:00
David Anderson 2d707927ab - scheduler: replace choose_download_url_by_timezone with
replace_download_url_by_timezone.


svn path=/trunk/boinc/; revision=17427
2009-03-02 23:47:11 +00:00
David Anderson dcc3bbe36f - scheduler: slight code cleanup
svn path=/trunk/boinc/; revision=17395
2009-02-26 03:03:35 +00:00
Eric J. Korpela 8f3abcc835 - Added checks for net/*.h, arpa/*.h, netinet/*.h and code to figure out
which of those files to include
    - Modified MAC address check to work on some non-Linux unixes.
      (mac_address.cpp)
    - Added suggested change to "already attached to project" checking.
      (ProjectInfoPage.cpp)
    - changed includes of standard c header files to their c++ equivalents
      (i.e. replaced <stdio.h> with <cstdio>) for namespace protection.
    - replaced "using namespace std;" with more explicit "using std::function" in
      several files.
    - Fixed bug in checking whether the os is OS/2 and added conditional OS_OS2
      to the build environment. (boinc_platform.m4,configure.ac)
    - Changed build environment to not use -nostandardlibs unless we are using
      G++ and static linkage is specified. (configure.ac)
    - Added makefiles and package building files for solaris CSW package manager.
    - Fixed bug with attempting to find login name using logname. (configure.ac)
    - Added ifdef HAVE_* protection around some include files commonly found in
      sys.
    - Added support for unified binary for x86_64/i686-pc-solaris.
      (cs_platforms.cpp)
    - generate_host_cpid() now uses MAC address on non-linux unix.
      (hostinfo_network.cpp)
    - Macro BOINC_SET_COMPILE_FLAGS now doesn't check gcc only flags on non-gcc
      compilers. (boinc_set_compile_flags.m4)
    - Library compiles no longer depend upon the library extension or require
      the library to be prefixed with lib.
    - More fixes for fcgi builds.
    - Added declaration of "struct ether_addr" and ether_ntoa().  Have not yet
      implemented ether_ntoa() for machines that don't have it, or where it is
      buggy.  (unix_util.h)
    - Added FCGI::perror() which calls FCGI_perror(). (boinc_fcgi.{h,cpp})
    - Fixed library Makefiles so that all required headers get installed.


svn path=/trunk/boinc/; revision=17388
2009-02-26 00:23:23 +00:00
David Anderson 20feef33f2 - fix possible crash on state file parse failures
svn path=/trunk/boinc/; revision=17371
2009-02-25 15:34:51 +00:00
David Anderson 15e9a87315 - client: add a debug flag <dcf_debug> for seeing changes in DCF
- scheduler: add a limit on NVIDIA driver version for CUDA apps
    (default 17500)

svn path=/trunk/boinc/; revision=17277
2009-02-17 03:16:25 +00:00