Commit Graph

105 Commits

Author SHA1 Message Date
Rytis Slatkevičius f239587bdb Sched: config option not to store stderr_out if exit_status==0 (to save on DB size). With help from Nicolas Alvarez.
svn path=/trunk/boinc/; revision=18528
2009-06-30 18:00:58 +00:00
David Anderson 2e5d9bd778 - scheduler: add new config option <max_wus_in_progress_gpus>.
The limit on jobs in progress is now
        max_wus_in_progress * NCPUS
        + max_wus_in_progress * NGPUS
    where NCPUS and NGPUS reflect prefs and are capped.
    Furthermore: if the client reports plan class for in-progress jobs
    (see checkin of 31 May 2009)
    then these limits are enforced separately;
    i.e. the # of in-progress CPU jobs is <= max_wus_in_progress*NCPUS,
    and the # of in-progress GPU jobs is <= max_wus_in_progress_gpu*NGPUS
- scheduler config: rename <cuda_multiplier> to <gpu_multiplier>
- scheduler: <max_wus_to_send> is now scaled by
    (NCPUS + gpu_multiplier*NGPUS)
- scheduler: don't keep scanning array if !work_needed()
- scheduler: moved array-scan logic from sched_send.cpp to sched_array.cpp
- scheduler: don't say "no work available" if jobs are available
    but work_needed() is initially false


svn path=/trunk/boinc/; revision=18255
2009-06-01 22:15:14 +00:00
David Anderson c2fda4db09 - scheduler: add <report_max> config parameter;
limits the # of completed results handled per scheduler RPC.
    This may be needed to avoid crashes due to memory allocation
    failure (each reported result uses about 128KB memory).
- web: In showing result lists,
    include "Validate error" results in the "Invalid" category.
    (Previously they didn't appear in any category)

svn path=/trunk/boinc/; revision=18104
2009-05-14 19:01:40 +00:00
David Anderson 12eb6057e5 - client, Mac: don't do res_init(). It causes a crash.
- client (Unix): if client crashes while benchmark processes are going,
    make sure they detect this and exit.
- back-end programs: remove hardwired assumptions about
    what directory they run in, and hence where config.xml is.
    E.g., daemons look for it in "..", others expect it in current dir.
    New approach: all the programs look for the project dir as follows:
    1) the environment var BOINC_PROJECT_DIR, if defined
    2) the current dir, if config.xml is there.
    3) else ".."
    This means you can run programs in either proj/bin/ or proj/,
    or (using BOINC_PROJECT_DIR) you can keep executables
    outside of the project dir.


svn path=/trunk/boinc/; revision=18042
2009-05-07 13:54:51 +00:00
David Anderson 41ed82f791 - scheduler: fix bugs that caused only 1 job to be sent
svn path=/trunk/boinc/; revision=17555
2009-03-07 01:00:05 +00:00
David Anderson 66ec889431 - scheduler: add <locality_scheduling_sticky_file>
and <locality_scheduling_workunit_file> options
    From Bernd M.

svn path=/trunk/boinc/; revision=17431
2009-03-03 00:25:41 +00:00
David Anderson aadf813336 - scheduler/feeder: add <locality_scheduler_fraction> option;
lets you intermix locality and job-cache scheduling
    From Bernd M.

svn path=/trunk/boinc/; revision=17429
2009-03-03 00:12:55 +00:00
David Anderson 2d707927ab - scheduler: replace choose_download_url_by_timezone with
replace_download_url_by_timezone.


svn path=/trunk/boinc/; revision=17427
2009-03-02 23:47:11 +00:00
Eric J. Korpela 8f3abcc835 - Added checks for net/*.h, arpa/*.h, netinet/*.h and code to figure out
which of those files to include
    - Modified MAC address check to work on some non-Linux unixes.
      (mac_address.cpp)
    - Added suggested change to "already attached to project" checking.
      (ProjectInfoPage.cpp)
    - changed includes of standard c header files to their c++ equivalents
      (i.e. replaced <stdio.h> with <cstdio>) for namespace protection.
    - replaced "using namespace std;" with more explicit "using std::function" in
      several files.
    - Fixed bug in checking whether the os is OS/2 and added conditional OS_OS2
      to the build environment. (boinc_platform.m4,configure.ac)
    - Changed build environment to not use -nostandardlibs unless we are using
      G++ and static linkage is specified. (configure.ac)
    - Added makefiles and package building files for solaris CSW package manager.
    - Fixed bug with attempting to find login name using logname. (configure.ac)
    - Added ifdef HAVE_* protection around some include files commonly found in
      sys.
    - Added support for unified binary for x86_64/i686-pc-solaris.
      (cs_platforms.cpp)
    - generate_host_cpid() now uses MAC address on non-linux unix.
      (hostinfo_network.cpp)
    - Macro BOINC_SET_COMPILE_FLAGS now doesn't check gcc only flags on non-gcc
      compilers. (boinc_set_compile_flags.m4)
    - Library compiles no longer depend upon the library extension or require
      the library to be prefixed with lib.
    - More fixes for fcgi builds.
    - Added declaration of "struct ether_addr" and ether_ntoa().  Have not yet
      implemented ether_ntoa() for machines that don't have it, or where it is
      buggy.  (unix_util.h)
    - Added FCGI::perror() which calls FCGI_perror(). (boinc_fcgi.{h,cpp})
    - Fixed library Makefiles so that all required headers get installed.


svn path=/trunk/boinc/; revision=17388
2009-02-26 00:23:23 +00:00
David Anderson 85a8e6a772 - scheduler: remove the config flag <have_cuda_apps>,
and add <cuda_multiplier>.
    The latter is used in calculating max jobs/day for a host;
    namely, it's host.max_results_day * (NCPUS + NCUDA*cuda_multiplier).
    Set it to 10 or so if you have CUDA apps.
- scheduler: don't overload effective_ncpus();
    instead, add two new functions,
    max_results_day_multiplier() and max_wus_in_progress_multiplier()
- scheduler: don't reduce max_results_day if we get an aborted job
    (it might have been aborted by the project;
    not appopriate to punish host in this case)

svn path=/trunk/boinc/; revision=16959
2009-01-20 00:54:16 +00:00
David Anderson 91e120b3f4 - scheduler: improve message formatting; add <debug_locality> flag
for locality scheduling messages

svn path=/trunk/boinc/; revision=16921
2009-01-15 20:23:20 +00:00
David Anderson 14f6f9a386 - scheduler: add <ignore_dcf> option;
use this temporarily when you've fixed FLOPS estimate


svn path=/trunk/boinc/; revision=16672
2008-12-12 17:03:54 +00:00
David Anderson 5039207e2c - scheduler: add <have_cuda_apps> config flag.
If set the "effective NCPUS" (which is used to scale
    daily_result_quota and max_wus_in_progress)
    is max'd with the # of CUDA GPUs.

svn path=/trunk/boinc/; revision=16246
2008-10-21 23:16:07 +00:00
David Anderson bb9d546a02 - scheduler: add <no_vista_sandbox> option.
If set, don't send work to sandboxed Vista clients
    (e.g., because of CUDA issue)

svn path=/trunk/boinc/; revision=16105
2008-10-01 19:48:52 +00:00
David Anderson 4ad5249bf2 - scheduler: various bug fixes in score-based schedule;
get rid of no_darwin_6 option


svn path=/trunk/boinc/; revision=16015
2008-09-17 23:35:16 +00:00
David Anderson cc7d507789 - DB interface: in update(), check that 1 row was updated
- API: in APP_INIT_DATA, enclose project preferences in tags
    so that it's legal XML
- scheduler: add <multiple_clients_per_host> option.
    Use this if your project runs on Condor or grids
    and (to use multicore machines) you're running
    multiple clients per host.
    This will skip the host lookup based on IP address.


svn path=/trunk/boinc/; revision=15954
2008-09-04 08:33:21 +00:00
David Anderson 53ccd10f13 - scheduler: add <debug_resend> config option to enable messages
about job resending

svn path=/trunk/boinc/; revision=15889
2008-08-19 03:00:17 +00:00
David Anderson b5b3fd43b7 - scheduler: make credit_multiplier stuff conditional on
<use_credit_multiplier> flag in config.xml

svn path=/trunk/boinc/; revision=15766
2008-08-06 23:30:22 +00:00
David Anderson 4f66bb4c95 - added copyright and license info to .C, .cpp, .h files
- scheduler: fix bug in adaptive replication:
    if send an unreplicated job to untrusted host,
    set both wu.target_nresults and wu.min_quorum to app.target_nresults.

svn path=/trunk/boinc/; revision=15762
2008-08-06 18:36:30 +00:00
David Anderson 007b3ba9dd - server compile fix for gcc 4.3
svn path=/trunk/boinc/; revision=15647
2008-07-21 22:29:10 +00:00
David Anderson 16b1305db7 - server code: at some point I made a global var "SCHED_CONFIG config",
mostly so that the parse function could assume
    that everything was initially zero.
    However, various back-end functions pass around SCHED_CONFIG&
    as an argument (also named "config").
    This creates a shadow, which is always bad.
    Worse is the possibility that some projects have back-end programs
    that have a SCHED_CONFIG variable that's automatic,
    and therefore isn't zero initially,
    and therefore isn't parsing correctly.

    To fix this, I changed the 2 vectors in SCHED_CONFIG into pointers,
    and have the parse routine zero the structure.
    I was tempted to remove the SCHED_CONFIG& args to back-end functions,
    but this would have broken some projects' code.
    I did, however, change the name from config to config_loc
    to avoid shadowing.

    Also fixed various other compiler warnings.

svn path=/trunk/boinc/; revision=15541
2008-07-02 17:24:53 +00:00
David Anderson 49eb1246cf - scheduler: added
- config option <matchmaker> for matchmaker scheduling
    - config options <mm_min_slots>, <mm_max_slots>, <job_size_matching>
        to control matchmaker scheduling
- scheduler: tweaks to matchmaker scheduling from Kevin Reed
- web: fixes to alternative stylesheet from Simek

svn path=/trunk/boinc/; revision=15281
2008-05-23 16:13:30 +00:00
David Anderson e6a5d0cbfb - scheduler: add new log flags debug_edf_sim_workload, debug_edf_sim_details
for getting info on EDF simulation;
    change output from seconds to hours
- API: remove extern "C" from graphics API
    (convince me it's needed before restoring)

svn path=/trunk/boinc/; revision=15148
2008-05-08 17:48:40 +00:00
David Anderson 05f703559f - scheduler: add preliminary support for "job size matching"
(attempt to send big jobs to fast hosts, small jobs to slow hosts).
    - have "census" compute mean/stdev of host speeds,
        write it to a file perf_info.txt
    - have feeder compute mean/stdev of sizes of jobs in shmem
    - have feeder read perf_info.txt into shmem
- scheduler: add some debugging messages for app version selection
- Add LGPL license to a few files
- upgrade/setup scripts: copy census to bin/


svn path=/trunk/boinc/; revision=15136
2008-05-06 19:53:49 +00:00
David Anderson 6e6fab3e7c - scheduler: clean up message log.
Merge redundant messages.
    Condition various messages on config flags.
- client (Unix) fix to CUDA detection if LD_LIBRARY_PATH is ""

svn path=/trunk/boinc/; revision=15122
2008-05-02 17:48:29 +00:00
David Anderson fbabb7cee7 - web: tweaks to host list
- scheduler: condition lots of log file writes on config flags
    (i.e. divide "debug" output into a bunch of categories, individually selectable)

svn path=/trunk/boinc/; revision=15101
2008-04-26 23:34:38 +00:00
David Anderson 5fd6c676cf - scheduler: fix bug where scheduler sends a WU when
an app version is not available for that platform

svn path=/trunk/boinc/; revision=15088
2008-04-23 23:34:26 +00:00
David Anderson ee0ca3973c - scheduler: add "distinct_beta_apps" option;
lets users filter out beta apps as well as others
    (from Nicolas Maire)

svn path=/trunk/boinc/; revision=14971
2008-03-27 21:39:02 +00:00
David Anderson 4e9fbac5e0 - admin web: touch reread_db in manage_app_versions.php
- DB code: remove "is_high_priority" stuff.
- scheduler: merge find_app_version() into get_app_version().
    Have the latter memoize its results (both positive and negative).
    Have it call app_plan() for apps with nonempty plan_class.
- scheduler: first steps towards improved selectability of log messages.
    It will eventually be like the client,
    where you can select among various types of messages.
- feeder: if can't unlink the reread_db trigger file, exit
    (else we'd go into an infinite loop)

svn path=/trunk/boinc/; revision=14940
2008-03-18 21:22:44 +00:00
David Anderson 95772cba77 - removed boinc_ncpus_available() and boinc_nthreads() calls.
The design has been changed to constant #threads per app version
    Various changes from Kevin Reed/WCG:
    - server: add workunit.rsc_bandwidth_bound: if nonzero,
        send this WU only to hosts with that much download bandwidth
    - assimilators: if a handler returns DEFER_ASSIMILATION,
        the WU remains in INIT state and will be handled when the
        next instance completes.
        Useful if you want the assimilator to see all instances.
    - scheduler: when setting result.outcome = DETACHED,
        set received_time to now
    - scheduler: removed the reliable_time and reliable_min_avg_credit
        options
    - scheduler/web: add optional <allow_non_preferred_projects>
        in project preferences.
        If present, user will accept work from non-selected apps
        if no work is available for selected apps
    - scheduler: improved messages for projects with multiple apps
    - scheduler: added config options
        <granted_credit_weight> and <granted_credit_ramp_up>.
        Used in calculating host.claimed_credit_per_cpu_sec,
        but I'm not sure how.
    - Added two new credit-granting formulas (validate_util.C):
        stddev_credit() and two_credit()
    - server DB: add rollback_transaction() and affected_rows() to DB_CONN

    NOTE: DB update required

svn path=/trunk/boinc/; revision=14870
2008-03-07 21:13:01 +00:00
David Anderson a09e19b8dc - scheduler: add a general method for excluding hosts from job distribution.
config.xml has optional <ban_os> and <ban_cpu> elements,
    which contain regular expressions matched against
    os_name\tos_version and p_vendor\tp_model.
    If a host matches either one, it's not sent jobs.
- scheduler: fix bug in job assignment
- scheduler: initial (incompleted, commented-out) code for
    matchmaker scheduling
- server programs: declare "SCHED_CONFIG config" in sched_config.C;
    remove declarations of it from all other .C files
    (because I added a vector to it, I can no longer use memset
    to initialize it to zero; instead, it must be a global variable,
    not an automatic)

svn path=/trunk/boinc/; revision=14783
2008-02-25 18:05:04 +00:00
David Anderson 79f6720547 - scheduler: add <max_ncpus> config option.
Use this with N=1 if your application primarily uses a coprocessor

svn path=/trunk/boinc/; revision=14772
2008-02-21 20:10:10 +00:00
David Anderson 54519a4ee1 - Server: add "job assignment" feature.
Lets you assign a WU to a particular host,
    to one or all hosts belonging to a user or team, or to all hosts.
    See http://boinc.berkeley.edu/trac/wiki/AssignedWork
    Disabled unless you include <enable_assignment> in config.xml
    Uses a new DB table.
    Tested but only a little.
- Server: code cleanup; moved result-handling to a new file,
    and removed the PLATFORM_LIST arg to everything
    (put it in SCHEDULER_REQUEST instead)

svn path=/trunk/boinc/; revision=14767
2008-02-21 00:47:50 +00:00
David Anderson 2be6f8e53a - Client: add <run_apps_manually> config flag.
This is for debugging apps (currently works only in Unix).
    What it does: when running an app,
    the client does everything except actually fork/exec the app,
    i.e. it sets up the slot dir, creates shared mem segment etc.
    It then continues as if the app were actually running,
    and you can then manually run your app under a debugger
    in the slot directory.
    Note: the client won't notice the termination of your app.
- API, Unix: in situations where the timer thread wants to exit
    (e.g. it notices a missing heartbeat).
    don't directly call boinc_exit(),
    since this touches data structures that the worker thread
    may be using concurrently.
    Instead, set a flag telling the worker thread to call boinc_exit()
    (which it will do from its signal handler)
    This is an attempt to fix problems reported by Bernd;
    I haven't tested it.
- scheduler: add config flag for uploading usage data
- web: show account key and weak account key on user page
- added some code for multithread support (not finished)

api/
    boinc_api.C


svn path=/trunk/boinc/; revision=14542
2008-01-13 00:12:14 +00:00
David Anderson 300d3ed48f - scheduler: change <max_wus_in_progress> to be per CPU, not per host
svn path=/trunk/boinc/; revision=14475
2008-01-06 22:23:51 +00:00
Frank Thomas 3bfc78b511 Updated the postal address of the Free Software Foundation in all license headers. See http://lists.ssl.berkeley.edu/pipermail/boinc_dev/2007-October/008939.html for reference.
svn path=/trunk/boinc/; revision=13804
2007-10-09 11:35:47 +00:00
Bruce Allen 6b1aa1cdf2 - scheduler: added config option to control (turn off) some aspects
of file deletion. If you set file_deletion_strategy to 0 (default)
      then this turns off file deletion in the locality scheduler.
      Set file_deletion_strategy to 1 to get current behavior.  Other
      values are not defined.  I'll document this in the Wiki.

    sched/
        sched_config.h
        sched_config.C
        sched_locality.C

svn path=/trunk/boinc/; revision=13594
2007-09-17 15:58:36 +00:00
David Anderson 823df51238 - client: allow up to a day (rather than work_buf_min()) to elapsed
between completing a result and reporting it.
- back end: added <httpd_user> config option:
    Web server user name (used by file deleter)
- back end: don't report unparsed XML in config.xml as an error

client/
    app.h
    work_fetch.C
lib/
    shmem.C
sched/
    file_deleter.C
    file_upload_handler.C
    sched_config.C,h
    show_shmem.C

svn path=/trunk/boinc/; revision=13039
2007-06-27 20:09:18 +00:00
David Anderson 981eef6b14 - Scheduler: changed the <homogeneous_redundancy> option
from a bool to an int:
    0 = no HR
    1 = HR with fine-grained classification (from MF Somers)
    2 = HR with coarse-grained classification (from WCG)

    Note: numerical equivalence depends on your application
    and how it's compiled.  Some apps may need classifications
    other than these two.  Contact me if so.

sched/
    sched_hr.C
    sched_config.C,h

svn path=/trunk/boinc/; revision=12934
2007-06-14 15:05:20 +00:00
David Anderson df5909d473 - scheduler: add no_darwin_6 option
(don't send work to Mac OS X 10.2 or earlier).
    This used to be #ifdef in by Einstein@home.
- scheduler: add no_amd_k6 option
    (don't send work to AMD K6 CPUs)

sched/
    handle_request.C
    sched_config.C,h

svn path=/trunk/boinc/; revision=12912
2007-06-12 21:40:32 +00:00
David Anderson 1e7e58875a - client: add cmdline options --exit_before_start and --exit_after_finish.
This cause the core client to exit immediately before or after
    running a job,
    letting you examine the contents of the slot directory.
- scheduler: changed max # of CPUs used in daily_result_quota
    limit from 4 to 8, and make it a compile-time parameter
- feeder/scheduler: make the number of work items in shared
    memory configurable (in config.xml).
    The element is <shmem_work_items>
- feeder: make the size of the work item query configurable
    (<feeder_query_size)
- feeder: remove code related to removing infeasible results
    from shared mem.
    This mechanism was never needed,
    and I think a timeout would accomplish the same effect.

client/
    app.C
    app_start.C
    client_state.C,h
    cs_cmdline.C
sched/
    feeder.C
    sched_array.C
    sched_config.C,h
    sched_send.C
    sched_shmem.C,h
    sched_util.C
    show_shmem.C

svn path=/trunk/boinc/; revision=12771
2007-05-29 21:45:01 +00:00
David Anderson 4f81994be6 - Add a mechanism for a project to shut down gracefully.
If <ended>1</ended> is included in config.xml,
    this gets send to clients,
    and they display a "Project ended - OK to detach" message.

client/
    client_state.C
    client_types.C,h
    scheduler_op.C
clientgui/
    VIewProjects.cpp
    VIewProjectsGrid.cpp
lib/
    gui_rpc_client.h
    gui_rpc_client_ops.C
    gui_rpc_client_print.C
sched/
    sched_config.C,h
    server_types.C

svn path=/trunk/boinc/; revision=12742
2007-05-24 16:07:21 +00:00
David Anderson 826e65f8d0 - client simulator: fix cpu_sched_rr_only bug
- boinc_cmd: canonicalize URLs
- user web: fix link to Top Teams
- back end: remove code related to alternate download dir

client/
    cpu_sched.C
    sim.C
html/
    inc/
        stats_sites.inc
    user/
        stats.php
lib/
    boinc_cmd.C
sched/
    file_deleter.C
    sched_config.C,h

svn path=/trunk/boinc/; revision=12694
2007-05-18 22:02:51 +00:00
David Anderson 01f4851323 - scheduler: add max_wus_in_progress option.
Limits total # of in-progress results per host
    (independently of #CPUs)

sched/
    sched_config.C,h
    sched_resend.C
    sched_send.C
    server_types.h


svn path=/trunk/boinc/; revision=12661
2007-05-14 15:21:38 +00:00
David Anderson a37403a673 - scheduler: add <workload_sim> config option.
If set, the scheduler will use EDF simulation,
    together with the in-progress workload reported by the client,
    to avoid sending results that
    1) will miss their deadline, or
    2) will cause an in-progress result to miss its deadline, or
    3) will make an in-progress result miss its deadline
        by more than is already predicted.
    If this option is not set, or if the client request doesn't
    include a workload description (i.e. the client is old)
    use the existing approach, which assumes there's no workload.
    NOTE: this is experimental.  Production projects should not use it.
- EDF sim: write debug stuff to stderr instead of stdout
- Account manager:
    - if an account is detach_when_done, set dont_request_more_work
    - check done_request_more_work even for first-time projects
- update_uotd: generate a file for use by Google gadget
- user_links(): use full URLs (so can use in Google gadget)

client/
    acct_mgr.C
    work_fetch.C
html/
    inc/
        uotd.inc
        util.inc
    user/
        uotd_gadget.php (new)
sched/
    Makefile.am
    edf_sim.C
    sched_config.C,h
    sched_resend.C
    sched_send.C,h
    server_types.C,h

svn path=/trunk/boinc/; revision=12639
2007-05-10 21:50:52 +00:00
David Anderson 1880a50cc5 David 19 Apr 2007
- Change things so that make_project installs a test application
        (uppercase) and the daemons to go along with it
        (sample_work_generator, sample_bitwise_validator, sample_assimilator).
        This is mostly working, not completely.

    apps
        upper_case.C
    html/ops/
        mass_email_script.php
    py/Boinc/
        setup_project.py
    sched/
        sched_config.C,h
        sample_work_generator.C
    tools/
        make_project
        project.xml
        Makefile.am

svn path=/trunk/boinc/; revision=12422
2007-04-19 22:11:25 +00:00
David Anderson 1ea3bb781e David 18 Apr 2007
- bug fixes for sample work generator

    sched/
        sched_config.C,h
        sample_work_generator.C


svn path=/trunk/boinc/; revision=12417
2007-04-19 03:12:13 +00:00
David Anderson 3431d99471 svn path=/trunk/boinc/; revision=12414 2007-04-18 20:49:58 +00:00
David Anderson 564504eff6 svn path=/trunk/boinc/; revision=12402 2007-04-17 17:12:44 +00:00
David Anderson 66071c9de1 svn path=/trunk/boinc/; revision=12373 2007-04-16 22:22:15 +00:00