Commit Graph

144 Commits

Author SHA1 Message Date
David Anderson 4f66bb4c95 - added copyright and license info to .C, .cpp, .h files
- scheduler: fix bug in adaptive replication:
    if send an unreplicated job to untrusted host,
    set both wu.target_nresults and wu.min_quorum to app.target_nresults.

svn path=/trunk/boinc/; revision=15762
2008-08-06 18:36:30 +00:00
David Anderson 848a8041aa - back-end code: fix compiler warnings (and maybe bugs);
to some retentive code cleanup


svn path=/trunk/boinc/; revision=15497
2008-06-26 21:20:07 +00:00
David Anderson 96eabc043b - server: added -simulation option to feeder, validator, and assimilator
(for use in server simulator)

svn path=/trunk/boinc/; revision=15344
2008-06-01 03:43:47 +00:00
David Anderson c25344e343 - fix FCGI compile
svn path=/trunk/boinc/; revision=15265
2008-05-20 22:36:43 +00:00
David Anderson 05f703559f - scheduler: add preliminary support for "job size matching"
(attempt to send big jobs to fast hosts, small jobs to slow hosts).
    - have "census" compute mean/stdev of host speeds,
        write it to a file perf_info.txt
    - have feeder compute mean/stdev of sizes of jobs in shmem
    - have feeder read perf_info.txt into shmem
- scheduler: add some debugging messages for app version selection
- Add LGPL license to a few files
- upgrade/setup scripts: copy census to bin/


svn path=/trunk/boinc/; revision=15136
2008-05-06 19:53:49 +00:00
David Anderson 13400c9516 Changes for multithread app support:
- update_versions: use __ (not :) as separator for plan class
- client: add plan_class to APP_VERSION;
    an app version is now identified by platform/version/plan_class
- client CPU scheduler: don't assume apps use 1 CPU
- client: add avg_ncpus, max_cpus, flops, cmdline to RESULT
- scheduler: implement app planning scheme

Other changes:

- client: if symlink() fails, make a XML soft link instead
    (for Unix running off a FAT32 FS)
- client: don't accept nonpositive resource share from AMS
- daemons and DB: check for error returns from enumerations,
    and exit if so.  Thus, if the MySQL server goes down,
    all the daemons will soon exit.
    The cron script will restart them every 5 min,
    so when the DB server comes back up so will the project.
- web: show empty max CPU % as ---
- API: get rid of all_threads_cpu_time option (always the case now)


svn path=/trunk/boinc/; revision=14966
2008-03-27 18:25:29 +00:00
David Anderson 4e9fbac5e0 - admin web: touch reread_db in manage_app_versions.php
- DB code: remove "is_high_priority" stuff.
- scheduler: merge find_app_version() into get_app_version().
    Have the latter memoize its results (both positive and negative).
    Have it call app_plan() for apps with nonempty plan_class.
- scheduler: first steps towards improved selectability of log messages.
    It will eventually be like the client,
    where you can select among various types of messages.
- feeder: if can't unlink the reread_db trigger file, exit
    (else we'd go into an infinite loop)

svn path=/trunk/boinc/; revision=14940
2008-03-18 21:22:44 +00:00
David Anderson 8098622210 - Validator framework: remove some consts, and other changes,
to allow validator to assign different credit
    to different instances of a job
- Scheduler: if can't open DB, return <project_is_down/>
    (fixes #578)
- clean up logic of modify_claimed_credit
- feeder: for -priority_order_create_time, use workunitid
    rather than create time (faster for the DB)
from Kevin Reed

svn path=/trunk/boinc/; revision=14908
2008-03-13 23:35:13 +00:00
David Anderson 95772cba77 - removed boinc_ncpus_available() and boinc_nthreads() calls.
The design has been changed to constant #threads per app version
    Various changes from Kevin Reed/WCG:
    - server: add workunit.rsc_bandwidth_bound: if nonzero,
        send this WU only to hosts with that much download bandwidth
    - assimilators: if a handler returns DEFER_ASSIMILATION,
        the WU remains in INIT state and will be handled when the
        next instance completes.
        Useful if you want the assimilator to see all instances.
    - scheduler: when setting result.outcome = DETACHED,
        set received_time to now
    - scheduler: removed the reliable_time and reliable_min_avg_credit
        options
    - scheduler/web: add optional <allow_non_preferred_projects>
        in project preferences.
        If present, user will accept work from non-selected apps
        if no work is available for selected apps
    - scheduler: improved messages for projects with multiple apps
    - scheduler: added config options
        <granted_credit_weight> and <granted_credit_ramp_up>.
        Used in calculating host.claimed_credit_per_cpu_sec,
        but I'm not sure how.
    - Added two new credit-granting formulas (validate_util.C):
        stddev_credit() and two_credit()
    - server DB: add rollback_transaction() and affected_rows() to DB_CONN

    NOTE: DB update required

svn path=/trunk/boinc/; revision=14870
2008-03-07 21:13:01 +00:00
David Anderson b20f2d915c - server: added a mechanism for submitting jobs as:
> boinc_submit --infile foo --outfile blah program --args
    This will run "program" on a remote host,
    with the given input, output files, and command-line args

    It manages everything for you.
    You don't have to worry about apps, app_versions,
    template files, wrapper job files, etc.
    See: http://boinc.berkeley.edu/trac/wiki/SingleJob

    Initial checkin - not debugged yet.

svn path=/trunk/boinc/; revision=14842
2008-03-04 23:50:38 +00:00
David Anderson 1106da6ec6 - transitioner: we're not supposed to ever see "assigned" WUs.
But if we do, set their transitioner time to plus infinity
    so that we don't see them again.
    (otherwise we go into an infinite loop)
- DB code: remove "high_priority" from queries not from scheduler
    (should probably remove them from there too)
- file_deleter: print error msg if apache user doesn't exist

svn path=/trunk/boinc/; revision=14835
2008-03-03 19:33:32 +00:00
David Anderson c8384113e4 - changes to many files to fix gcc4.3 compile
svn path=/trunk/boinc/; revision=14811
2008-02-27 23:26:38 +00:00
David Anderson a09e19b8dc - scheduler: add a general method for excluding hosts from job distribution.
config.xml has optional <ban_os> and <ban_cpu> elements,
    which contain regular expressions matched against
    os_name\tos_version and p_vendor\tp_model.
    If a host matches either one, it's not sent jobs.
- scheduler: fix bug in job assignment
- scheduler: initial (incompleted, commented-out) code for
    matchmaker scheduling
- server programs: declare "SCHED_CONFIG config" in sched_config.C;
    remove declarations of it from all other .C files
    (because I added a vector to it, I can no longer use memset
    to initialize it to zero; instead, it must be a global variable,
    not an automatic)

svn path=/trunk/boinc/; revision=14783
2008-02-25 18:05:04 +00:00
David Anderson 42d2abfa34 - scheduler: code cleanup, remove "SCHED_MSG_LOG::" from a zillion places
svn path=/trunk/boinc/; revision=14774
2008-02-21 21:00:58 +00:00
David Anderson 54519a4ee1 - Server: add "job assignment" feature.
Lets you assign a WU to a particular host,
    to one or all hosts belonging to a user or team, or to all hosts.
    See http://boinc.berkeley.edu/trac/wiki/AssignedWork
    Disabled unless you include <enable_assignment> in config.xml
    Uses a new DB table.
    Tested but only a little.
- Server: code cleanup; moved result-handling to a new file,
    and removed the PLATFORM_LIST arg to everything
    (put it in SCHEDULER_REQUEST instead)

svn path=/trunk/boinc/; revision=14767
2008-02-21 00:47:50 +00:00
Frank Thomas 3bfc78b511 Updated the postal address of the Free Software Foundation in all license headers. See http://lists.ssl.berkeley.edu/pipermail/boinc_dev/2007-October/008939.html for reference.
svn path=/trunk/boinc/; revision=13804
2007-10-09 11:35:47 +00:00
David Anderson bb5f54d31f - feeder/scheduler: fix bug where APP.homogeneous_redundancy
is defined as a bool instead of int
    (see its only nonzero value is 1, so the "coarse" HR type is ignored).

svn path=/trunk/boinc/; revision=13786
2007-10-05 22:32:47 +00:00
David Anderson 26a1c63b20 - feeder: add debugging feature:
if you send it a USR1 signal, it writes a summary
    of the shmem contents and HR state to the log file

svn path=/trunk/boinc/; revision=13107
2007-07-05 23:06:18 +00:00
David Anderson 7ea662932c - add new stats site
- back end: use boinc_mkdir to get permissions right

html/inc/
    stats_sites.inc
lib/
    filesys.C
sched/
    db_dump.C
    db_purge.C
    feeder.C

svn path=/trunk/boinc/; revision=13033
2007-06-26 21:14:58 +00:00
David Anderson a97556bdfd - feeder: added a new enumerator of DB_WORK_ITEM that,
on successive calls, scans through ALL the sendable
    jobs satisfying the select clause
    (it does this by ID order, so there's no order clause)
    This is used for HR, so that if a job has been committed
    to an HR class, we eventually get it.

    With extremely minimal testing, the new HR stuff seems to work.

db/
    boinc_db.C,h
sched/
    feeder.C
    sample_work_generator.C
    server_types.C

svn path=/trunk/boinc/; revision=12988
2007-06-22 23:48:37 +00:00
David Anderson c912a6506b - user web: fixed bug in private-message code
- back end: moved HR_INFO to a separate file;
    did some debugging in HR code

html/
    inc/
        forum_user.inc
        util.inc
    user/
        forum_pm.php
sched/
    Makefile.am
    census.C
    feeder.C
    hr.C,h
    hr_info.C,h (new)

svn path=/trunk/boinc/; revision=12978
2007-06-22 03:46:42 +00:00
David Anderson f5d94818dd - added "census", a program that counts up how much RAC
there is for each HR class, and writes it to a file.
    This will be used soon for HR support in the feeder.
- split the HR code into hr.C,h (stuff used by both census and scheduler)
    and sched_hr.C (stuff used only by the scheduler)
- database: change DB_CREDITED_JOB to treat workunitid
    as a double (which it is) rather than a long.
    BTW, long == int.
- fixed lots of compile warnings in the server code

db/
    boinc_db.C,h
lib/
    boinc_cmd.C
    miofile.C
    util.C
sched/
    Makefile.am
    census.C (new)
    feeder.C
    file_deleter.C
    file_upload_handler.C
    handle_request.C
    hr.C,h (new)
    main.C
    sample_assimilator.C
    sample_work_generator.C
    sched_array.C
    sched_hr.C,h
    sched_send.C
    server_types.C
    transitioner.C
    validator.C

svn path=/trunk/boinc/; revision=12970
2007-06-20 22:34:06 +00:00
David Anderson 6ad51e0452 - feeder: more code cleanup
sched/
    feeder.C


svn path=/trunk/boinc/; revision=12969
2007-06-20 20:23:07 +00:00
David Anderson 45f04aa04b - back end: make it possible to run multiple instances of make_work,
for WUs of different applications
    (need to count unsent results separately by app)
- feeder: major code cleanup
    - application interleaving (for -allapps) is now done
        by building a static slot-to-app array "app_indices".
        Fractional weights now work correctly.
    - enum sizes (for -allapps) are now precomputed
        in an array "enum_sizes"
    - rename "found" (confusing!!) to "collision"
    - swapped the names of mod_select_clause and select_clause,
        to reflect what they actually are
- file deleter: in finding oldest WU, order by id instead of create_time
    (there's no index on create_time)
- user web: show "merge by name" only to host owner
- add cpu_scheduler_period() member to GLOBAL_PREFS
    (so you don't have to multiply by 60 everywhere)
- infinite() fix for HPUX

client/
    cpu_sched.C
    cs_cmdline.C
    cs_scheduler.C
    rrsim_test.C
    sim.C
    work_fetch.C
html/user/
    hosts_user.php
lib/
    parse.h
    prefs.h
sched/
    feeder.C
    file_deleter.C
    make_work.C
    sample_work_generator.C
    sched_util.C,h
tools/
    updater.C

svn path=/trunk/boinc/; revision=12968
2007-06-20 16:27:27 +00:00
David Anderson 610d4cfa77 - client: allow suspension of non-CPU-intensive project or app
(undoes checkin of 25 Jan 2005; I don't remember why we
    made that change.  Next time put in checkin notes!)
- client: Linux: show error msg if can't open /proc/cpuinfo
    or /proc/meminfo; set memory size of 1 GB in latter case.
    Remove duplicate code that read /proc/meminfo
- 'start' script; handle <disabled>0</disable> in tasks and daemon
    elements of config.xml (can't handle <disable/>, however).
    
client/
    gui_rpc_server_ops.C
    hostinfo_unix.C
sched/
    feeder.C
    start

svn path=/trunk/boinc/; revision=12807
2007-06-03 19:37:45 +00:00
David Anderson 1e7e58875a - client: add cmdline options --exit_before_start and --exit_after_finish.
This cause the core client to exit immediately before or after
    running a job,
    letting you examine the contents of the slot directory.
- scheduler: changed max # of CPUs used in daily_result_quota
    limit from 4 to 8, and make it a compile-time parameter
- feeder/scheduler: make the number of work items in shared
    memory configurable (in config.xml).
    The element is <shmem_work_items>
- feeder: make the size of the work item query configurable
    (<feeder_query_size)
- feeder: remove code related to removing infeasible results
    from shared mem.
    This mechanism was never needed,
    and I think a timeout would accomplish the same effect.

client/
    app.C
    app_start.C
    client_state.C,h
    cs_cmdline.C
sched/
    feeder.C
    sched_array.C
    sched_config.C,h
    sched_send.C
    sched_shmem.C,h
    sched_util.C
    show_shmem.C

svn path=/trunk/boinc/; revision=12771
2007-05-29 21:45:01 +00:00
David Anderson 17a83ada30 - feeder: fixed SQL query again
sched/
    feeder.C

svn path=/trunk/boinc/; revision=12707
2007-05-22 02:58:01 +00:00
David Anderson 9b4eff7284 - feeder: the query change of 13 April broke the -random_order,
-priority_order, -priority_order_create_time,and -mod options
    (from Nicolas Maire)

sched/
    feeder.C

svn path=/trunk/boinc/; revision=12703
2007-05-21 15:06:43 +00:00
David Anderson 0daa5a8802 - translation.inc: backed out change of 7 May;
we need to use " to expand $PROJECT var
- scheduler:
    take work_buf_min into account in deciding whether
    a result will be reported by its deadline.
- all back end programs:
    better error message if can't parse config file

html/inc/
    translation.inc
sched/
    *.C

svn path=/trunk/boinc/; revision=12652
2007-05-11 16:30:13 +00:00
David Anderson 7ecdc2f44e - Scheduler: in reply, add <platform> element to each <app_version,
and add <platform> and <version_num> elements to <result>
        (server half of multi-version changes)
    - scheduler: make ssp a global; could eliminate from args everywhere

    db/
        boinc_db.h
    sched/
        feeder.C
        main.C,h
        sched_send.C
        sched_shmem.C,h
        server_types.C

svn path=/trunk/boinc/; revision=12535
2007-05-02 23:14:00 +00:00
David Anderson 3431d99471 svn path=/trunk/boinc/; revision=12414 2007-04-18 20:49:58 +00:00
David Anderson 66071c9de1 svn path=/trunk/boinc/; revision=12373 2007-04-16 22:22:15 +00:00
David Anderson 75a2b2c55b *** empty log message ***
svn path=/trunk/boinc/; revision=12322
2007-04-05 17:02:01 +00:00
David Anderson f7d648e350 *** empty log message ***
svn path=/trunk/boinc/; revision=12075
2007-02-13 21:41:07 +00:00
David Anderson c98a2415af *** empty log message ***
svn path=/trunk/boinc/; revision=11336
2006-10-22 00:42:44 +00:00
David Anderson cc5c1189c0 *** empty log message ***
svn path=/trunk/boinc/; revision=10623
2006-07-11 21:56:53 +00:00
David Anderson b9b5d0ed48 *** empty log message ***
svn path=/trunk/boinc/; revision=10622
2006-07-11 21:49:20 +00:00
Bruce Allen b7315f3b62 DAVID: your change to create_shmem() broke the sched build.
Check my 'fix' to feeder.C to see if this is what you had
intended.

svn path=/trunk/boinc/; revision=10298
2006-06-10 12:37:24 +00:00
David Anderson b2e6bd1b10 make_work
svn path=/trunk/boinc/; revision=10272
2006-06-08 05:54:40 +00:00
David Anderson a9183d00dc *** empty log message ***
svn path=/trunk/boinc/; revision=10097
2006-05-04 04:29:28 +00:00
David Anderson f6a01ea730 *** empty log message ***
svn path=/trunk/boinc/; revision=10084
2006-05-02 22:33:00 +00:00
David Anderson 2e7b35b9ec feeder/scheduler enhancements
svn path=/trunk/boinc/; revision=10083
2006-05-02 22:17:09 +00:00
David Anderson f9b7413d58 feeder/DB fixes
svn path=/trunk/boinc/; revision=9269
2006-01-19 21:46:25 +00:00
David Anderson fa9d97a1f3 feeder exit on DB error
svn path=/trunk/boinc/; revision=9191
2006-01-06 21:18:48 +00:00
Eric J. Korpela d9ee426071 Added explicit includes of "config.h" in many files to enable use of compilers
without a -include command line option.

svn path=/trunk/boinc/; revision=8896
2005-11-21 18:34:44 +00:00
Bruce Allen 4377095db0 Typo
svn path=/trunk/boinc/; revision=8698
2005-10-18 04:12:14 +00:00
Bruce Allen fec9e8d5fa Feeder: removed historical defines for PID file and lockfile.
Trigger file to re-read database is now reread_db in the project
root direcory, consistent with other BOINC trigger files, rather
than in the tmp_project_name subdirectory.
Log detection/removal of trigger file.

svn path=/trunk/boinc/; revision=8696
2005-10-18 04:08:12 +00:00
David Anderson 941d7e21d8 fix DEBUG symbol collision
svn path=/trunk/boinc/; revision=8189
2005-09-26 23:28:48 +00:00
David Anderson 20dcd8ae39 fix "no new work" bug
svn path=/trunk/boinc/; revision=8136
2005-09-22 00:15:43 +00:00
David Anderson 6adf04efda *** empty log message ***
svn path=/trunk/boinc/; revision=6684
2005-07-19 19:05:50 +00:00