Commit Graph

171 Commits

Author SHA1 Message Date
David Anderson 80bde8a4c6 - ops: add script to fix backslashes in posts etc.
- DB: add db_update.php entry for Eric's new table

svn path=/trunk/boinc/; revision=15700
2008-07-28 22:24:57 +00:00
Eric J. Korpela 8da770569f Typo fix
svn path=/trunk/boinc/; revision=15671
2008-07-25 01:42:23 +00:00
Eric J. Korpela a5a6f693cd - Implementation of automatic credit leveling for cpu based projects that
wish to use it.
- The script calculate_credit_multiplier (expected to be run daily as
      a config.xml task) looks at the ratio of granted credit to CPU time 
      for recent results for each app.  Multiplier is calculated to cause 
      median hosts granted credit per cpu second to equal to equal that 
      expected from its benchmarks.  This is 30-day exponentially averaged 
      with the previous value of the multplier and stored in the table 
      credit_multplier.
- When a result is received the server adjusts claimed credit by the
      value the multiplier had when the result was sent.


svn path=/trunk/boinc/; revision=15661
2008-07-22 23:36:55 +00:00
David Anderson 3054a5f6b3 - validator/assimilator: fix bug in 2 June checkin
(xml_doc_in wasn't included in DB query)

svn path=/trunk/boinc/; revision=15363
2008-06-05 18:36:31 +00:00
David Anderson 0e03df254b - Back end: add adaptive validation feature
(DB update required)
- Fixed typo in Eric's 5/28 checkin

svn path=/trunk/boinc/; revision=15357
2008-06-04 23:04:12 +00:00
David Anderson 6af9f66b4e - DB/feeder/scheduler: change app_version.xml_doc from blob to mediumblob,
and change the correspending structure field from 64KB to 256KB
    (could increase this if needed).
    This is needed to handle app versions with lots (> 100) of files
- change LARGE_BLOB_SIZE to BLOB_SIZE a bunch of places
- Change COPROCS from vector<COPROC> to vector<COPROC*>.
    Otherwise the right virtual functions of COPROCs don't get called

svn path=/trunk/boinc/; revision=14986
2008-03-31 16:19:45 +00:00
David Anderson 13400c9516 Changes for multithread app support:
- update_versions: use __ (not :) as separator for plan class
- client: add plan_class to APP_VERSION;
    an app version is now identified by platform/version/plan_class
- client CPU scheduler: don't assume apps use 1 CPU
- client: add avg_ncpus, max_cpus, flops, cmdline to RESULT
- scheduler: implement app planning scheme

Other changes:

- client: if symlink() fails, make a XML soft link instead
    (for Unix running off a FAT32 FS)
- client: don't accept nonpositive resource share from AMS
- daemons and DB: check for error returns from enumerations,
    and exit if so.  Thus, if the MySQL server goes down,
    all the daemons will soon exit.
    The cron script will restart them every 5 min,
    so when the DB server comes back up so will the project.
- web: show empty max CPU % as ---
- API: get rid of all_threads_cpu_time option (always the case now)


svn path=/trunk/boinc/; revision=14966
2008-03-27 18:25:29 +00:00
David Anderson 815b8fc043 Various preparation for handling multithreaded apps
and apps that use coprocessors.
There now can be several app_versions for the same
(app, platform, version_num) combination.
This changes a number of things.

- Added app_version.plan_class field to DB
- update_versions now looks for a :plan-class in the
    file or directory name, and puts it in the app_version's DB record
- Change uniqueness constraint to include plan_class
- Feeder: the feeder was putting non-deprecated app_versions
    in shared mem, and leaving it to the scheduler to
    find the latest version for a given platform.
    This is dumb.
    Instead, for each app/platform pair the feeder now
    finds the highest version number of a non-deprecated app version,
    and enumerates all non-deprecated app_versions with that
    app/platform/version
- Scheduler: add a BEST_APP_VERSION data structure that keeps track,
    for each app, what the best app_version is for this host.
    This saves the work of recomputing it for each job.

svn path=/trunk/boinc/; revision=14906
2008-03-13 22:57:24 +00:00
David Anderson 95772cba77 - removed boinc_ncpus_available() and boinc_nthreads() calls.
The design has been changed to constant #threads per app version
    Various changes from Kevin Reed/WCG:
    - server: add workunit.rsc_bandwidth_bound: if nonzero,
        send this WU only to hosts with that much download bandwidth
    - assimilators: if a handler returns DEFER_ASSIMILATION,
        the WU remains in INIT state and will be handled when the
        next instance completes.
        Useful if you want the assimilator to see all instances.
    - scheduler: when setting result.outcome = DETACHED,
        set received_time to now
    - scheduler: removed the reliable_time and reliable_min_avg_credit
        options
    - scheduler/web: add optional <allow_non_preferred_projects>
        in project preferences.
        If present, user will accept work from non-selected apps
        if no work is available for selected apps
    - scheduler: improved messages for projects with multiple apps
    - scheduler: added config options
        <granted_credit_weight> and <granted_credit_ramp_up>.
        Used in calculating host.claimed_credit_per_cpu_sec,
        but I'm not sure how.
    - Added two new credit-granting formulas (validate_util.C):
        stddev_credit() and two_credit()
    - server DB: add rollback_transaction() and affected_rows() to DB_CONN

    NOTE: DB update required

svn path=/trunk/boinc/; revision=14870
2008-03-07 21:13:01 +00:00
David Anderson 1106da6ec6 - transitioner: we're not supposed to ever see "assigned" WUs.
But if we do, set their transitioner time to plus infinity
    so that we don't see them again.
    (otherwise we go into an infinite loop)
- DB code: remove "high_priority" from queries not from scheduler
    (should probably remove them from there too)
- file_deleter: print error msg if apache user doesn't exist

svn path=/trunk/boinc/; revision=14835
2008-03-03 19:33:32 +00:00
David Anderson 54519a4ee1 - Server: add "job assignment" feature.
Lets you assign a WU to a particular host,
    to one or all hosts belonging to a user or team, or to all hosts.
    See http://boinc.berkeley.edu/trac/wiki/AssignedWork
    Disabled unless you include <enable_assignment> in config.xml
    Uses a new DB table.
    Tested but only a little.
- Server: code cleanup; moved result-handling to a new file,
    and removed the PLATFORM_LIST arg to everything
    (put it in SCHEDULER_REQUEST instead)

svn path=/trunk/boinc/; revision=14767
2008-02-21 00:47:50 +00:00
Frank Thomas 3bfc78b511 Updated the postal address of the Free Software Foundation in all license headers. See http://lists.ssl.berkeley.edu/pipermail/boinc_dev/2007-October/008939.html for reference.
svn path=/trunk/boinc/; revision=13804
2007-10-09 11:35:47 +00:00
David Anderson bb5f54d31f - feeder/scheduler: fix bug where APP.homogeneous_redundancy
is defined as a bool instead of int
    (see its only nonzero value is 1, so the "coarse" HR type is ignored).

svn path=/trunk/boinc/; revision=13786
2007-10-05 22:32:47 +00:00
David Anderson db08d30fee - scheduler: update host.n_bwdown
(this was omitted for some reason).
    From Kevin Reed.


svn path=/trunk/boinc/; revision=13431
2007-08-21 20:41:29 +00:00
David Anderson 3e340136da - feeder: simplified DB query
- make_project: remove space after URL in schedulers.txt

svn path=/trunk/boinc/; revision=13127
2007-07-09 20:09:49 +00:00
David Anderson a97556bdfd - feeder: added a new enumerator of DB_WORK_ITEM that,
on successive calls, scans through ALL the sendable
    jobs satisfying the select clause
    (it does this by ID order, so there's no order clause)
    This is used for HR, so that if a job has been committed
    to an HR class, we eventually get it.

    With extremely minimal testing, the new HR stuff seems to work.

db/
    boinc_db.C,h
sched/
    feeder.C
    sample_work_generator.C
    server_types.C

svn path=/trunk/boinc/; revision=12988
2007-06-22 23:48:37 +00:00
David Anderson f5d94818dd - added "census", a program that counts up how much RAC
there is for each HR class, and writes it to a file.
    This will be used soon for HR support in the feeder.
- split the HR code into hr.C,h (stuff used by both census and scheduler)
    and sched_hr.C (stuff used only by the scheduler)
- database: change DB_CREDITED_JOB to treat workunitid
    as a double (which it is) rather than a long.
    BTW, long == int.
- fixed lots of compile warnings in the server code

db/
    boinc_db.C,h
lib/
    boinc_cmd.C
    miofile.C
    util.C
sched/
    Makefile.am
    census.C (new)
    feeder.C
    file_deleter.C
    file_upload_handler.C
    handle_request.C
    hr.C,h (new)
    main.C
    sample_assimilator.C
    sample_work_generator.C
    sched_array.C
    sched_hr.C,h
    sched_send.C
    server_types.C
    transitioner.C
    validator.C

svn path=/trunk/boinc/; revision=12970
2007-06-20 22:34:06 +00:00
David Anderson d673440faf - Scheduler: increased the resolution of homogeneous redundancy (HR)
e.g. distinguish between models of Intel and AMD
- Scheduler: add a quick HR check that doesn't access the DB
- Transitioner: if a workunit has >0 error results and no success results,
    set its HR class to zero
From M.F. Somers.

db/
    boinc_db.C,h
sched/
    sched_array.C
    sched_hr.C,h
    transitioner.C

svn path=/trunk/boinc/; revision=12773
2007-05-29 23:41:31 +00:00
Matt Lebofsky 136ce49b84 svn path=/trunk/boinc/; revision=12536 2007-05-02 23:17:52 +00:00
Matt Lebofsky 846ec9fbb0 svn path=/trunk/boinc/; revision=12532 2007-05-02 18:51:51 +00:00
David Anderson 7e2b39c4db David 24 Apr 2007
- Server: add new database field host.error_rate.
        Stores a dynamic estimate of the fraction of the hosts's
        results that fail validation (updated by the validator).

    client/
        sim.C
        cs_prefs.C
    db/
        boinc_db.C,h
    html/ops/
        db_update.php
    sched/
        handle_request.C
        sched_msgs.h
        validator.C

svn path=/trunk/boinc/; revision=12466
2007-04-24 23:21:42 +00:00
David Anderson bc98f0e90c *** empty log message ***
svn path=/trunk/boinc/; revision=12367
2007-04-13 22:55:18 +00:00
David Anderson 75a2b2c55b *** empty log message ***
svn path=/trunk/boinc/; revision=12322
2007-04-05 17:02:01 +00:00
David Anderson 81effa6a7b *** empty log message ***
svn path=/trunk/boinc/; revision=12127
2007-02-21 16:26:51 +00:00
David Anderson 3967a1f924 *** empty log message ***
svn path=/trunk/boinc/; revision=11850
2007-01-12 17:42:29 +00:00
David Anderson adcd68afce *** empty log message ***
svn path=/trunk/boinc/; revision=11628
2006-12-06 20:59:29 +00:00
David Anderson 190f7d177a *** empty log message ***
svn path=/trunk/boinc/; revision=11337
2006-10-22 01:46:33 +00:00
David Anderson c98a2415af *** empty log message ***
svn path=/trunk/boinc/; revision=11336
2006-10-22 00:42:44 +00:00
David Anderson be1c8ff44f *** empty log message ***
svn path=/trunk/boinc/; revision=10641
2006-07-12 23:18:57 +00:00
David Anderson b9b5d0ed48 *** empty log message ***
svn path=/trunk/boinc/; revision=10622
2006-07-11 21:49:20 +00:00
Bruce Allen 086c94a309 When calling project-provided validator functions, populate the
WU.rsc_fpops_est field. This can be useful in
determining how much credit to assign, and preventing cheating.

DAVID: I was going to be a good guy and document this, but you
forgot to check validate_simple.php and validate_advanced.php
into CVS (:-).

svn path=/trunk/boinc/; revision=10297
2006-06-10 12:14:22 +00:00
Bruce Allen 033c6719a5 Scheduler: refuse to upload results if DB item is
(1) over and (2) validate state init and (3) NOT file_delete==INIT.

(David, please bless addition of file_delete_state to SCHED_RESULT_ITEM.)

svn path=/trunk/boinc/; revision=10245
2006-06-03 16:22:34 +00:00
Bruce Allen e37637661d Transitioner: nasty bug in logging, wrong result id/name
Scheduler: clearer logging for debugging purposes

svn path=/trunk/boinc/; revision=10243
2006-06-03 14:59:22 +00:00
David Anderson 9929015c1e xml_escape bug
svn path=/trunk/boinc/; revision=10238
2006-06-01 19:59:57 +00:00
Bruce Allen 26d08b603f Also need outcome...
svn path=/trunk/boinc/; revision=10229
2006-05-31 21:50:42 +00:00
Bruce Allen 490d4858bf scheduler: do a better job of accepting results from host machines
which may modify an EXISTING & OVER result in the database.

file_upload_handler: check for 'stop_upload' trigger file in the
project root directory.  If so, return ERR_TRANSIENT to hosts which
attempt uploads.  This is the equivalent of 'stop_sched' for the
scheduler.  (Eric, you may want to check that this is FCGI compatible).

svn path=/trunk/boinc/; revision=10227
2006-05-31 20:39:32 +00:00
David Anderson 210e22e537 *** empty log message ***
svn path=/trunk/boinc/; revision=10226
2006-05-31 17:19:29 +00:00
David Anderson f6a01ea730 *** empty log message ***
svn path=/trunk/boinc/; revision=10084
2006-05-02 22:33:00 +00:00
Bruce Allen 527e83ce0c Missing return value causes sched horror...
svn path=/trunk/boinc/; revision=9667
2006-03-17 05:54:06 +00:00
David Anderson 1b3e1fe484 scheduler race condition
svn path=/trunk/boinc/; revision=9666
2006-03-17 04:47:51 +00:00
David Anderson f471b89bf9 propagate batch to result
svn path=/trunk/boinc/; revision=9318
2006-01-26 00:19:05 +00:00
David Anderson f9b7413d58 feeder/DB fixes
svn path=/trunk/boinc/; revision=9269
2006-01-19 21:46:25 +00:00
Eric J. Korpela d9ee426071 Added explicit includes of "config.h" in many files to enable use of compilers
without a -include command line option.

svn path=/trunk/boinc/; revision=8896
2005-11-21 18:34:44 +00:00
Bruce Allen dc12b0895c Nuke unused variable
svn path=/trunk/boinc/; revision=8333
2005-09-29 22:14:59 +00:00
David Anderson b96bdc7c58 *** empty log message ***
svn path=/trunk/boinc/; revision=8137
2005-09-22 00:30:01 +00:00
David Anderson 20dcd8ae39 fix "no new work" bug
svn path=/trunk/boinc/; revision=8136
2005-09-22 00:15:43 +00:00
David Anderson 041b6f24c6 *** empty log message ***
svn path=/trunk/boinc/; revision=7492
2005-08-25 23:07:58 +00:00
Jeff Cobb 0680f23b68 *** empty log message ***
svn path=/trunk/boinc/; revision=7280
2005-08-11 23:53:24 +00:00
David Anderson b3dc4bcbe4 new account setup
svn path=/trunk/boinc/; revision=7205
2005-08-07 01:33:15 +00:00
David Anderson 3dfea63937 scheduler fix
svn path=/trunk/boinc/; revision=6860
2005-07-28 08:09:46 +00:00