Commit Graph

113 Commits

Author SHA1 Message Date
David Anderson 6e6fab3e7c - scheduler: clean up message log.
Merge redundant messages.
    Condition various messages on config flags.
- client (Unix) fix to CUDA detection if LD_LIBRARY_PATH is ""

svn path=/trunk/boinc/; revision=15122
2008-05-02 17:48:29 +00:00
David Anderson 8ba1188dd0 - Client/server protocol:
send <client_cap_plan_class/> if client understands
    app version plan class.
    The server checks for this instead of version > 6.11.
- clean up unix_util: .h files declare only (extern) interfaces;
    no reason for daemon() to be C

svn path=/trunk/boinc/; revision=15006
2008-04-02 19:05:08 +00:00
David Anderson 6af9f66b4e - DB/feeder/scheduler: change app_version.xml_doc from blob to mediumblob,
and change the correspending structure field from 64KB to 256KB
    (could increase this if needed).
    This is needed to handle app versions with lots (> 100) of files
- change LARGE_BLOB_SIZE to BLOB_SIZE a bunch of places
- Change COPROCS from vector<COPROC> to vector<COPROC*>.
    Otherwise the right virtual functions of COPROCs don't get called

svn path=/trunk/boinc/; revision=14986
2008-03-31 16:19:45 +00:00
David Anderson 13400c9516 Changes for multithread app support:
- update_versions: use __ (not :) as separator for plan class
- client: add plan_class to APP_VERSION;
    an app version is now identified by platform/version/plan_class
- client CPU scheduler: don't assume apps use 1 CPU
- client: add avg_ncpus, max_cpus, flops, cmdline to RESULT
- scheduler: implement app planning scheme

Other changes:

- client: if symlink() fails, make a XML soft link instead
    (for Unix running off a FAT32 FS)
- client: don't accept nonpositive resource share from AMS
- daemons and DB: check for error returns from enumerations,
    and exit if so.  Thus, if the MySQL server goes down,
    all the daemons will soon exit.
    The cron script will restart them every 5 min,
    so when the DB server comes back up so will the project.
- web: show empty max CPU % as ---
- API: get rid of all_threads_cpu_time option (always the case now)


svn path=/trunk/boinc/; revision=14966
2008-03-27 18:25:29 +00:00
David Anderson 4e9fbac5e0 - admin web: touch reread_db in manage_app_versions.php
- DB code: remove "is_high_priority" stuff.
- scheduler: merge find_app_version() into get_app_version().
    Have the latter memoize its results (both positive and negative).
    Have it call app_plan() for apps with nonempty plan_class.
- scheduler: first steps towards improved selectability of log messages.
    It will eventually be like the client,
    where you can select among various types of messages.
- feeder: if can't unlink the reread_db trigger file, exit
    (else we'd go into an infinite loop)

svn path=/trunk/boinc/; revision=14940
2008-03-18 21:22:44 +00:00
David Anderson 8098622210 - Validator framework: remove some consts, and other changes,
to allow validator to assign different credit
    to different instances of a job
- Scheduler: if can't open DB, return <project_is_down/>
    (fixes #578)
- clean up logic of modify_claimed_credit
- feeder: for -priority_order_create_time, use workunitid
    rather than create time (faster for the DB)
from Kevin Reed

svn path=/trunk/boinc/; revision=14908
2008-03-13 23:35:13 +00:00
David Anderson 815b8fc043 Various preparation for handling multithreaded apps
and apps that use coprocessors.
There now can be several app_versions for the same
(app, platform, version_num) combination.
This changes a number of things.

- Added app_version.plan_class field to DB
- update_versions now looks for a :plan-class in the
    file or directory name, and puts it in the app_version's DB record
- Change uniqueness constraint to include plan_class
- Feeder: the feeder was putting non-deprecated app_versions
    in shared mem, and leaving it to the scheduler to
    find the latest version for a given platform.
    This is dumb.
    Instead, for each app/platform pair the feeder now
    finds the highest version number of a non-deprecated app version,
    and enumerates all non-deprecated app_versions with that
    app/platform/version
- Scheduler: add a BEST_APP_VERSION data structure that keeps track,
    for each app, what the best app_version is for this host.
    This saves the work of recomputing it for each job.

svn path=/trunk/boinc/; revision=14906
2008-03-13 22:57:24 +00:00
David Anderson cc2f1a20a0 - lib: moved "run program as user" stuff to a separate file,
so it doesn't screw up the linkage of apps that don't use it
- start of server-side support for coprocessors

svn path=/trunk/boinc/; revision=14878
2008-03-10 21:59:27 +00:00
David Anderson 95772cba77 - removed boinc_ncpus_available() and boinc_nthreads() calls.
The design has been changed to constant #threads per app version
    Various changes from Kevin Reed/WCG:
    - server: add workunit.rsc_bandwidth_bound: if nonzero,
        send this WU only to hosts with that much download bandwidth
    - assimilators: if a handler returns DEFER_ASSIMILATION,
        the WU remains in INIT state and will be handled when the
        next instance completes.
        Useful if you want the assimilator to see all instances.
    - scheduler: when setting result.outcome = DETACHED,
        set received_time to now
    - scheduler: removed the reliable_time and reliable_min_avg_credit
        options
    - scheduler/web: add optional <allow_non_preferred_projects>
        in project preferences.
        If present, user will accept work from non-selected apps
        if no work is available for selected apps
    - scheduler: improved messages for projects with multiple apps
    - scheduler: added config options
        <granted_credit_weight> and <granted_credit_ramp_up>.
        Used in calculating host.claimed_credit_per_cpu_sec,
        but I'm not sure how.
    - Added two new credit-granting formulas (validate_util.C):
        stddev_credit() and two_credit()
    - server DB: add rollback_transaction() and affected_rows() to DB_CONN

    NOTE: DB update required

svn path=/trunk/boinc/; revision=14870
2008-03-07 21:13:01 +00:00
David Anderson b6cc885abf - server: make the special substring for assigned WUs
into a #define's symbol (ASSIGNED_WU_STR)
- scheduler: when send client command to abort a WU,
    include a reason code in the scheduler log

svn path=/trunk/boinc/; revision=14798
2008-02-26 17:24:29 +00:00
David Anderson 54519a4ee1 - Server: add "job assignment" feature.
Lets you assign a WU to a particular host,
    to one or all hosts belonging to a user or team, or to all hosts.
    See http://boinc.berkeley.edu/trac/wiki/AssignedWork
    Disabled unless you include <enable_assignment> in config.xml
    Uses a new DB table.
    Tested but only a little.
- Server: code cleanup; moved result-handling to a new file,
    and removed the PLATFORM_LIST arg to everything
    (put it in SCHEDULER_REQUEST instead)

svn path=/trunk/boinc/; revision=14767
2008-02-21 00:47:50 +00:00
David Anderson 7ea74282f4 - client: limit global prefs mod time to now
- server: limit global prefs mod time to now
    These changes address the situation where a server
    sends out prefs with mod time far in the future,
    and there's no way to undo them

svn path=/trunk/boinc/; revision=14664
2008-02-03 21:46:30 +00:00
David Anderson 2be6f8e53a - Client: add <run_apps_manually> config flag.
This is for debugging apps (currently works only in Unix).
    What it does: when running an app,
    the client does everything except actually fork/exec the app,
    i.e. it sets up the slot dir, creates shared mem segment etc.
    It then continues as if the app were actually running,
    and you can then manually run your app under a debugger
    in the slot directory.
    Note: the client won't notice the termination of your app.
- API, Unix: in situations where the timer thread wants to exit
    (e.g. it notices a missing heartbeat).
    don't directly call boinc_exit(),
    since this touches data structures that the worker thread
    may be using concurrently.
    Instead, set a flag telling the worker thread to call boinc_exit()
    (which it will do from its signal handler)
    This is an attempt to fix problems reported by Bernd;
    I haven't tested it.
- scheduler: add config flag for uploading usage data
- web: show account key and weak account key on user page
- added some code for multithread support (not finished)

api/
    boinc_api.C


svn path=/trunk/boinc/; revision=14542
2008-01-13 00:12:14 +00:00
Frank Thomas 3bfc78b511 Updated the postal address of the Free Software Foundation in all license headers. See http://lists.ssl.berkeley.edu/pipermail/boinc_dev/2007-October/008939.html for reference.
svn path=/trunk/boinc/; revision=13804
2007-10-09 11:35:47 +00:00
David Anderson 4e1f0b7019 - client: add a <data_dir> option in cc_config.xml;
tells the client to use this as the data directory
- scheduler: improve the message telling the client that
    more disk or memory is needed;
    tell them the minimum amount needed to
    send any of the jobs rejected,
    rather than the amount needed for the first job rejected
- manager: fix text in "connect now" dialog

svn path=/trunk/boinc/; revision=13387
2007-08-16 17:33:41 +00:00
David Anderson 797c464b3a - Back end: add a feature for "blackballing" hosts.
To do this, set host.max_results_day to -1.
    If you do this, scheduler requests from that host
    will get an error message, and will otherwise be ignored
    (no jobs in or out, no trickles).
- Scheduler: send_message() should be called ONLY if you're
    not going to call handle_request();
    otherwise we'll write two separate replies.
    To fix this, I added a separate function (send_error_message())
    that can be called within handle_request()
    to deal with error situations.
- Scheduler: moved debug_sched() to main.C
- Scheduler: moved logic to send "delete file" commands
    out of handle_request() into a separate function,
    send_file_deletes() in sched_locality.C.
    Remove #ifdef EINSTEIN_AT_HOMEs; maybe someday another project
    will use locality scheduling!

svn path=/trunk/boinc/; revision=13108
2007-07-06 16:37:00 +00:00
David Anderson 0bbe224c21 - scheduler: the "max_wus_in_progress" option only worked if
"resend_lost_results" option was used also
    (because the count of in-progress results was
    based from the DB query used by resend_lost_results).

    Fix: initialize the count of in-progress results to
    the list provided in the scheduler request.
- scheduler: add "--mark_jobs_done" flag; if set, all jobs
    sent are marked as done, and their WUs enabled for transition.
    This is used for simulation purposes,
    in conjunction with sched_driver.
- scheduler: if --batch option is set, don't check RPC seqnos
    (for simulation purposes)

svn path=/trunk/boinc/; revision=13101
2007-07-05 04:18:48 +00:00
David Anderson 01f4851323 - scheduler: add max_wus_in_progress option.
Limits total # of in-progress results per host
    (independently of #CPUs)

sched/
    sched_config.C,h
    sched_resend.C
    sched_send.C
    server_types.h


svn path=/trunk/boinc/; revision=12661
2007-05-14 15:21:38 +00:00
David Anderson a37403a673 - scheduler: add <workload_sim> config option.
If set, the scheduler will use EDF simulation,
    together with the in-progress workload reported by the client,
    to avoid sending results that
    1) will miss their deadline, or
    2) will cause an in-progress result to miss its deadline, or
    3) will make an in-progress result miss its deadline
        by more than is already predicted.
    If this option is not set, or if the client request doesn't
    include a workload description (i.e. the client is old)
    use the existing approach, which assumes there's no workload.
    NOTE: this is experimental.  Production projects should not use it.
- EDF sim: write debug stuff to stderr instead of stdout
- Account manager:
    - if an account is detach_when_done, set dont_request_more_work
    - check done_request_more_work even for first-time projects
- update_uotd: generate a file for use by Google gadget
- user_links(): use full URLs (so can use in Google gadget)

client/
    acct_mgr.C
    work_fetch.C
html/
    inc/
        uotd.inc
        util.inc
    user/
        uotd_gadget.php (new)
sched/
    Makefile.am
    edf_sim.C
    sched_config.C,h
    sched_resend.C
    sched_send.C,h
    server_types.C,h

svn path=/trunk/boinc/; revision=12639
2007-05-10 21:50:52 +00:00
David Anderson a3c22a1bf7 - client: add constructor/destructor for APP_VERSION, WORKUNIT, RESULT
- client: with anonymous platform, don't link app version twice

client/
    client_types.h
    client_state.C
    cs_apps.C
    cs_files.C
    cs_prefs.C
    cs_scheduler.C
    cs_statefile.C
    sim_util.C
html/
    uotd.inc
sched/
    edf_sim.h
    server_types.h

svn path=/trunk/boinc/; revision=12638
2007-05-10 16:23:10 +00:00
David Anderson 78807f659c - scheduler: slight code cleanup prior to adding EDF stuff
languages/translations/
    en.po
sched/
    sched_send.C
    server_types.h

svn path=/trunk/boinc/; revision=12631
2007-05-09 17:45:18 +00:00
David Anderson 445b3e8696 server EDF simulator
svn path=/trunk/boinc/; revision=12516
2007-04-30 23:35:39 +00:00
David Anderson 2e7b82b631 - scheduler: added (correct this time!) support for
<alt_platform> tags in scheduler requests.
    - file_deleter: add check for -dont_delete_batches 

    sched/
        file_deleter.C
        handle_request.C
        sched_array.C,h
        sched_locality.C,h
        sched_resend.C,h
        sched_send.C,h
        server_types.h

svn path=/trunk/boinc/; revision=12512
2007-04-30 21:19:24 +00:00
David Anderson 712d1c027f David 19 Apr 2007
- scheduler: accept elements of the form
        <alt_platform>
            <name>windows_x86</name>
        </alt_platform>
        in request messages.
        These are platforms that the client can handle, 
        other than its main one,
        listed in order of decreasing preferences.
        fixes #81.

    sched/
        handle_request.C
        server_types.C,h

svn path=/trunk/boinc/; revision=12423
2007-04-19 22:45:57 +00:00
David Anderson 66071c9de1 svn path=/trunk/boinc/; revision=12373 2007-04-16 22:22:15 +00:00
David Anderson 75a2b2c55b *** empty log message ***
svn path=/trunk/boinc/; revision=12322
2007-04-05 17:02:01 +00:00
David Anderson 7767ca1eb8 *** empty log message ***
svn path=/trunk/boinc/; revision=11492
2006-11-07 17:40:55 +00:00
David Anderson 190f7d177a *** empty log message ***
svn path=/trunk/boinc/; revision=11337
2006-10-22 01:46:33 +00:00
David Anderson c98a2415af *** empty log message ***
svn path=/trunk/boinc/; revision=11336
2006-10-22 00:42:44 +00:00
David Anderson d5fd6240e7 *** empty log message ***
svn path=/trunk/boinc/; revision=11253
2006-10-06 18:52:50 +00:00
David Anderson 6835cdac20 *** empty log message ***
svn path=/trunk/boinc/; revision=11244
2006-10-04 17:01:36 +00:00
David Anderson 7ce1144b73 *** empty log message ***
svn path=/trunk/boinc/; revision=11076
2006-09-01 21:06:30 +00:00
David Anderson b9b5d0ed48 *** empty log message ***
svn path=/trunk/boinc/; revision=10622
2006-07-11 21:49:20 +00:00
David Anderson 0f9b83a55b project files
svn path=/trunk/boinc/; revision=10517
2006-06-26 22:58:24 +00:00
Bruce Allen 29c352a869 Einstein@Home specific changes
svn path=/trunk/boinc/; revision=10192
2006-05-24 19:00:12 +00:00
David Anderson 2e7b35b9ec feeder/scheduler enhancements
svn path=/trunk/boinc/; revision=10083
2006-05-02 22:17:09 +00:00
Bruce Allen cc96fd006a Einstein@Home-specific hack to the locality scheduler. For new
WU, I need to separate two types of 'sticky/report on RPC' files.
One type is used for WU names of the form FILE__ and the others
are auxilliary.  David, after I've talked with you I'll either
generalize this notion or we can eliminate it.

svn path=/trunk/boinc/; revision=9131
2005-12-23 04:17:10 +00:00
David Anderson 413c161c36 *** empty log message ***
svn path=/trunk/boinc/; revision=9012
2005-11-30 22:52:23 +00:00
David Anderson 4bc558ffc8 disk usage fixes
svn path=/trunk/boinc/; revision=8442
2005-10-04 21:44:58 +00:00
David Anderson 99c9679b63 versions are major/minor/release
svn path=/trunk/boinc/; revision=7614
2005-08-31 00:18:36 +00:00
David Anderson 3f785e8bdd resend lost results
svn path=/trunk/boinc/; revision=6866
2005-07-28 10:13:30 +00:00
David Anderson 793fbf57ee *** empty log message ***
svn path=/trunk/boinc/; revision=6862
2005-07-28 09:00:19 +00:00
David Anderson 814f30d3b7 *** empty log message ***
svn path=/trunk/boinc/; revision=6486
2005-06-29 04:17:28 +00:00
David Anderson 213f2b1d1a *** empty log message ***
svn path=/trunk/boinc/; revision=6399
2005-06-22 06:02:59 +00:00
Bruce Allen fe47e56df6 scheduler: use locking to prevent more than one scheduler instance from
running for a given host.  This works by creating a file called
CGI_<HOSTID> in the cgi-bin/ directory, and using Posix advisory file
locking.  I have been testing this code for three days and am seeing
*some* invocations of this.  David, I'll send details to the dev mailing
list or talk with you about it later this week.  Note: this code probably
can be removed in the future, when the core client problems are fixed.
Also note: I don't know if this is compatible with the fast cgi sched.

svn path=/trunk/boinc/; revision=6172
2005-05-17 05:55:24 +00:00
David Anderson d8a43ac0fe *** empty log message ***
svn path=/trunk/boinc/; revision=5971
2005-04-29 06:55:44 +00:00
David Anderson 1c119cb037 *** empty log message ***
svn path=/trunk/boinc/; revision=5889
2005-04-18 18:42:29 +00:00
David Anderson 647c8122b3 *** empty log message ***
svn path=/trunk/boinc/; revision=5886
2005-04-18 17:54:03 +00:00
David Anderson b152c3f453 *** empty log message ***
svn path=/trunk/boinc/; revision=5530
2005-02-26 00:24:37 +00:00
Bruce Allen 0f6202fc9e When daily result quota exceeded, tell users what its value is for that host.
svn path=/trunk/boinc/; revision=5493
2005-02-22 17:53:17 +00:00