Commit Graph

180 Commits

Author SHA1 Message Date
David Anderson 478900c321 - web/scheduler: add project preference for not getting Intel GPU jobs 2013-04-20 12:25:55 -07:00
David Anderson c9c9f2bae0 - scheduler: code shuffle; new file sched_check.cpp contains functions
that decide whether a job can be sent to a host
2013-04-09 12:19:00 -07:00
David Anderson 12319ca82b - scheduler: add code (commented out for now) for new implementation
of score-based scheduling.
2013-04-09 11:10:50 -07:00
David Anderson 7cdb3a7080 - web: add advisory text to "Export as Notice" balloon
- scheduler: make max # of CPUs a #define, default 4096
2013-03-04 17:24:18 +01:00
David Anderson 1174b00aba - client/manager: tweaks to Intel GPU code 2013-03-04 15:27:25 +01:00
David Anderson ca652519cf - scheduler: log message tweaks
- Some C++ files in client had execute permissions (??).  Clear them.
2013-03-01 16:26:08 +01:00
David Anderson 42756406f5 - example app: get the graphics app to build on Fedora Core 14
Note: this required installing various packages,
    and tweaking the Makefile in various ways,
    some of them FC14-specific ways
2013-03-01 16:23:13 +01:00
David Anderson b2f9e30570 - scheduler: fix bug that cause NCI to be sent even when
client is not requesting work
- client: parse the <vbox_window> option in cc_config.xml


svn path=/trunk/boinc/; revision=26127
2012-09-19 23:21:04 +00:00
David Anderson 6b81e2ffc3 - scheduler: fix sending of NCI jobs.
We were failing to mark the cache entries as free.
- API: initialize GPU device # to -1;
    If client doesn't give us a device number, something is wrong
    and it's better to not start computing.


svn path=/trunk/boinc/; revision=26079
2012-09-06 23:44:03 +00:00
David Anderson 11a6e85632 - scheduler: support for projects with some non-CPU-intensive apps
(but not all) wasn't finished.
    New logic: if the project has an NCI app then:
    - make a list of NCI apps for which the client doesn't have
        a job in progress.
    - try to send one job for each of these apps
    - do this even if no work is being requested.
    - don't send jobs for NCI apps by other mechanisms

NOTE: the client logic isn't quite right for mixed NCI projects.
    If there's no job for a given NCI app,
    the client should do a scheduler RPC.
    This isn't critical so we won't do this now.


svn path=/trunk/boinc/; revision=26068
2012-09-01 04:58:12 +00:00
David Anderson b1d1e21de4 - remote job submission: start writing a general-purpose
cmdline tool for remote job submission (not done)
- remote job submission: support the 4 file modes described
    in the documentation (not done)


svn path=/trunk/boinc/; revision=26067
2012-08-31 06:11:06 +00:00
David Anderson 6b7fb36056 - scheduler: msg tweaks
svn path=/trunk/boinc/; revision=26066
2012-08-29 18:08:15 +00:00
David Anderson 96b6e172f9 - scheduler: improved log messages for limited locality scheduling
svn path=/trunk/boinc/; revision=26065
2012-08-29 03:09:10 +00:00
David Anderson 9ccb8fa38d - scheduler: add support for limited locality scheduling
- API: remove support for PPM files


svn path=/trunk/boinc/; revision=26062
2012-08-27 17:00:43 +00:00
David Anderson 0483da30f4 - server: compile fix
- server: add nfiles_on_host(),
    a support function for "locality scheduling lite".


svn path=/trunk/boinc/; revision=26061
2012-08-26 21:24:18 +00:00
David Anderson 13b6aefbd7 - scheduler: fix message typo
svn path=/trunk/boinc/; revision=25985
2012-08-05 01:40:19 +00:00
David Anderson f8c1665722 - client: keep track of the fraction of time that
1) a network connection is available and
    2) network communication is allowed and
    3) CPU computation is allowed
- If an app version is marked as needs_network,
    use the above fraction in estimating its rate of progress
- replace "core client" with "client" in comments.
- scheduler: message tweaks


svn path=/trunk/boinc/; revision=25803
2012-06-26 20:30:56 +00:00
David Anderson 8c71f6d59a - scheduler: add support for Intel GPUs, and restructure things
to make it easier to add other GPU types in the future


svn path=/trunk/boinc/; revision=25792
2012-06-25 23:09:45 +00:00
David Anderson fd0983b991 - web: server status page should show elapsed time, not CPU time
svn path=/trunk/boinc/; revision=25785
2012-06-22 07:35:54 +00:00
David Anderson 0cc0370f02 - client, GUI RPC: detect and export the PCI bus, device, and domain #s.
- scheduler: increase #GPU limit from 8 to 64


svn path=/trunk/boinc/; revision=25761
2012-06-15 20:49:11 +00:00
David Anderson 6888468d6d - scheduler: call get_reliability_and_trust() and get_prefs_info()
in send_work_setup(), so that they run before lost jobs are resent.
    Otherwise lost jobs could get sent using an app version
    that's prohibited by prefs


svn path=/trunk/boinc/; revision=25604
2012-04-26 21:40:20 +00:00
Wenjing Wu ccad62b912 - wrapper: when reading fraction-done file, read the last line
(or at least the last double).
    This accommodates a particular application (LAMMPS)
    that can only append to this file.
- CAS@home stuff


svn path=/trunk/boinc/; revision=25557
2012-04-13 09:44:01 +00:00
David Anderson 759c23ed27 - server: create a harness for testing validator code.
If you link your functions (init_result(), compare_results(),
    cleanup_result()) with validate_test.cpp,
    you'll get a program that you can run as
        validate_test file1 file2
    and it will compare the two files
    (this works only for validators that expect 1 file per result).

    I added a makefile, sched/makefile_validator_test,
    that you can use for this.
- server: shuffle code so that the above doesn't need to
    link MySQL libraries
- client: if we fetch a master file and it contains no scheduler URLs,
    show a message of class INTERNAL_ERROR
- client/scheduler: make CUDA_DEVICE_PROP.totalGlobalMem a double,
    and remove dtotalGlobalMem.
    Although NVIDIA reports RAM size as a size_t,
    there's no reason to store it as an integer after that.


svn path=/trunk/boinc/; revision=25542
2012-04-10 00:32:35 +00:00
David Anderson c22ac04ad0 - scheduler: don't max with min_work_buffer() in estimating
job turnaround time.
    min_work_buffer doesn't necessarily reflect network disconnect periods
- client simulator: compile fixes


svn path=/trunk/boinc/; revision=25521
2012-03-30 21:39:22 +00:00
David Anderson d677f2c1cd - scheduler: fix a bug in assigning deadlines to resent jobs.
From Kevin.
    NOTE: Kevin also reported a bug that resulted in
    way too large (e.g. year) deadlines.


svn path=/trunk/boinc/; revision=25458
2012-03-19 22:46:00 +00:00
David Anderson 5c8cdc3957 - scheduler: do homogeneous app version check before rereading WU
as well as after.


svn path=/trunk/boinc/; revision=25419
2012-03-14 06:56:48 +00:00
David Anderson 8f84416ab7 - scheduler: add log messages to show VM-related request items
- fix typos in db_update script


svn path=/trunk/boinc/; revision=25183
2012-02-02 00:18:42 +00:00
David Anderson 130d6ed4f0 - server: revamp the "assigned job" mechanism.
This now supports two main use cases:
    1) there's a job that you want to run once on all hosts,
        present and future
        (or all hosts belonging to a user, or to a team).
        The job is never transitioned, validated, or assimilated.
    2) There's a normal job for which you want to use only
        hosts belonging to a specific user (e.g. cluster or cloud hosts).
        This restriction can be made either when the job is created,
        or on the fly,
        e.g. as part of a scheme for accelerating batch completion.
        For the latter purposes we now provide a function
            restrict_wu_to_user(DB_WORKUNIT&, int userid);

        The job goes through the standard
        transitioner/validator/assimilator path.

    These cases are enabled by config flags
        <enable_assignment_multi/>
        <enable_assignment/>
    respectively.

    Assignment of type 2) are no longer stored in shared mem,
    so there is no limit on their number.

    There is no longer a rule that assigned job names must contain "asgn".

    NOTE: this requires a database update.


svn path=/trunk/boinc/; revision=25169
2012-01-30 22:39:13 +00:00
David Anderson 56e5a2c089 - scheduler: compute result.report_deadline BEFORE passing it
as an arg to update_wu_on_send()


svn path=/trunk/boinc/; revision=24567
2011-11-09 23:50:09 +00:00
David Anderson 7c201eba3f - DB: use %u when writing result IDs in SQL queries;
this is to support SETI@home, which ran out of result IDs
    and changed the DB field type to int unsigned.
    Note: eventually I'll make this change official
    and change the .h types as well.
- web: put <apps_selected> tags around <app_id> elements
    in project-specific prefs.


svn path=/trunk/boinc/; revision=24555
2011-11-09 07:41:49 +00:00
David Anderson 6d906cf523 - scheduler: fix bug in HR code (from Kevin)
svn path=/trunk/boinc/; revision=24534
2011-11-06 04:53:03 +00:00
David Anderson 497c08b5c2 - scheduler: in update_wu_on_send(), always try to update
the hr_class and app_version_id fields,
    with the where clause that they be either zero or the target value.
    This handles the cases where
    1) because of the failure of a results, the transitioner set
        the field back to zero;
    2) another scheduler set the field to the target value


svn path=/trunk/boinc/; revision=24513
2011-11-03 06:46:05 +00:00
David Anderson 743d687c05 - scheduler: bug fix from Kevin
svn path=/trunk/boinc/; revision=24503
2011-10-27 03:55:18 +00:00
David Anderson 2371bd641a - scheduler: typo in a SQL query
svn path=/trunk/boinc/; revision=24498
2011-10-26 22:05:50 +00:00
David Anderson 5d76e13277 - scheduler: tweaks to last night's checkin.
In the inner loop of scan_work_array() there are two WORKUNITs:
    - the one that's part of wu_result (in the shared-mem array)
    - a temp copy.
        quick_check() may modify this in host-specific ways
        (e.g., adjusting rsc_fpops_est or delay_bound).
        This is the one we pass to add_result_to_reply().
    When we reread hr_class and app_version_id from the DB,
    update both structs.


svn path=/trunk/boinc/; revision=24493
2011-10-26 16:51:10 +00:00
David Anderson 4b826b52a0 - scheduler: fix bug in the "homogeneous app version" (HAV) feature
(reported by Kevin Reed).
    The problem: cache inconsistency.
    If there are 2 results for the same WU in shared mem,
    and 2 scheduler instances get them around the same time,
    they can send them with different app versions.
    We already fixed this problem for HR by
    1) rereading the relevant WU fields while deciding
        whether to send the result
    2) doing a "careful update" of the WU field using a where clause
        to make sure it wasn't modified in the (short) interval
        since rereading it.
    I fixed the HAV problem in the same way,
    and merged the two mechanisms to combine the DB queries.

    Also:
    - The rereads are done in slow_check() (see below).
    - The careful updates are done in update_wu_on_send(),
        and this is called *before* doing careful updates on result fields.
        That way, if the WU updates fail, we don't have orphaned results.
    - already_sent_to_different_platform_careful() (sic)
        no longer does DB stuff, so it's merged with
        already_send_to_different_hr_class() (better name)

    NOTE: slow_check() is used in array scheduling only.
        Score-based scheduling uses other code,
        in which this bug is not yet fixed.
        Locality scheduling doesn't support HR or HAV at all.
        This should be unified.


svn path=/trunk/boinc/; revision=24484
2011-10-26 07:15:22 +00:00
David Anderson c2cdaacf89 - scheduler: fix bugs that broke work fetch for anonymous platform;
don't send irrelevant messages to anon platform clients


svn path=/trunk/boinc/; revision=24326
2011-10-03 23:43:53 +00:00
David Anderson a4cccec2cc - scheduler: revise [21428] to include non-anonymous-platform,
and change the ratio limit from 2 to 10.


svn path=/trunk/boinc/; revision=24217
2011-09-15 06:53:01 +00:00
David Anderson 760c16eb03 - scheduler: in work_needed(), ignore requests for resource
types for which we have no app versions
- client: if too many <coproc> elements in cc_config.xml,
    detect it and inform user


svn path=/trunk/boinc/; revision=24144
2011-09-08 18:55:55 +00:00
David Anderson 8870839583 - scheduler: use new XML parser for most request elements
- scheduler: parse NVIDIA as well as CUDA in app version descriptions
    (workaround for recent clients)


svn path=/trunk/boinc/; revision=24136
2011-09-06 22:53:48 +00:00
David Anderson 248282685e - scheduler: add a debug message
svn path=/trunk/boinc/; revision=24134
2011-09-06 19:34:15 +00:00
David Anderson 1bf54d11ff - client: send all running jobs a "reread app info" message
when global prefs change
    (they might have to suspend or resume network activity)


svn path=/trunk/boinc/; revision=24084
2011-08-30 21:34:27 +00:00
David Anderson 0059d6bf78 - scheduler: don't send user a message when there is no
app version for their platform for a particular app.
    The may be versions for other apps which don't have jobs right now.
    TODO: send a message if there are no versions of ANY app
    for any platform.
- fix makefile indentation, caused manager to not be built


svn path=/trunk/boinc/; revision=24052
2011-08-27 02:54:39 +00:00
David Anderson 3b73c8dc0a - db_purge: make zip compression work (from Teemu Mannermaa)
- get rid of a few compile warnings


svn path=/trunk/boinc/; revision=23789
2011-07-01 02:12:11 +00:00
David Anderson 9cfb88c3ea - scheduler: when creating HOST_APP_VERSION records,
initialize the n_jobs_today field correctly


svn path=/trunk/boinc/; revision=23637
2011-06-06 04:10:59 +00:00
David Anderson 436415cfe1 - scheduler, back end: add "homogeneous app version" feature.
Lets you specify, on a per-app basis,
    that all instances should be done using the same app version.
    This is for validation in the presence of GPUs.
- scheduler: code cleanup
    - Instead of adding a bunch of non-DB fields to RESULT,
        used a derived class SCHED_DB_RESULT.
    - Instead of storing a pointer to BEST_APP_VERSION in RESULT,
        store the structure itself.
        This simplifies the memory allocation situation.
- client: condition "Got server request to delete file" messages
    on <file_xfer_debug>


svn path=/trunk/boinc/; revision=23636
2011-06-06 03:40:42 +00:00
David Anderson 6d1133fb1d - scheduler: add <user_filter> config option.
If set, and a WU has nonzero batch,
    it is interpreted as a user ID,
    and the job will be sent only to hosts with that user ID.

    Note: the use of workunit.batch is arbitrary;
    we could also use workunit.opaque or other deprecated field.


svn path=/trunk/boinc/; revision=23556
2011-05-17 21:11:39 +00:00
David Anderson 8a4c3dccf3 - scheduler: if an in-progress limit is given in config_aux.xml,
and <per_proc> is not specified, default it to false.
- scheduler: add some log messages


svn path=/trunk/boinc/; revision=23555
2011-05-17 19:11:44 +00:00
David Anderson a2fc8edcae - scheduler: per-processor limits should be based on
"effective" # of processors (taking prefs into account)
    rather than the physical number


svn path=/trunk/boinc/; revision=23547
2011-05-13 22:04:10 +00:00
David Anderson 597320db39 - scheduler: compile fixes
svn path=/trunk/boinc/; revision=23281
2011-03-25 22:47:49 +00:00