Commit Graph

415 Commits

Author SHA1 Message Date
Eric J. Korpela 70baf36e93 - Added database table credit_multiplier and script to be run daily
(calculate_credit_multiplier) to determine what factor to multiply
   claimed credit by before insertion into the database.  Changes to scheduler
   to implement have not yet been checked in.


svn path=/trunk/boinc/; revision=15309
2008-05-28 17:28:19 +00:00
David Anderson b33276bee3 - compile fixes for gcc 4.x; fixes #592
svn path=/trunk/boinc/; revision=15007
2008-04-02 19:33:12 +00:00
David Anderson 6af9f66b4e - DB/feeder/scheduler: change app_version.xml_doc from blob to mediumblob,
and change the correspending structure field from 64KB to 256KB
    (could increase this if needed).
    This is needed to handle app versions with lots (> 100) of files
- change LARGE_BLOB_SIZE to BLOB_SIZE a bunch of places
- Change COPROCS from vector<COPROC> to vector<COPROC*>.
    Otherwise the right virtual functions of COPROCs don't get called

svn path=/trunk/boinc/; revision=14986
2008-03-31 16:19:45 +00:00
David Anderson 13400c9516 Changes for multithread app support:
- update_versions: use __ (not :) as separator for plan class
- client: add plan_class to APP_VERSION;
    an app version is now identified by platform/version/plan_class
- client CPU scheduler: don't assume apps use 1 CPU
- client: add avg_ncpus, max_cpus, flops, cmdline to RESULT
- scheduler: implement app planning scheme

Other changes:

- client: if symlink() fails, make a XML soft link instead
    (for Unix running off a FAT32 FS)
- client: don't accept nonpositive resource share from AMS
- daemons and DB: check for error returns from enumerations,
    and exit if so.  Thus, if the MySQL server goes down,
    all the daemons will soon exit.
    The cron script will restart them every 5 min,
    so when the DB server comes back up so will the project.
- web: show empty max CPU % as ---
- API: get rid of all_threads_cpu_time option (always the case now)


svn path=/trunk/boinc/; revision=14966
2008-03-27 18:25:29 +00:00
David Anderson 6ba8928537 - client (Unix): use symbolic links instead of XML link files.
This will allow applications to access shared libraries
    by their logical names,
    which simplifies things when you have multiple platforms,
    and the apps (or libraries) refer to shared libraries
    by names that don't include the platform.
    This shouldn't impact the API - boinc_resolve_filename()
    will open the file, see that it's not an XML link file,
    and the symbolic link will be used directly.
    At some point we can change boinc_resolve_filename() on Unix
    to be a no-op if the client is 6.2+
- client (Unix):
    Append the slot dir to LD_LIBRARY_PATH.

svn path=/trunk/boinc/; revision=14944
2008-03-19 16:54:36 +00:00
Reinhard Prix ebcbbb657e fixed internal library dependencies in Makefile.am's
svn path=/trunk/boinc/; revision=14918
2008-03-15 03:26:14 +00:00
David Anderson 815b8fc043 Various preparation for handling multithreaded apps
and apps that use coprocessors.
There now can be several app_versions for the same
(app, platform, version_num) combination.
This changes a number of things.

- Added app_version.plan_class field to DB
- update_versions now looks for a :plan-class in the
    file or directory name, and puts it in the app_version's DB record
- Change uniqueness constraint to include plan_class
- Feeder: the feeder was putting non-deprecated app_versions
    in shared mem, and leaving it to the scheduler to
    find the latest version for a given platform.
    This is dumb.
    Instead, for each app/platform pair the feeder now
    finds the highest version number of a non-deprecated app version,
    and enumerates all non-deprecated app_versions with that
    app/platform/version
- Scheduler: add a BEST_APP_VERSION data structure that keeps track,
    for each app, what the best app_version is for this host.
    This saves the work of recomputing it for each job.

svn path=/trunk/boinc/; revision=14906
2008-03-13 22:57:24 +00:00
David Anderson 95772cba77 - removed boinc_ncpus_available() and boinc_nthreads() calls.
The design has been changed to constant #threads per app version
    Various changes from Kevin Reed/WCG:
    - server: add workunit.rsc_bandwidth_bound: if nonzero,
        send this WU only to hosts with that much download bandwidth
    - assimilators: if a handler returns DEFER_ASSIMILATION,
        the WU remains in INIT state and will be handled when the
        next instance completes.
        Useful if you want the assimilator to see all instances.
    - scheduler: when setting result.outcome = DETACHED,
        set received_time to now
    - scheduler: removed the reliable_time and reliable_min_avg_credit
        options
    - scheduler/web: add optional <allow_non_preferred_projects>
        in project preferences.
        If present, user will accept work from non-selected apps
        if no work is available for selected apps
    - scheduler: improved messages for projects with multiple apps
    - scheduler: added config options
        <granted_credit_weight> and <granted_credit_ramp_up>.
        Used in calculating host.claimed_credit_per_cpu_sec,
        but I'm not sure how.
    - Added two new credit-granting formulas (validate_util.C):
        stddev_credit() and two_credit()
    - server DB: add rollback_transaction() and affected_rows() to DB_CONN

    NOTE: DB update required

svn path=/trunk/boinc/; revision=14870
2008-03-07 21:13:01 +00:00
David Anderson 4e3764df34 - server: debugged single-job submission feature.
It works, at least in my test case.
- boinc_submit script:
    show status every 10 seconds until completed
    add --wait option to show status for previous submission
- Added get_logical_name() in validate_util.C
- create_work: change default target_nresults from 5 to 2,
    max_error_results from 10 to 3,
    max_total_results from 20 to 10,
    max_success_results from 10 to 6
- boinc_cmd: show full usage if no args
- project setup: copy single_job_assimilator

svn path=/trunk/boinc/; revision=14847
2008-03-05 20:16:10 +00:00
David Anderson b20f2d915c - server: added a mechanism for submitting jobs as:
> boinc_submit --infile foo --outfile blah program --args
    This will run "program" on a remote host,
    with the given input, output files, and command-line args

    It manages everything for you.
    You don't have to worry about apps, app_versions,
    template files, wrapper job files, etc.
    See: http://boinc.berkeley.edu/trac/wiki/SingleJob

    Initial checkin - not debugged yet.

svn path=/trunk/boinc/; revision=14842
2008-03-04 23:50:38 +00:00
David Anderson 1106da6ec6 - transitioner: we're not supposed to ever see "assigned" WUs.
But if we do, set their transitioner time to plus infinity
    so that we don't see them again.
    (otherwise we go into an infinite loop)
- DB code: remove "high_priority" from queries not from scheduler
    (should probably remove them from there too)
- file_deleter: print error msg if apache user doesn't exist

svn path=/trunk/boinc/; revision=14835
2008-03-03 19:33:32 +00:00
David Anderson c8384113e4 - changes to many files to fix gcc4.3 compile
svn path=/trunk/boinc/; revision=14811
2008-02-27 23:26:38 +00:00
David Anderson 86857a5452 - Scheduler: fix bugs in work assignment feature
svn path=/trunk/boinc/; revision=14810
2008-02-27 22:26:37 +00:00
David Anderson b6cc885abf - server: make the special substring for assigned WUs
into a #define's symbol (ASSIGNED_WU_STR)
- scheduler: when send client command to abort a WU,
    include a reason code in the scheduler log

svn path=/trunk/boinc/; revision=14798
2008-02-26 17:24:29 +00:00
David Anderson 54519a4ee1 - Server: add "job assignment" feature.
Lets you assign a WU to a particular host,
    to one or all hosts belonging to a user or team, or to all hosts.
    See http://boinc.berkeley.edu/trac/wiki/AssignedWork
    Disabled unless you include <enable_assignment> in config.xml
    Uses a new DB table.
    Tested but only a little.
- Server: code cleanup; moved result-handling to a new file,
    and removed the PLATFORM_LIST arg to everything
    (put it in SCHEDULER_REQUEST instead)

svn path=/trunk/boinc/; revision=14767
2008-02-21 00:47:50 +00:00
David Anderson 69cfcebef4 - file upload handler: the FCGI version wasn't handling signatures,
because the code to read keys in FCGI was commented out
    (and was in fact wrong).
    I fixed it and uncommented it.
- make_project and upgrade: copy lib/crypt_prog to project/bin/

svn path=/trunk/boinc/; revision=14693
2008-02-06 19:32:51 +00:00
David Anderson a1664cc070 - scheduler: simplified Carl's checkin (avoid virtual destructors)
- API: small tweak for standalone

svn path=/trunk/boinc/; revision=14679
2008-02-05 21:26:43 +00:00
David Anderson 94fdec8b70 - scheduler: send log messages to file, rather than httpd error log,
when using FCGI (from Carl Christensen)

svn path=/trunk/boinc/; revision=14678
2008-02-05 20:16:57 +00:00
David Anderson daac96c05f - create_work: added -opaque option (to specify workunit.opaque)
svn path=/trunk/boinc/; revision=14605
2008-01-21 14:02:22 +00:00
David Anderson 5104b76439 - XML fix in backend lib
svn path=/trunk/boinc/; revision=14482
2008-01-07 22:04:54 +00:00
David Anderson 757c41b1fa - web: add RSS feed for notifications
- make_project: disable community-related periodic tasks
    for newly-created projects

svn path=/trunk/boinc/; revision=14468
2008-01-04 22:59:21 +00:00
David Anderson 57f0bbc6e5 - make_project: prepend "./" to config.xml script names in case "." is not
in Apache's search path
- web: show UOTD text correctly
- web: memoize profile_screening() in attempt to fix out-of-memory error
    when building profile pages on S@h
- web: move gallery.inc into update_profile_pages and remove it

svn path=/trunk/boinc/; revision=14460
2008-01-03 17:24:28 +00:00
David Anderson dddd1fc501 - fixed bug in upgrade
svn path=/trunk/boinc/; revision=14451
2008-01-01 18:07:13 +00:00
David Anderson f183b6f47f - web: add general-purpose notification mechanism.
Users can choose whether to get 1 email per notification,
    a daily "digest" email, or no email.
    (All notifications are shown on the Account page).
    Currently used for:
    - Friend requests and confirmations
    - Posts to subscribed threads
    - Private messages

    NOTE: To implement the "daily digest" feature, projects must add
    a periodic task for html/ops/notify.php to config.xml
- web: have project_footer() generate links for
    Account Page and Message Boards as well as Home
    NOTE: projects that want this change
    will have to modify their own project.inc.

svn path=/trunk/boinc/; revision=14447
2007-12-30 22:02:16 +00:00
David Anderson 40f594b07c - upgrade script: fix it, and add --web_only option (fixes #527)
- web: replace "Result" by "Task" a couple of places

svn path=/trunk/boinc/; revision=14446
2007-12-28 05:23:52 +00:00
David Anderson 11dac5e80c - make_project: add --web_only option.
This lets you create a project for Bossa or Bolt
    without compiling the C++ stuff, which is not used by Bossa or Bolt.
- DB schema: change type= to engine= to avoid MySQL warning messages

svn path=/trunk/boinc/; revision=14437
2007-12-21 23:39:25 +00:00
David Anderson b42c68e5dd - API: switch back to structure where the worker thread
calls getrusage() in a signal handler to find its CPU time.
    Apparently there are some kernels and/or thread libraries
    where getrusage() does NOT return the total CPU time of
    all threads in the process,
    so in these cases calling it from the timer thread doesn't work.
    
    This was originally changed because of concerns that
    calling getrusage() in a signal handler was dangerous;
    we'll see if any problems re-emerge.

    The alternative of reading /proc/PID/stats doesn't seem viable;
    format may not be standardized.

svn path=/trunk/boinc/; revision=14433
2007-12-21 19:23:22 +00:00
Eric J. Korpela fea4674524 Fix sql bug in vote_monitor
svn path=/trunk/boinc/; revision=13949
2007-10-24 20:14:20 +00:00
Eric J. Korpela 706b283b19 Added ability for forum moderators to vote for banishment of a poster
New db tables banishment_vote and banishment_votes
Daemon script vote_monitor for timing the votes
New configuration option <moderators_vote_to_ban/>



svn path=/trunk/boinc/; revision=13917
2007-10-19 20:01:36 +00:00
Frank Thomas fbcfeaf456 - Removed the svn:executable property from files that should not be executable,
like source code and text files. I skipped to check most files in html/
  and mac_*/ though.
- Added svn:executable to tools/watch_tcp because it has a shebang.


svn path=/trunk/boinc/; revision=13819
2007-10-10 09:25:40 +00:00
David Anderson 383b21dd5a - update_versions now generates <api_version> for all app versions
- user web: don't XML-escape team names in DB
- deleted obsolete files
- calloc() return check, fixes #277

svn path=/trunk/boinc/; revision=13817
2007-10-09 22:58:49 +00:00
David Anderson d50a183dcb - remove "updater" from tools/ Makefile
svn path=/trunk/boinc/; revision=13815
2007-10-09 18:29:51 +00:00
Rom Walton 38d4aecc0b Delete outdated .cvsignore files
svn path=/trunk/boinc/; revision=13814
2007-10-09 18:03:21 +00:00
Charlie Fenton f0aff10b8e Fix updater build on Windows after sandbox reorganization
svn path=/trunk/boinc/; revision=13810
2007-10-09 13:28:00 +00:00
Frank Thomas 3bfc78b511 Updated the postal address of the Free Software Foundation in all license headers. See http://lists.ssl.berkeley.edu/pipermail/boinc_dev/2007-October/008939.html for reference.
svn path=/trunk/boinc/; revision=13804
2007-10-09 11:35:47 +00:00
David Anderson c260a46b0d - update_versions_v6 doesn't set min_core_version
svn path=/trunk/boinc/; revision=13653
2007-09-25 17:13:38 +00:00
David Anderson bc5b979afb - Added new script "update_versions_v6"; use this instead of
update_versions to add version 6 apps.
    It looks for API_VERSION string in main executable,
    adds the API version to the app_version XML,
    and sets min_core_version to 6 for version 6+ apps
- API: include API_VERSION string
- convert tabs to spaces here and there
- scheduler: parse unused elements in <net_stats>
- ops/show_log.php: if no URL args, just show form (fixes #415)
- client: parse and store api_version (not used yet)

svn path=/trunk/boinc/; revision=13627
2007-09-21 18:10:54 +00:00
David Anderson 14bc500cab - user web: added mechanism for recording team membership changes.
These are stored in a new DB table, "team_delta".
    The team founder can view the history in HTML or XML
    (links on Management Functions page)

svn path=/trunk/boinc/; revision=13231
2007-07-27 18:30:10 +00:00
David Anderson 7594e57b05 - update_versions: don't accept a multi-file application
unless there's a main program with same name as dir
    (previously it was treating the first executable file found
    as the main program, with occasionally disastrous consequences).

svn path=/trunk/boinc/; revision=13201
2007-07-20 20:19:35 +00:00
David Anderson e1a0c552e2 - compile fixes for Win
client/
		acct_setup.C
		cs_prefs.C
		cs_trickle.C
	lib/
		filesys.C
		str_util.C,h
		util.C,h
	tools/
		updater.C
	win_build/
		sim.vcproj

svn path=/trunk/boinc/; revision=12985
2007-06-22 20:17:08 +00:00
David Anderson 6b4e912b27 - random compile warning fixes
client/
    auto_update.C
    file_names.h
    scheduler_op.C
    time_stats.C
lib/
    util.C,h
tools/
    create_work.C
    updater.C

svn path=/trunk/boinc/; revision=12971
2007-06-20 23:16:30 +00:00
David Anderson 45f04aa04b - back end: make it possible to run multiple instances of make_work,
for WUs of different applications
    (need to count unsent results separately by app)
- feeder: major code cleanup
    - application interleaving (for -allapps) is now done
        by building a static slot-to-app array "app_indices".
        Fractional weights now work correctly.
    - enum sizes (for -allapps) are now precomputed
        in an array "enum_sizes"
    - rename "found" (confusing!!) to "collision"
    - swapped the names of mod_select_clause and select_clause,
        to reflect what they actually are
- file deleter: in finding oldest WU, order by id instead of create_time
    (there's no index on create_time)
- user web: show "merge by name" only to host owner
- add cpu_scheduler_period() member to GLOBAL_PREFS
    (so you don't have to multiply by 60 everywhere)
- infinite() fix for HPUX

client/
    cpu_sched.C
    cs_cmdline.C
    cs_scheduler.C
    rrsim_test.C
    sim.C
    work_fetch.C
html/user/
    hosts_user.php
lib/
    parse.h
    prefs.h
sched/
    feeder.C
    file_deleter.C
    make_work.C
    sample_work_generator.C
    sched_util.C,h
tools/
    updater.C

svn path=/trunk/boinc/; revision=12968
2007-06-20 16:27:27 +00:00
David Anderson 46f999f0ba - Added project.xml entry for Playstation 3 / Linux
- Server status page: if the scheduler isn't running,
    it's almost certainly because "stop_sched" is present,
    not because Apache isn't running.
    Get rid of the use of "sched_pid".
    
tools/
    project.xml
html/ops/
    sample_server_status.php

svn path=/trunk/boinc/; revision=12931
2007-06-13 21:16:27 +00:00
David Anderson 58b39390b1 - Python tools:
- removed "add" (long deprecated)
    - version.py is needed only in test/;
        remove references to it in boinc_path_config.py,
        and don't copy it to project/bin

py/Boinc/
    boinc_path_config.py.in
    setup_project.py
tools/
    add (removed)

svn path=/trunk/boinc/; revision=12658
2007-05-11 22:24:14 +00:00
David Anderson c72634456d - make_project and upgrade now write a file 'local.revision'
in the project home directory.
        It contains the output of /usr/bin/svnversion,
        i.e. the code revision from which the server software was built.
    - Web files that create directories now give them g+rwx permissions

    html/inc/
        cache.inc
        translation.inc
    sched/
        edf_sim.C
    tools/
        make_project
        upgrade

svn path=/trunk/boinc/; revision=12521
2007-05-01 21:21:13 +00:00
David Anderson 5184dedb1e David 26 Apr 2007
- core client: set config settings to default values before
        parsing cc_config.xml.
        Otherwise, if you have a lot of log flags set,
        and tell the client to reread a config file that
        doesn't specify ANY log flags,
        you'll end up with all the flags still set.
        (from Sebastian Masch)
    - make_project: add cmdline args for update_stats in initial config

    tools/
        make_project
    client/
        log_flags.C,h

svn path=/trunk/boinc/; revision=12490
2007-04-26 21:57:42 +00:00
David Anderson b5b155e557 David 26 Apr 2007
- server: fix bug that causes PHP scripts not to run from Cron
        Fixes #129.  From Christian Beer.

    html/ops/
        update_profile_pages.php
    tools/
        run_in_ops

svn path=/trunk/boinc/; revision=12488
2007-04-26 20:24:47 +00:00
David Anderson bd06f5ae09 David 25 Apr 2007
- fixes #126 - generate cronjob file in make_project

    tools/
        make_project

svn path=/trunk/boinc/; revision=12480
2007-04-25 20:38:01 +00:00
David Anderson fe32380dcf David 23 Apr 2007
- make_project: remove --no_test_app option, add --test_app option
    - make_project: initial config in all cases has
        one_result_per_user_per_wu = 0
    - sample_assimilator: make it able to handle > 1 output file
    - boinc_cmd: make --help output more readable
    - User web: fix some undefined vars in translation.inc
        Rytis: let's try to fix PHP runtime warnings
    - DB: add default values for app table (avoid MySQL warnings)

    db/
        schema.sql
    html/
        inc/
            translation.inc
            uotd.inc
        ops/
            update_forum_activities.php
    lib/
        str_util.C
        boinc_cmd.C
    py/Boinc/
        setup_project.py
    sched/
        sample_work_generator.C
        sample_assimilator.C
    tools/
        make_project
        uc_wu, uc_result (new)

svn path=/trunk/boinc/; revision=12450
2007-04-23 16:14:47 +00:00
David Anderson b686f89d85 David 20 Apr 2007
- make_project: add periodic tasks to default config.xml:
        dp_dump, update_uotd, update_forum_activities, update_stats,
        update_profile_pages
    - backend_lib.C: refuse to create a job for which some necessary
        parameter (rsc_fpops_est, rsc_fpops_bound, etc.) is missing
    - Unix build: don't include std_fixes.h in config.h.
        std_fixes.h is (I think) deprecated
    - sched_util: default sched log level is MSG_NORMAL
    - client: add ".txt" to job log filename
    - compile fixes

    configure.ac
    api/
        reduce_main.C
    client/
        file_names.C
    lib/
        str_util.h
    sched/
        sample_work_generator.C
        sched_msgs.h
    tools/
        backend_lib.C

svn path=/trunk/boinc/; revision=12430
2007-04-20 17:14:08 +00:00