Commit Graph

427 Commits

Author SHA1 Message Date
David Anderson 234234569b - Improvements to single-job submission:
1) the assimilator creates a job_summary_WUID file saying
        - what host did the job
        - error status if any
        - info about missing output files
    2) all output files are now optional.
    3) the assimilator deletes temp files (templates, job dir file)

svn path=/trunk/boinc/; revision=15551
2008-07-04 19:41:58 +00:00
David Anderson a5a7332b6e svn path=/trunk/boinc/; revision=15549 2008-07-03 20:17:51 +00:00
David Anderson e9fe83cf8b - boinc_submit: add --jobs and --abort options
svn path=/trunk/boinc/; revision=15547
2008-07-03 18:41:13 +00:00
David Anderson 9299d34c20 svn path=/trunk/boinc/; revision=15545 2008-07-03 18:08:28 +00:00
David Anderson 540513fc7e - typo in make_project
svn path=/trunk/boinc/; revision=15542
2008-07-02 21:59:40 +00:00
David Anderson 16b1305db7 - server code: at some point I made a global var "SCHED_CONFIG config",
mostly so that the parse function could assume
    that everything was initially zero.
    However, various back-end functions pass around SCHED_CONFIG&
    as an argument (also named "config").
    This creates a shadow, which is always bad.
    Worse is the possibility that some projects have back-end programs
    that have a SCHED_CONFIG variable that's automatic,
    and therefore isn't zero initially,
    and therefore isn't parsing correctly.

    To fix this, I changed the 2 vectors in SCHED_CONFIG into pointers,
    and have the parse routine zero the structure.
    I was tempted to remove the SCHED_CONFIG& args to back-end functions,
    but this would have broken some projects' code.
    I did, however, change the name from config to config_loc
    to avoid shadowing.

    Also fixed various other compiler warnings.

svn path=/trunk/boinc/; revision=15541
2008-07-02 17:24:53 +00:00
David Anderson 72639010cd - create_work: couldn't connect to database because it had
an automatic var SCHED_CONFIG (hence nonzero)
- make_project: include xadd in list of instructions
- boinc_submit: fix bugs in --stdin

svn path=/trunk/boinc/; revision=15536
2008-07-01 23:40:46 +00:00
David Anderson 0f60f0d42b - scheduler: fix bug where a pointer to a 32-bit time field
was being cast to a pointer to 64-bit time field
    (this caused wrong results on 64-bit servers)
    From Sharov Artyom.

svn path=/trunk/boinc/; revision=15517
2008-06-30 23:05:16 +00:00
David Anderson 70eefb4828 - make_project: change the --test_app option so that it gets the
app executable from boinc_samples/example_app/uc2,
    rather than boinc/apps/upper_case.
    (boinc_samples must be installed in the same dir as boinc).

    The reason: the BOINC autoconf-based build process
    produces executables that have zillions of shared library dependencies,
    and generally won't run on machines other than
    the one where they were built.
    Executables built in boinc_samples/ don't have this problem.

svn path=/trunk/boinc/; revision=15509
2008-06-27 18:59:33 +00:00
David Anderson 4387e4491b - Fix emacs mode line in Makefile.am's
svn path=/trunk/boinc/; revision=15375
2008-06-08 22:18:52 +00:00
David Anderson 3054a5f6b3 - validator/assimilator: fix bug in 2 June checkin
(xml_doc_in wasn't included in DB query)

svn path=/trunk/boinc/; revision=15363
2008-06-05 18:36:31 +00:00
David Anderson 47d29e2b76 - make_project: when it's done it prints out a bunch of instructions.
These are more than 24 lines, so in the VMWare BOINC VM
    they scroll off the window and are lost.
    Instead, write them to a file (project/project.readme).
    Also, only write the message about example app if -test_app

svn path=/trunk/boinc/; revision=15358
2008-06-04 23:27:01 +00:00
Eric J. Korpela 70baf36e93 - Added database table credit_multiplier and script to be run daily
(calculate_credit_multiplier) to determine what factor to multiply
   claimed credit by before insertion into the database.  Changes to scheduler
   to implement have not yet been checked in.


svn path=/trunk/boinc/; revision=15309
2008-05-28 17:28:19 +00:00
David Anderson b33276bee3 - compile fixes for gcc 4.x; fixes #592
svn path=/trunk/boinc/; revision=15007
2008-04-02 19:33:12 +00:00
David Anderson 6af9f66b4e - DB/feeder/scheduler: change app_version.xml_doc from blob to mediumblob,
and change the correspending structure field from 64KB to 256KB
    (could increase this if needed).
    This is needed to handle app versions with lots (> 100) of files
- change LARGE_BLOB_SIZE to BLOB_SIZE a bunch of places
- Change COPROCS from vector<COPROC> to vector<COPROC*>.
    Otherwise the right virtual functions of COPROCs don't get called

svn path=/trunk/boinc/; revision=14986
2008-03-31 16:19:45 +00:00
David Anderson 13400c9516 Changes for multithread app support:
- update_versions: use __ (not :) as separator for plan class
- client: add plan_class to APP_VERSION;
    an app version is now identified by platform/version/plan_class
- client CPU scheduler: don't assume apps use 1 CPU
- client: add avg_ncpus, max_cpus, flops, cmdline to RESULT
- scheduler: implement app planning scheme

Other changes:

- client: if symlink() fails, make a XML soft link instead
    (for Unix running off a FAT32 FS)
- client: don't accept nonpositive resource share from AMS
- daemons and DB: check for error returns from enumerations,
    and exit if so.  Thus, if the MySQL server goes down,
    all the daemons will soon exit.
    The cron script will restart them every 5 min,
    so when the DB server comes back up so will the project.
- web: show empty max CPU % as ---
- API: get rid of all_threads_cpu_time option (always the case now)


svn path=/trunk/boinc/; revision=14966
2008-03-27 18:25:29 +00:00
David Anderson 6ba8928537 - client (Unix): use symbolic links instead of XML link files.
This will allow applications to access shared libraries
    by their logical names,
    which simplifies things when you have multiple platforms,
    and the apps (or libraries) refer to shared libraries
    by names that don't include the platform.
    This shouldn't impact the API - boinc_resolve_filename()
    will open the file, see that it's not an XML link file,
    and the symbolic link will be used directly.
    At some point we can change boinc_resolve_filename() on Unix
    to be a no-op if the client is 6.2+
- client (Unix):
    Append the slot dir to LD_LIBRARY_PATH.

svn path=/trunk/boinc/; revision=14944
2008-03-19 16:54:36 +00:00
Reinhard Prix ebcbbb657e fixed internal library dependencies in Makefile.am's
svn path=/trunk/boinc/; revision=14918
2008-03-15 03:26:14 +00:00
David Anderson 815b8fc043 Various preparation for handling multithreaded apps
and apps that use coprocessors.
There now can be several app_versions for the same
(app, platform, version_num) combination.
This changes a number of things.

- Added app_version.plan_class field to DB
- update_versions now looks for a :plan-class in the
    file or directory name, and puts it in the app_version's DB record
- Change uniqueness constraint to include plan_class
- Feeder: the feeder was putting non-deprecated app_versions
    in shared mem, and leaving it to the scheduler to
    find the latest version for a given platform.
    This is dumb.
    Instead, for each app/platform pair the feeder now
    finds the highest version number of a non-deprecated app version,
    and enumerates all non-deprecated app_versions with that
    app/platform/version
- Scheduler: add a BEST_APP_VERSION data structure that keeps track,
    for each app, what the best app_version is for this host.
    This saves the work of recomputing it for each job.

svn path=/trunk/boinc/; revision=14906
2008-03-13 22:57:24 +00:00
David Anderson 95772cba77 - removed boinc_ncpus_available() and boinc_nthreads() calls.
The design has been changed to constant #threads per app version
    Various changes from Kevin Reed/WCG:
    - server: add workunit.rsc_bandwidth_bound: if nonzero,
        send this WU only to hosts with that much download bandwidth
    - assimilators: if a handler returns DEFER_ASSIMILATION,
        the WU remains in INIT state and will be handled when the
        next instance completes.
        Useful if you want the assimilator to see all instances.
    - scheduler: when setting result.outcome = DETACHED,
        set received_time to now
    - scheduler: removed the reliable_time and reliable_min_avg_credit
        options
    - scheduler/web: add optional <allow_non_preferred_projects>
        in project preferences.
        If present, user will accept work from non-selected apps
        if no work is available for selected apps
    - scheduler: improved messages for projects with multiple apps
    - scheduler: added config options
        <granted_credit_weight> and <granted_credit_ramp_up>.
        Used in calculating host.claimed_credit_per_cpu_sec,
        but I'm not sure how.
    - Added two new credit-granting formulas (validate_util.C):
        stddev_credit() and two_credit()
    - server DB: add rollback_transaction() and affected_rows() to DB_CONN

    NOTE: DB update required

svn path=/trunk/boinc/; revision=14870
2008-03-07 21:13:01 +00:00
David Anderson 4e3764df34 - server: debugged single-job submission feature.
It works, at least in my test case.
- boinc_submit script:
    show status every 10 seconds until completed
    add --wait option to show status for previous submission
- Added get_logical_name() in validate_util.C
- create_work: change default target_nresults from 5 to 2,
    max_error_results from 10 to 3,
    max_total_results from 20 to 10,
    max_success_results from 10 to 6
- boinc_cmd: show full usage if no args
- project setup: copy single_job_assimilator

svn path=/trunk/boinc/; revision=14847
2008-03-05 20:16:10 +00:00
David Anderson b20f2d915c - server: added a mechanism for submitting jobs as:
> boinc_submit --infile foo --outfile blah program --args
    This will run "program" on a remote host,
    with the given input, output files, and command-line args

    It manages everything for you.
    You don't have to worry about apps, app_versions,
    template files, wrapper job files, etc.
    See: http://boinc.berkeley.edu/trac/wiki/SingleJob

    Initial checkin - not debugged yet.

svn path=/trunk/boinc/; revision=14842
2008-03-04 23:50:38 +00:00
David Anderson 1106da6ec6 - transitioner: we're not supposed to ever see "assigned" WUs.
But if we do, set their transitioner time to plus infinity
    so that we don't see them again.
    (otherwise we go into an infinite loop)
- DB code: remove "high_priority" from queries not from scheduler
    (should probably remove them from there too)
- file_deleter: print error msg if apache user doesn't exist

svn path=/trunk/boinc/; revision=14835
2008-03-03 19:33:32 +00:00
David Anderson c8384113e4 - changes to many files to fix gcc4.3 compile
svn path=/trunk/boinc/; revision=14811
2008-02-27 23:26:38 +00:00
David Anderson 86857a5452 - Scheduler: fix bugs in work assignment feature
svn path=/trunk/boinc/; revision=14810
2008-02-27 22:26:37 +00:00
David Anderson b6cc885abf - server: make the special substring for assigned WUs
into a #define's symbol (ASSIGNED_WU_STR)
- scheduler: when send client command to abort a WU,
    include a reason code in the scheduler log

svn path=/trunk/boinc/; revision=14798
2008-02-26 17:24:29 +00:00
David Anderson 54519a4ee1 - Server: add "job assignment" feature.
Lets you assign a WU to a particular host,
    to one or all hosts belonging to a user or team, or to all hosts.
    See http://boinc.berkeley.edu/trac/wiki/AssignedWork
    Disabled unless you include <enable_assignment> in config.xml
    Uses a new DB table.
    Tested but only a little.
- Server: code cleanup; moved result-handling to a new file,
    and removed the PLATFORM_LIST arg to everything
    (put it in SCHEDULER_REQUEST instead)

svn path=/trunk/boinc/; revision=14767
2008-02-21 00:47:50 +00:00
David Anderson 69cfcebef4 - file upload handler: the FCGI version wasn't handling signatures,
because the code to read keys in FCGI was commented out
    (and was in fact wrong).
    I fixed it and uncommented it.
- make_project and upgrade: copy lib/crypt_prog to project/bin/

svn path=/trunk/boinc/; revision=14693
2008-02-06 19:32:51 +00:00
David Anderson a1664cc070 - scheduler: simplified Carl's checkin (avoid virtual destructors)
- API: small tweak for standalone

svn path=/trunk/boinc/; revision=14679
2008-02-05 21:26:43 +00:00
David Anderson 94fdec8b70 - scheduler: send log messages to file, rather than httpd error log,
when using FCGI (from Carl Christensen)

svn path=/trunk/boinc/; revision=14678
2008-02-05 20:16:57 +00:00
David Anderson daac96c05f - create_work: added -opaque option (to specify workunit.opaque)
svn path=/trunk/boinc/; revision=14605
2008-01-21 14:02:22 +00:00
David Anderson 5104b76439 - XML fix in backend lib
svn path=/trunk/boinc/; revision=14482
2008-01-07 22:04:54 +00:00
David Anderson 757c41b1fa - web: add RSS feed for notifications
- make_project: disable community-related periodic tasks
    for newly-created projects

svn path=/trunk/boinc/; revision=14468
2008-01-04 22:59:21 +00:00
David Anderson 57f0bbc6e5 - make_project: prepend "./" to config.xml script names in case "." is not
in Apache's search path
- web: show UOTD text correctly
- web: memoize profile_screening() in attempt to fix out-of-memory error
    when building profile pages on S@h
- web: move gallery.inc into update_profile_pages and remove it

svn path=/trunk/boinc/; revision=14460
2008-01-03 17:24:28 +00:00
David Anderson dddd1fc501 - fixed bug in upgrade
svn path=/trunk/boinc/; revision=14451
2008-01-01 18:07:13 +00:00
David Anderson f183b6f47f - web: add general-purpose notification mechanism.
Users can choose whether to get 1 email per notification,
    a daily "digest" email, or no email.
    (All notifications are shown on the Account page).
    Currently used for:
    - Friend requests and confirmations
    - Posts to subscribed threads
    - Private messages

    NOTE: To implement the "daily digest" feature, projects must add
    a periodic task for html/ops/notify.php to config.xml
- web: have project_footer() generate links for
    Account Page and Message Boards as well as Home
    NOTE: projects that want this change
    will have to modify their own project.inc.

svn path=/trunk/boinc/; revision=14447
2007-12-30 22:02:16 +00:00
David Anderson 40f594b07c - upgrade script: fix it, and add --web_only option (fixes #527)
- web: replace "Result" by "Task" a couple of places

svn path=/trunk/boinc/; revision=14446
2007-12-28 05:23:52 +00:00
David Anderson 11dac5e80c - make_project: add --web_only option.
This lets you create a project for Bossa or Bolt
    without compiling the C++ stuff, which is not used by Bossa or Bolt.
- DB schema: change type= to engine= to avoid MySQL warning messages

svn path=/trunk/boinc/; revision=14437
2007-12-21 23:39:25 +00:00
David Anderson b42c68e5dd - API: switch back to structure where the worker thread
calls getrusage() in a signal handler to find its CPU time.
    Apparently there are some kernels and/or thread libraries
    where getrusage() does NOT return the total CPU time of
    all threads in the process,
    so in these cases calling it from the timer thread doesn't work.
    
    This was originally changed because of concerns that
    calling getrusage() in a signal handler was dangerous;
    we'll see if any problems re-emerge.

    The alternative of reading /proc/PID/stats doesn't seem viable;
    format may not be standardized.

svn path=/trunk/boinc/; revision=14433
2007-12-21 19:23:22 +00:00
Eric J. Korpela fea4674524 Fix sql bug in vote_monitor
svn path=/trunk/boinc/; revision=13949
2007-10-24 20:14:20 +00:00
Eric J. Korpela 706b283b19 Added ability for forum moderators to vote for banishment of a poster
New db tables banishment_vote and banishment_votes
Daemon script vote_monitor for timing the votes
New configuration option <moderators_vote_to_ban/>



svn path=/trunk/boinc/; revision=13917
2007-10-19 20:01:36 +00:00
Frank Thomas fbcfeaf456 - Removed the svn:executable property from files that should not be executable,
like source code and text files. I skipped to check most files in html/
  and mac_*/ though.
- Added svn:executable to tools/watch_tcp because it has a shebang.


svn path=/trunk/boinc/; revision=13819
2007-10-10 09:25:40 +00:00
David Anderson 383b21dd5a - update_versions now generates <api_version> for all app versions
- user web: don't XML-escape team names in DB
- deleted obsolete files
- calloc() return check, fixes #277

svn path=/trunk/boinc/; revision=13817
2007-10-09 22:58:49 +00:00
David Anderson d50a183dcb - remove "updater" from tools/ Makefile
svn path=/trunk/boinc/; revision=13815
2007-10-09 18:29:51 +00:00
Rom Walton 38d4aecc0b Delete outdated .cvsignore files
svn path=/trunk/boinc/; revision=13814
2007-10-09 18:03:21 +00:00
Charlie Fenton f0aff10b8e Fix updater build on Windows after sandbox reorganization
svn path=/trunk/boinc/; revision=13810
2007-10-09 13:28:00 +00:00
Frank Thomas 3bfc78b511 Updated the postal address of the Free Software Foundation in all license headers. See http://lists.ssl.berkeley.edu/pipermail/boinc_dev/2007-October/008939.html for reference.
svn path=/trunk/boinc/; revision=13804
2007-10-09 11:35:47 +00:00
David Anderson c260a46b0d - update_versions_v6 doesn't set min_core_version
svn path=/trunk/boinc/; revision=13653
2007-09-25 17:13:38 +00:00
David Anderson bc5b979afb - Added new script "update_versions_v6"; use this instead of
update_versions to add version 6 apps.
    It looks for API_VERSION string in main executable,
    adds the API version to the app_version XML,
    and sets min_core_version to 6 for version 6+ apps
- API: include API_VERSION string
- convert tabs to spaces here and there
- scheduler: parse unused elements in <net_stats>
- ops/show_log.php: if no URL args, just show form (fixes #415)
- client: parse and store api_version (not used yet)

svn path=/trunk/boinc/; revision=13627
2007-09-21 18:10:54 +00:00
David Anderson 14bc500cab - user web: added mechanism for recording team membership changes.
These are stored in a new DB table, "team_delta".
    The team founder can view the history in HTML or XML
    (links on Management Functions page)

svn path=/trunk/boinc/; revision=13231
2007-07-27 18:30:10 +00:00