Commit Graph

294 Commits

Author SHA1 Message Date
David Anderson e8e0de8cb9 server: fix bugs when using Python assimilator framework.
We needed to look for #defines in boinc_db_types.h as well as boinc_db.h
2013-06-10 11:37:40 -07:00
David Anderson 8e2524f55f Unix build: Makefile changes for "make install", from Steffen Moeller
"make install" followed by make_project should now work
2013-05-20 15:19:13 -07:00
David Anderson 0c430ce1fa Add support for multi-size apps
See http://boinc.berkeley.edu/trac/wiki/MultiSize
The components of this include:
- DB changes:
    add size_class to workunit and result
    n_size_classes to app; >1 means multi-size
- size_regulator daemon program: change results states
    from INACTIVE to UNSENT carefully
- size_census program; writes quantile info in flat files
- transitioner: when creating results for multi-size apps,
    set server state to INACTIVE
- sched shmem (feeder): read quantile info from flat files,
    store in shared memory
- scheduler (score-based scheduling): for multi-size apps,
    add component to score function for size class.
- show_shmem: show result size class
- make_work (and other callers of count_unsent_results()):
    count both INACTIVE and UNSENT
- create_work: add --size_class cmdline option

Also:
- if get MySQL errors in upgrade, don't rewrite db_version
2013-04-25 00:27:35 -07:00
David Anderson 0eaaca174a - make_project: don't write <cgi_url> in config.xml; it's not used 2013-04-13 21:32:10 -07:00
David Anderson 6f98cf9dc0 - project setup: create a default gui_urls.xml file 2013-03-05 14:52:38 +01:00
David Anderson d79c9584f2 - project creation/upgrade: install adjust_user_priorities 2013-03-04 17:39:24 +01:00
David Anderson 5cbccf7126 - server: add stage_file to project setup; add error checking to stage_file 2013-03-01 15:31:42 +01:00
David Anderson 32da1a7e37 - server: add support for having a mixture of CPU-intensive
and non-CPU-intensive applications.
    An app can be specified as non-CPU-intensive in project.xml,
    and this attribute can be set or cleared using the admin web interface.
    Note: support for this was added to the client in 2011,
    but we didn't add server-side support at that time.
    This change is in 6.12 and later clients.


svn path=/trunk/boinc/; revision=26060
2012-08-25 04:09:24 +00:00
David Anderson 000dbadc3d - install antique file deleter as periodic task (24 hours)
when create new project


svn path=/trunk/boinc/; revision=25322
2012-02-24 03:26:30 +00:00
David Anderson d03f697456 - storage stuff
svn path=/trunk/boinc/; revision=25307
2012-02-21 20:55:09 +00:00
David Anderson 4774eeda52 - make_project: don't try to copy nonexistent file; fixes #1166
- scheduler: change Vbox app plan function to accommodate
    single and multithreaded variants


svn path=/trunk/boinc/; revision=24884
2011-12-24 05:07:20 +00:00
David Anderson 53af2ae778 - client/API: pass bool using_sandbox in APP_INIT_DATA;
says whether the client is using account-based sandboxing.
    Needed by vboxwrapper.
- web: don't show authenticator in Your Account page.
    The problem with authenticators is that if a bad guys gets yours,
    they can log in to your account even if you change your password.
    Eventually we should not use them at all
    (i.e. in cookies and sent to client).
- make_project: copy python stuff (e.g. bin/start) for web-only projects;
    generate appropriate project.readme file for web-only projects.


svn path=/trunk/boinc/; revision=24447
2011-10-21 06:58:35 +00:00
David Anderson 3410abef6f - backend API: added function cancel_jobs(minid, maxid)
for canceling jobs
- added program cancel_jobs for canceling jobs
- DB interface: it's not an error if update_fields_noid()
    affects != 1 rows


svn path=/trunk/boinc/; revision=24413
2011-10-18 07:15:04 +00:00
David Anderson 75c6032043 - fix typo in DB schema
- msg tweaks in make_project


svn path=/trunk/boinc/; revision=24253
2011-09-22 03:15:21 +00:00
David Anderson 9d2ba11e09 - back end: extend the access control system for remote job submission
and other operations.
    You can now designate a user as "manager" for a particular app.
    They can then:
    - control job-submit permissions for that app
    - deprecate/undeprecate versions of the app.
    - abort jobs for that app

    You can also designate a user a manage for the project.
    They can then edit permissions and quotas,
    as well as performing the app-specific functions for all apps.

    This is described here:
    http://boinc.berkeley.edu/trac/wiki/MultiUser#Accesscontrol

    This required some changes to the DB schema.


svn path=/trunk/boinc/; revision=24250
2011-09-21 21:26:00 +00:00
David Anderson 27e05a3da9 - server: some stuff to prepare for distributed storage
- don't create result records for uploads and downloads.
        Just create a msg_to_client record.
    - the scheduler handles file-transfer results specially;
        it makes a vector of them, then calls a project-supplied function
        handle_file_xfer_results()
    - change the interface and implementation of put_file and get_file
- client write project sched priority in GUI RPC replies,
    but not to the state file


svn path=/trunk/boinc/; revision=23857
2011-07-19 20:52:41 +00:00
David Anderson e4f104f917 - Change the way update_versions works, and reimplement it in PHP.
Problems with the old version:
    - Lots of info encoded in file and directory names: messy kludge
    - The convention that directory name == main program name
        created a bad situation when using the wrapper:
        if you made a new version of your application,
        you were forced make a new version of the wrapper
        just so it could have a different name.
    The new version is described here:
    http://boinc.berkeley.edu/trac/wiki/AppVersionNew
- Remove some Python code that suppored the old version
- client: some fixes to dir_size() that might fox
    a buffer overflow when using symbolic links


svn path=/trunk/boinc/; revision=23525
2011-05-12 04:11:40 +00:00
David Anderson 5b3c9e872b - "upgrade" script: copy sched/fcgi if it exists
svn path=/trunk/boinc/; revision=23357
2011-04-08 19:36:26 +00:00
David Anderson 07982df97a - update_versions: if FILE.file_ref_info contains <open_name>,
use that instead of physical name.
    This lets you have logical names with slashes

svn path=/trunk/boinc/; revision=23042
2011-02-16 04:57:31 +00:00
David Anderson 4dbe2c8fdf - new script sched/transitioner_catchup.php: transitions all WUs
that need it, then exits.
    For big projects coming up after an outage, run this before bin/start
    to minimize startup overload
- tweaks to PHP libraries to allow them to be used
    by scripts anywhere in the dir hierarchy


svn path=/trunk/boinc/; revision=22959
2011-01-29 06:28:10 +00:00
David Anderson 58b345fce6 - server: disable upload certificates by default.
These increase server load and it's not clear they're needed.
        

svn path=/trunk/boinc/; revision=22934
2011-01-24 19:39:50 +00:00
David Anderson 9d35970994 - server scripts (e.g. update_versions):
compute MD5 checksums be reading files in pieces instead
    of reading whole file into memory.  From Tolu Aina

svn path=/trunk/boinc/; revision=22743
2010-11-23 22:16:24 +00:00
David Anderson 723c764963 - server: add scripts to submit jobs for demo application
svn path=/trunk/boinc/; revision=22676
2010-11-10 22:54:56 +00:00
David Anderson c0612ab77f - make_project: with --test_app, copy all the executables
(for many platforms) from samples/example_app/bin
- make_project: change name of example app from uppercase to example_app.
- update_versions: allow version numbers to not have decimal points
- sample work generator: make app name and template files
    command-line options;
    default to "example_app", "example_app_in.xml", "example_app_out.xml"

svn path=/trunk/boinc/; revision=22667
2010-11-10 00:10:32 +00:00
David Anderson d384377743 - client: fix harmless compile warning
- make_project: don't try to install non-existent file

svn path=/trunk/boinc/; revision=22656
2010-11-09 05:00:49 +00:00
David Anderson 8aa29bec33 - validator: fix another bug with --credit_from_wu
- make_project, update scripts: don't quit it user_profiles
    already exists


svn path=/trunk/boinc/; revision=22630
2010-11-05 17:15:27 +00:00
David Anderson 7fcd20e020 - make_project, upgrade scripts:
create a symbolic link from html/user/user_profile to html/user_profile
    (needed to make profiles work)

svn path=/trunk/boinc/; revision=22622
2010-11-03 22:42:01 +00:00
David Anderson e97a9177d3 - python DB layer: fields with values None or ''
where being omitted from query strings.
    This is incorrect.
    For example, suppose you have an app version with nonempty plan_class,
    then you try to add a version with no plan class.
    The query would omit the "and plan_class = ''"
    so it would match the existing app version and not add a new version.
    Reported by Rytis.
    Hopefully this won't break anything.


svn path=/trunk/boinc/; revision=22616
2010-11-03 18:25:24 +00:00
David Anderson d44e403fbf - admin: fix bug with "upgrade --server_only".
It wasn't copying html/ops/db_update.php,
    so it wasn't doing necessary DB updates.
    Fixed this by always copying html/ops/*.

    Even with this fix, there is a problem when using
    the --server_only or --web_only options of upgrade:
    if any DB updates are done, they may affect the
    server code that's not being updated, resulting in crashes.

    I added a warning message in this case,
    recommending that a full upgrade be done.

svn path=/trunk/boinc/; revision=22200
2010-08-11 20:08:13 +00:00
David Anderson fb851311e0 - server: various changes;
see http://boinc.berkeley.edu/trac/wiki/CreditNew

    Projects will need to update DB and recompile all back-end programs.

    Summary:
    - new way of computing credit
    - "reliable host" mechanism is per app version
    - "host punishment" mechanism is per app version
    - adjustment of wu.rsc_fpops_est provides the
        equivalent of per app version DCF
    - max jobs in progress is now per app
    - max jobs per RPC is now per app

    TODO:
    - reliable mechanism:
        - populate and use host_app_version.error_rate
        - populate host_app_version.turnaround
    - host punishment:
        - populate host_app_version.max_jobs_per_day
        - populate host_app_version.n_jobs_today
        - use app.max_jobs_per_day_init
    - job limits:
        - use app.max_jobs_in_progress, max_gpu_jobs_in_progress
        - use app.max_jobs_per_rpc
    - adjust wu.rsc_fpops_est
    - remove old credit stuff
        fpops_cumulative, credit_multiplier
        credit computation in scheduler

- AVERAGE class: use the Knuth algorithm (Wikipedia)


svn path=/trunk/boinc/; revision=21021
2010-03-29 22:28:20 +00:00
David Anderson c0ad0f3745 - update_versions: if file is already in download dir,
and is the same, don't copy (it might not be writeable)
- client: change "result" to "task" in user-visible messages

svn path=/trunk/boinc/; revision=20785
2010-03-03 23:52:20 +00:00
David Anderson c27f2486e7 - fix python deprecation warning (from Jeremy Cowles)
svn path=/trunk/boinc/; revision=20122
2010-01-11 06:10:17 +00:00
David Anderson 437087a053 - upgrade: add --server_only option
(updates scheduler and back-end programs, but not web code)
    fixes #968

svn path=/trunk/boinc/; revision=20052
2009-12-31 02:37:16 +00:00
David Anderson e1bdfc0ee8 - web: change the way project news is stored.
Old: in a flat file (html/project/project_news.inc)
    New: in a forum (called "News" by default)
    The script html/ops/news_convert.php copies news from
    old to new format.
    You'll also need to edit your index.php and use
    "show_news(0, 5)" to show news.
- web: added a "message of the day" mechanism.
    Edit html/user/motd.php to show a message.
    This will be shown as the first news item,
    but it's not archived (i.e., it's not a forum post)


svn path=/trunk/boinc/; revision=19949
2009-12-16 22:35:08 +00:00
David Anderson 3f9f76f5c2 - add "appmgr" script for managing apps and versions (from Gabor Gombas)
svn path=/trunk/boinc/; revision=18938
2009-08-28 18:25:26 +00:00
David Anderson d14527f79d - update_versions: if app version files already exist in download dir,
make sure they're the same (i.e., enforce file immutability).
    TODO: do this for input files too
- client: check return values from CUDA device detection functions

svn path=/trunk/boinc/; revision=18903
2009-08-24 19:08:05 +00:00
David Anderson 25cd6f456d - make_project and upgrade: copy file management programs to proj/bin/
svn path=/trunk/boinc/; revision=18885
2009-08-20 18:10:42 +00:00
David Anderson c653852718 svn path=/trunk/boinc/; revision=18840 2009-08-13 18:22:40 +00:00
David Anderson a1d18e7e11 svn path=/trunk/boinc/; revision=18839 2009-08-13 18:20:03 +00:00
David Anderson 69e363df2f svn path=/trunk/boinc/; revision=18838 2009-08-13 18:18:32 +00:00
David Anderson 6b36764b65 - start: don't print "XML parse error" message if run_state file
doesn't exist (e.g. when start new project)


svn path=/trunk/boinc/; revision=18837
2009-08-13 18:17:09 +00:00
David Anderson a0c483f5e6 - make_project/upgrade: install pymw assimilator
- make pymw_assimilator.py executable


svn path=/trunk/boinc/; revision=18481
2009-06-22 22:20:35 +00:00
David Anderson 817153c96b - setup script: remove old pymw stuff
svn path=/trunk/boinc/; revision=18455
2009-06-18 22:48:11 +00:00
David Anderson 577f14ff49 - python scripts: fix bugs, restore latest version of "start"
svn path=/trunk/boinc/; revision=18424
2009-06-15 23:24:15 +00:00
David Anderson ac14777bae - forget to add file
svn path=/trunk/boinc/; revision=18384
2009-06-11 23:06:02 +00:00
David Anderson b7406ef57c - server: make the python code honor BOINC_PROJECT_PATH.
It is now possible to use the same set of tools for multiple
    projects by setting BOINC_PROJECT_PATH.
    From Gabor Gombas.

svn path=/trunk/boinc/; revision=18376
2009-06-11 16:06:25 +00:00
David Anderson 6bc55040f0 - server: Use distutils for installing the python modules,
and move files not part of the BOINC package from py/Boinc to py/
    From Gabor Gombas.

svn path=/trunk/boinc/; revision=18375
2009-06-11 16:01:36 +00:00
David Anderson 4cf6b7793c - server: move python modules from PROJECT/bin to PROJECT/py/Boinc.
make_project now generates PROJECT/bin/boinc_path_config.py
    to ensure that the interpreter will find the modules.
    From Gabor Gombas.

svn path=/trunk/boinc/; revision=18374
2009-06-11 15:38:48 +00:00
David Anderson 144c106e19 - make_project and upgrade: add --srcdir option so you
can run these commands from any directory
    (mostly from Gabor Gombas; added a fix for svnversion)

svn path=/trunk/boinc/; revision=18356
2009-06-10 22:38:33 +00:00
David Anderson cb0c57c590 - python DB code: add clear cache method, and fix equality crash
(from Jeremy Cowles)

svn path=/trunk/boinc/; revision=18258
2009-06-02 04:07:53 +00:00