Commit Graph

541 Commits

Author SHA1 Message Date
David Anderson 004c5692e3 - create_work tool: change option names for assigned jobs
- admin web: show actual platform name in result summary page


svn path=/trunk/boinc/; revision=25175
2012-01-31 20:25:26 +00:00
David Anderson 130d6ed4f0 - server: revamp the "assigned job" mechanism.
This now supports two main use cases:
    1) there's a job that you want to run once on all hosts,
        present and future
        (or all hosts belonging to a user, or to a team).
        The job is never transitioned, validated, or assimilated.
    2) There's a normal job for which you want to use only
        hosts belonging to a specific user (e.g. cluster or cloud hosts).
        This restriction can be made either when the job is created,
        or on the fly,
        e.g. as part of a scheme for accelerating batch completion.
        For the latter purposes we now provide a function
            restrict_wu_to_user(DB_WORKUNIT&, int userid);

        The job goes through the standard
        transitioner/validator/assimilator path.

    These cases are enabled by config flags
        <enable_assignment_multi/>
        <enable_assignment/>
    respectively.

    Assignment of type 2) are no longer stored in shared mem,
    so there is no limit on their number.

    There is no longer a rule that assigned job names must contain "asgn".

    NOTE: this requires a database update.


svn path=/trunk/boinc/; revision=25169
2012-01-30 22:39:13 +00:00
David Anderson d281e43d6a - update_versions: for gzipped files,
send the size of the compressed file as well.
- client: parse and write the compressed size (FILE_INFO::gzipped_nbytes).
    For get_transfer GUI RPCs, if it's a compressed download send
    the compressed size.
    That way the manager will show the fraction done correctly.


svn path=/trunk/boinc/; revision=25152
2012-01-26 08:39:51 +00:00
Rom Walton 110a18b0a2 Quick Updates
svn path=/trunk/boinc/; revision=25118
2012-01-21 01:36:53 +00:00
Rom Walton 7e593bd53c Quick Updates
svn path=/trunk/boinc/; revision=25117
2012-01-21 01:33:44 +00:00
Rom Walton 563f532eb2 Quick Updates
svn path=/trunk/boinc/; revision=25116
2012-01-21 01:30:35 +00:00
David Anderson 44f173605e - client: add a mechanism for restartable download of compressed files.
(It turns out that the compression schemes supported by
        Apache and libcurl, suprisingly, aren't restartable.)

        if a <file_info> from the server contains <gzipped_url> tags,
        use those instead of the <url> tags,
        and flag the file as "download_gzipped".
        If this is the case, download NAME.gz and save it as NAME.gzt.
        When the download is complete, rename NAME.gzt to NAME.gz,
        and uncompress it to NAME.
        (this ensures that if NAME.gz is present, it's complete).
        Also do the uncompression, if needed, in verify_file().
        This ensures that the uncompression will eventually get done
        even if the client quits are crashes in the middle.

    - update_versions: if <gzip> is present in a <file_info>,
        add a gzipped copy in the download directory
        and add a <gzipped_url> elements to the app version's xml_doc.


svn path=/trunk/boinc/; revision=25112
2012-01-20 23:34:15 +00:00
David Anderson 33b607dab1 - upgrade script:
- prompt user for DB user/passwd if DB updates needed
    - error out immediately if any DB update fails
    Fixes #1169, #1170
- create_work: check for duplicate <file_info>s in input template
    NOTE: the input template syntax is pretty clunky


svn path=/trunk/boinc/; revision=24969
2012-01-02 07:33:08 +00:00
David Anderson bba4ce24ce - client: compute projects' disk share (based on resource share).
Report it (along with disk usage) in scheduler request messages.
    This will allow the scheduler to send file-delete commands
    if the project is using more than its share.
- client: add <disk_usage_debug> log flag
- create_work: add --help, show --command_line option


svn path=/trunk/boinc/; revision=24968
2012-01-02 05:53:42 +00:00
David Anderson 4e50e4b852 - update_versions: support <needs_network> flag in version.xml
svn path=/trunk/boinc/; revision=24901
2011-12-26 06:03:50 +00:00
David Anderson fa3f5a945e - bug fixes in remote job submission tools
(manage_privileges and get_output.php).
    From Nico Schlitter.
- change things so that
    make_project --test_app
    will produce input/output templates that are compatible
    with the remote job submission tools


svn path=/trunk/boinc/; revision=24774
2011-12-10 22:03:43 +00:00
David Anderson 8e689c7434 - update_versions: touch trigger file so that feeder rereads DB
svn path=/trunk/boinc/; revision=24458
2011-10-21 20:14:54 +00:00
David Anderson 53af2ae778 - client/API: pass bool using_sandbox in APP_INIT_DATA;
says whether the client is using account-based sandboxing.
    Needed by vboxwrapper.
- web: don't show authenticator in Your Account page.
    The problem with authenticators is that if a bad guys gets yours,
    they can log in to your account even if you change your password.
    Eventually we should not use them at all
    (i.e. in cookies and sent to client).
- make_project: copy python stuff (e.g. bin/start) for web-only projects;
    generate appropriate project.readme file for web-only projects.


svn path=/trunk/boinc/; revision=24447
2011-10-21 06:58:35 +00:00
David Anderson 3410abef6f - backend API: added function cancel_jobs(minid, maxid)
for canceling jobs
- added program cancel_jobs for canceling jobs
- DB interface: it's not an error if update_fields_noid()
    affects != 1 rows


svn path=/trunk/boinc/; revision=24413
2011-10-18 07:15:04 +00:00
David Anderson c4f234931e - fix typo in manage_privileges
svn path=/trunk/boinc/; revision=24276
2011-09-23 17:04:22 +00:00
David Anderson 9d2ba11e09 - back end: extend the access control system for remote job submission
and other operations.
    You can now designate a user as "manager" for a particular app.
    They can then:
    - control job-submit permissions for that app
    - deprecate/undeprecate versions of the app.
    - abort jobs for that app

    You can also designate a user a manage for the project.
    They can then edit permissions and quotas,
    as well as performing the app-specific functions for all apps.

    This is described here:
    http://boinc.berkeley.edu/trac/wiki/MultiUser#Accesscontrol

    This required some changes to the DB schema.


svn path=/trunk/boinc/; revision=24250
2011-09-21 21:26:00 +00:00
David Anderson 8c49c174c3 - preliminary stuff for mechanism where privileged users
can create apps and app versions
- crontab commands should be preceded by cd to project root


svn path=/trunk/boinc/; revision=24137
2011-09-07 06:54:44 +00:00
David Anderson f75ffd88f2 - server: factor process_input_template() into smaller pieces
svn path=/trunk/boinc/; revision=24108
2011-09-01 20:26:12 +00:00
David Anderson c529c3c50d - server: rename process_wu_template() to process_input_template(),
and move it to its own file


svn path=/trunk/boinc/; revision=24107
2011-09-01 19:58:27 +00:00
David Anderson abdb603bc4 - job creation: handling of extra XML in input templates didn't work.
- job creation: fix stripping of <output_template> tags
- client: fix off-by-1 error in buffer overflow logic for scanning keys.


svn path=/trunk/boinc/; revision=24057
2011-08-27 16:52:04 +00:00
David Anderson 49205acfd9 - job submission: fix bug where parameters specified in the
input template (e.g., delay bound) were ignored
- web: fix PHP warning


svn path=/trunk/boinc/; revision=24039
2011-08-25 05:08:47 +00:00
David Anderson c5c5975b44 - Improve interface of XML_PARSER.
Add parsed_tag and is_tag to the class,
    so that parsing functions don't need to declare them
    and pass them around.
- Complete the task of using XML_PARSER as the argument
    to all parsing functions.
    (Internally, many of these functions still use the old XML parser;
    that's the next step.)


svn path=/trunk/boinc/; revision=23978
2011-08-10 17:11:08 +00:00
David Anderson dec057eaab - update_versions: fix bug where files not listed in version.xml
are not processed correctly
- remote job submission: debug
- create_work: --rsc_fpops_est etc. should override the template file


svn path=/trunk/boinc/; revision=23942
2011-08-07 04:32:02 +00:00
David Anderson 5a8fd0afc7 - client: add optional <file_prefix> to APP_VERSION>.
If present, "file_prefix/" is prepended to the logical names
    of input and output files of jobs using that app version.
    I.e. for Vbox wrapper based app versions, file_prefix is "share",
    so that I/O files are put in a "share" subdirectory of the slot dir.
- update_versions: add support for
    <dont_throttle>
    <file_prefix>x</file_prefix>
    in version.xml


svn path=/trunk/boinc/; revision=23924
2011-08-03 18:14:45 +00:00
David Anderson 0004f43937 - update_versions: bug fixes
svn path=/trunk/boinc/; revision=23921
2011-08-02 17:20:18 +00:00
David Anderson 135c549475 - update_versions: add optional <gzip> element to
<file> element in version.xml.
    If present, update_versions will gzip the file in the download dir.
    Use this only if you have gzip-encoding enabled in Apache.


svn path=/trunk/boinc/; revision=23915
2011-08-02 07:38:45 +00:00
David Anderson 5ad821df21 - server: some remote job submission code. Not finished.
svn path=/trunk/boinc/; revision=23868
2011-07-22 22:47:41 +00:00
David Anderson 3c70004c5c - server: debug distributed storage functions
- generate upload signatures if needed
    - assign deadlines to file xfer jobs (default 1 week)
    - scheduler: ack completed file xfer results


svn path=/trunk/boinc/; revision=23866
2011-07-20 22:27:01 +00:00
David Anderson 9a4140ae30 - client/server: change the implementation of upload signatures
as described here: http://boinc.berkeley.edu/trac/wiki/ClientDataModel
    Compatibility: if your project is using upload signatures:
        - set ignore_upload_certificates
        - disable job creation
        - let your job queue drain
        - upgrade to new server software
        - clear ignore_upload_certificates
        - enable job creation


svn path=/trunk/boinc/; revision=23863
2011-07-20 20:08:05 +00:00
David Anderson 27e05a3da9 - server: some stuff to prepare for distributed storage
- don't create result records for uploads and downloads.
        Just create a msg_to_client record.
    - the scheduler handles file-transfer results specially;
        it makes a vector of them, then calls a project-supplied function
        handle_file_xfer_results()
    - change the interface and implementation of put_file and get_file
- client write project sched priority in GUI RPC replies,
    but not to the state file


svn path=/trunk/boinc/; revision=23857
2011-07-19 20:52:41 +00:00
David Anderson cc0ab61b96 - update_versions: fix typo that produced bad XML
svn path=/trunk/boinc/; revision=23793
2011-07-05 17:45:24 +00:00
David Anderson bfc04f03eb - back end: fix bugs in [23648] which caused create_work
to mess up input templates containing
    <copy_file/> or other attribute tags.
    XML_PARSER now contains a member element() for when
    you want to copy an element without knowing its structure.


svn path=/trunk/boinc/; revision=23790
2011-07-04 23:51:00 +00:00
David Anderson da74e5caf2 - back end: strip <output_template> tags from output templates
- web: main RSS feed should include all news items, even non-notices


svn path=/trunk/boinc/; revision=23783
2011-06-29 22:26:45 +00:00
David Anderson 8a9605e48c - web: add a web-service interface for remotely submitting, querying
and controlling batches of jobs
- web: add an administrative interface for controlling
    user permissions for submitting jobs
- web: add an interface where users can view and control
    their submitted jobs
See: http://boinc.berkeley.edu/trac/wiki/RemoteJobs
This is at a functional but rough stage.


svn path=/trunk/boinc/; revision=23762
2011-06-21 22:56:15 +00:00
David Anderson 6c29473890 - update_versions: read from stdin a different way
svn path=/trunk/boinc/; revision=23708
2011-06-11 15:30:10 +00:00
David Anderson 93add14614 - backend: use new XML parser for input template files
(so that they don't have to be 1 element/line)
    and also allow optional <input_template> root element
- fix bug in WORKUNIT DB interface


svn path=/trunk/boinc/; revision=23648
2011-06-07 04:12:49 +00:00
David Anderson 31faa14c43 - update_versions: flag main program as executable,
otherwise client will reject it


svn path=/trunk/boinc/; revision=23594
2011-05-24 19:16:54 +00:00
David Anderson b6140088e3 - update_versions: trim XML strings
svn path=/trunk/boinc/; revision=23569
2011-05-21 06:22:15 +00:00
David Anderson e861df4ba2 - update_versions: make <copy_file> work
svn path=/trunk/boinc/; revision=23559
2011-05-18 16:59:15 +00:00
David Anderson 649c094804 - another update_versions fix
svn path=/trunk/boinc/; revision=23553
2011-05-17 16:00:23 +00:00
David Anderson d9920ba520 - update_versions: fix bug in version number parsing (from Kevin)
svn path=/trunk/boinc/; revision=23552
2011-05-17 15:53:06 +00:00
David Anderson 51c652640f - create_work: allow multiple URLs for non-local input files
(from Zoltan Farkas)
- scheduler: fix message describing per-app limits


svn path=/trunk/boinc/; revision=23546
2011-05-13 18:40:10 +00:00
David Anderson 99525396ad - update_versions fix
svn path=/trunk/boinc/; revision=23527
2011-05-12 04:39:13 +00:00
David Anderson 606cf30667 - fix to update_versions
svn path=/trunk/boinc/; revision=23526
2011-05-12 04:14:53 +00:00
David Anderson e4f104f917 - Change the way update_versions works, and reimplement it in PHP.
Problems with the old version:
    - Lots of info encoded in file and directory names: messy kludge
    - The convention that directory name == main program name
        created a bad situation when using the wrapper:
        if you made a new version of your application,
        you were forced make a new version of the wrapper
        just so it could have a different name.
    The new version is described here:
    http://boinc.berkeley.edu/trac/wiki/AppVersionNew
- Remove some Python code that suppored the old version
- client: some fixes to dir_size() that might fox
    a buffer overflow when using symbolic links


svn path=/trunk/boinc/; revision=23525
2011-05-12 04:11:40 +00:00
David Anderson 2385c00b67 - wrapper: tell the client when we checkpoint
(else checkpoint_elapsed_time etc. don't get set)
- make_project: enable update_stats by default
- update_stats: add --min_age option


svn path=/trunk/boinc/; revision=23509
2011-05-06 12:33:12 +00:00
David Anderson c70cd934b4 - server: rename send_file to put_file.
Factor out put_file() and get_file() functions
    so they have a C++ API as well as command-line


svn path=/trunk/boinc/; revision=23425
2011-04-24 02:00:27 +00:00
David Anderson 6b0eba4641 - create_work and other tools: verify that the current dir,
parent dir, or BOINC_PROJECT_DIR actually is a project dir.
- client simulator: improvements


svn path=/trunk/boinc/; revision=23415
2011-04-21 17:04:42 +00:00
David Anderson 98b4c79e36 - client: if an app version has a missing GPU,
we still need to keep track of the usage
    and write it to the state file.
- get client simulator working again


svn path=/trunk/boinc/; revision=23413
2011-04-20 23:32:14 +00:00
David Anderson 9de5f69e39 - "upgrade" script: remove --project_root option,
and allow full path of project root dir
    instead of the short name


svn path=/trunk/boinc/; revision=23358
2011-04-09 03:10:29 +00:00