Commit Graph

108 Commits

Author SHA1 Message Date
David Anderson 9d2ba11e09 - back end: extend the access control system for remote job submission
and other operations.
    You can now designate a user as "manager" for a particular app.
    They can then:
    - control job-submit permissions for that app
    - deprecate/undeprecate versions of the app.
    - abort jobs for that app

    You can also designate a user a manage for the project.
    They can then edit permissions and quotas,
    as well as performing the app-specific functions for all apps.

    This is described here:
    http://boinc.berkeley.edu/trac/wiki/MultiUser#Accesscontrol

    This required some changes to the DB schema.


svn path=/trunk/boinc/; revision=24250
2011-09-21 21:26:00 +00:00
David Anderson e49f945908 - Validator: allow project-specific code to mark a result
is a "runtime outlier", i.e. its runtime does
    not correspond to the job's rsc_fpops_est.
    Runtime outliers are not counted in the statistics for
    elapsed time, turnaround time, and peak FLOPs count.

    The is intended for applications like SETI@home,
    some of whose jobs finish more or less instantly
    (this happens if the data contains a lot of interference).
    If a host happens to get a bunch of these short jobs,
    its statistics will get skewed: in essence, the server
    will think that the host is extremely fast,
    and will send it too many jobs.


svn path=/trunk/boinc/; revision=24225
2011-09-16 16:43:15 +00:00
David Anderson 8c49c174c3 - preliminary stuff for mechanism where privileged users
can create apps and app versions
- crontab commands should be preceded by cd to project root


svn path=/trunk/boinc/; revision=24137
2011-09-07 06:54:44 +00:00
David Anderson 6e5acbbe60 - web: remote job submission:
- add fields to batch table, extend APIs accordingly
    - require that example web interface run on BOINC server
        (this makes many things easier;
        an actual remote interface would require a bit more work)


svn path=/trunk/boinc/; revision=23881
2011-07-27 06:20:48 +00:00
David Anderson 8a9605e48c - web: add a web-service interface for remotely submitting, querying
and controlling batches of jobs
- web: add an administrative interface for controlling
    user permissions for submitting jobs
- web: add an interface where users can view and control
    their submitted jobs
See: http://boinc.berkeley.edu/trac/wiki/RemoteJobs
This is at a functional but rough stage.


svn path=/trunk/boinc/; revision=23762
2011-06-21 22:56:15 +00:00
David Anderson 436415cfe1 - scheduler, back end: add "homogeneous app version" feature.
Lets you specify, on a per-app basis,
    that all instances should be done using the same app version.
    This is for validation in the presence of GPUs.
- scheduler: code cleanup
    - Instead of adding a bunch of non-DB fields to RESULT,
        used a derived class SCHED_DB_RESULT.
    - Instead of storing a pointer to BEST_APP_VERSION in RESULT,
        store the structure itself.
        This simplifies the memory allocation situation.
- client: condition "Got server request to delete file" messages
    on <file_xfer_debug>


svn path=/trunk/boinc/; revision=23636
2011-06-06 03:40:42 +00:00
David Anderson e260b47bd5 - database: app.min_avg_pfc should default to 1, not 0
svn path=/trunk/boinc/; revision=21763
2010-06-17 16:44:33 +00:00
David Anderson 8b836a391b - database: remove unused fields from app table
svn path=/trunk/boinc/; revision=21728
2010-06-11 03:50:47 +00:00
David Anderson 5035007b90 - back end: new way of deciding:
- whether host is "reliable" for an app version
    - whether host is eligible for single replication for an app version
    - whether to use host scaling
    In each case, the answer is yes if the number of
    consecutive valid results is above a threshold.
    This replaces existing "error rate" and "scale probation" mechanisms.

    TODO: the # of consecutive valid results should also determine
        a limit on jobs in progress for an app version.
        Namely, if N is the threshold for host scaling, the limit should be
            ndevices*(max(1, consecutive_valid - N))
        The client currently doesn't supply enough
        app version info to do this.
        It could be approximated; that would give some protection
        against cherry-picking.
- credit: more conservative formulas for combining claimed credit
    among replicas.
    If there are normal replicas, we use a "low average"
    that weights each sample by the sum of the other samples.
    Otherwise we use the min (not the average) of the approximate samples.

NOTE: a DB update is required


svn path=/trunk/boinc/; revision=21230
2010-04-21 19:33:20 +00:00
David Anderson fb851311e0 - server: various changes;
see http://boinc.berkeley.edu/trac/wiki/CreditNew

    Projects will need to update DB and recompile all back-end programs.

    Summary:
    - new way of computing credit
    - "reliable host" mechanism is per app version
    - "host punishment" mechanism is per app version
    - adjustment of wu.rsc_fpops_est provides the
        equivalent of per app version DCF
    - max jobs in progress is now per app
    - max jobs per RPC is now per app

    TODO:
    - reliable mechanism:
        - populate and use host_app_version.error_rate
        - populate host_app_version.turnaround
    - host punishment:
        - populate host_app_version.max_jobs_per_day
        - populate host_app_version.n_jobs_today
        - use app.max_jobs_per_day_init
    - job limits:
        - use app.max_jobs_in_progress, max_gpu_jobs_in_progress
        - use app.max_jobs_per_rpc
    - adjust wu.rsc_fpops_est
    - remove old credit stuff
        fpops_cumulative, credit_multiplier
        credit computation in scheduler

- AVERAGE class: use the Knuth algorithm (Wikipedia)


svn path=/trunk/boinc/; revision=21021
2010-03-29 22:28:20 +00:00
David Anderson 295d4b54ea - server: major improvements to locality scheduling from Einstein@home.
Triggering the work generator is now done via the DB
    instead of flat files.

    Since only E@h uses locality scheduling,
    I kept the DB changes in a separate file (db/schema_locality.sql).
    There's a new field in the workunit table,
    and that's a required update (in db_update.php)
- manager: compile fix


svn path=/trunk/boinc/; revision=20807
2010-03-05 22:55:16 +00:00
David Anderson b817b8ed7a - admin web: changed protection scheme as follows:
- default: script can be run from command line;
        to run from web you need to be logged in as a particular user.
        (in addition to .htaccess protection).
    - if include $cli_only at start of script:
        can only be run from the command line.
    - if include $skip_auth_ops at start of script:
        can run from web without being logged in
        (this is set ONLY in login_action.php)
- admin web: added $cli_only = true to a bunch of scripts


svn path=/trunk/boinc/; revision=19230
2009-10-02 18:32:40 +00:00
David Anderson 0f9cda3ea3 - admin web: finished the above.
Got rid of auth_ops_cmdline() (not needed).
    auth_ops() is now called in util_ops.inc;
    every script in html/ops/ must include util_ops.inc.

Note: for existing projects, the admin web interface
    will now show "access denied".
    Projects must think about what access control policy they want,
    and specify it in project.inc


svn path=/trunk/boinc/; revision=19053
2009-09-15 18:14:37 +00:00
David Anderson 8f2a014f58 svn path=/trunk/boinc/; revision=19030 2009-09-08 20:37:40 +00:00
David Anderson da7e82fe15 - scheduler and back end: add new fields to result table:
elapsed_time: the elapsed time (runtime) as reported by client
    flops_estimate: the app's estimated FLOPS as reported by app_plan()
    app_version_id: the DB ID of the app_version used
        (or -1 if anonymous platform)
    TODO: show these in the web interfaces,
    and use them where appropriate

svn path=/trunk/boinc/; revision=19002
2009-09-03 20:26:31 +00:00
Jeff Cobb 15ccf7b778 Added table state_counts.
svn path=/trunk/boinc/; revision=18490
2009-06-23 21:45:22 +00:00
David Anderson da217e4c59 - upgrade: don't crash if file copy fails
- db_update.php: select database (not clear if this ever worked)
- translation: fix bug if have both BOINC and project translation

svn path=/trunk/boinc/; revision=18141
2009-05-18 04:18:47 +00:00
David Anderson 0b11a548dc - upgrade script: do DB update even if stop_web is present,
rather than spewing an HTML error message.
    Fixes #875


svn path=/trunk/boinc/; revision=17839
2009-04-17 02:08:05 +00:00
David Anderson e43e8a408d - client: major changes to enforce_schedule() to handle GPUs
svn path=/trunk/boinc/; revision=16178
2008-10-09 22:44:45 +00:00
David Anderson ebe3b090e8 - add a script "upgrade_db.php" that updates project DB structure
(after user confirmation).
    This is called from "upgrade", and can also be run by itself.

    NOTE: this mechanism will handle all DB updates going forward.
    Older updates must be done the old way (edit and run db_update.php)

- Web: let teams determine whether they're accepting new members


svn path=/trunk/boinc/; revision=16160
2008-10-08 16:48:11 +00:00
David Anderson 33ad1b3d40 - Added the above DB change to schema.sql
- Omit the "after" clause from update.
- Web: add missing arg; fixes #749
- server: more --help additions

svn path=/trunk/boinc/; revision=16136
2008-10-06 00:18:36 +00:00
Janus B. Kristensen f361f95313 - Added "Highlight special users"-feature
- Allow projects to specify how special users can be highlighted in their forums through CSS
- Special users are now highlighted by default
- Post body background is now 25% transparent in browsers that support transparency (white.css only)


svn path=/trunk/boinc/; revision=16134
2008-10-05 16:00:11 +00:00
David Anderson 9baf78fe5f - web: added copyright and license info to PHP files
svn path=/trunk/boinc/; revision=15758
2008-08-05 22:43:14 +00:00
David Anderson 80bde8a4c6 - ops: add script to fix backslashes in posts etc.
- DB: add db_update.php entry for Eric's new table

svn path=/trunk/boinc/; revision=15700
2008-07-28 22:24:57 +00:00
David Anderson 0e03df254b - Back end: add adaptive validation feature
(DB update required)
- Fixed typo in Eric's 5/28 checkin

svn path=/trunk/boinc/; revision=15357
2008-06-04 23:04:12 +00:00
David Anderson 6af9f66b4e - DB/feeder/scheduler: change app_version.xml_doc from blob to mediumblob,
and change the correspending structure field from 64KB to 256KB
    (could increase this if needed).
    This is needed to handle app versions with lots (> 100) of files
- change LARGE_BLOB_SIZE to BLOB_SIZE a bunch of places
- Change COPROCS from vector<COPROC> to vector<COPROC*>.
    Otherwise the right virtual functions of COPROCs don't get called

svn path=/trunk/boinc/; revision=14986
2008-03-31 16:19:45 +00:00
David Anderson bacf9512d1 - web: on completion of team foundership transfer,
clear ping_user and ping_time fields
- web: fix outdated update_forum_activities script

svn path=/trunk/boinc/; revision=14973
2008-03-28 03:05:19 +00:00
David Anderson 815b8fc043 Various preparation for handling multithreaded apps
and apps that use coprocessors.
There now can be several app_versions for the same
(app, platform, version_num) combination.
This changes a number of things.

- Added app_version.plan_class field to DB
- update_versions now looks for a :plan-class in the
    file or directory name, and puts it in the app_version's DB record
- Change uniqueness constraint to include plan_class
- Feeder: the feeder was putting non-deprecated app_versions
    in shared mem, and leaving it to the scheduler to
    find the latest version for a given platform.
    This is dumb.
    Instead, for each app/platform pair the feeder now
    finds the highest version number of a non-deprecated app version,
    and enumerates all non-deprecated app_versions with that
    app/platform/version
- Scheduler: add a BEST_APP_VERSION data structure that keeps track,
    for each app, what the best app_version is for this host.
    This saves the work of recomputing it for each job.

svn path=/trunk/boinc/; revision=14906
2008-03-13 22:57:24 +00:00
David Anderson 546c0b899a - server: fix botched checkin from 7 March.
The new field (workunit.rsc_bandwidth_bound)
    goes at the END of the record.
    Always do it this way!
- make_work: after creating a batch of new WUs,
    we were waiting 60 sec for the transitioner to
    create the results for them
    (so that our next count of unsent results would be correct).
    This is bogus; if e.g. the transitioner isn't running,
    we'll never get the results, and we'll keep creating WUs forever.

    Instead: explicitly wait for there to be results for
    the last WU from the batch just created.

- scheduler: parse <allow_non_preferred_apps>, <allow_beta_work> correctly.

svn path=/trunk/boinc/; revision=14875
2008-03-10 17:03:15 +00:00
David Anderson 95772cba77 - removed boinc_ncpus_available() and boinc_nthreads() calls.
The design has been changed to constant #threads per app version
    Various changes from Kevin Reed/WCG:
    - server: add workunit.rsc_bandwidth_bound: if nonzero,
        send this WU only to hosts with that much download bandwidth
    - assimilators: if a handler returns DEFER_ASSIMILATION,
        the WU remains in INIT state and will be handled when the
        next instance completes.
        Useful if you want the assimilator to see all instances.
    - scheduler: when setting result.outcome = DETACHED,
        set received_time to now
    - scheduler: removed the reliable_time and reliable_min_avg_credit
        options
    - scheduler/web: add optional <allow_non_preferred_projects>
        in project preferences.
        If present, user will accept work from non-selected apps
        if no work is available for selected apps
    - scheduler: improved messages for projects with multiple apps
    - scheduler: added config options
        <granted_credit_weight> and <granted_credit_ramp_up>.
        Used in calculating host.claimed_credit_per_cpu_sec,
        but I'm not sure how.
    - Added two new credit-granting formulas (validate_util.C):
        stddev_credit() and two_credit()
    - server DB: add rollback_transaction() and affected_rows() to DB_CONN

    NOTE: DB update required

svn path=/trunk/boinc/; revision=14870
2008-03-07 21:13:01 +00:00
David Anderson 54519a4ee1 - Server: add "job assignment" feature.
Lets you assign a WU to a particular host,
    to one or all hosts belonging to a user or team, or to all hosts.
    See http://boinc.berkeley.edu/trac/wiki/AssignedWork
    Disabled unless you include <enable_assignment> in config.xml
    Uses a new DB table.
    Tested but only a little.
- Server: code cleanup; moved result-handling to a new file,
    and removed the PLATFORM_LIST arg to everything
    (put it in SCHEDULER_REQUEST instead)

svn path=/trunk/boinc/; revision=14767
2008-02-21 00:47:50 +00:00
David Anderson f183b6f47f - web: add general-purpose notification mechanism.
Users can choose whether to get 1 email per notification,
    a daily "digest" email, or no email.
    (All notifications are shown on the Account page).
    Currently used for:
    - Friend requests and confirmations
    - Posts to subscribed threads
    - Private messages

    NOTE: To implement the "daily digest" feature, projects must add
    a periodic task for html/ops/notify.php to config.xml
- web: have project_footer() generate links for
    Account Page and Message Boards as well as Home
    NOTE: projects that want this change
    will have to modify their own project.inc.

svn path=/trunk/boinc/; revision=14447
2007-12-30 22:02:16 +00:00
David Anderson 13baee4302 - user web: add "friend" and "notification" features
- user web: code cleanup in Profile area
- GUI RPC: add missing "/" in auth2 RPC

svn path=/trunk/boinc/; revision=14394
2007-12-18 20:28:08 +00:00
David Anderson 4a81532043 - DB: add fulltext index on team.name
- user web: improve "team search" function,
    and get rid of the superceded "search by name" function
- user web: don't show team founder link to initiate transfer
- user web: show team types
- user web: fix "top teams by type" function
- client: tread NOT_FOUND error on file upload as transient;
    it means the project is missing its file_upload_handler

svn path=/trunk/boinc/; revision=14272
2007-11-20 22:58:14 +00:00
David Anderson 615d4dcf22 - user web: the "private-message notification by email" flag
should default to off.
    Note: it's necessary to clear everyone's flag.
    Projects will need to notify users (e.g. via a news item)
    that they need to explicitly set this pref if they want email
    notification of PMs

svn path=/trunk/boinc/; revision=14255
2007-11-19 02:17:57 +00:00
David Anderson 9970e2c71b - user web: bug fixes
svn path=/trunk/boinc/; revision=14204
2007-11-14 21:55:05 +00:00
David Anderson aa7b9f2547 - DB: fix typo in schema
svn path=/trunk/boinc/; revision=14141
2007-11-12 16:00:37 +00:00
David Anderson bf227ea8b1 - DB: remove "lang" from schema - not used
- DB: added parent_type to forum
- user web: BoincUser uses caching by default
- client: fixed small memory leak in statistics parsing

svn path=/trunk/boinc/; revision=14130
2007-11-10 00:32:42 +00:00
David Anderson fb8b4ffabb - user web: added the notion of "Team Admin": team members,
appointed by the founder, who have most of the same
    rights as the founder (edit team info, view/remove users).
    They can't, however, change the founder, or select/remove Admins.
- user web: added the ability for founder to delete an empty team
- user web: count/show all team members, not just those with credit
- user web: Team Search returns top team if form fields left blank
- Added db_update.php entry for Eric's banishment_vote stuff

svn path=/trunk/boinc/; revision=14108
2007-11-07 17:23:29 +00:00
David Anderson 27a7659f36 - Bossa: changed the DB infrastructure to let Bossa tables
live in a different database (and/or server) than the BOINC tables.
    Also eliminated the need to call a DB init function.
    This involved:
    1) adding <bossa_db_host> etc. entries to config.xml
    2) prefixing DB name to table name in all Bossa queries
        (this is needed if Bossa DB is on same server as BOINC DB,
        since we're using persistent connections)
- user web: added Montenegro, and changed Serbia and Montenegro to Serbia
    (requires DB update to make latter change in existing records)


svn path=/trunk/boinc/; revision=13962
2007-10-25 21:34:13 +00:00
David Anderson 38ba2854a0 svn path=/trunk/boinc/; revision=13750 2007-10-02 16:26:39 +00:00
David Anderson 2272891035 - user web: add "team search" page, which does a multi-criteria
search (keywords, country, type).
    Have the new-user scenario route the user through this page.
    Also, link to it rather than team.php for existing users.
- user web: use cookies rather than URLs to identify new users,
    and show "Welcome to X" when they reach their user page.
- user web: don't show user intermediate page for join/quit team.
    Just take them to their user page.

svn path=/trunk/boinc/; revision=13731
2007-10-02 03:19:03 +00:00
David Anderson aee80d8fea - simplify sql query
svn path=/trunk/boinc/; revision=13676
2007-09-26 17:54:35 +00:00
Rytis Slatkevičius 4b8aeabf91 User web: fix an annoyance when team founder transfer is requested and the requester leaves the team.
NOTE: DB UPDATE REQUIRED.

svn path=/trunk/boinc/; revision=13673
2007-09-26 15:46:54 +00:00
David Anderson ac5b3b8455 - client: don't crash if use --detach_project option
(fixes #283)
- client: canonicalize URLs in all cmdlines
- db_update.php: don't prompt for DB name/host; get it from config file
    (fixes #356)

svn path=/trunk/boinc/; revision=13243
2007-07-31 03:25:59 +00:00
David Anderson 14bc500cab - user web: added mechanism for recording team membership changes.
These are stored in a new DB table, "team_delta".
    The team founder can view the history in HTML or XML
    (links on Management Functions page)

svn path=/trunk/boinc/; revision=13231
2007-07-27 18:30:10 +00:00
Rytis Slatkevičius bcf948a03d user web: add a separate preference for private message notification emails.
'''NOTE: needs DB update, see html/ops/db_update.php'''

(Fix #167)

svn path=/trunk/boinc/; revision=12813
2007-06-04 19:35:11 +00:00
Rytis Slatkevičius 670bbd92d7 Donations system (running via PayPal).
Implements #211.
NOTE: DB updated needed. Documented in http://boinc.berkeley.edu/trac/wiki/PayPalDonations

svn path=/trunk/boinc/; revision=12754
2007-05-27 17:33:06 +00:00
Matt Lebofsky 136ce49b84 svn path=/trunk/boinc/; revision=12536 2007-05-02 23:17:52 +00:00
Matt Lebofsky 846ec9fbb0 svn path=/trunk/boinc/; revision=12532 2007-05-02 18:51:51 +00:00