- whether host is "reliable" for an app version
- whether host is eligible for single replication for an app version
- whether to use host scaling
In each case, the answer is yes if the number of
consecutive valid results is above a threshold.
This replaces existing "error rate" and "scale probation" mechanisms.
TODO: the # of consecutive valid results should also determine
a limit on jobs in progress for an app version.
Namely, if N is the threshold for host scaling, the limit should be
ndevices*(max(1, consecutive_valid - N))
The client currently doesn't supply enough
app version info to do this.
It could be approximated; that would give some protection
against cherry-picking.
- credit: more conservative formulas for combining claimed credit
among replicas.
If there are normal replicas, we use a "low average"
that weights each sample by the sum of the other samples.
Otherwise we use the min (not the average) of the approximate samples.
NOTE: a DB update is required
svn path=/trunk/boinc/; revision=21230
see http://boinc.berkeley.edu/trac/wiki/CreditNew
Projects will need to update DB and recompile all back-end programs.
Summary:
- new way of computing credit
- "reliable host" mechanism is per app version
- "host punishment" mechanism is per app version
- adjustment of wu.rsc_fpops_est provides the
equivalent of per app version DCF
- max jobs in progress is now per app
- max jobs per RPC is now per app
TODO:
- reliable mechanism:
- populate and use host_app_version.error_rate
- populate host_app_version.turnaround
- host punishment:
- populate host_app_version.max_jobs_per_day
- populate host_app_version.n_jobs_today
- use app.max_jobs_per_day_init
- job limits:
- use app.max_jobs_in_progress, max_gpu_jobs_in_progress
- use app.max_jobs_per_rpc
- adjust wu.rsc_fpops_est
- remove old credit stuff
fpops_cumulative, credit_multiplier
credit computation in scheduler
- AVERAGE class: use the Knuth algorithm (Wikipedia)
svn path=/trunk/boinc/; revision=21021
New policy: anon platform and old platform jobs
get average credit, possibly scaled by elapsed time.
We no longer attempt to guess what app version produced them.
svn path=/trunk/boinc/; revision=20816
Triggering the work generator is now done via the DB
instead of flat files.
Since only E@h uses locality scheduling,
I kept the DB changes in a separate file (db/schema_locality.sql).
There's a new field in the workunit table,
and that's a required update (in db_update.php)
- manager: compile fix
svn path=/trunk/boinc/; revision=20807
elapsed_time: the elapsed time (runtime) as reported by client
flops_estimate: the app's estimated FLOPS as reported by app_plan()
app_version_id: the DB ID of the app_version used
(or -1 if anonymous platform)
TODO: show these in the web interfaces,
and use them where appropriate
svn path=/trunk/boinc/; revision=19002
don't modify user preferences or CPID.
- client: fix bug that shows ATI version incorrectly
- database: host.posts has been repurposed as a salt (or seqno)
for a new type of weak authenticator that won't depend on password
- web code:
modify forum_preferences.posts instead of host.posts.
(actually, the former isn't used either, we just do a select count(*);
should fix this at some point).
svn path=/trunk/boinc/; revision=18865
wish to use it.
- The script calculate_credit_multiplier (expected to be run daily as
a config.xml task) looks at the ratio of granted credit to CPU time
for recent results for each app. Multiplier is calculated to cause
median hosts granted credit per cpu second to equal to equal that
expected from its benchmarks. This is 30-day exponentially averaged
with the previous value of the multplier and stored in the table
credit_multplier.
- When a result is received the server adjusts claimed credit by the
value the multiplier had when the result was sent.
svn path=/trunk/boinc/; revision=15661
(calculate_credit_multiplier) to determine what factor to multiply
claimed credit by before insertion into the database. Changes to scheduler
to implement have not yet been checked in.
svn path=/trunk/boinc/; revision=15309
and change the correspending structure field from 64KB to 256KB
(could increase this if needed).
This is needed to handle app versions with lots (> 100) of files
- change LARGE_BLOB_SIZE to BLOB_SIZE a bunch of places
- Change COPROCS from vector<COPROC> to vector<COPROC*>.
Otherwise the right virtual functions of COPROCs don't get called
svn path=/trunk/boinc/; revision=14986
and apps that use coprocessors.
There now can be several app_versions for the same
(app, platform, version_num) combination.
This changes a number of things.
- Added app_version.plan_class field to DB
- update_versions now looks for a :plan-class in the
file or directory name, and puts it in the app_version's DB record
- Change uniqueness constraint to include plan_class
- Feeder: the feeder was putting non-deprecated app_versions
in shared mem, and leaving it to the scheduler to
find the latest version for a given platform.
This is dumb.
Instead, for each app/platform pair the feeder now
finds the highest version number of a non-deprecated app version,
and enumerates all non-deprecated app_versions with that
app/platform/version
- Scheduler: add a BEST_APP_VERSION data structure that keeps track,
for each app, what the best app_version is for this host.
This saves the work of recomputing it for each job.
svn path=/trunk/boinc/; revision=14906
The new field (workunit.rsc_bandwidth_bound)
goes at the END of the record.
Always do it this way!
- make_work: after creating a batch of new WUs,
we were waiting 60 sec for the transitioner to
create the results for them
(so that our next count of unsent results would be correct).
This is bogus; if e.g. the transitioner isn't running,
we'll never get the results, and we'll keep creating WUs forever.
Instead: explicitly wait for there to be results for
the last WU from the batch just created.
- scheduler: parse <allow_non_preferred_apps>, <allow_beta_work> correctly.
svn path=/trunk/boinc/; revision=14875
The design has been changed to constant #threads per app version
Various changes from Kevin Reed/WCG:
- server: add workunit.rsc_bandwidth_bound: if nonzero,
send this WU only to hosts with that much download bandwidth
- assimilators: if a handler returns DEFER_ASSIMILATION,
the WU remains in INIT state and will be handled when the
next instance completes.
Useful if you want the assimilator to see all instances.
- scheduler: when setting result.outcome = DETACHED,
set received_time to now
- scheduler: removed the reliable_time and reliable_min_avg_credit
options
- scheduler/web: add optional <allow_non_preferred_projects>
in project preferences.
If present, user will accept work from non-selected apps
if no work is available for selected apps
- scheduler: improved messages for projects with multiple apps
- scheduler: added config options
<granted_credit_weight> and <granted_credit_ramp_up>.
Used in calculating host.claimed_credit_per_cpu_sec,
but I'm not sure how.
- Added two new credit-granting formulas (validate_util.C):
stddev_credit() and two_credit()
- server DB: add rollback_transaction() and affected_rows() to DB_CONN
NOTE: DB update required
svn path=/trunk/boinc/; revision=14870
Lets you assign a WU to a particular host,
to one or all hosts belonging to a user or team, or to all hosts.
See http://boinc.berkeley.edu/trac/wiki/AssignedWork
Disabled unless you include <enable_assignment> in config.xml
Uses a new DB table.
Tested but only a little.
- Server: code cleanup; moved result-handling to a new file,
and removed the PLATFORM_LIST arg to everything
(put it in SCHEDULER_REQUEST instead)
svn path=/trunk/boinc/; revision=14767
Users can choose whether to get 1 email per notification,
a daily "digest" email, or no email.
(All notifications are shown on the Account page).
Currently used for:
- Friend requests and confirmations
- Posts to subscribed threads
- Private messages
NOTE: To implement the "daily digest" feature, projects must add
a periodic task for html/ops/notify.php to config.xml
- web: have project_footer() generate links for
Account Page and Message Boards as well as Home
NOTE: projects that want this change
will have to modify their own project.inc.
svn path=/trunk/boinc/; revision=14447
This lets you create a project for Bossa or Bolt
without compiling the C++ stuff, which is not used by Bossa or Bolt.
- DB schema: change type= to engine= to avoid MySQL warning messages
svn path=/trunk/boinc/; revision=14437
- user web: improve "team search" function,
and get rid of the superceded "search by name" function
- user web: don't show team founder link to initiate transfer
- user web: show team types
- user web: fix "top teams by type" function
- client: tread NOT_FOUND error on file upload as transient;
it means the project is missing its file_upload_handler
svn path=/trunk/boinc/; revision=14272
For thread titles, show links to forum/category also
For posts, show complete post, with search terms highlighted.
- user web: in display of posts for a user,
show links to thread and forum also
- user web: posts/threads can become "orphaned" if their forum
disappears, e.g. team message board is deleted.
Handle this gracefully in search results and user post list
- user web: where HTML tags are allowed, say so
- user web: fix bug where wrap_postcounts weren't being shown in form
- DB: use "tinyint" instead of "tinyint(1) unsigned" etc.
svn path=/trunk/boinc/; revision=14267
should default to off.
Note: it's necessary to clear everyone's flag.
Projects will need to notify users (e.g. via a news item)
that they need to explicitly set this pref if they want email
notification of PMs
svn path=/trunk/boinc/; revision=14255
- DB: added parent_type to forum
- user web: BoincUser uses caching by default
- client: fixed small memory leak in statistics parsing
svn path=/trunk/boinc/; revision=14130
appointed by the founder, who have most of the same
rights as the founder (edit team info, view/remove users).
They can't, however, change the founder, or select/remove Admins.
- user web: added the ability for founder to delete an empty team
- user web: count/show all team members, not just those with credit
- user web: Team Search returns top team if form fields left blank
- Added db_update.php entry for Eric's banishment_vote stuff
svn path=/trunk/boinc/; revision=14108
Here's how things should eventually be:
- inc/db_conn.inc defines DbConn, a PHP class representing
a connection to a database with
various generic operations (update, insert etc.).
DbConn prepends the DB name to table names,
so you don't have to select a DB;
this means you can access multiple databases
(e.g. BOINC and Bossa) and use persistent connections.
ALL MYSQL-SPECIFIC CODE WILL EVENTUALLY BE HERE.
- inc/boinc_db.inc defines classes BoincUser, BoincTeam etc.,
representing the various BOINC DB tables.
All access to the DB should use these classes.
When you use this framework, no initialization calls are needed.
db.inc is deprecated and should eventually be removed.
I changed get_logged_in_user()
and all team-related code to use this framework.
- user web: show team founder link to "management functions"
even if they're not currently a member;
allow team founder to perform all operations even if not a member.
- turned on E_ALL error checking,
and fixed a bunch of warnings
and a couple of bugs ($team, not team)
- database: update schema/constraints for team fulltext search
- user web: forgot to change Serbia and Montenegro to Serbia
svn path=/trunk/boinc/; revision=13968
1) a WU is marked as ready for assimilation and has no errors;
2) it has no canonical result
In this case, the assimilate handler gets called anyway,
typically with the canonical result of the previous WU as arg.
Note: this situation doesn't arise normally;
it might happen if some results are deleted accidentally.
The fix:
- identify this situation, and set the WU.error_mask to a new code
(WU_ERROR_NO_CANONICAL_RESULT)
- zero out the "canonical_result" variable passed to the handler,
so even if the handler fails to check wu.error_mask,
at least it won't assimilate the same result twice.
Thanks to Hendrik Verhoek for finding this bug.
- DB schema: team table type is MyISAM, not InnoDB
svn path=/trunk/boinc/; revision=13938
New db tables banishment_vote and banishment_votes
Daemon script vote_monitor for timing the votes
New configuration option <moderators_vote_to_ban/>
svn path=/trunk/boinc/; revision=13917
These are stored in a new DB table, "team_delta".
The team founder can view the history in HTML or XML
(links on Management Functions page)
svn path=/trunk/boinc/; revision=13231
David Anderson