TODO: remove related code
- validator: update wu.canonical_credit correctly.
However, this field should be deprecated.
- validator: check for error return from assign_credit_set().
svn path=/trunk/boinc/; revision=21096
are written to the DB.
The incremental approach was bogus.
New approach:
host_app_version: write directly; R/W interval is tiny
app_version: maintain an explicit list of update samples
for both PFC and credit.
When the validator flushes its app_version cache,
do careful updates.
Note: when using double fields in careful updates,
you can't test for equality. Use abs(new-old) < 1e-N
svn path=/trunk/boinc/; revision=21057
see http://boinc.berkeley.edu/trac/wiki/CreditNew
Projects will need to update DB and recompile all back-end programs.
Summary:
- new way of computing credit
- "reliable host" mechanism is per app version
- "host punishment" mechanism is per app version
- adjustment of wu.rsc_fpops_est provides the
equivalent of per app version DCF
- max jobs in progress is now per app
- max jobs per RPC is now per app
TODO:
- reliable mechanism:
- populate and use host_app_version.error_rate
- populate host_app_version.turnaround
- host punishment:
- populate host_app_version.max_jobs_per_day
- populate host_app_version.n_jobs_today
- use app.max_jobs_per_day_init
- job limits:
- use app.max_jobs_in_progress, max_gpu_jobs_in_progress
- use app.max_jobs_per_rpc
- adjust wu.rsc_fpops_est
- remove old credit stuff
fpops_cumulative, credit_multiplier
credit computation in scheduler
- AVERAGE class: use the Knuth algorithm (Wikipedia)
svn path=/trunk/boinc/; revision=21021
However, MySQL's default is that "affected rows" is
rows actually modified, which is not what we want.
Use the CLIENT_FOUND_ROWS option in mysql_real_connect()
to change the semantics to "rows matched".
From Oliver Bock.
svn path=/trunk/boinc/; revision=20880
New policy: anon platform and old platform jobs
get average credit, possibly scaled by elapsed time.
We no longer attempt to guess what app version produced them.
svn path=/trunk/boinc/; revision=20816
Triggering the work generator is now done via the DB
instead of flat files.
Since only E@h uses locality scheduling,
I kept the DB changes in a separate file (db/schema_locality.sql).
There's a new field in the workunit table,
and that's a required update (in db_update.php)
- manager: compile fix
svn path=/trunk/boinc/; revision=20807
elapsed_time: the elapsed time (runtime) as reported by client
flops_estimate: the app's estimated FLOPS as reported by app_plan()
app_version_id: the DB ID of the app_version used
(or -1 if anonymous platform)
TODO: show these in the web interfaces,
and use them where appropriate
svn path=/trunk/boinc/; revision=19002
Old:
1) check deadline based on wu.delay_bound
2) in add_result_to_reply(), potentially modify wu.delay_bound,
e.g. because of retry acceleration
problem: reducing delay bound may cause deadline miss
New:
1) new function get_delay_bound_range()
(called from wu_is_infeasible_fast())
returns optimistic and pessimistic delay bounds.
Retry acceleration logic is here.
2) check deadline based on optimistic bound;
if that fails, check based on pessimistic bound.
Set wu.delay_bound to the one that worked.
Notes:
- get_delay_bound_range() needs result priority and report deadline,
and it's called before we read the full result.
So add these items to WORK_ITEM and WU_RESULT.
- get_delay_bound_range() could be customized for
project-specific deadline policy.
- add_result_to_reply() was becoming a toxic waste dump.
Deadline-related stuff should have been factored out in any case.
svn path=/trunk/boinc/; revision=18946
don't modify user preferences or CPID.
- client: fix bug that shows ATI version incorrectly
- database: host.posts has been repurposed as a salt (or seqno)
for a new type of weak authenticator that won't depend on password
- web code:
modify forum_preferences.posts instead of host.posts.
(actually, the former isn't used either, we just do a select count(*);
should fix this at some point).
svn path=/trunk/boinc/; revision=18865
after freeing the MYSQL_RES it came from.
(this didn't appear to cause any problems, but not good form).
Fixes#883
svn path=/trunk/boinc/; revision=17904
and add <cuda_multiplier>.
The latter is used in calculating max jobs/day for a host;
namely, it's host.max_results_day * (NCPUS + NCUDA*cuda_multiplier).
Set it to 10 or so if you have CUDA apps.
- scheduler: don't overload effective_ncpus();
instead, add two new functions,
max_results_day_multiplier() and max_wus_in_progress_multiplier()
- scheduler: don't reduce max_results_day if we get an aborted job
(it might have been aborted by the project;
not appopriate to punish host in this case)
svn path=/trunk/boinc/; revision=16959
put a textual summary of them in host.serialnum (currently unused)
- web: show coprocs on host detail page
- db_dump: include coproc info in host XML
svn path=/trunk/boinc/; revision=16697
is still alive before handling a request. If not, try to reconnect.
This will hopefully make things work better if MySQL goes down and up
when using FCGI.
svn path=/trunk/boinc/; revision=16112
we're the main program (otherwise we didn't lock it in
the first place, and a crash results). From Artyom Sharov.
- scheduler: add support for the GCL simulator,
which uses special versions of backend programs
that use virtual time,
and that wait for signals instead of sleep()ing.
To compile:
make clean
configure CXXFLAGS="-DGCL_SIMULATOR"
make
svn path=/trunk/boinc/; revision=16038
- API: in APP_INIT_DATA, enclose project preferences in tags
so that it's legal XML
- scheduler: add <multiple_clients_per_host> option.
Use this if your project runs on Condor or grids
and (to use multicore machines) you're running
multiple clients per host.
This will skip the host lookup based on IP address.
svn path=/trunk/boinc/; revision=15954
added "count" field to DB table to keep track of how many times
we've refreshed.
- show refresh schedule on main courses page
- default for random structure is all units, not 1
svn path=/trunk/boinc/; revision=15846
- scheduler: fix bug in adaptive replication:
if send an unreplicated job to untrusted host,
set both wu.target_nresults and wu.min_quorum to app.target_nresults.
svn path=/trunk/boinc/; revision=15762
wish to use it.
- The script calculate_credit_multiplier (expected to be run daily as
a config.xml task) looks at the ratio of granted credit to CPU time
for recent results for each app. Multiplier is calculated to cause
median hosts granted credit per cpu second to equal to equal that
expected from its benchmarks. This is 30-day exponentially averaged
with the previous value of the multplier and stored in the table
credit_multplier.
- When a result is received the server adjusts claimed credit by the
value the multiplier had when the result was sent.
svn path=/trunk/boinc/; revision=15661
If an app is hard, the scheduler always does the deadline check,
even if the client has no other jobs for this project.
And the estimated wallclock duration is multiplied by 1.3,
to avoid sending jobs to hosts that will barely make the deadline.
Hard apps are marked by setting weight = -1.
This is a total kludge, to avoid adding another field to app.
svn path=/trunk/boinc/; revision=15607
(calculate_credit_multiplier) to determine what factor to multiply
claimed credit by before insertion into the database. Changes to scheduler
to implement have not yet been checked in.
svn path=/trunk/boinc/; revision=15309
added client/scripts to default client build
removed sea from the default clientgui build
added locale/client to the default clientgui build
moved installed headers from $(includedir) to $(pkgincludedir) which
is $(includedir)/boinc by default.
removed redefinitions of $(includedir) from Makefiles.
- configure:
added locale/client/Makefile to AC_CONFIG_FILES
svn path=/trunk/boinc/; revision=15300
and change the correspending structure field from 64KB to 256KB
(could increase this if needed).
This is needed to handle app versions with lots (> 100) of files
- change LARGE_BLOB_SIZE to BLOB_SIZE a bunch of places
- Change COPROCS from vector<COPROC> to vector<COPROC*>.
Otherwise the right virtual functions of COPROCs don't get called
svn path=/trunk/boinc/; revision=14986
in server->client reply messages and in the client itself,
move app-planning info from RESULT to APP_VERSION.
This was necessary to allow anonymous platform info (app_info.xml)
to specify avg_ncpus, etc.
e.g., if someone wants to write a multithread version of SETI@home,
or a GPU/CUDA version,
they can run it using the anonymous platform mechanism
and it will be scheduled correctly.
If a server sends an existing APP_VERSION but with different
app-planning info, the client will accept and use the new info.
svn path=/trunk/boinc/; revision=14978
- update_versions: use __ (not :) as separator for plan class
- client: add plan_class to APP_VERSION;
an app version is now identified by platform/version/plan_class
- client CPU scheduler: don't assume apps use 1 CPU
- client: add avg_ncpus, max_cpus, flops, cmdline to RESULT
- scheduler: implement app planning scheme
Other changes:
- client: if symlink() fails, make a XML soft link instead
(for Unix running off a FAT32 FS)
- client: don't accept nonpositive resource share from AMS
- daemons and DB: check for error returns from enumerations,
and exit if so. Thus, if the MySQL server goes down,
all the daemons will soon exit.
The cron script will restart them every 5 min,
so when the DB server comes back up so will the project.
- web: show empty max CPU % as ---
- API: get rid of all_threads_cpu_time option (always the case now)
svn path=/trunk/boinc/; revision=14966
- DB code: remove "is_high_priority" stuff.
- scheduler: merge find_app_version() into get_app_version().
Have the latter memoize its results (both positive and negative).
Have it call app_plan() for apps with nonempty plan_class.
- scheduler: first steps towards improved selectability of log messages.
It will eventually be like the client,
where you can select among various types of messages.
- feeder: if can't unlink the reread_db trigger file, exit
(else we'd go into an infinite loop)
svn path=/trunk/boinc/; revision=14940
and apps that use coprocessors.
There now can be several app_versions for the same
(app, platform, version_num) combination.
This changes a number of things.
- Added app_version.plan_class field to DB
- update_versions now looks for a :plan-class in the
file or directory name, and puts it in the app_version's DB record
- Change uniqueness constraint to include plan_class
- Feeder: the feeder was putting non-deprecated app_versions
in shared mem, and leaving it to the scheduler to
find the latest version for a given platform.
This is dumb.
Instead, for each app/platform pair the feeder now
finds the highest version number of a non-deprecated app version,
and enumerates all non-deprecated app_versions with that
app/platform/version
- Scheduler: add a BEST_APP_VERSION data structure that keeps track,
for each app, what the best app_version is for this host.
This saves the work of recomputing it for each job.
svn path=/trunk/boinc/; revision=14906
The new field (workunit.rsc_bandwidth_bound)
goes at the END of the record.
Always do it this way!
- make_work: after creating a batch of new WUs,
we were waiting 60 sec for the transitioner to
create the results for them
(so that our next count of unsent results would be correct).
This is bogus; if e.g. the transitioner isn't running,
we'll never get the results, and we'll keep creating WUs forever.
Instead: explicitly wait for there to be results for
the last WU from the batch just created.
- scheduler: parse <allow_non_preferred_apps>, <allow_beta_work> correctly.
svn path=/trunk/boinc/; revision=14875
The design has been changed to constant #threads per app version
Various changes from Kevin Reed/WCG:
- server: add workunit.rsc_bandwidth_bound: if nonzero,
send this WU only to hosts with that much download bandwidth
- assimilators: if a handler returns DEFER_ASSIMILATION,
the WU remains in INIT state and will be handled when the
next instance completes.
Useful if you want the assimilator to see all instances.
- scheduler: when setting result.outcome = DETACHED,
set received_time to now
- scheduler: removed the reliable_time and reliable_min_avg_credit
options
- scheduler/web: add optional <allow_non_preferred_projects>
in project preferences.
If present, user will accept work from non-selected apps
if no work is available for selected apps
- scheduler: improved messages for projects with multiple apps
- scheduler: added config options
<granted_credit_weight> and <granted_credit_ramp_up>.
Used in calculating host.claimed_credit_per_cpu_sec,
but I'm not sure how.
- Added two new credit-granting formulas (validate_util.C):
stddev_credit() and two_credit()
- server DB: add rollback_transaction() and affected_rows() to DB_CONN
NOTE: DB update required
svn path=/trunk/boinc/; revision=14870
But if we do, set their transitioner time to plus infinity
so that we don't see them again.
(otherwise we go into an infinite loop)
- DB code: remove "high_priority" from queries not from scheduler
(should probably remove them from there too)
- file_deleter: print error msg if apache user doesn't exist
svn path=/trunk/boinc/; revision=14835
Lets you assign a WU to a particular host,
to one or all hosts belonging to a user or team, or to all hosts.
See http://boinc.berkeley.edu/trac/wiki/AssignedWork
Disabled unless you include <enable_assignment> in config.xml
Uses a new DB table.
Tested but only a little.
- Server: code cleanup; moved result-handling to a new file,
and removed the PLATFORM_LIST arg to everything
(put it in SCHEDULER_REQUEST instead)
svn path=/trunk/boinc/; revision=14767
which added only confusion.
Implement login directly, using cookies.
- All cookie setting/clearing now goes through two functions,
send_cookie() and clear_cookie().
These deal with path and expiry
(e.g. if you want to have different language or forum settings
on two projects on the same server, that now works).
svn path=/trunk/boinc/; revision=14735
in case someone else changed since we read it.
Hopefully this will fix a race condition
where WU results get sent to different HR classes.
(Alternatively we could use transactions,
or acquire the semaphore during read/update,
but this could impact performance).
svn path=/trunk/boinc/; revision=14710
causing a potential loss of precision.
Change it to double (same as atof())
- When moderator locks a thread, let them specify reason
svn path=/trunk/boinc/; revision=14662
Users can choose whether to get 1 email per notification,
a daily "digest" email, or no email.
(All notifications are shown on the Account page).
Currently used for:
- Friend requests and confirmations
- Posts to subscribed threads
- Private messages
NOTE: To implement the "daily digest" feature, projects must add
a periodic task for html/ops/notify.php to config.xml
- web: have project_footer() generate links for
Account Page and Message Boards as well as Home
NOTE: projects that want this change
will have to modify their own project.inc.
svn path=/trunk/boinc/; revision=14447
This lets you create a project for Bossa or Bolt
without compiling the C++ stuff, which is not used by Bossa or Bolt.
- DB schema: change type= to engine= to avoid MySQL warning messages
svn path=/trunk/boinc/; revision=14437
- May as well start keeping checkin notes for Bolt.
At this point the fundamental structure of Bolt is
more or less complete, I hope; many features are missing.
Items for this particular checkin:
- Make things work right when you resume a course at an
exercise answer page.
This creates a new View, whose prev_view_id link points
to the exercise view, not the answer page view
- When leave a page using "Up", terminate the view accordingly.
- Rather than using $_GET, Form vars are now passed to exercise primitives
in a global variable $bolt_ex_query_string, which they must parse.
svn path=/trunk/boinc/; revision=14414
Each account has an associated weak account key.
The user can see it by visiting weak_auth.php.
If you put the weak account key in the account file on a computer
(instead of the regular account key)
the computer will attach to the account.
But the weak account key cannot be used to log in
to the account on the web, or to do web RPCs, or anything else.
Note: this involves a scheduler change.
svn path=/trunk/boinc/; revision=14367
- user web: improve "team search" function,
and get rid of the superceded "search by name" function
- user web: don't show team founder link to initiate transfer
- user web: show team types
- user web: fix "top teams by type" function
- client: tread NOT_FOUND error on file upload as transient;
it means the project is missing its file_upload_handler
svn path=/trunk/boinc/; revision=14272
For thread titles, show links to forum/category also
For posts, show complete post, with search terms highlighted.
- user web: in display of posts for a user,
show links to thread and forum also
- user web: posts/threads can become "orphaned" if their forum
disappears, e.g. team message board is deleted.
Handle this gracefully in search results and user post list
- user web: where HTML tags are allowed, say so
- user web: fix bug where wrap_postcounts weren't being shown in form
- DB: use "tinyint" instead of "tinyint(1) unsigned" etc.
svn path=/trunk/boinc/; revision=14267
should default to off.
Note: it's necessary to clear everyone's flag.
Projects will need to notify users (e.g. via a news item)
that they need to explicitly set this pref if they want email
notification of PMs
svn path=/trunk/boinc/; revision=14255
- DB: added parent_type to forum
- user web: BoincUser uses caching by default
- client: fixed small memory leak in statistics parsing
svn path=/trunk/boinc/; revision=14130
appointed by the founder, who have most of the same
rights as the founder (edit team info, view/remove users).
They can't, however, change the founder, or select/remove Admins.
- user web: added the ability for founder to delete an empty team
- user web: count/show all team members, not just those with credit
- user web: Team Search returns top team if form fields left blank
- Added db_update.php entry for Eric's banishment_vote stuff
svn path=/trunk/boinc/; revision=14108
Here's how things should eventually be:
- inc/db_conn.inc defines DbConn, a PHP class representing
a connection to a database with
various generic operations (update, insert etc.).
DbConn prepends the DB name to table names,
so you don't have to select a DB;
this means you can access multiple databases
(e.g. BOINC and Bossa) and use persistent connections.
ALL MYSQL-SPECIFIC CODE WILL EVENTUALLY BE HERE.
- inc/boinc_db.inc defines classes BoincUser, BoincTeam etc.,
representing the various BOINC DB tables.
All access to the DB should use these classes.
When you use this framework, no initialization calls are needed.
db.inc is deprecated and should eventually be removed.
I changed get_logged_in_user()
and all team-related code to use this framework.
- user web: show team founder link to "management functions"
even if they're not currently a member;
allow team founder to perform all operations even if not a member.
- turned on E_ALL error checking,
and fixed a bunch of warnings
and a couple of bugs ($team, not team)
- database: update schema/constraints for team fulltext search
- user web: forgot to change Serbia and Montenegro to Serbia
svn path=/trunk/boinc/; revision=13968
1) a WU is marked as ready for assimilation and has no errors;
2) it has no canonical result
In this case, the assimilate handler gets called anyway,
typically with the canonical result of the previous WU as arg.
Note: this situation doesn't arise normally;
it might happen if some results are deleted accidentally.
The fix:
- identify this situation, and set the WU.error_mask to a new code
(WU_ERROR_NO_CANONICAL_RESULT)
- zero out the "canonical_result" variable passed to the handler,
so even if the handler fails to check wu.error_mask,
at least it won't assimilate the same result twice.
Thanks to Hendrik Verhoek for finding this bug.
- DB schema: team table type is MyISAM, not InnoDB
svn path=/trunk/boinc/; revision=13938
New db tables banishment_vote and banishment_votes
Daemon script vote_monitor for timing the votes
New configuration option <moderators_vote_to_ban/>
svn path=/trunk/boinc/; revision=13917
not the worker signal handler.
There's no reason to call it from the signal handler -
it returns the CPU for the entire process, not the calling thread.
And it may be asynch-signal-handler-unsafe.
- API: comment out checks for bad CPU times.
I don't think this is needed now, and in some cases it's wrong
(multi-threaded apps can accumulate CPU faster than real time)
- API, Unix: in boinc_calling_thread_cpu_time(), don't retry getrusage().
- Bossa: switch to better class structure (suggested by Nicolas Alvarez).
Haven't switched to mysqli yet, but will later.
Also various other Bossa fixes
svn path=/trunk/boinc/; revision=13855
like source code and text files. I skipped to check most files in html/
and mac_*/ though.
- Added svn:executable to tools/watch_tcp because it has a shebang.
svn path=/trunk/boinc/; revision=13819
update_versions to add version 6 apps.
It looks for API_VERSION string in main executable,
adds the API version to the app_version XML,
and sets min_core_version to 6 for version 6+ apps
- API: include API_VERSION string
- convert tabs to spaces here and there
- scheduler: parse unused elements in <net_stats>
- ops/show_log.php: if no URL args, just show form (fixes#415)
- client: parse and store api_version (not used yet)
svn path=/trunk/boinc/; revision=13627
These are stored in a new DB table, "team_delta".
The team founder can view the history in HTML or XML
(links on Management Functions page)
svn path=/trunk/boinc/; revision=13231
Make a single function that creates teams
and cleanses arguments.
- API: don't include config.h in parse.h.
This file is included from apps
(indirectly, via graphics_api.h)
so it shouldn't assume that config.h exists
svn path=/trunk/boinc/; revision=13212
To do this, set host.max_results_day to -1.
If you do this, scheduler requests from that host
will get an error message, and will otherwise be ignored
(no jobs in or out, no trickles).
- Scheduler: send_message() should be called ONLY if you're
not going to call handle_request();
otherwise we'll write two separate replies.
To fix this, I added a separate function (send_error_message())
that can be called within handle_request()
to deal with error situations.
- Scheduler: moved debug_sched() to main.C
- Scheduler: moved logic to send "delete file" commands
out of handle_request() into a separate function,
send_file_deletes() in sched_locality.C.
Remove #ifdef EINSTEIN_AT_HOMEs; maybe someday another project
will use locality scheduling!
svn path=/trunk/boinc/; revision=13108
on successive calls, scans through ALL the sendable
jobs satisfying the select clause
(it does this by ID order, so there's no order clause)
This is used for HR, so that if a job has been committed
to an HR class, we eventually get it.
With extremely minimal testing, the new HR stuff seems to work.
db/
boinc_db.C,h
sched/
feeder.C
sample_work_generator.C
server_types.C
svn path=/trunk/boinc/; revision=12988
there is for each HR class, and writes it to a file.
This will be used soon for HR support in the feeder.
- split the HR code into hr.C,h (stuff used by both census and scheduler)
and sched_hr.C (stuff used only by the scheduler)
- database: change DB_CREDITED_JOB to treat workunitid
as a double (which it is) rather than a long.
BTW, long == int.
- fixed lots of compile warnings in the server code
db/
boinc_db.C,h
lib/
boinc_cmd.C
miofile.C
util.C
sched/
Makefile.am
census.C (new)
feeder.C
file_deleter.C
file_upload_handler.C
handle_request.C
hr.C,h (new)
main.C
sample_assimilator.C
sample_work_generator.C
sched_array.C
sched_hr.C,h
sched_send.C
server_types.C
transitioner.C
validator.C
svn path=/trunk/boinc/; revision=12970
e.g. distinguish between models of Intel and AMD
- Scheduler: add a quick HR check that doesn't access the DB
- Transitioner: if a workunit has >0 error results and no success results,
set its HR class to zero
From M.F. Somers.
db/
boinc_db.C,h
sched/
sched_array.C
sched_hr.C,h
transitioner.C
svn path=/trunk/boinc/; revision=12773
and add <platform> and <version_num> elements to <result>
(server half of multi-version changes)
- scheduler: make ssp a global; could eliminate from args everywhere
db/
boinc_db.h
sched/
feeder.C
main.C,h
sched_send.C
sched_shmem.C,h
server_types.C
svn path=/trunk/boinc/; revision=12535
- Server: add new database field host.error_rate.
Stores a dynamic estimate of the fraction of the hosts's
results that fail validation (updated by the validator).
client/
sim.C
cs_prefs.C
db/
boinc_db.C,h
html/ops/
db_update.php
sched/
handle_request.C
sched_msgs.h
validator.C
svn path=/trunk/boinc/; revision=12466
WU.rsc_fpops_est field. This can be useful in
determining how much credit to assign, and preventing cheating.
DAVID: I was going to be a good guy and document this, but you
forgot to check validate_simple.php and validate_advanced.php
into CVS (:-).
svn path=/trunk/boinc/; revision=10297
(1) over and (2) validate state init and (3) NOT file_delete==INIT.
(David, please bless addition of file_delete_state to SCHED_RESULT_ITEM.)
svn path=/trunk/boinc/; revision=10245
which may modify an EXISTING & OVER result in the database.
file_upload_handler: check for 'stop_upload' trigger file in the
project root directory. If so, return ERR_TRANSIENT to hosts which
attempt uploads. This is the equivalent of 'stop_sched' for the
scheduler. (Eric, you may want to check that this is FCGI compatible).
svn path=/trunk/boinc/; revision=10227
' by \'. This makes the string longer. To guarantee that the
resulting longer string does not overflow storage, escape_string()
uses safe_strncpy() to copy the longer string back. But this
is too heavy-handed, since safe_strncpy() will blindly truncate
the string. Thus it might clobber \\ into \ or \' into \ which
causes trouble.
David, Rom, please double-check my changes, since escape_string()
is used all over the place.
svn path=/trunk/boinc/; revision=9163
David Anderson