Commit Graph

2385 Commits

Author SHA1 Message Date
Eric J Korpela 8f33da07fc Way to be consisent with the types of versions in plan classes. Not. 2015-02-12 17:02:30 -08:00
Eric J Korpela 53a5151cf2 - SCHED: Scheduler didn't have a means of parsing android version setting
from plan_class_spec.xml so the min/max_android_version settings weren't
  functioning
2015-02-12 16:58:06 -08:00
David Anderson 57e8f8cf5d sample_substr_validator: bug fix for --reject_if_present 2015-01-31 18:30:36 -08:00
David Anderson 0fd3ea7226 sample_substr_validator: add --reject_if_present option.
See http://boinc.berkeley.edu/trac/wiki/ValidationIntro
2015-01-28 13:53:12 -08:00
David Anderson 918dcd9179 db_purge: fix typo that cause SQL error 2015-01-26 10:20:13 -08:00
Julien Nabet d75222ebf3 cppcheck: Prefer prefix ++/-- operators for non-primitive types 2015-01-14 21:18:29 +01:00
David Anderson b1ee6b00c4 scheduler, FCGI: flush log file after each request. From Nicolas. 2015-01-09 20:40:02 -08:00
David Anderson 947b7b3f1e scheduler: support Android version limits in XML plan class spec
Note: Android clients report the Linux kernel version in host.os_version.
A month or two ago we added the Android version,
which can't always be inferred from the Linux version.
This will only work for devices running this new client.
2015-01-05 16:32:30 -08:00
David Anderson 86c81fc587 db_purge: add --app cmdline option 2014-12-23 16:35:14 -08:00
David Anderson afd8783b60 server: add <hr_class_static> option
For projects (like Lattice) that assign a WU's HR class when it's created,
we don't want the mechanism that clears the HR class
if there are error results and no in-progress of completed results.
This option suppresses this.
2014-12-10 23:18:40 -08:00
David Anderson 9411118774 client: fix bug where GPU jobs non suspended
There was a bug where, when you suspend GPU activity,
GPU jobs show as suspended but are not actually suspended.
This was because of recent changes to distinguish GPU and non-GPU coprocs.
Change things so that coprocs are by default GPUs.
If you want to declare a non-GPU coproc in your cc_config.xml,
you much put <non_gpu/> in its <coproc> element.
2014-12-08 20:38:56 -08:00
David Anderson 7100992f82 start script: skip 'run_in_ops' in forming log/out/lock/pid filenames
Tasks and daemons written in PHP are generally run with a command of the form
run_in_ops script.php
The start script would, by default, base log/out/lock/pid filenames
on "run_in_ops".
This forced you to specify the filenames in config.xml, e.g.
<output>script.out</output>

Fix this by skipping the run_in_ops and using the next string
to form default filenames.
2014-12-03 14:23:03 -08:00
David Anderson d2c760f102 scheduler: fix crash due to uninitialized var 2014-11-25 22:44:11 -08:00
David Anderson 373d345e0f scheduler: OS version constraints in plan class XML spec for Linux/Android
The OS version# constraints (os_min_version, os_max_version)
were only supported for Win and Mac.
Make them work for Linux and Android too.
Note: for Android, we get the Linux kernel version.
This isn't the same as the Android version, but it may be enough
2014-11-22 13:19:54 -08:00
David Anderson db501739a6 client: timestamp all writes to stderr 2014-11-14 13:07:24 -08:00
David Anderson ecc5ef9496 Assimilator: don't error out if unrecognized args
(app-specific code might parse them)
2014-11-12 09:58:14 -08:00
David Anderson 2829e7f368 sched_customize.cpp: add another example, and remove S@h-specific code 2014-10-31 15:02:13 -07:00
David Anderson 3471cb5e38 scheduler: add <cpu_vendor_regex> option to plan class XML spec
Lets you limit an app version to a particular CPU vendor (AMD, Intel).
2014-10-27 10:36:32 -07:00
David Anderson 6eabd34068 server: extend script_validator and script_assimilator to pass extra args
You can arrange to pass result IDs, runtime, etc.
to your validation and assimilation scripts.  See
http://boinc.berkeley.edu/trac/wiki/ValidationSimple#Usingscriptinglanguages
http://boinc.berkeley.edu/trac/wiki/AssimilateIntro#Usingscriptinglanguages
2014-10-24 14:22:07 -07:00
David Anderson 9f3fe162d0 make_project, upgrade: copy new server programs 2014-10-23 11:57:05 -07:00
David Anderson d2488ef87f server: add script_assimilator.cpp: support for script-based assimilators.
Also fix bug in script_validator.cpp
2014-10-22 10:24:40 -07:00
David Anderson 302aa8c7aa server: add script_validator: a validator framework for Python, Perl, bash etc.
Added script_validator, a validator that invokes scripts of your choice
to check and compare results.
These can be in any language you want.
2014-10-21 22:14:57 -07:00
David Anderson d1b2c57c37 client, Android: clear out project directory on project reset.
When a project is reset, the client deletes all the files from its dir
that BOINC knows about.
However, there may be other files, created by its apps.
On Android, where space is likely to be scarce,
delete these when the project is reset.

Maybe we should do this on all projects.
But we'd need to not to delete user-created files
like app_config.xml, and anonymous-platform stuff
like app_info.xml and the app files.
2014-10-21 22:06:22 -07:00
David Anderson 9ba4e985b6 server: fixes/improvements to validator_test 2014-10-14 10:25:28 -07:00
David Anderson 870ec73327 scheduler: fix crash bug caused by recent checkin 2014-10-09 08:56:03 -07:00
David Anderson 323d61bcee scheduler: fix bug that would send work for miner_asic only if other types requested also 2014-10-08 10:18:16 -07:00
David Anderson 1eda7de1ac scheduler: add <nthreads_cmdline> option to plan class XML spec
When using XML plan class spec, the "mt" plan class would
correctly set avg_ncpus, but wouldn't pass a cmdline arg to the app.
So you'd get jobs where BOINC thinks it's using N CPUs,
but it's actually using only one.
2014-10-07 04:03:35 -07:00
David Anderson 526aff6e0d scheduler: fix msg typo.
Note: the "reliable" mechanism isn't yet implemented for
score-based scheduling, and it's not real clear how to do this.
2014-09-23 12:37:50 -07:00
David Anderson 027689bd7c Plan class XML spec: let you exclude particular Vbox versions 2014-09-23 10:58:29 -07:00
David Anderson c2a0421074 scheduler: add support for miner_asic coprocessor type
I.e. treat miner ASICs as a distinct processor type;
send miner_asic jobs only if the client requests them.

Note: I was planning to do this in a more general way,
in which the scheduler wouldn't have a hard-wired list of processor types.
However, that would be a large code change,
so for now I just added miner_asic to the list of processor types
(nvidia, ati, intel_gpu),
and made various changes to get things to work.

Also: in the job dispatch logic, try to send coproc jobs
before CPU jobs.
That way if e.g. there's a limit on jobs in progress,
we'll preferentially send coproc jobs.
2014-09-21 21:08:09 -07:00
David Anderson 7a84dabbbc scheduler: fix bug that broke enforcement of per-app job limits
The old scheduler worked as follows:
    scan jobs; for each job
        get_app_version
        do various checks
        add_job_to_reply
The check for per-app job limits was in get_app_version,
and the incrementing of per-app job count is in add_job_to_reply.

The new (score-based) scheduler works as follows:
    scan jobs; for each job
        get_app_version
        add to list
    sort list by score
    scan list; for each job
        do various checks
        add_job_to_reply

So the limit check (in get_app_version) was ineffective
because it happens before we've incremented counts.

Fix: do the limit check (also) in the "scan list" loop.

Bigger picture: we need to restructure app version selection;
job limit enforcement doesn't belong there any more.
2014-09-16 11:07:36 -07:00
David Anderson c897dc8276 db_dump: split host.serialnum into client version/vbox version/coprocs parts 2014-09-11 18:06:05 -07:00
David Anderson 6c5849d817 scheduler: fix bug that caused no jobs to be sent to pre-6.7 clients 2014-09-03 15:35:36 -07:00
David Anderson 60a3582151 scheduler: if client reports zero elapsed time, set ET to CPU time 2014-09-03 10:35:52 -07:00
David Anderson a7c6b32ca2 scheduler: changes to allow the code signing key to be changed.
Aargh!  My checkin of 31 Aug 2011 causes the client to strip the trailing \n
from the code-signing key; this change affects 7.0+ clients.
This broke the documented protocol for changing code-signing keys,
since signatures for the new key with the trailing \n won't work.

Solution: when you change code-signing key,
you have to sign both the stripped and non-stripped versions
of the new public key.
See http://boinc.berkeley.edu/trac/wiki/CodeSigning

This required a change to the scheduler so that it sends
one signature or the other based on client version
(stripped for 7.0+, else unstripped).
This change is entirely within send_code_sign_key(),
in case you want to update just that.
2014-08-29 13:35:51 -07:00
David Anderson d21a062f89 validator: initialize per-app avg credit update times to now rather than 0
Hopefully this will make per-app avg credit add up to
something close to the overall average
2014-08-18 20:10:08 -07:00
David Anderson 8f0893b140 db_dump: export credit_by_app files in XML stats
It does this by running html/ops/export_credit_by_app.php.
I added a cmdline arg to this script saying where to write the files.
2014-08-17 23:29:41 -07:00
David Anderson dbd2d03a0d server/web: add support for per-application credit
See http://boinc.berkeley.edu/trac/wiki/PerAppCredit
If enabled (by the <credit_by_app> config flag)
validators will maintain on a per-(app, user, credit type) basis,
and same for teams,
in new DB tables credit_user and credit_team.
This info is displayed in the web site, on user and team pages,
using project-supplied functions to generate the HTML.

Note: update_stats doesn't decay the recent-average values
for per-app credit; I'll add this if needed.
2014-08-15 14:01:32 -07:00
David Anderson d77f322014 scheduler, show_shmem: if shared mem size mismatch, give details 2014-08-11 11:46:25 -07:00
David Anderson 9f1625a0a7 scheduler and client: fix mechanism for exposing user CPID to GUI
A month or two ago I added code to put user CPID in the project info
exported via GUI RPC, so that GUIs (like BoincTasks) could link
to user pages on stats sites.

However, I completely forgot that the CPID known to the client
(PROJECT::cross_project_id) is the "internal CPID",
while what gets exported to stats is the "external CPID",
which is MD5(internal CPID, email addr).

Solution: include the external CPID in scheduler replies,
store it in the client state file,
and export it in GUI RPCs as PROJECT::external_cpid.

This will eventually work for BoincTasks,
but only after projects update their server software,
and volunteers update their client software.
2014-08-08 12:58:08 -07:00
David Anderson 82da9693a3 scheduler: fix S@h compile error 2014-08-07 14:47:17 -07:00
David Anderson 7fb13c0eaf scheduler: use config.maintenance_delay instead of 3600 for client backoff
Use a configurable delay (default 3600) instead of hardwired 3600 if
- can't open database
- feeder not running
- shmem size inconsistency
- can't open log file
- can't parse config file
- can't find key file
- can't attach shmem
2014-07-28 09:00:14 -07:00
David Anderson b3392bb195 scheduler: fix bug in plan class spec logic 2014-07-25 15:04:04 -07:00
David Anderson 89b51ea43d scheduler: preliminary support for generic coprocessors
A "generic" coprocessor is one that's reported by the client,
but's not of a type that the scheduler knows about (NVIDIA, AMD, Intel).

With this commit the following works:
- On the client, define a <coproc> in your cc_config.xml
  with a custom name, say 'miner_asic'.
- define a plan class such as
  <plan_class>
    <name>foobar</name>
    <gpu_type>miner_asic</gpu_type>
    <cpu_frac>0.5</cpu_frac>
  <plan_class>
- App versions of this plan class will be sent only to hosts
  that report a coproc of type "miner_asic".
  The <app_version>s in the scheduler reply will include
  a <coproc> element with the given name and count=1.
  This will cause the client (at least the current client)
  to run only one of these jobs at a time,
  and to schedule the CPU appropriately.

Note: there's a lot missing from this;
- app version FLOPS will be those of a CPU app;
- jobs will be sent only if CPU work is requested
... and many other things.
Fixing these issues requires a significant re-architecture of the scheduler,
in particular getting rid of the PROC_TYPE_* constants
and the associated arrays,
which hard-wire the 3 fixed GPU types.
2014-07-25 12:40:35 -07:00
David Anderson 8710b5e260 scheduler: when parsing trickle-ups, initialize "variety" field to empty
... otherwise can get garbage in that field if request doesn't include variety
2014-07-21 09:42:29 -07:00
David Anderson 9a9041cf7d server: fix support for client break; show it on web 2014-07-16 21:08:18 -07:00
David Anderson 246f5a2d1e client/scheduler: add support for "client brand"
Currently the server doesn't know about different client "brands",
e.g. HTC Power to Give, Charity Engine, GridRepublic, etc.,
so there's no way to collect statistics about them.

Changes:
- client: at startup, read a "client brand" string from client_brand.txt
    (i.e. branded clients will have to include this file in their installer)
    Report this string in scheduler requests.
- scheduler: parse this request element,
    and store it in host.serialnum as [BOINC|7.4.2|brand]
2014-07-16 20:18:06 -07:00
David Anderson 7c43a9fbb0 antique file deleter: skip no_delete WUs in finding oldest WU 2014-07-16 19:25:15 -07:00
David Anderson 61fb82e99b scheduler: add <maintenance_delay> config option.
Tells clients minimum delay to next request when the project is down.
Previously this was hardwired at 3600 sec.
2014-07-14 14:13:33 -07:00
David Anderson 3b12ee0311 transitioner: don't exit if fail to find expected host_app_version record. 2014-07-10 20:51:46 -07:00