(but not all) wasn't finished.
New logic: if the project has an NCI app then:
- make a list of NCI apps for which the client doesn't have
a job in progress.
- try to send one job for each of these apps
- do this even if no work is being requested.
- don't send jobs for NCI apps by other mechanisms
NOTE: the client logic isn't quite right for mixed NCI projects.
If there's no job for a given NCI app,
the client should do a scheduler RPC.
This isn't critical so we won't do this now.
svn path=/trunk/boinc/; revision=26068
cmdline tool for remote job submission (not done)
- remote job submission: support the 4 file modes described
in the documentation (not done)
svn path=/trunk/boinc/; revision=26067
- Fix various #include issues.
CODING STYLE LAW (minimal inclusion principle):
If foo.cpp requires <blah.h>,
#include <blah.h> in foo.cpp, NOT foo.h
svn path=/trunk/boinc/; revision=25837
(or at least the last double).
This accommodates a particular application (LAMMPS)
that can only append to this file.
- CAS@home stuff
svn path=/trunk/boinc/; revision=25557
to be sent to non-targeted hosts.
The feeder was erroneously putting targeted jobs
in the shared mem cache.
Changes:
- The feeder only enumerates jobs for which
workunit.transitioner_flags is zero.
NOTE: this field is nonzero iff the job is assigned.
- create_work: when creating an assigned jobs,
set workunit.transitioner_flags appropriately
svn path=/trunk/boinc/; revision=25314
This now supports two main use cases:
1) there's a job that you want to run once on all hosts,
present and future
(or all hosts belonging to a user, or to a team).
The job is never transitioned, validated, or assimilated.
2) There's a normal job for which you want to use only
hosts belonging to a specific user (e.g. cluster or cloud hosts).
This restriction can be made either when the job is created,
or on the fly,
e.g. as part of a scheme for accelerating batch completion.
For the latter purposes we now provide a function
restrict_wu_to_user(DB_WORKUNIT&, int userid);
The job goes through the standard
transitioner/validator/assimilator path.
These cases are enabled by config flags
<enable_assignment_multi/>
<enable_assignment/>
respectively.
Assignment of type 2) are no longer stored in shared mem,
so there is no limit on their number.
There is no longer a rule that assigned job names must contain "asgn".
NOTE: this requires a database update.
svn path=/trunk/boinc/; revision=25169
send the size of the compressed file as well.
- client: parse and write the compressed size (FILE_INFO::gzipped_nbytes).
For get_transfer GUI RPCs, if it's a compressed download send
the compressed size.
That way the manager will show the fraction done correctly.
svn path=/trunk/boinc/; revision=25152
(It turns out that the compression schemes supported by
Apache and libcurl, suprisingly, aren't restartable.)
if a <file_info> from the server contains <gzipped_url> tags,
use those instead of the <url> tags,
and flag the file as "download_gzipped".
If this is the case, download NAME.gz and save it as NAME.gzt.
When the download is complete, rename NAME.gzt to NAME.gz,
and uncompress it to NAME.
(this ensures that if NAME.gz is present, it's complete).
Also do the uncompression, if needed, in verify_file().
This ensures that the uncompression will eventually get done
even if the client quits are crashes in the middle.
- update_versions: if <gzip> is present in a <file_info>,
add a gzipped copy in the download directory
and add a <gzipped_url> elements to the app version's xml_doc.
svn path=/trunk/boinc/; revision=25112
- prompt user for DB user/passwd if DB updates needed
- error out immediately if any DB update fails
Fixes#1169, #1170
- create_work: check for duplicate <file_info>s in input template
NOTE: the input template syntax is pretty clunky
svn path=/trunk/boinc/; revision=24969
Report it (along with disk usage) in scheduler request messages.
This will allow the scheduler to send file-delete commands
if the project is using more than its share.
- client: add <disk_usage_debug> log flag
- create_work: add --help, show --command_line option
svn path=/trunk/boinc/; revision=24968
(manage_privileges and get_output.php).
From Nico Schlitter.
- change things so that
make_project --test_app
will produce input/output templates that are compatible
with the remote job submission tools
svn path=/trunk/boinc/; revision=24774
says whether the client is using account-based sandboxing.
Needed by vboxwrapper.
- web: don't show authenticator in Your Account page.
The problem with authenticators is that if a bad guys gets yours,
they can log in to your account even if you change your password.
Eventually we should not use them at all
(i.e. in cookies and sent to client).
- make_project: copy python stuff (e.g. bin/start) for web-only projects;
generate appropriate project.readme file for web-only projects.
svn path=/trunk/boinc/; revision=24447
for canceling jobs
- added program cancel_jobs for canceling jobs
- DB interface: it's not an error if update_fields_noid()
affects != 1 rows
svn path=/trunk/boinc/; revision=24413
and other operations.
You can now designate a user as "manager" for a particular app.
They can then:
- control job-submit permissions for that app
- deprecate/undeprecate versions of the app.
- abort jobs for that app
You can also designate a user a manage for the project.
They can then edit permissions and quotas,
as well as performing the app-specific functions for all apps.
This is described here:
http://boinc.berkeley.edu/trac/wiki/MultiUser#Accesscontrol
This required some changes to the DB schema.
svn path=/trunk/boinc/; revision=24250
Add parsed_tag and is_tag to the class,
so that parsing functions don't need to declare them
and pass them around.
- Complete the task of using XML_PARSER as the argument
to all parsing functions.
(Internally, many of these functions still use the old XML parser;
that's the next step.)
svn path=/trunk/boinc/; revision=23978
are not processed correctly
- remote job submission: debug
- create_work: --rsc_fpops_est etc. should override the template file
svn path=/trunk/boinc/; revision=23942
If present, "file_prefix/" is prepended to the logical names
of input and output files of jobs using that app version.
I.e. for Vbox wrapper based app versions, file_prefix is "share",
so that I/O files are put in a "share" subdirectory of the slot dir.
- update_versions: add support for
<dont_throttle>
<file_prefix>x</file_prefix>
in version.xml
svn path=/trunk/boinc/; revision=23924
<file> element in version.xml.
If present, update_versions will gzip the file in the download dir.
Use this only if you have gzip-encoding enabled in Apache.
svn path=/trunk/boinc/; revision=23915
as described here: http://boinc.berkeley.edu/trac/wiki/ClientDataModel
Compatibility: if your project is using upload signatures:
- set ignore_upload_certificates
- disable job creation
- let your job queue drain
- upgrade to new server software
- clear ignore_upload_certificates
- enable job creation
svn path=/trunk/boinc/; revision=23863
- don't create result records for uploads and downloads.
Just create a msg_to_client record.
- the scheduler handles file-transfer results specially;
it makes a vector of them, then calls a project-supplied function
handle_file_xfer_results()
- change the interface and implementation of put_file and get_file
- client write project sched priority in GUI RPC replies,
but not to the state file
svn path=/trunk/boinc/; revision=23857
to mess up input templates containing
<copy_file/> or other attribute tags.
XML_PARSER now contains a member element() for when
you want to copy an element without knowing its structure.
svn path=/trunk/boinc/; revision=23790
and controlling batches of jobs
- web: add an administrative interface for controlling
user permissions for submitting jobs
- web: add an interface where users can view and control
their submitted jobs
See: http://boinc.berkeley.edu/trac/wiki/RemoteJobs
This is at a functional but rough stage.
svn path=/trunk/boinc/; revision=23762
(so that they don't have to be 1 element/line)
and also allow optional <input_template> root element
- fix bug in WORKUNIT DB interface
svn path=/trunk/boinc/; revision=23648
Problems with the old version:
- Lots of info encoded in file and directory names: messy kludge
- The convention that directory name == main program name
created a bad situation when using the wrapper:
if you made a new version of your application,
you were forced make a new version of the wrapper
just so it could have a different name.
The new version is described here:
http://boinc.berkeley.edu/trac/wiki/AppVersionNew
- Remove some Python code that suppored the old version
- client: some fixes to dir_size() that might fox
a buffer overflow when using symbolic links
svn path=/trunk/boinc/; revision=23525
fix bug that corrupted WU command lines.
The problem: we were using strcpy(p, p+n) to delete the
first n characters of p.
This is incorrect - the behavior of strcpy() is undefined
if its args overlap.
On some systems (e.q. AQUA's server) it does wacky things.
svn path=/trunk/boinc/; revision=23007
This allows a user to delete all traces of themselves from a project.
Namely:
- clear fields of user record: email_addr, authenticator,
name, country, postal_code
Note: record is not deleted
- clear the domain_name and last_ip_addr fields of hosts
Note: records are not deleted
- quit team
- delete private messages sent and received
- delete forum posts, subscriptions, and forum prefs
- delete profile and associated images
- server: compile fix
svn path=/trunk/boinc/; revision=23006
(for many platforms) from samples/example_app/bin
- make_project: change name of example app from uppercase to example_app.
- update_versions: allow version numbers to not have decimal points
- sample work generator: make app name and template files
command-line options;
default to "example_app", "example_app_in.xml", "example_app_out.xml"
svn path=/trunk/boinc/; revision=22667
URL, size, and MD5 of input files.
This supports "non-local" input files,
i.e. files not present on the project server.
NOTE: as implemented,
this requires a separate input template for each job.
It would be slightly better to let you specify the
URL/size/MD5 in the create_work() call.
From Zoltan Farkas (SZTAKI)
svn path=/trunk/boinc/; revision=22320
It wasn't copying html/ops/db_update.php,
so it wasn't doing necessary DB updates.
Fixed this by always copying html/ops/*.
Even with this fix, there is a problem when using
the --server_only or --web_only options of upgrade:
if any DB updates are done, they may affect the
server code that's not being updated, resulting in crashes.
I added a warning message in this case,
recommending that a full upgrade be done.
svn path=/trunk/boinc/; revision=22200
That produced a messed-up query that assigned garbage values to:
host_app_version.turnaround_var
host_app_version.turnaround_q
host_app_version.max_jobs_per_day
host_app_version.consecutive_valid
To repair these:
- set turnaround_var and turnaround_q to zero
- if max_jobs_per_day is outside of
(0..config.daily_result_quota)
set it to config.daily_result_quota
- if consecutive_valid is outside (0..1000), set it to zero
I added a script, html/ops/repair_21812.php, that does this;
if you ran server code between [21181] and [21812], run this script.
- scheduler/transitioner: add <debug_quota> log flag
- changed the build system to always use -Wall
(if we'd done this before, this bug wouldn't have happened)
- fixed a bunch of other compile warnings
svn path=/trunk/boinc/; revision=21812
see http://boinc.berkeley.edu/trac/wiki/CreditNew
Projects will need to update DB and recompile all back-end programs.
Summary:
- new way of computing credit
- "reliable host" mechanism is per app version
- "host punishment" mechanism is per app version
- adjustment of wu.rsc_fpops_est provides the
equivalent of per app version DCF
- max jobs in progress is now per app
- max jobs per RPC is now per app
TODO:
- reliable mechanism:
- populate and use host_app_version.error_rate
- populate host_app_version.turnaround
- host punishment:
- populate host_app_version.max_jobs_per_day
- populate host_app_version.n_jobs_today
- use app.max_jobs_per_day_init
- job limits:
- use app.max_jobs_in_progress, max_gpu_jobs_in_progress
- use app.max_jobs_per_rpc
- adjust wu.rsc_fpops_est
- remove old credit stuff
fpops_cumulative, credit_multiplier
credit computation in scheduler
- AVERAGE class: use the Knuth algorithm (Wikipedia)
svn path=/trunk/boinc/; revision=21021
or an auth_ops() function (not necessarily both).
New projects are given an .htaccess file with no .htpasswd,
so they must either create a .htpasswd file
or define auth_ops() in order to use admin web pages.
svn path=/trunk/boinc/; revision=20671
- remove obsolete and buggy code from transitioner (create_result() in backend_lib)
- account for 'mixed' scheduling in explain_to_user() in sched_send.cpp
- finish transition to configurable patterns for distinguishing files reported by the client
in the Einstein@home-specific part of send_work_locality in sched_locality
(removed previous hardcoded strcmps)
svn path=/trunk/boinc/; revision=20074
Old: in a flat file (html/project/project_news.inc)
New: in a forum (called "News" by default)
The script html/ops/news_convert.php copies news from
old to new format.
You'll also need to edit your index.php and use
"show_news(0, 5)" to show news.
- web: added a "message of the day" mechanism.
Edit html/user/motd.php to show a message.
This will be shown as the first news item,
but it's not archived (i.e., it's not a forum post)
svn path=/trunk/boinc/; revision=19949
make sure they're the same (i.e., enforce file immutability).
TODO: do this for input files too
- client: check return values from CUDA device detection functions
svn path=/trunk/boinc/; revision=18903
Daemons that are managed by 'start' go to $(libexecdir)/sched.
The (F)CGI stuff goes to $(libexecdir)/cgi-bin.
Finally, example applications go under $(libexecdir)/examples.
svn path=/trunk/boinc/; revision=18378
It is now possible to use the same set of tools for multiple
projects by setting BOINC_PROJECT_PATH.
From Gabor Gombas.
svn path=/trunk/boinc/; revision=18376
make_project now generates PROJECT/bin/boinc_path_config.py
to ensure that the interpreter will find the modules.
From Gabor Gombas.
svn path=/trunk/boinc/; revision=18374
- client (Unix): if client crashes while benchmark processes are going,
make sure they detect this and exit.
- back-end programs: remove hardwired assumptions about
what directory they run in, and hence where config.xml is.
E.g., daemons look for it in "..", others expect it in current dir.
New approach: all the programs look for the project dir as follows:
1) the environment var BOINC_PROJECT_DIR, if defined
2) the current dir, if config.xml is there.
3) else ".."
This means you can run programs in either proj/bin/ or proj/,
or (using BOINC_PROJECT_DIR) you can keep executables
outside of the project dir.
svn path=/trunk/boinc/; revision=18042
set the job params to reasonable values (see below),
and make it easy to change these values in the script
- create_work (function and script): change default job params:
FLOPs est: 1e9 => 3600e9
FLOPs bound: 1e10 => 86400e9
mem bound 100MB => 500MB,
disk bound 100MB => 1GB
delay bound: 100000s => 1 week
svn path=/trunk/boinc/; revision=17703
which compiles the translation files into the form
used by the web code.
Projects that don't use "upgrade" will need to do this manually
whenever translation files change.
svn path=/trunk/boinc/; revision=17652
which of those files to include
- Modified MAC address check to work on some non-Linux unixes.
(mac_address.cpp)
- Added suggested change to "already attached to project" checking.
(ProjectInfoPage.cpp)
- changed includes of standard c header files to their c++ equivalents
(i.e. replaced <stdio.h> with <cstdio>) for namespace protection.
- replaced "using namespace std;" with more explicit "using std::function" in
several files.
- Fixed bug in checking whether the os is OS/2 and added conditional OS_OS2
to the build environment. (boinc_platform.m4,configure.ac)
- Changed build environment to not use -nostandardlibs unless we are using
G++ and static linkage is specified. (configure.ac)
- Added makefiles and package building files for solaris CSW package manager.
- Fixed bug with attempting to find login name using logname. (configure.ac)
- Added ifdef HAVE_* protection around some include files commonly found in
sys.
- Added support for unified binary for x86_64/i686-pc-solaris.
(cs_platforms.cpp)
- generate_host_cpid() now uses MAC address on non-linux unix.
(hostinfo_network.cpp)
- Macro BOINC_SET_COMPILE_FLAGS now doesn't check gcc only flags on non-gcc
compilers. (boinc_set_compile_flags.m4)
- Library compiles no longer depend upon the library extension or require
the library to be prefixed with lib.
- More fixes for fcgi builds.
- Added declaration of "struct ether_addr" and ether_ntoa(). Have not yet
implemented ether_ntoa() for machines that don't have it, or where it is
buggy. (unix_util.h)
- Added FCGI::perror() which calls FCGI_perror(). (boinc_fcgi.{h,cpp})
- Fixed library Makefiles so that all required headers get installed.
svn path=/trunk/boinc/; revision=17388
- Update to libtool 1.5.24
- build environment: Major automake changes that I've been warning about
for some time.
- Now uses libtool to build libraries.
- Builds separate boinc_fcgi and sched_fcgi libraries for use with
FCGI server components.
- New macro "BOINC_CHECK_LIB_WITH" that executes a "AC_CHECK_LIB" on
a library only if --with-libname[=DIR] is specified on the configure
command line. This is to allow inclusion of libraries when the
ssl, gtk, wxWidgets, or other configuration is incorrect for static
libraries.
- Added a lot of "--with-*" for some libraries that might be required for
static builds.
- The sea directory has been moved to packages/generic. Changes to sea
and the associated scripts might be required to better make use of the
staging mechanism and shared libraries.
- Fixed includes of boinc_fcgi.h in many files.
- Fixed places where FCGI_FILE needs to be used implicitly.
- Fixed missing define of _SC_PAGESIZE on hosts that define only
_SC_PAGE_SIZE.
- Moved build of boinc_cmd (and source file) from lib to client
svn path=/trunk/boinc/; revision=16904
(after user confirmation).
This is called from "upgrade", and can also be run by itself.
NOTE: this mechanism will handle all DB updates going forward.
Older updates must be done the old way (edit and run db_update.php)
- Web: let teams determine whether they're accepting new members
svn path=/trunk/boinc/; revision=16160
we're the main program (otherwise we didn't lock it in
the first place, and a crash results). From Artyom Sharov.
- scheduler: add support for the GCL simulator,
which uses special versions of backend programs
that use virtual time,
and that wait for signals instead of sleep()ing.
To compile:
make clean
configure CXXFLAGS="-DGCL_SIMULATOR"
make
svn path=/trunk/boinc/; revision=16038
a modified boinc version.
- Added new header "boinc_fcgi.h" to be used instead of "fcgi_stdio.h".
This header defines I/O functions in the namespace FCGI rather than using
redefined functions the way "fcgi_stdio.h" does. This was causing a lot
of headaches when both <cstdio> and "fcgi_stdio.h" was called. Using
overloaded functions fixes this problem, except when the only difference
between functions is the return type (for example ::fopen() returns FILE*
and FCGI::fopen() returns FCGI_FILE*).
- Fixed some missing "#ifdef _WIN32" blocks in filesys.C
svn path=/trunk/boinc/; revision=15984
cause new projects to report an error when they first
attempt to start up.
- tools: Add the missing x86_64-apple-darwin platform type
used for 64-bit Mac OS X applications. I used the
description from the alpha project.
- MGR: delete obselete make files.
clientgui/
Makefile.linux.fedora
Makefile.linux.suse
Makefile.linux.ubuntu
html/user/
sample_index.php
tools/
project.xml
svn path=/trunk/boinc/; revision=15906
- scheduler: fix bug in adaptive replication:
if send an unreplicated job to untrusted host,
set both wu.target_nresults and wu.min_quorum to app.target_nresults.
svn path=/trunk/boinc/; revision=15762
client version. David -- this can probably be done
better and might indicate that some other info is not
being properly copied. Please review and fix if needed.
Before this fix, scheduler was complaining about
outdated core clients version 0 < min needed core 413.
tools for scheduler logging: change cgi.log.X to scheduler.log.X
in script that creates on-line logs for users to read
to help diagnose problems.
svn path=/trunk/boinc/; revision=15751
wish to use it.
- The script calculate_credit_multiplier (expected to be run daily as
a config.xml task) looks at the ratio of granted credit to CPU time
for recent results for each app. Multiplier is calculated to cause
median hosts granted credit per cpu second to equal to equal that
expected from its benchmarks. This is 30-day exponentially averaged
with the previous value of the multplier and stored in the table
credit_multplier.
- When a result is received the server adjusts claimed credit by the
value the multiplier had when the result was sent.
svn path=/trunk/boinc/; revision=15661
1) the assimilator creates a job_summary_WUID file saying
- what host did the job
- error status if any
- info about missing output files
2) all output files are now optional.
3) the assimilator deletes temp files (templates, job dir file)
svn path=/trunk/boinc/; revision=15551
mostly so that the parse function could assume
that everything was initially zero.
However, various back-end functions pass around SCHED_CONFIG&
as an argument (also named "config").
This creates a shadow, which is always bad.
Worse is the possibility that some projects have back-end programs
that have a SCHED_CONFIG variable that's automatic,
and therefore isn't zero initially,
and therefore isn't parsing correctly.
To fix this, I changed the 2 vectors in SCHED_CONFIG into pointers,
and have the parse routine zero the structure.
I was tempted to remove the SCHED_CONFIG& args to back-end functions,
but this would have broken some projects' code.
I did, however, change the name from config to config_loc
to avoid shadowing.
Also fixed various other compiler warnings.
svn path=/trunk/boinc/; revision=15541
an automatic var SCHED_CONFIG (hence nonzero)
- make_project: include xadd in list of instructions
- boinc_submit: fix bugs in --stdin
svn path=/trunk/boinc/; revision=15536
was being cast to a pointer to 64-bit time field
(this caused wrong results on 64-bit servers)
From Sharov Artyom.
svn path=/trunk/boinc/; revision=15517
app executable from boinc_samples/example_app/uc2,
rather than boinc/apps/upper_case.
(boinc_samples must be installed in the same dir as boinc).
The reason: the BOINC autoconf-based build process
produces executables that have zillions of shared library dependencies,
and generally won't run on machines other than
the one where they were built.
Executables built in boinc_samples/ don't have this problem.
svn path=/trunk/boinc/; revision=15509
These are more than 24 lines, so in the VMWare BOINC VM
they scroll off the window and are lost.
Instead, write them to a file (project/project.readme).
Also, only write the message about example app if -test_app
svn path=/trunk/boinc/; revision=15358
(calculate_credit_multiplier) to determine what factor to multiply
claimed credit by before insertion into the database. Changes to scheduler
to implement have not yet been checked in.
svn path=/trunk/boinc/; revision=15309
and change the correspending structure field from 64KB to 256KB
(could increase this if needed).
This is needed to handle app versions with lots (> 100) of files
- change LARGE_BLOB_SIZE to BLOB_SIZE a bunch of places
- Change COPROCS from vector<COPROC> to vector<COPROC*>.
Otherwise the right virtual functions of COPROCs don't get called
svn path=/trunk/boinc/; revision=14986
David Anderson