The validator handler can now pass unknown arguments to the project specific handler.
Projects that have their own validator need to implement the validate_handler_init() function and handle project specific arguments there. They also need to supply a validate_handler_usage() function that printf()'s a description of the custom options. For examples see sample_substr_validator.cpp or script_validator.cpp
The validator test harness was also adopted to use these new functions.
This brings the implementation of the validator framework on the same level as the assimilator framework where similar changes where made in 0038d27 and dd00440.
The xp.element_contents() functions strips the linebreak before the closing tag. But the closing tag needs to be on it's own line because of the limitations of BOINC XML parsing.
When in standalone mode (test framework) the name is actually the path supplied by the user. This will extract the filename from the given path and fill in the correct values of the OUTPUT_FILE_INFO object.
This is important so that the test framework is consistent with the actual behaviour and projects don't need to put in extra code to distinguish between test mode and normal mode.
A validator now has the possibility to mark a single result as "suspicious" by making init_result() return VAL_RESULT_SUSPICIOUS. If this is the single quorum result of an adaptive replication, this will trigger another task to be generated for validation.
The validator handler can now pass unknown arguments to the project specific handler.
Projects that have there own validator need to implement the validate_handler_init() function and handle project specific arguments there. They also need to supply a validate_handler_usage() function that printf()'s a description of the custom options. For examples see sample_substr_validator.cpp or script_validator.cpp
The validator test harness was also adopted to use this new functions.
This brings the implementation of the validator framework on the same level as the assimilator framework where similar changes where made in 0038d275c and dd004404a.
The HR (homogeneous redundancy) code was way out of date.
There were no classes, coarse or fine, for Android or ARM.
I added these.
I also changed/fixed the "coarse" classification to be (OS type + CPU type)
where OS type is Win/Mac/Linux/FreeBSD/Android,
and CPU type is Intel-compatible/PPC/ARM.
Previously all Linux hosts were considered equivalent.
Note: the fine classification of CPUs is based on models c. 2008.
This should be updated based on a study of FP discrepancies.
Add optional <mem_usage_base> and <mem_usage_per_cpu> elements
for MT plan classes.
These let you make the estimated memory usage depend on the # of CPUs used,
for example, for VM apps that run multiple jobs within the VM.
The mem usage is base + NCPUS*per_cpu.
This is used for:
1) to limit the # of CPUs, based on client's available memory
2) to determine workunit.rsc_memory_bound
(the value in the input template is overridden)
Notes:
1) We still need to read all the data from socket (copy_socket_to_null());
otherwise the client will block on the send, and never get the success reply.
The previous approach (read-only file) didn't do this.
All we're saving is disk I/O on server.
2) The client reports a result only after it know that
all its output files have successfully been uploaded.
It won't re-upload anything if that's the case.
Except for very specific cases, strncpy() should never be used.
It can result in a non-terminated string.
Also replace strncat() with strlcat(); the latter is simpler
because you don't have to calculate remaining buffer space.
This broke other things (e.g. get_file_size on that file).
We would accomplish the same thing a cleaner way,
i.e. notice the file is already there and of the right size.
The file upload handler checked for ".." in the filename.
Also check for control chars and for starting with /.
Put this into a separate function, is_valid_filename().
Otherwise, result file names can be inferred from result names.
An attacker with task A could find the name of the "wingman" task B,
upload fake files as B's output files,
upload the same files as A's output files,
report A as completed, and get unearned credit.
An issue with unicode strings in python 2.4 and 2.6 (and possibly 2.5) prevents shlex to split the command which leads to the daemon or task not starting. The unicode issue seems to be fixed in python 2.7. The exact error message is: "TypeError: execv() argument 1 must be (encoded string without NULL bytes), not str".
See: https://github.com/vinodc/gitlab-webhook-branch-deployer/issues/1
This opens the validator up to a result name spoofing attack where a bogus client can claim it processed the result reported by a different client for the same workunit.
If the printf() or close() calls change errno, the original lseek() error is lost. The logged error would differ from the message send to the client. This amends 005957a.
Suggested by Juha Sointusalo
- if a daemon or task should run in a shell, add <use_shell>1</use_shell> to the task entry in config.xml
this will spawn a "sh -c cmd" process that propagates signals to the child process (see 881863d)
- if a daemon or task has to use a shell (pipe or redirection present in cmd) and <use_shell> is not enabled:
don't execute the cmd and print an error message (other daemons and tasks are still started)
The for loop copies newly created objects into the vector and destroys the original objects. The resize() instantiates the objects directly in the vector. Suggested by Nicolás Alvarez.
David Anderson