Rooba
/
boinc
mirror of https://github.com/BOINC/boinc.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

When creating large numbers of WU which have large input files, 

the cost of computing the md5 sums becomes expensive.  Inserted
a simple caching mechanism into the utility function
process_wu_template().  For a given filename FILENAME, this looks
for a file called FILENAME.md5, which contains the md5sum and
length.  If the file exists, these values are used.  If not, such
a file is created.  Since (I think) David is planning to have a
database for files, this mechanism will eventually be obsolete.
Right now it disabled by default.  Use -DBOINC_CACHE_MD5 to
enable it.

svn path=/trunk/boinc/; revision=4965
This commit is contained in:
Bruce Allen 2004-12-30 04:16:00 +00:00
parent 097c237fda
commit e009537167
2 changed files with 52 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
checkin_notes
View File

@ -21995,3 +21995,19 @@ David 29 Dec 2004
api/
x_opengl.C
Bruce 30 Dec 2004
- When creating large numbers of WU which have large input files,
the cost of computing the md5 sums becomes expensive. Inserted
a simple caching mechanism into the utility function
process_wu_template(). For a given filename FILENAME, this looks
for a file called FILENAME.md5, which contains the md5sum and
length. If the file exists, these values are used. If not, such
a file is created. Since (I think) David is planning to have a
database for files, this mechanism will eventually be obsolete.
Right now it disabled by default. Use -DBOINC_CACHE_MD5 to
enable it.
tools/
backend_lib.C

40
tools/backend_lib.C
View File

@ -109,11 +109,43 @@ static int process_wu_template(
boinc_copy(top_download_path,path);
}
retval = md5_file(path, md5, nbytes);
if (retval) {
fprintf(stderr, "process_wu_template: md5_file %d\n", retval);
return retval;
retval=0;
#ifdef BOINC_CACHE_MD5
// see checkin-notes Dec 30 2004
{
// look for file named FILENAME.md5 containing
// md5sum and length
FILE *fp;
char md5name[512];
sprintf(md5name, "%s.md5", path);
if ((fp=fopen(md5name, "r"))) {
// found cached file info!
if (2==fscanf(fp, "%s %lf", md5, &nbytes))
retval=1; // indicates sucess getting cached info
fclose(fp);
}
}
#endif
if (!retval) {
retval = md5_file(path, md5, nbytes);
if (retval) {
fprintf(stderr, "process_wu_template: md5_file %d\n", retval);
return retval;
}
#ifdef BOINC_CACHE_MD5
else {
// Write file FILENAME.md5 containing md5sum and length
FILE *fp;
char md5name[512];
sprintf(md5name, "%s.md5", path);
if ((fp=fopen(md5name, "w"))) {
fprintf(fp,"%s %.15e\n", md5, nbytes);
fclose(fp);
}
}
#endif
} // (!retval)
dir_hier_url(
infiles[file_number], config.download_url,
config.uldl_dir_fanout, url