Log rotation

The log files generated by BOINC's daemons and servers can grow to gigabyte size in a few days or weeks, and will eventually fill up your disk. When this happens your project will grind to a halt. In addition, if you use db_dump to export statistics, directories with names of the form html/stats_2006_4_3_15_50_2 will build up, and you'll need to delete and possibly archive them. Most projects do this manually.

Initially you can deal with log files by hand, but eventually you may want an automated solution. Log rotation typically must be done while the project is stopped.

Log Compression

Here's a shell script (from Nicolas Alvarez) that tars and compresses log files (you'll still need to eventually deal with the compressed files). You can run this script as a periodic task in config.xml.

cd ..
./bin/stop
pushd ./log_servername
BACKUP_DIR=$(date --utc +backup_%Y_%m_%d)
mkdir $BACKUP_DIR
mv *.log *.out $BACKUP_DIR
( cd .. && ./bin/start& )
tar cjvf $BACKUP_DIR.tar.bz2 $BACKUP_DIR
rm -rf $BACKUP_DIR
popd

`logrotate`

Other projects use the Linux 'logrotate' program. WCG uses the following logrotate file, which they run while the project is online:

/our/log/directory/log_server1/*.log {
    compress
    rotate 10
    daily
    copytruncate
    olddir /our/log/directory/log_server1/archive
}

Predictor@home uses the following logrotate input file (they run this while the project is down for database backups):

/export/projects/predictor/log_predictor1/cgi.log{
    compress
    dateext
    maxage 365
    rotate 99
    size=+1096k
    notifempty
    missingok
    create 664 wwwrun users
    postrotate
        /etc/init.d/apache2 reload
    endscript
}

/export/projects/predictor/log_predictor1/file_upload_handler.log{
    compress
    dateext
    maxage 365
    rotate 99
    size=+1096k
    notifempty
    missingok
    create 664 wwwrun users
    postrotate
        /etc/init.d/apache2 reload
    endscript
}

/export/projects/predictor/log_predictor1/transitioner.log{
    compress
    dateext
    maxage 365
    rotate 99
    size=+1096k
    notifempty
    missingok
    create 644 boinc users
}

/export/projects/predictor/log_predictor1/taskmaster.log{
    compress
    dateext
    maxage 365
    rotate 99
    size=+1096k
    notifempty
    missingok
    create 644 boinc users
}

/export/projects/predictor/log_predictor1/assimilator_placeholder.log{
    compress
    dateext
    maxage 365
    rotate 99
    size=+1096k
    notifempty
    missingok
    create 644 boinc users
}

/export/projects/predictor/log_predictor1/feeder.log{
    compress
    dateext
    maxage 365
    rotate 99
    size=+1096k
    notifempty
    missingok
    create 644 boinc users
}
/export/projects/predictor/log_predictor1/sample_bitwise_validator.log{
    compress
    dateext
    maxage 365
    rotate 99
    size=+1096k
    notifempty
    missingok
    create 644 boinc users
}

/export/projects/predictor/log_predictor1/dataCollectorMFold.log{
    compress
    dateext
    maxage 365
    rotate 99
    size=+1096k
    notifempty
    missingok
    create 644 boinc users
}

Log rotation

Log Compression

logrotate

See Also

`logrotate`