Rotating Logfiles

[RebelCork]

Hi all,

Please find below my attempt at a quick script for rotating the logfiles created by the pineapple.

I am using ssmtp to email the logfiles back to me after 13 rotations ( Feel free to customise)

I also added a cron job to run the script every 30 mins

It's very rough, please feel free to add your suggestions, I'm not a coder, this is my first script for the pineapple.

It's not the best script ever, but it is doing the job so far for me (tested on MKIV running 1.10 beta)

It basically creates a folder on the usb drive called rotatelogs

It scans the /www/pineapple folder for any log files and moves them to the rotatelogs folder on the usb drive.

The user can also decide the rotation schedule (in days) for each individual file.

rotatelogs.conf

(I stored this in /www/pineapple/rotate)

# Configuration file for the rotate log script.
# Format is     name=duration   where 'name' can be any
# filename that appears in the /www/pineapple directory
# Duration is measured in days
#
# 30 mins = 0.020833333 day (0.2)
# 60 mins = 0.04 etc


phish.log=0.02
urlsnarf.log=0.04


# Anything with a duration of zero is not rotated
# i.e phish.log=0

rotatelogs.sh

#!/bin/sh
# rotatelogs - Rolls logfiles in /www/pineapple to /www/backuplogs for archival purposes
# uses a config file to allow customisation of how frequently
# each log should be backed up to the folder
# the config file is in logfilename=duration format
# where duration is in days
# if the logfile is not in the config file, the file will be backed up weekly
mkdir /usb/rotatelogs
logdir="/usb/rotatelogs"
config="/www/pineapple/rotate/rotatelogs.conf"
mv="/bin/mv"
default_duration=0.04   count=0

duration=$default_duration

if [ ! -f $config ] ; then echo "$0: no config file found. Can't proceed." >&2; exit 1
fi

if [ ! -w $logdir -o ! -x $logdir ] ; then echo "$0: You don't have the required permissions in $logdir" >&2; exit 1
fi

cd $logdir

# This is the meat of the program


for name in $(find /www/pineapple/*.log)
do

       count=$(( $count + 1 ))

       # grabbing this entry from the config file

       duration="$(grep "^${name}=" $config|cut -d= -f2)"

       if [ -z $duration ] ; then duration=$default_duration
       elif [ "$duration" = "0" ] ; then
       echo "Duration set to zero; Skipping $name"
       continue
       fi

# I have inputed 13 log files, one per hour

back1="${name}.1";
back2="${name}.2";
back3="${name}.3";
back4="${name}.4";
back5="${name}.5";
back6="${name}.6";
back7="${name}.7";
back8="${name}.8";
back9="${name}.9";
back10="${name}.10";
back11="${name}.11";
back12="${name}.12";
back13="${name}.13";

# if the most recently rolled log file (back1) has been modified within the
# the specific timeframe, then it wont be rotated.

if [ -f "$back1" ] ; then
 if [ -z $(find "$back1" -mtime +$duration -print 2>/dev/null) ]
 then
 echo -n "$name's most recent backup is more recent than $duration "
 echo "days: Skipping";        continue
 fi
fi

echo "Rotating log $name (using a $duration day schedule)"

# Rotating logfiles here, starting with the oldest first
# After 13 steps, the logs will be emailed.
# Change user@domain.com to your ssmtp settings

if [ -f "$back1" ] ; then
 ssmtp -v user@domain.com < "$back1"
fi
if [ -f "$back12" ] ; then
 echo "... $back12 -> $back13" ; $mv -f "$back12" "$back13"
fi
if [ -f "$back11" ] ; then
 echo "... $back11 -> $back12" ; $mv -f "$back11" "$back12"
fi
if [ -f "$back10" ] ; then
 echo "... $back10 -> $back11" ; $mv -f "$back10" "$back11"
fi
if [ -f "$back9" ] ; then
 echo "... $back9 -> $back10" ; $mv -f "$back9" "$back10"
fi
if [ -f "$back8" ] ; then
 echo "... $back8 -> $back9" ; $mv -f "$back8" "$back9"
fi
if [ -f "$back7" ] ; then
 echo "... $back7 -> $back8" ; $mv -f "$back7" "$back8"
fi
if [ -f "$back6" ] ; then
 echo "... $back6 -> $back7" ; $mv -f "$back6" "$back7"
fi
if [ -f "$back5" ] ; then
 echo "... $back5 -> $back6" ; $mv -f "$back5" "$back6"
fi
if [ -f "$back4" ] ; then
 echo "... $back4 -> $back5" ; $mv -f "$back4" "$back5"
fi
if [ -f "$back3" ] ; then
 echo "... $back3 -> $back4" ; $mv -f "$back3" "$back4"
fi
if [ -f "$back2" ] ; then
 echo "... $back2 -> $back3" ; $mv -f "$back2" "$back3"
fi
if [ -f "$back1" ] ; then
 echo "... $back1 -> $back2" ; $mv -f "$back1" "$back2"
fi
if [ -f "$name" ] ; then
 echo "... $name -> $back1" ; $mv -f "$name" "$back1"
fi
touch "$name"
chmod 0700 "$name"
done

if [ $count -eq 0 ] ; then
 echo "Nothing to do: No log files big enough or old enough to rotate"
fi
exit 0

Edited March 31, 2012 by RebelCork

[Sebkinne]

Hi rebelcork,

This looks good.

I just skimmed over this but I think that a program called logrotate can do a similar thing.

Maybe check that out.

Best,

Sebkinne

[RebelCork]

I was thinking of logrotate, but I wanted to keep something simple and able to run when I choose, not just daily, weekly etc, as some of my log files will get large fairly quickly.

I may alter the script to run and email each log-file once it's saved.

Thanks for the feedback, I'm honoured !

[PineDominator]

I was thinking of logrotate, but I wanted to keep something simple and able to run when I choose, not just daily, weekly etc, as some of my log files will get large fairly quickly.

I may alter the script to run and email each log-file once it's saved.

Thanks for the feedback, I'm honoured !

nice work.

when you rotate the logs is it adding the data together on the usb or is it creating new files every time?

I wonder if using a type of database would be cleaner and cooler to look through after?

that might be something tricky and complicated just spewing thoughts:-p

[RebelCork]

nice work.

when you rotate the logs is it adding the data together on the usb or is it creating new files every time?

I wonder if using a type of database would be cleaner and cooler to look through after?

that might be something tricky and complicated just spewing thoughts:-p

It's adding a new file, it is basically moving & renaming each log to log.1, log.2 etc.

My own aim is to have the logs emailed to me every couple of hours.

I might tweak the above script with the 'logrotate' command from the opkg packages, as I see it also supports emailing logfiles.

I don't want to add everything to one file, just in case there is an interuption in power when I am writing

Agin, it's a work in progress, so, please any suggestions are more than welcome