Linux – script to count the occurence of the particular string in the given time interval

bashcountlinuxscriptingsh

We are trying to write a script "sendemail.sh" to count the number of occurrence of a particular string in a log file "SendEmail.log" within the given interval.
We have a log file. In that we are searching for a pattern "ReqInputMsgLog" and need to count the number of times it occurred in the given period for eg: from "2014-08-19 11:30" to "2014-08-19 11:34". And our script look like this:

#!/bin/sh
enterdate=$1
echo $enterdate
enddate=$2
enterdate1=`date +%s -d $enterdate +"%Y-%m-%d %H:%M"`

echo $enterdate1
enddate1=`date +%s -d $enddate +"%Y-%m-%d %H:%M"`
echo $enddate
count=0
cat SendEmail.log | grep "ReqInputMsgLog" | awk -F "[" '{print $3}' | awk -F "," '{print $1}' > /con/scripts_server/file.txt
for line in `cat /con/scripts_server/file.txt`
do
logdate=`echo $line | awk -F : '{print $1":"$2}'`
if [[ $logdate < $enddate1 ]];
        then
        count=`expr $count + 1`
        fi
done
echo $count

But when we are trying to execute the script by the below command its not showing the proper count.

./sendemail.sh "2014-08-19 11:30" "2014-08-19 11:34"

Log file is very big one. Small chunk has been posted here.

INFO [SIBJMSRAThreadPool : 5] [2014-08-19 11:18:24,471] SendEmail - 8/19/14 11:18 AM,ECCF25B0-0147-4000-E000-1B830A3C05A9,ReqInputMsgLog,SendEmail,<?xml version="1.0" encoding="UTF-8"?>
<in:sendEmailRequestMsg xmlns:in="http://EmailMed/EmailMedInterface" xmlns:ns0="wsdl.http://EmailMed/EmailMedInterface" xmlns:ns1="http://EmailMed/EmailMedInterface" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:me="wsdl.http://EmailMed/EmailMedInterface" xsi:type="me:sendEmailRequestMsg">
<in:sendEmail xmlns:xci0="http://EmailMed/EmailMedInterface">
INFO [SIBJMSRAThreadPool : 7] [2014-08-19 11:18:14,235] SendEmail - 8/19/14 11:18 AM,ECCEFDB2-0147-4000-E000-1B830A3C05A9,ReqInputMsgLog,SendEmail,<?xml version="1.0" encoding="UTF-8"?>
<in:sendEmailRequestMsg xmlns:in="http://EmailMed/EmailMedInterface" xmlns:ns0="wsdl.http://EmailMed/EmailMedInterface" xmlns:ns1="http://EmailMed/EmailMedInterface" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:me="wsdl.http://EmailMed/EmailMedInterface" xsi:type="me:sendEmailRequestMsg">
<in:sendEmail xmlns:xci0="http://EmailMed/EmailMedInterface">
INFO [SIBJMSRAThreadPool : 7] [2014-08-19 11:18:14,241] SendEmail - xmlText: <?xml version="1.0" encoding="UTF-8"?>

after awk command we will get a file "/con/scripts_server/file.txt" which looks similar like below:

2014-08-19 11:28:03
2014-08-19 11:28:06
2014-08-19 11:28:17
2014-08-19 11:28:53
2014-08-19 11:29:02
2014-08-19 11:29:47
2014-08-19 11:29:57
2014-08-19 11:30:07
2014-08-19 11:30:17
2014-08-19 11:30:19
2014-08-19 11:30:19
2014-08-19 11:30:22
2014-08-19 11:30:25
2014-08-19 11:30:25
2014-08-19 11:30:36
2014-08-19 11:30:51
2014-08-19 11:30:56
2014-08-19 11:30:59
2014-08-19 11:30:59
2014-08-19 11:31:08
2014-08-19 11:31:25
2014-08-19 11:32:19
2014-08-19 11:32:22
2014-08-19 11:32:27
2014-08-19 11:32:28
2014-08-19 11:32:41
2014-08-19 11:32:49
2014-08-19 11:32:59
2014-08-19 11:33:27
2014-08-19 11:33:41
2014-08-19 11:34:07
2014-08-19 11:34:14
2014-08-19 11:34:21
2014-08-19 11:34:25
2014-08-19 11:34:38
2014-08-19 11:34:50
2014-08-19 11:34:58

Best Answer

Use the following to calculate the lines between two time variables. Put the following code in a file called countOcurrences.

#!/bin/bash

 awk "/$1/,/$2/"'{count++} END{ printf "There are %s lines\n",  count}' con/scripts_server/file.txt

Run it as follow.

./countOcurrences "2014-08-19 11:30:07" "2014-08-19 11:34:07"

If file.txt is filled with a new date/time each time a pattern match occurs, then the above will work.

Related Topic