Some Unix date operations

input_date_format="%m/%d/%Y"
target_date_format="%m%d%Y"


Inbdate=`date +"$input_date_format" --date="12/11/2011"`
Incdate=`date +"$input_date_format" --date="10/31/2011"`


bdate=`date +"$target_date_format" -d "$Inbdate"`
cdate=`date +"$target_date_format" -d "$Incdate"`
cdate1=`date +"$target_date_format" -d "$Incdate - 1 day"`

-----------------------------------------------------------------------
Some usage of if with regex
--------------------------------

if [[ "$app_id" = Nitesh* ]]
then
app="NiteshApp"
elif [[ "$app_id" = Kumar* ]]
then
app="Kumarapp"
fi

----------------------------------------------------------------------------
Exception handling
------------------------

a.)
sqlcommand=`echo "insert ignore into impressed_$value ( select * from impressed_""$value""_""$app_id"" )"`
echo "executing : $sqlcommand"
mysql -u$root -p$password -h$db_server -P$db_port -D$db_name -e"$sqlcommand"
if [ $? -ne 0 ]
then
echo "Exiting failed : $sqlcommand"
exit 2;
fi

b.) Debgging
#!/bin/sh -ex

--------------------------------------------------------------------------------
use of '\' to ignore new line
-------------------------------------

sqlcommand=`echo "insert into adx_adenginelog_tmp \
( select \
reqType, sessionID, timestamp, appID, phoneNum, numResults \
from log_$ldate where appid='$app_id' and timestamp >= '$ltimestamp' and timestamp <= '$ltimestamp_eod' );"`
echo "Executing: $sqlcommand"
mysql -u"$db_username" -p"$db_password" -h${db_server} -P${db_port} -D${db_name} -e"$sqlcommand"

Some common usable Unix commands

Simple linux command

Find out the PID of the program on a given port

sudo lsof -i :8080 | grep LISTEN

1. grep -R "search string" "file"
2. find "filename" -type d -iname "string" ---- i is to ignore case

3. ###### to find all newlines and replace it with comma, use AWK ##########
Nice awk tutorial : http://www.savs.hcc.edu.tw/~chuavv/awk/nawk_23.html

http://bogdan.org.ua/2010/11/16/how-to-replace-newlines-with-commas-tabs-etc-merge-lines.html


awk '{print $1}' result.txt | tr '\n' ',' > sq.sql

awk uses file field to be separated by space.

if a different fs is required then
awk 'BEGIN { FS = "," } ; { print $2 }' sq.sql

4. SED ### this is to replace a particular regex with another regex
suppose a file has one line as
a,b,c,d

but i want then line by line like
a
b
c
d

command : sed 's/,/\n/g' file.txt > new.txt

so nw.txt now has
a
b
c
d

SOME EXAMPLES:
Most of the characters used in SED do not need backslashing like the below example
sed 's/-h${db_server}/-h${db_server} -P${db_port}/g' init.sh

4. TR ### this command also replace a separator with another separator
Like in the previous example new.txt has
a
b
c
d

we want out as a,b,c,d
then use awk to parse it line by line and then tr to replace

awk '{print $1}' new.txt | tr '\n' ',' > new1.txt

so new1.txt has
a,b,c,d

5. CUT ###########
a.) http://en.kioskea.net/faq/1451-sed-delete-one-or-more-lines-from-a-file
b. ) http://linux.die.net/man/1/cut

Some Common Unix editor errors

Problem:

1. The script test.sh was zipped copied from one linux server to another and then unzipped
2. The script test.sh wwas given right permissions
3. Scripts were read and modified using vim editor on both servers
4. Both the servers ( to and from ) have the same version of linux and red-hat
>>dmesg
dmesg command gives system details and showed that linux versions were same

While test.sh execution , below error occured

Error
/bin/sh^M: bad interpreter: No such file or directory

Cause of Error: Still not sure how ^M char could be found.
However using dos2unix test.sh solved the problem
for all the scripts use >> dos2unix *.sh ( but 'dos2unix *' does not work ?? weird !! )
b.) another solution is open using vi and then :set fileformat=unix also solved the problem

Links:
http://pbxinaflash.com/forum/showthread.php?t=3738

http://www.gizmola.com/blog/archives/87-Linux-shell-scripting-bad-interpreter-No-such-file-or-directory.html

http://www.edaboard.com/thread219178.html

some more array operations

Some help links
arithmetic operations :
http://bash.cyberciti.biz/guide/Perform_arithmetic_operations

for arrays:
http://www.arachnoid.com/linux/shell_programming.html
http://tldp.org/LDP/abs/html/arrays.html

master_slaves.txt is a file having lines separrated by ':' that needs to converted to array and then processed

>>cat master_slaves.txt
master_config:root:venableroot:3306:ml1
slave_config:ml2:ml3

master= array of values from line 1
slave = array of values from line 2


---------------------------------- script -----------------------------------
!/bin/bash -ex
p_r="test"
filename="$p_r/master_slave.txt"
master=()
slaves=()

while read line
do
table=$line

splits=(`echo "$table" | tr ':' ' '`)

if [ "${splits[0]}" = "master_config" ]
then
master=(${splits[*]})
elif [ "${splits[0]}" = "slave_config" ]
then
slave=(${splits[*]})
else
continue
fi

if [ "$master" ] && [ "$slave" ]
then

lenm="${#master[*]}"
echo "lenm=$lenm"
lens="${#slave[*]}"
echo "lens=$lens"

## append master to slave
slave[$lens]=${master[4]}
lens=$(( lens + 1 ))

i=0;
while [ $i -lt $lenm ]
do


echo "master:${master[$i]}"
i=$(( i + 1 ))
done

i=0;
while [ $i -lt $lens ]
do

echo "master:${slave[$i]}"
i=$(( i + 1 ))
echo "i=$i"

done

else
echo "master and slave config improper. check $filename"
fi

done < ${filename}
echo "over"

split a string using 'tr' and convert into array

Below script reads a file line by line
Each line is a string delimited by underscore ( '_' ).Script splits each string by this delimiter and extracts the last index,which is the date part of the string
and produces a new string

if the lines of status.txt are

impression_1_10142011
impression_1_10152011

then output of the below script is

impression_2_10142011
impression_2_1015201
-----------------------------------------------------------
#!/bin/sh

filename="status.txt"

while read line
do
table=$line
splits=(`echo "$table" | tr '_' ' '`)
tabletmp=`echo "impression_2_""${splits[2]}"`
echo "table tmp=$tabletmp"
done < ${filename}

echo "over"

-------------------------------------------------------------

Other ways of splitting a string by delimiters
a.
>> echo "nitesh:kumar:jag:gita" | awk -F: '{print $3}'
>> jag

b.
>> echo "nitesh:kumar:jag:gita" | cut -d: -f4
>> gita

c.
>> echo "nitesh" | awk '{print substr($0,2)}'
>> itesh

d.
>> echo "nitesh" | cut -c2
>> i

e.
>> echo "nitesh" | cut -c2-
>> itesh

f.
>> echo "nitesh" | cut -c2,4
>> ie

g.
>> echo "nitesh" | cut -c2-4
>> ite

h.
>> df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 10321208 2687988 7108932 28% /
/dev/sdb 423135208 203084 401438084 1% /mnt
none 8963788 0 8963788 0% /dev/shm
/dev/sdg 1032123136 953849496 25844840 98% /data

then
>> df | grep "\ /data" | awk '{print $5}'
>> 98%

i. TO get process id of a process 'crond'
>> ps -ef | grep "crond" | grep -v grep | awk '{print $2}'
>> 988