Shell Script in a Nutshell

How to check what port a DB2 instance is listening to

2009-02-04T09:10:00.000-08:00

Hi,

right now we faced a problem that we didn't know what port a db2 was listening to and to solve the issue we issued the following commands in a Solaris box (should also work in AIX or RedHat):

//check what port DB2 should be using:
$ db2 get dbm cfg | grep SVCENAME

//if it's not numeric, check the number in /etc/services:
$ grep db2c_db2svc01 /etc/services

//and if you want to confirm that DB2 is indeed listening to that port, issue:
$ netstat -a | grep DB2

//Or in Windows client, check if you can get a tcp connection
c:\>telnet server_ip server_port

cheers,

Borys Marcelo

How to retrieve lines before and after in a AIX / Solaris box when -B and -A options are not available

2009-02-04T04:22:00.000-08:00

Hi,

when you find yourself trying to retrieve the lines before and after of an occurrence of an exception or keyword in a log file and you discover that in AIX or Solaris box you don't have such a options in grep command, the below command will help you in that issue:

nawk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=a}b{r[NR%b]=$0}' b=10 a=10 s="NullPointerException" server.log

where:

b means Before and is the same as the -B option in gnu grep
a means After and is the same as the -A option in gnu grep
s means the keyword or string you are looking for

cheers,

Borys Marcelo

Getting rid of ^M character while printing out a report

2008-01-11T12:40:00.000-08:00

Hi, recently I got in turns with a problem in a CSV report. The report was printed out correctly but because of a ^M a new line had been added and the report was generated with extra lines. A mishmash. But fortunately, I've found a solution:

In the variable that holds your value to be printed out in a report, say CSV, you need to use awk to remove the ^M (carrot M)

DETAILS=`echo $DETAILS | awk -v RS="^M" -v ORS="" '{print}'`

but BE SURE you make

^M USING "CTRL-V CTRL-M" NOT BY TYPING "CARROT M"! because if you don't, ^M will not be replaced at all as you are typing it as literal not as a metacharacter.

* CRTL+V=^
  CRTL+M=M

^M= carriage return


the above worked for me, please post here if you have any questions. Thanks.

Linux is not Windows

2008-01-04T13:55:00.000-08:00

Here is a good article for new Linux users that explain very well that Linux is not a Windows and what the new user have to expect of it:

http://linux.oneandoneis2.org/LNW.htm

How to recover your xorg configuration in Ubuntu / Linux?

2008-01-04T06:06:00.000-08:00

Hi,

first of all, it's a good practice to keep a backup of the xorg configuration file right after Ubuntu installation finishes. Because you will already have a running xorg configuration that you will be able to use in case of something getting wrong, say, after a 3D video card installation or something liked. Then, let's walk through the steps:

1 - If you cannot see anything when GDM starts, you need to start linux in a safe-mode or just issue CRTL+ALT+F1 to see a terminal prompt
2 - Log into you Linux as usual and go to /etc/X11/
3 - issue ls -l xorg* and you will see xorg configuration files:
$ ls -l xorg*
-rw-r--r-- 1 root root 2390 Nov 23 00:28 xorg.bak_rodando
-rw-r--r-- 1 root root 2516 Nov 19 21:11 xorg.bichado
-rw-r--r-- 1 root root 2390 Jan 4 09:25 xorg.conf
-rw-r--r-- 1 root root 2300 Nov 18 01:16 xorg.conf~
-rw-r--r-- 1 root root 2305 Nov 18 01:33 xorg.conf.backup
-rw-rw-r-- 1 root root 2719 Nov 22 23:55 xorg.conf.backup-ati-fglrx
-rw-r--r-- 1 root root 803 Nov 17 03:58 xorg.conf.original-0
-rw-r--r-- 1 root root 823 Nov 18 01:41 xorg.conf.original-1

4 - If this is the first time you are restarting your computer, than you can use the xorg.original-0
5 - then as root or with sudo command issue
sudo cp xorg.conf.original-0 xorg.conf
say y[es] if prompt for a confirmation.
6 - Now if you tried everything and yet you are not able to see anything when GDM starts, you will need to generate a brand new xorg conf file. To do so, issue as root or with sudo:

sudo

 dpkg-reconfigure xserver-xorg

follow on screen instructions and you will have your xorg conf to previous state.


please post a comment whether any question.

Borys

How to use IF statement more properly

2007-12-27T04:58:00.000-08:00

Hi,

when I first developed a sh script I ended up with a bunch of If's clauses since the only IF statement I had heard about it was the
if [ ]
then

fi

this because if you are using IF statement with square brackets, you can only test for a unique assertion. But, what if you need to do a short circuit using || and &&? If you are not able to do that you will probably end with a bunch of ifs to accommodate this deficiency. But there is a solution, you need to use the IF statement with a double parenthesis, like this:

* no extra space between (( and $a and between 10 and )) allowed
if (($a < 0 && $b < 10))
then
#statements go here
fi

How to return the number correspondent to any given Month name

2007-12-26T08:26:00.000-08:00

Hi,

today I'm facing a problem: How to return the number corresponding to Jan, Feb, Mar.... in the date String returned by
chage -l ?

For example, when I issue chage -l myusername it gives me:

Last password change : Nov 17, 2007
Password expires : Feb 15, 2008
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 90
Number of days of warning before password expires : 7

So, I made a grep command in union with a gawk command to bring me the 3 last pieces of this lines for any line containing expires in their name. So it gives me

Feb 15, 2008
never
never

so, with Feb 15, 2008 in my hands, what I could do with that? Nothing, because I cannot use Feb to compare with another date in order to get number of days for the next user account expiration. Given this scenario, we have to options:

1 - Create a function with a whole bunch of ifs in order to switch between Name / Number like:

if [ $1 == "Jan" ]
then
return 1
and so...

2 - use the built in Linux date command:

/bin/date --d="Feb 15, 2008" +%m

the above gives you the number 11 as result. Then you can issue /bin/date +%m and get current month and do the needful to calculate days before account expiration or password expiration.

Please, post here if you have any questions, thanks.

How to increase the screen resolution available on Ubuntu

2007-12-24T17:17:00.000-08:00

Hi,

recently I removed my Ubutun installation to install Mandriva One distro. It was full of bug, so I decided to give a shot to OpenSuse distro. It was terrible as well, nothing worked out-of-the-box and even the sound was not working due to some Alsa packages not being installed. (alsa-firmware, alsa-tools and alsa-utils by the way). So, I decided to go back to Ubuntu and reinstalled it again. But I forgot to set the screen resolution to 1024 - 768 (the max resolution my laptop can afford) and I ended up with a awful resolution of 800-600. Hopefully, I found a blog with some instructions on how to increase the screen resolution to 1024-768 again and to a bigger resolution if you want. The site is:

Site

How to evaluate a variable via concatenation or concatenating 2 strings?

2007-12-21T11:07:00.001-08:00

Sometimes we want to evaluate a value of a variable A where the name of variable A is a concatenation of B + C. I'll give you an example:

MY_BRANDS=(VOLKS FORD FIAT)
MY_VEHICLES(TRUCK CAR MOTO)

MAX_VALUE_FOR_A_VOLKS_CAR=100
MIN_VALUE_FOR_A_VOLKS_CAR=10

MAX_VALUE_FOR_A_VOLKS_TRUCK=10000
MIN_VALUE_FOR_A_VOLKS_TRUCK=1000

MAX_VALUE_FOR_A_VOLKS_MOTO=10
MIN_VALUE_FOR_A_VOLKS_MOTO=1

No imagine that you have a function that gives to you the MAX or MIN value for any given BRAND and VEHICLE. Without concatenation you will end up with a plenty of If elif or switch with a plenty of Cases. But we can do it via concatenation and evaluation:

#1 first of all create a tmp var

VAR_TMP_AUX=""

#now we need to evaluate the concatenation and assign it to VAR_TMP_AUX

#How to pass parameters
#$1 = MAX or MIN
#$2 = BRAND
#$3 = VEHICLE
giveMeTheMaxOrMinValueForAVehicleOfAnyGivenBrand()
{
PREFIX_VAR_AUX=$1
BRAND_VAR_AUX=$2
VEHICLE_VAR_AUX=$3

COMMON_PART="_VALUE_FOR_A_"

eval "VAR_TMP_AUX=\${$PREFIX_VAR_AUX$COMMON_PART$BRAND_VAR_AUX"_"$VEHICLE_VAR_AUX}"
# ^
# |
# this is necessary (the \$) because the eval command forces the evaluation of $ not as it was literal but as operator
return $VAR_TMP_AUX
}

that's it!

Serching patterns and displaying columns using awk

2007-12-12T08:38:00.000-08:00

Imagine a file products.txt with the following structure:

ID TYPE NAME WEIGHT COUNTRY DESCRIPTION PRICE OVERALL
0 SPORT SHOES SHOES 1 USA Nike Shoes 100 25
1 SOUVENIR PENCIL 0.2 BRAZIL CROWN PENCIL 10 78
2 SOUVENIR WATCH 0.8 CHINA Nike Watch 99.99 30

then imagine that you need to generate a report based on that file and the report should only display souvenirs and the columns TYPE, NAME COUNTRY and PRICE. What you are going to do? Create a java / dot.net / c++ / c / c# program? Create a grep command? no... What we need to do here is to use awk to retrieve what columns we really want to:

cat products.txt | awk '/SOUVENIR/ {$2, $3, $5, $7}'

here you go! A simple awk program that does much! If you want to know more, please visit the below link for a complete awk tutorial:

http://www.vectorsite.net/tsawk.html

Watching streaming videos in Firefox provided via mms protocol

2007-11-22T11:26:00.000-08:00

For the Linux addicts, I've found a site with some steps to get Firefox properly configured in order to watch streaming videos across web sites that provides videos via mms protocol. To do so follow the steps below:

As *ROOT*:

1 - # yum -y install mplayer
2 - # yum -y install mplayerplug-in
3 - # yum -y remove totem-mozplugin
4 - # yum -y install gconf-editor (only if you haven't installed yet)
5 - # gconf-editor
5.1 - Drill down into: desktop >> gnome >> url-handlers >> mms
5.1.1 - Change the key: totem "%s" to: mplayer "%s"
5.2 - Close the gconf-editor application
6 - Open up the Firefox explorer and in the url put the following:
6.1 - about:config
6.1.1 - Right click and go to New>>String
Set name as: network.protocol-handler.app.mms
Set value as: /usr/bin/X11/mplayer

6.1.2 - Right click and go to New>>Boolean
Set name as: network.protocol-handler.external.mms
Set value as: True
7 - Restart Firefox and try to watch videos in your favorite site or you can find an mms video here --> www.globo.com

8 - If the video load but won't start, follow step 9:

9 - Right click and got to CONFIGURE
9.1: Set Output Video as: X11
9.2: Set Output Audio as : Alsa
9.3: Set Minimum Cache Size as: Whatever rate you want
9.4: Set Percent of Media to Cache as: Whatever rate you want
9.5: Tick all check-boxes below, right after ” Percent of Media to Cache” with only one exception --> “Play Media directly from site”, which you need to leave unchecked.
10 - Refresh the page and try again.

The above works for me. I'm using Open Client 2.1 RHE 5

Please, let me know if you have questions. Thanks.

chmod Tutorial

2007-11-20T05:53:00.000-08:00

Who has ever been stuck giving file access permission before? Well, not anymore! I've just found a link with a useful and easy tutorial about chmod:

http://catcode.com/teachmod/index.html

The site even gives you the ability to execute chmod via web!

How to block a port using iptables command line

2007-11-08T09:49:00.000-08:00



IPTABLES -A INPUT -p tcp --dport 25 -j REJECT

IPTABLES -A INPUT -p tcp --dport 25 -j ACCEPT

where you see 25, one can change to whatever port we want to keep blocked.

more information here:

http://iptables-tutorial.frozentux.net/iptables-tutorial.html#INTRODUCTION

How to use Gzip and Gunzip commands for compressing/uncompressing a file

2007-10-26T14:41:00.000-07:00

Sometimes we need to uncompress/compress .gz files in AIX and we don't know what command to use or we don't know what options to use. So, here are some examples of how using tar, gzip and gunzip:

Let's start with gzip:

Usage - It's most used to compress files

Example:

gzip file.txt --> compress file.txt, removing the original file and creating file.txt.gz

gzip -c file.txt > file.txt.gz --> compress file.txt, keeping the original file, redirecting file compression to stdout and the > file.txt.gz redirects the stdout to file.txt.gz, which creates the file

gzip -c9 file.txt > file.txt.gz --> same of the above, but option 9 makes a better compression

gzip -c1 file.txt > file.txt.gz --> same of the above but option 1 makes a faster compression

gzip -cv[91] file.txt > file.txt.gz --> same of the above but option v shows the underlying process verbosity.

GUNZIP:

gunzip file.txt.gz --> uncompress file.txt.gz, removing it, and creating the file file.txt

gunzip -c file.txt.gz > file.txt --> uncompress file.txt.gz, keeping the original file, redirecting file uncompression to stdout and the > file.txt redirects the stdout to file.txt, which creates the file

How to create a function in a shell script

2007-10-26T14:33:00.000-07:00

If you want to create a function in your sh script, the first thing you need to do is to declare your function like this:

Code:

myFunction()
{

echo "This is a function"

}

Now, if you want to call the function you just created, you need to do this:

Quote:

myFunction()
{

echo "This is a function"

}

#calling myFunction
myFunction

Now, if you want to call your function and to receive a parameter send as argument on its body, you need to do this:

Code:

#first of all, a parameter get retrieved by using the $ character and the parameter number. i.e. MyVar=$1 for the first parameter, and so.
myFunction()
{

echo "This is a function"
echo "This is the first parameter sent $1--> "$1
echo "This is the second parameter sent $2--> "$2
MyVar=$1" <--- value of the first parameter | "$2" Value of the second parameter"
echo "You get the point! "$MyVar

}

#calling myFunction, passing 2 parameters as arguments
param1="This is the first parameter"
param2="This is the second parameter"
myFunction "$param1" "$param2"

Get to know Regular Expressions

2007-10-25T04:48:00.001-07:00

Do you like Regular Expressions? If not, well I think you have never had a chance to know it very well. The following site has all about regular expression. From the beginning to advanced. It's a very good site to start using regex all over your code!

http://www.regular-expressions.info/

Sed Tutorial

2007-10-24T14:19:00.001-07:00

I found a very useful tutorial where we can learn about sed command and it's good to keep it in your bookmarks : )

http://www.grymoire.com/Unix/Sed.html

String replacement

2007-10-24T13:59:00.000-07:00

Here is what you need to know about string replacement:

if you want to replace a character from a string, just do this:

MY_STR="12345"

MY_STR=${MY_STR//[1-3]*/' '}

echo $MY_STR # result --> 45

So, for replacing a key in a string, you need to use the ${string//replace_this/for_this}
replace_this could be a regex like our first example.

Here is an url that a whole bunch of REGEX that you can put in your replace_this regex, have fun:

http://www.baremetalsoft.com/baregrep/regex.php

Sending emails in an AIX, UNIX, Linux command line

2007-10-24T12:09:00.000-07:00

Below are two examples of how we can send an email using shell command in a Unix like OS:

Example 1 - Sending a simple note:

mail -s "Here goes the subject" email_to_1@your_company.com, email_to_2@your_company.com < your_email_body_text_stored_in_a_tmp_file.txt

options:

-c send_copy_to_first_email@your_company.com, send_copy_to_second_email@your_company.com, and so

-b send_blind_carbon_copy_to_first_email@your_company.com, and so

Example 2 Sending attachments:

mutt -s "Here goes the subject" -a my_attachment.any_extension email_to_1@your_company.com, email_to_2@your_company.com < your_email_body_text_stored_in_a_tmp_file.txt

or, you can use the command: nail with the same options as in mutt.

Please, let me know if you have questions

How to retrieve / return the whole / full line of a grep result set using for ... in

2007-10-24T11:28:00.000-07:00

I don't know if you guys have already faced an issue like this... But, recently I had to write a shell script where I had to iterate through a result set retrieved from a grep command, and while it was iterating through out the keys, I noticed that the collection of keys returned by grep had been split up by blank spaces and it wasn't what I was looking for. I wanted to retrieve each line by iteration. So, as I was not familiar with AWK, I decided to implement a function for doing the hard work.

Basically, the function receives two parameters: the 1st is the Key and 2nd is the delimiter.

Here is the function and how to call it inside a sh script. Enjoy it!

#External Vars
P_VEZ_AUX=0
STR_READY_TO_USE=""
MAIN_KEY_AUX=""

getTheWholeLineInAGrepCollection()
{

KEY_AUX_2="$1"
KEY_DELIMITER=$2

if [ "$KEY_AUX_2" == $KEY_DELIMITER ]
then

if [ "$P_VEZ_AUX" -eq 0 ]
then
let P_VEZ_AUX=1
MAIN_KEY_AUX=$MAIN_KEY_AUX$KEY_AUX_2
return 0
else

STR_READY_TO_USE=$MAIN_KEY_AUX
MAIN_KEY_AUX=$KEY_AUX_2
return 1
fi

else

MAIN_KEY_AUX=$MAIN_KEY_AUX" "$KEY_AUX_2

return 0

fi

}

#Then, to calling the function

# -A 1 --> extended results 1 line After matched result
# -0 --> returns only what is inside the regex

KEYS=`egrep -A 1 "NullPointerException" SystemOut.log | egrep -o "at"\ [A-Za-z.-_\(]`

#but... before we begin, as the number of words may vary, we most first test the number of #matching against our delimiter, which in this case is 'at'

TAM_KEYS=`echo "$KEYS" | grep -c 'at'`

#now we most check if TAM_KEYS is zero

if [ "$TAM_KEYS" -eq 0 ]
then
echo "No matching records"
else

#now, if our string begins with the delimiter, you probably will face problems in case a grep statement returning only a single line
#To avoid that kind of error, we most concatenate our delimiter at the end of our KEYS collection of words in order to have our STR_READY_TO_USE properly set and to have a return of 1
#or we can simply not enter in the for loop as we already have the whole line in the KEYS var

if [ "$TAM_KEYS" -eq 1 ]
then

#adding the delimiter as the end of our KEYS collection as we have a single line and as our
#delimiter comes at the front of each line
KEYS=$KEYS" at" #don't forget about the extra space at the front of delimiter

fi

for KEY in KEYS
do

#In case a line being:
#at br.com.minhaempresa.actions.MinhaAction
# thus, keys will be sent 3 times and each iteration will occurs three times as well:
#"at", " ", "br.com.minhaempresa.actions.MinhaAction"

#However, if we want the whole line, isolated keys won't help at all.
#Finally, we use then our super function for doing the hard work:
#$KEY is the key returned on each iteration and "at" is our delimiter:

getTheWholeLineInAGrepCollection $KEY "at"

#retrieving return value

RET=$?

# and the we need to verify if RET equals 1, which indicates our String is ready to use

if [ "$RET" -eq 1 ]
then

# var --> STR_READY_TO_USE , is being set by our function after completing String concatenation

echo "full line by a given delimiter ---> " $STR_READY_TO_USE

fi

done

fi #end of if for TAM_KEYS == 0

key-words, tags tag : shell script bash sh for while grep egrep AIX Linux Unix loop iteration variable var

Please, let me know if you have questions!

Retornando uma linha por vez ao receber um resultado do grep, utilizando for ... in

2007-10-24T09:46:00.000-07:00

Eu não sei se alguém já passou por esse tipo de probleminha... Mas recentemente tive que escrever um shell script onde o resultado de um grep era recebido em uma variável e atráves do FOR eu varria os resultados retornados pelo grep command. Só que o problema era que os resultados eram retornados separados por " " spaços. E a cada iteração do for, uma única palavra era retornado, o que impossibilitava que eu pegasse essa palavra e utilizaçe como key para outro grep.

Como não tinha conhecimento de AWK, que depois fiquei sabendo que faz exatamente o que eu queria, acabei fazendo uma função no próprio shell script que busca por um delimitador da coloção de resultados que o grep retorna e te avisa quando a linha que vc quer, dado o delimitador, estiver pronta. Eis o código: (espero que ajude alguém com o mesmo problema!)

#Variáveis externas
P_VEZ_AUX=0
STR_READY_TO_USE=""
MAIN_KEY_AUX=""

getLinhaCompletaDeUmGrep()
{

KEY_AUX_2="$1"
KEY_DELIMETER=$2

if [ "$KEY_AUX_2" == $KEY_DELIMETER ]
then

if [ "$P_VEZ_AUX" -eq 0 ]
then
let P_VEZ_AUX=1
MAIN_KEY_AUX=$MAIN_KEY_AUX$KEY_AUX_2
return 0
else

STR_READY_TO_USE=$MAIN_KEY_AUX
MAIN_KEY_AUX=$KEY_AUX_2
return 1
fi

else

MAIN_KEY_AUX=$MAIN_KEY_AUX" "$KEY_AUX_2

return 0

fi

}

#daí para chamar a função

# -A 1 --> resultado estendido a 1 linha After (após) o resultado encontrado
# -0 --> traz only (somente) o que estiver especificado no regex

KEYS=`egrep -A 1 "NullPointerException" SystemOut.log | egrep -o "at"\ [A-Za-z.-_\(]`

#mas... antes de mais nada, como a quantidade de palavras na coleção é variável,
#devemos primeiro testar a quantidade de matches para o delimitador primeiro, no nosso caso é o 'at'.

TAM_KEYS=`echo "$KEYS" | grep -c 'at'`

#agora checamos se o TAM_KEYS é 0 (zero)

if [ "$TAM_KEYS" -eq 0 ]
then
echo "Nenhum match encontrado"
else

#agora se o seu delimitador fica no começo como o do exemplo
#você enfrentará problemas caso existam somente uma linha retornada pelo grep.
#Para isso devemos concatenar o nosso delimitador no final da coleção keys
#assim a nossa função vai setar a variável STR_READY_TO_USE e retornar 1
#ou simplemente nao entre no for e já utilize o KEYS como sua linha, o que realmente ela é

if [ "$TAM_KEYS" -eq 1 ]
then

#adicionando o delimitador no final quando nosso delimitador vir no começo
# e quando só existirem uma ocorrência do delimitador na coleção retornada pelo grep
KEYS=$KEYS" at" #nao se esqueça do espaço antes do delimitador

fi

for KEY in KEYS
do

#caso a linha seja:
#at br.com.minhaempresa.actions.MinhaAction
# então o key vai ser enviado três vezes, e a iteração ocorrerá três vezes também:
#"at", " ", "br.com.minhaempresa.actions.MinhaAction"

#Porém, como queremos a linha completa, esses resultados isolados não nos importa, mas sim a linha completa.

#utilizamos então nossa função para isso: $KEY é o resultado iterado e o "at" é o delimitador que queremos

getLinhaCompletaDeUmGrep $KEY "at"

#pegamos o retorno

RET=$?

# e verificamos se o retorno é 1, o que indica que nossa String está pronta para ser usada

if [ "$RET" -eq 1 ]
then

# a vraiável --> STR_READY_TO_USE , está sendo setada pela nossa função após completar a concatenação

echo "Temos uma linha completa concatenada da coleção do grep, dado um delimitador ---> " $STR_READY_TO_USE

fi

done

fi #if do TAM_KEYS -eq 0

qualquer dúvida, manda um post.