Linux - Some useful applications of sed and grep

1) sed - how to find and replace within a file

This is one of the basic application of sed

sed -i 's/find-pattern/replace-pattern/g' filename


2) sed - How to search between two patterns:

sed -n '/start-pattern/,/end-pattern/p' filename

This is useful when you want to limit your search to a region within a file

e.g Between two time points

sed -n '/01:44:39/,/02:41:44/p' abc.log


3) sed - how to prefix/suffic something to the start/end of every line

You know this well if you are exposed to regular expression (makes way for some most powerful text manipulations).

^ - Denotes beginning

$ - Denotes ending

e.g Comment out some lines in a config file

sed -i 's/^/#/g' filename

e.g Add some comments to every line at the end

sed -i 's/$/<your text>/g' filename

4) grep multiple strings

grep 'warning\|error\|critical' /var/log/messages

5) grep and display few lines above the matched pattern

grep -A  2 'warning\|error\|critical' /var/log/messages

6) grep and display few lines below the matched pattern

grep -B  2 'warning\|error\|critical' /var/log/messages

tar creation with option C and effect of globbing

There was a need to archive some logs for investigation...

Without much thought, I fire this command and there it was..bummer!!


[root@mysys rmlater]# tar -zcvf /var/tmp/test.tgz -C /root/myapp/log/ 
appProc.log.*
tar: appProc.log.*: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors

[root@mysys rmlater]# ll /root/myapp/log/appProc.log.*
-rw-r--r-- 1 root root 338928 Feb 22 10:47 /root/myapp/log/appProc.log.0
-rw-r--r-- 1 root root 460000 Feb 20 22:40 /root/myapp/log/appProc.log.1
-rw-r--r-- 1 root root 460000 Feb 18 21:39 /root/myapp/log/appProc.log.2
-rw-r--r-- 1 root root 462685 Feb 16 20:38 /root/myapp/log/appProc.log.3
-rw-r--r-- 1 root root 907643 Feb 14 20:29 /root/myapp/log/appProc.log.4

[root@mysys rmlater]# pwd
/var/tmp/rmlater

Solution 1:
tar -zcvf /var/tmp/test.tgz /root/myapp/log/appProc.log.*

Solution 2:
cd /root/myapp/log/; tar -zcvf /var/tmp/test.tgz appProc.log.*

Difference between Solution 1 and Solution 2:
1) The files are archived in package structure opt/cpf/oma/esymacstarter/. You will see the 

appProc.log.* under opt/cpf/oma/esymacstarter/ when you extract the test.tgz

2) The files are archived into test.tgz without any package structure

Now looking at the error:
tar: appProc.log.*: Cannot stat: No such file or directory

tar -C option => Instructs tar to change the working directory and perform archiving

Reason why it failed: Due to the wildcard pattern and how it is interpreted by shell
Shell expands the files into matching files as per the wild card pattern. This operation is 
called "globbing"...

When shell does globbing, it does so from the current working directory - It checks for 
files matching "appProc.log.*" in /var/tmp/rmlater

[root@mysys rmlater]# ll
total 0
-rw-r----- 1 root root 0 Feb 22 11:25 opt


No files matching appProc.log.* => Cannot stat: No such file or directory

Why use -C option then:

It comes handy when when you want to go to multiple directories and archive different files from them. Most importantly, when you want to put them into one tgz file!!

One use case I can think of is collecting logs for different SW modules from more than one 
folder. In case you want to explore tar options in detail, check out this excellent resource GNU tar manual

Linux: cpu information processor vs core vs socket

When you run /proc/cpuinfo on linux system, you get the cpu information of the hardware.

I try to explain how to interpret the cpu infomation in order to understand the number of physical processors and whether it is single or dual or quad core and whether hyperthreading is enabled.


processor:    Defines the number of processors on the system (logical)
physical id:   Number of CPU Sockets (physical) - Also referred as Chip - This is what is plugged onto mother board.
core id:        Identifies a particular logical core
cpu cores:    Number of cores (1-Single/2-Dual/4-Quad/8-Octa)
siblings:        Indicates whether hyper threading enabled. If "Number of cores" == siblings, then hyper threading is OFF ; else ON

For e.g look at the below sample:


[root@myhome ~]# cat /proc/cpuinfo | /bin/egrep 'processor|physical |core|sibl' | tr -d "\n" | sed s/processor/\\nprocessor/g | grep -v ^$ | sort -u
processor       : 0physical id  : 1siblings     : 8core id              : 0cpu cores    : 4
processor       : 1physical id  : 1siblings     : 8core id              : 2cpu cores    : 4
processor       : 2physical id  : 1siblings     : 8core id              : 1cpu cores    : 4
processor       : 3physical id  : 1siblings     : 8core id              : 3cpu cores    : 4
processor       : 4physical id  : 1siblings     : 8core id              : 0cpu cores    : 4
processor       : 5physical id  : 1siblings     : 8core id              : 2cpu cores    : 4
processor       : 6physical id  : 1siblings     : 8core id              : 1cpu cores    : 4
processor       : 7physical id  : 1siblings     : 8core id              : 3cpu cores    : 4

Physical id  => 1 => Single CPU Socket/Chip

Cpu cores   => 4 => Quad Core Processor

This system has 1 Physical CPU Socket - Quad Core Processor configured with hyper threading ON...