Linux Tips And Tricks

Pin0

Keep or Remove The First Line From a File

Stephane Morin — Wed, 21 Jun 2017 00:00:03 +0000

If you need to easily remove or keep the line at the beginning of the file you should use awk , it`s a powerful tool for simple or complex text manipulation.

Here is an example to show you how to REMOVE the line 1 to 4 of a file.

cat file | awk 'NR>4'

Here is an example to show you how to KEEP the line 1 to 4 of a file.

cat file | awk 'NR<=4'

NR in awk means : Number of Rows.

Pin0

Remove Character From End of String in Linux

Stephane Morin — Tue, 20 Jun 2017 00:00:12 +0000

Remove Character From End of String in Linux

Often you need to remove one or many character at the end of a string, but you don`t know the string lenght. When that happen there is a way to extract the character that you want no matter the lenght.

For that technique to work you need to use rev and cut , rev is a program that reverse the string it received and cut is a tool that extract part of string , either by field or by character.

In this example I will show you how to extract the last 4 charactere of a string.

echo $string | rev | cut -c 1-4 | rev

and with a value of "This is a great Rock Band" in the variable string, this will output :

Band

echo $string

Send string to standard output.

rev

Reverse the string.

cut -c 1-4

Extract character one to four.

rev

Reverse the string again to put it back in the orginal order.

Pin0

Grep

Stephane Morin — Thu, 15 Jan 2015 16:58:21 +0000

Grep is one of the most usefull tool in linux or unix. With grep you can extract lines that correspond to certain criteria.

The most basic way to use grep is the following :

Syntax

grep

Let suppose a file name “name” that contain the name of people.

Here is what it contain:

Dany Moore

Steve Hawkins

Roger Hipkiss

Loy Hinton

Mandy Moore

Candy Diaz

Fernando Diaz

if you do :

1 Grep Moore name

you will get :

Dany Moore

Mandy Moore

Or you can do :

2 cat name | grep Moore

And still get the same result!

The Grep RegEx Command

Stephane Morin — Sat, 10 Jan 2015 00:53:19 +0000

If you consider the following file :

drwxrwxr-- 2 morins morins 4096 Feb 21 2014 Desktop
drwxrwxr-- 2 morins morins 4096 Feb 21 2014 Templates
drwxrwxr-- 2 morins morins 4096 Feb 21 2014 Public
drwxrwxr-- 2 morins morins 4096 Feb 21 2014 Pictures
drwxrwxr-- 2 morins morins 4096 Feb 21 2014 Music
drwxrwxr-- 2 morins morins 4096 Feb 21 2014 Downloads
drwxrwxr-- 5 morins morins 4096 Oct 7 16:40 kdenlive
drwxrwxr-- 2 morins morins 4096 Oct 7 16:42 Videos
-rw-rw-r-- 1 morins morins 337597 Oct 7 16:53 out.ogv
drwxrwxr-- 2 morins morins 4096 Oct 7 20:19 Documents
-rw-r--r-- 1 morins morins 447 Dec 8 19:30 color.txt
-rwxrwxr-- 1 morins morins 408 Dec 8 19:30 color.sh
-rwxrwxr-- 1 morins morins 624 Dec 8 19:30 color4.sh
-rwxrwxr-- 1 morins morins 623 Dec 8 19:30 color3.sh
-rwxrwxr-- 1 morins morins 1126 Dec 8 19:30 color2.sh
-rwxrwxr-- 1 morins morins 3076 Dec 8 19:30 test2
-rwxrwxr-- 1 morins morins 662 Dec 8 19:30 test
-rwxrwxr-- 1 morins morins 72 Dec 8 19:30 scripts.sh
-rwxrwxr-- 1 morins morins 68 Dec 8 19:30 quotes.sh
-rw-r--r-- 1 morins morins 88 Dec 8 19:30 name
-rwxrwxr-- 1 morins morins 134 Dec 8 19:30 format.sh
-rw-r--r-- 1 morins morins 8980 Dec 8 19:30 examples.desktop
-rw-r--r-- 1 morins morins 229 Dec 8 19:30 test.txt
-rw-r--r-- 1 morins morins 229 Dec 8 19:30 test3.txt
-rwxrwxr-- 1 morins morins 4373 Dec 8 19:30 test3
-rw-r--r-- 1 morins morins 229 Dec 8 19:30 test2.txt
-rwx------ 1 morins morins 113 Dec 8 19:55 t
-rwx------ 1 morins morins 228 Dec 30 22:57 test_zenity.sh

You do the command :

1

ls –ltr | grep d

you will get :

Ok you get all the folder , which is what you want , but you also get the file examples.desktop which you don’t want.

So instead you will do :

2

ls –ltr | grep “^d”

you get :

The (carret) ^ indicate the beginning of the file , so it means every lines that start with letter ‘d’

Now if you want let say to have all the file that end with an ‘s’.

You will do the grep regex command :

3

ls –ltr | grep “s$”

and you get :

drwxrwxr-- 2 morins morins 4096 Feb 21 2014 Templates

drwxrwxr-- 2 morins morins 4096 Feb 21 2014 Pictures

drwxrwxr-- 6 morins morins 4096 Oct 12 2015 Downloads

drwxrwxr-- 2 morins morins 4096 Oct 25 2015 Videos

drwxrwxr-x 2 morins morins 4096 Nov 1 2015 scripts

drwxrwxr-- 2 morins morins 4096 Mar 6 21:11 Documents

It means all lines that contain ‘s’ immediately folowed by the end of the line.

Now let say you want to select all lines with a folder of month feb.

You could do :

4

s –ltr | grep “^d.*Feb”

you would obtain :

That means , give me the lines that start with a ‘d’ , have any charactere many time, folow by ‘Feb’ , don’t forget the ‘F’ in caps is essential , because linux is case sensitive.

Here are some basic symbol definition :

Symbol	Definition
^	Beginning of the line
$	End of the line
.	Any charactere
*	One or many time the preceding charactere or symbol
[]	Selection of possible charactere

you can also do :

5

ls –ltr | grep “[FND]eb”

the result is :

Like that you will get any lines that contain either Feb, Neb or Deb , but as in our list only the folder correspond to that and there is no lines with either Neb or Deb we got the same lines as per the preceding example.

For example if you do :

6

ls –ltr | grep “[PV]i”

you get :

drwxrwxr-- 2 morins morins 4096 Feb 21 2014 Pictures

drwxrwxr-- 2 morins morins 4096 Oct 7 16:42 Videos

Because it’s the only lines that contain either ‘Pi’ or ‘Vi”.

The Reverse Grep

Stephane Morin — Mon, 05 Jan 2015 03:58:46 +0000

The reverse grep

Grep –v stand for inverse. With grep –v you can extract lines that correspond to the opposite of your search criteria.

Let suppose a file name “name” that contain the name of many people , here is what it contain:

Dany Moore

Steve Hawkins

Roger Hipkiss

Loy Hinton

Mandy Moore

Candy Diaz

Fernando Diaz

If you do :

cat name | grep –v moore

you will get :

Steve Hawkins

Roger Hipkiss

Loy Hinton

Candy Diaz

Fernando Diaz

Or you can do :

grep –v moore name

And still get the same result!

Pin0

Case incensitive extraction with Grep

Stephane Morin — Sat, 03 Jan 2015 20:35:30 +0000

Grep -i

Grep –i Stand for case insensitive. With grep –i you can extract lines that correspond to your search criteria no matter the cases.

Let suppose a file name “name” that contain the name of many people.

Here is what it contain:

File

Dany Moore

Steve Hawkins

Roger Hipkiss

Loy Hinton

Mandy Moore

Candy Diaz

Fernando Diaz

if you do :

you will get :

cat name | grep –i moore

File

Dany Moore

Mandy Moore

Like you can see , your searching for moore with the 'm' in lowercase and you still get the line that contain moore with the 'M' in uppercase or else like you known linux is case sensitive and if you omit the -i option it will return no line like the following :

cat name | grep moore

Or you can do :

grep –i moore name

And still get the same result!

Pin0

Remove control-m from text

Stephane Morin — Tue, 02 Sep 2014 23:39:02 +0000

Remove Control-M From Text

Multiple ways to remove control-m from text in linux.

I will try to cover most of them in this post.

The main reason to convert the file is because windows add control-m at the end of each line, which is problematic in linux because it will be interpreted as a character as the rest of the line and will eventually make some scripts failed.

The easiest method is to transfer your file from windows to unix in ASCII format. If you transfer the file in Binary or you ignore the format in which it is transfer, better verify , cause there is a lot of chance your file inherited control-m characters. When you transfer your file , it will be with an ftp or sftp software , most of them are really obvious and you have a settings for the format of the transfer, simple choose ASCII.

To see if a file contain control-m all you need to do is :

cat –v filename

vi filename

To transfer a file without control-m :

Just transfer the file in ascii mode.

To remove control-m from text :

Here is some example of how to do it :

Using ‘dos2unix’ and ‘unix2dos'

(the easiest method, if dos2unix is installed on your system)

dos2unix filename ---- To remove control-m
Unix2dos filename ---- To put back control-m at the end of the line

Using ‘tr’

tr –d ‘r’ < file_to_modify > resulting_file
tr –d ‘^M’ < file_to_modify > resulting_file
In both cases after you need to move back the resulting file into the original file

---- To do the ^M , press [ctrl]+v followed by [ctrl]+m

Using ‘vi’ --- Very easy also , if the file is not too big

vi filename --- Position yourself on the first line
:%s/^M//g --- Then press enter , and voila !