Dr. Mark Humphrys

School of Computing. Dublin City University.

Home      Blog      Teaching      Research      Contact

Online coding site: Ancient Brain

coders   JavaScript worlds


CA170      CA668      CA686

Online AI coding exercises

Project ideas

Shell utilities


grep                    search for a string

grep string file

(output of prog) | grep string | grep otherstring

-i    ignore case
-v    return all lines that do NOT match

More complex solution to long-line problem:
  1. Use egrep to display from 0 to max 40 chars each side of the string:
    egrep -o ".{0,40}string.{0,40}" file
    . means any char
    0 to max n of any char is .{0,n}

string matching / regular expressions

^                       start-of-line
$                       end-of-line
.                       any character

where "c" stands for the character:

c*			0 or more instances of c
cc*			1 or more instances of c
grep "  *"		1 or more spaces
.*                  any sequence of characters

where "c" has a special meaning, e.g. is $ or ., etc:

\c                  the character itself
grep "\."           the '.' character itself

recall the two forms of string:

grep '\$'	works (searches for the "$" char instead of end-of-line)
grep "\$"	fails (double quote treatment of $ is different to single quote treatment of $)
grep "\\$"	works


From xkcd.


cut     extract columns or fields of text on command-line

To extract columns  30  to end of line of the ls listing:

  ls -l | cut -c30-

In grep output, extract the 1st field, with delimiter ":"

 grep string *html | cut -f1 -d':' 

Extract the 2nd to end fields, with delimiter ":"

 grep string *html | cut -f2- -d':' 

Q. Why "-f2-" ?
Why not "-f2" ?


sed     "stream editor" - find and replace text on command-line (and other things)

sed 's|oldstring|newstring|'    change first match on each line
sed 's|oldstring|newstring|g'   change all matches 

'|' is just my choice of a separator.
Other people like '/'
We can actually use any character as a separator (whatever comes first after "s").

e.g. ls listing that highlights web pages:

 ls -l 	| sed "s|\.html| [Web page]|"

e.g. ls listing that changes how my username appears:

 ls -l 	| sed "s|$USER|ME|"

sed examples

To insert a new line

To put new lines in front of and after every HTML tag

To substitute back in the pattern we matched

# \( ... \) to mark a pattern
# \1 to reference it later

# e.g. change:
# (start of line)file.html: ...
# to:
# <a href=file.html>file.html</a>: ...

# search for:
# ^\(.*\.html\):
# change to:
# <a href=\1>\1</a>:

grep -i $1 *html |

	sed -e "s|^\(.*\.html\):| <a href=\1>\1</a>: |g"



dirname, basename

useful cutting and pasting with filenames


$ echo $HOME /users/group/me $ dirname $HOME /users/group $ basename $HOME me $ dirname `dirname $HOME` /users

head and tail

Display the first 100 lines of the output:
grep string files | head -100 
Display the last 30 lines of the logfile:
cat logfile | tail -30 


date                         looks like: "Tue Feb 17 16:28:33 GMT 2009"
CURRENTDATE=`date`           remember backquotes
echo $CURRENTDATE             

date "+%b %e"                looks like: "Jan 21"  

date "+%b.%e.log"            can add things to the string 

file=`date "+%b.%e.log"`    
echo $file             

Using date to get unique filename

Say web server in response to client needs to make a temporary file.
Use date to get a new filename that is unique to the current second:
timenow=`date +%H%M%S`
Unique to second and nanosecond:
date "+%H.%M.%S.%N"

Alternative ways of getting unique filename

About random-number generators:

A pseudo-random-number generator typically generates a series of random-looking numbers based on a start seed.
If the seed is identical, the algorithm will generate the same numbers.
So we want a different start seed each time we run.
Seed could be based on the current time (second and nanosecond).

JavaScript random number generation demo.
Usage: Freeze the seed key. Set parameters. Then generate random numbers.
Can view JavaScript code to see how it works.

ancientbrain.com      w2mind.org      humphrysfamilytree.com

On the Internet since 1987.

Wikipedia: Sometimes I link to Wikipedia. I have written something In defence of Wikipedia. It is often a useful starting point but you cannot trust it. Linking to it is like linking to a Google search. A starting point, not a destination. I automatically highlight in red all links to Wikipedia and Google search and other possibly-unreliable user-generated content.