Linux and Unix - Manipulating Files

Below is some common commands that I would like to know better so figured, why not and made a post about them:

• wc -w <file> #This counts the amount of words in a file.

• wc -l <file> #This counts the number of lines in a file.

• head <file> #This reviews the first set of text at the beginning of the file. It can be piped out to more using "| more" .

• tail <file> #Same as above but the end of thr file. This is commonly used to go ahead and tail a log file and watch it as it does stuff, for this however it requires the -f switch.

• grep <option> <keyword> <file(s)> #Here is where things get a little more complciated. With Grep you can either search files for certain keywords with no options or using options such as -<num> to view num of lines before and after where it was found, or -i for searching without case sensitivity. Keep in mind you can also search many files just using wildcards.

• sed '/<textyouwanttoreplace>/<textyourreplacingitwith>/<g for all or nothing for only one occurrence>' <file> #Well thats a handful eh? Basically sed then place quotes around what you want to replace and what your replacing it with with slashes between them. If you use the G switch it changes every instance in the file, if you do not it only does it once. If you want to change line by line, use this. If you are changing csv or database files, use awk

• awk <opt> '{ <command> }' <filename> #Well this doesnt look complex does it. Well it is, using an example command of print $1 it will go ahead and issue the print command on the first line of each line. this is where things get interesting. You can string multiple statements together in the command to have it manipulate back oht in a fashion that you want. Using options you can use -f which will allow you to use the proceding character as the seperator such as -f, making it a comma as the seperator. This can get quite complicated depending on the commands entered.

• cmp <file1> <file2> #This allows static comparisons of the two files. If the two files are the same nothing will comment out otherwise as the first occurrence it will tell you the line/char difference

• diff <file1> <file2> #This will actually show the differences line by line between the two files. It will use the '<' and '>' to declare which file is which. Sdiff is better.

• sort <file> <outputfile> #Basically does what you think, grabs line by line and sorts them alphabetically. This can be piped out to 'uniq' which allows only unique lines to be stored.

• tee #This one is literally used as a pipe to redirect the output file from standard off to another application as for text. Basically using sort above, you could sort it, then pipe it to tee, and then pipe it to mail or another similar application.

Wildcards using Grep:

. = Matches any 1 character.
* = Finds 0 or more version of the preceding item. Ex: a*b would match ab, aab, aaab
^ = Matches string that only starts a line.
$ = Matches string that only ends a line.
\ = Allows special characters to be escaped, such as *,^,$
[ ] = Allows regular expressions, such as 0-9 or a-z