I may have mentioned a couple of times before that I enjoy using the built-in Terminal utility. A few years ago, I often found it useful for troubleshooting permissions problems, for example, and as TMO’s Jeff Butts points out, there are tons of under-the-hood changes you can make by knowing how to use certain commands. Or heck, by copying and pasting them in from the Internet. The Terminal doesn’t care if you know what you’re doing, which is why it’s always a great idea to be careful what you type in it! That’s my PSA for today, folks.
Anyway, one command that’s simple to use (at least at its most basic level) yet handy is grep. As Terminal’s manual page on it explains:
What this means in more simple terms is that you can use grep to pull lines that contain search terms out of a text file. Here’s how it works: Let’s pretend this text file of mine has many hundreds of lines of data that I need to paw through.
If what I need to do is take all of the lines containing “test” and separate them from the lines containing everything else, that’s really simple for grep. What I’ll do first is open Terminal, of course—it lives in my Applications > Utilities folder—and after it gives me a prompt to start (ending with a dollar sign), I’ll type this:
So I’ve entered the grep command followed by my search term—“test”—and now I just have to tell Terminal which file to run things on. An easy way I can do this is by making sure to type a space after my search term, and then I’ll drag and drop the file I want to search onto the Terminal window, like so:
Once I do that, the program’ll fill in the path to the file for me, easy-peasy.
If I then press Return on my keyboard, the Terminal window will fill up with the lines that match my search! Neat!
When you get to this point with your own grepping, which is probably a word I just made up, you can copy and paste the resulting data out of Terminal. Or if you’re more fancy, you could use a right-angle bracket (“>”) to “pipe” the info out into a new file by adding on to your command, like this:
You have to know a bit about how to use file paths to really understand what you’re doing here, but in plain English, my command above says “find lines matching ‘test’ in the ‘Grep.txt’ document on Melissa’s desktop, then put the resulting data into a file called ‘testfile.txt,’ also on her desktop.” Be careful, though, that you aren’t pointing Terminal to an existing file! If you already have a “testfile.txt” on your desktop, this command will overwrite it.
Finally, one more detail—if your search term contains any special characters (such as spaces, say), you’ll have to put a backslash in front of them to make Terminal interpret them correctly. So for example, if you wanted to find lines containing “scary sewer clown,” you’d type this:
grep scary\ sewer\ clown /Path/To/File
The grep command is actually incredibly powerful, and some of its functionality is admittedly way over my head. Still, even peons like me can use it to do simple line-searches! If you’re braver than I and would like to delve into it more deeply, just open Terminal and type man grep at the prompt to read more of its manual page. And let me know if you use it for anything fun!
Yes, Terminal can be fun. YES IT CAN.
Wow, Scott!! That is an amazing amount of info! Thank you so much for taking the time to put that together; it’s very useful. 🙂
Grep came from the original Unix editor ed where it describes the command:
g/re/p
Globally fine the regular expression “re” and print.
The key to using grep is the “re” or regular expressions. Let me give you four regular expression examples that should add tons of functionality to the command:
Search for any character is the “.” (period). For example, if your regular expression is “b.b” then it will give every line containing a “b” followed by any character then followed by another “b”.
Repeat the last items zero or more times. If you change the regular expression to “b.b” then it will find lines with “bb”, “bob”, “boob”, “bib”, and even “bop bop”. Remember, space is any character! However, if you want there to be at least a single character between the b’s so that it does not match the “bb” then the regular expression would be “b..b”.
Please note that there are other ways of doing this and something called extended regular expressions. If you want to use extended regular expressions you have to add -E to the command line. Then you can use the “+” character which means one or more of the previous match. In our example, the extended regular expression would be “b.+b”.
If you want to find something at the beginning of a line, you can anchor it by adding the ^ (caret) to the front of the regular expression. For example, to find all of the lines beginning with a capital T, the regular expression would be “^T”.
Just like you can anchor a regular expression to the beginning of the line, you can anchor it to the end of the line. Find all lines ending with a lower-case s, the regular expression would be “s$”. However, if you want to find all lines ending in a period, you would write the regular expression as “.$”. The backslash means to escape the next character or alter its meaning. In this case, it changes the interpretation of the period from meaning any character to just being a period.
In the basic regular expression, escaping a character turns it into a literal interpretation. For example, to find all lines with a dollar sign you would have to write the regular expression as “\$” so that the regular expression parser does not interpret the dollar sign to mean the end of the line. However, in extended regular expressions, there are other uses for the backslash. This is too much to go into now but I bring it up to make you aware.
Of course, you can combine regular expressions to come up with some interesting searches.
How about counting just the number of blank lines in a file:
–> grep -c “^$” somefile
The -c option to grep is to just produce a count.
But wait, the lines may be blank but there could be spaces. Ok, then you can enter a regular expression like:
–> grep -c “^ *$” somefile
(there’s a space after the caret)
How about finding the number of non-blank lines:
–> grep -c -v “^ *$” somefile
The -v option inverts the search. Instead of counting the files that match the regular expression, count the files that do not match the regular expression.
In the original “ed” editor, the command to do this was to change the g to a v. Thus “v/re/p” would find the lines that do not match the regular expression.
I better stop because I can do this all night! I love to use regular expressions to transform text into something more useful. It is behind a lot of scripts I use to scrape html pages and reformat them for my use!
Thanks for the comment, vpndev! Very helpful information you’ve added, as always!
Great tip, Melissa.
To expand slightly on your caution about spaces in the search term (you mentioned scary\ sewer\ clown), this applies to anything in the command line. Because a space is a delimiter for purposes of Terminal commands.
So you need the backslash also in the file name parameter if your file name contains a space. So your example /Path/To/File is fine but /Path/To/My Input File is not. You can solve this either with backslashes, i.e. /Path/To/My\ Input\ File, or by wrapping in double-quotes “/Path/To/My Input File”.
Thanks also for mentioning the trick of dragging the file icon into the Terminal window. Very cool. Even more cool is that this automagically takes care to escape any spaces in the name.