ngrep -- searching network packets like Unix grep

ngrep is grep turned in a sniffer and can be used for watching network traffic. It is based with the libpcap library, which provides packet-capturing functionality. ngrep allows regular expression style filters to be used to select traffic to be displayed.

ngrep was developed and is maintained by Jordan Ritter, and the project’s home page is http://ngrep.sourceforge.net.

ngrep strives to provide most of GNU grep's common features, applying them to the network layer. ngrep is a pcap-aware tool that will allow you to specify extended regular or hexadecimal expressions to match against data payloads of packets. It currently recognizes IPv4/6, TCP, UDP, ICMPv4/6, IGMP and Raw across Ethernet, PPP, SLIP, FDDI, Token Ring and null interfaces, and understands BPF filter logic in the same fashion as more common packet sniffing tools, such as tcpdump and snoop.

Searching network traffic for patterns in real time is a useful technique for debugging a variety of network problems.  Here are some examples of how ngrep can be used:

Example: Processing PCAP dump files, looking for patterns

I had a friend who worked at Network Solutions and among the things he did was analyze huge 500M+ PCAP dump files of DNS traffic, looking for patterns and anomalies. ngrep was an invaluable tool for this purpose; it allowed him to take one instance of a network dump and search it quickly and repeatedly for patterns in the data packets.

To save a PCAP dump file from ngrep is very easy; simply run ngrep as you normally would but add one more command line option: ``-O some.file.dump'' (the name of the file is largely irrelevant). To illustrate another feature of ngrep, we will use the ``-T'' option (print time differential information).

ngrep has the following general syntax:

ngrep [options] pattern [filter]

where pattern is a regular expression to search for in network packets, and the optional filter is an expression indicating the sorts of network packets to which to pay attention. This filter, technically known as a Berkeley packet filter (BPF), consists of a series of keywords specifying rules for selecting packets (BPF filters are also used by packet-dumping utilities). These keywords specify the source and/or destination host, network, protocol, and/or port. See the manual page for information about constructing BPF expressions. If the filter is omitted, then all packets are included.

Using a filter in combination with a search string usually makes searching more efficient and ngrep’s output more readable. As an example, consider this ngrep command that looks for packets containing output from the finger command among all network traffic:

$ ngrep "No Plan\."

The command searches for a recognizable string from the finger output. If you run this on most systems and run a finger command within earshot of that host, you’ll see the relevant packet intermixed with lots of number signs and dot series, and it will often be repeated several times. A number sign is printed for each packet examined, and dots indicate output interruptions. If you use the following command instead, then you won’t get the ugly output, and ngrep will do less work:

# ngrep -q "No Plan\." port finger

Now, only packets to or from the finger port (79) are examined. The -q option suppresses the number signs.

Here is a more complex ngrep command that locates some specific FTP connection operations. It examines FTP-related packets sent from host hamlet to host ophelia and searches their data for the string USER:

# ngrep -q -t "USER" src host hamlet and dst host ophelia and tcp port 21
T 2002/09/24 14:11:15.413069 192.168.9.212:32813 -> 192.168.9.84:21 [AP]
  USER chavez..

T 2002/09/24 14:32:07.776476 192.168.9.212:32814 -> 192.168.9.84:21 [AP]
  USER amber..

For each packet, the output begins with a letter indicating the protocol (TCP here), followed by a time stamp (requested with the -t option), and then the source and destination host and port. The second line displays the packet’s data. A command like this is one way of capturing all FTP connection attempts. As this example indicates, complex filters can be created by joining clauses with “and”. You can also use the “or” and “not” logical operators as well as parentheses for grouping (the latter will need to be escaped to protect them from the shell).

ngrep can be very useful when testing and debugging network services. For example, I found it very useful when I enabled the secure versions of LDAP. Everything seemed to work fine after I was finished, but I wanted to verify that the secure version of the protocol was being used. If it was, then I would not be able to detect any clear text passwords in LDAP traffic. An ngrep command like this one enables me to test this functionality:

# ngrep 'badpassword' src host ldapserver and \( port 636 or port 389 \)

This command watches the ports corresponding to SSL and TLS-secured LDAP. While this command was running, I ran three queries: one using the normal ldap protocol, one using the ldaps protocol, and a third using a GUI LDAP client in TLS mode. All three queries succeeded and displayed the appropriate records. The ngrep command returned output only for the first one, so it was clear that the password had been encrypted in the latter two cases. ngrep was a better choice than a general packet dumper for this job because it limited its scope to exactly the packets in which I was interested.

This example illustrates that ngrep can be useful for not finding things as well as finding them. In this case, the lack of output was what I hoped for. Those of you who had qualms about the finger example earlier can apply this principle as well: because running a finger daemon is not a good idea in most environments, such a ngrep command can function as a security trap. If finger traffic appears on the network, ngrep will detect it and let you know there is a problem.

ngrep has several other useful options:

-i — Perform a case-insensitive search.
-A n — Display the n packets following each matched packet.
-d dev — Use the specified network device.
-O file — Save matching packets in file in addition to displaying them.
-X — Interpret the search pattern as hexadecimal.

ngrep is useful for a wide variety of tasks ranging from testing network applications to monitoring network traffic. It is also quite useful for debugging specific operations or programs on busy systems because of its ability to extract very narrow ranges of packets for examination.

Getting and Installing ngrep

Source code for ngrep is available from http://ngrep.sourceforge.net/, as are some binary packages. I'll only review installing the source code because the binary packages are fairly straightforward.

On a Red Hat 6.2 system, you'll need to install libpcap before you can install ngrep. This package is available from http://www.tcpdump.org/release. As of this writing, the most recent version is libpcap-0.5.2.tar.gz. After download, I put things like this into /usr/local/src; you should do something like:

$ tar xvzf libpcap-0.5.2.tar.gz
$ cd libpcap_0_5rel2
$ ./configure
$ make
$ su
Password:********
# make install-incl
# make install-man
# exit

Your next step is to build ngrep itself. ngrep source code can be downloaded from http://www.packetfactory.net/Projects/ngrep. After downloading it, follow the steps

$ tar xzvf ngrep-1.38.tar.gz
$ cd ngrep
$ ./configure
$ make
$ su
Password:********
# make install
# exit

Congratulations! At this point, you should have a working copy of ngrep installed on your system.

To start using ngrep, you'll need to decide what pattern you want to search for. These can be either libpcap-style descriptions of network traffic or GNU grep-style regular expressions describing the contents of traffic. In the following example, you'll grab any packet containing the pattern ssword and display it in the alternative format (which I think is a lot more readable). The results

[root@cherry /root]# ngrep -x ssword
interface: eth0 (192.168.1.0/255.255.255.0)
match: ssword
################################
T 192.168.1.20:23 -> 192.168.1.10:1056 [AP]
 50 61 73 73 77 6f 72 64  3a 20             Password:
#########################exit
59 received, 0 dropped
[root@cherry /root]#

Each hash mark in this example represents a packet not containing the pattern you're searching for; any packets containing the pattern are displayed. In this example, you followed the basic syntax of ngrep: ngrep <options>[pattern]. You used only the -x option, which sets the alternative display format.

A number of additional twists are available for the way that you can use ngrep. Chief among them is the capability to include libpcap-style packet filtering. libpcap provides a fairly simple language for filtering traffic.

Filters are written by combining primitives with conjunctions (and and or). Primitives can be preceded with the term not.

Primitives are normally formed with an ID (which can be numeric or a symbolic name preceded by one or more qualifiers). There are three kinds of qualifiers: type, direction, and protocol.

• Type qualifiers describe what the ID refers to. Allowed options are host, net, and port. If no type is given, the primitive defaults to host. Examples of type primitives are host crashtestdummy, net 192.168.2, or port 80.
• Directional qualifiers indicate which direction traffic is flowing in. Allowable qualifiers are src and dst. Examples of direction primitives are src cherry, dst mango, and src or dst port http. This last example shows two qualifiers being used with a single ID.
• Protocol qualifiers limit the captured packets to those of a single protocol. In the absence of a protocol qualifier, all IP packets are captured (subject to other filtering rules). Protocols that can be filtered are TCP, UDP, and ICMP.You might use a protocol qualifier such as icmp or tcp dst port telnet.

Primitives can be negated and combined to develop more complex filters. If you want to see all traffic to rose except Telnet and FTP data, you could use the filter host dst rose and not port telnet and not port ftp-data.

Some command-line switches are worth noting as well.

Command-Line Switches for ngrep

using ngrep can help you quickly match and display packets during your troubleshooting. If you've got an application level problem, ngrep can help you isolate the problem.

For example, if I was trying to make a connection from cherry (192.168.1.10) to cuke (192.168.2.10) and the connection was failing, I might troubleshoot the problem like this:

Describe the symptoms—Cherry cannot make a connection to hosts on remote network, but it can connect to hosts on other networks. Other hosts on cherry's network can connect to hosts on the remote network.

Understand the environment—The hosts involved are cherry, rhubarb (the gateway to the remote network), and cuke.

List hypotheses—My problems might be a misconfiguration of cherry or of an intervening router.

Prioritize hypothesis and narrow focus—Because cuke seems to be the only host affected, start looking there. If you can't solve the problem on cuke, move to rhubarb.

Create a plan of attack—I can try to ping cuke from cherry while using ngrep to see what traffic I am sending, like this: ngrep host cherry.

Act on the plan—As you start pinging cuke, you can see the results of the ngrep session:

[root@cherry /root]# ngrep -e -x host 192.168.1.10
interface: eth0 (192.168.1.0/255.255.255.0)
filter: ip and ( host 192.168.1.10 )
#
I 192.168.1.10 -> 192.168.2.10 8:0
 eb 07 00 00 31 86 a7 39  5e cd 0e 00 08 09 0a 0b ....1..9 ^.......
 0c 0d 0e 0f 10 11 12 13  14 15 16 17 18 19 1a 1b ................
 1c 1d 1e 1f 20 21 22 23  24 25 26 27 28 29 2a 2b .... !"#$%&'()*+
 2c 2d 2e 2f 30 31 32 33  34 35 36 37       ,-./01234567
#
I 192.168.1.1 -> 192.168.1.10 5:1
 c0 a8 01 0b 45 00 00 54  25 f2 00 00 40 01 d0 52 ....E..T%[email protected]
 c0 a8 01 0a c0 a8 02 0a  08 00 dc 67 eb 07 00 00 ...........g....
 31 86 a7 39 5e cd 0e 00  08 09 0a 0b 0c 0d 0e 0f 1..9^...........
 10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f ................
 20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f  !"#$%&'()*+,-./
 30 31 32 33 34 35 36 37  b4 04 01 00 06 00 00 00 01234567........
 00 10 00 00 01 00 00 00  e8 40 00 00       .........@..
exit
2 received, 0 dropped
[root@cherry /root]#

This shows two packets. The first is an ICMP packet of Type 8 and Code 0, a ping request. It is destined for cuke. The second is an ICMP packet of Type 5 and Code 1 and ICMP Redirect.This is coming from mango, the gateway to the rest of the world.

Test the results—You shouldn't expect to see mango involved at all. If you look at the ICMP Redirects being sent (using the -v switch), you can see that you're being redirected to the 192.168.1.11 address, not rhubarb.

Apply results of the test to the hypothesis—If you're not sending your traffic to the right gateway, it will never get to the right place.You should be able to solve this by adding a route to the 192.168.2.0/24 network on cherry (a quick check of working hosts shows that this is the way they're configured).You'll probably want to fix the bad route on mango as well.

Iterate as needed—When you've made the change and tested it, you know that it works and don't need to go any further.

Recommended Links

Softpanorama Recommended

Top articles

Sites

ngrep - network grep

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019