Packet Sniffing is a colloquial term that refers to the art of network traffic analysis. Contrary to common sense, things like emails and web pages don’t traverse the internet in one piece. They are broken down into thousands of small data packets and sent across the internet in that manner. In this article, we round up the best free network analyzers and packet sniffers.
There are many, many tools out there that will collect network traffic and most of them use pcap (Unix-like systems) or libcap (Windows systems) at their core to do the actual collection. Another set of tools exists to help analyze that data because even a small amount of data can result in thousands of packets which can be hard to navigate. Almost all of these tools collect in the same way; it’s the analysis which differentiates them.
Network traffic analysis requires an understanding of how networking works. There’s no tool that will magically remove the requirement for an analyst to understand the basics of networking such as the TCP three-way handshake which is used to initiate a connection between two devices. Analysts should also have some understanding of the types of network traffic that exist on a normally functioning network such as ARP and DHCP traffic. This knowledge is essential because analyzing tools will just show you what you ask for – it’s up to you to know what to ask for. If you’re not sure how your network looks normally, it can be hard to ensure you’re digging for the right thing in the mass of packets you’ve collected.
Best Packet Sniffers and network Analyzers
Let’s start at the top and work our way down into the nitty gritty basics. If you’re dealing with an enterprise level network, you’ll need the big guns. While almost everything uses tcpdump at its core (more on that later), enterprise-level tools can address the other complicated issues such as correlating traffic from many servers, providing intelligent query tools to spot issues, alerting on exception cases, and producing nice graphs that management demands.
Enterprise level tools tend to focus on network traffic flow rather than judging packet content. By that I mean that the focus of most sysadmins in enterprise is to keep the network humming along without performance bottlenecks. When bottlenecks occur, the goal is usually to determine if the problem is the network or an application on the network. On the other side of the coin, these enterprise-level tools are usually able to see so much traffic that they can help predict when a network segment will saturate which is a critical element of capacity management.
SolarWinds is a very broad suite of IT management tools. The tool that is more relevant to this article is the Deep Packet Inspection and Analysis tool. Collecting network traffic is fairly easy. Using tools like WireShark, basic level analysis isn’t a show stopper either. But not all situations are that cut and dried. In a very busy network it may be hard to determine even some very basic things such as:
- What application on the network is creating this traffic?
- If the application is known (say, a web browser) where are people spending most of their time?
- Which connections take the longest and are bogging down the network?
Most network devices just use each packet’s metadata to ensure the packet gets where it is going. The contents of the packet are unknown to the network device. Deep Packet Inspection is different; it means that the actual contents of the packet are inspected in order to learn more about it. Critical network information that cannot be gleaned from the metadata can be discovered in this way. Tools like those provided by SolarWinds can provide more meaningful data than simply traffic flow.
Other techniques for managing high volume networks include NetFlow and sFlow. Each has their strengths and weaknesses and you can read more about NetFlow and sFlow techniques here.
Network analysis in general is an advanced topic that is half experience and half training. It’s possible to train someone to understand every detail about network packets, but unless that person also has knowledge of the target network, and some experience to identify anomalies, they won’t get very far. The tools I’ve listed in this article can be used by experienced network admins who already know what they’re looking for, but aren’t sure which tools are best. They can also be used by more junior sysadmins to gain experience with how networks look during day-to-day operation, which will help identify issues later on.
The fundamental tool of almost all network traffic collection is tcpdump. It is an open source application that comes installed on almost all Unix-like operating systems. Tcpdump is an excellent collection tool and comes complete with a very complex filtering language. It’s important to know how to filter the data at collection time in order to end up with a manageable chunk of data to analyze. Capturing all data from a network device on even a moderately busy network can create too much data to analyze easily.
In some rare cases, allowing tcpdump to output its capture directly to your screen may be enough to find what you’re looking for. For example, in writing this article, I captured some traffic and noticed that my machine was sending traffic to an IP I did not recognize. It turns out that my machine was sending data to a Google IP address of 126.96.36.199. Since I did not have any Google products running, nor Gmail open, I did not know why this was happening. I examined my system and found this:
[ ~ ]$ ps -ef | grep google user 1985 1881 0 10:16 ? 00:00:00 /opt/google/chrome/chrome --type=service
It seems that even when Chrome is not running in the foreground it remains running as a service. I would not have necessarily noticed this without a packet analysis to tip me off. I re-captured some more tcpdump data but this time told tcpdump to write the data to a file that I opened in Wireshark (more on that later). Here’s that entry:
Tcpdump is a favourite tool among sysadmins because it is a command-line tool. This means that it doesn’t require a full-blown desktop to run. It is unusual for production servers to provide a desktop because of the resources that would take, so command-line tools are preferred. As with many advanced tools, tcpdump has a very rich and arcane language that takes some time to master. A few of the very basic commands involve selecting the network interface from which to collect data, and writing that data to a file so it can be exported for analysis elsewhere. The
-w switches are used for this.
# tcpdump -i eth0 -w tcpdump_packets tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes ^C51 packets captured
This produces a capture file:
file tcpdump_packets tcpdump_packets: tcpdump capture file (little-endian) - version 2.4 (Ethernet, capture length 262144)
The standard tcp capture file is a
pcap file. It is not text so it can only be read by an analysis program that knows how to read pcap files.
Most useful open source tools are eventually cloned to other operating systems. When this happens, the application is said to have been ported over. Windump is a port of tcpdump and behaves in very similar ways.
One major difference between Windump and tcpdump is that Windump needs the Winpcap library installed prior to being able to run Windump. Despite both Windump and Winpcap being provided by the same maintainer, they are separate downloads.
Winpcap is an actual library that needs to be installed. But, once it is installed, Windump is an exe file that needs no installation so it can just run. That may be something to keep in mind if you’re running a Windows network. You don’t necessarily need Windump installed on every machine since you can just copy it over as needed, but you will want Winpcap installed in order to support Windump.
As with tcpdump, Windump can output network data to the screen for analysis, be filtered in the same way, and also write data to a pcap file for analysis offsite.
Wireshark is probably the next best known tool in any sysadmin’s toolkit. It can not only capture data, but also provides some advanced analysis tools. Adding to its appeal, Wireshark is open source, and has been ported over to almost every server operating system that exists. Starting life named Etheral, Wireshark now runs everywhere, including as a standalone portable app.
If you’re analyzing traffic on a server with a desktop installed, Wireshark can do it all for you. It can collect the data, and then analyze it all in one spot. However, desktops are not common on servers, so in many cases you’ll want to capture the network data remotely and then pull the resulting pcap file into Wireshark.
At first launch, Wireshark allows you to either load an existing pcap file, or start capturing. If you elect to capture network traffic, you can optionally specify filters to pare down the amount of data Wireshark collects. Since its analysis tools are so good, it’s less important to ensure you surgically identify the data at collection time with Wireshark. If you don’t specify a filter, Wireshark will simply collect all network data that your selected interface observes.
One of the most useful tools Wireshark provides is the ability to follow a stream. It’s probably most useful to think of a stream as an entire conversation. In the screenshot below we can see a lot of data has been captured, but what I am most interested in is that Google IP. I can right-click it and Follow the TCP Stream to see the entire conversation.
If you’ve captured traffic elsewhere, you can import the pcap file using Wireshark’s File -> Open dialogue. The same filters and tools that can be used for natively captured network data are available for imported files.
Tshark is a very useful cross between tcpdump and Wireshark. Tcpdump excels at collecting data and can very surgically extract only the data you want, however it is limited in how helpful it can be for analysis. Wireshark does a great job at both collection and analysis, but since it has a heavy user interface, it can’t be used on headless servers. Enter tshark; it captures and analyzes but does the latter on the command line.
Tshark uses the same filtering conventions as Wireshark which should be no surprise since they’re essentially the same product. This command tells tshark to only bother capturing the destination IP address as well as some other interesting fields from the HTTP part of the packet.
# tshark -i eth0 -Y http.request -T fields -e ip.dst -e http.user_agent -e http.request.uri 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /images/title.png 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /images/styles/phoenix.css 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /images/code/jquery_lightbox/jquery_lightbox/js/jquery-1.2.6.pack.js 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /images/styles/index.css 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /images/images/title.png 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /favicon.ico 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /favicon.ico
If you want to capture to a file you can use the
-w switch to write it, and then use tshark’s
-r (read mode) switch to read it.
# tshark -i eth0 -w tshark_packets Capturing on 'eth0' 102 ^C
Read it, either on the same server, or transfer it to some other analysis server.
# tshark -r tshark_packets -Y http.request -T fields -e ip.dst -e http.user_agent -e http.request.uri 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /contact 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /reservations/ 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /reservations/styles/styles.css 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /res/code/jquery_lightbox/jquery_lightbox/js/jquery-1.2.6.pack.js 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /res/styles/index.css 172.20.0.122 Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0 /res/images/title.png
6. Network Miner
Network Miner is a very interesting tool that falls more into the category of a forensic tool rather than a straight-up packet sniffer. The field of forensics typically deals with the investigation and collection of evidence and Network Miner does that job well for network traffic. Much like WireShark can follow a TCP stream to recover an entire TCP conversation, Network Miner can follow a stream in order to reconstruct files that were sent over the network.
To capture live traffic, Network Miner should be strategically placed on the network to be able to observe and collect the traffic you’re interested in. It won’t introduce any of its own traffic onto the network, so it operates very stealthily.
Network Miner can also operate in offline mode. You can use the tried and true tcpdump tool to catpure packets at a point of interest on your network, and then import the pcap files into Network Miner. It will then attempt to reconstruct any files or certificates it finds in the capture file.
Network Miner is built for Windows, but by using Mono, it can be run on any OS that has a Mono framework such as Linux and macOS.
There’s a free version to get you started that has a decent array of features. If you want more advanced capabilities such as GeoIP location and custom scripting, you’ll need to purchase a professional license.
7. Fiddler (HTTP)
Fiddler is not technically a network packet capture tool, but it is so incredibly useful that it made the list. Unlike the other tools listed here which are desgined to capture adhoc taffic on the network from any source, Fiddler is more of a desktop debugging tool. It captures HTTP traffic and while many browsers already have this capability in their developer tools, Fiddler is not limited to browser traffic. Fiddler can capture any HTTP traffic on the desktop including that of non-web applications.
Many desktop applications use HTTP to connect to web services and without a tool like Fiddler the only way to capture that traffic for analysis is using tools like tcpdump or WireShark. However, those tools operate at the packet level so analysis includes reconstruction of those packets into HTTP streams. That can be a lot of work in order to perform some simple HTTP investigation and Fiddler comes to the rescue. Fiddler can help discover cookies, certificates, and payload data coming in or out of those apps.
It helps that Fiddler is free and, much like Network Miner, it can be run within Mono on any other operating system that has a Mono framework.
Capsa network analyzer has several editions, each with varying capabilities. At the first level, Capsa free, the software essentially just captures packets and allows some very graphical analysis of them. The dashboard is very unique and can help novice sysadmins pinpoint network issues quickly even with little actual packet knowledge. The free level is aimed at people who want to know more about packets, and build up their skills into full-fledged analysts.
The free version knows how to monitor over 300 protocols, it allows for email monitoring and also it is able to save email content, and also supports trigger. The triggers can be used to set alerts for specific situations which means Capsa can also be used in a support capacity to some extent.
Capsa is only available for Windows 2008/Vista/7/8 and 10.
With the tools I have mentioned, it is not a big leap to see how a systems administrator could build an on-demand network monitoring infrastructure. Tcpdump, or Windump, could be installed on all servers. A scheduler, such as cron or Windows scheduler, could kick off a packet collection session at some time of interest and write those collections to a pcap file. At some later time, a sysadmin can transfer those packets to a central machine and use Wireshark to analyze them. If the network is so large that this isn’t feasible, then enterprise level tools like the SolarWinds suite can help tame all that network data into a manageable data set.