NetFlow is a network protocol developed by Cisco that notes and reports on all IP conversations passing through an interface. NetFlow is stateful and works in terms of the abstraction called a flow: that is, a sequence of packets that constitutes a conversation between a source and a destination, analogous to a call or connection. If you have intelligent switches and/or routers, they may support NetFlow, and you can add software or appliance-based probes that export NetFlow.
A NetFlow exporter device collects data on the IP traffic entering/exiting the device; it inspects packets and groups them into flows by inspecting particular fields: the source and destination addresses, protocols, ports, etc. Data on observed flows is rolled up from the packets and cached locally (in the flow cache), then it’s periodically exported to the collector based on active and inactive timeouts. NetFlow thus only handles IP, focusing on OSI Layers 3 and 4. Its knowledge of the IP protocols enables it to interpret packets and work in terms of flows.
Here’s our list of the best netFlow collectors and analyzers:
- SolarWinds Real-Time NetFlow Analyzer (FREE DOWNLOAD) A free tool for analysis of network traffic with the NetFlow, IPFIX, J-Flow, and Netstream standards.
- SolarWinds NetFlow Traffic Analyzer (FREE TRIAL) The leading network traffic analyzer. Runs on Windows Server.
- ManageEngine NetFlow Analyzer (FREE TRIAL) A traffic analyzer that installs on Windows Server and Linux and deploys the NetFlow, IPFIX, J-Flow, NetStream standards.
- Paessler PRTG NetFlow, sFlow, and J-Flow sensors that form part of a network, server, and application monitor. Installs on Windows Server.
- Nprobe and ntopng A straightforward network monitoring system in both free and paid versions.
- Plixer Scrutinizer A cybersecurity activity monitor that is available for installation, as a cloud-based service, or as an appliance.
- Nagios XI and Core An extensive network monitoring system in both free (Nagios Core) and paid (Nagios XI) versions.
- Kentik Detect A cloud-based service that can analyze your on-premises traffic.
- WhatsUp Gold A network monitor that runs on Windows Server and has a tr4affic analysis add-on module.
- Splunk A well-known and highly respected packet sniffer that can collect data by analysis through more sophisticated tools.
- Elastic Stack Log file collection and analysis tools that can be adapted to work with NetFlow.
- Influxdata’s TICK Stack Telegraf, Influxdb, Chronograf, and Kapacitor are network data collection and analysis tools that can use sFlow and SNMP.
NetFlow Types and Extensions
Flexible NetFlow and IPFIX provide the ability to have vendor-extensible templates for tweaking the set of packet fields of interest. NetFlow v9 and IPFIX also add the ability to monitor Layer 2 fields. Random Sampled NetFlow adds the option of doing sampling to NetFlow (sampling is mandatory in sFlow).
The differences between NetFlow and sFlow
Avi Freedman makes an apt analogy to monitoring vehicular traffic: “… while NetFlow can be described as observing traffic patterns (‘How many buses went from here to there?’), with sFlow you’re just taking snapshots of whatever cars or buses happen to be going by at that particular moment.”
Here are the main differences between the two technologies.
Accuracy and scalability
NetFlow’s partisans have long argued that NetFlow can be more accurate than sFlow. NetFlow aggregates data about all packets into flows locally at the device; thus it can’t by happenstance miss a conversation by failing to sample the relevant packets. This granularity of NetFlow is attractive for examining traffic with an individual host. It’s easy to see per-host details, notice localized anomalies, and investigate particular flows. But as traffic volume mushrooms, it becomes less and less feasible to collect every flow. If you’re not doing sampling, scalability becomes an issue.
sFlow is thus more scalable than traditional NetFlow. However, sampling has the downside that there may be gaps in visibility. The packets sampled may not reflect every flow (for instance, short bursts). For detecting and drilling down to investigate security issues, this can be significant.
Device performance at high volumes
As noted above, sFlow does minimal work on the network device, versus NetFlow which uses the device’s CPU and RAM to implement the flow cache. This can become a problem with high-speed devices where many conversations are concentrated onto a link. The additional CPU load on top of the “real work” the device is doing increases based on the number of flows per second, and can consume a significant fraction of the CPU per a Cisco whitepaper (PDF). In contrast, sFlow generally does its packet sampling in the switching/routing ASIC, letting the network device’s CPU concentrate on its core job.
At volumes of hundreds of gigabits per second, such as in edge routing and large data centers, traffic engineering becomes the central concern; the focus is on large-scale patterns and abrupt shifts in volume. Fine-grained visibility into individual hosts becomes less significant. Now sampling starts to become the clear winner. Because of this, NetFlow has added the option of Sampled NetFlow, which makes NetFlow scalable — but loses that accurate high granularity of traditional NetFlow.
NetFlow is IP only (with some Layer 2 support added recently). Thus legacy protocols (e.g., Appletalk, IPX) and other non-Internet protocols do not show up. In contrast, sFlow can cover Layers 2 through 7.
sFlow can have lower latency than NetFlow. A device collecting NetFlow metrics in its flow cache exports them periodically based on active and inactive timeouts. Thus reports on recent and ongoing conversations may be delayed, depending on the timeouts. In contrast, sFlow sends collected packet prefixes and counters in real-time. If sub-minute latency is a concern — and your monitoring/analysis tooling supports it — sFlow may be the better choice.
When your network grows to the point that seeing what’s going on has become tricky, tools leveraging NetFlow may be the solution. Below, we look at several popular NetFlow-based network monitoring and analysis tools for Windows. All are sophisticated, having a considerable learning curve; so online training and good support are important.
SolarWinds produces a suite of products providing comprehensive support for network monitoring and management. The Real-Time NetFlow Analyzer is a free tool that provides real-time insight into your current flows. The free version is focused on displaying the current and recent state of your bandwidth usage. It’s limited to one NetFlow interface and 60 minutes of data. Flow technologies supported include NetFlow, Juniper’s J-Flow, IPFIX, and Huawei’s netstream.
The analyzer identifies which devices/IP addresses, apps, and users are consuming the most bandwidth. The user interface displays inbound and outbound traffic for the chosen NetFlow exporter; traffic can be sorted and displayed in various ways. The user interface’s tree explorer summarizes NetFlow traffic, parsing it into applications, conversations, domains, endpoints, and protocols. Each can be expanded into an inclusive graph for drilling down to explore particular aspects. The tree views and graphs update in real-time.
Installation is via a standard Windows setup wizard, and the NetFlow Configurator is included to assist in configuring the NetFlow collector and your devices that support various NetFlow variants.
If your key devices support NetFlow, and you’re looking for a lean and clear viewport into your current and recent bandwidth usage, the SolarWinds Real-Time NetFlow Analyzer fits the bill.
For a more powerful and feature-rich version, SolarWinds for-cost option, the Network Traffic Analyzer, is covered below.
Another free traffic analysis tool that you could try is the SolarWinds Flow Tool Bundle. This is a useful traffic sample collector that employs Cisco NetFlow v5. As well as collecting traffic samples, the tool includes a traffic flow simulator, which will enable you to preview the effects on the network of extra traffic volumes, or changes in hardware layout.
The SolarWinds NetFlow Traffic Analyzer (NTA) is the for-cost step up from their free tool, the Real-Time NetFlow Traffic Analyzer. NTA is a module in the Network Performance Monitor (NPM), so you must accommodate the costs and platform requirements of both. NTA and NPM both are available in a 30-day fully-functional trial.
NTA might well be called the Network Traffic Analyzer since it handles not just the original Cisco Netflow but many of its variants from other manufacturers, as well as NetFlow’s primary alternative, sFlow.
Once installed, NPM and NTA offer you a wide range of sophisticated facilities for managing multi-vendor networks. It features bandwidth monitoring, traffic analysis, performance analysis, alerts, customizable reports, policy optimization, and more.
The NetFlow Traffic Analyzer gathers flow data exported by the flow-enabled devices tracked by the SolarWinds network monitoring software.
The default NetFlow Traffic Analyzer Summary has multiple sections like Top 5 Applications, Top 5 Endpoints, Top 5 Conversations, Top 10 Sources by % Utilization, etc.
As a flow analyzer, NTA identifies the users, applications, and protocols consuming the most bandwidth. You can sort by ports, source, destination, and protocols, and view traffic patterns over minutes, days, or months.
NTA and NPM are enterprise-grade packages, so even the free trial will consume considerable resources on your system. If you have a sophisticated network with NetFlow-enabled devices, NTA’s capabilities are worth exploring. For details on NTA, see our SolarWinds NetFlow Traffic Analyzer review.
The ManageEngine NetFlow Analyzer provides real-time visibility into network bandwidth and traffic patterns. The tool visualizes traffic by applications, conversations, protocols, etc. Alerts can be set based on traffic thresholds. There are a variety of useful predefined reports, ranging from troubleshooting oriented to capacity planning and billing. Custom search reports can be created.
The NetFlow Analyzer has a suite of NetFlow-oriented tools for managing complex networks. The web-based user interface has a default dashboard with several real-time pie charts, including a heat map showing the status of monitored interfaces, top applications, top protocols, top conversations, recent alarms, top QoS, and more.
Hovering over a graphic usually provides an explanatory pop-up, and clicking on any graphic drills down to more details on the selected element. There are specific displays for detecting security issues. Dashboards are customizable.
Alerts show up as pop-ups on the user interface. Multi-site traffic can be analyzed; there is a smartphone app for mobile monitoring and alerting.
Flow technologies supported include NetFlow, IPFIX, J-Flow, NetStream, and several others. The tool leverages advanced features of Cisco devices, including support for adjusting the traffic shaping and QoS policies on your network.
The ManageEngine NetFlow Analyzer provides a range of capabilities for managing complex networks making heavy use of NetFlow. The free version allows unlimited monitoring for 30 days but then reverts to monitoring only two interfaces. ManageEngine has a variety of related products to expand beyond NetFlow traffic-oriented analysis into a full network management suite. You can download a 30-day free trial.
The Paessler PRTG Network Monitor is a “batteries included” solution that monitors bandwidth utilization, the availability and health of devices on your network, and more. PRTG can monitor multiple sites, WAN, VPN, and cloud services. The free version provides unlimited sensors for a month, and thereafter is limited to 100 sensors; a sensor is an individual data stream, so each device will typically require several sensors.
In PRTG’s user interface, a primary view is the device tree showing all devices on your network and the sensors monitoring each. Devices include firewalls, routers, access points, servers, workstations, virtual servers, storage, etc. The device tree is supplemented by table views of sensors, logs, and alarms, as well as various charts and graphs for bandwidth, etc. Tables can be sorted and filtered.
Drilling down through the tree view reveals indicators and metrics at every level. Settings, like scan interval, are inherited and can be overridden at lower levels in the device tree. Alerts can similarly be set at every level, so you can arrange to be notified about events and threshold transitions of a particular critical device, or rolled up from an overall aspect of your network. Alerts can be transmitted in multiple ways, including SMTP email and SMS text messaging.
The devices-and-sensors abstraction shapes the dashboards and reports too. Custom dashboards can be created, including interactive maps. There is a range of predefined reports, and facilities for designing custom reports; reports can also be scheduled.
Traffic analysis facilities include built-in NetFlow support. For flow protocols, PRTG supports NetFlow, sFlow, and J-Flow. Other protocols/mechanisms used include SNMP, WMI, and packet sniffing. Paessler calls these detection systems, such as the NetFlow collector, “sensors.”
Installation is straightforward. There is a setup wizard, as well as a video providing step-by-step guidance. At installation, the core server’s local probe does auto-discovery to identify devices and set up sensors. Additional sensors (including NetFlow collectors) can be added manually; a video provides instructions.
The core server is Windows only. Monitoring of a single site can be done via the web application, but the simultaneous view of multiple core servers requires using the enterprise app on Windows. A mobile app is also provided. One clever addition is that PRTG provides QR codes that can be pasted on particular devices for a quick look-up and statusing in the mobile app. PRTG supports clustering for fault tolerance: you can set up failover instances of the monitor.
Though PRTG is all-in-one so you don’t need multiple products and licenses to gain comprehensive monitoring, a key question to evaluate is how many sensors your network needs, and what will be the long-term cost of the sensor-based licensing model as you grow. To evaluate, you can download a 30-day trial of the software here.
ntopng is an open-source web-based traffic analysis tool that does passive network monitoring based on flow data and statistics extracted from observed traffic. ntopng does the packet capture itself; to receive flow data it depends on nProbe, a NetFlow/IPFIX exporter/collector. Flow protocols include NetFlow v9, IPFIX, and NetFlow-lite.
The community version of ntopng is free. The professional (small business) and enterprise versions require a paid license, but are free to educational and nonprofit organizations. nProbe can be test-driven for free but a fully functioning version requires a paid license. So the use of NetFlow data is limited (unless you qualify for a free license).
ntopng’s web-based user interface rolls up data into traffic (e.g., top talkers), flows, hosts, devices, and interfaces. Most categories have multiple views, a mix of charts, tables, and graphs; and in each you can drill down to explore in depth and cross-reference. Tables can be sorted – so for instance, selecting the throughput column on the flows table shows the current top bandwidth users.
The flow display shows application protocols (e.g. Facebook, YouTube). Latencies and TCP statistics (e.g. packet loss) are displayed. Observed hosts/IP addresses can be displayed on a map via geolocation. Alerts can be set on hosts based on many criteria, and will show up as an icon in the user interface.
The professional version can save and display historical application usage statistics, do active monitoring via SNMP, generate custom traffic reports, and several other additional features.
The installation package for both ntopng and nProbe is a zip file containing a standard Windows setup wizard. The installer will install winpcap (for packet sniffing) if needed.
Since ntopng is open source, there is considerable scope for extending it. Data can be exported to MySQL, ElasticSearch, and LogStash, where is can be merged into the reports stored by your Syslog server.
Plixer Scrutinizer is a sophisticated flow-oriented traffic analysis system with particular focus on security forensics (it’s called the “Scrutinizer Incident Response System”). It supports both NetFlow and sFlow.
Scrutinizer can be installed as a dedicated physical appliance, as a virtual machine running on a server, or as a SaaS solution running in the cloud (public or hybrid). It’s a sophisticated system, so even the free trial on a virtual machine demands considerable resources (e.g., a dedicated 16GB of RAM).
Scrutinizer is designed for high performance and scalability from small to very large environments. It provides a rich range of analysis and reporting features.
The trial includes full access for 30 days. After that, the free version has a limit of 10K flows collected per second, five hours of raw flows kept, and one week of historical summaries maintained. The paid version includes notifications, dashboard customization, custom reports, scheduled email reports, and support. License pricing depends on the platform chosen and the number of flow exporters to be supported.
Nagios is an enduring standard in network monitoring. Nagios Core is the open-source free version, and Nagios XI is the commercial for-cost variant with additional features and automated assistance for configuration. Nagios has a reputation for being powerful, reliable, scalable, and extremely customizable – and being complex to configure.
The free version has a learning curve but also an active community. It monitors servers, services, and applications, just like the commercial version. It includes reporting by email and SMS, a basic user interface (including the network map), and basic reports.
Nagios Core lacks auto-discovery, and you must learn to set up and maintain complex configurations. On the plus side, it does give you a lot of flexibility to customize and extend the tool. Community-developed addons can perform discovery and help you get started with configuration.
You can use the free 60-day trial to evaluate the for-cost version. If you elect to go with the free version when the trial is done, you can save the auto-generated config files from
/usr/local/nagios/etc before uninstalling your eval copy. You can then use those files as your starting point for your new installation’s configuration.
The commercial version Nagios XI has a richer range of features, including automated support for discovering your devices and hosts, automatically configuring the tool, and commercially-supported addons. It has a much more sophisticated user interface and more advanced reporting that covers trends, capacity planning assistance, etc.
Nagios XI is built to run on Red Hat Linux and CentOS. For Windows, use a VM appliance with Hyper-V or VMware. It includes an auto-discovery tool and a configuration wizard for adding a new device, host, or application.
Once Nagios XI is installed and monitoring, the Operations Screen gives you a high-level view of the current state of the network, and the Operations Center lets you drill down to the items mentioned.
The Host Status page shows a summary of metrics for the monitored hosts. You can drill down to an individual host to see details including performance graphs, capacity planning info, alarms, etc.
The Service Status page summarizes the state of the monitored services.
Nagios is a well-regarded solution for network monitoring. As with other tools that offer a fully-free vs commercial version tradeoff, you must decide whether you have (or will develop) the expertise and time to use the free tool, or whether it would be more cost-effective to pay for the automation and support of the commercial version.
Kentik Detect, in contrast to the traffic analyzer tools above, is a pure Software-as-a-Service (SaaS) system. As such, it offers the scalability of the cloud.
Networks are growing, and off-premises network resources are more vital to success. Thus, traffic data is becoming big data, and cloud-based big data solutions start to make sense.
Kentik aims to capture the details of multiple types of data, provide a unified view of all of it, and provide interfaces for accessing the data and integrating with other systems. Kentick Detect is composed of a custom high-availability time-series datastore (Kentik Data Engine) and a UI (Kentik Portal). Protocols include Netflow, IPFIX, sFlow, SNMP, and BGP.
Kentik Portal is a web-based interface (of course) and provides a growing range of configurable dashboards.
The Data Explorer permits ad-hoc exploration of the collected network data. You can quickly drill down and filter on potentially billions of records, obtaining views in the form of tables and graphs.
Alerting to notify you of unusual conditions can be set up by creating policies that define when an alert will enter alarm state. Alerts can be sent by various media, including email, Slack, paging, etc.
WhatsUp Gold is a well-known network monitoring tool from IPSwitch that’s feature-rich yet is straightforward. It’s available in both a free starter edition and a 30-day trial to evaluate the paid one.
WhatsUp Gold monitors network traffic, servers, virtual servers, cloud services, and applications. The free version is a free five-point license for monitoring up to five resources (e.g., five servers).
WhatsUp Gold must be installed on Windows. Setup is simple and uses auto-discovery. The user interface provides multiple views with an interactive network map and the ability to drill down to investigate issues.
WhatsUp Gold’s list view shows the discovered hosts and devices, summarizing their characteristics and status.
The map view is an interactive map for visualizing your network’s components and their statuses. You can drill down to inspect the availability and performance of individual nodes.
Traffic analysis facilities work with a wide range of flow-enabled devices, including NetFlow, sFlow, NetFlow-Lite, IPFIX, and J-Flow.
Dashboards are customizable. WhatsUp Gold provides many canned reports, including reports for bandwidth and utilization; you can design customized reports, too.
The top 10 view shows critical statuses in your network.
You can configure alerts to notify you when senders or receivers exceed bandwidth thresholds, when interfaces exceed utilization thresholds, etc. There are multiple possible methods for notification, including email and SMS. Triggered actions give the ability to execute actions automatically as responses to alerts.
The free edition of WhatsUp Gold is a straightforward and fully-featured tool for monitoring and managing a small shop. Graduating to the for-cost version lets you move up to covering large networks.
10. Roll Your Own
Perhaps none of the above pre-packaged NetFlow analyzers are customizable enough or powerful enough to meet your needs. Maybe you’re sure you can do better, or you just want to experiment with analyzing the data yourself. There are multiple packages for time-series data capture and analytics available that make this quite doable. Several are free open-source software; some are not. Some can be integrated with prepackaged analyzers, such as Plixer and ntopng.
Here are a few possibilities to check out.
Splunk is a for-cost package for searching, monitoring, and analyzing/visualizing big data. Splunk captures real-time data and provides web-based facilities for analyzing and visualization. Splunk has an add-on for NetFlow, and one for IPFIX.
The ELK Stack – Elasticsearch, Logstash, and Kibana – is an open-source analytics toolset typically used with data that resembles log messages. Elasticsearch is a popular distributed search and analytics engine. Logstash is a data collection and log-parsing engine. Kibana is a browser-based data visualization dashboard for analytics and search. Logstash includes a codec for processing multiple versions of NetFlow data.
Several groups have used the ELK Stack with NetFlow. Cisco has a guide for doing it, and there are several other articles online. People have built systems using the ELK Stack with other popular components, such as the Riemann distributed system monitoring and alerting tool. An alternative to logstash is fluentd.
Influxdata‘s TICK Stack – Telegraf, Influxdb, Chronograf, and Kapacitor – is a set of Go-based open-source tools for capturing, monitoring, and analyzing/visualizing time-series metrics data. Telegraf collects performance metrics; InfluxDB is a time series database; Chronograf performs real-time visualization of InfluxDB data; and Kapacitor is a streaming/batch data-processing engine that can do monitoring and alerting of views of InfluxDB data. The TICK Stack has been used with network statistics from sFlow and SNMP.
Another powerful tool, sometimes used with Influxdb is Grafana, an open-source package for time-series analytics and visualization. Grafana is analogous to Kibana, but where Kibana is log-message oriented, Grafana is metrics-oriented.
Choosing a NetFlow Analyzer
The table below shows a summary of each of these options.
|1. SolarWinds Real-Time NetFlow Analyzer||Free Download||Windows||SOHO|
|2. SolarWinds NetFlow Traffic Analyzer||Free Trial||Windows||SMB to large enterprises|
|3. ManageEngine NetFlow Analyzer||Free Trial|
For-cost tool with free starter edition for small shops
|Windows, Linux||SMB to large enterprises|
|4. Paessler PRTG||Free Trial|
For-cost tool with free starter edition for small shops
|Windows||SMB to large enterprises|
|5. Nprobe and ntopng||For-cost (unless non-profit)||Windows, Linux||SMB to large enterprises|
|6. Plixer Scrutinizer||For-cost tool with free starter edition for small shops||Hardware appliance, Windows or Linux VM, SaaS||SMB to large enterprises|
|7. Nagios XI and Core||Free open-source tool, or for-cost tool with support/enhancements||Linux, or on Windows in a VM appliance||SMB to large enterprises|
|8. Kentik Detect||For-cost tool||SaaS||SMB to large enterprises|
|9. WhatsUp Gold||For-cost tool with free starter edition for small shops||Windows||SMB to large enterprises|
|10. Roll your own||Components, paid or free open source||Varies||SMB to large enterprises|
Multiple excellent tools for network monitoring and traffic analysis are available. Small organizations have an array of free choices, and large or growing organizations have many for-cost options.
In recent years, open source solutions have become widely implemented for many types of networking software and also for business and security applications. A benefit of open source projects is that anyone can read the code that drives the software. By that enquiry, you can be sure that there is no malicious code hidden inside the program.
Usually, open source projects are maintained by volunteers. The benefit of enthusiast-developed software is that it can be given away for free. The downside of this setup is that the free tools aren’t professionally managed and can contain bugs. The lack of income of free software means that the organizations that maintain it don’t have the funds to keep up with security standards or fix problems with the code.
When you consider using open source software for network monitoring and analysis, check out the packages that interest you and test them thoroughly before you commit the network to it. Consider paying for network analysis tools in order to get guaranteed performance and also support from the commercial organizations that provide that paid software.
Anyone who wants to contribute the effort to learn has a toolbox of powerful components that you can use to roll your own solution. Your final choice depends on the size and complexity of your network, the expertise you bring (or want to develop), and how you expect your network to evolve in the future.