Virus attacks are no longer the only threats to the integrity of your network. Advanced persistent threats have become the new intrusion method that is grabbing the headlines. Under this scenario, hackers gain access to your network, control logs, and manipulate user records to gain entry whenever they want. They can even sell off your internet bandwidth as a cover for other cybercriminals.
Network intrusion detection tools and systems are now essential for network security. Fortunately, these systems are very easy to use and most of the best IDSs on the market are free to use. In this review, you will read about the ten best intrusion detection systems that you can install now to start protecting your network from attack. We cover tools for Windows, Linux and Mac.
Here’s 10 Intrusion Detection Systems and Tools:
- Security Onion
Read on to learn more about intrusion detection and to find out why we recommend these tools:
Host Intrusion Detection Systems (HIDS)
Host-based intrusion detection, also known as host intrusion detection systems or host-based IDS, examine events on a computer on your network rather than the traffic that passes around the system. This type of intrusion detection system is abbreviated to HIDS and it mainly operates by looking at data in admin files on the computer that it protects. Those files include log files and config files.
A HIDS will back up your config files so you can restore settings should a malevolent virus loosen the security of your system by changing the setup of the computer. Another key element that you want to guard against is root access on Unix-like platforms or registry alterations on Windows systems. A HIDS won’t be able to block these changes, but it should be able to alert you if any such access occurs.
Each host the HIDS monitors must have some software installed on it. You can just get your HIDS to monitor one computer. However, it is more typical to install the HIDS on every device on your network. This is because you don’t want to overlook config changes on any piece of equipment. Naturally, if you have more than one HIDS host on your network, you don’t want to have to log into each one in order to get feedback. So, a distributed HIDS system needs to include a centralized control module. Look for a system that encrypts communications between host agents and the central monitor.
Network Intrusion Detection Systems (NIDS)
Network-based intrusion detection, also known as a network intrusion detection system or network IDS, examines the traffic on your network. As such, a typical NIDS has to include a packet sniffer in order to gather network traffic for analysis.
The analysis engine of a NIDS is typically rule-based and can be modified by adding your own rules. With many NIDS, the provider of the system, or the user community, will make rules available to you and you can just import those into your implementation. Once you become familiar with the rule syntax of your chosen NIDS, you will be able to create your own rules.
Chaining back to traffic collection, you don’t want to dump all of your traffic into files or run the whole lot through a dashboard because you just wouldn’t be able to analyze all of that data. So, the rules that drive analysis in a NIDS also create selective data capture. For example, if you have a rule for a type of worriesome HTTP traffic, your NIDS should only pick up and store HTTP packets that display those characteristics.
Typically, a NIDS is installed on a dedicated piece of hardware. High-end paid-for enterprise solutions come as a piece of network kit with the software pre-loaded onto it. However, you don’t have to pay out big bucks for the specialist hardware. A NIDS does require a sensor module to pick up traffic, so you may be able to load it onto a LAN analyzer, or you may choose to allocate a computer to run the task. However, make sure the piece of equipment that you choose for the task has enough clock speed to not slow down your network.
HIDS or NIDS?
The short answer is: both. A NIDS will give you a lot more monitoring power than a HIDS. You can intercept attacks as they happen with a NIDS, whereas a HIDS only notices anything is wrong once a file or a setting on a device has already changed. However, just because HIDSs don’t have as much activity as NIDSs doesn’t mean that they are less important.
The fact that the NIDS is usually installed on a stand-alone piece of equipment means that it doesn’t drag down the processors of your servers. However, the activity of HIDS is not as aggressive as that of NIDS. A HIDS function can be fulfilled by a lightweight daemon on the computer and shouldn’t burn up too much CPU. Neither system generates extra network traffic.
Detection methods: Signature-based or Anomaly-based IDS
Whether you are looking a host intrusion detection system or a network intrusion detection system, all IDSs use two modes of operation — some may only use one or the other, but most use both.
- Signature-based IDS
- Anomaly-based IDS
The signature-based method looks at checksums and message authentication. Signature-based detection methods can be applied just as well by NIDS as by HIDS. A HIDS will look at log and config files for any unexpected rewrites, whereas a NIDS will look at the checksums in packets and message authentication integrity of systems such as SHA1.
The NIDS may include a database of signatures that packets known to be sources of malicious activities carry. Fortunately, hackers don’t sit at their computers typing like fury in order to crack a password or access the root user. Instead, they use automated procedures supplied by well-known hacker tools. These tools tend to generate the same traffic signatures every time because computer programs repeat the same instructions over and over again rather than introducing random variations.
Anomaly-based detection looks for unexpected or unusual patterns of activities. This category can also be implemented by both host and network-based intrusion detection systems. In the case of HIDS, an anomaly might be repeated failed login attempts, or unusual activity on the ports of a device that signify port scanning.
In the case of NIDS, the anomaly approach requires establishing a baseline of behavior to create a standard situation against which ongoing traffic patterns can be compared. A range of traffic patterns are considered acceptable, and when current real-time traffic moves out of that range, an anomaly alert is provoked.
Choosing an IDS method
Sophisticated NIDSs can build up a record of standard behavior and adjust their boundaries as their service life progresses. Overall, both signature and anomaly analysis is much simpler in operation and easier to set up with HIDS software than with NIDS.
Signature-based methods are much faster than anomaly-based detection. A fully comprehensive anomaly engine touches on the methodologies of AI and can cost a lot of money to develop. However, signature-based methods boil down to the comparison of values. Certainly, in the case of HIDS, pattern matching with file versions can be a very straightforward task that anyone could perform themselves using command line utilities with regular expressions. So, they don’t cost as much to develop and are more likely to be implemented in free intrusion detection systems.
A comprehensive intrusion detection system needs both signature-based methods and anomaly-based procedures.
Defend the network with an IPS
Now we need to consider intrusion prevention systems (IPSs). IPS software and IDSs are branches of the same technology, because you can’t have prevention without detection. Another way to express the difference between these two branches of intrusion tools is to call them passive or active. A straightforward intrusion monitoring and alerting system is sometimes called a “passive” IDS. A system that not only spots an intrusion, but takes action to remediate any damage and block further intrusion activity from a detected source, is also known as a “reactive” IDS.
Reactive IDSs, or IPSs, usually don’t implement solutions directly. Instead, they interact with firewalls and applications by adjusting settings. A reactive HIDS can interact with a number of networking aides to restore settings on a device, such as SNMP or an installed configuration manager. Attacks on the root user, or admin user in Windows, usually aren’t dealt with automatically as the blocking of an admin user or changing the system password would result in locking the system administrator out of the network and servers.
Many users of IDSs report a flood of false positives when they first install their defense systems. As IPSs automatically implement defense strategy on detection of an alert condition. Incorrectly calibrated IPSs can cause havoc and bring your legitimate network activity to a standstill.
To minimize the network disruption that can be caused by false positives, you should introduce your intrusion detection and prevention system in stages. Triggers can be tailored and you can combine warning conditions to create custom alerts. The statement of actions that need to be performed on the detection of threats is termed a policy. The interaction of intrusion detection and prevention procedures with firewalls should be particularly fine tuned to prevent your business’s genuine users from being locked out by over-tight policies.
Intrusion detection systems by type and operating system
The producers of IDS software focus on Unix-like operating systems. Some produce their code according to the POSIX standard. In all of these cases, that means that Windows is excluded. As the Mac OS operating systems of Mac OS X and macOS are based on Unix, these operating systems are much better catered to in the IDS world than in other software categories. The table below explains which IDSs are host-based, which are network-based, and which operating systems each can be installed on.
You may read some reviews that claim that Security Onion can be run on Windows. It can if you first install a virtual machine and run it through that. However, for the definitions in this table, we only count software as being compatible with an operating system if it can be installed directly.
Top Intrusion Detection Tools & Software
Intrusion Detection Systems for Unix
To restate the information in the table above into a Unix-specific list, here are the HIDS and NIDS you can use on the Unix platform.
Host intrusion detection systems:
Network intrusion detection systems:
Intrusion Detection Systems for Linux
Here are lists of the host intrusion detection systems and network intrusion systems that you can run on the Linux platform.
Host intrusion detection systems:
- Security Onion
Network intrusion detection systems:
- Security Onion
- Open WIPS-NG
Intrusion Detection Systems for Windows
Despite the popularity of Windows Server, the developers of intrusion detection systems don’t seem to be very interested in producing software for the Windows operating system. Here are the few IDSs that run on Windows.
Host intrusion detection systems:
Network intrusion detection systems:
Intrusion Detection Systems for Mac OS
Mac owners benefit from the fact that Mac OS X and macOS are both based on Unix and so there are far more intrusion detection system options for Mac owners than those who have computers running the Windows operating system.
Host intrusion detection systems:
Network intrusion detection systems:
Recommended intrusion detection systems
Now you have seen a quick rundown of host-based intrusion detection systems and network-based intrusion detection systems by operating system, in this list we go deeper into the details of each of the best IDS:
OSSEC stands for Open Source HIDS Security. It is the leading HIDS available and it is completely free to use. As a host-based intrusion detection system, the program focuses on the log files in the computer where you install it. It monitors the checksum signatures of all your log files to detect possible interference. On Windows, it will keep tabs on any alterations to the registry. On Unix-like systems it will monitor any attempts to get to the root account. Although OSSEC is an open source project, it is actually owned by Trend Micro, a prominent security software producer.
The main monitoring application can cover one computer or several hosts, consolidating data in one console. Although there is a Windows agent that allows Windows computers to be monitored, the main application can only be installed on a Unix-like system, which means Unix, Linux or Mac OS. There is an interface for OSSEC for the main program, but this is installed separately and is no longer supported. Regular users of OSSEC have discovered other applications that work well as a font-end to the data gathering tool: include Splunk, Kibana, and Graylog.
The log files covered by OSSEC include FTP, mail, and web server data. It also monitors operating system event logs, firewall and antivirus logs and tables, and traffic logs. The behavior of OSSEC is controlled by the policies that you install on it. These can be acquired as add-ons from the large user community that is active for this product. A policy defines an alert condition. Those alerts can be displayed on the console or sent as notifications via email. Trend Micro offers support for OSSEC for a fee.
Snort is the industry leader in NIDS, but it is still free to use. This is one of the few IDSs around that can be installed on Windows. It was created by Cisco. The system can be run in three different modes and can implement defense strategies, so it is an intrusion prevention system as well as an intrusion detection system.
The three modes of Snort are:
- Sniffer mode
- Packet logger
- Intrusion detection
You can use snort just as a packet sniffer without turning on its intrusion detection capabilities. In this mode you get a live readout of packets passing along the network. In packet logging mode, those packet details are written to a file.
When you access the intrusion detection functions of Snort, you invoke an analysis module that applies a set of rules to the traffic as it passes by. These rules are called “base policies,” and if you don’t know which rules you need, you can download them from the Snort website. However, once you become confident in the methodologies of Snort, it is possible to write your own. There is a large community-base for this IDS and they are very active online on the community pages of the Snort website. You can get tips and help from other users and also download rules that experienced Snort users have developed.
The rules will detect events such as stealth port scans, buffer overflow attacks, CGI attacks, SMB probes, and OS fingerprinting. The detection methods depend on the specific rules being used and they include both signature-based methods and anomaly-based systems.
Snort’s fame has attracted followers in the software developer industry. A number of applications that other software houses have created can perform deeper analysis on the data collected by Snort. These include Snorby, BASE, Squil, and Anaval. Those companion applications help you make up for the fact that the interface for Snort isn’t very user-friendly.
Suricata is probably the main alternative to Snort. There is a key advantage that Suricata has over Snort, which is that it collects data at the application layer. This overcomes a blindness that Snort has to signatures split over several TCP packets. Suricata waits until all of the data in packets is assembles before it moves information into analysis.
Although the system works at the application layer ,it is able to monitor protocol activity at lower levels, such as IP, TLS, ICMP, TCP, and UDP. It examines real-time traffic for different network applications including FTP, HTTP, and SMB. The monitor doesn’t just look at packet structure. It can examine TLS certificates and focus on HTTP requests and DNS calls. A file extraction facility lets you examine and isolate suspicious files with virus infection characteristics.
Suricata is compatible with Snort and you can use the same VRT rules written for that NIDS leader. Those third-party tools, such as Snorby, BASE, Squil, and Anaval that integrate with Snort can also bolt on to Suricata. So, accessing the Snort community for tips and free rules can be a big benefit for Suricata users. A built-in scripting module allows you to combine rules and get a more precise detection profile than Snort can give you. Suricata uses both signature and anomaly-based methods.
Suricata has a clever processing architecture that enables hardware acceleration by using many different processors for simultaneous, multi-threaded activity. It can even run partly on your graphics card. This distribution of tasks keeps load from bearing down on just one host. That’s good because one problem with this NIDS is that it is quite heavy on processing. Suricata has a very slick-looking dashboard that incorporates graphics to make analysis and problem recognition a lot easier. Despite this expensive-looking front-end, Suricata is free of charge.
Bro is a free NIDS that goes beyond intrusion detection and can provide you with other network monitoring functions as well. The user community of Bro includes many academic and scientific research institutions.
The Bro intrusion detection function is fulfilled in two phases: traffic logging and analysis. As with Suricata, Bro has a major advantage over Snort in that its analysis operates at the application layer. This gives you visibility across packets to get a broader analysis of network protocol activity.
The analysis module of Bro has two elements that both work on signature analysis and anomaly detection. The first of these analysis tools is the Bro event engine. This tracks for triggering events, such as a new TCP connection or an HTTP request. Each event is logged, so this part of the system is policy-neutral — it just provides a list of events in which analysis may reveal repetition of actions or suspiciously diverse activity generated by the same user account.
The mining of that event data is performed by policy scripts. An alert condition will provoke an action, so Bro is an intrusion prevention system as well as a network traffic analyzer. The policy scripts can be customized but they generally run along a standard framework that involves signature matching, anomaly detection, and connection analysis.
You can track HTTP, DNS, and FTP activity with Bro and also monitor SNMP traffic, enables you to check on device configuration changes and SNMP Trap conditions. Each policy is a set of rules and you are not limited to the number of active policies or the protocol stack layers that you can examine. At lower levels, you can watch out for DDoS syn flood attacks and detect port scanning.
Bro can be installed on Unix, Linux, and Mac OS.
Sagan is a host-based intrusion detection system, so this is an alternative to OSSEC and it is also free to use. Despite being a HIDS, the program is compatible with data gathered by Snort, which is a NIDS system. This compatibility also extends to the other tools that can be used in conjunction with Snort, such as Snorby, BASE, Squil, and Anaval. Data sources from Bro and Suricata can also feed into Sagan. This tool can be installed on Unix, Linux, and Mac OS. Although you can’t run Sagan on Windows, you can feed windows event logs into it.
Strictly speaking, Sagan is a log analysis tool. The element that it lacks to make it a stand-alone NIDS is a packet sniffer module. However, on the plus side, this means that Sagan doesn’t require dedicated hardware and it has the flexibility to analyze both host logs and network traffic data. This tool would have to be a companion to other data gathering systems in order to create a full intrusion detection system.
Some nice features of Sagan include an IP locator, which enables you to see the geographical location of the IP addresses that are detected as having suspicious activities. This will enable you to aggregate the actions of IP addresses that seem to be working in concert to form an attack. Sagan can distribute its processing over several devices, lightening the load on the CPU of your key server.
This system includes script execution, which means that it will generate alerts and perform actions on the detection of intrusion scenarios. It can interact with firewall tables to implement IP bans in the event of suspicious activity from a specific source. So, this is an intrusion prevention system. The analysis module works with both signature and anomaly detection methodologies.
Sagan doesn’t make it onto everyone’s list of the best IDSs because it doesn’t truly qualify as an IDS, being a log file analyzer. However, its HIDS with a splash of NIDS concept makes it an interesting proposition as a hybrid IDS analysis tool component.
For a blend of IDS solutions, you could try the free Security Onion system. Most of the IDS tools in this list are open source projects. That means that anyone can download the source code and change it. That’s exactly what the developer of Security Onion did. He took elements from the source code of Snort, Suricata, OSSEC, and Bro and stitched them together to make this free Linux-based NIDS/HIDS hybrid. Security Onion is written to run on Ubuntu and it also integrates elements from front-end systems and analysis tools including Snorby, Sguil, Squert, Kibana, ELSA, Xplico, and NetworkMiner.
Although Security Onion is classified as a NIDS, it does include HIDS functions as well. It will monitor your log and config files for suspicious activities and check on the checksums of those files for any unexpected changes. One downside of the Security Onion’s comprehensive approach to network monitoring is its complexity. It has several different operating structures and there isn’t really sufficient learning material online or bundled in to help the network administrator get to grips with the full capabilities of the tool.
Network analysis is conducted by a packet sniffer, which can display passing data on a screen and also write to a file. The analysis engine of Security Onion is where things get complicated because there are so many different tools with different operating procedures that you may well end up ignoring most of them. The interface of Kibana provides the dashboard for Security Onion and it does include some nice graphs and charts to ease status recognition.
Both signature-based and anomaly-based alert rules are included in this system. You get information on device status as well as traffic patterns. All of this could really do with some action automation, which Security Onion lacks.
“Advanced Intrusion Detection Environment” is a lot to write, so the developers of this IDS decided to abbreviate its name to AIDE. This is a free HIDS that focuses on rootkit detection and file signature comparisons for Unix and Unix-like operating systems, so it will work on Mac OS and Linux as well.
If you have considered Tripwire, you would be better off looking at AIDE instead, because this is a free replacement for that handy tool. Tripwire has a free version, but a lot of the key functions that most people need from an IDS are only available with the paid-for Tripwire, so you get a lot more functionality for free with AIDE.
The system compiles a database of admin data from config files when it is first installed. That creates a baseline and then any changes to configurations can be rolled back whenever changes to system settings are detected. The tool includes both signature and anomaly monitoring methods. System checks are issued on demand and do not run continuously, which is a bit of a shortfall with this HIDS. As this is a command line function, though, you can schedule it to run periodically with an operating method, such as cron. If you want near real-time data, you could just schedule it to run very frequently.
AIDE is really just a data comparison tool and it doesn’t include any scripting language, you would have to rely on your shell scripting skills to get data searching and rule implementation functions into this HIDS. Maybe AIDE should be considered more as a configuration management tool rather than as an intrusion detection system.
If you have heard about Aircrack-NG, then you might be a little cautious of this network-based IDS because it was developed by the same entrepreneur. Aircrack-NG is a wireless network packet sniffer and password cracker that has become part of every wifi network hacker’s toolkit.
In WIPS-NG we see a case of poacher-turned-gamekeeper. This free software is designed to defend wireless networks. Although Aircrack-NG can run on a range of operating systems, Open WIPS-NG only runs on Linux. The name “WIPS” stands for “wireless intrusion prevention system,” so this NIDS both detects and blocks intrusion.
The system includes three elements:
There are plans to allow a WIPS-NG installation to monitor multiple sensors. However, at the moment, each installation can only include one sensor. That shouldn’t be too much of a problem because you can achieve multiple tasks with just the one sensor. The sensor is a packet sniffer, which also has the ability to manipulate wireless transmissions in mid-flow. So the sensor acts as the transceiver for the system.
The information gathered by the sensor is forwarded to the server, which is where the magic happens. The server program suite contains the analysis engine that will detect intrusion patterns. Intervention policies to block detected intrusions are also produced at the server. The actions required to protect the network are sent as instructions to the sensor.
The interface module of the system is a dashboard that displays events and alerts to the systems administrator. This is also where settings can be tweaked and defensive actions can be adjusted or overridden.
Samhain, produced by Samhain Design Labs in Germany, is a host-based intrusion detection system that is free to use. It can be run on one single computer or on many hosts, offering centralized data gathering on the events detected by the agents running on each machine.
The tasks performed by each agent include file integrity checking, log file monitoring, and port monitoring. The processes look for rootkit viruses, rogue SUIDs (user access rights), and hidden processes. The system applies encryption to communications between agents and a central controller in multi-host implementations. Connections for the delivery of log file data include authentication requirements, which prevent intruders from hijacking or replacing the monitoring process.
The data gathered by Samhain enables analysis of activities on the network and will highlight warning signs of intrusion. However, it will not block intrusion or clear out rogue processes. You will need to keep backups of your configuration files and user identities in order to resolve the problems that the Samhain monitor reveals.
One problem with hacker and virus intrusion is that the intruder will take steps to hide. This includes killing off monitoring processes. Samhain deploys a stealth technology to keep its processes hidden, thus preventing intruders from manipulating or killing the IDS. This stealth method is called “steganography.”
Central log files and configuration backups are signed with a PGP key to prevent tampering by intruders.
Samhain is an open source system that can be downloaded for free. It was designed along POSIX guidelines to make it compatible with Unix, Linux, and Mac OS. The central monitor will aggregate data from disparate operating systems.
Fail2Ban is a free host-based intrusion detection system that focuses on detecting worrisome events recorded in log files, such as excessive failed login attempts. The system sets blocks on IP addresses that display suspicious behavior. These bans usually only last a few minutes, but that can be enough to disrupt a standard automated brute force password cracking scenario. This policy can also be effective against DoS attacks. The actual length of the IP address ban can be adjusted by an administrator.
Fail2Ban is actually an intrusion prevention system because it can take action when suspicious activity is detected and doesn’t just record and a highlight possible intrusion.
Therefore, the system administrator has to be careful about access policies when setting up the software because a prevention strategy that is too tight could easily lock out bona fide users. A problem with Fail2Ban is that it focuses on repeated actions from one address. This doesn’t give it the ability to cope with distributed password cracking campaigns or DDoS attacks.
Fail2Ban is written in Python and it is able to write to system tables to block out suspicious addresses. These automatic lockouts occur in Netfilter, iptables, PF firewall rules, and the hosts.deny table of TCP Wrapper.
The attack monitoring scope of the system is defined by a series of filters that instruct the IPS on which services to monitor. These include Postfix, Apache, Courier Mail Server, Lighttpd, sshd, vsftpd, and qmail. Each filter is combined with an action to perform in the event of an alert condition being detected. The combination of a filter and an action is called a “jail.”
This system is written to the POSIX standard, so it can be installed on Unix, Linux, and Mac OS operating systems.
How to select an IDS
The hardware requirement of network-based intrusion detection systems may put you off and push you towards a host-based system, which is a lot easier to get up and running. However, don’t overlook the fact that you don’t need specialist hardware for these systems, just a dedicated host.
In truth, you should be looking at getting both a HIDS and a NIDS for your network. This is because you need to watch out for configuration changes and root access on your computers as well as looking at unusual activities in the traffic flows on your network.
The good news is that all of the systems in our list are free of charge, so you could try out a few of them. The user community aspect of these systems may draw you towards one in particular if you already have a colleague that has experience with it. The ability to get tips from other network administrators is a definitive draw to these systems and makes them even more appealing than paid-for solutions with professional Help Desk support.
If your company is in a sector that requires security standard compliance, such as a PCI, then you really are going to need an IDS in place. Also, if you hold personal information on members of the public, your data protection procedures need to be up to scratch to prevent your company being sued for data leakage.
Although it probably takes all of your working day just to keep on top of your network admin in tray, don’t put off the decision to install an intrusion detection system. Hopefully, this guide has given you a push in the right direction. If you have any recommendations on your favorite IDS and if you have experience with any of the systems mentioned in this guide, leave a note in the comments section below and share your thoughts with the community.
Comparitech networking guides
- Top 10 LAN monitoring tools for 2018
- The definitive guide to DHCP
- The definitive guide to SNMP
- The ultimate guide to mobile device management (MDM) in 2018
- The ultimate guide to BYOD in 2018
- Top 10 server management & monitoring tools for 2018
- The best free NetFlow analyzers and collectors for Windows
- 6 of the best free network vulnerability scanners and how to use them
- 8 best packet sniffers and network analyzers for 2018
- Best free bandwidth monitoring software and tools to analyze network traffic usage
Other information on network monitoring
- Wikipedia: Intrusion detection system
- Tech Target: Intrusion detection system (IDS)
- CSO: What is an intrusion detection system?
- Lifewire: Introduction to intrusion detection systems
- YouTube: Intrusion Detection Systems