Open Source Intelligence Tools – or OSINT tools – are not as intimidating as they sound. You see, we live in an age where the value of information, which is a commodity in its own rights, has continued to increase over time. Everyone, it seems, needs to have it.
And yet, surprisingly, the amount of information – about almost anything under the sun – that is available, to anyone who can be bothered to look, has also grown immensely. This is where the eight best OSINT tools we will soon see come into play as we learn to dig deep to uncover all this data.
Here’s our list of the eight best OSINT tools:
- OSINT Framework – a website directory of data discovery and gathering tools for almost any kind of source or platform.
- SpiderFoot – an OSINT tool to scrape data from over 100 data sources on personal, network, and business entities.
- Google Dorks – OSINT data gathering method using clever Google search queries with advanced arguments.
- Shodan – a search engine for online devices and a way to get insights into any weaknesses they may have.
- Maltego – an OSINT tool for gathering information and bringing it all together for graphical correlation analysis.
- Metasploit – a powerful penetration testing tool that can find network vulnerabilities and even be used to exploit them.
- Recon-ng – an open-source web reconnaissance tool developed in Python and continues to grow as developers contribute to its capabilities.
- Aircrack-ng – a wifi network security testing and cracking tool that can be used both defensively and offensively to find compromised networks.
What is OSINT?
OSINT – short for Open Source Intelligence – is the art of searching for, collecting, and summarizing information that is freely, and publicly, available on the Internet for the purpose of using it as a source of intelligence.
This public information can be about an individual, a business or corporate entity, a network, a nation, or any other source of relevant data. And, as the “open source” part of OSINT indicates, there is no need to employ sneaky or illegal tactics to obtain it.
After all, why would anyone want to resort to illegal activities when the data they need is freely available from Internet sources like websites, blog posts, social media platforms, search engine result pages (SERPs), and other public-facing digital assets, just to name a few?
Why would we need OSINT for business?
The scope of this article will be limited to a business and its network. The person doing the research is assumed to be an administrator trying to protect the network.
And so, as an administrator of a business network, the main reasons for using OSINT would be:
- Penetration testing: a great use for OSINT would be to gather all the information that is available out there and see if any of it can lead to an indication that your network has been compromised.
- Breach detection: if there is data out on the Internet that you didn’t share it could mean you have been hacked and have had data stolen. Monitoring the Internet using OSINT could give you an early start in damage control and even catch the people behind the data theft. Alternatively, it could simply be that a public-facing (or peripheral) device hasn’t been secured well enough and could be leaking data. Either way, an OSINT tool will give you a heads up.
- Ethical hacking: turn the table and gather information on a source-target; find out everything you can about competitors and use it to gain an insight into their way of doing things. Remember, as long as you abide by the OSINT ethical hacking rules, you will be on the right side of the law. Never cross that line – no matter how strong the temptation is.
- Chatter monitoring: use OSINT to listen to what is being said about you and yours. Perhaps you have a reputation to maintain, a brand to protect, or a network to secure. Monitor traffic and packets to see what is being directed your way; use the tools to find out all you can before an attack happens.
Finally; remember it isn’t just businesses that use OSINT. Governments and their agencies also use it to gather data on undeclared assets that belong to persons or organizations of interest, for example. With the right tools, a business can find out if there are any such probes aimed their way by simply looking at the searches, queries, and any network penetration attempts that are being made.
What types of OSINT do we have?
OSINT tools can be divided into three main categories:
- Discovery tools: are used to search for the information that is out there. A great example would be Google. Although it may seem like it is a simple search engine, there is really nothing simple about the information it can discover when an OSINT expert has a go at it, as we will soon see.
- Scraping tools: once discovered, the data must be “scraped” and collected somewhere safe. These tools make sure only the required data is filtered for extraction to avoid bulky transfers (which could alert the source) and also avoid unnecessary data that could muck up the information that is to be extracted from it.
- Aggregation tools: once the data has been stored safely, it needs to be mined and sifted through to convert it into usable These tools are used to combine related data bits into a larger picture and present it in a way that will show relations and connections between datasets and bring it all together in a consumable format.
Of course, there are tools that have all three functionalities included in one package.
OSINT gathering tactics
There are three methods of OSINT intelligence gathering:
- Passive: this is the “normal” way of digging for information; usually done by scouring the web with applications like Google search, Bing Maps, and Yandex images. This method is hard to detect as no probing is involved and only archived information is collected.
- Semi-Passive: here too, scouring the Internet is involved, to find the data; but software solutions are also involved to non-intrusively gather information about a network, for example, and send the data off to collection servers. No brute force attacks or in-depth querying is involved.
- Active: in this scenario, the information is collected by directly extracting it from the target; although no malicious software is involved in breaching their security. Remember, although it is publicly available, just sitting unprotected on their servers and networks, it could still be perceived as hacking. This type of probing can be detected because it involves scanning of networks to find open ports, for example. Once the data has been discovered, the next step involves getting it into storage servers for further analysis.
This brings us to the point where we have to warn you about using OSINT tools without hiding your identity. Always assume that your target will find out about the intelligence probe and might even try to go after you – legally or otherwise. Learn how to hide your identity by using VPNs, fake accounts, and TOR, and other anonymity tactics.
What kind of information can you gather with OSINT?
To be honest, you could probably extract any information that is in digital format. There is no such thing as a secure online presence. Once a device is exposed to the Internet, someone, somewhere, could probably find a way to it.
The only truly secure system is one that is powered off, cast in a block of concrete, and sealed in a lead-lined room with armed guards – and even then I have my doubts. – Gene Spafford
Search engines like Google can give you insights into data that is not only shared on the web but also, with the help of advanced arguments, allow you to delve deeper to find files and information that hasn’t been shared intentionally.
Then again, using tools like Google Earth, you can see some of the remotest parts of the planet and even accidentally uncover state secrets. On the other hand, you can also catch live events with the help of unsecured security cameras, unprotected CCTVs, and even a Google mapping car.
To put it all in perspective: all that is needed to start finding information on a person is a single phone number. Once we have that, it is easy for anyone to build an OSINT tool – from scratch – that can extract information like name, location, and social media account details which can then be used to dig further for even more personal or financial information.
The Best OSINT tools
Before we begin, we need to remind you: the information provided in this post is for informational purposes only; you – and only you – will be held responsible for any misuse of said information.
With that in mind, and without further ado, here are the eight best OSINT tools:
This is perhaps one of the most popular OSINT tools out there. The thing is that OSINT Framework is more of a website with a directory of tools rather than just one single tool. And, it is perhaps this ability to find all the tools you may need to dig up all the information on a target, in one place, that makes it the go-to option for information gathering.
Another reason this is a popular collection is that many of the best OSINT tools are written or created for a Linux environment. This directory, meanwhile, has many tools that can be run from a browser and, even when the installation is needed, there are options for most major operating systems.
The collection of OSINT tools can help dig up information using anything from a simple telephone number, IP address, or email addresses. There are even options for venturing into the Dark Web or the ability to analyze malicious files. So, proceed with caution.
There are tutorials and games included to get beginners started with the digging-for-information game. Need a VM for a research campaign? You can find a list of software solutions under “Virtual Machines.”
Almost all of the tools that are linked to an OSINT Framework are free while the few remaining ones might ask for a small subscription fee.
With SpiderFoot we have a tool that can be downloaded and installed on a Linux or Windows machine and also a cloud version that can be run from a browser.
SpiderFoot HX – the free online version – offers five scans per month. Once the target has been selected (or a bunch of them in the forms of domain names, user names, IP addresses, etc.) the tool starts to run a major scan that automatically starts to dig for troves of information from over 100 public data sources.
Once the scan is complete, an alert is sent via email and the report is immediately available in a dashboard that is easy to navigate and drill into. It integrates with other apps like Slack, which makes it an even more powerful tool as more than one person can collaborate on the project.
The data it returns includes the number of compromised passwords, SSL certificate details, hosting information, physical addresses related to the input parameter, and so much more.
In fact, this tool can be used offensively to gather some in-depth information on a network and its users. And, as if that weren’t enough, the source code of the whole project is freely available for anyone who wants to tweak it to meet their specific needs.
Anyone who takes Google’s search capability for granted, or underestimates the power that lies behind this search engine’s capability to dig deep and come up with some interesting information, is a fool.
With the right arguments, anyone can find files or documents that may seem securely stored. In fact, one of the first things to do during a penetration test is to use Google Dorks to see what can already be accessed without any data mining tools.
As you may have understood, Google Dorks is not a tool, per se. It is a data querying method that involves querying for information using advanced – and clever – search arguments in Google Search.
Here’s how it works: websites are automatically indexed when Google bots crawl them. Now, unless sites with sensitive data or folders specifically block the bots (using noindex meta tags), their contents will be made available as search results for specific Google queries.
The concept here is to enable any user to delve deep into a server’s annals to come up with data corresponding to various arguments. The beauty of it is that Google has a large list of arguments that can address queries for almost any type of data including usernames and passwords.
There is no one website to go for the ultimate compilation of clever Google syntaxes; that means you will need to do a Google search for that too. But, for your reference, we have one of the most popular Google Dorks sites: Google Hacking Database on Exploit Database. Enthusiasts from all over the world update this registry daily.
Again, be aware that this is a powerful OSINT tool that can uncover sensitive information that could get you in trouble simply because you downloaded, or even looked at it.
Shodan is a querying digital intelligence gathering tool. It is a search engine that can be used to find information on IP addresses, ports, and any Internet-connected devices. It can be used to gather information on servers belonging to businesses or even cities, for example.
To start using it, simply type in any business and you get information on the devices that the business uses including honeypot ICS, location, services (HTTP, etc.), and even any vulnerabilities the devices might have.
The results are grouped by network names or IP addresses. Host information includes what operating systems are being used, open ports, type of Internet server, website design language, and much more. Classless Inter-Domain Routing (CIDR), or IP range, network scanning for bulk information is also possible.
Some queries may only work for the US – but, there are plenty more tools that help search for information from the rest of the world. You can start by typing in a query for a country to get the number of unique IP addresses they have registered.
Using this tool becomes a breeze once you have learned the Shodan syntax which is similar to Google Search. For example, querying for “Org: Organization_Name” gives you the information related to the devices that belong to an organization.
With such commands, users can run a query to list open surveillance or web cameras and even grab snapshots from them.
Although the main purpose of this tool is for reconnaissance, some commands can be actually used to perform penetration testing. In the right hands, this is a powerful tool that can lay bare the weaknesses of a network.
This OSINT tool is helpful in finding information on individuals as well as organizations. It can run on Linux, Windows, and macOS.
Although you need to register with Maltego Community to start digging for information, which is a mighty tool as it is, you can also buy the premium version for even more advanced features.
Once signed in, you get a “Graph” window where you do your research. The query results are displayed in the form of a bubble graph that shows the relations of each “transform” results – as Maltego query scripts are known.
To start the information gathering process, you first enter the main entity you are researching – an individual, organization, phone number, etc. – and run the available transforms to see the results. For example, it can be used to map networks to see how the servers on it are linked and if, perhaps, they have been compromised. The resulting information can be filtered or further “transformed” for even more in-depth data analysis.
Although this tool is very easy to use, as you simply start from one piece of information and start to progressively build on it, it is also very powerful and never disappoints in its result delivery.
There is nothing shy about this tool; on the contrary, it is a bold weapon that can be used to get all the required information on a target – be it a host or a network – and then exploit any vulnerability that may have been discovered. This is usually done by sending out a payload that executes commands.
With Metasploit, users can upload, download, listen to, or alter files they have found. In the case of mobile devices, they can even capture screenshots and activate the camera and microphone for remote eavesdropping.
This is a no-nonsense tool that can cause real damage – and get you in trouble – if abused. It has seven modules that can be used for different intelligence gathering campaigns: auxiliary, payloads, evasion, encoders, exploits, post, and NOP.
These modules tackle specific issues like getting past defenses (encoders), running scripts, and code by exploiting buffer overflows (NOP), or performing tasks after compromising a system (post), for example.
Once someone has access to a system, they can practically own every single device on it. The scary thing about this OSINT tool is that it can deliver payloads to devices running almost any type of operating system out there: Windows, macOS, Linux, Android, and many more.
Metasploit, itself, can be run from Linux, macOS, and Windows.
Here is another tool that is great at getting information from open, public records. Although the interface could appear to be a bit daunting at first – because of the CLI – it really is an easy tool to master after spending a few days playing around with it.
On the contrary, anyone that is proficient at working in a Unix/Linux environment will find this to be a familiar tool.
Recon-ng has default modules that are also open source, and then there is a marketplace to add even more features. And because it is an open-source tool, it continues to evolve and grow as the developer community continues to contribute to it. Written in Python, Recon-ng is designed exclusively for web-based open source reconnaissance. Therefore, it can’t be used for exploits.
But, still, once the information has been collected, it is stored in a database which can then be used to generate insightful custom reports.
Aircrack-ng is a wireless network security penetration testing tool that has four main functions:
- Packet monitoring – capturing of frames and collecting WEP IVs (Initialization Vectors); if a GPS is added, it can log the position of APs (access points).
- Penetration testing – by performing packet injection attacks, fake access points, replay attacks, and more to test a network’s security.
- Performance analysis – testing wifi and driver capabilities.
- Password security testing – password cracking on WEP and WPA PSK (WPA 1 and 2).
Although the tool was developed primarily for Linux, there are versions for Windows, OS X, and FreeBSD. The fact that it is a fully CLI tool means that it can be easily tweaked to meet unique requirements using custom scripts.
Here are a selection of tools that can further enhance the performance and reach of the tools we have seen above:
Wireshark – this popular free, open source packet sniffing tool is one of the best penetration testing applications that lets you see if there are any unprotected protocols like FTP, Telnet, and SSH travelling in a network.
Nmap – this is another popular “old-timer” that is still used to keep an eye on network security; it can be used for discovery or testing purposes to see host statuses and gather information like shared data, operating systems, and much more to uncover vulnerabilities. As time goes by, it has gotten more powerful and now has a GUI (Zenmap).
PhoneInfoga – squeeze as much information as possible from a phone number; this tool works globally, for phone numbers from across the world. The only catch: it needs Python.
TinEye – in a world where the problem of fake news is being exacerbated with expert Photoshop manipulations, this reverse search engine uses image identification instead of keywords or metadata. It is a simple, browser-based tool.
We would like to hear about other OSINT tools you use or think should be on this list. Tell us about them; leave a comment below.