.png)
The history of OSINT extends beyond the emergence of digital technologies and the Internet. During the Cold War era, OSINT gained significant importance as an intelligence discipline, particularly for gathering intelligence on the Soviet Union and China.
After the Cold War, OSINT's potential and range were further enhanced by major global developments in technology, commerce, and politics. The proliferation of media publications, the invention of television, and the arrival of the Internet, in particular, have all expanded and diversified the intelligence community's access to open sources.
In today's digital world, information is more accessible than ever before. However, with the vast amount of information available, it can be difficult to find the relevant data required for intelligence gathering purposes. Open Source Intelligence (OSINT) is a method of collecting, analyzing and disseminating information from publicly available sources to generate valuable intelligence. This article will explore what OSINT is, how it is used to gather information, who uses OSINT and for what purposes.
What is OSINT?
OSINT is a type of intelligence gathering that uses publicly available information to generate intelligence. This information can be gathered from a variety of sources, including social media, news articles, public records, and other online resources. OSINT is considered to be one of the most valuable sources of intelligence gathering as it can provide a wealth of information that is not available through other means.
How to gather information using OSINT?
Gathering information using OSINT requires a specific set of skills and techniques. There are various steps involved in the OSINT gathering process, which are outlined below.
Define the objectives: The first step in gathering information using OSINT is to define the objectives of the intelligence gathering. The objectives should be specific, measurable, achievable, relevant, and time-bound (SMART).
Identify the sources: The next step is to identify the sources of information. This can include social media platforms, news websites, public records, and other online resources.
Collect the data: Once the sources have been identified, the data can be collected. This can be done manually or through the use of specialized tools designed for OSINT gathering.
Analyze the data: After the data has been collected, it needs to be analyzed to extract the relevant information. This can involve using data visualization tools, data mining techniques, and other analytical methods.
Disseminate the information: The final step is to disseminate the information to the relevant stakeholders. This can include government agencies, law enforcement, and other organizations.
OSINT Methods:
There are various methods used to gather OSINT information. Some of the most common methods are:
Internet Searching - The internet is a vast source of information, and search engines can be used to find information on almost anything. Using advanced search techniques, such as Boolean operators, can help to narrow down search results and find more relevant information.(Google, Bing, Yahoo, Wayback Machine, Whois)
Social Media Monitoring - Social media platforms such as Twitter, Facebook, and LinkedIn are rich sources of information. By monitoring social media accounts, it is possible to gain insights into the activities, interests, and relationships of individuals and organizations.
Data Mining - Data mining involves using software to analyze large datasets and extract useful information. This can be done on websites, social media platforms, and other sources.
Public Records Search - Many countries have public records that are available to the public. These records can include information on property ownership, criminal records, and other legal proceedings.
Human Intelligence - Human intelligence involves gathering information through personal interactions with individuals. This can include interviews, surveys, and other methods.
Geospatial Analysis - Geospatial analysis involves using geographic information systems (GIS) to analyze spatial data. This can be used to identify patterns and relationships between locations and events.
Image Analysis - Image analysis involves using software to analyze images and extract useful information. This can be used to identify objects, people, and locations.
Deep web: The deep web consists of any non-indexed web pages (sites that are not reachable by internet search engines).
Dark web: The dark web is only accessible through darknets. Darknets can be small peer-to-peer or friend-to-friend networks, as well as large networks like Tor and I2Ps. Many sites on the dark web host illegal content.
Who is using OSINT and for what purposes?
OSINT is used by a wide range of organizations and individuals for various purposes. Some of the most common users of OSINT include:
Law Enforcement: Law enforcement agencies use OSINT to gather intelligence on criminal activities. OSINT can provide valuable information about the location, movements, and activities of criminal organizations.
Military: The military uses OSINT to gather intelligence on potential threats, including terrorist organizations and hostile nations. OSINT can provide information on the capabilities, intentions, and movements of these groups.
Intelligence agencies: Intelligence agencies use OSINT to gather intelligence on a wide range of topics, including political, economic, and military issues. OSINT can provide valuable insights into the intentions and capabilities of foreign governments and organizations.
Private sector: The private sector uses OSINT for a range of purposes, including due diligence, competitive intelligence, and brand management. OSINT can provide information about the reputation, financial stability, and strategic plans of other businesses.
OSINT Techniques
OSINT reconnaissance (recon) techniques fall into one of two main categories: passive and active.
Passive recon involves gathering information about a target network or device without directly engaging with the system. OSINT analysts rely on third-party information using passive recon tools, such as Wireshark, which analyzes network traffic in real-time for Windows, Mac, Unix, and Linux systems. They piece together these different OSINT data points to find and map patterns.
Active recon directly engages with the target system, offering more accurate and timely information. OSINT analysts use active recon tools like Nmap, a network discovery tool that provides a granular view of a network's security.
Targets are more likely to notice active scanning as intrusion detection systems (IDS) or intrusion prevention systems (IPS) can detect attempts to access open ports and scan for vulnerabilities.
While information security teams need to adopt unique OSINT techniques specific to their organizational needs, following a general process helps lay the foundations for effective intelligence gathering.
The Open Web Application Security Project (OWASP) outlines a 5-step OSINT process:
Source Identification
Determine where to find the information for the specific intelligence requirement.
Harvesting
Gather relevant information from the identified source.
Data Processing
Process the identified source's data and extract meaningful insights.
Analysis
Combine the processed data from multiple sources.
Reporting
Create a final report on findings.
Listed below are some useful open source intelligence tools.
Babel X is a multilingual Internet search tool that finds publicly available information from sources like social media, forums, news sites, and blogs across 200 different languages. It filters relevant information into different categories for OSINT analysis.
BuiltWith is a website profiling tool that shows current and historical information about a website's technology usage, technology versions, and hosting.
Creepy is an open source intelligence gathering tool that collects geolocation information through social networking platforms.
DarkSearch is a dark web search engine that allows organizations to research and access sites directly through Tor2Web.
GHunt is an OSINT tool used to find data associated with Google accounts, including account owner name, Google ID, YouTube, and other services like Photos and Maps.
Google Dorking
Google Dorking, also known as a Google Dork, involves using advanced search queries to find security and configuration information about websites.
Greg.app is a search engine that searches code from public repositories on GitHub.
Intel Owl is an OSINT tool that gathers threat intelligence data about a specific file, an IP, or a domain through a single API request.
Intelligence X is a search engine and data archive that Searches Tor, I2P, data leaks, and the public web by email, domain, IP, CIDR, Bitcoin address, and more.
Maltego is an OSINT and graphical link analysis tool for gathering and connecting information for investigative tasks.
O365 Squatting is a Python tool used to check inputted domains against O365 infrastructure to identify typo-squatted domains that do not appear in DNS requests.
The OSINT framework is an online directory that lists open source tools for OSINT gathering, sorted by source type.
reNgine is an automated reconnaissance framework used for OSINT gathering that streamlines the recon process.
Recon-ng is an open source intelligence gathering tool used to conduct web-based reconnaissance.
Searchcode is a source code search engine that indexes API documentation, code snippets, and open source (free software) repositories.
Shodan is a search engine used for gathering intelligence information from a variety of IoT devices like webcams, routers, and servers.
Social Mapper is an OSINT tool that uses facial recognition to correlate social media profiles across different sites on a large scale.
Spiderfoot is a reconnaissance tool that automatically queries over 100 public data sources (OSINT) to gather intelligence on IP addresses, domain names, email addresses, names, and more.
Sublist3r is a python tool designed to enumerate subdomains of websites, using search engines such as Google, Yahoo, Bing, Baidu, and Ask.
theHarvester is a penetration testing tool used to gather information about emails, subdomains, hosts, employee names, open ports, and banners from different public sources like search engines, PGP key servers, and SHODAN computer database.
TinEye is a reverse image search engine and image recognition tool.
Zmap is a network tool used for Internet-wide network surveys.
Conclusion
OSINT is a valuable method of intelligence gathering that can provide a wealth of information from publicly available sources. The OSINT gathering process involves defining objectives, identifying sources, collecting data, analyzing data, and disseminating information. OSINT is used by a wide range of organizations and individuals, including law enforcement, military, intelligence agencies, and the private sector, for various purposes. With the increasing amount of information available online, OSINT is becoming an increasingly important tool for intelligence gathering.
Very interesting ! I am very interested in using technnology for "documentation".