What is Open Source Intelligence?

Table of content

Open Source Intelligence, or OSINT for short, refers to the collection of information from public sources to be used in the context of intelligence. In the "world of the Internet" there are many states whose impact on our lives has both advantages and disadvantages. Originally, the term OSINT originated in the military domain and referred to the strategic collection of publicly available information and its analysis to gain actionable intelligence. The scope of OSINT is not limited to cybersecurity, but also to corporate, business, and military intelligence or other areas where information is important.

To begin defending against cyber attacks, it is important to understand the information that can be collected about your organization from public sources. The dark web and OSINT collection do just that.

Comprehensive OSINT techniques identify an organization's IP addresses, ports, sub-domains, services, technologies used, sensitive data and credentials to map its attack vectors.

What are the Types?

The reconnaissance types in an OSINT analysis can be divided into two categories, passive and active.

Passive methods are those that do not require interaction with the target systems and therefore cannot be detected.

In active data collection, analysts interact with target systems, which may involve the use of advanced techniques or simple interactions such as registering on an organization's website to obtain materials available only to registered users.

The active methods are detectable by security mechanisms such as IPS/IDS, so it is necessary to be much more careful here to avoid detection.

While analysts must apply specific OSINT techniques tailored to the needs of the target organization, following the general approach helps lay the groundwork for effective information gathering.

The Procedure

Identifying Sources

The first phase involves identifying sources from which information can be obtained. The sources are documented internally according to the predefined processes in order to capture enough details to refer to when needed.

Collecting Data

The security analyst gathers information from various sources such as social media, search engines, OSINT tools, deep and dark web, and other relevant sources pertaining to the target organization.

Data Processing and Integration

Our experienced security analysts process the collected information into actionable insights by searching for information that can help in enumerating.

Data Analysis

Using effective OSINT tools, the security analyst performs data analysis on the processed information.

Delivery of results

Once the OSINT analysis is complete, the results are reported and provided to the customer or Red Team for use or presented in an adversarial simulation.

The best OSINT Tools

Maltego

Maltego was developed by Paterva and is a tool in Kali Linux (ships with Community Edition). Maltego helps to perform extensive reconnaissance against targets using several built-in transformations. To use Maltego, the user must first be registered on the Paterva website.

After registration, the user can create or launch a new machine to run transformations on the target. Once configured, the machines must be started. There are several footprints built into Maltego that can be run against the target. Maltego will start executing all transformations with the Maltego servers.

The expected results could be that the domain to IP conversion has taken place, the network block is identified, the AS number is also identified, the locations and other phrases are also identified. These are all icons in Maltego, and there is a detailed view of all of them. Researchers can continue this process to get more information about the target. An absolutely recommended tool to track the footprints of a single entity on the Internet.

Recon-Ng

Recon-Ng is another useful tool for conducting reconnaissance and is also integrated with Kali Linux. Recon-Ng has several modules built in, and its use is somewhat similar to Metasploit.

Workspaces can be created to perform any operations within them. Once the workspace is created, the user is redirected to that workspace. Once the user is in the workspace, the domain can be specified with add domain "domain name". After the domains are added in recon-ng, recon-ng modules can be used to extract information about that domain. There are some excellent modules such as bing_domain_web and google_site_web to find additional domains related to the original target domain. The output of these domains are all indexed domains to these search engines. Another handy module is bing_linkedin_cache, which can be used to find the email addresses associated with the domain, which can then be used for social engineering. So, with other modules, we can get additional information about targets. Thus, recon-ng is a great tool and should not be missing in the toolbox of researchers.

theHarvester

theHarvester is again a great tool for gathering information about the specified target. theHarvester is built into Kali, is very fast and much easier to use than recon-ng to gather basic information. It can be observed trying to fetch results from Google, Bing, PGP key servers, etc.

Below are the details that we can get from theHarvester: Email address related to the domain and results from hosts and virtual hosts found in search engines.

So we can see that theHarvester is also very useful for extracting information from the specified targets and is very useful with all its features.

Shodan

Shodan is touted as the "search engine for hackers" because it provides a huge footprint of devices connected online. It is a gold mine for researchers to see exposed devices.

For example, one can see the connected webcams, netcams, traffic lights, etc. Below are some of the use cases of Shodan: Testing "Available Assets" with RDP port open, "Default Passwords" and Assets with VNC viewer.

So Shodan is an excellent tool for finding the fingerprint of connected assets. Researchers can easily imagine how far they can go with it to gather information in depth.

Google Dorks

Search engines provide us with a lot of information, and they also index a lot of information that can be used to gather information about a target. Google Dorking provides such information by using some operators that are otherwise difficult to extract by simple searches. Below are some of the operators used in Google Dorking:

(title): Searches for mentioned words in the page ent

Searches for mentioned words in the URL.

Filetype: This is used to find file types.

Ext: This is used to identify files with specific extensions. Remember to use it to search for such files as .log, which should not be indexed.

Intext: This helps to search for a specific text on the page.

Example based on E-mail Address

Using this flow as an example, we will see what other information you want to grab if you have an email address. Important information that one wants to grab is information about the person behind it, this includes the full name of the owner of the email address as well as the employer. Additionally, one tries to find social media accounts as well as other usernames that are related to the email address. Furthermore, it is also important to check whether this email address is already included in password leaks.

Figure 1: IntelTechniques.com OSINT Workflow Chart: Email Address

As you can see in the picture, the goal is to find out as much information as possible about the person behind the email address in order to use this information for further OSINT research or to attack this person directly if you have enough information.

More flows on parameters like domain, username, etc. can be found in the following Github repo - https://github.com/willc/OSINT-flowcharts

What can You Do with the Information?

Basically, gathering information is usually the groundwork for later cyberattacks, such as:

Social engineering, e.g. phishing and email spoofing.
Building botnets to carry out distributed denial of service (DDoS) attacks.
Brute force attacks
Doxing
Injecting malware, such as spyware, ransomware and other types of malware.

This, among other reasons, makes it important for organizations to have effective information risk management in place to address OSINT misuse. Especially in the event of public data leaks and software vulnerabilities, processes should be in place to respond to them immediately.

Conclusion

In summary, it's hard to stay private and control all the information floating around the web these days. You can't control everything that circulates about you, but it is important to at least be aware of it. It goes without saying that in the digital age, information plays a key role, and those who know how to find it will always be one step ahead. That thought alone should be enough to drive OSINT measures in companies to mitigate cyber risks.