Overcoming Common OSINT challenges

Open source investigations (OSI) have become increasingly important and commonplace in various sectors, primarily journalism, intelligence agencies, research, and law enforcement.

However, investigators face several challenges when conducting an OSI, which can hinder the effectiveness of their investigations.

In this post, we will explore some of the most common challenges OSI practitioners face and provide strategies for overcoming them.

Challenges of Open Source Investigations

Carrying out an OSI can be complex and time-consuming, requiring careful attention to detail and a willingness to pursue leads and follow the evidence wherever it may lead.

Even the most skilled and dedicated investigators can face various challenges when conducting investigative research. Here are some of the most common:

Challenge #1: Information Overload

One of the primary challenges OSI practitioners face is the sheer amount of data available online.

As we know, a big part of OSI relies on leveraging Open Source Intelligence (OSINT), which is extracting Publicly Available Information (PAI) from several sources, analyzing it, and generating actionable intelligence from it.

This process is easier said than done. Within the PAI, there are various sources that deliver vast volumes of data in different formats.

Sorting through it to find relevant and reliable information can be challenging. It can often feel like you are drowning in data and don’t know how to stay afloat.  To overcome this challenge, investigators should use specialized tools and techniques to manage data.

For instance, data visualization tools can help you gain a bird's eye view of the information you have and help you understand the bigger picture and, by extension, determine where you ought to dig deeper.

Another tip would be to use Automated data processing tools to filter irrelevant information and focus only on relevant signals.

Challenge #2: Managing Diverse Types of Information

As we mentioned, Open source data comes in various formats, including text, images, and videos. This inconsistency in the data makes it challenging to compare and analyze information across multiple sources.

To overcome this challenge, investigators should establish an effective organization and structure for data management by creating a shared data architecture that makes sense for your purposes and using a notation system for tracking and handling information.

This last step is instrumental when it comes time to look at information security management (we will touch upon that in a bit).

Overall, the idea is to look for common dimensions in the data that can be used as a base, whether that’s time, space (geolocation), a set typology of event categories, or whatever is most sensible for your research.

Challenge #3: Maintaining Accuracy and Reliability

Accuracy and reliability are critical in OSI. Investigators must ensure that the information they collect and analyze is accurate and reliable, as the insights they generate must be defensible to affect desired change.

To overcome this challenge, investigators should verify sources and use corroborating evidence. This means checking the authenticity of the source and cross-checking the information with other sources to ensure accuracy and reliability.

It's critical here to have a shared, systematized approach to source validation and evaluation and to, ideally, be able to count on the support of data fusion tools that facilitate the corroboration process.

Having a clear audit trail of how the conclusions were reached and who contributed to the assessment within the team will save you big headaches down the line when questions or requests for clarifications arise further down in the information supply chain.

Challenge #4: Difficulty Collaborating

OSI often involves multiple investigators and stakeholders, requiring effective collaboration and communication. Collaboration can be challenging, mainly when investigators work remotely.

To overcome this challenge, investigators must establish clear communication channels and protocols for collaboration.

They can use collaborative tools such as Slack to communicate and share information effectively but must be mindful of security.

Challenge #5: Security and Privacy

OSI can pose significant risks to security and privacy, particularly when handling sensitive information. Investigators must take precautions to mitigate these risks.

To overcome this challenge, investigators should use secure platforms for data collection, storage (and sharing), analysis, and reporting. They should also follow best practices for data protection, such as end-to-end data encryption (in transit and at rest), multi-factor authentication, and role-based access controls (RBAC).

The last point is critical. From a cyber security standpoint, you can have the most impenetrable fortress in the world, but if someone inadvertently opens a door from the inside, malicious actors can find a way in.

Well-designed RBAC protocols ensure that users only have access to what they need to perform their work, not exposing other information they don’t need in the first place to unnecessary risk.

Challenge #6: Technology Integration

OSI often require multiple specialized tools and software to collect, process, and analyze the raw data they need to work with. However, using multiple tools can be challenging and time-consuming.

We have worked with many clients that operate legacy systems that are interconnected by fragile links that can quickly break whenever a node in the network suffers any change (planned or not).

To overcome this challenge, investigators should prefer tools that cover all aspects of the information supply chain (from collection to storage to analysis and reporting), thereby reducing integration risks and saving you information security control headaches, time, and money.

Challenge #7: Efficient Use of Time and Resources

Some data management processes can be time-consuming, taking the focus from core research activities. This can be a significant challenge for investigators who have to balance multiple tasks, often against the clock.

To overcome this challenge, investigators should automate repetitive tasks such as manual data collection and basic analytical processing using automated workflows, extract-transform-load (ETL) processes, and Artificial Intelligence (AI).

Conclusion

OSI is an increasingly important arrow in the quiver for various industries but comes with several challenges.

The seven challenges discussed in this article include information overload, managing diverse types of information, maintaining information accuracy and reliability, collaborating effectively, protecting security and privacy, dealing with technology integration issues, and ensuring efficient use of limited resources. 

To overcome these challenges, investigators must :

• Use specialized tools and techniques for data management,
• Establish an effective organization and structure for data management,
• Verify sources and use corroborating evidence,
• Establish clear communication channels and protocols for collaboration,
• Use secure platforms for data collection and analysis that play well with each other,
• Automate repetitive tasks.

By following these strategies, open source investigators can overcome these challenges and make the most of the open source investigation process for better results.

How can AKTEK iO Help with OSI

Leveraging multiple years of experience servicing investigative teams around the world, the AKTEK iO platform can help investigators overcome OSI challenges in multiple ways:

• AKTEK iO can ingest and facilitate data collection from various sources, including social media platforms, public government records, anonymous phone data, natural hazard and geopolitical event data, web traffic data, and more.
• It allows investigators to centralize their data from multiple sources on a shared architecture and provides advanced search capabilities to find and retrieve relevant information for their research purposes quickly.
• AKTEK iO helps reduce the risk of errors that can occur with manual data entry and ensures that the data is consistent and kept freshly updated across the platform, leveraging a relational database where everything is interlinked.
• Investigators and analysts can work collaboratively and securely on complex, multi-source research projects in real-time with AKTEK iO. Its case management system also helps keep everyone on track, and the information produced is systematized for maximum impact on the output when communicated to the consumers.
• AKTEK iO lives up to and exceeds your security and data privacy management expectations by not only offering encryption in transit and at rest, multi-factor authentication, and everything else you have come to expect but also offering micro-permissions for role-based access control, ensuring complete control over who can see or do what with your organization’s information.

From monitoring war crimes to investigative work in illicit money flows, our clients benefit from AKTEK iO’s end-to-end data management system for research and investigation, allowing them to focus on what they do best, surfacing evidence to hold individuals, organizations, and governments accountable.