Investigating, Authenticating, Disseminating: The Challenges of Open-Source Intelligence for the U.S. Government and Solutions to Address It
In the wake of Russia's invasion of Ukraine, the need for improved open-source intelligence (OSINT) in US information warfare has become increasingly apparent. A new report, authored by Major Brian Cheng, US Army Reserve, Major Scott Fisher, US Army Reserve, and Major Jason C. Morgan, US Army Reserve, outlines key areas for improvement.
The authors highlight the challenges faced during the process, including navigating the messaging-related restrictions of the Geneva Conventions, and the collection, vetting, and analysis of content from various channels. The US European Command Information Operations and Special Activities Division was tasked with developing response plans for the invasion, and the task shifted to operationalizing those plans once the conflict began.
However, a fraction of the required content was collected due to high demand for intelligence capabilities during the war. To address this issue, the report suggests several improvements.
Firstly, the development of advanced data collection technologies is crucial. US OSINT must counter increasingly sophisticated anti-scraping measures like honeypots and IP blocking used by adversaries. This requires more resilient proxy networks and adaptive scraping tools.
Secondly, real-time processing and AI integration at the tactical edge is essential. The ability to rapidly convert vast, multi-domain data streams into actionable intelligence is critical. This requires ruggedized edge computing platforms combined with AI models designed for autonomous operation even with limited connectivity, reducing latency in intelligence delivery to warfighters.
Thirdly, improved AI-driven decision-making models are necessary. Moving beyond traditional sequential frameworks such as the OODA loop, integrating AI-powered “Matrix Operations” can enable simultaneous multi-domain situational awareness and faster coordination, essential for irregular warfare and information operations in conflicts like Ukraine.
The report also emphasizes the need for policy and security frameworks for AI and OSINT. Ensuring that AI tools used in intelligence gathering adhere to strict data protection, record management, and transparency requirements preserves operational security and public trust.
Expanded collaboration via information sharing is another key area. Open data sharing among allied and private sector partners enhances OSINT’s scope and depth, addressing cyber threats and misinformation campaigns by harnessing broad public and commercial datasets, including the dark web.
The Russia-Ukraine conflict underscores OSINT’s value in exposing espionage networks, tracking misinformation, and analyzing real-time battlefield developments. However, it also highlights the need for more adaptive technology to overcome adversarial countermeasures, faster AI-enabled intelligence processing close to the front lines, and enhanced multi-domain coordination models to stay ahead in the information war.
Together, these improvements will strengthen US capabilities to match and exceed the sophisticated information warfare strategies seen in Russia’s invasion of Ukraine. The authors advocate for the adaptation of the federal government to designate an organization for the role of open-source lead, and suggest establishing a new open-source intelligence agency or a model similar to the National Counterterrorism Center or the CIA's former Foreign Broadcast Information Service or a better-resourced Open Source Center/Enterprise.
The content was rarely in English, requiring tools and people who understand target languages. Vetting open-source content to determine authenticity was a major challenge. Despite these difficulties, the authors express their frustrations came from trying to solve problems that have been known for years and highlighted in multiple studies.
In a post-Covid world of work-from-home preferences, this organization may have recruiting advantages compared to organizations working principally with classified material. The authors recommend that to avoid the problems outlined, an organization needs to be tasked to collect, analyze, and share open-source content as a primary function.
- The new report emphasizes the importance of advanced data collection technologies in the realm of US open-source intelligence (OSINT), given the increasing use of anti-scraping measures by adversaries.
- Rapid real-time processing and AI integration at the tactical edge is critical for the rapid conversion of vast, multi-domain data streams into actionable intelligence.
- Improved AI-driven decision-making models are necessary, moving beyond traditional sequential frameworks, to enable simultaneous multi-domain situational awareness and faster coordination.
- The report stresses the need for policy and security frameworks for AI and OSINT, ensuring that AI tools adhere to strict data protection, record management, and transparency requirements.
- Expanded collaboration via information sharing among allied and private sector partners is key, harnessing broad public and commercial datasets for addressing cyber threats and misinformation.
- The authors propose the designation of a federal organization for the role of open-source lead, and suggest the establishment of a new OSINT agency or a model similar to the National Counterterrorism Center or the CIA's former Foreign Broadcast Information Service.
- In the post-Covid world of work-from-home preferences, this organization may have recruiting advantages compared to organizations working principally with classified material, and should be tasked to collect, analyze, and share open-source content as a primary function.
