Photo by https://unsplash.com/@marjanblan

BACKGROUND

Artios partnered with Grey Dynamics (GD), specialists in business intelligence and Open-Source Intelligence (OSINT) techniques, to map evidence of progress observed in the Mine Action sector. The aim of this open-source change mapping study was to collate and review evidence of strategy impact and outcomes rather than focus on outputs of the client’s work.

The diagram below (Belcher et al 2020[1]) demonstrates the relationship of control to influence and interest against outputs, outcomes and impact. This is a good framework to consider evidence and communication of progress as well as manage expectations of how much resolution or confidence should be sought at each level.

Developing an OSINT approach to outcome mapping

Artios and GD used a hybrid model of systematic review and OSINT analysis, with two concurrent and independent reviews. These reviews asked two questions about available evidence: 1) What does the data say? and 2) Where could evidence be and what could it be saying?

To answer the question, ‘What does the data say?’, GD leveraged OSINT techniques in a systematic search taking a ‘clean look’ at publicly available data. To dial in on searches GD used tools such as Google Dorks and analysts considered factors such as frequency of key words, connectivity of certain sites, strength of ‘voices’, and types of evidence that trend.

The question of, ‘Where could evidence be and what could it be saying?’, was led by a Mine Action specialist from Artios who ‘knew where to start looking’ to build a well-informed Mine Action focused source list via Google and academic journal searches as well as accessing bibliographies from key Mine Action publications.

The final attribution of linkages of evidence to client interventions (i.e. contribution, correlation, and causation) were not in scope.

Recommendation: Use OSINT within an evidence informed decision making approach

Open-Source Intelligence (OSINT) review is considered a useful tool with potential for tracking progress and change in Mine Action as part of a wider results framework. OSINT/Business Intelligence techniques are not a panacea as was found in this study. It is best to harness any big data (what and how) insights by combining with thick data (why and context), as well as follow on analysis for quality and significance by informed human analysts. Human analysis is needed to recognise the complex, foggy chains of causality – to ‘join the dots’ and make assessments. Essentially OSINT should be used as a tool to follow the evidence informed decision-making model.

Artios has previously worked on evidence informed decision making in a year-long partnership with Queen’s University Belfast, on how evidence informs policy decisions in Stabilisation and the sociocultural impact of landmines on communities. Evidence informed decision making has also been used in Artios’ strategic research work for the UK Government, teaming up with industry research experts: Transparency Solutions, Somalia and Kiris Group, UK.


[1] A refined method for theory-based evaluation of the societal impacts of research, Brian M Belcher, Rachel Davel, Rachel Claus, 2020, https://pubmed.ncbi.nlm.nih.gov/32025508/

Get in touch