Improving knowledge management across government

Working with Cabinet Office we’re applying cutting-edge AI techniques to improve how government makes its knowledge accessible for better policy development.

Imagine a stack of paper 20,000 miles high in the centre of Whitehall. The volume of unstructured information held by government departments is vast, it’s estimated that by 2023 there will be 53 billion documents in “digital heaps” held across government.

We were engaged to explore how machine learning and natural language processing technology could be applied to help understand the contents of these “digital heaps”, so that policymakers can learn from previous work and KIMs (knowledge and information managers) can meet their legislative obligations.

Across a 9-week discovery we engaged with more than 40 stakeholders across 13 government departments in order to surface user needs and develop value propositions with a technology strategy that would address both the need for better exploitation of knowledge across government and assist KIMs with meeting their obligations around retention and deletion.

In the subsequent alpha phase our team rapidly prototyped a product in an iterative process using one-week sprints, refining the design based on 4-5 usability testing sessions within each sprint. Our data scientists undertook three separate experiments to test the fundamental ML and NLP approaches that would underpin the product in a parallel stream of work.

Within very tight timeframes we demonstrated the value of three different core technologies that would underpin the beta product, co-design and prototype an MVP beta product and validate our work with a wide cross-government audience.

Using NLP we were able to successfully identify high-value documents, improving on the accuracy of the current processes with a >80% reduction in incorrectly identified docs (i.e. false positives).