Foundations of Good Public Sector Data Management

If the UK government can create firm platforms for its use of data, civil servants heard at a recent Dun & Bradstreet webinar, the country could make a forward leap as great as during the industrial revolution. But to realise the potential, today’s civil service and political leaders will need to show as much ambition and vision as their Georgian and Victorian forebears

When the COVID-19 pandemic swept the USA in the summer of 2020, the government unleashed unprecedented amounts of public money to soften the damage caused by lockdowns. But deciding to spend cash was the easy part. The tough questions were: how much to spend, and how best to spend it?

Data provided the answers, according to Sally Block, Vice President of Global Sales for Government Solutions at business data and analytics firm Dun & Bradstreet. Public and private sector data scientists were able to quickly tell the government that those hardest-hit by lockdown would include retail small businesses, hospitality, transportation, and minority-owned businesses. These insights helped decide how money was disbursed through paycheque protection schemes and disaster loans.

“We took the data, we looked at it and we embraced it. But the data led to action, and the action led to policy,” Block told a live, online audience of civil servants during a recent Dun & Bradstreet webinar exploring the complex but pivotal topic of master data management.

The international audience heard how public sector leaders and private sector consultants are using canonical records – easily-accessible ‘golden records’ holding reliable, verified data in standardised formats – to unlock the potential of data to transform public service delivery.

Data drives service change, drives policy change

The most obvious way in which golden records can simplify access to public services is through “tell us once” schemes. Under these mechanisms, citizens or businesses need only report major events – such as births and deaths – to a single government body, which then cascades the information out to all the other parts of the public sector required to act or update their records. In the case of bereavement, for example, the deceased’s driving licence is cancelled automatically, their council tax bills are stopped, and pension payments ended.

The idea is extremely simple – but deceptively so, because it hides a change in the nature of how government works, according to Dr Ziv Baida, a consultant on public sector data projects and the author of the blog Insights Unboxed. He walked the panel through the implication of one such idea: ‘trade single windows’. These are public service innovations that allow importers to tell a government about a consignment entering their territory just once, so that food safety, customs and ports authorities are all informed through one interaction.

Golden records’ holding reliable, verified data in standardised formats, can unlock the potential of data to transform public service delivery

On the surface level, this is where the process ends, but Dr Baida explained that governments which have implemented the most effective trade single windows went several steps further. First they coordinated inspections, so that customs and health inspectors would look over a consignment at the same time, shrinking the delay to the shipment. In other cases, a single person could carry out checks on multiple agencies’ behalf.


Ultimately, trade single windows led to fundamentally political decisions about which authority had jurisdiction, and whether the roles normally played by one could be taken up by another to increase efficiency.

Dr Baida described the principal at work here as “start small, think big”. A relatively simple change to data input can lead to a fundamental reorganisation of the way a company experiences a government process, with benefits for trade efficiency and therefore the wider economy. The lesson to draw, Dr Bada said, is that, “because data can generate value, it must be treated as an asset.”

Put garbage in, get garbage out

If data is going to have a transformative impact on public service delivery, it needs to be reliable, easily accessible, and structured in a way that can be interrogated simply by other departments. In the trade single window example, for instance, the starting point was that all relevant public sector IT systems need to be interoperable and work with data in a single common format required of the shipper.

For government departments, this first step itself may be daunting. Rather than simply collecting data, as they have done expertly for centuries, it implies ordering it in agreed, standardised formats before feeding it into public systems. “Quality of data is a critical success factor for realising the potential value of that data,” said Baida. “As we say, if we put garbage in, we get garbage out.”

He urged governments to tackle the data quality problem as early as possible, factoring it into data collection so that any information coming to a government already conforms to agreed standards and patterns. Forms, for example, should have limited ‘free text’ fields requiring answers in prose, and the terms used on them should be common to all departments.

A relatively simple change to data input can lead to a fundamental reorganisation of the way a company experiences a government process

Departments also need a robust, shared way to identify key entities such as citizens, companies, vehicles and cargo, and agreed terminology for describing them. Ideally, panellists suggested, these naming formats should extend both to the private sector and to overseas governments, creating the prospect that ultimately data generated anywhere in the world could be easily manipulated, compared and worked with seamlessly.


The pandemic has also shown the need to enrich data, improving public bodies’ understanding of the context within which an entity sits, according to Imam Hoque, Chief Product Officer at the consultancy Quantexa.

Consider freight coming into a country through a trade single window during the pandemic. The content and origins of the shipment are important, said Hoque – but so is the logistics company handling it, the specific driver, and who he or she may have been in contact with previously.

The idea is that an individual data point cannot sit in isolation but needs to be seen within a network of other data. “Only then can you apply your wonderful analytical models and get better outcomes and better decisions,” Hoque said.

A new industrial revolution

If all this sounds futuristic and unrealistic, Thom Townsend, the Executive Director of OpenOwnership, has some sympathy. He leads a non-profit which seeks to identify who owns specific data, so that it can be brought into the open and become part of the networks that Hoque describes. And Townsend, the former Head of Data Policy for the UK government, acknowledged that for rank-and-file civil servants the scale of change required can look intimidating.

“Thinking about this audience, [there may be] lots of people within government departments who will spend their time screaming into the void about the availability of data and how hard it is to get things done,” Townsend said. Politicians at the head of government must lead the way, he argued.

An individual data point cannot sit in isolation but needs to be seen within a network of other data

He compared the economic opportunity that data currently offers the UK to the transformative impact of building the railways during the industrial revolution. To unlock this potential, said Townsend, senior leaders today must show the same level of ambition as their Victorian-era predecessors.


“150 years ago, we invested a huge sum of money in building the infrastructure we needed to be a modern economy,” Townsend said. “The challenge that we face today as a country is for the government to match that.”

Massive investments would be required in computing power and data centres to provide the infrastructure on which a truly digital government can be built, he added. Only then will the softer, strategic work of changing attitudes in government pay dividends.

Winning political support

If the need is for ministers to take the lead and commit vast quantities of cash to the cause of golden records, how can individual civil servants persuade them to take the leap?

Townsend suggested a degree of realpolitik, suggesting that civil servants highlight how investing in data could enhance a ministerial career. An ideal time to lobby for a data investment, for example, might be when the absence of data prevents a minister from providing a robust response to a parliamentary question.

Other participants said departments need to educate senior officials, who can then make the case for data investment to ministers. Block from Dun & Bradstreet reported how various US government agencies have appointed chief data officers. Officials with sufficient seniority are more likely to succeed in persuading political office holders to commit funds to data projects.

Once investment starts flowing, the hope is that its returns will demonstrate to politicians and the public alike the value of continued efforts and funding. “The cultural barriers [will] come down when the outcomes far exceed any of the trepidations,” Block said.

Making private data public

One area where political intent will be needed is in persuading private sector organisations to open up their data as well. Townsend took the panel through the fascinating example of how the World Health Organisation worked with pharmaceutical companies to prevent the growth of antimicrobial resistance to medications.

The initiative required pharmaceutical companies to share data with rivals about how their drugs are working in the field; and also, to potentially withdraw products from sale for periods of time, minimising the risk that microbes develop resistance. But how can governments persuade companies to provide access to data, where there is an inherent commercial disadvantage?

For most participants, this seemed to be a clear example of a situation where governments should act in the public interest by compelling companies to share information, either through legislation or regulation.

“There are some cases where other methods won’t work, and you do need to be a bit more heavy-handed. I think this is one of them,” said Joshua Harris-Kirkwood, Head of Data Availability and CDEI Sponsorship at the UK’s Department for Digital, Culture, Media and Sport.

Harris-Kirkwood also suggested that an entirely new industry is likely to emerge to intermediate between governments and companies. This could potentially make privately-held information public in a smooth manner, so it can be used to frame policy.

He described how the UK government relied on anonymised location data from Google, Apple and others to understand how population movements were changing during the pandemic. Efforts were successful and fed directly into pandemic responses but involved government bodies striking painstakingly negotiating a series of legal agreements with individual companies.

He foresees the emergence of professional “data intermediary as a service” companies, which strike agreements with large data providers to access their information on an ongoing basis. These intermediaries would then develop in-house solutions which allow them to combine data from multiple sources to provide answers to policy questions in real time.

“When they’re doing this on a regular basis, for different purposes, that will enable you to scale the kind of intervention needed [to answer policy questions] and make it happen more quickly,” he said.

As states grapple with how to manage data as an asset for the public good, it is clear that changes will be needed not only within government, but between the state and private sector. Much work is required – but if data professionals, senior officials and elected leaders grasp the opportunities and invest the resources, the rewards will soon vastly exceed the costs.

The full webinar: ‘Master Builders: The Foundations of Good Data Management for Public Sector’, can be watched via the link below.


Watch the webinar