Bridging the gap
A vision for the future of open data
The Open Data Charter Principles were set by governments and civil society as part of a global consultative process in 2015. In the three years since, it has become clear that the potential impact of open data is not yet being realised, despite some good individual examples. The sector is maturing, learning from what has worked and what has not. As the Charter’s Principles are increasingly shaping the way policies around the world are created and delivered, we believe that refreshing the principles to include what has been learnt will help bridge the impact gap.
In the coming months, these principles will be reviewed and updated through a process that will include input from stakeholders and the Charter’s Implementation Working Group, and which will be culminating in a meeting at the 2018 International Open Data Conference in Buenos Aires in September.
This blog aims to provide a foundation for this process, outlining the Charter team’s reflections on what the future of work in this sector should look like. It draws on what we have learnt from our own research and publications, as well as those of our partners, and the environment scans being collected for the State of Open Data report.
Insights from the ODC team and network
Early work by the open data sector has typically focussed on getting data out into the public domain. The emphasis has been on ensuring that institutions had data platforms that were openly available, that they were releasing a large number of datasets and to a usable standard.
The attitude was largely that of “build it and they will come”, and that it was enough for those institutions simply to make data available. This initial approach was helpful in putting open data on the map, and in delivering an increasing number of initiatives that have opened data in both private institutions and in government.
However, this approach has yet to lead to widespread fundamental changes in institutions, in approaches to building a sustainable data infrastructure, or a good body of evidence on how open data is adding value in a cost-effective way.
In the three years since the Open Data Charter principles were developed, there have also been developments in the uses of open data that were not considered in that first iteration. An increased realisation of the use of data to feed AI algorithms, for example, highlights the need to address the use of data and its openness in the use of Artificial Intelligence.
The Open Data Charter team have identified four key points that we believe will most benefit from attention during reviews over the months to come:
1) Publish with purpose
There is already a slow shift towards designing work around the idea of open data as a means to solve key policy problems. The Charter’s 2018 strategy, Publish with Purpose, identifies “a growing recognition that opening up data in isolation is less effective than it can be if targeted at solving specific policy problems”.
We need to cement and accelerate this shift. Implementation of open data projects needs to involve collaboration with those with the mandate and credibility to address key policy challenges.
In our paper on bridging the impact gap, we wrote that “moving from transparency to accountability isn’t just about publishing data and using data. Instead there’s a more subtle process of collecting data, sharing data, processing data, taking action and hopefully getting a policy response”.
Last year Global Integrity explored how many organisations “are looking to move beyond an emphasis on making data available and open, some are giving additional thought to questions of users, use and impact, and others still are adopting an approach which starts with problems rather than with data.” Prioritisation of data release should be based on demand for data from citizens, CSOs and other public sector actors who can use it.
In addition, if we hope to achieve the potential of open data, as GovLab has indicated, “we need a better — and more granular — understanding of the enabling conditions that lead to success”, an idea echoed in the Charter’s own anti-corruption guide.
2) Alignment with privacy and protection
Our findings on implementing the anticorruption guide revealed that “while the Charter’s principles recognise the fundamental importance of respecting privacy, this needs to be balanced with the commitment to be ‘open by default’. Privacy concerns can sometimes be used as a smokescreen to not publishing public information at all.”
The State of Open Data’s scan on privacy, notes that there has been “considerable development of tools/methods for evaluating data for privacy issues prior to release as open data” and identifies a gap on balancing of privacy and transparency goals, especially in the global south.
There needs to be a renewed and realistic focus on how open data is aligned with privacy and data protection concerns. This should include a focus on how governmental initiatives handle anonymisation and privacy of key datasets, but also of private sector data.
Tim Davies and Jeni Tennison identified that “often the dataset that most matter, and that could have the most impact if they were open, do not belong to governments”.
A full and frank conversation about the relative costs and benefits must be had, with a view to establishing and recognising the value of collecting and publishing data in an open way.
3) Data infrastructure that is open by default
The first Charter Principle is ‘open by default’, and this needs to extend beyond datasets, to the broader investments in building data infrastructures. “Historical databases weren’t built with open data in mind”, we establish in our report on cities, “If ‘open by default’ is applied to a city’s broader data management systems it can allow better internal data sharing, as well as improving access to information for citizens”.
A presumption of openness should be integrated into the ways data collection and management are designed and the standards that guide how they are organised.
Our report on bridging the impact gap identified that “to improve as a sector and deliver impact — we need to understand the complex system connecting data to impact. Then we need to coordinate our investments and program so that they support the system as a whole.”
It is also helpful to think of the integration of initiatives into broader government reform processes that can sustain them. As we wrote in our paper on embedding change to open data, “there has been a lot of progress at getting countries to open up data, but there’s a serious risk of backsliding when governments change. There are some key ways of embedding transparency reforms, including: de-politicise and institutionalise the policy, broaden ownership and generating results that resonate”.
The same report also highlights that “Policy and standard development is not keeping up with the pace of change. It’s important for data to be released using shared standards, but city officials often struggle to find relevant ones for particular datasets”.
4) Openness in Artificial Intelligence
The data and algorithms used in Artificial Intelligence (AI) technology and applications are an increasing feature of public life. The Web Foundation has highlighted how “the availability of quality data is crucial for AI systems to learn and function effectively. If AI systems are to reshape the world, we must have the opportunity to shape them. And a wider group of people must have access to datasets necessary to build AI technologies”.
We need to see what data is both feeding and teaching these systems so that its use supports just and equitable outcomes, an idea further explored in our article for CPI.
In addition, the State of Open Data identifies the need for further consideration of “open data and its relationship to privacy in the context of big data and AI”.
In relation to the use of AI in civic life, the Omidyar Network’s report on scrutiny of automated decision notes that “advocates, policymakers, and technologists have begun demanding that these automated decisions be explained, justified, and audited. There is a growing desire to ‘open the black box’ of complex algorithms and hold the institutions using them accountable”.
Next steps and how to get involved
The Open Data Principles were originally agreed in 2015, together with a commitment to revisit the text after a few years to ensure that they remained fit for purpose. The Open Data Charter will launch its collaborative process to update these Principles this month.
The process will be led by the Charter’s Implementation Working Group, which is opening an invitation to new members who wish to be involved. It will also shortly be launching a survey to gather views on how the Principles might be updated. To participate in this process, via the survey or by joining the Implementation Working Group, send an email to info@opendatacharter.org.
