Compiling Data Inventories

Collaborative Working in Action

Photo by NASA on Unsplash

by Philippa Fogarty, X4 Consulting

Since 2018, X4 Consulting has been working with Stats NZ to compile data inventories across multiple government agencies, as part of New Zealand’s Open Government Data Programme.

These inventories allow government to clearly identify what data they collect and hold, as well as the status of those datasets. This is a key piece of work, designed to accelerate the building of a well-managed data infrastructure, provide greater transparency and visibility of available data, and enable economic and social benefits for all New Zealand through data driven innovation.

While typically data inventories have been focussed within a single agency, below is a case study that outlines how government agencies worked collaboratively on a data inventory to uncover and document a rich source of marine geo-spatial data across multiple agencies.

Background

Considerable efforts and resources are invested across New Zealand to collect, manage and use marine geospatial information (MGI) for a wide range of purposes. Greater collaboration can increase the value of these investments.

The New Zealand Marine Geospatial Information Working Group (NZMGI-WG) was established in February 2019, to facilitate national collaboration across marine stakeholders in New Zealand and develop a national programme of work to make marine geospatial data ‘FAIR’ (i.e. Findable, Accessible, Interoperable, Reusable).

The Key Players

Through the Open Government Data Programme at Stats NZ, Land Information New Zealand (LINZ) started the work on a national stocktake to ensure the data is ‘findable’.

X4 was involved with developing inventories for the Department of Conservation, Greater Wellington Regional Council and the Ministry for Primary Industries.

Each inventory will contribute to the national marine geospatial stocktake, be used to populate holdings on data.govt.nz, and can be used by agencies to maximise the value of their data.

The Approach

The data inventories capture an overarching view of the marine geospatial data holdings at each organisation. The inventory process was designed to:

• Establish a base level inventory of marine geospatial data held by each organisation
• Create and collect a base level of attributes describing this data
• Identify key stakeholders currently working with and managing marine data
• Start building a picture of the marine geospatial data landscape (i.e. where the information is held)
• Initiate a conversation around prioritising datasets for public release and/or internal access

The Process

Using the standardised marine geospatial data inventory template from the NZMGI-WG and Stats NZ, our high-level process was as follows:

• Review and capture publicly available information
• Review and capture any immediately obvious internal data holdings
• Consult representatives across the business to discuss and review data sources, and capture any missing data holding
• Publish the initial inventory on data.govt.nz and provide recommended next steps in conjunction with Stats NZ
• Deliver a maintenance guide for each agency to help keep their inventories up to date

The Outcome

A wealth of marine geospatial data holdings was uncovered across the three organisations — over 9,000 datasets!

The findings were broken down for each organisation and graphics were used to tell a story about their holdings.

Organisations confirmed which datasets could be made publicly available and these were subsequently listed to data.govt.nz (along with the inventory itself).

The Benefits

The data inventory is much more than a list of holdings — it can provide multiple benefits to an organisation, as summarised below:

What worked well

Power in a collective

We identified an invaluable pool of knowledge and passion in staff across each organisation. The collective approach for the data inventory project allowed these individuals to come together and discuss not only the data holdings but broader data management issues relating to how the data is being collected, collated, and described.

Identifying duplication

Undertaking a data inventory covering one theme across three agencies allowed us to quickly identify duplicated sets of data held across the organisations.

This, in turn, raised opportunities to discuss whether savings could be made by having data sets managed by one agency and shared with many.

Starting the discussion on data

The work on the data inventory provided the opportunity to start a discussion with staff right across the organisation regarding the value of their data assets. The organisations were surprised by the depth and richness of their data holdings.

We were able to tell the story of the data holdings to the agencies formally in presentations to Governance Groups and informally through presentations at morning tea get togethers. These presentations help to lift awareness across the agency about the data, the value it has to them and others and the importance of caring for it.

Lessons learned

Each data inventory project gives valuable insight into what we can do to continually maximise the value of a data inventory. The collective approach showed us that the following lessons:

One methodology, with an agile implementation

When working on a collective inventory it is essential that consistent metadata is collected across the agencies, but flexibility will be needed when dealing with different levels of data maturity within each organisation.

Not all surprises are pleasant

The inventory is not a data cleansing exercise. It is about presenting findings on data holdings, warts and all. This means that there will be duplicates — sometimes an agency can hold the same data in four different repositories. Gaps in metadata become apparent.

Working as a collective means that any shortcomings will be exposed to a wider audience. This requires a high level of trust, which fortunately in this instance was present across all agencies involved.

Good is good enough

Timebox effort at the beginning and work with the expectation that there will be gaps at the end. This is particularly important when there is a collective timeframe to be met. Determine what is the minimum output.

Be upfront with agencies regarding their required input

Agencies will be required to co-ordinate access to people and data, provide context on data supplied, and be prepared to make decisions regarding releases of data. This can take time.

What’s next?

Data holdings are dynamic, so the data inventory is intended to be a living document.

• Each organisation will be provided with a maintenance guide, which sets out our recommendations to continue maximising the use and value of the data inventory.
• Provides guidance around what is required for maintenance, including an outline of the suggested maintenance principles, and a high-level maintenance review process
• These principles and processes can then be tailored more specifically to each organisation alongside their information management teams and current practices

The Power of Many

Ongoing success will be dependent on the priority the inventory is given by each organisation and their understanding of the importance of data. A collective approach holds agencies to account for not only undertaking the data inventory, but for continuing to invest time and effort to fully realising the value of their data assets.