04Obstacles to overcome

While there are clear benefits to be gained from employing an Open Data Policy at both a government and organisational level, there are also concerns that can either prevent the release of data or hinder its release. The case studies illustrate that while some cities are happy to make their data publicly available, many are not.

Making big data publicly available can deliver significant benefits, but there are legitimate concerns associated with making any data publicly available. As outlined in section 1, a lack of demand for data can lower the benefits seen from making data freely available, which can reduce organisations’ commitment to an Open Data Policy. In addition, concerns around privacy, cost, continued availability and lack of skills to make sense of vast amounts of data present barriers. These are explored below.

• Privacy: Public and private organisations are wary of allowing data to be published if it is possible to identify individuals directly from the data or by linking datasets. However, social media sites such as Facebook suggest that, in specific circumstances, people may be willing to trade privacy around certain issues for access, provided they understand the benefits of giving up their privacy. As such, private organisations may be prepared to publicly share their own data if it means free access to the data of others, or the ability to have free access to any apps developed using the data for a set period of time.
• Cost of publishing data: There is a time and financial cost to ensuring data validity before release and presenting the data in a user friendly format. New York has reported that it is concerned it is spending resources organising data for release that no one is using.²⁷ While this is a valid concern, it is likely that in the UK the benefits of releasing data are underestimated. While the UK is able to measure the costs of publishing data, there is no model for estimating the benefits from re-use of data.²⁸ Significant amounts of time and money do not need to be spent ensuring data is complete before it is published – publishing incomplete/unverified data may actually help improve the accuracy of data, as individuals and industry experts are better placed to identify and correct local/ sector inaccuracies. As the hack events described in case study 4 illustrate, it is not necessarily dependent on the data holders themselves to develop user friendly ways of displaying information – individuals in other organisations can step in and create apps, which further reduces the cost of publication.
• Uncertainty over continued availability: In order for organisations to make use of data, some assurance that the data will be updated on a continuing and regular basis is generally required. If user believe the release of data is a one off they are less likely to use it.²⁹ To overcome this problem the Open Data Institute (ODI) has developed Open Data Certificates.³⁰ One criterion for being awarded a Certificate is the frequency with which the provider has committed to update the data. While there is no legal requirement for organisations to certify their data, those that do and achieve an award of ‘standard’ or above provide potential users with the confidence that the data will be published regularly.
• Lack of specific skills: A large amount of skill is required when working with vast amounts of data, particularly when linking various datasets together and presenting this data in a usable format. BIS highlighted that currently there is a skill shortage in the UK and that the lack of data scientists may prevent the full value of open data being realised.³¹ However, moves to encourage investment in data linking projects are underway, with the Economic and Social Research Council announcing in April 2013 that it will provide funding of £64million to projects that link up government data, survey data or organisational data in a secure way and/or develop apps that present this data in a user-friendly manner.

Many of these obstacles could be mitigated by increasing the demand for data. In particular, increasing the use of data will increase the benefits of releasing it and lower the average cost of publishing data. As such, organisations that see their data is being regularly used are more likely to continue to publish it. A greater use of data may also lead to more demand for data scientists, more training courses in this specialism and so an improvement to the UK’s skill levels.