Metadata issues
Metadata are needed to explain the content of data, its provenance and context. It also allows a researcher to match and link multiple sources. “Metadata needs to be organized for data sharing to be productive, for the simple reason that users need to know what the data is. becomes increasingly pressing as the number of useful and brazil rcs data potentially useful data sources increases…. potential will only be realized if the metadata is well-documented and well-managed.” Robin Bloor “Does Big Data Mean Big Metadata?
However big data can present challenges when attempting to assign metadata to a data collection. Are all variables labelled adequately, are meaningful and intepretable, do they match the documentation (if there is any!), and is there enough information on derived data and weights (if any exist)?
An example illustrating the important role of metadata for big data is the UK’s Energy Demand Research Project: Early Smart Meter Trials, 2007-2010 (study number 7591) held at the Data Service. As along a ‘long and thin’ dataset comprising millions of rows of smart electricity and gas meter readings at half hourly intervals from over 14,000 households, the documentation provided by the depositor was very limited, and details like which company supplier the readings to refer to was not available. This example highlights the importance of staff curating big data sources undertaking further detective work to establish better provenance, where this is even possible. In many cases, with big data, there be no direct link to a ‘depositor.