Data Quality Indicators
We use Data Quality Indicators to rank the data quality of the sources used, and to provide insights into different aspects that might affect the quality of the data.
This tool was originally developed by Weidema and Wesnæs (1996) and used in life-cycle assessments. It was adjusted for urban-level material flow work by Hoekman and Von Blottnitz (2016), and that adjusted matrix was used as a starting point. Some additional modifications were made for our work. Most importantly, "Data classification" replaces an indicator formerly defined as "Additional steps", and the ratings to "Geographic correlation" were adjusted to better account for the significant impact that downscaling from a national scale might have on data quality in our case.
Similar to its application in LCA, the pedigree scores are “semi-quantitative” and serve only as identification numbers. They should not be aggregated, but rather only be used as a reference to better understand the quality of the different data sources.
Reliability
| Rating | Description |
|---|---|
| 1 | Verified data based on measurements |
| 2 | Verified data partly based on assumptions or non-verified data based on measurements |
| 3 | Non-verified data partly based on assumptions |
| 4 | Qualified estimate (e.g. by industrial expert) |
| 5 | Non-qualified estimate or unknown origin |
Completeness
| Rating | Description |
|---|---|
| 1 | Representative data from a sufficient sample of sites over an adequate period to even out normal fluctuations |
| 2 | Representative data from a smaller number of sites but for adequate periods |
| 3 | Representative data from an adequate number of sites but from shorter periods |
| 4 | Representative data but from a smaller number of sites and shorter periods or incomplete data from an adequate number of sites and periods |
| 5 | Representativeness unknown or incomplete data from a smaller number of sites and/or from shorter periods |
Temporal correlation
| Rating | Description |
|---|---|
| 1 | Time period is equal to the period of study |
| 2 | Up to one year of difference to year of study |
| 3 | More than one but no more than three years of difference to year of study |
| 4 | More than three but no more than five years of difference to year of study |
| 5 | Age of data unknown or more than five years of difference |
Geographical correlation
| Rating | Description |
|---|---|
| 1 | Data from area under study |
| 2 | Average date from larger area in which the area under study is included and with which it shares comparable conditions. |
| 3 | Average date from smaller area inside the area under study and with which it shares comparable conditions. |
| 4 | Average date from larger area in which the area under study is included and with which it does not share comparable conditions. |
| 5 | Data from unknown area or area with very different conditions |
Access
| Rating | Description |
|---|---|
| 1 | Publicly and readily available data |
| 2 | Data are not publicly available but can be easily and freely obtained by anyone |
| 3 | Specific effort required to obtain data (e.g. only through formal requests, granted on a per-case basis, or by paying the owner) |
| 4 | Data are only accessible to very specific users (e.g. government or partner organizations) |
| 5 | Data are only accessible to the organization holding the data |
Frequency
| Rating | Description |
|---|---|
| 1 | New data collected on an annual basis |
| 2 | New data collected every 2-3 years |
| 3 | New data collected every 4-5 years |
| 4 | New data collected every 6+ years |
| 5 | No scheduled data collection interval |
Informality and illegality
| Rating | Description |
|---|---|
| 1 | No illegal or informal flows, or they are fully included |
| 2 | Illegal or informal flows estimated at no more than 5% |
| 3 | Illegal or informal flows estimated at 5-15% |
| 4 | Illegal or informal flows estimated at 15-30% |
| 5 | Illegal or informal flows estimated at more than 30% or impossible to quantify |
Classification compatibility
| Rating | Description |
|---|---|
| 1 | Data classification is identical to the chosen classification system |
| 2 | Data classified using a different yet compatible classification system |
| 3 | Data classified using a different classification system that is not fully compatible; up to 50% of the data require reclassification |
| 4 | Data classified using a different classification system that is not fully compatible; more than 50% of the data require reclassification |
| 5 | Data classified using a different classification system that is not compatible |