Competency: Data Quality

Customer Profile:

Federal civilian agency.

Customer Challenge/Need:

Ability to maximize the accuracy of a large data collection program. Requirements included:

• Accurately reflect the information provided by respondents
• Contain no biases relative to error distribution
• Accurately predict accuracy output by field

Evolver Solution:

Evolver employed an Average Outgoing Quality Limit (AOQL) methodology to ensure overall high accuracy of data. This involves the establishment of a sampling plan, which includes:

• Sampling rate: the proportion of captured fields that must be inspected in order to confidently measure the incoming accuracy. Note that any errors detected during inspection are corrected.
• Error threshold: the proportion of errors that can be detected within the sample of a decision group beyond which all previously uninspected fields in the decision group must be inspected (and corrected, if necessary). Although all errors detected in the initial sample are corrected, they indicate the number of errors that remain in the uninspected fields.
• Minimum sample size: the minimum number of fields that must be inspected (given the sampling rate and error threshold) for the AOQL methodology to ensure the required outgoing accuracy.

Evolver developed a model that generates sampling plans that are optimal with respect to achieving the required overall accuracy at the lowest cost. The sampling plan generated by this model is used to control the processing within data capture and ensures that the overall accuracy requirements are satisfied. In order to ensure that no error biases for any demographic groups exist in the captured data, various post-capture analyses are performed using sophisticated statistical tools. These analyses include:

• Association analysis: the identification of combinations of characteristics (related to both demography and the capture process) that have a disproportionately high incidence of error.
• Control charting: the identification of conditions where the accuracy of data with specific characteristics varies significantly over time from its historical level.
• Business intelligence: the use of data cubes and business intelligence tools to provide rapid assessment of the accuracy of various groupings of data.

Whenever pockets of disproportionate errors are found, an impact analysis is conducted and a determination is made to decide whether all previously-processed data with the same characteristics should be re-processed. The Evolver solution establishes a new standard in the use of statistical controls to ensure data quality. Further, our solution will also mitigate the risk error distribution disproportionately associated with any demographic group of respondents.

For more information on our complete offering of statistical capabilities call Evolver, Inc. at 703.742.4090.