Not logged in.

Quick Search - Contribution

Contribution Details

Type Journal Article
Scope Discipline-based scholarship
Title A Methodology for Quantifying the Effect of Missing Data on Decision Quality in Classification Problems
Organization Unit
Authors
  • Michael Feldman
  • Adir Even
  • Yisrael Parmet
Item Subtype Original Work
Refereed Yes
Status Published in final form
Language
  • English
Journal Title Communications in Statistics. Theory and Methods
Publisher Taylor & Francis
Geographical Reach international
ISSN 0361-0926
Volume 47
Number 11
Page Range 2643 - 2663
Date 2018
Abstract Text Decision-making is often supported by decision models. This study suggests that the negative impact of poor data quality (DQ) on decision making is often mediated by biased model estimation. To highlight this perspective, we develop an analytical framework that links three quality levels – data, model, and decision. The general framework is first developed at a high-level, and then extended further toward understanding the effect of incomplete datasets on Linear Discriminant Analysis (LDA) classifiers. The interplay between the three quality levels is evaluated analytically - initially for a one-dimensional case, and then for multiple dimensions. The impact is then further analyzed through several simulative experiments with artificial and real-world datasets. The experiment results support the analytical development and reveal nearly-exponential decline in the decision error as the completeness level increases. To conclude, we discuss the framework and the empirical findings, elaborate on the implications of our model on the data quality management, and the use of data for decision-models estimation.
Digital Object Identifier 10.1080/03610926.2016.1277752
Other Identification Number merlin-id:14529
PDF File Download from ZORA
Export BibTeX
EP3 XML (ZORA)
Additional Information This is an Accepted Manuscript of an article published by Taylor & Francis in Communications in Statistics. Theory and Methods on 2017, available online: http://wwww.tandfonline.com/10.1080/03610926.2016.1277752.