An Examination of Questionnaire Evaluation by Expert Reviewers
Citations Over TimeTop 13% of 2010 papers
Abstract
Expert reviews are frequently used as a questionnaire evaluation method but have received little empirical attention. Questions from two surveys are evaluated by six expert reviewers using a standardized evaluation form. Each of the questions has validation data available from records. Large inconsistencies in ratings across the six experts are found. Despite the lack of reliability, the average expert ratings successfully identify questions that had higher item nonresponse rates and higher levels of inaccurate reporting. This article provides empirical evidence that experts are able to discern questions that manifest data quality problems, even if individual experts vary in what they rate as being problematic. Compared to a publicly available computerized question evaluation tool, ratings by the human experts positively predict questions with data quality problems, whereas the computerized tool varies in success in identifying these questions. These results indicate that expert reviews have value in identifying question problems that result in lower survey data quality.
Related Papers
- → Reliability of water distribution networks due to pumps failure: comparison of VSP and SSP application(2012)5 cited
- → Helicopter Reliability Researching Based on Neural Networks(2014)1 cited
- → DETERMINING QUALITY REQUIREMENTS AT THE UNIVERSITIES TO IMPROVE THE QUALITY OF EDUCATION(2018)
- → Reliability Analysis for Trans-Scale Safety and Arming Device Based on FORM+G-PCM Method(2018)
- → Research on digital twin-based reliability evaluation of electric power optical communication network(2023)