Validating the Contribution-Weighted Model: Robustness and Cost-Benefit Analyses

Citations Over TimeTop 11% of 2016 papers

Abstract

We use results from a multiyear, geopolitical forecasting tournament to highlight the ability of the contribution weighted model [Budescu DV, Chen E (2015) Identifying expertise to extract the wisdom of crowds. Management Sci. 61(2):267–280] to capture and exploit expertise. We show that the model performs better when judges gain expertise from manipulations such as training in probabilistic reasoning and collaborative interaction from serving on teams. We document the model’s robustness using probability judgments from early, middle, and late phases of the forecasting period and by showing its strong performance in the presence of hypothetical malevolent forecasters. The model is highly cost-effective: it operates well, even with random attrition, as the number of judges shrinks and information on their past performance is reduced.

Related Papers

• → Physician-Friendly Machine Learning: A Case Study with Cardiovascular Disease Risk Prediction(2019)71 cited
• → PSTMonitor: Monitor synthesis from probabilistic session types(2022)7 cited
• → A Vulnerability Exploits Detection Method Based on Binary Program Behavior Analysis(2012)
• The Application of Datasnap to Internet Exploition(2002)
• A new approach to deploying private mobile network exploits(2016)