Simulation Experiments on (the Absence of) Ratings Bias in Reputation Systems
Citations Over TimeTop 10% of 2017 papers
Abstract
As the gig economy continues to grow and freelance work moves online, five-star reputation systems are becoming more and more common. At the same time, there are increasing accounts of race and gender bias in evaluations of gig workers, with negative impacts for those workers. We report on a series of four Mechanical Turk-based studies in which participants who rated simulated gig work did not show race- or gender bias, while manipulation checks showed they reliably distinguished between low- and high-quality work. Given prior research, this was a striking result. To explore further, we used a Bayesian approach to verify absence of ratings bias (as opposed to merely not detecting bias). This Bayesian test let us identify an upper- bound: if any bias did exist in our studies, it was below an average of 0.2 stars on a five-star scale. We discuss possible interpretations of our results and outline future work to better understand the results.
Related Papers
- → A content-driven reputation system for the wikipedia(2007)446 cited
- → Cooperation in the spatial public goods game with the second-order reputation evaluation(2019)51 cited
- → Partner selection supported by opaque reputation promotes cooperative behavior(2016)1 cited
- A Model of Reputation of Agent Based on Acquaintances Chain(2009)