A Keyword Based Approach to Understanding the Overpenalization of Marginalized Groups by English Marginal Abuse Models on Twitter

Citations Over Time

Abstract

Harmful content detection models tend to have higher false positive rates for content from marginalized groups. In the context of marginal abuse modeling on Twitter, such disproportionate penalization poses the risk of reduced visibility, where marginalized communities lose the opportunity to voice their opinion on the platform. Current approaches to algorithmic harm mitigation, and bias detection for NLP models are often very ad hoc and subject to human bias. We make two main contributions in this paper. First, we design a novel methodology, which provides a principled approach to detecting and measuring the severity of potential harms associated with a text-based model. Second, we apply our methodology to audit Twitter's English marginal abuse model, which is used for removing amplification eligibility of marginally abusive content. Without utilizing demographic labels or dialect classifiers, we are still able to detect and measure the severity of issues related to the over-penalization of the speech of marginalized communities, such as the use of reclaimed speech, counterspeech, and identity related terms. In order to mitigate the associated harms, we experiment with adding additional true negative examples and find that doing so provides improvements to our fairness metrics without large degradations in model performance.

Related Papers

• → Review on Parameterization Schemes of Visibility in Fog and Brief Discussion of Applications Performance(2021)13 cited
• Studies on the visibility index-classified system in Shanghai(2005)
• ASSESSMENT OF NIGHTTIME ROADWAY VISIBILITY(1957)
• Analysis of Characteristics and Trends of Atmospheric Visibility in Shizuishan City of Ningxia(2012)
• Change Trend and Prediction for Future of Visibility in Kashgar City(2013)