OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings

Citations Over TimeTop 10% of 2021 papers

Abstract

Language representations are known to carry certain associations (e.g., gendered connotations) which may lead to invalid and harmful predictions in downstream tasks. While existing methods are effective at mitigating such unwanted associations by linear projection, we argue that they are too aggressive: not only do they remove such associations, they also erase information that should be retained. To address this issue, we propose OS-CAR (Orthogonal Subspace Correction and Rectification), a balanced approach of mitigation that focuses on disentangling associations between concepts that are deemed problematic, instead of removing concepts wholesale. We develop new measurements for evaluating information retention relevant to the debiasing goal. Our experiments on genderoccupation associations show that OSCAR is a well-balanced approach that ensures that semantic information is retained in the embeddings and unwanted associations are also effectively mitigated.

Related Papers

• → Principles of first and third angle orthographic projection(2020)1 cited
• → Short-cut algorithms for the learning subspace method(2005)2 cited
• Calculation of an Orthographic Projection with the Projection Theorem(2001)
• Method of projection subspace construction for recognizing similar targets(2011)
• The Design, Construction, and Use of a Tri-color Projection Box to be Used in the Instruction of Orthographic Projection(1969)