Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support

Citations Over TimeTop 1% of 2022 papers

Abstract

Various tools and practices have been developed to support practitioners in identifying, assessing, and mitigating fairness-related harms caused by AI systems. However, prior research has highlighted gaps between the intended design of these tools and practices and their use within particular contexts, including gaps caused by the role that organizational factors play in shaping fairness work. In this paper, we investigate these gaps for one such practice: disaggregated evaluations of AI systems, intended to uncover performance disparities between demographic groups. By conducting semi-structured interviews and structured workshops with thirty-three AI practitioners from ten teams at three technology companies, we identify practitioners' processes, challenges, and needs for support when designing disaggregated evaluations. We find that practitioners face challenges when choosing performance metrics, identifying the most relevant direct stakeholders and demographic groups on which to focus, and collecting datasets with which to conduct disaggregated evaluations. More generally, we identify impacts on fairness work stemming from a lack of engagement with direct stakeholders or domain experts, business imperatives that prioritize customers over marginalized groups, and the drive to deploy AI systems at scale.

Related Papers

• → Emily Dickinson Face to Face: Unpublished Letters with Notes and Reminiscences(1933)16 cited
• Face to face with the past: uncovering the real lives of Roman Britain(2015)
• On China's WTO Entry Will Exert an Influence to Trade Unions' Work and Chinese Trade Unions Will Face Challenges(2001)
• Acid attacks and women: problem and solution(2021)
• → The Domain of Social Work: What is It(1981)