Beyond the ML Model: Applying Safety Engineering Frameworks to Text-to-Image Development
Citations Over TimeTop 14% of 2023 papers
Abstract
Identifying potential social and ethical risks in emerging machine learning (ML) models and their applications remains challenging. In this work, we applied two well-established safety engineering frameworks (FMEA, STPA) to a case study involving text-to-image models at three stages of the ML product development pipeline: data processing, integration of a T2I model with other models, and use. Results of our analysis demonstrate the safety frameworks – both of which are not designed explicitly examine social and ethical risks – can uncover failure and hazards that pose social and ethical risks. We discovered a broad range of failures and hazards (i.e., functional, social, and ethical) by analyzing interactions (i.e., between different ML models in the product, between the ML product and user, and between development teams) and processes (i.e., preparation of training data or workflows for using an ML service/product). Our findings underscore the value and importance of examining beyond an ML model in examining social and ethical risks, especially when we have minimal information about an ML model.
Related Papers
- → Workflow Reuse in Practice: A Study of Neuroimaging Pipeline Users(2014)15 cited
- A Workflow Event Logging Mechanism and Its Implications on Quality of Workflows(2010)
- Workflow Analysis using Graph Kernels.(2010)
- → Utilizing Tags for Scientific Workflow Recommendation(2019)2 cited
- Configurable and Collaborative Scientific Workflows(2014)