Researchers from MIT, Worcester Polytechnic Institute, and Google have introduced a new debiasing method for vision-language models (VLMs) called WRING (Weighted Rotational DebiasING). This technique aims to reduce bias without amplifying other biases, directly addressing the "Whac-A-Mole dilemma" that plagues existing projection debiasing methods.
The Challenge of Bias in AI
Bias in AI systems can stem from training data and model architecture, leading to poor performance in critical real-world settings. For example, a dermatology AI might misclassify lesions on certain skin tones. A common approach to fix this—projection debiasing—removes biased information but can inadvertently amplify other biases, a phenomenon known as the "Whac-A-Mole dilemma."
How WRING Works
WRING is a post-processing approach that adjusts coordinates in a model's high-dimensional space. Its goal is to make the model unable to distinguish between groups within a certain concept, while leaving all other relationships intact.
WRING is designed to be applied to pre-trained VLMs without retraining, making it efficient and minimally invasive.
In tests, WRING successfully reduced bias for a target concept without increasing bias in other areas. The method is currently limited to CLIP (Contrastive Language-Image Pre-training) models.
Expert Perspectives
Walter Gerych, first author and assistant professor at Worcester Polytechnic Institute, stated that WRING is efficient and does not require additional training.
Marzyeh Ghassemi, MIT associate professor, highlighted the practical challenge of amplifying biases: "When removing racial bias in a clinical staff image retrieval model, you could increase gender bias."
Funding and Next Steps
This research was supported by several awards, including a National Science Foundation CAREER Award, an AI2050 Award Early Career Fellowship, a Sloan Research Fellow Award, the Gordon and Betty Moore Foundation Award, and an MIT-Google Computing Innovation Award.
The researchers aim to extend WRING to generative language models, similar to ChatGPT, to broaden its impact.