Quantifying Societal Bias Amplification in Image Captioning

We study societal bias amplification in image captioning. Image captioning models have been shown to perpetuate gender and racial biases, however, metrics to measure, quantify, and evaluate the societal bias in captions are not yet standardized. We provide a comprehensive study on the strengths and...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) s. 13440 - 13449
Hlavní autoři: Hirota, Yusuke, Nakashima, Yuta, Garcia, Noa
Médium: Konferenční příspěvek
Jazyk:angličtina
japonština
Vydáno: IEEE 01.06.2022
Témata:
accountability
Computational modeling
Computer vision
Equalizers
fairness
Focusing
Measurement
Predictive models
privacy and ethics in vision; Vision + language
Skin
Transparency
ISSN:1063-6919
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We study societal bias amplification in image captioning. Image captioning models have been shown to perpetuate gender and racial biases, however, metrics to measure, quantify, and evaluate the societal bias in captions are not yet standardized. We provide a comprehensive study on the strengths and limitations of each metric, and propose LIC, a metric to study captioning bias amplification. We argue that, for image captioning, it is not enough to focus on the correct prediction of the protected attribute, and the whole context should be taken into account. We conduct extensive evaluation on traditional and state-of-the-art image captioning models, and surprisingly find that, by only focusing on the protected attribute prediction, bias mitigation models are unexpectedly amplifying bias.
ISSN:1063-6919
DOI:10.1109/CVPR52688.2022.01309