Accuracy Comparison Across Face Recognition Algorithms: Where Are We on Measuring Race Bias?

Previous generations of face recognition algorithms differ in accuracy for images of different races (race bias). Here, we present the possible underlying factors (data-driven and scenario modeling) and methodological considerations for assessing race bias in algorithms. We discuss data-driven facto...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on biometrics, behavior, and identity science Ročník 3; číslo 1; s. 101 - 111
Hlavní autoři: Cavazos, Jacqueline G., Phillips, P. Jonathon, Castillo, Carlos D., O'Toole, Alice J.
Médium: Journal Article
Jazyk:angličtina
Vydáno: United States IEEE 01.01.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Accuracy
Algorithms
Artificial neural networks
Bias
Computational modeling
Convolutional neural networks
deep convolutional neural networks
Demographics
Face recognition
Face recognition algorithm
Faces
Image quality
Modelling
Object recognition
Population statistics
Prediction algorithms
Principal component analysis
Race
race bias
the other-race effect
deep convolutional neural networks
the other-race effect
face recognition algorithm
race bias
ISSN:2637-6407, 2637-6407
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Previous generations of face recognition algorithms differ in accuracy for images of different races (race bias). Here, we present the possible underlying factors (data-driven and scenario modeling) and methodological considerations for assessing race bias in algorithms. We discuss data-driven factors (e.g., image quality, image population statistics, and algorithm architecture), and scenario modeling factors that consider the role of the "user" of the algorithm (e.g., threshold decisions and demographic constraints). To illustrate how these issues apply, we present data from four face recognition algorithms (a previous-generation algorithm and three deep convolutional neural networks, DCNNs) for East Asian and Caucasian faces. First, dataset difficulty affected both overall recognition accuracy and race bias, such that race bias increased with item difficulty. Second, for all four algorithms, the degree of bias varied depending on the identification decision threshold. To achieve equal false accept rates (FARs), East Asian faces required higher identification thresholds than Caucasian faces, for all algorithms. Third, demographic constraints on the formulation of the distributions used in the test, impacted estimates of algorithm accuracy. We conclude that race bias needs to be measured for individual applications and we provide a checklist for measuring this bias in face recognition algorithms.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2637-6407
2637-6407
DOI:10.1109/TBIOM.2020.3027269