Five points to check when comparing visual perception in humans and machines

With the rise of machines to human-level performance in complex recognition tasks, a growing amount of work is directed toward comparing information processing in humans and machines. These studies are an exciting chance to learn about one system by studying the other. Here, we propose ideas on how...

Full description

Saved in:
Bibliographic Details
Published in:Journal of vision (Charlottesville, Va.) Vol. 21; no. 3; p. 16
Main Authors: Funke, Christina M., Borowski, Judy, Stosio, Karolina, Brendel, Wieland, Wallis, Thomas S. A., Bethge, Matthias
Format: Journal Article
Language:English
Published: United States The Association for Research in Vision and Ophthalmology 01.03.2021
Subjects:
Artificial Intelligence
Humans
Image Processing, Computer-Assisted - methods
Learning - physiology
Pattern Recognition, Automated - methods
Pattern Recognition, Visual - physiology
Problem Solving
Recognition, Psychology
Visual Perception - physiology
ISSN:1534-7362, 1534-7362
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:With the rise of machines to human-level performance in complex recognition tasks, a growing amount of work is directed toward comparing information processing in humans and machines. These studies are an exciting chance to learn about one system by studying the other. Here, we propose ideas on how to design, conduct, and interpret experiments such that they adequately support the investigation of mechanisms when comparing human and machine perception. We demonstrate and apply these ideas through three case studies. The first case study shows how human bias can affect the interpretation of results and that several analytic tools can help to overcome this human reference point. In the second case study, we highlight the difference between necessary and sufficient mechanisms in visual reasoning tasks. Thereby, we show that contrary to previous suggestions, feedback mechanisms might not be necessary for the tasks in question. The third case study highlights the importance of aligning experimental conditions. We find that a previously observed difference in object recognition does not hold when adapting the experiment to make conditions more equitable between humans and machines. In presenting a checklist for comparative studies of visual reasoning in humans and machines, we hope to highlight how to overcome potential pitfalls in design and inference.
Bibliography:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 23
CMF and JB are both first authors on this work.
WB, TSAW and MB are joint senior authors.
ISSN:1534-7362
1534-7362
DOI:10.1167/jov.21.3.16