Mv-Math: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts

Multimodal Large Language Models (MLLMs) have shown promising capabilities in mathematical reasoning within visual contexts across various datasets. However, most existing multimodal math benchmarks are limited to single-visual contexts, which diverges from the multi-visual scenarios commonly encoun...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 19541 - 19551
Main Authors: Wang, Peijie, Li, Zhong-Zhi, Yin, Fei, Ran, Dekang, Liu, Cheng-Lin
Format: Conference Proceeding
Language:English
Published: IEEE 10.06.2025
Subjects:
Benchmark testing
Codes
Cognition
Computational modeling
Computer vision
Large language models
math reasoning
Mathematical models
multi-image reasoning
multimodal reasoning
Pattern recognition
Systematics
Visualization
ISSN:1063-6919
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Multimodal Large Language Models (MLLMs) have shown promising capabilities in mathematical reasoning within visual contexts across various datasets. However, most existing multimodal math benchmarks are limited to single-visual contexts, which diverges from the multi-visual scenarios commonly encountered in real-world mathematical applications. To address this gap, we introduce MV-MATH: a meticulously curated dataset of 2,009 high-quality mathematical problems. Each problem integrates multiple images interleaved with text, derived from authentic K-12 scenarios, and enriched with detailed annotations. MV-MATH includes multiple-choice, free-form, and multi-step questions, covering 11 subject areas across 3 difficulty levels, and serves as a comprehensive and rigorous benchmark for assessing MLLMs' mathematical reasoning in multi-visual contexts. Through extensive experimentation, we observe that MLLMs encounter substantial challenges in multi-visual math tasks, with a considerable performance gap relative to human capabilities on MV-MATH. Furthermore, we analyze the performance and error patterns of various models, providing insights into MLLMs' mathematical reasoning capabilities within multi-visual settings. The data and code: https://eternal8080.github.io/MV-MATH.github.io/.
ISSN:1063-6919
DOI:10.1109/CVPR52734.2025.01820