A Framework of Cross-Modal Learning for Solving Geometry Problems

This paper presents a framework of deep cross-modal learning networks for solving geometry problems. Existing geometry solvers either focus on single-modal problems or multi-modal problems, and they cannot fit each other. In this paper, we propose a cross-modal learning architecture for solving geom...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Proceedings of IEEE International Conference on Teaching, Assessment, and Learning for Engineering (Online) s. 506 - 512
Hlavní autori: Guo, Fucheng, Jian, Pengpeng, Wang, Yanli, Wang, Qingjiang
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 05.12.2021
Predmet:
Automatic solver
Cognition
cross-modal learning
Education
Feature extraction
Geometry
geometry problems
Head
shared encoder-decoder
Transformers
Transforms
ISSN:2470-6698
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:This paper presents a framework of deep cross-modal learning networks for solving geometry problems. Existing geometry solvers either focus on single-modal problems or multi-modal problems, and they cannot fit each other. In this paper, we propose a cross-modal learning architecture for solving geometry problems, which can effectively solve both text and image-text geometry problems. To attack the representation of cross-modal features is a key challenge in understanding geometry problems that this paper adopts a shared encoder, in which the text and (or) image features are masked by using self-attention units and a multi-layer transformer is used to realize the interaction between cross-modal features. Further, we adopt a shared decoder to decode the single-modal features or the series sequence of multi-modal features according to the input of the encoder is single-modal or multi-modal problems. The representation of the decoder is transferred to task-specific heads for geometry relations extraction, theorems reasoning and geometry problem solving. The proposed algorithm can produce the solutions of geometry problems in a readable way. The experimental results shown that the proposed algorithm has good property and performance in solving geometry problems.
ISSN:2470-6698
DOI:10.1109/TALE52509.2021.9678945