Neuro-symbolic program corrector for introductory programming assignments

Automatic correction of programs is a challenging problem with numerous real world applications in security, verification, and education. One application that is becoming increasingly important is the correction of student submissions in online courses for providing feedback. Most existing program r...

Full description

Saved in:
Bibliographic Details
Published in:2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE) pp. 60 - 70
Main Authors: Bhatia, Sahil, Kohli, Pushmeet, Singh, Rishabh
Format: Conference Proceeding
Language:English
Published: New York, NY, USA ACM 27.05.2018
Series:ACM Conferences
Subjects:
Automated Feedback Generation
Computing methodologies > Machine learning > Machine learning approaches > Neural networks
Maintenance engineering
Neural guided search
Neural Program Correction
Prediction algorithms
Programming
Recurrent neural networks
Semantics
Software and its engineering > Software creation and management > Search-based software engineering
Software and its engineering > Software organization and properties > Software functional properties > Correctness > Functionality
Syntactics
neural guided search
automated feedback generation
neural program correction
ISBN:9781450356381, 1450356389
ISSN:1558-1225
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Automatic correction of programs is a challenging problem with numerous real world applications in security, verification, and education. One application that is becoming increasingly important is the correction of student submissions in online courses for providing feedback. Most existing program repair techniques analyze Abstract Syntax Trees (ASTs) of programs, which are unfortunately unavailable for programs with syntax errors. In this paper, we propose a novel Neuro-symbolic approach that combines neural networks with constraint-based reasoning. Specifically, our method first uses a Recurrent Neural Network (RNN) to perform syntax repairs for the buggy programs; subsequently, the resulting syntactically-fixed programs are repaired using constraint-based techniques to ensure functional correctness. The RNNs are trained using a corpus of syntactically correct submissions for a given programming assignment, and are then queried to fix syntax errors in an incorrect programming submission by replacing or inserting the predicted tokens at the error location. We evaluate our technique on a dataset comprising of over 14,500 student submissions with syntax errors. Our method is able to repair syntax errors in 60% (8689) of submissions, and finds functionally correct repairs for 23.8% (3455) submissions.
ISBN:9781450356381
1450356389
ISSN:1558-1225
DOI:10.1145/3180155.3180219