A Comparison of Transformer and LSTM Encoder Decoder Models for ASR

We present competitive results using a Transformer encoder-decoder-attention model for end-to-end speech recognition needing less training time compared to a similarly performing LSTM model. We observe that the Transformer training is in general more stable compared to the LSTM, although it also see...

Full description

Bibliographic Details

Published in:	2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) pp. 8 - 15
Main Authors:	Zeyer, Albert, Bahar, Parnia, Irie, Kazuki, Schluter, Ralf, Ney, Hermann
Format:	Conference Proceeding
Language:	English
Published:	IEEE 01.12.2019
Subjects:	attention Convergence Convolutional codes Data models Decoding Encoding end-to-end ASR Hidden Markov models LSTM Training Transformer
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!