A non-sequential refinement approach to improve word embeddings using GPU-based string matching algorithms

Unlike other word embedding models that learn word vectors for a collection of words sequentially, this paper proposes a non-sequential refinement approach to improve the vectors of particular words non-sequentially using a string matching algorithm to speed up the process. The key idea is to change...

Full description

Saved in:
Bibliographic Details
Published in:Cluster computing Vol. 24; no. 4; pp. 3123 - 3134
Main Authors: Naderalvojoud, Behzad, Ozsoy, Adnan
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2021
Springer Nature B.V
Subjects:
Algorithms
Computer Communication Networks
Computer Science
Embedding
Hypotheses
Language
Learning
Neural networks
Operating Systems
Probability
Processor Architectures
Semantics
String matching
Training
Word embedding
Semantic model
String matching
GPGPU
ISSN:1386-7857, 1573-7543
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Unlike other word embedding models that learn word vectors for a collection of words sequentially, this paper proposes a non-sequential refinement approach to improve the vectors of particular words non-sequentially using a string matching algorithm to speed up the process. The key idea is to change the order of training in the embedding learning model and force it to learn the vector of a particular word completely before skipping to other target words. The learned vector of the given word and its context vectors are then used to train other target words. In this case, later words can be trained by the word vectors that are more accurate. In this study, the effect of training order in the Skip-gram model is investigated and a quantitative and qualitative comparison is made between the learned vectors in the word similarity task. To speed up the process, a GPU based string matching algorithm is used to find the occurrences of the given word in the training corpus. Incorporating the GPU-based string matching algorithm into the Skip-gram model to refine particular word vectors is, to our best knowledge, the first use case in the literature. Additionally, we provide in-depth analysis of GPU parallelization and identification of string matching algorithms that are suitable for integrating into word embedding models.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1386-7857
1573-7543
DOI:10.1007/s10586-021-03321-4