A non-sequential refinement approach to improve word embeddings using GPU-based string matching algorithms

Unlike other word embedding models that learn word vectors for a collection of words sequentially, this paper proposes a non-sequential refinement approach to improve the vectors of particular words non-sequentially using a string matching algorithm to speed up the process. The key idea is to change...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Cluster computing Ročník 24; číslo 4; s. 3123 - 3134
Hlavní autoři: Naderalvojoud, Behzad, Ozsoy, Adnan
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.12.2021
Springer Nature B.V
Témata:
Algorithms
Computer Communication Networks
Computer Science
Embedding
Hypotheses
Language
Learning
Neural networks
Operating Systems
Probability
Processor Architectures
Semantics
String matching
Training
Word embedding
Semantic model
String matching
GPGPU
ISSN:1386-7857, 1573-7543
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Unlike other word embedding models that learn word vectors for a collection of words sequentially, this paper proposes a non-sequential refinement approach to improve the vectors of particular words non-sequentially using a string matching algorithm to speed up the process. The key idea is to change the order of training in the embedding learning model and force it to learn the vector of a particular word completely before skipping to other target words. The learned vector of the given word and its context vectors are then used to train other target words. In this case, later words can be trained by the word vectors that are more accurate. In this study, the effect of training order in the Skip-gram model is investigated and a quantitative and qualitative comparison is made between the learned vectors in the word similarity task. To speed up the process, a GPU based string matching algorithm is used to find the occurrences of the given word in the training corpus. Incorporating the GPU-based string matching algorithm into the Skip-gram model to refine particular word vectors is, to our best knowledge, the first use case in the literature. Additionally, we provide in-depth analysis of GPU parallelization and identification of string matching algorithms that are suitable for integrating into word embedding models.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1386-7857
1573-7543
DOI:10.1007/s10586-021-03321-4