Large Language Models With Contrastive Decoding Algorithm for Hallucination Mitigation in Low‐Resource Languages

ABSTRACT Neural machine translation (NMT) has advanced with deep learning and large‐scale multilingual models, yet translating low‐resource languages often lacks sufficient training data and leads to hallucinations. This often results in translated content that diverges significantly from the source...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:CAAI Transactions on Intelligence Technology Ročník 10; číslo 4; s. 1104 - 1117
Hlavní autori: Hongying, Zan, Javed, Arifa, Abdullah, Muhammad, Rashid, Javed, Faheem, Muhammad
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Wiley 01.08.2025
Predmet:
ChatGLM
contrastive decoding
hallucination
LLAMA
LLM
low resource NMT
ISSN:2468-2322, 2468-6557, 2468-2322
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:ABSTRACT Neural machine translation (NMT) has advanced with deep learning and large‐scale multilingual models, yet translating low‐resource languages often lacks sufficient training data and leads to hallucinations. This often results in translated content that diverges significantly from the source text. This research proposes a refined Contrastive Decoding (CD) algorithm that dynamically adjusts weights of log probabilities from strong expert and weak amateur models to mitigate hallucinations in low‐resource NMT and improve translation quality. Advanced large language NMT models, including ChatGLM and LLaMA, are fine‐tuned and implemented for their superior contextual understanding and cross‐lingual capabilities. The refined CD algorithm evaluates multiple candidate translations using BLEU score, semantic similarity, and Named Entity Recognition accuracy. Extensive experimental results show substantial improvements in translation quality and a significant reduction in hallucination rates. Fine‐tuned models achieve higher evaluation metrics compared to baseline models and state‐of‐the‐art models. An ablation study confirms the contributions of each methodological component and highlights the effectiveness of the refined CD algorithm and advanced models in mitigating hallucinations. Notably, the refined methodology increased the BLEU score by approximately 30% compared to baseline models.
Bibliografia:The authors are highly grateful to their affiliated universities and institutes for providing research facilities. The research work of M. Faheem is supported by VTT Technical Research Center of Finland.
Funding
ISSN:2468-2322
2468-6557
2468-2322
DOI:10.1049/cit2.70004