Enhancing Multi-Turn Dialogue Generation with MoE-Based Multi-Latent Variable Fusion

Generating relevant and diverse responses remains challenging for non-pretrained small models in open-domain multi-turn dialogues. To address the one-to-many mapping problem caused by conversational uncertainty, we propose a novel framework integrating Variational Autoencoder (VAE) with a Mixture-of...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2025 IEEE 6th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) s. 1 - 4
Hlavní autoři: Cui, Zishun, Sun, Xiao
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 11.04.2025
Témata:
Artificial intelligence
Autoencoders
Context modeling
Dialogue generation
Diversity reception
Information technology
Latent variables
Mixture of Experts
Multi-turn dialogue
Seminars
Uncertainty
Variational Autoencoder
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Generating relevant and diverse responses remains challenging for non-pretrained small models in open-domain multi-turn dialogues. To address the one-to-many mapping problem caused by conversational uncertainty, we propose a novel framework integrating Variational Autoencoder (VAE) with a Mixture-of-Experts (MoE) model. The VAE component learns three distinct latent variables: background, roles, and topic, to capture dynamic conversational factors, while the MoE module employs context-aware gating to dynamically activate domain-specific experts based on these variables. Extensive experiments on multi-turn dialogue datasets show that our approach outperforms state-of-the-art non-pretrained baselines, particularly in improving the relevance and diversity of responses.
DOI:10.1109/AINIT65432.2025.11035140