Enhancing Multi-Turn Dialogue Generation with MoE-Based Multi-Latent Variable Fusion

Generating relevant and diverse responses remains challenging for non-pretrained small models in open-domain multi-turn dialogues. To address the one-to-many mapping problem caused by conversational uncertainty, we propose a novel framework integrating Variational Autoencoder (VAE) with a Mixture-of...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2025 IEEE 6th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) S. 1 - 4
Hauptverfasser: Cui, Zishun, Sun, Xiao
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 11.04.2025
Schlagworte:
Artificial intelligence
Autoencoders
Context modeling
Dialogue generation
Diversity reception
Information technology
Latent variables
Mixture of Experts
Multi-turn dialogue
Seminars
Uncertainty
Variational Autoencoder
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Generating relevant and diverse responses remains challenging for non-pretrained small models in open-domain multi-turn dialogues. To address the one-to-many mapping problem caused by conversational uncertainty, we propose a novel framework integrating Variational Autoencoder (VAE) with a Mixture-of-Experts (MoE) model. The VAE component learns three distinct latent variables: background, roles, and topic, to capture dynamic conversational factors, while the MoE module employs context-aware gating to dynamically activate domain-specific experts based on these variables. Extensive experiments on multi-turn dialogue datasets show that our approach outperforms state-of-the-art non-pretrained baselines, particularly in improving the relevance and diversity of responses.
DOI:10.1109/AINIT65432.2025.11035140