Enhancing Multi-Turn Dialogue Generation with MoE-Based Multi-Latent Variable Fusion

Generating relevant and diverse responses remains challenging for non-pretrained small models in open-domain multi-turn dialogues. To address the one-to-many mapping problem caused by conversational uncertainty, we propose a novel framework integrating Variational Autoencoder (VAE) with a Mixture-of...

Full description

Saved in:
Bibliographic Details
Published in:2025 IEEE 6th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) pp. 1 - 4
Main Authors: Cui, Zishun, Sun, Xiao
Format: Conference Proceeding
Language:English
Published: IEEE 11.04.2025
Subjects:
Artificial intelligence
Autoencoders
Context modeling
Dialogue generation
Diversity reception
Information technology
Latent variables
Mixture of Experts
Multi-turn dialogue
Seminars
Uncertainty
Variational Autoencoder
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Generating relevant and diverse responses remains challenging for non-pretrained small models in open-domain multi-turn dialogues. To address the one-to-many mapping problem caused by conversational uncertainty, we propose a novel framework integrating Variational Autoencoder (VAE) with a Mixture-of-Experts (MoE) model. The VAE component learns three distinct latent variables: background, roles, and topic, to capture dynamic conversational factors, while the MoE module employs context-aware gating to dynamically activate domain-specific experts based on these variables. Extensive experiments on multi-turn dialogue datasets show that our approach outperforms state-of-the-art non-pretrained baselines, particularly in improving the relevance and diversity of responses.
DOI:10.1109/AINIT65432.2025.11035140