HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference

The Mixture of Experts (MoE) architecture has demonstrated significant advantages as it enables to increase the model capacity without a proportional increase in computation. However, the large MoE model size still introduces substantial memory demands, which usually requires expert offloading on re...

Full description

Bibliographic Details

Published in:	2025 62nd ACM/IEEE Design Automation Conference (DAC) pp. 1 - 7
Main Authors:	Zhong, Shuzhang, Sun, Yanfan, Liang, Ling, Wang, Runsheng, Huang, Ru, Li, Meng
Format:	Conference Proceeding
Language:	English
Published:	IEEE 22.06.2025
Subjects:	Computational modeling Dynamic scheduling Hands Heuristic algorithms Inference algorithms Load modeling Prefetching Rendering (computer graphics) Resource management Schedules
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!