ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching

The Transformer architecture has significantly advanced natural language processing (NLP) and has been foundational in developing large language models (LLMs) such as LLaMA and OPT, which have come to dominate a broad range of NLP tasks. Despite their superior accuracy, LLMs present unique challenge...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA) S. 1005 - 1017
Hauptverfasser:	Zhao, Youpeng, Wu, Di, Wang, Jun
Format:	Tagungsbericht
Sprache:	Englisch
Veröffentlicht:	IEEE 29.06.2024
Schlagworte:	Accuracy Heuristic algorithms Large language models Machine Learning Systems Memory management Tensors Throughput Transformers
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!