XShift: FPGA-efficient Binarized LLM with Joint Quantization and Sparsification

Binarization is a promising approach to significantly reduce computational complexity by replacing multiplications with hardwareefficient XNOR operations. However, the binarization of LLM activations often leads to severe accuracy degradation, while weight-only binarization fails to eliminate multip...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2025 62nd ACM/IEEE Design Automation Conference (DAC) s. 1 - 7
Hlavní autori: Zhou, Shuai, Tian, Huinan, Meng, Sisi, Chen, Jianli, Yu, Jun, Wang, Kun
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 22.06.2025
Predmet:
Accuracy
Decoding
Encoding
Field programmable gate arrays
Inference algorithms
Memory management
Quantization (signal)
Scalability
Systolic arrays
Transforms
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Binarization is a promising approach to significantly reduce computational complexity by replacing multiplications with hardwareefficient XNOR operations. However, the binarization of LLM activations often leads to severe accuracy degradation, while weight-only binarization fails to eliminate multipliers due to the Self-Attention mechanism. Furthermore, LLMs exhibit distinctive channel-level data distribution characteristics and differing computational and memory requirements between the Pre-fill and Decoding stages, necessitating a specialized inference framework. In response, we introduce XShift, an algorithm-hardware co-design framework optimized for efficient binarized LLM inference on FPGAs. XShift incorporates three key contributions: (1) a hardwarefriendly XNOR-Shift Encoding (XSE) format that transforms traditional multiplications into XNOR and shift operations, ensuring scalability and precision; (2) Hardware Adaptive Outlier and Sparsity (HAOS) techniques, which exploit channel-level data distribution and systolic array architectures for optimized quantization and sparsification; and (3) a dedicated hardware accelerator featuring an XNOR-Shift Systolic Array (XSSA) and an enhanced Base-2 SoftMax Converter (BSMC), designed to address the specific computational demands of binarized LLMs. Experimental evaluations on the Alveo U280 and U50 FPGA demonstrate that XShift achieves a \mathbf{1 0 - 1 5 x} reduction in DSP resource usage while surpassing existing accelerators and GPUs in inference performance. Specifically, XShift delivers an average speedup of 4.174.76 x and a 14.29-6.95 x improvement in energy efficiency, alongside lower perplexity compared to other low-precision LLM techniques. These results underscore the potential of XShift for edge deployment of LLMs.
DOI:10.1109/DAC63849.2025.11133363