Primer: Fast Private Transformer Inference on Encrypted Data

It is increasingly important to enable privacy-preserving inference for cloud services based on Transformers. Post-quantum cryptographic techniques, e.g., fully homomorphic encryption (FHE), and multi-party computation (MPC), are popular methods to support private Transformer inference. However, exi...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2023 60th ACM/IEEE Design Automation Conference (DAC) S. 1 - 6
Hauptverfasser: Zheng, Mengxin, Lou, Qian, Jiang, Lei
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 09.07.2023
Schlagworte:
Computational modeling
Cryptographic Protocol
Cryptography
Design automation
Fully Homomorphic Encryption
Multi-party computation
Natural language processing
Private Inference
Solids
Transformer
Transformers
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:It is increasingly important to enable privacy-preserving inference for cloud services based on Transformers. Post-quantum cryptographic techniques, e.g., fully homomorphic encryption (FHE), and multi-party computation (MPC), are popular methods to support private Transformer inference. However, existing works still suffer from prohibitively computational and communicational overhead. In this work, we present, Primer, to enable a fast and accurate Transformer over encrypted data for natural language processing tasks. In particular, Primer is constructed by a hybrid cryptographic protocol optimized for attention-based Transformer models, as well as techniques including computation merge and tokens-first ciphertext packing. Comprehensive experiments on encrypted language modeling show that Primer achieves state-of-the-art accuracy and reduces the inference latency by 90.6% ∼ 97.5% over previous methods.
DOI:10.1109/DAC56929.2023.10247719