Non‐linear activation function approximation using a REMEZ algorithm

Here a more accurate piecewise approximation (PWA) scheme for non‐linear activation function is proposed. It utilizes a precision‐controlled recursive algorithm to predict a sub‐range; after that, the REMEZ algorithm is used to find the corresponding approximation function. The PWA realized in three...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IET circuits, devices & systems Ročník 15; číslo 7; s. 630 - 640
Hlavní autori: Chiluveru, Samba Raju, Tripathy, Manoj, Bibhudutta
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: John Wiley & Sons, Inc 01.10.2021
Wiley
Predmet:
Algorithms
CMOS memory circuits
Comparative analysis
content‐addressable storage
integrated circuit design
memristor circuits
Monte Carlo methods
ISSN:1751-858X, 1751-8598
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Here a more accurate piecewise approximation (PWA) scheme for non‐linear activation function is proposed. It utilizes a precision‐controlled recursive algorithm to predict a sub‐range; after that, the REMEZ algorithm is used to find the corresponding approximation function. The PWA realized in three ways: using first‐order functions, that is, piecewise linear model, second‐order functions (piecewise non‐linear model), and hybrid‐order model (a mixture of first‐order and second‐order functions). The hybrid‐order approximation employs the second‐order derivative of non‐linear activation function to decide the linear and non‐linear sub‐regions, correspondingly the first‐order and second‐order functions are predicted, respectively. The accuracy is compared to the present state‐of‐the‐art approximation schemes. The multi‐layer perceptron model is designed to implement XOR‐gate, and it uses an approximate activation function. The hardware utilization is measured using the TSMC 0.18‐μm library with the Synopsys Design Compiler. Result reveals that the proposed approximation scheme efficiently approximates the non‐linear activation functions.
ISSN:1751-858X
1751-8598
DOI:10.1049/cds2.12058