A Microscaling Multi-Mode Gain-Cell Computing-in-Memory Macro for Advanced AI Edge Device

The microscaling (MX) format is an emerging data representation that quantizes high-bitwidth floating-point (FP) values into low-bitwidth FP-like values with a shared-scale (SS) exponent. When implemented with computing-in-memory (CIM), MX allows an attractive tradeoff between accuracy and hardware...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE journal of solid-state circuits pp. 1 - 14
Main Authors:	Tien, Jen-Chun, Wu, Ping-Chun, Khwa, Win-San, Sanjay Lele, Ashwin, Su, Jian-Wei, Cheng, Chiao-Yen, Hsu, Jun-Ming, Chen, Yu-Chen, Hsieh, Le-Jung, Bai, Jyun-Cheng, Kao, Yu-Sheng, Lou, Tsung-Han, Wu, Jui-Jen, Lo, Chung-Chuan, Liu, Ren-Shuo, Hsieh, Chih-Cheng, Tang, Kea-Tiong, Chang, Meng-Fan
Format:	Journal Article
Language:	English
Published:	IEEE 2025
Subjects:	Accuracy Adders Artificial intelligence (AI) Artificial neural networks Common Information Model (computing) Common Information Model (electricity) Computer architecture computing-in-memory (CIM) Data transfer Energy consumption gain cell (GC) Hardware In-memory computing microscaling (MX) multiply-and-accumulate (MAC)
ISSN:	0018-9200, 1558-173X
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	The microscaling (MX) format is an emerging data representation that quantizes high-bitwidth floating-point (FP) values into low-bitwidth FP-like values with a shared-scale (SS) exponent. When implemented with computing-in-memory (CIM), MX allows an attractive tradeoff between accuracy and hardware efficiency for specific neural network (NN) workloads. This work presents the first multi-mode gain-cell (GC) CIM macro capable of processing MX, integer (INT), and FP multiply-and-accumulate (MAC) operations with high energy efficiency (EEF) and area efficiency (AEF). The proposed macro employs four important innovations: 1) a multi-mode input processing unit (M2-IPU) with SS-variance-aware MAC flow (SS-VAF) for SS processing and SS alignment within the CIM macro to reduce system-to-CIM data transfer and compute energy; 2) a pattern-aware hybrid adder tree (PAH-ADT), which improves EEF and AEF by optimizing the common input patterns; 3) an accumulation-aware data flow (A2-DF) that adjusts the write path based on accumulation size to reduce data transfer energy; and 4) a 3.xT GC, which boosts data retention time (DRT) by increasing parasitic capacitance without additional area overhead. A 16-nm FinFET 216-kb MX-INT-FP multi-mode GC-CIM macro achieved 133.5 TFLOPS/W for MX-MAC with MXINT8 input, MXINT8 weight, and FP32 output; and 91.9 TFLOPS/W for FP-MAC with BF16 input, BF16 weight, and FP32 output.
AbstractList	The microscaling (MX) format is an emerging data representation that quantizes high-bitwidth floating-point (FP) values into low-bitwidth FP-like values with a shared-scale (SS) exponent. When implemented with computing-in-memory (CIM), MX allows an attractive tradeoff between accuracy and hardware efficiency for specific neural network (NN) workloads. This work presents the first multi-mode gain-cell (GC) CIM macro capable of processing MX, integer (INT), and FP multiply-and-accumulate (MAC) operations with high energy efficiency (EEF) and area efficiency (AEF). The proposed macro employs four important innovations: 1) a multi-mode input processing unit (M2-IPU) with SS-variance-aware MAC flow (SS-VAF) for SS processing and SS alignment within the CIM macro to reduce system-to-CIM data transfer and compute energy; 2) a pattern-aware hybrid adder tree (PAH-ADT), which improves EEF and AEF by optimizing the common input patterns; 3) an accumulation-aware data flow (A2-DF) that adjusts the write path based on accumulation size to reduce data transfer energy; and 4) a 3.xT GC, which boosts data retention time (DRT) by increasing parasitic capacitance without additional area overhead. A 16-nm FinFET 216-kb MX-INT-FP multi-mode GC-CIM macro achieved 133.5 TFLOPS/W for MX-MAC with MXINT8 input, MXINT8 weight, and FP32 output; and 91.9 TFLOPS/W for FP-MAC with BF16 input, BF16 weight, and FP32 output.
Author	Hsu, Jun-Ming Hsieh, Le-Jung Cheng, Chiao-Yen Su, Jian-Wei Tien, Jen-Chun Chen, Yu-Chen Lou, Tsung-Han Lo, Chung-Chuan Liu, Ren-Shuo Wu, Jui-Jen Kao, Yu-Sheng Sanjay Lele, Ashwin Tang, Kea-Tiong Chang, Meng-Fan Wu, Ping-Chun Khwa, Win-San Hsieh, Chih-Cheng Bai, Jyun-Cheng
Author_xml	– sequence: 1 givenname: Jen-Chun orcidid: 0009-0005-3129-1554 surname: Tien fullname: Tien, Jen-Chun email: jimytien@gapp.nthu.edu.tw organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 2 givenname: Ping-Chun orcidid: 0009-0008-8481-3012 surname: Wu fullname: Wu, Ping-Chun organization: Taiwan Semiconductor Manufacturing Company (TSMC), Hsinchu, Taiwan – sequence: 3 givenname: Win-San orcidid: 0000-0002-6283-3564 surname: Khwa fullname: Khwa, Win-San organization: Taiwan Semiconductor Manufacturing Company (TSMC), Hsinchu, Taiwan – sequence: 4 givenname: Ashwin orcidid: 0000-0002-2440-905X surname: Sanjay Lele fullname: Sanjay Lele, Ashwin organization: Taiwan Semiconductor Manufacturing Company (TSMC), Hsinchu, Taiwan – sequence: 5 givenname: Jian-Wei surname: Su fullname: Su, Jian-Wei organization: Industrial Technology Research Institute (ITRI), Hsinchu, Taiwan – sequence: 6 givenname: Chiao-Yen orcidid: 0009-0009-4898-2537 surname: Cheng fullname: Cheng, Chiao-Yen organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 7 givenname: Jun-Ming orcidid: 0009-0006-7126-4275 surname: Hsu fullname: Hsu, Jun-Ming organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 8 givenname: Yu-Chen orcidid: 0009-0006-8079-9087 surname: Chen fullname: Chen, Yu-Chen organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 9 givenname: Le-Jung surname: Hsieh fullname: Hsieh, Le-Jung organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 10 givenname: Jyun-Cheng orcidid: 0009-0002-5525-4166 surname: Bai fullname: Bai, Jyun-Cheng organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 11 givenname: Yu-Sheng orcidid: 0009-0006-2008-4004 surname: Kao fullname: Kao, Yu-Sheng organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 12 givenname: Tsung-Han orcidid: 0009-0000-3268-4345 surname: Lou fullname: Lou, Tsung-Han organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 13 givenname: Jui-Jen orcidid: 0009-0007-6161-3539 surname: Wu fullname: Wu, Jui-Jen organization: Taiwan Semiconductor Manufacturing Company (TSMC), Hsinchu, Taiwan – sequence: 14 givenname: Chung-Chuan orcidid: 0000-0001-7737-7250 surname: Lo fullname: Lo, Chung-Chuan organization: Institute of Systems Neuroscience, NTHU, Hsinchu, Taiwan – sequence: 15 givenname: Ren-Shuo orcidid: 0000-0002-5311-4955 surname: Liu fullname: Liu, Ren-Shuo organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 16 givenname: Chih-Cheng orcidid: 0000-0003-4070-5059 surname: Hsieh fullname: Hsieh, Chih-Cheng organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 17 givenname: Kea-Tiong orcidid: 0000-0002-9689-1236 surname: Tang fullname: Tang, Kea-Tiong organization: National Tsing Hua University (NTHU), Hsinchu, Taiwan – sequence: 18 givenname: Meng-Fan orcidid: 0000-0001-6905-6350 surname: Chang fullname: Chang, Meng-Fan organization: NTHU, Hsinchu, Taiwan
BookMark	eNpFkM1qwzAMgM3oYG23Bxjs4BdwJ_819jFkXdfRsEM32E7BOHLJSJOStIW-_Rxa2MmypE9I34SMmrZBQh45zDgH-_y-2WQzAULP5JwnVqkbMuZaG8YT-T0iYwBumBUAd2TS97_xq5ThY_KT0rzyXdt7V1fNlubH-lCxvC2RLl3VsAzrmmbtbn88xDKLmRx3bXemuYsUDW1H0_LkGo8lTVd0UW6RvuCp8nhPboOre3y4vlPy9br4zN7Y-mO5ytI180LAgRnkPMjSJi4BMDYEa7RFH7Sdx8h7pX1cNQjtlCoBrNcREw4FBGfmIZFTwi9zhyv6DkOx76qd684Fh2JwUwxuisFNcXUTmacLUyHifz8XIK1M5B8cQGC7
CODEN	IJSCBC
ContentType	Journal Article
DBID	97E RIA RIE AAYXX CITATION
DOI	10.1109/JSSC.2025.3617944
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef
DatabaseTitle	CrossRef
DatabaseTitleList
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1558-173X
EndPage	14
ExternalDocumentID	10_1109_JSSC_2025_3617944 11203937
Genre	orig-research
GroupedDBID	-~X .DC 0R~ 29I 4.4 5GY 6IK 97E AAJGR AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS ACIWK ACNCT AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS F5P HZ~ IFIPE IPLJI JAVBF LAI M43 O9- OCL P2P PZZ RIA RIE RNS TAE TN5 3EH 41~ 5VS AAYXX AETIX AGSQL AI. AIBXA ALLEH CITATION EJD H~9 IAAWW IBMZZ ICLAB IFJZH UKR VH1
ID	FETCH-LOGICAL-c220t-8e11f3d97a70089ff9859ecf596985cc45c144f25a44d009c52202ae20fa86f73
IEDL.DBID	RIE
ISICitedReferencesCount	0
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001596989200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0018-9200
IngestDate	Sat Nov 29 07:08:45 EST 2025 Sat Oct 25 03:11:26 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c220t-8e11f3d97a70089ff9859ecf596985cc45c144f25a44d009c52202ae20fa86f73
ORCID	0000-0002-9689-1236 0009-0006-8079-9087 0009-0006-7126-4275 0009-0008-8481-3012 0000-0003-4070-5059 0000-0002-6283-3564 0000-0002-2440-905X 0009-0006-2008-4004 0000-0001-6905-6350 0000-0002-5311-4955 0000-0001-7737-7250 0009-0009-4898-2537 0009-0002-5525-4166 0009-0007-6161-3539 0009-0000-3268-4345 0009-0005-3129-1554
PageCount	14
ParticipantIDs	ieee_primary_11203937 crossref_primary_10_1109_JSSC_2025_3617944
PublicationCentury	2000
PublicationDate	2025-00-00
PublicationDateYYYYMMDD	2025-01-01
PublicationDate_xml	– year: 2025 text: 2025-00-00
PublicationDecade	2020
PublicationTitle	IEEE journal of solid-state circuits
PublicationTitleAbbrev	JSSC
PublicationYear	2025
Publisher	IEEE
Publisher_xml	– name: IEEE
SSID	ssj0014481
Score	2.4641902
Snippet	The microscaling (MX) format is an emerging data representation that quantizes high-bitwidth floating-point (FP) values into low-bitwidth FP-like values with a...
SourceID	crossref ieee
SourceType	Index Database Publisher
StartPage	1
SubjectTerms	Accuracy Adders Artificial intelligence (AI) Artificial neural networks Common Information Model (computing) Common Information Model (electricity) Computer architecture computing-in-memory (CIM) Data transfer Energy consumption gain cell (GC) Hardware In-memory computing microscaling (MX) multiply-and-accumulate (MAC)
Title	A Microscaling Multi-Mode Gain-Cell Computing-in-Memory Macro for Advanced AI Edge Device
URI	https://ieeexplore.ieee.org/document/11203937
WOSCitedRecordID	wos001596989200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-173X dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014481 issn: 0018-9200 databaseCode: RIE dateStart: 19660101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA9ueNCDnxPnFzl4ErK1adM2xzLnF2wIU5inkiYvMhid7EPwv_elrToPHryFNA_aX5K-X5L38iPk0guMVoYbpgUohvwWmEJewKS2wsst8FxUYhPxcJiMx_KxTlYvc2EAoAw-g44rlmf5ZqZXbqusi9zApZLGDdKI46hK1vo-MsB1RiWP5-MMxr6vjzB9T3YfRqMeLgW56ASRG4DhLye0pqpSOpWb3X--zh7ZqdkjTavu3icbUByQ7bU7BQ_JS0oHLshugeBjBS0zbJmTPKO3alKwHkyntNJywMcMawYu2PaDDhRaUeSwNK3jAmh6T_vmFeg1uP9Jizzf9J96d6zWT2Cac2_JEvB9GxgZqxg9vbRWJkICdoKMsKR1KDSiZrlQYWiQa2nkYh5XwD2rksjGwRFpFrMCjgmVkUZrgx9rbKggkH6u3dQVCZgch0GbXH0Bmr1V12Rk5fLCk5lDP3PoZzX6bdJyYP40rHE8-aP-lGw582rj44w0l_MVnJNN_b6cLOYX5Sj4BPzHrzM
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dT8IwEG8UTdQHPzHiZx98Mils3crWR4IoKBATMMGnpbRXQ0LA8GHif-91m4oPPvjWdOuy_e66-1171yPk2guMVoYbpgUohvwWmEJewKS2whta4EORFZuIut14MJBPebJ6mgsDAGnwGZRdM93LN1O9dEtlFeQGLpU0WicbIgy5l6VrfW8aoKeRFcjzcQ6j9PNNTN-TlYder47OIBfloOpUMPxlhlbqqqRm5W7vny-0T3Zz_khrmcAPyBpMDsnOyqmCR-SlRjsuzG6O8GMHTXNsmSt6Ru_VaMLqMB7TrJoDXmbY03Hhth-0o3AURRZLa3lkAK21aMO8Ar0F90cpkue7Rr_eZHkFBaY59xYsBt-3gZGRitDWS2tlLCSgGGQVW1qHQiNqlgsVhgbZlkY25nEF3LMqrtooOCaFyXQCJ4TKqsbRBj_W2FBBIP2hdpNXxGCGqAglcvMFaPKWHZSRpA6GJxOHfuLQT3L0S6TowPy5Mcfx9I_-K7LV7HfaSbvVfTwj2-5R2TLIOSksZku4IJv6fTGazy5TjfgES_eyeg
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Microscaling+Multi-Mode+Gain-Cell+Computing-in-Memory+Macro+for+Advanced+AI+Edge+Device&rft.jtitle=IEEE+journal+of+solid-state+circuits&rft.au=Tien%2C+Jen-Chun&rft.au=Wu%2C+Ping-Chun&rft.au=Khwa%2C+Win-San&rft.au=Sanjay+Lele%2C+Ashwin&rft.date=2025&rft.pub=IEEE&rft.issn=0018-9200&rft.spage=1&rft.epage=14&rft_id=info:doi/10.1109%2FJSSC.2025.3617944&rft.externalDocID=11203937
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9200&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9200&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9200&client=summon