A Method to Solve overflow or underflow errors Resulting from Activation functions in Convolutional Neural Networks
In this paper, a method is proposed to correct overflow errors of resulting values that can occur when the cross-entropy loss function is used as a softmax and ReLU as an activation function of the convolutional neural network. Since conventional softmax and cross-entropy functions include exponenti...
Uloženo v:
| Vydáno v: | International Information Institute (Tokyo). Information Ročník 20; číslo 11; s. 8151 - 8158 |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Koganei
International Information Institute
01.11.2017
|
| Témata: | |
| ISSN: | 1343-4500, 1344-8994 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | In this paper, a method is proposed to correct overflow errors of resulting values that can occur when the cross-entropy loss function is used as a softmax and ReLU as an activation function of the convolutional neural network. Since conventional softmax and cross-entropy functions include exponential and log computations, they have a problem of overflow or underflow errors, which produce a range of numbers such as convergence to zero or infinite divergence when a value of the final output layer becomes too large. In the present paper, the above problem is solved by transforming an equation without affecting the functions of the conventional softmax and cross-entropy functions. Thus, training can be achieved to correcting the overflow or underflow error problem regardless of a size of values in the final output value. |
|---|---|
| AbstractList | In this paper, a method is proposed to correct overflow errors of resulting values that can occur when the cross-entropy loss function is used as a softmax and ReLU as an activation function of the convolutional neural network. Since conventional softmax and cross-entropy functions include exponential and log computations, they have a problem of overflow or underflow errors, which produce a range of numbers such as convergence to zero or infinite divergence when a value of the final output layer becomes too large. In the present paper, the above problem is solved by transforming an equation without affecting the functions of the conventional softmax and cross-entropy functions. Thus, training can be achieved to correcting the overflow or underflow error problem regardless of a size of values in the final output value. |
| Author | Jeong, JunMo Choi, SeJin Kim, ChiYong |
| Author_xml | – sequence: 1 givenname: JunMo surname: Jeong fullname: Jeong, JunMo – sequence: 2 givenname: SeJin surname: Choi fullname: Choi, SeJin – sequence: 3 givenname: ChiYong surname: Kim fullname: Kim, ChiYong |
| BookMark | eNotjttKAzEURYNUsNb-Q8DngTNJppM8luINqoKX55KZnNGpY47mMv1929qntTdsFvuSTTx5PGPTUipVaGPU5JhloSqACzaPcQsAEsAsoJyyuOSPmD7J8UT8lYYROY0YuoF2nALP3p0KhkAh8heMeUi9_-BdoG--bFM_2tST51327SFE3nu-Ij_SkA_dDvwJczgi7Sh8xSt23tkh4vzEGXu_vXlb3Rfr57uH1XJd-FJUqTB1J4VEY1UDQkFVCtWoTpTCmbKprYRGK7RYt9qBcChquUAFC9RtbZ3WTs7Y9b_3J9Bvxpg2W8phfyhuBOw9-7E28g-RLlwE |
| ContentType | Journal Article |
| Copyright | Copyright International Information Institute Nov 2017 |
| Copyright_xml | – notice: Copyright International Information Institute Nov 2017 |
| DBID | 7SC 8FD 8FE 8FG ABUWG AFKRA ARAPS BENPR BGLVJ BVBZV CCPQU DWQXO HCIFZ JQ2 L7M L~C L~D P5Z P62 PHGZM PHGZT PKEHL PQEST PQGLB PQQKQ PQUKI PRINS |
| DatabaseName | Computer and Information Systems Abstracts Technology Research Database ProQuest SciTech Collection ProQuest Technology Collection ProQuest Central (Alumni) ProQuest Central UK/Ireland Advanced Technologies & Computer Science Collection ProQuest Central Technology collection East & South Asia Database ProQuest One Community College ProQuest Central SciTech Premium Collection ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Premium ProQuest One Academic ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China |
| DatabaseTitle | Technology Collection Technology Research Database Computer and Information Systems Abstracts – Academic ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Computer Science Collection Computer and Information Systems Abstracts ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Central Korea ProQuest Central (New) Advanced Technologies Database with Aerospace Advanced Technologies & Aerospace Collection ProQuest One Academic Eastern Edition ProQuest Technology Collection ProQuest SciTech Collection East & South Asia Database Computer and Information Systems Abstracts Professional Advanced Technologies & Aerospace Database ProQuest One Academic UKI Edition ProQuest One Academic ProQuest One Academic (New) |
| DatabaseTitleList | Technology Collection |
| Database_xml | – sequence: 1 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| EISSN | 1344-8994 |
| EndPage | 8158 |
| GroupedDBID | 7SC 8FD 8FE 8FG ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS ARAPS BENPR BGLVJ BPHCQ BVBZV CCPQU DWQXO HCIFZ JQ2 L7M L~C L~D P62 PHGZM PHGZT PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PROAC |
| ID | FETCH-LOGICAL-n125t-97f323e9a4b02405124b4f212d91b7a30b84eae7c8d02de2736e406e8c7ad88d3 |
| IEDL.DBID | P5Z |
| ISSN | 1343-4500 |
| IngestDate | Sun Jul 13 03:47:17 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Issue | 11 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-n125t-97f323e9a4b02405124b4f212d91b7a30b84eae7c8d02de2736e406e8c7ad88d3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| PQID | 2021240689 |
| PQPubID | 936334 |
| PageCount | 8 |
| ParticipantIDs | proquest_journals_2021240689 |
| PublicationCentury | 2000 |
| PublicationDate | 20171101 |
| PublicationDateYYYYMMDD | 2017-11-01 |
| PublicationDate_xml | – month: 11 year: 2017 text: 20171101 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | Koganei |
| PublicationPlace_xml | – name: Koganei |
| PublicationTitle | International Information Institute (Tokyo). Information |
| PublicationYear | 2017 |
| Publisher | International Information Institute |
| Publisher_xml | – name: International Information Institute |
| SSID | ssj0003009601 |
| Score | 1.6699216 |
| Snippet | In this paper, a method is proposed to correct overflow errors of resulting values that can occur when the cross-entropy loss function is used as a softmax and... |
| SourceID | proquest |
| SourceType | Aggregation Database |
| StartPage | 8151 |
| SubjectTerms | Activation Algorithms Artificial neural networks Classification Divergence Entropy Error correction Methods Neural networks Neurons |
| Title | A Method to Solve overflow or underflow errors Resulting from Activation functions in Convolutional Neural Networks |
| URI | https://www.proquest.com/docview/2021240689 |
| Volume | 20 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVPQU databaseName: Advanced Technologies & Aerospace Database customDbUrl: eissn: 1344-8994 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0003009601 issn: 1343-4500 databaseCode: P5Z dateStart: 20111101 isFulltext: true titleUrlDefault: https://search.proquest.com/hightechjournals providerName: ProQuest – providerCode: PRVPQU databaseName: East & South Asia Database customDbUrl: eissn: 1344-8994 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0003009601 issn: 1343-4500 databaseCode: BVBZV dateStart: 20111101 isFulltext: true titleUrlDefault: https://search.proquest.com/eastsouthasia providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 1344-8994 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0003009601 issn: 1343-4500 databaseCode: BENPR dateStart: 20111101 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV07T8MwELagZWDhIUA8SuWBNVISO008oVC1YqGqCkgVSxXbF1SpiiEO5e_jc1MYkFiYoihD4td9913u7iPkplCJHshYBgAMCQobBEXpzlXKhHT8WRSRUF5sIp1MsvlcTNuAm23TKrc20RtqbRTGyJGkR4g-mbh9ew9QNQr_rrYSGruki10SULphmrx8x1iYd9A3nIuzgCdh-MvkehwZH_73C47IQetB0nyz5MdkB6oTYnP64LWgaWPoo1mtgWJuZrkyn9TUFCvFNjdQ16a2dAYWMwmrV4r1JTRXW5UzikDn9yJdVnRoqnW7N90rsZGHv_jMcXtKnsejp-F90OopBJVzY5pApCWLGYiCS-xs5qCeS166QWkRybRgocw4FJCqTIexBufYDMCNFzKVFjrLNDsjncpUcE6olFLwOEwgDksOLJHCES9ZRqVOojSU6oL0tnO4aA-FXfxM4OXfj6_Ifozo6Uv-eqTT1B9wTfbUulnauk-6d6PJdNb3a_0FzQW3Gw |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1LS8NAEB5qK-jFByo-qu5BvQWSbF578NBWS0sfiNZSvNRsMpFCSTSJLf4pf6O721QPgrcePIUQ2GWZnfnm28zsB3DhB3bocJNriFQSFOpofiT8yqWMC_7MfIMFSmzC7fe90YjdleBz2QsjyyqXMVEF6jAJ5Bm5JOmGRB-PFRWUHfyYC36WXbdvhDEvTbN5O2i0tEJCQIsFcucacyNqUmS-xeVlXgLdLG5FYqSQGdz1qc49C310Ay_UzRAFljsoJkEvcP3Q80Iqxr16fdOkSpX8m1tIdqxBxXNEblKGSn1Yfxp-n-pQRQkWLM-immXr-q8gr5Cruf3f1rwDW0WOTGqLTb0LJYz3IKuRnlK7JnlCHpLpDImsPo2myZwkKZG9cIsXTNMkzcg9ZrJWMn4hsoOG1IKljhuRUK68jUxi0kjiWeF9Ykp5VYl6qNr4bB8eV7L0AyjHSYyHQDjnzDJ1G009spDanAlqySMjCm3D1XlwBNWlzcaF22fjH4Md__35HDZag1533G33OyewacpcQTU4VqGcp-94CuvBLJ9k6Vmxwwg8r9rmXyLvEwE |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1NS8MwGA5zinjxAxU_puagx2LbtGtzEBmbwzEdww8YXmaTvJHBaLWtG_41f51J2upB8LaDp1IKTcP78eRJ3zcPQqcR90WTucwCIJqgkKYVSRVXAaFM8WcaOZQbsYlgMAhHIzqsoc-qF0aXVVY50SRqkXC9R65JuqPRJ6TnsiyLGHa6l69vllaQ0n9aKzmNwkX68DFX9C276HWUrc9ct3v10L62SoUBK1bAnls0kMQlQCOP6bO-FPh5zJNqIEEdFkTEZqEHEQQ8FLYrQEF9E9Q3QMiDSIShIOq9S2g5UBxTlxMO_afv_R1iyEHB9zxieb5t_0r3BsO6G_959ptovVw541bh6luoBvE2ylr41mhg4zzB98l0BljXpMppMsdJinWHXHEDaZqkGb6DTFdQxi9Y99XgFq_U3bAGeBODeBLjdhLPyphUQ-oDTMzFVMxnO-hxIfPcRfU4iWEPYcYY9VzbB9eWHhCfUUU4mXSk8J3AZnwfNSr7jctkkI1_jHfw9-MTtKqsOr7pDfqHaM3VCwjT9dhA9Tx9hyO0wmf5JEuPjath9Lxo634BarQaZw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Method+to+Solve+overflow+or+underflow+errors+Resulting+from+Activation+functions+in+Convolutional+Neural+Networks&rft.jtitle=International+Information+Institute+%28Tokyo%29.+Information&rft.au=Jeong%2C+JunMo&rft.au=Choi%2C+SeJin&rft.au=Kim%2C+ChiYong&rft.date=2017-11-01&rft.pub=International+Information+Institute&rft.issn=1343-4500&rft.eissn=1344-8994&rft.volume=20&rft.issue=11&rft.spage=8151&rft.epage=8158&rft.externalDBID=HAS_PDF_LINK |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1343-4500&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1343-4500&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1343-4500&client=summon |