On the computational power of winner-take-all
This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational models that involve competitive stages have so far been neglected in computational complexity theory, although they are widely used in comput...
Uloženo v:
| Vydáno v: | Neural computation Ročník 12; číslo 11; s. 2519 |
|---|---|
| Hlavní autor: | |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
United States
01.11.2000
|
| Témata: | |
| ISSN: | 0899-7667 |
| On-line přístup: | Zjistit podrobnosti o přístupu |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational models that involve competitive stages have so far been neglected in computational complexity theory, although they are widely used in computational brain models, artificial neural networks, and analog VLSI. Our theoretical analysis shows that winner-take-all is a surprisingly powerful computational module in comparison with threshold gates (also referred to as McCulloch-Pitts neurons) and sigmoidal gates. We prove an optimal quadratic lower bound for computing winner-take-all in any feedforward circuit consisting of threshold gates. In addition we show that arbitrary continuous functions can be approximated by circuits employing a single soft winner-take-all gate as their only nonlinear operation. Our theoretical analysis also provides answers to two basic questions raised by neurophysiologists in view of the well-known asymmetry between excitatory and inhibitory connections in cortical circuits: how much computational power of neural networks is lost if only positive weights are employed in weighted sums and how much adaptive capability is lost if only the positive weights are subject to plasticity. |
|---|---|
| AbstractList | This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational models that involve competitive stages have so far been neglected in computational complexity theory, although they are widely used in computational brain models, artificial neural networks, and analog VLSI. Our theoretical analysis shows that winner-take-all is a surprisingly powerful computational module in comparison with threshold gates (also referred to as McCulloch-Pitts neurons) and sigmoidal gates. We prove an optimal quadratic lower bound for computing winner-take-all in any feedforward circuit consisting of threshold gates. In addition we show that arbitrary continuous functions can be approximated by circuits employing a single soft winner-take-all gate as their only nonlinear operation. Our theoretical analysis also provides answers to two basic questions raised by neurophysiologists in view of the well-known asymmetry between excitatory and inhibitory connections in cortical circuits: how much computational power of neural networks is lost if only positive weights are employed in weighted sums and how much adaptive capability is lost if only the positive weights are subject to plasticity.This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational models that involve competitive stages have so far been neglected in computational complexity theory, although they are widely used in computational brain models, artificial neural networks, and analog VLSI. Our theoretical analysis shows that winner-take-all is a surprisingly powerful computational module in comparison with threshold gates (also referred to as McCulloch-Pitts neurons) and sigmoidal gates. We prove an optimal quadratic lower bound for computing winner-take-all in any feedforward circuit consisting of threshold gates. In addition we show that arbitrary continuous functions can be approximated by circuits employing a single soft winner-take-all gate as their only nonlinear operation. Our theoretical analysis also provides answers to two basic questions raised by neurophysiologists in view of the well-known asymmetry between excitatory and inhibitory connections in cortical circuits: how much computational power of neural networks is lost if only positive weights are employed in weighted sums and how much adaptive capability is lost if only the positive weights are subject to plasticity. This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational models that involve competitive stages have so far been neglected in computational complexity theory, although they are widely used in computational brain models, artificial neural networks, and analog VLSI. Our theoretical analysis shows that winner-take-all is a surprisingly powerful computational module in comparison with threshold gates (also referred to as McCulloch-Pitts neurons) and sigmoidal gates. We prove an optimal quadratic lower bound for computing winner-take-all in any feedforward circuit consisting of threshold gates. In addition we show that arbitrary continuous functions can be approximated by circuits employing a single soft winner-take-all gate as their only nonlinear operation. Our theoretical analysis also provides answers to two basic questions raised by neurophysiologists in view of the well-known asymmetry between excitatory and inhibitory connections in cortical circuits: how much computational power of neural networks is lost if only positive weights are employed in weighted sums and how much adaptive capability is lost if only the positive weights are subject to plasticity. |
| Author | Maass, W |
| Author_xml | – sequence: 1 givenname: W surname: Maass fullname: Maass, W organization: Institute for Theoretical Computer Science, Technische Universität Graz, Austria |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/11110125$$D View this record in MEDLINE/PubMed |
| BookMark | eNo1j01LxDAYhHNYcb_8Ax6kJ2_RN2maNEdZ_IKFvbjnkqZvsZomtUlZ_PdWXOcy8MwwMGuy8MEjIdcM7hiT_B5KrZWUADkAMFFytSCrX0hnqpZkHePHnEgGxSVZslnAeLEi9OCz9I6ZDf0wJZO64I3LhnDCMQttduq8x5Em84nUOLclF61xEa_OviHHp8e33QvdH55fdw97aoWWiVrGBHKbI9SmtLq1GmuluREFNlBbqwAaXrRSWxA6t4BS8lw0CqWYLxjNN-T2b3cYw9eEMVV9Fy06ZzyGKVaKCwWclXPx5lyc6h6bahi73ozf1f9B_gNUslFa |
| CitedBy_id | crossref_primary_10_1016_S0893_6080_03_00205_3 crossref_primary_10_1523_JNEUROSCI_0155_18_2018 crossref_primary_10_1109_TMC_2023_3294188 crossref_primary_10_1242_jeb_234963 crossref_primary_10_1023_B_NEPL_0000016837_13436_d3 crossref_primary_10_1109_TNN_2006_890804 crossref_primary_10_1007_s12559_023_10181_0 crossref_primary_10_1177_1477153516675910 crossref_primary_10_1177_1073858416631966 crossref_primary_10_1371_journal_pcbi_1009434 crossref_primary_10_1126_science_add8468 crossref_primary_10_1523_JNEUROSCI_2186_12_2012 crossref_primary_10_1016_j_ins_2012_10_011 crossref_primary_10_1016_j_neunet_2007_12_044 crossref_primary_10_1109_TNN_2008_2003287 crossref_primary_10_1109_TSMC_2016_2627579 crossref_primary_10_1016_j_neucom_2019_09_038 crossref_primary_10_1016_j_neucom_2014_01_074 crossref_primary_10_1038_s41598_017_18666_3 crossref_primary_10_3389_fncom_2015_00012 crossref_primary_10_1371_journal_pcbi_1013081 crossref_primary_10_3389_fnins_2019_01085 crossref_primary_10_1016_j_jcss_2004_04_001 crossref_primary_10_1109_TAC_2016_2578645 crossref_primary_10_1109_TNN_2011_2169086 crossref_primary_10_1162_neco_a_01047 crossref_primary_10_1088_2634_4386_ace64c crossref_primary_10_3390_s19020437 crossref_primary_10_1371_journal_pcbi_1003037 crossref_primary_10_1016_j_jmp_2020_102447 crossref_primary_10_1016_j_neucom_2017_06_070 crossref_primary_10_1155_2008_386059 crossref_primary_10_1016_j_cviu_2010_07_008 crossref_primary_10_1088_2634_4386_acc6e8 crossref_primary_10_3389_fncir_2020_00040 crossref_primary_10_1016_j_amc_2012_07_057 crossref_primary_10_1016_j_neulet_2008_02_036 crossref_primary_10_1016_j_neuron_2023_03_015 crossref_primary_10_1088_2634_4386_ad8c78 crossref_primary_10_1162_neco_2008_03_08_734 crossref_primary_10_1007_s12559_008_9000_9 crossref_primary_10_1016_j_arcontrol_2022_05_001 crossref_primary_10_3389_fnins_2018_00961 crossref_primary_10_7554_eLife_08362 crossref_primary_10_1088_2752_5724_accd87 crossref_primary_10_1109_ACCESS_2024_3479968 crossref_primary_10_1038_s41467_018_06899_3 crossref_primary_10_1038_s43588_021_00184_y crossref_primary_10_1016_j_ins_2021_01_059 crossref_primary_10_1109_TCYB_2021_3079457 crossref_primary_10_1007_s10827_018_0708_6 crossref_primary_10_1038_s42256_020_0187_0 crossref_primary_10_1088_2634_4386_ad850f crossref_primary_10_1371_journal_pcbi_1004039 crossref_primary_10_1016_S0304_3975_02_00097_X crossref_primary_10_1007_s42514_020_00037_6 crossref_primary_10_1016_j_asoc_2018_02_054 crossref_primary_10_3389_fncom_2017_00020 crossref_primary_10_3390_s22020440 crossref_primary_10_1016_j_neunet_2009_03_020 crossref_primary_10_1038_s41565_023_01391_6 crossref_primary_10_1109_JETCAS_2017_2771392 crossref_primary_10_1371_journal_pcbi_1004558 crossref_primary_10_3758_s13428_023_02295_y crossref_primary_10_1016_j_biosystems_2024_105164 crossref_primary_10_1080_23746149_2021_1894234 crossref_primary_10_1080_09540091_2014_956288 crossref_primary_10_1371_journal_pcbi_1012973 crossref_primary_10_1523_JNEUROSCI_3989_14_2015 crossref_primary_10_1038_s41467_022_28487_2 crossref_primary_10_1093_hmg_ddt539 crossref_primary_10_3389_fncom_2014_00068 crossref_primary_10_1007_s00285_024_02136_2 crossref_primary_10_1080_01677063_2020_1719407 crossref_primary_10_1007_s11370_024_00533_6 crossref_primary_10_1016_j_neunet_2007_04_020 crossref_primary_10_1109_TNNLS_2012_2202400 crossref_primary_10_1109_JPROC_2014_2313954 crossref_primary_10_1016_j_neucom_2025_130391 crossref_primary_10_1007_s00422_023_00956_x crossref_primary_10_1039_D0NR07865D crossref_primary_10_1109_TVLSI_2013_2245351 crossref_primary_10_1016_j_heliyon_2020_e05347 crossref_primary_10_1088_1361_6528_acebf5 crossref_primary_10_1109_JETCAS_2017_2777784 crossref_primary_10_1016_j_jsis_2010_06_001 crossref_primary_10_1371_journal_pcbi_1010270 crossref_primary_10_3758_s13428_018_1067_y crossref_primary_10_1162_NECO_a_00029 crossref_primary_10_1162_NECO_a_00304 crossref_primary_10_1016_j_neucom_2005_04_012 crossref_primary_10_1371_journal_pcbi_1009045 crossref_primary_10_1016_j_neunet_2021_07_033 crossref_primary_10_1162_neco_2009_07_08_829 crossref_primary_10_1016_j_automatica_2018_11_001 crossref_primary_10_1109_TNN_2006_881046 crossref_primary_10_1038_s41598_021_92170_7 crossref_primary_10_1109_LED_2023_3240419 crossref_primary_10_1109_TNN_2010_2050781 crossref_primary_10_1088_2634_4386_aca710 crossref_primary_10_1109_JAS_2022_105731 crossref_primary_10_1146_annurev_neuro_27_070203_144152 crossref_primary_10_1162_089976602760407955 crossref_primary_10_7554_eLife_73783 crossref_primary_10_1162_089976602760408008 crossref_primary_10_7554_eLife_51473 crossref_primary_10_1088_2634_4386_ac7c8a crossref_primary_10_1109_MSP_2019_2928376 crossref_primary_10_1109_TNNLS_2016_2582517 crossref_primary_10_1109_JLT_2020_3000670 crossref_primary_10_7554_eLife_101506 crossref_primary_10_1088_1361_6528_ab86e8 crossref_primary_10_1109_JPROC_2015_2444094 crossref_primary_10_1109_TSMC_2022_3189479 crossref_primary_10_1109_TCSI_2025_3549060 crossref_primary_10_3390_app12052679 crossref_primary_10_7554_eLife_101506_3 crossref_primary_10_1109_JETCAS_2023_3330069 crossref_primary_10_1088_2634_4386_ad2d5c crossref_primary_10_1002_aisy_202100054 crossref_primary_10_1007_s00422_005_0023_y crossref_primary_10_1016_j_neucom_2020_11_072 crossref_primary_10_1038_s43588_021_00157_1 crossref_primary_10_1109_TNNLS_2023_3237381 crossref_primary_10_1162_neco_a_01242 crossref_primary_10_1162_NECO_a_00091 crossref_primary_10_1007_s00429_021_02347_z crossref_primary_10_1016_j_neunet_2006_06_006 crossref_primary_10_1007_s00521_022_07345_8 crossref_primary_10_1007_s10489_014_0525_1 crossref_primary_10_1007_s12559_012_9141_8 crossref_primary_10_1016_j_patcog_2006_04_041 crossref_primary_10_1371_journal_pcbi_1003994 crossref_primary_10_1109_ACCESS_2020_3027966 crossref_primary_10_1016_S0304_3975_02_00099_3 crossref_primary_10_1016_j_asoc_2025_113115 crossref_primary_10_1016_j_neunet_2014_09_003 crossref_primary_10_1038_s44335_025_00024_6 crossref_primary_10_3389_fnsys_2015_00119 crossref_primary_10_1038_s41467_018_05517_6 crossref_primary_10_1162_neco_2010_03_09_980 crossref_primary_10_1016_j_neuron_2022_10_026 crossref_primary_10_4218_etrij_2020_0052 crossref_primary_10_1073_pnas_2312992121 crossref_primary_10_1162_neco_a_01113 crossref_primary_10_1126_science_adu1327 crossref_primary_10_1371_journal_pone_0099681 crossref_primary_10_1016_j_neucom_2011_10_017 crossref_primary_10_1016_j_ecoinf_2013_10_006 crossref_primary_10_1016_j_neunet_2004_07_003 crossref_primary_10_1155_2018_4573631 crossref_primary_10_1162_neco_2008_06_08_804 crossref_primary_10_1145_3304103 crossref_primary_10_1088_2516_1091_acb51c crossref_primary_10_1038_s41586_018_0289_6 crossref_primary_10_3389_fncom_2017_00070 crossref_primary_10_1007_s42113_020_00095_7 crossref_primary_10_1039_D0SC05860B crossref_primary_10_3389_fnins_2017_00269 crossref_primary_10_1016_j_cub_2021_08_051 crossref_primary_10_1088_2632_072X_ac3ad4 crossref_primary_10_1109_TNN_2010_2052631 crossref_primary_10_1371_journal_pcbi_1007579 crossref_primary_10_1016_j_ecocom_2024_101083 crossref_primary_10_1002_adfm_201804844 crossref_primary_10_1162_089976603322518731 crossref_primary_10_3389_fnins_2018_00665 crossref_primary_10_1002_adma_201808032 crossref_primary_10_1016_j_neunet_2007_12_036 crossref_primary_10_1073_pnas_1917551117 crossref_primary_10_1523_JNEUROSCI_1259_18_2018 crossref_primary_10_1162_neco_a_01074 |
| ContentType | Journal Article |
| DBID | CGR CUY CVF ECM EIF NPM 7X8 |
| DOI | 10.1162/089976600300014827 |
| DatabaseName | Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic |
| DatabaseTitle | MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic MEDLINE |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Computer Science |
| ExternalDocumentID | 11110125 |
| Genre | Research Support, Non-U.S. Gov't Journal Article Comparative Study |
| GroupedDBID | --- -~X .4S .DC 0R~ 123 36B 4.4 41~ 53G 6IK AAJGR AALMD AAYOK ABAZT ABDBF ABDNZ ABEFU ABIVO ABJNI ACGFO ACUHS ACYGS ADIYS ADMLS AEGXH AEILP AENEX AFHIN AIAGR ALMA_UNASSIGNED_HOLDINGS ARCSS AVWKF AZFZN BEFXN BFFAM BGNUA BKEBE BPEOZ CAG CGR COF CS3 CUY CVF DU5 EAP EAS EBC EBD EBS ECM ECS EDO EIF EJD EMB EMK EMOBN EPL EPS EST ESX F5P FEDTE FNEHJ HVGLF HZ~ H~9 I-F IPLJI JAVBF MCG MINIK MKJ NPM O9- OCL P2P PK0 PQQKQ RMI SV3 TUS WG8 WH7 XJE ZWS 7X8 ABUFD ABVLG AMVHM |
| ID | FETCH-LOGICAL-c496t-c114e2c3e0ba8c9fc9eb792a45ed0bcc700d25f69c0493c0e66234d7e64660a92 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 249 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000165399700003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0899-7667 |
| IngestDate | Sun Nov 09 09:36:01 EST 2025 Wed Feb 19 01:31:45 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 11 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c496t-c114e2c3e0ba8c9fc9eb792a45ed0bcc700d25f69c0493c0e66234d7e64660a92 |
| Notes | ObjectType-Article-2 SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 23 |
| PMID | 11110125 |
| PQID | 72470218 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_72470218 pubmed_primary_11110125 |
| PublicationCentury | 2000 |
| PublicationDate | 2000-11-01 |
| PublicationDateYYYYMMDD | 2000-11-01 |
| PublicationDate_xml | – month: 11 year: 2000 text: 2000-11-01 day: 01 |
| PublicationDecade | 2000 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | Neural computation |
| PublicationTitleAlternate | Neural Comput |
| PublicationYear | 2000 |
| SSID | ssj0006105 |
| Score | 2.1769476 |
| Snippet | This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | 2519 |
| SubjectTerms | Animals Brain - physiology Models, Neurological Models, Theoretical Neural Networks (Computer) Neuronal Plasticity |
| Title | On the computational power of winner-take-all |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/11110125 https://www.proquest.com/docview/72470218 |
| Volume | 12 |
| WOSCitedRecordID | wos000165399700003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV07T8MwED4VysBCeVOeHlitRokftYSEEKJioXQAqVvlnB0JUSWBFvj7nN1ETIiBJYOjSNH5fPed7_EBXIaZVC71ggvyHlzkaMgOqoxLpxKjncCiEJFsQo_Hw-nUTDpw1fbChLLK1iZGQ-0qDHfkA50KHfzRdf3GA2dUyK02BBpr0M0IyISCLj39mRWumgJGiii4Vkq3LTMqHYQ1WgoaHmKEYap_B5jR0Yx6__vFbdhqACa7WWnEDnR8uQu9lryBNWd5D_hjyQj8MYxvmitBVgfSNFYV7CtycvGlffXczuf78Dy6e7q95w13Akdh1JIjxTk-xcwnuR2iKdD4XJvUCuldkiPqhLZIFsoghQgZJl4RDhJOeyVILtakB7BeVqU_AkbysRQHOklIT8jcGBnSqSIn8GA9wbk-XLTimJFuhoSDLX31sZi1AunD4Uqis3o1QiPGHeQa5fGf357AZux-j-1_p9At6FT6M9jAz-XL4v08bjk9x5OHbwWQsmg |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=On+the+computational+power+of+winner-take-all&rft.jtitle=Neural+computation&rft.au=Maass%2C+W&rft.date=2000-11-01&rft.issn=0899-7667&rft.volume=12&rft.issue=11&rft.spage=2519&rft_id=info:doi/10.1162%2F089976600300014827&rft_id=info%3Apmid%2F11110125&rft_id=info%3Apmid%2F11110125&rft.externalDocID=11110125 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0899-7667&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0899-7667&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0899-7667&client=summon |