A Fundamental Tradeoff Between Computation and Communication in Distributed Computing

How can we optimally trade extra computing power to reduce the communication load in distributed computing? We answer this question by characterizing a fundamental tradeoff between computation and communication in distributed computing, i.e., the two are inversely proportional to each other. More sp...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on information theory Jg. 64; H. 1; S. 109 - 128
Hauptverfasser:	Songze Li, Maddah-Ali, Mohammad Ali, Qian Yu, Avestimehr, A. Salman
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York IEEE 01.01.2018 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Benchmark testing coded multicasting coded TeraSort Coding Communication Computation computation-communication tradeoff Computer networks Distributed computing Distributed databases Distributed processing Electrical engineering Electronic mail Encoding Information theory Load distribution (forces) Lower bounds MapReduce Nodes Optimization Stress concentration Tradeoffs
ISSN:	0018-9448, 1557-9654
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	How can we optimally trade extra computing power to reduce the communication load in distributed computing? We answer this question by characterizing a fundamental tradeoff between computation and communication in distributed computing, i.e., the two are inversely proportional to each other. More specifically, a general distributed computing framework, motivated by commonly used structures like MapReduce, is considered, where the overall computation is decomposed into computing a set of "Map" and "Reduce" functions distributedly across multiple computing nodes. A coded scheme, named "coded distributed computing" (CDC), is proposed to demonstrate that increasing the computation load of the Map functions by a factor of r (i.e., evaluating each function at r carefully chosen nodes) can create novel coding opportunities that reduce the communication load by the same factor. An information-theoretic lower bound on the communication load is also provided, which matches the communication load achieved by the CDC scheme. As a result, the optimal computation-communication tradeoff in distributed computing is exactly characterized. Finally, the coding techniques of CDC is applied to the Hadoop TeraSort benchmark to develop a novel CodedTeraSort algorithm, which is empirically demonstrated to speed up the overall job execution by 1.97× -3.39×, for typical settings of interest.
AbstractList	How can we optimally trade extra computing power to reduce the communication load in distributed computing? We answer this question by characterizing a fundamental tradeoff between computation and communication in distributed computing, i.e., the two are inversely proportional to each other. More specifically, a general distributed computing framework, motivated by commonly used structures like MapReduce, is considered, where the overall computation is decomposed into computing a set of "Map" and "Reduce" functions distributedly across multiple computing nodes. A coded scheme, named "coded distributed computing" (CDC), is proposed to demonstrate that increasing the computation load of the Map functions by a factor of r (i.e., evaluating each function at r carefully chosen nodes) can create novel coding opportunities that reduce the communication load by the same factor. An information-theoretic lower bound on the communication load is also provided, which matches the communication load achieved by the CDC scheme. As a result, the optimal computation-communication tradeoff in distributed computing is exactly characterized. Finally, the coding techniques of CDC is applied to the Hadoop TeraSort benchmark to develop a novel CodedTeraSort algorithm, which is empirically demonstrated to speed up the overall job execution by 1.97× -3.39×, for typical settings of interest. How can we optimally trade extra computing power to reduce the communication load in distributed computing? We answer this question by characterizing a fundamental tradeoff between computation and communication in distributed computing, i.e., the two are inversely proportional to each other. More specifically, a general distributed computing framework, motivated by commonly used structures like MapReduce, is considered, where the overall computation is decomposed into computing a set of “M” and “Reduce” functions distributedly across multiple computing nodes. A coded scheme, named “coded distributed computing” (CDC), is proposed to demonstrate that increasing the computation load of the Map functions by a factor of r (i.e., evaluating each function at r carefully chosen nodes) can create novel coding opportunities that reduce the communication load by the same factor. An information-theoretic lower bound on the communication load is also provided, which matches the communication load achieved by the CDC scheme. As a result, the optimal computation-communication tradeoff in distributed computing is exactly characterized. Finally, the coding techniques of CDC is applied to the Hadoop TeraSort benchmark to develop a novel CodedTeraSort algorithm, which is empirically demonstrated to speed up the overall job execution by 1.97× -3.39×, for typical settings of interest.
Author	Qian Yu Songze Li Maddah-Ali, Mohammad Ali Avestimehr, A. Salman
Author_xml	– sequence: 1 surname: Songze Li fullname: Songze Li email: songzeli@usc.edu organization: Dept. of Electr. Eng., Univ. of Southern California, Los Angeles, CA, USA – sequence: 2 givenname: Mohammad Ali surname: Maddah-Ali fullname: Maddah-Ali, Mohammad Ali email: maddah_ali@sharif.edu organization: Dept. of Electr. Eng., Sharif Univ. of Technol., Tehran, Iran – sequence: 3 surname: Qian Yu fullname: Qian Yu email: qyu880@usc.edu organization: Dept. of Electr. Eng., Univ. of Southern California, Los Angeles, CA, USA – sequence: 4 givenname: A. Salman surname: Avestimehr fullname: Avestimehr, A. Salman email: avestimehr@ee.usc.edu organization: Dept. of Electr. Eng., Univ. of Southern California, Los Angeles, CA, USA
BookMark	eNp9kM1LAzEQxYNUsFXvgpcFz1uTzdfmWKvVQsHL9hyyyayktNmazSL-927Z4sGDp-EN783wfjM0CW0AhO4InhOC1WO1ruYFJnJeSC4UVxdoSjiXuRKcTdAUY1LmirHyCs26bjdIxkkxRdtFtuqDMwcIyeyzKhoHbdNkT5C-AEK2bA_HPpnk25CZ4E760Advx40P2bPvUvR1n8CdzT583KDLxuw7uD3Pa7RdvVTLt3zz_rpeLja5LRRJuZVCAnW8loRJIXDDXAm4NKWR0llGRS1KWzNuLOWusUraghpmBVNUsoJgeo0exrvH2H720CW9a_sYhpe6IJLxoSWhg0uMLhvbrovQaOvHSikav9cE6xNCPSDUJ4T6jHAI4j_BY_QHE7__i9yPEQ8Av_YSc4Iloz__fn4D
CODEN	IETTAW
CitedBy_id	crossref_primary_10_1109_TIT_2021_3127910 crossref_primary_10_1007_s11063_024_11505_3 crossref_primary_10_1109_TCC_2024_3370834 crossref_primary_10_1109_TCOMM_2021_3049821 crossref_primary_10_1109_TII_2018_2857203 crossref_primary_10_1103_PhysRevLett_130_150602 crossref_primary_10_1109_TCOMM_2020_3020549 crossref_primary_10_1109_TCOMM_2021_3056089 crossref_primary_10_1109_TIT_2019_2954338 crossref_primary_10_1109_TCOMM_2023_3326499 crossref_primary_10_1109_TIT_2024_3423710 crossref_primary_10_1109_TIT_2023_3291216 crossref_primary_10_1109_TCOMM_2024_3395332 crossref_primary_10_1109_TIT_2025_3556384 crossref_primary_10_1109_TDSC_2025_3532850 crossref_primary_10_1109_TSP_2019_2912139 crossref_primary_10_1109_TCOMM_2021_3053018 crossref_primary_10_1109_TCOMM_2021_3087788 crossref_primary_10_1109_TIT_2019_2955384 crossref_primary_10_1109_TCOMM_2020_2992721 crossref_primary_10_3390_su142113809 crossref_primary_10_1109_TCOMM_2021_3054906 crossref_primary_10_1109_TSP_2024_3452035 crossref_primary_10_1109_TCOMM_2022_3211932 crossref_primary_10_1109_TIT_2023_3321918 crossref_primary_10_1109_TCOMM_2024_3381671 crossref_primary_10_1109_TCC_2021_3050012 crossref_primary_10_1109_TIT_2024_3382257 crossref_primary_10_1109_TIT_2022_3158828 crossref_primary_10_1109_TWC_2021_3062264 crossref_primary_10_1109_JSAC_2020_3036961 crossref_primary_10_1109_TIT_2019_2957577 crossref_primary_10_1109_TCOMM_2024_3381711 crossref_primary_10_1002_jcc_27459 crossref_primary_10_1109_TIT_2020_2975021 crossref_primary_10_1109_JSAC_2022_3180811 crossref_primary_10_1109_TNET_2020_2992989 crossref_primary_10_1109_TMC_2025_3570907 crossref_primary_10_1109_TIT_2025_3573708 crossref_primary_10_1007_s11432_024_4408_1 crossref_primary_10_1109_TPDS_2025_3566080 crossref_primary_10_1109_TIT_2024_3392685 crossref_primary_10_1109_TCC_2022_3228243 crossref_primary_10_1109_TIT_2024_3352020 crossref_primary_10_3390_info13070342 crossref_primary_10_1109_TCOMM_2020_2970950 crossref_primary_10_1109_TCOMM_2023_3274144 crossref_primary_10_1109_TIT_2019_2904055 crossref_primary_10_3390_e23080985 crossref_primary_10_1109_TIFS_2020_2972166 crossref_primary_10_1109_JSAC_2021_3126076 crossref_primary_10_1109_TCOMM_2023_3345421 crossref_primary_10_1109_TIT_2020_2964547 crossref_primary_10_1109_TCOMM_2020_2975160 crossref_primary_10_1109_JSTSP_2022_3156756 crossref_primary_10_1109_LWC_2021_3065101 crossref_primary_10_1109_TWC_2024_3395368 crossref_primary_10_1109_TIT_2022_3198031 crossref_primary_10_1109_LCOMM_2019_2934436 crossref_primary_10_1109_LCOMM_2021_3114222 crossref_primary_10_1109_TIT_2019_2959760 crossref_primary_10_1109_JIOT_2019_2907245 crossref_primary_10_1109_TIT_2019_2924621 crossref_primary_10_1109_TCOMM_2023_3275194 crossref_primary_10_3390_e26060448 crossref_primary_10_1109_ACCESS_2021_3094888 crossref_primary_10_1109_TIT_2022_3206868 crossref_primary_10_1109_TIFS_2019_2940895 crossref_primary_10_1186_s13638_023_02299_4 crossref_primary_10_1109_LCOMM_2021_3140100 crossref_primary_10_1109_LCOMM_2019_2930513 crossref_primary_10_1109_TCC_2021_3098593 crossref_primary_10_1109_LCOMM_2021_3125800 crossref_primary_10_1109_TIT_2023_3247860 crossref_primary_10_1109_TNET_2022_3181234 crossref_primary_10_1109_TCOMM_2022_3230779 crossref_primary_10_1109_JIOT_2020_3040657 crossref_primary_10_1109_TCOMM_2020_2988506 crossref_primary_10_1109_TIT_2025_3528083 crossref_primary_10_1109_TNSM_2023_3336067 crossref_primary_10_1109_TCOMM_2024_3492071 crossref_primary_10_1016_j_ssci_2019_08_022 crossref_primary_10_1109_TIT_2024_3373128 crossref_primary_10_1109_TNET_2021_3058685 crossref_primary_10_1109_TIT_2022_3143199 crossref_primary_10_1109_JSAC_2018_2869952 crossref_primary_10_3390_s20216153 crossref_primary_10_1109_JIOT_2022_3143229 crossref_primary_10_1109_TIFS_2020_3009610 crossref_primary_10_1080_09640568_2024_2413879 crossref_primary_10_1109_JIOT_2022_3175733 crossref_primary_10_1109_TWC_2023_3307140 crossref_primary_10_3390_e27030311 crossref_primary_10_1109_TNNLS_2024_3414325 crossref_primary_10_1016_j_envres_2021_112572 crossref_primary_10_1109_JIOT_2021_3132031 crossref_primary_10_1080_17477778_2022_2043792 crossref_primary_10_6023_A23110496 crossref_primary_10_1109_TIT_2025_3541808 crossref_primary_10_1109_TIFS_2019_2925723 crossref_primary_10_1109_TIT_2021_3095471 crossref_primary_10_1109_TVT_2021_3070723 crossref_primary_10_1109_JIOT_2020_2974045 crossref_primary_10_1109_TCOMM_2021_3087628 crossref_primary_10_1109_TCOMM_2025_3529253 crossref_primary_10_1109_TCOMM_2020_3016648 crossref_primary_10_1109_TWC_2022_3187427 crossref_primary_10_1109_TIT_2019_2927558 crossref_primary_10_1109_TCOMM_2024_3446641 crossref_primary_10_1109_TMC_2019_2963668 crossref_primary_10_1109_COMST_2020_3007787 crossref_primary_10_1109_TWC_2021_3091465 crossref_primary_10_1109_JIOT_2023_3305708 crossref_primary_10_1109_JSAC_2018_2844939 crossref_primary_10_1109_TIT_2020_2993965 crossref_primary_10_1109_TIT_2020_2966197 crossref_primary_10_1002_cpe_5593 crossref_primary_10_1109_TCSS_2024_3361465 crossref_primary_10_3390_e24091284 crossref_primary_10_1109_TIT_2021_3066005 crossref_primary_10_1109_TIT_2018_2888494 crossref_primary_10_1109_TNET_2024_3372698 crossref_primary_10_1109_TWC_2020_2985039 crossref_primary_10_1016_j_comnet_2025_111381 crossref_primary_10_1109_TCOMM_2023_3347772 crossref_primary_10_1109_TIT_2020_3029396 crossref_primary_10_1109_TCOMM_2023_3238373 crossref_primary_10_1109_TMC_2021_3097380 crossref_primary_10_1007_s12083_018_0695_4 crossref_primary_10_1109_TIT_2017_2785237 crossref_primary_10_1109_TIT_2025_3559736 crossref_primary_10_1109_TSP_2023_3238888 crossref_primary_10_1109_TVT_2024_3391836 crossref_primary_10_1109_TCOMM_2023_3298981 crossref_primary_10_1109_TIT_2019_2929328 crossref_primary_10_1109_ACCESS_2020_3043825 crossref_primary_10_1109_TCOMM_2018_2869791 crossref_primary_10_1109_TNET_2021_3075377 crossref_primary_10_1109_TPDS_2023_3276888 crossref_primary_10_1109_COMST_2021_3091684 crossref_primary_10_3390_e25020266 crossref_primary_10_1109_TWC_2022_3165658 crossref_primary_10_1109_TNSE_2021_3095040 crossref_primary_10_1109_LSP_2023_3321561 crossref_primary_10_1007_s11277_024_11061_y crossref_primary_10_1109_TNET_2019_2919553 crossref_primary_10_1145_3766543 crossref_primary_10_1109_ACCESS_2019_2953172 crossref_primary_10_1109_TIT_2021_3132043 crossref_primary_10_1109_TNET_2020_3003907 crossref_primary_10_1109_TIFS_2023_3326970 crossref_primary_10_1016_j_jnca_2025_104280 crossref_primary_10_1109_TVT_2021_3131395 crossref_primary_10_1109_TIFS_2024_3450288 crossref_primary_10_3390_e26100881 crossref_primary_10_1109_TIT_2022_3157835 crossref_primary_10_1109_TIT_2021_3139009 crossref_primary_10_1109_TSIPN_2020_2998223 crossref_primary_10_1109_JIOT_2021_3137757 crossref_primary_10_1109_TCOMM_2019_2940671 crossref_primary_10_1109_LSP_2023_3299531 crossref_primary_10_1109_TIT_2020_2999675 crossref_primary_10_1109_TSP_2022_3173726 crossref_primary_10_1109_TWC_2024_3453403 crossref_primary_10_1109_TIT_2023_3283967
Cites_doi	10.1109/TIT.2015.2504556 10.1109/ISIT.2016.7541478 10.1109/TIT.2010.2103753 10.1109/TIT.2007.904785 10.1109/ISIT.2014.6875212 10.1017/CBO9781139058452 10.1109/ISIT.2017.8006961 10.1145/2189750.2150984 10.14778/1687627.1687731 10.1109/ISIT.2017.8007057 10.1109/18.915643 10.1109/ALLERTON.2016.7852225 10.1109/ALLERTON.2015.7447112 10.1145/800135.804414 10.1109/GLOCOMW.2016.7848828 10.1109/MCOM.2017.1600894 10.1109/18.50368 10.1109/ICC.2017.7996730 10.1109/JSAC.2013.130404 10.1145/1165389.945450 10.1109/TNET.2003.818197 10.1145/1272998.1273005 10.1109/ISIT.2017.8006963 10.1109/CLOUD.2013.107 10.1109/TIT.2014.2306938 10.1109/TNET.2017.2702605 10.1109/18.850663 10.1109/e-Science.2009.53 10.1109/ISIT.2003.1228459 10.1145/2723372.2742790 10.1145/2342509.2342513 10.1109/ISIT.2016.7541612 10.1109/IPDPSW.2017.33 10.1145/1402946.1402967 10.1145/288090.288094 10.1109/TIT.2006.874540 10.1109/TNET.2014.2317316 10.1109/TIT.1979.1056022 10.1145/2043164.2018448 10.1109/JIOT.2016.2584538
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2018
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2018
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/TIT.2017.2756959
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Computer Science
EISSN	1557-9654
EndPage	128
ExternalDocumentID	10_1109_TIT_2017_2756959 8051074
Genre	orig-research
GrantInformation_xml	– fundername: NSA grantid: H98230-16-C-0255 funderid: 10.13039/100009226 – fundername: ONR grantid: N000141612189 funderid: 10.13039/100000006 – fundername: Defense Advanced Research Projects Agency grantid: HR001117C0053 funderid: 10.13039/100000185 – fundername: NSF grantid: CCF-1408639; NETS-1419632 funderid: 10.13039/100000001 – fundername: Intel
GroupedDBID	-~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACGFS ACGOD ACIWK AENEX AETEA AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD F5P HZ~ H~9 IAAWW IBMZZ ICLAB IDIHD IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS RXW TAE TN5 VH1 VJK AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c291t-c767e3d5b7147660f4d8e08a8a77dc436b68cb45ac35dfc97c23a4c6493742103
IEDL.DBID	RIE
ISICitedReferencesCount	335
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000418754100007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0018-9448
IngestDate	Sun Oct 05 00:18:42 EDT 2025 Sat Nov 29 03:31:39 EST 2025 Tue Nov 18 21:25:32 EST 2025 Tue Aug 26 16:59:35 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	1
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c291t-c767e3d5b7147660f4d8e08a8a77dc436b68cb45ac35dfc97c23a4c6493742103
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-4282-3307 0000-0002-3222-1874
PQID	2174501413
PQPubID	36024
PageCount	20
ParticipantIDs	ieee_primary_8051074 crossref_citationtrail_10_1109_TIT_2017_2756959 proquest_journals_2174501413 crossref_primary_10_1109_TIT_2017_2756959
PublicationCentury	2000
PublicationDate	2018-Jan. 2018-1-00 20180101
PublicationDateYYYYMMDD	2018-01-01
PublicationDate_xml	– month: 01 year: 2018 text: 2018-Jan.
PublicationDecade	2010
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on information theory
PublicationTitleAbbrev	TIT
PublicationYear	2018
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref56 ref12 ref15 ref14 ref53 ref52 kushilevitz (ref16) 2006 ref54 zaharia (ref5) 2010; 10 isard (ref43) 2007; 41 ref17 ref19 ref18 li (ref55) 2017 tandon (ref40) 2017 ref50 ref46 ref45 ref48 ref47 (ref33) 2017 ref41 ref44 (ref10) 2017 ref49 ref8 dutta (ref37) 2016 zaharia (ref36) 2008; 8 ref9 ref3 ref6 kiamari (ref35) 2017 ref34 dean (ref4) 2004 ref31 ref30 o’malley (ref32) 2008 ref2 ref1 ref38 chu (ref42) 2007; 19 (ref11) 2017 (ref51) 2017 guo (ref7) 2013 ref24 ref23 ref26 ref25 ref20 ref22 ref21 yu (ref39) 2017 ref28 ref27 ref29
References_xml	– year: 2006 ident: ref16 publication-title: Communication Complexity – ident: ref22 doi: 10.1109/TIT.2015.2504556 – ident: ref24 doi: 10.1109/ISIT.2016.7541478 – ident: ref28 doi: 10.1109/TIT.2010.2103753 – ident: ref18 doi: 10.1109/TIT.2007.904785 – ident: ref23 doi: 10.1109/ISIT.2014.6875212 – ident: ref25 doi: 10.1017/CBO9781139058452 – ident: ref34 doi: 10.1109/ISIT.2017.8006961 – year: 2017 ident: ref11 publication-title: Hadoop TeraSort – ident: ref6 doi: 10.1145/2189750.2150984 – year: 2017 ident: ref39 article-title: Polynomial codes: An optimal design for high-dimensional coded matrix multiplication publication-title: Proc Adv Neural Inf Process Syst (NIPS) – start-page: 1 year: 2004 ident: ref4 article-title: MapReduce: Simplified data processing on large clusters publication-title: Proc USENIX Symp on Operating System Design and Implementation – ident: ref44 doi: 10.14778/1687627.1687731 – ident: ref54 doi: 10.1109/ISIT.2017.8007057 – ident: ref17 doi: 10.1109/18.915643 – ident: ref47 doi: 10.1109/ALLERTON.2016.7852225 – year: 2017 ident: ref35 publication-title: On heterogeneous coded distributed computing – ident: ref1 doi: 10.1109/ALLERTON.2015.7447112 – ident: ref12 doi: 10.1145/800135.804414 – ident: ref41 doi: 10.1109/GLOCOMW.2016.7848828 – ident: ref56 doi: 10.1109/MCOM.2017.1600894 – ident: ref14 doi: 10.1109/18.50368 – start-page: 2100 year: 2016 ident: ref37 article-title: Short-dot: Computing large linear transforms distributedly using coded short dot products publication-title: Proc Adv Neural Inf Process Syst (NIPS) – ident: ref26 doi: 10.1109/ICC.2017.7996730 – ident: ref19 doi: 10.1109/JSAC.2013.130404 – volume: 8 start-page: 7 year: 2008 ident: ref36 article-title: Improving MapReduce performance in heterogeneous environments publication-title: OSDI – ident: ref50 doi: 10.1145/1165389.945450 – ident: ref30 doi: 10.1109/TNET.2003.818197 – volume: 19 start-page: 281 year: 2007 ident: ref42 article-title: Map-reduce for machine learning on multicore publication-title: Proc Adv Neural Inf Process Syst – volume: 41 start-page: 59 year: 2007 ident: ref43 article-title: Dryad: Distributed data-parallel programs from sequential building blocks publication-title: ACM SIGOPS Oper Syst Rev doi: 10.1145/1272998.1273005 – year: 2017 ident: ref33 publication-title: Open MPI Open Source High Performance Computing – ident: ref38 doi: 10.1109/ISIT.2017.8006963 – ident: ref9 doi: 10.1109/CLOUD.2013.107 – ident: ref20 doi: 10.1109/TIT.2014.2306938 – ident: ref49 doi: 10.1109/TNET.2017.2702605 – ident: ref29 doi: 10.1109/18.850663 – year: 2008 ident: ref32 article-title: Terabyte Sort On Apache Hadoop – ident: ref45 doi: 10.1109/e-Science.2009.53 – ident: ref31 doi: 10.1109/ISIT.2003.1228459 – ident: ref46 doi: 10.1145/2723372.2742790 – ident: ref52 doi: 10.1145/2342509.2342513 – ident: ref2 doi: 10.1109/ISIT.2016.7541612 – year: 2017 ident: ref55 article-title: Architectures for coded mobile edge computing publication-title: Proc Fog World Congress – year: 2017 ident: ref10 publication-title: Amazon Elastic Compute Cloud (Amazon EC2) – ident: ref3 doi: 10.1109/IPDPSW.2017.33 – year: 2017 ident: ref51 publication-title: Apache Hadoop Distributed File System – volume: 10 start-page: 10 year: 2010 ident: ref5 article-title: Spark: Cluster computing with working sets publication-title: Proc 2nd USENIX Conf HotCloud – ident: ref48 doi: 10.1145/1402946.1402967 – ident: ref15 doi: 10.1145/288090.288094 – ident: ref27 doi: 10.1109/TIT.2006.874540 – ident: ref21 doi: 10.1109/TNET.2014.2317316 – start-page: 107 year: 2013 ident: ref7 article-title: iShuffle: Improving Hadoop performance with shuffle-on-write publication-title: Proc 10th Int Conf Auton Comput – ident: ref13 doi: 10.1109/TIT.1979.1056022 – start-page: 3368 year: 2017 ident: ref40 article-title: Gradient coding: Avoiding stragglers in distributed learning publication-title: Proc Int Conf Mach Learn (ICML) – ident: ref8 doi: 10.1145/2043164.2018448 – ident: ref53 doi: 10.1109/JIOT.2016.2584538
SSID	ssj0014512
Score	2.68326
Snippet	How can we optimally trade extra computing power to reduce the communication load in distributed computing? We answer this question by characterizing a...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	109
SubjectTerms	Benchmark testing coded multicasting coded TeraSort Coding Communication Computation computation-communication tradeoff Computer networks Distributed computing Distributed databases Distributed processing Electrical engineering Electronic mail Encoding Information theory Load distribution (forces) Lower bounds MapReduce Nodes Optimization Stress concentration Tradeoffs
Title	A Fundamental Tradeoff Between Computation and Communication in Distributed Computing
URI	https://ieeexplore.ieee.org/document/8051074 https://www.proquest.com/docview/2174501413
Volume	64
WOSCitedRecordID	wos000418754100007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1557-9654 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014512 issn: 0018-9448 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB7a4kEPVlvF-iIHL4Lb7naTzO6xPoqCFA8t9LZkkywUZCt9-PtNsg8qiuBtD5mw7DfZmcnMfANwE3MUEU3RywJfeNR42F4ap9RTPmoMUmbMiqPMf8XJJJrP47cG3NW9MFprV3ym-_bR5fLVUm7tVdkgshqEtAlNRF70atUZA8qCghk8MAfYxBxVStKPB9OXqa3hwr6lOo8tK-mOCXIzVX78iJ11Gbf_915HcFh6kWRUwH4MDZ13oF1NaCDlge3AwQ7dYBdmIzK2jR8Fnz8xZkrpZZaR-6JWixTyDioickW-NY-QRU4eLcuuHZClVbnY7HsCs_HT9OHZK8cqeHIYBxtPIkcdKpZiQJFzP6Mq0n4kIoGoJA15yiOZUiZkyFQmY5TDUFDJqfFkqIkQw1No5ctcnwFBzgygtrDG4Kt1JLJMGuvHpEhZxlD3YFB96USWnON29MV74mIPP04MNonFJimx6cFtLfFR8G38sbZrsajXlTD04LICMykP5DqxkZdNoQbh-e9SF7A_tLrjblcuobVZbfUV7MnPzWK9una69gWOBdC1
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8MwDLbGQAIODDYQgwE5cEGiWx9J0x7HY9rEmDh00m5VmqTSJNShPfj9JOlDQyAkbj3EUdXPqe3Y_gxwG_qUBTihVurYzMLKw7aSMMGWsKmkTkKUWTGU-WM6mQSzWfhWg_uqF0ZKaYrPZFc_mly-WPCNvirrBVqDKN6BXYKxa-fdWlXOABMn5wZ31BFWUUeZlLTDXjSKdBUX7Wqy81Dzkm4ZITNV5cev2NiXQeN_b3YMR4Ufifo58CdQk1kTGuWMBlQc2SYcbhEOtmDaRwPd-pEz-iNlqIRcpCl6yKu1UC5vwEIsE-hb-wiaZ-hJ8-zqEVlSFIvVvqcwHTxHj0OrGKxgcTd01hanPpWeIAl1MPV9O8UikHbAAkap4NjzEz_gCSaMe0SkPKTc9RjmPla-DFYxoncG9WyRyXNA1CcKUl1aoxCWMmBpypX9I5wlJCVUtqFXfumYF6zjevjFe2yiDzuMFTaxxiYusGnDXSXxkTNu_LG2pbGo1hUwtKFTghkXR3IV69hLJ1Ed7-J3qRvYH0av43g8mrxcwoGr9cjctXSgvl5u5BXs8c_1fLW8Nnr3BZFo0_w
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Fundamental+Tradeoff+Between+Computation+and+Communication+in+Distributed+Computing&rft.jtitle=IEEE+transactions+on+information+theory&rft.au=Li%2C+Songze&rft.au=Mohammad+Ali+Maddah-Ali&rft.au=Yu%2C+Qian&rft.au=Avestimehr%2C+A+Salman&rft.date=2018-01-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=0018-9448&rft.eissn=1557-9654&rft.volume=64&rft.issue=1&rft.spage=109&rft_id=info:doi/10.1109%2FTIT.2017.2756959&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9448&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9448&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9448&client=summon