Transformer: A New Paradigm for Building Data-Parallel Programming Models

Cloud computing drives the design and development of diverse programming models for massive data processing. the transformer programming framework aims to facilitate the building of diverse data-parallel programming models. transformer has two layers: a common runtime system and a model-specific sys...

Full description

Saved in:
Bibliographic Details
Published in:IEEE MICRO Vol. 30; no. 4; pp. 55 - 64
Main Authors: Wang, Peng, Meng, Dan, Han, Jizhong, Zhan, Jianfeng, Tu, Bibo, Shi, Xiaofeng, Wan, Le
Format: Journal Article
Language:English
Published: Los Alamitos IEEE 01.07.2010
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
actor model
All-Pairs
Cloud computing
Clouds
Computation
Computational modeling
data flow
data intensive computing
Data models
Data processing
Design engineering
Libraries
Load modeling
MapReduce
Mathematical models
Parallel processing
Programming
Programming languages
programming model
Receivers
Runtime
Transformers
ISSN:0272-1732, 1937-4143
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Cloud computing drives the design and development of diverse programming models for massive data processing. the transformer programming framework aims to facilitate the building of diverse data-parallel programming models. transformer has two layers: a common runtime system and a model-specific system. using transformer, the authors show how to implement three programming models: dryad-like data flow, MapReduce, and All-Pairs.
AbstractList Cloud computing drives the design and development of diverse programming models for massive data processing. The Transformer programming framework aims to facilitate the building of diverse data-parallel programming models. Transformer has two layers: a common runtime system and a model-specific system. Using Transformer, the authors show how to implement three programming models: Dryad-like data flow, MapReduce, and All-Pairs.
Cloud computing drives the design and development of diverse programming models for massive data processing. The Transformer programming framework aims to facilitate the building of diverse data-parallel programming models. Transformer has two layers: a common runtime system and a model-specific system. Using Transformer, the authors show how to implement three programming models: Dryad-like data flow, MapReduce, and All-Pairs. [PUBLICATION ABSTRACT]
Author Peng Wang
Xiaofeng Shi
Le Wan
Jizhong Han
Dan Meng
Jianfeng Zhan
Bibo Tu
Author_xml – sequence: 1
  givenname: Peng
  surname: Wang
  fullname: Wang, Peng
– sequence: 2
  givenname: Dan
  surname: Meng
  fullname: Meng, Dan
– sequence: 3
  givenname: Jizhong
  surname: Han
  fullname: Han, Jizhong
– sequence: 4
  givenname: Jianfeng
  surname: Zhan
  fullname: Zhan, Jianfeng
– sequence: 5
  givenname: Bibo
  surname: Tu
  fullname: Tu, Bibo
– sequence: 6
  givenname: Xiaofeng
  surname: Shi
  fullname: Shi, Xiaofeng
– sequence: 7
  givenname: Le
  surname: Wan
  fullname: Wan, Le
BookMark eNptkM1LxDAQxYMouK5evHopXgShOkmapvXmt8JW96DnMttOl0jaaNJF_O_NsuJBPA0z7_cej9lj24MbiLFDDmecQ3leVWcC4qLVFpvwUuo045ncZhMQWqRcS7HL9kJ4AwAloJiwxxePQ-ic78lfJJfJE30mc_TYmmWfxHNytTK2NcMyucER07VkLdlk7t3SY9-vlcq1ZMM-2-nQBjr4mVP2enf7cv2Qzp7vH68vZ2kjOYyplpQTKiwwLwrR6VY2WUtSchJNyfNOx9K4EKUuFyJTuUZsFABJBOQdX3Ryyk42ue_efawojHVvQkPW4kBuFeqCF4WMeTySx3_IN7fyQyxXayVygCJTETrdQI13IXjq6ndvevRfNYd6_dO6qur1T6MnwvAHbsyIo3HD6NHY_y1HG4shot9spRQH4PIbhJaB9Q
CODEN IEMIDZ
CitedBy_id crossref_primary_10_1109_TC_2012_103
crossref_primary_10_1016_j_jpdc_2011_03_006
crossref_primary_10_1109_TPDS_2011_257
crossref_primary_10_1007_s11704_012_2118_7
crossref_primary_10_1016_j_future_2014_09_008
crossref_primary_10_1109_TPDS_2011_144
Cites_doi 10.2200/S00193ED1V01Y200905CAC006
10.1145/1376616.1376726
10.1109/IPDPS.2008.4536311
10.1145/1582716.1582723
10.1145/1013208.1013209
10.7551/mitpress/1086.001.0001
10.1155/2005/962135
10.1145/1272996.1273005
10.1145/1327452.1327491
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Jul/Aug 2010
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Jul/Aug 2010
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
F28
FR3
DOI 10.1109/MM.2010.75
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
ANTE: Abstracts in New Technology & Engineering
Engineering Research Database
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
Engineering Research Database
ANTE: Abstracts in New Technology & Engineering
DatabaseTitleList Technology Research Database
Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 1937-4143
EndPage 64
ExternalDocumentID 2144886751
10_1109_MM_2010_75
5551001
Genre orig-research
Feature
GroupedDBID -DZ
-~X
.DC
0R~
29I
3EH
4.4
5GY
5VS
6IK
97E
AAFWJ
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACGFO
ACGFS
ACGOD
ACIWK
ACNCT
AENEX
AETEA
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
AZLTO
BEFXN
BFFAM
BGNUA
BKEBE
BKOMP
BPEOZ
C1A
CS3
DU5
EBS
EJD
HZ~
H~9
IBMZZ
ICLAB
IEDLZ
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
OHT
P2P
PQQKQ
RIA
RIE
RNI
RNS
RZB
TAE
TN5
TWZ
VH1
YZZ
ZCG
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
F28
FR3
ID FETCH-LOGICAL-c310t-73e6ea5a8a6882f7d3c4de331e2c916f7937ab2979b24567aac500e3a0a1f1bf3
IEDL.DBID RIE
ISICitedReferencesCount 16
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000280949100009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 0272-1732
IngestDate Sun Sep 28 01:41:03 EDT 2025
Sun Nov 09 06:30:27 EST 2025
Sat Nov 29 06:18:37 EST 2025
Tue Nov 18 22:16:09 EST 2025
Wed Aug 27 02:47:56 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 4
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c310t-73e6ea5a8a6882f7d3c4de331e2c916f7937ab2979b24567aac500e3a0a1f1bf3
Notes SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-2
content type line 23
PQID 752600845
PQPubID 37061
PageCount 10
ParticipantIDs crossref_primary_10_1109_MM_2010_75
proquest_journals_752600845
ieee_primary_5551001
proquest_miscellaneous_818833311
crossref_citationtrail_10_1109_MM_2010_75
PublicationCentury 2000
PublicationDate 2010-July-Aug.
2010-07-00
20100701
PublicationDateYYYYMMDD 2010-07-01
PublicationDate_xml – month: 07
  year: 2010
  text: 2010-July-Aug.
PublicationDecade 2010
PublicationPlace Los Alamitos
PublicationPlace_xml – name: Los Alamitos
PublicationTitle IEEE MICRO
PublicationTitleAbbrev MM
PublicationYear 2010
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref12
ref11
chaiken (ref9) 2008; 1
ref2
yu (ref10) 2008
agha (ref14) 1986
ref7
ref4
dean (ref1) 2004; 6
ref6
ref5
hindman (ref3) 2009
thusoo (ref8) 2009; 2
johnston (ref13) 2004; 36
References_xml – volume: 6
  start-page: 137
  year: 2004
  ident: ref1
  article-title: MapReduce: Simplified Data Processing on Large Clusters
  publication-title: Proc Symp Operating Systems Design and Implementation
– ident: ref12
  doi: 10.2200/S00193ED1V01Y200905CAC006
– ident: ref7
  doi: 10.1145/1376616.1376726
– volume: 1
  start-page: 1265
  year: 2008
  ident: ref9
  article-title: Scope: Easy and Efficient Parallel Processing of Massive Datasets
  publication-title: Proc Int'l Conf Very Large Data Bases (VLDB)
– ident: ref4
  doi: 10.1109/IPDPS.2008.4536311
– ident: ref5
  doi: 10.1145/1582716.1582723
– volume: 36
  start-page: 1
  year: 2004
  ident: ref13
  article-title: Advances in Dataflow Programming Languages
  publication-title: ACM Computing Surveys
  doi: 10.1145/1013208.1013209
– year: 1986
  ident: ref14
  publication-title: Actors A Model of Concurrent Computation in Distributed Systems
  doi: 10.7551/mitpress/1086.001.0001
– ident: ref6
  doi: 10.1155/2005/962135
– ident: ref2
  doi: 10.1145/1272996.1273005
– ident: ref11
  doi: 10.1145/1327452.1327491
– volume: 2
  start-page: 1626
  year: 2009
  ident: ref8
  article-title: Hive?Warehousing Solution over a Map-Reduce Framework
  publication-title: Proc Int'l Conf Very Large Data Bases (VLDB)
– start-page: 91
  year: 2009
  ident: ref3
  article-title: A Common Substrate for Cluster Computing
  publication-title: Proc of Workshop on Hot Topics in Cloud Computing
– start-page: 1
  year: 2008
  ident: ref10
  article-title: DryadLINQ: A System for General-Purpose Distributed Data-Parallel Computing Using a High-Level Language
  publication-title: Proc 5th Oper Syst Design Implementation Symp
SSID ssj0005208
Score 2.0240536
Snippet Cloud computing drives the design and development of diverse programming models for massive data processing. the transformer programming framework aims to...
Cloud computing drives the design and development of diverse programming models for massive data processing. The Transformer programming framework aims to...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 55
SubjectTerms actor model
All-Pairs
Cloud computing
Clouds
Computation
Computational modeling
data flow
data intensive computing
Data models
Data processing
Design engineering
Libraries
Load modeling
MapReduce
Mathematical models
Parallel processing
Programming
Programming languages
programming model
Receivers
Runtime
Transformers
Title Transformer: A New Paradigm for Building Data-Parallel Programming Models
URI https://ieeexplore.ieee.org/document/5551001
https://www.proquest.com/docview/752600845
https://www.proquest.com/docview/818833311
Volume 30
WOSCitedRecordID wos000280949100009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 1937-4143
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0005208
  issn: 0272-1732
  databaseCode: RIE
  dateStart: 19810101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PT4MwFG7m4sGLU6dxTk0TvZiIaylQ8DZ_LJrIssM0u5FSilnCmAHm328fv7JEL94IfZDyStuvfV_fh9C1Ry2ul7LEoFFoG5YdUiOMIOU-92TMaEh4qUXw8canU3ex8GYddNuehVFKleQzdQeXZSw_WssNbJWNbD29EzistcO5U53V2qZzlKOuyU2DcmbWqUgp8Ua-X1G4gEu4NfmUaiq_huByXpn0_lejA7Rf40c8rhr8EHVUeoR6jTYDrrtqH73OG0Sqsns8xnowwzORiWj5ucL6Nn6o9bDxkyiEAUVJohI8q_haKygBnbQkP0bvk-f544tRyyYYUmO1wuBMOUrYwhWOhs8xj5i0IsUYVabUYDCGjHgiND3uhRD15EJImxDFBBE0pmHMTlA3XafqFGESQ5DGtBzHlaDJoRc3rgj1w7oluQaDA3TT-DOQdU5xkLZIgnJtQbzA9wPwfcC17VVr-1Vl0vjTqg9ebi1qBw_QsGmmoO5kubaG7PouVAO3pbp3QMhDpGq9yQMNR1ymP52e_f3eIdqr6ADAvz1H3SLbqAu0K7-LZZ5dln_YD5nLzP0
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PT4MwFG6WaaIXp07jnD-a6MVEXEthBW_zx7LFsewwzW6klGKWMGaA-ffbB2wxcRdvhD5IeaXt176v70Po1qUW10tZYtAwsA3LDqgRhJByn7syYjQgvNAi-Bjx8diZzdxJDd1vzsIopQrymXqAyyKWHy7lCrbKOrae3gkc1tqxLcsk5Wmt34SOYtw1uWlQzswqGSklbsfzShIXsAl_TT-FnsqfQbiYWfqN_9XpEB1UCBL3yiY_QjWVHKPGWp0BV521iYbTNSZV6SPuYT2c4YlIRTj_XGB9Gz9Vitj4ReTCgKI4VjGelIytBZSAUlqcnaD3_uv0eWBUwgmG1GgtNzhTXSVs4YiuBtARD5m0QsUYVabUcDCCnHgiMF3uBhD35EJImxDFBBE0okHETlE9WSbqDGESQZjGtLpdR4Iqh17eOCLQD-u25BoOttDd2p--rLKKg7hF7BerC-L6nueD732ubW82tl9lLo2tVk3w8saicnALtdfN5FfdLNPWkF_fgWrgTanuHxD0EIlarjJfAxKH6U-n59vfe432BlNv5I-G47c22i_JAcDGvUD1PF2pS7Qrv_N5ll4Vf9sPbJTQRA
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Transformer%3A+A+New+Paradigm+for+Building+Data-Parallel+Programming+Models&rft.jtitle=IEEE+MICRO&rft.au=Wang%2C+Peng&rft.au=Meng%2C+Dan&rft.au=Han%2C+Jizhong&rft.au=Zhan%2C+Jianfeng&rft.date=2010-07-01&rft.issn=0272-1732&rft.volume=30&rft.issue=4&rft.spage=55&rft.epage=64&rft_id=info:doi/10.1109%2FMM.2010.75&rft.externalDBID=NO_FULL_TEXT
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0272-1732&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0272-1732&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0272-1732&client=summon