Research on Join Operation of Temporal Big Data in Distributed Environment

Distributed system is an ideal choice for processing temporal large data join operation,but the existing distributed system cannot support the original temporal join query and cannot meet the processing requirements of temporal large data with low latency and high throughput.Therefore,a two-level in...

Full description

Saved in:
Bibliographic Details
Published in:Ji suan ji gong cheng Vol. 45; no. 3; pp. 20 - 25,31
Main Author: ZHANG Wei,WANG Zhijie
Format: Journal Article
Language:Chinese
English
Published: Editorial Office of Computer Engineering 01.03.2019
Subjects:
temporal big data| distributed memory computing| temporal join| two-level index| partition method| spark framework
ISSN:1000-3428
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Distributed system is an ideal choice for processing temporal large data join operation,but the existing distributed system cannot support the original temporal join query and cannot meet the processing requirements of temporal large data with low latency and high throughput.Therefore,a two-level index memory solution scheme based on Spark is proposed.The global index is used to prune the distributed partitions,and the local temporal index is used to query the partitions in order to improve the efficiency of data retrieval.A partition method is designed for temporal data to optimize global pruning.Experimental results based on real and synthetic datasets show that the scheme can significantly improve the processing efficiency of temporal join operation.
AbstractList Distributed system is an ideal choice for processing temporal large data join operation,but the existing distributed system cannot support the original temporal join query and cannot meet the processing requirements of temporal large data with low latency and high throughput.Therefore,a two-level index memory solution scheme based on Spark is proposed.The global index is used to prune the distributed partitions,and the local temporal index is used to query the partitions in order to improve the efficiency of data retrieval.A partition method is designed for temporal data to optimize global pruning.Experimental results based on real and synthetic datasets show that the scheme can significantly improve the processing efficiency of temporal join operation.
Author ZHANG Wei,WANG Zhijie
Author_xml – sequence: 1
  fullname: ZHANG Wei,WANG Zhijie
  organization: 1.Department of Computer Science and Engineering,Shanghai Jiaotong University,Shanghai 200240,China; 2.School of Data and Computer Science,Sun Yat-Sen University,Guangzhou 510006,China
BookMark eNo9jNtKw0AUReehgm31H8YPSDxzySR51LZqS6Eg9TmcudUJ7UyZRMG_t6j4tNiLzZqRSUzREXLHoGStqpv7vgzDEEsGAIWQvCkBKq64mpDpv7sms2HoASTnAFOyeXWDw2zeaYp0k0Kku7PLOIbLTJ7u3emcMh7pYzjQJY5IL49lGMYc9MfoLF3Fz5BTPLk43pArj8fB3f5xTt6eVvvFS7HdPa8XD9vCMqHGohGulspXDIxSVrhKcbRGc22E4I0WzFfIGDCNntemko22TjpUXrYGq7YRc7L-7dqEfXfO4YT5q0sYuh-R8qHDPAZzdJ2pOQoAq0QLskbbomY1s7KtGuadBvENEAdeAQ
ContentType Journal Article
DBID DOA
DOI 10.19678/j.issn.1000-3428.0052626
DatabaseName DOAJ Open Access Full Text
DatabaseTitleList
Database_xml – sequence: 1
  dbid: DOA
  name: DOAJ Directory of Open Access Journals
  url: https://www.doaj.org/
  sourceTypes: Open Website
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EndPage 25,31
ExternalDocumentID oai_doaj_org_article_c72a300d639047ad9ab171d49581feb0
GroupedDBID -0Y
5XA
5XJ
92H
92I
ABJNI
ACGFS
ALMA_UNASSIGNED_HOLDINGS
CCEZO
CUBFJ
CW9
GROUPED_DOAJ
TCJ
TGT
U1G
U5S
ID FETCH-LOGICAL-d136t-83e746f510c66d3e562adcb2bc3328b31f5a1101baf27c548bde4ea6f49ca5983
IEDL.DBID DOA
ISSN 1000-3428
IngestDate Mon Nov 10 19:22:27 EST 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 3
Language Chinese
English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-d136t-83e746f510c66d3e562adcb2bc3328b31f5a1101baf27c548bde4ea6f49ca5983
OpenAccessLink https://doaj.org/article/c72a300d639047ad9ab171d49581feb0
ParticipantIDs doaj_primary_oai_doaj_org_article_c72a300d639047ad9ab171d49581feb0
PublicationCentury 2000
PublicationDate 2019-03-01
PublicationDateYYYYMMDD 2019-03-01
PublicationDate_xml – month: 03
  year: 2019
  text: 2019-03-01
  day: 01
PublicationDecade 2010
PublicationTitle Ji suan ji gong cheng
PublicationYear 2019
Publisher Editorial Office of Computer Engineering
Publisher_xml – name: Editorial Office of Computer Engineering
SSID ssj0042200
Score 2.2206867
Snippet Distributed system is an ideal choice for processing temporal large data join operation,but the existing distributed system cannot support the original...
SourceID doaj
SourceType Open Website
StartPage 20
SubjectTerms temporal big data| distributed memory computing| temporal join| two-level index| partition method| spark framework
Title Research on Join Operation of Temporal Big Data in Distributed Environment
URI https://doaj.org/article/c72a300d639047ad9ab171d49581feb0
Volume 45
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAON
  databaseName: DOAJ Directory of Open Access Journals
  issn: 1000-3428
  databaseCode: DOA
  dateStart: 20160101
  customDbUrl:
  isFulltext: true
  dateEnd: 99991231
  titleUrlDefault: https://www.doaj.org/
  omitProxy: false
  ssIdentifier: ssj0042200
  providerName: Directory of Open Access Journals
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1NSwMxEA1SRPQgfuI3Ebxum03S7OZobYv0UD1U6G3Jp1Rkt7Srv9_JptV68uJ1N1mWPDLzBmbeQ-hOEUdyb31ipOUJMGKXKJWLxFrNtUidSbNoNpGNx_l0Kp83rL5CT1iUB44H1zEZVYwQC5mU8ExZqTRst8Dr89Q73VTrJJPrYirGYE4piToEBKIMMOwddNtM5kFo7rw116v9_a7daJ7Q36r9TXoZHqD9FS_E9_F_DtGWK4_Q3oZa4DEarbvkcFXiUTUr8dPcRQBx5fEkiky9497sFfdVrTCs6Add3GBp5Swe_My0naCX4WDy8JisrBASmzJRJzlzGRceLpARwjIHrEVZo6k2jNFcs9R3FSTyVCtPMwNViLaOOyU8l0Z1Zc5OUausSneGMOXGcu5gtRIAh1fd4Iqgw4CqDOJf56gXjqGYR7WLIuhPNw8AlWKFSvEXKhf_8ZFLtAv0RMaOryvUqhcf7hptm896tlzcNIB_ATtHrdg
linkProvider Directory of Open Access Journals
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Research+on+Join+Operation+of+Temporal+Big+Data+in+Distributed+Environment&rft.jtitle=Ji+suan+ji+gong+cheng&rft.au=ZHANG+Wei%2CWANG+Zhijie&rft.date=2019-03-01&rft.pub=Editorial+Office+of+Computer+Engineering&rft.issn=1000-3428&rft.volume=45&rft.issue=3&rft.spage=20&rft.epage=25%2C31&rft_id=info:doi/10.19678%2Fj.issn.1000-3428.0052626&rft.externalDBID=DOA&rft.externalDocID=oai_doaj_org_article_c72a300d639047ad9ab171d49581feb0
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1000-3428&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1000-3428&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1000-3428&client=summon