In EDS ansehen

SpecK: Composition of Stream Processing Applications over Fog Environments

Gespeichert in:
Bibliographische Detailangaben
Titel: SpecK: Composition of Stream Processing Applications over Fog Environments
Autoren: Battulga, Davaadorj, Miorandi, Daniele, Tedeschi, Cédric
Weitere Verfasser: U-Hopper srl Trento, Design and Implementation of Autonomous Distributed Systems (MYRIADS), Inria Rennes – Bretagne Atlantique, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-SYSTÈMES LARGE ÉCHELLE (IRISA-D1), Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA), Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom Paris (IMT)-Institut Mines-Télécom Paris (IMT)-Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut Mines-Télécom Paris (IMT)-Institut Mines-Télécom Paris (IMT)-Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom Paris (IMT)-Institut Mines-Télécom Paris (IMT), European Project: 765452,h2020,H2020-MSCA-ITN-2017,FogGuru(2017)
Quelle: DAIS 2021 - 21st International Conference on Distributed Applications and Interoperable Systems ; https://inria.hal.science/hal-03259975 ; DAIS 2021 - 21st International Conference on Distributed Applications and Interoperable Systems, Jun 2021, Valetta, Malta. pp.38-54, ⟨10.1007/978-3-030-78198-9_3⟩
Verlagsinformationen: HAL CCSD
Publikationsjahr: 2021
Bestand: Université de Rennes 1: Publications scientifiques (HAL)
Schlagwörter: stream processing, deployment, geographically distributed platforms, [INFO.INFO-OS]Computer Science [cs]/Operating Systems [cs.OS], [INFO.INFO-DC]Computer Science [cs]/Distributed, Parallel, and Cluster Computing [cs.DC]
Geographisches Schlagwort: Valetta, Malta
Beschreibung: International audience ; Stream Processing (SP), i.e., the processing of data in motion, as soon as it becomes available, is a hot topic in cloud computing. Various SP stacks exist today, with applications ranging from IoT analytics to processing of payment transactions. The backbone of said stacks are Stream Processing Engines (SPEs), software packages offering a high-level programming model and scalable execution of data stream processing pipelines. SPEs have been traditionally developed to work inside a single datacenter, and optimised for speed. With the advent of Fog computing, however, the processing of data streams needs to be carried out over multiple geographically distributed computing sites: Data gets typically pre-processed close to where they are generated, then aggregated at intermediate nodes, and finally globally and persistently stored in the Cloud. SPEs were not designed to address these new scenarios. In this paper, we argue that large scale Fog-based stream processing should rely on the coordinated composition of geographically dispersed SPE instances. We propose an architecture based on the composition of multiple SPE instances and their communication via distributed message brokers. We introduce SpecK, a tool to automate the deployment and adaptation of pipelines over a Fog computing platform. Given a description of the pipeline, SpecKcovers all the operations needed to deploy a stream processing computation over the different SPE instances targeted, using their own APIs and establishing the required communication channels to forward data among them. A prototypical implementation of SpecK is presented, and its performance is evaluated over Grid'5000, a large-scale, distributed experimental facility.
Publikationsart: conference object
Sprache: English
Relation: info:eu-repo/grantAgreement//765452/EU/FogGuru: Training the Next Generation of European Fog Computing Experts/FogGuru
DOI: 10.1007/978-3-030-78198-9_3
Verfügbarkeit: https://inria.hal.science/hal-03259975
https://inria.hal.science/hal-03259975v1/document
https://inria.hal.science/hal-03259975v1/file/dais.pdf
https://doi.org/10.1007/978-3-030-78198-9_3
Rights: info:eu-repo/semantics/OpenAccess
Dokumentencode: edsbas.DF3B6AE7
Datenbank: BASE
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://inria.hal.science/hal-03259975#
    Name: EDS - BASE (s4221598)
    Category: fullText
    Text: View record from BASE
  – Url: https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=EBSCO&SrcAuth=EBSCO&DestApp=WOS&ServiceName=TransferToWoS&DestLinkType=GeneralSearchSummary&Func=Links&author=Battulga%20D
    Name: ISI
    Category: fullText
    Text: Nájsť tento článok vo Web of Science
    Icon: https://imagesrvr.epnet.com/ls/20docs.gif
    MouseOverText: Nájsť tento článok vo Web of Science
Header DbId: edsbas
DbLabel: BASE
An: edsbas.DF3B6AE7
RelevancyScore: 934
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 933.9462890625
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: SpecK: Composition of Stream Processing Applications over Fog Environments
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Battulga%2C+Davaadorj%22">Battulga, Davaadorj</searchLink><br /><searchLink fieldCode="AR" term="%22Miorandi%2C+Daniele%22">Miorandi, Daniele</searchLink><br /><searchLink fieldCode="AR" term="%22Tedeschi%2C+Cédric%22">Tedeschi, Cédric</searchLink>
– Name: Author
  Label: Contributors
  Group: Au
  Data: U-Hopper srl Trento<br />Design and Implementation of Autonomous Distributed Systems (MYRIADS)<br />Inria Rennes – Bretagne Atlantique<br />Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-SYSTÈMES LARGE ÉCHELLE (IRISA-D1)<br />Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA)<br />Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes)<br />Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)<br />Institut Mines-Télécom Paris (IMT)-Institut Mines-Télécom Paris (IMT)-Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes)<br />Institut Mines-Télécom Paris (IMT)-Institut Mines-Télécom Paris (IMT)-Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA)<br />Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)<br />Institut Mines-Télécom Paris (IMT)-Institut Mines-Télécom Paris (IMT)<br />European Project: 765452,h2020,H2020-MSCA-ITN-2017,FogGuru(2017)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: DAIS 2021 - 21st International Conference on Distributed Applications and Interoperable Systems ; https://inria.hal.science/hal-03259975 ; DAIS 2021 - 21st International Conference on Distributed Applications and Interoperable Systems, Jun 2021, Valetta, Malta. pp.38-54, ⟨10.1007/978-3-030-78198-9_3⟩
– Name: Publisher
  Label: Publisher Information
  Group: PubInfo
  Data: HAL CCSD
– Name: DatePubCY
  Label: Publication Year
  Group: Date
  Data: 2021
– Name: Subset
  Label: Collection
  Group: HoldingsInfo
  Data: Université de Rennes 1: Publications scientifiques (HAL)
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22stream+processing%22">stream processing</searchLink><br /><searchLink fieldCode="DE" term="%22deployment%22">deployment</searchLink><br /><searchLink fieldCode="DE" term="%22geographically+distributed+platforms%22">geographically distributed platforms</searchLink><br /><searchLink fieldCode="DE" term="%22[INFO%2EINFO-OS]Computer+Science+[cs]%2FOperating+Systems+[cs%2EOS]%22">[INFO.INFO-OS]Computer Science [cs]/Operating Systems [cs.OS]</searchLink><br /><searchLink fieldCode="DE" term="%22[INFO%2EINFO-DC]Computer+Science+[cs]%2FDistributed%22">[INFO.INFO-DC]Computer Science [cs]/Distributed</searchLink><br /><searchLink fieldCode="DE" term="%22Parallel%22">Parallel</searchLink><br /><searchLink fieldCode="DE" term="%22and+Cluster+Computing+[cs%2EDC]%22">and Cluster Computing [cs.DC]</searchLink>
– Name: Subject
  Label: Subject Geographic
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Valetta%22">Valetta</searchLink><br /><searchLink fieldCode="DE" term="%22Malta%22">Malta</searchLink>
– Name: Abstract
  Label: Description
  Group: Ab
  Data: International audience ; Stream Processing (SP), i.e., the processing of data in motion, as soon as it becomes available, is a hot topic in cloud computing. Various SP stacks exist today, with applications ranging from IoT analytics to processing of payment transactions. The backbone of said stacks are Stream Processing Engines (SPEs), software packages offering a high-level programming model and scalable execution of data stream processing pipelines. SPEs have been traditionally developed to work inside a single datacenter, and optimised for speed. With the advent of Fog computing, however, the processing of data streams needs to be carried out over multiple geographically distributed computing sites: Data gets typically pre-processed close to where they are generated, then aggregated at intermediate nodes, and finally globally and persistently stored in the Cloud. SPEs were not designed to address these new scenarios. In this paper, we argue that large scale Fog-based stream processing should rely on the coordinated composition of geographically dispersed SPE instances. We propose an architecture based on the composition of multiple SPE instances and their communication via distributed message brokers. We introduce SpecK, a tool to automate the deployment and adaptation of pipelines over a Fog computing platform. Given a description of the pipeline, SpecKcovers all the operations needed to deploy a stream processing computation over the different SPE instances targeted, using their own APIs and establishing the required communication channels to forward data among them. A prototypical implementation of SpecK is presented, and its performance is evaluated over Grid'5000, a large-scale, distributed experimental facility.
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: conference object
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: NoteTitleSource
  Label: Relation
  Group: SrcInfo
  Data: info:eu-repo/grantAgreement//765452/EU/FogGuru: Training the Next Generation of European Fog Computing Experts/FogGuru
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/978-3-030-78198-9_3
– Name: URL
  Label: Availability
  Group: URL
  Data: https://inria.hal.science/hal-03259975<br />https://inria.hal.science/hal-03259975v1/document<br />https://inria.hal.science/hal-03259975v1/file/dais.pdf<br />https://doi.org/10.1007/978-3-030-78198-9_3
– Name: Copyright
  Label: Rights
  Group: Cpyrght
  Data: info:eu-repo/semantics/OpenAccess
– Name: AN
  Label: Accession Number
  Group: ID
  Data: edsbas.DF3B6AE7
PLink https://erproxy.cvtisr.sk/sfx/access?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.DF3B6AE7
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/978-3-030-78198-9_3
    Languages:
      – Text: English
    Subjects:
      – SubjectFull: Valetta
        Type: general
      – SubjectFull: Malta
        Type: general
      – SubjectFull: stream processing
        Type: general
      – SubjectFull: deployment
        Type: general
      – SubjectFull: geographically distributed platforms
        Type: general
      – SubjectFull: [INFO.INFO-OS]Computer Science [cs]/Operating Systems [cs.OS]
        Type: general
      – SubjectFull: [INFO.INFO-DC]Computer Science [cs]/Distributed
        Type: general
      – SubjectFull: Parallel
        Type: general
      – SubjectFull: and Cluster Computing [cs.DC]
        Type: general
    Titles:
      – TitleFull: SpecK: Composition of Stream Processing Applications over Fog Environments
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Battulga, Davaadorj
      – PersonEntity:
          Name:
            NameFull: Miorandi, Daniele
      – PersonEntity:
          Name:
            NameFull: Tedeschi, Cédric
      – PersonEntity:
          Name:
            NameFull: U-Hopper srl Trento
      – PersonEntity:
          Name:
            NameFull: Design and Implementation of Autonomous Distributed Systems (MYRIADS)
      – PersonEntity:
          Name:
            NameFull: Inria Rennes – Bretagne Atlantique
      – PersonEntity:
          Name:
            NameFull: Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-SYSTÈMES LARGE ÉCHELLE (IRISA-D1)
      – PersonEntity:
          Name:
            NameFull: Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA)
      – PersonEntity:
          Name:
            NameFull: Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes)
      – PersonEntity:
          Name:
            NameFull: Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
      – PersonEntity:
          Name:
            NameFull: Institut Mines-Télécom Paris (IMT)-Institut Mines-Télécom Paris (IMT)-Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes)
      – PersonEntity:
          Name:
            NameFull: Institut Mines-Télécom Paris (IMT)-Institut Mines-Télécom Paris (IMT)-Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA)
      – PersonEntity:
          Name:
            NameFull: Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique)
      – PersonEntity:
          Name:
            NameFull: Institut Mines-Télécom Paris (IMT)-Institut Mines-Télécom Paris (IMT)
      – PersonEntity:
          Name:
            NameFull: European Project: 765452,h2020,H2020-MSCA-ITN-2017,FogGuru(2017)
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2021
          Identifiers:
            – Type: issn-locals
              Value: edsbas
            – Type: issn-locals
              Value: edsbas.oa
          Titles:
            – TitleFull: DAIS 2021 - 21st International Conference on Distributed Applications and Interoperable Systems ; https://inria.hal.science/hal-03259975 ; DAIS 2021 - 21st International Conference on Distributed Applications and Interoperable Systems, Jun 2021, Valetta, Malta. pp.38-54, ⟨10.1007/978-3-030-78198-9_3⟩
              Type: main
ResultId 1