In EDS ansehen

Robust external hash aggregation in the solid state age

Gespeichert in:
Bibliographische Detailangaben
Titel: Robust external hash aggregation in the solid state age
Autoren: Kuiper, L.N. (Laurens), Boncz, P.A. (Peter), Mühleisen, H.F. (Hannes)
Publikationsjahr: 2024
Bestand: CWI's Institutional Repository (Centrum voor Wiskunde en Informatica)
Schlagwörter: Relational databases, Database query processing, Aggregation
Beschreibung: Analytical database systems offer high-performance in-memory aggregation. If there are many unique groups, temporary query intermediates may not fit RAM, requiring the use of external storage. However, switching from an in-memory to an external algorithm can degrade performance sharply. We revisit external hash aggregation on modern hardware, aiming instead for robust performance that avoids a 'performance cliff' when memory runs out. To achieve this, we introduce two techniques for handling temporary query intermediates. First, we propose unifying the memory management of temporary and persistent data. Second, we propose using a page layout that can be spilled to disk despite being optimized for main memory performance. These two techniques allow operator implementations to process larger-than-memory query intermediates with only minor modifications. We integrate these into DuckDB's parallel hash aggregation. Experimental results show that our implementation gracefully degrades performance as query intermediates exceed the available memory limit, while main memory performance is competitive with other analytical database systems.
Publikationsart: conference object
Sprache: English
Relation: https://ir.cwi.nl/pub/34360
DOI: 10.1109/ICDE60146.2024.00288
Verfügbarkeit: https://ir.cwi.nl/pub/34360
https://doi.org/10.1109/ICDE60146.2024.00288
Dokumentencode: edsbas.AFB0FD60
Datenbank: BASE
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://ir.cwi.nl/pub/34360#
    Name: EDS - BASE (s4221598)
    Category: fullText
    Text: View record from BASE
  – Url: https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=EBSCO&SrcAuth=EBSCO&DestApp=WOS&ServiceName=TransferToWoS&DestLinkType=GeneralSearchSummary&Func=Links&author=Kuiper%20LN
    Name: ISI
    Category: fullText
    Text: Nájsť tento článok vo Web of Science
    Icon: https://imagesrvr.epnet.com/ls/20docs.gif
    MouseOverText: Nájsť tento článok vo Web of Science
Header DbId: edsbas
DbLabel: BASE
An: edsbas.AFB0FD60
RelevancyScore: 901
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 901.007141113281
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Robust external hash aggregation in the solid state age
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Kuiper%2C+L%2EN%2E+%28Laurens%29%22">Kuiper, L.N. (Laurens)</searchLink><br /><searchLink fieldCode="AR" term="%22Boncz%2C+P%2EA%2E+%28Peter%29%22">Boncz, P.A. (Peter)</searchLink><br /><searchLink fieldCode="AR" term="%22Mühleisen%2C+H%2EF%2E+%28Hannes%29%22">Mühleisen, H.F. (Hannes)</searchLink>
– Name: DatePubCY
  Label: Publication Year
  Group: Date
  Data: 2024
– Name: Subset
  Label: Collection
  Group: HoldingsInfo
  Data: CWI's Institutional Repository (Centrum voor Wiskunde en Informatica)
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Relational+databases%22">Relational databases</searchLink><br /><searchLink fieldCode="DE" term="%22Database+query+processing%22">Database query processing</searchLink><br /><searchLink fieldCode="DE" term="%22Aggregation%22">Aggregation</searchLink>
– Name: Abstract
  Label: Description
  Group: Ab
  Data: Analytical database systems offer high-performance in-memory aggregation. If there are many unique groups, temporary query intermediates may not fit RAM, requiring the use of external storage. However, switching from an in-memory to an external algorithm can degrade performance sharply. We revisit external hash aggregation on modern hardware, aiming instead for robust performance that avoids a 'performance cliff' when memory runs out. To achieve this, we introduce two techniques for handling temporary query intermediates. First, we propose unifying the memory management of temporary and persistent data. Second, we propose using a page layout that can be spilled to disk despite being optimized for main memory performance. These two techniques allow operator implementations to process larger-than-memory query intermediates with only minor modifications. We integrate these into DuckDB's parallel hash aggregation. Experimental results show that our implementation gracefully degrades performance as query intermediates exceed the available memory limit, while main memory performance is competitive with other analytical database systems.
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: conference object
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: NoteTitleSource
  Label: Relation
  Group: SrcInfo
  Data: https://ir.cwi.nl/pub/34360
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1109/ICDE60146.2024.00288
– Name: URL
  Label: Availability
  Group: URL
  Data: https://ir.cwi.nl/pub/34360<br />https://doi.org/10.1109/ICDE60146.2024.00288
– Name: AN
  Label: Accession Number
  Group: ID
  Data: edsbas.AFB0FD60
PLink https://erproxy.cvtisr.sk/sfx/access?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.AFB0FD60
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1109/ICDE60146.2024.00288
    Languages:
      – Text: English
    Subjects:
      – SubjectFull: Relational databases
        Type: general
      – SubjectFull: Database query processing
        Type: general
      – SubjectFull: Aggregation
        Type: general
    Titles:
      – TitleFull: Robust external hash aggregation in the solid state age
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Kuiper, L.N. (Laurens)
      – PersonEntity:
          Name:
            NameFull: Boncz, P.A. (Peter)
      – PersonEntity:
          Name:
            NameFull: Mühleisen, H.F. (Hannes)
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-locals
              Value: edsbas
ResultId 1