Tuning DNN Model Compression to Resource and Data Availability in Cooperative Training

Model compression is a fundamental tool to execute machine learning (ML) tasks on the diverse set of devices populating current-and next-generation networks, thereby exploiting their resources and data. At the same time, how much and when to compress ML models are very complex decisions, as they hav...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE/ACM transactions on networking Ročník 32; číslo 2; s. 1 - 16
Hlavní autoři: Malandrino, Francesco, di Giacomo, Giuseppe, Karamzade, Armin, Levorato, Marco, Chiasserini, Carla Fabiana
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.04.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Adaptation models
Alternative energy sources
Availability
Complexity
Computational modeling
Costs
Data compression
Data models
Decisions
Distributed learning
Dynamic programming
Energy consumption
Energy costs
Graph representations
Graphical representations
Machine learning
model pruning
Network latency
network support to machine learning
Optimization
Polynomials
Switches
Task analysis
Training
ISSN:1063-6692, 1558-2566
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Model compression is a fundamental tool to execute machine learning (ML) tasks on the diverse set of devices populating current-and next-generation networks, thereby exploiting their resources and data. At the same time, how much and when to compress ML models are very complex decisions, as they have to jointly account for such aspects as the model being used, the resources (e.g., computational) and local datasets available at each node, as well as network latencies. In this work, we address the multi-dimensional problem of adapting the model compression, data selection, and node allocation decisions to each other: our objective is to perform the DNN training at the minimum energy cost, subject to learning quality and time constraints. To this end, we propose an algorithmic framework called PACT, combining a time-expanded graph representation of the training process, a dynamic programming solution strategy, and a data-driven approach to the estimation of the loss evolution. We prove that PACT's complexity is polynomial, and its decisions can get arbitrarily close to the optimum. Through our numerical evaluation, we further show how PACT can consistently outperform state-of-the-art alternatives and closely matches the optimal energy consumption.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1063-6692
1558-2566
DOI:10.1109/TNET.2023.3323023