Artykuł w czasopiśmie
Brak miniatury
Licencja
ClosedAccessDostęp zamknięty
 

Using Unused: Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk

Uproszczony widok
dc.abstract.enModern HPC workload managers and their careful tuning contribute to the high utilization of HPC clusters. However, due to inevitable uncertainty it is impossible to completely avoid node idleness. Although such idle slots are usually too short for any HPC job, they are too long to ignore them. Function-as-a-Service (FaaS) paradigm promisingly fills this gap, and can be a good match, as typical FaaS functions last seconds, not hours. Here we show how to build a FaaS infrastructure on idle nodes in an HPC cluster in such a way that it does not affect the performance of the HPC jobs significantly. We dynamically adapt to a changing set of idle physical machines, by integrating open-source software Slurm and OpenWhisk. We designed and implemented a prototype solution that allowed us to cover up to 90% of the idle time slots on a 50k-core cluster that runs production workloads.
dc.affiliationUniwersytet Warszawski
dc.conference.countryStany Zjednoczone
dc.conference.datefinish2022-11-18
dc.conference.datestart2022-11-13
dc.conference.placeDallas
dc.conference.seriesInternational Conference for High Performance Computing, Networking, Storage and Analysis
dc.conference.seriesInternational Conference for High Performance Computing, Networking, Storage and Analysis
dc.conference.seriesshortcutSC
dc.conference.shortcutSC22
dc.conference.weblinkhttps://sc22.supercomputing.org/
dc.contributor.authorŻuk, Paweł
dc.contributor.authorRządca, Krzysztof
dc.contributor.authorMalawski, Maciej
dc.contributor.authorLagosz, Bartłomiej
dc.contributor.authorPawlik, Maciej
dc.contributor.authorPrzybylski, Bartłomiej
dc.date.accessioned2024-01-26T11:29:30Z
dc.date.available2024-01-26T11:29:30Z
dc.date.issued2022
dc.description.financePublikacja bezkosztowa
dc.identifier.doi10.1109/SC41404.2022.00045
dc.identifier.urihttps://repozytorium.uw.edu.pl//handle/item/124387
dc.identifier.weblinkhttp://xplorestaging.ieee.org/ielx7/10046045/10045783/10046086.pdf?arnumber=10046086
dc.languageeng
dc.pbn.affiliationcomputer and information sciences
dc.relation.pages1-15
dc.rightsClosedAccess
dc.sciencecloudnosend
dc.subject.ensupercomputer
dc.subject.enfunction as a service
dc.subject.enFaaS
dc.subject.enserverless
dc.subject.enhigh-performance computing
dc.subject.enHPC
dc.titleUsing Unused: Non-Invasive Dynamic FaaS Infrastructure with HPC-Whisk
dc.typeJournalArticle
dspace.entity.typePublication