Using balanced data placement to address I/O contention in production environments
Designed for capacity and capability, HPC I/O systems are inherently complex and shared among multiple, concurrent jobs competing for resources. Lack of centralized coordination and control often render the end-to-end I/O paths vulnerable to load imbalance and contention. With the emergence of data-...
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Dokumenttyp: | Kapitel/Artikel Konferenzschrift |
| Sprache: | Englisch |
| Veröffentlicht: |
2016
|
| In: |
2016 28th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)
Year: 2016, Pages: 9-17 |
| DOI: | 10.1109/SBAC-PAD.2016.10 |
| Schlagworte: | |
| Online-Zugang: | Resolving-System, Volltext: http://dx.doi.org/10.1109/SBAC-PAD.2016.10 Verlag, Volltext: https://ieeexplore.ieee.org/document/7789318/ 
|
| Verfasserangaben: | Sarah Neuwirth, Feiyi Wang, Sarp Oral, Sudharshan Vazhkudai, James H. Rogers and Ulrich Bruening |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1575446774 | ||
| 003 | DE-627 | ||
| 005 | 20220814144431.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 180523s2016 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1109/SBAC-PAD.2016.10 |2 doi | |
| 035 | |a (DE-627)1575446774 | ||
| 035 | |a (DE-576)505446774 | ||
| 035 | |a (DE-599)BSZ505446774 | ||
| 035 | |a (OCoLC)1341009969 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 28 |2 sdnb | ||
| 100 | 1 | |a Neuwirth, Sarah |d 1986- |e VerfasserIn |0 (DE-588)1159979707 |0 (DE-627)1023096536 |0 (DE-576)505429004 |4 aut | |
| 245 | 1 | 0 | |a Using balanced data placement to address I/O contention in production environments |c Sarah Neuwirth, Feiyi Wang, Sarp Oral, Sudharshan Vazhkudai, James H. Rogers and Ulrich Bruening |
| 264 | 1 | |c 2016 | |
| 264 | 4 | |c © 2016 | |
| 300 | |a 9 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 23.05.2018 | ||
| 520 | |a Designed for capacity and capability, HPC I/O systems are inherently complex and shared among multiple, concurrent jobs competing for resources. Lack of centralized coordination and control often render the end-to-end I/O paths vulnerable to load imbalance and contention. With the emergence of data-intensive HPC applications, storage systems are further contended for performance and scalability. This paper proposes to unify two key approaches to tackle the imbalanced use of I/O resources and to achieve an end-to-end I/O performance improvement in the most transparent way. First, it utilizes a topology-aware, Balanced Placement I/O method (BPIO) for mitigating resource contention. Second, it takes advantage of the platform-neutral ADIOS middleware, which provides a flexible I/O mechanism for scientific applications. By integrating BPIO with ADIOS, referred to as Aequilibro, we obtain an end-to-end and per job I/O performance improvement for ADIOS-enabled HPC applications without requiring any code changes. Aequilibro can be applied to almost any HPC platform and is mostly suitable for systems that lack a centralized file system resource manager. We demonstrate the effectiveness of our integration on the Titan system at the Oak Ridge National Laboratory. Our experiments with a synthetic benchmark and real-world HPC workload show that, even in a noisy production environment, Aequilibro can improve large-scale application performance significantly. | ||
| 650 | 4 | |a ADIOS-enabled HPC applications | |
| 650 | 4 | |a Aequilibro | |
| 650 | 4 | |a Aggregates | |
| 650 | 4 | |a balanced data placement | |
| 650 | 4 | |a balanced placement I/O method | |
| 650 | 4 | |a BPIO | |
| 650 | 4 | |a end-to-end I/O performance improvement | |
| 650 | 4 | |a flexible I/O mechanism | |
| 650 | 4 | |a High Performance Computing | |
| 650 | 4 | |a I/O contention | |
| 650 | 4 | |a I/O resources | |
| 650 | 4 | |a Load management | |
| 650 | 4 | |a noisy production environment | |
| 650 | 4 | |a Oak Ridge National Laboratory | |
| 650 | 4 | |a parallel processing | |
| 650 | 4 | |a Performance evaluation | |
| 650 | 4 | |a platform-neutral ADIOS middleware | |
| 650 | 4 | |a Resource management | |
| 650 | 4 | |a Scalability | |
| 650 | 4 | |a software performance evaluation | |
| 655 | 7 | |a Konferenzschrift |0 (DE-588)1071861417 |0 (DE-627)826484824 |0 (DE-576)433375485 |2 gnd-content | |
| 700 | 1 | |a Brüning, Ulrich |d 1954- |e VerfasserIn |0 (DE-588)1047086840 |0 (DE-627)777543397 |0 (DE-576)400390639 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |a International Symposium on Computer Architecture and High Performance Computing (28. : 2016 : Los Angeles, CA) |t 2016 28th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD) |d Piscataway, NJ : IEEE, 2016 |g (2016), Seite 9-17 |h xix, 224 Seiten |w (DE-627)1656469251 |w (DE-576)505445735 |z 9781509061082 |7 nnam |
| 773 | 1 | 8 | |g year:2016 |g pages:9-17 |g extent:9 |a Using balanced data placement to address I/O contention in production environments |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1109/SBAC-PAD.2016.10 |x Resolving-System |x Verlag |3 Volltext |
| 856 | 4 | 0 | |u https://ieeexplore.ieee.org/document/7789318/ |x Verlag |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20180523 | ||
| 993 | |a ConferencePaper | ||
| 994 | |a 2016 | ||
| 998 | |g 1047086840 |a Brüning, Ulrich |m 1047086840:Brüning, Ulrich |d 700000 |d 720000 |e 700000PB1047086840 |e 720000PB1047086840 |k 0/700000/ |k 1/700000/720000/ |p 6 |y j | ||
| 998 | |g 1159979707 |a Neuwirth, Sarah |m 1159979707:Neuwirth, Sarah |d 700000 |d 720000 |e 700000PN1159979707 |e 720000PN1159979707 |k 0/700000/ |k 1/700000/720000/ |p 1 |x j | ||
| 999 | |a KXP-PPN1575446774 |e 3010008287 | ||
| BIB | |a Y | ||
| JSO | |a {"recId":"1575446774","language":["eng"],"note":["Gesehen am 23.05.2018"],"type":{"bibl":"chapter","media":"Online-Ressource"},"title":[{"title":"Using balanced data placement to address I/O contention in production environments","title_sort":"Using balanced data placement to address I/O contention in production environments"}],"person":[{"family":"Neuwirth","given":"Sarah","roleDisplay":"VerfasserIn","display":"Neuwirth, Sarah","role":"aut"},{"role":"aut","display":"Brüning, Ulrich","roleDisplay":"VerfasserIn","given":"Ulrich","family":"Brüning"}],"relHost":[{"title":[{"title":"2016 28th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)","subtitle":"proceedings : 26 - 28 October 2016, Los Angeles, California, USA","title_sort":"2016 28th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)"}],"part":{"text":"(2016), Seite 9-17","extent":"9","year":"2016","pages":"9-17"},"titleAlt":[{"title":"SBAC-PAD 2016"},{"title":"28th International Symposium on Computer Architecture and High Performance Computing"}],"disp":"International Symposium on Computer Architecture and High Performance Computing (28. : 2016 : Los Angeles, CA)2016 28th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)","note":["Gesehen am 23.05.2018"],"type":{"media":"Online-Ressource","bibl":"book"},"recId":"1656469251","language":["eng"],"corporate":[{"role":"aut","display":"International Symposium on Computer Architecture and High Performance Computing (28., 2016, Los Angeles, CA)","roleDisplay":"VerfasserIn"},{"roleDisplay":"Herausgebendes Organ","display":"Institute of Electrical and Electronics Engineers","role":"isb"}],"origin":[{"dateIssuedKey":"2016","publisher":"IEEE","dateIssuedDisp":"2016","publisherPlace":"Piscataway, NJ"}],"id":{"eki":["1656469251"],"isbn":["9781509061082"]},"physDesc":[{"extent":"xix, 224 Seiten"}]}],"physDesc":[{"extent":"9 S."}],"id":{"doi":["10.1109/SBAC-PAD.2016.10"],"eki":["1575446774"]},"origin":[{"dateIssuedDisp":"2016","dateIssuedKey":"2016"}],"name":{"displayForm":["Sarah Neuwirth, Feiyi Wang, Sarp Oral, Sudharshan Vazhkudai, James H. Rogers and Ulrich Bruening"]}} | ||
| SRT | |a NEUWIRTHSAUSINGBALAN2016 | ||