Optimizing the data-collection time of a large-scale data-acquisition system through a simulation framework
The ATLAS detector at CERN records particle collision “events” delivered by the Large Hadron Collider. Its data-acquisition system identifies, selects, and stores interesting events in near real-time, with an aggregate throughput of several 10 GB/s. It is a distributed software system executed on a...
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
8 July 2016
|
| In: |
The journal of supercomputing
Year: 2016, Jahrgang: 72, Heft: 12, Pages: 4546-4572 |
| ISSN: | 1573-0484 |
| DOI: | 10.1007/s11227-016-1764-1 |
| Online-Zugang: | Verlag, Volltext: http://dx.doi.org/10.1007/s11227-016-1764-1 Verlag, Volltext: https://link.springer.com/content/pdf/10.1007%2Fs11227-016-1764-1.pdf Verlag, Volltext: https://link.springer.com/article/10.1007/s11227-016-1764-1 
|
| Verfasserangaben: | Tommaso Colombo, Holger Fröning, Pedro Javier Garcìa, Wainer Vandelli |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1560392827 | ||
| 003 | DE-627 | ||
| 005 | 20220813185931.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 170703s2016 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1007/s11227-016-1764-1 |2 doi | |
| 035 | |a (DE-627)1560392827 | ||
| 035 | |a (DE-576)490392822 | ||
| 035 | |a (DE-599)BSZ490392822 | ||
| 035 | |a (OCoLC)1340976502 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 28 |2 sdnb | ||
| 100 | 1 | |a Colombo, Tommaso |d 1986- |e VerfasserIn |0 (DE-588)1136286225 |0 (DE-627)892497033 |0 (DE-576)490391966 |4 aut | |
| 245 | 1 | 0 | |a Optimizing the data-collection time of a large-scale data-acquisition system through a simulation framework |c Tommaso Colombo, Holger Fröning, Pedro Javier Garcìa, Wainer Vandelli |
| 264 | 1 | |c 8 July 2016 | |
| 300 | |a 27 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 03.07.2017 | ||
| 520 | |a The ATLAS detector at CERN records particle collision “events” delivered by the Large Hadron Collider. Its data-acquisition system identifies, selects, and stores interesting events in near real-time, with an aggregate throughput of several 10 GB/s. It is a distributed software system executed on a farm of roughly 2000 commodity worker nodes communicating via TCP/IP on an Ethernet network. Event data fragments are received from the many detector readout channels and are buffered, collected together, analyzed and either stored permanently or discarded. This system, and data-acquisition systems in general, are sensitive to the latency of the data transfer from the readout buffers to the worker nodes. Challenges affecting this transfer include the many-to-one communication pattern and the inherently bursty nature of the traffic. The main performance issues brought about by this workload are addressed in this paper, focusing in particular on the so-called TCP incast pathology. Since performing systematic studies of these issues is often impeded by operational constraints related to the mission-critical nature of these systems, we developed a simulation model of the ATLAS data-acquisition system. The resulting simulation tool is based on the well-established, widely-used OMNeT++ framework. This tool was successfully validated by comparing the obtained simulation results with existing measurements of the system’s behavior. Furthermore, the simulation tool enables the study of the theoretical behavior of the system in numerous what-if scenarios and with modifications that are not immediately applicable to the real system. In this paper, we take advantage of this to analyze the behavior of the system using different traffic shaping and scheduling policies, and with network hardware modifications. This analysis leads to conclusions that could be used to devise future system enhancements. | ||
| 700 | 1 | |a Fröning, Holger |d 1976- |e VerfasserIn |0 (DE-588)133209466 |0 (DE-627)538678658 |0 (DE-576)299696189 |4 aut | |
| 700 | 1 | |a García, Pedro Javier |e VerfasserIn |0 (DE-588)1136286659 |0 (DE-627)892497688 |0 (DE-576)490392466 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t The journal of supercomputing |d Dordrecht [u.a.] : Springer Science + Business Media B.V, 1987 |g 72(2016), 12, Seite 4546-4572 |h Online-Ressource |w (DE-627)271350202 |w (DE-600)1479917-0 |w (DE-576)11061514X |x 1573-0484 |7 nnas |a Optimizing the data-collection time of a large-scale data-acquisition system through a simulation framework |
| 773 | 1 | 8 | |g volume:72 |g year:2016 |g number:12 |g pages:4546-4572 |g extent:27 |a Optimizing the data-collection time of a large-scale data-acquisition system through a simulation framework |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1007/s11227-016-1764-1 |x Verlag |x Resolving-System |3 Volltext |
| 856 | 4 | 0 | |u https://link.springer.com/content/pdf/10.1007%2Fs11227-016-1764-1.pdf |x Verlag |3 Volltext |
| 856 | 4 | 0 | |u https://link.springer.com/article/10.1007/s11227-016-1764-1 |x Verlag |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20170703 | ||
| 993 | |a Article | ||
| 994 | |a 2016 | ||
| 998 | |g 133209466 |a Fröning, Holger |m 133209466:Fröning, Holger |d 700000 |d 720000 |e 700000PF133209466 |e 720000PF133209466 |k 0/700000/ |k 1/700000/720000/ |p 2 | ||
| 998 | |g 1136286225 |a Colombo, Tommaso |m 1136286225:Colombo, Tommaso |d 700000 |d 720000 |e 700000PC1136286225 |e 720000PC1136286225 |k 0/700000/ |k 1/700000/720000/ |p 1 |x j | ||
| 999 | |a KXP-PPN1560392827 |e 2973053501 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"name":{"displayForm":["Tommaso Colombo, Holger Fröning, Pedro Javier Garcìa, Wainer Vandelli"]},"language":["eng"],"person":[{"display":"Colombo, Tommaso","given":"Tommaso","family":"Colombo","role":"aut"},{"family":"Fröning","role":"aut","display":"Fröning, Holger","given":"Holger"},{"family":"García","role":"aut","display":"García, Pedro Javier","given":"Pedro Javier"}],"recId":"1560392827","note":["Gesehen am 03.07.2017"],"type":{"media":"Online-Ressource","bibl":"article-journal"},"title":[{"title":"Optimizing the data-collection time of a large-scale data-acquisition system through a simulation framework","title_sort":"Optimizing the data-collection time of a large-scale data-acquisition system through a simulation framework"}],"id":{"doi":["10.1007/s11227-016-1764-1"],"eki":["1560392827"]},"physDesc":[{"extent":"27 S."}],"relHost":[{"origin":[{"dateIssuedDisp":"1987-","publisher":"Springer Science + Business Media B.V ; Kluwer","publisherPlace":"Dordrecht [u.a.] ; Dordrecht [u.a.]","dateIssuedKey":"1987"}],"title":[{"title_sort":"journal of supercomputing","subtitle":"an international journal of high-performance computer design, analysis and use","title":"The journal of supercomputing"}],"id":{"zdb":["1479917-0"],"issn":["1573-0484"],"eki":["271350202"]},"physDesc":[{"extent":"Online-Ressource"}],"recId":"271350202","disp":"Optimizing the data-collection time of a large-scale data-acquisition system through a simulation frameworkThe journal of supercomputing","type":{"media":"Online-Ressource","bibl":"periodical"},"pubHistory":["1.1987 -"],"note":["Gesehen am 02.11.05"],"language":["eng"],"part":{"year":"2016","volume":"72","issue":"12","pages":"4546-4572","extent":"27","text":"72(2016), 12, Seite 4546-4572"}}],"origin":[{"dateIssuedKey":"2016","dateIssuedDisp":"8 July 2016"}]} | ||
| SRT | |a COLOMBOTOMOPTIMIZING8201 | ||