Contribution to a conference proceedings/Contribution to a book

GSI-2026-00371

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png From terabytes to petabyte: scaling of the archiving system for FAIR

Rapp, V. (Corresponding author)GSI* ; Kerssemakers, J.GSI* ; Klimczyk, K. ; Salabura, P.

2025
JACoW Publishing Geneva, Switzerland
ISBN: 978-3-95450-255-4

Please use a persistent id in citations: doi:10.18429/JACOW-ICALEPCS2025-TUPD109  doi:10.15120/GSI-2026-00371

Abstract: With the recent rise of AI and various machine learning models, the importance of storing and managing data generated by control systems is greater than ever before. In 2016, GSI began developing an archiving system to collect, store, and retrieve data from the diverse accelerator devices managed by the GSI control infrastructure. The system was successfully deployed in production in 2021. To evaluate its capabilities and suitability for operational needs, the system was initially launched with a limited storage capacity of 50 TB and reduced computing power. With a current data volume of over 100 GB per day, the archiving system quickly exceeded its initial limits. However, the experience gained in day-to-day operations thus far has allowed us to better understand our use-cases and identify areas for further improvement. In preparation for the anticipated start of FAIR operations in 2027, the system will require significant scaling to meet future demands. Therefore, this is an opportune moment to review and refine the system’s architecture based on the experience gained so far. This paper outlines the challenges encountered with the current implementation and presents the solutions that will be incorporated into the system for FAIR operations.

Keyword(s): Accelerator Physics ; MC16 - MC16: Data Management and Analytics

Note: Published by JACoW Publishing under the terms of the Creative Commons Attribution 4.0 license.

---

Contributing Institute(s):

1. Accelerator Controls (ACO)

Research Program(s):

1. 621 - Accelerator Research and Development (POF4-621) (POF4-621)
2. FAIR Phase-0 - FAIR Phase-0 Research Program (GSI-FAIR-Phase-0) (GSI-FAIR-Phase-0)

Experiment(s):

1. no experiment theory work (theory)

Appears in the scientific report 2025

---

Database coverage:
;

---