Name: Parallel IO NHR Workshop
Start: 2024-05-07T11:00:00+02:00
End: 2024-05-08T17:00:00+02:00
Location: Bundesstraße 45a

Parallel IO NHR Workshop

from Tuesday, 7 May 2024 (11:00) to Wednesday, 8 May 2024 (17:00)

Monday, 6 May 2024
Tuesday, 7 May 2024

11:00 Reception coffee
Reception coffee
11:00 - 12:00
Room: R034
12:00 I/O in Climate Modeling - Panos Adamidis
I/O in Climate Modeling
- Panos Adamidis
12:00 - 12:40
Room: R034 Opening talk
12:40 Lustre at DKRZ: Stripping Strategies & Best Practices - Carsten Beyer (DKRZ)
Lustre at DKRZ: Stripping Strategies & Best Practices
- Carsten Beyer (DKRZ)
12:40 - 13:20
Room: R034 TBD
13:20 Field notes from explorations on a big file system - Florian Ziemen (DKRZ)
Field notes from explorations on a big file system
- Florian Ziemen (DKRZ)
13:20 - 14:00
Room: R034 In this talk I will present observations on various aspects of the use of DKRZ's 120 TB lustre file system, regarding throughput and usage patterns. They range from back-of-the-envelope calculations via aggregate statistics to the usage of an individual dataset, and may provide valuable insights into the users' needs and things to consider in the development of I/O solutions and the procurement of future storage systems.
14:00 Coffee Break
Coffee Break
14:00 - 14:15
Room: R034
14:15 IO Benchmarking in HPC Systems - Jannek Squar Anna Fuchs (Universität Hamburg)
IO Benchmarking in HPC Systems
- Jannek Squar
- Anna Fuchs (Universität Hamburg)
14:15 - 14:55
Room: R034 Description: TBD
14:55 I/O performance in CLAIX23 infrastructure - Radita Liem (RWTH Aachen)
I/O performance in CLAIX23 infrastructure
- Radita Liem (RWTH Aachen)
14:55 - 15:35
Room: R034 In this topic, I would like to compare the I/O performance changes from the system that we benchmark using IO500. Additionally, we want to see the performance of ICON in the grand scheme of things within CLAIX since currently we are working with DKRZ in the Green HPC project
15:35 Coffee Break And Discussion
Coffee Break And Discussion
15:35 - 16:10
Room: R034
16:10 Parallel HDF5 25 years after - Elena Pourmal
Parallel HDF5 25 years after
- Elena Pourmal
16:10 - 16:50
Room: R034 25 years have passed since the first release of parallel HDF5. The software is still under active development to address constantly evolving HPC requirements. In our talk we will give an overview of the current state of parallel HDF5 library and its new compression and sub-filing capabilities. We will also talk about HDF5 tuning knobs for the HPC applications that were developed over the years.
16:50 hiopy - Optimizing Model Output for Analysis - Tobias Kölling
hiopy - Optimizing Model Output for Analysis
- Tobias Kölling
16:50 - 17:30
Room: R034 As climate models reach the kilometer scale, horizontal model grids outgrow the size of computer screens as well as the capacity of the human eye. In consequence, a model output analyst can't observe the full output at once, but will always use subsets or coarser versions of the data. The time to an analysis result can be reduced dramatically, if output datasets are optimized for the changed read workload. Hiopy is a new way of writing ICON model output, which utilizes YAC and the Zarr format to create such optimized datasets directly from the running model for immediate consumption.
17:30 Panel Discussion
Panel Discussion
17:30 - 18:00
Room: R034
19:00 https://www.rudolphs-hamburg.de/
https://www.rudolphs-hamburg.de/
19:00 - 22:00
Room: R034
Contributions

19:00 Dinner
Wednesday, 8 May 2024

09:00 Receiption Coffee
Receiption Coffee
09:00 - 09:30
Room: R034
09:30 Enabling purposeful use of large-volume Earth System Modelling datasets: ideas and concepts explored at DKRZ - Karsten Peters-von Gehleт
Enabling purposeful use of large-volume Earth System Modelling datasets: ideas and concepts explored at DKRZ
- Karsten Peters-von Gehleт
09:30 - 10:10
Room: R034 Current state-of-the-art and upcoming Earth System Model (ESM) simulations produce output on the order of single- to double digit petabytes per individual climatic timescale-spanning simulation. Creating an infrastructure environment enabling the purposeful analysis of such data amounts requires revamping data handling paradigms for ESM datasets. We present concepts, ideas and prototypes developed along the requirements of the ESM-community to enable efficient ESM output access and analysis across the storage hardware hierarchy at DKRZ
10:10 Databases for HPC and Parallel IO - Jay Lofstead
Databases for HPC and Parallel IO
- Jay Lofstead
10:10 - 10:50
Room: R034 Early explorations into using an RDBMS as a data store for parallel IO workloads led to a conclusion that the technology was ill fitted for the task. The community has accepted this “wisdom” and been reluctant to support any new efforts into investigating databases. I think it is time to revisit.
10:50 Discussion
Discussion
10:50 - 11:30
Room: R034

11:30 Lunch
Lunch
11:30 - 12:30
Room: R034
12:30 Leveraging Flexible Storage System Components for HPC Research - Michael Kuhn (OVGU)
Leveraging Flexible Storage System Components for HPC Research
- Michael Kuhn (OVGU)
12:30 - 13:10
Room: R034 Abstract: Research has become increasingly data-driven, putting additional pressure on the underlying storage systems. Gaining insights into the their behavior is critical understanding and optimizing I/O performance. However, existing storage systems often lack the necessary functionality and are difficult to modify and extend. Therefore, the Parallel Computing and I/O research group is developing several storage system components within the JULEA and Haura projects, making it possible to cover the entire storage stack from application I/O interfaces to block device access. This allows rapidly prototyping new approaches and optimizations.
13:10 Using the DAOS Storage APIs with Weather and Climate Applications - Michael Hennecke
Using the DAOS Storage APIs with Weather and Climate Applications
- Michael Hennecke
13:10 - 13:50
Room: R034 The Distributed Asynchronous Object Storage (DAOS) is an open source scale-out storage system that is designed from the ground up to support Storage Class Memory (SCM) and NVMe storage in user space (https://docs.daos.io/). This presentation provides an overview of the DAOS architecture, and describes the various APIs that are available to the user to benefit from the performance advantages that DAOS offers in comparison to traditional parallel filesystems like GPFS or Lustre. We will also solicit feedback from the community to guide future development efforts.
13:50 Discussion
Discussion
13:50 - 14:30
Room: R034

14:30 Coffee Break
Coffee Break
14:30 - 14:50
Room: R034

14:50 Working Groups and Discussion
Working Groups and Discussion
14:50 - 16:00
Room: R034

16:00 Summary
Summary
16:00 - 16:30
Room: R034 TBD