In this episode of Call the Doctor, Scot Breitenfeld introduces the h5fuse.sh script, an updated utility for consolidating sub-files into a single hdf5 file. Using Cabana as an example application, he showcases the script's practicality. Breitenfeld presents performance metrics, comparing writing with no local storage to writing as a single shared file on the Leicester file system, with the latter demonstrating notable improvement. He also explores the effectiveness of Progressive File Layout on Frontier, which surprisingly outperforms manual configuration of stripe size and count. However, execution times vary, ranging from 5 to 35 seconds per node.
Throughout the discussion, potential enhancements are mentioned, such as employing more sub-filing threads and multiple processes per node. Breitenfeld also suggests the possibility of a workshop feedback survey to gather user input. He reports AMR failures on AArch64 architecture and highlights upcoming releases (1.13.3 and 1.14) addressing bug fixes and enhancements. The presentation concludes with addressing ongoing challenges related to NFS compatibility and outlining plans for further development in this domain
This session happened on August 22, 2023. You can also watch this video online.
Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help users tackle real-world HDF problems from a common cold to severe headaches and offer relief where that’s possible. As time permits, we will include how-tos, offer advice on tool usage, review your code samples, teach you survival in the documentation jungle, and discuss what’s new or just around the corner in the land of HDF.
Join us every Tuesday at 12:20 p.m. central (US/Canada.) on Zoom!
In this episode of Call the Doctor, Scot Breitenfeld introduces the h5fuse.sh script, an updated utility for consolidating sub-files into a single hdf5 file. Using Cabana as an example application, he showcases the script's practicality. Breitenfeld presents performance metrics, comparing writing with no local storage to writing as a single shared file on the Leicester file system, with the latter demonstrating notable improvement. He also explores the effectiveness of Progressive File Layout on Frontier, which surprisingly outperforms manual configuration of stripe size and count. However, execution times vary, ranging from 5 to 35 seconds per node.
Throughout the discussion, potential enhancements are mentioned, such as employing more sub-filing threads and multiple processes per node. Breitenfeld also suggests the possibility of a workshop feedback survey to gather user input. He reports AMR failures on AArch64 architecture and highlights upcoming releases (1.13.3 and 1.14) addressing bug fixes and enhancements. The presentation concludes with addressing ongoing challenges related to NFS compatibility and outlining plans for further development in this domain
This session happened on August 22, 2023. You can also watch this video online.
Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help users tackle real-world HDF problems from a common cold to severe headaches and offer relief where that’s possible. As time permits, we will include how-tos, offer advice on tool usage, review your code samples, teach you survival in the documentation jungle, and discuss what’s new or just around the corner in the land of HDF.
Join us every Tuesday at 12:20 p.m. central (US/Canada.) on Zoom!