The HDF Group's Call the Doctor

New HSDS Features Coming Soon and a Design Proposal for Long Running Tasks - John Readey

January 09, 2024 The HDF Group
The HDF Group's Call the Doctor
New HSDS Features Coming Soon and a Design Proposal for Long Running Tasks - John Readey
Show Notes

In this episode of Call the Doctor, The HDF Group's John Readey discusses upcoming features in the HSDS release. Since John is in China, the session is pre-recorded, with software engineer Matt Larson available to answer questions. The discussed features include shape reduction, broadcasting, UTF-8 fixed strings, quick scan, N-bit and scale offset filters, enhanced array type support, field ops for compound types, support for long attribute names, non-UTF-8 encodable attributes, multi-op attributes, long link names, and hyper chunking. John also introduces the concepts of async tasks for long-running operations and using Parquet for encoding chunks with variable-length types.  The design doc for async tasks can be viewed on github.  The episode includes a demonstration of the attribute multi-op feature, showcasing its efficiency compared to a serial approach.

You can participate in the discussion of features for HSDS 0.9 on the forum.

This session happened on January 9, 2024. You can also watch this episode.

Call the Doctor is a series of weekly, unscripted, live events! The HDF Group’s staff members will answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help users tackle real-world HDF problems from a common cold to severe headaches and offer relief where that’s possible. As time permits, we will include how-tos, offer advice on tool usage, review your code samples, teach you survival in the documentation jungle, and discuss what’s new or just around the corner in the land of HDF.

Join us every Tuesday at 12:20 p.m. central (US/Canada.) on Zoom!