hdf5-global-heap · shipping · also HDF5 global heap collection

HDF5 global heap

The HDF5 global heap stores file objects, including variable-length payloads, that datasets access through heap identifiers.

Related Observatory article: HDF5 global heap on hesela.com

A file can contain multiple global-heap collections. They are reached through references rather than a single directory of all collections; "global" does not mean one contiguous region. Variable-length elements in dataset storage contain a sequence length and heap identifier instead of the complete payload. [1]

This is an on-disk structure, not the process-memory heap. It is also distinct from HDF5 fractal heaps, whose uses and filtering rules must not be assumed to apply to global heaps. [1]

Dataset filters do not process the separate variable-length payload heaps. A compressed chunk can therefore hold compressed descriptors while leaving referenced strings uncompressed. Raw chunk storage is insufficient to budget the complete file; the linked experiment compares three representations with byte-exact readback. [2] [3]

Relations

related
HDF5, HDF5 chunked layout, HDF5 raw-data chunk cache

Citations

  1. HDF5 File Format Specification: III.E Global Heap and IV.B Data Object Data Storage (accessed 2026-10-20)
  2. HDF Group: H5Pset_filter, dataset filters and heap scope (accessed 2026-10-20)
  3. Hesela: variable-length string storage controls and measurements (accessed 2026-10-20)

JSON: /data/terms/hdf5-global-heap.json

Published by Hesela team · Updated

Report a correction · Editorial standards · Reuse guidance

Cite this page
Hesela team. HDF5 global heap. Hesela. 2026-10-20. https://hesela.dev/terms/hdf5-global-heap/