HDF5Buffer#

class fields.buffers.core.HDF5Buffer(array: Dataset, __is_file_owner__: bool = False)[source]#

A buffer that wraps a lazily-loaded HDF5 dataset using h5py.Dataset.

This buffer enables disk-backed storage of field data, making it ideal for large datasets that exceed memory constraints. Internally, it wraps a persistent HDF5 dataset and exposes a NumPy-compatible interface for slicing, indexing, and arithmetic operations.

Unlike in-memory buffers like ArrayBuffer or UnytArrayBuffer, this buffer does not eagerly load its contents into memory. Instead, all reads and writes are performed through a memory-mapped interface, and computations are supported through seamless integration with NumPy ufuncs.

If units are provided (either at creation or via a unit-aware input), they are stored as a string attribute on the dataset and automatically reapplied when data is accessed.

Examples

Create a disk-backed buffer with units:

>>> buff = HDF5Buffer.from_array([1, 2, 3],
...                       "mydata.h5",
...                       "dataset",
...                       units=None,
...                       overwrite=True,dtype='f8',
...                       create_file=True)
>>> buff
HDF5Buffer(shape=(3,), dtype=float64, path='/dataset')

See also

ArrayBuffer: In-memory NumPy array backend.
UnytArrayBuffer: In-memory unit-aware buffer.
BufferBase: Abstract buffer interface.

Methods

`__init__`(array[, __is_file_owner__])	Create an `HDF5Buffer` object.
`as_array`()	Return a NumPy array view of the full dataset (units stripped).
`as_core`()	Return the raw backend array object stored in this buffer.
`as_repr`()	Return this buffer as an `unyt_array` if the buffer has units; otherwise as a `ndarray`.
`as_unyt_array`()	Return a unit-aware unyt_array view of the full dataset.
`astype`(dtype, *args[, inplace])	Return a copy of this buffer with a different data type.
`broadcast_to`(shape, *args[, inplace])	Broadcast an array to a new shape.
`can_handle`(obj)	Return `True` if obj can be wrapped by this buffer class.
`can_handle_list`()	Return a list of type names that this buffer can wrap.
`close`([force])	Close the underlying HDF5 file, if owned by this buffer.
`convert_to_base`([unit_system, equivalence])	Convert this buffer in-place to base units for the given unit system.
`convert_to_units`(units[, equivalence])	Convert the buffer data to the specified physical units in-place.
`copy`(args, *kwargs)	Return a deep copy of this buffer.
`create_dataset`(file, name[, shape, dtype, ...])	Create a new HDF5 dataset with optional unit metadata.
`empty`(shape, *args[, dtype])	Create an HDF5-backed buffer filled with zeros.
`empty_like`(other, args, *kwargs)	Create a new buffer allocation matching the shape of another buffer.
`expand_dims`(axis, *args[, inplace])	Expand the shape of this buffer by inserting a new axis.
`flatten`(*args[, order, inplace])	Return a flattened 1D view of this buffer.
`flush`()	Flush any buffered data to disk without closing the file.
`from_array`(obj[, file, path, dtype, units])	Construct an HDF5-backed buffer from an array-like object or existing HDF5 dataset.
`full`(shape, *args[, fill_value, dtype])	Create an HDF5-backed buffer filled with a constant value or quantity.
`full_like`(other[, fill_value])	Create a new buffer filled with a constant value and matching the shape of another buffer.
`in_units`(units, *args[, equivalence, ...])	Return a new copy of this buffer cast to the specified physical units.
`ones`(shape, *args[, dtype])	Create an HDF5-backed buffer filled with ones.
`ones_like`(other, args, *kwargs)	Create a new buffer filled with ones and matching the shape of another buffer.
`open`(file, path, *[, mode, close_on_exit])	Open an existing HDF5 dataset and return it as a buffer.
`replace_dataset`([shape, dtype, data, units])	Replace the current HDF5 dataset with a newly created one.
`reshape`(shape, *args[, inplace])	Return a reshaped copy of this buffer.
`resolve`(array_like, *args[, buffer_registry])	Resolve and instantiate a buffer subclass for an arbitrary array-like input.
`squeeze`(*args[, axis, inplace])	Remove singleton dimensions from this buffer.
`to`(units, *args[, equivalence, ...])	Return a new buffer (or array) with values cast to the specified units.
`to_value`(units[, equivalence])	Return a NumPy array of values converted to the specified physical units.
`transpose`([axes, inplace])	Return a transposed copy of this buffer.
`zeros`(shape, *args[, dtype])	Create an HDF5-backed buffer filled with zeros.
`zeros_like`(other, args, *kwargs)	Create a new buffer filled with zeros and matching the shape of another buffer.

Attributes

`c`	Shorthand for as_core().
`d`	Shorthand for as_array().
`dtype`	Data type of the array.
`file`	The open h5py.File object backing this buffer.
`filename`	Absolute path to the file backing this buffer.
`has_units`	Whether the buffer carries unit metadata.
`is_open`	True if the underlying dataset is still attached to an open file.
`is_owner`	Whether this buffer owns the file (i.e., responsible for closing it).
`name`	Internal HDF5 path to the dataset.
`ndim`	Number of dimensions.
`shape`	Shape of the underlying array.
`size`	Total number of elements.
`units`	Return the physical units stored as an HDF5 attribute.
`v`	Shorthand for as_unyt_array().

HDF5Buffer#

This Page