How to traverse a hdf5 file using h5py

h5py

How do I traverse all the groups and datasets of an hdf5 file using h5py?

I want to retrieve all the contents of the file from a common root using a for loop or something similar.

Best Answer

visit() and visititems() are your friends here. Cf. http://docs.h5py.org/en/latest/high/group.html#Group.visit. Note that an h5py.File is also an h5py.Group. Example (not tested):

def visitor_func(name, node):
    if isinstance(node, h5py.Dataset):
         # node is a dataset
    else:
         # node is a group

with h5py.File('myfile.h5', 'r') as f:
    f.visititems(visitor_func)

Related Solutions

Python – how to export HDF5 file to NumPy using H5PY

h5py already reads files in as numpy arrays, so just:

with h5py.File('the_filename', 'r') as f:
    my_array = f['array_name'][()]

The [()] means to read the entire array in; if you don't do that, it doesn't read the whole data but instead gives you lazy access to sub-parts (very useful when the array is huge but you only need a small part of it).

Python – Incremental writes to hdf5 with h5py

Per the FAQ, you can expand the dataset using dset.resize. For example,

import os
import h5py
import numpy as np
path = '/tmp/out.h5'
os.remove(path)
with h5py.File(path, "a") as f:
    dset = f.create_dataset('voltage284', (10**5,), maxshape=(None,),
                            dtype='i8', chunks=(10**4,))
    dset[:] = np.random.random(dset.shape)        
    print(dset.shape)
    # (100000,)

    for i in range(3):
        dset.resize(dset.shape[0]+10**4, axis=0)   
        dset[-10**4:] = np.random.random(10**4)
        print(dset.shape)
        # (110000,)
        # (120000,)
        # (130000,)

Best Answer

Related Solutions

Python – how to export HDF5 file to NumPy using H5PY

Python – Incremental writes to hdf5 with h5py

Related Topic