Python – Appending more datasets into an existing Hdf5 file without deleting other groups and datasets

dataseth5pyhdf5python

I have an HDF5 file which contains groups and subgroups inside which there are datasets. I want to open the file and add some datasets to the groups. I took the following approach which is quite simple in python.

    import h5py
    f = h5py.File('filename.h5','w')
    f.create_dataset('/Group1/subgroup1/dataset4', data=pngfile)
    f.close()

The before file looked like this

After File looks like this

but I want it to not delete other datasets and groups but rather just append the dataset4 in the line.

Best Answer

Just like with the Python open() function, 'w' will truncate any existing file. Use the 'a' mode to add content to the file:

import h5py
f = h5py.File('filename.h5','a')
f.create_dataset('/Group1/subgroup1/dataset4', data=pngfile)
f.close()

Related Solutions

Python – How to determine the size of an object in Python

Just use the sys.getsizeof function defined in the sys module.

sys.getsizeof(object[, default]):

Return the size of an object in bytes. The object can be any type of object. All built-in objects will return correct results, but this does not have to hold true for third-party extensions as it is implementation specific.

Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to.

The default argument allows to define a value which will be returned if the object type does not provide means to retrieve the size and would cause a TypeError.

getsizeof calls the object’s __sizeof__ method and adds an additional garbage collector overhead if the object is managed by the garbage collector.

See recursive sizeof recipe for an example of using getsizeof() recursively to find the size of containers and all their contents.

Usage example, in python 3.0:

>>> import sys
>>> x = 2
>>> sys.getsizeof(x)
24
>>> sys.getsizeof(sys.getsizeof)
32
>>> sys.getsizeof('this')
38
>>> sys.getsizeof('this also')
48

If you are in python < 2.6 and don't have sys.getsizeof you can use this extensive module instead. Never used it though.

Python – a mixin, and why are they useful

A mixin is a special kind of multiple inheritance. There are two main situations where mixins are used:

You want to provide a lot of optional features for a class.
You want to use one particular feature in a lot of different classes.

For an example of number one, consider werkzeug's request and response system. I can make a plain old request object by saying:

from werkzeug import BaseRequest

class Request(BaseRequest):
    pass

If I want to add accept header support, I would make that

from werkzeug import BaseRequest, AcceptMixin

class Request(AcceptMixin, BaseRequest):
    pass

If I wanted to make a request object that supports accept headers, etags, authentication, and user agent support, I could do this:

from werkzeug import BaseRequest, AcceptMixin, ETagRequestMixin, UserAgentMixin, AuthenticationMixin

class Request(AcceptMixin, ETagRequestMixin, UserAgentMixin, AuthenticationMixin, BaseRequest):
    pass

The difference is subtle, but in the above examples, the mixin classes weren't made to stand on their own. In more traditional multiple inheritance, the AuthenticationMixin (for example) would probably be something more like Authenticator. That is, the class would probably be designed to stand on its own.

Best Answer

Related Solutions

Python – How to determine the size of an object in Python

Python – a mixin, and why are they useful

Related Topic