Python – mechanism to identify compressed file type and uncompress

compressionpythonrartarzip

A compressed file can be classified into below logical groups
a. The operating system which you are working on (*ix, Win) etc.
b. Different types of compression algorithm (i.e .zip,.Z,.bz2,.rar,.gzip). Atleast from a standard list of mostly used compressed files.
c. Then we have tar ball mechanism – where I suppose there are no compression. But it acts more like a concatenation.

Now, if we start addressing the above set of compressed files,
a. Option (a) would be taken care by python since it is platform independent language.
b. Option (b) and (c) seems to have a problem.

What do I need
How do I identify the file type (compression type) and then UN-compress them?

Like:

fileType = getFileType(fileName)  
switch(fileType):  
case .rar:  unrar....
case .zip:  unzip....

etc

So the fundamental question is how do we identify the compression algorithm based on the file (assuming the extension is not provided or incorrect)? Is there any specific way to do it in python?

Best Answer

This page has a list of "magic" file signatures. Grab the ones you need and put them in a dict like below. Then we need a function that matches the dict keys with the start of the file. I've written a suggestion, though it can be optimized by preprocessing the magic_dict into e.g. one giant compiled regexp.

magic_dict = {
    "\x1f\x8b\x08": "gz",
    "\x42\x5a\x68": "bz2",
    "\x50\x4b\x03\x04": "zip"
    }

max_len = max(len(x) for x in magic_dict)

def file_type(filename):
    with open(filename) as f:
        file_start = f.read(max_len)
    for magic, filetype in magic_dict.items():
        if file_start.startswith(magic):
            return filetype
    return "no match"

This solution should be cross-plattform and is of course not dependent on file name extension, but it may give false positives for files with random content that just happen to start with some specific magic bytes.

Related Solutions

Python – What’s the canonical way to check for type in Python

To check if o is an instance of str or any subclass of str, use isinstance (this would be the "canonical" way):

if isinstance(o, str):

To check if the type of o is exactly str (exclude subclasses):

if type(o) is str:

The following also works, and can be useful in some cases:

if issubclass(type(o), str):

See Built-in Functions in the Python Library Reference for relevant information.

One more note: in this case, if you're using Python 2, you may actually want to use:

if isinstance(o, basestring):

because this will also catch Unicode strings (unicode is not a subclass of str; both str and unicode are subclasses of basestring). Note that basestring no longer exists in Python 3, where there's a strict separation of strings (str) and binary data (bytes).

Alternatively, isinstance accepts a tuple of classes. This will return True if o is an instance of any subclass of any of (str, unicode):

if isinstance(o, (str, unicode)):

Python – How to determine a Python variable’s type

Use the type() builtin function:

>>> i = 123
>>> type(i)
<type 'int'>
>>> type(i) is int
True
>>> i = 123.456
>>> type(i)
<type 'float'>
>>> type(i) is float
True

To check if a variable is of a given type, use isinstance:

>>> i = 123
>>> isinstance(i, int)
True
>>> isinstance(i, (float, str, set, dict))
False

Note that Python doesn't have the same types as C/C++, which appears to be your question.

Best Answer

Related Solutions

Python – What’s the canonical way to check for type in Python

Python – How to determine a Python variable’s type

Related Topic