Python – In python, make a tempfile in the same directory as another file

filepythontemporary-files

I need to update a file. I read it in and write it out with changes. However, I'd prefer to write to a temporary file and rename it into place.

temp = tempfile.NamedTemporaryFile()
tempname = temp.name
temp.write(new_data)
temp.close()
os.rename(tempname, data_file_name)

The problem is that tempfile.NamedTemporaryFile() makes the temporary file in /tmp which is a different file system. This means os.rename() fails. If I use shlib.move() instead then I don't have the atomic update that "mv" provides (for files in the same file system, yadda, yadda, etc.)

I know tempfile.NamedTemporaryFile() takes a "dir" parameter, but data_file_name might be "foo.txt" in which case dir='.'; or data_file_name might be "/path/to/the/data/foo.txt" in which case dir="/path/to/the/data".

What I'd really like is the temp file to be data_file_name + "some random data". This would have the benefit of failing in a way that would leave behind useful clues.

Suggestions?

Best Answer

You can use:

prefix to make the temporary file begin with the same name as the original file.
dir to specify where to place the temporary file.
os.path.split to split the directory from the filename.

import tempfile
import os
dirname, basename = os.path.split(filename)
temp = tempfile.NamedTemporaryFile(prefix=basename, dir=dirname)
print(temp.name)

Practical Tips

<package>.__path__ lets you identify the location(s) of a specific package: (details)

$ python -c "import setuptools as _; print(_.__path__)"
['/usr/lib/python2.7/dist-packages/setuptools']

<module>.__file__ lets you identify the location of a specific module: (difference)
```
$ python3 -c "import os as _; print(_.__file__)"
/usr/lib/python3.6/os.py
```

Run pip show <package> to show Debian-style package information:

$ pip show pytest
Name: pytest
Version: 3.8.2
Summary: pytest: simple powerful testing with Python
Home-page: https://docs.pytest.org/en/latest/
Author: Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, Florian Bruhin and others
Author-email: None
License: MIT license
Location: /home/peter/.local/lib/python3.4/site-packages
Requires: more-itertools, atomicwrites, setuptools, attrs, pathlib2, six, py, pluggy

Python – How to get file creation & modification date/times in Python

Getting some sort of modification date in a cross-platform way is easy - just call os.path.getmtime(path) and you'll get the Unix timestamp of when the file at path was last modified.

Getting file creation dates, on the other hand, is fiddly and platform-dependent, differing even between the three big OSes:

On Windows, a file's ctime (documented at https://msdn.microsoft.com/en-us/library/14h5k7ff.aspx) stores its creation date. You can access this in Python through os.path.getctime() or the .st_ctime attribute of the result of a call to os.stat(). This won't work on Unix, where the ctime is the last time that the file's attributes or content were changed.
On Mac, as well as some other Unix-based OSes, you can use the .st_birthtime attribute of the result of a call to os.stat().
On Linux, this is currently impossible, at least without writing a C extension for Python. Although some file systems commonly used with Linux do store creation dates (for example, ext4 stores them in st_crtime) , the Linux kernel offers no way of accessing them; in particular, the structs it returns from stat() calls in C, as of the latest kernel version, don't contain any creation date fields. You can also see that the identifier st_crtime doesn't currently feature anywhere in the Python source. At least if you're on ext4, the data is attached to the inodes in the file system, but there's no convenient way of accessing it.

The next-best thing on Linux is to access the file's mtime, through either os.path.getmtime() or the .st_mtime attribute of an os.stat() result. This will give you the last time the file's content was modified, which may be adequate for some use cases.

Putting this all together, cross-platform code should look something like this...

import os
import platform

def creation_date(path_to_file):
    """
    Try to get the date that a file was created, falling back to when it was
    last modified if that isn't possible.
    See http://stackoverflow.com/a/39501288/1709587 for explanation.
    """
    if platform.system() == 'Windows':
        return os.path.getctime(path_to_file)
    else:
        stat = os.stat(path_to_file)
        try:
            return stat.st_birthtime
        except AttributeError:
            # We're probably on Linux. No easy way to get creation dates here,
            # so we'll settle for when its content was last modified.
            return stat.st_mtime

Best Answer

Related Solutions

Python – How to find the location of the Python site-packages directory

Practical Tips

Python – How to get file creation & modification date/times in Python

Related Topic