Apache HTTP server – How to determine on the server if a download finished and of which file

apache-2.2file-transfer

We have an Apache HTTP server up and running, which serves dynamically created zip archives to the users which are possibly several 100s of megabytes in size. As we create a new file with every click on the "download" button (even if the content did not change…), we will most likely run into disc capacity problems.

I have a rather dumb script running that deletes all files that have been created 90+minutes ago. Obviously not that nice of a solution.

I was wondering how to determine from the command line of the server when a download finished, successfully or not. In that case I can delete that file as it will not be served twice. Thinking about it, it might even be sufficient to check if the server is using a file or not as it is created right before it is served to the client.

Thanks!

Best Answer

Since deleting a file that a process has an open file handle on leaves the file there until the handle is closed you could just delete them immediately, and when Apache closes the file it'll be removed from disk.

Related Solutions

Linux – Corrupted zip file after using split and cat on Linux

Identical MD5 hashes suggest that the transfer has worked well.

More than 2G filesize sounds suspiciously like some pointer size issue - maybe the zip in question doesn't handle that well? more than (ca) 2G would be a negative number in 32 bit... Can you unzip the file on the system where you zipped it? Do both systems differ? Is one 64bit, the problematic 32 bit? What are the filesystems on both systems? Can you find another zip utility?

If you have a chance to retransmit the content, you might want to use tar.gz or keep file size lower than that value. gzip compressed content should handle this better. Zip stores the contents (index) at the end of the file.

Edit: Yup, see here:

In practice, the real limit may be 2 GB on many systems, due to UnZip's use of the fseek() function to jump around within an archive. Because's fseek's offset argument is usually a signed long integer, on 32-bit systems UnZip will not find any file that is more than 2 GB from the beginning of the archive [...]

Static file download from browser breaking in varnish but works fine in Apache

You are hitting the Varnish send_timeout limit. The default value for send_timeout was 600s, with Varnish 3.0 it was changed to 60s. This may intefere with downloads taking longer than 60s.

You can check the value of the send_timeout parameter with varnishadm:

varnishadm param.show send_timeout

This will output something like:

send_timeout           60 [seconds]
                       Default is 60
                       Send timeout for client connections. If the HTTP
                       response hasn't been transmitted in this many
                       seconds the session is closed. 
                       See setsockopt(2) under SO_SNDTIMEO for more
                       information.

                       NB: This parameter may take quite some time to
                       take (full) effect.

You can set it to 600s with:

varnishadm param.set send_timeout 600s

To make this setting persistent, you have to add "-p sendtimeout 600" to the startup parameters of Varnish. This depends on the distribution you are using. In case of Debian/Ubuntu yo may want to edit /etc/default/varnish.

Best Answer

Related Solutions

Linux – Corrupted zip file after using split and cat on Linux

Static file download from browser breaking in varnish but works fine in Apache

Related Topic