NGINX – Can NGINX ‘proxy_cache_valid’ Be Relative to Last-Modified

I've been beating my head against the wall trying to get live DASH and HLS video streams to be cached correctly by an NGINX caching HTTPS reverse-proxy server.

My current thinking is that my main problem may be that the NGINX proxy_cache_valid directive appears to be relative to when the file entered the cache. What I think I need is for cache retention to be relative to the Last-Modified header field. NGINX provides access to this via sub_filter_last_modified on and $upstream_http_last_modified, but I haven't yet found how to get "Last-Modified + 10 seconds" set as the cache expiration time.

1. Is there a way to set proxy_cache_valid to "Last-Modified + 10 seconds"?

Another possible approach I found is for the backend to provide the X-Accel-Expires header field with a @ (ref: http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid).

2. Is X-Accel-Expires the "most correct" way for NGINX to cache real-time media content?

3. Are there other, better ways I should investigate?

Thanks, in advance. (This is my third bite at this apple, my prior questions being too vague and clueless to get useful answers.)

Some background / overview / context for those unfamiliar with Live video streaming over HTTP:

The video flow starts with an MPEG Transport Stream (TS) provided by a digital TV tuner (ATSC @ 1080i) that is fed to an MPEG transcoder (ffmpeg with hardware assist) that in real-time generates 3 resolutions (720p, 480p, 360p) in CBR (Constant Bit Rate) that are then fed to a packager (e.g., GPAC or Shaka) that generates the dynamic media files (manifests/playlists + audio + video) for bandwidth-adaptive streaming (e.g., fading WiFi causes the resolution to decrease, which will automatically increase when bandwidth improves).

New file sets are created every 2-10 seconds (the "segment time"), depending on how the packager is configured; we're presently using 10 second segments to minimize the number of transfers, though we hope to eventually go to 2 seconds to minimize latency. All these files are served up by NGINX via a simple page listing the available live streams.

To give you an idea of the scale, a single 1080i input generates 3 CBR streams, each of which gets packaged as DASH and HLS, for 6 output streams created from each input stream. In total, the packager outputs 10 files every 10 seconds for each 1080i input. With 24 1080i inputs (not uncommon in many cities), that's 240 new files every 10 seconds, or 86,400 new files every hour. The cache is sized for 1 minute of history, or 1,440 media files (<1 GB).

The video and audio segment files (which contain MPEG data) are created with unique names, so letting them LRU out of a full cache is no problem. But the DASH manifests and HLS playlists are formatted text files that are rewritten every segment time (to list the new audio and video segment files), so the manifests/playlists must be replaced when updated.

The tuner-trancoder-packager is great at its job, but it's not a great web server past a dozen clients, primarily due to the overhead of HTTPS. The external caching HTTPS reverse-proxy is needed to minimize this load, its job being to look like a single client no matter how many actual video users there are. However, the video users shouldn't be able to tell that the caching reverse-proxy is present, so having always-correct caching is vital.

I'm presently seeing video from the cache occasionally freeze, though it plays fine directly from the video server. HLS sooner than DASH, but both will freeze in 1-8 hours.

The video server and the caching reverse-proxy both run NGINX to leverage (tune/optimize) the combined feature set.