Nginx – HAProxy URL rewrite on 404 error

haproxynginx

How to make HAProxy rewrite to a different back-end when the first is missing the file? What I need is errorloc but making a rewrite instead of redirect, so the client is not aware of redirect.

We have developed an application with NginX in mind, which was both load balancing reverse proxy and web server for static files. Application is based on Opa framework which requires sticky sessions based on cookies – supported by both NginX and HAproxy. The application feature we have problem with is dynamic content generation. It generates images on demand, but after generation it is saved on disk and can can be accessed statically with deterministic path.

Problem was easily solved with NginX – it tries to read local file and use load balanced back-end only if the file is missing (not generated yet):

server {
  server_name wkaliszu.pl;
  location /thumb {
    root /path_on_disk/to_cached_content;
    expires 7d;
    # try to access already generated content
    try_files $uri @wkaliszu;
  }
  location / {
    # reverse proxy to the application
    [...]
  }
  location @wkaliszu {
    # reverse proxy to the application
    [...]
  }
}

Server was migrated and now uses HAPproxy for load balancing, which is not web server and does not supports this feature. Now dynamic software generation is performed each time client tries to access the resource, what is much slower and wastes resources. It would be fine if it could use next back-end if the first (simple caching web server for static files) failed with error 404, but I cannot find a way to do it simple way. Redirecting /thumb to NginX, which tries to read static file and again rewrites to HAproxy with new HTTP header only comes to my mind, but I would like to find something better.

Best Answer

HAProxy's backends are either up or down (or on their way to being up/down).

There are various ways to check a backend's health but I am not aware of any that provide per-request based tracking. Once a request fails, that backend would be marked as down or would be failing (on its way to being considered down).

This is very different logic than your Nginx setup which was routing requests on a per-request basis.

I see a couple of options here:

  • Nginx as Caching Proxy
  • Use Apps Servers for Static Content
  • Use a CDN

Caching Proxy

In HAProxy, you would use ACLs to route static content requests to a specific backend. Those backend nodes would run nginx in with a caching proxy. If nginx had the file cached, it would just serve it. If not, it would call your backend.

Use App Servers for Static Content

If your app servers are efficient at serving static content, you may not need to split the request in haproxy. Just send all request to your application backends. Build logic into them to serve static content if available and if not send the request to the backend.

CDN Option

If you can use a dedicated domain for the static content, you may be able to use a CDN. At the CDN, you just point to source URL to your application nodes. You can then control caching at the CDN level. This is similar to the Nginx caching above except the CDN provider is handling it for you.

Related Topic