HAProxy – Multiple sites, multiple acl’s

access-control-listapache-2.4blockinghaproxy

We are busy with setting up HAProxy. Almost everything is working, except setting it up without multiple sites with different acl's.

What we want:

Using HAProxy with multiple sites, all on the same IP, all with the same backends but with different IP Blockings/Username-Password protection.

What we have working:

Get incoming traffic to backend (with an acl to use https instead of http).

The situation without haproxy/varnish:

Now, we are using apache to handle the acl (allow,deny ip's and use username-password protection).

We tried to capture this through X-Forwarded-For inside apache. However, the web servers themselves can no longer communicate with each other. We had to remove the allow role for the subnet (loadbalancer and webserver are in the same subnet). With this allow, apache don't look at the x-forward-header, and will allow all traffic. Without it, the webservers cannot communicate with each other.

Does anyone have an example or mindset for us to get there?

Now: internet->firewall->apache

What we want: internet->firewall->haproxy(ssl offloading)->varnish->apache (haproxy and varnish running on the same machine)

Best Answer

There are a few ways to set this up. The most simple way would probably be with the use of some acls.

Eg.:

Defining the different websites

acl site1 hdr_end(host) site1.com
acl site2 hdr_end(host) site2.net
acl site3 hdr_end(host) site3.org

(I used hrd_end on the host header, so that any subdomain as well as without subdomain would match. You of course need to determine if this is also a desired condition in your case.)

Defining the allowed sources for the sites

acl allowed_site1 src 1.2.3.4/32 1.2.4.5/32
acl allowed_site2 src 2.3.4.5/32 3.4.5.0/24
acl allowed_site3 src 4.5.0.0/16

Denying all that does not match

http-request deny if site1 !allowed_site1
http-request deny if site2 !allowed_site2
http-request deny if site3 !allowed_site3

(Basically this reads as: deny the request if the request is for the specified site, but is not on the list of allowed IP addresses)

Related Solutions

Nginx/haproxy/IIS – Configuration questions and concerns

That's a similar setup to ours, except we don't have varnish in the middle. The nginx and HAProxy "servers" are on the same box. nginx does SSL offload for us, and then HAProxy sends the requests to one of several web servers. We wanted the enhanced load balancing from HAProxy so we have both. The entire process works great.

The only gotcha we found was the client ip address. We couldn't find a way to tell IIS to log the x-forwarded-for IP as the clientip in its logs, but we did find that ASP.Net can ask IIS to append some additional information to the end of the request string in the log, so we stick the client IP there. Not ideal, but it works. nginx is set to add a x-forwarded-for header, HAProxy is not set to add forwarded for information (since it would simply point to nginx). Also, nginx seems to only add the header for the first request in a keep-alive session -- again, not ideal, but it's good enough for us.

Excerpt of our nginx config (HAProxy is listening on 127.0.0.1:5000):

location / {
    proxy_pass      http://127.0.0.1:5000;
    proxy_set_header        Host    $host;
    proxy_set_header        X-Forwarded-For $remote_addr;
    proxy_buffering off;
    client_body_buffer_size 64k;
}

ASP.Net code (in global.asax) to add the ip to the request header:

protected void Application_BeginRequest(Object sender, EventArgs e)
{
    //If there's an X-ForwardedFor header (Proxy) append it to the query string for logging purposes.
    if (HttpContext.Current.Request.Headers["X-Forwarded-For"] != null)
        HttpContext.Current.Response.AppendToLog((HttpContext.Current.Request.QueryString.Count == 0 ? "" : "&") + "X-Forwarded-For=" + HttpContext.Current.Request.Headers["X-Forwarded-For"]);

Haproxy 1.5 specific source IP address show 503 SC in haproxy log

See the SC-- in the log entry?

This is a problem with your back-end server, or a network issue between HAProxy and the back-end... not on the front side, and not related to the IP address of the connecting client.

From "Session state at disconnect" in the docs:

SC The [back-end] server or an equipment between it and haproxy explicitly refused the TCP connection (the proxy received a TCP RST or an ICMP message in return). Under some circumstances, it can also be the network stack telling the proxy that the server is unreachable (eg: no route, or no ARP response on local network). When this happens in HTTP mode, the status code is likely a 502 or 503 here.

http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#8.5

Best Answer

Related Solutions

Nginx/haproxy/IIS – Configuration questions and concerns

Haproxy 1.5 specific source IP address show 503 SC in haproxy log

Related Topic