If the status check is returning a 401, AWS sees the instance as unhealthy and takes it out of the ELB. Since you have no healthy instances, the ELB is down.
Note that AWS accesses /heartbeat.php
via IP, not via DNS name. Check the response using curl -I http://<your instance IP>/heartbeat.php
and adjust your server config if it's not being served by the default virtualhost.
f you know any thing about this (maybe you are a network engineer) you probably see right away what the problem is.
The problem is rooted in these concepts:
- Ephemeral ports
- Stateless and Stateful
- Connection Tracking
- NACLs are Stateless
- Security Groups are Stateful
Read about it here:
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Security.html
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_SecurityGroups.html
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_ACLs.html
https://en.wikipedia.org/wiki/Ephemeral_port
The corrected confiuration is below. The change that really corrected the connectivity between the Public and Private subnets was enabling response traffic on the Ephemeral ports for both In Bound Public subnet NACL and Out Bound Private subnet NACL rules.
I also cleaned up some redundant and insecure Out Bound rules in the NACLS and Security Groups.
VPC NACL
In Bound
80 0.0.0.0/0 Allow
Out Bound
32768-65535 0.0.0.0/0 Allow
Public Subnet NACL
InBound
80 0.0.0.0/0 Allow
32768-65535 172.30.4.0/0 Allow
Out Bound
32768-65535 0.0.0.0/0 Allow
5432 172.30.4.0/24 Allow (PostgreSQL)
Private Subnet NACL
InBound
5432 172.30.1.0/24 Allow
Out Bound
32768-65535 172.30.1.0/24 Allow
VPC Security Group
InBound
80 VPC-Security-Group-ID Allow
Out Bound
Public Subnet Security Group
InBound
80 0.0.0.0/0 Allow
Out Bound
5432 172.30.4.0/0 Allow
Private Subnet Security Group
InBound
5432 172.30.1.0/24 Allow
Out Bound
This is a working configuration. Hopefully this helps someone. If you see anyhting that can be improved let me know and let me know if you have questions.
Best Answer
Some thoughts: