DNS issues with different internal domain and external domain

I'm going to go ahead and make a couple of assumptions in my answer. Correct me if I'm wrong.

1. You only have 1 public IP and it's the one assigned to the WAN interface of your router.

2. Your router is a SOHO model that can only be assigned a single IP on its WAN interface. i.e. if you purchased additional static IPs from your ISP, you'd need to replace your router to handle this.

I have a local DNS server inside my network and have set up a few A Name records to some of my internal sites (i.e. app1.domain.com, app2.domain.com) and the dns resolves internally and loads the sites just fine.

I'm assuming that your internal DNS servers are configured to return the RFC1918 private addresses of the servers. This makes the DNS zones on this server useless to people outside of your LAN. If they get a response of 10.1.1.x from across the Internet, it's discarded.

Now I have a godaddy domain as well and want to allow external access for a few of these sites. I made the same A Name records in godaddy and pointed them to my single WAN IP. Before I was port forwarding all incoming port 80 traffic on the router directly to the IP of my internal web server just as an ad-hoc solution and it worked fine.

Ok. This will work, but it's not very scalable, as you've seen.

Now I have a second web server that I need to expose on a different internal server. So for the port 80 traffic I need to send app1 to my internal ip 10.1.1.23 and app2 to 10.1.1.24 when clients try to connect from the outside.

So I modified my router to forward external port 80 traffic to 10.1.1.11 (my internal DNS server) on port 53 for both tcp/udp. The DNS server is a Windows Server 2008 R2 machine. It looks like the firewall exceptions are also setup to allow incoming traffic on port 53, but anytime I try to access the site from an external client, nothing loads. I know the godaddy is forwarding the traffic to my WAN because I can do a tracert that resolves to my WAN IP.

This is where you get weird, my friend. Really, really weird. You're forwarding an HTTP request to a DNS server and translating the port as well. That's like taking a letter written in English addressed to someone in the USA, grabbing it out of the Post Office, and dropping it in the mailbox of someone in Myanmar and wondering why you haven't received a reply yet.

HTTP and DNS are totally different protocols. HTTP is what serves up your web pages. DNS translates a fully-qualified name to an IP address. They aren't interchangeable. The DNS server is going to take one look at the HTTP packets that you're forwarding to it and say "screw this stuff, no idea what that says" and will then just throw it in the trash, much like your new pen pal in Myanmar.

OK, so now that you know where you've gone wrong, what can you do about it?

1. Get an additional public IP and assign it to your router. This assumes that your router can handle this. If it can't, you might have to get a new one, or use option #2. If you get another IP and a router that can handle it, you just set up Go Daddy to use the second IP for your second server and then forward port 80 requests on the new IP to the new server, much like you did with your first IP and your first server. This will require you to have one public IP for each new server you add.

2. Set up a reverse proxy and point everything to your one IP address. You can use things like apache's mod_proxy, or IIS7's ARR for this. You'd tell Go Daddy's DNS to send all traffic for all of your external sites to the one IP that you already have. On your router, you'd forward requests on port 80 to a new server that's acting as a reverse proxy. This server would be configured to act sort of like a "broker" for inbound connections. You'd configure that server to forward all requests to it to the right server based on the hostname being requested.

I figured out a workaround but it's far from perfect. Ideally I'd have liked to add "*.mydomain.com" as a catch all forwarding to external DNS (my registrar) and only point the relevant ones to internal servers. Unfortunately OS X server doesn't currently allow wildcard entries so I've had to add everything manually pointing out/in where appropriate.