Ssl – HTTPS streaming and dynamic load balancer using Wowza and AWS

amazon-web-serviceshttp-streamingload balancingsslvideo streaming

We are using Wowza instances to perform live streaming on AWS infrastructure. We have a script that monitors load balancer and launches/kills instances when required based on the number of connections/viewers per edge server. It woks well for HTTP/RTMP/RTMPT streaming.
We are now trying to implement dynamic load balancing for HTTPS streaming and automate the scaling process. The main question for us is how to automate it in a best possible way?
We don't know the IPs of the edge servers that will be launched, therefore, can't create the DNS records in advance.

One possible solution would be to reserve some number of elastic IPs (i.e. 20) and the same number of DNS names and use them during scaling. But this will set a limit on the number of servers we can launch dynamically.

Is there a more elegant solution that we can try to implement?

Our LB infrastructure is inside AWS VPC and we have a star certificate that we can use.
Also, Wowza Load Balancer only used to tell the viewer the IP of the available edge server. After that, the viewer connects to the edge server directly.

Best Answer

You could configure the instances to discover their identity on launch, such as by passing the instance's desired Internet hostname in an instance tag or through instance metadata, and have each node send an API request to Route 53 to configure its own A-record, essentially claiming its Internet identity, when it boots. It could send a second request to remove that record on graceful shutdown.

This is a mechanism I use with spot instances.

Since you mentioned that your domain isn't in Route 53, you would need a workaround for that, unless you could move the entire DNS hosting to Route 53, which there is no reason not to do other than political/corporate objections. You can retain your domain registration with the current registrar, and simply use Route 53 for hosting the DNS, of course.

Here are some possible workarounds:

You could get a new domain name and put it in Route 53. The downside here is that you'd need a new certificate.

You could create a subdomain of your domain, delegating the subdomain to Route 53 in your base diamond's DNS servers with NS records, but you'd still need a new certificate, because the *.example.com cert is only valid for one level of hostnames: foo.example.com works as expected, but foo.bar.example.com does not. This would require a certificate for *.bar.example.com.

So, both of those alternatives need a new certificate.

Another option would be to use Let's Encrypt and have each server go out and get a free SSL cert for itself, programmatically, on startup. This would allow you to use whatever domain or subdomain, without additional certificate expense.

Finally, you could use a bit of DNS hackery. This will seem counter-intuitive to some, but since Route 53 is an authoritative-only (not recursive) DNS service, it works as expected.

Create a hosted zone for your existing domain in Route 53. This is valid, even if your DNS isn't hosted with Route 53.

The hosted zone will have 4 name servers assigned, in generally this format:

ns-aaaa.awsdns-bb.com.
ns-cccc.awsdns-dd.net.
ns-eeee.awsdns-ff.org.
nd-gggg.awsdns-hh.co.uk.

The a, b, c, etc., are numbers that are assigned by the system.

Take these records, and use them to delegate specific hostnames from your existing DNS server to Route 53.

media-100 IN NS ns-aaaa.awsdns-bb.com.
media-100 IN NS ns-cccc.awsdns-dd.net.

Repeat the process for each of your "pool" hostnames, for each of the four AWS DNS servers assigned. This is still somewhat limiting in the sense that it restricts your pool to a finite size, but unlike the reserved elastic IP solution, this approach has no hard maintenance cost (Elastic IP addresses are billed $0.005 for each address for each hour that the address is not being used as the primary Elastic IP of a running instance).

Of course, your main DNS service might support wildcards, in which case all you would need is a single set of records:

media-* IN NS ns-aaaa.awsdns-bb.com ; etc., one for each of the 4 hostnames.

These NS records work by delegating the resolution of each of these hostnames to Route 53. Since nothing on the Internet is configured to send requests to those specific 4 Route 53 name servers for any other hostnames in your domain, the fact that Route 53 doesn't know about your entire zone does not break anything.

A potentially common misconception is that a domain can only be provisioned once in Route 53... but in fact, you can have multiple hosted zones for exactly the same domain name, in the same or even in different AWS accounts, because each of those hosted zones will be answered by the specific 4 name servers that Route 53 to that zone... So even if you later moved your entire DNS to Route 53, if you copied the other zone from the other DNS provider, as is, into a new hosted zone in Route 53 and made that one authoritative, this solution would not present a conflict.

Related Topic