Load Balancing DNS Servers: UDP / TCP

Assuming that all your back-end machines are "active" and able to respond to requests all you really need is a load balancer front-end.

A good load balancer will be able to keep track of the number of connections going to each host and dynamically distribute new connections to avoid swamping one of the back-end systems (by "good" I mean "expensive", like Cisco Content Switches/Content Switch Service Modules). Price goes hand in hand with features here: Content switches are pretty high up on the solutions tier.

I've got no experience with HAProxy, but it sounds like it can do least-connection load balancing like content switches, so this would probably be a good choice (and at a much more attractive price point). I'm not sure if HAProxy can do source-tracking (send all connections from the same IP to the same back-end) though.

A few steps down down the pf firewall (or the pfsense customized distribution) can do load balancing (random or round-robin, I don't believe they can do "weighted least connections" as a balancing option like the content switches). Source tracking is implemented in pf, though you may have to play with how long that information is retained to avoid problems with connections getting moved from one server to another.
If you're already using pf/pfsense as your firewall this is a no-cost option: We use this in my current deployment with good results, but our connections aren't as long-lived as yours.

The canonical solution to this is to not rely on end user IP address, but instead use a Layer 7 (HTTP/HTTPS) load balancer with "Sticky Sessions" via a cookie.

Sticky sessions means the load balancer will always direct a given client to the same backend server. Via cookie means the load balancer (which is itself a fully capable HTTP device) inserts a cookie (which the load balancer creates and manages automagically) to remember which backend server a given HTTP connection should use.

The main downside to sticky sessions is that beckend server load can become somewhat un-even. The load balancer can only distribute load fairly when new connections are made, but given that existing connections may be long-lived in your scenario, then in some time periods load will not be distributed entirely fairly.

Just about every Layer 7 load balancer should be able to do this. On Unix/Linux, some common examples are nginx, HAProxy, Apsis Pound, Apache 2.2 with mod_proxy, and many more. On Windows 2008+ there is Microsoft Application Request Routing. As appliances, Coyote Point, loadbalancer.org, Kemp and Barracuda are common in the low-end space; and F5, Citrix NetScaler and others in high-end.

Willy Tarreau, the author of HAProxy, has a nice overview of load balancing techniques here.

About the DNS Round Robin:

Our intent was for the Round Robin DNS TTL value for our api.company.com (which we've set at 1 hour) to be honored by the downstream caching nameservers, OS caching layers, and client application layers.

It will not be. And DNS Round Robin isn't a good fit for load balancing. And if nothing else convinces you, keep in mind that modern clients may prefer one host over all others due to longest prefix match pinning, so if the mobile client changes IP address, it may choose to switch to another RR host.

Basically, it's okay to use DNS round robin as a coarse-grained load distribution, by pointing 2 or more RR records to highly available IP addresses, handled by real load balancers in active/passive or active/active HA. And if that's what you're doing, then you might as well serve those DNS RR records with long Time To Live values, since the associated IP addresses are highly available already.