Openvpn: how to get clients to use network names available on server

Yes, you can. By default, OpenVPN server doesn't allow multiple clients to connect with the same pair of cert/key; each client must have its own cert/key with an unique common name. However, multiple clients are allowed to connect to the OpenVPN server using the same pair of cert/key if duplicate-cn (allowing duplicate common name) is present in OpenVPN server's configuration file.

I think your setup is going to break or not work:

Your OpenVPN client configuration uses server.foo.com's public address to connect to the OpenVPN server. This address will be looked up prior to establishing the VPN tunnel, obviously.

You're trying to push a DNS entry for server.foo.com with a OpenVPN IP after the tunnel is established with the dnsmasq config. Either the OpenVPN client ignores it (since it already knows about server.foo.com as it had to look it up to establish the tunnel) or it will respect it, and then drop the tunnel because the OpenVPN client configuration will point to a now nonexistent IP address. The latter may likely happen during your OpenVPN session, depending on the TTL of the DNS server for server.foo.com's public IP.

TL;DR: you are basically trying to tell your client conflicting information about server.foo.com. I can't think of a good way to do what you have in mind. An alternative might be to set up a second DNS entry A record vpnserver.foo.com which points to the same IP as server.foo.com, and then change your OpenVPN config to use that.