Linux – How to make ssh/rsync/etc use a VLAN network interface

You can't easily do what you want.

Or how do I configure a different DNS Name Server for eth0 vs eth1?

The name lookup for a hostname happens through standard system libraries and isn't associated in any way with a particular "connection". In fact, at the time the DNS query happens, there is no connection, because your application hasn't even figured out the address to which it's going to connect (which is why it's using DNS in the first place).

How do I get it to respect the DNS settings in ifcfg rather than a default for resolv.conf?

The Linux resolver only has a single, global configuration (/etc/resolv.conf). There is no per-interface, per-domain, or per-connection setting of any sort. The settings in /etc/sysconfig/network-scripts/... are only used to populate /etc/resolv.conf, and generally if you specify DNS1 and DNS2 in these files, the last interface to come up will be what you see in /etc/resolv.conf.

Is there a better way of handling this?

Can you tell us what you're actually trying to accomplish? We might be able to suggest better solutions if you can tell us more about your specific situation.

Absolutely you can. You're looking for 802.1q tagging, which is the underpinning of multiple VLANs on a switch.

Configure your server-facing ports like this...

interface GigabitEthernet0/1
 ! tell the switch to encapsulate frames with dot1q on this trunk port
 switchport trunk encapsulation dot1q
 ! make this a trunk port unconditionally
 switchport mode trunk
 ! allow VLANs 220, 221, and 223 to be sent on this port
 switchport trunk allowed vlan 220,221,223
 ! send vlan 221 frames untagged
 ! you could omit this command, but you'll need to move your eth0 config to a subinterface as well
 switchport trunk native vlan 221
 ! this is an end-device, and doesn't need to be treated like a switch for spanning-tree
 spanning-tree portfast trunk
 ! remove the (ignored) access vlan setting, as this is now a trunk
 no switchport access vlan

On your Ubuntu server you'll need to configure the subinterfaces to be associated to each appropriate VLAN. This can be done manually like this ...

# create a new subinterface tied to dot1q vlan 221
ip link add link eth0 name eth0.221 type vlan id 221
# change the MTU to allow for the inserted dot1q tag (4 bytes)
ifconfig eth0 mtu 1504
# bring the new interface up
ifconfig eth0.221 up

Now you can associate the correct IP address to new interface eth0.221, along with the appropriate default gateway and subnet mask.

If this doesn't work for you, please include a pastebin link to your switch config as well as the output of "ip -d link show" on your Ubuntu server.

Hope this helps!