Resolvconf coupled with dnsmasq and lxd

ansiblecontainersdnsmasqresolv.conf

Q: How do I programmatically add a persistent resolvconf entry for my lxd based dnsmasq?

I have a very base installation of Ubuntu 16.04 server.

I have installed and configured lxd and created a container.

lxd lxd/bridge-domain   string  lxd
# lxcbr0 is being replaced by lxdbr0
lxd lxd/bridge-upgrade-warning  note    
lxd lxd/bridge-ipv6-address string  fd6d:c772:b128:ebfd::1
lxd lxd/bridge-ipv6-netmask string  64
# Choices: gnome, kde, xfce, lxde, cinnamon, mate
lxd lxd/bridge-empty-error  note    
lxd lxd/bridge-ipv6-nat boolean true
lxd lxd/bridge-name string  lxdbr0
lxd lxd/bridge-ipv4-netmask string  24
lxd lxd/bridge-ipv4 boolean true
lxd lxd/bridge-dnsmasq  string  
lxd lxd/bridge-ipv6 boolean true
lxd lxd/use-existing-bridge boolean false
lxd lxd/bridge-ipv4-dhcp-first  string  10.41.63.2
lxd lxd/bridge-ipv4-dhcp-leases string  252
lxd lxd/bridge-random-warning   note    
lxd lxd/bridge-ipv4-dhcp-last   string  10.41.63.254
lxd lxd/bridge-http-proxy   boolean false
lxd lxd/setup-bridge    boolean true
lxd lxd/bridge-ipv4-nat boolean true
lxd lxd/update-profile  boolean true
lxd lxd/bridge-ipv4-address string  10.41.63.1

Running container:

+-------+---------+--------------------+-----------------------------------------------+------------+-----------+
| NAME  |  STATE  |        IPV4        |                     IPV6                      |    TYPE    | SNAPSHOTS |
+-------+---------+--------------------+-----------------------------------------------+------------+-----------+
| first | RUNNING | 10.41.63.27 (eth0) | fd6d:c772:b128:ebfd:216:3eff:fea7:69c6 (eth0) | PERSISTENT | 0         |
+-------+---------+--------------------+-----------------------------------------------+------------+-----------+

LXD processes:

root       886     1  0 11:28 ?        00:00:00 /usr/bin/lxcfs /var/lib/lxcfs/
lxd       4129     1  0 13:04 ?        00:00:00 dnsmasq -s lxd -S /lxd/ -u lxd --strict-order --bind-interfaces --pid-file=/run/lxd-bridge//dnsmasq.pid --dhcp-no-override --except-interface=lo --interface=lxdbr0 --dhcp-leasefile=/var/lib/lxd-bridge//dnsmasq.lxdbr0.leases --dhcp-authoritative --listen-address 10.41.63.1 --dhcp-range 10.41.63.2,10.41.63.254 --dhcp-lease-max=252 --dhcp-range=fd6d:c772:b128:ebfd::1,ra-only --listen-address fd6d:c772:b128:ebfd::1
root      4146     1  0 13:04 ?        00:00:18 /usr/bin/lxd --group lxd --logfile=/var/log/lxd/lxd.log
root      4209     1  0 13:05 ?        00:00:00 [lxc monitor] /var/lib/lxd/containers first

A lookup of the container name returns nothing, however:

# dig first @10.41.63.1
...
first.          0   IN  A   10.41.63.27
...

Equally, due to lxd config:

# dig first.lxd @10.41.63.1
...
first.lxd.          0   IN  A   10.41.63.27
...

This is because of:

# lsof -i :53
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
dnsmasq 4129  lxd    7u  IPv4  27561      0t0  UDP 10.41.63.1:domain 
dnsmasq 4129  lxd    8u  IPv4  27562      0t0  TCP 10.41.63.1:domain (LISTEN)
...

ifconfig reports:

eno1      Link encap:Ethernet  HWaddr b8:ae:ed:ea:69:41  
          inet addr:192.168.0.3  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::baae:edff:feea:6941/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:116948 errors:0 dropped:3 overruns:0 frame:0
          TX packets:59726 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:161031770 (161.0 MB)  TX bytes:4656462 (4.6 MB)
          Interrupt:16 Memory:df100000-df120000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:177 errors:0 dropped:0 overruns:0 frame:0
          TX packets:177 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:13388 (13.3 KB)  TX bytes:13388 (13.3 KB)

lxdbr0    Link encap:Ethernet  HWaddr fe:5c:57:cd:bd:72  
          inet addr:10.41.63.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fd6d:c772:b128:ebfd::1/64 Scope:Global
          inet6 addr: fe80::d08c:f9ff:fed9:5cd1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:39 errors:0 dropped:0 overruns:0 frame:0
          TX packets:78 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:5064 (5.0 KB)  TX bytes:11128 (11.1 KB)

vethD7L5U9 Link encap:Ethernet  HWaddr fe:5c:57:cd:bd:72  
          inet6 addr: fe80::fc5c:57ff:fecd:bd72/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:39 errors:0 dropped:0 overruns:0 frame:0
          TX packets:75 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:5610 (5.6 KB)  TX bytes:10322 (10.3 KB)

The bridge came up when I started my first container.

resolve.conf contains:

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 192.168.0.1

… because there is no NetworkManager and no dnsmasq package installed (which is fine)

Finally, the network interfaces file contains:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eno1
iface eno1 inet static
    address 192.168.0.3
    netmask 255.255.255.0
    network 192.168.0.0
    broadcast 192.168.0.255
    gateway 192.168.0.1
    # dns-* options are implemented by the resolvconf package, if installed
    dns-nameservers 192.168.0.1

As an important aside, I am using ansible to mange all of this.

If course, if I prepend the appropriate nameserver by hand, it will work.

nameserver 10.41.63.1
nameserver 192.168.0.1

If it's the other way around, it won't, as expected.

Equally, it doesn't survive resolvconf -u or a reboot.

Uninstalling resolvconf feels like a failure.

Creating /etc/resolvconf/resolv.conf.d/tail will obviously not work

base and head seem like a kludge

Diving into the middle of init.d feeling like getting too down and dirty.

What I'm looking for is a method that's a hook into the up or down of the bridge where I can at least try a resolvconf -a, even though I'm not sure it will get it in the right order.

That's the problem.

UPDATE: In fact, all I need is:

echo "nameserver 10.41.63.1" | resolvconf -a lo.dnsmasq

resolv.conf now becomes:

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.41.63.1
nameserver 192.168.0.1

Which is what I need.

So where does that hook go for the instantiation of the bridge?

Best Answer

You can configure DNS nameservers on the loopback* interface. These will always be listed. The following should keep the listed nameserver(s) in place.

# The loopback network interface
auto lo
iface lo inet loopback
    dns-nameservers 192.168.0.1
Related Topic