How to make lxd containers accessible to the LAN

iplocal-area-networklxc

If I use Virtualbox to spin up a VM, I can select "bridged" as the network adapter type and this results in the guest/virtual NIC being connected to my physical LAN and hence getting a LAN IP from my router (via DHCP).

I want to achieve this same functionality, but instead of using Virtualbox, I want to use lxc/lxd containers.

How can I achieve this?

Edit 1

I'm running Ubuntu and I tried following this guide:

https://insights.ubuntu.com/2015/11/10/converting-eth0-to-br0-and-getting-all-your-lxc-or-lxd-onto-your-lan/

…but it doesn't help. Should the original host interface have an IP afterwards? Because it doesn't when I try the manual bridging.

Edit 2

If it helps, my lxd/lxc host is a Virtualbox virtual machine running Ubuntu, set up with bridged networking to my physical Ethernet NIC.

Edit 3

If I use tcpdump to monitor icmp traffic on the bridge interface, the physical/host interface and the container/virtual interface, only the container/virtual does not get any traffic. The other two do.

Edit 4

According to this guide:

http://www.microhowto.info/troubleshooting/troubleshooting_ethernet_bridging_on_linux.html

I have no issues with my bridge setup.

Yet as mentioned in "Edit 3", the container interface is not getting any traffic. Need to work out why, but I'm not sure how to…

I have a feeling it has something to do with routes.

The container has no routes, whereas the host does.

Edit 5

Using tcpdump to monitor arp traffic, shows that arp traffic is actually getting to the container/virtual interface.

So it's just icmp traffic that isn't.

Edit 6

If I set a static IP in the container (via /etc/network/intefaces*), that allows me to ping the container from the host (which is a Virtualbox machine).

If I then change the network configuration in Virtualbox to allow promiscuous traffic, I can then ping the container from my physical machine (the host of the Virtualbox machine).
Yet from here, I am still unable to ping beyond my physical LAN, from within the container.

The last step, if I manually add a "default" route in the container like so:

route add default gw 192.168.0.1 eth0

that allows me to ping outside of the physical LAN from inside the container.

So unless someone else can offer an explanation (I'll wait before posting an answer), I'm guessing the lack of container DHCP support (via bridging) has something to do with the fact that lxc/lxd is using netmasq to handle DHCP (and DNS).

Best Answer

If your LXD host is actually a virtual machine, ensure that the virtual machine's network adaptor is configured to promiscuous mode, so that way LXD container traffic passes from the physical to the virtual network.
Set a static IP address in the lxd container(s), because DHCP (from your physical gateway) doesn't seem to work.

In my 6th edit I said that I needed to manually add a default route in the container, but that's not true. I only needed to do that because I forgot to specify the gateway LAN IP address in the /etc/network/interfaces file. So it's not an LXD issue, just don't forget to specify it.

Related Solutions

Ubuntu – Make LXC containers directly accessable with ipv6

It seems to me that you are conflating a number of different things here. First, I doubt that the net mask on your server's ethernet port is actually /128. I suspect it's something else (/64 perhaps) and that you're on a shared segment with a bunch of other customers.

Judging by the output of your "ip -6 a" command:

4: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 2607:5300:60:714::1/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::ea40:f2ff:feed:106f/64 scope link 
       valid_lft forever preferred_lft forever
8: lxcbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 
    inet6 2607:5300:60:714::2/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::b07b:e3ff:fe33:22e7/64 scope link 
       valid_lft forever preferred_lft forever
18: vethPVJQ6M: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 fe80::fcb7:57ff:fe3c:bcd1/64 scope link 
       valid_lft forever preferred_lft forever

I would say that the /128 on the interfaces is an error. Your prefix appears to be 2607:5300:60:714::/64 (most likely).

Assuming that's correct, then you'll need to set up your interfaces file as follows (add your IPv4 as needed):

auto lxcbr0
iface lxcbr0 inet6 static
  bridge_ports eth0
  bridge_fd 0
  address 2607:5300:60:714::1
  net mask 64
  gateway 2607:5300:60:7ff:ff:ff:ff:ff

Note: It's not clear how you reach 2607:5300:60:7ff::/64 to get to your default gateway. It would be very useful to know how your provider expects you to configure your network or to have a first-hand look at any documentation they provided. Best guess from here is that the 2607:5300:60:714::/64 network is on the same link as 2607:5300:60:7ff::/64. That 2607:5300:60:7ff::/64 is used for the provider's infrastructure. It's unclear whether you get the entire 2607:5300:60:714:/64 or whether that's shared with other customers on the same link.

Assuming you have the freedom to assign addresses from within that range, then all you really need to do is connect your containers to the same lxcbr0 interface and assign each container's address to that bridge interface.

Again, this is just a best guess based on the data you provided. Without knowing your provider's actual configuration, it's impossible to tell for sure.

Ubuntu – Copy lxd containers between hosts

The later release (non-beta) of lxd (v2.0) seems to have resolved my issue. The steps, which may be found in the excellent documentation here, are:

Publish an image (without stopping the container) on host A;

$ lxc publish --force container_name --alias image_name
Container published with fingerprint: d2fd708361...a125d0d5885

Export the image to a file;

$ lxc image export image_name 
Output is in dd2fd708361...a125d0d5885.tar.gz

Copy the file to host B, and import;

$ lxc image import dd2fd708361...a125d0d5885.tar.gz --alias image_name
Transferring image: 100%

Launch the container (from the image) on host B;

$ lxc launch image_name container_name
Creating container_name
Starting container_name

In some instances the publish command may lead to a split xz tar-ball --- but both formats are supported. Simply import the meta-data and rootfs components with

    lxc image import <metadata tarball> <rootfs tarball> --alias image_name

Best Answer

Related Solutions

Ubuntu – Make LXC containers directly accessable with ipv6

Ubuntu – Copy lxd containers between hosts

Related Topic