LXC – No Network Connectivity in Routed Mode

linux-networkinglxclxdnetworking

I'm experimenting with lxc/lxd in Vagrant, but i'm quite new to it. I managed to create running container, but I cannot ping anything (including 8.8.8.8) from inside of it. I can ping its IP from my top-level non-virtual system, but it refuses SSH connections. I can enter the container only directly from the direct container's host (Vagrant) by using lxc exec my-container /bin/bash.

I tried to setup my container in the routed mode, and I still want it, for the learning purposes. The LXD/LXC documentation seems to be somewhat lacking though.

I tried to follow this instruction: https://blog.simos.info/how-to-get-lxd-containers-get-ip-from-the-lan-with-routed-network/ but it didn't work for me in the end. I could miss something, because I'm not well versed in the linux networking yet.

My Vagrant host is running on Ubuntu 20.04.

My LXC container is running on Debian 10.

LXC configuration on my Vagrant host:

config:
  core.https_address: '[::]:8443'
  core.trust_password: true
networks: []
storage_pools:
- config:
    source: /home/luken/lxd-storage-pools
  description: ""
  name: default
  driver: dir
profiles:
- name: default
  config: {}
  description: ""
  devices:
    root:
      path: /
      pool: default
      type: disk
- name: mail-server
  config:
    user.network-config: |
      version: 2
      ethernets:
        eth0:
          addresses:
          - 192.168.33.11/32
          nameservers:
            addresses:
            - 8.8.8.8
            search: []
          routes:
          -   to: 0.0.0.0/0
            via: 169.254.0.1
  description: Mail Server LXD profile
  devices:
    eth0:
      ipv4.address: 192.168.33.11
      nictype: routed
      parent: eth1
      type: nic
cluster: null

ip addr in my Vagrant host:

luken@luken-tech-test:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:be:4a:e8 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
       valid_lft 76347sec preferred_lft 76347sec
    inet6 fe80::a00:27ff:febe:4ae8/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:65:e6:28 brd ff:ff:ff:ff:ff:ff
    inet 192.168.33.2/24 brd 192.168.33.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe65:e628/64 scope link 
       valid_lft forever preferred_lft forever
6: vetha8400046@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fe:48:28:3e:e4:fa brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 169.254.0.1/32 scope global vetha8400046
       valid_lft forever preferred_lft forever
    inet6 fe80::fc48:28ff:fe3e:e4fa/64 scope link 
       valid_lft forever preferred_lft forever

ip addr in my container:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 9a:14:96:30:67:43 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.33.11/32 brd 255.255.255.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::9814:96ff:fe30:6743/64 scope link 
       valid_lft forever preferred_lft forever

ip r in my Vagrant host:

default via 10.0.2.2 dev eth0 proto dhcp src 10.0.2.15 metric 100 
10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15 
10.0.2.2 dev eth0 proto dhcp scope link src 10.0.2.15 metric 100 
192.168.33.0/24 dev eth1 proto kernel scope link src 192.168.33.2 
192.168.33.11 dev vetha8400046 scope link

ip r in my container:

default via 169.254.0.1 dev eth0 
169.254.0.1 dev eth0 scope link

Is there anything I missed (probably a lot)?

Best Answer

There are several things are required to make your scheme work:

  1. Check the forwarding on the host interfaces: ip -4 netconf show dev vetha8400046 and ip -4 netconf show dev eth0. You should see the enabled forwarding in the output. Otherwise enable it with the sysctl -w net.ipv4.conf.<iface>.forwarding=1 command. Add the corresponded strings into the /etc/sysctl.conf file to make it persistent.

  2. Check the routing on the host: ip -4 route get 8.8.8.8 from 192.168.33.11 iif vetha8400046. You should see the valid route through your default gateway (... via 10.0.2.2 dev .... Check also the reverse route with the ip -4 route get 192.168.33.11 from 8.8.8.8 iif eth0.

  3. Ping the container from host and vice versa.

  4. Setup the NAT on the host: iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE. Ping some external host from the container. The counters of this rule in the iptables-save -c should be non-zero.

  5. Ask the question if you stuck.

Related Topic