Getting ipv6 via radvd/dhcpd6 in an LXC guest working

ipv6lxcnixos

what i want

my setup has a dynamic amount of LXC containers and therefore i need some dynamic ipv6 address allocation. the interface brNC-internet is a simple bridge which is mapped into the LXC based container.

i need a way to assign ipv6 addresses in LXC containers to the internet interface of each. manually doing so works, as shown below but this should be automated using dhcpd6 (or something similar).

i'd love to try:

stateful dhcp: where a client is reassigned the same address when rebooted within some time limit (days, weeks)
stateless dhcp: a client gets just any address for the time it is up

note: i can't use radvd with SLAAC because my network prefix is /66 and radvd requires at least a /64 for this, see this.

note: i'd like to focus on ipv6 for the moment.

note: i'm using nixos linux and i did probably just configure something wrongly, have a firewall rule which breaks things or a general misunderstanding about ipv6 or even some LXC specific internal. in either case, please point out what i could be trying next.

my problem

i've setup radvd and dhcpd6 on the host (a VM) but while radvd is able to push a ipv6 default gateway and a prefix the client never seems to talk to the dhcpd6 server using dhcpcd from the client.

if i disable the dhcpcd client on the guest LXC instance i can assign a ipv6 address within the prefix and ping6 to google works:

[root@10:/]# ip a replace 2a01:4f8:221:3744:4000::4 dev internet

[root@10:/]# ping -6 -I internet 2a00:1450:4001:80b::2003
PING 2a00:1450:4001:80b::2003(2a00:1450:4001:80b::2003) from fe80::10af:ffff:fef4:318a internet: 56 data bytes
From fe80::2044:c6ff:fef3:cd5d%internet icmp_seq=1 Destination unreachable: Beyond scope of source address
64 bytes from 2a00:1450:4001:80b::2003: icmp_seq=2 ttl=55 time=5.36 ms
64 bytes from 2a00:1450:4001:80b::2003: icmp_seq=3 ttl=55 time=5.26 ms
64 bytes from 2a00:1450:4001:80b::2003: icmp_seq=4 ttl=55 time=5.27 ms

i've also tried to disable the firewalls on both host and LXC client but no change.

dhcpcd from lxc

dhcpcd -6  --config /nix/store/7n7ysqf92rlafihs9dm2gzsbh06cw64z-dhcpcd.conf
DUID 00:01:00:01:22:4f:af:36:ee:ae:40:b5:d7:d3
internet: IAID 98:0e:c7:c8
internet: soliciting an IPv6 router
internet: Router Advertisement from fe80::2044:c6ff:fef3:cd5d
forked to background, child pid 1238

tcpdump on the VM

tcpdump -i brNC-internet ip6



14:36:18.618554 IP6 fe80::dc5d:98ff:fe0e:c7c8 > ff02::2: ICMP6, router solicitation, length 16
14:36:18.618681 IP6 status.nixcloud.io > fe80::dc5d:98ff:fe0e:c7c8: ICMP6, router advertisement, length 112
14:36:20.578582 IP6 status.nixcloud.io > ff02::1: ICMP6, router advertisement, length 112
14:36:24.059514 IP6 status.nixcloud.io > fe80::dc5d:98ff:fe0e:c7c8: ICMP6, neighbor solicitation, who has fe80::dc5d:98ff:fe0e:c7c8, length 32
14:36:24.059598 IP6 fe80::dc5d:98ff:fe0e:c7c8 > status.nixcloud.io: ICMP6, neighbor advertisement, tgt is fe80::dc5d:98ff:fe0e:c7c8, length 24

dhcpd6 log on the vm

journalctl -u dhcpd6 -f
-- Logs begin at Tue 2018-02-13 02:31:51 CET. --
Mar 30 14:00:08 status.nixcloud.io dhcpd[17605]: Wrote 0 NA, 0 TA, 0 PD leases to lease file.
Mar 30 14:00:08 status.nixcloud.io dhcpd6[17605]: Bound to *:547
Mar 30 14:00:08 status.nixcloud.io dhcpd[17605]: Bound to *:547
Mar 30 14:00:08 status.nixcloud.io dhcpd[17605]: Listening on Socket/5/brNC-internet/2a01:4f8:221:3744:4000::/66
Mar 30 14:00:08 status.nixcloud.io dhcpd[17605]: Sending on   Socket/5/brNC-internet/2a01:4f8:221:3744:4000::/66
Mar 30 14:00:08 status.nixcloud.io dhcpd6[17605]: Listening on Socket/5/brNC-internet/2a01:4f8:221:3744:4000::/66
Mar 30 14:00:08 status.nixcloud.io dhcpd6[17605]: Sending on   Socket/5/brNC-internet/2a01:4f8:221:3744:4000::/66
Mar 30 14:00:08 status.nixcloud.io systemd[1]: dhcpd6.service: Can't open PID file /run/dhcpd6/dhcpd.pid (yet?) after start: No such file or directory
Mar 30 14:00:08 status.nixcloud.io dhcpd6[17615]: Server starting service.
Mar 30 14:00:08 status.nixcloud.io systemd[1]: Started DHCPv6 server.

my setup

host vm

ip -6 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2a01:4f8:221:3744::1:26/128 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fefb:d8d0/64 scope link
       valid_lft forever preferred_lft forever
3: enp0s2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::5054:ff:fe08:4db9/64 scope link
       valid_lft forever preferred_lft forever
4: brNC-hostonly: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::5880:6aff:fe77:cd16/64 scope link
       valid_lft forever preferred_lft forever
5: brNC-internet: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2a01:4f8:221:3744:4000::2/128 scope global
       valid_lft forever preferred_lft forever
    inet6 fc00::261/128 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::2044:c6ff:fef3:cd5d/64 scope link
       valid_lft forever preferred_lft forever
121: vethVIEF2K@if120: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::fc55:8fff:fe5d:a50a/64 scope link
       valid_lft forever preferred_lft forever
123: vethPIKVFW@if122: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::fc4f:adff:fe61:180b/64 scope link
       valid_lft forever preferred_lft forever
133: vethC387B7@if132: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::fc5c:16ff:fe64:444b/64 scope link
       valid_lft forever preferred_lft forever
135: vethE0I0FG@if134: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::fcbf:d4ff:fe33:a9d0/64 scope link
       valid_lft forever preferred_lft forever


ip -6 r
2a01:4f8:221:3744::1:26 dev enp0s3 proto kernel metric 256 pref medium
2a01:4f8:221:3744:4000::2 dev brNC-internet proto kernel metric 256 pref medium
2a01:4f8:221:3744:4000::/66 dev brNC-internet proto kernel metric 256 expires 9700sec pref medium
fc00::26 dev enp0s3 metric 1024 pref medium
fc00::261 dev brNC-internet proto kernel metric 256 pref medium
fe80::/64 dev enp0s3 proto kernel metric 256 pref medium
fe80::/64 dev brNC-hostonly proto kernel metric 256 pref medium
fe80::/64 dev brNC-internet proto kernel metric 256 pref medium
fe80::/64 dev enp0s2 proto kernel metric 256 pref medium
fe80::/64 dev vethVIEF2K proto kernel metric 256 pref medium
fe80::/64 dev vethPIKVFW proto kernel metric 256 pref medium
fe80::/64 dev vethE0I0FG proto kernel metric 256 pref medium
fe80::/64 dev vethC387B7 proto kernel metric 256 pref medium
default via fc00::26 dev enp0s3 metric 1024 pref medium

dhcpd configuration

cat /nix/store/5zvcjwvlj9n7cvrppkw1mxsxwhxwx3cm-dhcpd.conf
default-lease-time 600;
max-lease-time 7200;
authoritative;
ddns-update-style interim;
log-facility local1; # see dhcpd.nix

subnet6 2a01:4f8:221:3744:4000::/66 {
  #range6 2a01:4f8:221:3744:4000::/66 temporary;
  option dhcp6.name-servers 2a01:4f8:0:1::add:1010, 2a01:4f8:0:1::add:9999, 2a01:4f8:0:1::add:9898;
}

radvd configuration

cat /nix/store/89j6hg4qhnd9nyijf9p2dcr4f5ygjz6r-radvd.conf
interface brNC-internet {
  AdvSendAdvert on;
  MinRtrAdvInterval 3; 
  MaxRtrAdvInterval 10;
  prefix 2a01:4f8:221:3744:4000::/66 {
    AdvOnLink on; 
    AdvAutonomous off; 
  };
  RDNSS 2a01:4f8:0:1::add:1010 2a01:4f8:0:1::add:9999 2a01:4f8:0:1::add:9898 { };
};

guest LXC

[root@10:/]# ip -6 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
132: hostonly@if133: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::484f:50ff:fe0c:fa62/64 scope link 
       valid_lft forever preferred_lft forever
134: internet@if135: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2a01:4f8:221:3744:4000::4/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::10af:ffff:fef4:318a/64 scope link 
       valid_lft forever preferred_lft forever

[root@10:/]# ip -6 r
2a01:4f8:221:3744:4000::4 dev internet proto kernel metric 256 pref medium
2a01:4f8:221:3744:4000::/66 dev internet proto kernel metric 256 expires 86395sec pref medium
fe80::/64 dev hostonly proto kernel metric 256 pref medium
fe80::/64 dev internet proto kernel metric 256 pref medium
default via fe80::2044:c6ff:fef3:cd5d dev internet proto ra metric 1024 expires 25sec hoplimit 64 pref medium

lxc config

lxc.uts.name = 10

# Fixme also support other architectures?
lxc.arch = x86_64
# Not needed, just makes spares a few cpu cycles as LXC doesn't have
# to detect the backend.
#lxc.rootfs.backend = dir
lxc.rootfs.path = /var/lib/lxc/10/rootfs
lxc.init.cmd = /init/container/init
#lxc.rootfs = /var/lib/lxc/10/rootfs

# Ensures correct functionality with user namespaces. Since mknod is not possible stuff like
# /dev/console, /dev/tty, /dev/urandom, etc. need to be bind mounted. Note the order
# of the file inclusion here is important.
lxc.include = /nix/store/3hz7xkd86pzrvr4z53fa079q61qar02x-lxc-2.1.1/share/lxc/config/common.conf
lxc.include = /nix/store/3hz7xkd86pzrvr4z53fa079q61qar02x-lxc-2.1.1/share/lxc/config/userns.conf

## Network
# see also https://wiki.archlinux.org/index.php/Linux_Containers
lxc.net.0.type = veth
lxc.net.0.name = hostonly
#lxc.net.0.ipv4.address = 10.101.0.63 (we assign this using nix, not from lxc)
lxc.net.0.flags = up
lxc.net.0.link = brNC-hostonly

lxc.net.1.type = veth
lxc.net.1.name = internet 
lxc.net.1.flags = up
lxc.net.1.link = brNC-internet


# Specifiy {u,g}id mapping.
lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536

# FIXME apparmor support
# Nixos does not provide AppArmor support.
#lxc.aa_profile = unconfined
#lxc.aa_allow_incomplete = 1
lxc.apparmor.profile = unconfined
lxc.apparmor.allow_incomplete = 1

# Tweaks for systemd.
lxc.autodev = 1

# Additional mount entries.
lxc.mount.entry = /nix/store nix/store none defaults,bind.ro 0.0
lxc.mount.entry = /nix/var/nix/profiles/nixcloud-container-10 init none defaults,bind.ro 0 0

# Mount entries that lead to a cleaner boot experience.
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
lxc.mount.entry = mqueue dev/mqueue mqueue rw,relatime,create=dir,optional 0 0

# LXC autostart
lxc.start.auto = 0


lxc.rootfs.path = dir:/var/lib/lxc/10/rootfs

Best Answer

ok, here we go:

after 3 full days of trying different configurations, reading about 40 webpages and a hell of luck, here is how i got it working:

needed a special dhcpcd.conf (default won't work)
needed a special dhcpd6.conf (default won't work either)
needed a special radvd.conf (there was no default)
and only with the nixos firewall disabled (will post the rules modification when i found out what needs to be done) -> systemctl stop firewall

the missing firewall rules in the nixos firewall were those:

ip6tables -A INPUT -p tcp -m tcp -m multiport -i brNC-internet -j ACCEPT --dports 546,547
ip6tables -A INPUT -p udp -m udp -m multiport -i brNC-internet -j ACCEPT --dports 546,547

summary: the state of ipv6 documentation, ipv6 default configurations (in ubuntu server) and best practices guides is a shame and shows once more why ipv6 it is not deployed on more sites and with that i mean servers (not even talking about client configurations as laptops or mobile devices here).

my current setup deploys the ipv6 gateway via radvd and assigns ipv6 addresses using dhcpd6 in parallel as SLAAC can't be used in my setup as my prefix is /66 and needs to be (/64, /63, ..., a smaller number than 64). see http://www.teaparty.net/technotes/home-ipv6.html (section radvd) for more details.

note on dhclient: oh, and unlike dhcpcd which requires a very special configuration to work using dhclient on my ubuntu test machines i got a lease without any modification to the dhclient configuration. this is a huge plus compared to the dhcpcd implementation.

documentation

note on documentations and blogs: thanks so much to sixxs.net and the authors of these webpages. without your great work i couldn't have made it!

radvd configuration

  interface brNC-internet {
    AdvSendAdvert on;
    MinRtrAdvInterval 3;
    MaxRtrAdvInterval 10;
    #prefix 2a01:4f8:221:3744:4000::/66 {
    #  AdvOnLink on;
    #  AdvAutonomous off;
    #};
    #RDNSS 2a01:4f8:0:1::add:1010 2a01:4f8:0:1::add:9999 2a01:4f8:0:1::add:9898 { };

dhcpcd output

[root@11:~]# dhcpcd --config /root/dhcpcd6.conf 
DUID 00:01:00:01:22:52:a7:e3:0a:79:bb:c7:9c:d6
internet: IAID fc:bf:59:37
internet: IAID 00:00:00:01
internet: confirming prior DHCPv6 lease
internet: REPLY6 received from fe80::e4d2:fbff:feab:81dd
internet: adding address 2a01:4f8:221:3744:4000::300/128
internet: renew in 40000, rebind in 64000, expire in 86400 seconds
forked to background, child pid 12363

dhcpcd.conf (client configuration)

# Inform the DHCP server of our hostname for DDNS.
hostname

# Rapid commit support.
# Safe to enable by default because it requires the equivalent option set
# on the server to actually work.
option rapid_commit

# options to request from the DHCP
option domain_name_servers, interface_mtu

# A ServerID is required by RFC2131.
require dhcp_server_identifier

# only configure ipv6
ipv6only

# disable routing solicitation
noipv6rs

# don't touch these interfaces at all
denyinterfaces hostonly

interface internet
 # enable routing solicitation get the default IPv6 route
 #ipv6rs
 # request a normal (IA_NA) IPv6 address with IAID 1
 ia_na 1

dhcpd6 configuration

  services.dhcpd6 = {
    enable = true;
    interfaces = [ "brNC-internet" ];
    extraConfig = ''
      ddns-update-style interim;
      ddns-updates on;
      ddns-domainname "your.domain.com";
      ddns-rev-domainname "ip6.arpa";
      allow client-updates;
      update-conflict-detection false;
      update-optimization false;
      authoritative;
      option domain-name-servers dns.your.domain.com;
      default-lease-time 86400;
      preferred-lifetime 80000;
      allow leasequery;
      option dhcp6.name-servers 2001:0db8:edfa:1234::1;
      option dhcp6.domain-search "your.domain.com","domain.com";
      #include "/etc/rndc.key";
      option dhcp6.preference 255;

      subnet6 2a01:4f8:221:3744:4000::/66 {
        #range6 2a01:4f8:221:3744:4000::/66 temporary;
        range6 2a01:4f8:221:3744:4000::129 2a01:4f8:221:3744:4000::300;
        option dhcp6.name-servers 2a01:4f8:0:1::add:1010, 2a01:4f8:0:1::add:9999, 2a01:4f8:0:1::add:9898;
        # option dhcp6.gateway 2001:db8:2:3::1;
      } 
    ''; 
  };

tcpdump output

tcpdump -i brNC-internet ip6
13:47:01.854794 IP6 fe80::d4e8:fcff:febf:5937 > status.nixcloud.io: ICMP6, neighbor solicitation, who has status.nixcloud.io, length 32
13:47:01.854827 IP6 status.nixcloud.io > fe80::d4e8:fcff:febf:5937: ICMP6, neighbor advertisement, tgt is status.nixcloud.io, length 24
13:47:05.649860 IP6 status.nixcloud.io > ff02::1: ICMP6, router advertisement, length 24
13:47:06.772849 IP6 fe80::d4e8:fcff:febf:5937.dhcpv6-client > ff02::1:2.dhcpv6-server: dhcp6 solicit
13:47:06.773021 IP6 status.nixcloud.io.dhcpv6-server > fe80::d4e8:fcff:febf:5937.dhcpv6-client: dhcp6 advertise
13:47:06.773344 IP6 fe80::d4e8:fcff:febf:5937.dhcpv6-client > ff02::1:2.dhcpv6-server: dhcp6 request
13:47:06.774004 IP6 status.nixcloud.io.dhcpv6-server > fe80::d4e8:fcff:febf:5937.dhcpv6-client: dhcp6 reply
13:47:06.777782 IP6 fe80::d4e8:fcff:febf:5937 > ff02::16: HBH ICMP6, multicast listener report v2, 2 group record(s), length 48
13:47:07.071788 IP6 fe80::d4e8:fcff:febf:5937 > ff02::16: HBH ICMP6, multicast listener report v2, 2 group record(s), length 48
13:47:07.423792 IP6 :: > ff02::1:ff00:300: ICMP6, neighbor solicitation, who has 2a01:4f8:221:3744:4000::300, length 32

dhcpd6 log

Apr 01 13:35:41 status.nixcloud.io dhcpd6[9225]: Copyright 2004-2016 Internet Systems Consortium.
Apr 01 13:35:41 status.nixcloud.io dhcpd6[9225]: All rights reserved.
Apr 01 13:35:41 status.nixcloud.io dhcpd6[9225]: For info, please visit https://www.isc.org/software/dhcp/
Apr 01 13:35:41 status.nixcloud.io dhcpd6[9225]: Wrote 0 NA, 0 TA, 0 PD leases to lease file.
Apr 01 13:35:41 status.nixcloud.io dhcpd[9225]: Wrote 0 NA, 0 TA, 0 PD leases to lease file.
Apr 01 13:35:42 status.nixcloud.io dhcpd6[9225]: Bound to *:547
Apr 01 13:35:42 status.nixcloud.io dhcpd[9225]: Bound to *:547
Apr 01 13:35:42 status.nixcloud.io dhcpd[9225]: Listening on Socket/5/brNC-internet/2a01:4f8:221:3744:4000::/66
Apr 01 13:35:42 status.nixcloud.io dhcpd[9225]: Sending on   Socket/5/brNC-internet/2a01:4f8:221:3744:4000::/66
Apr 01 13:35:42 status.nixcloud.io dhcpd6[9225]: Listening on Socket/5/brNC-internet/2a01:4f8:221:3744:4000::/66
Apr 01 13:35:42 status.nixcloud.io dhcpd6[9225]: Sending on   Socket/5/brNC-internet/2a01:4f8:221:3744:4000::/66
Apr 01 13:35:42 status.nixcloud.io systemd[1]: Started DHCPv6 server.
Apr 01 13:35:42 status.nixcloud.io dhcpd6[9227]: Server starting service.
Apr 01 13:44:47 status.nixcloud.io dhcpd6[9227]: Solicit message from fe80::d4e8:fcff:febf:5937 port 546, transaction ID 0x693A9D00
Apr 01 13:44:47 status.nixcloud.io dhcpd6[9227]: Picking pool address 2a01:4f8:221:3744:4000::300
Apr 01 13:44:47 status.nixcloud.io dhcpd6[9227]: Advertise NA: address 2a01:4f8:221:3744:4000::300 to client with duid 00:01:00:01:22:52:a7:e3:0a:79:bb:c7:9c:d6 iaid = 1 valid for 86400 seconds
Apr 01 13:44:47 status.nixcloud.io dhcpd6[9227]: Sending Advertise to fe80::d4e8:fcff:febf:5937 port 546
Apr 01 13:44:47 status.nixcloud.io dhcpd6[9227]: Request message from fe80::d4e8:fcff:febf:5937 port 546, transaction ID 0x8694C500
Apr 01 13:44:47 status.nixcloud.io dhcpd6[9227]: Reply NA: address 2a01:4f8:221:3744:4000::300 to client with duid 00:01:00:01:22:52:a7:e3:0a:79:bb:c7:9c:d6 iaid = 1 valid for 86400 seconds
Apr 01 13:44:47 status.nixcloud.io dhcpd6[9227]: Sending Reply to fe80::d4e8:fcff:febf:5937 port 546
Apr 01 13:45:38 status.nixcloud.io dhcpd6[9227]: Release message from fe80::d4e8:fcff:febf:5937 port 546, transaction ID 0x176D6200
Apr 01 13:45:38 status.nixcloud.io dhcpd6[9227]: Client 00:01:00:01:22:52:a7:e3:0a:79:bb:c7:9c:d6 releases address 2a01:4f8:221:3744:4000::300
Apr 01 13:45:38 status.nixcloud.io dhcpd6[9227]: Sending Reply to fe80::d4e8:fcff:febf:5937 port 546
Apr 01 13:45:44 status.nixcloud.io dhcpd6[9227]: Solicit message from fe80::d4e8:fcff:febf:5937 port 546, transaction ID 0x9D658700
Apr 01 13:45:44 status.nixcloud.io dhcpd6[9227]: Advertise NA: address 2a01:4f8:221:3744:4000::300 to client with duid 00:01:00:01:22:52:a7:e3:0a:79:bb:c7:9c:d6 iaid = 1 valid for 86400 seconds
Apr 01 13:45:44 status.nixcloud.io dhcpd6[9227]: Sending Advertise to fe80::d4e8:fcff:febf:5937 port 546
Apr 01 13:45:44 status.nixcloud.io dhcpd6[9227]: Request message from fe80::d4e8:fcff:febf:5937 port 546, transaction ID 0xBF064200
Apr 01 13:45:44 status.nixcloud.io dhcpd6[9227]: Reply NA: address 2a01:4f8:221:3744:4000::300 to client with duid 00:01:00:01:22:52:a7:e3:0a:79:bb:c7:9c:d6 iaid = 1 valid for 86400 seconds
Apr 01 13:45:44 status.nixcloud.io dhcpd6[9227]: Sending Reply to fe80::d4e8:fcff:febf:5937 port 546
Apr 01 13:46:20 status.nixcloud.io dhcpd6[9227]: Unable to add forward map from 11.your.domain.com to 2a01:4f8:221:3744:4000::300: timed out

Related Solutions

OpenVPN IPV6 Tunnel Radvd

Your question really got me hooked up, since I may use the very same solution in another network I'm managing. I experimented with it and it is indeed possible! (I just love Linux...).

I created a Netkit laboratory that models your situation. You can download the laboratory here (1.8 KiB).

You do not need to install Netkit if you are not interested in testing it in your machine. You can just get the package above and see the .startup files for the various machines. In case you want to test the laboratory, it needs a filesystem with "radvd" installed, which is not included in Netkit's standard filesystem. Check the README of the filesystem package to see how to mount it in your machine and then use apt-get update && apt-get install radvd.

The lab contains 6 machines: v6site (some V6 Internet site you want to access), v6isp (your ISP), r1 (your first router with V4 and V6 connectivity), r2 (your second router that connects to r1 via OpenVPN), pc1 and pc2 (machines connected and served IPv6 by r2).

I used the RFC 3849 documentation prefix 2001:DB8::/32 in the example, instead of the random example addresses you used. Also, I used FEC0::/96 for the OpenVPN endpoints, which is obsolete. In your deployment, it is recommended that you use a small prefix from your Unique Local Address instead.

Clarification: RFC 3849 defines the prefix 2001:DB8::/32 to be used for example and documentation purposes (for global unicast). Instead of choosing any random IPv6 address, people are encouraged to use addresses in the 2001:DB8::/32 prefix as a wildcard in examples that will be changed to something else in the actual deployment. In this question, first 2001:acb:132:acb::/64, then 2001:123:123:11a1::/64. In the answer, I just replaced both with addresses in the documentation prefix. When you apply the answer to your real scenario, just look for every occurrence of 2001:DB8:: addresses and replace them with your actual addresses.

Tunnel endpoints also need addresses. The addresses used in tunnel endpoints do not need to be externally routeable, since they are used only internally. You used addresses starting with FED1:: and FED2::, while I used address starting with FECO::. These addresses were originally defined in RFC 3513. They are the equivalent to IPv4's private-use addresses 10.0.0.0/8, 192.168.0.0/16 and 172.16.0.0/12. Due to problems, they were later deprecated in RFC 3879 in favor of Unique Local Addresses (ULA) in RFC 4193. ULAs have a "random" prefix unique for each end-user. The benefit is that if for any reason you route between these networks, using tunnels for example, they will be able to talk to each other without address translation (while clashes may and do occur when using 192.168.0.0/16, for example). The page linked before this clarification will help you create your own ULA prefix (and possibly register it, but it is not necessary to register).

There is no real problem in using site-local addresses like FECx or FEDx in tunnel endpoints. They are deprecated, but that does not make them wrong. It is just recommended to use ULAs.

The overall configuration of your first router (r1) is bellow. Follow the comments for a better understanding.

# Enable forwarding for IPv6 (between eth0 <-> tun0)
sysctl -w net.ipv6.conf.all.forwarding=1

## ISP V6 Internal network
# Since there is no host specific address, we pick an address in the /64
# prefix. Note that this address is the same in two different prefixes:
# ..11a1::/64 and ..11a0::/59. This requires a proxing hack in R2.
# Optimally, you would have an address in the /59 prefix to use here,
# outside the delegated /64 prefix.
ip link set eth0 up
ip addr add 2001:db8:1:11a1::1/59 dev eth0
ip route add default via 2001:db8:1:11a0::1 dev eth0

## V4 Internet
ip link set eth1 up
ip addr add 192.168.1.1/24 dev eth1

## OpenVPN tunnel via IPv4 Internet to R2
# This is the most basic configuration of OpenVPN. No encryption, no security,
# no nothing. DO NOT USE THIS OUTSIDE THIS LABORATORY.
openvpn --dev tun --tun-ipv6 --daemon

while ! ip link show tun0 2>/dev/null
do
    echo "Waiting for OpenVPN to connect..."
    sleep 1
done

# Configure OpenVPN endpoints. Choose a distinct small prefix for the endpoints
# and use it to route the the /64 prefix to R2.
ip link set tun0 up
ip addr add fec0::1/96 dev tun0
ip route add 2001:db8:1:11a1::/64 via fec0::2 dev tun0

The overall configuration for the second router (r2):

# Enable forwarding for IPv6 (between eth0 <-> tun0)
sysctl -w net.ipv6.conf.all.forwarding=1

## Internal Company IPv6 Network
# The router address is arbitrary.
ip link set eth0 up
ip addr add 2001:db8:1:11a1::ffff/64 dev eth0

## V4 Internet
ip link set eth1 up
ip addr add 192.168.1.2/24 dev eth1

## OpenVPN tunnel via IPv4 Internet to R1
# This is the most basic configuration of OpenVPN. No encryption, no security,
# no nothing. DO NOT USE THIS OUTSIDE THIS LABORATORY.
openvpn --remote 192.168.1.1 --dev tun --tun-ipv6 --daemon

# Wait for OpenVPN...
while ! ip link show tun0 2>/dev/null
do
    echo "Waiting for OpenVPN to connect..."
    sleep 1
done

# Configure OpenVPN endpoints. See comments for R1 above.
# Note that we route ALL IPv6 traffic through the tunnel.
ip link set tun0 up
ip addr add fec0::2/96 dev tun0
ip route add default via fec0::1 dev tun0

# R1 address is in our private network (eth0, see above), but on the other
# side of the tunnel. We need a more specific route specifically for it.
# Also, make this router (R2) act as a neighbor proxy so that other
# machines on the private network can see R1 through the tunnel.
# This is a hack that would be avoided if we had a bigger prefix than
# /64, or if R1 had a host-specific address outside of the /64.
ip route add 2001:db8:1:11a1::1/128 via fec0::1 dev tun0
ip neigh add proxy 2001:db8:1:11a1::1 dev eth0

## Routing advertisement daemon
# NOTE: The standard Netkit filesystem does not have radvd, it has to be
# installed manually with `apt-get update && apt-get install radvd` in
# the model fs.
chmod 644 /etc/radvd.conf
radvd

The configuration for the PCs connected to r1 (eth0) is extremely simple, thanks to radvd:

sysctl -w net.ipv6.conf.all.autoconf=1
ip link set eth0 up

This is the most important configuration. The other details (including a copy of r2's /etc/radvd.conf) are in the laboratory package above.

Configuring ipv6 over radvd using sixxs, routing not working

Have you actually requested a subnet, and not just the tunnel? You need another subnet routed to your tunnel endpoint, you CANNOT use the tunnel subnet directly:

https://www.sixxs.net/faq/connectivity/?faq=usingsubnet

https://www.sixxs.net/home/requestsubnet/

How do I give connectivity to other hosts on my subnet?

The easiest way to use your subnet is to assign a /64 per network and then setup a Router Advertisement server. As SixXS serves out per tunnel a /48 (a so called site-prefix) as subnets you have the possiblity of having 65535 /64's and thus subnets inside your site. A /48 is an end-site and should thus not be delegated to other administrators.

Under Linux this Router Advertisement (RA) server is called radvd, *BSD (KAME stack) calls it rtadvd. Clients can then be configured using RFC 2462 aka "IPv6 Stateless Address Autoconfiguration".

Note well that in tunnels from the /64 only ::1 (the PoP) and ::2 (your endpoint) can be used as the rest is not routed.

The 'default routed /64' is a routed subnet though and is routed to the endpoint of your tunnel and thus can be used directly for connecting hosts on a seperate link behind your tunnel. If you need to serve multiple /64s behind your endpoint you will need to request a subnet though.

A site is defined as a network with one single administration. The moment a change occurs in administration, one is in a different site. Thus if you have 1 building operated by administration group X and another building operated by admin Y then those are two sites. Of course, when group X and Y, both administratively fall under group A, they can still be taken as to be a single site where wanted.

what i want

my problem

dhcpcd from lxc

tcpdump on the VM

dhcpd6 log on the vm

my setup

host vm

dhcpd configuration

radvd configuration

guest LXC

lxc config

Best Answer

documentation

radvd configuration

dhcpcd output

dhcpcd.conf (client configuration)

dhcpd6 configuration

tcpdump output

dhcpd6 log

Related Solutions

OpenVPN IPV6 Tunnel Radvd

Configuring ipv6 over radvd using sixxs, routing not working

Related Topic