Linux – IPv6 Neighbor Discovery/Routing Failing for Kernel 3.10

ipv6linuxlxcrouting

Stupid mistakes led to all of this, read Update 5

Intro

I'm trying to setup a Linux Router (LXC, 3.10.0-123.el7.x86_64) with IPv6.

The provider is Hetzner and I got 2 subnets, one /56 and a /64. They have setup the routing to my LL Address and my default gateway is fe80::1.

I have enabled ipv6 and ipv4 forwarding in sysctl.conf:

net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1

and I configured the IPTables to Accept everything.

I know that ICMPv6 is the main utility used for routing in IPv6 – so that's not blocked ;-). I also added -A FORWARD -j ACCEPT.

I have configured the second (::2) ip of my /56 subnet to eth1 and the second (::2) of the /64 to eth0.

eth0 is the uplink interface
eth1 is the lan interface

Main problem

The internal clients with ipv6 addresses from the /56 can't ping the outside world, but the packets and response is correctly routed up until my router, which just drops the packet without any hint.

e.g. ping6 ipv6.google.com results in timeouts.

but on the router's uplink I get:

20:43:13.350932 IP6 client-ipv6-ip > fra07s64-in-x00.1e100.net: ICMP6, echo request, seq 1, length 64
20:43:13.355143 IP6 fra07s64-in-x00.1e100.net > client-ipv6-ip: ICMP6, echo reply, seq 1, length 64
20:43:14.350572 IP6 client-ipv6-ip > fra07s64-in-x00.1e100.net: ICMP6, echo request, seq 2, length 64
20:43:14.354609 IP6 fra07s64-in-x00.1e100.net > client-ipv6-ip: ICMP6, echo reply, seq 2, length 64
20:43:15.350630 IP6 client-ipv6-ip > fra07s64-in-x00.1e100.net: ICMP6, echo request, seq 3, length 64
20:43:15.355072 IP6 fra07s64-in-x00.1e100.net > client-ipv6-ip: ICMP6, echo reply, seq 3, length 64
20:43:16.350656 IP6 client-ipv6-ip > fra07s64-in-x00.1e100.net: ICMP6, echo request, seq 4, length 64
20:43:16.354748 IP6 fra07s64-in-x00.1e100.net > client-ipv6-ip: ICMP6, echo reply, seq 4, length 64

on the client only the echo request is visible.

What could be the problem? Are there any additional settings so that my router is accepting anything that's sent to it?

Up until now I only used NAT and never full blown routing on my Linux boxes.

Thanks for any pointers, I hope it's just a simple sysctl parameter to set …

Update 1

I replaced the /56 subnet by "subnet-2" and the /64 by "subnet-1".

ip -6 a:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
61: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 subnet-1::2/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:2eff:fe01:1/64 scope link 
       valid_lft forever preferred_lft forever
63: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 1000
    inet6 subnet-2::2/56 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe01:1/64 scope link 
       valid_lft forever preferred_lft forever

ip -6 r:

unreachable ::/96 dev lo  metric 1024  error -101
unreachable ::ffff:0.0.0.0/96 dev lo  metric 1024  error -101
unreachable 2002:a00::/24 dev lo  metric 1024  error -101
unreachable 2002:7f00::/24 dev lo  metric 1024  error -101
unreachable 2002:a9fe::/32 dev lo  metric 1024  error -101
unreachable 2002:ac10::/28 dev lo  metric 1024  error -101
unreachable 2002:c0a8::/32 dev lo  metric 1024  error -101
unreachable 2002:e000::/19 dev lo  metric 1024  error -101
subnet-1::/64 dev eth0  proto kernel  metric 256 
subnet-2::/56 dev eth1  proto kernel  metric 256 
unreachable 3ffe:ffff::/32 dev lo  metric 1024  error -101
fe80::/64 dev eth0  proto kernel  metric 256 
fe80::/64 dev eth1  proto kernel  metric 256 
default via fe80::1 dev eth0  metric 1024

System is up2date, except for the kernel (e.g. it's not loaded) since that will require a restart and I haven't had issues with the current one. But if it's a known issue with 3.10.0-123.13.2.el7 I can restart to upgrade it to 3.10.0-229.14.1.el7.

Update 2

tcpdump -i eth0 ip6

shows incoming packets.

I added a firewall rule to mangle e.g.

*mangle
-A PREROUTING -j NFLOG

to log to /var/log/messages, but nothing is logged there!

Update 3

    Bridge "br0"
        Port "router.eth0"
            Interface "router.eth0"
        Port "br0"
            Interface "br0"
                type: internal
        Port "eth0"
            Interface "eth0"
    Bridge "br1"
        Port "db01.eth0"
            Interface "db01.eth0"
        Port "mail01.eth0"
            Interface "mail01.eth0"
        Port "br1"
            Interface "br1"
                type: internal
        Port "team1"
            Interface "eth3"
            Interface "eth2"
        Port "router.eth1"
            Interface "router.eth1"

Update 4

I fucked up, had the wrong MAC-address on the interface (long story short – fixed it now).

So now the packets are accepted and forwarded, but the response is not sent to the default gateway:

Router:

22:09:22.638405 IP6 ping_ip > client_ip: ICMP6, echo request, seq 243, length 64
22:09:22.754936 IP6 router-ll > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fe80::1, length 32
22:09:22.757001 IP6 2a01:4f8::a:b:10 > router-ll: ICMP6, neighbor advertisement, tgt is fe80::1, length 32
22:09:22.757044 IP6 router-ll > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has router-ll, length 32
22:09:23.639651 IP6 ping_ip > client_ip: ICMP6, echo request, seq 244, length 64
22:09:23.756969 IP6 router-ll > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fe80::1, length 32
22:09:23.759007 IP6 router-ll > ff02::1:ff00:2: ICMP6, neighbor solicitation, who has router-ll, length 32
22:09:23.759240 IP6 2a01:4f8::a:b:10 > router-ll: ICMP6, neighbor advertisement, tgt is fe80::1, length 32
22:09:24.640677 IP6 ping_ip > client_ip: ICMP6, echo request, seq 245, length 64

Client:

22:11:41.640978 IP6 ping_ip > client_ip: ICMP6, echo request, seq 382, length 64
22:11:41.640998 IP6 client_ip > ping_ip: ICMP6, echo reply, seq 382, length 64
22:11:42.642993 IP6 ping_ip > client_ip: ICMP6, echo request, seq 383, length 64
22:11:42.643013 IP6 client_ip > ping_ip: ICMP6, echo reply, seq 383, length 64

ip -6 neigh:

fe80::216:3eff:fe01:1 dev eth1 lladdr 00:16:3e:01:00:01 router STALE
subnet-2::1:1 dev eth0  FAILED
fe80::216:3eff:fe15:1 dev eth1 lladdr 00:16:3e:15:00:01 DELAY
fe80::216:3eff:fe15:c dev eth1 lladdr 00:16:3e:15:00:0c STALE
fe80::1 dev eth0  FAILED
subnet-2::1:1 dev eth1 lladdr 00:16:3e:15:00:01 REACHABLE
fe80::216:2eff:fe01:1 dev eth0  INCOMPLETE
subnet-2::1:12 dev eth1 lladdr 00:16:3e:15:00:0c STALE
subnet-1::2 dev eth0  FAILED

the failed entry for 1:1 is here because I shortly had that address on eth0 to debug the ping issue from before …

Update 5

So as it happens with changing the MAC-address I didn't change the LL Address. I wasn't aware that those 2 addresses are linked 100%.

The main problem was that the old router died and I copied the LL Address from there didn't change the corresponding MAC-address → it didn't work.

Then … changed the LL back (which was the mistake) and changed the MAC-address (in 2 separate steps a few hours apart).

For anyone else the L2 MAC und the IPv6 LL Address need to be corresponding, and they need to match with what a router is sending.

To debug use tcpdump -i eth0 ip6 -en then you can see the MAC-address and LL addresses and compare them with your interface, if any of them, and the one configured at Hetzner as routing address don't match, nothing works!

So again, sorry for anyone who read that, I somehow missed to explain that it was on a different router before and I installed this one because the last one died. Obvious fix …

Best Answer

If you are experiencing weird routing issues with packets missing, follow these steps:

ip l and note the MAC Addresses

tcpdump -en -i eth0 ip6 (or the corresponding interface) and compare them, if they match all is good

ip -6 a compare the Link Local Addresses (scope local) to your mac address (here is a calculator: http://ben.akrin.com/?p=1347 , there are a few)

if it's still not working I guess the problem lies in the iptables, use logging as much as possible :)

Related Solutions

Linux – IPv6 works only after ping to routing box

Are the clients' prefixes being manually assigned? Usually they should automatically find the router via Neighbor Discovery Protocol (during which the router sends out advertisements and assign them prefixes automatically), but it sounds like that step might be missing.

Additionally, the router advertisement should be including its link-layer address as an option in the ICMP header of the router advertisement. If this field is missing, the client won't know how to send data to the router. It sounds like this could be the case. The client doesn't know how to reach the router, until it sends out the Neighbor Discovery message and it receives an Neighbor Advertisement from the router (with the router flag in the ICMP message set).

To include the source link layer address in your router advertisements, add the following into your radvd.conf

AdvSourceLLAddress on;

Linux – Native IPv6: routing between eth0 and eth1

I notice one specific problem in your routing table. It specifies 2001:db8:14::/48 as directly attached to eth0. That should have been a /64. But I don't see how that could explain the symptoms.

One piece of information is clearly missing. The ISP router would have to be configured with a gateway address for 2001:db8:14::/48. If that gateway address is 2001:db8:14::2, the routing should be working correctly. If it is something else, then packets from outside cannot reach anything on your LAN, only the router WAN address would be reachable. That would explain why 2001:db8:14:a::1 is not reachable from outside. But 2001:db8:14::2 should still have been reachable from inside, which is puzzling.

The only way forward in such a situation is to repeat the pings, which did not work, and this time be observing the network traffic on both interfaces of the router with tcpdump or equivalent.

When sending packets from outside, the eth0 interface on the router should see neighbor discovery for the gateway that the ISP assigned for your prefix. If sending packets to any address in your /48 results in neighbor discovery for the exact same address, then that address is the gateway address you should be assigning to eth0. There is a few other ways this can turn out, in that case you need to update your question with information about what traffic you actually see.