No internet on VIrtual Machines and external (physical) network is unreachable from VM’s

cloud computingopenstack

I have installed openstack (packstack) on centos 6.4 with neutron. The openstack is installed on single virtual machine using vmware.

First i have created external network using following command

quantum net-create public --router:external=True

Then i added my external network subnet (ip pool not used in external network)

Then i created router

Then i set my router gateway to external network

I created security group to allow ssh and icmp.

On second step i created private network with dhcp enabled

Then created router interface and attach it to my private network

On third step i launched the instance with private network

On fourth step i generated floating ip of external network and associates it to instance

Problem Statement:

Virtual machines are getting IP's (private network) from dhcp and communicating with each other but there is No internet on VM's

VM's cannot ping any external network device.

I am using Centos 6.4. Ip route shows

192.168.186.0/24 dev eth1  proto kernel  scope link  src 192.168.186.166  metric 1 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1 
10.16.48.0/22 dev br-ex  proto kernel  scope link  src 10.16.51.208
169.254.0.0/16 dev eth0  scope link  metric 1002
169.254.0.0/16 dev br-ex  scope link  metric 1017
default via 10.16.48.1 dev br-ex

Where 10.16.48.0/22 is my external network on eth0 (for internet)

10.16.186.0/24 is another interface on eth1

quantum agent-list shows

----+-------+----------------+
| id                                   | agent_type         | host                  | alive | admin_state_up |
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| 4c709a4c-bf0c-4e03-a0f5-2d938fee7ae1 | L3 agent           | localhost.localdomain | :-)   | True           |
| 960a7806-dc4d-4ab1-99b0-a79dbc31600f | Open vSwitch agent | localhost.localdomain | :-)   | True           |
| d9f545e2-4a6c-43f4-8037-b807cbe27fc5 | DHCP agent         | localhost.localdomain | :-)   | True           |
+--------------------------------------+--------------------+-----------------------+-------+----------------+

Best Answer

So you have created a VMWare Virtual Server, and in this VM, installed OpenStack, which itself hosts VMs?

Then the Virtual Switch the VMWare VM is connected to, blocks every traffic from your OpenStack VMs. This is a security measure, to make sure only VMWare VMs can reach the Virtual Switch and the outside network. This is a layer 2 trouble, I am quite sure the arp -an command in OpenStack VMs does not show you gateway.

All you have to do is allow the VMWare Virtual Switch to receive multiple MAC addresses.

You can find how to do this onthis post

EDIT FOR YOUR COMMENT:

I don't know. What I think I know, is that you can't reach you router, and as consequence, the outside network, even at a Layer 2 (check for this that no MAC address is shown for the gateway when launching the arp -an command) because the VSwitch is dropping paquets whom mac-addresses are not those of the OpenStack server.

Then, you can either disabled this feature in the VSwitch, or install on your OpenStack server an ARP-proxy service. The idea is to spoof MAC-address of your VMs which will then appear on the VSwitch as the OpenStack server itself.

Related Solutions

Linux – Shutting down Openstack server and virtual machines

This is more about libvirt than OpenStack itself. When the libvirt daemon is shutting down during host shutdown, it tries to gracefully shut down all of the guests. If any don't shut down within a grace period, they get unceremonially killed. So in answer to your question, it depends whether your guests shut down in a reasonable timeframe or not.

Nat – How to make a linux VM working as a router

You are on the right track with your firewall rules. I do things the "old fashioned" way - write a script, put it in /etc and call it from /etc/rc.local. However you like to do it, here's what works for me.

The OS is Debian Jessie 64 bit via netinstall with only "standard system utilities" selected at tasksel time, running in VirtualBox on a Mint desktop. The eth0 is connecting to my LAN via a bridged interface and DHCP. The eth1 is the LAN side of a network of VM machines I use for experimenting. Copy/paste the firewall script to /etc/rc.firewall, make it executable, and call it in /etc/rc.local.

#!/bin/bash
# copyright me, licensed to you freely
# a very simple set of iptables commands 
# to allow forwarding between ethernet
# devices

# make sure forwarding is enabled in the kernel
echo 1 > /proc/sys/net/ipv4/ip_forward

# where is iptables located?
iptables=`which iptables`

# flush all existing rules
$iptables -F

# this is for NAT
# enable masquerading
/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

# don't forward packets from off-lan to lan if
# they are a brand new connection being initiated
$iptables -A FORWARD -i eth0 -o eth1 -m state --state NEW -j REJECT

# if the packets come from off-lan but they are
# related to a connection that was established from
# within the lan, go ahead and forward them
$iptables -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT

# whatever traffic comes from the lan to go to
# the world allow thru
$iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT

When all that is done and run, you should be able to see something like

root@router:~# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 08:00:27:e6:43:df  
          inet addr:192.168.1.126  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fee6:43df/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:729 errors:0 dropped:0 overruns:0 frame:0
          TX packets:382 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:61777 (60.3 KiB)  TX bytes:46468 (45.3 KiB)

root@router:~# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 08:00:27:af:50:e2  
          inet addr:10.99.99.1  Bcast:10.99.99.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:feaf:50e2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:828 (828.0 B)

root@router:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0
10.99.99.0      0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
root@router:~# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
REJECT     all  --  anywhere             anywhere             state NEW reject-with icmp-port-unreachable
ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
root@router:~# cat /proc/sys/net/ipv4/ip_forward
1
root@router:~#