I've setup OpenStack – my first time. I've followed the installation guide (on Ubuntu 14.04) and I believe I've got everything correct. However, when trying to create an instance I get the error:
Build of instance
e948205a-2287-4b0e-9829-f2c6fe3a93eb
aborted: Unable to establish
connection to
http://127.0.0.1:9696/v2.0/extensions.json
I've been debugging this on and off for around a week now, and I've exhausted every solution I've found on Google, and that I can think of.
Neutron is listening on the port:
netstat -ln | grep 9696 tcp 0
0 0.0.0.0:9696 0.0.0.0:*
LISTEN
I can also connect to Neutron from the command line:
root@controller:/var/log# neutron ext-list
+---------------------------+-----------------------------------------------+
| alias | name |
+---------------------------+-----------------------------------------------+
| default-subnetpools | Default Subnetpools |
| network-ip-availability | Network IP Availability |
| network_availability_zone | Network Availability Zone |
| auto-allocated-topology | Auto Allocated Topology Services |
| ext-gw-mode | Neutron L3 Configurable external gateway mode |
| binding | Port Binding
...
I can authenticate with a token and get a response from Neutron manually with the curl test below, this responds with expected data.
curl -i -H'X-Auth-Token: MY_TOKEN' http://controller:9696/v2.0/extensions.json
My neutron api endpoints:
| 440735e5dd8d468e89de9a8077ea9491 | RegionOne | neutron | network | True | public | http://controller:9696 |
| 5985d944b02b4148a87cafae8df006c8 | RegionOne | neutron | network | True | internal | http://controller:9696 |
I've checked all of the Nova and Neutron logs, and i can't see any errors in there, I've recreated my Neutron endpoints, I've checked and double checked configs. I'm at a loss how to continue debugging or where the solution lies.
Edit with Updated Info:
root@controller:~# openstack catalog list
+----------+----------+--------------------------------------------------------------------------+
| Name | Type | Endpoints |
+----------+----------+--------------------------------------------------------------------------+
| neutron | network | RegionOne |
| | | admin: http://controller:9696 |
| | | RegionOne |
| | | internal: http://controller:9696 |
| | | RegionOne |
| | | public: http://controller:9696 |
| | | |
| nova | compute | RegionOne |
| | | admin: http://controller:8774/v2.1/225f1282f5034949937f5ef71c503dd0 |
| | | RegionOne |
| | | internal: http://controller:8774/v2.1/225f1282f5034949937f5ef71c503dd0 |
| | | RegionOne |
| | | public: http://controller:8774/v2.1/225f1282f5034949937f5ef71c503dd0 |
| | | |
| keystone | identity | RegionOne |
| | | admin: http://controller:35357/v3 |
| | | RegionOne |
| | | public: http://controller:5000/v3 |
| | | RegionOne |
| | | internal: http://controller:5000/v3 |
| | | |
| glance | image | RegionOne |
| | | admin: http://controller:9292 |
| | | RegionOne |
| | | public: http://controller:9292 |
| | | RegionOne |
| | | internal: http://controller:9292 |
| | | |
+----------+----------+--------------------------------------------------------------------------+
Neutron networks i've created via Horizon and the cli
root@controller:~# neutron net-list
+--------------------------------------+------------------+--------------------------------------------------+
| id | name | subnets |
+--------------------------------------+------------------+--------------------------------------------------+
| 8a50aef6-b687-483d-ab49-f43460eebdd6 | My Admin Network | d4f840cc-31b9-4d32-963d-3a2110ae5765 10.0.0.0/24 |
| 8bb223ef-828d-4c9a-aa16-15ac13d244f6 | ext-net | |
| 98a2606a-3017-48fb-8b60-e18c4a4f1083 | My Network | b76352ac-dc61-4d75-ba7a-b5b4d76705d0 10.0.0.0/24 |
+--------------------------------------+------------------+--------------------------------------------------+
My Neutron Ext list is below:
root@controller:~# neutron ext-list
+---------------------------+-----------------------------------------------+
| alias | name |
+---------------------------+-----------------------------------------------+
| default-subnetpools | Default Subnetpools |
| network-ip-availability | Network IP Availability |
| network_availability_zone | Network Availability Zone |
| auto-allocated-topology | Auto Allocated Topology Services |
| ext-gw-mode | Neutron L3 Configurable external gateway mode |
| binding | Port Binding |
| agent | agent |
| subnet_allocation | Subnet Allocation |
| l3_agent_scheduler | L3 Agent Scheduler |
| tag | Tag support |
| external-net | Neutron external network |
| net-mtu | Network MTU |
| availability_zone | Availability Zone |
| quotas | Quota management support |
| l3-ha | HA Router extension |
| provider | Provider Network |
| multi-provider | Multi Provider Network |
| address-scope | Address scope |
| extraroute | Neutron Extra Route |
| timestamp_core | Time Stamp Fields addition for core resources |
| router | Neutron L3 Router |
| extra_dhcp_opt | Neutron Extra DHCP opts |
| dns-integration | DNS Integration |
| security-group | security-group |
| dhcp_agent_scheduler | DHCP Agent Scheduler |
| router_availability_zone | Router Availability Zone |
| rbac-policies | RBAC Policies |
| standard-attr-description | standard-attr-description |
| port-security | Port Security |
| allowed-address-pairs | Allowed Address Pairs |
| dvr | Distributed Virtual Router |
+---------------------------+-----------------------------------------------+
With the create command with debug, i now have some further info in the actual error (will be best to copy and paste this out to read it):
{u'message': u'Build of instance 3e2f5678-6a4a-478b-8953-d10db929d367 aborted: Unable to establish connection to http://127.0.0.1:9696/v2.0/extensions.json', u'code': 500, u'details': u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1926, in _do_build_and_run_instance\n filter_properties)\n File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2083, in _build_and_run_instance\n \'create.error\', fault=e)\n File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__\n self.force_reraise()\n File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type_, self.value, self.tb)\n File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2067, in _build_and_run_instance\n instance=instance)\n File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__\n self.gen.throw(type, value, traceback)\n File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2244, in _build_resources\n reason=six.text_type(exc))\n', u'created': u'2016-04-29T13:56:55Z'} |
Best Answer
you can debug this by checking if the neutron service is up and running and registered correctly in your service catalog.
look at your
keystone catalogit will ouput the API endpoints
then what is the output of the
neutron net-listcommandin your neutron.conf are you binding to 127.0.0.1 ? - you should bind to all IP's
Neutron uses api extensions or 'plugins' and also agents for DHCP, security groups and NAT- can you go on to say what extension you are using?
probably this will be openvswitch as your plugin and network agents by default are usually dnsmaq, and iptables.
you can check the openvswitch services in /etc/ini.d/ and look at the switch database with the following show commands
also, for debugging the extension http error, you can enable debug logging in the neutron conf or pass in the --debug option on the cli e.g.