Eucalyptus problem

cloudeucalyptus

I have some problems regarding to eucalyptus private cloud building. And two versions of this problem here, a simple one, and a long one.

Simple:

Anyone familiar with private cloud could draw me a picture to illustrate the 'IP SCHEME' of he cloud? I'm having trouble to decide where should be each component to.
If I want to install CLC, Walrus, CC, SC, NC all on different physical machines, what should be the network topology?

More information: I have 12 physical machines running the cloud now, but I can't attach volumes to instances. I wondered where should I put storage controller to, public network? under CLC? under CC? same level with CC? No picture, no explanation found at all. And I tried everything I can, instances are running, ok to access them, just the damn volume thing bothered me for a whole week.

Long version:

  1. I have successfully configured eucalyptus on 4 physical servers and it runs great.
  2. The four servers are: CLC, SC, CC, NC
  3. The problem is I'm not very sure where to put SC, Since the topology of my cloud is like this:

alt text

  1. Now that CC are connected to two switches : ENTSWitch and PrivateSwitch. And has two IPs: public ip->10.11.25.115, private IP-> 10.11.20.1. Node controller is connected to privateswitch only and has private IP 10.11.20.2
  2. This works fine by now, using managed-novlan mode, node controller can deploy instances and get public ip addresses for them. Everything is sweet.
  3. Here comes the problem, I can create volume on storage controller. But the volume is failed to be attached to instances.
  4. Wondering if it's because node controller is under a different network with storage controller.

Tried to solve the problem by doing following:

  • A. maybe i shouldn't connect storage controller to ENTSwitch?. Changed SC from public network t o private network
  • B. Try to register storage controller again but failed. Reason: Cloud can't reach private network.
  • C. Try to register sc on CC, can't, it must be registered on CLC
  • D. Add route to CLC, now CLC and SC can connect to each other.
  • E. Register successfully, still can't attach volume.
  • F: Error log:
    [root@HeroNodeServer1 images]# cat /var/log/eucalyptus/nc.log | grep Volume
    [Fri Aug 20 16:36:12 2010][015662][EUCAINFO ] doAttachVolume() invoked (id=i-33BD0649 vol=vol-59C90629 remote=/dev/etherd/e0.1 local=/dev/sdb)
    [Fri Aug 20 16:37:53 2010][015662][EUCAERROR ] AttachVolume() failed (err=-1) XML=
    [Fri Aug 20 16:37:53 2010][015662][EUCAERROR ] ERROR: doAttachVolume() failed error=1
    [Fri Aug 20 16:37:53 2010][015662][EUCAINFO ] doAttachVolume() invoked (id=i-33BD0649 vol=vol-59C90629 remote=/dev/etherd/e0.1 local=/dev/sdb)
    [Fri Aug 20 16:37:53 2010][015662][EUCAERROR ] AttachVolume() failed (err=-1) XML=
    [Fri Aug 20 16:37:53 2010][015662][EUCAERROR ] ERROR: doAttachVolume() failed error=1
    [Fri Aug 20 16:37:53 2010][015662][EUCAINFO ] doAttachVolume() invoked (id=i-33BD0649 vol=vol-59C90629 remote=/dev/etherd/e0.1 local=/dev/sda5)
    [Fri Aug 20 16:37:53 2010][015662][EUCAERROR ] AttachVolume() failed (err=-1) XML=
    [Fri Aug 20 16:37:53 2010][015662][EUCAERROR ] ERROR: doAttachVolume() failed error=1
    [Fri Aug 20 16:37:58 2010][015662][EUCAINFO ] doAttachVolume() invoked (id=i-33BD0649 vol=vol-59C90629 remote=/dev/etherd/e0.1 local=/dev/sda4)
    [Fri Aug 20 16:39:38 2010][015662][EUCAERROR ] AttachVolume() failed (err=-1) XML=
    [Fri Aug 20 16:39:38 2010][015662][EUCAERROR ] ERROR: doAttachVolume() failed error=1
    [Fri Aug 20 16:39:38 2010][015662][EUCAINFO ] doAttachVolume() invoked (id=i-33BD0649 vol=vol-59C90629 remote=/dev/etherd/e0.1 local=/ev/sdp)
    [Fri Aug 20 16:39:38 2010][015662][EUCAERROR ] ERROR: doAttachVolume() failed error=1
    [Fri Aug 20 16:39:38 2010][015662][EUCAINFO ] doAttachVolume() invoked (id=i-33BD0649 vol=vol-59C90629 remote=/dev/etherd/e0.1 local=/dev/sdp)
    [Fri Aug 20 16:39:38 2010][015662][EUCAERROR ] AttachVolume() failed (err=-1) XML=
    [Fri Aug 20 16:39:38 2010][015662][EUCAERROR ] ERROR: doAttachVolume() failed error=1
    [Fri Aug 20 16:41:15 2010][015662][EUCAINFO ] doAttachVolume() invoked (id=i-33BD0649 vol=vol-5A030630 remote=/dev/etherd/e0.2 local=/dev/sdp)
    [Fri Aug 20 16:42:55 2010][015662][EUCAERROR ] AttachVolume() failed (err=-1) XML=
    [Fri Aug 20 16:42:55 2010][015662][EUCAERROR ] ERROR: doAttachVolume() failed error=1
    [Fri Aug 20 16:43:12 2010][015662][EUCAINFO ] doAttachVolume() invoked (id=i-33BD0649 vol=vol-5A030630 remote=/dev/etherd/e0.2 local=/dev/sdc)
    [Fri Aug 20 16:44:52 2010][015662][EUCAERROR ] AttachVolume() failed (err=-1) XML=
    [Fri Aug 20 16:44:52 2010][015662][EUCAERROR ] ERROR: doAttachVolume() failed error=1
    [Fri Aug 20 16:50:02 2010][015662][EUCAINFO ] doAttachVolume() invoked (id=i-33BD0649 vol=vol-59C90629 remote=/dev/etherd/e0.1 local=/dev/sdd)
    [Fri Aug 20 16:51:42 2010][015662][EUCAERROR ] AttachVolume() failed (err=-1) XML=
    [Fri Aug 20 16:51:42 2010][015662][EUCAERROR ] ERROR: doAttachVolume() failed error=1

Questions:

  1. Where on earth should I attach SC to? public network or private network?
  2. If it should be attached to private network, how to register SC if CLC can't reach private network
  3. Why two solutions all failed to attach volume to instance?

================================================================

Here comes second one:

  1. I'm currently trying to do some more test on it.
  2. I want to expand 4-physical architecture to 16-physical architecture.
  3. No matter how many machines I want to use, there's a question I had for a long time without being solved by tons of documents read.
  4. The new architecture I designed is like following picture alt text
  5. In the new topology, I assumed that storage controller should be attached to private network
  6. In this model, instances running on node controller should have two IP addresses assigned, one private, one "public"
  7. The public IP address should be in the network same as CC right?
  8. That means instances can be accessed by the network which CC located in.
  9. And CLC should have a gateway service running to let instances accessing outer networks

Questions:

  1. Is there any problem about the design?
  2. If instances only can get "Public IP" which is under same network with Cluster Controller, how are clients suppose to access them from outer networks?
  3. Means, how clients from out of the cloud access Instances through CLC?
  4. Does CLC has the same mechanism like CC to assign a "public IP" for instances so that it can be accesses?

================================

That'd all I need to ask.

Thank you very much for reading this messy post, and any kind of reply is highly appreciated!

Best Answer

The following answers all assumes that the network mode is MANAGED-NOVLAN (this probably all apply for MANAGED mode also).

Section 1:

-1) Unless you have specific needs preventing it, I would recommend coupling the SC with the CC.

-2) In Managed mode, the CC sits between the public network and the private network of (one of) the cloud cluster(s). If the SC is on the CC machine it can than communicate with both networks. If you decide to have the SC on a separate server on the private network, you'll need to create a NAT rule (with iptables) on the CC to enable communications between the SC and the CLC through the CC Firewall. The CLC will need that rule to communicate with the SC.

-3) In solution 1; your SC was not connected to the private switch (this is a must). In solutions 2; you probably did not created that iptables rule on the CC and the SC was then isolated behind the CC.

Section 2:

-1) One correction: merge the switches CTI and switch1, It should be the same switch that is the connection to the public network. I would also suggest (unless absolutely needed) to merge the CLC and WC into one machine and to merge the SCs on their respective CCs. This diagram (http://cssoss.files.wordpress.com/2010/05/eucalyptus_cloud.png?w=600&h=467) of a UEC cloud can help you.

-2) When you create a new instance (euca-run-instance) Eucalyptus will create a new iptable rule on the CC server managing the node on which the instance is running. That rule permit communications through the CC. This NAT traversal route the communications between publics IPs and their corresponding private IPs.

-3) The clients accesses the instances through the corresponding CC. The CLC is only there to manage the cloud but, once an instance is started, the CC handles the communications.

-4) In short: No.

Related Topic