Cloudformation bootstrap with puppet and ubuntu server

amazon ec2amazon-cloudformationamazon-web-servicespuppet

I've looked over the Puppet Master Cloudformation Template and understand it is using CloudInit baked into the Amazon Linux AMI. However, I'd like to use an existing Ubuntu 11.10 server image and still bootstrap the server as a Puppet Master. Reading through the CloudFormation and BootStrapping documentation it would seem this is possible but no adequate logs are generated on the server or when the stack is created. Short of creating a custom AMI is there a way to do this with an existing AMI?

Best Answer

I decided to go the custom script route since I couldn't get the MetaData resource to install and configure packages with apt (see: https://s3.amazonaws.com/cloudformation-examples/BoostrappingApplicationsWithAWSCloudFormation.pdf).

The custom bash script does the install work to bootstrap the puppet master instance. I needed it to be in JSON format so I used this handy Ruby script from http://allanfeid.com/content/using-amazons-cloudformation-cloud-init-chef-and-fog-automate-infrastructure:

#!/usr/bin/ruby
require 'rubygems'
require 'json'

if ARGV.size != 1
  puts "Usage: #{$0} <file>"
  exit 1
end

def escape(string)
  parse = JSON.parse({ 'json' => string }.to_json)
  parse['json']
end

data = ''
File.open(ARGV[0]) {|f| data << f.read}
p escape(data)

# ./json_encode.rb combined-userdata.txt

This should work for now, but it would be nice to eventually manage the bootstrap procces as intended with cfn-init MetaData and apt. For example, this still doesn't work for me with apt:

"Resources": {
  "MyInstance": {
    "Type": "AWS::EC2::Instance",
"Metadata" : {
      "AWS::CloudFormation::Init" : {
        "config" : {
          "sources" : {
            :
          },
"packages" : {
            :
          }
"files" : {
            :
          }
"services" : {
            :
          }
        }
      }
    },
"Properties": {
      :
    }
  }
}

UPDATE

An easier solution is to also use Ubuntu CloudInit (https://help.ubuntu.com/community/CloudInit) and have it install the packages for you. My template for EC2 Puppet Client instances looks like this now:

"webserver" : {
   "Type" : "AWS::EC2::Instance",
   "Properties" : {
      "KeyName" : { "Ref" : "KeyName" },
      "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
      "InstanceType": { "Ref" : "InstanceType" },               
      "SecurityGroups" : [ { "Ref" : "PuppetGroup" }, { "Ref" : "WebServerGroup" } ],
      "UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
      "#cloud-config","\n",
      "packages:","\n",
        "- ruby","\n",
        "- ruby-dev","\n",
        "- rubygems","\n",
        "- puppet","\n",
        "- vim-puppet","\n",
        "- puppet-el","\n",
        "puppet:","\n",
        "conf:","\n",
        "agent:","\n",
        "server: ",,"\n",
        "runcmd:","\n"
      ]]}}
   }
}

That's basically the CloudConfig boot strap process minus the runcmd magic that makes it work. I'm still trying to figure out how to get my puppet manifests down to the puppet master and assigned to nodes but that should help anyone who's using an Ubuntu UEC AMI.

Related Topic