Specify root volume size for ec2 instance in cloudformation

amazon-cloudformationamazon-web-services

Inherited an amazon hosted infrastructure consisting of a load balancer, a DB server, and several webservers inside an autoscaling group; with deployment coordinated by CloudFormation.

Problem is, the spun webservers have a puny root volume (8gb), and sometimes during the webserver lifetime the disk gets filled with logs and temporary files and some services stop working.

I found the part where the machine definition is declared (I think):

...
"Properties": {
            "ImageId": {
                "Fn::FindInMap": [
                    "AWSRegionArch2AMI",
                    {
                        "Ref": "AWS::Region"
                    },
                    {
                        "Fn::FindInMap": [
                            "AWSInstanceType2Arch",
                            {
                                "Ref": "InstanceType"
                            },
                            "Arch"
                        ]
                    }
                ]
            },
            "InstanceType": {
                "Ref": "InstanceType"
            },
            "SecurityGroups": [
                {
                    "Ref": "WebServerSecurityGroup"
                }
            ],
            "KeyName": {
                "Ref": "KeyName"
            },
            "UserData": {
                "Fn::Base64": {
                    "Fn::Join": [
...

Instance type is ultimatetly defined somewhere else:

"InstanceType": {
  "Description": "WebServer EC2 instance type",
  "Type": "String",
  "Default": "c4.xlarge",
  "AllowedValues": [
    "t1.micro",
    "t2.nano",
     # ... more allowed values
    "cg1.4xlarge"
  ],
  "ConstraintDescription": "must be a valid EC2 instance type."
},

But I have no clue how to declare the root volume size, nor if it possible at all; and I can't see anywhere an ebs volume being declared.

This is a linux machine, and the root device is /dev/xvda1

[ec2-user@ip- /]$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      7.8G  1.6G  6.2G  21% /
devtmpfs        3.7G   60K  3.7G   1% /dev
tmpfs           3.7G     0  3.7G   0% /dev/shm

Tried the solution suggested by @Jason (although replacing sda1 with xvda1) but CloudFormation wont spin up any new machines after doing that.

I can see that the launch configuration for the machine is updated accordingly after I upload the new template with these changes:

But new machines won't spin up automatically any more. 🙁

Best Answer

I had been stuck for days to change root-disk size using Cloudformation, , this is how it works for me..

If you use /dev/xvda1 the instance will fail, the correct way is to use /dev/xvda.

Using YAML format:

BlockDeviceMappings:
- DeviceName: "/dev/xvda"
   Ebs:
      VolumeSize: 15

Related Solutions

CloudFormation/EC2 – access IP address of EC2 instance in the template

After some digging in the AWS docs, i've found that you can reference the properties of resources created in the same template, as long as they a) are defined before your reference b) don't point to the resource being created ( no circular dependencies )

b) implies that to find out an EC2 node's own IP i indeed have to call the aws URL ceejayoz mentions above

This is my definition of an Instance resource for a Cassandra node. Note that i have placed markers in the configuration file to be able to safely insert correct values, but you could always revert to simply replacing the default values ( e.g. "initial_token: 0" with "initial_token: 123" )

    "Cas1" : {

        "Type" : "AWS::EC2::Instance",
        "Metadata" : {
            "AWS::CloudFormation::Init" : {
                "config" : {
                    "packages" : {
                        "yum" : {
                            "opscenter-free" : [],
                            "pyOpenSSL" : []
                        }
                    }
                },

                "services" : {
                    "sysvinit" : {
                        "cassandra" : {
                            "enabled" : "true",
                            "ensureRunning" : "true"
                        },
                        "opscenterd" : {
                            "enabled" : "true",
                            "ensureRunning" : "true"
                        }
                    }
                }
            }
        },

        "Properties" : {
            "SecurityGroups" : [ { "Ref" : "CommonSecurityGroup" }, { "Ref" : "OpsCenterSecurityGroup" } ],
            "KeyName" : { "Ref" : "KeyName" },
            "InstanceType" : { "Ref" : "CassandraInstanceType" },
            "ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" },{ "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "CassandraInstanceType" }, "Arch" ] } ] },
            "UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
            "#!/bin/bash -v\n",

            "MY_IP=`(curl http://169.254.169.254/latest/meta-data/local-ipv4)`\n",
            "sed -i 's/REPLACE_WITH_TOKEN/0/g' /etc/cassandra/conf/cassandra.yaml\n",
            "sed -i 's/REPLACE_WITH_MY_IP/'$MY_IP'/g' /etc/cassandra/conf/cassandra.yaml\n",
            "sed -i 's/REPLACE_WITH_SEED_IP/'$MY_IP'/g' /etc/cassandra/conf/cassandra.yaml\n",
            "sed -i 's/REPLACE_WITH_MAX_HEAP_SIZE/8G/g' /etc/cassandra/conf/cassandra-env.sh\n",
            "sed -i 's/REPLACE_WITH_HEAP_NEWSIZE/4G/g' /etc/cassandra/conf/cassandra-env.sh\n",

            "yum update -y aws-cfn-bootstrap\n",

            "/opt/aws/bin/cfn-init -s ", { "Ref" : "AWS::StackName" }, " -r Cas1 ",
            "    --access-key ",  { "Ref" : "HostKeys" },
            "    --secret-key ", {"Fn::GetAtt": ["HostKeys", "SecretAccessKey"]},
            "    --region ", { "Ref" : "AWS::Region" }, " || error_exit 'Failed to run cfn-init'\n",

            "sed -i 's/127.0.0.1/'$MY_IP'/g' /etc/opscenter/opscenterd.conf\n",
            "sed -i 's/#passwd_file/passwd_file/g' /etc/opscenter/opscenterd.conf\n",
            "echo 'xxx:xxx:admin' > /etc/opscenter/.passwd\n",

            "rm -fR /var/lib/cassandra/data\n",
            "rm -fR /var/lib/cassandra/commitlog\n",
            "mkdir /var/lib/cassandra/data\n",
            "mkdir /var/lib/cassandra/commitlog\n",
            "chown -R cassandra:cassandra /var/lib/cassandra\n",

            "service cassandra start\n",
            "service opscenterd start\n"

            ]]}}
        }

    },

Configuration of subsequent nodes are the same, except the line

            "sed -i 's/REPLACE_WITH_SEED_IP/'$MY_IP'/g' /etc/cassandra/conf/cassandra.yaml\n",

References the IP address of the seed ( first ) node

            "sed -i 's/REPLACE_WITH_SEED_IP/", { "Fn::GetAtt" : ["Cas1", "PrivateIp"] }, "/g' /etc/cassandra/conf/cassandra.yaml\n",

With regards to adding the instance IPs to security groups, this seems unnecessary, you can just create a group and add Ingress resources to it in which you reference the group as the Source

    "CommonSecurityGroup" : {
        "Type" : "AWS::EC2::SecurityGroup",
        "Properties" : {
            "GroupDescription" : "Enables",
            "SecurityGroupIngress" : [ {
                "IpProtocol" : "tcp",
                "FromPort" : "22",
                "ToPort" : "22",
                "CidrIp" : "0.0.0.0/0"
            } ]
        }
    },

    "OpsCenterSecurityGroup" : {
        "Type" : "AWS::EC2::SecurityGroup",
        "Properties" : {
            "GroupDescription" : "Enables",
            "SecurityGroupIngress" : [ {
                "IpProtocol" : "tcp",
                "FromPort" : "8888",
                "ToPort" : "8888",
                "CidrIp" : "0.0.0.0/0"
            },
            "SecurityGroupIngress" : [ {
                "IpProtocol" : "tcp",
                "FromPort" : "61620",
                "ToPort" : "61621",
                "SourceSecurityGroupName" : { "Ref": "CommonSecurityGroup" }
            } ]
        }
    },

    "CassandraThriftIngress" : {
        "Type" : "AWS::EC2::SecurityGroupIngress",
        "Properties" : {
            "GroupName": { "Ref": "CommonSecurityGroup" },
            "IpProtocol" : "tcp",
            "FromPort" : "9160",
            "ToPort" : "9160",
            "SourceSecurityGroupName" : { "Ref": "CommonSecurityGroup" }
        }
    },

    "CassandraData" : {
        "Type" : "AWS::EC2::SecurityGroupIngress",
        "Properties" : {
            "GroupName": { "Ref": "CommonSecurityGroup" },
            "IpProtocol" : "tcp",
            "FromPort" : "7000",
            "ToPort" : "7001",
            "SourceSecurityGroupName" : { "Ref": "CommonSecurityGroup" }
        }
    },

    "CassandraJMX" : {
        "Type" : "AWS::EC2::SecurityGroupIngress",
        "Properties" : {
            "GroupName": { "Ref": "CommonSecurityGroup" },
            "IpProtocol" : "tcp",
            "FromPort" : "7199",
            "ToPort" : "7199",
            "SourceSecurityGroupName" : { "Ref": "OpsCenterSecurityGroup" }
        }
    },

Specifying a VPC in a CloudFormation template for an EC2 instance

For EC2 Instances, you get to skip specifying the VPC ID and instead, just specify the subnet you want the instance in. From there, the VPC is assumed.

Inside of your "Properties" array, add the following:

"SubnetId" : "subnet-XXXXXXXX"

Best Answer

Related Solutions

CloudFormation/EC2 – access IP address of EC2 instance in the template

Specifying a VPC in a CloudFormation template for an EC2 instance

Related Topic