Add ElastiCache Cluster to Elastic Beanstalk Application

amazon-elasticacheamazon-vpcamazon-web-serviceselastic-beanstalk

I have an Elastic Beanstalk app setup utilizing the docker platform. Everything runs smoothly and I can redeploy the app using the eb deploy cli command. The app is setup on a custom VPC (something I barely understand) and is accessible via the web.

I want to add a Memcached ElastiCache cluster to my app. I tried creating the cluster manually via the console, but I can't ever seem to connect to it (no matter how I set it up) from my app (I've tried telnetting from the Elastic Beanstalk instance). I have added a .ebextensions directory to my app and have tried adding a directive for EB to create the cache cluster for me. I have tried using the AWS example template, but that didn't work. I have tried changing settings and have even tried simplifying the directive down to the very minimum required parameters to get EB to create the cluster. Nothing I am doing seems to work. Below is my current file:

Resources:
  MyElastiCache:
    Type: "AWS::ElastiCache::CacheCluster"
    Properties:
      CacheNodeType: "cache.t2.small"
      NumCacheNodes: "1"
      Engine: "memcached"
      VpcSecurityGroupIds: 
        - sg-XXXXXXXX
      CacheSubnetGroupName: "subnet-XXXXXXXX"

For the VpcSecurityGroupIds, I have tried both EC2 security groups and VPC security groups. Neither seem to help. The CacheSubnetGroupName is the id of a subnet within my VPC.

It is very frustrating because it takes 5 minutes to test the most minimal change as EB has to redeploy my app every time I try something new. Any help would be greatly appreciated.

Edit: Here is my latest attempt.

Resources:
  MyElastiCacheSubnetGroup:
    Type: "AWS::ElastiCache::SubnetGroup"
    Properties:
      Description: "Blah blah blah"
      SubnetIds:
        - subnet-XXXXXXXX
        - subnet-XXXXXXXX
        - subnet-XXXXXXXX
        - subnet-XXXXXXXX
  MyElastiCache:
    Type: "AWS::ElastiCache::CacheCluster"
    Properties:
      CacheNodeType: "cache.t2.small"
      NumCacheNodes: "1"
      Engine: "memcached"
      VpcSecurityGroupIds: 
        - sg-XXXXXXXX
      CacheSubnetGroupName:
        Ref: "MyElastiCacheSubnetGroup"

And the error to go with it: Service:AmazonCloudFormation, Message:Stack named 'awseb-e-amyvnbtvps-stack' aborted operation. Current state: 'UPDATE_ROLLBACK_IN_PROGRESS' Reason: The following resource(s) failed to create: [MyElastiCacheSubnetGroup].

Best Answer

The VpcSecurityGroupIds property is a list of one or more VPC security groups that will be assigned to the cache cluster.

Make sure one of those security groups is allowing connections on port 11211 from either your EB application's EC2 instance's security groups, or a CIDR that includes your EB application's EC2 instances. Without this rule, any connections will fail.

The CacheSubnetGroupName is not a VPC subnet.

You need to create a Cache Subnet Group before you create your Cache Cluster in a VPC. A Cache Subnet Group is a collection of VPC subnets. See the following for more information:

http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/SubnetGroups.html

You can create the Cache Subnet Group resource in your .ebextensions file:

Resources:
  MyElastiCacheSubnetGroup:
    Type: "AWS::ElastiCache::SubnetGroup"
    Properties:
      Description: "Something informational"
      SubnetIds:
        - subnet-XXXXXX
  MyElastiCache:
    Type: "AWS::ElastiCache::CacheCluster"
    Properties:
      CacheNodeType: "cache.t2.small"
      NumCacheNodes: "1"
      Engine: "memcached"
      VpcSecurityGroupIds: 
        - sg-XXXXXXXX
      CacheSubnetGroupName: 
        Ref: "MyElastiCacheSubnetGroup"
Related Topic