Way to create AWS autoscaling lifecycle hooks with CloudFormation without a race condition

amazon ec2amazon-cloudformationamazon-web-services

I am trying to use AWS autoscaling lifecycle hooks in a template that encapsulates the following things:

  1. AWS::AutoScaling::AutoScalingGroup with associated scale up/down policies, launch configuration, IAM role, etc.
  2. 2 of AWS::AutoScaling::LifecycleHook for EC2 launching/terminating events.
  3. AWS::SQS::Queue (in a simplified example) where lifecycle notifications get posted.
  4. AWS::IAM::Role role for the autoscaling group to post notifications to the SQS queue.

When an ASG is launched, the queue ends up with two test notifications from the creation of lifecycle hooks, but no notifications for instance launch.

And here is the race condition.

AWS::AutoScaling::LifecycleHook object references AWS::AutoScaling::AutoScalingGroup (and hence depends on it). This dictates the order in which CloudFormation creates resources (the group is created first).

The problem is that the group starts launching instances before the hook creation is complete (instance launch is not a part of the template, so it starts executing in parallel). By the time the hook is created, there are no more events to post as the instances were already created.

Is there any way to work around it and catch launch events at stack launch time?

Best Answer

It's not an ideal solution, but would a two-pass stack creation be a valid workaround?

  1. Set DesiredCapacity property in the AutoScalingGroup resource to 0 on initial stack creation. This allows the LaunchConfiguration, AutoScalingGroup, and LifecycleHook resources to be created without actually launching any instances.
  2. Set DesiredCapacity to your desired count (> 0) on a subsequent stack update. This should launch your desired instances after the LifecycleHook has been created.