I am trying to configure an autoscaling setup in AWS where the Node Launch Template includes encrypting the root volume (EBS). I have configured a service linked role, and a CMK in Amazon KMS with an IAM policy as per the documentation.
However, I am getting the following error when the ASG looks to create the instances:
Launching a new EC2 instance: i-0123456789xxx. Status Reason: Instance became unhealthy while waiting for instance to be in InService state. Termination Reason: Client.InternalError: Client error on launch
The troubleshooting documentation simply points at the original documentation and suggests the IAM policy isn't configured properly – I'm struggling to work out what's not right though.
The service linked role is configured on the ASG: SLR on ASG, and the SLR has the correct permissions in the IAM policy for the key that was used to encrypt the volume:
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::0123456789:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
},
"Action": [
"kms: Encrypt",
"kms: Decrypt",
"kms: ReEncrypt*",
"kms: GenerateDataKey*",
"kms: DescribeKey"
],
"Resource": "*"
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::0123456789:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
},
"Action": [
"kms: CreateGrant",
"kms: ListGrants",
"kms: RevokeGrant"
],
"Resource": "*",
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
}
Note that launching the same AMI manually, specifying the root volume is encrypted with the same key, works. This suggests a problem with the SLR perhaps?
Or, do I need to create an AMI where the root volume is already encrypted?
UPDATE 11/05/2020:
Turns out there was a formatting error – there is a space after each colon in the Actions section. Removing that has fixed it and it works as expected now.
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::0123456789:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::0123456789:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "*",
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
}
Best Answer
I had the same problem and resolved it by adding the Service-Linked Role for Auto Scaling to the Key policy of the pertinent key (AWS Console -> KMS -> Customer managed keys -> YOUR_KEY -> 'edit' under the Key policy tab) as follows: