AWS Elastic Beanstalk – Environment Variable Update Fails

The recommended and supported way to upgrade your AWS Beanstalk environment is documented here and managed platform updates are discussed here, honestly I'd stick to that if you want things to be easy (and that's what Beanstalk is all about), you'll theoretically only get the non-breaking updates and AWS will manage the process so there's no downtime.

So I just want to reiterate that managed platform updates are probably what you or anyone else coming here from Google will want, but if you want to know the latest AWS provided AMI for your Beanstalk environment it can be done fairly trivially with AWS CLI (thanks to sane naming conventions from Amazon on their AMIs).

Starting with an instance from your environment, describe the instance to get the current AMI (skip if you already know the current AMI).

aws ec2 describe-instances --instance-ids i-0909613f35ec0ffee --query 'Reservations[*].Instances[*].ImageId' --output text

ami-35290a56

Take the resulting AMI ID and describe it.

aws ec2 describe-images --image-ids ami-35290a56 --query 'Images[*][Architecture, Hypervisor, Name, RootDeviceType, VirtualizationType]' --output json

[
    [
        "x86_64",
        "xen",
        "aws-elasticbeanstalk-amzn-2016.03.0.x86_64-python34-hvm-201603290718",
        "ebs",
        "hvm"
    ]
]

We can use the output of the above as input to a new, sorted describe-images but this time we replace the timestamps with * wildcard symbols, like so:

aws ec2 describe-images --filters 'Name=architecture,Values=x86_64' 'Name=virtualization-type,Values=hvm' 'Name=owner-alias,Values=amazon' 'Name=name,Values=aws-elasticbeanstalk-amzn-*.x86_64-python34-hvm-*' --query 'sort_by(Images[*], &Name)[-1].ImageId' --output text

ami-1be5de78

Due to the power of lexical sorting and ISO 8601, we end up with the latest AMI, which in my example is ami-1be5de78.

aws ec2 describe-images --image-ids ami-1be5de78 --query 'Images[*].Name' --output text

aws-elasticbeanstalk-amzn-2016.09.0.x86_64-python34-hvm-201612200708

Again, I wouldn't recommend you try to change to this AMI by hand, Beanstalk has provisions to do all this for you!

I've evaluated Elastic Beanstalk in addition to other AWS offerings while trying to improve our hand-rolled AWS instances. The reasons I chose not to use it were due to complications that would arrise migrating my existing application and not with the offering itself. The catch is that you don't have as much control about application deployment / configuration of the servers. If you are starting a new application then it may be helpful to not deal with those things right now, if you have an existing application then it is more of a challenge to fit in the Beanstalk model.

Beanstalk provides a similar offering to Heroku and other PaaS vendors but not much of a benefit to those who just want to focus on making their application. You do at least get to determine the virtualized resources to a greater degree than other PaaS vendors.

Problems that I was running into with my application(s):

• Git based deployments - I love them but our repo is 1+ GB. Rather large to push on a regular basis. This repo also contains about 40 applications (which really should be split) but that would take some time. Uploading any sort of package could work but most of our applications would take a large amount of work to make it into a package.

• Integration with other services - From what I've seen Beanstalk sort of assumes that anything you are connecting to is a single service. This works well if your services are behind and ELB but our were separate nodes that we hit through HAProxy running on each application server. If you are running your datastores and other services as a single endpoint, you should be fine.

In my evaluation I also included OpsWorks and CloudFormation. OpsWorks has similar integration issues with how existing automation was working for these applications. CloudFormation didn't do much more than what some Python scripts and Chef were already taking care of for us.

I wound up choosing to use AWS Autoscaling Groups instead with some automation provided by Asgard. This was the smallest change to existing configuration/application code and provided us with the benefits we were looking for, simple management of multiple servers available through the AWS API.

The restrictions put in place by Elastic Beanstalk on your application are very helpful. You will have to make sure that your application is mostly stateless, provides an endpoint for a service, and relies on other services for state. If you are trying to make reusable stand-alone services that multiple applications in Beanstalk is a great start.

If/when you get to the point of wanting more configuration OpsWorks is a great next step. The predefined roles should make the transition easy to start with and it provides an automation framework around Chef to help coordinate provisioning of multiple servers.