Where to Store Application Configuration

configurationconfiguration-management

Until now for each of our applications we store application configuration in the same repository as the code that uses the configuration. We have environment specific yml files (dev, test, .., prod) that contain plain text as well as encrypted configuration values. Those get deployed for example as "Application Settings" in Azure. Depending on the type of application (Java Tomcat, .NET webapp, etc.) The configuration values are fetched at runtime from environment variables or configuration files directly.

Now, we would like to decouple the release of the code from release of the configuration. A configuration change should not require us to rebuild and redeploy the application (process, approval, etc). This would be for patches for instance where the application version x.y will fetch the latest configuration for x.y.*.

The natural way to do this is to move the configuration values into a dedicated service that the applications will fetch them from, a REST API, backed by a database for instance. The question is where that external repository gets populated from, where the configuration lives from then on. Two options I see are keeping the configuration in the repository of the application it configures or having a "Configuration" repository holding the configuration of all applications.

I am a big proponent of keeping the code and configuration together in one repository. I just can't really quantify the upsides. Here are a few I can think of.

  • Keeping configuration isolated from other applications'. Other application's configuration changes won't affect each other. (Changing a seemingly unshared configuration)
  • Review of code and configuration at the same time in the same merge request.

I feel very strongly about this but I don't know why.

I don't see any downsides of keeping the configuration together with the code and no upsides of moving all configuration into the same repository.

Why is it better to store the configuration alongside the code?
Why is it better to store all configuration in the same repository?

Best Answer

The configuration values are fetched at runtime from environment variables or configuration files directly.

It sounds like you already have the possibility of decoupled deployments because you say that you pass in configuration at runtime. It sounds like your process is the thing that's getting in the way of decoupled config deployments. Configuration is only coupled to the software if it's part of the artifact at build time and cannot be changed at runtime. I think you're conflating repositories with deployments. Not everything that's in the same repository must be deployed together, and things that are in separate repositories can be deployed together.

This would be for patches for instance where the application version x.y will fetch the latest configuration for x.y.*.

What do you mean when you say application version x.y will fetch configuration for x.y.*? The application shouldn't be fetching config. The config should be passed in by whatever starts the application (through environment variables or command line arguments).

I think it's pretty important to keep the "developer's" config with the code. Your application code should have a config that is appropriate for a developer to clone the repository, build, and run the software on their local system. I think the dev, test, and prod configs should not be in the same repository for a couple reasons:

  1. Security - Even though you've encrypted sensitive information in the config, it's still not a great idea to keep this in VCS with the code. E.g. you wouldn't want to keep it in an open-source repository. Such a practice also encourages the key to be shared among developers so that anybody on the team can update the config when it changes.
  2. Flexibility - Your codebase should be designed to run with any configuration, it shouldn't try to predict how a specific production deployment will be configured. There can be infinite variations of the config, why should your code repository contain specific ones? If you want to create more environments you'll end up with a proliferation of config files.
  3. Separation of Responsibility - In many companies the personnel who write the software are not the same people who set up the infrastructure it will run on. I think it's easier for the team that manages the infrastructure and deployment of software to manage the configuration of the software. Instead I think the software engineers should create and maintain the description of the configuration and the operations team should manage which hosts/databases/passwords etc. get filled in.

The natural way to do this is to move the configuration values into a dedicated service that the applications will fetch them from, a REST API, backed by a database for instance

I don't see this as much of an improvement of your current setup. Indeed it is much more complex. Now you have to configure your original system to talk to the configuration service. Also running the application locally would now require running an additional service. In the spirit of dependency-injection, software should not configure itself--configuration should be passed in. If you were to develop a service like you're describing it would be better to fetch the configuration from the service then start the application with the fetched config instead of the application fetching its own config.

I am a big proponent of keeping the code and configuration together in one repository. I just can't really quantify the upsides. Here are a few I can think of.

  • Keeping configuration isolated from other applications'. Other application's configuration changes won't affect each other. (Changing a seemingly unshared configuration)

There is nothing stopping you from keeping applications' configurations separate from each other and certainly deploying one configuration shouldn't necessitate redeploying all configurations.

  • Review of code and configuration at the same time in the same merge request.

I think this is odd. Does this mean before the code is reviewed, you must decide the production configuration? If my change requires a new database, do I have to go to the infrastructure team, create all databases for all environments, fill in the hostnames, usernames, and passwords? Again if the description of the configuration is in the source repository then that can be reviewed without dealing with the production configuration.

Related Topic