I have existing infrastructure in Terraform and have been using it for a while. Recently I had swapped the AWS credentials of my local laptop (the creds stored in ~/.aws/credentials
) and it stopped working until I re-set those credentials back.
The problem is that I'm declaring the creds in the Terraform source itself but it doesn't seem to be using them at all.
terraform {
backend "s3" {
bucket = "example_tf_states"
key = "global/vpc/us_east_1/example_state.tfstate"
encrypt = true
region = "us-east-1"
}
}
provider "aws" {
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
region = "${var.region}"
}
variable "access_key" {
default = "<hidden_for_stack_exchange_post>"
}
variable "secret_key" {
default = "<hidden_for_stack_exchange_post>"
}
variable "region" {
default = "us-east-1"
}
The access ID permissions are 100% good.
I am using the same account ID and secret key both for the aws configure
settings that go into ~/.aws/credentials
as I am in the above Terraform variable declarations.
Everything works fine as long as the creds are in ~/.aws/credentials
but as soon as the OS level credentials are gone (ie rm ~/.aws/credentials
) I get the following when trying to run Terraform operations, such as terraform plan
:
Failed to load backend:
Error configuring the backend "s3": No valid credential sources found for AWS Provider.
Please see https://terraform.io/docs/providers/aws/index.html for more information on
providing credentials for the AWS Provider
Please update the configuration in your Terraform files to fix this error.
If you'd like to update the configuration interactively without storing
the values in your configuration, run "terraform init".
If I re-populate the ~/.aws/credentials
by running aws configure
it will work fine again.
I'm not understanding — if my provider
setting is explicitly declaring the credentials to use inside the Terraform source code, why does my OS-level AWS configuration matter at all?
How can I make Terraform only use the creds defined in my Terraform configuration and ignore what's in my OS user profile?
Edit, it's Terraform v0.11.7
Edit: Please note that I'm trying to solve the issue on why the statically declared creds are not being utilized in the provider declaration. Not looking for alternative methods or workarounds. Thanks.
Best Answer
Your first question
The error message "Failed to load backend: Error configuring the backend "s3"" is referring to your Backend S3 configuration.
Look in the file
./.terraform/terraform.tfstate
and you will see the S3 Backend configuration.The Terraform S3 Backend is different than the Terraform AWS Provider. The error message "No valid credential sources found for AWS Provider." is misleading. It implies that the AWS Provider configuration is used, which is false. S3 Backend credentials are configured separately and stored in the
terraform.tfstate
file.Your OS-level AWS configuration matters because if no S3 Backend credentials are specified, as documented here https://www.terraform.io/docs/backends/types/s3.html, then Terraform defaults to using the following, in order:
You didn't specify any credentials in your S3 Backend config so terraform is defaulting to the AWS Shared Credentials File.
Your S3 Backend configuration contains no credentials.
Your second question,
First, Backends cannot contain interpolation, see https://www.terraform.io/docs/backends/config.html. So you cannot use any variables in the Backend config. e.g. this config is invalid
If you want to specify AWS credentials when running
terraform init
you specify backend configuration as options.terraform init --backend-config="access_key=your_access_key" --backend-config="secret_key=your_secret_key"
This produces a S3 Backend config that looks like this, stored in the
./.terraform/terraform.tfstate
file:Again, the S3 Backend credentials are configured separately from your AWS Provider credentials.
Re-run
terraform init
and specify the credentials on the command line as--backend-config
options to fix your error.