Your first question
if my provider setting is explicitly declaring the credentials to use
inside the Terraform source code, why does my OS-level AWS
configuration matter at all?
The error message "Failed to load backend: Error configuring the backend "s3"" is referring to your Backend S3 configuration.
Look in the file ./.terraform/terraform.tfstate and you will see the S3 Backend configuration.
The Terraform S3 Backend is different than the Terraform AWS Provider. The error message "No valid credential sources found for AWS Provider." is misleading.
It implies that the AWS Provider configuration is used, which is false. S3 Backend credentials are configured separately and stored in the terraform.tfstate file.
Your OS-level AWS configuration matters because if no S3 Backend credentials are specified, as documented here https://www.terraform.io/docs/backends/types/s3.html, then Terraform defaults to using the following, in order:
- Environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
- AWS Shared credentials file, default value is "~/.aws/credentials".
You didn't specify any credentials in your S3 Backend config so terraform is defaulting to the AWS Shared Credentials File.
Your S3 Backend configuration contains no credentials.
terraform {
backend "s3" {
bucket = "example_tf_states"
key = "global/vpc/us_east_1/example_state.tfstate"
encrypt = true
region = "us-east-1"
}
}
Your second question,
How can I make Terraform only use the creds defined in my Terraform
configuration and ignore what's in my OS user profile?
First, Backends cannot contain interpolation, see https://www.terraform.io/docs/backends/config.html. So you cannot use any variables in the Backend config. e.g. this config is invalid
terraform {
backend "s3" {
bucket = "example_tf_states"
key = "global/vpc/us_east_1/example_state.tfstate"
encrypt = true
region = "us-east-1"
access_key = ${var.access_key}
secret_key = ${var.secret_key}
}
}
If you want to specify AWS credentials when running terraform init you specify backend configuration as options.
terraform init --backend-config="access_key=your_access_key" --backend-config="secret_key=your_secret_key"
This produces a S3 Backend config that looks like this, stored in the ./.terraform/terraform.tfstate file:
{
"version": 3,
"serial": 1,
"lineage": "bd737d2d-1181-ed64-db57-467d14d2155a",
"backend": {
"type": "s3",
"config": {
"access_key": "your_access_key",
"secret_key": "your_secret_key"
},
"hash": 9345827190033900985
},
Again, the S3 Backend credentials are configured separately from your AWS Provider credentials.
Re-run terraform init and specify the credentials on the command line as --backend-config options to fix your error.
Best Answer
I would try out https://github.com/dtan4/terraforming as it can actually dump resources to tfstates.
Another shortcut: grab all resources from aws-cli, run a terraform Import in These, followed by a terraform destroy.