Can I use same state file in terraform for multiple terraform files

terraform

Suppose in project directory I have 2 .tf files and I want to have same state file for both .tf file. Now if execute plan for one .tf file my state file got updated but when I am trying to execute plan for other .tf file my state file got updated with latest file and it destroy the previous.tf files resources

Best Answer

Its not really clear what you're trying to accomplish - perhaps you could clarify?

If you have multiple files in directory, Terraform should plan and apply both of them against the same state file (using the Backend declaration). As the doc says:

"Terraform loads all configuration files within the directory specified in alphabetical order. The configuration within the loaded files are appended to each other. This is in contrast to being merged."

If you wanted to keep the .tf files separate, you could run Terraform from different directories (thus creating multiple state files) or alternatively, use Workspaces.

You can have multiple workspaces tied to a single backend (or state). This allows multiple distinct instances of that configuration to be deployed. These previously were referred to as "environments" but this was renamed in 0.10.

From the document:

Named workspaces allow conveniently switching between multiple instances of a single configuration within its single backend. A common use for multiple workspaces is to create a parallel, distinct copy of a set of infrastructure in order to test a set of changes before modifying the main production infrastructure.

To use a workspace, you would run the command like this:

$ terraform workspace new bar
Created and switched to workspace "bar"!

You're now on a new, empty workspace. Workspaces isolate their state,
so if you run "terraform plan" Terraform will not see any existing state
for this configuration.

Then you can reference workspace interpolation within your resources to dynamically alter what is deployed:

resource "aws_instance" "example" {
  count = "${terraform.workspace == "default" ? 5 : 1}"

  # ... other arguments
}

Related Solutions

How to identify libvirt_volume id in terraform for reuse in split configuration files

After further investigation I found the answer to my own question:

The base_volume_id has to be used with the full path to the target volume (default pool has the path: /var/lib/libvirt/images).

The state file contained the component name separated with '/'. This caused an issue to write the state file on the cloud storage. I fi

Terraform – Choosing Credentials for a Remote State File

Your first question

if my provider setting is explicitly declaring the credentials to use inside the Terraform source code, why does my OS-level AWS configuration matter at all?

The error message "Failed to load backend: Error configuring the backend "s3"" is referring to your Backend S3 configuration.

Look in the file ./.terraform/terraform.tfstate and you will see the S3 Backend configuration.

The Terraform S3 Backend is different than the Terraform AWS Provider. The error message "No valid credential sources found for AWS Provider." is misleading. It implies that the AWS Provider configuration is used, which is false. S3 Backend credentials are configured separately and stored in the terraform.tfstate file.

Your OS-level AWS configuration matters because if no S3 Backend credentials are specified, as documented here https://www.terraform.io/docs/backends/types/s3.html, then Terraform defaults to using the following, in order:

Environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
AWS Shared credentials file, default value is "~/.aws/credentials".

You didn't specify any credentials in your S3 Backend config so terraform is defaulting to the AWS Shared Credentials File.

Your S3 Backend configuration contains no credentials.

terraform {
  backend "s3" {
    bucket = "example_tf_states"
    key    = "global/vpc/us_east_1/example_state.tfstate"
    encrypt = true
    region     = "us-east-1"
  }
}

Your second question,

How can I make Terraform only use the creds defined in my Terraform configuration and ignore what's in my OS user profile?

First, Backends cannot contain interpolation, see https://www.terraform.io/docs/backends/config.html. So you cannot use any variables in the Backend config. e.g. this config is invalid

terraform {
  backend "s3" {
    bucket = "example_tf_states"
    key    = "global/vpc/us_east_1/example_state.tfstate"
    encrypt = true
    region     = "us-east-1"
    access_key = ${var.access_key}
    secret_key = ${var.secret_key}
  }
}

If you want to specify AWS credentials when running terraform init you specify backend configuration as options.

terraform init --backend-config="access_key=your_access_key" --backend-config="secret_key=your_secret_key"

This produces a S3 Backend config that looks like this, stored in the ./.terraform/terraform.tfstate file:

{
    "version": 3,
    "serial": 1,
    "lineage": "bd737d2d-1181-ed64-db57-467d14d2155a",
    "backend": {
        "type": "s3",
        "config": {
            "access_key": "your_access_key",
            "secret_key": "your_secret_key"
        },
        "hash": 9345827190033900985
    },

Again, the S3 Backend credentials are configured separately from your AWS Provider credentials.

Re-run terraform init and specify the credentials on the command line as --backend-config options to fix your error.

Best Answer

Related Solutions

How to identify libvirt_volume id in terraform for reuse in split configuration files

Terraform – Choosing Credentials for a Remote State File

Related Topic