Providers and State in Terraform

Gashida T101 Study Week 4 Learning

What is a Provider?

A Provider in Terraform is essentially a plugin that allows Terraform to interact with and manage resources in various cloud services, SaaS platforms, and other APIs. It acts as a bridge between Terraform and the service’s API, translating Terraform configurations into API calls to create, update, or delete resources.

Provider Tiers

Terraform categorizes providers into three main tiers:

Official: These are plugins developed and maintained by the Terraform team at HashiCorp. They are considered the most reliable and well-supported.

• Namespace: hashicorp
• Example: hashicorp/aws, hashicorp/google

Partner: These plugins are developed by companies that have partnered with Terraform. They typically provide management capabilities for the partner’s cloud or SaaS products. To become a Partner provider, the company must be part of the HashiCorp Technology Partner Program.

• Namespace: The organization name
• Example: mongodb/mongodbatlas

Community: These are plugins created and maintained by individuals or groups in the Terraform community. While they can be very useful, extra caution is advised when using them.

• Namespace: Individual or organization account name
• Example: DeviaVir/gsuite

There’s also an additional tier called Archived, which represents older versions of providers that are no longer maintained.

Basic Provider Configuration

Provider configuration is typically set up using the provider block in your Terraform configuration files. Here's an example of a basic provider configuration.

# main.tf
terraform {
  required_version = "~> 1.3.0"  # Specifies the required Terraform version
  
  required_providers {
    random = {
      version = "~> 3.0"
    }
    architech-http = {
      source  = "architect-team/http"
      version = "~> 3.0"
    }
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.56.1"
    }
  }
}

• The random provider is from the Official Tier, so the source attribute can be omitted.
• The architech-http provider is not from the Official Tier, so the source must be explicitly specified.
• The aws provider is from the Official Tier, but we've included the source for clarity.

You can assign custom local names to providers. While it’s generally recommended to use the default names for clarity, there might be situations where custom names are useful.

terraform {
  required_providers {
    my-custom-aws = {   # Custom local name for the AWS provider
      source  = "hashicorp/aws"
      version = "~> 5.56.1"
    }
  }
}

When you have multiple providers with similar names or functionalities, you can explicitly specify which provider to use for a particular resource or data source.

terraform {
  required_providers {
    architect-http = {
      source  = "architect-team/http"
      version = "~> 3.0"
    }
    http = {
      source  = "hashicorp/http"
    }
    aws-http = {
      source  = "terraform-aws-modules/http"
    }
  }
}

data "http" "example" {
  provider = aws-http
  url      = "https://checkpoint-api.hashicorp.com/v1/check/terraform"

  request_headers = {
    Accept = "application/json"
  }
}

In this example, we have three HTTP-related providers. By using provider = aws-http in the data block, we explicitly specify that this particular data source should use the aws-http provider.

Sometimes, you might need to use the same provider with different configurations. This is common when working with multiple regions or accounts (API keys) in cloud providers like AWS.

# Configure the default AWS provider
provider "aws" {
  region = "ap-southeast-1"
}

# Configure an additional AWS provider for Seoul region
provider "aws" {
  alias  = "seoul"
  region = "ap-northeast-2"
}

# Resource using the default provider (ap-southeast-1)
resource "aws_instance" "app_server1" {
  ami           = "ami-06b79cf2aee0d5c92"
  instance_type = "t2.micro"
}

# Resource using the Seoul provider
resource "aws_instance" "app_server2" {
  provider      = aws.seoul
  ami           = "ami-0ea4d4b8dc1e46212"
  instance_type = "t2.micro"
}

• The first aws provider is used as the default (no alias).
• The second aws provider has an alias "seoul" and is configured for a different region.
• Resources can specify which provider to use with the provider argument.
• After applying your Terraform configuration, you can use the Terraform CLI to verify that resources were created in the correct regions:

$ terraform apply

$ terraform state list

$ echo "aws_instance.app_server1.public_ip" | terraform console
# Output: "13.215.47.148"

$ echo "aws_instance.app_server1.availability_zone" | terraform console
# Output: "ap-southeast-1b"

$ echo "aws_instance.app_server2.public_ip" | terraform console
# Output: "43.203.243.238"

$ echo "aws_instance.app_server2.availability_zone" | terraform console
# Output: "ap-northeast-2a"

Defining Provider Requirements & Provider Installation

When executing Terraform, provider requirements can be defined in the terraform block using the required_providers block. This allows you to specify multiple providers and their details.

terraform {
  required_providers {
    <provider_local_name> = {
      source  = "[<hostname>/]<namespace>/<type>"
      version = "<version_constraint>"
    }
    # Additional providers can be defined here
  }
}

The source attribute specifies where the provider is hosted, its namespace, and type.

• Hostname: Optional. Defaults to registry.terraform.io if omitted.
• Namespace: Represents the organization publishing the provider in public registries or private registries in Terraform Cloud.
• Type: The name of the platform or service managed by the provider.

source = "hashicorp/aws"
source = "mycorp.com/myns/mytype"

The version attribute specifies version constraints for the provider. This follows semantic versioning rules.

version = "~> 3.0"  # Any version in the 3.x series, but at least 3.0

When you run terraform init, Terraform downloads the specified providers from the given source, adhering to the version constraints. To ensure consistent use of specific versions across your team or environments, you can…

1. Define versions in the terraform block
2. Share the .terraform.lock.hcl lock file in your code repository

It’s worth noting that providers defined in the required_providers block will be installed even if no resources from that provider are used in your configuration. If you don't use the required_providers block, Terraform will infer the required providers from the resources in your configuration and install the latest versions.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
    azurerm = {
      source  = "hashicorp/azurerm"
      version = ">= 2.26"
    }
    random = {
      source  = "hashicorp/random"
      version = "3.1.0"
    }
  }
}

# Provider configuration
provider "aws" {
  region = "us-west-2"
}

provider "azurerm" {
  features {}
}

# Resource using the aws provider
resource "aws_instance" "example" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
}

# Resource using the azurerm provider
resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "West Europe"
}

# Resource using the random provider
resource "random_pet" "server" {
  length = 2
}

Provider Interchangeability ?

While it might seem tempting to switch between providers for similar services (like AWS and GCP), it’s not a straightforward process. Each provider has its own resource names and configuration options, making direct substitution challenging.

However, there are similarities between providers for similar services, which can make the transition easier if needed. Understanding these similarities can help when moving between cloud platforms or services.

What is State in Terraform?

Terraform state is a crucial component that keeps track of the current state of your infrastructure managed by Terraform. It’s typically stored in a file named terraform.tfstate in JSON format.

Terraform’s state management helps maintain idempotency. When you run terraform plan or terraform apply, Terraform compares the desired state (defined in your .tf files) with the current state (stored in the state file). If there's a mismatch, Terraform calculates the necessary changes to bring the infrastructure in line with the desired state.

The state file is crucial for this process, as it contains the mapping between your Terraform resources and the real-world infrastructure. If the state file is lost or corrupted, Terraform loses track of the existing resources, potentially leading to duplicate resource creation or other issues.

How Terraform Synchronizes State

https://getcoal.medium.com/terraform-the-importance-of-the-state-file-bc95d6e94180

Terraform compares three elements to determine the necessary actions.

1. Configuration files (*.tf)
2. Existing state (terraform.tfstate)
3. Actual resource configuration in the cloud provider

By analyzing these elements, Terraform identifies changes and applies them accordingly. The process typically involves the following steps.

1. State comparison
2. Refresh
3. Plan
4. Apply

During the plan and apply phases, Terraform uses specific symbols to indicate the type of change for each resource.

• + create: A new resource will be created
• - destroy: An existing resource will be deleted
• -/+ replace: A resource will be deleted and then recreated (Note: You can use the lifecycle block with create_before_destroy = true to reverse this order)
• ~ update in-place: An existing resource will be modified without being replaced

Let’s explore various scenarios and how Terraform behaves in each.

Scenario 1: “Starting from scratch”

Imagine you’ve just written your Terraform configuration file, but you haven’t run Terraform yet. There’s no state file, and no actual resource exists in your cloud environment. When you run Terraform, it will create the resource for you.

Let’s start with a simple example of creating new IAM users.

locals {
  name = "sigrid_test"
}

resource "aws_iam_user" "sigrid_user1" {
  name = "${local.name}1"
}

resource "aws_iam_user" "sigrid_user2" {
  name = "${local.name}2"
}

When you run terraform apply for the first time, Terraform will create these resources.

$ terraform init && terraform plan && terraform apply -auto-approve
# ... output omitted ...
# Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Scenario 2: “Oops, something went wrong last time”

In this case, you have your configuration file, and Terraform has a state file (maybe from a previous attempt), but the actual resource doesn’t exist in your cloud environment. This could happen if a previous creation attempt failed midway. Terraform will try to create the resource again.

If you manually delete the resources outside of Terraform…

$ aws iam delete-user --user-name sigrid_test1
$ aws iam delete-user --user-name sigrid_test2

Terraform will detect the discrepancy and plan to recreate the resources.

$ terraform plan
# ... output omitted ...
# Plan: 2 to add, 0 to change, 0 to destroy.

Interestingly, if you use the -refresh=false option, Terraform won't detect the manual changes:

$ terraform plan -refresh=false
# No changes. Your infrastructure matches the configuration.

Scenario 3: “Everything’s in perfect harmony”

This is the ideal scenario. Your configuration file matches what’s in the state file, which matches what actually exists in your cloud environment. Terraform doesn’t need to do anything because everything is as it should be.

When the configuration, state, and actual resources are in sync, Terraform takes no action.

$ terraform apply -auto-approve
$ cat terraform.tfstate | jq .serial
# 8

$ terraform apply -auto-approve
$ cat terraform.tfstate | jq .serial
# 8

Notice that the serial number in the state file remains unchanged, indicating no modifications were made.

Scenario 4: “Time to clean up”

In this case, you’ve removed a resource from your configuration file, but it still exists in both the state file and the actual cloud environment. When you run Terraform, it will recognize that you no longer want this resource and will delete it for you.

If you remove a resource from your Terraform configuration, Terraform will plan to destroy that resource.

locals {
  name = "sigrid_test"
}

resource "aws_iam_user" "sigrid_user1" {
  name = "${local.name}1"
}

# sigrid_user2 has been removed

Applying this change will result in:

$ terraform apply -auto-approve
# ... output omitted ...
# Apply complete! Resources: 0 added, 0 changed, 1 destroyed.

Scenario 5: “Who put that there?”

This is an interesting one. There’s a resource in your cloud environment, but Terraform doesn’t know anything about it (it’s not in your configuration or state files). This could happen if someone manually created a resource outside of Terraform. In this case, Terraform will ignore it — it won’t create, modify, or delete anything.

If you accidentally delete the Terraform state file:

$ rm terraform.tfstate*

Terraform loses track of the managed resources:

$ terraform plan
# Plan: 1 to add, 0 to change, 0 to destroy.

Attempting to apply this plan will result in an error because the resources already exist:

$ terraform apply -auto-approve
# Error: creating IAM User (sigrid_test1): operation error IAM: CreateUser, ...

To recover from a deleted state file, you can use the terraform import command:

$ terraform import aws_iam_user.sigrid_user1 sigrid_test1
# aws_iam_user.sigrid_user1: Importing from ID "sigrid_test1"...
# ... output omitted ...
# Import successful!

This command reconnects the existing resource with Terraform’s state management. However, note that you need to import each resource individually, which can be cumbersome for large infrastructures.

When you’re dealing with an existing infrastructure that wasn’t originally managed by Terraform, manually importing each resource can be a tedious and time-consuming process. This is where Terraformer comes in handy. Terraformer is a tool developed by Google that allows for bulk importing of existing cloud resources into Terraform state.

1. Easy Installation: Terraformer can be easily installed using package managers like Homebrew.

brew install terraformer

2. Wide Resource Support: It supports a variety of cloud providers, including AWS, and can import multiple resource types in one go.

3. Bulk Import: Instead of importing resources one by one, you can import multiple resources of different types with a single command:

terraformer import aws - resources=vpc,subnet,route_table,igw,sg,nat - path-pattern="{output}/" - connect=true - regions=ap-northeast-2 - profile=aws-env-profile

4. Automatic Terraform File Generation: Terraformer not only imports the state but also generates the corresponding Terraform configuration files (.tf files) for the imported resources.

5. Flexible Output: You can specify how you want the output files to be organized using the --path-pattern option.

6. Region and Profile Support: It allows you to specify the AWS region and profile, making it easy to work with multiple AWS accounts.

While Terraformer significantly simplifies the import process, there are a few additional steps to ensure everything works smoothly:

1. You may need to install the appropriate AWS provider plugin for Terraform.
2. Some manual adjustments to the generated provider.tf file might be necessary.
3. You might need to run a provider replacement command:

terraform state replace-provider -auto-approve - -/aws hashicorp/aws

After these steps, you can run terraform init and terraform plan to verify that everything has been imported correctly.

This approach is particularly useful when you’re dealing with a large existing infrastructure or when you’re transitioning from manual management to Infrastructure as Code with Terraform. It saves a significant amount of time and reduces the likelihood of errors that might occur during manual imports.

Creating and Managing a VPC

// main.tf

provider "aws" {
  region = "ap-northeast-2"
}

resource "aws_vpc" "example_vpc" {
  cidr_block = "10.10.0.0/16"

  tags = {
    Name = "terraform-study"
  }
}

$ terraform init && terraform plan && terraform apply -auto-approve

$ cat terraform.tfstate
#    "serial": 1,
#    ...
#    "instances": [ 
#       { "id": "vpc-0123456789abcdef0",
#         "tags": { "Name": "terraform-study" }, 
#    ...

$ echo "aws_vpc.example_vpc.id" | terraform console
# Output: "vpc-0123456789abcdef0"

$ echo "aws_vpc.example_vpc.tags.Name" | terraform console
# Output: "terraform-study"

Changing VPC tags afterwards:

resource "aws_vpc" "example_vpc" {
  cidr_block = "10.10.0.0/16"

  tags = {
    Name = "updated-terraform-study"
  }
}

$ terraform apply -auto-approve

Now, let’s compare the state files:

$ diff terraform.tfstate terraform.tfstate.backup
4c4
<   "serial": 2,
---
>   "serial": 1,
39c39
<               "Name": "updated-terraform-study"
---
>               "Name": "terraform-study"

Challenges in Team Environments

Using local state files in a team setting presents several challenges. All team members need access to the latest state file; Preventing simultaneous applies by multiple team members. Additionally, there’s a need for separating state for different environments (dev, staging, production).

While version control systems (VCS) like Git seem like a solution, they introduce their own set of problems: 1) Forgetting to pull/push the latest state changes; 2) Incorrect merging of state files. 3) Git doesn’t provide state locking mechanisms. 4) Lastly, sensitive data in plain text accessible to anyone with git access.

To address these issues, popular remote backend options include AWS S3 with DynamoDB, Azure Blob Storage, Google Cloud Storage, and HashiCorp Consul sheds light on. For example, the configuration below stores the state file in an S3 bucket, encrypts it, and allows for state locking, when used with a DynamoDB table.

First, let’s create an S3 bucket to store our Terraform state:

git clone https://github.com/sungwook-practice/t101-study.git example
cd example/state/step3_remote_backend/s3_backend

# Modify the terraform.tfvars file to set the bucket name
# bucket_name = "<your-nickname>-tf1014-remote-backend"
# For example: bucket_name = "sigrid-tf1014-remote-backend"

# Create the S3 bucket
terraform init && terraform plan && terraform apply -auto-approve

# Verify the creation
terraform state list
aws s3 ls
# Expected output: 2024-07-XX 01:07:50 sigrid-tf1014-remote-backend

Modify the provider.tf file to set the backend to S3.

terraform {
  backend "s3" {
    bucket = "sigrid-tf1014-remote-backend"
    key    = "terraform/state-test/terraform.tfstate"
    region = "ap-northeast-2"
    # dynamodb_table = "terraform-lock" # We'll add this later
  }
}

terraform init
terraform apply -auto-approve

terraform state list

# Verify that no local state file exists
ls *.tfstate

# Check the tfstate file in the S3 bucket
MYBUCKET=sigrid-tf1014-remote-backend

aws s3 ls s3://$MYBUCKET --recursive --human-readable --summarize

At this point, we’ve confirmed that the state is stored in S3 instead of a local file. However, this setup doesn’t prevent concurrency issues. Let’s add DynamoDB to implement locking.

Terraform requires a DynamoDB table with a primary key named LockID to use for locking:

cd ../dynamodb
cat main.tf

# Create the DynamoDB table
terraform init && terraform plan && terraform apply -auto-approve

# Verify the creation
terraform state list
terraform state show aws_dynamodb_table.terraform_state_lock

# Check the DynamoDB table
aws dynamodb list-tables --output text
aws dynamodb describe-table --table-name terraform-lock | jq
aws dynamodb describe-table --table-name terraform-lock --output table

Modify the provider.tf file to include the dynamodb_table attribute.

terraform {
  backend "s3" {
    bucket         = "sigrid-tf1014-remote-backend"
    key            = "terraform/state-test/terraform.tfstate"
    region         = "ap-northeast-2"
    dynamodb_table = "terraform-lock"
  }
}

$ terraform init -migrate-state

To demonstrate the locking mechanism, let’s modify a resource and apply the changes.

resource "aws_vpc" "main" {
  cidr_block = var.vpc_cidr
    
  tags = {
    Name = "terraform 123" # Changed from "terraform VPC"
  }
}

terraform apply

# Don't enter "yes" immediately

While the apply command is waiting for confirmation, check the DynamoDB table in the AWS Console. You should see a lock entry similar to this:

{
  "ID":"d210a853-3361-527a-dc34-8f6b310ba77b",
  "Operation":"OperationTypeApply",
  "Info":"",
  "Who":"sigridjin",
  "Version":"1.8.5",
  "Created":"2024-07-XXT16:38:55.21022Z",
  "Path":"sigrid-tf1014-remote-backend/terraform/state-test/terraform.tfstate"
}

After confirming the apply operation, the lock will be released.

S3 versioning allows you to keep multiple variants of an object in the same bucket.

# Check S3 bucket contents
aws s3 ls s3://$MYBUCKET --recursive --human-readable --summarize

# Check versioned files
aws s3api list-object-versions --bucket $MYBUCKET | jq

You should see multiple versions of the state file, demonstrating that S3 is maintaining a history of changes.

# Delete VPC resources
terraform destroy -auto-approve

# Delete DynamoDB table
cd ../dynamodb
terraform destroy -auto-approve

# Delete S3 bucket
cd ../s3_backend
terraform destroy -auto-approve

# If the S3 bucket is not empty, use these commands to remove all objects and delete markers
aws s3api delete-objects \
    --bucket $MYBUCKET \
    --delete "$(aws s3api list-object-versions \
    --bucket "$MYBUCKET" \
    --output=json \
    --query='{Objects: Versions[].{Key:Key,VersionId:VersionId}}')"
aws s3api delete-objects --bucket $MYBUCKET \
    --delete "$(aws s3api list-object-versions --bucket "$MYBUCKET" \
    --query='{Objects: DeleteMarkers[].{Key:Key,VersionId:VersionId}}')"
# Now you can delete the S3 bucket
terraform destroy -auto-approve

Workspaces in Terraform

When building infrastructure, it’s often necessary to create separate environments for different stages of development, such as dev (development), stage (staging), and prod (production). Terraform offers two primary methods to manage these distinct environments: isolation via file layout and the use of Terraform workspaces.

One approach to separating environments is to use a specific file layout, organizing your Terraform configurations into different directories for each environment.

Here’s an example of how your project structure might look.

.
├── common
│   └── variables.tf
├── dev
│   ├── vpc
│   │   ├── main.tf
│   │   └── variables.tf
│   ├── main.tf
│   └── variables.tf
├── prod
│   ├── vpc
│   │   ├── main.tf
│   │   └── variables.tf
│   ├── main.tf
│   └── variables.tf
└── stage
    ├── vpc
    │   ├── main.tf
    │   └── variables.tf
    ├── main.tf
    └── variables.tf

This method allows for complete independence between environments, giving you the flexibility to configure each one as needed. However, it can lead to code duplication and become difficult to manage as the number of resources grows.

To reduce duplication, you can create common .tf files that are shared across environments. For instance, you might have a common/variables.tf file that defines variables used in all environments.

Isolation via Terraform Workspaces

Terraform workspaces provide a way to create quick, isolated environments within the same configuration. This feature is particularly useful when you want to maintain similar configurations across different environments.

Let’s walk through a practical example of using Terraform workspaces.

First, let’s check our current workspace:

$ terraform workspace list
* default

Create a main.tf file with the following content:

resource "aws_instance" "sigrid_srv1" {
  ami           = "ami-0ea4d4b8dc1e46212"
  instance_type = "t2.micro"
  tags = {
    Name = "t101-study"
  }
}

Initialize Terraform and apply the configuration:

$ terraform init && terraform plan && terraform apply -auto-approve

After applying, you can check the state and IP addresses:

$ terraform state list
$ cat terraform.tfstate | jq -r '.resources[0].instances[0].attributes.public_ip'
$ cat terraform.tfstate | jq -r '.resources[0].instances[0].attributes.private_ip'

Now, let’s create a new workspace called mywork1:

$ terraform workspace new mywork1
$ terraform workspace show
mywork1

You’ll notice a new subdirectory is created:

$ tree terraform.tfstate.d
terraform.tfstate.d
└── mywork1

When you run terraform plan in the new workspace, Terraform will create a plan to add a new resource, as this workspace has its own state file:

$ terraform plan
# ... Plan: 1 to add, 0 to change, 0 to destroy.

$ terraform apply -auto-approve

Now you can compare the resources in different workspaces:

$ terraform workspace list
  default
* mywork1

$ cat terraform.tfstate | jq -r '.resources[0].instances[0].attributes.public_ip'

$ cat terraform.tfstate.d/mywork1/terraform.tfstate | jq -r '.resources[0].instances[0].attributes.public_ip'

You can create additional workspaces as needed:

$ terraform workspace new mywork2
$ terraform plan && terraform apply -auto-approve

To clean up resources, select each workspace and destroy the resources:

$ terraform workspace select default
$ terraform destroy -auto-approve

$ terraform workspace select mywork1
$ terraform destroy -auto-approve

$ terraform workspace select mywork2
$ terraform destroy -auto-approve

Advantages of Terraform Workspaces

1. Manage different environments using the same root module and Terraform configuration.
2. Experiment with changes without affecting existing provisioned environments.
3. Manage different resource results from the same configuration, similar to Git branch strategies.

Disadvantages of Terraform Workspaces

1. State files are stored in the same repository (local or backend), making it impossible to manage state access permissions separately.
2. May require numerous conditional statements in Terraform configurations if environments don’t require identical resources.
3. Cannot completely separate authentication elements for provisioning targets.
4. The biggest drawback is the impossibility of perfect isolation.

• All workspaces within a Terraform configuration typically use the same backend to store state. This means that if you’re using a remote backend (like S3), all your workspaces’ state files are in the same bucket. This can potentially lead to security concerns, as anyone with access to this backend can theoretically access the state of all environments.
• Workspaces use the same Terraform configuration files. While you can use variables to differentiate between environments, the core configuration is shared. This can make it challenging to have significantly different setups between environments without complex conditional logic. It also make it difficult to have environment-specific variables without declaring all possible variables in the shared configuration.
• Terraform itself doesn’t provide a way to restrict access to specific workspaces. If someone has access to run Terraform commands, they potentially have access to all workspaces.

To address these limitations, consider:

1. Using directory-based layouts with separate root modules for each environment.
2. Utilizing Terraform Cloud workspaces, which provide more robust isolation and management features.