ARTICLE
The Lifecycle of a Terraform Resource
From Terraform in Action by Scott Winkler
This article discusses the lifecycle of a terraform resource.
__________________________________________________________________
Take 37% off Terraform in Action by entering fccwinkler into the discount code box at checkout at manning.com.
__________________________________________________________________
Process Overview
Imagine you’re a resource managed by Terraform. How did you come to be? What’s your destiny? And where do you go when you die? Although there’s a lot involved in the process, it boils down to lifecycle function hooks and how they’re handled in Terraform. For this scenario, we’re going to be using the local provider to provision a text file called art_of_war.txt. This file is going to contain the first couple stanzas of Sun Tzu’s masterpiece, “The Art of War”. If you’ve never read “The Art of War”, it’s a classic in military strategy and I highly recommend it. Our overall architecture is illustrated by Figure 1.
After we finish creating the resource (1), we’ll modify the configuration code and have Terraform update the contents of the text file accordingly (2 & 3). We’ll also simulate a configuration drift by altering the art_of_war.txt file out of band and have Terraform restore the file to what it should be (2 & 3 again).
Lastly, we’ll clean-up the resource with a terraform destroy and verify that the file is gone (4). Along the way we’ll observe how the state file changes over time, and how these changes affect the execution plans generated by Terraform. The steps we’ll follow are depicted in figure 2.
One last point to keep in mind as we work through the scenario is that all resources implement Terraform’s official resource schema. This resource schema is an interface which specifies, among other things, that all resources implementing the interface must define function hooks which are called when certain lifecycle events are triggered. Create(), Read(), Update() and Delete() are the major lifecycle function hooks, although there are a few other lesser used/more specialized ones. I’ll point out when Terraform invokes each of the major method hooks during the lifecycle of the “Art of War” file, as it’s important for gaining an understanding of what’s happening when Terraform manages your resource. A depiction of how function hooks are related to the local provider, and Terraform in general, is portrayed in figure 3.
Declaring a Local File Resource
Let’s get started by creating a new workspace for Terraform. This can be any folder on your computer, as long as it doesn’t contain Terraform configuration files. In this folder, make a new file called main.tf and insert the following code from Listing 1 in.
TIP: The <<- sequence indicates an indented heredoc string. Anything between the opening identifier and the closing identifier (EOT) is interpreted literally. Leading whitespace is ignored (unlike traditional heredoc syntax).
Listing 1 Contents of main.tf
terraform { #A
required_version = "~> 0.12"
required_providers {
local = "~> 1.2"
}
}
resource "local_file" "literature" { #B
content = <<-EOT #C
Sun Tzu said: The art of war is of vital importance to the State.
It is a matter of life and death, a road either to safety or to
ruin. Hence it is a subject of inquiry which can on no account be
neglected.
EOT
filename = "art_of_war.txt" #D
}#A Configuring the behavior of Terraform itself
#B Declaring a local file resource with name “literature”
#C Using heredoc syntax to set the attribute value to a multi-line string
#D The name of the file after it’s created
Two configuration blocks are in Listing 1. The first block: terraform {…} is a special configuration block for configuring Terraform itself. Its primarily use is for enforcing dependency requirements before the code can be run; in this particular case it specifies a fuzzy equals relationship on Terraform v0.12 and local provider v1.2. This means that as long as you aren’t using Terraform v1.0 or local provider v2.0 (both of which could cause breaking changes), then you’re good. We don’t have the provider installed yet, but we will after the initialization process is complete.
The second block is a resource block for the local_file resource, which is defined by the local provider, and allows us to manage text files locally. You can learn more about this resource and what other arguments it accepts by looking up the corresponding provider documentation, but it’s able to create a file with a given content and filename. In this scenario, the content contains the first couple stanzas of Sun Tzu’s masterpiece “The Art of War” and the file name is art_of_war.txt.
Notice also the use of heredoc syntax for setting the content attribute to a multi-line string literal. It’s more convenient to do this than writing out a long string. If we had a long string literal, we could also read from an existing local file.
Initializing the Workspace
At this point, Terraform isn’t aware of your workspace, let alone that it’s supposed to manage any local_file resource, because it hasn’t been initialized yet. Let’s fix that by running a terraform init. You always need to run terraform init at least once, but often you’ll need to run it more than once, as you write and rewrite your configuration code to have new provider or module dependencies. Fortunately, this command is idempotent, which means it has no additional side effects if it is called more than once with the same input parameters. Feel free to run wild with terraform init. The command and output are shown below:
$ terraform init
Initializing the backend...
Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "local" (terraform-providers/local) 1.2.2...
The following providers don’t have any version constraints in configuration, and the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it’s recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
* provider.local: version = "~> 1.2"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands detect it and remind you to do this if necessary.You can view the files that were created as a result of terraform init with Tree. If you have a mac, you can easily install this with Homebrew using the command: brew install tree. After you install Tree, the following command lists files in your workspace:
$ tree -a
.
├── .terraform
│ └── plugins
│ └── darwin_amd64
│ ├── lock.json
│ └── terraform-provider-local_v1.2.2_x4
└── main.tf
3 directories, 3 filesAs you can see, terraform init creates a hidden .terraform folder in the current working directory, populated with our one provider dependency (we have no module dependencies for this scenario). Because we declared a local_file resource in main.tf, Terraform’s smart enough to realize there’s an implicit dependency on terraform-provider-local, which it proceeds to look up and download from the provider registry. You can verify this by adding the line: provider “local” {} at the top of main.tf and observing that the outcome doesn’t change on a subsequent terraform init.
TIP: If you’re going to be using implicitly declared providers, at least version lock them with a terraform settings block (exactly as we did with the local provider). This ensures any deployment’ is always repeatable and immutable.
Generating an Execution Plan
Before we create the resource with a terraform apply, we can preview what Terraform intends to do by having it output the result of an execution plan. Do this by running the command terraform plan in the terminal. The command and output are shown below.
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state’s used to calculate this plan, but won’t be persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform performs the following actions:
# local_file.literature will be created
+ resource "local_file" "literature" { #A
+ content = "Sun Tzu said: The art of war is of vital importance to the State.\n\nIt is a matter of life and death, a road either to safety or to \nruin. Hence it is a subject of inquiry which can on no account be\nneglected.\n"
+ filename = "art_of_war.txt"
+ id = (known after apply) #B
}
Plan: 1 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Note: You didn't specify an "-out" parameter to save this plan, and Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run. #C#A Terraform wants to perform a create on this resource
#B The computed id that Terraform assigns to every new resource
#C You could choose to save the output of terraform plan and use in as input to a terraform apply
TIP: You should always run terraform plan after making changes to your configuration code. In addition to letting you know what Terraform intends to do, it acts as a linter, informing you of any syntax or dependency errors. Like terraform init, this is an idempotent function with no additional side effects, and calling it multiple times doesn’t change the result.
WARNING: Terraform plans can fail for many reasons, such as if your configuration code was invalid, or your state file was created by a newer version of Terraform than the one installed on your machine, or even network throttling due to making a lot of remote API calls all at once. Sometimes, albeit rarely, the plan fails as a result of a bug in a provider’s source code. You need to carefully read whatever error message you get to know for sure. For more verbose logs, you can set the environment variable TF_LOG to debug or trace, e.g. export TF_LOG=debug
As you can see from the output, Terraform is letting us know that it wants to create a local file with the content and filename which we explicitly set. It also wants to set its own computed attribute called id. The computed attribute id is a special attribute which all resources have. It’s not a meta-argument or optional input, and you can’t directly set it, but you can read the value and use it for declaring explicit dependencies. id is mainly used internally by Terraform when calculating diffs in state and “proving” the identify of a resource during the read operation (i.e. if the ids for the same resource are different, then Terraform panics and throws an exception).
Although this particular terraform plan should have exited quickly, some plans can take a long time to complete. It all has to do with how spaghetti your Terraform code is, and if there’s lots of existing resources that Terraform needs to refresh and perform diff calculations on. Although the output of the plan we performed is fairly straightforward, there’s a lot going on that you should be aware of. The three main stages of generating the execution plan are:
- Loading Files — Where Terraform parses your configuration and state files and loads the data into memory. In our example, Terraform is loading
main.tfandterraform.tfstate(which doesn’t exist and is therefore considered empty). - Determining Actions to Take — This step is all about figuring out what needs to be done to achieve the desired state. In this case, we have a single resource that needs to be created, and Terraform determines that a
Create()action should be performed. - Respecting Dependencies — Actions need to occur in the right order, otherwise chaos ensues. If we had more than one resource, this step would be more interesting, but even for this simple example there are implicit dependencies that need to be respected.
Figure 4 is a detailed flow diagram showing what happens during this terraform plan. We’ll see how this diagram compares to execution plans generated later.
We haven’t yet talked about the dependency graph, but it’s a big part of Terraform and every Terraform plan generates one for respecting implicit and explicit dependencies between resource/provider nodes. Terraform has a special command for visualizing the dependency graph, terraform graph. This command outputs a dotfile which can be converted to a digraph using a variety of tools. You can see the resultant graph I produced for this workspace from the preceding DOT graph[1] in figure 5.
The dependency graph for this workspace has a few nodes, including one for the local provider, one for the local_file resource, and a few other meta nodes which correspond to miscellaneous actions. During an apply, Terraform walks the dependency graph to ensure that everything occurs in the right order.
Creating the Local File Resource
Next let’s run a terraform apply and see how the output compares to the execution plan we created from terraform plan. The command and output are shown below.
$ terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform performs the following actions:
# local_file.literature will be created
+ resource "local_file" "literature" {
+ content = "Sun Tzu said: The art of war is of vital importance to the State.\n\nIt is a matter of life and death, a road either to safety or to \nruin. Hence it is a subject of inquiry which can on no account be\nneglected.\n"
+ filename = "art_of_war.txt"
+ id = (known after apply)
}
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform performs the actions described above.
Only 'yes' is accepted to approve.
Enter a value:It’s no coincidence that this looks suspiciously like the output of terraform plan we saw earlier. The execution plan created by terraform apply is the same plan that was created by terraform plan. All we did was discard the execution plan we created earlier; this new plan is run once you confirm the manual approval step.
NOTE: Because calculating the execution plans twice in a row is obviously inefficient, there’s an optional argument to pass in the path of a previous plan’s output file. In practice, I suggest not worrying about this minor optimization, as you probably have bigger bottlenecks in your deployment pipeline than this. Additionally, it’s easy to make a mistake and pass in the wrong plan file to an apply, which could lead to unexpected behavior in your deployment.
The manual approval step in terraform apply is a safeguard meant to protect you from dangerous operations. During an apply, Terraform can create and destroy real infrastructure, and this obviously has real consequences. If you were to make a mistake or typo in your configuration code, you could potentially take down not only your application, but all the infrastructure your application depends on to run. This is why taking a few seconds to skim the result of the plan before confirming it is usually a good idea.
For this workspace, there shouldn’t be any unexpected behavior because it’s fairly straightforward. All Terraform does is call the Create() method on the local_file resource to create a text file on our local machine, as shown in figure 6.
Type “yes” into the cli at the prompt to approve the manual confirmation step. Your output is as follows.
$ terraform apply
…
Enter a value: yes
local_file.literature: Creating...
local_file.literature: Creation complete after 0s [id=907b35148fa2bce6c92cba32410c25b06d24e9af]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.Two files were created as a result of this command, art_of_war.txt, and terraform.tfstate file. You can view these files in the working directory by using the following command.
$ tree
.
├── art_of_war.txt
├── main.tf
└── terraform.tfstate
0 directories, 3 filesThe terraform.tfstate file (also known as the state file) is unsurprisingly, a generated file that Terraform uses to track the state of all the resources it manages. It’s used to perform diffs during the plan and detect configuration drift. Snippet 1 shows what the current state file looks like.
Snippet 1 Contents of terraform.tfstate
{
"version": 4, #A
"terraform_version": "0.12.0", #A
"serial": 1,
"lineage": "7bfcbd71-70f8-c31f-4349-03a49864baa2", #B
"outputs": {},
"resources": [
{
"mode": "managed",
"type": "local_file",
"name": "literature",
"provider": "provider.local",
"instances": [
{ #C
"schema_version": 0, #C
"attributes": { #C
"content": "Sun Tzu said: The art of war is of vital importance to the State.\n\nIt is a matter of life and death, a road either to safety or to \nruin. Hence it is a subject of inquiry which can on no account be\nneglected.\n", #C
"filename": "art_of_war.txt", #C
"id": "907b35148fa2bce6c92cba32410c25b06d24e9af", #C
"sensitive_content": null #C
} #C
} #C
]
}
]
}#A Version information about the state file. Newer versions are incompatible with older versions
#B The unique id assigned to the state file when it’s created
#C Stateful information about the art_of_war.txt resource
WARNING: It’s important not to edit, delete or otherwise tamper with the terraform.tfstate file, or else Terraform could potentially lose track of the resources it manages. It’s possible to restore a corrupted or missing state file, but it’s extremely difficult and time consuming to do this.
We can verify that the art_of_war.txt matches what we expect. This can easily be done by cat-ing the file in the cli. The command and output are shown below.
$ cat art_of_war.txt
Sun Tzu said: The art of war is of vital importance to the State.
It is a matter of life and death, a road either to safety or to
ruin. Hence it is a subject of inquiry which can on no account be
neglected.That’s all for this article. If you want to learn more about the book, you can check it out on our browser-based liveBook reader here and in this slide deck.
[1]DOT is a graph description language. DOT graphs are files with the filename extension dot. Various programs can process and render DOT files in graphical form
