Terraform Workflow

How many people claim to know terraform, but do not understand the workflow? Do you? If not, follow along and I'll guide you through some need to know info that will help you understand the terraform workflow a bit better.

One of my buddies, Samuel Tillman, posted a question about Terraform, and I am sure he's going to get a lot of responses. Unfortunately, I'm not sure I would be happy with most of them, as I suspect they will be lacking a lot of nuance and detail.

You can see Sam's question here: terraform_workflow

It's actually a decent and fair question, even if Sam did put some of the steps out of order. I suspect he did that on purpose, or maybe he wasn't even thinking about it as an ordered operation when he wrote the question, which would be equally fair.

Edit/Correction: I owe Sam an apology here. I had said he was wrong, but I had actually overlooked some nuanced details when I was trying to simplify this without getting into the weeds of how the workflow works. To be clear, my statement was the wrong one.

So, let's get down to basics. What is the 'Terraform Workflow'? What does each step do, or why is it important?

Terraform Workflow

• Terraform Format terraform fmt
• Terraform Init terraform init
• Terraform Validate terraform validate
• Terraform Format
• Terraform Plan terraform plan (-w outfile.name)
• Terraform Apply terraform apply (outfile.name)

Now, I have put these in an order, but there are two steps that can technically be put in either order. Validate and Format can be run in either order, and it doesn't really make much of a difference (or does it?). I personally prefer to put validate first, and then format. I'll explain why in a moment, but for now, let's move on to the actual workflow explanation.

Ok, to clarify what happened and why it is important:

1. terraform format is scopeless. It operates on files. Simple. Done. I actually have this as part of my IDE, so it's technically performed before the other steps anyway. My own oversight here too.
2. terraform init -- This is where things get sketchy, because I had oversimplified things tremendously, even though I know the details myself. tf init has to be run before tf validate because validate requires definitions for things like provider schema, module definitions, etc. Now, let's say you have already done a tf init, and you're not changing providers, modules, or anything else. You may not even need the init here (and this is the mode I was thinking from). However! This is also still a bit misleading and incorrect, because to do an init you must have some form of tf resource definitions. (Even if its just providers at this stage). There is yet another nuanced gotcha where init can be done on an empty directory, but this is unique to tf test (I believe.)

Terraform Format

This one should be pretty self explanatory. Just thinking about the name, I am pretty confident you can guess what it does. Yep. It formats your terraform code files so that they are nice and neat and properly indented and things. Nothing overly fancy, but still a good idea because it helps keep your code consistently formatted, at minimum.

Terraform Init

What is terraform init, and what does it do? terraform init, or tf init if you're lazy like most of us, is the first second step for preparing a terraform project for use AND for refreshing module definitions, etc. When you run this command, it does a couple things, including sets up the actual terraform workspace. It will do things like creating the .terraform directory, where modules and definitions go, it will create the lockfile(s) needed, and even create the initial state file. Whether your state is remote stored or not, you will still have a local copy of the state that is reconciled when you execute terraform actions.

Terraform Validate

This is one of the first steps for a real IAC workflow with terraform. It is pretty basic, but highly useful. tf validate goes through your project code and looks for syntax errors. It cannot do advanced testing and things, only syntax. Now, I said before that I prefer validate before format, now I'll let you guess why first.

... ... Ok, you give up? If you do format first, then you do validate, then there is a chance that you may have to go and fix your code if validate found something, then you'll have to go and do another format anyway. By doing validate first, you can remove a potential duplicate step. It's not much, but we're automation experts. We don't like wasting time or energy (even a computer's!). Even in my own setup, this is after format because of my own ide/editor config.

Terraform Plan

Now, this is where we start getting into the real meat of things. What exactly is plan? Well, to simplify it, terraform plan is a dry run of a terraform execution. It doesn't make any changes to you infra, but it does go through your local state, and compares the information to the operational state of your provider. It will look for any signs of drift as well as look for cases where taint/replace operations needs to occur. How does it do this? This is where we get a bit more detailed. In the TF state file, there is a ton of metadata saved. Things like 'name', 'id', and many other metadata objects that represent what an item that is operating 'is'. Now, not all objects in state are actually treated as 'mandatory' when reconciling. I'm not going to get that far into the weeds with this post, but it is important to know that reconciliation is the comparison of local state metadata to operational state metadata. There is another reason that tf plan is important though, and that is because of the ability to write an outfile (-w outfile.name). Many TF novices undervalue tf plan. If you write an out file (-w outfile.name), you can then use that exact outfile as the execution instruction set, which we will cover next.

Terraform Apply

Lastly, we come to tf apply. The crux of terraform. This is the step where terraform will load your local state file, read the remote state, and actually reconcile the differences. With plan, we just determined what changes need to be made. With tf apply, we are actually going to execute those changes. If you have come this far, you have already learned about -w, and yes, we can use that outfile as our operating steps for the reconciliation. We do that by running tf apply outfile.name. It will use that outfile as the set of steps needed to reconcile our infra.

Now, there is more to be known about these steps, and I will put together a best practices guide that builds on top of this soon, but this should at least be enough to cover 'the workflow' itself.

Happy terraforming!

-Villain