File Directories
Before we get into dynamization, let’s take another look at a typical structure of a Terraform folder, for example in a code repository.
Although it is technically possible, Terraform configurations are usually not stored in a single file, especially in an enterprise context. In principle, the developer is free in the number, and naming of the files. However, some practice has taken hold. Often there is a division in the style:
Outsourced modules are located in separate folders or their own remote repositories. Modules are a big topic in their own right in Terraform, which is why we won’t cover them for now, but later. In this part, we will only talk about the structure of the root module, which is the entry point in any Terraform project.
In the main.tf file – or with the name of the resources to be created, e.g. network.tf – there are resource blocks, local definitions and maybe necessary data queries. The var files and the outputs are related to variable usage. The mandatory provider configurations, for example for the version or different subscriptions, are outsourced in a providers.tf.
Variables
Automation only works sensibly if a dynamic can be brought into the code through certain parameterization. With Terraform, this is done, among other things, with variables that are defined in the three variants which can occur:
- Input Variables
- Locals
- Outputs
Let’s look at them.
Input Variables
If you come from native ARM development, you would probably compare Terraform input variables with the parameters from the ARM templates.
Variables are defined in blocks. Besides the name itself, at least the type must be defined within the block. In addition, there are many optional features. A description should be set for the purpose, a default value is also possible.
Another interesting feature is the “sensitive” property for sensitive information such as passwords. The values are then masked in the various console outputs.
Attention: in the state file, the values are always stored in plain text.
Validation of the variable can also be configured. For this purpose, a validation block is defined and a condition is specified for verification. This allows numbers to be checked for value ranges or, for example, strings for a regex pattern.
Setting the Variable Values
If a variable has been given a default value, this counts as an optional variable in the world of Terraform. For all other variables, a value must be assigned. A number of options are available for this. The value assignment is done according to a defined order, using the last value found. A value from a previous step is overwritten. The sequence is:
- Environment variable
- .tfvars file
- Using console arguments
For environment variables, the prefix TF_VAR_ must be used before the variable name. The configuration files are automatically evaluated if certain naming patterns are adhered to. Either the file extension – tfvars or json – is preceded by only “terraform” or between an individual name and the extension is “.auto“. Key-value pairs are accepted for .tfvars files, or JSON objects are accepted with the other option. «Auto» files would overwrite the other values, the order within the file system is taken into account.
- terraform.tfvars.json
- *.auto.tfvars
Variables can also be passed directly with the Terraform commands via the -var argument for the single input or with a file path and the addition – -var-file.
If none of these steps are successful, the value must be entered via the console, Terraform plan and apply stops at this point. Especially with DevOps pipelines, this can lead to surprises if the variable has been forgotten and the pipeline does not continue to run as a result.[1]
Locals
Locals follow a similar logic as the variables from ARM templates. As with input variables, they are used to avoid repetition and “hardcoding” in the configuration. However, they are not entered into the configuration from the outside, but are defined within it or evaluated at runtime. Locals are a link between a name and an expression. They are particularly useful when a certain logic or function is to be used for valuation.
Some caution is required when referencing the locals in the configuration, which in principle works as with variables: Access is via the local keyword, but is always defined with locals.
Our storage account name is a typical example of locals. As variable input comes a part of the naming pattern. However, since the name is integrated into the endpoint URLs of the storage, a random number is needed for worldwide uniqueness, for example:
In addition, the expressions supported by Terraform, including function calls, can be used – for example, to generate a random number. [2]
Outputs
An output is a kind of return value that describes, for example, an attribute of a resource and can be output in the console. In addition, outputs are used to transfer information between different configurations from a module structure – more on this in the next posts.
A corresponding block must be defined for the definition:
The return values are usually listed in the outputs.tf file. All that is required is an expression for the value. Description, value masking, and defined dependencies on other resources are optional. Output values are only output after a successful apply step. In our example, we output the region and the web endpoint from the storage account for demo:
Functions
Terraform can help make code more dynamic and streamlined with the help of numerous predefined functions. There are many “little helpers” especially in the categories:
- Strings (Substring, Regex, Upper-Lower, Formatting)
- Numbers and lists (max-min, concatenation, length, search and sorting)
- Hashing and encoding (Base64, json yaml encoding, md5 and sha256)
- Date and time (formats, manipulations, timestamps)
- Network functions (IP and subnet calculations)
Most of them should be known in principle, for the details please refer to the documentation. Unfortunately, it is not possible to create individual functions. [3]
From our practice, the following favorites have emerged so far:
String interpolation
The composition of string parts is probably one of the most common use cases for dynamizing values, especially to adhere to naming conventions and uniqueness for globally individual attributes. With the help of ${expression}, e.g. variables, locals, functions and even data retrieval can be easily integrated in a string:
Merge
The merge function can merge multiple objects and override default values – or keep them as a fallback. merge compares the keys in the objects and, if they do not exist, inserts them into the resulting object. Values of existing keys are overwritten, strictly according to the order in the function call.
As an example, suppose that the following would be defined as the standard for a Tags object in the variables:
In the locals, the following is also defined:
the result of a merge call with
would then end up in an output of:
try()
Another useful function is try(). The passed parameters are evaluated one by one and the first valid value is returned. Expressions or function calls can also be used for the parameters, the last parameter is usually a safe fallback value.
Example: (tags.abc doesn’t exist and stage isn’t a Boolean-compatible string)
This would return: []
Test features – Terraform console
Also a very useful tool from Hashicorp’s construction kit is the terraform console command. This starts a kind of interactive mode and the values of variables, locals or resource configurations can be evaluated and tested. This is especially helpful for composite or calculated values and if the configuration has become a bit more extensive. This form of evaluation is also much faster than waiting for a plan or apply command.
Suppose there is the following locals definition, as well as variable values defined in a tfvars file:
For example, these queries are possible then:
The state is also used, i.e. values of already deployed resources can also be queried in this way. The lookup function is useful when searching map objects.
In the next part we will look at conditions, loops and constructs.
[1] https://developer.hashicorp.com/terraform/language/values/variables#variable-definition-precedence
[2] https://developer.hashicorp.com/terraform/language/values/locals
[3] https://developer.hashicorp.com/terraform/language/functions