Terraform Definitions
Terraform langugage definitions explained
Website Visitors:Terraform is an open source tool used to provision infrastructure on AWS. It allows developers to write declarative configuration files that describe their cloud resources. In this tutorial, we’ll explain basic terraform definitions and their usage.
Let’s look into Terraform concepts and commands.
Init
The terraform init
command is used to initialize a working directory containing Terraform configuration files. This is the first command that should be run after writing a new Terraform configuration or cloning an existing one from version control. It is safe to run this command multiple times. Terraform init will download all the providers and modules listed in the .tf file. There can be multiple providers in a single .tf file.
Variable
A variable is a symbolic name associated with a value. Variables are reusable values from terraform configuration. It is always good practice to separate variables from the actual configuration files as they are easy to modify when specified in a separate file outside the actual script. Also, when using variables, sensitive information like access keys or any other confidential information will not be shared when you share the terraform configuration script.
In Terraform, “variables” almost always refers to input variables, which are key/value pairs used in a run. Terraform modules can declare variables to allow customization. For child modules, the parent module provides a value when calling the module; for the root module, values are provided at run time.
Benefit of using variables is if you declare variables only once, you can use it anywhere in the script. When changing the values, you just have to change the variable value only once.
If you’re familiar with traditional programming languages, it can be useful to compare Terraform modules to function definitions:
-
Input variables are like function arguments.
-
Output values are like function return values or script output values.
-
Local values are like a function’s temporary local variables.
Input variables let you customize aspects of Terraform modules without altering the module’s own source code. This allows you to share modules across different Terraform configurations, making your module composable and reusable.
Input variables are the variables that you specify as inputs to your terraform script to perform specific activity in the script. For example, if you want to create an instance in AWS, one of the parameters you should specify is the region name. You have to pass this region name into your script. This is sent as input variable.
|
|
To use this variable in your script, use var.variable_name
.
Type Constraints
You can specify variable type in the variable as shown below. Here, we are defining the number of instances to deploy. It should be a numeric value. So, we specify type=number
in the variable declaration. If you enter anything other than a number, you will get an error.
|
|
Specifying the type value is optional but this will be useful when we specify a wrong value. When type is mentioned, terraform displays the type details in the error on which value to enter i.e., number or string etc…
Default Values
Let’s go back to the number of instances variable. If needed, user can enter number of instances to deploy. If user does not enter any value, it should deploy one instance. In this case, we have to specify a default value as, default = 1
so that one instance will be deployed if the variable value is not specified. You can also add type to the default value.
|
|
Terraform variables types
The Terraform language uses the following types for its values:
string
: a sequence of Unicode characters representing some text, like"hello"
.number
: a numeric value. Thenumber
type can represent both whole numbers like15
and fractional values like6.283185
.bool
: a boolean value, eithertrue
orfalse
.bool
values can be used in conditional logic.list
(ortuple
): a sequence of values, like["us-west-1a", "us-west-1c"]
. Elements in a list or tuple are identified by consecutive whole numbers, starting with zero. You can call them in your script as var.VarListName[0], var.VarListName[1] and so on..map
(orobject
): a group of values identified by named labels, like{name = "Mabel", age = 52}
. In your script you can then call them as var.VarMapName[“name”] or var.VarMapName[“age”] and so on..
Strings, numbers, and bools are sometimes called primitive types. Lists/tuples and maps/objects are sometimes called complex types, structural types, or collection types.
Finally, there is one special value that has no type:
null
: a value that represents absence or omission. If you set an argument of a resource tonull
, Terraform behaves as though you had completely omitted it — it will use the argument’s default value if it has one, or raise an error if the argument is mandatory.null
is most useful in conditional expressions, so you can dynamically omit an argument if a condition isn’t met.
Default terraform variable type is string.
Let’s look into examples for each of the variable types.
String
Strings are simple series of characters that are used to store names.
|
|
List
Terraform list is a comma separated list of strings in square brackets. List starts from zero.
|
|
When you want to use a value from the list, use the variable name with the index number as shown below.
|
|
Map
Maps in terraform are key/value combinations. We can then reference to the item anywhere as needed.
|
|
When you want to reference an item from the map list, you have to use the key as shown below.
|
|
Boolean
A boolean type allows either true or false for a variable. You can deploy a variable with boolean type as shown below.
|
|
You can then refer to the variable in your script under a module as
|
|
Assigning values to variables
There are multiple ways in which you can add variables to your terraform script, as explained below.
Providing variable value in the main.tf file
Here, you are specifying the value of the variable directly in the main.tf script file.
|
|
Specify variables in variables.tf file
You can create a separate file with the default name, variables.tf and run the terraform commands. While executing main.tf file, terraform will look for the default file, variables.tf and search for the variables.
|
|
In the below main.tf file we have to use the same variable name given in variables.tf file.
|
|
-var
parameter like: terraform plan -var="region=ap-south-1"
Specifying variables in files
If you need to set multiple variables in the script, then you can specify them in files with any of the below file names.
-
terraform.tfvars
-
terraform.tfvars.json
-
filename.auto.tfvars
-
filename.auto.tfvars.json
If you want to use a custom name for your variable file, you should specify that file name in the plan or apply or destroy commands as shown below:
|
|
If you are using variables.tf and terraform.tfvars files, you SHOULD declare an empty variable in variables.tf file and then assign value to that variable in terraform.tfvars file.
variables.tf vs tfvars
A variables.tf file is used to define the variables type and optionally set a default value.
A terraform.tfvars file is used to set the actual values of the variables.
You could set default values for all your variables and not use tfvars files at all.
Actually the objective of splitting between the definitions and the values, is to allow the definition of a common infrastructure design, and then apply specific values per environment.
Using multiple tfvars files that you give as an argument allows you to set different values per environment : secrets, VM size, number of instances, etc.
terraform.tfvars vs variables.tf difference
Normally in the variables.tf file we can specify value to the variable like region = "us-east-1"
but in the actual usage, in variables.tf file, you will specify empty variable like shown below:
variables.tf file
|
|
Next in terraform.tfvars file you will specify a value of that variable. So variables.tf file is for defining the variables, and ANYNAME.tfvars file is for assigning value to that variable as shown below:
|
|
Single variable
If you have only one variable in your script, then you can directly specify the variable value in terraform command like, terraform plan -var="region=ap-south-1"
. This will be useful if you do not want to specify access key or secret key in any file. Example: terraform plan -var="access_key=XXXXXXXXXXXX" -var="secret_key=XXXXXXXXXX"
Sepcify variable name at the time of execution
By default if you do not specify any value to a variable, terraform looks for terraform.tfvars file for variables. If it doesn’t find one, it prompts you to enter variable value when you run terraform plan or apply commands.
In this case, you should create variables.tf file and enter the variable names and leave them empty as shown below:
|
|
main.tf file
|
|
Using Environmental Variables:
You can use the export command and export your variable details in the terminal and then run terraform plan or apply to apply the changes. For this, you should also add a TF_VAR_ prefix in the command as shown below:
export TF_VAR_region="ap-south-1"
Next you can directly run terraform plan or apply commands etc…
Using TF_VAR command
Using environment variables to set a value.
TF_VAR_parameterName = "value" terraform plan
Ex: TF_VAR_instance_type= "t2.micro" terraform plan
Separate Terraform code as per the resource
You can create separate files with the extension .tf and have code for creating that resource alone. For example, you are working with aws. For vpc configuration you can create vpc.tf and enter all the vpc variables values there like cidr_block = “10.0.0.0/16” etc.. or call these variables from variables.tf file. In your main.tf file you will not have any code, just the provider details thats it. For seggregating resources clearly you can create each resource into different files like vpc.tf, ec2.tf etc..
|
|
vpc.tf file content
|
|
Variables file extensions
-
.tfvars and .tfvars.json are files for declaring terraform variables.
-
terraform.tfvars.json should contain json format text.
-
Terraform also processes *.auto.tfvars extension. This is used for complex environments when working with multiple teams.
When you declare variables in the root module of your configuration, you can set their values using CLI options and environment variables. When you declare them in child modules, the calling module should pass values in the module block.
Terraform variables precedence order
Default value –> Environment value –> File (terraform.tfvars or terraform.auto.tfvars or *.auto.tfvars) –> commandline (-var or -var-file)
commandline takes highest preference and goes on till the default value.
Provider
A plugin for Terraform that makes a collection of related resources available. A provider plugin is responsible for understanding API interactions with some kind of service and exposing resources based on that API.
Each provider adds a set of resource types and/or data sources that Terraform can manage. Every resource type is implemented by a provider; without providers, Terraform can’t manage any kind of infrastructure. Most providers configure a specific infrastructure platform (either cloud or self-hosted). Providers can also offer local utilities for tasks like generating random numbers for unique resource names.
Terraform providers are generally tied to a specific infrastructure provider, which might be an infrastructure as a service (IaaS) provider (like AWS, GCP, Microsoft Azure, OpenStack), a platform as a service (PaaS) provider (like Heroku), or a software as a service (SaaS) provider (like Terraform Cloud, DNSimple, CloudFlare).
In simple words using the provider you specify the vendor that you create/modify resources on. If you have to use multiple providers of same infrastructure provider like multiple AWS providers, you can specify alias for each one of them and call the data source later. Ex, if you have to create 2 VMs under 2 different azure subscriptions, you can create one provider as azure and add “alias = subscription1” and so on.
Resource
In Terraform’s configuration language: A block that describes one or more infrastructure objects. Resources can be things like virtual networks, compute instances, or higher-level components such as DNS records.
In other Terraform-related contexts: An infrastructure object of a type that could be managed by Terraform.
A resource block in a configuration instructs Terraform to manage the described resource — during a run, Terraform will create a matching real infrastructure object if one doesn’t already exist, and will modify the existing object if it doesn’t match the desired configuration. Terraform uses state to keep track of which real infrastructure objects correspond to resources in a configuration.
Terraform uses cloud provider APIs to create, edit, and destroy resources.
Terraform providers are responsible for defining resource types and mapping transitions in a resource’s state to a specific series of API calls that will carry out the necessary changes.
- In Terraform, a resource is a piece of infrastructure that you want to manage, such as an AWS EC2 instance, an Azure virtual machine, or a Google Cloud Storage bucket.
- Resources are provisions that Terraform creates, updates, or deletes to match the desired state declared in your configuration.
- Resources typically represent things that you can create, update, or delete, such as servers, networks, databases, etc.
- When you define a resource in your Terraform configuration, Terraform will manage the lifecycle of that resource based on the configurations you provide.
Data Source (data)
Data sources provide information about entities that are not managed by the current Terraform configuration. Using data sources, you can retrieve data from your existing infrastructure and use it in your Terraform code. Once you have defined a data source, you can use the data elsewhere in your Terraform configuration.
Let’s say your company has a custom AWS AMI for linux web server. If you want to deploy a new AWS instance from that ami, you need that ami ID to use it in the terraform code. In this case, you’d use a data source to retrieve the ami id information from your AWS account and update that ID in the instance creation parameters. More information is available HERE
- Data sources allow Terraform to fetch information or access data that already exists outside of Terraform, such as existing AWS S3 buckets, AWS VPC configurations, etc.
- Data sources are read-only, meaning Terraform uses them to obtain information to use elsewhere in your configuration, but it doesn’t manage their lifecycle.
- Data sources provide a way to reference existing infrastructure or configuration details within your Terraform configuration.
Output
Data exported by a Terraform module, which can be displayed to a user and/or programmatically used by other Terraform code. Using output you can get some information after a task is completed. Example, get aws instance ip after it is created. Output variables are used when you want to combine terraform output to other system like jenkins, for example. These output values are declared in an output block as shown below. Value after the word output, is the name of the output.
|
|
Terraform File Structure
Instead of adding all the resource blocks, variable blocks and output blocks in the same single file, it is always recommended to separate files as per their role.
|
|
State
If you execute the same terraform script and run terraform apply again and again, it wont delete and deploy the resources again. Until you make a change to terraform script file and run terraform apply again, it wont make any change.
How does Terraform know whether to apply the configuration change to your infrastructure or no?
Terraform knows this by creating a tfstate file when you run terraform apply. Every change that you make in your infrastructure through Terraform is noted in this tfstate file.
Terraform stores information about your infrastructure in a state file. This state file keeps track of resources created by your configuration and maps them to real-world resources.
Terraform’s cached information about your managed infrastructure and configuration. This state is used to persistently map the same real world resources to your configuration from run-to-run, keep track of metadata, and improve performance for large infrastructures.
Without state, Terraform has no way to identify the real resources it created during previous runs. Thus, when multiple people are collaborating on shared infrastructure, it’s important to store state in a shared location, like a free Terraform Cloud organization.
This state is stored by default in a local file named “terraform.tfstate”, but it can also be stored remotely, which works better in a team environment. Terraform uses this local state to create plans and make changes to your infrastructure. Prior to any operation, Terraform does a refresh to update the state with the real infrastructure.
Terraform’s filename.tf file contains the resources to be deployed or modified. Actual resources are deployed when you run the terraform apply command. After the resources are provisioned, terraform stores the state of the resources in a state file and it is stored locally by default.
You can run terraform state list
command to see the resources in a state. You can then use terraform state show resource_name
to view more details about a resource in the state. By default terraform show
command shows the state details. Terraform maintains the deployed configuration in this state file.
For every change you make and apply to your code, the “serial” keyword value changes in tfstate file. We cant control that number. It changes automatically.
State is saved when we execute terraform apply. State contains all values created by your provider when resources are created. Terraform uses the difference between plan and state to find changes to the configuration, which inturn generates a new plan that deploys those changes.
State Lock
If supported by your backend, Terraform will lock your state for all operations that could write state. This prevents others from acquiring the lock and potentially corrupting your state. If you are working alone on the config files, then locking may not be that useful. If multiple team members are working on the config files, locking prevents others from making changes to the environment.
State locking happens automatically on all operations that could write state. You won’t see any message that it is happening. If state locking fails, Terraform will not continue. You can disable state locking for most commands with the -lock flag but it is not recommended.
If acquiring the lock is taking longer than expected, Terraform will output a status message. If Terraform doesn’t output a message, state locking is still occurring if your backend supports it. Your state file can be on a local or remote location.
When terraform apply is executed, it creates 2 files. one for lock file and other for state file. Lock file is used by terraform to prevent two instances from executing an apply operation simultaneously. State file provides terraform with mapping between representation of resources in configuration file and actual resources that are deployed. By default these files are stored locally on the root folder of the configuration script.
Plan
The terraform plan command is used to create an execution plan. Terraform performs a refresh, unless explicitly disabled, and then determines what actions are necessary to achieve the desired state specified in the configuration files.
This command is a convenient way to check whether the execution plan for a set of changes matches your expectations without making any changes to real resources or to the state. For example, terraform plan might be run before committing a change to version control, to create confidence that it will behave as expected. The optional -out argument can be used to save the generated plan to a file for later execution with terraform apply, which can be useful when running Terraform in automation.
One of the stages of a run, in which Terraform compares the managed infrastructure’s real state to the configuration and variables, determines which changes are necessary to make the real state match the desired state, and presents a human-readable summary to the user. The counterpart of an apply. Terraform plan, which only performs a plan. It can optionally output a plan file, which terraform apply can use to perform that exact set of planned changes.
Terraform plan only creates an execution plan displaying what infrastructure changes are applied when you run the apply command. This is similar to running -whatif parameter in PowerShell.
If you have created a plan file before the infrastructure is changed (created or destroyed), that plan file becomes stale. You cannot use it again.
If you have created a plan file before the infrastructure is changed and completely deleted all the resources after few mins. Even in this case, that plan file becomes stale. You cannot use it again.
If you look at the output, you can easily determine if it is plan or state. For plan it will be “Known after apply” but in state file it will show the resource names directly.
Provisioners
Provisioners can be used to model specific actions on the local machine or on a remote machine in order to prepare servers or other infrastructure objects for service. If you are creating a FTP server, you create a VM and specify commands to install ftp software under provisioners section in your TF file. There are some pre-defined provisioners which are: chef Provisioner, file Provisioner, habitat Provisioner, local-exec Provisioner, puppet Provisioner, remote-exec Provisioner, salt-masterless Provisioner. To install ftp on the new server you have to use remote-exec provisioner. Similarly to copy a some data from the remote machine to our local machine, we have to use local-exec provisioner.
If something fails while using provisioners, it will be marked as tainted, and script will continue.
Functions
The Terraform language includes a number of built-in functions that you can call from within expressions to transform and combine values. The general syntax for function calls is a function name followed by comma-separated arguments in parentheses: max(5, 12, 9)
The Terraform language does not support user-defined functions, and so only the functions built in to the language are available for use.
You can experiment with the behavior of Terraform’s built-in functions from the Terraform expression console, by running the terraform console command:
max(5, 12, 9)
12
There are different types of functions like Numeric, String, Collection, Encoding, Filesystem, Date and Time, IP Network, Type conversion, and Hash and Crypto. Checkout this Link for complete functions information.
Modules
Modules are containers for multiple resources that are used together. A module consists of a collection of .tf
and/or .tf.json
files kept together in a directory. Modules are the main way to package and reuse resource configurations with Terraform. Using modules you can deploy infrastructure with very few lines of code.
The Root Module
Every Terraform configuration has at least one module, known as its root module, which consists of the resources defined in the .tf
files in the main working directory.
Child Modules
A Terraform module (usually the root module of a configuration) can call other modules to include their resources into the configuration. A module that has been called by another module is often referred to as a child module.
Child modules can be called multiple times within the same configuration, and multiple configurations can use the same child module.
Published Modules
In addition to modules from the local filesystem, Terraform can load modules from a public or private registry. This makes it possible to publish modules for others to use, and to use modules that others have published.
The Terraform Registry hosts a broad collection of publicly available Terraform modules for configuring many kinds of common infrastructure. These modules are free to use, and Terraform can download them automatically if you specify the appropriate source and version in a module call block.
Also, members of your organization might produce modules specifically crafted for your own infrastructure needs. Terraform Cloud and Terraform Enterprise both include a private module registry for sharing modules internally within your organization.
Using Modules
-
Module Blocks documents the syntax for calling a child module from a parent module, including meta-arguments like
for_each
. -
Module Sources documents what kinds of paths, addresses, and URIs can be used in the
source
argument of a module block. -
The Meta-Arguments section documents special arguments that can be used with every module, including
providers
,depends_on
,count
, andfor_each
.
Graph
The terraform graph command is used to generate a visual representation of either a configuration or execution plan. The output is in the DOT format, which can be used by GraphViz to generate charts.
After your terraform script file is ready, run terraform graph
command to create a dot format output. Copy the whole output to Webgraphviz URL and paste it and generate a diagram.
Apply
The terraform apply command executes the actions proposed in a Terraform plan.
The most straightforward way to use terraform apply is to run it without any arguments at all, in which case it will automatically create a new execution plan (as if you had run terraform plan) and then prompt you to approve that plan, before taking the indicated actions.
Another way to use terraform apply is to pass it the filename of a saved plan file you created earlier with terraform plan -out=filename
in which case Terraform will apply the changes in the plan without any confirmation prompt. This two-step workflow is primarily intended for when running Terraform in automation.
You can also use destroy command with plan as terraform plan -destroy -out=test.plan
and then run terraform apply test.plan
Use -auto-approve
parameter to skip the prompt when running terraform apply command. Full apply command usage is given HERE
Console
The terraform console
command provides an interactive console for evaluating expressions. This command provides an interactive command-line console for evaluating and experimenting with expressions. You can use it to test interpolations before using them in configurations and to interact with any values currently saved in state. If the current state is empty or has not yet been created, you can use the console to experiment with the expression syntax and built-in functions. The console holds a lock on the state, and you will not be able to use the console while performing other actions that modify state. This console is also called repl (read, evaluate, print, loop )
To close the console, enter the exit
command or press Control-C or Control-D.
For configurations using the local backend only, terraform console
accepts the legacy command line option -state
.
After the resources are deployed using terraform you can open terraform console command and view the specific resource info by executing the resource name from your tf file as shown below. Here, HomeLabVPC is the vpc deployed using terraform.
|
|
Show
The terraform show
command is used to provide human-readable output from a state or plan file. This can be used to inspect a plan to ensure that the planned operations are expected, or to inspect the current state as Terraform sees it. By default terraform show
command shows the state details.
Machine-readable output is generated by adding the -json
command-line flag.
Note: When using the -json
command-line flag, any sensitive values in Terraform state will be displayed in plain text. For more information, see Sensitive Data in State.
JSON Output
For Terraform state files (including when no path is provided), terraform show -json
will show a JSON representation of the state. If you execute terraform show
command without any parameters, it will display the output of latest state file. If there is no state file present, it will display “no state.” as the output.
For Terraform plan files, terraform show -json
will show a JSON representation of the plan, configuration, and current state.
If you’ve updated providers which contain new schema versions since the state was written, the state needs to be upgraded before it can be displayed with show -json
. If you are viewing a plan, it must be created without -refresh=false
. If you are viewing a state file, run terraform refresh
first.
The output format is covered in detail in JSON Output Format.
Backend
If the whole project is maintained by one person then saving state file locally might look good. But what if multiple people are working on a single terraform project? Here, terraform state file is stored to a remote share path. You can setup terraform to store the state file to this remote share path. This remote share path is called backend.
A backend defines where Terraform stores its state data files.
Terraform uses persisted state data to keep track of the resources it manages. Most non-trivial Terraform configurations either integrate with Terraform Cloud or use a backend to store state remotely. This lets multiple people access the state data and work together on that collection of infrastructure resources.
Available Backends
By default, Terraform uses a backend called local, which stores state as a local file on disk. You can also configure one of the built-in backends listed below:
local, remote, artifactory, azurerm, consul, cos, etcd, etcdv3, gcs, http, Kubernetes, manta, oss, pg, s3, swift
Some of these backends act like plain remote disks for state files, while others support locking the state while operations are being performed. This helps prevent conflicts and inconsistencies.
The built-in backends listed are the only backends. You cannot load additional backends as plugins.
There are several ways to supply the remaining arguments:
File: A configuration file may be specified via the init command line. To specify a file, use the -backend-config=PATH option when running terraform init. If the file contains secrets it may be kept in a secure data store, such as Vault, in which case it must be downloaded to the local disk before running Terraform.
Command-line key/value pairs: Key/value pairs can be specified via the init command line. Note that many shells retain command-line flags in a history file, so this isn’t recommended for secrets. To specify a single key/value pair, use the -backend-config=“KEY=VALUE” option when running terraform init.
Interactively: Terraform will interactively ask you for the required values, unless interactive input is disabled. Terraform will not prompt for optional values.
For more information checkout this terraform link.
Set timeout limits
When you run terraform plan
command or terraform apply
command, it might take sometime in deploying the resources. If you want to calculate the time taken to execute the command, you can do it with time
parameter.
You can use time
command with plan command as shown below.
|
|
It will show the time taken to perform the operation.
|
|
Similarly, for apply command,
|
|
This same time command will also work for destroy command.
Some resource types provide a special
timeouts
nested block argument that allows you to customize how long certain operations are allowed to take before being considered to have failed. For example,aws_db_instance
allows configurable timeouts for create, update and delete operations.
Adding Comments
In Terraform you can use three different ways in adding comments.
#
- To comment single line.
|
|
//
- To comment single line.
|
|
/*
and */
- To comment multiple lines. Add /*
at starting of first line and add */
at end of last line
|
|
Validate
The terraform validate
command validates the configuration files in a directory, referring only to the configuration and not accessing any remote services such as remote state, provider APIs, etc.
Validate runs checks that verify whether a configuration is syntactically valid and internally consistent, regardless of any provided variables or existing state. It is thus primarily useful for general verification of reusable modules, including correctness of attribute names and value types.
It is safe to run this command automatically, for example as a post-save check in a text editor or as a test step for a re-usable module in a CI system.
Validation requires an initialized working directory with any referenced plugins and modules installed. Run below command to use the validate command.
terraform validate
Suggested Article
Now that you know about Terraform definitions, continue reading our other article Terraform ForEach Loop. All other DevOps categories are listed here. Have a look at the posts as per your topic of interest.
Conclusion
In this tutorial, we’ve explained Terraform definitions and its basic usage. We’ve also demonstrated examples for all the topics that are discussed.
We hope you have learned something new in this article. Please feel free to share your thoughts about this article in the comments section below.
Your inbox needs more DevOps articles.
Subscribe to get our latest content by email.