profile
viewpoint

Ask questionsfor_each attribute for creating multiple resources based on a map

Hi,

We are missing a better support for loops which would be based on keys, not on indexes.

Below is an example of the problem we currently have and would like Terraform to address:

  • We have a list of Azure NSG (Network Security Group) rules defined in a hash. E.g.
locals {
  rules = {
    rdp_from_onprem = {
      priority         = 100
      protocol         = "TCP"
      destination_port = "3389"
      source_address   = "10.0.0.0/8"
    }

    winrm_from_onprem = {
      priority         = 110
      destination_port = "5985-5986"
      source_address   = "10.0.0.0/8"
    }

    dynatrace_security_gateway = {
      priority         = 120
      destination_port = "9999"
    }
  }
}
  • This allows us to keep the Terraform resource definition DRY and use a loop to create all the rules:
resource "azurerm_network_security_rule" "allow-in" {
  count                       = "${length(keys(local.rules))}"
  name                        = "allow-${element(keys(local.rules), count.index)}-in"
  direction                   = "Inbound"
  access                      = "Allow"
  priority                    = "${lookup(local.rules[element(keys(local.rules), count.index)], "priority")}"
  protocol                    = "${lookup(local.rules[element(keys(local.rules), count.index)], "protocol", "*")}"
  source_port_range           = "*"
  destination_port_range      = "${lookup(local.rules[element(keys(local.rules), count.index)], "destination_port", "*")}"
  source_address_prefix       = "${lookup(local.rules[element(keys(local.rules), count.index)], "source_address", "*")}"
  destination_address_prefix  = "${lookup(local.rules[element(keys(local.rules), count.index)], "destination_address", "*")}"
  resource_group_name         = "${azurerm_resource_group.resource_group.name}"
  network_security_group_name = "${azurerm_network_security_group.nsg.name}"
}
  • So far, so good. However since the resources and their state are uniquely identified by the index and not by their name, we can't simply change the rules later.
    • We can add new rules only at the end of the hash.
    • We can remove rules only at the end of the hash.
    • We can modify the rules, as long as their position in the hash doesn't change.
    • But we can never remove any other rule or change their position in the hash. This seems to be very restrictive and basically means we had to stop using this approach and define all individual rules as individual azurerm_network_security_rule resources.

As you can guess, if we e.g. remove the first item from the hash, Terraform would not see that as a removal of the first resource (index 0), but rather removal of the last resource (index 2) and a related unexpected change of all the other resources (old index 1 becomes new index 0, old index 2 becomes new index 1).

Unfortunately this can also cause Azure provider to fail, because it may get into a conflict where an actual resource (old index 1) still exists in Azure, but Terraform now tries to modify the actual resource (old index 0) to have the same properties, but that is not possible (e.g. NSG priority and port have to be unique).

I've shown an example with 3 rules, but in reality we can have 50 rules and the tf file is 5x longer and more difficult to manage with individual resources compared to using a hash.

We would like to use hashes in Terraform in such a way that a position of an element inside a hash doesn't matter. That's why many other languages provide two ways of looping - by index (e.g. for i=0; i<list.length;i++) and by key (foreach key in list).

I'm sure that smart guys like you can figure out how to make this work in Terraform.

Thanks

hashicorp/terraform

Answer questions apparentlymart

I'm not sure what part of the upgrade guide you are referring to, but I suspect you're thinking of dynamic blocks, which are not the same thing as resource-level for_each even though they also use an argument named for_each.

dynamic blocks allow dynamic creation of nested blocks within resources. This issue is about using for_each on the resource itself, as a replacement for count.

The reason these things are different is that blocks within a resource block (aside from the meta-arguments) are just a normal part of the resource's object representation and so the dynamic block just behaves as a sort of macro, as if you had manually written out several blocks. Resource-level for_each must integrate with Terraform's built-in resource addressing, so it behaves slightly differently.

For example, if you were to write the following:

locals {
  things = {
    foo = bar
    baz = boop
  }
}

resource "null_resource" "example" {
  for_each = local.things

  triggers = {
    key   = each.key
    value = each.value
  }
}

...Terraform would see this as declaring two resource instances with the following addresses:

  • null_resource.example["foo"]
  • null_resource.example["baz"]

This is not the same as what would've happened if you just wrote two separate resource blocks, since in that case they would've been required to have distinct names. It's also different than using count, because the instance keys are the strings "foo" and "baz", rather than numeric indices 0, 1. This is the important detail that makes this better than count: Terraform will then correlate the indices by these string keys rather than by their positions in a sequence, so no particular ordering is implied or assumed.

What we did for Terraform 0.12 is prepared Terraform's internal models and file formats to support instance keys being strings. The remaining work is to change the current "expand" logic that handles count to also deal with for_each, and to track the expression result so that each.key and each.value will return suitable results inside those blocks.

useful!

Related questions

failed to save provider manifest: open .terraform/plugins/linux_amd64/lock.json: permission denied hot 4
The argument "host" is required, but no definition was found. hot 3
Unable to run 0.12upgrade hot 3
Module cannot find alias AWS provider in 0.12.0 hot 3
Error: Invalid template interpolation value hot 2
Terraform v0.11.1 : Error downloading modules: Error loading modules: open .terraform/modules/3f10921295c292995128e9e36eb: no such file or directory hot 2
MalformedPolicyDocument: Policy document should not specify a principal. hot 2
Error in Terraform 0.12.0: This object has no argument, nested block, or exported attribute hot 2
Feature Request - Allow list/array in 'query' in 'external' data source hot 2
Terraform provider downloads fail with TLS handshake timeout hot 2
'terraform init' failed with 'Registry service unreachable.' error hot 2
`Unreadable module directory` error is not clear for nested modules hot 2
Provider Development: Expected type 'string', got unconvertible type '[]interface {}' - with complicated block hot 2
"Error: Provider configuration not present" when aliased provider is used hot 2
Error loading state: state snapshot was created by Terraform v0.12.7, which is newer than current v0.12.6 hot 2
Github User Rank List