🔥Let’s Do DevOps: Terraform Dynamic IAM Policy Construction
This blog series focuses on presenting complex DevOps projects as simple and approachable via plain language and lots of pictures. You can do it!
Hey all!
I was brought into DevOps in a shop that heavily utilized SparkleFormation. Sparkle (or SFN) is a ruby-based tool that constructs CloudFormation stacks. It allows DevOps folks to write minimal configurations, and then run the SFN constructor tool against them, and SFN will read your defaults (security, logging, config, etc.) and output a valid CloudFormation stack.
This has its drawbacks — obviously CloudFormation is only valid in AWS-land, but additionally, the SFN DSL (Domain Specific Language), or syntax, is painful and hard to work with. There is some security through obscurity, but primarily this difficult language operates as a tax on your teams — not good.
Terraform solves lots of these problems in an intuitive, easy way. Terraform is multi-cloud and the concept of modules can replace much of SFN’s constructor processes and gains. However, there is one process I haven’t been able to easily replicate — defaults.
In SFN, you can set a default list of Amazon Account IDs, and utilize this list to generate IAM policies, or any other kind of json/yaml-encoded policy, in an easy way. Terraform supports this, but not in an intuitive way.
Let’s build it together and you can start using this powerful pattern.
IAM Resources — Basic
I won’t go too deep into what AWS IAM is here — suffice to say, they are security and access control documents that control who can do what, and what actions can be taken. They’re incredibly powerful, but also complex, which means automating and process-atizing them is a huge gain for your company and your dev teams.
The AWS terraform team has helpfully created some purpose-built resources that help us build IAM Json docs and utilize them. The first is a data source called aws_iam_policy_document (link). Data sources generally reach out to the provider to learn about the environment. That isn’t the case here — the AWS terraform team is mis-using a data source to construct a json IAM document, which is very cool:
| data "aws_iam_policy_document" "example" { | |
| statement { | |
| sid = "1" | |
| actions = [ | |
| "s3:ListAllMyBuckets", | |
| "s3:GetBucketLocation", | |
| ] | |
| resources = [ | |
| "arn:aws:s3:::*", | |
| ] | |
| } | |
| } |
You’re then able to use this data source to output a json-encoded version of your policy:
| resource "aws_iam_policy" "example" { | |
| name = "example_policy" | |
| path = "/" | |
| policy = data.aws_iam_policy_document.example.json | |
| } |
This works great, but it’s again, static. We want to be able to send a list of account IDs and have IAM build us complex policies. Let’s talk about how we can build some iterative, complex policies.
IAM Resources — Iterative
First, let’s establish a local data source of account IDs. You don’t need to have this locally — you could store this in a central data store, or have the parent pass this information to the module that’s building these IAM policies. We’re using a local data store as a simple way to demonstrate.
| locals { | |
| account_ids = [ | |
| "111111111111", | |
| "222222222222", | |
| "333333333333", | |
| ] | |
| } |
Okay, now we have a list of account IDs. Now let’s build our json-encoded policy document. This IAM policy will only have a single statement — permitting IAM access to an entire other account by granting access to the “root” user, which means any user in that other account. However, we want it to update automatically when that passed list updates.
To do that, we set the principal as type AWS and set an identifier (required when multiple values here) of the local list of account IDs. Note that we use a for loop to iterate over that list and encoded it in the expected format.
Below, we set the actions — permit ECR pulling, an AWS action for an ECR (Image Repo for Container Images).
| data "aws_iam_policy_document" "aws_ecr_repository_policy" { | |
| statement { | |
| sid = "KeyVaultPolicyForAccounts" | |
| effect = "Allow" | |
| principals { | |
| type = "AWS" | |
| identifiers = [for k in local.account_ids : "arn:aws:iam::${k}:root"] | |
| } | |
| actions = [ | |
| "ecr:GetDownloadUrlForLayer", | |
| "ecr:BatchGetImage", | |
| "ecr:BatchCheckLayerAvailability", | |
| ] | |
| } | |
| } |
Okay, now we have a valid IAM policy document, all ready to go. However, we haven’t build the policy yet — right now this all lives within Terraform, no real resources are built.
Thankfully, we’ve done the hard part — now we are able to simply reference that data source’s output of json, which is a json encoded policy document.
| resource "aws_ecr_repository_policy" "aws_ecr_repository_policy" { | |
| repository = "dynamicBuilt" | |
| policy = data.aws_iam_policy_document.aws_ecr_repository_policy.json | |
| } |
And boom, we have a policy that is able to update itself when the passed values are updated.
IAM Resources — Iterative — Multiple Statements
This is of course a limited example — what if we wanted multiple statements based on these same values?
Well, we can do that also. Rather than a single iam policy doc, we use a for_each to build several of them, one each for the passed in account IDs. The SID must be unique for each statement or IAM complains, so we append the value, which of course should be unique.
| data "aws_iam_policy_document" "aws_ecr_repository_policy" { | |
| for_each = toset(local.account_ids) | |
| statement { | |
| sid = "KeyVaultPolicyForAccount${each.value}" #Must be unique in policy, so appending account ID | |
| effect = "Allow" | |
| principals { | |
| type = "AWS" | |
| identifiers = ["arn:aws:iam::${each.value}:root"] | |
| } | |
| actions = [ | |
| "ecr:GetDownloadUrlForLayer", | |
| "ecr:BatchGetImage", | |
| "ecr:BatchCheckLayerAvailability", | |
| ] | |
| } | |
| } |
We can’t directly use these json documents — after all, a Policy expects to be passed a single json document, not several! However, the AWS Terraform team is amazing, and built this modality into the same document, so we pass each document from above into a new copy of the same resource type, as a json entry.
This resource natively combines the json docs into a single valid policy document.
| data "aws_iam_policy_document" "combined" { | |
| source_policy_documents = [ | |
| for k, v in data.aws_iam_policy_document.aws_ecr_repository_policy : v.json | |
| ] | |
| } |
And then, just as before, we’re able to utilize the combined documents as input for our compiled policy.
| resource "aws_ecr_repository_policy" "aws_ecr_repository_policy" { | |
| repository = "dynamicBuilt" | |
| policy = data.aws_iam_policy_document.combined.json | |
| } |
Summary
This pattern is flexible, and allows us to build IAM policy docs in a “constructor” pattern, similar to how SparkleFormation and other constructors do. This makes life simpler and easier for our Devs, which makes our environments more secure.
I want to thank Jamie Phillips on the Hashicorp Ambassadors group for steering me towards this solution. Jamie has a great write-up on his website that goes over a similar pattern:
https://www.phillipsj.net/posts/terraforming-aws-iam-policies
Best of luck out there!
kyler