🔥Let's Do DevOps: Updating Your TF and Action Dependencies on GitHub Repos with Dependabot🔥
Never let new module versions go unnoticed!
This blog series focuses on presenting complex DevOps projects as simple and approachable via plain language and lots of pictures. You can do it!
Hey all!
Recently I was asked to implement Dependabot for our Terraform/OpenTofu repos, and do we have a lot of them! As your terraform implementation matures, you’ll end up with lots of Repos that contain modules. These modules represent best practice deployments of all the resource types you need to manage in your environment.
Your Project repos will end up calling these Resource modules, and deploying them lots of times. This works great, and is very scalable, but it does lead to a problem - you’ll create a new version of a Resource module, and it’s hard to tell exactly which other Project repos (or other Resource Repos, even) are calling your Resource Repo, and which should probably be updated since there’s a new version.
That’s where Dependabot comes in - it has a lot of other uses, for all types of software, but we’ll stay firmly focused on the ecosystem I know the best - Terraform and OpenTofu (which I use as synonyms here and in all my articles).
You’ll also end up with lots of Actions files in your repos. GitHub has now developed some tools for centralizing the Actions files in a single repo, and calling them from each repo, but those tools are still pretty remedial, so I’d bet you have sometimes hundreds of copies of Actions in your repos.
Keeping up with all the step definitions can be a pain, even if you’ve centralized most of your logic in other Action-focused Repos. Dependabot can help with that too!
Implementation is incredibly simple, and doesn’t require any special license.
Dependabot File Customization
First of all, the dependabot file lives here in your repo:
.github/dependabot.yml
It must live in exactly that location in order to work.
Let’s look through this file, and see what it can and should contain. First, I have some information commented out at the top - notably the link to the Dependabot file customization options available from GitHub. All the flags and customizations we’ll talk about are from that link.
On line 7, we define dependabot version 2. At the time of writing, you must specify version 2, there is no version 1 or 3 yet.
| # To get started with Dependabot version updates, you'll need to specify which | |
| # package ecosystems to update and where the package manifests are located. | |
| # Please see the documentation for all configuration options: | |
| # https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file | |
| # Version 2 is required | |
| version: 2 |
If you are only updating dependencies from public registries and only consuming public modules, you don’t require any special SSH keys or PATs. However, most of the implementations I’ve seen are loading TF modules, and sometimes Actions from internal (non-public) Repos.
To do that, you have to give Dependabot special access to your internal Repos, and you do that by defining a Registry. Registries can be of any type (there are tons of them!) but the most useful one for this use case is a git Registry. That’ll let both Actions and Terraform clone Dependent Repos. There is a special note here
The password on line 7 isn’t loaded from the regular Repo Secrets. It’s loaded from the Dependabot Secrets.
Let’s talk about that more.
| # Permit accessing internal repositories | |
| registries: | |
| github: | |
| type: git | |
| url: https://github.com | |
| username: x-access-token | |
| password: ${{secrets.GH_PAT}} |
When you’re configuring an Action, you have access to the Repo Secrets. However, Dependabot doesn’t have access to those secrets, either when it’s running to check versions, or when it builds a PR to test your code.
You’ll still (of course) need to access Secrets, like PATs and SSH keys, with your Dependabot-driven workflows, so to fix that, we have to create “Dependabot Secrets”. You can do that from the Repo or Org settings, and you’ll find an entry for Dependabot secrets separate and apart from Actions secrets.
In the future GitHub is going to implement check boxes to make secrets available in other contexts, like on Dependabot. Today they exist as entirely separate entities.
I personally added the exact same secrets, with the same name and contents, that Dependabot will use, so the Actions can work exactly the same way. If you use different names you’ll need to write logic in your Actions to read secretX if run in normal context, and secretY if run by Dependabot. You can, but same name are easier.
We can now define several different Package Ecosystems that can be kept up to date. First, let’s define the “GitHub Actions” ecosystem. That’ll keep the version of Action tasks that we’re calling up to date. Let’s go line by line.
On line 4, we have it read the entire repo. This is probably overkill, and we could filter to just the .github/workflows location. But who cares.
On line 6, we tell it what registries it’s allowed to read - these are the special authorizations we defined earlier in this file. That means for this check it can use our GitHub PAT to read any non-public Action tasks it requires.
On line 8, we schedule an inventory. We set it up as daily, but weekly is also an option.
On line 10, we can set a prefix for the commit-messages Dependabot creates. This is optional, but I like to see all commits prefixed for clarity.
On line 14, we specify some labels to assign to the PRs created by Dependabot. I picked “dependencies” for this update type.
And finally, on line 17, we can assign a team or user to be the reviewers for the PRs. This is obviously a fake team, but you can use other internal teams if you’d like.
| # Scan for github actions and terraform module updates | |
| updates: | |
| - package-ecosystem: "github-actions" # See documentation for possible values | |
| directories: | |
| - "*" # Location of package manifests | |
| registries: | |
| - "github" | |
| schedule: | |
| interval: "daily" | |
| commit-message: | |
| # Prefix all commit messages with Dependabot" | |
| prefix: "Dependabot" | |
| # Specify labels for github-actions pull requests | |
| labels: | |
| - "dependencies" | |
| # Add reviewers | |
| reviewers: | |
| - "OrgName/pr_approvers" |
We could close out and be done, but I also want Dependabot to keep us up to date with the Terraform (Hopefully OpenTofu is coming soon now that the OpenTofu project has started to build out their module Registry). Let’s again go line by line. it’s mostly the same.
One line 2, we define the directories to look through to find module calls. Wildcard works great here, because your TF code might be in folders, and it might call modules from anywhere.
On line 4, we again tell Dependabot it can use the “github” registry we defined above to find non-public TF modules.
On line 6, we do something interesting - we tell Dependabot to build “groups” of updates, and put them into the same PRs. That means if it notices that there are several different dependencies that all need to be updated, it won’t create multiple PRs, it’ll combine. I’m still testing this out, and we might revert it.
On line 10, we tell it to check daily, and on line 12, we prefix all commits with “Dependabot”.
On line 16, we assign all Dependabot PRs with label “dependencies”, and on line 19, we assign a team to all these PRs.
| - package-ecosystem: "terraform" | |
| directories: | |
| - "*" | |
| registries: | |
| - "github" | |
| groups: | |
| terraform-modules: | |
| patterns: | |
| - "*" | |
| schedule: | |
| interval: "daily" | |
| commit-message: | |
| # Prefix all commit messages with Dependabot" | |
| prefix: "Dependabot" | |
| # Specify labels for github-actions pull requests | |
| labels: | |
| - "dependencies" | |
| # Add reviewers | |
| reviewers: | |
| - "OrgName/pr_approvers" |
Turn it On, Troubleshoot Failures
As soon as you commit dependabot to the default branch, it turns on. There are no settings or anything to enable it. If it doesn’t turn on, make sure your file is in this exact path in the default branch of the Repo:
.github/dependabot.yml
If your dependabot actions fail, it’ll print out error messages and email you, exactly like normal Action runs do.
And if your Dependabot PRs fail when your normal Actions should work, you’re likely missing a Secret. Make sure to copy any secrets you require from the regular Actions store to the Dependabot store.
And that’s all!
Summary
In this article, we walked through what Dependabot can do, and why you should use it. Then we turned it on for several different ecosystems, including Actions tasks as well as Terraform/OpenTofu (TF) dependencies.
We talked about the limitations present around Secrets and cloning Module Repos, namely that Dependabot only has access to the Dependabot Secrets, so you’ll need to copy those over in your Repo or Org.
We turned on Dependabot and will now have an easier time keeping up to date with lots of our dependencies, forever. Go us!
Good luck out there.
kyler