🔥Let’s Do DevOps: GitHub Power User — Find All PRs with Specific Title Org-Wide with Rest API
This blog series focuses on presenting complex DevOps projects as simple and approachable via plain language and lots of pictures. You can do it!
Hey all!
Recently, I’ve been getting our GitHub Enterprise up to snuff. For other CI platforms, you’d set policies at the Enterprise level that all the Orgs and Repos would inherit — e.g., permissions. There’s also usually robust search for things.
However, those features are either not quite present or are presented as a “you can do it… with the API”. So let’s do it. With the API!
Let’s dive into why you might need to search your whole Org for every Repo, and look at each PR. If you’d rather skip to code, scroll to the bottom of this article and click on the GitHub link — all code provided so you can build it too!
The Problem
In some CI tools, you have the ability to set a single Action/Automation as a required run, and set a single pipeline that must run for every PR to qualify to be merged. There are lots of justifications for this — a tool that does static analysis of code, a code linter, a tool which validates that organization best-practices were followed.
GitHub lets you do this only at the single-Repo level. Which sounds totally reasonable until you need to deploy a tool like I described above to hundreds of repos. That’s a huge, annoying problem. And I have a write-up coming for the tool I’ve built to do just that — iterate through all repos, add Action(s), commit them, push to remote, and open a PR.
However, imagine we’re past that. You’ve now opened 180 PRs and need to track how many remain open so you can poke the teams that control those repos and haven’t yet taken care of merging them yet. Manually searching for those PRs is an annoying problem.
One which can be solved with GitHub’s robust REST API and a bit of scripting magic. The script I wrote is linked at the bottom of this write-up. Let’s step through each section and what it does. That’s not much to it at all.
Input Mapping
I wrote this tool in bash/shell for a linux/mac environment, but it could be pretty easily ported to powershell and windows (if you really wanted to touch icky windows). If you port it, please message me!
So we have a script that can search an organization for a PR with title n . Both of those items shouldn’t be derived from anywhere. First of all, admins are often parts of multiple GitHub orgs, so let’s have the user tell us exactly which Org to search.
Second, the PR title — we could maybe derive from a commit or something, but easier to just have someone tell us. So to call our script, it’d look like this:
| ./findAllPrsWithName.sh your_org_name "The title of your PR" |
Our script maps those inputs to named variables right away for the humans in the room. Bash uses a numeric reference to input arguments, e.g. $1 for the first argument and $2 for the first argument. It’s remedial, but it’s all we need for this simple script.
| # Input mapping | |
| GH_ORG=$1 | |
| PR_TITLE=$2 |
Validate the Arguments
Next we do some validations to make sure the arguments are all provided, and as much as we can, provided properly.
First let’s make sure the GITHUB_TOKEN var is populated. This is what github cli tools use to authenticate to GitHub. If it’s blank (-z tests if a var is blank), then we print some info for the caller to read and correct, and gracefully exit.
| #GITHUB_TOKEN | |
| if [ -z "$GITHUB_TOKEN" ]; then | |
| echo "Your GITHUB_TOKEN variable appears blank, make sure to export it into this terminal, like this" | |
| echo "export GITHUB_TOKEN=ghp_xxxxx" | |
| exit 1 | |
| fi |
Then we validate that the arguments are provided for the org and PR title using the same strategy — check if the VARs are blank. If yes, print helpful info and exit.
| # Org | |
| if [ -z "$GH_ORG" ]; then | |
| echo "Your GH_ORG variable appears blank, make sure to call this script with the org as your first argument, like this:" | |
| echo "./findAllPrsWithName YourOrgNameHere \"PR name goes here\"" | |
| exit 1 | |
| fi | |
| # PR Title | |
| if [ -z "$PR_TITLE" ]; then | |
| echo "Your PR_TITLE variable appears blank, make sure to call this script with the repo title as your second argument, like this:" | |
| echo "./findAllPrsWithName YourOrgNameHere \"PR title goes here\"" | |
| exit 1 | |
| fi |
Next we check if we’re receiving too many arguments. We shouldn’t ever receive a $3. If we do, something’s gone wrong. The most common “gone wrong” is probably that the caller didn’t quote their PR title, and it contains spaces. Bash interprets spaces as separating different vars, which messes up our script something fierce. We print an error and gracefully exit.
| # Warn about receiving too many vars | |
| if [ ! -z "$3" ]; then | |
| echo "Receiving too many arguments, you might need to quote your PR title, like this:" | |
| echo "./findAllPrsWithName YourOrgNameHere \"PR title goes here\"" | |
| exit 1 | |
| fi |
Next we check if the gh tool (GitHub CLI) is installed. We require it to work, so let’s check if it’s here and active. If the gh --help response is less than a few lines, something’s wrong — it normally prints dozens of help lines.
| # GH CLI Installed | |
| if [ $(gh --help | wc -l) -le 3 ]; then | |
| echo "It doesn't look like you have the GitHub CLI installed, install it here: https://cli.github.com/" | |
| exit 1 | |
| fi |
State Your Intentions
Now that we’ve passed all our validations, we print out all the information we’re searching for. The github Org, the title of the PRs we’ll be hunting for, and some information about what we can use this information for.
For instance, did you know you can pass a valid html URL to the gh tool and you can close PRs programmatically? I’ve closed about 70 at a time in a batch and it works flawlessly 💥
| # State search params | |
| echo "#######" | |
| echo "# Searching org $GH_ORG for PRs with title \"$PR_TITLE\"" | |
| echo "# If any are found, their URLs will be printed below" | |
| echo "# These URLs can be used to interact with them, e.g.:" | |
| echo "# (example) gh pr close https://github.com/YourOrgName/RepoName/pull/1234" | |
| echo "#######" |
What Repos?
Notice we haven’t provided a list of repos. Rather, we want the gh tool to grab that info for us. That helps to keep our list up to date (to the second!). We ask the gh tool to list all the repos in our Org, then use cut twice to separate the repo name from all the information it spits out at us.
| # Grab all repos | |
| org_repos=$(gh repo list $GH_ORG -L 1000 | cut -d "/" -f 2 | cut -f 1) |
The resulting list looks like this, and is stashed in the $org_repos variable.
| repo1 | |
| repo2 | |
| repo3 | |
| repo4 | |
| repo5 |
Start Searching!
Now we start the cool stuff. Let’s iterate over each repo our gh tool found in our Org, and search each one for a PR with the title we passed it. If there is a result, print it. If not, move on.
On line 1, we do a while loop to read over our list of repos (not on line 10 how we’re passing that var into our while loop to iterate over).
On line 3 we are setting GITHUB_URL to the results of a curl command. That curl sends a REST API call to github to get all information about all pull requests in a particular Repo in our Org. Then we use the very awesome and powerful jq tool to parse the json response and filter for a title, and then grab only the .html_url, which is the browser-compatible URL you can visit.
We could jump right to printing out the results, which works great if the repo contains a PR with that name. However, if it doesn’t, $GITHUB_URL will contain an empty line, so why print it? On line 7, we check if the line is blank (echo the var, use the awk 'NF' tool to filter out blank lines, then wc -l to count the number of lines. If there is a non-zero result, there is a PR (or maybe more than 1 PR), so print it!
| while IFS= read -r GH_REPO; do | |
| unset GITHUB_URL | |
| GITHUB_URL=$(curl -s \ | |
| -H "Accept: application/vnd.github+json" \ | |
| -H "Authorization: Bearer $GITHUB_TOKEN" \ | |
| https://api.github.com/repos/$GH_ORG/$GH_REPO/pulls 2>&1 | jq -r ".[] | select (.title==\"$PR_TITLE\")| .html_url") | |
| if [ $(echo $GITHUB_URL | awk 'NF' | wc -l) -gt 0 ]; then | |
| echo $GITHUB_URL | |
| fi | |
| done <<< "$org_repos" |
Summary
We identified a problem with GitHub — namely, needing to build dozens or hundreds of PRs in lots of repos, and how hard it can be to find them quickly. Then we covered the tool I’ve built that you can use to find the HTML URL of each PR across an entire GitHub Org in any Repo (which could be hundreds or thousands of repos).
We covered how the tool works — REST API calls, and how you can use the tool to make your life easier.
You can find the code here:
GitHub - KyMidd/FindAllGitHubPrsByTitle
You can't perform that action at this time. You signed in with another tab or window. You signed out in another tab or…github.com
Best of luck out there folks!
kyler