How to access Terraform modules in another private repo using GitHub Actions

Having Terraform modules in a separate repo is a common practice to easily reuse modules over projects. But what’s the best way to get access to this private repo using GitHub Actions.

Alternatives

Using the Github Token is not good enough. Because the token has access to the current repo only. It is very secure to use this token because it expires after the job is done.

${{ secrets.GITHUB_TOKEN }} or ${{ github.token }}

Using the Personal Access Token (PAT) could probably work. But is not practical. Because it’s someone’s token. What if this person leaves. And if someone can steal this token, it can do harm.

Another option could be to create a service account in Github. Especially for this purpose. The disadvantage is that this requires another license for this user. And you will need to create a PAT for this service account. With the same risks discussed above when this token is stolen.

Get access using GitHub Apps

The way to get access in a maintainable and secure way is by using GitHub Apps.

Actually, the GITHUB_TOKEN is also created by a GitHub App. But this app has only access to the current repo.

The app configures a token on your current repo. Just like the GitHub Token, but this token will have access to other repos.

Let’s see how we can make the following code work:

module "resource_group" {  
  source = "git::git@github.com:techdrivennl/blogpost-terraformmodules-modules.git//modules/resource-group?ref=main"

  name = var.resource_group_name
}

Steps by step

Prerequisites:

1. We have 2 repos. One for modules and let’s call the other repo the project repo.
   Here can you find an example of the module repo.
   And here you can find an example of the project repo where main.tf is referencing a module in the module repo.
   In my demo environment both repos are public, so you can see the details. But this solution works great for private repos.
   
2. Make sure you have configured an App registration so we can authenticate using OIDC. This results in authentication without a password or secret. To make OIDC work a federated credential needs to be configured on the App Registration to Github. You could create a federation for the main branch. But this is very limited because you will need to create a federation per branch you might be working on. I would advise configuring a federation for an environment. And run the job in de GitHub action in an environment. This looks like this in my demo environment:
   

   

   

3. The Client Id of the App Registration, Subscription Id and TenantId are configured as a Github Secret on the environment with, in my case the name, dev.
   

   

4. The SPN has enough rights on the subscription to provision the needed resources. For example contributor on subscription level.

When you would create a GitHub Action workflow, the Terraform init would fail because it’s not able to download the referenced module:

git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Error: Failed to download module

Could not download module "resource_group" (main.tf:1)

Now we have an SPN, the details of the SPN as secrets on an environment in GitHub, we can move on to access the private module repo:

Using a GitHub App to authenticate to private repos:

1. Create a GitHub App. This can be done both on Organization or on your Account level.
   To create a GitHub App on the Organizational level go to the settings of the organization, scroll down and open up Developer Settings and choose GitHub Apps. Then click New GitHub App.
   Fill in: GitHub App name, a valid URL in Homepage URL (which will not be used. So you can fill in anything valid), and Disable the active checkbox for Webhook. Select at least Content Read-only under Repository permissions. Then click Create GitHub App.
   
2. Now edit the created GitHub App. Scroll down to Private Keys and generate a private key. A pem file is downloaded and the private key is created.
   
3. The private key and the App ID must be added as secret. It could be handy to add those secrets as Organization secrets. The names for the secrets in my case are TECHDRIVEN_APP_CLIENTID and TECHDRIVEN_APP_SECRET. Make sure that the secret is available for private repositories (or select the repositories where the secrets should appear)
   
4. Now all is in place to change the GitHub action workflow. We are going to add 2 actions.
   In the first step, we need an action that generates a token based on the GitHub App. You can choose one of the following actions:
   navikt/github-app-token-generator
   jnwng/github-app-installation-token-action
   tibdex/github-app-token
   In my pipeline I’m using the last one.
   
   The second step is to use this token so git can use it to access other repos using this credentials. This can be done using the action: de-vri-es/setup-git-credentials. After applying this action git is able to access the repos based on the Repository permissions configured on the app.
   
   The two actions in the GitHub workflow looks like this:

       - name: "Get Application Token"
         id: get_app_token
         uses: tibdex/github-app-token@v1
         with:
           app_id: ${{ secrets.TECHDRIVEN_APP_CLIENTID }}
           private_key: ${{ secrets.TECHDRIVEN_APP_SECRET }}
   
       - name: Extend Git credentials
         uses: de-vri-es/setup-git-credentials@v2.0.10
         with:
           credentials: https://user:${{ steps.get_app_token.outputs.token }}@github.com

The complete GitHub Action workflow

The following code contains a very simple workflow that shows how to access Terraform modules in another private repo:

name: Terraform Plan

on:
  workflow_dispatch:

env:  
  TERRAFORM_VERSION: "1.3.7"  
  TERRAFORM_DIRECTORY: "terraform"

permissions:
  id-token: write
  contents: read
  pull-requests: write

jobs:
  terraform:    
    runs-on: ubuntu-latest   
    environment: dev
    steps:
    
    - name: Checkout PR
      uses: actions/checkout@v3

    - name: Setup Terraform
      uses: hashicorp/setup-terraform@v2
      with:
        terraform_version: ${{ env.terraform_version }}
        terraform_wrapper: false

    - name: "Get Application Token"
      id: get_app_token
      uses: tibdex/github-app-token@v1
      with:
        app_id: ${{ secrets.TECHDRIVEN_APP_CLIENTID }}
        private_key: ${{ secrets.TECHDRIVEN_APP_SECRET }}

    - name: Extend Git credentials
      uses: de-vri-es/setup-git-credentials@v2.0.10
      with:
        credentials: https://user:${{ steps.get_app_token.outputs.token }}@github.com

    - name: Log in using OIDC
      uses: azure/login@v1.4.7
      with:
        client-id: ${{ secrets.ARM_CLIENT_ID }}
        subscription-id: ${{ secrets.ARM_SUBSCRIPTION_ID }}
        tenant-id: ${{ secrets.ARM_TENANT_ID }}        

    - name: Terraform Init      
      working-directory: ${{ env.TERRAFORM_DIRECTORY }}
      shell: bash
      run: |
        terraform init -upgrade \
          -backend-config=storage_account_name=sablogterraform \
          -backend-config=container_name=terraformstate \
          -backend-config=key=demostate \
          -backend-config=resource_group_name=rg-blog-terraform \
          -backend-config=use_oidc=true
      env:
        ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID }}
        ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID }}
        ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID }}
        
    - name: Terraform Plan      
      working-directory: ${{ env.TERRAFORM_DIRECTORY }}
      shell: bash
      run: |
        terraform plan
      env:
        ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID }}
        ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID }}
        ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID }}

Conclusion

Using these 2 actions in combination with the GitHub App you are able to access private modules in another repo. The complete sources can be found in the following GitHub repos:

https://github.com/techdrivennl/blogpost-terraformmodules-project
https://github.com/techdrivennl/blogpost-terraformmodules-modules