Testing external contributions using GitHub Actions secrets

As cloud-native applications become more complex and rely on more third-party services, testing becomes increasingly difficult. One of the most significant challenges for open source projects is testing contributions against complex services that require authentication and are particularly hard to mock.

In this blog post, we will explore a simple method for securely running this kind of integration tests on external pull requests, using the GitHub Actions pull_request_target trigger and GitHub environments to prevent unauthorized runs:

Configuration

1. Create some encrypted secrets; a secret named EXAMPLE will be used to illustrate the next sections.

2. Create an environment named external and add some trusted GitHub users or teams as required reviewers; they’ll be responsible for approving every run triggered by external contributors.

   

Workflow

⚠️ Warning: using the pull_request_target event without the cautionary measures described below may allow unauthorized GitHub users to open a “pwn request” and exfiltrate secrets; see also this [1, 2, 3] blog post series from GitHub Security Lab and this Stack Overflow answer.

on: pull_request_target

jobs:
  authorize:
    environment:
      ${{ github.event_name == 'pull_request_target' &&
      github.event.pull_request.head.repo.full_name != github.repository &&
      'external' || 'internal' }}
    runs-on: ubuntu-latest
    steps:
      - run: true

  test:
    needs: authorize
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          ref: ${{ github.event.pull_request.head.sha || github.ref }}
      - run: printenv EXAMPLE
        env:
          EXAMPLE: ${{ secrets.EXAMPLE }}

This workflow will be triggered by the pull_request_target event, which is similar to the pull_request event, but it always passes secrets to workflows triggered from fork pull requests.

The authorize job checks if the workflow was triggered from a fork pull request. In that case, the external environment will prevent the job from running until it’s approved. Otherwise (i.e. when pull requests belong to the main repository), the job will run without requiring explicit approval.

The test job is where secrets would be used. It needs the previous job, so it will never run without explicit approval. The security of this approach is based on the idea of a human approving every run after making sure that there is no malicious code on them, hence it also overrides the ref from actions/checkout to run on the pull request branch rather than on the main branch.

Alternatives

Admittedly, adding this authorize job to the workflow isn’t particularly elegant but, as of January 2023, GitHub doesn’t provide any official guidance on how to achieve a similar result in simpler ways.

• In 2020, GitHub introduced an option to send secrets to workflows from fork pull requests, but it only has effect on fork pull requests from private repositories.
• In 2021, GitHub introduced an option to require approval for all the outside collaborators, but the pull_request_target event will trigger regardless of the approval settings.

Other common alternatives include: skipping tests that need access to secrets, disabling forks, and using pull request labels or code review approvals to control the execution of tests.

Security Testing

This approach has been tested by sporadic security researchers who found our repositories while looking for the pull_request_target trigger, but none of them (#1130 [1] & #1322) were able to bypass this protection. If you find out a way of bypassing it, please feel free to put our bug bounty program to good use.

Now you have it! As far as we know, this is currently the most elegant GitHub Actions configuration for testing pull requests from public repository forks using secrets. As maintainers of a lot of open source software, this is close to our hearts!

Here are some example usages for cml and mlem.

Do you have any better alternative or maybe a similar use case and want to discuss more? Join us in Discord!