GatekeeperQA

Developer quick start

Updated

Run automated accessibility checks locally or in GitHub Actions using the MIT-licensed @gatekeeperqa/cli@0.3.0. Use Node.js 24. You do not need access to GatekeeperQA's source repository or a hosted scanner account. Local and CI execution do not consume public web scanner allowances.

1. Install the published CLI

From your application's directory (with an existing package.json):

npm install --save-dev --save-exact @gatekeeperqa/cli@0.3.0
npx --no-install playwright install chromium

For an empty directory, run npm init -y first. On a Linux CI runner, install Chromium with system dependencies using npx --no-install playwright install --with-deps chromium.

2. Choose the pages and policy

Create gatekeeperqa.json:

{
  "urls": ["http://127.0.0.1:3000/"],
  "failOn": ["critical", "serious"],
  "warnOn": ["moderate", "minor", "unknown"],
  "timeoutMs": 30000
}

Start your application and wait until it is ready. Replace the URL with a page you own or have permission to test. The CLI scans 1–20 explicit HTTP(S) URLs; it does not crawl the site. URLs containing credentials, query strings or fragments are rejected. Login flows, interactions and custom readiness checks are not supported.

3. Scan and read the result

npx --no-install gatekeeperqa --config gatekeeperqa.json --output reports

Open reports/report.html for the readable report. Other outputs are report.json, report.md, and report.xml (JUnit). The CLI does not directly export PDF; PDF is available through the browser interfaces.

ResultExit codeWhat to do
PASS0Review scope and remaining manual checks; this is not a conformance certification.
WARN0Review findings below the failure threshold and unresolved automated checks.
FAIL1Inspect failed policy thresholds or page execution errors before accepting the run.
Setup/configuration error2Read the command log, correct the problem, and rerun. A new report may not exist.

An incomplete page cannot produce a passing gate. Disabling severity thresholds does not suppress execution failures or unresolved-check warnings. Use a fresh output directory in automation: an unsuccessful run must not be confused with reports from an earlier run.

4. Use the GitHub Marketplace Action

Get GatekeeperQA Accessibility Evidence on GitHub Marketplace. Version 1.0.0 uses the published CLI 0.3.0 with locked dependencies. No GatekeeperQA account or API key is required.

This is a job snippet, not a complete workflow file. Add it to an existing workflow, after choosing its trigger. Commit the configuration from step 2, start your application, and wait for its health check before scanning. This example pins the tested release commit.

permissions:
  contents: read
jobs:
  accessibility:
    runs-on: ubuntu-24.04
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false
      # Install/start your app and wait for its health check here.
      - uses: gatekeeperqa/accessibility-action@c7ba35dab80e9d6f65ecc36440a205df15c9235e # v1.0.0
        with:
          config: gatekeeperqa.json
          artifact-name: accessibility-reports

New to GitHub Actions? Use the complete manual-run workflow below to scan an authorized public URL without installing or starting your own application in CI. It runs the same CLI directly. For localhost targets, keep your application startup and readiness steps in the job above.

The Action saves JSON, Markdown, HTML and JUnit evidence before enforcing the result. PASS and WARN succeed; FAIL and ERROR fail the job. Reports are retained for seven days. Setup failures can produce no report; unreachable pages fail the gate. Use a unique artifact name for each invocation.

For untrusted pull requests, use pull_request, not pull_request_target, and do not expose secrets. Review report contents and artifact access before scanning private pages. Merge blocking requires separate branch-protection settings. Automated checks support human review and do not certify conformance.

Release notes and source · Successful verification run

5. First run without starting a local application

Copy Download accessibility.yml into your repository as .github/workflows/accessibility.yml. The complete copy-ready workflow is below. It installs the published package, saves reports before enforcing the gate, and requires only read access.

After committing it to your default branch, open Actions → Accessibility check → Run workflow, enter an authorized public page URL, and run it. No GatekeeperQA API key is required.

name: Accessibility check
on:
  workflow_dispatch:
    inputs:
      target_url:
        description: Public URL you own or are authorized to test
        required: true
        type: string
permissions:
  contents: read
jobs:
  accessibility:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
        with:
          node-version: '24'
      - name: Install scanner
        working-directory: ${{ runner.temp }}
        run: |
          mkdir gatekeeperqa
          cd gatekeeperqa
          npm init -y
          npm install --save-exact @gatekeeperqa/cli@0.3.0
          npx --no-install playwright install --with-deps chromium
      - name: Scan authorized URL
        id: scan
        working-directory: ${{ runner.temp }}/gatekeeperqa
        env:
          SCAN_URL: ${{ inputs.target_url }}
        shell: bash
        run: |
          node -e 'require("fs").writeFileSync("gatekeeperqa.json",JSON.stringify({urls:[process.env.SCAN_URL]}))'
          set +e
          npx --no-install gatekeeperqa --config gatekeeperqa.json --output reports
          result=$?
          set -e
          echo "exit-code=$result" >> "$GITHUB_OUTPUT"
          if [ "$result" -eq 2 ]; then
            mkdir -p reports
            echo "Scanner setup or configuration failed. No passing result issued." > reports/error.txt
          fi
      - name: Save reports
        if: ${{ always() && steps.scan.outputs.exit-code != '' }}
        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
        with:
          name: accessibility-reports
          path: ${{ runner.temp }}/gatekeeperqa/reports
          retention-days: 7
          if-no-files-found: error
      - name: Enforce result
        if: ${{ always() }}
        env:
          RESULT: ${{ steps.scan.outputs.exit-code }}
        run: |
          case "$RESULT" in
            0) exit 0 ;;
            1) echo "Accessibility gate failed. Review the reports."; exit 1 ;;
            *) echo "Scanner did not complete successfully."; exit 2 ;;
          esac

Find the report after a failure

Open the workflow run and download accessibility-reports from Artifacts. Unzip it and open report.html. JSON, Markdown and JUnit files are included for completed scans. For setup/configuration errors, inspect the failed step's log and error.txt if present. If installation failed before scanning, there may be no artifact. Retention is seven days.

The workflow intentionally uploads evidence before failing the job. Do not add continue-on-error to a production quality gate.

A green GitHub check is not the same as a PASS report

The scan command returns exit code 0 for both PASS and WARN. GitHub therefore shows a successful job for either result in this workflow. Open the report to distinguish them. A successful report upload only means the files were saved.

Do not weaken the policy just to make the job green. Keep automated findings, scan failures and manual outcomes distinct. Read an annotated report example.

Troubleshoot the first failed step

Chromium is missing or will not start

Check the Install scanner log. Run the documented Playwright install command in the same directory as the installed CLI; Linux runners need --with-deps chromium. Confirm Node.js 24 and that package installation completed before the scan step.

The page could not be scanned

Check whether the runner can reach the target and receives a successful HTML response. A URL that works on your laptop may be unavailable to a hosted runner. If using localhost, start the application on that runner and wait for it to be ready. Check for login redirects, bot challenges, server errors and non-HTML responses; the CLI does not perform sign-in or dismiss barriers.

The scan times out

Confirm application readiness and connectivity first. The CLI waits for the page load event and runs automated checks within timeoutMs. For a legitimately slow page, adjust the configuration within the supported 1,000–120,000 ms range. In the downloadable workflow, add that property to the JSON generated by the Scan authorized URL step. A longer timeout does not add custom interaction or content-readiness checks.

No report or artifact appears

Check the earliest failed step. Installation failures happen before the scan and may leave no artifact. Exit code 2 can mean invalid arguments, configuration, runtime setup or output-write problems; a complete report is not guaranteed. Look for error.txt when the workflow created it. Confirm the upload path matches the scan output directory and that the seven-day artifact retention has not elapsed. Never use an older report as evidence for the failed run.

The job is green but the report says WARN

This is the documented exit-code behavior, not a missing failure. Read the warning evidence and assign the unresolved work. Branch protection requires the check to succeed; it does not turn WARN into FAIL.

The result does not match the change in the pull request

Confirm the scanned URL serves the PR build. The manual workflow scans a deployed page and does not check out or build your application. Compare the same URLs, policy and page states after deploying the fix.

Adapt it for pull requests

The workflow above is a manual check of a deployed URL, not a check of changed PR code. For PR validation, add a pull_request trigger and either start the checked-out application on the runner or use that PR's deployed preview URL. Wait for readiness before scanning. A scan of your existing production site does not validate un-deployed PR changes.

Use read-only repository permissions, avoid pull_request_target for untrusted code, and do not expose secrets to fork PRs. Configure the resulting check as required in repository branch protection or rulesets if it should block merging. Reports may contain page content; choose targets and artifact visibility accordingly.

Optional account and handoff tools

Save a CLI report to your account

With CLI 0.2.0 or later, you can explicitly upload a report to My reports. Sign in, open Upload CLI reports to this account, and create an upload-only token. Store it as GATEKEEPERQA_UPLOAD_TOKEN in your environment or CI secret store. Do not put it in source code or command arguments.

# Check locally; no token or network request needed:
npx --no-install gatekeeperqa upload --report reports/report.json --dry-run
# Review the evidence first, then explicitly send it:
npx --no-install gatekeeperqa upload --report reports/report.json --confirm-upload

The upload sends URLs, findings, captured HTML/text and source mappings. Review for private information first. It does not run a hosted scan or fetch local/staging URLs. Unknown fields, local review metadata and rendered downloads are not imported. This does not add login handling or storage-state scanning to the CLI.

Refresh My reports to review, share or compare the result. Use the same URLs and scan settings for repeat runs. Uploaded reports preserve their original scan dates and are labeled as user-supplied evidence. For PDF, open the reviewed HTML and choose Print / save as PDF.

Uploads do not use hosted scan credits. Limits: 10 uploads/account and 100 shared/day (UTC), 20 distinct pages and 2 MB of JSON. The latest 10 saved reports, including uploads, stay for up to 30 days from saving. A new report can remove an older report and revoke its share link. Identical retained reports are reused without extending expiry. Upload privacy and token details.

Use upload secrets only in trusted CI runs; never expose them to fork pull requests or untrusted code. Upload success exits 0 even for a FAIL report, so preserve and enforce the original scan exit code separately. The downloadable workflow keeps its existing artifact upload and quality gate; hosted upload is optional.

Reuse a named page list

In your signed-in scanner account, open Reusable test sets. Create a name and enter one complete URL per line. You can save up to 10 sets, each with 1–20 URLs. Select Use for scan to fill the hosted form, review the pages and confirm permission; no scan starts automatically. Hosted limits stay at five public pages on one website. Use the CLI for larger or local sets.

For CLI 0.3.0 or later, copy the ID and current page-list revision from the set's CLI upload command:

gatekeeperqa upload --report reports/report.json --test-set SET_ID --test-set-revision 1 --dry-run
gatekeeperqa upload --report reports/report.json --test-set SET_ID --test-set-revision 1 --confirm-upload

Replace SET_ID and the revision with your saved values. Keep your upload token in GATEKEEPERQA_UPLOAD_TOKEN. The report must contain exactly the set's URLs; the server checks ownership and the current revision. Dry-run cannot perform those server checks. Upload tokens cannot read or edit test sets.

Renaming keeps the page-list revision. Changing URLs creates a new one, and earlier runs keep their original snapshot. Choose Report group to focus history, comparison and the progress chart on a set and revision. Old unassigned reports stay unassigned. A set name or matching URL list does not prove the page state and testing conditions were the same.

Test sets remain until deleted. Their reports still share your account's latest-10/30-day retention. Deleting a set leaves saved reports and existing share links intact until their normal expiry or revocation. Test-set privacy and retention.

Prepare a GitHub issue from a saved report

  1. Sign in to the scanner, open a saved report, and select a finding or manual check in Saved tester review.
  2. Save any review changes, then choose Create GitHub issue.
  3. Enter the destination as owner/repository. Check that you have access and whether the repository is public.
  4. Review and edit the title and body. The draft includes the page, rule, available evidence, mapped WCAG references, suggested next steps and retest guidance. Manual checks remain unconfirmed.
  5. Choose Open GitHub issue form. Review the draft on GitHub and submit it there. No issue is created automatically.

Captured HTML is excluded unless you opt in. Saved reviewer names, notes and source mappings are excluded. URL credentials, query strings and fragments are removed, but page paths, selectors and optional HTML can still reveal private information. Remove sensitive content before opening GitHub. The edited draft travels in URL parameters to GitHub and may remain in browser history.

For long drafts, use Copy GitHub issue body, open the issue form, and paste the body. The interface explains when this fallback is needed. Your repository and draft are not saved on our server. GitHub sign-in, repository access, issue templates and permissions still apply.

This action is available in hosted saved reports, including uploaded CLI runs. It does not add a GitHub token, automatic PR comments, issue synchronization or a new CLI release. Open your saved reports.

Scope and limitations

The published npm release is version 0.3.0. Later hosted reporting changes are not automatically included. Automatic PR comments are not implemented. Automated checks support human assessment; PASS is a configured quality-gate result, not full WCAG or Section 508 conformance.

Turn a failed accessibility check into a useful review

Start with the uploaded HTML report, confirm the target URL and check for execution errors. A scan of production does not validate a pull request that has not been deployed there.

  1. Locate the affected element and reproduce its page state.
  2. Use the evidence to prepare a fix; see the form-label example.
  3. Repeat the automated check on the changed application.
  4. Record manual-review evidence for interactions that automation did not cover.

Keep the original report and new run together; use the retesting checklist to record the build, page state, policy and manual outcomes. The npm workflow preserves reports before enforcing the result. A WARN exits successfully, so reviewers must still read the report. Branch protection is a separate repository setting; merely adding a workflow does not require it before merging.

Follow the complete video walkthrough

Watch the developer workflow with captions and a full transcript, then use the instructions above to repeat the steps on a page you are authorized to test.

Compare GatekeeperQA, WAVE and axe DevTools features and limitations.