CI/CD

CI/CD is currently in beta. To learn more about joining the beta program, reach out to our team at beta@levelaccess.com.

Continuous Accessibility (CA11y) enables you to automate accessibility testing within your development workflows. By integrating directly with CI/CD pipelines, it helps you identify and resolve accessibility issues before deployment, ensuring compliance with accessibility standards.

On this page:

• Project setup
• Accessibility scan configuration
• Install the package
• Build the environment
• First analysis
• Pull Request Analysis
• Additional configuration and troubleshooting

• Self-hosted scans: Ideal for organizations with strict security or compliance requirements, allowing them to run accessibility checks within their own infrastructure.
• Managed scans: Suitable for teams looking for a cloud-hosted solution with minimal setup, where reports and results are processed and stored within the Continuous Accessibility dashboard.

With seamless integration into platforms like GitHub, Bitbucket and GitLab, Continuous Accessibility streamlines accessibility testing, providing automated reports, quality gates, and pull request analysis to maintain high accessibility standards throughout development.

Project setup 

Add a project

1. Access the dashboard by locate and selecting the “+” button in the top-right corner to start adding a project.
   
2. Select the source:
   • For GitHub, Bitbucket, Azure DevOps, or GitLab, select the relevant option to import the project directly. Sign in and authorize UserWay to read repositories and team information as prompted.
   • For unsupported platforms or manual setup, select “Add repository manually.” Enter a unique project name and main branch (already added).
     
     
     
     
3. Configuring Project Settings
   • Project name: Use a slug-compatible format (lowercase letters, numbers, hyphens only) to ensure compatibility across environments.
   • Main branch name: Specify the primary development branch (e.g., “main” or “develop”). The main branch has to be analyzed first in your project.
     
4. Optional settings
   • URL Parameters: Activate this setting to capture and analyze pages with unique URL parameters or hashes.
   • Page URL templates: Specify templates for dynamic URLs (e.g., /products/:id) to ensure accurate page detection.

Accessibility scan configuration

Set up accessibility scanning using our tools, integrated with end-to-end testing frameworks. We support Cypress, Playwright, Puppeteer, and Selenium for test automation.

Install the package

Install the relevant accessibility scan package for the framework. Run the command below in the project root directory:

• Cypress:

npm install --save-dev @userway/a11y-cypress

• Playwright:

npm install --save-dev @userway/a11y-playwright

• Puppeteer:

npm install --save-dev @userway/a11y-puppeteer

• Selenium (Java):

Add the Maven dependency to `pom.xml`:
<dependency>
  <groupId>org.userway</groupId>
  <artifactId>a11y-selenium-java</artifactId>
  <version>${USERWAY_SELENIUM_VERSION}</version>
</dependency>

• Webdriver IO:

npm install --save-dev @userway/a11y-webdriverio

• Selenium Python and C# are also supported.

Configure the accessibility analysis

1. Initialize the setup by importing and invoking the setupUserway() function within the test framework to configure accessibility scans.
2. Trigger accessibility scans:
   • Add userwayAnalysis() (or cy.userwayAnalysis() in Cypress) to test cases. This function performs static page analysis at specific points in test execution and saves accessibility reports.
3. Specify report options:
   • Reports are saved in the uw-a11y-reports directory. Customize report paths or include additional options, like element screenshots and violation tables, as needed.

Example for Cypress:

import { setupUserway } from '@userway/a11y-cypress';
setupUserway({ reportPath: './uw-a11y-reports' });
it('performs accessibility analysis', () => {
  cy.visit('http://example.com'); cy.userwayAnalysis();
});

Build the environment

Configure the CI/CD build environment to automatically trigger accessibility scans and manage reports.

Add the repository variables

Add a secure repository variable or secret with the USERWAY_TOKEN (already added) in each platform:

• GitHub: Open Settings > Secrets and variables > Actions, and add USERWAY_TOKEN.
• Bitbucket: Navigate to Repository settings > Repository variables, and add USERWAY_TOKEN.
• GitLab: Open Settings > CI/CD > Variables, and add USERWAY_TOKEN.
• Azure DevOps: Set up a service connection in the pipeline settings.

Update the CI/CD pipeline configurations

To integrate our solution within CI/CD pipelines, update your workflow file with the following steps:

1. Install the @userway/cicd-cli package:

   npm install --save-dev @userway/cicd-cli

2. Create the userway.config.ts file in the project root:

   import type { Config } from '@userway/cicd-core';
   
   export default {
     organization: 'your-org-slug',
     project: 'your-project-name',
     token: process.env.USERWAY_TOKEN,
     reportPaths: ['./uw-a11y-reports'],
   } satisfies Config;

3. Add the action or pipe to your workflow file:
   
   • GitHub (.github/workflows/build.yml):

     name: UserWay Accessibility Analysis
       uses: UserWayOrg/cicd-action@main
       with:
         token: ${{ secrets.USERWAY_TOKEN }}

   • Bitbucket (bitbucket-pipelines.yml):

     pipe: userway/cicd-bitbucket:1.1.8
       variables:
         USERWAY_TOKEN: $USERWAY_TOKEN

4. Before pushing changes to your branch and running CI/CD workflows, ensure that your userway.config.ts file and environment variables are correctly configured. This step helps prevent errors during the automated processes:

   1. Run end-to-end (E2E) tests locally to generate the scan artifacts. These artifacts will be saved in the /uw-a11y-reports/ folder.
      Refer to Accessibility scan configuration for these steps.

   2. Use the following command in your IDE’s terminal to upload the generated scan artifacts to the Continuous Accessibility (ca11y) backend service:

      npx userway

      • This command uploads the files from the /uw-a11y-reports/ folder.
      • If the upload is successful, you’ll see the results displayed on the CA11y dashboard.

   3. After verifying your configuration locally, proceed to run your CI/CD pipeline. This step ensures your configuration is validated within the pipeline.

First analysis

After setting up the project, trigger the first analysis to verify the configuration.

Run the first analysis

1. Confirm that the main branch is configured correctly. This branch will be analyzed by default.
2. After you finished setting up the files in your project, push these changes to your main branch to initiate the first scan of your project.

Review your results in the CA11y dashboard

1. After your pipeline is completed you'll see a passed or failed Quality Gate status in your dashboard.

Pull Request Analysis

Set up Pull Request (PR) analysis to maintain accessibility standards as code changes are introduced.

Configure branch and PR analysis

1. Trigger scans on branch updates:
   • Confirm that branches created for new features or fixes are analyzed by configuring the pipeline to scan all branches or specific PRs.
2. Review quality gates:
   • Set criteria for accessibility compliance in the PR analysis to enforce standards. This step can prevent the merging of PRs that introduce new accessibility issues.
3. Adjust and re-run analysis as needed:
   
   • Re-run scans or adjust quality gates based on team requirements and review feedback to maintain accessibility standards across updates.

Additional configuration and troubleshooting

Optional configurations

• TypeScript support: Add relevant packages to tsconfig.json for enhanced TypeScript compatibility.
• Ignoring Report Files: Add uw-a11y-reports to .gitignore to prevent report files from being committed.

Troubleshooting tips

• Failed Analysis: Review logs for errors in setup or permissions.

  • Try to re-run your scan with the verbose mode enabled to see more details:

    // userway.config.ts
    { verbose: true }
    
    // cli command
    yarn userway --verbose

• Token Issues: Confirm that the USERWAY_TOKEN environment variable is correctly set.
• Local Testing: Run npx userway locally with the project token to verify configurations before pushing to CI/CD.