Conformance score

Who can use this feature? 

• Any user with access to manual evaluations.  
• Available for all packages with manual evaluations.       

This article describes how to calculate, improve, and understand the conformance score for your evaluations. Also, learn how to differentiate it from the accessibility health level score. For more information, refer to Check evaluation results and Check the Conformance report.

Note: You must manually select the applicable standard used in testing, for evaluations that lack this information. You cannot change a standard, after you select it. If you selected a wrong standard, this will result in an inconsistent report. However, you can contact your Customer Success Manager or submit a Support request, and we can change the selection you made.

On this page:

• What is the conformance score?
• How is the conformance score calculated?
• How do I improve my conformance score?
• What's the difference between the conformance score and the accessibility health score? 
• FAQs

What's the conformance score?

The conformance score is a measure of accessibility based on evaluations. It shows the degree to which a digital asset meets or violates the tested success criteria.  

Over time, the score can be a measure of relative change. Notice your scores improve as you fix accessibility issues.

Note: You'll only notice changes in your conformance score after validation testing from Level Access.

To view a conformance score, select a digital asset, select Evaluations, and then the Conformance tab. For details, go to Conformance report.

How is the conformance score calculated?

The conformance score is inspired by the VPAT framework. It measures accessibility of each criterion using findings by severity and assigning a status that has a corresponding score. The final conformance score is the average score of all tested criteria. There are three possible calculations for the conformance score:

• Default calculation: Uses the three default severity levels.
• Four severity levels: For customers that have one additional custom severity.
• Five severity levels: For customers with two additional custom severities.

Unless you have custom severities set up in the platform, the conformance score uses the default calculation. Learn more about custom severities.

Note: Conformance score calculation differs from the accessibility health score calculation.

Default calculation

The table below describes the conformance statuses and corresponding score based on three severity levels.

Four severity levels

The table below describes the conformance statuses and corresponding score based on four severity levels.

Five severity levels

The table below describes the conformance statuses and corresponding score based on five severity levels.

How do I improve my conformance score?

Note: You'll only notice changes in your conformance score after validation testing from Level Access.

The conformance score is calculated in a way that gives you clear next steps to improve it based on tested criteria and finding severity. Because the conformance score is the average score of all tested criteria, it's best to focus on one criterion at a time and to fix findings in the order of severity, highest to lowest. Once you fix all findings under a single severity, the score for that criterion improves and impacts the overall conformance score.

Follow these best practices to see changes:

• Focus on one criterion at a time.
• Fix from highest severity to lowest.
• Focus on A and AA success criteria first, then shift your focus to AAA.
• Select your ideal prioritization option and follow the recommendations.

If you don't follow these best practices, your score may not improve because it's calculated using an ordered framework based on finding severity. The framework also requires that every finding under a severity level to be fixed to notice score improvements. For example, if you fix 9/10 critical findings, your score won't improve. Whereas if you fix 10/10 critical findings, you'll notice a score improvement.

The table below describes the score increases for a single criterion when all findings at a given severity level are fixed.

What's the difference between the conformance score and the accessibility health score?

The conformance score and accessibility health score are both measures of accessibility. At this time, the conformance score measures manual evaluations based on tested criteria and the accessibility health score measures on-demand scans and monitoring using rules. Beyond just what they measure, the key difference is that the conformance score uses a structured framework that provides a clear path to improved accessibility. 

FAQs

Where can I learn more about manual selection of standards? 

Review Check evaluation results and the section FAQs about manual selection of standards. 

How the statuses of the findings affect the score?

Why is the score lower for the A criteria if there are more criteria to meet in AA?

WCAG levels are cumulative — Level AAA includes all criteria from A and AA, as well as additional advanced checks. At a high level, the conformance score is calculated based on the number of criteria that are:

• Supported (no findings),
• Partially supported (high or low severity findings), and
• Not applicable (which are treated as fully compliant and positively impact the score).

For example, assuming approximate counts for WCAG 2.2:

• Level A → 31 criteria → each one weighs ~3.22% (100 ÷ 31)
• Level AA → 55 criteria → each one weighs ~1.81%
• Level AAA → 86 criteria → each one weighs ~1.16%

Consequently, if the same criterion is marked as “Not Supported“ (has critical findings) across all three levels, it will have a greater negative impact on the Level A score than on the Level AAA score. Additionally, because “Not Applicable” criteria are fully credited, they can lift the overall score. Because higher WCAG levels include more criteria, they often have more that don’t apply to certain digital assets — which results in a higher final score.

For WCAG standard, does selecting AA include the A criteria?

Yes, WCAG versions are cumulative. For example, WCAG 2.2 AA contains the success criteria of the WCAG 2.2 A.