Over-reliance on automated PDF accessibility checkers

The PDF accessibility checker built into Acrobat Professional (versions 11 onwards) is a useful tool. However, it is important to understand that no automated checker will test the accessibility of a PDF and give you a yes or no answer.

I have written about this previously elsewhere, but am prompted to do so again after hearing twice in a week of web managers of large corporates commissioning external graphic designers to create accessible PDFs. In both cases the designers were simply instructed that their documents must pass the Adobe PDF accessibility checker, and nothing more.

The benefits of automated PDF accessibility checkers

The Acrobat PDF accessibility checker has many excellent features that can help greatly in identifying and fixing potential problems.

The screenshot above shows the checker’s results panel highlighting a number of potential problems. In many cases you can simply right/ctrl-click an item to get a menu that includes the magic word “Fix”. Often, simply clicking this will fix the problem, or else provide simple guidance on how to do so.

All well and good, so far. 

So what’s the problem?

Although this is a very useful tool it can only identify certain types of problem. It in no way constitutes a reliable test of whether the document is accessible or not

False positives (type 1 errors) (convicting an innocent party)

For example, the checker is capable of the occasional type 1 error. That is, it may tell you that a skipped heading level is a problem when it isn’t (not all skipped headings are accessibility problems).

False negatives (type 2 errors) (guilty party goes free)

But more importantly perhaps, the checker is capable of returning numerous type 2 errors. For example, it cannot tell you if:

• Headings or lists are tagged as ordinary paragraphs
• A table of contents is present and correctly linked and tagged
• Appropriate bookmarks are present
• Page numbering is set correctly
• Appropriate alt text has been assigned to images
• Footnotes are accessible
• Reading order is correct for screen readers
• Reading order is correct for non-screen reader assistive technologies
• Internal links have appropriate zoom settings
• The correct document language has been set
• An appropriate document title has been set
• There are spaces missing at line ends, or inappropriate spaces in the middle of words
• Background colours or image transparencies will cause problems for people requiring customised colour schemes
• The file name is too long for some assistive technologies to be able to open the document (this could, of course, be checked for, but currently isn’t)

The above list is by no means exhaustive, and yet, getting these things wrong can seriously compromise the accessibility of your documents.

Hence, it is just not safe to rely on automated checkers alone to benchmark the accessibility of your PDFs. If you are serious about accessibility, you must test them manually as well, using a wide variety of checks, including with a range of assistive technologies (not just screen readers).