Certification and testing

AUTHOR’S NOTE – You’re reading the HTML version of a chapter from the book Building Accessible Websites (ISBN 0-7357-1150-X). Copyright © Joe Clark, 2002 (about the author). All rights reserved. ¶ Back to Contents

Rather like being forced to apply for a permit to hold a demonstration, it seems contradictory to be expected to certify one’s Website. The Web is supposed to be “anarchic,” right? Anything goes.

That, at least, is the dream. The reality? Compliance, whether legally enforced or not, is an increasingly pressing criterion in Web development. You may be required to certify that your Website is accessible, or may voluntarily choose to do so. Then there’s the issue of testing your site to prove that it is accessible.

Ideally, both certification and testing would be automated processes akin to spellchecking in a word processor. I will continue my tradition of heresy and iconoclasm by explaining just how difficult automated certification and testing actually are. Along the way, you will learn how to develop and post accessibility policies.


In this chapter:

Certifying accessibility

In industrial applications (like manufacturing an engine component), we are faced with the issues of spec and tolerance – what the standard or specification requires and how close you come to meeting it.

In Web accessibility, the specification from which all others derive is the Web Accessibility Initiative’s Web Content Accessibility Guidelines (WCAG). You’ve got three levels of compliance, as I described in Chapter 5, “The structure of accessible pages.”

In practical terms, Level A compliance is easiest to attain and does the greatest good in accessibility terms because Priority 2 and 3 requirements are harder to meet, serve smaller and smaller audiences, and apply to less and less common Web techniques.

Certification and policies

A terminology issue here. A certification level forms only one part of an accessibility policy. Certification levels relate to WCAG requirements; a policy stipulates the promises and requirements you and your team will abide by.

In practice, there may not be all that much of a difference. Your policy may simply say “Our pages conform to Level AA compliance as articulated by the Web Content Accessibility Guidelines,” short and sweet. Or your lawyers may demand something more lengthy and qualified.

Choosing a certification level

Unless you are working for a government department or the rare “private” organization with actual requirements, you will not be obligated to certify the accessibility of your sites. If you actually are required or simply elect to do so, you need to decide which level of certification you wish to claim.

Do not mistake this for an easy decision. The salient issue is the future vs. the past.


Through concerted effort and after a lot of internal training (based largely on this book), it may be possible for your Web team to attain, for example, Level AA compliance on all new Web pages and templates. OK. What do you do with the two thousand database-generated pages already available? It is not impossible to retrofit even many thousands of old pages, no matter what anyone tells you; it is merely expensive to do so. If you’ve got the money, do it.

I would be very surprised if even a single reader of this book were willing and able to retrofit more than a dozen old Web pages. I myself am an expert on the topic and should have tremendous motivation to get my own house in order, but of the 200 or more old documents on my various Websites, I have rewritten, upgraded, and retrofitted only about 60. That does not imply that the remaining pages are inaccessible; they are merely unappraised or less than ideal (due largely to the sloppy HTML 2.0–era coding practices, a phase all Web old-timers went through).

It is much more likely that you will adopt an approach similar to the U.S. government’s Section 508 regulations (described in Appendix A, “Accessibility and the law”): Old pages may remain unappraised and potentially inaccessible as long as they are untouched. But if an old page is updated or altered in any way, no matter how minor, full-on accessibility must be provided. Also, a page must be upgraded on reasonable request from a visitor.

Such a policy in no way represents shirking your responsibilities or copping out. In human-rights legislation throughout the Western world, accommodation must be provided for people with disabilities up to but not beyond the point of undue hardship or burden. There are limits to what can be done, and those limits are not always the same as refusing to do the work because it is very difficult and expensive. Even very difficult or expensive accessibility provisions may nonetheless be required; undue hardship is defined differently in every case. But there is no blanket requirement to do absolutely everything conceivable to improve access. (It may surprise you to hear that I fully support this limitation. It is a necessity in the real world.)

Accordingly, I can’t give you a specific rule to live by here. I cannot offer a magic threshold combining staff complement, budget, and back catalogue of existing Web pages above or below which you must or must not retrofit those pages. I will, however, offer some guidelines that are consistent with other advice given in this book:

What to look at?

When retrofitting old pages, you have to decide what issues to consider, a decision intricately related to certification level (Level A vs. AA vs. AAA). It would be nice to be able to automate the process.

Some utilities and services almost make that possible. The CD-ROM included with this book contains a copy of the A-Prompt program from the Adaptive Technology Resource Centre at the University of Toronto, which walks you through topic after topic in Web accessibility for selected pages. (The program is small enough and self-explanatory enough that I’m not documenting it here. Later versions may support Mac OS under Java, but for now A-Prompt is Windows-only.)

You can find a few other accessibility testing services, many of them available for a fee, on the Web. There’s a whole list over at the Web Accessibility Initiative site: w3.org/WAI/ER/existingtools.html.

These programs will attempt to identify problems and let you fix them. I’m going to rain on your parade a bit here and tell you that there is no substitute for human oversight of each retrofitted page. Even if you have an automated tool that alerts you to a missing alt text, you still have to write it. That will probably require looking at the graphic itself (or at least the filename – you can probably tell at a glance that spacer.gif will take alt=""). Then you may further wish to add a title, and, in rare cases, a longdescription. Can software do that for you? Hardly.

I will strengthen my reputation (notoriety, shurely?!) for heresy and iconoclasm by advising that you might as well forget about Bobby (cast.org/bobby), the alleged Website accessibility testing tool that is always immediately mentioned by everyone who doesn’t know the first thing about accessibility.

Bobby is just too primitive and outdated, and, far from solving the problem of human intervention, Bobby makes matters worse, flooding your monitor with error messages along the lines of “This may be a problem. You should look at it.” Um, yeah. I should look at it, and I don’t need computer software nagging me about it.

At any rate, to explore a real-world example, if you find that the same shared graphics are unannotated on page after page, you’ll yearn for some kind of automated modification system, not just an automated identification system. Perhaps a site-wide search-and-replace will do, unless you were using sloppy HTML 2.0–era coding practices at the time and were not consistent in the order of attributes inside a tag. (Did you write the tag as <img height width> sometimes and <img width height> some other times?) As with automating accesskey and tabindex (Chapter 8, “Navigation”), this is the sort of thing a database should do for you, not that any such program actually exists.

For every retrofitted page, then, you the designer or developer must survey the page’s condition and make informed, specific, conscious decisions on how to fix problems. Such a procedure cannot be automated.

Accessibility advocates are hesitant to admit all this; we all seem to want to pretend that accessibility is easy. Conversely, when confronted with the prospect of retrofitting large numbers of pages, developers tend to immediately reject the notion out of hand. Both camps are overreacting. Accessibility generally is easy, as access advocates claim – for new pages, at least. And it is a lot of work to fix up old pages, as developers claim. The degree to which it is actually practicable varies case by case, and that determination must remain yours.

But there is a parallelism at work here. Just as repairing access defects in old pages requires human judgement and equivocation, so does evaluating the entire issue. You must decide how much work you are willing and able to do, and, while doing it, you must decide how that work will express itself.

New pages

Life is so much simpler when it comes to producing new pages. Here you have no choice. You absolutely must include accessibility features. Priority 1 or Level A access will do just fine to start, unless your content demands a higher level (as with online video). I’m not talking about legal or regulatory requirements, though you may have those to deal with, too; this entire book is based on the premise that you cannot build a Website in the 21st century and beyond without including accessibility features, no matter how modest.

Making a final decision

You are under no obligation to decide on the same certification level for old and new pages. The following combinations are all defensible depending on the resources at your disposal:

Any policy that was reasonably considered after a full and thorough understanding of accessibility issues and, after honest assessment of what your team can actually do as opposed to what it feels like doing, will suffice.

Line-item exceptions

Remember the issue of spec and tolerance?

What happens if you can meet every WCAG provision except one or two? What if, in other words, you cannot accommodate certain line items in Priority 1, 2, or 3 requirements?

The most likely case is, I suppose, an inability to make multimedia accessible on a page where everything else meets the spec. In any case such as this, you had better have a very good explanation for such noncompliance. If you do, then a posted policy must explain which line items you do not comply with. Stating that you meet all guidelines but one is arguably better than nothing and has the virtue of honesty. Moreover, the policy can list specific pages or sections that do not meet a given guideline, or you could add such a notification to the pages or sections themselves.

By implication, then, you should not refuse to post a policy merely because you think you must meet every component of a WCAG requirement before doing so. Yes, technically you cannot pick and choose which items to comply with, but here in the real world it is occasionally unavoidable. And remember, there’s always a way to meet every part of the Guidelines, particularly for Priority 1; in the tough case of accessible multimedia, over time you will be able to provide a transcript at the very least. At that point, you may have fulfilled the entirety of that level of WCAG compliance, at which point you should update your policy.

Writing and posting the policy

The first rule of writing an accessibility policy is “Cut the crap.” Don’t waste our time with bromides of this sort:

[Company name] strives to provide market-leading user experience to its customers. To that end, we are proud to embrace cooperative standards for performance and user satisfaction. One of many [Company name] initiatives in this regard is accessibility to the handicapped. [Company name] undertakes to make best efforts to work toward ensuring that all parts of Companyname.com are accessible to handicapped people. Specifically, this page and many others at Companyname.com are optimized for World Wide Web Consortium (W3C) Web Accessibility Initiative (WAI) Web Content Accessibility Guidelines (WCAG) Level A (single-A)/Priority 1 conformance.

Don’t waste our time, OK? Life’s too short.

Tell us, in a sentence or two, to what standard your pages are written. (See the previous section to learn how simple it can be.) You may optionally wish to specify other standards you adhere to, like XHTML or Dublin Core metadata.

You have a few options in making your accessibility policy known. Absolutely place a link to it on your homepage (or all homepages, if you have multiple entry points, as a multilingual site might). It’s a must. A link saying “Accessibility policy” is all you need. I suppose you could give the link a certain accesskey and a relatively low tabindex. Any “About us” or corporate-information page must carry a link. So must a copyright or privacy-policy page, if there is one.

Optionally, you may include a link to the policy in page footers throughout your site. This may be gilding the lily a bit. It is theoretically possible to add such a notation even to old pages if they are constructed from a database where footers are specified by include files (or moral equivalent). Alternatively, for pages where your best current efforts aren’t very good (e.g., a page with multimedia you have not made accessible yet), you may wish to include a link to the access policy. You could even write slightly different wordings for different applications – one for general usage, another for troublesome pages, a third for new pages. (I would set up separate documents rather than using HTML anchors within the same document. You want people to know unequivocally what policy applies.)

You may also list accesskey assignments in your accessibility policy (as mentioned in Chapter 8, “Navigation”). That may be too specific a level of detail for a site-wide or company-wide policy.

Now, this book is not a thoroughgoing, general-interest manual on accessibility. I am not attempting to counsel you on how to reno your bathrooms so a person in a wheelchair can use them. Nonetheless, this is a golden opportunity for you to consider “enterprise-wide” accessibility, which could cover everything from workplace accommodations to TTY telephone access to provision of materials in alternate formats like audiotape.

If you don’t have such a sweeping policy ready, don’t wait for one before posting a Web accessibility policy. The latter could, however, be a nice talking point for and instigator of the former.

Legal compliance

I am not providing an opinion one way or another on the issue of complying with legal requirements apart from stating the obvious: Follow the requirements to the best of your ability. In practice, most of my readers affected by legal compliance work under the ægis of U.S. Section 508 regulations, which are not exactly the easiest things to understand, making full compliance a matter open to debate. If you are forced to comply with a regulation, you may also be forced to provide a statement of compliance; for all I know, eventually a form of outside auditing may be instituted to prove your claims. Meeting WCAG specifications may be quite different from these legal requirements, though in practice achieving at least Priority 1 accessibility gets you most of the way to meeting every other Web-accessibility rule on the planet.

If you are not actually required to meet certain standards, even those of the WCAG, but do so anyway and post an accessibility policy, it may be easier or harder for a visitor to file a complaint or a lawsuit alleging inaccessibility, that is, discrimination or unequal treatment. There is no jurisprudence in this area at time of writing, and even if there were, it wouldn’t necessarily apply where you happen to live. It seems self-evident that stating your level of accessibility compliance indicates forethought and responsibility, but that is mere supposition on my part.


How do you test Websites for accessibility? The same way you test Websites in general – with real people.

Usability testing has become an accepted component of the more expensive and sophisticated Websites. If nothing else, prototypes are tested on a couple of people in the office, or the developers themselves attempt to perform certain functions as if they were neophytes, or the company’s existing quality-assurance department, if there is one, is pressed into service to test the prototype. You can hire consultants and labs to perform user testing for you, if you can afford it.

However, while experts may be able to pretend to be neophytes, nondisabled people are not very good at pretending to be disabled. To a certain extent, it is possible: You can unplug the mouse and attempt to operate your site purely via keyboard, including the use of all navigation elements. (Suddenly accesskey and especially tabindex don’t seem so obscure after all.)

So how do you simulate being blind? You can’t just unplug the monitor or turn the brightness down completely, because then how do you interact with the system? You need a screen reader, don’t you?

What about simulating visual impairment? You can’t just bump up the font size on your Website; that is not how low-vision people use computers: The entire screen is magnified using software.

I’m sure you can see where this is heading. To do a better-than-half-arsed job of testing for accessibility, at the very least you need your own adaptive technology and, preferably, you need to include actual disabled users in your user testing.

Buying your own

I can highly recommend investing a bit of money in a screen reader. Several are in wide use, all on Windows: Jaws (by Freedom Scientific), Window-Eyes (GW Micro), and IBM Home Page Reader. You might as well forget about Macintosh screen readers until Alva Access Group upgrades OutSpoken for OS X.

In rough terms, screen readers cost a thousand bucks American, which isn’t a lot of money for a mid-sized firm. That’s about five hours of a project manager’s time at typical rates. If you’re a Web consultancy, you can bill the cost to a client, or swing a deal with several clients to split the cost – and keep the program for your own use.

Adding to the overhead is the absolute requirement to run the screen reader on a separate computer. These programs are finicky and unstable; you do not want their tendrils infiltrating an important production machine. On the plus side, they run just fine on old iron.

So for a couple of thousand dollars American, you can test all your designs (for the life of the software license term) for screen-reader compatibility. Yes, you will probably test against one screen reader only, and I can certainly tell you that the three leading products interpret HTML – and generally behave – quite differently. You’ll also receive different levels of support when asking for help and reporting bugs (lots of it and immediately from GW Micro, next to none of it and begrudgingly from the other two).

Someone in the office will have to learn the labyrinthine keystroke commands to run the program. You may have to draw straws for that task, but I suspect you’ll find a keener on staff who is just geeky enough to find the idea kewl. Definitely use headphones for speech output, though having a good speaker (one is enough) on hand for meetings and presentations can be nice.

Do not unplug the monitor or attempt to simulate blindness while running the screen reader. You must be able to compare the full visual information against the auditory version. (Similarly, to evaluate captioning, you must leave the sound on, and to evaluate audio description, you must watch the visuals.) It may be somewhat overwhelming at first to correlate vision and speech in this way. You will get the hang of it after a few hours of acclimation, and remember that you can always rerun a test as many times as necessary until you catch everything. You can also slow down the speech rate.

If all this seems too expensive, note that each of the screen readers mentioned here can be downloaded in a demo version. Additionally, you can buy a time-limited version of Jaws (for 60 days’ use) for next to nothing. It would still be advisable to install it on a nonessential machine.

As for visual impairment: Since there’s not a whole lot you can do to craft your pages for magnification of the complete screen (see Chapter 9, “Type and colour”), it is strictly optional to buy a screen magnifier and run your own tests.

Disabled testers

Including actual disabled people is the gold standard of accessibility testing. It is, however, essentially impossible in practice.

It may make more sense to bring the test suite to the disabled user. OK – but where do you find them? And aren’t you running special surveillance software on your test systems to monitor keystrokes and mouse movements and to time responses?

Also, remember that in all cases you need to test for multiple relevant disabilities – blindness, mobility impairment, maybe deafness if you run multimedia. That will require more than one disabled testing population.

Consider the experience of the most famous usability consultancy on the planet, the Nielsen Norman Group, home of Jakob Nielsen. (Yes, I know, OK? Jakob Nielsen. I know.) The firm conducted a set of usability tests with disabled subjects while I was writing this book. Kara Pernice Coyne of the Group refused to disclose even how many subjects were involved, but confirmed that the Group was forced to search high and wide for blind and visually-impaired subjects, going so far as to get in touch with other accessibility experts and more or less beg them for contacts. Do you think you can do better? I could not.

Disabled employees

It is also possible that you yourself are a disabled person who does usability testing, either for your employer or as a consultant. Or you could be a disabled employee doing other work who is pressed into testing service as other employees might be.

This is, admittedly, rather unlikely. The public image presented by the Internet industry is one of ecumenism and equal opportunity, where no configuration of sexual orientation, hairstyle, embedded hardware like nose rings, fashion sense, or ethnicity could possibly be an impediment to getting hired if you have the right skills. In reality, Web firms hire within a specific range of employee types; while the range is larger than, say, the accounting business, there are actual limits to who is acceptable and who isn’t. (As an example, I am entirely unacceptable to Web firms for reasons of personality. On the upside, I can’t stand them, either.)

As described at the outset of this book, disability is upsetting and foreign to nondisabled people. Employers in every sector save for public service are unwilling to hire a disabled employee if it would require any accommodations whatsoever. While illegal, this bias is in wide effect. A blind Web programmer, for example, may be disqualified from consideration because of a claim that it is necessary to view and evaluate the appearance of client Websites, or at least because the blind employee is assumed to work more slowly than someone who can see. (It’s usually true. But accuracy may be higher. Still, discrimination is often irrational, so I won’t bother trying to argue against the bias.)

I would be very pleased to hear of even a single Web consultancy, or Web-related department of an old-economy consultancy, with a blind or mobility-impaired or even deaf employee working on Web design. I expect not to enjoy such a pleasure. In the hypothetical case that such a person does exist, I would urge the employer and employee to work out a way in which the disabled person’s experience and skills could help the testing process.

Having someone on staff can be a quick and easy way to find obvious problems. Do keep in mind, though, that a blind person cannot simulate the accessibility barriers faced by someone using switch access. On-staff disabled testers do not eliminate the need for more thorough testing, either in-house or out-of-house.

A problem in search of a solution

There is no immediately obvious or attainable solution for the problem of testing Websites with actual disabled users. Ideally, the industry will adopt the following changes:

Doing pretty much anything that puts actual disabled people to work testing Web accessibility is all to the good. If you can figure out a way to make it work, however imperfectly, I say go for it.

Bottom-Line Accessibility Advice

Basic accessibility

Write and post an accessibility policy. State the pages to which it applies – for example, new pages only or pages retrofitted after a certain date.

Intermediate accessibility

Certify the accessibility of your pages against a known standard, such as the Web Content Accessibility Guidelines priority levels. Note any known or deliberate deviations from those standards.

Advanced accessibility

Previous   ¶   Contents   ¶   Next