An E-commerce site was redesigned with Web standards in mind. The revised site used semantic HTML markup that usually passes validation tests and also incorporated many common accessibility features. A study was carried out with screen-reader users to determine how well compliance with Web standards and accessibility guidelines translated into actual usability and accessibility.
Two participants were surveyed. One participant could complete all tasks in the study, sometimes with difficulty, while another participant, using an earlier screen-reader version, could accomplish no tasks in the study. Tasks took twice as long for screen-reader users than for a nondisabled subject also studied.
Specific features of standards-compliant coding are a clear factor in the first participant’s experience, while an older screen reader’s inability to handle valid HTML is a factor in the case of the second participant.
The World Wide Web Consortium’s Web Content Accessibility Guidelines (WCAG) are the de facto international standard for creation of Web sites accessible to people with disabilities. WCAG offers three compliance or priority levels; the lowest level, Priority 1, is adequate for many sites, though some site administrators prefer the medium compliance level, Priority 2. (A separate survey of 1,000 U.K. Web sites found that none met the highest level, Priority 3, which is generally viewed as unattainable on commercial Web sites.)
A standards-compliant site – with valid HTML and CSS – can automatically achieve some of the Priority 1 guidelines (like text equivalents for images, required in HTML), while also automatically meeting a Priority 2 guideline (3.2, which requires valid code).
But does standards compliance automatically translate to usability and accessibility for people with disabilities? This study attempted to shine light on that question.
The site tested is an E-commerce operation that sells software and hardware. Its recent redesign for standards compliance included, among other changes, a reconfiguration of its navigation structure to use tabbed navigation; traditional lists of links in navbars; and browsing and search mechanisms. The site also allows a visitor to compare a number of products by selecting them and choosing a Compare function.
Accessibility for people with disabilities is a broad field, and a single experiment could not hope to encompass all disabled group. It was decided to focus on a group for whom standards compliance is known to be important – screen-reader users.
Dummy accounts were set up, with userIDs of the form screenreaderA through screenreaderF and passwords of the form passwordA through passwordF. All participants were named Chris as far as the site was concerned. Participants were assured that any purchases would be purely hypothetical. To safeguard privacy, the experimenter’s credit card was used to check out products; sales on this E-commerce site were voided afterward.
A list of tasks on the candidate E-commerce site was developed. Each task was fine-tuned to be plausible and representative of common visitor functions. Where product searches were involved, tasks were selected to produce a manageably-small set of product listings. (The products used as examples – a mousing surface, a French keyboard, and a game controller – were unlikely purchases for English-speaking blind consumers, but were sufficient to test the site’s functions.)
Participation was limited to Toronto-area residents who used any make or model of screen reader. Service organizations (CNIB, Balance); mailing lists (NFB:AE, WAI-IG, WebAIM, Blind-L, UVIP-Web-Test); a usability social club (Toronto Interacts); disabled-student-services offices at University of Toronto, York University, and Ryerson University; and personal contacts were all engaged. Budgets did not permit paying honoraria, though the equivalent of gift certificates were later offered.
Five participants volunteered for the study. Due to various commitments and conflicts (often job-related), only two were able to carry out the experiment. Results must be viewed as limited but suggestive. Qualitative research of this sort resists statistical analysis, so any effort to impose statistical significance on the results would be misplaced.
A nondisabled subject was also used as a point of reference, though this should not be taken as any kind of control group.
A privacy policy was developed for the study. Each participant read an online version and signed a printed version. Among its important features was anonymization and generalization of personal information. The only details revealed in this report are those that do not uniquely identify the participant; the screen-reader users in this study are named only as Participant A and Participant B, while the nondisabled subject is named Participant C.
For completeness, participants’ workspaces were photographed (with identifying objects removed or obscured).
According to the prospectus for the project, screen-reader users would be visited at their home or office or any other place where they typically browsed the Web. They would use their own equipment customized to their own specifications. This is a departure from traditional usability testing involving nondisabled subjects, who are often invited into the researchers’ facility to use the researchers’ own equipment. That protocol is not practicable with users of assistive technology, who have considerable investment in setting up preferences, keystrokes, and general work environments that conform to their needs.
Participant A was interviewed in her office, Participant B in his home. The experimenter went through a script with each participant interview.
Sessions were approximately transcribed, and these narratives are available:
Screen-reader users participating in the study were asked to complete a verbal questionnaire to collect information about their age, sex, and visual and other disabilities; computer, screen-reader, and browser usage; and general experience of the Web. The nondisabled subject did not fill out this survey. The data in the questionnaire have been anonymized and generalized from those responses.
Participants A and B are expert screen-reader users who are accustomed to real-world Web usage. Nonetheless, results differed markedly between Participant A and Participant B, chiefly due to the screen-reader version in use.
Participant A completed the test in 1 hour 10 minutes, Participant B in 1 hour 3 minutes, and nondisabled Participant C in 26 minutes.
In Participant B’s session, the speech produced by the screen reader clearly did not represent the page as seen on the computer monitor. Entire sections of the page were simply not read aloud even after Participant B used specific Jaws keystrokes to explore the full page. Yet these were the same pages that Participant A was able to read, usually with little trouble and with only occasional recourse to navigation keystrokes and manual exploration. In short, Participant B’s screen reader was not rendering the page as intended by the author.
Participant A used a more-recent version of the Jaws screen reader (version 5) than Participant B (version 4.51). According to the Jaws release notes, support for basic HTML and CSS has improved in each version. It’s open to question why a leading screen reader, retailing for up to $1,732, would understand basic HTML and CSS only in a fifth full version release. Even so, Participant B’s failure can be chalked up to inadequate HTML and CSS support in his adaptive technology.
Both screen-reader users were accustomed to spending much longer on many tasks than nondisabled people. They did not view the time spent on this study’s tasks as excessive (even though Participant B could not complete any of the tasks). We can tentatively conclude that shopping online takes about twice as long for screen-reader users than for nondisabled people. Due to the sequential nature of screen readers – where users listen to or read Braille output of a Web site in linear order – it may not be possible to significantly improve this time difference even with highly-compliant sites.
Choice of terms turned out to be important. The experimental script asked participants to “log into” the site, though the site itself uses the term “Sign-in.” Participant A was able to find the sign-in section after checking through a list of links. Changing the terminology to “Sign in/Log in”; or using “Sign in” and adding “Log in” in text marked up as visibility: hidden; or simply adding a title="Log in" to the sign-in link might marginally improve usability.
ul li a); the site could add richness by also marking up the tabs as headings (ul li h4 a). That markup is slightly more semantic than the site’s existing HTML and would make the tabs available to two different Jaws functions (List of Links and List of Headings).Participant A was confused by the need to update quantities in a cart before checkout. That step is farther down on the page and in the source code – easy enough to notice in a visual display, but nearly the last information heard in a screen reader. If this step can be eliminated, it should be. If not, place a warning nearer the top of the page listing the steps that visitors need to take to check out correctly, including updating quantities.
Participant A could not use the site’s Compare function, which requires the visitor to select checkboxes alongside products displayed in an HTML table. The checkboxes, though marked up in valid HTML, are too far removed from the rest of the data in the table to be understandable in a screen reader. In part this is an issue of associating HTML labels with the checkbox control. Some possibilities for remediation:
label markup using the for="" attribute to explicitly associate it with the form field being labeled. The HTML specification places no restrictions on placement of a form label inside or outside the table cell that contains the form field being labeled.It is generally unwise to tailor site content or markup for a specific browser (since that contradicts the principles of Web standards), but some small improvements could be helpful in this case.
title element of each page contains a toll-free phone number and the hostname www. and causes lengthy delays in speech output. The punctuation used in the phone number is read literally at some Jaws verbosity levels, and www. adds ten syllables.It appears that mere standards compliance does not always translate into full accessibility for screen-reader users. First of all, the screen readers themselves fall down on the job of interpreting compliant HTML and CSS. Moreover, some site designs whose interfaces are understandable when viewed in a gestalt do not translate well into sequential viewing.
Even with the limited cohort base in this study, it is apparent that relatively minor site improvements could remediate most of the problems encountered in the survey.
– 30 –
2005.03.24 ¶ Joe Clark