Typography and appearance

Captioning is a medium that is all about written words. Typography is the instrument of captioning. But throughout the history of the medium, typography has been overlooked or mishandled. That history has reiterated itself with online captioning.

Issues

Captions (and subtitles) are typographic forms with a unique set of properties.

1. They are transcriptions or translations of spoken words. Some viewers – even many viewers with hearing impairments – will read the captions and listen to the audio, making captions a multimodal form.
2. Captions move (as by scrolling, crawling, or painting on) and appear and disappear. Though viewers can often record a production and watch it again, in broad terms you've got one chance to read a caption before it disappears.
3. Viewing conditions include both reflected and emitted light – reflected on movie screens and projections, emitted on TV and computers.
4. Technology affects resolution and colour: First-run feature films can do more than an old NTSC television can, for example.
5. Viewing conditions can be poor. People can sit too far from their televisions, or watch TV lying down, or use smudged and scratched reflectors in Rear Window® installations.
6. Some viewers can be expected to have multiple impairments. Even very slight blurry vision (as caused by an outdated eyeglass prescription) can severely diminish caption reading (Thorn & Thorn 1996).
7. Captioners and developers pay no especial attention to typography.

Screenfonts

This activity did not have a mandate or funding that encompasses a full exploration of caption typography, but we will note that custom-crafted screenfonts for captioning and subtitling are all but nonexistent.

The RNIB/Bitstream typeface Tiresias Screenfont has received attention as a claimed solution to every “subtitling” problem, but the typeface has a host of design flaws. It lacks even an italic, and Tiresieas has not been fully tested with representative audiences (deaf/hard-of-hearing, visually-impaired, and hearing).

In the field of online captioning, no custom-made and tested screenfonts exist. (Type designer David Berlow, however, tells this activity that his Charcoal font for Apple was designed in part for such usage.)

Today’s type

In closed captioning using video players’ own files, authors must specify a font and hope it exists on the viewer's computer system. For RealPlayer, things are worse still, since, according to Magpie documentation, RealPlayer understands only a predefined list of Windows fonts, with other predefined lists of substitute fonts for Macintosh and Unix. Font sizes, foreground and background colours, and alignment cannot be reliably specified across all platforms.

Some typefaces custom-designed for onscreen viewing are easy to find, notably Verdana and Georgia by Matthew Carter and Trebuchet by Vincent Connare. Nonetheless, those faces were not designed for reading captions and have not been tested for those functions. Captioners may, unfortunately, select any font they wish, including typefaces intended for printing (e.g., Palatino, Bookman), or 0lder system fonts imperfectly converted to outline form (e.g., Geneva).

Online captioning, then, gives authors no custom-made and tested screenfonts, lets them select any font they want, yet also does not guarantee that the font will actually be displayed for the viewer. While this is probably not the worst imaginable scenario, it is bad enough.

Visual impairment

It is interesting to note that Tiresias was developed as a captioning font for visually-impaired people. Yet the font was only lightly tested with that group, and the fact that most captioning viewers have normal vision was not considered.

Online captioning has advantages over television in that fonts are somewhat mutable. Authors can present normal- and large-sized fonts if they wish, for example, though that is often technically difficult. (Sometimes entirely separate streams are needed.) Colours can be selected, too, with reasonable reliability.

These facts confer some advantages to low-vision people who read captions, since they usually require bigger type at higher contrast than people with normal vision. However, neither players nor the underlying file types like SMIL make it easy to select different forms of type. In essence, not only has typography been ignored in the entire system, even the idea that a viewer may wish to customize the type has also been ignored.

Rendering

Rendering means the presentation of a typeface in a form perceivable to a viewer. On today's computer screens, fonts are rendered as dots or pixels. In some cases (as on liquid-crystal displays), those pixels can be divided into subpixels and individually manipulated to improve the appearance of a font. ClearType is most prominent technology, but CoolType can be found in Adobe Acrobat, and subpixel rendering is built into Mac OS X. (See articles at Typographi.ca and by Gruber and Riley.)

Rendering issues that impair legibility and readability include:

1. Caption fonts will look better with subpixel rendering, but such rendering can only improve what’s already there. With small sizes producing stem weights of a single pixel, some characters become hard to distinguish because there aren't enough pixels to manipulate. Stems may shrink or diminish when seen against some background colours.
2. Italics may not be available in some fonts, though software may render an oblique of a roman font. Jagged edges are more pronounced in such cases. Italics also make for a poor fit on the same line as roman characters in most captioning examples.
3. Foreground and background colours are oftensometimes unwisely chosen.
4. Leading is difficult or impossible to control, as are linebreaks in some cases, resulting in sometimes-lengthy lines of type with too little space between the lines.

Character encoding

Captions require reliable character encoding in order to survive the many-step process from authoring to storage to transmission to rendering.

In principle, Unicode character encoding can be used in captions.

• SMIL permits Unicode encoding by virtue of its being XML.
• SAMI does not support Unicode.
• RealText supports ISO-8859-1 and a few other encodings, but not Unicode.
• QTtext can employ Unicode encoding, though QuickTime can auto-guess another encoding if necessary.

While Unicode encoding is preferably because of its higher reliability (especially when using character entities), Unicode also encompasses some older encodings that can themselves be sufficient. Anything that can reliably transmit characters from author to viewer suffices.

Cross-platform issues

Incorrect character encoding is an especially pressing issue:

• when captions are mastered on one operating system but viewed on another (Windows-made captions viewed on Macintosh are a notorious example)
• when characters outside the Latin repertoire of the ISO 8859-1 encoding are used (as in Turkish or Icelandic), as these tend to be missing from some fonts or encoded specially on the author's computer
• when proprietary systems with nonstandard encoding are used to create a caption file or transcript (Microsoft Word exports are a notorious example)
• when files with correct encoding are saved in another format, including “text” (an ambiguous term), in which case higher-order characters like typographer’s quotation marks and dashes can be overregularized to neutral quotes and hyphens
• when users of one language's keyboard and operating system are confronted with another language (e.g., Scandinavians captioning English-language video on keyboards without apostrophes)

Difficulty for non-experts

Unfortunately, these problems are not easy to solve for grassroots caption authors. Unless the characters they use are solely limited to those printed on the keycaps of a U.S. keyboard, trouble can arise. And once the character encoding of a file is destroyed, it’s hard to recover the encoding.

Not only is fixing the problem difficult, so is explaining it, which should not be necessary in the first place. Authoring tools should make no mistakes with character encoding.

Examples

Character-encoding mismatch likely due to platform differences

Replacement of the ü character in kümmen is typical of a Windows-to-Macintosh transcoding error.

Characters outside common repertoires

These Beowulf and Sir Gawain and the Green Knight examples require Old English characters, some of which are still used in living languages like Icelandic and Faroese. Both use graphics for the Old English characters (and text for the modern English in the Sir Gawain case).

Research priorities

Clearly, caption typography pressingly requires research and development.

Fonts

Custom-made screenfonts need to be developed and tested with deaf/hard-of-hearing, visually-impaired, and hearing audiences.

Encoding

Software must take all the guesswork and confusion out of the task of using exactly the right characters in captions.

Reference

Thorn, F., & Thorn, S., “Television captions for hearing-impaired people: a study of key factors that affect reading performance” Human Factors, 1996.