Typographic requirements for captioning for HDTV

Note: I gave a presentation at the Deaf Way conference at Gallaudet University in 1989 on the topic of typographic requirements for captioning for HDTV. I was, of course, prescient: Everything I said we needed we do actually need, and most of it we are getting. I even predicted Unicode. By incredible coincidence, a later paper by other authors (who knew of this paper) made very similar points using very similar language. ¶ Note that this paper is over a decade old. I have no compunctions about leaving its overwritten pretentiousness intact. I’ve grown up a bit since then. The visuals accompanying the presentation are unavailable, but they’re not very important, either.

Introduction

Before I begin this discussion, let me offer my thanks to the Deaf Way for hosting such a forum as this. I also owe thanks to Al Gleason and Peter Reich of the University of Toronto, without whose help I would not be here today. Interested audiencemembers will find a much more detailed version of this paper in the Deaf Way proceedings.

Let us consider the phenomenon of captioning typography. Why? Because it's fundamentally true that captioning is a typographic medium. It takes the form of written words displayed on a screen, and for that reason, we should take typographic principles into account in all our discussions of the appearance of captions.

But as I have described previously in articles on captioning typography, captioners have to mix and match older typographic idioms from print with brand-new techniques created especially for captioning. The results vary in their levels of æsthetic and communicative success; some captioning styles are better than others at communicating a soundtrack, and some captioning styles are certainly more attractive than others. We have to keep the two criteria of communicative success and æsthetic satisfaction in mind when we talk about how captions look.

In this presentation, I offer a typographic manifesto for an enlightened captioning system one designed for the high-definition television of the near future. I will describe some of the problems of today's captioning and state some possible technological solutions in the areas of fonts, linguistic capabilities, and other requirements.

The present day

The captioning technology predominant in Canada and the U.S. is the Line 21 system, a fixture of modern television since 1981. For all its utility as a basic captioning system, in the latter '80s the Line 21 system has begun to show its age. Line 21 captions are functional as a means of making TV accessible to deaf and hard-of-hearing people, but in 1989 we deserve more than mere functionality. I hope to show that the more we put into a captioning system, the more we'll get out of it. The key phrase here is: “Use your imagination.” It's usually true that what we think is possible is usually way behind what actually is possible.

But here are some specifics about Line 21 captioning. In fact, the limitations of the Line 21 technology itself have determined how our captions will look as much as the artistic choices of individual captioners. Let me show you some examples of how the captioning technology itself cramps our style. One of the crucial factors in the legibility of typefaces is the presence of descenders. Descenders are the part of some lowercase letters which hangs below the common baseline of all the characters. Here is a diagram showing what descenders are [shows an overhead slide]. The arrows point to the descender parts of the letters g, y, p, q, and j.

Lowercase letters without descenders are much less legible; unfortunately, Line 21 closed-captioning forces captioners to caption in capital letters almost all the time, since the lowercase is illegible at prolonged reading. I'm going to show you a slide now which will illustrate the poor legibility of lowercase caption type [Slide 1]. Note that the descenders on the p and g are scrunched up into the bodies of the letters instead of hanging down as readers expect. Research has verified the obvious fact that reading extended text in all-capitals is more taxing and invites more errors than upper- and lowercase text.

From a Canadian standpoint, Line 21 typography is too underpowered to deal with the linguistic reality of television. The designers of the Line 21 system were apparently unaware of the fact that TV shows in Canada and the U.S. aren't limited to the English language. Although programs in French and Spanish are commonplace on the airwaves – Canada alone has several French-language TV networks – today's caption decoders contain essentially no accented characters to accommodate those programs. Most of the accents in decoder fonts are in lowercase. Consider the irony that captioners must set captions in uppercase most of the time. These facts combine to make it all but impossible to caption French and Spanish properly under any conditions. Proper captioning of other accented languages is just as difficult. As it is, French- and Spanish speaking captioning viewers have to put up with unaccented captions set entirely in uppercase; this slide in French [Slide 2] shows what that looks like.

Captioners are severely limited in positioning of captions on the screen; captions cannot always go where they need to regardless of the needs of the program. The slow transmission rate of captions impedes captioning of music and other fast-paced programming and makes it difficult to caption in two languages at once.

Line 21 captioning is laden with problems.

Regrettably, we must admit that Line 21 captioning has serious design flaws which have hindered the quest for typographic excellence in captioning. Knowing the problem is half the battle. Now we have a responsibility to consider ways to guarantee more exhaustive capabilities for future captioning systems.

By now everyone knows that high-definition television (HDTV) is on the way in the next several years if not right away. Regardless of the transmission standard(s) on which the authorities decide, high-definition components will be densely computerized devices. So, coïncidentally, are caption decoders. As such, closed-captioning has a natural ally in HDTV.

From both political and technical standpoints, it is important that persons interested in captioning and subtitling worldwide unite to propose a standard for closed-captioning of high-definition TV, a system I'll call HDCC. This HDCC standard will crucially depend on hardware in each high-definition TV set.

In my opinion, the most sensible plan is to require, by legislation or industry standard, that each high-definition set contain standardized hardware whose main purpose is the generation of characters. That means we should insist that each high-definition TV set contain a circuit board, or ROM, or other device to display captions and subtitles.

Typo-imperatives

Let us project ourselves into a fantasy world of the near future, in which we can design an HDCC system with all the necessary and desirable features we want. What features are required? Let's start with the most basic: worldwide compatibility.

It will be wise to devise an HDCC system which can be implemented on all high-definition sets no matter where in the world they are manufactured and sold. An obvious reason for this is the cost saving which manufacturers will enjoy if they can produce for a very large market. It shouldn't be difficult to devise an HDCC system which will work with all the high-definition formats which will ultimately be in place worldwide.

More interestingly, consider our requirements in fonts, or typefaces.

It's imperative that HDCC designers not repeat the sins which resulted in the illegibility and ugliness of closed-captions today. It will be a good investment to spend part of the HDCC development budget on hiring recognized digital typographers to adapt present-day fonts to HDCC work or, better still, to design new fonts for this new medium.

What will we ask of these typographers? To solve our current problem, which is the limitation of one typeface with few variations. We will need a range of captioning fonts, in different scripts for different languages, which will satisfy a high standard of typographic usefulness and æsthetic excellence. We need several fonts, in different sizes, with many more variations like italics, small capitals, and bold and extrabold versions. That way we can differentiate speakers more readily, better represent the tone of voice of different speakers, and in general make captions on different programs look different. For the first time, captioners could become graphic designers. Imagine how our captions could look in different fonts and colours (more on that later), and compare that to what we have now.

Since our fantasy HDCC system will have to function worldwide, we must plan to include the widest possible range of character sets to set the largest variety of languages. Moreover and this is a consideration which many people overlook programs in one language often contain dialogue in another, which captioners have to either transcribe phonetically, or caption without accents, or just sidestep by saying that a character is speaking, say, German without captioning the actual German wording.

The best and worst example of this is the opening to the film Indiana Jones and the Temple of Doom, in which an actress sings a Chinese song to the music of “Anything Goes.” When she reaches the refrain, she switches back to English and sings out, “Anything goes!” The captioned version of this panlingual cabaret makes no sense at all in the Line 21 technology, and all the funky polyglot humour is lost.

To avoid these problems, HDCC fonts must include:

Latin alphabets: We must be able to set every language written in Latin characters, from English to Icelandic to Welsh to Turkish to Vitnamese.
Cyrillic, or Russian, alphabets, capable of setting all levels of all the languages written in Cyrillic alphabets.
Greek.
Hebrew.
Pi, or special, characters; and finally...
Japanese and Chinese

Although at first glance it may seem a mind boggling task to include umpteen thousand characters in a captioning font, it is actually a relatively painless procedure. In fact, the Japanese have already implemented such a system. Chinese characters will require only slightly more work.

Obviously some scripts are missing from this list. Yet the above alphabets will serve at least half the literate peoples of the world, and nothing stops us from downloading a different font if we need it.

Finally, it's important to be able to combine any alphabets on any one line. Someone speaking English but discussing Arabic, for example, needs to be captioned in both languages. In our fantasy HDCC system, a caption line containing English and Arabic in sequence should be no problem.

Positioning and transmission

HDCC must permit us virtually unlimited flexibility in positioning captions. Also, we must look beyond the American experience to determine how fast our captions should be transmitted. In Canada, it would be useful to be able to transmit at least English and French simultaneously; Europeans have already expressed a need to transmit five languages at once. Transmission rates, therefore, should be set at the highest possible level so that at least five languages, in any combination, can be transmitted at once.

Even in America, such a capability would be highly valuable as captioning grows from a service for deaf viewers to a tool for literacy in different languages. Use your imagination maybe the children of Spanish speaking immigrants would like to read subtitles in Spanish.

Special applications

Since we are planning for the utmost sophistication in our fantasy HDCC system, we should ask for a few rather recherché features. Research has asserted that deaf people are more likely to have visual impairments than hearing people, and that those visual impairments are more likely to be corrected improperly. Furthermore, recently published research has shown that even minor visual impairment drastically reduces people's ability to understand today's Line 21 closed-captions. Clearly that is as much the fault of today's fonts as the vision-loss patterns of deaf and hearing people.

Given this, we should require that every HDTV set feature a connector to attach an offscreen display of captions. If our fonts aren't big enough for some people, they can use their own caption displays in whatever sizes are available.

Also, the captioning needs of poetic drama and of music video have been pitifully ignored thus far. For such programming, we need animated captions for example, captions which move across the screen and follow Dorothy and her friends as she walks down the yellow-brick road. For animated captions, we need very high transmission rates and a nearly unlimited ability to place captions where we want on the screen. At the moment, captioners cannot do justice to either Shakespeare or Madonna, and this must change in the high-definition age.

Colour-coördination

Captioners in England today and at the WGBH Caption Centre in the '70s are proud of their use of colour in captions to show who is speaking, among other uses. More locally, Line 21 captioners have been discouraged from colour-coding their captions for technical reasons.

In HDCC, captioners must have wide control over the colour of text and the colour, size, and opacity of background. Captioners should be trusted to use their imaginations and manipulate such colour options well, at least after some training.

Captioners must have control over the size and opacity of the background. It is not always necessary to add a background to a caption to make it easy to read today's black backgrounds on captions are unnecessary when the picture under the captions is dark in the first place. And in some cases a translucent or frosted background obscures the least amount of the picture without reducing the legibility of the caption. It also looks nice.

Finally, we must provide enough room in the transmission rate and in memory to download fonts, logos, and special characters, perhaps during moments when the captioning channels are unoccupied.

Politics and money

All of these plans will require not just money but a unity and strength of will for which the captioning firms are not noted. Select any two captioning firms in Canada and the U.S. at random and you will probably find that they do not get along. The National Captioning Institute has the most money and power, and NCI sets the largest number of trends in captioning.

Yet so far no one, not even juggernaut NCI, has shown much interest in planning for the inevitable future of high-definition television. It seems to me that this reluctance is shortsighted and dangerous. It was only by good luck that Line 21 was free for use in closed-captioning, and only by luck that consumer video equipment could read Line 21 data. As I have demonstrated already, Line 21 captioning technologies seem to have been thrown together with little (and probably no) expert typographic planning. We must not make the same mistakes again, nor must we rely on good luck to shine on us when we come to implement a closed captioning system for HDTV.

I therefore call on all interested individuals and firms in the fields of captioning, subtitling, deafness, HDTV, accessibility, literacy, linguistics, and typography to unite to form a coalition which will produce a worldwide standard for captioning of high-definition TV. We may apply to become a subcommittee of some professional organization or we may stand alone, but no matter what we do, we must come together to secure funding and legislative backing for implementing the very highest quality captioning and subtitling system for high-definition TV.

We must start to work now in order to organize all the technical and æsthetic details and to deal with the political necessities of pressing for standardized hardware and software. We owe it to ourselves to start to work now on this, the wave of the captioning future.

Thank you for your attention.