Joe Clark: joeclark.org (E-mail)
You are here: Homepage > Site info > Get me working for you > Usability at epinions.com
Note: This isn't a live document. I don't keep it continuously updated to reflect the ongoing reality at epinions.com. The assessment reflects the state of the site in late 1999. From what I've been told, usability at Epinions has improved somewhat since then, but I am not tracking the usability of the site – not without a regular paycheque, at any rate.
I conducted an informal usability analysis of Epinions.com in November 1999. Epinions is a now-famous startup site in which anyone may review cultural works, products, and other items. What's more, one's actual reviews are then reviewed by other readers, creating a feedback loop. U.S. residents can earn money (in pitifully small amounts) for the Epinions they write.
I extracted a couple of record reviews from my homepage and attempted to cut and paste them wholesale into Epinions. In so doing, I bypassed what most Epinions members would consider the most nerve-wracking, painful, and trying chunk of the process – writing the actual epinion – and could focus on specific UI issues.
I had actually wanted to upload articles on all of the following:
and a great many lesser-known artistes, like Kathleen Yearwood, Les Rita Mitsouko, Kronos Quartet, King's X, Luc De Larochellière, Yello, and Cabaret Voltaire.
We were told, on the epinion site itself and via postings to UI-related mailing lists, that we are now able to rate absolutely any book, record, or movie. This is false. It's difficult even to write the first rating for an album by an artist whose other works are rated.
Knowing, just knowing in the pit of my stomach, that this is what would happen, I started out with Madonna. God knows there have to be Madonna reviews in the database, I thought. A search for Something to Remember by Madonna turned up nothing.
This is insane, I thought. Searching just for Madonna turned up nothing even though the instructions clearly state "Search by Artist and/or Album name."
The problem seems to be Epinions' database structure. Something to Remember by Madonna is listed as one item: Something to Remember - Madonna. It's indivisible. Yet plugging that text into any search field I could find did not produce a hit. (In fact, search at Epinions is completely broken and unreliable, worse than even first-generation Internet search engines. I'll get to that later.)
I ultimately had to look through the alphabetical list, eventually finding Something to Remember at the unhelpful URL of epinions.com/musc_mu-201443. The only reason there was a listing for Something to Remember is because a professional epinion had been linked to it. I am the first and only reviewer of the album. (The term "professional epinion" casually disparages everyone else's. How about "commercial Epinions," or "outside Epinions," or "outside reading"?)
Had there not been any listing whatsoever for that album, I could not have created one myself, even though Madonna already has her own category. My only option is to make a request using a form. Someone (or maybe some bot) has to read that form and create the listing for you.
(And what does the Message field on that form mean? Won't people mistakenly fill in their epinion there? Or is the expectation that people who have already been frustrated at every turn in their attempts to review an album, who've already been asked to fill out the artist and album title on that form and probably elsewhere, will now write a sentence explaining what should be self-evident?)
So for nearly all the artistes and/or albums I wanted to review, I was stopped dead in my tracks.
Responsible designers should not reqire visitors to ask permission to review something. At a site whose raison d'être is reviews, that's outrageous. Anyone should be able to start a new artist or title subcategory. A search, if it produces no hits, should then ask "Would you like to create this category?" We then get into problems of misspellings and bad typing, but let's review the options. Isn't it better to send each of these user-created, brand-new review categories to an Epinions category editor to proofread post facto than to force the reviewer and Epinions to jump through hoops just to set up the entry, as is currently done?
It gets worse. I somehow locate the listing for Moby. I find two album reviews. I want to write a review of a third album, and I haven't gone through the rigamarole I just described; I'm pretending I'm a newbie and I am now pursuing another of the forks in the road. I see the two existing reviews, plus the Member Epinions bar and, below that, the header for an epinion that is not linked to any album. If I hit the Write an Epinion button in the bar, what will my epinion be linked to? I haven't selected anything, and nowhere has the system let on that I am allowed to review an artist (viz., Moby) rather than an album. As it turns out, hitting that button takes me to a generic review screen. The generic review as a concept is what Epinions is emphatically not all about.
The system has its heart in the right place when it flags possible spelling errors, but frankly, that kind of machine pre-editing should be something people opt into. All the words flagged as misspelled in my Epinions were either correctly spelled or were morphemes embedded in terms more complex than the software could handle. A spellchecker that highlights "misspelled" words, but does not let you correct them right then and there and does not suggest alternatives, is not a spellchecker; it's a nag. Combined with a search system that refuses to show me what I later find out really was there all along, Epinions degrades my trust by making me do too much of the work of database construction and verification of data integrity. Trust is a big deal at Epinions, yes?
As another example of data integrity, check the reviews for these categories:
They're identical. What if the performance of the Circulon sets were actually different from the individual pieces? (What if they tossed in a crappy 6" frypan that devalued the rest of the set?) Humans should scrutinize categories to make sure they are genuinely distinct and for good reason.
Further, artist names seem to have been decided on by someone who isn't a native speaker of English. Beastie Boys don't need a trailing "the." I'm not sure any band needs a "the" in its URL, though the name listing can show it: Doors, The; The, The.
Black Crowes and Black Clint don't have structurally similar names, though the rendering wrongly suggests that; they should read Black Crowes and Black, Clint, with the mandatory comma.
And just what is an "Editorial Dummy"? That phrase turned up as a top-level category in several of my searches, including this one (though the results may have changed since writing this paragraph). It seems to mean "[Stand-in text for a category we have not permanently named yet]." As such, "dummy" does not really mean "nitwit," but a site based on people's opinions should avoid such a word altogether.
On virtually every page at a level below the www.epinions.com greeting page, we are given the chance to sign up for automatic E-mail notifications if a category changes. Except the system applies this to pages that make no sense whatsoever: "Send me email when new opinions on 'M' appear."
No one has perfected Internet searching yet. But Epinions currently does everything wrong. If I search for something to remember
or "something to remember" the database returns identical streams of irrelevant hits. It should infer that both entries are phrases. Maddeningly, it particularly does not understand that I mean the latter search to find a phrase consisting of those three words. (Perhaps the preposition could be ignored, but Epinions should still know to search for "something remember" in that order.)
Epinions, as currently structured, deals with individual items and does not traffic in the infinite permutations of English grammar. Its search results should be limited to individual items, meaning that many possible misunderstandings of search phrases can be ignored altogether. If I enter "cheap Florida vacation" into Google, it is not really wrong for Google to return hits on cheap vacations or Florida vacations; it's just not quite what I want. But if I search for "something to remember" at Epinions, chances are I am asking for that exact phrase, because we are searching for exact items with reviews.
What I think we need to do is add human intelligence to the search function. We need to have real people look over the list of what visitors are searching for and learn from that listing. We then re-train the database to produce good hits for searches the engine's internal logic cannot handle. This is a labour-intensive activity; to do it properly, we'd have to review all searches – sorted, ideally, by frequency – and fix them up individually. This technique mediates between the full-on-automated approach of, say, Altavista and the human-taxonomic approach of, say, dmoz (or Yahoo, not that there's any evidence at all that their human staff add any value to the present-day Yahoo database). This is not a new idea: Even Microsoft does it. (See related article not pertaining to Microsoft.)
In any event, even if visitors choose to search "everything" (the default in the pull-down menu), some sections of the site should remain unsearched in most cases. Searching for everything is wrong, the title of a Moby album, should absolutely not produce a list of member names. A search for everythingiswrong or everything_is_wrong plausibly should do so. There is just no way that a search phrase including spaces could refer to member IDs. If I really want to do that, the search-results page should then ask me if I want to.
It is also doubtful, at least at the current preview stage, that a search for everything is wrong would be intended as a free-text search of the words in Epinions reviews.
The search engine must be reprogrammed never to return a hit in which none of the search terms is visible. Searching for "everything is wrong" produced 13 hits with none of those words in the results – not even "is." (The letter sequence is was indeed found in several member IDs, which is the kind of dumb result the database should be reprogrammed never to yield.)
If a search returns exactly one hit, it should bring me there immediately rather than asking me to select that hit.
My graphic-design background will come in handy in discussing the ways Epinions currently displays topic matches. Here is some actual text from a search result:
Editorial Dummy >Platinum Albums of 1998 > > Before These Crowded Streets - Dave Matthews Band On Video >Childrens > > Nightmare Before Christmas On Video >Childrens > > Land Before Time Fiction: By Genre >Old_Fiction > > Before You Sleep Biography & Autobiography >General > > Before the Wind : The Memoir of an American Sea Captain, 1808-1833
Where to begin?
→ [→], → [→], and → [→], which, to my knowledge, only iCab supports. (Maybe MSIE 5 does. Maybe Opera will. Maybe Mozilla will.)
Otherwise we're stuck with > or the guiellemet, ». I don't know a browser that can't render the latter. If Epinions does not make it possible to search using those delimiters – e.g., entering Biography & Autobiography > General into a search field – then we don't have to worry about the fact that nearly all Windows users and most Mac users won't be able to type the »; it will be for display only.
<FONT SIZE="-1"> to reduce the visual impact of the > or » characters. There is no reason for them to be the same font size as the text.
The current results fail various tests: English orthography, punctuation consistency (Epinions is not quite sure how many spaces to surround the > characters with; the answer is one), and readability and scannability. They're a mess, with, in particular, a lot of bold. Yahoo and dmoz results are a dream by comparison. But it is quite easy to fix, and readily XMLable. I've set up a Web page with some improvements.
Thankfully, Epinions.com avoids unnecessary graphics. Still, there are a few usability and accessibility errors. Surely, with the 21st century about to faxmodemed in, we have evolved past the use of tiny spacer GIFs (with the delightful filename spacer.gif!) to provide vertical separation in the Music section's genre listings. They take time to load but are invisible. I am quite aware of the uses of whitespace. It's just that this is the wrong way to get it. That's why God gave us CSS.
There are practically no alt texts anywhere on the site. Actually, spacer.gifs have alt="", which is correct and helpful, but much less helpful than having all the graphics for rating stars equipped with alts. It's not exactly hard:
alt="*****"
alt="****"
alt="***"
alt="**"
alt="*"or, preferably (since screen readers may choke on the asterisks):
alt="[Five stars]"
alt="[Four stars]"
alt="[Three stars]"
alt="[Two stars]"
alt="[One star]"Words are preferable to numbers because they are easier to differentiate. Compare:
alt="[5 stars]"
alt="[4 stars]"
alt="[3 stars]"
alt="[2 stars]"
alt="[1 star]"(The Toronto Star approach of [Rating Star] [Rating Star] [Rating Star] is the worst of all possible worlds. You'd think they'd get it right, having the word "star" in the paper's name.)
It's kind of ridiculous to force users who do not or cannot surf with graphics to load individual graphic files just to see a rating. We feel similar impatience when the browser loads every other graphic first.
The system should automatically include an alt with a member's userID or real name when displaying a photo of the person.
URLs at Epinions should be predictable. A case can be made that there will be too many reviews of individual items to make them typable. A URL like http://www.epinions.com/musc_mu-201443 is unavoidable for a specific album.
I should, however, be able to type epinions.com/music into my browser and have it work. (Note the lack of www. and the use of a real word, music, as opposed to musc.) Trying that now gives me merely www.epinions.com (a step above a 404 message, but only a step).
We should again use a human-devised synonym system to permit visitors to make educated guesses. All the following should work, for example, and lead at least to the unwieldy "real" URL http://www.epinions.com/hmgd-Kitchen-Cookware_and_Cutlery:
Also, what's with the different hostnames? http://media.epinions.com/ versus http://www.epinions.com/musc-Moby. Some keen users will spot the hostnames and try to extrapolate. Accordingly, music.epinions.com should work as a hostname; it doesn't.
Epinions may wish to retain four-character category names, like musc and hmgd, but human-readable, human-typable English words should map to them.
The systems should also be highly error-compliant. If I try to type www.epinions.com/musc-BeastieBoys or www.epinions.com/musc-Beastie-Boys or www.epinions.com/musc-Beastie_Boys or www.epinions.com/beastieboys, any should work and funnel me to the laughably unmemorable "correct" URL www.epinions.com/musc-Beastie_Boys__The_.
Epinions' use of hyphen and underscore in URLs invites errors. The system should fix our errors for us.
Epinions will, I trust, expand the range of acceptable review subjects to include, in the case of music, entities below the level of the album. Individual remixes and music videos should be reviewable. It should even be possible to review different remix music videos (e.g., in Madonna's case, Stereo MCs' remix of "Frozen," or "You'll See" and «Verra»). For movies, it should be possible to review dubbing, subtitling, captioning, audio description, video transfer, sound quality, soundtracks, and other production features.
First, "log in" and "log out" are verb phrases; "login" and "logout" are nouns. The homepage is really using the verb phrases while writing them as nouns.
If showing all my Epinions, the system should not tell me it's displaying 1 to 2 of 2 Epinions. (If I had only one, would it say "1 to 1 of 1"?) It should simply display all of them. It should attempt to do so in one screen unless there are a huge number, like more than 20. No matter what Jakob Nielsen may claim, people will scroll in this case.
And finally, how do I get back to the homepage from my Edit Profile screen?
Epinions.com represents an excellent idea with relatively poor implementation – but not all the deficiencies are at the conceptual level and thus difficult and expensive to fix. Many little usability errors could be reprogrammed in a matter of days.