Hipster Whining, or How Not to Do an Expert Review

The way you present the findings from an expert review of a client’s web site is at least as important, if not more so, than the findings themselves. If done poorly, the constructive criticism you offer will, at best, be ignored. At worst, you’ll end up alienating the client and reflecting poorly on your own abilities as a researcher.

The NYTimes’ You’re the Boss blog provides an excellent example of what can go horribly wrong with expert reviews. The blog’s author offers to review small business web sites and mobile apps, then share his findings and the business owner’s reaction. A recent entry explored why a suitcase web site’s marketing efforts failed to increase site traffic. The follow-up post listed what the blog’s author and readers thought of the marketing effort and the product’s web site. It also listed the business owner’s near complete dismissal of the feedback. My favorite quote from the business owner:

“I disagree with the many comments that say I need a ‘call to action’ and that the site isn’t ‘sales oriented.’ The site is designed to explain a product that is unique and benefits from explanation. It is easy to buy one.”

Ooh, and this one:

“There was a lot of hipster whining about our Web site, but very little actionable advice”

Ouch.

Look, I know that there is a big difference between a journalist writing a business-focused article about a business-related problem and a usability researcher doing an expert review for a client. But the way this review was handled and especially the way it was received suggest a couple of red flags to look out for when presenting your findings from expert reviews and usability tests.

Test with actual users

The intended audience for this product–a carry-on bag designed to minimize wrinkles–includes frequent fliers, e.g., business travelers and flight crew. But the “users” providing feedback are the readers of this small business blog. There might be frequent fliers among the readers providing feedback, but who knows? Because no effort has been made to ensure the folks providing the feedback are like the folks who would use the product, it is much easier for the owner to dismiss the feedback.

Say something positive

In the entire 1,700-word article, I didn’t see a single piece of positive feedback. Start out with a spoonful of sugar–honest, positive feedback–lest the patient refuse to take the medicine at all.

Focus on the “why” not the “what”

It’s so easy to move directly into douchebag territory when giving constructive criticism. An example of the feedback from the comments section of the first article:

“It is a terrible Web site …. The paragraphs are disjointed, with way too many words …. The Web site construction is flat and dated as well.”

Wow. Great. It’s “terrible.” That’s so helpful. It doesn’t sound subjective at all. And “disjointed.” What does that even mean? Out of sequence, incoherent, choppy?

The problem with vague categorizations like this (“flat and dated”) is that these descriptions don’t deal with why they are a problem. Ultimately, these are problems because they caused problems for people. For prospective customers. The problem isn’t that the paragraphs are “disjointed,” it’s that  users didn’t understand the terms of the contest, or how the product is different from its competitors. Simply labeling the problem makes it look like one person’s opinion rather than a fundamental problem with the site.

Don’t be insulting

“The product itself looks great, but obviously no one with marketing experience was involved.”

Classy.

“Introducing a mediocre Web site or online promotion with a minimal budget and little thought is worse than a waste of money. It sends a message that you are unprofessional or incompetent or both.”

Would you be open to hearing more after feedback like that? I wouldn’t.

Tread carefully when answering questions you weren’t asked

It seems that the business owner’s main objection to the feedback is that it’s about the wrong thing. He wanted to know how to better promote his contest. What he got was feedback about the web site and how the contest is presented there.

Granted, it often happens that when a client asks, “what’s wrong with X” it’s actually not X that’s the problem. But if you want the client to listen to your feedback, I find it helps to answer the initial question first, then get into what the larger related issues might be.

Finally, prep your client

Nobody likes surprises, not when the surprise is bad news or a lengthy discussion of how you are currently screwing up your business. Based on the knee-jerk reaction of the business owner–and let me be clear, I think he’s making a huge mistake dismissing the volumes of constructive, reasonable criticism he did receive–it looks like he just wasn’t prepared for the feedback he got. A frank, high-level discussion of preliminary findings can do wonders for getting the client to be a little more receptive.

Resources

If you want to know more about The Art of Constructive Criticism, Psychology Today has a nice article. If you didn’t see Adam Connor and Aaron Irizzary’s presentation at the IA Summit on Discussing Design: The Art of Critique, do it now. Or listen to the podcast.

How about you? What techniques do you use to make expert reviews easier for the client to accept?

This entry was posted in Featured and tagged . Bookmark the permalink.

Comments are closed.