Atomic Direct, LTD

I’ve spent nearly 20 years improving infomercial sales based on audience testing our shows in research. To do this, we don’t use dial groups. That surprises many of my colleagues so I’m regularly asked “why”.

Two critical reasons.

Dials don’t reveal the truths that make the difference between success and failure.

Dials primarily reveal minute-by-minute detail that’s unrelated to sales success. And pre-occupation with those details leads production teams to avoid dealing with things that are far more important.

What Is Dial Group Research? Dial research facilities have been around for decades – eking out a living with the theory that turning a dial reveals what consumers could never tell us if they had to speak it out loud. And dials remain fairly popular for looking at politician’s speeches, new sitcom ideas, and 1/2 hour infomercials.

In case you haven’t observed dials, here’s how it goes.

Everyone in a group is given a dial with markings from 0 to 10. Sometimes there’s a keypad with the dial for entering numerical data.

Basic demographic data about the group is input using the dials & keypads.

Then, they are shown something to evaluate and asked to turn the dial up to indicate “good” and down to indicate “bad”. (Good or bad what? That’s something you have to very carefully decide before doing the research.)

And after all that, there’s a very short focus group.

In the end, clients and their producers are given a tape with a line graph of average numbers superimposed and a report filled with details. And its what directors and producers love – microscopic minute-by-minute feedback about all those little choices they made. But does that matter? Not really.

Case Study: Dial Readings Don’t Correspond to On-air Success. Early in my DRTV career, Tyee’s production team made a new version of an existing 1/2 hour infomercial. The new show failed where the old show succeeded. Since the client had used dials to evaluate the old show they chose to test the new show the same way – same facility, same moderator, same dial approach.

And we found… Nothing. No difference. The dials went up and down at roughly the same times. The same topics drove dial movement. And the dials always dropped when the viewer was asked to order.

In the end, even after group discussion, all we knew was that the new show generated half as many sales but looked just like the old show on the dials.

But, Isn’t a Dial Reading Unbiased? Some defenders suggest dials are unbiased (link here). What they probably mean is it’s just a device and there’s no way to read a “3″ as anything but a “3″. Well, that part is true. But having a computer accurately read the device setting is a minuscule part of potential bias areas.

The first bias in dial groups happens because participants notice the action of those around them and this influences their dial choices. How bad is this bias? I’m sure the dial salespeople will tell you “not at all”. But, it’s human nature that your dial action will increase if you notice more dial action around you. And that inherently suggests a significant risk of “herd behavior”. (Herd behavior is a risk in focus groups, too. But a perceptive moderator can quickly detect it in the discussion and break it up. That is impossible during dial viewing.)

Major Bias is Introduced When Trying to Figure Out What the Numbers Mean. Let’s assume the average dial reading changes at a certain point. What kind of things might influence the change? An amazingly rich breadth of human possibilities:

Was it the voice used for the voice over?
Was it the words being said?
Was it the idea behind the product?
Do they own something that solves this need already?
Do they dis-like your brand?
Was it the color?
Did the product remind them of something from their childhood?
Was it indigestion from the dried out sandwiches the facility served?
Were they momentarily distracted by something earlier in the day – like when their boss said something that sounded mean?

So out of this wide range of possibilities, what do dials record? A series of single numbers – but we don’t know what influenced them.

That means dial numbers are mere shadows of what’s really going on with people. They can never be more than that and that’s a very serious problem. Remember the shadow puppet game we played as kids? If all you can look at is the shadow, a hand quite easily looks like a rabbit, a moose, or an eagle.

Interpreting shadows of the viewer experience makes the dial research process an amazing Rorshach test – one that reveals most clearly the prejudices of your producer, director, and agency.

Studies Show That Dial Testing Reveals Only a Slice of Reality. A few years ago a team evaluated a set of commercials with dials as well as other moment by moment methods of research. They found there was no correlation among the methods – ups and downs on each method were not predictive of ups and downs on the others. Further, discussing dials and another method one researcher concludes:

While both of these commonly used measures are telling us something important about the commercial, it has been shown repeatedly that each is measuring something different about expected ad performance (See Kastenholz et al, 2004.) (Link here.)

In other words, dials are now proven to record some slice of human reality. Do you know which slice that is? Is it a reliable predictor of sales impact?

Incidentally, all observational research has this interpretational bias problem. For more, here’s a post about Paleontologists and how observation can massively mis-lead even the best scientists.

THE REALLY SERIOUS PROBLEM: DIALS DON’T DELIVER INSIGHT. If we knew the answers, we wouldn’t need research. So, research must find the unexpected – the things that make a big difference.

The true catastrophe of dials groups is that I’ve always ended up at the same place: detail without insight. Those few times when the report reveals something useful, it always comes from the discussion afterward and not from the dials.

So why risk mis-interpretation, producer distraction, and outright mis-communication with dials? At my company, we never waste our client’s money using dials.

We Use Focus Group Audience Tests to Improve Results. Over the past 20 years I’ve regularly used focus group audience testing to increase sales. For finished infomercials, we’ve doubled, tripled and even jumped results by much more. We’ve successfully analyzed what worked, didn’t work, and decided what should be changed. And we use groups to review rough cuts – learning key things that help focus final changes.

All this works because within focus groups we learn what matters – those things that generate profit or help you avoid costs that aren’t going to drive sales.

But What About The Focus Group Skeptic? Skeptics will trot out the old focus group canards.

“People influence each other in groups.” Of course they do. Once we’ve watched the entire show, that’s the whole point. People reveal deep things far more within a discussion format than they’d ever reveal by themselves or with dials.

“People don’t reveal their true feelings in focus groups.” Of course they do – if the group is run by an experienced moderator using appropriate and thoroughly prepared stimuli. In fact, groups reveal far more about true feeling than dials ever could.

For more about the strengths of focus groups check out this blog post and this one.

Drive Infomercial Success with Research – the Right Research. Infomercial success requires the big things: a product we lead people to care about, demonstrations that convince viewers, testimonials that are credible and meaningful, and a “deal” that adds up to a great value.

Unfortunately, too many creative and production teams are far better at getting the details right than they are at delivering on the big picture. The most detail oriented teams are also those who fall in love with dial groups – because they feed their mis-perception about what’s important. And that leads to what I find the bane of our industry: A-grade production values that mask C-grade or D-grade communication.

And that’s why you need to avoid dials – so that you can find, and focus on, the issues that matter.

Copyright 2011 – Doug Garnett – All Rights Reserved

Have you heard the “Jobs Excuse”? When someone introduces a bad idea with “well Steve Jobs says” or “…just like Apple…”. It’s an old name dropping game that hopes to make even horrid ideas sound good.

In the world of market research, we hear it most often through one popular quote from Mr. Jobs:

“It’s really hard to design products by focus groups. A lot of times, people don’t know what they want until you show it to them.” (BusinessWeek, May 1998)

The Good. You have to read this quote carefully because what it says is:

- Focus groups aren’t good places to design products.
- Only you know what’s possible thru technology
- People can’t project ahead to tell you what to build.

He is absolutely right. Far too often, focus groups are asked to answer things consumer participants don’t know and can’t imagine. It’s an exceptionally poor use of research.

In advertising, this type or research often asks consumers to decide the colors used in ads or to project what a finished ad might look like based on storyboards or sketches (it’s hard enough for experts to do that – much less consumers). In the extreme, it can even lead to asking consumers to design their ads – as ill considered as asking consumers to design products.

The real reason for these research abuses is to avoid responsibility for bad decisions in engineering, design, or marketing (“but focus groups said…”). But if you are letting focus group participants make your decisions then you will get what you deserve.

So, yes. What Steve Jobs said is absolutely correct.

The Bad. Unfortunately, nobody uses this quote for what it says – they use it for what they wish it said. And what they wish it said was “don’t use focus groups”.

I hear this lie regularly from agency types and creative teams who simply don’t like the answers they get from research. And I hear it from design teams who just want to be left alone to design the usual meaningless innovations that come from humanity-free engineering. So they invoke the name of Jobs to try to avoid research altogether.

Research and Innovation. In fact, research is an exceptionally strong tool for finding consumer realities that can make the difference between innovation success and failure. But you can’t use the kind of responsibility avoiding research I noted above. And, sadly, many researchers (especially those in the large firms) lack the unusual set of skills needed to work early in the innovation cycle.

But research creates the foundation for successful innovation:

- to learn about consumer lives so you can deliver the best human value.
- to learn about whether/how the innovations you have designed fit with humanity.
- to learn how to pull together the messages that will lead people to buy your innovation.
- to learn how communication changes as you move from the earliest buyers to mass markets.

(My Shelf Potato blog discusses cases where failure to communicate caused innovation to fail – or where communication rescued innovations that were sitting on the shelf and unloved.)

Jobs’ Research Legacy. I don’t claim to know what Jobs thinks – others might. But what he had said publicly, while usually perceptively true, has left behind a sad, mixed legacy.

And those who attempt to avoid research with the “Jobs excuse” should be identified for what they are: people who don’t want to find out how badly their product or advertising misses with the people who buy their products.

Postscript: I wrote this post just over 3 weeks prior to Steve Jobs’ death. The world will miss his brilliant innovations.

Copyright 2011 – Doug Garnett – All Rights Reserved

WordPress tells me this is my 100th post (the celebration commences immediately). So is there a more fitting topic than content based marketing’s need to “feed the beast”?

Want a higher Klout score (thinking it actually means something)? Then work, work, work. Read more, comment more, pass along more, write more. And connect with people you don’t know, but who have higher scores. (By the way, don’t ignore all that other work you have to do – or your family.)

Want more visitors to your blog? Post more, work harder on what you post. Then fight hard to find places to publicize your blog. (And don’t forget to sell product, answer the phones, create new things, make ads, and all that other marketing.)

Want Twitter followers? Then Tweet constantly and find highly provocative things to Tweet about. Isolate those meme’s that drive people to sign up to follow you. (And, don’t worry that the best meme’s have to be simplified so far that they end up separated from truth.)

Steve Schildwachter posted about the beast recently in an interesting post titled Five Reasons Not to Fear Social Media Burnout.

He covers some well known people who complain about the burnout factor. (One of these being AdAge’s eminent industry commentator Bob Garfield.) Some have even dropped back to lurk rather than actively participate.

Steve rightly reminds us social media is quite young and still ill developed. (I agree with Steve’s points – except the idea that we should take heart because companies are spending more in social media. Companies are like lemmings and will increase spending on new media options just because everyone else is – not because that’s where the money delivers the best results.)

For Companies, the Risk Isn’t Personal Burnout – It’s Idea Burnout. While agreeing with Steve, I think companies who rely too much on social media are taking a huge burnout risk. Not because individuals burn out – but because their ideas burnout.

Look around and you’ll see that companies who were the early darlings of social media are falling off the radar. Remember the “Will It Blend?” campaign? Those videos still exist, but they’ve dropped off the radar despite the fact that the company keeps making new ones.

Remember the Old Spice guy? W+K has a big challenge keeping that campaign in front of people. They may have to invent another one to try to get the same level of viral action. (Fortunately TV advertising drove the viral response – so maybe they have a formula they can re-use.)

Informative Content Offers a Bigger Challenge. What about informative content instead of entertainment content? Corporate informative content seems to keep getting worse – and most lacks any real value.

It’s not much different from where printed content has come. Brochure’s decayed over the past 20 years to where most lack anything fresh – and don’t get read. (Car dealer brochures are the worst – offering nothing significant except a list of features and colors that are probably out of date.) Is your content too much like a deadly-dull brochure?

Even worse, examples in a magazine I recently read about corporate content marketing looked like those boring company magazines written in the 1970′s.

Perhaps the problem is content farms. Too much content farm work is as meaningless as dairy cows mooing on the way to be milked.

And Then There are Corporate Facebook Pages… Talk about meaningless drivel.

I’ve written that most people really don’t want to be your friend. Seems that we can now conclude the primary corporate post people want to see on Facebook starts with “30% off on …”.

There Is Hope. It’s really not as bleak as I’ve dramatized. Personally, I love finding a topic I care about and developing what I hope are thoughtful comments about it. While saying this, I also know that dedication to writing something meaningful creates longer gaps between posts than I like. But I think that’s what’s needed – content that’s meaningful.

So here’s a recommendation:

Limit content marketing to be only a portion of your marketing mix.

Merchandise well written bits over longer periods.

Develop a BS meter that filters out the ideas that lack insight.

Don’t get caught up in social media scoring. Klout scores probably don’t measure anything important (unless you’re a Klout score consultant).

And good luck. I think I might have a grasp of what this means. Yet it’s a strange new world we all live within.

Now…what should I do for my 101st post?

Copyright 2011 – Doug Garnett – All Rights Reserved