From ad-hoc research to real-time insight: the changes we need to see, now.


A planning perspective on Future Foundation’s ‘The Future of Insight’ conference: implications for planning, in client businesses and in agencies.

I presented this article at the above conference as a prezi, which really doesn’t work without the voice-over! So here’s the full contents.

For those who were there, you saw me bitch about the lack of innovation in the recently re-branded marketing and innovation ‘insight’ industry.

With the notable exception of the constantly experimenting Brainjuicer and its irrepressible John Kearon (whose thoughts about corporate innovation practices in his ‘Marketing Has Been the Death of Innovation’ paper I’ve gratefully borrowed) there has been little in terms of exciting new ways to probe the human mind in a way that delivers marketers insights on the fly. Indeed, the whole industry is struggling to work with those more digitally minded (clients, consumers or agencies), nor to create research solutions for niche but inmersive experiences that provide insights in ways Marketing Directors appreciate.

Online communities and their dashboard tools have been a liberation. Likewise, the kind of qual/quant surveying Brainjuicer started out with provide rich but quantifiable data. But beyond their efforts in testing innovation and exploring the web for insight, I’d argue the research industry has been slow to service the needs of modern marketers. Discuss.

The very anatomy of insight is changing

We’re moving from adhoc research to responsive insight, slowly.

But like all change it’s painful and scattered.

Insight is everywhere, it’s just not peddled well by research agencies or by any agency for that matter.

Brilliant insight is as likely to be tiny, tactical and changeable as earth-moving and revolutionary.

I thought I’d share a problem with have…after all, a problem shared…

Insight is only valuable if proven through action, mostly sales-driving marketing.

So it’s the application of insight through what we do that we need to focus on, especially to prove the discipline’s value within the organization.

The problem is that in a connected world, marketing should mean always-on collaboration with audiences, working with consumer reactions and making ourselves more and more relevant to them.

However, few clients are structured to deal with continuous, iterative, responsive communication and few agencies have structured themselves to deliver this either (because the clients aren’t structured that way yet).

We rely on pricey and protracted up-front planning.

But planning, even keeping best practices in mind, is just guessing.

The only military analogies and lore worth keeping from previous eras of marketing strategy is the wisdom that all plans go out of the window at first contact with the enemy.

Most of the time we don’t know how the plan is going, not in time to do anything about it anyway.

So we need real-time insight to tell us.

Once we have an idea of what to improve, we need the latest insight to guide us, continuously, across bought, owned and earned touchpoints.

But this rarely happens.

So testing the quality of insight through marketing is mostly too slow, confusing and inconclusive.

In delivering clients useful, timely insight to improve marketing on the fly, we are failing.

What about proving the value of insight through the resulting innovations in services and products?

Well, I propose that most of our market research bypasses real ‘innovation’ (new approaches that create disproportionate value by combining ways of working or technologies) and concentrates instead on ‘optimisation’ (small improvements to what we’re already doing).

True category-beating innovation doesn’t tend to fit well with research. Trend research is useful, but testing ideas in consumer research can often kill them or strip the originality from them (in marketing as in innovation) because people don’t tend to respond well to uncomfortably new ideas. Even skilled qual researchers can’t probe the subtle responses to truly eye-opening innovation well.

Real innovation, it appears, is the domain of visionary start-ups, not risk-averse big business.

8 of the Interbrand top 10 brands in the world (and almost half of the top 50) originated the category in which they operate.

Yet the centralised, risk averse marketing-sciences practiced by large businesses since the 1960s has served to merely ‘tend the garden’ these innovations created.

With the exception of a handful of brands, true innovation in FMCG, technology, banking, apparel and other industries has been left to start-ups and fringe players with the corporates buying those that succeed.

Bacardi bought the biggest new thing in vodka of the 90s Grey Goose, then the noughties hero 42 Below, Google has bought every other hyped start-up in Silicon Valley, Unilever the iconic Ben and Jerry’s, Cisco bought the category busting Flip video camera, Coca-Cola bought trailblazing Innocent and Glaceau. The list of big corporates paying top price for innovations they could easily have developed themselves goes on and on.

The corporate exceptions tend to be businesses led by a strong vision, who support their own ideas with gut instinct over consumer research and cultivate a healthy acceptance of failure.

Ibuka, the Chairman of Sony, one of the few big corporations to have created new categories said, “The consumer doesn’t know what they want, it is for us to invent it for them”.

Others that behave in this way are the Virgin group, Apple, Google, Facebook and 3M. Google has bought many of their successes but are committed to experimenting ferociously.

Criminally, the marketing services industry itself fails to innovate. With the exception of groundbreaking research ideas like DigiViduals from Brainjuicer, few new approaches to gathering insight appear to have been created.

So when it comes to big business successes through innovation, we in the insight industry have struggled to provide much demonstrable value.

So from a planning perspective (as a buyer as well as collator of insight), I’d beg the following of our industry:

Support ‘Flexible guessing’

If plans are there to be broken, let’s use scenarios.

Let’s propose what might happen, then adopt agile methodology to change what we do on the fly.

If social networks can respond to user behaviour to roll out new versions and whole new services every few weeks, why can’t the rest of the brands online do the same?

Let’s devise short, medium and long-term activity.

Let’s work out how to react on launch, what to anticipate and what to commit to across all touchpoints.

Thinking long term is critical. Successful approaches need to be learned. Starbucks has its most senior and talented people dedicated 24/7 building long term relationships through it’s most important marketing activity, its platform on Facebook.

While embracing the future, let’s keep a link to the past – history is bound to repeat itself if we don’t plunder the archives for lessons and ensure the inspiring young Turks know them.

Support ‘Responsive doing’

So in a connected world, we should be testing, learning and changing in real time collaboration with our communities

Let’s light lots of fires as we will never know what will take-off and what won’t.

Let’s respect our own hunches. Faster horses would never have spawned a multi-billion dollar industry. Serendipity is more useful than tight processes when it comes to making new ideas from new connections.

Let’s do it in small ways, and amplify what works, and let’s do it in small groups. Autonomy fosters passion and vision and that’s where great ideas come from so maverick thinkers within organisations must be supported with time and cash.

Let’s love failure. Dyson prototype number 1527 was the one that worked, Virgin has more failed businesses than successes.

And let’s re-organise ourselves to deliver this. In client businesses and agencies alike, insight and marketing functions need to be nimble and always-on. This takes bold internal change.

To conclude

Insight into the state of our own profession, like that shared today by the Future Foundation, is a great place to start.

The insight and marketing services industry appears caught like a rabbit in the headlights. Basking in new found relevance, but creaking with processes, tools and systems hopelessly out of date.

We need insight’s innovation and help.

If we are to help clients embrace a rapid, real-time, collaborative marketing environment we’d better get on and innovate pretty swiftly. Else the Googles and Facebooks of this world will eat our lunch.

8 Responses to “From ad-hoc research to real-time insight: the changes we need to see, now.”

  1. Hi Martin,

    I love how you criticize traditional market research, but I cringe a bit with a couple of things.

    First, “real time” insight is important, sure, but what happens when there is no DEEP insight to begin with? Imagine that you have somehow optimized your agency (not to mention your CLIENTS!) to collect and act upon real-time insight. Ok, then you have a bunch of junior planners who have no real background in the topic? Where do you get the deep understanding and cultural sensitivity that is required? You may be too busy running after the latest whiff of customer discontent spreading through Facebook.

    In short, you may not have the depth of understanding you need to really really understand a social phenomenon. Think “verstehen.”

    Second, I believe the market research industry is indeed guilty of simply “asking customers what they want.” But that does not mean that research itself should be thrown out of the innovation process. Indeed, design-based research approaches are specifically targeted in “inventing” what consumers implicitly want but do not know what it is. That is the real skill of a good qualitative researcher — bringing out something the consumer doesn’t even know herself.

    We do this all the time with ethnographic methods, which are not “new” or “sexy” but are certainly useful!

    But anyway, I applaud your call to action. The Fordist Market Research Company should be put to rest, forthwith.

    • 2 martin bailie

      Hi Sam, I agree with your points.

      Deep, solid insight that more tactical realtime ideas are build onto is critical. Clients hold this a lot of the time, so they are obviously the key partner.

      Getting the balance is the most difficult bet. Too often we have the universal insights but nothing to help apply them in the most relevant manner. People brandish ‘the insight’ as though that is all the work they need to do to deliver successful comms – the delpoyment and learning elements become secondary, so the theory of the insight trumps in the field learning.

      I’m certainly not advocating ditching research, and the smartest moderators look beyond what people say. It’s just that we haven’t seen many variations from this trusted format. Which is a massive shame. And ironic for people advising on ‘innovation’.

  2. Agreed. Planners must innovate insight, especially by collaborating in real-time, loving failure and thinking like start-ups. Well said.


  1. 1 From ad-hoc research to real-time insight: the changes we need to see, now. (via LOVE+MONEY) « matthewcopeland has moved
  2. 2 ambidextrous thoughts » From ad-hoc research to real-time insight: the changes we need to see, now. (via LOVE+MONEY)
  3. 3 Forget innovation, start to optimise – Getting marketing insight the solution focused way « Superbly Human
  4. 4 Consequences for small businesses of Martin Baillie’s radical views on the future of market research | Hallam Marketing

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: