Archive for the ‘measurement’ Category

Net Promoter Score – a flawed science?

Friday, January 29th, 2010

Yesterday, I attended a WOM UK espresso breakfast briefing with Professor of Consumer Behaviour Dr Robert East from our very own Kingston University.

Dr East and his MBA students have been researching consumer behaviour and word for mouth for 10 years.

Without serious funding they don’t have the data to prove the validity of their model conclusively, but their research indicates that the well accepted Net Promoter Score word of mouth measure for predicting growth is flawed.

NPS asks the question ‘would you recommend this brand’. Respondents are then rated 0 – 10, with 0-6 ranking as detractors, 7 and 8 passives and 9 and 10 as promoters. The NPS score is then calculated by subtracting the average of the detractors from the average of the promoters.

Dr East contends that in only measuring the volume of given word of mouth, positive versus negative without measuring received word of mouth or the impact of WOM in general, NPS misses a trick. Not to mention the fact that the people most likely to offer negative word of mouth are ex-customers who are not surveyed.

The resources of Kingston University don’t lend themselves to the level of data crunching research required to prove the validity of Dr East’s methodology. This is a shame as I think the results could be rather interesting. I’ll certainly be keeping my eyes open for a potential research partner for his department.

I’m also keen to see more thinking done around the customer experience in relation to professional services. With such a complicated purchasing relationship, can a measure as simple as NPS be effective in measuring a client/agency relationship? Can we ever hope to achieve the kind of high NPS scores enjoyed by companies such as online retailers or supermarkets that have traditionally invested heavily in how they shape their customers’ experience?

Dr East argues that there is far more positive word of mouth than negative and that negative word of mouth doesn’t necessarily have the most impact. But if I’ve taken one thing away from the talk it is that we need to be addressing past clients in addition to existing when it comes to assessing our own performance.

At the end of the day NPS may be an imperfect tool for measurement. And it may be better suited to consumer retail. But we have to start somewhere because if you can’t measure it, you can’t manage it.

Reblog this post [with Zemanta]

How bit.ly will change the world

Monday, August 10th, 2009

On the surface, URL shortening services such as http://bit.ly are a great idea, because they can turn long, messy web addresses like this:

http://business.timesonline.co.uk/tol/business/industry_sectors/support_services/article6788774.ece

Into nice tidy addresses like this:

http://bit.ly/g4ol9

This is especially useful for posting to Twitter, since it saves valuable space, but a lot of people have got into the habit of using URL shortening services all the time.

There’s an obvious problem with this from an SEO point of view. For a start, the shortened URL contains no anchor text, and secondly they do not pass on PageRank.  Since these two things are fundamental to Google’s ranking algorithm, any links to your company website that use a URL shortening service are practically worthless in terms of SEO value. They will do nothing to improve your site’s ranking for the relevant keywords.

[EDIT] As pointed out in the comments, it seems bit.ly and other URL shortening services do pass on PageRank (a few of these services do not) but the anchor text issue is still a problem, links without embedded keywords don’t provide much value.

But that’s not all. As the recent closure of http://tr.im has illustrated, sometimes URL shortening services go out of business and that means that all those millions of links on the internet which use that service will suddenly stop working.

So the long and short of it is: for online PR purposes URL shortening services are best avoided where possible, but sometimes they’re necessary, like on Twitter.

Here’s the really interesting bit

But there’s more to this story. They may have certain disadvantages and risks, but as long as Twitter is going strong, it’s fairly certain that bit.ly will be doing quite nicely too (did somebody say buyout?) and that creates an interesting situation.

Twitter is a hotbed of viral activity, with news and trends being retweeted backwards and forwards, spreading across the web faster than ever before. Given that bit.ly is rapidly becoming the de-facto URL shortening service, it is an amazing and unprecedented position in that it has access to a live, detailed view of these trends as they are developing.

Before anybody else knows what’s making an impact on the web on any given day, the people who run bit.ly will already have a clear picture of what people are looking at, what is spreading around the web, and how it’s spreading. If they’re smart, they will already be analysing that in all sorts of clever and interesting ways to figure out how they can extract value from it.

For most web users, bit.ly is just a handy way to make unwieldy URLs a bit more manageable, but for businesses it’s a goldmine of up to the minute data on consumer trends and behaviour, on an amazing scale. I wouldn’t be at all surprised to see bit.ly selling customised dashboards to provide businesses with snapshots of that data in the future.

Twitter’s business model may still be a bit hazy, but it’s certainly created fertile ground for bit.ly to develop into what could be one of the most powerful and valuable business tools on the web.

[UPDATE] It seems tr.im has decided to stick around after all – although the service’s owners have a few things to say about the relationship between Twitter and bit.ly

It’s time online PR got serious about measurement

Monday, March 23rd, 2009

One of the most interesting changes taking place as the PR industry evolves into an online discipline is the increased emphasis on measurement. In the old days of offline PR, little attention was paid to systematically analysing the relative success of campaigns, because measuring PR is hard to do. Measures that attempted to pin a quantifiable return on investment to PR activity, such as Advertising Value Equivalent were woolly at best.

But that’s all changing and it seems that PR isn’t getting a free ride any more. All of a sudden, acronyms like ROI and KPI are being applied to an industry that has long been used to justifying its budget with a ring-binder full of shiny, laminated press-clippings. There is now a far greater expectation that PR should provide hard evidence of its impact on the bottom line.

There are two things PR agencies can do about this:

  • Keep banging on about how the qualitative nature of PR makes it impossible to measure in the same way as other disciplines
  • Figure out how we can use all of these new metrics which are available in the online world to put together some kind of robust and repeatable framework for measuring the value of PR activity with some degree of consistency

You might like to take a guess at which of those approaches is likely to win the most new business.

Of course, there’s already a lot of healthy debate and discussion about PR measurement in the blogosphere, and it’s no surprise that a lot of the big names in PR have their own ideas about the most effective approach.

On the one hand, it’s good that there’s so much interest in solving the problem, but on the other hand, it looks unlikely that an industry-wide consensus will be reached any time soon. Obviously, everybody wants to implement a measurement standard that best represents their own strengths, and it doesn’t help that any discussion on the subject invariably gets sidetracked into an esoteric debate about the nature of influence.

Obviously there’s still a long way to go before this is anywhere close to being solved, but at least there now seems to be broad acceptance that rigorous measurement will be key to the PR industry’s future.

However the measurement debate unfolds, I think it’s absolutely key to ensure that the metrics used are properly aligned with the client’s business goals.  All too often, arbitrary KPIs are chosen simply to provide a tick-box for PR staff to show that they’ve done some work, with little consideration into how exactly they help the business achieve its ultimate aims. And that’s a far worse situation to be in than the old days where we relied solely on qualitative reporting, which at least had some kind of value.