Why audience measurement is so difficult

There are numerous resources for online audience measurement available today. One of the earliest forms was the page counter which appeared soon after the now retired scrolling text on web pages. 

Today some of the online analytics include Google Analytics, Quantcast, Nielsen, comScore, Hitwise, and others. Though there seems to be quite a bit of controversy and debate over who's numbers are right, who's are wrong and why. A recent example has been the coverage over comScore and Google's Ad Planner and Trends for Web sites. After the announcement of Google's Ad Planner, comScore's stock price took a real hit. Headlines like Google Comscore Killer? didn't help the situation and helped stir a lot of reaction across the web. Once things started to settle down new headlines of a different tone "Google Ad Planner No comScore-Killer After All" started to appear.

A primary reason why audience measurement is so difficult, I believe is due to a lack of understanding of some of the fundamentals of audience measurement. This includes principals around sampling, panel recruitment and measurement. I am a data-junkie and data is only good as long as it is within context and when there is a thorough understanding of it's limitations, methodology and assumptions.

I think Fred Wilson summarizes things well in his post Checking Out Google Trends for Web sites. 

. . .  Everyone who provides third party measurement starts out with a data set that is skewed in some way. The trick is to understand how your data set is skewed and apply statistics to take that bias out. Firms like ComScore, Compete, Hitwise, and NetRatings who sell their data have invested heavily for many years in reconciling their data to server logs and internal analytics. And that makes their data better.

Instead of everyone wondering who's data is better, it would be nice to see an unbiased review of the different data sources summarizing the benefits, disadvantages, limitations, methodology and use cases.  

Buffalo + Lion + Croc = YouTube reality show

I've personally had enough of reality tv. It's taken me some time to get into the YouTube thing and in part it may be my lack of patience with Idol-tv, so-you-think-you-can-[fill in whatever] and basic let's watch people fail specials.

Reality tv (or near-reality-tv) has been around for quite some time, some of the pioneers included Tiny Talent Time, the Newlywed Game, America's Funniest Videos, and of course Jerry Springer. How else do you explain the popularity of American Idol and Trailer Park Boys.

The beauty of YouTube is you can get your fix of reality entertainment in 30 second doses. One of my favorites for quite some time is the video below.

The lost "R"

It was about 15 years ago that the three "R" campaign really started. The three R's being Reduce, Reuse and Recycle. While the three circular arrows are still quite common, you don't hear mention to the three R's so much any more.

Looking closer at these, I think as a whole we've become relatively good at two of the three R's.

1. Recycle. Technology for recycling and participation in recycling programs is quite impressive. At one point not long ago people couldn't imagine separating their household waste, now blue boxes are all over and some are lost for a moment if they are some place without one. Recycling has become a major industry and waste diversion initiatives generally work.
2. Reuse. Maybe not as successful (or as measurable) but people are much more aware of the need to reuse products before sending them off to the garbage or recycle. This is very apparent with children who don't like to waste anything and reuse it for something new. Hopefully this generation will help us turn the corner.

The lost 'R' is Reduce. While the others have achieved modest success, our society refuses to reduce. We live in a consumer society and purchasing and consuming is the furthest from the 'Reduce' mentality.

Why have we lost this R? I think in part because of the scale that the mega-marts of the world (don't want to mention any names) are able to achieve. Things are built cheaply and sold cheaply, making the short term perception much more desirable than the long term view.  The expectation that people have is that things should be cheap.

The two items that really get me are bottled water and backyard canopies. A lot has been written up on bottled water so I won't dwell on that one.  As for backyard canopies, you know these. They are cheap at every major chain, last about a season then end up in the garbage in the spring, only to have a brand new one in the yard a week later.

The wake-up call that I'm trying to make is that before you jump on that great deal, think twice about what you are really getting. Typically if you spend just a little bit more, you can get something that is much better quality. In the long run I would argue that it is cheaper - and helps contribute to the lost R.

Look around your house, surely you can find a dozen examples. If that's not convincing enough, when you do your spring or fall cleaning, take note of what you are getting rid of.

The Reduce mentality is of course much much more than this. I'm just sick of cheaply built and sold products.

Email panel response rates

The topic of response rates and online email panels can provide some interesting discussion amongst research professionals. Response rates in general are important for all research and is sometimes referred to as a measure of panel quality or a measure of risk on data quality. Outside of some statistical reasons, there is growing discussion on how to calculate and report response rates - or if you should at all.

Calculating response rates for an online email panel can be rather challenging given some of the questions around methodology, panel recruitment, email delivery, email bounce, shared accounts and so on. 

Online sample is a hot commodity these days, the industry is strong despite some real challenges. Though while I say it is 'hot', it is not yet a 'commodity' as methodology is still charging non-commodity prices. Enough on that for now so back to response rates.

Whether a researcher expecting a online sample provider to report response rates, or a sample provider reporting response rates, both need to realize some of the short-comings of such a metric. As an example, what does it really mean if someone says their panel can get a 70% + response rate? On the other hand, should anyone expect this from an online email panel.

Here are a few scenarios where a 70%+ response rate on an email panel may be possible:

1. You, as a panel manager, do everything right (or at least most things). Recruitment is from credible sources, respondents are not 'touched' too many times that they get fatigued, respondents aren't ignored so much that they forget about you. Surveys are fun, engaging and relevant. Incentives are reasonable but not unrealistic. Your respondent, has developed some level of loyalty to your brand.
2. Non-responsive panel managers are purged on a regular basis. This leaves you with very responsive members, thus a higher response rate.
3. Email members who have responded in the last few days.
4. Primarily email 'new' members. New members are mostly likely to participate when they are fresh.
5. A panel consisting primarily of professional respondents who take everything.
6. Simply lie about the response rate.

Most cases are a mix of the above and hopefully #6 doesn't ever happen. In many cases email panel managers simply don't report response rates since low numbers don't look attractive and many attribute it to a reflection on panel quality.

So the next time you ask for a response rate, or report one, consider carefully what your metric really reflects. Is a 70% response rate really much better than a 4% response rate?

And for the record, the list above is not exhaustive, just a few extremes. Please add yours to the list.

T-Rex on the streets

Last fall I was out in Cupertino and came across a T-Rex outside of the Apple headquarters. These car/motorcyle cross vehicles made by Campagna look killer.

The particular models that we saw were 100% electric and sounded beautiful. Apparently performance and handling is impressive with a low stance and instant torque. 

Most impressive is that these are a great example of thinking outside of the box. Electric powered cars are a good step forward, but there is a real ceiling that is hit when you are trying to apply it to a 3,000 pound vehicle. 

I assume they are relatively expensive but if they could be mass produced, it would be the coolest car on the street!

When a good employee becomes a not-so-good employee

There is a lot written on what makes a good employee or employer. These tend to focus around attributes such as dedication, professionalism etc etc. What I want to explore is while these are true, what about when a good employee becomes a not-so-good employee? And of course the same goes for an employer.

The employee and employer relationship can be very delicate. It typically starts off well and both are eager to impress each other, whether to provide a good service or provide a good working environment. The reality is that even under the best intentions and best circumstances this relationship can become strained. I am sure that everyone can think of a few situations where this has become the case.

What I am alluding to here is that many employees have trouble identifying when they've been at a job too long, have reached their potential, and instead of growing, they become disgruntled employees over time. I am sure that everyone has been guilty of this at least once in their career. This of course, applies to some industries and environments more than others but for the sake of this discussion, we'll keep it generic.

The employer can be as guilty as well. They may develop a loyal connection to these long time employees who have probably helped through some tough and good times. Though at some point they are unable to provide new opportunities for growth and advancement.

This is a very unfair system that starts to grow on both parts. The employer recognizing the commitment and history of these employees may tend to promote due to loyalty and not on merit. The result is employees who are eventually promoted to a level of incompetency. This can lead to an employees dissatisfaction at a job.

I'm not implying that anyone should be fired in a case like this, but it is important that both the employee and employer are aware of this. In an ideal world an employee can address this with their employer and if an employee is ready to make a change, to support it and help them with new opportunities. The worst thing an employer can do is try and entice them to stay by simply offering them more money. It's a very short term fix only.

Ultimately, don't stay at a job out of comfort and fear of change. It is much better to leave when things are good and you feel that you've accomplished something. Having been on both sides of the employment relationship, I believe very few employees take full benefit of the opportunities that an employer can offer, even outside of their immediate firm.

For employers, I believe that a primary goal is to provide them with an opportunity to do a good job, and build their skills which will help them land and succeed in their next job.

Why we must endure high oil prices

There is no doubt that the continued rise in oil prices will have impacts across our economy and daily lives for quite some time. A decade ago if you were told that oil was going to surpass $100 per barrel, most people would have responded with "I would sell my car" or "the economy would crumble".

We are feeling an impact in the economy but it has not crumbled (not yet at least), and people are still buying cars. Regardless, while it is painful, we must endure these high prices for very good reasons.

The main reason is that we now have a problem. The oil industry has been in a depressive state for many decades; the result is that little investment and innovation is created. North America has enjoyed a long history of cheap fuel prices (except for a few minor blips) and as a result has developed around this abundance of cheap fuel. Big cars, big highways, inefficient transportation are a natural side effect. Why can't we just produce or refine more fuel? The reason is that when an industry has little innovation or investment in technology, don't expect refineries to be built.

Amidst these high prices also exists an opportunity. This is an opportunity for change and innovation. Hybrid vehicle technology and alternate fuel sources have been around for years. Though while fuel prices have been low, there has been little incentive to work towards an alternative. Today, people are starting to make better choices or at least being aware of choices, hybrid and alternate fuel technologies are on everyones mind and I can ensure you that investment and technology becomes a major focus.

This may surprise some, but we will NEVER run out of oil. That is, as long as prices keep going up (even if the supply really is depleting), this increase in price will drive innovation. Problems and opportunity create innovation. At some point, changes in habits and technology will provide a solution to this problem.

Many nations have endured and prospered with relatively high oil prices, and there are many ways of coping. It may mean a few changes, but change in general is painful. For now we must endure high oil prices and the inconvenience that comes with it.

I have my own ideas on how we can cope, and everyone probably has their own. Though I urge those who merely complain to take some action.