Marketing to Women #11 – The Challenges of Research

Wouldn't it be nice?

I’ve spent the last four years in market research, almost entirely focused on analyzing secondary data sources (i.e. getting data and/or conclusions from someone else and analyzing them for some other purpose). It’s a fantastic way to do research, mostly because the hard work has already been done and you, the secondary researcher, simply has to pull it all together, see a pattern and develop conclusions from the available work. It’s (relatively) fast, cheap and easy compared to surveys, focus groups and other primary research methods.

The problem is that the studies aren’t always right. Maybe it was bad methodology. Maybe there wasn’t enough data. Maybe the margin-of-error was too large or the sample size was too small or the test subjects weren’t random enough for the data to mean anything. Sometimes the study is developed in order to reach a specific conclusion (do you think industry-sponsored studies are always conducted for the benefit of the public?) or maybe there was a political reason why there just had to be a conclusion to justify the money spent on the study. It’s a minefield.

“85% of purchases are made or influenced by women.”

That’s a big deal, right? And it’s critical to the argument of the importance of focusing on women in the marketplace. Is it true? I don’t know, because I can’t find the data to support it. Despite the fact that it is used in nearly every presentation/article/book I’ve come across, I can’t seem to find where it originated. The problem gets worse as the game of statistics telephone continues — I’ve seen it said that 85% of purchases are made by women, which is a whole lot different than adding influencing and buying together. What is the 85% in the first place — is that on a dollar basis or a number-of-purchases basis? And what the hell does “influence” mean, anyway? Does she have veto power, is she subtly hinting at a decision, or are men asking for women’s input? These details are important.

Let’s assume that the 85% number is based on the number of actual decisions to keep it easy (i.e. buying a car is equivalent to buying a bottle of shampoo). I’ve completely made up the following data (seriously, please don’t cite me as a source for these bogus graphs) but let’s see how this might look depending on different methodologies and conclusions:

There are four scenarios above: the first is that women are making 85% of the decisions, which is a pretty amazing number; the second is that men and women are separately making 15% of the decisions, but influence each other on 70% of purchases; the third is that decisions are different for each gender with regard to both income source and influence; and the fourth differs by how women are influencing the decision. Wouldn’t those differences matter to your analysis?

Why does any of this matter?

We’re often attracted to data that justifies things we already believe, known as confirmation bias. In a recent post discussing policy research, which is particularly susceptible to confirmation bias, Will Wilkinson says:

Fairly few political commentators know enough to decide which research papers are methodologically convincing and which aren’t. So we often end up touting the papers that sound right, and the papers that sound right are, unsurprisingly, the ones that accord most closely with our view of the world.

This might seem unusually wonky, but I think it’s critical to what we’re all trying to accomplish. One of the ways we can convince marketers of both genders to pay attention to this subject is to give solid evidence that marketing to women is critical to sales success. If we’re throwing around numbers that sound unrealistic or are based on easily-refuted research, we’re making the focus on women easier to dismiss.

On a personal level, I tend to think the 85% number is close to accurate on a dollars basis (that’s huge) and that women have an amazing amount of veto power in purchasing decisions whether or not she’s writing the check. What are the percentages? Whose money is each person spending? Those things matter and would change my behavior in how I reach out to consumers.

Do any of you know where to find well-written research in this space? Do YOU have the source for the 85% number? Have you ever seen a piece of research that you found questionable? Have you ever made a decision based on bad research?