Big Data, Predictive Analytics & Marketing Research Strike Out in Election 2016
Nov 10, 2016
At around 7pm, Tuesday evening, on about any one of a dozen Election Coverage programs, you would have heard someone say “We see Clinton reaching more than 300 electoral votes…”
And around 9pm, that very same night, those very same people were starting to say “We see Trump reaching more than 300 electoral votes….”
At this time, it looks like President-elect Trump earned about 279.
How could all the masters of data been so wrong?
The results suggest pollsters may have wildly underestimated the number of hidden Trump voters — people who stampeded to the ballot box on Election Day but never showed up on the radar of surveyors. Source: How Did Pollsters Get Trump, Clinton Election So Wrong?
Know your Universe.
Early reports have focused on pollsters ignoring those that didn’t vote in the last presidential election and focusing on “those that voted in the last presidential election”. 20/20 hindsight – maybe define your universe as “registered voters” rather than the smaller subset of “registered voters that voted in previous presidential election”?
Defining your universe clearly, concisely, accurately is key to getting the insights you desire. Asking the wrong people or failing to include everyone that needs to be included can definitely lead to some nasty surprises at the end of the day.
"There's some suggestion that Clinton supporters are more likely to say they're a Clinton supporter than Trump supporters are to say they're a Trump supporter," Arie Kapteyn, director of the University of Southern California’s (USC) Dornsife Center for Economic and Social Research.
“Everybody lies. The only variable is about what.”
I toss that one out to all the House fans. But as Arie Kapteyn stated (see above), the quality and accuracy of the data collected from the right people may be in question.
Surveys, interviews…they are built on a simple premise which is that honesty is assumed. Now, people sometimes tell you what they think you want to hear. And they sometimes tell you what they think is honest and accurate – but it isn’t because of a variety of reasons including limited/unique exposure to something.
“The very premise of polling is based on the idea that voters will be completely honest with total strangers,” said veteran GOP operative Ned Ryun, who runs a grassroots group called American Majority. Source How Did Everyone Get It So Wrong?
But at the end of the day, the belief is that if you ask the right people the right questions you will get insight into the “right answer” that is statistically viable (applicable to the group as a whole). The outliers will be identified and the insight you desire will show itself.
In research, bias occurs when “systematic error [is] introduced into sampling or testing by selecting or encouraging one outcome or answer over others”. Bias can occur at any phase of research, including study design or data collection, as well as in the process of data analysis and publication. Source: Identifying and Avoiding Bias in Research
Bias.
There seems to have several “systemic errors” including the definition of the universe (study design).
…some voters were apparently sheepish about admitting to a human pollster that they were backing Trump. But the L.A. Times/USC poll was based on an internet survey of a recruited group of voters. Source: How Did Pollsters Get Trump, Clinton Election So Wrong?
Data Collection Methodology.
There was some talk about how open and honest people are when participating in a telephone survey (speaking with a human) and answering an online survey (no human interaction with higher perception of anonymity). Interesting concept that needs to be tested and proven to be accurate. But right now, that and $2.50 gets you a Pumpkin Spiced Latte.
Surveys and Research are wasted time, energy and money!
Nope. Don’t believe that for a second. But what we have seen is just how easy it is for some research to go so bad – imagine what might have occurred if this work was performed by people that don’t do it for a living! So the next time you have a need, get an expert involved.
Next, don’t blindly accept whatever is recommended. Ask questions. Push. Make them explain and defend their recommendations. Ask why they recommend something and why they didn’t recommend something else because even though they are experts, they are typically not as close to your world as you are and you may have some insight that could have a significant impact on what they suggest.
Remember, when you get your findings - understand that they are a reflection of what was when the research was performed and things change so you might want to revisit the research on a regular basis in order to make sure the findings are still applicable or if there is change happening that you might need to address. That’s why we HATE one off research projects – they tend to cause people to think that they findings are carved in stone for all eternity and that leads to missing shifts in the market that can kill your business.
Lastly, when it comes to marketing research, remember that you don’t want to take the findings and go all in – you need to do some real live testing. In marketing, you have to use research as a guide, and that you should test to validate the results of the study. Why? Because what they tell you in the research has yet to be proven in the market – and you need to make sure that what they said they would do is what they really do.
Just because they told you they will buy your widget in the color red doesn’t mean they really will buy your widget in red when/if they are presented to color options. Bottom line – test, measure, analyze, modify and repeat!
Patrick McGraw is VP of Higher Educaton Marketing Services and has more than 25 years experience in market research, competitive intelligence, business intelligence including database marketing and CRM, strategic planning, brand development and management as well as operations/campaign management. His work has consistently helped his clients and employers develop and implement more efficient ways to attract and retain profitable customers, enter new markets and launch new products. His areas of focus include the education, hospitality, travel and tourism, hi-tech, telecommunications, financial services, and retail industries on both the agency and customer sides.
Add Pingback