Polls should never be trusted for accuracy when read on their own, according to the AFL-CIO’s Mike Podhorzer, a former union political director.
“It almost feels like margin of error has become quaint,” Podhorzer said on a panel at Wispolitics.com’s Midwest Polling Summit last week. “It meant something when it was probability samples, and now it really doesn’t mean anything.”
But not all panelists were in agreement.
Nick Mastronardi, CEO of Polco, said he thinks margin of error still has its place. But that modern polling science hasn’t yet found a perfect way to account for it.
“You can have different confidence levels and margin of error on each of the different partitions of your samples,” Mastronardi said.
Fellow panelist Ashlee Stephenson, vice president and national political director with the U.S. Chamber of Commerce, said part of the problem with margin of error in polls is how the polls are conducted.
She said each poll can get its information in person, over the phone, online or some combination of everything. And most polling agencies will use the same margin of error assumptions no matter how they collected the information.
“There’s a real lack of publicly available or well-known standards for margin of error on a specific mode,” Stephenson said.
She called polls, especially ones that focus on specific groups, to be “directional at best.”
One of the reasons for this is that typical polls won’t represent certain societal groups as much when there is a challenge in reaching out to them, according to Ray Block, associate professor of political science and African American studies at Pennsylvania State University.
Block mentioned the trouble traditional polls have with translating certain English messages into other languages.
“And that’s more than taking something in English and turning it into Spanish, for example,” Block said. “It is literally translating the messages so that the type of message, even if it isn’t a verbatim translation, works for that community.”
Another issue with traditional polling, according to Podhorzer, is that most people are reading election predictions wrong.
He used the 2016 election as an example.
Podhorzer said when analytics gave then-presidential candidate Hillary Clinton a 66 percent chance of victory, those numbers were not based on consistent odds like a coin toss. It meant elections in the past that shared some similarities had her type of candidate winning 66 percent of the time.
“That’s really different,” Podhorzer said. “If things have changed from this election to prior elections, then it’s just way off.”
The panel, led by Marquette University Law School Poll Director Charles Franklin, agreed there are many flaws in current polling practices, such as the challenge with gathering information on certain demographics and the cost and time challenges that come with “doing it right.”
Panelists said campaigns these days don’t even necessarily need traditional polls to be successful.
Stephenson called the Trump era “unlike any other.” She said Trump is using a multitude of talking points in rallies and online fundraising and “just seeing what sticks.”
“It’s going to be interesting going forward seeing what kind of campaigns can survive using very serious expensive academic message-testing versus those who follow in [Trump’s] mold and just slingshot it,” Stephenson said.
Franklin called it “a pretty effective strategy” for campaigns to try open testing over polling, “as long as there’s not a price to be paid for a message that flops.”
The summit was held Oct. 9 in Madison, and featured top experts talking about the issues and trends that will affect campaign 2020 and key Midwest battlegrounds, including Wisconsin. See previous coverage here.