In February, 90 days remained before the June primary election and the mayoral campaign for Carl DeMaio sent out an email to potential donors.
It touted a SurveyUSA poll commissioned by KGTV, Channel 10.
The poll showed DeMaio up by 1 point over Bob Filner, 25 percent to 24 percent. The email, signed by Mike Turk, cited a list of apparently eye-popping facts to come out of the poll. But, most important for his purpose, was the fact that Nathan Fletcher and Bonnie Dumanis were trailing badly.
It was time for the city’s center-right establishment to board the DeMaio train. Look at the poll!
Bob Filner is counting on you to sit on the sidelines.
Even better, Bob Filner would love for you to donate to one of the two nominal candidates in the race.
While the polls show they have no traction to win a place in the runoff, they can engage in nasty, negative campaigning to create a messier primary.
DO NOT let that happen!
DeMaio ended up winning the primary, with Filner close behind, just like SurveyUSA said.
Now, the latest SurveyUSA poll has Filner up 7 points. And now, yes, SurveyUSA polls are not to be trusted. This weekend, as U-T San Diego was teasing its new poll in the mayor’s race, U-T writer Craig Gustafson remarked on Twitter about one aspect that would make the poll unlike SurveyUSA’s.
Stephen Puetz, one of DeMaio’s chief campaign consultants, responded with some snark: “The analysis on any legitimate poll will use the phrase “unlike surveyusa… ” he wrote.
That U-T poll did come out and, unlike SurveyUSA’s, it had DeMaio up by a shocking 10 points.
This one, DeMaio’s team liked.
Unlike the national stage, our local races don’t get many independent polls, which is why I highly anticipated the U-T’s. Polls from the campaigns that come out have shown a much tighter race.
Now, the U-T’s ownership has made no bones about its preference for DeMaio, running two front-page editorials on his behalf, donating to his campaigns and otherwise saying DeMaio’s rival, Filner, is a bad person.
Despite that, I trust the newsroom’s effort to try to poll the race. And I’m working off the assumption they gave it their best shot.
That said, the poll seems like a major outlier. Even the Republican Party’s own poll over the same period showed a dead heat.
Labor and associated friends of Filner released a poll with Filner up by 3 points.
So what’s going on?
I asked Vladimir Kogan, now a political science professor at Ohio State, to give me his take. He thinks what was most odd is how the U-T’s pollsters, the Glover Park Group, weighted the sample of respondents.
After all, the sample is always what’s most important about a poll.
Here were his three points:
I. Weighting on Party Identification Was Odd
Kogan:
Weighting is done to correct for non-random survey non-response. The generally accepted practice is to assign weights based on factors for which the population parameters are known. For example, we know from the 2010 Census (assuming the Census is correct) how many people in San Diego are white, how many are black, etc. These are population parameters in that they are largely fixed and known ahead of time, so we can know how different our survey sample is from the population. No reputable poll I know of assigns weights based on party ID, because party ID is not a population parameter. Party ID, like vote choice, changes during the course of the campaign.
Here’s more on that point, from Nate Silver, the poll guru at The New York Times.
II. Weighted by Party of Registered Voters but Filtered for Likely Voters
If you’re going to weigh by party, maybe don’t do it this way.
Again, Kogan:
In the methods note, the U-T says their poll was of “likely voters.” Yet, right below, they say they assigned weights based on population of registered voters. That is a problem, since not all registered voters are likely to vote. For example, if their sample weights are constructed to make decline to state voters the same proportion of their sample as this group’s share of the registered voter population, their sample would greatly overestimate this group’s voice, since DTS voters are substantially less likely to vote than partisans.
On the site, the U-T says it reached 626 registered voters, of whom 614 were deemed likely to vote. That doesn’t seem like much of a screening.
III. Unusual Exclusions
Kogan and many Filner fans on Twitter found something bizarre about some groups of potential respondents that the U-T and its pollsters excluded. From the poll:
“Employees of a newspaper, television/radio station, marketing/public opinion research company or the city of San Diego — or who live with someone employed in one of those fields — were excluded,” it said.
If city employees would improperly skew a poll, maybe we should not let them vote at all!
Finally, I don’t like to get into analysis of the horse race. I’m no good at it. But I found this worth trying to understand.
A couple more points:
• There does seem to be a trend toward DeMaio. A SurveyUSA poll showed a 12-point Filner lead over DeMaio in late September. Then on Oct. 15, SurveyUSA showed a 7-point lead for Filner. Now labor and other Filner fans are touting a 3-point lead. Filner’s campaign may not mind a little tension as well to keep their own troops motivated.
• The U-T’s been wrong, badly, before on polling. At about this time, in 2010, the U-T’s commissioned pollsters declared that voters were split on Proposition D, the sales tax increase. Voters were not split. D went down in flames.
It was a different polling firm.
Still, DeMaio’s campaign is likely not assuming he has a 10-point lead.
I’m Scott Lewis, the CEO of Voice of San Diego. Please contact me if you’d like at scott.lewis@voiceofsandiego.org or 619.325.0527 and follow me on Twitter (it’s a blast!):
Like VOSD on Facebook.
Disclosure: Voice of San Diego members and supporters may be mentioned or have a stake in the stories we cover. For a complete list of our contributors, click here.