New Poll May Not Be So Authoritative
A new telephone poll commissioned by WGRZ TV has already been posted with a story in the online version of Buffalo Business First. This, the “final poll” commissioned by the TV station from Survey USA, puts Mayor Byron Brown ahead of challenger Mickey Kearns.
Survey USA also conducted a poll for WTVD-TV in Raleigh-Durham, NC last fall, for the Presidential election. There, three previous Survey USA polls had put McCain up by eight, five, and four points, while the fourth one put him up 20. Said McCain would get 58% of the vote, Obama 38%.
On election day, Obama won North Carolina and picked up 15 electoral votes.
So remember, polls are good space fillers for media outlets, but they aren’t always accurate, and they don’t even have to be, no offense to Survey USA.
People seem to love ‘em, though, so I figured I’d get a little mileage off this one, seeing as somebody else paid for it.
Potentially related posts (automatically generated):
Phone polls may be dinosaurs. Too many people who vote don’t have listed phones, therefore it is impossible to get a true random sample, and all statistical inference falls apart.
Comment by Peter A Reese — September 14, 2009 @ 9:58 am
Another point of interest is how the poll leaves out Hispanic voters. “Too few respondents of this type were interviewed for this data to be meaningful,” the survey explains.
Comment by Buck Quigley — September 14, 2009 @ 10:43 am
It’s probably worth noting that the poll we released in North Carolina showing McCain up 20 points was released on September 9, nearly two months before the election:
http://www.surveyusa.com/client/PollReport.aspx?g=9d3db8ba-996f-414f-b9d0-5bc16f50d34d
Our final poll, released 24 hours before the election, showed the race tied, with McCain nominally ahead by 1 point, and carefully explained that turnout would determine the winner:
http://www.surveyusa.com/client/PollReport.aspx?g=dbe1ab15-bdcc-4503-9a25-caa47a4952d1
The actual result was a 0.4% Obama win.
You can see the fully graphed tracking over seven releases by clicking the “1/T” icon on either of those links. Polls are, of course, a snapshot of opinion at a given time, and the 9/9 poll went into the field two days after the Republican National Convention ended. It also, perhaps coincidentally, had the highest percentage of self-identified Republicans of any of our polls in that contest, and the lowest percentage of self-identified Democrats, if you look at the “composition” lines:
http://www.surveyusa.com/client/PollTrack.aspx?g=e31e80a6-67b7-43a1-81fd-ef5f0a9aaa12&x=74531,1
http://www.surveyusa.com/client/PollTrack.aspx?g=e31e80a6-67b7-43a1-81fd-ef5f0a9aaa12&x=74531,2
It’s possible that shift was simply due to an “odd sample,” which can happen from time to time — statistically, about 5% of the time — but it’s also possible it was reflective of a genuine but temporary shift in Republican enthusiasm at that moment in time. (It’s of course much easier to explain these shifts in opinion and try to determine if you’re looking at a genuine shift in opinion or an “outlier” well after the fact, though I would point out that the electorate in both our Buffalo mayor poll releases is fairly similar, though not identical, to one another.)
Comment by Ken Alper — September 14, 2009 @ 10:51 am
Thanks for getting back to us with such a detailed response so quickly, Ken. (Ken is the News Director for Survey USA.)
After the election tomorrow, we’ll be able to check the accuracy of Survey USA’s final poll for the Buffalo Mayoral primary.
Comment by Buck Quigley — September 14, 2009 @ 12:03 pm
Buck, your post was very one-sided and lacking integrity because you mentioned Survey USA’s polling two months before the Nov 2008 election and omitted any mention of their final poll which was perfectly accurate within their stated margin of error.
http://abclocal.go.com/wtvd/story?section=news/politics&id=6485717
Good for Survey USA for calling you out on your bias. Your response didn’t even own up to what you knew you did in your b.s. posting.
Artvoice has embarrassed itself several times in its election coverage.
Comment by AV = Kearns Mouthpiece — September 14, 2009 @ 9:26 pm
Dear AV=Kearns Mouthpiece,
Sorry you feel that way. You’re right, their final poll in the presidential election “was perfectly accurate within their stated margin of error.”
Unfortunately, it too was wrong in picking McCain to win.
The point of my post remains that polling is not always accurate.
Thanks for writing.
Comment by Buck Quigley — September 15, 2009 @ 9:36 am
Saying “it too was wrong in picking McCain to win” just shows real (or more likely fake) ignorance on Buck Quigley’s part about what margin of error means.
So does his saying: “The point of my post remains that polling is not always accurate.”
Survey USA’s poll Oct 30 to Nov 2 was indeed accurate. It said McCain was ahead by 1% in N.C. and they CLEARLY said the margin of error is +/-3.8%.
That’s reporting a statistical dead heat which is NOT “picking McCain to win”, and it’s difficult to believe you don’t realize that.
Here’s what the poll reported, verbatim:
“Question 1 – If the election for President were today, would you vote for … (choices rotated) Republican John McCain? Democrat Barack Obama? Or one of the other candidates? (Margin of error +-3.8%)
49% McCain (R)
48% Obama (D)
2% Other
2% Undecided”
http://abclocal.go.com/wtvd/story?section=news/politics&id=6485717
Reporting a 1% lead with +/-3.8% margin of error is NOT predicting either candidate will win. Obama’s margin of victory in N.C. was 0.4%, making that poll fully accurate.
Comment by AV = Kearns Mouthpiece — September 15, 2009 @ 3:15 pm
Dear AV=Kearns Mouthpiece,
I don’t ordinarily reply to anonymous comments, since I think they’re cowardly. However, in your case I’ll make an exception.
Survey USA final poll:
Brown 51%
Kearns 44%
Actual election results:
Brown 63%
Kearns 37%
Please wrap up your discussion about the accuracy of automated telephone polling.
Thanks!
Comment by Buck Quigley — September 16, 2009 @ 9:12 am
Inaccuracy of their Buffalo poll doesn’t change the b.s. you reported about Survey USA. Only someone with awful reading skills would think anyone said all polls are always accurate.
Your post was wrong to cite only their early N.C. poll that was 2 months before the vote. After Survey USA called you out about that, you then reported a wrong fact about their later poll.
You said their later poll “was wrong in picking McCain to win”. Only a clueless “reporter” would think it picked McCain to win. It said McCain was up by 1% with +/-3.8% margin of error. That’s not picking anybody to win. The 0.4% vote margin is easily in the range. They were right, your reporting was wrong twice.
Condolences about Mickey losing even worse than predicted.
Comment by AV = Kearns Mouthpiece — September 16, 2009 @ 10:16 pm