(This is my first Patch blog. Hope you dig it).
Doesn’t matter if you’re a Republican or Democrat - each day’s new deluge of polling results from the news media has to be driving us all nuts. How can Romney be 3% points up on a Thursday and 3% behind on a Friday? Did 6% of the US voting population suddenly change their minds overnight????
I’m pretty sure something else is going on here: It has to do with the ratings-hunger of news media outlets and their inclination to hype the “horse-race.” One way they do this is to scan the day’s polling results from various sources and “cherry-pick” the ones they like best.
Result: we’re fed apples-to-oranges comparisons from day-to-day – “Robo-Phone Survey A” (with a small sample size and few questions) gets compared with “Phone Interview Survey B” (with an average sample size and a different order of questions) and this, in turn, is followed by “Mail-in Survey C” (with a huge sample size and long list of questions). It should come as no surprise that polling results over time bounce around like Angry Birds.
This isn’t to say that polls can’t be illuminating and useful – they just have to be used wisely. Here are a few examples of how to do just that:
- Bookmark the Real Clear Politics website (www.realclearpolitics.com) and
check it regularly. They post and track all the various polling results (&
also provide survey dates and sample size). They do this both nationally and
for a dozen or so key battleground states.
- Zero in on two specific factors: (1) polling results from the same source over a period of time. (for example, I like to track the latest Rassmussen polling data along with earlier Rassmussen results to see if Obama’s or Romney’s lead is widening, narrowing or staying the same. FOX, CBS, NBC and WSJ also poll regularly). (2) Have any new results been posted within the past week? (Sometimes the most recent statewide data may be more than a month old).
- Factor in Sample Size from each poll – Small samples tend to result in bigger statistical “bounce.” Results from larger surveys tend to be more stable from month to month.
Be skeptical whena headline announces a sudden dramatic shift in voter sentiment. (It could just be MSNBC, FOX or CNN looking to stir things up). Use a website like Real Clear to fact check the polling date. More often than not, you’ll discover that little if anything has changed very much. That said – reach for the Maalox and enjoy the horse-race.
The trick is for the viewer to understand what any singular poll truly represents, and that is a "snapshot in time" as many pundits like to say. It is important to recognize that polls are typically administered over a period that is a few days or weeks prior to the actual release of the results. A good illustration would be to look at tracking poll numbers for candidates during the Republican primary season. It has always been pretty clear that Herman Cain was not going to get the Republican nomination. Yet, his poll numbers would spike from time to time, the result of some highly successful speech or fundraiser that happened to occur during the period that said poll was gathering its data. A political poll can only be representative of data it has had the ability to obtain. Polls make no assumption of future events.
* More about FiveThirtyEight: http://en.wikipedia.org/wiki/FiveThirtyEight * Nate Silver's election forecast and accompanying column: http://fivethirtyeight.blogs.nytimes.com/
I suspect (but can't prove) that numbers are sometimes jerryrigged in an "opposite direction" (i.e. lowballing a preferred candidate early-on in order to make his/her numbers look better later on when it counts. It's not always the pollsters; sometimes it's the media trying to support a contrived narrative by cherrypicking the polling results they prefer.