I am going to take a moment of publisher privilege and write about a topic that is not specifically atomic, nuclear, or energy related. As I read about election predictions based on polling numbers, I am reminded of a story I learned during one of my statistics classes.
We had just learned the mathematical formula for determining the required sample size for a specified "error rate". It turns out that if you test 1000 randomly selected representatives of a given group, you will be able to predict the behavior of that group to a precision of +- 3%.
To illustrate the importance of the descriptive words - "random" and "representative" - the instructor told the famous story of the polls that predicted that Dewey would handily beat Truman in the 1948 election. That election took place in the early days of political polling. The poll was taken by telephone at a time when telephones were still luxury items. Since Dewey's supporters tended to be in the higher income classes compared to Truman's, the telephone polls included a systematic sample bias that made their results wildly inaccurate in predicting the actions of the people who voted.
There are a number of observers who are pointing out the fact that the 2010 elections might contain a different kind of telephone bias, since there is a growing population of people who do not own traditional wired telephones. Similar stories ran during the 2008 election season. According to The Pew Research Center, a recent phone sample that specifically includes cell phones in addition to land lines shifted a few points towards candidates from the Democratic Party.
Based on my own admittedly non-random sampling, I suspect that there are other technological developments that are making phone polling less and less representative of the voting population. Though I generally take the time to vote, I NEVER take the time to talk to a pollster. Like most of the people that I call, I only answer the phone if I recognize the caller ID. I let voicemail handle the sales people and the robodialers. I never return those calls.
In the rare instance when I am distracted enough to answer the phone without looking at the caller ID, I hang up without comment as soon as I recognize an autodialer, a poll or a sales pitch. (I have been told I am being rude when I quickly hang up, but I figure I am saving the person on the other end time. Is that rude or not?)
The final technological issue that is becoming more important is the growing use of texting rather than talking. Both of my twenty something daughters are more likely to answer a text from a friend than a telephone call from a stranger - quite frankly, so am I. Are my responses to attempts to intrude in my life common enough to cause an additional bias in the polls towards answers given by people with more time on their hands and less interest in using modern technology?
I am not sure what political positions the people who use cell phones, texting, and caller ID favor, but my prediction is that the final results of the upcoming election will be substantially different from the results predicted by using polls produced from surveying people who actually answer randomly placed calls and take the time to answer questions from surveyors.
Beware of Poll Predictions Based on Poorly Representative Samples
Authored by:
Rod Adams
Rod Adams gained his nuclear knowledge as a submarine engineer officer and as the founder of a company that tried to develop a market for small, modular reactors from 1993-1999. He began publishing Atomic Insights in 1995 and began producing The Atomic Show Podcast in March 2006. He now works for B&W mPower, but his posts on the Internet reflect his personal views and not necessarily the ...
Other Posts by Rod Adams
» Already a member? Login now to comment!
» Not a member? Register to comment!
A guest says:
Ironically people with more time on their hands - vote in higher numbers. Maybe those people that answer the poll have a higher interest in politics in general and are really more likely to vote anyhow. Each pollster puts their name and reputation on the line when they conduct a poll. They get more work the next election on the bias of how good their last poll was. So check out which poll predictions are accurate this season and listen to them next season.Bill Woods says:
Nate Silver has written about this. It's a pretty murky picture:
What seems reasonably clear is this: If you use the types of weighting techniques that Pew uses, you’ll have a lot of problems if you don’t include cellphones in your sample. But this won’t necessarily hold if you use a different set of weighting methods. SurveyUSA, for instance, which conducts surveys by automated script (i.e. “robopolls”), attempted a similar study and found little evidence of bias.
http://fivethirtyeight.blogs.nytimes.com/2010/10/14/bypassed-cellphones-biased-polls/
Rebecca Lutzy says:
Rod - I extended your publisher priviledge to The Energy Collective and chose to re-post this insightful post. Although not explicitly about energy and environment policy, the outcome of races in the upcoming midterm election will have a signifcant impact on the future of energy and climate policy in the U.S., perhaps a larger impact than the 2008 election.
Scott Edward Anderson is a consultant, blogger, and media commentator who blogs at The Green Skeptic. More »
Christine Hertzog is a consultant, author, and a professional explainer focused on Smart Grid. More »
Gary Hunt Gary is an Executive-in-Residence at Deloitte Investments with extensive experience in the energy & utility industries. More »
Jesse Jenkins is a graduate student and researcher at MIT with expertise in energy technology, policy, and innovation. More »
Jim Pierobon helps trade associations/NGOs, government agencies and companies communicate about cleaner energy solutions. More »
Geoffrey Styles is Managing Director of GSW Strategy Group, LLC and an award-winning blogger. More »
The Energy Collective
- YOU
- Rod Adams
- Scott Edward Anderson
- Charles Barton
- Barry Brook
- Dick DeBlasio
- Simon Donner
- Big Gav
- Michael Giberson
- James Greenberger
- Lou Grinzo
- Tyler Hamilton
- Christine Hertzog
- David Hone
- Gary Hunt
- Jesse Jenkins
- Sonita Lontoh
- Jesse Parent
- Jim Pierobon
- Vicky Portwain
- Tom Raftery
- Joseph Romm
- Robert Stavins
- Robert Stowe
- Geoffrey Styles
- Alex Trembath
- Gernot Wagner
- Dan Yurman

About Social Media Today




