Thursday, November 1, 2012

A Very Good Look at Polling and Poll Analysis

Dan McLaughlin (also known as Baseball Crank) has an excellent article at RedState discussing polls, and why we need to take them with a grain of salt this year.  It is a very long article, but well worth the read if you want to understand the underlying math and assumptions that goes into using small samples to predict the behavior of large groups.

Yesterday, in one of the comments, I mentioned that I am less concerns about "movement" in the polls right now, than I am with the facts on the ground.  We can get caught up in a small polling bump for Obama and start worrying about its meaning.  But small bumps in polling averages have very little import 5 days from an election.  As Dan says in his article:

Obama is getting clobbered with independent voters, traditionally the largest variable in any election and especially in a presidential election, where both sides will usually have sophisticated, well-funded turnout operations in the field. He’s on track to lose independents by double digits nationally, and the last three candidates to do that were Dukakis, Mondale and Carter in 1980. And he’s not balancing that with any particular crossover advantage (i.e., drawing more crossover Republican voters than Romney is drawing crossover Democratic voters).

We are now in the period where the campaigns are executing their ground game.  What is important to the campaigns is executing their early voting operations and getting their voters to the polls on election day.  Swaying the last few undecided voters toward your message is of much less importance to winning.

For the most part, the War for Independents is over.  Obama started the war by trying to damage the Romney brand using 5 months of negative campaigning.  The negative War on Women and Bain ads are still running in Florida.  Romney instead tried to woo them by convincing them "we can't afford another 4 years".  He got their attention during the first debate, and has solidified their support during the last two weeks by promising to "reach across the aisle".  This is the same message that Obama used in 2008 to great effect.  It is also a message that Obama can not use this year.  His governing style over the last 4 years would make that claim laughable.  As we enter the last 5 days, Independents are favoring Romney by an average of 8 points in all the current national polls.  This is about a 20 point swing from 2008, when Obama dominated.

Now we are in the period of the early voting ground game.  This is where Obama won the election in 2008.  He ran up such an advantage among early voters that year, that in many states, such as Florida and North Carolina, even though he lost the vote on Election Day, he still won the state.  But Obama's early voting advantage has collapsed.  Over on Breitbart they discuss Pew's finding that Obama is suffering from a 26% drop in early votes.  Over at AEI, Henry Olsen takes a look at the Ohio early votes and find support for Romney's claim that they are turning out new GOP voters, while Obama is underperforming in counties that support him.  A key paragraph from his analysis:

The numbers are particularly strong for Romney in the southeastern coal country on or near the Ohio River. From Scioto county in the south to Columbiana county in the north, early voting shares range from a low of 63.5% in Monroe to 82.7% in Columbiana. (Athens County, an Obama stronghold because of Ohio University, touches the Ohio River- its early voting share is only 57.4%). To compare, the early voting shares in the largest and strongest Obama counties (Cuyahoga, Lucas, Franklin, Summit, and Lorain) never top 61.0% (Cuyahoga).
If these early voting trends continue, then Romney will have erased Obama's strongest advantage in the race.  It will all come down to turnout on election day.  This is an area that Reince Priebus at the RNC has focused on specifically for the last two years.  They tested the methods in June in Wisconsin during the Scott Walker recall, and were able to over perform all expectations during election day.  Walker was able to win 60 of 72 counties, and cruised to an easy victory, well beyond expectations.

So take a deep breath and focus on the important things now.  Watch where the the campaigns are going (Wisconsin), and take heart that the Romney campaign is where they wanted to be, and are executing their final week strategies.

One final note from Dan's article.  I want to note his unknowing acknowledgement that my efforts here are on the right track.

One of the more widely-discussed efforts to fix the problem of topline poll data varying by turnout models is Dean Chambers’, which takes the internals of each poll and re-weights them for a more Romney-friendly turnout model. In concept, what Chambers is doing is on the right track, because it lets us separate how much of the poll toplines is due to the sentiments of different groups and how much is due to assumptions about turnout.

Hat tip to Matt for pointing out this article to me.


  1. Good article by McLaughlin. And as a sabermetric follower that respects Nate Silver, it's as simple as this:

    "Nate Silver’s much-celebrated model is, like other poll averages, based simply on analyzing the toplines of public polls."

    Elections are not like predicting baseball results, where you have numerical FACTS and use knowledgeable opinion to interpret them how you like. Silver's percentages on winning are utter nonsense.

    Dean Chambers, while an embarrassment for Republicans with his divisive spew, serves a purpose. He's putting his neck on the line and saying the majority of polls are skewed. He's probably wrong with how extreme he is, but it's something.

    Silver is only doing what all of us following polls do, weight them on whatever preference he has (and refuses to show). Since he's admitting to being an Obama supporter, is writing for the New York Times and formerly had a Kos blog, I can guess where he stands.

    1. The LAST thing I expected to see, checking out your blog, was sports and wrestling :)

      My problem with Silver is that he isn't applying the same rigor to the underlying data sets that he does in baseball. IMO, the least reliable numbers in any poll are the topline summary.

  2. It was a great article (and I say this as someone who left RedState for dead because I didn't like the direction they were going).

    The big takeaway for me is that the models and math go out the window once you start challenging the underlying assumptions. It's garbage in, garbage out and the topline numbers are garbage . It's very logical, and I'm convinced, and I really don't think it's a "far out" concept.

    I think my "fear" is, I hate being in the position of saying "all the experts are wrong and i'm right" because it starts to feel like wish casting and I do remember some conservatives making cases like this that McCain was actually going to win in 2008. this is completely different however, especially since reputable polling agencies like Gallup have been showing Romney winning handily.

    Right now though, liberals have basically put their entire polling argument on ohio, they're not even really trying to make the case that Obama will win the popular vote, just that he'll be able to sneak in on a single state because of turnout and 4 years of organizing. I honestly believe the MSM polling agencies are happy to adjust their models to keep hope alive.

  3. Great article that explains it very well.

  4. Replies
    1. Honestly, everyone is. I'm looking at the data, everything looks solid. Romney has everything lined up to win this. I can't think of anything that could change the direction now.

      The only points of nervousness I am getting are from the Romney campaign itself, talking about how close it is. But if it was that close they wouldn't be trying to expand the map.

      But the fundamentals haven't changed. All of the polls say the economy sucks and Romney is better able to fix it.

    2. Serious question (I love the site and analysis by the way, sites like yours have kept me sane lately).

      If the Romney campaign knew they had a great chance of winning, might they still make it look close anyway? Fear is a powerful motivator. I know plenty of people will crawl over broken glass that is on fire and shoots acid at them in order to vote in this election. But hypothetically, could talking about it in terms of it still being close just be a little bit of insurance to light a fire in people and encourage them to take nothing for granted?

    3. I think that is a real possibility. I don't want to claim to know what the campaign knows and doesn't know, but it is very possible that the Romney folks are looking at very good numbers, but don't want their troops to get complacent. They want them working around the clock, because fear motivates.

      Typically, a campaign has a range of internals that measure using multiple turnout models. It is possible that they are internally publishing a pessimistic one.

      Today I heard from someone working inside that folks are talking about Ohio is close and they think they will win but barely. But I keep looking at the Ohio poll internals and the early vote numbers, and I can't see how he can lose. I'm seeing at least a 4 point win in Ohio.

      But I don't know. Maybe I'm missing something important.

    4. I'm with you Dave, I don't see how we lose this one. If we do, I will be floored.

    5. Thanks for the hopeful dose of sanity, Dave. And if somehow you're wrong in the end, at least you did a good job making us feel better all month.

      Although I know that wasn't the purpose. In fact, right before the first debate, you had a lot of O's popping into your models.

    6. What you're missing, perhaps, is the possibility of a last-minute groundshifter like the DUI reveal in 2000 -- turned an easy win into a squeaker (with the Dem again aided by bogus polling, this time on the pre-poll-close FL call).

      Pretty sure Romney is squeaky clean, though, and Benghazi is tying down most of Obama's dog-wagging options. Still, something could always happen.

  5. Interesting stuff, except I think what you are writing here is much better than Dean's stuff. Both are doing good work, but yours is more insightful.

  6. My views on the state of the election right now, are that the Romney camp is getting exceptional numbers all across the board. Swing states are almost all going their way and now he has turned states such as Michigan, PA, Minn. into true battleground states. They have so much money to spend and so many motivated voters to speak to, that they expanding the map just for the hell of it, just in case something goes wrong, just add to the many ways they win the electorate. I am highly confident in a Romney win.

  7. If you're a campaign, you never want to say you're losing but you also don't want to say you're comfortably ahead. The best answer is always we're "narrowly winning". That keeps morale high but no one gets lazy.

    You have to keep an attitude of "all hands on deck" until the polls close.

    Also, polling has its limits, even internal ones. Eventually it's about turnout and it's hard to say exactly how well a turnout operation "works" before it actually goes into effect, my guess is they're narrowly winning with something like a D+3 model. I honestly believe though that Republican turnout this year will exceed even optimistic scenarios, but we'll have to wait and see.

  8. Well, IMO some of the state polls are close and rather than leaving it to chance (and risk having a recount situation), might try to get a nice buffer to avoid any "controversy" to which the slew of lawyers working for Obama will try to find anything to give their client some kind of... in (rather than out).

    If Romney does win, I have to wonder what kind of backlash some of these pollsters will face. I know within the media it won't matter but there's a society within this field that I hope others accountable.

    To me, when the media shows these polls, they almost always display the number of voters surveyed and the MOE, but I think they should also have the D/R/I distribution when available. Also, RCP should exclude any poll that doesn't publish their D/R/I. It gives context and might hold some of these pollsters accountable for how and why they come up with their numbers.

    1. Backlash? Probably none, after all Zogby is still employed.

      Can you tell doing this has made me pretty cynical about polling? :)

    2. So true and you cynical? No way!

      It does piss me off that nothing will be done to those pollsters who are way off (meaning they'll continue to be placed in the RCP averages and get guest slots on TV). There are some whose models or techniques are off and that's fine so long as they adjust it, but there are others who seem to skew it to fit a narrative.

      BTW, IYO what percentage would you place in Romney winning?

    3. I'd guess about 80%. The only data point we have that he could lose is the top line numbers of the polls.

      Put another way, I give the Democrats less than a 20% chance of getting the election day turnout they will need.

  9. I'm not going to lie. I'm getting a little worried after Rasmussen, ABC and even PPP found a substantial dip in Romney's lead among Independents, though I do find it a bit interesting that most of the top lines didn't change much. I can't see anything that would make Independents abandon Romney with less than a week to go before the election. All of this is making me half-dizzy/half-sick.

    1. Yeah, it is odd. I can't imagine Hurricane Sandy did that much to sway Independents like that. OTOH, things are chaotic it seems and while they won't place a lot of blame on Obama, he won't get the benefit.

      Also, "even PPP"? Aren't they a Democratic polling firm? It would stand to reason they might do something. Rasmussen? That would be worrisome though what was the distribution on that poll?

      Anyway, in the end between the voter enthusiasm gap and Romney's still strong support amongst Independents, I still see a decent win.

    2. That doesn't worry me. I never believed the numbers like 21%. I've always thought Romney would be around 8% with the Independents, which is where he is right now. That matches the lead Obama had with them in 2008. If Hope and Change can only get 8%...