What is Fact and what is Conjecture?

In the world of polling and election prediction, there are actually very few facts.  Most of what we deal with are conjectures based on a small set of data that we think might give us clues about future outcomes.  There are even some surprising things that we take as gospel, that are actually not proven facts.  We think they are right, but we really don't know.  We can't prove them.

Facts:

 

In the current discussion of the 2012 election the following items are provable facts:

• In the 2008 election 69,456,897 voted for Obama and 59,934,814 voted for McCain.
• In the 2004 election 62,040,610 voted for Bush and 59,028,444 voted for Kerry.
• In 2008 there were 8,322,657 more voters than in 2004.
• In 2012 there are 490,000 less voters in Ohio.
• In 2012 there are 215,600 less voters in Cleveland and Cuyahoga County.
• In 2012 Iowa has moved from 110,000 voter advantage for Democrats to a 20,000 voter advantage for Republicans.
• In 2012 Florida Democrat voter advantage over Republicans has dropped 170,000.
• There are other measurable shifts of registration advantage in Nevada, New Mexico, North Carolina, Colorado, and Pennsylvania.
• The response rate that Pew receives in it's polling has dropped to 9%. 

 

Conjecture:

Pretty much everything else involved in polling is a conjecture.  The D/R/I split in 2004, 2008, and 2010 that we use?  They are all based off exit polling which we think is accurate, but we can't prove.  The only facts are actual vote totals, which are not divided into partisan identification as part of the vote counting, and actual registrations currently in the hands of the Boards of Election.  In some states, such as Ohio and Virginia, we don't even know the partisan ID of registered voters.

To provide an extreme example, it is possible, but very highly unlikely, that all 69 million of Obama's voters were Democrats, and all Republicans and Independents voted for McCain, less those people who were individually interviewed and said they were a Republican or Independent who voted for Obama.  The exit polls just happened to poll only the couple thousand folks who fall into these categories.

You could also get a couple million monkeys to randomly pound away on a keyboard and have them produce Hamlet.

 

So What?

Based on historical precedent, polls have proven to be predictive with a specific margin of error.  So if the polls say that things are generally tied, then things are likely generally tied.  Individual polls can be very wrong.  Even Rasmussen has been wrong by 3 or more points in the past.  Every cycle, the pollsters try to correct for past errors (at least reputable ones do).  But their results are always based on a specific model of turnout.

In the case of some pollsters, they aren't trying to correct for an expected turnout model.  They are sampling 1000 respondents and reporting on the results.  If they have a D+12 sample, then that is just what they found.

But historically, we know that a D+12 result in the electorate is highly unlikely (again this is conjecture, not a fact).  So how can a responsible polling firm end up with a D+12 result?

In 2004 we learned that certain segments of the population are more interested in responding to polls than others.  In 2004, the exit polls showed a decisive victory for Kerry.  Historically, exit polls were considered the most accurate of all polls, because you were asking how people actually voted, rather than how they intended to vote.  But in 2004 we found that even these polls can skew dramatically due to self selection by respondents.  Kerry voters were highly motivated to tell pollsters who they voted for, while Bush voters were not.

Look again at the final fact that I mention above.

If there are skews in the polling, it is possible that Democrats are more motivated to respond to polls.  We have no insight into the actual thoughts or intents of the 91% who refuse to participate.  We only assume that their opinions and demographics are no different than people who were not contacted in the first place.

It is also possible that in 2012 Democrats will turn out at a 12% higher rate than Republicans.  Even after the election ends, we really won't know.  We will take some exit polls and report a conjecture that the polls reflect the demographics of the voters.