Monday, January 24, 2011

Hard Numbers

In case you think that presidential polls at this stage mean anything, Le Monde offers a sobering reminder of just how wrong they were 18 months before the last three presidential elections. The problems are numerous: too many candidates, aleatory press coverage, and, perhaps most interesting, misleading comparisons demonstrating Arrow's impossibility theorem at work. All right, so the application of Arrow's theorem isn't rigorous: different polls involve different groups, different questions, and different combinations of candidates. But the principle remains: person A may prefer Valls to Royal to Besancenot in poll X and wind up voting for Sarkozy over Montebourg in round 2 of 2012. Voters are hard pressed to sort out their preferences when there are too many candidates to choose from, and there is no consistent way of aggregating the preferences over the broad field to arrive at a consistent group choice in a binary confrontation. For the same reason, we tend to overinterpret "presidential mandates." The ultimate binary choice required in a presidential system of the French type yields the illusion that "the nation" has made a consistent choice between two competing sets of preferences. But underlying that choice is a chaotic sea of individual preferences that cannot be consistently combined.


bernard said...

my personal view on this would be that, simply, most people can't be bothered with an election that early. Those who actually give an opinion in these pollls are the politics-obsessed (like myself). And the point of an election cmpaign is not to convince the convinced, but to convince the unconvinced. This view has served me well in the past in predicting election due time.

Anonymous said...

Many people, most people can't be bothered at this point.
In fact, the directors of IFOP and BVA both stated that DSK's high "popularity" came from those who don't know/don't care about politics: they've heard the name but they don't associate him with any political party or policy (including his own time as a ministre for Jospin.) They fear that if he comes back and he's associated with the PS, his numbers will drop sharply (and then it'll be too late for him).
Hollande, they stated, benefitted from a nice "echo room" from the press, i.e, they like what he says, they like him, so he seems to have broader appeal than he really does.
Royal remains very strong with the 18-25 group as well as with the "classes populaires".
Regarding that poll from 2006: surely the socialists didn't use that to choose her, there must have been something else: 49/51 with a 4 points margin of error is far from enough to grant someone 60% votes, surely?!

Passerby said...

I agree with the comments that the election is too far away for people to be concerned.
Of course year-long militants will already have their preferences for certain party leaders. But for most people it's just too early.

My feeling (based on zero statistical analysis), is that for non-militants (i.e the majority of voters) up to the election day impressions on any candidate can change quickly.
Of course it's unlikely that a self-defined "de droite" will vote for the PS candidate, but sympathy for "les Verts" or FN, may swing the vote during the first round.

I can't think of any candidate in recent elections that had enough legitimacy or charisma to create some kind of French Obamania. With pales candidates, it's hard to get excited.

Until your choice is narrowed to 2 candidates. That's when people are force to ask themselves who they want (or don't want) to be the next president. No matter what complex political sympathies one may have, it becomes choosing a person.
That's why the last election was so polarizing. If you can't stand Royal, you'll vote for Sarkozy no matter his flaws. Just like on the other side the motto was "tout sauf Sarko".

My 2 cents.

gregory brown said...

First of all an editorial comment -- you noted recently your 500Kth page view and I suspect it is much higher since I am likely far from the only one to read primarily the RSS feed for the past year at least. And secondly, thank you for such erudition as Arrow's Theorem of which I had never heard but should have.

Now as to the substance, I would add a point that has baffled me since I started following French presidential elections closely in 1988 -- that the use of a so-called "representative sample" method is universal in France among academic as well as commercial polling (and, I presume, the parties). Whereas in the US, "random sample" method is almost universal (and more common in the UK, where representative sample is used as well).

While random sample polling can be very poor as a predictor of voter behavior if the sample size is too small (my local paper commits this error regularly by sampling fewer than 300 voters for congressional or statewide races) or if the sample is not truly random (ie excludes cell phone users or Spanish-language speakers or those who do not respond to the phone the first time called -- all of which led to very bad polling in the NV Senate race this past fall), it is still the case that most social scientists with whom I've spoken believe that a properly conceived and executed random sample survey is much more predictive than a so-called "representative sample" survey, precisely because it is so hard to know, a priori, what demographic sample is "representative" of the overall electorate.

And to complicate matters, French surveys tend to throw out "no opinion" responses or to assimilate them into a specific response if the survey respondent, when prompted, chooses one of the available candidates. This cuts down on the number of interviews that need to be conducted, but badly weakens the methodological rigor of the survey.

I've been told by journalists in France that it would be too expensive to conduct a "random sample" survey properly for a news agency. They therefore opt for what is offered by the Instituts de sondage -- "representative sample" surveys.

The point then, in response to Art's post, seems to me that a big issue is methodological -- not that voters change their mind between the survey and the election but that their actual intentions are poorly recorded in the surveys that are conducted.

Unknown said...

Thanks for the comment. Information on French polling methods can be found here:

gregory brown said...

Art thanks for that link. To me the most interesting (and contestable) passage is the following. The interesting part is the statement that IPSOS uses "representative" sampling because it cannot afford to redial each randomly selected respondent in a "random sample" pool. This, however, is precisely why so many "random sample" polls were off in the American elections last fall.

The contestable part is the contention that while "representative sample" method does not allow calculation of margin of error, the margin is "equal or inferior to" the MOE of a random sample poll. There is simply no evidence to support that claim; indeed, from a statistical point of view it is nonsensical to make inferences about margin of error when the margin of error cannot be calculated.

I'm still in the dark as to how French surveys handle undecided respondents.

Par rapport à la méthode aléatoire, celle des quotas a l’avantage d’être plus rapide. Avec l’aléatoire, les sondés ne sont pas ' interchangeables '. Cela signifie que la personne tirée au sort doit être recontactée autant de fois que nécessaire. Grâce aux quotas, il est possible de remplacer un sondé par un autre qui a les mêmes caractéristiques socio-démographiques. Cela permet de réaliser un sondage dans des délais plus courts.

L’inconvénient majeur de la méthode des quotas est de ne pas permettre de calculer scientifiquement la marge d’erreur du sondage. Les lois statistiques qui permettent de la déterminer ne valent théoriquement que pour les sondages aléatoires. En pratique, on considère cependant que la marge d’erreur des sondages par quotas est égale ou inférieure à celle des sondages aléatoires.

Unknown said...

What you say is true, but I have a friend in the polling business who tells me that even random sample American polls are not truly random. Each polling house knows from experience that "random" samples are not truly random because certain groups are systematically underrepresented for one reason or another: no phone, cell phone, reluctance to respond to questions, antipollster bias, etc. So they devise methods for "correcting" the results of random calling based on accumulated knowledge of past errors. This is one reason why polls exhibit "house effects," which, as you know if you follow Nate Silver's excellent blog on polling, can be considerable. True random sampling is not attainable in this imperfect world, so professional pollsters compensate. Social scientists, who have less experience and "fingerspitzengefühl" and a robust faith in theory, may not be the most reliable source of information about this phenomenon.

FrédéricLN said...

On the technical polling issues: you are both true. The Ipsos claim is "une horreur" in the eyes of any statistician. (Yes, I'm one).

But the real issue with polls 15 months before the election (hey Art, it's 15, not 18), is that the people just haven't started to think about it, as we have no… primary elections - or, only the PS will have some.

Many people may answer very seriously to the question, but their answer will just mirror the present state of media coverage. (Marine Le Pen after the FN congress, Eva Joly when she made her announcement, and so on).

The Le Monde paper is right, but why only 3 elections ? The issue has been the same at all presidential elections in France, including 1848, when there were no polls, but there has been an outcome nobody would have expected months before (namely the election of Napoleon's nephew).