February 10, 2013 | 12:28 pm

The Rosner-Fuchs Exchange, Part 1: How Yair Lapid Took Us By Surprise

Posted by Shmuel Rosner

Professor Camil Fuchs

Professor Camil Fuchs, who has been a valuable contributor to Rosner's Domain for the past year, is a veteran Israeli statistician. He is a Professor of Statistics at Tel Aviv University, where he has also served as head of the Department of Statistics and Operations Research, and as the chairman of the School of Mathematical Sciences. Professor Fuchs, who is the official  pollster of Haaretz daily and channel 10 news, has been one of the leading and most reliable polling experts in Israel for many years.

In the first part of this exchange about the results of the recent Israeli elections, Professor Fuchs tries to explain how and why Yair Lapid took virtually all the Israeli pollsters by surprise. 

Dear Camil,

It’s been two weeks or so since the elections, and I guess it’s time for some tough questions. But let me start by reminding our readers that: a) You are one of Israel’s leading pollsters b) For the past year you’ve been making Rosner’s Domain better by running very interesting data-driven features as part of what we call the J-Meter c) One of these features- the Israel Poll Trend- projected a 66-67 right-religious political bloc, and 33-34 mandates for Likud-Beiteinu in its final estimation (4 days before the elections) d) In the last polls you carried out prior to Election Day (such as this one) you estimated that Yesh Atid would get 11 mandates and that the Labor Party would be the second largest party.

Obviously, pollsters weren’t quite accurate in their final polls and, unsurprisingly, a barrage of poll-criticisms was fast coming. In an article about the “fiasco”, University of Haifa's Professor Gabriel Weimann rejected “arguments, like the one made by pollster Rafi Smith on the day after the election, which claim that the big change occurred right before the election, after polling was no longer permitted". "A professional pollster should have identified this change, if only as a trend, long before," Weimann believes, "The pollsters are just trying to cover up their mistake. They were wrong, big time."

So, two questions must be asked:

1. Were you wrong “big time”?

2a. If you were – why?

2b. If you weren’t- why do people think you were?

Since I want this to be a dialogue in which we discuss the Israeli election results with some depth, I’m going to leave you with these two simple questions – and ask the other questions I have for you in the next round.

Best,

Shmuel

Dear Shmuel,

I've been expecting these tough questions, but I have to say that I expected them to come from critics, not from friends…

But seriously, I’m glad you asked. The short answer is that both Professor Weimann’s claim- that "a professional pollster should have identified this change, if only as a trend, long before"- and Rafi Smith’s claim- that if only he had been permitted to take another poll he would have detected the change- are, in my opinion, wrong. Yes- in my opinion, they are both wrong. I don’t know about “big time”, but they are wrong nevertheless. Let me explain:

Lapid’s trend could not have been identified long before, (as Weimann claims) simply because it wasn’t there. In the last poll, taken on the Friday before Tuesday’s elections, 24% were still undecided. Furthermore, even after subtracting those who mentioned a favorite party when pressed and asked “nevertheless, to which party do you lean most?”,15% were still undecided.  And they remained undecided until the very last moment, when they heard the loudspeakers say “we're closing, make up your mind!”

And they did make up their minds- they voted Lapid. We don’t know how many voters were like that, but there were a lot of them. They were genuinely undecided and at the very last moment Lapid was a “let’s give him a chance”-alternative. I don’t believe that an earlier poll could have detected this trend prior to the actual elections (as Rafi Smith suggested). If the elections had taken place, say, two weeks later, the undecided would still have remained undecided to the very last moment and the trend would have probably still remained virtually undetected.

Let me refer you to Roger Cohen’s article from the NY Times from February 2, 2013 (ten days after the Israeli elections): He illustrates Lapid’s trend through the story of a middle class couple and mentions that “Anda (the wife) said that two hours before voting she was undecided, but concluded that Lapid might do something because he understood 'how much better off we might be' if entitlements for the ultra-Orthodox and investment in West Bank settlements were not draining the country.”

Of course, a statistician needs more than a single observation in order to decide whether to accept or to reject a statistical hypothesis. But as an illustration, this observation will do. Now, of course we can analyze and explain a-posteriori why the cluster of votes for Lapid “makes sense”. There are several interesting aspects about this issue. But as I said, this is all a-posteriori, and it's known that it is easier to explain than to predict.

To focus on your question, though- no, we weren’t wrong 'big-time'. Regarding those who think that we were: I believe that a) They choose to ignore the level of accuracy of our other predictions (after all, 34 parties were on the ballot); and b) I guess it makes them feel good when they feel that they have a reason to criticize.

By the way, I must include you among those who took their criticism one step too far: You mention that our final Israel Poll Trend post projected “a 66-67 right-religious political bloc, and 33-34 mandates for Likud-Beiteinu”. This is true, but your statement leaves out (deliberately?) some important data: First, as you know, the trend was based on all the polls published in Israel that week. Secondly and closely related- in our final comments prior to Election Day, we specifically mentioned in the Poll Trend that:  

Two of those polls were conducted by the same polling company and published in the same day (one for a major newspaper and one for a radio station). In both of those polls, the level of support recorded for the Prime Minister’s party was of 37 seats and the level of support for his Likud-Right-Religious bloc was equivalent to 71 and 72 seats, respectively. Among the other 12 polls, the highest level of support recorded for the Prime Minister’s party was of 35 seats, and the highest level of support recorded for his Likud-Right-Religious bloc was of 65 seats. So, are those two polls - the polls recording high level support for the  Prime Minister’s party and for his bloc, a statistical outlier?  If they are, and the circumstances might make one suspicious about them, then the gap is narrower than the one recorded here.

As you probably realized even then, I was indeed suspicious about those two polls.

On a final note- While the last point on the poll trend was based on 14 polls, let me just mention that in my last poll (published in Ha’aretz, and included in the index), I projected 32 seats for Likud-Beitenu and a 63-57 spread between the blocs. I think that's not so bad, given that the Likud-Beitenu eventually got 31 mandates and there was a 61-59 spread between the blocs.

Best regards

Camil.

0 Comments — Leave your comment