In case you’re wondering how the public is reacting to President Obama and John Kerry’s nuclear deal – and why I am so skeptical of political polling – please read the following data, drawn from a just released Washington Post/ABC News poll:
Let’s start with the Washington Post’s write-up. Can you see what is wrong about it? I’ll give you a few seconds.
BZZZZZZZZZZZZZZ. Ok, time’s up.
The write-up says that, by nearly 2 -1 the public supports a deal that would “prevent the production of nuclear weapons”. But that is NOT what the question asks. The question asks if they support a deal that “makes it harder” for Iran to do so; i.e. Iran can produce them, it just has to work harder to do so. Unbelievable mistake.
Now let’s move on from the misbegotten analysis, and look at the data.
The first question, by itself, would probably warm the heart of President Obama and Secretary Kerry. After all, about 6 in ten respondents support their deal, right? But why were respondents given only a choice of “makes it harder” to produce nuclear weapons, rather than, say, “makes it extremely difficult” or “prevents” Iran from doing so?
What does “makes it harder” even mean? From very easy to easy? From very hard to nearly impossible? Something in between? Without context, it is a meaningless question. But that’s what was asked.
Then, in the second question, respondents indicate how confident they are that the Obama/Kerry deal would prevent Iran from developing nuclear weapons. PREVENT. Not “makes it harder”.
So these two questions, which are supposed to tie together and tell a story, do not even coincide with each other.
Between this and the analytical gaffe we previously discussed, I have to wonder who is doing research for the Washington Post/ABC News these days. As someone with many, many years in both quantitative and qualitative marketing research, including the writing of countless questionnaires, be assured that neither the analysis nor the questions you just read would ever have gotten past my desk.
But, plodding onward, let’s look at the second question’s results. A grand total of 4% is “very confident” the deal would prevent Iran from developing nuclear weapons. The other 96% has at least some level of doubt – with about 2/3 of them “not so” or “not at all” confident Iran would be deterred.
You’d think this renders the first question irrelevant, wouldn’t you? After all, what does it mean to support the goals of a deal if you do not believe the deal will result in those goals being met?
Yet the data show that 67% of people who are “not so” confident Iran would be prevented from producing nuclear weapons and 31% of the “not at all” confident people, support the deal anyway — which should send a chill up the back of anyone who believes in logic and common sense.
So there you have it. A poor analysis, badly written questions, and screwy responses.
If you ever wondered why I am skeptical about political polling, this should give you an answer.