MASTER’S NOTEBOOK
Hole in the Poll
Glaring Opportunities for Statisticians
Mani Lakshminarayanan and Amarjot Kaur, ASA Committee on Applied Statisticians
A
mericans (at least those interested in politics) were engrossed for nonresponders, and margin of error. In addition, it would be of
with what people were saying about Democratic candidate interest to know how the polls were able to provide point estimates
Hillary Clinton after the New Hampshire presidential pri- for the Republican side that were remarkably close.
maries. Two months later, the media are still debating what hap- The time is right for the ASA and the statistical community to
pened and remain puzzled over some of the pre-election polls post- play a visible role in clarifying misconceptions and explaining sta-
New Hampshire (e.g., Super Tuesday primaries/caucuses). tistical intricacies. We must address the usefulness, potential biases,
What we find most interesting about the media is how little and value these pre-election polls provide to voters. We have an
attention they have paid to the scientific aspects and methodologies obligation to address the use of correct statistical methodology and
behind opinion/exit polls. Rather, they have spent their time on the resulting interpretation. Also, statisticians must take a closer
the emotional and spontaneous or expected reactions from voters. look at how pre-election polls are conducted and whether con-
David Brooks, in his op-ed piece that appeared in The New York clusions drawn from them are statistically sound. After doing so,
Times on January 18, wrote about how uncertain our emotional and we may agree with the pundits that voting is an uncertain, emo-
intuitive decisions are about who we prefer, and then how we try to tional, intuitive decision—a decision that, for a significant group of
rationalize our choices with post-hoc reasoning. He also referred to a people, is finalized at the last minute. However, if we find that we
statement made by Nobel Prize winner Daniel Kahneman: “People don’t agree, then we have the responsibility to provide alternative
often act without knowing why they do what they do.” statistical methodologies and join hands with social and political
Theories about why the opinion polls conducted by organiza- scientists to better interpret such data.
tions such as Zogby International, Rasmussen Reports, and Gallup
failed to predict the New Hampshire primary results have so far
systematically ignored the statistical aspects behind the survey
methodologies. Statistician Andrew Gelman discusses some of
null
The time is right for the ASA and the
these aspects on his blog at www.stat.columbia.edu/~gelman/blog. statistical community to play a visible role
He points to the likely voter screen and its potential deficiencies
and the problems in survey weighting, especially when the Iowa
in clarifying misconceptions and explaining
caucus turnout was unusual.
statistical intricacies.
Descriptions of survey methodologies used in telephone surveys
conducted by Rasmussen and Zogby are included on their respec-
null
tive web sites. One issue of interest is their use of a single, digitally
recorded voice to conduct the surveys during specific times. In Based on the information we informally gathered from the mem-
this case, one should ask how likely one would be to answer calls bers of the ASA’s Section on Survey Research Methods and other
from an unknown number, especially about political issues. There sources, there appear to be innumerable opportunities for master’s-
is significant loss of information here, due to nonresponse, about level colleagues to contribute and participate in this real-world prob-
whether the person being surveyed is really an “unlikely voter.” lem in survey methodology. Additionally, surfing the web sites of
In spite of the media’s attention to just one side of the issue, some of these polling organizations (e.g., www.rasmussenreports.com,
they are given copious amounts of technical/statistical information www.zogby.com, www.gallup.com) further reveals there exists career
that are not discussed or shared with the general public. Some of opportunities for master’s-level colleagues as research associates/
the high-level issues include sample size, types of questions asked, experts who can contribute to attitudinal and behavioral research
sampling scheme, sample selection and its probabilities, weighting using survey methods and analytical modeling. n
MARCH 2008 AMSTAT NEWS 35
Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36  |  Page 37  |  Page 38  |  Page 39  |  Page 40  |  Page 41  |  Page 42  |  Page 43  |  Page 44  |  Page 45  |  Page 46  |  Page 47  |  Page 48  |  Page 49  |  Page 50  |  Page 51  |  Page 52  |  Page 53  |  Page 54  |  Page 55  |  Page 56  |  Page 57  |  Page 58  |  Page 59  |  Page 60  |  Page 61  |  Page 62  |  Page 63  |  Page 64  |  Page 65  |  Page 66  |  Page 67  |  Page 68  |  Page 69  |  Page 70  |  Page 71  |  Page 72  |  Page 73  |  Page 74  |  Page 75  |  Page 76  |  Page 77  |  Page 78  |  Page 79  |  Page 80  |  Page 81  |  Page 82  |  Page 83  |  Page 84