Further Reading
“ Computing Weights for American National Election Study Survey Data”
http://www.electionstudies.org/resources/ papers/nes012427.pdf
Weights Are Not Magic Weights are a wonderful tool, but they are not magic. They fix minor biases or random peculiarities in survey numbers, but they cannot make surveys representative of population groups that are missing altogether. (In technical parlance, they can’t fix noncoverage or extreme nonresponse bias.) For instance, if a survey is conducted only in English, and the sample ends up with too few immigrants, weights can- not make the survey properly representative of immigrants who do not speak English.
ster can randomly draw respondents. Instead, real surveys differ from the ideal in three ways: 1) They start by se- lecting households, often by calling phone numbers at random; 2) Within households, they select one respon- dent; and 3) More often than not, no one in the selected household will take part in the survey, and the pollster has to try the next one.
Weights at Work: A Simple Illustration Weights account for these peculiarities of sampling by de- termining how much each respondent counts—that is, the number of people (or the portion of the whole, however tiny) each person represents. Suppose women are more co- operative on a given survey than men, so when it is fin- ished, there are 600 responses from women and 400 from men. If we know the population is 52 percent female and 48 percent male, it’s clear that this 60-40 split isn’t a rep- resentative sample. Weights fix this by counting the male responses a little more heavily than the female ones. In this example, the women would each be assigned a weight of 52 ÷ 60 = .867, and the men would each get a weight of 48 ÷ 40 = 1.20. Once weighted, the sample comes out to 52 percent female and 48 percent male, matching the general population.
How to Weight: From Design Effects to Poststratification Creating weights is moderately technical, but fortunately it’s not rocket science. Here are the top ten things that any consumer or producer of polling data should know about how weights are properly constructed. (These are the basic concepts. For detailed step-by-step technical instructions on weighting, see “Computing Weights for American National Election Study Survey Data,” a report available online. See above for Web address.)
Weights determine how much each person in a sample counts. They are required because real-world surveys differ from the simplified ideal taught in Statistics 101.
Weighting is not optional. The only kind of survey worth
paying for is a scientific survey, and with few exceptions real-world scientific surveys require weights to accurately describe the population. There’s no such thing as a free lunch. Weights improve sur-
vey accuracy, but this accuracy typically comes at a cost of increasing the variance of survey estimates. In simple terms, weighting increases the survey’s margin of error.
June 2011 | Campaigns & Elections 29
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62