Statistical Corrections
To reduce bias and improve precision, the figures
Table 6 Analysis of Survey Item #9
presented in the results section have been corrected
“How did your exposure to the first statistics course in
for two important phenomena. The first is that not
college affect your interest in statistics?”
all past examinees who were eligible for inclusion
Standard
in the sample were equally likely to be selected.
Estimate of
95% Confidence Limits
Response Value Error of
Percent
For example, because the author was interested in
Percent
Lower Upper
controlling for the differences in survey responses
(a)(b)(c)(d)(e)
across years, a proportionally larger number of past
Greatly Decreased 7.8% 1.59 4.6% 11.0% examinees from the early years of the relevant peri-
od were drawn (see the appendix to the full report
Somewhat Decreased 5.7% 1.34 3.1% 8.4%
at
http://professionals.collegeboard.com/data-reports-
No Effect 51.5% 3.51 44.6% 58.5% research/cb/ap-statistics-education-choices.) The expec-
tation was that it would be more difficult to reach
Somewhat Increased 25.4% 2.54 20.3% 30.5%
these individuals because the addresses on record
Greatly Increased 9.5% 3.05 3.4% 15.6%
were from the time of the AP examination, which
Source: The College Board
could be as many as eight years old. For that reason,
Note: Estimates have been corrected for (1) sample selection probability
the response data were corrected for selection prob-
and (2) unit nonresponse probability.
ability or the chance that each individual from the
nearly 230,000 AP examinee population would be
chosen as one of the 2,143 sampled individuals.
The second important phenomenon for which
the figures to follow were corrected was survey (or
Table 7 Analysis of Survey Item #14
unit) nonresponse. Some of the sampled individu-
“What is or was your major in college?”
als were more likely to respond to the survey than
others. Indeed, the appendix shows that response
Standard
Estimate of
95% Confidence Limits
Response Value Error of
rates were substantially higher for females, generally
Percent
Percent
Lower Upper appear to be higher for individuals receiving higher
(a)(b)(c)(d)(e)
AP Statistics exam grades, and may differ slightly by
U.S. region. The College Board has demographic
Statistics 1.3% 0.70.0% 2.7%
and exam performance data for all examinees, and
Mathematics with a
0.4% 0.10.1% 0.7%
therefore can use these data to estimate the chance
concentration in Statistics
that each sampled individual would respond to the
Mathematics 5.6% 1.13.5% 7.7% survey. Consider the individuals who were very like-
Engineering or
ly to respond when they received the survey (e.g.,
10.0% 1.17.9% 12.1%
Computer Science
high-scoring females in later testing years); they may
Biology, Chemistry,
be fundamentally different in terms of their survey
Health Sciences, Physics, 15.6% 2.011.6% 19.6%
responses than those who are inherently unlikely to
or other Natural Science
respond (e.g., low-scoring males in earlier testing
years). The unadjusted approach that ignores this
Economics or Business 17.5% 1.714.2% 20.9%
fact would result in figures that over-represent the
Education 4.9% 0.63.7% 6.0%
subpopulation that was likely to respond, and hence
potentially would introduce a bias in the final fig-
Political Science,
ures. It is for this reason that the figures were cor-
Psychology, Sociology, or 23.0% 2.118.9% 27.1%
other Social Science
rected for survey nonresponse.
Ignoring these differences in sample selection
Agriculture 0.2% 0.20.0% 0.6%
and response probability would likely result in
Not Yet Decided 0.7% 0.50.0% 1.7%
biased results, and when the unadjusted analy-
ses were run, they typically differed substantially
Other 20.8% 1.418.0% 23.6%
from what is presented. For example, assuming
Source: The College Board that applying these two corrections results in
Note: Estimates have been corrected for (1) sample selection probability approximately unbiased estimates, the unadjusted
and (2) unit nonresponse probability. approach overestimates the proportion of students
who indicated a greatly or somewhat increased
interest in statistics as a response to exposure to
the AP Statistics course and underestimates the
proportion of students whose interest greatly or
10 AMSTAT NEWS MAY 2009
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76 |
Page 77 |
Page 78 |
Page 79 |
Page 80 |
Page 81 |
Page 82 |
Page 83 |
Page 84