A risk-limiting audit can be four confirmed the provisional
couched as a hypothesis test. outcomes without requiring a full
The “risk” is the chance that the hand count.
audit stops before a full hand Risk-limiting audits are typi-
count when the outcome is cally performed in stages. Each
wrong. To control that risk sta- stage involves drawing a probabil-
tistically, it makes sense to take ity sample of batches, comparing
the null hypothesis to be that a the preliminary results with hand
full hand count would contradict counts of the audit trail for those
the apparent outcome. A Type batches, and calculating a p-value
I error occurs if the audit stops for the hypothesis that the out-
short of a full hand count when come of the election is incor-
the outcome is wrong. The goal rect. The calculation involves the
is to efficiently control the Type reported votes by batch for the
I error rate—that is, to count as entire race, the observed discrep-
few ballots as possible while limit- ancies between the preliminary
ing the risk of stopping too early counts and the hand counts, the
when a full count would change sampling design (including sam-
the outcome. ple sizes, stratification, and the
Early statistical work on vote- protocol for advancing from one
tabulation audits includes SAFE stage to the next), and the desired
(statistically accurate, fair, and limit on the risk. If the discrepan-
efficient), which gave the num- cies in a large enough sample are
ber of audit batches to select by sufficiently small, there is strong
simple random sampling as a evidence that the outcome is cor-
function of the margin of victory, rect, so the audit can stop; other-
precinct sizes, and a parameter wise, the audit progresses to the
specifying the maximum plausi- next stage—to collect more evi-
ble level of error in any precinct. dence, perhaps eventually requir-
SAFE ensures that if the outcome ing a full hand count.
is wrong, there is a large chance This sample-and-test strategy
the audit will uncover at least requires a transparent, public,
one precinct with an error, even trustworthy method of gener-
if the error is concealed in as few ating random samples, public
precincts as possible. SAFE is not counting, published procedures,
a fully fleshed-out, risk-limiting and calculations and theorems
protocol; it provides no guidance that few elections officials or
for what to do when the audit others have the background to
finds discrepancies, which are understand. Perhaps the biggest
virtually inevitable even when the open problems for methodolo-
outcome is right. gists in post-election audits now
Philip B. Stark of the University concern logistical barriers to
of California at Berkeley and his adoption: developing simpler,
colleagues have developed several more efficient, and more usable
methods for risk-limiting audits. methods; developing efficient
In collaboration with county elec- ways to audit dozens of contests
tion officials, they conducted four simultaneously; and developing
risk-limiting audits in California data ‘plumbing,’ software tools
in 2008: Marin County (a small and turn-key solutions that make
Tally sheets for auditing Measure W, Yolo County,
measure requiring a supermajor- the methods accessible to elec-
California, November 2008
ity and a county-wide measure), tions officials and transparent to
Santa Cruz County (County the public. Another important
Supervisor, District 1), and Yolo task is to estimate the costs and
County (a bond measure). The specific resources needed to sup-
audits limited the risk—the Type port credible audits.
I error rate—to at most 25%. All
JUNE 2009 AmstAt News 39
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76 |
Page 77 |
Page 78 |
Page 79 |
Page 80 |
Page 81 |
Page 82 |
Page 83 |
Page 84 |
Page 85 |
Page 86 |
Page 87 |
Page 88 |
Page 89 |
Page 90 |
Page 91 |
Page 92 |
Page 93 |
Page 94 |
Page 95 |
Page 96 |
Page 97 |
Page 98 |
Page 99 |
Page 100