perspective, the cost has decreased because of the not subject to change. Among these
substitution of the less-expensive treatment. are that the population estimates will
Therefore, one has to consider whether the treat- reflect the census concept of usual resi-
ments available for specific diseases have changed dence, the most recent census counts
recently, and whether the total number of people will serve as the estimates base, and
being treated has changed (i.e., with lower cost priority is given to a lack of bias because
options, more people might seek treatment). BEA of the use in funds distribution. The
is trying to identify diseases where spending by estimates must be produced within dead-
treatment vs. spending by disease is important. The lines determined by U.S. Census Bureau staff,
work is still preliminary and further complicated by estimates within a vintage must sum to others
the tendency for some diseases to occur in groups. of that vintage, and each vintage must include a
Still, some of the early data indicate treatment-based time series from the last census.
measures suggest higher increases in health care costs Methodological principles call for soundness
than do the disease-based measures. Findings so (solid reasoning), accountability (understandable
far—based on an extensive list of diseases—suggest by many parties), availability of data (for all areas of
treatment of disease–based measures usually show the United States), availability of resources, robust-
lower growth in health care costs. ness (insensitive to small departures from assump-
tions), comparability, adaptability, parsimony, and
Population Estimates Testing and
reasonableness (in terms of accuracy, demograph-
Evaluation Research ic appropriateness, and external comparisons).
Victoria Velkoff of the U.S. Census Bureau described
Accuracy is defined as the degree of closeness to the
the bureau’s plans for evaluating their population
2010 Census values. Devine listed four properties
estimates against the results of the 2010 Census (a
of “good” estimates, drawn from a 1970s review of
program they call Estimates Evaluation E2). The
the census estimates program by the Committee
objective is to evaluate not only the methods they
on National Statistics. These include low aver-
currently use, but a number of alternative methods.
age numeric error, low average percent error, few
Currently, the U.S. Census Bureau uses the
extreme percent errors, and the absence of bias for
administrative records method (ADREC) for coun-
subgroups. Devine also identified the following five
ty estimates and the housing unit method (HU) for
selected measures of accuracy, chosen after a review
subcounty estimates. Velkoff recalled the Housing
of 19 measures:
Unit Based Estimates Research Team (HUBERT)
project undertaken in response to recommendations
1. Average numeric error (root mean
that they consider HU methods for the county esti-
square error)
mates. The HUBERT results found that the ADREC
2. Average percent error (MAPE)
method has produced more accurate estimates than
3. Extreme percent error (N greater than an
HU methods in most counties. However, some HU
established threshold)
method proponents are concerned that ADREC
showed no advantage with respect to bias, so the
4. Bias (MALPE)
bureau is taking a further look at HU methods. The 5. Accuracy of share of total population
objective is to document the ultimate decision fol- (absolute error of shares)
lowing the 2010 Census.
Phase one of the evaluation is to develop estima- Table 1 shows measures for 2000 county popula-
tion principles and select accuracy measures. Phase tion estimates for ADREC and HU methods.
two is to select alternative methods to test, develop
official and alternative estimates, and carry out the
evaluation. The timeline calls for the completion of
Table 1—Measures for 2000 County Population
estimates for ADReC and HU Methods
phase one in April 2009 (already done), the determi-
nation of alternative methods by July 2009, the pro-
duction of evaluation estimates by December 2010,
ADREC HU
the completion of evaluations by October 2011,
Root mean square 10,341 13,464
and decisions on post-2010 methodology by spring mAPe 3.2 5.9
2012. Evaluations will cover total population and
extreme percent error (10%) 109 502
demographic characteristics, and the U.S. Census
Bureau will look to external researchers to help eval-
MALPe 1.5 1.4
uate specific methods, such as ratio correlation.
Accuracy of shares 6,574,332 12,173,807
Jason Devine of the U.S. Census Bureau described
the underlying principles of the estimates—items
AUGUST 2009 AmstAt News 17
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76