Skip navigation

Arrest-Related Deaths Program Assessment: Technical Report BJS 2015

Download original document:
Brief thumbnail
This text is machine-read, and may contain errors. Check the original document to verify accuracy.
Arrest-Related Deaths
Program Assessment
Technical Report
Duren Banks
Lance Couzens
Caroline Blanton
Devon Cribb
RTI International

NCJ 248543
March 2015

Contents
Executive summary .....................................................................................................................1
1 Background ..............................................................................................................................3
2 Methodology ............................................................................................................................4
2.1 Probabilistic case-level linking ......................................................................................4
2.1.1 Calculation of similarity scores .........................................................................5
2.1.2 Thresholds for matching ....................................................................................6
2.1.3 Linking algorithm ..............................................................................................6
2.1.4 Monte carlo replication ......................................................................................8
2.2 Capture-recapture analytic approach .............................................................................8
2.2.1 Simple poststratified Lincoln-Petersen approach ..............................................9
2.2.1.1 Assumptions........................................................................................9
2.2.1.2 Strengths and limitations...................................................................10
2.2.2 Poststratified Lincoln-Petersen approach with constant ratio
adjustment for unobserved agencies .............................................................10
2.2.2.1 Assumptions......................................................................................11
2.2.2.2 Strengths and limitations...................................................................11
2.2.3 Data assumptions and coverage estimate implications ....................................12
3 Findings ..................................................................................................................................13
3.1 Arrest-related death coverage ......................................................................................13
3.2 Variability in ARD methodology across states ............................................................23
3.2.1 ARD law enforcement homicide reporting methodology................................27
3.2.2 Location of SRC ..............................................................................................29
3.3 Number of years at least one death was reported to ARD and SHR ...........................31
4 Conclusions and recommendations ........................................................................................32
4.1 Relative strengths and weaknesses of the ARD program methodology ......................32
4.1.1 Types of cases that were covered well by the ARD program ..........................32
4.1.2 Identification approaches that are associated with better coverage .................32
4.2 Recommendations ........................................................................................................33
4.3 Study limitations ..........................................................................................................34
References .................................................................................................................................37
Appendix A: Similarity scoring for ARD/SHR matching.........................................................38
Introduction ........................................................................................................................38
iii

Continuous variables ..........................................................................................................38
Categorical variables ..........................................................................................................39
Calculating the overall score ..............................................................................................40
Appendix B: Linking algorithm ................................................................................................41
Estimate the agency population size and agency undercoverage adjustment factor .........41
Classify cases as matchable or unmatchable .....................................................................41
Probabilistic linking between ARD and SHR ....................................................................42
Recombine cases ................................................................................................................43
Stratify by individual-level characteristics ........................................................................43
Estimate population size and variance by stratum .............................................................43
Estimate population size ....................................................................................................44
Adjust population size for agency undercoverage .............................................................44
Estimate ARD coverage .....................................................................................................44
Replication .........................................................................................................................44
Appendix C: Race/Hispanic origin coding ................................................................................46
Hispanic origin in the SHR ................................................................................................46

List of tables
Table 1: ARD and SHR law enforcement homicides, by match outcome, 2003–09
and 2011 ................................................................................................................ 16
Table 2: ARD law enforcement homicides, by match outcome, 2003–09 and 2011 .............. 16
Table 3: ARD and SHR law enforcement homicides, by matching variable and match
outcome, 2003–09 and 2011 ................................................................................. 19
Table 4: ARD law enforcement homicides, by matching variable and match outcome,
2003–09 and 2011 ................................................................................................. 20
Table 5: ARD law enforcement homicide decedent characteristics, by source, 2003–
09 and 2011 ........................................................................................................... 22
Table 6: ARD coverage estimates and confidence intervals, by estimation approach,
2003–09 and 2011 ................................................................................................. 23
Table 7: ARD law enforcement homicides, by region, state, and source, 2011 ..................... 25
Table 7: Primary methodology for identifying arrest-related deaths, by state, 2011 .............. 27
Table 8: SRC agency location, 2011 ....................................................................................... 29
Table 9: Number of years at least one death was reported to ARD and SHR......................... 31

iv

List of figures
Figure 1: ARD and SHR coverage and overlap of the universe of law enforcement
homicides in the United States, with no agency adjustment, 2003–09 and
2011....................................................................................................................... 14
Figure 2: Estimated proportion of law enforcement homicide universe covered by
each source, with no agency adjustment, 2003–09 and 2011 ............................... 15
Figure 3: Percent of observed ARD law enforcement homicides, by match score,
2003–09 and 2011 ................................................................................................. 17
Figure 4: Percent of law enforcement homicide universe covered by ARD, by
estimation method, 2003–09 and 2011 ................................................................. 24
Figure 5: Law enforcement homicides, by source and primary reporting method,
2011....................................................................................................................... 28
Figure 6: Law enforcement homicides, by source and SRC location, 2011 ........................... 30
Figure 7: ACS 5-year Hispanic origin, by county ................................................................... 48

v

Executive summary
The Bureau of Justice Statistics (BJS) designed the Arrest-Related Deaths (ARD) program to be
a census of all deaths that occur during the process of arrest in the United States. The manner in
which these data were collected varied from state to state, and often depended on the data
systems available to the state reporting coordinators (SRCs) responsible for data collection
throughout the state, the involvement of local law enforcement agencies or medical
examiner’s/coroner’s offices, and other support that the SRC may have had to conduct the data
collection. This variability in approach has led to questions about whether these data collection
methods were capable of capturing the universe of arrest-related deaths and law enforcement
homicides in particular. BJS requested RTI International to conduct an assessment of the ARD
program to evaluate (1) the coverage of the program in comparison to Supplementary Homicide
Reports (SHRs) maintained by the Federal Bureau of Investigation (FBI) and (2) various aspects
of the current program methodology. The coverage assessment matched law enforcement
homicides captured by the ARD program to those found in the SHR justifiable homicide file,
followed by a capture-recapture analysis to provide information on the scope and characteristics
of cases eligible for inclusion in the ARD program that are captured in one or both of these data
systems. The ARD law enforcement homicides and SHR justifiable homicide files are similar;
however, some law enforcement homicides that were not classified as justifiable are not
identified in the SHR.
RTI calculated the size of the law enforcement homicide population in the United States and the
ARD program coverage using two methods to estimate the lower and upper bounds of ARD
coverage. We found that over the study period from 2003 through 2009 and 2011, the ARD
program captured, at best, 49% of all law enforcement homicides in the United States. The lower
bound of ARD program coverage was estimated to be 36%. These findings indicate that the
current ARD program methodology does not allow a census of all law enforcement homicides in
the United States.
The ARD program captured approximately 49% of law enforcement homicides, while the SHR
captured 46%. An estimated 28% of the law enforcement homicides in the United States are not
captured by either system. However, the methodology for identifying ARD cases has changed
over the observation period. In 2011, the ARD program was estimated to cover between 59%
and 69% of all law enforcement homicides in the United States, depending on the estimation
method used. While this coverage estimate still does not result in a census, it does suggest
improvements over time in the overall approach to identifying law enforcement homicides and
reporting them to the ARD program.
We found considerable variability between states in the proportion of law enforcement
homicides that are reported to the ARD program only, the SHR only, or to both sources in 2011.
Twelve states reported only to the ARD program in 2011, while no states reported cases only to
the SHR. Additional analyses to explore the effect of case identification methodology and SRC
affiliation failed to identify a specific ARD methodology that was associated with better program
coverage in 2011.

Page 1

The current analyses only compared ARD program coverage to the SHR. Other sources may also
provide additional information about the extent of law enforcement homicides in the United
States or in selected jurisdictions, and coverage of the various data collection systems. These
sources include the Fatal Injury Reports that are part of the National Vital Statistical System
maintained by the Centers for Disease Control and Prevention and direct reports from local law
enforcement agencies. In addition, the ARD program assessment examined only arrested-related
deaths that are the result of law enforcement homicides. Arrest-related deaths due to illness,
overdose, accidents, and other manners of death are likely even more difficult to identify and, if
included, could have a significant downward impact on our coverage estimates. However, no
other national data collection exists that examines arrest-related deaths due to a manner other
than law enforcement homicide. If BJS pursues a collection to measure law enforcement
homicides or all manners of arrest-related deaths in the United States, changes must be made to
the data collection methodology to support more complete coverage.

Page 2

1 Background
By definition, the Arrest-Related Deaths (ARD) program was to be a national census of law
enforcement homicides that occurred during the process of arrest or during an attempt to obtain
custody by a state or local law enforcement agency in the United States. The program was
implemented by the Bureau of Justice Statistics (BJS) in 2003 as part of its Deaths in Custody
Reporting Program (DCRP) in response to the Deaths in Custody Reporting Act of 2000 (DICRA).
In addition to the ARD program, the DCRP includes collections that measure deaths occurring in
jails and state prisons.
The ARD program methodology relied on state reporting coordinators (SRCs) in each of the 50
states and the District of Columbia. SRCs were responsible for understanding the scope and
definition of the ARD program, identifying eligible cases in their state, and working within
available resources to collect and report information about those cases. The manner in which these
data were collected varied from state to state, and often depended on the data systems available to
the SRC, the involvement of local law enforcement agencies or medical examiner’s/coroner’s
offices, and other support that the SRC may have had to conduct the data collection. This variability
in approach led to questions about whether the existing data collection methods were capable of
capturing the universe of arrest-related deaths and law enforcement homicides in particular.
The Supplementary Homicide Reports (SHRs) represent another national-level data source that
provides information on law enforcement justifiable homicides in the United States. 1 The SHR is
part of the Uniform Crime Reporting (UCR) Program conducted by the Federal Bureau of
Investigation (FBI). While the UCR summary program provides aggregate annual counts of the
number of homicides occurring in the United States, the SHR data provide additional details about
each homicide incident, including the jurisdiction, month, year, victim and offender demographic
characteristics, weapon, the circumstances surrounding the incident (e.g., argument, robbery, gangrelated), and the relationship between the victim and offender, if known.
To determine whether the ARD program met the criteria for a valid and reliable statistical
collection, BJS tasked RTI to undertake a program assessment to better understand the utility of the
ARD data collection approach and to effectively assess alternative approaches. RTI conducted the
ARD program assessment to evaluate (1) the homicide coverage of the program in comparison to
the Supplementary Homicide Reports (SHRs) and (2) the current program methodology. The
coverage assessment matched ARD cases to the SHR. A capture-recapture analysis provided
information on the scope and characteristics of cases eligible for inclusion in the ARD program
that are captured in one or more of these data systems. The program methodology assessment
further compared ARD cases identified by various approaches currently employed by the state
reporting coordinators (SRCs) and the ARD staff, including media searches, access to state
reporting systems, and direct reporting from law enforcement agencies. The program assessment
process and findings are documented below.

1
The SHR homicide also includes a small number of law enforcement-related homicides that are not classified as
justifiable, but those cases cannot be identified.

Page 3

2 Methodology
The Arrest-Related Deaths (ARD) program relied on a census collection from agencies that
experienced one or more deaths to estimate the number of arrest-related deaths in a particular
calendar year. Because the census was (1) obtained through self-reporting by agencies, as
opposed to targeted collection of a probability sample derived from a robust agency frame, and
(2) included no agencies that did not experience a death, it is difficult to make inferences
regarding any undercoverage using information from the ARD program alone. In light of these
constraints, a coverage assessment that utilizes comparable data from an auxiliary source is
appropriate.
The U.S. Department of Justice, through the Federal Bureau of Investigation (FBI), releases
annual law enforcement-reported homicide data in the form of Supplementary Homicide Reports
(SHRs). Because the SHR data provide incident-level information on homicides, they can be
used to help assess the coverage of the ARD data collection. Whereas the ARD collection
documents all manners of death that occur during the arrest process or in the presence of law
enforcement, the SHR is limited to homicides and therefore does not include suicide, accidents,
illness or natural causes, drug intoxications, and other manners of death not considered to be
homicides. Therefore, to make comparisons between these data collections, each must be
restricted to a similar population of cases. For the analyses presented below, we restricted the
SHR to only the subset of homicides in which an offender is killed by the police (justifiable
homicides) and subset the ARD to those incidents defined as homicides committed by law
enforcement. Both of these subsets are limited to incidents involving state or local law
enforcement agencies. The SHR and ARD data sets include all law enforcement homicides
reported in the years that the ARD program produced annual datasets: 2003 through 2009 and
2011. Some states did not report any SHR data in some years. With the exception of Florida,
which did not report any SHR data for the entire period, no additional information was added to
the FBI’s SHR file for other states. Homicide cases for Florida were collected independently
from the FBI collection and added to the SHR file for this analysis.
By linking ARD decedent records to those captured in the SHR, it is possible to make inferences
about the total size of the law-enforcement homicide population and, thereby, about the
population coverage achieved in the ARD program. This analytic technique, known as capturerecapture, is typified by studies of wild animal populations, though it is widely used in the study
of human populations as well, particularly in coverage evaluation for censuses (Wolter, 1986).
Though the ARD coverage assessment is based on traditional capture-recapture methods, unique
characteristics of ARD data necessitate special techniques, including probabilistic record linkage
between data sources and adjustments for unobserved agencies.

2.1 Probabilistic case-level linking
Capture-recapture analysis depends on the ability to link records across data sources. In wildlife
applications, this requirement is easily met as captured animals are identified through tagging. In
the context of a coverage assessment utilizing an auxiliary data source not collected as part of the
coverage study, this requirement becomes more difficult. Because decedents are not uniquely
identified across data sources (by name or Social Security number, for example), record linkage
must be based on comparable decedent demographic and incident characteristics in an attempt to
Page 4

create a unique link. However, linkages made without unique identifiers are inherently uncertain,
decedents are matched between ARD and SHR sources probabilistically based on measured
similarity between records, within a monte carlo replication framework. This means that
although a given ARD decedent will be linked to either no SHR decedents or a single SHR
decedent in a particular replicate, the nature of the link for that ARD decedent (whether it is
linked and, if so, to which SHR decedent) is allowed to vary across replicates according to the
probabilities associated with particular record pairings, based on measured similarity between
cases. Probabilistic linking is random in nature and incorporates additional uncertainty into the
process of estimating population coverage; monte carlo replication allows for this additional
uncertainty to be captured in variance estimates.

2.1.1 Calculation of similarity scores
Records cannot be uniquely identified across sources so linking decedent records is done
probabilistically, based on the measured similarity between two records. The first step in
determining link probabilities for a given ARD decedent and the decedent’s potential matches in
the SHR is to calculate a similarity score. Similarity is based on the following variables that are
common to both ARD and SHR:


Originating Agency Identifier (ORI)



Sex



Year



Date reported/date of death



Age



Race/Hispanic origin

Decedent records are considered 0% similar if they do not match on at least ORI, sex, and year
and are not eligible for linking. For records that have common values for these three variables
across sources, scores are calculated based on the values of the date, age, and race/ Hispanic
origin variables. These measures include a mix of continuous and categorical variables;
therefore, the method for calculating the similarity score must account for both types.
The portion of the similarity score determined by the continuous variables (date and age) is
calculated as a function of the standard Euclidean distance for two variables, such that two
records having identical values across sources have a similarity of 1. The portion of the similarity
score determined by the continuous variables approaches 0 as dates and ages become more
dissimilar.
The portion of the similarity score determined by the race/Hispanic origin values is based on a
penalized scoring approach described in Boriah et al. (2008). This approach assigns a similarity
of 0 if the variables disagree across sources and a value of less than 1 when the variables agree.
The amount that matching records’ similarity score is discounted is determined by the
probability that the race/Hispanic origin value would be in agreement by random chance alone.
This approach is based on the premise that matches that have a high probability of occurring by
random chance should not count as much as those that would be observed very rarely by chance

Page 5

alone. The final similarity score is calculated as a weighted mean of the components based on
continuous variables (weight = 2/3) and race/Hispanic origin (weight = 1/3).
Once similarity scores are calculated for each potential pairing of a given ARD decedent and
SHR decedents from the same ORI, sex, and year, these values are used to create an additional
score. This score represents the combined dissimilarity between the ARD case and all of its
potential matches. Following calculation of this dissimilarity score, all scores are scaled such that
their sum is 1. These scaled scores are treated as relative probabilities and determine how cases
are linked. A detailed description of similarity score calculation, including all formulas, may be
found in appendix A.

2.1.2 Thresholds for matching
Potential record pairings fall into one of two categories: those that are within a predetermined
threshold of similarity and those that are not. If two records each have the same value of ORI,
sex, year, and race/Hispanic origin, there is a difference in the dates of report and death less than
or equal to 30 days, and there is a difference in recorded age of less than or equal to 5 years 2, the
pairing is thought to be within an acceptable threshold to consider the records as representing the
same decedent. In these instances, the scaled similarity scores (match probabilities) for all
pairings for the same ARD decedent that are not within threshold are set to 0. This includes the
scaled dissimilarity score, which represents the joint probability across potential pairings that no
SHR record represents the same decedent. This means that when an ARD decedent has one or
more pairings that are within threshold, the probability that the record will not be linked at all is
0. In some cases, a single ARD case may have more than one pairing that is within threshold. In
these instances, a link will take place according to probabilities resulting from rescaling the
similarity scores among only the within-threshold pairings such that they sum to 1. ARD
decedent records that have a single potential pairing that is within threshold undergo the same
rescaling process, resulting in a single potential pairing with a link probability of 1; these
pairings are thereby referred to as certainty pairings. For ARD decedent records with no potential
within-threshold pairings, linking occurs according to the probabilities associated with all
potential pairings and the calculated probability that no pairing is correct. Thus, these decedent
records may remain unlinked, and in the event that all potential pairings are very dissimilar, this
will be the most probable outcome.

2.1.3 Linking algorithm
The first step in the linking process is separating ARD and SHR decedent records into two
groups: those that have ORI, sex, and year values common across data sources and those that do
not. The latter group is considered unlinkable; those records with common values of ORI, sex,
and year are linked probabilistically according to the following steps (a more technical
description is provided in appendix B):

2

Age values in the SHR are considered imprecise and have been observed to clump around 5-year marks.
Additionally, the SHR date is the month and year the homicide was reported. These factors necessitate a certain
level of leeway in the criteria for linking and are what drive the linking threshold definitions.

Page 6

1. Randomly sort the ARD cases within ORI, sex, and year. The first step in linking ARD
decedent records to the SHR is a random sort. Because the ARD decedents are processed
sequentially and a linked SHR decedent is unavailable for linking to other ARD records
processed after the link is made, it is important to ensure that that the order in which the
ARD cases are processed is not fixed. This is necessary for monte carlo replication as
described in 2.1.4.
2. For each ARD case, calculate a similarity score corresponding to each SHR case with
the same ORI, sex, and year. In this step, similarity scores are calculated as described in
2.1.1, above. Note that for any ARD decedent record that is not the first to be processed in its
ORI, sex, and year classification, there may be SHR decedents ineligible for linking, as they
are linked to other ARD records processed previously. The similarity scores for such
ineligible pairs are set to 0.
3. Treating the scores from (2) above as probabilities, calculate the dissimilarity score (the
probability that the ARD case is not represented by any SHR decedent in the same
ORI, sex, and year). Given the small number of variables that overlap between the ARD and
SHR and the inability to perfectly match across sources, it is important to assign a nonzero
probability to the possibility that the ARD decedent is not represented by any of the SHR
decedent records in the same ORI, sex, and year.
4. Determine the number of cases that are within threshold. Each potential pairing for a
given ARD decedent is checked against the threshold criteria described in 2.1.2. If one or
more pairings are within threshold, all other scores (including the one described in (3),
above) are set to 0.
5. Scale scores. Though the scores described above may be individually interpreted as
probabilities, their sum may be greater than 1, complicating both their joint interpretation and
the process of linking. In this step, each of the scores is divided by their sum, maintaining
their relative proportions. Note that as a result of (2) and (4) above, several outcomes are
possible:
i.

No potential SHR decedent records have been previously linked to other ARD
records, and there are no within-threshold potential pairings—in this case there
will be a nonzero probability that no linkage will be made.

ii.

No potential SHR decedent records have been previously linked to other ARD
records and there is one potential within-threshold pairing—in this case the
within-threshold pairing is a certainty.

iii.

No potential SHR decedent records have been previously linked to other ARD
records and there are multiple potential within-threshold pairings—in this case, a
linkage will be made only according to the probabilities among within-threshold
pairings, as all other probabilities have been set to 0 prior to scaling.

iv.

One or more potential SHR decedent records have been previously linked to other
ARD records and there are one or more potential SHR records available for
linking —these cases are then treated identically to i. through iii., above.

v.

One or more potential SHR decedent records have been previously linked to other
ARD records and there are no potential SHR records remaining for linking—in

Page 7

this case, the probability that the ARD decedent record is not represented in the
SHR is 1 and the record is not linked.

2.1.4 Monte carlo replication
The inability to precisely match decedents between the ARD and the SHR necessitates a
probabilistic linking approach, but this technique is inherently random in nature, achieving the
most probable outcomes only in expectation. By repeating the linking algorithm many times, the
variability resulting from that randomness can be incorporated into final estimates.
The monte carlo framework leverages two points at which randomness enters the linking
process: the use of random numbers for sorting ARD decedent records and the use of random
numbers to select from potential pairings. This means that uncertainty in record linking resulting
from the lack of overlapping variables, and/or errors in variables that do overlap, will be
incorporated in final estimates.
Generation of random number streams depends on selecting ‘seed’ values—numbers that are fed
into the algorithm as a starting point for generating the stream. By setting random number seeds
in a predictable way (for repeating the results) that is a function of both the replicate number and
the ORI, sex, and year grouping, variation inherent to the probabilistic linking can be correctly
accounted for. Given the resource-intensive nature of probabilistic linking in general, a replicate
count of 1,000 was selected to balance resource usage and precision.

2.2 Capture-recapture analytic approach
Once decedent records have been linked across the ARD and SHR, capture-recapture methods
can be used to estimate the total population of law enforcement homicide and, thereby, the
proportion of that population covered by the ARD program. This analysis makes use of the
commonly applied Lincoln-Petersen (LP) estimator, with modifications for bias amelioration
necessitated by the characteristics of the ARD program.
Use of the basic LP estimator implies acceptance of several assumptions, including the
following:
a. The population of interest is closed geographically and in time and is of fixed size;
b. Correct matches may be made between sources;
c. Each data source is free of spurious cases, such as those that are out of scope or that
are duplicates;
d. The probability of inclusion is independent across lists;
e. The probability of inclusion is constant within lists; and
f. The probability of inclusion is constant across lists.
For the purposes of evaluating ARD coverage, some of the above assumptions may be accepted
as true, while others must be accepted with qualification and, in some cases, in conjunction with
specific analytic techniques designed to minimize the biases incurred as a result of their likely
violation. Given the goal of the analysis—to determine if ARD coverage is high enough to
Page 8

justify continuing the program in its current form—violation of some assumptions is not as
problematic as would be the case in other applications because the direction of incurred bias is
known. This allows for reasonable assertions of upper limits for ARD coverage, which provides
sufficient information given the circumstances.
In addition to the assumptions presented as a.–f. listed above, this analysis must account for an
additional, and seemingly unique, challenge: estimating the impact that unobserved law
enforcement agencies with law enforcement homicides have on decedent population coverage.
Because agencies with law enforcement homicides report inconsistently to the ARD and SHR,
proper evaluation of coverage must account for whether or not agencies that are not observed in
either source experienced law enforcement homicides, and, if so, how many. Sections 2.2.1 and
2.2.2, below, address two methods of conducting the capture-recapture analysis, including
assumptions made and the benefits of each technique. A detailed, technical description of the
techniques employed is provided in appendix B.

2.2.1 Simple poststratified Lincoln-Petersen approach
This approach is the simpler of the two described here, as it does not incorporate any correction
for unobserved agencies. In other words, under this approach there is an implied assumption that
any agency not observed in either the ARD or the SHR has not experienced a law enforcement
homicide, or, put another way, this method assumes that every agency with homicides is
observed in at least one source. Though this assumption is strong (and very likely violated), the
results of this method are still quite useful given the context in which they’re being used.
Specifically, assuming that all unobserved agencies are so-called ‘true zeroes’ (they have not
experienced any cases of law enforcement homicide), results in a coverage estimate that
represents the best-case scenario. Thus, the simple, poststratified LP estimate may be interpreted
as an upper bound for ARD coverage, with the true coverage rate falling somewhere below.
From a decision-making perspective, knowing the best-case estimate may be sufficient,
especially if the estimated coverage is considered unacceptable.

2.2.1.1 Assumptions
Assumptions a. and c. from 2.2 are accepted as valid in this approach. With both sources limited
to cases from 2003–09 and 2011, and all arising from U.S. agencies, assumption a. is reasonable.
Because each source has been subset to the common subpopulation of justifiable law
enforcement homicides, and BJS has verified that no duplicates are included in the files
provided, assumption c. is also reasonable. 3
As stated previously, given the inability to uniquely identify decedents across data sources,
assumption b. cannot be reasonably made. Conducting the analyses within a monte carlo

3

As part of the data quality checks, the ARD files were sorted several ways (i.e., state, year, name, date of death,
and date of birth) to identify duplicate records. This method was successful in identifying duplicate entries even
when the decedent had different names listed (e.g., first and last name inverted or middle name used as last name).
In addition, every ARD case was cross-referenced with the DCRP Jails file to identify duplicates across DCRP
collections. BJS used a similar method with the SHR file and removed any duplicates identified.

Page 9

replication framework allows for incorporation into final estimates of error incurred through
violation of this assumption, and for this reason, no single replicate can be evaluated for linking
correctness. Individual linkages may be made in a single replicate with low probability, though
across replicates they will occur rarely. Taken as a whole, across all replicates, average linking
outcomes represent the best matches given the available agency and individual information.
Assumption d., commonly referenced in the context of behavioral response in capture-recapture
literature, cannot be validated and potential biases incurred through violation cannot be mitigated
or quantified given the information available. It is worth noting, however, that the nature of this
bias may not be particularly important for this analysis. In the context of this study, a behavioral
response would mean that those responsible for collecting and reporting data to the ARD and
SHR programs would be more or less likely to report to one program if they had already reported
to the other. Because (1) it is reasonable to assume that such an effect could work in both
directions (i.e., a record may be more or less likely to be captured in one source if it has been
previously captured in the other) and (2) there is no reason to believe that one source consistently
captures decedent records before the other, any bias resulting from behavioral responses is
assumed to be diffused and to contribute little, if any, directional bias to final coverage estimates.
Homogeneity of capture probabilities implied by assumptions e. and f. cannot be guaranteed
within or across data sources, so the LP estimator of population size may contain downward bias,
leading to an overestimation of coverage (Wolter, 1986). The common method for dealing with
this bias—and the one employed in this analysis—is poststratification. Once decedent records
are linked, they are grouped into strata corresponding to the cross-classification of available
demographic characteristics. The simple LP estimator is then used within strata and the results
are aggregated over the complete population. This approach is based on the idea that, once
grouped, capture probabilities become homogeneous within strata. Poststratification on these
variables may not account for all bias because the number of reliable characteristics with enough
samples per category common to both sources is low (race/Hispanic origin and age only). Again,
in the context of this analysis, remaining bias may have little to no effect on the analytic utility of
the results, as the direction of the bias (if not its magnitude) is known. Heterogeneity in capture
probabilities yield population size estimates that are deflated and coverage rates that are
artificially high. Thus, final coverage estimates can be interpreted as the upper bounds on the
actual coverage rate.

2.2.1.2 Strengths and limitations
This simple LP estimator method has the notable benefit of being the simpler of the two methods
implemented, as it does not require estimation of the number of unobserved agencies with law
enforcement homicides. Additionally, this method yields an estimate that represents the best-case
scenario for ARD coverage. From the perspective of a user wanting to know the most accurate
coverage estimate, rather than its upper boundary, this method is not ideal.

2.2.2 Poststratified Lincoln-Petersen approach with constant ratio adjustment
for unobserved agencies
Unlike the approach described in 2.2.1, the poststratified LP method rejects the assumption that
any agency not observed in either the ARD or the SHR has not experienced a law enforcement
Page 10

homicide. It therefore must account for two additional unknown factors: the number of
unobserved agencies with law enforcement homicides and the number of homicides experienced
among those agencies.
Under this approach, the simple LP estimator is used to estimate the total number of agencies
that experienced law enforcement homicides in the reference period. By treating the ratio of this
estimate over the number of agencies actually observed across both sources as an agency
undercoverage adjustment factor, decedent population estimates corresponding to each
poststratum (as described in 2.2.1) may be extrapolated to represent the decedent population
among unobserved agencies. Because the decedent population is inflated by a constant factor,
this approach implies a strong assumption that missing agencies with homicides are missing at
random (i.e., the distribution of missing agencies in terms of decedent counts is identical to that
of the observed agencies). 4 Notably, agencies that are captured in both the ARD and SHR have,
on average, more homicides than those observed in only one source (5.2 vs. 1.5), and it may be
reasonable to infer that unobserved agencies are more like the latter group. In other words,
assuming that missing agencies are just like those that have been observed likely results in an
inflationary bias in the decedent population size estimate and, therefore, a deflationary bias in the
corresponding ARD coverage estimate. This implies that the coverage estimate derived using a
constant ratio adjustment for unobserved agencies represents a worst-case scenario, or lower
bound, for ARD coverage. Presenting the coverage estimates derived using this method along
with the one described in 2.2.1 yields a bounded range representing the most extreme
assumptions regarding the nature of unobserved agencies. Given the requirements of the
analysis, this is likely sufficient and benefits from making as few assumptions as possible.

2.2.2.1 Assumptions
Beyond the assumptions inherent to the agency undercoverage adjustment factor noted above,
the assumptions for this model are identical to those described in 2.2.1.1 and are not repeated
here.

2.2.2.2 Strengths and limitations
The postratified LP methodology provides additional benefit over that described in 2.2.1 in that it
rejects the (likely false) assumption that all unobserved agencies are so-called true zeroes. The
notable downsides of this approach are the necessity of making additional assumptions regarding
the nature of unobserved agencies—each additional assumption introduces new sources of
potential error. Additionally, as this method treats unobserved agencies as being like those that
are observed, in terms of the volume of law enforcement homicides, it potentially yields a
deflated coverage estimate.

4

Use of a simple, unstratified LP estimator likely results in deflationary bias in the law enforcement homicide
agency population estimate; its use is necessitated by resource constraints that precluded collection and inclusion of
agency-level characteristics for poststratification.

Page 11

2.2.3 Data assumptions and coverage estimate implications
The sections discussed above deal with theoretical assumptions underpinning the LP estimator.
As noted, these assumptions have been addressed where possible and, in all cases, their potential
effects on coverage estimates have been discussed. Given the context in which these estimates
are being used and the careful statements regarding their interpretation, the analysis is robust to
violations of these assumptions. Fundamental assumptions about the data, however, are largely
unverifiable.
For certain data items, specifically date and race/Hispanic origin variables that are used in
linking and that are measured differently across sources, comparability is maximized through
recoding (as noted in appendixes A and C) and the impact of remaining inconsistencies can be
inferred based on the amount of variability in final estimates across replicates, as the linking
procedure results in cases being linked probabilistically even when dates and race/Hispanic
origin codings are not in agreement across data sources. Because the range of final ARD
coverage estimates across replicates is less than 2 percentage points, it is reasonable to infer that
inconsistencies in these variables across sources do not have a large impact. Furthermore, the
small range of replicate estimates indicates that final estimates are not being driven by the
probabilistic linking.
Across replicates, 68% of decedent records remain unmatched after linking. Of these unmatched
records, 53% come from ARD and an overwhelming 82% of those are not even eligible to enter
the probabilistic linking procedure because they have no match in the SHR on ORI, year, and
sex. Given the relative rarity of female law enforcement homicides and the low estimated
probability of decedent records appearing in the incorrect year, ORI can be viewed as the linking
variable with by far the largest impact on final estimates. Data quality issues relating to ORI
codes may therefore affect estimated coverage estimates, though, unfortunately, errors resulting
from problems with ORI codes are of an unknown magnitude and direction.

Page 12

3 Findings
3.1 Arrest-related death coverage
As requested by BJS, RTI estimated the size of the law enforcement homicide population in the
United States and the Arrest-Related Death (ARD) program coverage using two methods. The
simple, poststratified Lincoln-Petersen method assumes that all nonreporting law enforcement
agencies did not have any law enforcement homicides during the observation period. This
approach provides the highest possible ARD coverage estimate and the lowest estimate of the
total population size (also referred to as the “right bookend” approach). Following this approach,
the ARD coverage of all law enforcement homicides is estimated to be 48.73%, with a 95%
confidence interval ranging from 47.28% to 50.18%. In other words, the ARD program captured
about half of all law enforcement homicides in the United States during the observation period.
The second coverage estimate assumes that all nonreporting agencies have the same number of
law enforcement homicides as agencies that did report to either the Supplementary Homicide
Reports (SHRs) or the ARD program. The poststratified Lincoln-Petersen approach with a
constant ratio adjustment for unobserved agencies provides the “left bookend” or lower estimate
of ARD coverage. Under this approach, the ARD program is estimated to capture 36.43% of all
law enforcement homicides in the United States, with a 95% confidence interval ranging from
34.50% to 38.35%.
Figure 1 is a Venn diagram of the ARD and SHR coverage following the simple, poststratified
Lincoln-Petersen method (the first coverage estimate described above), which assumes that all
nonreporting law enforcement agencies did not have any law enforcement homicides during the
observation period. Under these assumptions, the universe of law enforcement homicides in the
United States across all years examined (2003–09 and 2011) is 7,427, or an average of 928 law
enforcement homicides per year. The ARD program captures approximately 49% of these
homicides, while the SHR captures 46%. About 28% of the law enforcement homicides in the
United States are not captured by either system (see figure 2).

Page 13

Figure 1: ARD and SHR coverage and overlap of the universe of law enforcement
homicides in the United States, with no agency adjustment, 2003–09 and 2011

Page 14

Figure 2: Estimated proportion of law enforcement homicide universe covered by each
source, with no agency adjustment, 2003–09 and 2011

Unobserved
28%

SHR only
23%

ARD only
26%

ARD and SHR
23%

RTI implemented a monte carlo replication approach to match SHR and ARD cases (see table 1).
There were an estimated 5,324 law enforcement homicides reported to either system in the
observed years. More than two-thirds of those homicides were not able to be matched across data
sources. The level of unmatched cases was greatest in 2005 (75.6%) and lowest in 2011 (60.4%).
Certainty matches across sources are created when cases match on the originating agency
identifier (ORI) number, year, and sex, within 1 month of date of death (or date of report for
SHR) and within 5 years for age. Across all observed years, approximately 21% of all law
enforcement homicides reported to either source were certainty matches, ranging from 15% in
2008 to 26% in 2009 and 2011. These certainty matches represent 94.5% of ARD cases with one
or more SHR pairings satisfying the certainty criteria noted above (not shown). The remaining
5.5% have more than one match that satisfies the certainty criteria (4.9% have two matches, and
the remaining 0.6% have between three and six matches).

Page 15

Table 1: ARD and SHR law enforcement homicides, by match outcome, 2003–09 and 2011
Probability match score
No match
ARD program data
collection year
All LE homicides

AVG N

0.01 – 0.25
%

AVG N

0.26 – 0.50

%

AVG N

%

0.51 – 0.75
AVG N

%

0.76 – 0.99
AVG N

%

Certainty match

Total

AVG N

%

AVG N

%

3,643.9

68.4

221.0

4.2

190.1

3.6

150.8

2.8

6.8

0.1

1,111.9

20.9

5,324.5

100

2003

424.6

71.3

23.6

4.0

19.3

3.2

15.2

2.5

0.1

0.0

112.6

18.9

595.3

100

2004

416.7

70.6

18.9

3.2

19.4

3.3

11.8

2.0

0.6

0.1

123.1

20.8

590.4

100

2005

456.7

75.6

19.8

3.3

16.8

2.8

14.8

2.5

1.8

0.3

94.4

15.6

604.3

100

2006

469.8

70.9

22.7

3.4

27.0

4.1

24.4

3.7

0.1

0.0

118.4

17.9

662.4

100

2007

450.1

66.2

25.2

3.7

23.2

3.4

21.1

3.1

1.5

0.2

158.9

23.4

680.0

100

2008

482.6

73.4

37.1

5.6

21.1

3.2

18.5

2.8

0.0

0.0

98.0

14.9

657.3

100

2009

443.4

62.7

30.1

4.3

27.7

3.9

17.2

2.4

1.2

0.2

187.1

26.5

706.7

100

2011

500.1

60.4

43.6

5.3

35.7

4.3

27.8

3.4

1.5

0.2

219.4

26.5

828.0

100

Note: The 'AVG N' represents the mean count across all replicates.

Table 2: ARD law enforcement homicides, by match outcome, 2003–09 and 2011
Probability match score
ARD program
data collection
year

0.01 – 0.25

No match

0.26 – 0.50

AVG
N

1,939.5

53.6

221.0

6.1

190.1

5.3

150.8

4.2

6.8

0.2

2003

206.3

54.7

23.6

6.3

19.3

5.1

15.2

4.0

0.1

2004

201.4

53.7

18.9

5.0

19.4

5.2

11.8

3.1

2005

229.3

60.8

19.8

5.3

16.8

4.5

14.8

3.9

2006

254.4

56.9

22.7

5.1

27.0

6.0

24.4

2007

225.0

49.5

25.2

5.5

23.2

5.1

All LE homicides

%

%

AVG
N

0.76 – 0.99

%

AVG N

AVG
N

0.51 – 0.75
%

AVG
N

%

Certainty match
AVG N

Total

%

AVG N

%

1,111.9

30.7

3,620.0

100

0.0

112.6

29.9

377.0

100

0.6

0.2

123.1

32.8

375.0

100

1.8

0.5

94.4

25.0

377.0

100

5.5

0.1

0.0

118.4

26.5

447.0

100

21.1

4.6

1.5

0.3

158.9

34.9

455.0

100

2008

229.3

56.8

37.1

9.2

21.1

5.2

18.5

4.6

0.0

0.0

98.0

24.3

404.0

100

2009

232.7

46.9

30.1

6.1

27.7

5.6

17.2

3.5

1.2

0.2

187.1

37.7

496.0

100

2011

361.0

52.4

43.6

6.3

35.7

5.2

27.8

4.0

1.5

0.2

219.4

31.8

689.0

100

Note: The 'AVG N' represents the mean count across all replicates.

Page 16

There were 3,620 cases reported to ARD in the years observed (see table 2). Across all years,
54% of those were not able to be matched to an SHR case, and 31% were a certainty match.
Probability matches included 11.4% of ARD cases with a low probable match (probability score
of 0.50 or less), and 4.4% with a probability match score between 0.51 and 0.99. Among all
matched ARD cases, 47.6% were linked to a single SHR case across replicates (not shown), and
74.7% were matched to five or fewer distinct SHR cases. The maximum number of distinct SHR
pairings for a single ARD case across replicates was 24. Across years, the lowest overlap with
the SHR occurred in 2005 when 61% of all ARD cases were not able to be matched to an SHR
case. In 2009, however, 38% of ARD cases were a certainty match to SHR, 15% were a probable
match, and 47% were a no match. Figure 3 further shows the distribution of match scores for
cases reported to ARD across all years observed.

Figure 3: Percent of observed ARD law enforcement homicides, by match score, 2003–09
and 2011
No match

0.01 - 0.25

0.26 - 0.50

0.51 - 0.75

0.76 - 0.99

Certainty match

100.0
90.0
80.0
70.0

30.7

29.9

25.0

24.3

26.5

32.8

34.9

37.7

31.8

60.0
50.0
40.0
30.0
20.0
10.0
0.0

53.6

54.7

53.7

60.8

56.9

49.5

56.8

46.9

52.4

All years

2003

2004

2005

2006

2007

2008

2009

2011

Table 3 describes characteristics of law enforcement homicides reported to either the SHR or the
ARD program by match score. As expected, law enforcement homicide victims with an
unknown age are more likely to have no match. Law enforcement homicide victims that are
identified as white, non-Hispanic (75%) are most likely to have a no match compared to other
race/Hispanic origin groups, while victims of an “other” race are most likely to have a certainty
match (26%) compared to unknown or other unspecified race/Hispanic origin groups.

Page 17

The characteristics of cases reported to the ARD program by probability match score are
described in table 4. Cases with victims of any unknown age are more likely to have no match
compared to any known age group. Similarly, cases with victims of an unknown race/Hispanic
origin are more likely to have a low probability match score compared to any known
race/Hispanic origin groups. ARD cases with a victim identified as Hispanic (28%) are least
likely to result in a certainty match compared to other known race/Hispanic origin groups.
As described in appendix C, the race/Hispanic origin variable is measured differently by each
data source, and there is a high amount of missing data regarding Hispanic origin in the SHR.
This measurement issue may account for some of the differences in probability match scores by
race/Hispanic origin group. A subgroup analysis on states with larger proportions of Hispanic
populations was conducted to provide further insight into whether the difference in probability
match scores across race/Hispanic origin groups is meaningful. RTI calculated coverage
estimates among five states that report Hispanic origin to SHR greater than 75% (Arizona,
California, North Carolina, Pennsylvania, and Texas). Following the simple, poststratified
Lincoln-Peterson approach (upper bound), the ARD coverage in these five states is 71.91% (95%
confidence interval range from 69.47% to 74.34%). With agency adjustment (providing the
lower bound), the estimated ARD coverage is 54.90% in these five states (95% confidence
interval range from 50.52% to 59.29%). Given the wide range of these results, it is likely that
they are confounded with other, unmeasured agency characteristics, such as agency size,
reporting requirements, and the prevalence of law enforcement homicides. That is, the estimates
with these five states were intended to act as sensitivity analyses for measuring the Hispanic
origin of victims of arrest-related homicides; however, they suggest that other variables may be
driving ARD program coverage estimates.

Page 18

Table 3: ARD and SHR law enforcement homicides, by matching variable and match outcome, 2003–09 and 2011
Probability match score
No match

0.01 – 0.25

AVG N

%

AVG N

%

0.76 – 0.99
AVG N

%

Certainty match
AVG N

%

Total

%

3,643.9

68.4

221.0

4.2

190.1

3.6

150.8

2.8

6.8

0.1

1,111.9

20.9

5,324.5

100

Male

3,500.4

68.3

220.4

4.3

189.7

3.7

143.9

2.8

6.0

0.1

1,063.4

20.8

5,123.7

100

Female

140.6

71.1

0.6

0.3

0.4

0.2

6.9

3.5

0.8

0.4

48.5

24.5

197.8

100

17 or younger

115.4

70.0

10.7

6.5

5.4

3.3

2.8

1.7

0.0

0.0

30.5

18.5

164.9

100

18–24

850.9

67.8

52.4

4.2

52.4

4.2

39.9

3.2

1.1

0.1

258.5

20.6

1,255.2

100

25–34

1,087.1

67.4

69.4

4.3

64.7

4.0

46.0

2.9

0.9

0.1

344.1

21.3

1,612.0

100

35–44

786.3

68.0

41.6

3.6

35.5

3.1

37.3

3.2

2.2

0.2

252.7

21.9

1,155.6

100

45–54

509.5

70.5

30.0

4.2

21.1

2.9

14.6

2.0

0.8

0.1

147.1

20.3

723.3

100

55 or older

246.2

69.9

12.3

3.5

8.0

2.3

6.7

1.9

0.0

0.0

79.0

22.4

352.2

100

Unknown

48.5

79.1

4.5

7.4

3.0

5.0

3.5

5.7

1.7

2.8

0.0

0.0

61.3

100

White, nonHispanic

1,764.9

75.0

27.9

1.2

47.4

2.0

36.5

1.5

3.2

0.1

472.5

20.1

2,352.3

100

Black, nonHispanic

1,093.6

64.9

77.0

4.6

70.5

4.2

36.2

2.1

2.2

0.1

404.2

24.0

1,683.7

100

Hispanic, any
race

581.7

59.2

89.1

9.1

56.1

5.7

54.9

5.6

1.4

0.1

199.7

20.3

982.7

100

All LE homicides

%

0.51 – 0.75

AVG N

Matching variable

AVG N

0.26 – 0.50

AVG N

%

Sex

Age

Race/ Hispanic
origin

Other

86.0

63.2

4.8

3.5

4.9

3.6

4.9

3.6

0.0

0.0

35.5

26.1

136.2

100

Unknown

117.8

69.5

22.2

13.1

11.2

6.6

18.3

10.8

0.0

0.0

0.0

0.0

169.6

100

Note: The 'AVG N' represents the mean count across all replicates.

Page 19

Table 4: ARD law enforcement homicides, by matching variable and match outcome, 2003–09 and 2011
Probability match
No match
Matching variable

0.01 – 0.25
%

AVG N

%

0.51 – 0.75
AVG N

%

0.76 – 0.99
AVG N

%

Certainty match
AVG N

Total

AVG N

%

1,939.5

53.6

221.0

6.1

190.1

5.3

150.8

4.2

6.8

0.2

Male

1,865.7

53.5

220.4

6.3

189.7

5.4

143.9

4.1

6.0

Female

73.8

56.3

0.6

0.5

0.4

0.3

6.9

5.3

0.8

17 or younger

57.5

53.8

10.7

10.0

5.4

5.0

2.8

2.7

0.0

0.0

30.5

28.5

107.0

100

18–24

427.7

51.4

52.4

6.3

52.4

6.3

39.9

4.8

1.1

0.1

258.5

31.1

832.0

100

25–34

555.0

51.4

69.4

6.4

64.7

6.0

46.0

4.3

0.9

0.1

344.1

31.9

1,080.0

100

35–44

426.7

53.6

41.6

5.2

35.5

4.5

37.3

4.7

2.2

0.3

252.7

31.7

796.0

100

45–54

297.3

58.2

30.0

5.9

21.1

4.1

14.6

2.9

0.8

0.2

147.1

28.8

511.0

100

55 or older

143.1

57.5

12.3

4.9

8.0

3.2

6.7

2.7

0.0

0.0

79.0

31.7

249.0

100

Unknown

32.2

71.5

4.5

10.1

3.0

6.7

3.5

7.8

1.7

3.8

0.0

0.0

45.0

100

White, nonHispanic

926.6

61.2

27.9

1.8

47.4

3.1

36.5

2.4

3.2

0.2

472.5

31.2

1,514.0

100

Black, nonHispanic

550.8

48.3

77.0

6.8

70.5

6.2

36.2

3.2

2.2

0.2

404.2

35.4

1,141.0

100

Hispanic, any
race

322.9

44.6

89.1

12.3

56.1

7.7

54.9

7.6

1.4

0.2

199.7

27.6

724.0

100

All LE homicides

AVG N

0.26 – 0.50

%

AVG N

%

1,111.9

30.7

3,620.0

100

0.2

1,063.4

30.5

3,489.0

100

0.6

48.5

37.1

131.0

100

Sex

Age

Race/ Hispanic
origin

Other

45.8

47.8

4.8

5.0

4.9

5.1

4.9

5.2

0.0

0.0

35.5

37.0

96.0

100

Unknown

93.2

64.3

22.2

15.3

11.2

7.7

18.3

12.6

0.0

0.0

0.0

0.0

145.0

100

Note: The 'AVG N' represents the mean count across all replicates.

Page 20

Table 5 describes the characteristics of law enforcement homicides that were reported to the
ARD program, the SHR, or both. No differences emerged in cases reported to either system or to
both in terms of age and race/ Hispanic origin of the victim. The vast majority of all law
enforcement homicides resulted from a firearm, with no difference in the proportion of
homicides resulting from other means (e.g., other weapon) across data sources. Across ARD
program years, there does not appear to be a difference in the characteristics of cases captured by
the ARD Program, the SHR, or both.

Page 21

Table 5: ARD law enforcement homicide decedent characteristics, by source, 2003–09 and 2011
Percent of all law enforcement homicide victims
2003–06

All years
Decedent characteristics

ARD
only

SHR only

ARD and
SHR

3.4 %

2.9 %

ARD
only

2007–09

SHR only

ARD and
SHR

3.2 %

3.0 %

ARD
only

SHR
only

2011
ARD and
SHR

ARD
only

SHR
only

ARD and
SHR

Age
17 or younger

3.0 %

3.3 %

3.3 %

4.1 %

3.5 %

1.5 %

1.1 %

1.7 %

18–24

22.1

24.8

24.1

22.2

24.9

25.1

22.0

25.3

23.8

21.9

22.2

22.5

25–34

28.6

31.2

31.2

29.3

31.4

31.4

29.0

30.6

31.5

26.3

33.6

30.2

35–44

22.0

21.1

22.0

23.4

23.3

23.6

21.1

18.9

20.8

20.3

18.0

21.0

45–54

15.3

12.5

12.7

14.2

11.2

10.5

14.5

13.3

13.4

19.9

16.2

16.0

55 or older

7.4

6.1

6.3

5.7

5.3

5.3

8.2

6.6

6.2

10.0

8.4

8.5

Unknown

1.7

1.0

0.8

2.0

0.7

1.2

2.0

1.4

0.7

0.2

0.5

0.1

Race/ Hispanic origin
White, non-Hispanic

47.8 %

49.2 %

35.0 %

49.8 %

49.9 %

35.2 %

43.4 %

47.0 %

35.3 %

51.0 %

55.2 %

33.8 %

Black, non-Hispanic

28.4

31.8

35.1

26.5

30.1

33.2

33.3

34.4

35.7

23.9

29.8

38.0

Hispanic, any race

16.7

15.2

23.9

17.1

15.9

25.5

16.2

14.6

22.5

16.3

13.3

23.2

Other

2.4

2.4

3.0

3.1

2.9

4.4

1.7

2.0

2.4

1.9

0.5

1.2

Unknown

4.8

1.4

3.1

3.5

1.1

1.8

5.4

1.9

4.1

6.9

1.2

3.7

Manner of death
Firearm

96.3 %

96.0 %

97.0 %

99.4 %

97.0 %

99.9 %

90.6 %

94.9 %

92.5 %

99.4 %

94.8 %

100 %

Other

3.7

4.0

3.0

0.6

3.0

0.1

9.4

5.1

7.5

0.6

5.2

0.0

Missing

0.0

0.0

0.0

0.1

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0

Page 22

3.2 Variability in ARD methodology across states
As described in Section 3.1, RTI calculated the size of the law enforcement homicide population
in the United States and the ARD program coverage using two methods. The first, a simple,
poststratified Lincoln-Petersen method, assumes that all nonreporting law enforcement agencies
did not have any law enforcement homicides during the observation period and therefore has no
agency adjustment. The second coverage estimate assumes that all nonreporting agencies have
the same number of law enforcement homicides on average as agencies that did report to either
the SHR or the ARD program. This poststratified Lincoln-Petersen assumes that the population
of interest is closed geographically and in time and is of fixed size, with a constant ratio
adjustment for unobserved agencies. Table 6 and figure 4 show coverage estimates using each
method by the year observed. The highest coverage rate with no agency adjustment was found in
the most recent collection year of 2011 (69%).

Table 6: ARD coverage estimates and confidence intervals, by estimation approach, 2003–
09 and 2011
No agency adjustment

Year

95%
Lower bound

Coverage
estimate

Agency adjustment

95% Upper
bound

95% Lower
bound

Coverage
estimate

95%
Upper
bound

All study years

47.28 %

48.70 %

50.18 %

34.50 %

36.43 %

38.35 %

2003

36.87

41.84

46.82

21.96

26.61

31.27

2004

37.30

42.89

48.49

24.41

29.46

34.52

2005

33.67

38.55

43.43

19.90

24.25

28.59

2006

42.50

46.99

51.48

28.73

33.37

38.02

2007

46.45

50.51

54.58

35.18

39.61

44.04

2008

36.50

40.81

45.12

22.57

26.82

31.07

2009

51.37

54.98

58.59

35.00

39.64

44.29

2011

65.12

69.01

72.90

53.85

58.58

63.30

Page 23

Figure 4: Percent of law enforcement homicide universe covered by ARD, by estimation
method, 2003–09 and 2011

Percent of estimated law enforcement homicides

100%

80%

69%

No agency adjustment
60%
42%

43%

40%

55%

51%

47%

59%
41%

39%

40%

40%
20%

27%

29%

33%
27%

24%

Agency adjustment
0%
2003

2004

2005

2006

2007

2008

2009

2011

Year

The methodology for identifying ARD cases has changed over the observation period, and
figure 4 suggests that the methods used in more recent years are associated with better coverage.
To better understand the methodologies used to identify arrest-related deaths in 2011, RTI
conducted interviews with state reporting coordinators (SRCs). These interviews explored the
primary method each SRC used for identifying cases, whether direct reporting from law
enforcement agencies was involved, and whether the SRCs conducted any sort of follow-up or
verification of cases reported to them. In addition, RTI tracked information about reporting
methods and SRC location throughout the 2011 data collection period. This information was
used to further examine the utility of various methodologies for identifying arrest-related deaths,
as implemented by the SRCs during the 2011 data collection.
The remaining analyses examine differences in case matching by state and reporting method.
RTI grouped states by the primary data collection methodology and the location of the SRCs.
Then, RTI conducted analyses to compare the number of cases reported only to ARD, only to
SHR, or to both programs. Because the methodology for reporting was determined largely
through the SRC interviews and information gained during the 2011 data collection period, these
analyses are limited to 2011 data only. Table 7 describes state-level variation in reporting. In
2011, 83% of law enforcement homicides reported to either system were estimated to be
captured in the ARD program. Approximately 139 of the estimated 828 law enforcement
homicides captured by either system were reported only to the SHR.

Page 24

There is considerable variability among states in the proportion of law enforcement homicides
that are reported only to the ARD program, only to SHR, or to both sources. Three states
reported to both systems but had no estimated overlap between the two (Iowa, Minnesota, and
Arkansas). In other words, no law enforcement homicides reported to ARD were matched to
homicides found in the SHR, and the SHR deaths were not matched to any of the ARD deaths in
these three states. Twelve states reported all law enforcement homicides captured by either
system to ARD: Connecticut, Massachusetts, New Hampshire, New York, Kansas, North
Dakota, Alabama, Mississippi, West Virginia, Alaska, Idaho, and Montana. No states reported
law enforcement homicides only to the SHR. Reporting to ARD is estimated to capture at least
75% of cases reported to either system for 38 states, 11 of which reported at least 20 law
enforcement homicides to either system in 2011. Missouri, Maryland, Hawaii, and Nevada were
each estimated to report more cases to the SHR than the ARD program. However, ARD still
captured 82% of the estimated cases in Maryland and 84% of the cases in Nevada, and Hawaii
had very few arrest-related deaths reported to either system. In Missouri, half of the cases
reported to either system were found in the ARD, but 87% of the cases reported to either system
were found in the SHR. Rhode Island, Vermont, Delaware, and Wyoming did not report any law
enforcement homicides to either system in 2011.
Table 7: ARD law enforcement homicides, by region, state, and source, 2011
ARD only
Region and state

SHR only

ARD and SHR

AVG N

Row %

Col %

AVG N

Row %

Col %

AVG N

Row %

Col %

361.3

43.6

100

139.3

16.8

100

327.7

39.6

100

50.0

55.6

13.8

9.0

10.0

6.5

31.0

34.4

9.5

Connecticut

2.0

100

0.6

0.0

0.0

0.0

0.0

0.0

0.0

Maine

2.6

30.6

0.7

2.6

30.6

1.9

3.4

38.9

1.0

Massachusetts

5.0

100

1.4

0.0

0.0

0.0

0.0

0.0

0.0

New Hampshire

4.0

80.0

1.1

0.0

0.0

0.0

1.0

20.0

0.3

New Jersey

4.0

25.0

1.1

4.0

25.0

2.9

8.0

50.0

2.4

26.0

100

7.2

0.0

0.0

0.0

0.0

0.0

0.0

Pennsylvania

6.4

23.3

1.8

2.4

8.7

1.7

18.6

68.0

5.7

Rhode Island

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0

Vermont

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0

78.7

50.6

21.8

31.7

20.4

22.8

45.3

29.1

13.8

Illinois

9.1

28.4

2.5

1.1

3.4

0.8

21.9

68.2

6.7

Indiana

8.7

46.6

2.4

5.7

30.6

4.1

4.3

22.8

1.3

Iowa

4.0

80.0

1.1

1.0

20.0

0.7

0.0

0.0

0.0

Kansas

6.0

85.7

1.7

0.0

0.0

0.0

1.0

14.3

0.3

Michigan

8.0

44.4

2.2

5.0

27.8

3.6

5.0

27.8

1.5

Minnesota

2.0

50.0

0.6

2.0

50.0

1.4

0.0

0.0

0.0

Missouri

2.4

13.2

0.7

9.4

51.2

6.8

6.6

35.6

2.0

Nebraska

1.0

33.3

0.3

1.0

33.3

0.7

1.0

33.3

0.3

North Dakota

1.0

100

0.3

0.0

0.0

0.0

0.0

0.0

0.0

Total
Northeast

New York

Midwest

Page 25

ARD only
Region and state

SHR only

ARD and SHR

AVG N

Row %

Col %

AVG N

Row %

Col %

AVG N

Row %

Col %

29.5

88.0

8.2

0.5

1.4

0.3

3.5

10.6

1.1

South Dakota

3.0

42.9

0.8

3.0

42.9

2.2

1.0

14.3

0.3

Wisconsin

4.0

50.0

1.1

3.0

37.5

2.2

1.0

12.5

0.3

125.8

41.0

34.8

52.8

17.2

37.9

128.2

41.8

39.1

Alabama

7.0

100

1.9

0.0

0.0

0.0

0.0

0.0

0.0

Arkansas

3.0

75.0

0.8

1.0

25.0

0.7

0.0

0.0

0.0

Delaware

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0

District of
Columbia

0.4

6.6

0.1

0.4

6.6

0.3

4.6

86.8

1.4

Florida

20.8

24.2

5.8

19.8

23.1

14.2

45.2

52.7

13.8

Georgia

12.9

43.1

3.6

3.9

13.0

2.8

13.1

43.8

4.0

Kentucky

8.0

61.5

2.2

3.0

23.1

2.2

2.0

15.4

0.6

Louisiana

3.7

42.8

1.0

1.7

20.0

1.3

3.3

37.2

1.0

Maryland

2.8

13.4

0.8

3.8

18.2

2.7

14.2

68.3

4.3

Mississippi

7.0

87.5

1.9

0.0

0.0

0.0

1.0

12.5

0.3

12.0

80.0

3.3

1.0

6.7

0.7

2.0

13.3

0.6

Oklahoma

5.1

31.5

1.4

3.1

19.1

2.2

7.9

49.4

2.4

South Carolina

5.4

52.1

1.5

3.4

32.9

2.5

1.6

15.0

0.5

Tennessee

2.1

15.8

0.6

2.1

15.8

1.5

8.9

68.5

2.7

Texas

23.1

47.1

6.4

7.1

14.5

5.1

18.9

38.4

5.8

Virginia

10.5

56.8

2.9

2.5

13.6

1.8

5.5

29.6

1.7

2.0

100

0.6

0.0

0.0

0.0

0.0

0.0

0.0

106.7

38.7

29.5

45.7

16.6

32.8

123.3

44.7

37.6

Alaska

1.0

50.0

0.3

0.0

0.0

0.0

1.0

50.0

0.3

Arizona

5.0

21.7

1.4

3.0

13.0

2.1

15.0

65.3

4.6

California

52.7

34.7

14.6

27.7

18.3

19.9

71.3

47.0

21.8

Colorado

10.1

59.1

2.8

5.1

29.8

3.7

1.9

11.1

0.6

Hawaii

0.1

4.4

0.0

2.1

68.1

1.5

0.9

27.4

0.3

Idaho

3.0

100

0.8

0.0

0.0

0.0

0.0

0.0

0.0

Montana

2.0

100

0.6

0.0

0.0

0.0

0.0

0.0

0.0

Nevada

1.6

9.4

0.4

2.6

15.4

1.8

12.4

75.2

3.8

New Mexico

5.9

49.4

1.6

1.9

15.6

1.3

4.1

35.0

1.3

Oregon

3.2

61.7

0.9

0.2

4.3

0.2

1.8

33.9

0.5

Utah

2.6

39.4

0.7

1.6

24.3

1.1

2.4

36.3

0.7

19.6

58.3

5.4

1.6

4.7

1.1

12.4

37.0

3.8

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0

0.0

Ohio

South

North Carolina

West Virginia
West

Washington
Wyoming

Note: These are the mean counts across all replicates.

Page 26

3.2.1 ARD law enforcement homicide reporting methodology
Primary data collection methodology was determined through interviews with the SRCs in 2012
and from information gathered during the 2011 data collection period (see table 7). Most states
(N = 26) relied solely on media to identify ARD-eligible cases. ARD program staff identified
cases in four states, also relying primarily on media sources. Eight states used direct reporting
from local law enforcement agencies, and seven states relied on direct reporting from medical
examiner’s/coroner’s offices. Four states accessed the National Violent Death Reporting System
(NVDRS). Finally, two states had a legislative mandate to report arrest-related deaths.
Table 7: Primary methodology for identifying arrest-related deaths, by state, 2011

Legislative
mandate

Direct law
enforcement
agency report

Direct medical
examiner’s/coroner’s
report

National
Violent Death
Reporting
System
KY

Media only

CA

AL

CO

TX

DC

ME

KS

MA

MO

NM

UT

NH

NC

FL

ND

NV

SC

HI

OH

TN

WA

ID

PA

IL

RI

IN

SD

IA

VA

LA

WV

MD

MS

MI

MT

VT

ArrestRelated
Death
program
staff

AK

MN

AR

OK

AZ

NE

GA

OR

CT

NY

WI

DE

NJ

WY

Note: States are listed using standard U.S. postal abbreviations.

Figure 5 shows that there was not much variability in the proportion of observed law
enforcement homicides that were found in the ARD program based on the primary identification
method. The proportion of observed cases found in ARD ranged from a low of 80% for states
that relied primarily on direct law enforcement agency reports to a high of 85% for states that
relied on reports from medical examiner’s or coroner’s offices. The lack of association between
the primary identification method and the proportion of cases found in ARD could be due to
imprecise methods for categorizing states. For example, there may be additional variability
within primary methodologies. Some states with direct law enforcement agency reports may
have received these from only a few agencies in the state, and without a backup form of
identification, were missing many law enforcement homicides. The majority of states relied on
media reports, but there was likely to be considerable variability in the number of media sources
reviewed, whether such review was proactive or reactive, and whether media-identified reports
were then verified. Furthermore, the information RTI received about primary identification
methodology may not have reflected the true methods that the SRC uses in that state, due to
incomplete or outdated information collected during the SRC interviews or during the 2011 data
collection period.

Page 27

Figure 5: Law enforcement homicides, by source and primary reporting method, 2011
ARD only

90%

SHR only

Total cases (avg across replicates)
900

828
17%

16%

17%

15%
20%

80%

19%

800
19%
700

70%

600

60%
500
50%

371

400

40%
300

30%

201
200

20%

102

10%

72

42

41

ARD program
staff
identification
only

Access to and use
of NVDRS

0%

100
-

All states

Media report
only

Legislative
mandate

Direct medical
examiner/
coroner's office
report

Direct law
enforcement
agency report

Primary identification methodology

Page 28

Total number of law enforcement homicides observed

Proportion of observed cases captured in each source

100%

ARD and SHR

3.2.2 Location of SRC
States were also categorized by the location of the SRC. The greatest number of SRCs (25) were
located in a planning, commission, or statistics agency, such as a statistical analysis center, a
research or data agency within the state’s department of criminal justice, or a planning
organization (see table 8). SRCs in 13 states were situated in a state-level law enforcement or
public safety department. There were three SRCs in a state prosecutor’s office, three in a medical
examiner/coroner’s office or health agency, and three in a state corrections agency.
Figure 6 describes the proportion of observed law enforcement homicides that were found in
ARD by SRC location. SRCs located in state prosecutors’ offices and statistical analysis or other
research centers, and those located in “other” locations, each had approximately 87% of the
observed cases found in ARD, and only 13% of the observed cases found only in SHR. SRCs
located in state law enforcement agencies and state corrections agencies had the lowest
proportion of observed cases found in ARD (76% and 75%, respectively). State law enforcement
agencies may not have collected data regularly from local law enforcement agencies to report to
ARD, or they may have captured data only from a subset of agencies in the state. Data reported
to the SHR, on the other hand, came directly from local law enforcement agencies. SRCs located
in state corrections agencies may not have had the direct access or necessary relationships to
regularly collect information on law enforcement homicides that occurred during the process of
arrest. Further analysis may be warranted to determine how staff in state law enforcement
agencies work with local agencies to identify and collect information about law enforcement
homicides.

Table 8: SRC agency location, 2011
State law
enforcement
agency or
public
safety dept

State
prosecutors
office

Medical
examiner/coroner’s
office or health
agency

CO

NH

ME

FL

ND

NC

IN

HI

TX

OR

LA

MA
MI
MN

IL

UT

State
corrections
agency
ID

Planning,
commission, or
statistics agency
AK

NM

AR

AZ

NY

GA

CA

OH

WI

CT

OK

WY

DE

PA

MO

IA

VT

RI

KS

VA

SC

KY

WV

SD

MD

AL

TN

NV

MS

WA

NE

MT

DC

NJ

Note: States are listed using standard U.S. postal abbreviations.

Page 29

ArrestRelated
Death
program
staff

Figure 6: Law enforcement homicides, by source and SRC location, 2011
ARD only

90%

SHR only

Total cases (avg across replicates)
900

828
17%

13%

13%

13%
19%

80%

24%

25%

800
700

70%

600

60%

500

451
50%

400

40%
30%

300

221

200

20%
55

10%

42

30

29

0%

100
-

All states

Statistical analysis
or other research
center

State law
enforcement
agency

State prosecutors ARD program staff State corrections
office
agency
SRC location

Page 30

Other

Total number of observed law enforcement homicides

Proportion of observed homicides found in each source

100%

ARD and SHR

3.3 Number of years at least one death was reported to ARD and SHR
Table 9 describes the number of agencies reporting at least one death for the 8 years observed in
these analyses, and the estimated number of deaths reported by those agencies. Overall, more
unique agencies reported to ARD in the years observed, 2003 through 2009 and 2011 (N =
1,261), compared to SHR (N = 881). For both systems, the majority of these unique agencies
reported a death in only 1 of the 8 years examined. Fifteen agencies reported at least 1 death to
ARD in all 8 years, and 28 agencies reported at least 1 death to SHR in all 8 years. The number
of unique agencies reporting deaths in 7 or 8 years is higher for the SHR system compared to the
ARD program. This finding suggests that the SHR is more likely to include data from agencies
that consistently have at least one law enforcement homicide each year, as compared to the ARD
program. Agencies that do not have law enforcement homicides each year are more likely to be
found in the ARD data.

Table 9: Number of years at least one death was reported to ARD and SHR
ARD
Number of years at least one
death was reported
Total

SHR

Number of
agencies

Number of
deaths

Number of
agencies

Number
of deaths

1,261

3,620

881

3,385

8

15

696

28

1075

7

13

348

17

368

6

23

278

20

240

5

23

262

18

149

4

42

273

40

304

3

81

358

77

295

2

189

478

167

401

1

875

927

514

553

Page 31

4 Conclusions and recommendations
The capture-recapture analysis described above estimates that, at best, the Arrest-Related Deaths
(ARD) program captured approximately half of all law enforcement homicides in the United
States during the study period (2003–09, 2011). This estimate assumes that all unobserved law
enforcement agencies had zero law enforcement homicides. If those unobserved agencies did
have a similar number of law enforcement homicides as those that did report to the ARD
program or the Supplementary Homicide Reports (SHRs), the ARD coverage estimate is reduced
to 36%. This information alone may be sufficient for BJS to determine whether to continue the
ARD data collection program. Additional considerations for BJS to explore regarding continuing
data collection through the ARD program are described below.

4.1 Relative strengths and weaknesses of the ARD program methodology
A primary strength of the ARD program is that it was the only data collection that attempted to
enumerate all arrest-related deaths in the United States. Unlike the SHR, or any other national- or
state-level database, it also included accidental and natural deaths that occurred during the
process of arrest in addition to law enforcement homicides. Although the current analyses did not
identify any specific program methodologies that were more likely to identify law enforcement
homicides than others, we did not examine their utility for identifying other types of arrestrelated deaths.

4.1.1 Types of cases that were covered well by the ARD program
Analyses of decedent characteristics did not reveal any meaningful differences in capture rates
by age, sex, or race/Hispanic origin. The 2011 data collection year did exhibit some differences
across decedent characteristics compared to previous years, but these differences were not
confined to cases found only in the ARD program.

4.1.2 Identification approaches that are associated with better coverage
The analyses described above did not identify specific methodologies that were associated with
increased coverage. There was little variation in the percentage of observed cases found in ARD
(as opposed to the SHR) based on the primary methodology employed by the SRCs. For
example, our analyses indicated that 20% of all observed homicides were found only in the SHR
for states where the SRC relied primarily on direct reports from law enforcement agencies to
identify homicides. In other words, ARD captured 80% of the observed homicides in these
states. Eighty-five percent of the observed law enforcement homicides in states that relied
primarily on data from medical examiner’s or coroner’s offices were found in the ARD program.
Among these states, 15% of the cases observed in 2011 were found only in the SHR. Examining
state-level variations in reporting and matching those directly to the time put in by the SRC, the
location of the SRC, and the primary identification methodology will likely shed more light on
effective identification practices. BJS may also further examine the policies and practices in
California and Texas. Both states have mandated reporting legislation, but our analysis suggests
that approximately a fifth of the observed cases in California in 2011 were reported to the SHR
only, while 14% of the observed cases in Texas in 2011 were reported to the SHR only.

Page 32

4.2 Recommendations
While the capture-recapture analysis estimated that approximately half of all law enforcement
homicides were not captured in the ARD program during the study period, the estimated rate of
coverage for ARD in 2009 and 2011 was higher than the earlier years examined. In 2003, we
estimated that the ARD program captured, at best, 42% of the universe of law enforcement
homicides in the United States. In 2011, that coverage estimate increased to 69%. However, the
analyses described above did not uncover any significant variations in coverage for observed
homicides based on the location of the SRC or the primary identification methodology used in
2011. It is possible that all of the states were using more rigorous methodologies in 2011 as
compared to earlier data collection years. Even states that relied primarily on media to identify
cases may have been applying a comprehensive and regular approach to identifying these cases
and following up with other agencies to verify the death. In fact, most states relied on multiple
approaches to identify cases, but only the primary method was analyzed in this report.
In general, more of the observed law enforcement homicides in 2011 were found in the ARD
program compared to the SHR. There was still considerable state-to-state variation in reporting to
one or both programs, however, and our analyses failed to identify specific reasons for this
variation, such as the SRC’s location or the primary identification methodology used.
A combination of data from both the SHR and the ARD is estimated to capture, at best, 72% of all
law enforcement homicides in the United States across all years observed. In 2011, the ARD
program captured, at most, an estimated 69% of the universe. Despite demonstrated improvements
in the ARD program coverage over time, there are still a large number of law enforcement
homicides that go unreported, indicating that the ARD is not a census.
A first step to increasing the coverage of ARD is to define a consistent data collection approach
across states, as opposed to the variable SRC-driven data collection methodology employed in
2011. More guidance or even requirements for where the SRC is located, how cases are identified,
and verification processes may increase the reliability and coverage of the program. However, such
an approach would increase the burden of this voluntary program, and would need to be balanced
with the utility of the data for data providers and stakeholders at both the national and the local
level.
The analyses described above only examined one alternative source for identifying law
enforcement homicides. BJS may further study other potential alternatives to the SRC-focused data
collection methodology to determine if any provide more cost-effective or more valid and reliable
means of identifying arrest-related deaths, including primary data collection or secondary analyses
of existing data sources.
Primary data collection methods such as direct surveys of law enforcement agencies or medical
examiner’s/coroner’s offices could identify law enforcement homicides and the circumstances
surrounding those homicides in the jurisdictions surveyed. Direct surveys would involve an
additional burden than the ARD program’s SRC-focused approach through the cost of fielding the
study and the time required to collect the data, as well as the burden on survey respondents. If the
survey has a sufficient response rate and demonstrated validity, however, it may be an accurate
measure of law enforcement homicides in the United States.

Page 33

Less burdensome methodologies could include data collection from open sources. Open sources
are publicly available data compiled through targeted searches, such as Google Alerts and review
of specific websites devoted to law enforcement homicides. Open sources also include public
documents such as press releases from states’ attorney general offices, medical examiners, or state
police agencies. For example, the New York City Police Department releases an annual report
describing all law enforcement homicides in that jurisdiction. Finally, fee-based services such as
www.webclipping.com can be used to identify information from a large number of media outlets.
Reliance on open source methods, while cost-effective, has a number of limitations. For example,
media sources are often limited to “newsworthy” deaths and may be biased toward higher profile
cases. Furthermore, there may be considerable regional and local variation in media coverage for
law enforcement homicide-related events. Google Alerts and web-clipping services are also
constrained by the search terms provided, which must be balanced to include all potentially eligible
law enforcement homicides while not resulting in an unmanageable number of “alerts” to review.
The timeliness of open source searches presents another limitation, because information on webbased news platforms may be available for a limited time. Most importantly, however, the quality
of information from open sources is unknown, and the accuracy, reliability, and type of content
provided is likely to vary over time, by source, and in comparison to official records.
Other alternative measures of law enforcement homicides include existing data maintained by the
Federal Bureau of Investigation (FBI) and the Centers for Disease Control and Prevention (CDC).
Supplementary Homicide Reports (SHRs) are collected nationally as part of the FBI’s Uniform
Crime Reporting Program (UCR), where information is voluntarily provided by law enforcement
agencies to their state UCR programs and then to the FBI. The SHR has the potential to identify or
confirm ARD cases categorized as a “homicide by law enforcement officer(s).” Other publicly
available data sources that may also measure law enforcement homicides include the FBI’s
National Incident Based Reporting System (NIBRS), and CDC’s National Vital Statistics System
(NVSS) and its National Violent Death Reporting System (NVDRS) Each of these data sources
has known limitations. For example, data from NIBRS, NVDRS, and local law enforcement
agency websites are not national in scope, and are available only for certain jurisdictions. Data
from the SHR and NVSS are national in scope, but do not contain unique identifiers that would
allow precise case-level matching and comparison with the cases submitted to the ARD program.
Studies have further documented inconsistencies in reporting to the ARD, SHR, and NVSS both
within and across states (Loftin et al., 2003; Mumola, 2007; RTI International, 2012). The ability
of any of these systems to accurately capture the universe of all arrest-related deaths as defined by
the ARD program, or law enforcement homicides in particular, is currently unknown.

4.3 Study limitations
The findings described above have several limitations, some of which may be addressed in
additional analyses as indicated by BJS. First, the study is limited by the approach to including all
probable and certainty matches in the analyses reported in exhibits 8, 10, 12, and 14. BJS may
choose to limit the cases identified as a match to those with a probability score cutoff of 0.50 or
higher, or to limit them to certainty matches only. The analyses presented here represent the “best
case scenario” for ARD; any assumption changes will lower the number and proportion of
observed cases that overlap across the two systems and increase the number and proportion of
observed cases found in one system only.

Page 34

The most significant limitation of this study, however, is measurement discrepancies between the
SHR and ARD on matching variables, and the lack of a unique identifier to positively link cases
across the systems. The monte carlo replicate approach implemented here provides the most
rigorous means of addressing this limitation, but the shortcomings in the matching variable
measurement remain. Findings were presented here as estimates of “cases,” but all estimates are
based on the replicate approach, so any conclusions about the actual number of cases matched or
covered by either system are not warranted. Instead, we can only estimate the level of matching
and overlap across the two systems based on the information provided.

Page 35

References
Boriah, S., Chandola, V., & Kumar, V. 2008. Similarity measures for categorical data: A
comparative evaluation. In proceedings of the eighth SIAM International Conference on
Data Mining. 243–254.
Loftin, C., Wiersema, B., McDowall, D. & Dobrin, A. (2003). Underreporting of justifiable
homicides committed by police officers in the United States, 1976–1998. American
Journal of Public Health, 93(7), 1117–1121.
Mumola, C.J. (2007). Arrest-related deaths in the United States, 2003–2005. Bureau of Justice
Statistics Special Report. (NCJ 219534). Washington, DC: U.S. Department of Justice,
Office of Justice Programs. Retrieved from
http://bjs.ojp.usdoj.gov/index.cfm?ty=pbdetail&iid=379.
RTI International. (May 2012). Arrest-related deaths program: Program assessment, alternative
approaches to data collection, and recommendations for program enhancement. Draft
report submitted to the Bureau of Justice Statistics. Washington, DC: U.S. Department of
Justice, Office of Justice Programs.
Wolter, K.M. (1986). Some coverage error models for census data. Journal of the American
Statistical Association, 81 (394): 338–346.

Page 37

Appendix A: Similarity scoring for ARD/SHR matching
Introduction
Calculation of similarity scores in the context of matching data from the Arrest-Related Deaths
(ARD) program with the Supplemental Homicide Reports (SHRs) requires a method that
accounts for both continuous and categorical variables. Cases will be matched between data
sources and within the originating agency identifier (ORI) code and victim sex on three
measures: date of incident/date of death, age, and race/Hispanic origin, with the first two
measures being continuous and the latter categorical. Given this mix of variables, RTI proposes
calculating scores using a mixture of methods as follows.

Continuous variables
For the date and age variables, we propose use of a standardized Euclidean distance, combined
with a similarity transformation.
Let
𝑁𝑁𝑆𝑆𝑆𝑆𝑆𝑆 = number of records in SHR (within a given year)

𝑁𝑁𝐴𝐴𝐴𝐴𝐴𝐴 = number of records in ARD (within a given year)

𝑖𝑖

𝑗𝑗

= SHR record index where 𝑖𝑖 ∈ [1, 𝑁𝑁𝑆𝑆𝑆𝑆𝑆𝑆 ]

= ARD record index where 𝑗𝑗 ∈ [1, 𝑁𝑁𝐴𝐴𝐴𝐴𝐴𝐴 ]

𝑑𝑑𝑆𝑆𝑆𝑆𝑆𝑆,𝑖𝑖 = SHR date (measured in number of days since January 1, 1960 5) for record i
𝑑𝑑𝐴𝐴𝐴𝐴𝐴𝐴,𝑗𝑗 = ARD date (measured in number of days since January 1, 1960) for record j

𝑎𝑎𝑆𝑆𝑆𝑆𝑆𝑆,𝑖𝑖 = SHR age for record i

𝑎𝑎𝐴𝐴𝐴𝐴𝐴𝐴,𝑗𝑗 = ARD age for record j
𝑑𝑑
−𝑚𝑚𝑒𝑒𝑒𝑒𝑒𝑒(𝑑𝑑
)
𝑑𝑑́𝑆𝑆𝑆𝑆𝑆𝑆,𝑖𝑖 = 𝑆𝑆𝑆𝑆𝑆𝑆,𝑖𝑖𝑠𝑠𝑠𝑠𝑠𝑠(𝑑𝑑 ) 𝐴𝐴𝐴𝐴𝐴𝐴
𝐴𝐴𝐴𝐴𝐴𝐴

𝑑𝑑
−𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚(𝑑𝑑𝐴𝐴𝐴𝐴𝐴𝐴 )
𝑑𝑑́𝐴𝐴𝐴𝐴𝐴𝐴,𝑗𝑗 = 𝐴𝐴𝐴𝐴𝐴𝐴,𝑗𝑗
)
𝑠𝑠𝑠𝑠𝑠𝑠(𝑑𝑑𝐴𝐴𝐴𝐴𝐴𝐴

5

The midpoint of the reported SHR month will be used to calculate the number of days since January 1, 1960, the
standard date convention used by SAS statistical software.

Page 38

𝑎𝑎́ 𝑆𝑆𝑆𝑆𝑆𝑆,𝑖𝑖 =
𝑎𝑎́ 𝐴𝐴𝐴𝐴𝐴𝐴,𝑗𝑗 =

𝑎𝑎𝑆𝑆𝑆𝑆𝑆𝑆,𝑖𝑖 −𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚(𝑎𝑎𝐴𝐴𝐴𝐴𝐴𝐴 )
𝑠𝑠𝑠𝑠𝑠𝑠(𝑎𝑎𝐴𝐴𝐴𝐴𝐴𝐴 )

𝑎𝑎𝐴𝐴𝐴𝐴𝐴𝐴,𝑗𝑗 −𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚(𝑎𝑎𝐴𝐴𝐴𝐴𝐴𝐴 )
𝑠𝑠𝑡𝑡𝑡𝑡(𝑎𝑎𝐴𝐴𝐴𝐴𝐴𝐴 )

Then the standardized Euclidean distance between records i and j on date and age is given by
2
2
𝐷𝐷𝑖𝑖,𝑗𝑗 = ��𝑑𝑑́𝑆𝑆𝑆𝑆𝑆𝑆,𝑖𝑖 − 𝑑𝑑́𝐴𝐴𝐴𝐴𝐴𝐴,𝑗𝑗 � + �𝑎𝑎́ 𝑆𝑆𝑆𝑆𝑆𝑆,𝑖𝑖 − 𝑎𝑎́ 𝐴𝐴𝐴𝐴𝐴𝐴,𝑗𝑗 �

This distance is then transformed to a similarity score as
𝑆𝑆𝑐𝑐𝑐𝑐𝑐𝑐;𝑖𝑖,𝑗𝑗 =
where

1
1 + 𝐷𝐷𝑖𝑖,𝑗𝑗

0 < 𝑆𝑆𝑐𝑐𝑐𝑐𝑐𝑐;𝑖𝑖,𝑗𝑗 ≤ 1

Categorical variables
Rather than simply counting matches on categorical variables as 1 and mismatches as 0, we
propose the use of a penalized scoring approach similar to that described in Boriah et al., 2008. 6
This approach is based on the premise that matches that have a high probability of occurring by
random chance should not count as much as those that would be observed very rarely by chance
alone. For each categorical variable (currently only race/Hispanic origin), the similarity between
two records i and j is given by the following.
Let
𝑝𝑝̂ (𝑥𝑥)

= the sample probability that 𝑋𝑋 = 𝑥𝑥

Then the similarity between records i and j on variable X is given by
1 − 𝑝𝑝̂ (𝑥𝑥𝑆𝑆𝑆𝑆𝑆𝑆 )𝑝𝑝̂ (𝑥𝑥𝐴𝐴𝐴𝐴𝐴𝐴 ) if 𝑥𝑥𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑥𝑥𝐴𝐴𝐴𝐴𝐴𝐴
𝑆𝑆𝑋𝑋 = �
0
otherwise
0 ≤ 𝑆𝑆𝑋𝑋 < 1

6

The proposed method is similar to the Goodall3 approach described by Boriah et al., but adapted for two data
sources.

Page 39

Calculating the overall score
The final similarity score for records i and j is the weighted mean of the various components and
is given by the following:
2

1

𝑆𝑆𝑖𝑖,𝑗𝑗 = 3 𝑆𝑆𝑐𝑐𝑐𝑐𝑐𝑐;𝑖𝑖,𝑗𝑗 + 3 𝑆𝑆𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅/𝐸𝐸𝐸𝐸𝐸𝐸;𝑖𝑖,𝑗𝑗

Page 40

Appendix B: Linking algorithm
Estimate the agency population size and agency undercoverage adjustment
factor
To ensure that coverage estimates apply to the population of law enforcement (LE) homicides
across all agencies and not only to the population of LE homicides occurring in agencies that
have been observed, we must estimate the total population size of agencies with LE homicides
(reported or unreported). To achieve this, we will use the Lincoln-Petersen (LP) estimator 7 as
follows:
(1)

� = (𝑛𝑛𝐴𝐴𝐴𝐴𝐴𝐴 +1)(𝑛𝑛𝑆𝑆𝑆𝑆𝑆𝑆 +1) − 1
𝑁𝑁
𝑛𝑛
+1
𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆

In (1), nARD is the number of agencies in ARD, nSHR is the number of agencies in SHR, and
� is given by
nARD||SHR is the number of agencies represented in both sources. The variance of N
the following:
�� =
(2) 𝑉𝑉𝑉𝑉𝑉𝑉�𝑁𝑁

�𝑛𝑛𝐴𝐴𝐴𝐴𝐴𝐴 +1��𝑛𝑛𝑆𝑆𝑆𝑆𝑆𝑆 +1��𝑛𝑛𝐴𝐴𝐴𝐴𝐴𝐴 −𝑛𝑛𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆 ��𝑛𝑛𝑆𝑆𝑆𝑆𝑆𝑆 −𝑛𝑛𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆 �
2
�𝑛𝑛𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆 +2�
𝑛𝑛𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆

The number of agencies observed across ARD and SHR is given by
𝑛𝑛 = 𝑛𝑛𝐴𝐴𝐴𝐴𝐴𝐴 + 𝑛𝑛𝑆𝑆𝐻𝐻𝐻𝐻 − 𝑛𝑛𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆

and the agency undercoverage adjustment factor is given by
𝑅𝑅 =

�
𝑁𝑁
𝑛𝑛

and its variance is given by
𝑉𝑉𝑉𝑉𝑉𝑉(𝑅𝑅) =

�)
𝑉𝑉𝑉𝑉𝑉𝑉(𝑁𝑁
𝑛𝑛2

Classify cases as matchable or unmatchable
For the purposes of record linkage, cases are separated into two groups determined by whether or
not it is possible to match them between sources, within characteristic classifications that must
be the same: originating agency identifier (ORI) code, sex, and year. Cases from either source

7
By incorporating agency-level characteristics into a stratified LP estimate, we may reduce bias induced by the
assumption that agencies are observed with constant probability across agency characteristics.

Page 41

that cannot be matched on these three characteristics are considered unmatchable and are
excluded from subsequent probabilistic linking.

Probabilistic linking between ARD and SHR
For cases considered matchable across sources, probabilistic linking is used to determine
whether a case in the Arrest-Related Deaths (ARD) program is the same as a case in the
Supplementary Homicide Reports (SHRs). Within each distinct combination of ORI, sex, and
year, probabilistic linking proceeds according to the following steps:
a. Randomly sort the ARD cases
b. For each ARD case i, calculate a similarity score, 𝑆𝑆𝑖𝑖,𝑗𝑗 , corresponding to each SHR
case j

c. Create a 0/1 indicator, 𝑢𝑢𝑖𝑖,𝑗𝑗 , indicating whether SHR case j has already been linked
to another ARD record
′
d. Define 𝑆𝑆𝑖𝑖,𝑗𝑗
= 𝑆𝑆𝑖𝑖,𝑗𝑗 ∗ (1 − 𝑢𝑢𝑖𝑖,𝑗𝑗 )

e. Create a 0/1 indicator, 𝑡𝑡𝑖𝑖,𝑗𝑗 , for whether or not each SHR case is within acceptable
threshold for being considered the same as the ARD case in question (an SHR
case is considered within threshold if the difference in dates of incident and death
is less than or equal to 30 days, the ages are less than or equal to 5 years apart,
and the race/Hispanic origin matches exactly); Note that if 𝑢𝑢𝑖𝑖,𝑗𝑗 = 1 then 𝑡𝑡𝑖𝑖,𝑗𝑗 = 0
f. Treating the scores as unscaled probabilities, calculate the probability that the
ARD case is not represented in SHR as follows
𝑘𝑘

′
𝑆𝑆𝑖𝑖,𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 = ��1 − 𝑆𝑆𝑖𝑖,𝑗𝑗
� (1 − 𝑡𝑡𝑖𝑖,𝑗𝑗 )
𝑗𝑗=1

g. Determine the number of cases that are within threshold as

𝑘𝑘

𝑛𝑛𝑡𝑡 = � 𝑡𝑡𝑖𝑖,𝑗𝑗
𝑗𝑗=1

′′
′
h. If 𝑛𝑛𝑡𝑡 > 0 then define 𝑆𝑆𝑖𝑖,𝑗𝑗
= 𝑆𝑆𝑖𝑖,𝑗𝑗
∗ 𝑡𝑡𝑖𝑖,𝑗𝑗

′′
′
i. Otherwise, if 𝑛𝑛𝑡𝑡 = 0, then define 𝑆𝑆𝑖𝑖,𝑗𝑗
= 𝑆𝑆𝑖𝑖,𝑗𝑗

𝑝𝑝𝑖𝑖,𝑗𝑗 =

j. Define 𝑝𝑝𝑖𝑖,𝑗𝑗 , the probability that ARD case i is the same as SHR case j as
′′
𝑆𝑆𝑖𝑖,𝑗𝑗

𝑗𝑗

′′
𝑆𝑆𝑖𝑖,𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 + ∑1 𝑆𝑆𝑖𝑖,𝑗𝑗

k. Define the probability of no match as

Page 42

𝑝𝑝𝑖𝑖,𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 =

𝑆𝑆𝑖𝑖,𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛

𝑗𝑗

′′
𝑆𝑆𝑖𝑖,𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 + ∑1 𝑆𝑆𝑖𝑖,𝑗𝑗

l. Define the cumulative probability of record linkage as

𝑗𝑗

𝑐𝑐𝑖𝑖,𝑗𝑗 = � 𝑝𝑝𝑖𝑖,𝑞𝑞
𝑞𝑞=1

m. Select a random variate 𝑥𝑥𝑖𝑖 , from 𝑋𝑋~𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈(0,1)

n. Link ARD case i with SHR case j if case j is the first case (going from 1 to k) such
that 𝑐𝑐𝑖𝑖,𝑗𝑗 > 𝑥𝑥𝑖𝑖

o. If this condition is not satisfied, ARD case i will be linked to no SHR cases with
probability 𝑝𝑝𝑖𝑖,𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛

Recombine cases

Once matchable cases have gone through the probabilistic linking process, they are recombined
with cases previously determined to be unmatchable due to a lack of overlap on key variables
(ORI, sex, and year). Once recombined, the number of individual homicides in ARD, in SHR,
and in both ARD and SHR are tabulated; these are denoted as 𝑚𝑚𝐴𝐴𝐴𝐴𝐴𝐴 , 𝑚𝑚𝑆𝑆𝑆𝑆𝑆𝑆 , and 𝑚𝑚𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆 ,
respectively.

Stratify by individual-level characteristics

Since homogeneity of capture probabilities cannot be assumed within or across lists, the LP
estimator of population size may contain downward bias, leading to an overestimation of
coverage (Wolter, 1986). A common method for amelioration of this bias—and the one we
employ—is poststratification. Once record linkage has occurred, cases will be grouped into 𝑠𝑠
strata, defined by the cross-classification of demographic characteristics. If these characteristics
are correlated with capture probability, bias incurred through use of the LP estimator will be
reduced – the choice of characteristics is limited to information available in both sources (age,
race/Hispanic origin, and sex).

Estimate population size and variance by stratum

Within each stratum 𝑙𝑙, with 1 ≤ 𝑙𝑙 ≤ 𝑠𝑠, obtain the LP estimate of population size and its
associated variance as
(3)

�𝑙𝑙 = �𝑚𝑚𝑙𝑙,𝐴𝐴𝐴𝐴𝐷𝐷 +1�(𝑚𝑚𝑙𝑙,𝑆𝑆𝑆𝑆𝑆𝑆 +1) − 1
𝑀𝑀
𝑚𝑚
+1
𝑙𝑙,𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆

� 𝑙𝑙 is given by the following:
The variance of M
� 𝑙𝑙 � =
(4) 𝑉𝑉𝑉𝑉𝑉𝑉�M

�𝑚𝑚𝑙𝑙,𝐴𝐴𝐴𝐴𝐴𝐴 +1��𝑚𝑚𝑙𝑙,𝑆𝑆𝑆𝑆𝑆𝑆 +1��𝑚𝑚𝑙𝑙,𝐴𝐴𝐴𝐴𝐴𝐴 −𝑚𝑚𝑙𝑙,𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆 ��𝑚𝑚𝑙𝑙,𝑆𝑆𝑆𝑆𝑆𝑆 −𝑚𝑚𝑙𝑙,𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆 �
𝑚𝑚2𝑙𝑙,𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝐻𝐻𝑅𝑅 �𝑚𝑚𝑙𝑙,𝐴𝐴𝐴𝐴𝐴𝐴||𝑆𝑆𝑆𝑆𝑆𝑆 +2�

Page 43

Estimate population size
By summing the stratum-specific population sizes, we obtain the overall population size among
observed agencies:
𝑠𝑠

� = � 𝑀𝑀
�𝑙𝑙
𝑀𝑀
𝑙𝑙=1

� is given by the following:
The variance of 𝑀𝑀
𝑠𝑠

� � = � 𝑉𝑉𝑉𝑉𝑉𝑉(𝑀𝑀
�𝑙𝑙 )
𝑉𝑉𝑉𝑉𝑉𝑉�𝑀𝑀
𝑙𝑙=1

Adjust population size for agency undercoverage

To obtain an estimate of the overall population size, 𝑃𝑃�, among all agencies with LE homicides,
including those unobserved across lists, we apply the ratio from 1 as follows:
�
𝑃𝑃� = 𝑅𝑅 ∗ 𝑀𝑀

The variance of 𝑃𝑃� is given by the following:

� 2 𝑉𝑉𝑉𝑉𝑉𝑉(𝑅𝑅) + 𝑅𝑅 2 𝑉𝑉𝑉𝑉𝑉𝑉�𝑀𝑀
��
𝑉𝑉𝑉𝑉𝑉𝑉�𝑃𝑃�� = 𝑀𝑀

Estimate ARD coverage

ARD coverage is estimated as the number of cases observed in ARD, divided by the estimated
population size:
(5)

𝐶𝐶 =

𝑚𝑚𝐴𝐴𝐴𝐴𝐷𝐷
𝑃𝑃�

The variance of 𝐶𝐶 is given by the following:
(6)

𝐶𝐶 2

𝑉𝑉𝑉𝑉𝑉𝑉(𝐶𝐶) = 𝑃𝑃�2 ∗ 𝑉𝑉𝑉𝑉𝑉𝑉(𝑃𝑃�)

Replication

To achieve final estimates of coverage and variance that take into account uncertainty inherent to
the probabilistic record linkage, the process above is repeated 1,000 times. In each of these
replicates the linking procedure yields different results, and this variability is incorporated as
follows:
Let 𝑟𝑟 be the replicate index, where 1 ≤ 𝑟𝑟 ≤ 1,000, then the overall population size estimate is
given by
1,000

1
𝑃𝑃� =
� 𝑃𝑃�𝑟𝑟
1,000
𝑟𝑟=1

The variance of 𝑃𝑃� is given by the following:

Page 44

1,000

1,000

𝑟𝑟=1

𝑟𝑟=1

1
1
𝑉𝑉𝑉𝑉𝑉𝑉�𝑃𝑃�� =
� � 𝑉𝑉𝑉𝑉𝑉𝑉�𝑃𝑃�𝑟𝑟 �� +
� � (𝑃𝑃�𝑟𝑟 − 𝑃𝑃�)2 �
1,000
999

The final coverage estimate and variance are obtained by replacing 𝑃𝑃� and 𝑉𝑉𝑉𝑉𝑉𝑉(𝑃𝑃�) with the above
in equations (5) and (6), respectively.

Page 45

Appendix C: Race/Hispanic origin coding
Before matching and capture-recapture analyses may begin, RTI must recode race/Hispanic
origin to be comparable between the Arrest-Related Deaths (ARD) program and Supplementary
Homicide Report (SHR) data sources. This requirement presents a problem in that race/Hispanic
origin is collected in a combined measure for the ARD and in two separate variables for the
SHR. Since separating the combined ARD measure into two variables comparable to those that
exist in the SHR would yield a missing race for all records currently coded as Hispanic, it is
much more desirable to create a combined measure on the SHR file that is comparable to that in
the ARD data. Given high levels of missingness in the SHR Hispanic origin measure, this
approach is also not without problems.

Hispanic origin in the SHR
At the national level, Hispanic origin in the SHR is highly underreported, with a missingness rate
of 53% in the years 2003–11.
Cumulative
V22

Frequency Percent Frequency
2051

53.44

(H) Hispanic origin

2051
600

(N) Not of Hispanic origin
(U) Unknown or not reported

15.63
1101
86

2651
28.69
2.24

3752
3838

To examine whether SHR respondents may be using the Hispanic origin variable as an extra race
category, when needed (i.e., when the deceased was obviously Hispanic), we explored the
hypothesis that if the Hispanic origin responses that were not missing were mostly 1, it would be
a strong indication that this was true. As shown in the frequency above, this is not the case. So,
we dug a little deeper to see where the missing Hispanic origin cases were coming from, with the
following result:
State N Reported Hisp.

CALIF 993 0.9073515

ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ

COLO 80

0

ALA

CONN

0

10

0

ALASKA 18

DC

17

0

ARIZ 177 0.8983051

DEL

1

0

ARK

FL

373

0

16

0

4

0

Page 46

GA

82 0.4512195

HAWAII 8

0

IDAHO 3

0

N MEX 46
NY

0

53

NEBR

0

11 0.1818182

ILL 137

0

NEV

79

0

IND

0

OHIO 44

0

50

IOWA 17

0

OKLA 92 0.5978261

KANS

0

OREG 33 0.6666667

14

KY

19

0

PA

185 0.9621622

LA

65

0

RI

1

SC

20

MAINE 18
MASS
MD

0

11

0

169

0

0
0

S DAK 7

0

TENN 73

0

MICH 78 0.1153846

TEXAS 320 0.8937500

MINN 31

UTAH 23

MISS
MO

2

0

VI

3

103 0.6116505

VA

45

VT

3

MONT
NC

0

7

0

42 0.7619048

N DAK 1
NH

2

NJ

102

0
0
0

W VA

0
0
0
0

2

0

WASH 98 0.2346939
WIS
WYO

Page 47

47 0.4255319
3

0

Figure 7: ACS 5-year Hispanic origin, by county

Highlighted in the table at left are the states where we would expect to find the largest Hispanic
populations. Of the 2,068 cases occurring in these states, 72% come from states with high
Hispanic origin coverage. This implies that the situation may not be as bad at the national level
as it might seem based only on the frequency above.
Assuming we were to create a combined race/Hispanic origin variable for the SHR, the
distribution of white, black, and Hispanic nationally in 2003–09 would be 45.89%, 33.89%, and
16.28%, respectively. In the ARD, the corresponding breakdown is 42.06%, 31.03%, and
20.36%.
If we subset to states with reporting rates for Hispanic origin greater than 75% (Arizona,
California, North Carolina, Pennsylvania, and Texas), the SHR distribution of white, black, and
Hispanic becomes 35.54%, 25.20%, and 34.01%, respectively. The corresponding breakdown in
the ARD, when subset to the same states, is 32.38%, 26.39%, and 34.70%.
Given these results, we follow an approach in which we create a combined race/Hispanic origin
measure in the SHR and conduct both a national analysis as well as one subset to the states with
high Hispanic origin reporting. Using only these five states would cover half of our national
sample and would give us some insight into how sensitive the capture-recapture results are to the
issue of data regarding Hispanic origin.

Page 48

The Bureau of Justice Statistics of the U.S. Department of Justice is
the principal federal agency responsible for measuring crime, criminal
victimization, criminal offenders, victims of crime, correlates of crime,
and the operation of criminal and civil justice systems at the federal, state,
tribal, and local levels. BJS collects, analyzes, and disseminates reliable and
valid statistics on crime and justice systems in the United States, supports
improvements to state and local criminal justice information systems, and
participates with national and international organizations to develop and
recommend national standards for justice statistics. William J. Sabol is
acting director.
This report was written by Duren Banks, Lance Couzens, Caroline Blanton,
and Devon Cribb, RTI International. Erin Kennedy, Crystal Daye, Lance
G. Couzens, Devon S. Cribb, Caroline Blanton, and Duren Banks, RTI
International provided statistical review and verification of the report.
Lynne McConnell and Jill Thomas edited the report. Barbara Quinn produced
the report.
March 2015, NCJ 248543

Office of Justice Programs
Innovation • Partnerships • Safer Neighborhoods
www.ojp.usdoj.gov