Economic Evaluation of Svori Reentry Initiative Dec 2009
Download original document:
This text is machine-read, and may contain errors. Check the original document to verify accuracy.
An Economic Evaluation of the Serious and Violent Offender Reentry Initiative THE MULTI-SITE EVALUATION OF THE SERIOUS AND VIOLENT OFFENDER REENTRY INITIATIVE December 2009 Alexander J. Cowell Pamela K. Lattimore RTI International 3040 East Cornwallis Road Research Triangle Park, NC 27709 Phone: (919) 541-8754 Fax: (919) 541-5985 Cowell@rti.org RTI International 3040 East Cornwallis Road Research Triangle Park, NC 27709 Phone: (919) 485-7759 Fax: (919) 541-5985 Lattimore@rti.org John Roman The Urban Institute 2100 M Street, N.W. Washington, DC 20037 Phone: (202) 261-5774 Jroman@urban.org This project was supported by Grant No. 2004-RE-CX-002 awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. Points of view in this document are those of the authors and do not necessarily represent the official position or policies of the U.S. Department of Justice. Acknowledgments The Multi-site Evaluation of the Serious and Violent Offender Reentry Initiative (SVORI) was supported by grant number 2004-RE-CX-002 from the National Institute of Justice (U.S. Department of Justice) and was conducted by RTI International and the Urban Institute. Points of view are those of the authors and do not necessarily represent those of the U.S. Department of Justice. Principal Investigators Pamela K. Lattimore, RTI International Christy A. Visher, University of Delaware and Urban Institute Report Authors Alexander J. Cowell, RTI International John Roman, Urban Institute Pamela K. Lattimore, RTI International Staff Contributors Christy A. Visher, University of Delaware and Urban Institute Brendan J. Wedehase, RTI International Aaron Sundquist, Urban Institute Carly Knight, Urban Institute Michael Mills, RTI International We also acknowledge the contributions of the site liaisons from RTI and the Urban Institute, who documented the implementation of SVORI programming across the sites and facilitated data collection for the impact study. In addition, we are grateful for the hard work and dedication shown by our field interviewers, supervisors, and data collection task leader throughout the data collection period. RTI and the Urban Institute thank the SVORI project directors, other program and research staff from the SVORI sites, and iii staff at the facilities where interviews were conducted. We greatly appreciate the assistance and support received from these individuals. Finally, RTI and the Urban Institute acknowledge the invaluable assistance and direction provided by the members of our external advisory group. For more information about the SVORI Multi-site Evaluation, please visit our Web site at http://www.svori-evaluation.org/. iv Abstract Statement of Purpose The Serious and Violent Offender Reentry Initiative (SVORI) funded agencies in 2003 to develop programs to improve criminal justice, employment, education, health, and housing outcomes for released prisoners. Sixty-nine agencies received federal funds to develop 89 programs. The SVORI multi-site evaluation was funded by the National Institute of Justice in the spring of 2003. Sixteen programs (12 adult and 4 juvenile programs) were included in an impact evaluation to determine the effectiveness of the programming provided under SVORI. Nearly 2,400 returning prisoners were interviewed during the course of the evaluation. This report presents a detailed analysis of costs of programming before release and a costbenefit analysis comprising costs and benefits from both before and after release. Research Subjects The sample includes 722 men from 4 of the 12 adult programs and 79 juvenile males from 1 of the 4 juvenile programs. Each program had a SVORI sample and a comparison sample. The 4 adult programs were selected based on meeting at least one of three criteria: a large study size, a good comparison match or random assignment to study condition, and likely availability of administrative records. The juvenile site had a large number of study participants and reliable cost data available. Study Methods A self-report survey provided data on service receipt, rearrest, and reincarceration. Survey waves were administered approximately 1 month before release, and 3, 9, and 15 months after release. The literature and site-specific records provided data on the cost of each service, arrest, and day v An Economic Evaluation of the Serious and Violent Offender Reentry Initiative incarcerated. The survey and cost data were combined for the analyses. Propensity score weights were applied to improve the comparability of the SVORI and comparison groups and the results of the weighted analyses are reported. Major Findings Estimates on pre-release service costs varied considerably from a low of an additional SVORI cost of $658 per offender in Pennsylvania to $3,480 in the South Carolina juvenile program. The cost-benefit results confirm that enhanced reentry through SVORI resulted in more resources being spent on services to these offenders. For men, the greater service cost persisted throughout all three follow-up waves. For South Carolina juveniles, greater service costs were seen before and 3 months after release. Relative to the comparison group, criminal justice costs for the SVORI group were lower 3 months after release, but higher 9 and 15 months after release; importantly, none of these findings were statistically significant at conventional levels. Estimates of net costs (service costs combined with criminal justice costs) had large confidence intervals and there was no evidence that net costs for the SVORI group were higher or lower than the comparison group. Conclusions The SVORI economic evaluation indicated that enhanced reentry was successful in delivering services to offenders, both before and after release into the community. The cost-benefit analysis was inconclusive and could not determine whether net costs for the SVORI group were higher or lower than the comparison group. Given the richness of the SVORI evaluation data, further work assessing the impact on criminal justice costs may be rewarding. These future directions will help address gaps in current knowledge. First, future work is needed to further examine the degree to which enhanced reentry programming may be associated with possible reductions in criminal justice costs. This may be perhaps most successfully done by using administrative rather than survey data on arrests and reincarceration and expanding the analysis to all of the 16 sites used in the main evaluation for which reliable data are vi Abstract available. A second likely fruitful area is to expand the kinds of benefits examined to include outcomes such as employment. vii Contents Section Abstract Executive Summary Introduction Page v ES-1 1 The Purpose of This Report ......................................... 1 Literature on Reentry Costs and Benefits ...................... 3 Rationale ................................................................. 8 Methodology 9 Overview of the Economic Evaluation ........................... 9 Sample ...................................................................10 Data .......................................................................13 Analysis ..................................................................22 Results 31 The Cost of Discretionary Pre-release Services..............31 Pre-release Cost Estimates by Domain.........................34 Sensitivity Analyses of Pre-release Cost Estimates ........37 Cost-Benefit Analysis Results .....................................37 Summary of Results .................................................44 Conclusions 47 Discussion of Findings ...............................................47 Implications for Policy and Practice .............................48 Implications for Future Research ................................52 References 55 Appendix A: Program Descriptions A-1 Appendix B: Potential Sources of Duplication Bias B-1 ix Exhibits Number 1. Page Criteria and Outcomes of the Site Selection Process for the Economic Evaluation 11 Relative Merits of Alternative Approaches to Measuring Quantity 14 3. Administrative Data Availability, by Site 15 4. Sources for Quantity of Services Received 17 5. Sources for Prices of Services Received 17 6. Estimates and Data Sources of Arrest Costs (2007$) 21 7. Estimates and Data Sources of Incarceration Costs per Offender per Night (2007$) 22 8. Five Incremental Steps of the Pre-release Services Cost Estimation Protocol 23 9. Conducting CBA: Estimating Net Costs 27 2. 10. Four Incremental Steps of the Cost-Benefit Analysis Estimation Protocol 29 11. Annual Incremental Pre-release Costs of Service Provision per Offender (2007$) 31 12. Annual Pre-release Incremental Cost Estimates of Discretionary Service Provision and Bounds from Sensitivity Analyses (2007$) 33 13. Annual Pre-release Cost of Service Provision to Men, by Domain (2007$) 35 14. Annual Pre-release Cost of Service Provision to South Carolina Juveniles, by Domain (2007$) 36 15. Number of Services Included in the Cost Analysis, by Site and Domain 36 16. Results of Sensitivity Analyses on Pre-release Costs (2007$) 38 17. Main CBA Findings for the Adult Programs: Monthly Net Cost per Offender, by Study Period (2007$) 39 xi xii 18. Sensitivity Analyses for Cost-Benefit Analysis of Four Sites for Adult Male Offenders (2007$) 42 19. Main CBA Findings for the South Carolina Juvenile Program: Monthly Net Cost per Offender, by Study Period (2007$) 43 20. Sensitivity Analyses for the Cost-Benefit Analysis of the South Carolina Juvenile Site (2007$) 45 Executive Summary Decision makers who are in charge of planning and funding prisoner reentry programs and policies need to know the cost of reentry services and the benefits that accrue to society as a result of these services. In 2003, the U.S. Departments of Justice, Labor, Housing and Urban Development, and Health and Human Services established the Serious and Violent Offender Reentry Initiative (SVORI), a large-scale funding stream that provided more than $100 million to 69 grantees to develop programming, training, and state-of-the-art reentry strategies at the community level. The multi-site evaluation of SVORI examined the initial implementation of 16 different reentry programming models in 2004–2005. The economic evaluation focused on offenders reentering society from five sites: programming for men in Iowa, Ohio, Pennsylvania, and South Carolina and programming for juvenile males in South Carolina. This report presents the results of an economic evaluation of the enhanced reentry efforts funded through SVORI. It provides the results from two separate analyses: Pre-release service cost analysis. A cost analysis was conducted of those services provided before prisoners were released into the community in 2004 and 2005. Estimates are provided at the site level and throughout the course of the 12 months immediately preceding release. Cost-benefit analysis. A cost-benefit analysis (CBA) was conducted of enhanced reentry, both pre- and postrelease. The analysis assessed the degree to which possible reductions in criminal justice costs (arrests plus incarceration) through enhanced reentry offset the likely increased service costs. For this analysis, the four sites ES-1 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative for men were combined, and average monthly cost estimates are presented. The findings draw on the most detailed and reliable data available, and use the same methods across all five sites. RESULTS Pre-release service costs vary considerably across sites. Estimates of the additional pre-release cost attributable to enhanced reentry varied from $658 per offender for the Pennsylvania adult program to $3,480 for the South Carolina juvenile program. Case management contributed largely to prerelease costs. The exception was in Iowa, where the difference was driven by employment/education/life skills services. The cost-benefit analysis results showed that at no point after release were the estimates of differences between the enhanced reentry and comparison groups statistically significant. Because the results were generally not statistically significant, there was no strong evidence either that the SVORI group cost more or that it cost less than the comparison group at each of the three follow-up periods. Enhanced reentry succeeded in providing offenders more services, both for men and juveniles. For men, the findings suggested that, even as long as 15 months after release, the average monthly service cost was $97 higher for those in the enhanced reentry group than in the comparison group. For juveniles, the findings suggested that service provision for enhanced reentry was particularly high within the first 3 months after release. The difference in average monthly costs for the enhanced reentry and the comparison groups was $330. CONCLUSIONS A limitation of the findings is that the estimates were not sufficiently precise to detect differences in overall criminal justice costs at any point in time after release. The imprecision in estimate may be the result of several factors; for example, the evaluation took place while initial efforts to enhance reentry were still underway and, thus, services may not have been sufficiently established to have an effect on criminal justice costs. Another limitation is that detailed information on the number and the specific type of service events was not available. A final limitation is that results may not generalize to ES-2 Executive Summary other jurisdictions. There was no reentry model to which grantees were expected to adhere. The data are for four sites for men and one for juvenile males, and each of the sites demonstrably delivered services differently. The SVORI economic evaluation indicated that enhanced reentry was successful in delivering services to offenders, both before and after release into the community. Given the richness of the SVORI evaluation data, further work assessing the impact on criminal justice costs may be particularly rewarding. These future directions will help address gaps in current knowledge. First, future work is needed to further examine the degree to which enhanced reentry may be associated with possible reductions in criminal justice costs. This may be perhaps most successfully done by using administrative rather than survey data on arrests and reincarceration and expanding the analysis to every one of the 16 sites used in the main evaluation for which reliable data are available. A second likely fruitful area is to expand the kinds of benefits examined to include outcomes such as employment. ES-3 Introduction THE PURPOSE OF THIS REPORT Decision makers who are in charge of planning and funding prisoner reentry programs and policies need to know the cost of reentry services and the benefits that accrue to society as a result of these services. In 2003, the U.S. Departments of Justice, Labor, Housing and Urban Development, and Health and Human Services established the Serious and Violent Offender Reentry Initiative (SVORI), a large-scale funding stream that provided more than $100 million to 69 grantees to develop programming, training, and state-of-the-art reentry strategies at the community level. The multi-site evaluation of SVORI examined 16 different reentry programming models, and the economic evaluation focused on five of these models to ascertain the impact, costs, and benefits of reentry programming. The study focused on offenders reentering society in 2004–2005 as part of the initial implementation of enhanced reentry. This report presents the results of an economic evaluation of the enhanced reentry efforts funded through SVORI. It provides the results from two separate analyses: Pre-release service cost analysis. A detailed cost analysis was conducted of those services provided before prisoners were released into the community in 2004 and 2005. Estimates are provided at the site level and throughout the course of the 12 months immediately preceding release. Cost-benefit analysis. A cost-benefit analysis (CBA) of enhanced reentry was conducted, both pre- and postrelease. Monthly estimates are provided, and estimates for all four adult sites are combined. 1 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative A key feature of the economic evaluation is that it leverages important strengths of the main outcome evaluation: Data are site specific. The economic evaluation follows the outcome evaluation in recognizing that sites differed greatly in the way in which SVORI funds were spent. Thus, information was collected from each site. Because of concerns about small sample sizes, when using statistical significance tests, estimates were aggregated across sites. The design is longitudinal. Offender services were tracked at several points in time, including before release into the community (pre-release) and once they were put back into the community (post-release). A comparison group of individuals received services as usual. So that the impact of SVORI funding on service utilization and outcomes could be fully understood, the experiences of participants receiving services funded through SVORI have been compared to those receiving services under reentry as usual. The measures are at the individual (rather than at the site) level. The most accurate estimates are those that account for person-to-person variation in service receipt and criminal justice events, rather than rely on broad, program-level averages. Pre-release Service Cost Analysis This cost analysis examined how resources were spent on offenders before entering the community from four adult sites— Iowa, Ohio, Pennsylvania, and South Carolina—and one juvenile site, South Carolina. Pre-release services are of particular interest to decision makers, such as those in departments of corrections, because they are largely within the direct control of grant recipients. After release, services that offenders receive are typically provided by a variety of providers, and decision makers often have less control over provision and access. Estimates are for the 12 months before release, that is, annual costs. Because most corrections agencies understand, at a broad level, the cost of standard reentry, the most practical advantage of a cost analysis is to understand the additional costs of enhanced reentry resources (Drummond, O'Brien, Stoddart, & Torrance, 2000). Thus, in addition to the cost of service provision per site, the analysis provided estimates on the costs per SVORI participant over and above a comparison 2 Introduction participant. These estimates are referred to as “incremental costs.” In addition to an estimated incremental cost for the program as a whole, the results also describe the incremental costs for specific domains of costs, such as educational services. Disaggregating the estimates in this manner is particularly useful because it reflects the way in which appropriations and budgets are disaggregated and recognizes that, beyond the SVORI grant, different components of reentry programs in the community may have different funding sources. Because of state-to-state variation, separate analyses for each of the five programs selected for the cost analysis were conducted. Cost-Benefit Analysis The CBA addressed the degree to which expenditures on services were offset by reductions in the key outcomes of rearrest and reincarceration. It accounts for the fact that services—before and after release—are designed to help offenders contribute positively to society. The analysis combined the data from the pre-release cost analysis with estimates of post-release costs, which included costs incurred by obtaining additional services and those incurred through rearrest and reincarceration. The analysis then compared, at key time points, average costs for two groups: those in enhanced reentry and those in a comparison group. Relative to the comparison group, the enhanced reentry group would likely have higher service costs but lower arrest and incarceration costs, and total costs overall would likely be lower. For this analysis, all adult sites’ data (the juvenile site data were analyzed separately) were combined. This is because limited sample size threatened the possibility of finding statistically significant results. It also was in keeping with the design of the main outcome evaluation, which combined all sites in its analyses. LITERATURE ON REENTRY COSTS AND BENEFITS The challenge to researchers investigating reentry policies and programs is that the term “reentry” does not refer to a bounded, defined mission, but rather to the period surrounding release from jail or prison when issues related to desistance may be addressed. A CBA of reentry is therefore a CBA of all 3 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative the social service programming associated with reentry, where the only commonality is the ultimate outcome of interest, desistance. Reentry programming as it is currently understood is nascent—the first broad federal funding of reentry programming began with the SVORI funding in 2003, and many, if not most, local reentry efforts began concurrently. A number of challenges exist to conducting useful economic impact analyses over and above the challenges of identifying the average treatment effect of the component programs. These challenges include costs that are often spread across numerous service providers, organizing entities that coordinate or facilitate linkages to services (but do not directly provide services), limited data measuring the amount (dosage) of services (inputs), and benefits that are diffuse and that may accrue to many parties not directly affected by the intervention. The preferred strategy for measuring the total costs of a criminal justice system–based intervention is the bottom-up approach. This approach estimates the cost of the intervention for each individual by adding up the cost of all the services and goods received by that individual. The total cost is then calculated by adding up the cost of all the individuals who participated in the intervention. One recent study tested the effect of the Maryland Reentry Partnership Initiative (REP) on crime using a cohort of prisoners released from the Maryland Transition Center in Baltimore, Maryland, between March 2001 and January 2005 (Roman, Brooks, Lagerson, Chalfin, & Tereshchenko, 2007; Western, 2006). Retrospective administrative data compared 229 REP clients to a contemporaneous cohort of 370 prisoners released from the Maryland Transition Center to neighborhoods in Baltimore City that were not in the REP catchment area. Outcomes were measured during the post-release period, which averaged 38 months. Fewer REP clients (72.0% compared to 77.6%) committed at least one new crime in the study period. There were no significant differences in time to rearrest, likelihood of a new conviction, number of new convictions, or time to a new conviction. The reduction in new offending was enough to return about $3 in benefits for every dollar in new costs. The total net benefit—total benefits minus total costs—to the citizens of Baltimore from the REP program is about $7.2 million, or about $21,500 per REP participant. 4 Introduction In practice, few evaluations follow this procedure because the CBA is an add-on to an impact evaluation. Data that are critical to the estimation of economic impacts (such as the quantity of services consumed by the comparison group) are often not collected, either because they are not relevant to the impact evaluation or because the reentry program links to so many different kinds of services that estimating costs and benefits is impractical. Most CBAs rely instead on top-down estimates of average cost using an accounting approach. Another limitation of current studies results from the dominance of relatively high-cost (or high-benefit) but rare events (such as an escape or a murder) on costs or benefits. In cases where outliers explain some or all of the difference in cost-effectiveness, those effects cannot be modeled using traditional statistical or econometric approaches. For example, a recent evaluation using a randomized control trial of a reentry court in Indiana found about $3 in benefits for every dollar in costs. However, most of the benefit to the community ($1.3 million of the $1.7 million total) was due to a reduction in homicides from an expected 0.42 homicides to zero. That is, half of the program’s benefit is explained by an expectation that part of a homicide (with an extremely large associated benefit) was prevented. Excluding these savings reduces the benefitcost ratio to less than two. Two other important and closely related limitations of the usual cost-benefit approach to reentry evaluation are worth considering—the effect of researching “stovepipe” programs and the effects of programs on communities. As noted, many reentry programs are designed to bundle services, where programs that attempt to address the whole range of offender deficits are hypothesized to be more likely to increase desistance than programs that focus on a single issue. However, data limitations often require evaluators to focus on the costs and benefits of only a single service or a few services in the bundle, because primary data collection in the evaluation of a program providing large numbers of services is often impractical. The net result is that both costs and benefits are systematically undercounted. Since the extent of the undercounting cannot be observed with a limited sampling frame, it is difficult to know how to interpret the resulting estimates. 5 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative Current approaches also tend to be offender-centric. That is, cost estimates focus on costs associated with treating and processing offenders, and benefits estimates focus on avoided costs from crime prevention, including benefits to victims who are not victimized and benefits to public agencies that do not need to expend resources to investigate, prosecute, and incarcerate offenders for those prevented crimes. This approach excludes important economic consequences, especially on the benefits side, from effective reentry programs. As Western (Western, 2006, p. 355) noted, “[A]n expanded commitment to prisoner reentry policy has collateral benefits that are also somewhat invisible. Children and spouses will benefit from the improved literacy and subsidized employment of their fathers and partners even if they have trouble finding work.” Other collateral consequences beyond the offenders’ immediate families are also excluded, including the offenders’ neighbors and other residents of their communities, all of whom can benefit from the successful reintegration of former prisoners. Thus, CBA of reentry should perhaps consider externalities from these programs and sum the costs and benefits to those who are not party to the reentry exchange of services, but nevertheless experience the consequences. In practice, measurement of these externalities is extremely difficult. As a result, the research literature is sparse, and CBAs of prisoner reentry programs are rare. That said, there is a broad literature describing the potential efficacy or effectiveness of various components of reentry. Several reviews of reentry program evaluations have examined the available research on what works with regard to reentry and/or rehabilitative programming (Aos, Miller, & Drake, 2006; Lipsey & Cullen, 2007; MacKenzie, 2006; Petersilia, 2004; Seiter & Kadela, 2003). Among the conclusions of these reviews are that intensive supervision programs with a clear treatment component show a sizeable impact on recidivism (Aos et al., 2006) and that programs focusing on individual-level change, including cognitive change, education, and drug treatment, are more effective than those based on a control or deterrent philosophy (Lipsey & Cullen, 2007; MacKenzie, 2006). Few reentry programs have been studied using a random assignment research design. One exception is the evaluation of the Opportunity to Succeed (OPTS) program. OPTS was designed to reduce substance abuse relapse and criminal 6 Introduction recidivism by providing comprehensive, case-managed reentry services to felony offenders who had drug offense histories. The evaluation found that OPTS clients had significantly higher levels of full-time employment during the first year after prison release. In turn, high levels of employment were associated with reductions in self-reported commission of person and property crimes, as well as reductions in drug dealing during a 1-year follow-up period (Rossman, Sridharan, Gouvis, Buck, & Morley, 1999). Another exception, Project Greenlight, was developed from research and best practice models to create an evidence-based reentry program (Wilson & Davis, 2006). However, an evaluation of the model and its implementation in 2004 in New York revealed that the program did not replicate past best practice. Instead, Project Greenlight modified past practice to fit institutional requirements, was delivered ineffectively, did not match individual needs to services, and failed to implement any post-release continuation of services and support (Marlowe, 2006; see also Visher, 2006; Wilson & Davis, 2006). The evaluation found that the program participants performed significantly worse than a control group on multiple measures of recidivism after 1 year. The evaluators attributed these findings to a combination of implementation difficulties, program design, and a mismatch between participant needs and program content. In response to the evaluation report, Marlowe (2006) argued that the evidence base for the program was flawed from the beginning. Rhine and colleagues (2006) were more optimistic about reentry programming in general, but argued that Project Greenlight was not sufficiently different from other failed reentry programs, nor was the treatment delivered appropriately. In summary, a critical problem for community-based reentry programs (and the evaluation) is that they often are not integrated into an overall “continuum of care” strategy that links prison and community-based treatment, and they are not managed or monitored according to procedures designed to help guide or maximize their effectiveness (Andrews & Bonta, 1998). Numerous challenges encumber the research assessing the effectiveness of programs for formerly incarcerated individuals, whether focused on reentry or general rehabilitation. Foremost among the challenges is the lack of theoretical models that articulate behavior change among former prisoners. Within any 7 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative particular substantive area, there are also problems of fidelity in that a particular service approach may manifest itself in different ways under different programs and circumstances. As a result, it is often difficult to generalize research findings from one program to others, and substantial variability exists among the outcome variables examined (e.g., employment, homelessness, health). Finally, there are problems related to the research itself, as rigorous experimental designs—including the use of comparison groups (randomly assigned or otherwise)—are rare in this research literature. There are also temporal restrictions in this research, where few studies examine outcomes beyond 1 year. RATIONALE This analysis represents one of the first attempts to conduct a large-scale cost-benefit evaluation of reentry programs. Additionally, there are at least three contributions to the literature. First, the evaluation covers one grant mechanism across several grantees. Even though grantees had considerable discretion to use the funds as needed, the analysis uses the same methodology for data collection and estimation across all sites examined. Thus, estimates can be compared across sites. Second, this analysis attempts to capture the considerable diversity of reentry programming exercised by states. Thus, the analysis is comprehensive. A third contribution is that the analysis uses well matched comparison as part of the study design. In the absence of random assignment to reentry services, such a design is desirable for drawing inferences about the costs and benefits of enhanced reentry. 8 Methodology OVERVIEW OF THE ECONOMIC EVALUATION The perspective of the analyses—which determines whose costs and benefits are measured—is the criminal justice system. State agencies in the criminal justice system underwrite all services received by offenders (e.g., anger management) and certainly pay for the law enforcement and court resources used in rearresting and the prison resources used in reincarcerating offenders. For the cost analysis, separate analyses were conducted for each of five programs selected. Pre-release rather than postrelease costs were detailed because resources are largely within the direct control of the Department of Correction before offenders are released. After release, services that offenders receive are typically provided by a variety of providers, and decision makers have less control over provision and access. Estimates are provided at the site level because evidence in the main outcome evaluation indicates significant state-to-state variation in programming. The CBA combines all costs—both pre- and post-release—and all monetized benefits. The costs of reentry are the value of resources used in programming to help offenders reenter society. These costs include the programs, reentry planning, and case management that occur before release, as well as the continued provision of services after release. The benefits are the value to criminal justice agencies of arrests and the number of days in prison that are avoided because offenders receive enhanced reentry services. For the CBA all four sites for men were combined. Combining sites increased the statistical power for the point estimates. The results indicate whether and the degree to which benefits outweighed costs. 9 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative SAMPLE The sample comprises 722 men from four adult sites and 79 juveniles from one juvenile male program. The five programs included in the economic evaluation are programs for men in Iowa, Ohio, Pennsylvania, and South Carolina, and a program for boys in South Carolina. Because relatively few women participated in services at these sites, women are not included in the analyses. Detail on each site is described in Appendix A. All programs had similar goals: Prevent reoffending. Enhance public safety. Redeploy and leverage existing resources. Assist the offender to avoid crime. Help the offender engage in prosocial community activities and meet family responsibilities. Ensure the sustainability of the program beyond federal funding. Each program had a pre-release phase that almost exclusively provided services within the correctional facility and a postrelease phase that helped secure services for offenders once they were in the community. Further detail can be found in the appendix. The programs were selected for the study by first conducting a systematic screen of each evaluation site along the following criteria, summarized in Exhibit 1. Each of the criteria addressed critical considerations from a statistical, practical, or substantive point of view. Criterion 1 was that the sample size should be as large as possible. To help ensure statistically meaningful results, it was preferred that the SVORI and comparison samples in each site number in the hundreds. A common challenge in economic evaluation is that program cost estimates may vary greatly across the sample being studied, making it difficult to find statistically significant results (e.g., Cowell, Broner, & Dupont, 2004; Cowell, Lattimore, & Krebs, in press). The difficulty stems from the fact that some people consume relatively few resources, whereas others are associated with relatively high expenditures. Because the spread of cost estimates around the measure of central tendency is large, the standard error of 10 Methodology Exhibit 1. Criteria and Outcomes of the Site Selection Process for the Economic Evaluation Criterion Explanation Final Decision Large sample size (n) A statistical consideration. A larger n gives greater statistical power. For adult sites with a quasiexperimental design, anticipated n at baseline must be greater than 100. Thus at least one large site—South Carolina adult—was included. Strong quasiexperimental or experimental design A substantive consideration. A better Include sites where a strong counterfactual provides stronger estimates for comparison group can be policy making. reliably identified and sites that randomly assign offenders to SVORI. Thus the two random assignment sites—Iowa and Ohio—were included. Likely availability of administrative records A practical consideration. Administrative records may provide key metrics for number of service units received and cost of service. Important limiters are which services are covered in the data, which offenders are covered in the data, for which dates data are available, and completeness and reliability of data fields. Include sites that promise to have unusually good administrative data. Thus Pennsylvania was included. Juvenile program A substantive consideration. Little information exists on the costs and benefits of reentry programming for juveniles. Include at least one juvenile program. This site was South Carolina juvenile. estimate is large, which means that statistical significance may be difficult to achieve. Mean costs for offenders in the SVORI study may be high because at least one of three features was present in every site: Participants had heterogeneous needs. Some offenders received many services and others received none. The cost per unit of service was high. Other reports from the SVORI evaluation show the large extent to which offenders have heterogenous needs (Lattimore, Visher, & Steffey, 2008). Many offenders, for example, need substance use treatment. To the extent that services are provided to those in need, offenders who receive treatment will have a higher cost than those who do not. Finally, high costs per service unit will drive up the variation in the data. This is seen throughout the literature on health services, particularly 11 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative for inpatient or residential services (Cowell et al., 2004; Cowell, Broner, Hinde, & Aldridge, 2009). In the case of SVORI, this situation is most likely to manifest itself in the case of substance use treatment, in which residential services incur a high cost per offender. A strong study design also helps increase statistical power (Criterion 2). The strongest study design is typically a randomized experiment. Two sites in the study, Iowa and Ohio, randomized participants into SVORI or a comparison group. In the case of a quasi-experimental design, a stronger design is one in which the comparison group is very similar in key characteristics to the SVORI sample. As documented elsewhere (Lattimore et al., 2008), a careful selection protocol for the comparison group at each site paid dividends insofar as analyses indicated that the SVORI and comparison samples matched well at baseline across many measures. The availability of administrative records (Criterion 3) was a practical consideration. An offender survey was administered to all participating offenders at each site and at four important points in time, just before release and at three points in time after release. This survey could not capture all the preferred information for the economic evaluation. Sites were thus selected that had the promise of providing collateral administrative data on service utilization. Criterion 4 was that at least one program selected should be targeted at juveniles. Juvenile justice reentry programming differs in many ways (conceptualization, approach, funding, and service provision) from adult criminal justice programming. Moreover, relatively little is understood about the resources needed and benefits derived from juvenile justice programs (Cowell et al., in press). Five programs met at least one of these criteria. By implementing random assignment to SVORI or comparison study conditions, Iowa and Ohio had the strongest possible design. The two other adult programs—South Carolina and Pennsylvania—had both a sufficiently large sample size and a sufficiently strong quasi-experimental sample design. The South Carolina juvenile program had a relatively large number of juveniles, could provide administrative data, and was able to provide particularly detailed budgetary information for the economic evaluation. 12 Methodology DATA The data for analyses came from several sources. Because of the variation in the types of services and the nature of the data available, the data sources for service costs were particularly diverse. Data came from a mixture of the offender surveys, site-specific documentation, the literature, and expert opinion. Data on the costs of arrests came from the Bureau of Justice Statistics (BJS) and the literature, and data on incarceration came from BJS. All dollar estimates were adjusted to 2007 costs using the Consumer Price Index of the Bureau of Labor Statistics (BLS). The offender survey was conducted as part of the main SVORI evaluation and contained several questions that were used for the two economic analyses. The survey instrument was quite comprehensive and is described more fully in Lattimore et al. (2008). Data on service use, arrest, and incarceration were collected at four points in time: 1 month before release and 3, 9, and 15 months after release. Receipt of Services (Pre- and Post-release) In estimating pre-release costs per offender and the additional costs for post-release services for the CBA, a key challenge was to obtain detailed information on the quantity of services used by each offender. This challenge is quite common to cost analyses in criminal justice settings (e.g., Cowell, Broner, Aldridge, & Hinde, 2009; Cowell et al., 2004; Cowell, Broner, Hinde, et al., 2009; Cowell et al., in press). This fact has implications for the design and protocol of the study. As noted in the conceptual discussion above, the units in which price and quantity are measured must be compatible. Thus, although obtaining price information is certainly not straightforward, the source and method of estimation of price estimates are contingent on the source of quantity data. The preferred sources for this report were offender survey data for the mean quantity of services used and site-specific sources and the literature for the price of the services. This approach was determined from several alternatives, each of which was explored, and these are summarized in Exhibit 2. The most straightforward of all methods is the simple top-down approach, whereby the grant funding is divided by the number of offenders served. This method is commonly used in economic evaluations in many settings. It requires relatively 13 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative Exhibit 2. Relative Merits of Alternative Approaches to Measuring Quantity Measure Budget Type of Analysis It Supports A straightforward, topdown approach Budget or expenditure A top-down approach by category providing more detailed estimates Advantages Disadvantages • Straightforward • Consistent across programs • Of limited use for decision making • Estimates are limited to only those programs directly funded with the grant or budget • More detailed information than a standard top-down approach • Not consistent across programs • Less detailed than a bottom-up approach • Estimates are limited to only those programs directly funded with the grant or budget Administrative data A detailed bottom-up analysis • Detailed information • Measures of actual service provision • Estimates are limited to only those programs or agencies providing data • Programs vary in completeness and reliability of data Survey data A detailed bottom-up analysis • Available for all relevant services • Consistent across programs • Relies on self-report • Measures require transformations and assumptions to be used in economic analysis little detail on costs, utilization, and client flow. In its simplest form, the top-down estimate is the amount of the grant divided by the number of grant recipients. An additional advantage is that it can be often be implemented the same way across sites and programs, so estimates can be compared. The drawback of this approach is that it provides little insight into the components of the cost. Thus, decision makers are unable to determine, for example, how changes in the mix of services may influence costs. The top-down approach can be enhanced to provide information on the main components of the costs of a reentry program. This enhancement requires grant or expenditure information broken out by these main components and data on the numbers of offenders served under each of these components. For example, in preliminary analyses for South Carolina, data on 14 Methodology the budget of the Department of Corrections was aggregated into four categories (e.g., educational services). This was then combined with estimates from the SVORI evaluation offender survey on the differential service receipt between SVORI offenders and comparison offenders. With a number of necessary assumptions, this provided the incremental cost per offender for SVORI. The drawback of this approach is that the methods used must be site specific, so the estimates may not be readily compared across sites. Administrative data, such as billing or management information system data, may contain both quantity and price information. Billing data, for example, may contain data on each service event for each type of service and the charge or bill for that service (Cowell, Broner, Hinde, et al., 2009). In the case of the SVORI evaluation programs, sites were able to provide administrative data. However, these data did not allow for an estimation approach that was common across programs, and for some sites, it was not possible to provide reliable incremental cost estimates because data on comparison group members were not available. As shown in Exhibit 3, the administrative data were neither consistent nor complete across sites. Exhibit 3. Administrative Data Availability, by Site Site Administrative Data Available? Is Service Dosage Available? On Both SVORI and Comparison? Both Pre- and Post-release? Iowa Yes Not always Yes Yes Ohio Yes No Yes No, pre-release only Pennsylvania Yes Yes No, SVORI only No, pre-release only South Carolina (adult) No Not applicable Not applicable Not applicable South Carolina (juvenile) Yes No No, SVORI only No, pre-release only The offender survey that was used to provide information on quantity of services for this report is described fully elsewhere (Lattimore et al., 2008). The survey instrument was administered at four points in time, approximately 3 weeks before release, and 3 months, 9 months, and 15 months after release. As part of the survey, each offender reported whether he received, over a period of time, each of more than 80 15 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative different types of services. The services are categorized into the following seven domains: coordination services, transition services, family services, child services, mental health services, substance use services, and employment/education/life skills services. The advantages of using the offender survey data are that the approach is consistent across programs and that all relevant services are covered. The disadvantages of this data source are that it relies on self-reported information and requires additional transformations and assumptions to be used in the analysis. No information is available on the dosage (or number or quantity) of services received, and the survey questions were not intended to cover exclusive services. Thus, an offender reporting receipt of case management and a needs assessment may well have received the needs assessment as part of the case management. Quantity of Services Because the survey data only provide information on whether an offender received a service, supplementary information was required on the quantity of services received. The preferred source of information on quantity was site-specific documentation, obtained from on-site visits and existing program literature, such as program descriptions, syllabi, or supporting materials for contracts. Other supplementary sources included the broader literature and opinions from site program staff and substantive experts. Sensitivity analyses (below) varied these and other assumptions to assess the degree to which the study conclusions depended on them. It also should be noted that estimates of quantity and price were obtained only for those services for which there was a statistically significant difference in the proportion of SVORI and comparison group participants reporting receipt at each site. Exhibit 4 documents the sources of data used for each domain. Price per Service Received Information on the price for each service came from a combination of program- and service-specific collateral sources and the literature. Exhibit 5 summarizes the sources for price measures used in the analysis. Price data were obtained for services in which there was a statistically significant difference between the proportion of the SVORI and the comparison group participants receiving the service. 16 Methodology Exhibit 4. Sources for Quantity of Services Received Service Domain Source Coordination services during Site visit report, interviews with site staff, SVORI grant proposal, Reentry incarceration Policy Council Web site Transition services during incarceration Site visit report, interviews with site staff, SVORI grant proposal Family services during incarceration Ohio offender services contracts (used in all sites) Child services during incarceration Ohio offender services contracts (used in all sites) Mental health services Literature, Ohio offender services contracts (used in all sites), Medicaid reimbursement rates Substance abuse services Literature, Ohio offender services contracts (used in all sites), Medicaid reimbursement rates Employment/education/ life skills services Ohio offender services contracts, Iowa service documentation (both used in all sites) Note: Information was obtained for a service only if the offender survey indicated that the proportion of SVORI offenders receiving the service was significantly different from the proportion of comparison offenders. Exhibit 5. Sources for Prices of Services Received Domain Source Coordination services during Interviews with site staff, BLS wage data, SVORI grant proposal incarceration Transition services during incarceration Interviews with site staff, BLS wage data, SVORI grant proposal Family services during incarceration BLS wage data Child services during incarceration BLS wage data Mental health services Literature, Ohio offender services contracts, Medicaid reimbursement rates, BLS wage data Substance abuse services Literature, Ohio offender services contracts, Medicaid reimbursement rates, BLS wage data Employment/education/life skills services BLS wage data Note: Price information was obtained all medical services. For all other services, price information was obtained only if the offender survey indicated that the proportion of SVORI offenders (by site) receiving the service was significantly different from the proportion of comparison offenders. Number of Arrests and Nights Incarcerated Counts of the number of arrests and nights incarcerated came from the offender surveys. The measure of arrest was the number of times since the last interview that the respondent had been booked into jail or prison, as based on responses at each of the three follow-up interview waves. The measure of 17 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative nights incarcerated was the number of nights incarcerated since the last interview, as indicated at each of the three follow-up waves. The data contained missing values for incarceration days that should, in fact, be nonmissing, and these items were imputed. Review of the data and survey documentation suggested that these missing items were an artifact of the skip pattern in the offender survey. At 9 and 15 month, surveys respondents who had been incarcerated before the 3-month follow up were asked questions on service receipt, but not on arrest and incarceration. Nine months after release, there were 25 such respondents, and at 15 months, there were 107. The imputed number of days incarcerated depended on which of two categories respondents fell into. The first category comprised those who had been interviewed in the previous wave and who had been incarcerated the entire time between interviews. These people had responded with information on incarceration in their previous interview. The date of entering the current incarceration was earlier than the date of their previous wave’s interview. At 9 months, 16 of the 25 respondents to be imputed were in this category, and at 15 months, 86 of 107 were in this category. These respondents were thus assigned the number of days from their prior interview until their current interview as the number of days incarcerated. The second category comprised the remainder of respondents with missing incarceration data but with reported service utilization data (9 people at 9 months, 21 people at 15 months). They were assumed to have been incarcerated throughout the period of the previous interview. These people had not been interviewed in the previous wave and, in the current wave, reported a date of incarceration before the date of the previous wave’s interview. These people were thus assigned the number of days from date of incarceration to date of interview as the number of days incarcerated. Alternative data sources were explored, including administrative data on arrests from the Federal Bureau of Investigation’s (FBI) National Crime Information Center (NCIC) and administrative data on reincarceration from states’ Departments of Corrections (DOC). Unfortunately, although the NCIC data provided reliable counts of arrests for every adult 18 Methodology site in the analyses, the DOC data were not complete for one adult site. Because a guiding principle of the analysis was to use the same data source and approach for all sites whenever possible, the offender survey data were therefore used to provide estimates for reincarceration. Moreover, the offender survey data were also used for rearrests. Offender reports and administrative records may not agree on counts of criminal justice events and because arrest and incarceration events are connected, mixing the two types of sources would likely to lead to inconsistent counts of arrests and nights incarcerated (e.g., some people may have reported nights incarcerated with no accompanying arrest indicated in the administrative data). Thus, rather than mix the two types of data on criminal justice events, offender survey data were used for both arrests and incarceration. Cost per Arrest The costs per arrest were constructed from estimates in three publications (Cohen, Miller, & Rossman, 1994; Durose & Langan, 2003; Miller, Cohen, & Rossman, 1993) and Uniform Crime Reports (UCR) estimates were constructed from the FBI (2009). The cost per arrest by type of arrest—violent, nonviolent, and drug—was obtained from Miller et al. (1993) and Cohen et al. (1994). The estimates included eight major components of costs, from initial contact, investigation, and arrest through sentencing. Each individual component had a raw cost associated with it. For example, on average (in 2007 dollars), it costs $334 to book someone, and a trial costs, on average, $55,088. For each of the three types of arrest, each component was then appropriately weighted according to the number of arrests that included that component. The weights were taken from estimates from Durose and Langan (2003), which provides information on the number of convictions that result from a guilty verdict at trial. The cost per type of arrest was then the sum of the weighted components. For example, 9.9% or about 0.1 of violent arrests went to trial. Thus, in the estimate of the cost of a violent arrest, $5,454—which is about 0.1 of the raw cost of $55,088— was the weighted component for the trial cost of an arrest for a violent offense. This amount was added to the seven other weighted components to obtain the cost of a violent arrest. 19 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative To obtain one overall cost per arrest, the costs of the three types of arrest were aggregated. This aggregation was necessary because the counts of arrest did not provide a ready breakdown by type of offense. Although the NCIC data used to track arrests do include very detailed charge information, multiple charges can exist for one arrest. Thus, to determine the type of arrest would have required determining which charge was most applicable for every arrest with multiple charges, a task that was beyond the scope of this study. To aggregate the arrest costs, the estimates were weighted using UCR estimates for 2007 on the relative proportion of violent, nonviolent, and drug offenses. Separate weights were computed for each of the four states included in the study. The methodology used to calculate the cost of an arrest has a number of advantages and disadvantages. One advantage is that the methodology is consistent across the four states in the study, while also providing some meaningful variation by state. The methodology also has high scientific integrity in that it uses an approach that is accepted by researchers and has passed peer review. Finally, the estimates are very comprehensive in that they account for resources used in the criminal justice system at every stage, from the police response to the initial incident to sentencing at trial. However, one disadvantage is that many components of the estimates are not derived from state-specific sources; for example, the cost of a trial covers all states. Sensitivity analyses (the methods of which are described below) used an alternative estimate of the cost of an arrest to assess the degree to which conclusions were robust to the specific arrest cost used. Alternative costs drew on BJS (2009) employment and expenditure statistics to determine each state’s police and judicial expenditures in 2007. Police costs were adjusted by subtracting costs for nonsworn employees. That estimate was divided by the total costs by the number of arrests for that state in 2007. The advantage of using this approach is that it is based on state-level data. The disadvantage is that the estimates will be measured with significant error. For example, the judicial costs include civil cases, whereas arrest costs should typically exclude civil cases. Exhibit 6 presents the arrest costs by state for the main analyses and for the sensitivity analyses. 20 Methodology Exhibit 6. Estimates and Data Sources of Arrest Costs (2007$) Type of Analysis Main analysis a Sensitivity analysis b Pennsylvania South Carolina (Adult) South Carolina (Juvenile) Iowa Ohio 7,030 7,012 7,088 7,081 7,095 7,091 17,063 8,174 5,224 5,224 Note: Estimates are for the average costs. Costs are rounded to the nearest dollar. a Calculated from Cohen et al. (1994); Durose and Langan (2003); and FBI (2009). b Calculated from BJS (2009) and FBI (2009). For sites of adult offender programs, estimates of the cost per night in prison came from BJS (2009). The estimates are on the operating expenses, as reported by states to the U.S. Census Bureau’s annual Survey of Government Finances in 2001 (Stephan, 2004). Expenditures are the amounts paid for prison operations, including interest on debt and, thus, some capital costs. The estimates are net of revenue generated from activities such as farm and industrial production and services. An average 12% of operating expenses, representing medical care costs, was deducted from the estimates. As noted above, medical care service costs for offenders were separately esitmated. Thus, deducting medical care costs helped avoid double-counting the cost of medical services. The BLS data have both advantages and disadvantages for the analysis. Because the data were available from a single, reliable source—a federal agency—the data were likely the most reliable and could be readily compared across states. However, the BLS estimates may be obsolete. Although estimates were adjusted for general price inflation, the structure and technologies underlying costs may change over time. For example, if a state were to reduce its ratio of guards to inmates from one year to the next, then costs would decrease faster than general inflation. Alternative and more recent estimates were available from individual state sources. These alternative estimates were used in sensitivity analyses to assess the degree to which study conclusions were robust to variations in the estimates used (as described more fully below). For the juvenile site in South Carolina, no national-level data source was found. Instead, the analysis used one state-level source for the estimates. It should be noted that, although the source claims that a portion of the estimated cost was covered by county funds (the rest being covered by the state), the full 21 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative amount was used in analyses. Exhibit 7 summarizes the estimates used in the main analyses and the alternative estimates that were used in the sensitivity analyses. Exhibit 7. Estimates and Data Sources of Incarceration Costs per Offender per Night (2007$) Type of Analysis Main analysis a Sensitivity analysis Iowa Ohio Pennsylvania South Carolina (Adult) 68 76 90 50 72 b 58 c 76 d 33 South Carolina (Juvenile) 137 e — Note: Estimates are for the average (per offender) costs. Costs are rounded to the nearest dollar. a Source: Stephan (2004). b Source: Iowa Department of Corrections (2008). c Source: Ohio Department of Rehabilitation and Correction (2009). d Source: Beard (2009). e Source: South Carolina Department of Corrections (2008). ANALYSIS Pre-release Costs To estimate the impact of enhanced reentry on the costs of prerelease services, information from the offender survey on whether an offender received a service was combined with information from the sites and the literature on the amount of service received (i.e., dosage) and its cost. To maximize the utility of this analysis, the results are provided at the site level. Exhibit 8 summarizes the five incremental steps of the estimation protocol. Step 1. Determine Which Services to Include in the Estimation Develop and Apply Decision Rules The offender survey gathered information on a very rich set of potential services. In the Wave 1 survey interview, for example, respondents were asked whether they had received each of 41 services. These services were not necessarily exclusive of one another. For example, a service thought of by the respondent as anger management may equally have been thought of as counseling. The analysis included those services for which there was a statistically significant difference in the proportion of SVORI respondents endorsing the question and the proportion 22 Methodology Exhibit 8. Five Incremental Steps of the Pre-release Services Cost Estimation Protocol of the comparison group endorsing the question. This rule followed the goal of the study, which was to estimate incremental costs—the difference between the SVORI group and the comparison. To apply this decision rule, the analysis included only those services where the difference between the mean proportions was significant at the 5% level. Modify Decision Rules Finally, in sensitivity analysis the 5% decision rule was modified to include more services (see Step 5, below). Step 2. Address Potential Duplication Across Offender Survey Items Determine Most Likely Sources of Duplication A key exercise of the economic evaluation was to diagnose and address potential sources of double-counting that would occur if the offender survey responses were used naively to estimate costs. The offender survey was designed to gather information across as comprehensive a set of services as possible; the goal 23 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative was to determine needs and whether those needs were met. Because it was important to minimize respondent burden, the survey did not explicitly reveal whether any two service items were received in the same class or service event. For example, the survey asks separately whether the respondent met with a case manager and whether the respondent received a needs assessment. However, the survey data did not indicate whether the needs assessment was delivered as part of meeting with the case manager. This limitation of the survey is a key challenge for the economic evaluation. In estimating the costs, if service categories are not exclusive of one another, then the estimates may include duplicated services. To continue the example, if the cost of case management is added to the cost of a needs assessment, then that double-counts the cost of needs assessments received as part of case management. The first activity in this step was to assess which measures were most vulnerable to potential duplication bias. This entailed consulting site visit reports and other existing documentation on how SVORI funds were intended to be spent. The result was a comprehensive set of services that could suffer from duplication bias. The four categories of potential duplication are detailed in Appendix B. After prioritizing the potential types of duplication using the categorization schema, the offender survey data were examined by site. The primary method of assessing duplication was to rely on simple tabulations and cross-tabulations using Stata software. To help develop the code for these tabulations, contingency diagrams of likely points of duplication were developed. Step 3. Estimate Incremental Pre-release Cost per Offender Ensure Compatibility across Data Sources To estimate the cost per offender it was important to ensure that prices and dosage were in compatible units. Thus, a systematic check ensured, for example, that information on the typical number of group counseling sessions was matched to a price estimate of a single group counseling session. 24 Methodology Compute Incremental Cost per Offender Once all prices and quantities were in compatible units, estimating the incremental cost per offender was conceptually straightforward. For each service to each offender, the receipt of a service was multiplied by the product of the dose and the price. The cost was summed within offenders and averaged within SVORI group, the group averages were compated. This exercise was conducted across all statistically significant services combined and within service domain. Step 4. Conduct Sensitivity Analyses In economic evaluations, sensitivity analyses are used to vary the assumptions in a model to determine the degree to which the conclusions are robust. As noted above, some aspects of the data on price and dosage relied on assumptions. Importantly, any given cost estimate involves distinct combinations of these assumptions, and the number of potential sensitivity analyses increases greatly as the number of assumptions made in the analysis grows. Thus, all assumptions made in the analyses at each site were documented. The assumptions were then ranked by their likely individual impact on the results and grouped similar types of assumptions. Finally, combinations of the most important types of assumptions for the sensitivity analyses were created. The sensitivity analysis simultaneously varied three different sets of model assumptions. This analysis provided a maximum and minimum bound around the estimate. The first set of assumptions are on the content, scope, and cost of case management and other key services. Because case management comprised a large portion of costs (see results below), any assumptions regarding this service would potentially have a large impact on cost. These assumptions include those made regarding whether and the degree to which services overlapped. In most sites, case management, for example, was understood to cover numerous reentry activities, many of which were separate questions in the offender survey. Thus, to avoid double-counting, these activities would not have been included as a separate cost. Set two comprised assumptions on the resources used for each service included in the analysis. The resource information included the value of the resources (e.g., the cost of a case manager) and the quantity of the resource. This quantity 25 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative measure was either the number of distinct events (e.g., the number of reentry needs assessments) or the composition of each event (e.g., the number of mental health counseling sessions and the length of time of each session). The third set comprised one assumption, the rule that determined which services were included in the analysis. The baseline analysis included for consideration only those services for which there was a significant difference between the SVORI and comparison group at the 5% level. In the sensitivity analysis, the significance level was expanded to 10%, thus widening the scope of services that were included. Step 5. Provide Bounds of Incremental Costs The estimates from this analysis were provided within bounds determined by the sensitivity analysis. A series of iterative tasks helped fine-tune these bounds. The preliminary estimates from the sensitivity analyses were arrayed in order of magnitude. The estimates were then compared to grant budgets to determine whether the bounds were feasible. The types of assumptions that gave the largest and smallest estimates were reassessed at this stage in terms of their substantive validity and relative importance. The assumptions were subsequently modified as necessary, and updated estimates were derived. The estimates from the iterative process provide maximum and minimum bounds around the incremental costs. The bounds provide the range of estimates for incremental costs within which it can be said with some certainty that the actual cost lies. The bounds may help decision makers budget for contingencies appropriately and will be used in further costeffectiveness and cost-benefit analyses. It should be noted that these bounds are driven solely by the data and the assumptions used in estimating the costs, so they need not be symmetrical around the main estimate of incremental cost for the study. This means that the base incremental cost may be closer to one bound than another; this emphasizes the fact that the bounds should not be interpreted as confidence intervals. Confidence intervals around point estimates are a statistical concept that draws on the sampling variation in the data and the assumed statistical distribution from which the data are drawn; they are, by construction, symmetrical and have a very different interpretation from the bounds in the sensitivity analyses. By 26 Methodology the nature of the data, no such sampling variation exists to estimate pre-release costs. Because the analysis relies heavily on assumptions, the sensitivity bounds are reported as part of the main findings on the pre-release cost estimates. Cost-Benefit Analysis The CBA follows a protocol similar to that used for the prerelease cost analysis, above. Net costs per offender are the sum of the costs of three different measures: service costs, arrest costs, and incarceration costs. These were summed for every offender in every analysis period (i.e., pre-release and 3, 9, and 15 months after release) for which survey data were available (Exhibit 9). Note that for the pre-release period, arrest and incarceration costs were not assessed, because all offenders were incarcerated. Results are expressed as average costs per person per month. Exhibit 9. Conducting CBA: Estimating Net Costs In each analysis period and for each offender, three types of costs were estimated: service costs, arrest costs, and incarceration costs. The sum is the net cost per offender per analysis period. 27 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative The estimates of the costs of services used in the CBA combine post-release with pre-release cost data. The benefits took the form of averted costs and were simply included in the net cost estimate. Arrest costs were computed by multiplying the number of arrests in the period by the cost per arrest. Similarly, incarceration costs were computed by multiplying the number of nights incarcerated by the cost per night of incarceration. Calculating the net cost per offender was the first of four steps in the overall CBA estimation protocol (Exhibit 10). The CBA estimates were reported by combining all four adult sites. This contrasts with the analysis of the costs of pre-release services, which were reported at the site level. Combining sites for the CBA helped improve the ability to find statistically significant findings, because the number of observations used in the analyses was increased. The analysis of pre-release costs made no attempt to establish whether differences found were statistically significant. Combining sites was also consistent with the approach of the overall SVORI evaluation. The main findings for three offender populations studied—men, women, and boys—are all presented by combining sites. Thus, by combining all adult sites, the CBA findings can be used in tandem with the outcome findings of the overall SVORI evaluation. The average net cost was taken across all offenders in the two groups, SVORI and comparison (Step 2). Propensity scoring was used to adjust for potential confounders. By making statistical contrasts between the groups receiving enhanced reentry and the comparison receiving standard reentry, the analysis accounted for the potential influence of confounders, examples of which include differences in criminal history and age. For the four adult sites combined and the juvenile site, propensity scores were estimated as part of the main outcome evaluation and are described more fully in the methodology report for the evaluation, Lattimore and Steffey (2009). Propensity scores were used as analytic weights in assessing, at each period, the difference between the average net costs of the SVORI and comparison groups. Summarized here is the approach used to estimate the propensity scores. 28 Methodology Exhibit 10. Four Incremental Steps of the Cost-Benefit Analysis Estimation Protocol There were three main sequential steps used to estimate the propensity scores. First, logistic regression was used to regress the probability of being assigned to the SVORI group (versus the comparison). The largest number of possible covariates was used, including measures of criminal history and demographic characteristics. Data for all four adult sites were combined, and a separate analysis was conducted for the juvenile site. In the second step, the predicted probability of being assigned to the SVORI group was used to create analytic weights. How the predicted probabilities were combined depends on the treatment effect of interest. The treatment effect expresses the hypothetical experiment being conducted with the analysis. In the case of the SVORI evaluation, the hypothetical experiment of interest is comparing when all offenders are hypothetically offered enhanced reentry to when all offenders are hypothetically offered standard reentry services. The effect of interest is known as the population average treatment effect (PATE). The analytic weights for the SVORI group were the reciprocal of the predicted probability (p). The analytic weights for the comparison group were the reciprocal of the quantity (1 – p). 29 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative In the third step, a balance check was conducted for every covariate used in Step 1 above (the regression of SVORI status). The balance check ensures that, for each of the covariates, the statistical distribution among SVORI and comparison offenders is similar. If balance is not achieved, then further steps are needed to correct for potential confounding. The propensity weights were used in statistical contrasts of the average net cost for SVORI and comparison offenders at each of the four analysis periods. The result indicated the degree to which net costs were lower for enhanced reentry. As described above for the pre-release cost analysis, sensitivity analyses were conducted and used those results to put bounds on the estimates (Steps 3 and 4). For the juvenile site, South Carolina, the analysis used propensity scores derived for all four adult sites in the main evaluation pooled together. Pooling sites ensured that sufficient statistical power was obtained to derive precise propensity scores. However, when applied to South Carolina, the propensity scores gave good balance for only approximately half of the first-stage covariates (depending on the wave of data examined). An alternative using doubly robust regression was explored by estimating, at each of the four waves, a weighted multivariate regression. In this alternative approach, the dependent variable was the net cost, and the covariates in the regression were a SVORI indicator and all variables from the multivariate regression used to create the PATE weights; the analytic weights were the PATE weights. However, there was an insufficient number of observations to support a multivariate regression. The regression estimates were implausibly large in absolute magnitude and were bounded by very wide confidence intervals. 30 Results THE COST OF DISCRETIONARY PRERELEASE SERVICES The main results of the pre-release cost estimates are shown in Exhibit 11. Recall that the main result of interest is the incremental cost per offender, which is the difference between the average cost for a SVORI offender and that of a comparison offender. The main estimates are presented for the year before release. Recall that the estimates include the following sets of services: medical, mental health, and substance abuse services, and any other service for which the proportion of the SVORI group receiving the service was significantly different from the comparison. The table shows the estimated cost per SVORI offender; the estimated cost per comparison offender; the ratio of per-offender SVORI costs to the comparison; and the difference between the two, which is the incremental cost. Exhibit 11. Annual Incremental Pre-release Costs of Service Provision per Offender (2007$) Cost Iowa SVORI Comparison SVORI: Comparison ratio b Incremental cost a Ohio Pennsylvania South Carolina (Adult) South Carolina (Juvenile) $3,809 $3,361 $3,497 $2,378 $6,889 $2,646 $1,663 $2,839 $898 $3409 1.4:1 2:1 1.2:1 2.6:1 2:1 $1,163 $1,698 $658 $1,480 $3,480 Note: Estimates are for the average (per offender) costs over the course of 12 months before release. Costs are rounded to the nearest dollar, and ratios are rounded to the first significant decimal. a SVORI: Comparison ratio = SVORI cost divided by Comparison cost. For example, 2.5:1 means that, among those services included in the analysis, $2.50 was spent on SVORI for every $1 spent on a comparison offender. b Incremental Cost = SVORI cost – Comparison cost. The estimates indicate a considerable range of incremental costs, varying from $658 per offender for the Pennsylvania adult program to $3,480 for the South Carolina juvenile program. This likely reflects considerable variation both in regular reentry services, received by the comparison group, 31 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative and enhanced programming, provided to the SVORI group. Variation in reentry programming has been noted in several other products of the SVORI evaluation, including a report on men’s pre-release services and offender characteristics (Lattimore et al., 2008). In that report, offenders at three sites—Iowa, Ohio, and Pennsylvania—reported receiving many different services, especially those in the SVORI group. In South Carolina, the reported number of different types of services received was far lower. However, the report also indicated significant variation across sites in the kinds of services that offenders reported receiving. That finding helps underline the importance of avoiding using the incremental cost estimate alone to categorize the four sites. The adult sites, for example, ostensibly seem to be separated into two groups of incremental costs: Pennsylvania, where the incremental cost was relatively low ($658), and Iowa, Ohio, and South Carolina, in which the incremental cost was high ($1,163, $1,698, and $1,480 respectively). However, examining the absolute level of cost and the ratio of SVORI to comparison costs indicates that categorizing sites in such a one-dimensional manner would be misleading. Iowa, Ohio, and Pennsylvania have similar absolute costs while the South Carolina adult site has lower absolute costs. The difference in absolute costs is caused by the difference in the number of standard services offered to all offenders. In the South Carolina adult site the typical comparison group receives relatively few services ($898), whereas in Iowa, Ohio, and Pennsylvania the typical offender receives more services ($2,646, $1,663, and $2,839 respectively). The sites can also be grouped by the ratio of SVORI costs to comparison costs. Iowa and Pennsylvania have a lower ratio (1.4:1 and 1.2:1 respectively) while the Ohio and South Carolina adult sites have high ratios (2:1 and 2.6:1, respectively). Exhibit 12 shows the bounds from the sensitivity analyses around each estimate. Although this graphic suppresses the values behind the display, the values are reported in the text. Three of the five sites (Iowa, Ohio, and South Carolina juvenile) have a large range between the maximum and minimum values around the estimate, shown as the full length of the lines surrounding the estimate. The other two sites (Pennsylvania and South Carolina adult) have a very small range. Across all five sites, the range is greatest for the South Carolina juvenile 32 Results Exhibit 12. Annual Pre-release Incremental Cost Estimates of Discretionary Service Provision and Bounds from Sensitivity Analyses (2007$) $5,000 $4,500 $4,000 Incremental Cost $3,500 $3,000 $2,500 $2,000 $1,500 $1,000 $500 $0 Iowa Ohio Pennsylvania South Carolina Adult South Carolina Juvenile Note: For each site, the incremental cost estimate (reported in Exhibit 11) is shown as a dot. The estimate is the difference between the SVORI and comparison mean costs of those services included in the analysis. Services were over a 12-month span of time before release. The maximum and minimum bounds from the sensitivity analyses are shown as the termini of lines stretching from the estimate. Values are suppressed in the graphic but reported in the text. site, at $1,500. Across the adult sites, the range of estimates is greatest for Ohio at $1,270 followed by Ohio ($821), South Carolina adult ($256), and Pennsylvania ($77). Providers and decision makers using these estimates should understand that alternative assumptions change the estimate of how much more than the comparison a typical SVORI offender costs. 1 1 The fact that the incremental cost is closer to one bound than another may be surprising. The asymmetry is because the bounds are not conceptually the same as statistical confidence intervals. Standard confidence intervals around point estimates are typically symmetrical by construction and the point estimate is halfway between the lower and upper bound. However, sensitivity analyses follow a conceptually different process from that used to derive confidence intervals. Sensitivity analyses follow a predetermined set of rules that need not be symmetric; confidence intervals are a statistical construct that relies on sampling variation in the data. 33 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative PRE-RELEASE COST ESTIMATES BY DOMAIN To understand further the variation in incremental cost across sites, we examined the domains of services included in the analysis. Exhibit 13 summarizes the incremental cost per domain of service for the adult sites; Exhibit 14 is for the juvenile site. Although sites varied insofar as which domains contributed to the analysis, in all sites, six domains contained at least one service that was included in the costs: coordination services, transition services, mental health services, substance use services, employment/education/life skills services, and medical/dental services. Thus, in all sites, these six domains contributed to the overall incremental cost. In all sites except Iowa, coordination services were the bulk of the difference in pre-release service costs between the two groups of offenders (Exhibits 13 and 14). Of these four sites (five sites minus Iowa), the proportion of the increment accounted for by coordination services was typically 75% or higher; for example, in the South Carolina adult site it was 79% ($1,169 as a percentage of $1,480). In Pennsylvania, the increment disaggregated by service domain is unusual. For the substance use services domain, the mean costs associated with the comparison group were, on average, $825 per offender greater than for the SVORI group. Thus, the proportional increment in costs due to SVORI was actually greater for the coordination services domain (increment = $1,556) than the sum across all domains ($658). Exhibit 13 also shows that, in Iowa the pattern of service receipt was quite different from the other sites. In Iowa, the employment/education/life skills services domain accounted for 40% ($467) of the overall increment per offender. The coordination services and substance use services domains accounted for most of the remaining incremental cost. To help understand why there are differences across sites in which domains contribute to the incremental cost, we also examined the number of services in each domain that were included in the analysis (Exhibit 15). Iowa had an unusually high number of employment/education/life skills services included in the analysis. That same domain in Iowa had six separate services included in the analysis, while no other site had more than three services from that domain. It should also be noted that, across all domains, the Iowa site had the largest 34 5 1 — 48 1,182 247 1000 $2,646 1 3 — 41 1,769 714 925 $3,809 SVORI $162 Comparison $356 Incremental Costa 467 −75 $1,163 — 7 641 1 −3 $194 SVORI 159 675 $3,361 23 66 553 — 7 $1,878 Comparison 40 827 $1,664 6 49 275 — 8 $459 Incremental Costa 119 −152 $1,697 17 17 278 — 0 $1,418 SVORI 8 628 $3,497 — 29 1,112 — 1 $1,720 16 675 $2,839 — 46 1,937 — 1 $164 Comparison –8 −47 $658 — −17 –825 — 0 $1,556 South Carolina (Adult) 68 978 $2,378 — 13 103 1 6 $1,208 34 683 $898 — 33 105 0 3 $40 33 295 $1,480 — −20 −2 1 2 $1,169 Note: Estimates are for the average (per offender) costs. Costs are rounded to the nearest dollar; thus, differences and sums may vary by $1. Dollar signs are suppressed in central cells of the table. a Incremental cost = SVORI cost – Comparison cost. Coordination services during incarceration Transition services during incarceration Family services during incarceration Child services during incarceration Mental health services Substance use services Employment/education/life skills services Medical/dental Total Service Domain Incremental Costa Pennsylvania SVORI Ohio Comparison Iowa Incremental Costa Exhibit 13. Annual Pre-release Cost of Service Provision to Men, by Domain (2007$) Results 35 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative Exhibit 14. Annual Pre-release Cost of Service Provision to South Carolina Juveniles, by Domain (2007$) Service Domain Coordination services during incarceration Transition services during incarceration Family services during incarceration Child services during incarceration Mental health services Substance use services Employment/education/life skills services Medical/dental Total SVORI $5,062 38 — — 83 591 104 1011 $6,889 South Carolina (Juvenile) Incremental Comparison Costa $1,592 $3,470 59 −21 — — — — 60 23 730 −139 27 77 942 69 $3,409 $3,480 Note: Estimates are for the average (per offender) costs. Costs are rounded to the nearest dollar; thus, differences and sums may vary by $1. Dollar signs are suppressed in central cells of the table. a Incremental cost = SVORI cost – Comparison cost. Exhibit 15. Number of Services Included in the Cost Analysis, by Site and Domain Service Domain Coordination services during incarceration Transition services during incarceration Family services during incarceration Child services during incarceration Mental health services Substance use services Employment/education/life skills services Medical/dental Total Iowa Ohio Pennsylvania South Carolina (Adult) South Carolina (Juvenile) 5 4 4 5 2 12 2 1 3 1 3 — 3 7 — 1 3 7 — — 3 7 1 — 3 7 — — 3 7 6 5 41 3 5 25 1 5 21 3 5 27 1 5 19 number of services (41) included in the analysis. South Carolina had the next highest number of services (27) included in the analysis. The large number of services for Iowa reflects the fact that service provision there is perhaps more aligned than other sites toward a standard package of services. Hence, offenders in the SVORI group received the complete package of services over and above standard reentry programming received by offenders in the comparison group. In all adult sites—including 36 Results Iowa—the coordination services domain contained four or five distinct services. Additional analyses showed that the case management service in particular contributed to this domain. For the juvenile site, two distinct services were included in this domain, and, again, the cost comprised mainly case management (result not shown). SENSITIVITY ANALYSES OF PRE-RELEASE COST ESTIMATES Exhibit 16 presents the findings from the sensitivity analyses by site. In economic evaluations, sensitivity analyses are used to determine the degree to which the conclusions are robust to any assumptions made in forming the estimates. As noted above, some aspects of the data on price and dosage relied on assumptions, and every cost estimate involves combinations of these assumptions. All the assumptions made were identified and documented by site. As an example of an assumption, the available data indicated whether an individual received individual counseling but did not indicate the number of sessions that individual received. To estimate the cost for individual counseling, an assumption was thus made on the number of sessions received, conditional on receiving the service at all. In the sensitivity analysis the assumption on the number of sessions was changed to see how it affected the costs. The sensitivity analyses thus produce a maximum, a minimum, and a range (maximum – minimum) around the estimate. At $77, the range for Pennsylvania was the tightest of all five sites. The small range indicates that changing the assumptions had little effect on the incremental cost of SVORI. South Carolina adult had the next smallest range at $256, followed by Iowa ($821), Ohio ($1,270), and South Carolina juvenile ($1,500). COST-BENEFIT ANALYSIS RESULTS CBA Results for Men Our CBA provides estimates of monthly net costs (services plus arrests plus incarceration) at each of the four waves of the offender survey. In keeping with the main outcome evaluation, results are not reported by site. Exhibit 17 provides the estimates of the main findings for the four sites with adult male 37 38 $2,221 $2,679 $2,003 $5,613 Ohio Pennsylvania South Carolina adult South Carolina Juvenile $2,797 $646 $2,056 $1,152 $1,870 $2,816 $1,357 $623 $1,069 $845 Compari- Incremental son Costa $6,889 $2,378 $3,497 $3,361 $3,808 SVORI $3,409 $898 $2,839 $1,663 $2,646 Comparison $3,480 $1,480 $658 $1,697 $1,162 Incremental Costa Base Estimate $8,360 $2,772 $4,467 $4,527 $5,094 SVORI $4,044 $1,160 $3,767 $2,188 $3,428 Comparison Maximum 4,316 $1,612 $700 $2,339 $1,666 Incremental Costa Note: The table describes the range of costs per person from the sensitivity analyses. Numbers may not sum or difference because of rounding. a Incremental cost = SVORI cost – Comparison cost (per person). $2,715 SVORI Iowa State Minimum Exhibit 16. Results of Sensitivity Analyses on Pre-release Costs (2007$) 1,500 $256 $77 $1,270 $821 Range An Economic Evaluation of the Serious and Violent Offender Reentry Initiative Results Exhibit 17. Main CBA Findings for the Adult Programs: Monthly Net Cost per Offender, by Study Period (2007$) Cost Number of observations Services SVORI Comparison Difference (SVORI – Comparison) Arrests SVORI Comparison Difference (SVORI – Comparison) Incarceration SVORI Comparison One Month before Release Three Months after Release Nine Months after Release Fifteen Months after Release 722 459 484 516 $242 [$229, $256] $121 [$107, $135] $315 [$269, $361] $99 [$71, $128] $222 [$183, $260] $117 [$78, $156] $165 [$138, $193] $68 [$45, $92] $121** [$102, $141] $216** [$161, $269] $105** [$50, $160] $97** [$60, $133] $384 [$228, $540] $599 [$77, $1121] $519 [$410, $628] $441 [$334, $549] $565 [$399, $730] $505 [$386, $624] N/A −$215 [−$762, $332] $78 [−$76, $231] $60 [−$145, $264] $1,931 [$1,886, $1,975] $1,923 [$1,871, $1,976] $60 [$25, $96] $66 [$29, $104] $351 [$273, $430] $379 [$280, $478] $648 [$543, $754] $556 [$442, $670] −$6 [$57, $46] −$28 [−$155, $99] $92 [−$64, $248] $444 [$273, $616] $665 [$137, $1193] $870 [$715, $1,025] $821 [$657, $984] $1,213 [$1,013, $1,413 $1,061 [$887, $1,236] −$221 [−$778, $336] $49 [−$177, $275] $152 [−$115, $417] [$2,124, $2,222] $2,044 [$1,985, $2,103] $759 [$580, $939] $765 [$233, $1,296] $1,092 [$929, $1,254] $938 [$772, $1,103] $1,378 [$1,176, $1,580] $1,130 [$956, $1,304] $129** [$51, $205] −$6 [−$569, $558] $154 [−$79, $387] $248* [−$19, $516] N/A N/A Difference $8 [−$62, $77] (SVORI – Comparison) Criminal justice (arrest + incarceration) subtotal $1,931 SVORI Comparison [$1,886, $1,975] $1,923 [$1,871, $1,976] Difference $8 [−$62, $77] (SVORI – Comparison) Total net cost (services + arrests + incarceration) $2,173 SVORI Comparison Difference (SVORI – Comparison) Note: Estimates are for the average monthly (per offender) costs. Costs are rounded to the nearest dollar. Estimates use propensity score weights, using a population average treatment effect formulation. Each average cost estimate is presented with a 95% confidence interval in brackets. **p < 0.05. *p < 0.10. 39 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative offenders. Statistical significance of the difference between the enhanced reentry and comparison groups is indicated by asterisks below the tables in this section. One asterisk indicates marginal significance (p < 0.10); two asterisks indicate significance at p < 0.05. A positive estimate indicates that the group receiving enhanced reentry cost more, on average, whereas a negative estimate indicates potential cost savings. The total net cost estimates (bottom set of rows in the tables) indicate that, as expected, there was a statistically significant net monthly cost over the 12 months before release (p < 0.05). At that point, enhanced reentry through SVORI had $129 higher net costs per offender per month. Almost all of this ($121 of the $129 difference) is service programming. Offenders had almost the same incarceration costs and no arrest costs in this period. None of the differences in net costs at each of the three periods after release are statistically significant at conventional levels. Three months after release, the average monthly net cost of the two groups was very close. The $6 cost savings for the SVORI group was small. Nine months after release, enhanced reentry was associated with $154 higher monthly costs (not statistically significant; p > 0.10; 95% confidence interval [CI] = −79,387). At 15 months after release, enhanced reentry was associated with $248 higher monthly net costs, which is marginally statistically significant (p < 0.10; 95% CI = −19,516). The estimates indicate that, throughout the period of study, enhanced reentry was associated with more resources being used for services. The difference between the enhanced reentry and comparison groups in monthly costs for services spiked at the 3-month interview—at $216 (p < 0.05)—and then diminished over time. At the 9-month follow-up interview, the monthly cost difference was $105 (p < 0.05), and at the 15-month interview the difference was $97 (p < 0.05). The difference between the enhanced reentry and comparison groups in criminal justice costs—which comprise arrest plus incarceration costs—is not statistically significant at any of the three follow-up waves. The estimates indicate that at the 3-month follow-up, the comparison group incurred $221 higher monthly criminal justice costs, whereas at 9 and 15 months, the enhanced reentry group incurred higher monthly criminal justice costs at $49 and $152, respectively. However, because 40 Results of the lack of precision in the estimate, the sign and magnitude of any of these findings cannot be reliably interpreted. Examining the component costs of criminal justice costs— arrests and incarceration—provides some additional insight. At no follow-up wave is the difference between the enhanced reentry and comparison groups statistically significant for either arrest or incarceration costs. There are notable instances by wave and SVORI group where the arrest cost or incarceration cost estimates have particularly wide confidence intervals. Consider, for example, average monthly arrest costs for the comparison group 3 months after release. The estimate is $559 and has a wide 95% confidence interval of $77 to $1,129. The results of the sensitivity analyses for men are presented in Exhibit 18. These analyses provide minimum and maximum estimates around the main findings. Overall, the sensitivity analyses do not change the conclusions from the main findings. Service costs for the enhanced reentry group were higher at every wave, with the difference peaking 3 months after release. As for the main analysis findings, differences in criminal justice costs were not significantly different at any of the waves. This holds for criminal justice costs as a whole, as well as for its two components, arrests and incarceration. Exhibit 19 shows the main findings for the South Carolina juvenile reentry site. The differences in monthly service costs between the SVORI enhanced reentry group and the comparison were pronounced before release ($282; p < 0.05) and 3 months after release ($330; p < 0.05). Thereafter, service costs were similar for the two groups. At the four sites for adult male offenders, there were no statistically significant differences in criminal justice costs (arrest and incarceration costs combined) at any of the followup interviews. Also, the data did not reveal reliable differences in arrest costs at any of the three follow-up periods. However, there was evidence that incarceration costs were higher for the juveniles in enhanced reentry for at least one of the follow-up periods. Average monthly incarceration costs for juveniles in the SVORI group were higher than the comparison by $486 at 9 months (p < 0.05; 95% CI = $47, $915) and by $859 at 15 months (p < 0.05; 95% CI = $180, $1,537). 41 42 $1,600 [$1,534, $1666] $679 [$154, $1,204] $802 [$654, $950] $923 [$774, $1,071] $1,759 [$1,702,$ 1,815] $624 [$459, $789] $928 [$783, $1,074] $1,170 [$983, $1,356] One month before release Three months after release Nine months after release Fifteen months after release $247* [–$7, $486] $126 [–$81, $334] –$55 [–$608, $498] $159** [$72, $245] Differencea $1,378 [$1,176, $1,580] $1,092 [$929, $1,254] $759 [$580, $939] $2,173 [$2,124, $2,222] SVORI $1,130 [$956, $1,304] $938 [$772, $1,103] $765 [$233, $1,296] $2,044 [$1,985, $2,103] Comparison $248* [–$19, $516] $154 [–$79, $387] –$6 [–$569, $558] $129** [$51, $205] Differencea Net Cost Base Estimate $1,489 [$1,269, $1,708] $1,227 [$1,043, $1,411] $923 [$700, $1,146] $2,271 [$2,218, $2,324] SVORI $1,211 [$1,023, $1,399] $1,131 [$896, $1,365] $915 [$352, $1,478] $2,103 [$2,040, $2,165] Comparison $278* [–$13, $567] $96 [–$203, $396] $8 [–$600, $616] $168** [$86, $251] Differencea Maximum Net Cost $31 $58 $63 $39 Range Note: The table describes the range of average monthly (per offender) costs from the sensitivity analyses. Estimates use propensity score weights, using a population average treatment effect formulation. Costs are rounded to the nearest dollar, and numbers may not sum because of rounding. Each estimate is presented with a 95% confidence interval in brackets. a Difference = SVORI cost – Comparison cost (per person). **p < 0.05. *p < 0.10. Comparison SVORI Study Period Minimum Net Cost Exhibit 18. Sensitivity Analyses for Cost-Benefit Analysis of Four Sites for Adult Male Offenders (2007$) An Economic Evaluation of the Serious and Violent Offender Reentry Initiative Results Exhibit 19. Main CBA Findings for the South Carolina Juvenile Program: Monthly Net Cost per Offender, by Study Period (2007$) One Month before Release Three Months after Release Nine Months after Release Fifteen Months after Release 79 56 62 65 $570 [$467, $672] $288 [$224, $352] $430 [$365, $496] $100 [$65, $136] $145 [$114, $176] $121 [$62, $180] $95 [$63, $127] $46 [$24, $67] $282** [$158, $406] $330** [$253, $407] $24 [–$45, $93] $49** [$9, $89] $453 [–$53, $959] $150 [–$136, $436] $844 [$299, $1,389] $849 [–$442, $2,139] $341 [$51, $631] $760 [$265, $1,255] N/A $303 [–$302, $908] –$5 [–$1,457,$1,448] –$419 [–$1,012, $175] $4,110 [$4,110, $4,110] $4,110 [$4,110, $4,110] $15 [–$8, $38] $72 [–$65, $209] $587 [$199, $975] $101 [–$41, $243] $1,021 [$380, $1,662] $162 [$25, $299] Difference $0 –$57 [$0, $0] [–$202, $88] (SVORI – Comparison) Criminal justice (arrest + incarceration) subtotal $4,110 $468 SVORI $486** [$57, $915] $859** [$180, $1,537] $1,431 [$660, $2,202] $950 [–$388, $2,288] $1,362 [$699, $2,024] $921 [$390, $1,453] $481 [–$1,119, $2,082] $441 [–$439, $1,319] Cost Number of observations Services SVORI Comparison Difference (SVORI – Comparison) Arrests SVORI Comparison Difference (SVORI – Comparison) Incarceration SVORI Comparison Comparison N/A N/A [$4,110, $4,110] $4,110 [$4,110, $4,110] [–$47, $983] $222 [–$201, $645] Difference $0 $246 [$0, $0] [–$448, $940] (SVORI – Comparison) Total net cost (services + arrests + incarceration) $4,680 $899 SVORI Comparison Difference (SVORI – Comparison) [$4,577, $4,782] $4,398 [$4,334, $4,462] [$387, $1,410] $322 [–$93, $738] $1,576 [$807, $2,345] $1,071 [–$268, $2,410] $1,456 [$799, $2,114] $967 [$433, $1,501] 282** [$158, $406] 577* [–$110, $1262] $505 [–$1,096, $2,106] $489 [–$387, $1,366] Note: Estimates are for the average monthly (per offender) costs. Costs are rounded to the nearest dollar. Estimates use propensity score weights, using a population average treatment effect formulation. Each average cost estimate is presented with a 95% confidence interval in brackets. **p < 0.05. *p < 0.10. 43 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative The number of observations in the juvenile data varies from 56 observations at 3 months post-release to 79 observations before release. This number proved too low to yield reliable results in additional multivariate analyses that control for potential confounders (e.g., age). The low number of observations also likely contributes to the large confidence intervals around many of the estimates. The results of the sensitivity analyses for the juvenile site are presented in Exhibit 20. These analyses do little to change the main conclusions, either with regard to overall costs or to each of the three components of costs (i.e., services, arrests, and incarceration). SUMMARY OF RESULTS The main metric of pre-release costs for each site is the incremental cost—that is, the difference in the average cost for a SVORI offender and a comparison offender before release from prison. The results indicate a considerable range of incremental costs for the 12 months before release. Costs varied from $658 per offender for the Pennsylvania adult program to $3,480 for the South Carolina juvenile program. Two other metrics of cost—reflecting the absolute level of service provision and the ratio of SVORI costs to the comparison—also indicate substantial variation across sites. Finally, estimates of the ratio of SVORI to comparison costs were particularly high for the South Carolina adult program. There, for every dollar spent on the services included in the analysis for a comparison offender, approximately $2.60 were spent on a SVORI offender. The domain that contributed most to the difference in prerelease costs seems to be case management. The exception was in Iowa, where the difference was driven by employment/ education/life skills and substance use services. This finding is reflected in the sensitivity analyses. These indicate that there is a large range around the base estimates, and the results seem particularly sensitive to changes in case management costs. The CBA results confirm that enhanced reentry through SVORI resulted in more resources being spent on services to offenders. For men, the greater service cost persisted throughout all three follow-up waves. For South Carolina 44 $4,347 [$4,293, $4,402] $224 [–$72, $521] $800 [–$194, $1,795] $729 [$328, $1,131] $4,575 [$4,487, $4,664] $663 [$283, $1,043] $1,236 [$611, $1,861 $1,312 [$679, $1,944 One month before release Three months after release Nine months after release Fifteen months after release $583 [–$193, $1,358] $436 [–$782, $1,653] $439* [–$63, $940] $228** [$121, $335] Differencea $1,456 [$799, $2,114] $1,576 [$807, $2,345] $899 [$387, $1,410] $4,680 [$4,577, $4,782] SVORI $967 [$433, $1,501] $1,071 [–$268, $2,410] $322 [–$93, $738] $4,398 [$4,334, $4,462] Comparison $489 [–$387, $1,366] $505 [–$1,096, $2,106] $577* [–$110, $1,262] $282** [$158, $406] Differencea Net Cost Base Estimate $1,474 [$817, $2,131] $1,621 [$853, $2,389] $1,012 [$502, $1,523] $4,800 [$4,682, $4,918] SVORI $979 [$445, $1,513] $1,110 [–$230, $2,450] $358 [–$55, $771] $4,450 [$4,375, $4,525] Comparison $495 [–$381, $1,372] $511 [–$1,090, $2,112] $654* [–$29, $1,338] $350** [$206, $493] Differencea Maximum Net Cost $671 $75 $215 $122 Range Note: The table describes the range of average monthly (per offender) costs from the sensitivity analyses. Estimates use propensity score weights, using a population average treatment effect formulation. Costs are rounded to the nearest dollar, and numbers may not sum or difference because of rounding. Each estimate is presented with a 95% confidence interval in brackets. a Difference = SVORI cost – Comparison cost (per person). ** p < 0.05. *p < 0.10. Comparison SVORI Study Period Minimum Net Cost Exhibit 20. Sensitivity Analyses for the Cost-Benefit Analysis of the South Carolina Juvenile Site (2007$) Results 45 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative juveniles, greater service costs were seen before and 3 months after release; 9 and 15 months after release their difference in service costs was not statistically significant. Estimates of net costs—which combine service costs with criminal justice costs—were imprecisely estimated. Because the results were generally not statistically significant, there was no clear evidence either that the SVORI group cost more or that it cost less than the comparison group at any of the follow-up periods. Sensitivity analyses varied key assumptions underlying the services, arrest, and incarceration cost estimates. The summary of those analyses indicate that those results did little to change the conclusions reached in the main findings. Because the service costs in particular relied on many assumptions, considerable attention was given to the sensitivity analyses for service costs. Even if the analyses were to entirely set aside the service cost estimates and focus on criminal justice costs alone, the results of the CBA remain equivocal. The estimates of differences in criminal justice costs between the enhanced reentry and comparison groups were not statistically significant for any of the follow-up waves for either men or juveniles. We discuss the implications of these findings and directions for future research in the Conclusions section. 46 Conclusions DISCUSSION OF FINDINGS In an environment of heavily scrutinized state budgets, information on how much reentry services cost should be of great use to policy makers, providers, and researchers. In 2003, the U.S. Departments of Justice, Labor, Housing and Urban Development, and Health and Human Services established SVORI, a large-scale program providing more than $100 million to 69 grantees to develop programming, training, and state-of-the-art reentry strategies at the community level. The multi-site evaluation of SVORI examined 16 different reentry programming models, and the economic evaluation focused on five sites to ascertain the impact, costs, and benefits of reentry programming. This report presents the results of an economic evaluation of the enhanced reentry efforts funded through SVORI. It provides the results from two separate analyses: an analysis of service costs before release and a cost-benefit analysis that spanned both before and after release. The results should be understood in the context of the study’s potential limitations. The analyses of the juvenile data faced two particular challenges that limit the interpretation of their findings. First, there were too few observations to provide precision on the estimates, even with unadjusted comparisons of means. A second limitation was that analytic weights were derived from all four adult sites to attempt to control for potential confounders in the South Carolina site. The covariates at that site were not balanced when using these propensity weights, meaning that potential confounding from other covariates remained. Additional multivariate regression could not yield reliable estimates, because the sample was insufficiently large. 47 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative As noted throughout the report, the data used to measure service utilization in all sites were not ideal. Rather than tracking detail on the number of days, events, or units of each service received by each offender, the data indicated only whether an offender reported receiving a specific service. Moreover, the raw data cannot indicate the context of receiving each service; for example, the raw data do not reveal whether an offender receiving group mental health counseling and anger management received both in the same session. To address both of these limitations, the data were supplemented with a number of sources, including interviews with providers and stakeholders on typical service provision, syllabi of classes and programs, expert opinion, and the literature. Two additional data limitations pertain to the measures of criminal justice events. First, the offender survey provided counts of the number of bookings at each interview wave. Each booking event was assumed to be associated with one arrest. Although this assumption is unlikely accurate in the event of every booking, no additional data were available to fine-tune the mapping between booking and arrests. In the absence of an alternative estimate of the ratio of arrests-to-bookings in published literature or reports, this assumption was not varied in the sensitivity analyses. A second limitation was that the survey did not distinguish between a count of jail and prison nights; all nights incarcerated were assumed to be nights in prison. This assumption was also not varied in the sensitivity analyses. A final limitation is that the results may not be greatly generalized from the sites included in the analysis. The nature of the funding mechanism gave grant recipients considerable lattitude in how to spend the funds. Moreover, the sites were selected because they were particularly strong in at least one of four design criteria, Although this reasoning increased the chances of finding interpretable results, it also set the sites apart from other sites in the analysis. IMPLICATIONS FOR POLICY AND PRACTICE The study focused on offenders reentering society in 2004– 2005, a period during which states were beginning to implement enhanced reentry. Since that time, many of the states conducting reentry have developed their programs 48 Conclusions further. This ongoing evolution of programming, to some degree, means that the target of any evaluation is constantly moving. Nevertheless, the results of the evaluation are instructive because it examined an unprecedented, one-time injection of funding across states. Although all states received additional funding in the range of $1 million to $2 million, what states did with those funds to enhance reentry clearly varied and likely depended on several factors, such as the status quo of reentry services before the SVORI grant (e.g., Lattimore et al., 2008), the vision of decision makers on how additional funds should be spent, the idiosyncrasies of the population being targeted for services, the state and local policies that shaped the operating environment for those services, and the numerous constraints that service providers and departments of corrections faced in enhancing service receipt. The focus of this report on incremental costs— meaning the resource difference between business-as-usual and enhanced reentry in 2004–2005—helps account for the baseline differences in states at the time of reentry. Among the adult sites, the results indicate a considerable range of incremental costs, varying from $658 per offender for the Pennsylvania adult program to $3,480 per offender in South Carolina’s juvenile program. The domain that typically contributed most to the difference in pre-release costs was case management, indicating that case management was the service most heavily targeted for enhanced reentry. The exception was in Iowa, where the difference was driven by employment/education/life skills services. This finding is reflected in the sensitivity analyses. These analyses indicate that there is a large range around the base estimates, and the results seem particularly sensitive to changes in case management costs. The CBA provided estimates of net costs, which combined service, arrest, and incarceration costs. At none of the three points studied after release were the estimates of differences between the enhanced reentry and comparison groups statistically significant. This finding holds for both men and juveniles. Because the results were generally not statistically significant, there was no strong evidence either that the SVORI group cost more or that it cost less than the comparison at each of the follow-up periods. 49 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative Enhanced reentry through SVORI succeeded in providing offenders more services for both men and juveniles. This was apparent not just for the pre-release period, but also after release, when the state had far less control over offenders’ actions. For men, the findings suggested that, even as long as 15 months after release, the average monthly service cost was $97 higher for those in the enhanced reentry group than in the comparison. For juveniles, the findings suggested that service provision for enhanced reentry was particularly high within the first 3 months after release. The difference in average monthly costs for the enhanced reentry and the comparison groups was $330. The natural variation in the data perhaps limits the ability to detect differences in costs between the groups. Despite having several hundred observations in the data for men, the criminal justice costs have wide 95% CIs or high standard errors of estimates. The reason for this statistical imprecision is likely the large variation across offenders in their individual costs of arrest and incarceration. Some people are arrested a lot, whereas others are not arrested at all; likewise, some people are incarcerated for almost the entire study period, whereas others are not incarcerated at all. When expressed in dollar terms, this gives a large standard deviation. That, in turn, drives a large standard error of estimate of the mean. The estimates in this report should be put in context of the broader literature on economic evaluations of programs for offenders and the policy environment within which SVORI was operating. With regard to the context of the broader literature, consider the findings in the literature on the costs and benefits of providing basic substance abuse programming. Such programming targets a high-need population, like the one examined by this evaluation, with a clearly defined, evidencebased regimen of services. Also, unlike any other prison-based service, such as case management, there are a number of published peer-reviewed studies available in the literature on substance abuse services. The drawback of comparing to substance abuse programming is that it is fairly expensive; however, it is only one service, whereas reentry programming is intended to cover many services. McCollister et al. (2003) used detailed methods to derive cost estimates for the Amity program in California in the early 1990s. The authors found that the program typically lasted just under 1 year before 50 Conclusions release and cost $3,886 per offender and that the aftercare program was cost-effective. b Daley et al. (2004) examined the Connecticut substance abuse programming in the mid-1990s. The authors examined four tiers of programming ranging from a low-intensity, week-long daily drug education class to a therapeutic treatment community, similar to the Amity program. The estimates (in adjusted 2007 dollars) ranged from $250 for the drug education class to $7,533 for attendees and $20,520 for completers of the therapeutic community. The authors found that the more resource-intensive tiers of programming—such as the therapeutic community—were cost-beneficial. Both McCollister and Daly came to conclusions that support the programs being studied, perhaps because the programs were well-funded demonstration projects. Given that the additional resources leveraged by SVORI amounted to no more than $1,200 per offender in adult sites, the pre-release findings suggest that the grant funds were spread thin over the reentering offender population. This gives context to the finding of equivocal results, which indicated that the data did not indicate differences in criminal justice costs between the enhanced reentry and comparison groups. In addition to understanding the context of findings from the literature, it is also important to understand the context of the policy environment in which SVORI and its evaluation took place. The evaluation took place during the first years of sites’ implementation of enhanced reentry. Developing and implementing the panoply of services for a comprehensive reentry program within a prison environment is likely to take time to fully realize. The results presented here and in accompanying reports of the broader evaluation (e.g., Lattimore et al., 2009) indicate that additional funding can, at a minimum, lead to an increase in the types and amounts of necessary services that are provided to men and juveniles before and after release from prison. The findings are, thus, consistent with sites establishing a foundation upon which to build better programs. Winterfield, Lindquist, and Brumbaugh (2007) reported in 2006 that most states, according to surveys b The estimate was $2,708 in 1993 dollars; inflation was adjusted for using the CPI (U.S. Bureau of Labor Statistics, 2009). All estimates quoted from the literature were similarly adjusted. 51 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative of the SVORI program directors, were continuing to build on the programs that they had established with SVORI grant funds. IMPLICATIONS FOR FUTURE RESEARCH Because of the richness of the study data, future research will likely be fruitful. The arrest costs examined in this report did not distinguish by type of arrest. Although information on the cost per arrest by type was available, project resources did not allow for uniquely coding each arrest event by type. Such an exercise could alter the findings if there was a statistically significant relationship between being in the SVORI group and arrest type. Other areas of future research include further examination of the administrative sources of criminal justice events. Recall that administrative incarceration data were not reliable for one of the four adult sites. To maintain cross-site integrity in approach and to ensure consistency between arrest and incarceration data sources, the study used offender survey data. A different analysis might use only administrative data on criminal justice events and expand the analysis sites to all sites in the broader SVORI evaluation that have reliable data. Future work should also consider whether insights can be gained by alternative statistical approaches to modeling costs. It is unclear whether differences in costs would be demonstrated if alternative and more sophisticated modeling approaches were adopted. Future work may explore fitting different statistical distributions on costs, thereby transforming the cost estimates (by using, say, the natural logarithm of costs). Other work may separately model the probability of an event occurring in a period and the number of times it occurs conditional on the event occurring at all. For example, one might model being arrested at all in a study period and then the number of arrests conditional on being arrested at all. A final avenue of future work would be to alter the perspective of the economic evaluation. Recall that perspective addresses whose costs and whose benefits are evaluated, and in this analysis the perspective is the criminal justice system. Findings in the main analysis suggest that employment is improved by enhanced reentry, for example. The perspective might be expanded from the criminal justice system to the broader perspective of all taxpayer-supported resources. Although it 52 Conclusions answers a different research question than that addressed in this report, such an analysis may be of great importance to policy makers. Broadening the perspective widens the number of outcomes that must be monetized and included in the analysis. Thus, considerably more project resources may be required to appropriately address the question. The extensive SVORI evaluation data set provides an opportunity for future research to explore each of the above questions. The evaluation also has set in place an infrastructure and approach on which future analyses should build. Importantly, as reentry planning evolves and grows from the preliminary stages examined in this evaluation, future research should update the findings described in this report. 53 References Andrews, D. A., & Bonta, J. (1998). The psychology of criminal conduct (2nd ed.). Cincinnati, OH: Anderson Publishing. Aos, S., Miller, M., & Drake, E. (2006). Evidence-based adult corrections programs: What works and what does not. Olympia, WA: Washington State Institute for Public Policy. Beard, J. A. (2009). Costs and population, 2009-10 budget presentation. Retrieved July 20, 2009, from http://www.cor.state.pa.us/stats/lib/stats/Costs_Pop.pdf Cohen, M. A., Miller, T. R., & Rossman, S. B. (1994). The costs and consequences of violent behavior in the United States. In A. J. Reiss, Jr., & J. A. Roth (Eds.), Understanding and preventing violence: Vol 4. Consequences and control. Washington, DC: National Academy Press. Cowell, A. J., Broner, N., Aldridge, A. P., & Hinde, J. M. (2009). The cost of building a jail diversion program and diverting individuals. Manuscript submitted for publication. Cowell, A. J., Broner, N., & Dupont, R. (2004). The costeffectiveness of criminal justice diversion programs for people with serious mental illness co-occurring with substance abuse: Four case studies. Journal of Contemporary Criminal Justice, 20(3), 292-314. Cowell, A. J., Broner, N., Hinde, J., & Aldridge, A. P. (2009). A cost-benefit analysis of a jail diversion program. Manuscript submitted for publication. Cowell, A. J., Lattimore, P. K., & Krebs, C. P. (in press). A costbenefit study of a juvenile breaking the cycle program. Journal of Research on Crime and Delinquency. Daley, M., Love, C. T., Shepard, D. S., Petersen, C. B., White, K. L., & Hall, F. B. (2004). Cost-effectiveness of Connecticut's in-prison substance abuse treatment. Journal of Offender Rehabilitation, 39(3), 69-92. 55 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative Drummond, M. F., O'Brien, B., Stoddart, G. L., & Torrance, G. W. (2000). Methods for the economic evaluation of health care programmes (2nd ed.). Oxford, England: Oxford University Press. Durose, M., & Langan, P. A. (2003). Felony sentences in state courts, 2000. Washington, DC: U.S. Bureau of Justice Statistics. Federal Bureau of Investigation (2009). Crime in the United States, 2007. Retrieved July 3, 2009, from http://www.fbi.gov/ucr/cius2007/index.html Iowa Department of Corrections (2008). FY2008 annual report. Retrieved July 29, 2009, from http://www.doc.state.ia.us/Documents/2008AnnualRepo rt.pdf Lattimore, P. K., & Steffey, D. M. (2009). The Multi-Site Evaluation of SVORI: Methodology and analytic approach. Research Triangle Park, NC: RTI International. Lattimore, P. K., Steffey, D. M., & Visher, C. A. (2009). Prisoner reentry experiences of adult males: Characteristics, service receipt, and outcomes of participants in the SVORI Multi-site Evaluation. Research Triangle Park, NC: RTI International. Lattimore, P. K., Visher, C. A., & Steffey, D. M. (2008). Prerelease characteristics and service receipt among adult male participants in the SVORI Multi-site Evaluation. Research Triangle Park, NC: RTI International. Lipsey, M. W., & Cullen, F. T. (2007). The effectiveness of correctional rehabilitation: A review of systematic reviews. Annual Review of Law and Social Science, 3, 297-320. MacKenzie, D. L. (2006). What works in corrections: Reducing the criminal activities of offenders and delinquents. New York: Cambridge University Press. Marlowe, D. B. (2006). When "what works" never did: Dodging the "scarlet M" in correctional rehabilitation. Criminology and Public Policy, 5(2), 339-346. McCollister, K. E., French, M. T., Prendergast, M. L., Sacks, S., & Hall, E. A. (2003). Is in-prison treatment enough? A cost-effectiveness analysis of prison-based treatment and aftercare services for substance-abusing offenders. Law & Policy, 25(1), 63-82. 56 References Miller, T. R., Cohen, M. A., & Rossman, S. B. (1993). Victim costs of violent crime and resulting injuries. Health Affairs (Millwood), 12(4), 186-197. Ohio Department of Rehabilitation and Correction (2009). Monthly factsheet: June 2009. Retrieved July 29, 2009, from http://www.drc.state.oh.us/web/Reports/FactSheet/Jun e%202009.pdf Petersilia, J. (2004). What works in prisoner reentry? Reviewing and questioning the evidence. Federal Probation, 68(2), 4-8. Rhine, E., Mawhorr, T., & Parks, E. C. (2006). Implementation: The bane of effective correctional programs. Criminology and Public Policy, 5(2), 347-358. Roman, J., Brooks, L. E., Lagerson, E., Chalfin, A., & Tereshchenko, B. (2007). Impact and cost-benefit analysis of the Maryland reentry partnership initiative. Washington, DC: Urban Institute. Rossman, S. B., Sridharan, S., Gouvis, C., Buck, J., & Morley, E. (1999). Impact of the opportunity to succeed. OPTS aftercare program for substance-abusing felons: Comprehensive final report. Washington, DC: Urban Institute. Seiter, R. P., & Kadela, K. R. (2003). Prisoner reentry: What works, what does not, and what is promising. Crime & Delinquency, 49(3), 360-388. South Carolina Department of Corrections (2008). Cost per inmate fiscal years 1988-2008. Retrieved July 29, 2009, from http://www.doc.sc.gov/research/BudgetAndExpenditure s/PerInmateCost1988-2008.pdf Stephan, J. J. (2004). State prison expenditures, 2001. (NCJ 202949). Washington, DC: U.S. Bureau of Justice Statistics. U.S. Bureau of Justice Statistics (2009). Justice expenditure and employment extracts. Retrieved July 2, 2009, from http://www.ojp.usdoj.gov/bjs/pubalp2.htm#J U.S. Bureau of Labor Statistics (2009). CPI calculator. Retrieved May 15, 2009, from http://www.bls.gov/data/inflation_calcularot.htm 57 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative Visher, C. A. (2006). Effective reentry programs [Editorial introduction]. Criminology and Public Policy, 5(2), 299302. Western, B. (2006). Punishment and inequality in America. New York: Russell Sage Foundation Publications. Wilson, J. A., & Davis, R. C. (2006). Good intentions meet hard realities: An evaluation of the project greenlight reentry program. Criminology and Public Policy, 5(2), 303-338. Winterfield, L., Lindquist, C., & Brumbaugh, S. (2007). Sustaining adult reentry programs after SVORI. The Multi-site Evaluation of the Serious and Violent Offender Reentry Initiative: Reentry research in action. Retrieved December 18, 2009, from http://www.svorievaluation.org/%5Cdocuments%5Creports%5CRRIA_20 07_Adult_Sustainability.pdf 58 Appendix A: Program Descriptions IOWA Background The Iowa Department of Corrections (IDOC) received a $2 million grant and established the Keys Essential to Your Success (KEYS) program in Polk County. It served male and female adult offenders returning to Polk County from any state correctional facility. Similar to other states, Iowa has experienced a dramatic increase in its prison population over the past few decades. State officials attribute this increase to a high prevalence of recidivism among returning adults, in addition to other factors. Polk County, which includes the city of Des Moines, was targeted for intervention because it is the largest county in the state of Iowa and accounts for a significant percentage of annual prison admissions. Moreover, the county had community-based resources already in place to assist in offender reintegration, making the area ideal for SVORI resources. The program engaged a host of services and resources, focusing on creating linkages and fostering coordination among existing institutions to address the needs of returning adults who presented the greatest risk for reoffending. Services were aimed at increasing employment, reducing substance use, and improving mental health among offenders. The services— rendered both pre- and post-release—were intended to address the criminogenic patterns of returning adults while increasing their prosocial skills and ensuring offender accountability. A-1 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative Eligibility Criteria The project drew participants from any state correctional institution who had been identified as returning to Polk County following release. In addition, returning adults must have been between the ages of 18 and 48 for women or between the ages of 18 and 40 for men, assessed with a Level of Service Inventory (LSI-R) score of 25 or higher, confined for at least 12 months, given a tentative discharge date at least 18 months from the beginning of the project start date, identified as having specific employment needs, and identified as having a mental illness and/or issues with substance abuse. Eligible participants were transferred to Newton Correctional Facility and Correctional Release Center, Fort Dodge Correctional Facility, or the Iowa Correctional Institution for Women. These three sites were chosen because they already had the programmatic and resource capacities necessary to address vocational development and treat mental health and substance abuse disorders. Description of Services Offenders who met the eligibility criteria were chosen randomly by the Iowa Division of Criminal and Juvenile Justice Planning to either enter KEYS or be part of the comparison group. Comparison subjects came from the three correctional institutions listed above, as well as the Fort Des Moines Community Correctional Center and the Women’s Residential Correctional Facility. The treatment group received a life skills curriculum prior to release and intensive case management in the community. The comparison group received conventional in-prison programming and parole supervision. Intervention for the KEYS participants proceeded in three phases: (1) the incarceration phase, which lasted approximately 12 weeks prerelease; (2) the transition phase, which spanned offenders’ return to the community, at which point they initially reported to their case manager, employment counselor, and parole officer; and (3) the after-care phase, which continued for 12 months following release. A-2 Appendix A — Program Descriptions The IDOC was the lead agency under the KEYS project. The IDOC manages eight judicial Departments of Correctional Services (DCSs) throughout the state, which administer services to offenders. The Fifth Judicial District DCS, which encompasses Polk County, provided the services either directly or through contractual agreements with outside agencies to program participants. Among the pre-release services were risk and needs assessments, case management, mental health counseling, education and housing assistance, and anger management. Post-release services included, among other things, life skills training, family counseling, transportation assistance, and other unmet needs such as clothing and food. The incarceration phase included a 12-week life skills class taught in the three correctional institutions for 17.5 hours each week by Des Moines Area Community College (DMACC) employees. The course included presentations from the Iowa Workforce Development group, the Institute for Social and Economic Development, and other local and community organizations. Reentry case managers from DMACC visited participants between one and three times to develop a release plan based on their risk and needs assessment. At release, participants were placed under the supervision of parole officers assigned to the KEYS program by the Fifth Judicial District. During this time, the returning participants received a suite of services (managed by DMACC employees) identified during their risk and needs assessment. Case managers focused on mentoring, education, and employment. To facilitate a smooth transition back into the community, every offender spent roughly 3 hours with their case manager in the first month after release. The participants spent less time with their case managers in subsequent periods. Most participants in KEYS participated in a work-release program, while others were released directly into the community. Participants who were not employed attended Job Club three times a week, as well as two 1-hour group classes a week. An additional employment program paid, in full or in part, the first 3 months of wages for the offender who found employment. The program’s goal was to provide incentives to employers who were willing to hire SVORI clients. Reentry planning for the comparison group started at intake with a risk and needs assessment. The results of the A-3 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative assessment were used to develop a case plan and transition/release plan. These plans laid out what services would best benefit the offender; services could include education, vocational training, and motivational enhancement. At 6 months prior to release the offenders were transferred to a minimum security/transition facility near the release community. During the last 6 months of the sentence the offender focused on life skills, meaningful work, family reunification, and release planning. Ohio Background The Ohio Department of Rehabilitation and Correction (ODRC) received a $1,998,014 grant with which it implemented the Community Oriented Reentry (CORE) program. CORE is a comprehensive approach to helping serious, violent, high-risk and high-need incarcerated offenders between the ages of 18 and 35 return to their communities and families. All offenders in the program must have served at least 12 consecutive months in confinement. CORE’s goals were to assist offenders returning home to avoid recidivism, find stable housing, receive substance abuse and mental health treatment, sustain longterm employment, reunite with their families, and become productive, law-abiding citizens in their communities. Prior to receiving the SVORI grant, the ODRC’s reentry program consisted of basic programs and services that varied by institution. These programs did not include reentry teams, family involvement, or coordinated outreach into the community to which the offender would return. In 2002, a strategic document by ODRC concluded that community participation, collaboration, and partnerships were integral to the success of reentering offenders. This document was used as a template of change for the CORE program. Eligibility Criteria CORE services were directed to adult high-risk/high-need offenders (spanning substance abuse, mental health, education, and employment) who were convicted in and returning to Allen, Cuyahoga, and Franklin counties. Additionally, offenders were ages 18 to 35 at the time of release, incarcerated for a minimum of 12 consecutive months, and on community supervision for at least 1 year. The comparison group had the A-4 Appendix A — Program Descriptions same eligibility criteria as the CORE participants and was selected from the pool of eligible and voluntary offenders during the randomization process in the enrollment period. Description of Services The program emphasized effective connections with the Adult Parole Authority and community service providers and linking participants to needed resources such as medical care and housing before they were released to avoid a delay in services. CORE worked through 12 institutions that were returning offenders to three counties. Each facility had slightly different programming and catered to different populations, as necessary. CORE activities began about a year before release and comprised screening and needs assessments, the development of accountability plans, and service programming in the facility. After release, CORE comprised an accountability plan meeting and needs assessment, several post-release meetings, and a post-release assessment. Each offender was initially processed through a reception center, during which a risk assessment was administered. If the inmate scored as being at high risk and in high need, the individual was eligible for enhanced reentry via CORE. Inmates volunteering to enter CORE were randomly assigned to either CORE or a control group. Pre-release services were delivered at the facility housing the inmate. CORE delivered regular reentry services by offering a reentry management team (RMT), comprising a community case manager, an institutional case manager, a parole officer, and treatment staff. The RMT met monthly to discuss the offender’s forthcoming reentry plan. Once the offender began CORE, the RMT administered a more focused needs assessment, developed a service plan with the offender, and helped the offender access needed services. Two employment services in particular were funded with the SVORI grant. The same providers offered services to both the CORE and control group inmates, but CORE participants had priority over other prisoners for services. For example, for substance abuse treatment, preference was given to CORE inmates. Medical and dental services also were available if needed, and CORE participants did not receive priority with these services. At release, both CORE participants and individuals in the control group were placed under community supervision. A community A-5 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative reentry coordinator conducted intensive case management with CORE participants and worked closely with the parole officer and community service providers. Within 72 hours of release, the CORE participant met with the case manager and parole officer to review the reentry plan to ensure that all needs were being met. The main difference between CORE and the control services in the period after release was that CORE participants had access to financial assistance, resources for securing documents, and public transportation vouchers. Additionally, CORE participants had priority for all post-release services provided by an agency or organization that had a contract with ODRC. For the first 3 months after release, the RMT conducted weekly meetings with CORE participants. The RMT during postrelease had the same constituents as pre-release except it excluded most of the institutional staff. Six months postrelease, the frequency of the RMT meetings was typically reduced to meeting with the CORE participant every other month. Pennsylvania Background The Pennsylvania Department of Corrections received a grant of $1,990,990 and established the Erie Pennsylvania Reentry Project (EPRP), which focused on adult offenders returning to Erie County from September 2003 through 2006. This county was targeted because of both the volume of offenders returning to this geographic area and the higher than average recidivism rates of the returning adults. Erie typically has relatively high unemployment and poverty rates, due, in part, to declines in manufacturing jobs for low-skilled and minority laborers. The socioeconomic challenges in Erie are particularly detrimental to a returning offender population that has limited skills, has minimal work experience, and is disproportionately minority. Eligibility Criteria The project intended to serve both male and female adults from any of the secure state correctional institutions (SCI) who indicated they were returning to Erie County upon release. Project participants were identified by staff members within the state institutions and, in addition to their Erie residence, must have been between the ages of 18 and 35 upon release or rerelease on state parole by the Pennsylvania Board of Probation and Parole (PBPP). Entry into the program was voluntary; yet, A-6 Appendix A — Program Descriptions following the offenders’ enrollment in EPRP, program participation became a special condition of release. Offenders selected for the program were those receiving the highest LSI-R scores and/or those who had already violated the conditions of their parole—two indicators of substantial risk of recidivating. Offenders selected for the comparison group were those who met the eligibility criteria but who did not volunteer for SVORI. The comparison group thus comprised state parolees, state re-parolees (those who had violated their parole and served an additional portion of their original sentence), technical parole violators with community parole center placements, and pre-release individuals between the ages of 18 and 35 who were returning to Erie County. Description of Services To address the needs of the adults returning to Erie County and thereby reduce the rate of recidivism, the EPRP provided a continuum of services pre- and post-release. Among the prerelease services were risk and needs assessments, substance abuse and mental health treatment, medical and dental services, and life and parenting skills training. The post-release services included risk and needs assessments, education, housing assistance, and mentoring. To leverage existing resources, the Greater Erie Community Action Committee (GECAC) was the point of contact for all services rendered in the EPRP. The GECAC has been providing services to Erie County residents for more than 30 years with the goal of eliminating poverty and improving life quality. In fact, adults returning to Erie County from prison have received GECAC services for several years. GECAC provides a complement of services appropriate for ex-offenders, including drug and alcohol treatment and employment training, which were supplemented under the SVORI grant. Eligible participants were initially transferred to Albion SCI (men) or Cambridge Springs SCI (women) to begin the threepart EPRP programming. Phase 1 occurred while offenders were in prison at Albino SCI or Cambridge Springs SCI and continued for approximately 1 year. During this phase, a case manager (community service specialist) and a parole officer met with offenders to prepare them for their impending release and to provide details about their participation in the program. Phase 1 A-7 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative also included the delivery of services previously mentioned, such as risk and needs assessments and job readiness training. Following this phase, all EPRP participants were paroled to a community corrections facility in Erie County (Phase 2). Men were generally paroled to the Erie County Community Correctional Center (CCC), which is a state-run CCC that provides housing and services to parolees. Women were paroled to Gaudenzia, which is a privately operated CCC that also provides housing and treatment services to parolees. Occasionally, men who have been identified as having substance abuse problems are paroled to Gaudenzia as well. The majority of service receipt took place in Phase 2. GECAC provided services to parolees or contracted with other agencies to provide services directly to the program participants. As mentioned, these services included life skills training, mental health and substance abuse counseling, and general case management. The average length of stay for EPRP participants in these facilities was 90 days. This service provision continued through the final phase (Phase 3), when clients were released from the CCC into the community. Individuals in the comparison group were paroled to Gateway Erie, a secure community corrections facility that provides housing and limited residential treatment. Gateway Erie provides a portion of the services to its residents, but most services are delivered by an external provider following referral. Referred services are financed either by the offender or the Erie County Department of Public Welfare. South Carolina (Adult Program) Background The South Carolina Department of Corrections (SCDC) used a $1,000,002 grant to fund the Going Home program. Before receiving the SVORI grant, the SCDC’s reentry program consisted of services that were on a “first come, first serve” basis. Most inmates were required to take educational and vocational training components, and if substance abuse and mental health issues were identified during an offender’s needs assessment, those services were provided as well. There was little collaboration among state agencies on prisoner reentry issues and, as a result, a smooth transition from correctional institutions back to the community was difficult to achieve. A-8 Appendix A — Program Descriptions Because the SCDC had experienced budget cuts over the past several years, the inmates were receiving less of the vital services they needed. Reentry programs had substance abuse and mental health counselors to address specific health needs but did not have transition coordinators to coordinate services and organize transition plans. There was a disconnect in services and transition planning, and although the services may have been available to most individuals, SCDC lacked a single process to ensure that an offender going through reentry could access all the services that he or she needed. Eligibility Criteria The target population was male inmates ages 17 to 35 who were classified as high risk and had a requirement for community supervision in South Carolina and a minimum of 1 year of incarceration remaining. Offenders were identified during a risk and needs assessment conducted by SCDC staff at Reception and Evaluation Centers, where all offenders are directed before being sent to correctional institutions. Entry into Going Home was voluntary. The program was statewide and served offenders no matter where the inmates needed to continue their post-release treatment. Offenders were first placed in Reception and Evaluation Centers where they were assessed to see if they would be eligible to participate in Going Home prior to entering correctional facilities. If offenders were deemed eligible and volunteered to participate in Going Home, they were then sent to one of seven correctional institutions. Description of Services The pre-release phase for Going Home lasted 18 months, on average. Information from needs and risk assessments was used by transition coordinators to create each offender’s individual action plan (IAP). The IAP always involved a life skills class, education courses, and counseling and assessment from the Vocational Rehabilitation Department. Life skills classes concentrated on key areas such as lifestyle change and attitude adjustment. The program had a set 8-week curriculum, with classes averaging 60 to 90 minutes in length. The transition coordinator—one of which was located at each of the seven Going Home facilities—provided the reentry services that were not provided by formal agencies or classes and often managed a caseload of 50 individuals. The transition coordinators also A-9 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative were responsible for identifying community resources from the counties surrounding their facility. A key element of the program was the transition team. This team consisted of the transition coordinator, a representative from the South Carolina Department of Probation, Parole and Pardon Services, a classification caseworker, the participating offender, associated institutional staff, and, where appropriate, a member of the participating offender’s family. The transition team was designed to provide better communication among all parties regarding the participating offender’s reentry. The transition team actively participated during the participant’s pre-release phase and met once a month to discuss the participant’s progress. Comparison inmates received only education and services from the Department of Vocational Rehabilitation during their prerelease phase, and, when needed, received substance abuse and mental health services. Case management for this group was handled by classification case managers whose main responsibility was to classify offenders and manage caseloads of approximately 250 to 350 inmates. The main difference in post-release services between Going Home participants and the comparison group was that the transition coordinator during pre-release had actively identified existing community resources to match the participant’s needs after release from the correctional institution. Both Going Home participants and comparison inmates were under the supervision of a parole officer after release. However, parole officers were not aware of Going Home and, therefore, did not distinguish participants from comparison inmates. Stakeholders indicated that the SVORI grant did not appear to affect postrelease supervision in any way. South Carolina (Juvenile Program) Background The South Carolina Department of Juvenile Justice (SCDJJ) received a $999,989 grant with which it implemented the Reintegration Initiative. The program targeted indeterminately sentenced juvenile offenders to be conditionally released to the community after serving sentences ranging from 3 months to 54 months. A third of the indeterminately sentenced juvenile offenders possessed a prior commitment to custody, and more A-10 Appendix A — Program Descriptions than 80% had four or more delinquency referrals to the family courts. Although the program’s main aim was to reduce recidivism, the SCDJJ also sought to make the transition from correctional institution to the community more seamless. Eligibility Criteria Reintegration Initiative participants could come from any of the state’s secure, long-term facilities, but they had to reside in Orangeburg, Dorchester, Calhoun, Florence, York, Spartanburg, Marion, or Kershaw Counties for post-release. The participating facilities included four maximum-security institutions and any medium-security community corrections facility. Eligible offenders were those aged between 14 and 18 committed to and released from any of South Carolina’s secure, long-term, or wilderness camp SCDJJ facility for a serious or violent crime or a technical violation related to a serious or violent offense. Additionally, those eligible must have been assessed as high risk, indeterminately sentenced for a minimum sentence of 3 to 6 months, determinably sentenced for at least 90 days with probation supervision to follow release, returning to one of eight counties on release, and supervised by a SCDJJ community caseworker. Finally, juvenile offenders selected to participate in the Reintegration Initiative participated on an involuntary basis. Incoming juvenile offenders who were required to transfer to an adult facility at the age of 18 were excluded. The comparison group was selected based on the abovementioned criteria, except that they were incoming juvenile offenders that were returning to counties that were not participating in the Reintegration Initiative. Possible offenders who were returning to counties with similar demographic and juvenile criminal history statistics were comparison subjects. Description of Services The Reintegration Initiative set a goal to create collaborative partnerships, and in doing so, the SCDJJ obtained memoranda of understanding with 26 state and local partners. At the local level, each county created a planning and review team that consisted of the county’s key partners. Juvenile offenders were first assessed at one of three regional evaluation centers and then assigned to a program. Within a week of enrolling a juvenile offender into a correctional facility, A-11 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative a community caseworker met with the participant’s institutional caseworker to construct a plan for initial treatment. Within 30 days of arrival, the institutional reentry team, consisting of the institutional caseworker, security staff, educational staff, a psychologist or psychiatrist, classification staff, and a community caseworker, met to discuss and review the development plan that was formulated at the regional evaluation center and developed a more comprehensive reentry plan. The institutional caseworker and the community caseworker were responsible for service coordination while the participant was committed to the institution. Beginning 90 days before an offender’s release, the community caseworker and the institutional caseworker coordinated the transition from pre-release to post-release. An aftercare treatment plan was completed during this phase, and important dates and times for school admission and appointments with service providers were included. The level of supervision and the role of a community support team also were established in the plan. At release, the community caseworker met with the participant and the participant’s family to review the aftercare treatment plan and parole guidelines. For at least 90 days after release, the community caseworker assisted the participant with school or vocational job program enrollment and scheduling of appointments. The community caseworker had the responsibility of overall service coordination, monitoring participant progress, and implementing graduated incentives and sanctions. A separate planning and review committee reviewed cases monthly. To enter the final phase of reentry, individuals had to be in compliance with the conditions of release and school or job attendance. The final phase of reentry lasted as long as a participant was under parole supervision or SCDJJ probation, typically from 6 months to 2 years. The intensive supervision and community services were gradually phased out, and the community support team assumed the primary support to the participant and the family. A community caseworker continued to monitor the participant’s progress and maintained contact with the participant’s school, vocational program, or employer. A-12 Appendix B: Potential Sources of Duplication Bias Four possible levels of duplication in assessing service costs were identified: 1. Duplication is unlikely. The rule of including in the analysis only services with a statistically significant difference between the proportion of SVORI and comparison recipients helps eliminate some potential areas of duplication. Consider, for example, if after eliminating services with no significant difference in receipt, the only question remaining in the employment domain is a question on receipt of employment readiness services. There is, therefore, unlikely to be any duplication between that service and any other. If the analysis had included all employment and education services, the potential for duplication would have been higher. 2. Duplication is within domain but only via a gate question. For example, “Have you received any educational services?” is a gate question: a participant is only asked about the type of educational services if he endorses this question. One strategy in the presence of gate questions is to assume that one standard type of educational service was received. This approach solves the problem of potentially double-counting any educational services that are, thereafter, endorsed. It also means only one price is required, although it has the limitation that different types of education services could have different prices. 3. Duplication is within domain and does not involve a gate question. For example, services described as “drug education” and “group counseling” are both found within B-1 An Economic Evaluation of the Serious and Violent Offender Reentry Initiative the substance use services domain. They may be delivered in the same class or in separate classes. 4. Duplication is across domains, where ostensibly different services in different domains are actually provided in the same event. For example, an offender could be receiving residential treatment for substance use, which is part of the substance use services domain. Group counseling might be part of the residential treatment, which is in the mental health services domain. The offender would answer that he received both services. If these were both provided in the residential treatment stay, costing both services would overestimate the incremental cost. B-2