Utah Audit of Drug Offender Reform Act 2009
Download original document:
This text is machine-read, and may contain errors. Check the original document to verify accuracy.
Office of the LEGISLATIVE AUDITOR GENERAL State of Utah REPORT NUMBER 2009-03 January 2009 A Performance Audit of the Drug Offender Reform Act (DORA) DORA is intended to provide selected felony offenders with drug treatment and increased community supervision in order to reduce the costs associated with future criminal behavior. DORA began as a three-year pilot program in 2005 but expanded statewide in 2007, prior to the conclusion of the pilot study, at an annual cost of about $8 million. Ideally, program effectiveness should be judged over many years as data becomes available to know whether DORA participants are less likely to commit future crimes. However, early evidence does not demonstrate reduced criminal behavior. Therefore, it remains unknown whether the expected savings will be realized. Legislative leadership asked us to conduct a limited review of DORA focusing on the cost savings generated by the program. In a September 2008 meeting, legislators were told that “DORA has saved the state over $23 million in the last year.” For this report, we tried to verify these cost savings. In addition, we reviewed the history of DORA, studied the results of the DORA Pilot Program Evaluation, and reviewed some of the implementing practices of participating state and local agencies. It was not within the scope of this audit to focus on other goals of DORA—such as long-term offender or societal outcomes. Office of the Utah Legislative Auditor General DORA is a process for rehabilitating offenders with substance abuse addiction and provides resources for treatment and supervision. 1 This report addresses three main points: It is too early to draw conclusions from the DORA pilot program, but participants do not demonstrate reduced criminal behavior. • Although established as a pilot program to evaluate its effectiveness, DORA was expanded statewide before pilot program results were available. • While it is too early to draw firm conclusions, the report on the DORA pilot program does not demonstrate reduced criminal behavior from DORA participants, and, as a result, cost savings to the state have not been realized. • State agencies implementing DORA should review some practices. The Division of Substance Abuse and Mental Health should review apparent inconsistencies in cost and type of treatment provided to ensure funds are used efficiently. The Department of Corrections should review DORA cost accounting practices to ensure funds are spent on DORA offenders. DORA Program Expanded Statewide Before Pilot Program Had Expired DORA was expanded statewide before the three year pilot program was completed. DORA began as a three-year pilot program in 2005 but was expanded statewide after just two years in 2007 before pilot program results were available. DORA increases up-front costs because it provides services that offenders might not otherwise receive. Future savings are envisioned if fewer crimes are committed by DORA participants, thus reducing the costs of incarceration. DORA Pilot Was Established To Test Program Effectiveness DORA intended to be a collaborative effort among state and local agencies. Legislation establishing the Drug Offender Reform Act (DORA) Pilot Program passed during the First Special Session of the 2005 Legislature. Senate Bill 1004 created a three-year pilot program in the Third Judicial District and appropriated $1.4 million to implement the program on a limited basis and study its effectiveness. The purpose of the pilot program was to examine the impact of providing enhanced services to selected felony offenders. DORA is 2 A Performance Audit of the Drug Offender Reform Act (January 2009) intended to be a collaborative effort among state and local agencies to identify and treat convicted felons who will benefit from substance abuse treatment and who do not require incarceration. Eligible offenders are identified based on both a drug assessment and a risk assessment. Both assessments are administered prior to sentencing and are used to advise the court about the offender’s suitability for DORA. • • The Addiction Severity Index (ASI) measures a person’s drug dependency. The ASI is administered by substance abuse staff to evaluate whether the offender will benefit from treatment. The Level of Service Inventory (LSI) measures an offender’s level of risk. The LSI is administered by corrections staff to determine whether an offender can be released into the community or should be incarcerated. Through DORA, judges are provided with specific information, prior to sentencing, about offenders’ substance abuse patterns and recommended treatment options. Initially, only felony offenders convicted of violating Utah’s Controlled Substances Act were eligible for DORA. With the passage of Senate Bill 185 during the 2006 General Session, the DORA pilot program criteria were expanded to accept all felony offenders who had an assessed drug problem. DORA Program Was Expanded Statewide Before Pilot Results Were Available Before the three-year pilot program had expired, the 2007 Legislature passed S.B. 50 – Drug Offenders Reform Act and appropriated $8 million for fiscal year 2008 for a statewide program. The statewide DORA program requires the courts to order every offender convicted of a felony on or after July 1, 2007 to participate in a screening and assessment prior to sentencing. Based on the ASI and LSI assessments, the court may order the offender into the DORA program. Although DORA was expanded statewide beginning in July 2007, pilot program results were not yet available. As part of the original DORA pilot, the Commission on Criminal and Juvenile Justice (CCJJ) contracted with the Utah Criminal Justice Center at the University of Utah to evaluate the DORA pilot program. A report called Evaluation of the Drug Offender Reform Act: DORA Pilot Office of the Utah Legislative Auditor General Senate Bill 50 appropriated $8 million in FY08 for DORA and required every felony offender to participate in screening and assessments for DORA. CCJJ contracted with the University of Utah’s Criminal Justice Center to evaluate the pilot program. 3 (hereafter, DORA Pilot Evaluation Report) was released in November 2008, and some of its results are discussed later. The statewide DORA program allows firsttime parolees to participate in the program. The statewide DORA also allows offenders who are being granted parole for the first time to participate in the program. Depending on an offenders ASI and LSI scores, the Board of Pardons and Parole may order the offender to participate in DORA as a condition of parole. The Department of Corrections (DOC) reports that the number of parole offenders admitted to DORA in fiscal year 2008 is 265, or 34 percent of the total number of DORA admissions. Five state agencies have received DORA funds. As shown in Figure 1, in fiscal years 2008 and 2009, the Division of Substance Abuse and Mental Health (DSAMH) and the DOC received the largest portion of the funding for treatment and supervision. Figure 1. DORA Appropriations for Fiscal Years 2008 and 2009. Most of the DORA appropriation goes toward treatment and supervision. Agency Division of Substance Abuse and Mental Health Department of Corrections Administrative Office of the Courts Board of Pardons and Parole Commission on Criminal and Juvenile Justice Total Funding FY 2008 $ 4,850,000 Funding FY 2009* $ 4,683,300 3,039,600 3,419,500 50,400 0 36,000 40,500 24,000 19,500 $ 8,000,000 $ 8,162,800 Purpose ASI assessment, treatment, case management LSI assessment, supervision, case management case management case analysis and case management administration, research, and evaluation of DORA * As reduced by special session in September 2008 DORA was funded $9 million for FY09, but funding was reduced to $8.2 million in the 2008 Second Special Session. 4 For fiscal year 2009, the Legislature originally appropriated $9 million; however, the funding was decreased to $8.2 million during the 2008 Second Special Session. The special session eliminated funding to the Administrative Office of the Courts and reduced funding to the DSAMH and CCJJ. In addition, some unspent funds from the fiscal year 2008 appropriation were eliminated. A Performance Audit of the Drug Offender Reform Act (January 2009) Early Evidence Does Not Demonstrate DORA Cost Savings Not enough DORA recipients from the pilot program have completed treatment for sufficient time to draw firm conclusions about DORA effectiveness. However, early indicators do not show that DORA recipients are less likely than non-DORA offenders to engage in criminal behavior. Thus, proof is not available that DORA services will prevent future criminal behavior and its associated costs, such as prison costs. In addition, we are concerned about the accuracy of some data used to evaluate DORA. To enable a valid analysis in the future, data integrity concerns should be addressed. DORA Pilot Program participants have not been out of treatment long enough to measure effectiveness. More Time Is Needed for a Better Pilot Program Evaluation The DORA Pilot Evaluation Report released in November 2008 stated that it is too soon to measure the program’s impact. Because the objective of DORA is to reduce future criminal behavior of recently convicted felons, enough time needs to pass before a meaningful evaluation can occur. The DORA pilot program provided (or is still providing) services to offenders who had recently been convicted of felonies. Since the objective of DORA is to reduce future criminal behavior, more time is needed to evaluate program effectiveness. We reviewed the DORA Pilot Evaluation Report and met with staff at the Utah Criminal Justice Center to better understand the available information. However, we did not have access to the study’s data and did not complete an independent analysis of the data. The study’s results are presented in two separate groups depending on whether offenders began the program before or after March 26, 2006; at that time, the program was expanded to include felons who were not necessarily convicted of drug offenses. Thus, • Time 1 includes offenders who began the program between July 1, 2005 and March 25, 2006. Time 1 consists of 85 DORA participants, all of whom were convicted of a drug felony offense in Salt Lake County. Office of the Utah Legislative Auditor General 5 • The control groups used in the DORA Pilot Program Evaluation are not ideal for comparing with the DORA participants. The time that has passed since offenders completed probation is a key factor in evaluating program effectiveness. 6 Time 2 includes offenders who began the program between March 26, 2006 and November 20, 2006. Time 2 consists of 134 DORA participants who were convicted of a drug-related or other felony offense in Salt Lake County. The study includes non-DORA comparison groups from Salt Lake and Davis counties. The study did not use an experimental design that randomly assigned offenders to DORA and non-DORA groups. However, UCJC staff report they attempted to make the comparison groups as similar as possible to the DORA participants. In general, the comparison groups seem fairly similar to the DORA group. The Salt Lake comparison group has a somewhat more severe criminal history and the Davis group has a somewhat less severe criminal history than DORA participants. However, many in the non-DORA groups did not have ASI assessments, thus adding to concerns about the comparability of the groups. We are uncertain why offenders in the Salt Lake comparison group were not provided DORA services. Of course, the Davis comparison group was geographically ineligible for the DORA pilot program. The time since program completion is a key factor in evaluating program effectiveness. Figure 2 provides data from the pilot study that shows some offenders in both Time 1 and Time 2 have not yet exited probation. Of those who have completed probation, the average time is about 16 months off probation for Time 1 and 9 months for Time 2. The range of time those averages are based on is extreme. A Performance Audit of the Drug Offender Reform Act (January 2009) Figure 2. Selected Pilot Study Information for DORA Participants and Comparison Groups. Many offenders remain on probation or have not exited probation long enough to allow a good assessment of DORA effectiveness. DORA Salt Lake Davis Sample Size 85 103 134 Number Exited Probation Average Days Off Probation 63 485 83 497 107 470 Range of Days Off Probation Time 2 58-796 19-936 2-1009 134 88 108 63 155 98 285 2-761 333 36-637 304 10-820 Time 1 (only drug felony convictions) (drug or other felony convictions) Sample Size Number Exited Probation Average Days Off Probation Range of Days Off Probation Source: DORA Pilot Evaluation Report (Completed November 2008) For example, Figure 2 shows that 63 Time 1 DORA offenders have exited probation for an average length of 485 days. However, the span of days off probation ranges from 58 to 796. Cost Savings Remain Uncertain The DORA model has been designed to identify offenders with drug problems early on, and treat these offenders, therefore reducing future criminal behavior. By reducing future criminal activity, savings can be achieved within the criminal justice system through reduced costs of incarceration. Also, there should be fewer crime victims, and more reformed criminals will be productively employed. The DORA model has been designed to identify offenders with drug problems early on, and treat these offenders, therefore reducing future criminal behavior. One important goal of DORA is to reduce Utah’s future need for prison and jail beds, thereby saving taxpayer funds. At the time DORA was proposed state agencies estimated that substance abuse treatment per client costs about $3,200 - $4,200 annually, while the cost of a prison bed was $25,700 annually. Similarly, when a statewide DORA bill was being considered by the Legislature, legislators were told that DORA could save the state $838 million over a 10-year period. This cost savings was based on 70 to 85 percent of prison inmates qualifying for DORA. Office of the Utah Legislative Auditor General 7 DORA Increases Current Costs. As shown in Figure 1, DORA costs about $8 million per year. The DORA model is to spend somewhat more now by providing more treatment and supervision to avoid having to spend much more later in prison costs. While future cost savings remain uncertain, current costs increase. DORA does not reduce prison costs because DORA participants are very unlikely to be sentenced to prison. It is important to understand that DORA is not considered as a prison diversion program, so it does not reduce current prison costs. An offender who was convicted of a serious offense, or posed a serious risk to public safety would not be eligible to participate in DORA. CCJJ staff told us, and the results of pilot program evaluation report confirm, that DORA recipients would be very unlikely to otherwise be sentenced to prison. Early results from the Pilot Evaluation do not indicate a reduction in criminal activity by DORA participants. Future DORA Savings Are Uncertain. Expected DORA savings are based on the premise that DORA recipients are less likely to commit future crimes, thus saving future prison costs. We reviewed the DORA Pilot Evaluation Report for evidence of reduced criminal activity. While it remains early, we did not see evidence of crime reduction that could lead to future cost savings. Figure 3 shows data from the pilot study on criminal activity of Time 1 offenders who have exited probation. The data does not indicate any reduction in criminal behavior. In fact, a higher percentage of DORA recipients appear to have engaged in subsequent criminal activity than the comparison groups. We show the information for Time 1 offenders because they began the program sooner and have longer time for follow-up. Figure 3. Selected Post-Supervision Data for Time 1 Offenders. Early indicators do not show that DORA is making a positive impact on offenders when compared to the control groups. Number Exited Probation Number with New Arrests Number with New Convictions Number with New Prison Commitments DORA Salt Lake Davis 63 22 (35%) 5 (8%) 3 (5%) 83 22 (27%) 4 (5%) 4 (5%) 107 25 (23%) 5 (5%) 1 (1%) Source: DORA Pilot Evaluation Report 8 A Performance Audit of the Drug Offender Reform Act (January 2009) While Figure 3 does not show that DORA recipients are less likely to commit crimes, it is too soon to know. More time is needed to draw firm conclusions due to the small number of DORA participants who have exited probation and accrued a reasonable follow-up period. Better data about DORA’s impact on future criminal behavior will be available when all offenders have two or three years of time off probation. Some Time 1 offenders remain on probation, and those who have exited probation average just 16 months of post-supervision time. Time 2 offenders have even less post-probation time. The impact of DORA on future criminal behavior will be better measured after offenders have been off probation for 2-3 years. In the future, the criminal justice system may see reduced costs that can be attributed to DORA. However, since it is a new program more time is needed to measure its ability to reduce future criminal behavior and thus reduce costs of the criminal justice system. Data Integrity Concerns Should Be Addressed In addition to needing more time, the ongoing analysis of DORA requires that appropriate outcome measures be based on reliable data. The DORA Oversight Committee has a Research and Evaluation Subcommittee to evaluate the impact and results of DORA. This subcommittee is focusing on two types of impacts—impacts on the criminal justice system and the impacts on the offenders who receive treatment and supervision. One challenge the subcommittee is addressing is the reliability and validity of data about DORA participants. DSAMH collects admissions and treatment data from the 13 local substance abuse authorities for their database. DOC also collects data including DORA admissions and supervision. However, there are no special data sets, or central collection for DORA data. The DORA data are taken from databases from these agencies that are used for other agency activities. A few examples of data concerns are discussed below. Reliability and validity of DORA data is a concern. There is no central collection process for DORA data. Tracking of DORA Offenders by Treatment Providers May Not Be Adequate. DSAMH uses national outcome measures for participators in substance abuse treatment programs and can apply the same outcome measures to DORA participants in the statewide program. The outcome measures used by DSAMH include an increase in substance abuse abstinence, decrease in homelessness, increase in Office of the Utah Legislative Auditor General 9 employment, and decrease in arrests. DSAMH does not have a unique identifier in their database to identify DORA Pilot Program participants; however, DSAMH can identify participants in the statewide DORA implementation. Data discrepancies have been found between DSAMH and Corrections. Inconsistencies still exist in the admissions data for fiscal year 2008. Data on DORA Participants Is Not Consistent. Discrepancies exist when DORA data from the DSAMH is compared to the DOC’s data. Initially for fiscal year 2008, the DSAMH reported 845 admissions, and the DOC reported 775 admissions (488 on probation, 231 on parole, 56 other). The Research and Evaluation Subcommittee as well as the DOC and the DSAMH have been reviewing the 2008 DORA data and making corrections to provide more accurate data on DORA participants. Updated admissions data for fiscal year 2008 show that DSAMH reports 815 admissions, and DOC reports 781 admissions (516 on probation and 265 on parole). Inconsistencies still exist in the admissions data. Inconsistency in the admissions data may be due to data-matching problems, such as offenders listed by different names in one database, incomplete or incorrect offender information, duplicated data, or other reasons. Some Data in Pilot Evaluation Report Raises Questions. The DORA pilot study relied on data from Salt Lake and Davis County substance abuse authorities and the DOC to complete the evaluation. ASI assessment data was available for the DORA participants and the Salt Lake County comparison group, but was not available for the Davis County comparison group. In addition, Figure 4 shows an example of questionable DORA data. Figure 4. Selected Assessment and Treatment Data from DORA Pilot Evaluation Report. Some data for DORA offenders raises questions about data validity. Description Percent with Assessments Time 1 93 Time 2 96 Percent with Treatment Admissions 87 90 Percent who Received Substance Abuse Treatment During Probation 85 92 Source: DORA Pilot Evaluation Report Figure 4 indicates a small inconsistency in DORA offender treatment admissions. For Time 1, local substance abuse authority records show 10 A Performance Audit of the Drug Offender Reform Act (January 2009) slightly more treatment admissions (line 2) than are shown in DOC records (line 3); for Time 2, the opposite is the case. Reliable and valid data needs to be available to accurately evaluate and measure the impact of DORA and to determine if DORA is operating as intended. The DORA Research and Evaluation subcommittee should continue working with treatment and supervising agencies to improve data quality. Reliable and valid data needs to be available to accurately evaluate and measure the impact of DORA. DORA Implementing Agencies Should Review Some Processes Our audit work also reviewed some of the implementing practices, although not in great depth. After interviewing CCJJ staff and reviewing oversight meeting minutes, we concentrated on the treatment and supervising agencies that spend most of the DORA funds. As a new program, DORA’s practices and policies are still being developed. This section includes some issues we think agencies should consider in the future. The DORA model includes increases in both treatment and supervision of offenders. The biggest change brought by DORA seems to be that selected felony offenders are much more likely to get drug treatment. Figure 5 has data from the DORA Pilot Evaluation Report showing that the DORA offenders were much more likely than the comparison groups to have treatment admissions. However, the difference between the DORA and comparison groups in frequency of contact with Adult Probation and Parole (AP&P) agents is more modest. It is notable that the pilot study data, shown in the table below, indicates that much more contact occurred between agents and treatment providers for the DORA groups. However, this greater contact is at least partially due to many comparison group offenders not having treatment providers. Office of the Utah Legislative Auditor General The Pilot Evaluation showed that DORA offenders were more likely to have treatment admissions and to have contact between their AP&P agents and treatment providers. 11 Figure 5. Selected Treatment and Supervision Information for DORA Participants and Comparison Groups. DORA offenders appear to receive much more drug treatment and somewhat more intensive supervision than comparison groups. DORA Salt Lake Davis Time 1 (Only Drug Felony Convictions) Numbers with Treatment Admissions Numbers with Contact Between AP&P Agent and Treatment Provider Average Days Between P.O. Contacts 74 (87%) 47 (46%) 43 (32%) 82 (97%) 39 (38%) 32 (24%) 17 24 27 120 (90%) 44 (41%) 25 (16%) 113 (84%) 36 (33%) 32 (21%) 23 25 Time 2 (Drug or Other Felony Convictions) Numbers with Treatment Admissions Numbers with Contact Between AP&P Agent and Treatment Provider Average Days Between P.O. Contacts 21 Source: DORA Pilot Evaluation Report As discussed in the prior section, there are questions about the accuracy of some of this data. For example, it is unclear how, for both the DORA group in Time 1 and the Davis group in Time 2, the number of participants with treatment admissions is less than the number of participants with contact between their supervising agent and treatment provider. The remainder of this report addresses the use of DORA funds by the two main implementing agencies, DOC and DSAMH. It was beyond the scope of this report to audit DORA implementation in detail, but this section discusses some issues of concern. DSAMH Should Review Apparent Inconsistencies Among Local Authorities Local authority financial records provided by DSAMH suggest that DORA operates inconsistently throughout the state. For fiscal year 2008, the Legislature appropriated $4,850,000 to the DSAMH for DORA treatment services. About $100,000 was retained by the division for administrative expenses, and the remainder was allocated to local authorities based on adult population and probation or parole admissions. Only 58 percent of DORA funds was disbursed to local authorities in fiscal year 2008. 12 For this first year of statewide operations of the DORA program, fiscal year 2008, only 58 percent of the available DORA funds was actually disbursed to the local authorities (leaving about $2 million A Performance Audit of the Drug Offender Reform Act (January 2009) unspent). An initial lack of staff and an initial lack of referrals were both cited by local authority officials as factors that prevented DORA from being fully operational in some regions during the beginning months of statewide DORA. This start-up period contributed to some of the cost inconsistencies observed among the local authorities. Cost per Case Varies Among Local Authorities. Figure 6 shows the DSAMH calculated average cost per DORA case for each local authority. Statewide, the DSAMH reports a $3,300 average, as shown in the following table. The average cost per case varies significantly among the local authorities. Figure 6. The Average FY 08 Cost per DORA Case Was Inconsistent Among Local Authorities. Even among urban and rural regions that would be expected to have comparable populations, the average DORA cost per case is often very dissimilar. In fiscal year 2008, the average cost per DORA case throughout the state was $3,300. According to the DSAMH’s data, local authorities in both urban and rural areas have very divergent average costs. Although division staff told us the cost differences represent real differences among local authorities, the DSAMH has not yet identified the exact reasons for these differences in average cost. The following section offers some possible explanations for the inconsistencies. DSAMH should analyze the reasons for cost differences. Local Authorities May Vary in Their Treatment Approaches. One reason for cost differences among local authorities could be that they provide different types of treatment. It appears that the proportion of admissions to the different service types varies among Office of the Utah Legislative Auditor General 13 the local authorities. Figure 7 shows the admissions types for DORA clients at the four largest local authorities. Figure 7. Fiscal Year 2008 Distribution of DORA Admissions to Different Treatment Options by the Four Largest Local Authorities. Local authorities vary in the type of treatment admissions for DORA offenders. The ratio of admissions to each service type varies among the local authorities. Note: Offenders may have multiple admissions. Some differences are expected. However, the figure indicates that Weber is much more likely to have residential admissions than other local authorities while Utah County is much more likely to have intensive outpatient admissions. We do not know whether the offender population’s therapeutic needs in each jurisdiction are really as diversified as suggested by the data in the graph above. One director explained there may be some inconsistency in applying the process for admitting DORA clients. He explained that the American Society of Addiction Medicine’s (ASAM) Patient Placement Criteria is used as a tool for determining the level of care. He stated that each local authority is responsible for providing its own ASAM training and that the training is not standardized throughout the state. Multiple factors appear to affect a DORA client’s care level placement. 14 Other directors indicated other possible causes for differences in service types. One suggested that clients may not always truthfully report their drug abuse patterns; such inconsistency could affect the client’s placement. Another director explained that, initially, not all A Performance Audit of the Drug Offender Reform Act (January 2009) judges complied with the ASAM scoring model and ordered clients to residential care who may, more appropriately, have been ordered to jail. Different treatment philosophies may also play a part. For example, Utah County has identified that it has a large prescriptionopiate-offender population which can be assisted through rehabilitation by being given an opiate-effects-blocking prescription drug called Vivitrol. By being administered Vivitrol, an offender’s risk of relapse is theoretically reduced, and there is less of a need for many clients to enter residential treatment. Not all local authorities use DORA funds to pay for prescription drugs like Vivitrol to treat their dependent DORA clients. Some DORA Funding May Be Used To Supervise Non-DORA Offenders The Department of Corrections (DOC) receives nearly 40 percent of the DORA appropriation for their part in supervising offenders. The accounting of those funds raises questions because supervision costs for DORA and non-DORA offenders are not clearly separated. In addition, the DORA Pilot Evaluation Report does not indicate a great increase in the frequency of contact with DORA offenders; however, agents report their supervision practices for DORA offenders are much different. Corrections received nearly 40 percent of DORA appropriations, 70 percent of which went to personnel. Accounting for DORA Funds Raises Questions. Most DORA funds spent by the DOC are used to pay for personnel. In fiscal year 2008, about 70 percent of the DOC’s DORA expenditures were used for personnel and 15 percent for vehicles. As of June 2008, DOC reports the DORA organizational units included 31 AP&P agents, two supervisors, and two urinalysis technicians. Additionally, four personnel were hired to conduct assessments, help train other agencies on DORA processes, and work with the transitioning of offenders to regional offices. Although these personnel are funded exclusively by DORA funds, they do not work exclusively with DORA offenders. According to case lists provided to us by the DOC, as of June 2008, DORA agents supervised 722 DORA offenders and 746 non-DORA offenders. On the other hand, DOC also reported that 123 DORA offenders were supervised by non-DORA agents. Based on this data, it seems that Office of the Utah Legislative Auditor General DORA agents are funded exclusively with DORA funds, but supervise non DORA offenders as well. 15 some DORA funds may subsidize the supervision of non-DORA offenders. However, DOC also reports that non-DORA funds are being used to cover some DORA costs. There are practical reasons for the mixed DORA and non-DORA caseloads described above. For example, many of the DORA agents— including all agents from the Salt Lake Region where the pilot program took place—were previously non-DORA agents with nonDORA caseloads who transferred into a DORA organizational unit. Rather than transferring existing cases, the agents kept them. In addition, some rural areas may not have enough DORA cases for a full-time agent, so a mixed caseload makes sense. After DORA enabled DOC to hire 31 additional agents, statewide caseloads decreased from 75 to 65. It appears that the DORA funds have enabled DOC to provide increased supervision to non-DORA as well as DORA offenders. According to the DOC, average caseloads for non-DORA agents have decreased from about 75 to 65 since 2006, despite an increase of nearly 1,300 offenders. Meanwhile, the DORA agents have a caseload of about 44, although less than half of the cases are DORA offenders. The DOC should be able to more accurately account for DORA funding. By allocating both DORA and non-DORA agents’ salaries based on the percentage of DORA cases an agent has, DORA funds would only be used for DORA cases. In the future, DOC may be able to reduce its number of DORA agents if the number of DORA cases increases in rural regions, and as non-DORA offenders assigned to DORA agents complete probation. With 845 DORA cases and 31 agents, the average caseload without a mixed caseload would be just 27 cases. The pilot study shows only small increases in the amount of contact between agents and DORA participants. DORA Offenders May Not Receive Substantially More Supervision. Figure 5 shows the average number of days between agent and offender contact, as reported in the DORA Pilot Evaluation Report. The number of days between contacts was somewhat lower for both Time 1 and Time 2 DORA offenders, but not by a lot. The study shows that there has been increased contact between agents and DORA offenders, but the differences are not very large despite the decreased caseload. The DOC has standards for frequency of contact between agents and offenders on probation, as shown in Figure 8. 16 A Performance Audit of the Drug Offender Reform Act (January 2009) Figure 8. Department of Corrections Levels of Supervision. The DOC has specific standards for the amount of supervision an offender should receive based on their level of supervision. Level of Supervision Intensive LSI Score Number of Contacts 41-54 2 office contacts and 2 residence contacts per month 1 Office contact and 1 residence contact per month 1 office contact per month and 1 residential contact every 60 days 1 contact every 90 days High 24-40 Moderate 14-23 Low 0-13 For DORA offenders, however, there is no specific standard for the frequency of contacts with agents. According to the Department of Corrections’ Standards of Supervision, DORA caseloads are established to provide closer supervision and a coordinated supervision of drug offenders. DORA focuses on close and effective relationships and collaboration with treatment providers and a mutually supportive role. DOC does not have a specific standard for the frequency of contacts with DORA participants. In our conversations with seven agents from five of the six regions in the state, all agents mentioned that the level of supervision is higher with DORA offenders. Agents report that meeting with family members and employers are important components of supervision in addition to face-to-face meetings with offenders. Agents from the two biggest regions both said that they try to visit offenders twice per month, once in the office and once in the community (home or work visits). However, there is no specific standard that specifies the number of contacts an agent should have with a DORA offender. In addition to contact with offenders, agents say that contact with treatment providers is important. In speaking with agents, we were consistently told that smaller caseloads allow agents to work closely with the treatment provider. We observed this in practice at a meeting where AP&P agents and treatment providers discussed offenders’ progress in the treatment program. DORA agents report working closely with treatment providers. According to an AP&P supervisor as well as several agents, with DORA the response to a failure in treatment is less likely to be Office of the Utah Legislative Auditor General 17 punitive. Instead, the agent can work with the treatment provider and the offender to determine how to best help the offender and prevent future problems. Data in the DORA Pilot Evaluation Report indicated that significantly fewer offenders had their probation revoked after non-compliance events like substance use and other supervision violations. The agents were more likely to work with the offender rather than having them removed from the community. Recommendations 1. We recommend that the DORA Research and Evaluation Subcommittee continue to monitor the Pilot Program participants to determine post-supervision outcomes. 2. We recommend that the DORA Research and Evaluation Subcommittee continue to work with the agencies that provide DORA data to correct data errors. 3. We recommend that the Division of Substance Abuse and Mental Health review and evaluate the differences for DORA offenders’ cost differences among local authorities and provide additional guidance if needed. 4. We recommend that the Department of Corrections more accurately account for the DORA funding. 5. We recommend that the Department of Corrections establish clear guidelines for the number of contacts that DORA offenders should receive from AP&P agents. 18 A Performance Audit of the Drug Offender Reform Act (January 2009) Agency Response Office of the Utah Legislative Auditor General 19 This Page Left Blank Intentionally 20 A Performance Audit of the Drug Offender Reform Act (January 2009) •. ~ ~ i - +•., "- .... ~ , .+ State of Utah COMMISSION ON CRIMINAL AND JUVENILE JUSTICE Jon M. Huntsman, Jr. Governor Robert S. Yeates Executive Director Suite E330, East Capitol Complex • Salt Lake City, Utah 84114 801-538-1031 • Fax: 801-538-1024 • www.justice.utah.gov January 13, 2009 John M. Schaff, CIA Legislative Auditor General W315 Utah State Capitol Complex Salt Lake City, Utah 84114-5315 Re: A Performance Audit of the Drug Offender Reform Act (DORA) Report No. 2009-03 Dear Mr. Schaff: Thank you for the opportunity to provide a response to the Performance Audit of the Drug Offender Reform Act (DORA). The agencies involved in the audit appreciate the work of the Legislative auditors and the courtesy shown as they worked with our staff. We are grateful for the observations and recommendations that identify important areas where we can improve DORA and DORA’s outcomes. DORA is unique in state government for many reasons, not the least of which is the collaboration it has promoted among diverse agencies across the substance abuse treatment and criminal justice systems. In keeping with this collaboration, the three agencies which were given the opportunity to respond to this audit – Corrections, Human Services, and the Commission on Criminal and Juvenile Justice – have chosen to offer a single, coordinated response. We concur with the recommendations of the audit and have already begun the work required to implement them. Before we respond to the specific recommendations, we would like to take this opportunity to make a few general comments about DORA and the audit. The audit identifies aspects of DORA that represent the difficulties inherent in implementing any sweeping policy reform. Nonetheless, DORA’s vision remains on track. From its inception, those involved in the implementation of the program have understood that it would be an incremental process. This is common in any new program such as DORA, particularly one that requires collaboration among a variety of state agencies, branches of government, local government entities, and service providers in the community. DORA Goals The audit report states the intent of DORA as being “to provide selected felony offenders with drug treatment and increased community supervision to reduce the costs associated with future criminal John M. Schaff, CIA Re: A Performance Audit of the Drug Offender Reform Act (DORA): Report No. 2009-03 January 13, 2009 Page 2 behavior”. While this is certainly one of the goals of DORA, it is important to note that the intent of DORA also includes several process and outcome goals in addition to reduced costs. DORA represents a change in the way Utah handles offenders with drug problems by addressing the root cause of many crimes – substance abuse – with the ultimate objectives of reducing both substance abuse and crime and their associated impacts on individuals and society. DORA’s process goals may be summarized as follows: - “Smarter sentencing” is accomplished by providing the judge with specific information about the offender’s substance abuse problem and treatment needs prior to sentencing. DORA does not mandate treatment, but provides judges with this information and at the same time provides funding for treatment services. - “Smarter treatment” is achieved by providing funding to create more treatment slots in both the community and in correctional facilities, and by conducting a comprehensive assessment of the offender’s substance abuse and determining the appropriate level of treatment (e.g., outpatient treatment, intensive outpatient treatment, day treatment, residential treatment) for the offender. - “Smarter supervision” means the treatment provider and the Adult Probation and Parole (AP&P) agent are working for the same goals, and offenders/clients receive the same message from both treatment and AP&P. AP&P and substance abuse treatment staff meet regularly to discuss goals and progress, and more information is available to both parts of the system which improves effectiveness and offender accountability. DORA’s outcome goals may be summarized as follows: - Reduce substance abuse and crime/recidivism - Reduce Utah’s future needs for prison and jail beds - Create more law-abiding and taxpaying citizens - Create safer neighborhoods Successful, long-term implementation of DORA will do more than just save money; it will literally save lives and sustain families and communities. The audit mentions that Legislators were told DORA “could save the State $838 million over a 10-year period.” It does not explain that these figures were based on costs avoided through full funding for the statewide implementation of DORA. Only half of the initial funding request was appropriated, which would reduce the estimate of costs avoided by at least 50%. In addition, the audit does not explain that most of these avoided costs were not in state government or even local government operations, but in reduced victimization costs resulting from thefts and injuries. Less than 30% of the projected avoided costs were in government services, both state and local. John M. Schaff, CIA Re: A Performance Audit of the Drug Offender Reform Act (DORA): Report No. 2009-03 January 13, 2009 Page 3 DORA Pilot Study The audit acknowledges that it is too soon to draw conclusions from the DORA pilot study regarding the impact of DORA on later criminal activity. Large numbers of participants are still on probation and follow-up times for those who have exited probation are short. Generally, criminal justice research requires at least one year of follow-up post-supervision for recidivism comparisons. More than 30% of the DORA pilot participants are still on probation, and the fact that they are continuing on probation after more than two years suggests that they have a good chance of being successful. In criminal justice research, it is important to remember that failures generally happen quickly, but successes require time and effort to achieve. It is possible that longer follow-up of the pilot participants will show greater rates of success for the DORA participants. While the audit mentions the failure of the pilot study to show a lower rate of criminal activity for DORA participants, the audit does not acknowledge the positive results found in the DORA pilot. The study found that essential elements of DORA predicted successful completion of probation, including: completing a treatment admission, agent and offender contact in the community, and shorter time lags between conviction and probation start. Specific Recommendations Recommendation 1: The audit recommends that the DORA pilot participants continue to be followed by the DORA Oversight Committee’s Research and Evaluation Subcommittee to determine their long-term outcomes. We support this recommendation and plan to continue to follow these offenders and provide annual updates on their outcomes. Recommendation 2: The audit notes problems with the accuracy and completeness of the data on DORA participants provided by the Division of Substance Abuse and Mental Health (DSAMH) and the Department of Corrections and recommends that efforts be made to correct these errors. We wholeheartedly support this recommendation and will continue to work on these issues. Data collection is improving. Sharing public and private data among a variety of entities is always challenging. The DORA Research Committee shares the auditors’ concerns related to the reliability and validity of data, and continues to work toward a solution. In time, as the program is fully implemented and stabilized, we are confident the data concerns will be resolved. Completely accurate data have been and will continue to be the goal of the DORA evaluation project. However, we recognize that hundreds of different individuals working in Corrections’ AP&P offices, in the prisons, at the Division of Substance Abuse and Mental Health, and in the Local Substance Abuse Authority and other treatment provider agencies are collecting and entering data and some inaccuracies are unavoidable. In addition, data collected on substance abuse treatment are subject to strict Federal and professional privacy regulations that, while they are important to client confidentiality, make data cleanup very difficult. In spite of these challenges, the Research and Evaluation Subcommittee will continue to make every effort to provide the most complete and accurate data possible on DORA. John M. Schaff, CIA Re: A Performance Audit of the Drug Offender Reform Act (DORA): Report No. 2009-03 January 13, 2009 Page 4 Recommendation 3: The Division of Substance Abuse and Mental Health has already begun the recommended review and evaluation of the differences in costs for DORA offenders among the local authorities. As the Division reviewed and evaluated the cost differences prior to the report’s publication, the Division concluded that there is no one exact reason for the differences in average cost. In fact, there are multiple reasons, including the ones mentioned in the audit report. One reason not mentioned in the audit report is the difference in start-up costs for the new program among different areas of the state. Some local authorities were able to absorb the initial additional client load with their already existing staff and programs, while others had to hire new staff members that weren’t fully utilized until the client population had increased. Recommendation 4: The Department of Corrections is thankful for the opportunity to reiterate its commitment to the vision of DORA and is confident that as the program is fully implemented, there will be limited mixing of DORA and non-DORA funds. o The building of DORA caseloads and treatment needs is incremental. With the exception of rural areas in Utah that will continue to require mixed caseloads, as there are simply fewer DORA clients in those regions, DORA-funded agents will ultimately supervise DORA caseloads only. At this time, however, the Department would be concerned if agents in our urban regions were required to create a separate reporting stream in documenting DORA and non-DORA time. Their responsibilities are already considerably expanded, and a focus is needed, now more than ever, to place a premium on time spent supervising offenders in the community. o In the interim, UDC has provided necessary services for DORA clients from its ongoing, non-DORA supervision budget. For example, initial screening costs, most supervision of DORA-funded agents, support staff, and administrative costs are not supported by DORA funds. Corrections remains confident that most DORA funds are used almost exclusively for DORA-related operations; however, as noted above, non-DORA funds have also been used to sustain this vital program. Recommendation 5: The audit recommends that Corrections establish clear guidelines for the number of contacts that DORA offenders should have with their AP&P agents. While this is an important measure, contact between an agent and the client is but one part of that agent’s involvement with the client. Outcome analysis of DORA will include many different measures of agent supervision, such as the number of contacts and days between contacts with DORA clients. There are many collateral contacts both with the client and the treatment provider that are not being documented. By only collecting the number of direct contacts between agents and offenders, we certainly understate the true amount of work and contact between the DORA client and the agent. Realizations like this are common when developing an outcome evaluation protocol, and we appreciate the auditors in bringing this particular issue to the surface. The auditors recognized the need for “clear guidelines” for the amount of contact with DORA clients. UDC agrees with this recommendation. As the DORA program matures, with the help of John M. Schaff, CIA Re: A Performance Audit of the Drug Offender Reform Act (DORA): Report No. 2009-03 January 13, 2009 Page 5 the Research and Evaluation Subcommittee, we continue to gain clearer insight into what these guidelines should entail. Our intent is to use the analysis results to determine “best practices” in terms of establishing DORA-specific contact guidelines. The DORA partner agencies would like to thank the Audit team for their professionalism and courtesy. We recognize that our shared aim is to provide the most effective state services possible to the people of our state. Sincerely, Lisa-Michele Church, Executive Director Utah Department of Human Services 1~~- 1~~ Thomas E. Patterson, Executive Director Utah Department of Corrections .~ ~ /~ ,,y I '.I Robert S. Yeates, Executive Director Utah Commission on Criminal and Juvenile Justice