Wsipp Evidence Based Substance Abuse Mental Health Treatment Washington State 2006
Download original document:
This text is machine-read, and may contain errors. Check the original document to verify accuracy.
Washington State Institute for Public Policy 110 Fifth Avenue Southeast, Suite 214 • PO Box 40999 • Olympia, WA 98504-0999 • (360) 586-2677 • www.wsipp.wa.gov June 2006 EVIDENCE-BASED TREATMENT OF ALCOHOL, DRUG, AND MENTAL HEALTH DISORDERS: POTENTIAL BENEFITS, COSTS, AND FISCAL IMPACTS FOR WASHINGTON STATE During the mid-1990s, the Washington State legislature began to enact statutes to promote an “evidence-based” approach to several public policies. While the term evidence-based has not always been precisely defined in legislation, it has generally been constructed to describe a program or policy supported by a rigorous outcome evaluation clearly demonstrating effectiveness. Additionally, to determine if taxpayers receive an adequate return on investment, the legislature has also started to require benefit-cost analyses of certain state-funded programs and practices. Washington’s initial experiments with evidence-based and cost-beneficial public policies began in the state’s juvenile justice system. The legislature funded several nationally known and rigorously researched programs designed to reduce the reoffending rates of juveniles. At the same time, the legislature eliminated the funding of a juvenile justice program when a careful evaluation revealed that it was failing to reduce juvenile crime. Thus, the term evidencebased does not mean simply adding new programs, it also means eliminating programs when research indicates they do not work. Following this successful venture into evidencebased public policy, Washington began to introduce the approach in other fields including adult corrections, child welfare, and K–12 education. Extending the Evidence-Based Concept to the Treatment of Alcohol, Drug, and Mental Health Disorders. The 2005 Legislature directed the Washington State Institute for Public Policy (Institute) to examine the potential benefits Washington could obtain if it adopted an evidence-based approach for alcohol, drug, and mental illness treatment. This report describes our “bottom-line” findings as well as our research approach. _____________________________________________ Suggested citation for this report: Steve Aos, Jim Mayfield, Marna Miller, and Wei Yen. (2006). Evidence-based treatment of alcohol, drug, and mental health disorders: Potential benefits, costs, and fiscal impacts for Washington State. Olympia: Washington State Institute for Public Policy. Summary The Washington State Institute for Public Policy was directed by the 2005 Washington Legislature to estimate whether “evidence-based” treatment for people with alcohol, drug, and mental health disorders offers economic advantages. Do benefits outweigh costs? And, if so, what is the magnitude of the potential fiscal savings to government, as well as the total net benefits to all of Washington? Methods To answer these questions, we systematically reviewed the “what works” literature regarding treatments for people with alcohol, drug, and mental health disorders. We then estimated the monetary value of the benefits, including factors such as improved performance in the job market, reduced health care and other costs, and reduced crime-related costs. Findings 1. Evidence-based treatment works. We found that the average evidence-based treatment can achieve roughly a 15 to 22 percent reduction in the incidence or severity of these disorders—at least in the short term. 2. The economics look attractive. We found that evidenced-based treatment of these disorders can achieve about $3.77 in benefits per dollar of treatment cost. This is equivalent to a 56 percent rate of return on investment. From a narrower taxpayer’s-only perspective, the ratio is roughly $2.05 in benefits per dollar of cost. 3. The potential is significant. We estimate that a reasonably aggressive implementation policy could generate $1.5 billion in net benefits for people in Washington ($416 million are net taxpayer benefits). The risk of losing money with an evidence-based treatment policy is small. Background: The Omnibus Treatment of Mental and Substance Abuse Disorders Act of 2005 This research assignment originated in a much larger bill enacted during the 2005 legislative session: the Omnibus Treatment of Mental and Substance Abuse Disorders Act. A major goal of the Act is to reform how publiclyfunded mental health and chemical dependency programs are provided in Washington. In passing the omnibus Act, the 2005 Legislature found that: “Persons with mental disorders, chemical dependency disorders, or co-occurring mental and substance abuse disorders are disproportionately more likely to be confined in a correctional institution, become homeless, become involved with child protective services or involved in a dependency proceeding, or lose those state and federal benefits to which they may be entitled as a result of their disorders.” 1 Further, the Legislature found that: “Prior state policy of addressing mental health and chemical dependency in isolation from each other has not been cost-effective and has often resulted in longer-term, more costly treatment that may be less effective over time.” 2 Among the several actions adopted in the 2005 Act to address these general concerns, the Legislature indicated its intention to: “Improve treatment outcomes by shifting treatment, where possible, to evidence-based, research-based, and consensus-based treatment practices and by removing barriers to the use of those practices.” 3 The Basic Questions for the Study Within the context of the Act’s overall goals, the language directing the Institute’s study is shown in the sidebar on this page. In brief, the Legislature directed the Institute to answer the following “bottom-line” questions: 9 Does evidence-based treatment for people with alcohol, drug, or mental health disorders make economic sense? 9 Do benefits outweigh costs? 9 And, if so, what is the potential magnitude of the fiscal savings to government, and what are the total net benefits to all of Washington? In addition to directing the Institute to answer these questions, the omnibus Act also required the Institute to evaluate the effectiveness of the Act’s pilot programs, which are designed to test several new implementation approaches (see the sidebar on page 6 for a brief description of the pilot program study). Legislative Study Language Engrossed Second Substitute Senate Bill 5763, Chapter 504, Laws of 2005, Sec. 605. “The Washington state institute for public policy shall study the net short-run and longrun fiscal savings to state and local governments of implementing evidencebased treatment of chemical dependency disorders, mental disorders, and co-occurring mental and substance abuse disorders. The institute shall use the results from its 2004 report entitled "Benefits and Costs of Prevention and Early Intervention Programs for Youth" and its work on effective adult corrections programs to project total fiscal impacts under alternative implementation scenarios. In addition to fiscal outcomes, the institute shall estimate the long-run effects that an evidence-based strategy could have on statewide education, crime, child abuse and neglect, substance abuse, and economic outcomes. The institute shall provide an interim report to the appropriate committees of the legislature by January 1, 2006, and a final report by June 30, 2006.” The Institute received an appropriation of $80,000 to conduct the study. 1 2 3 E2SSB 5763, Chapter 504, Laws of 2005, Section 101. Ibid. Ibid., Section 101(3). 2 Research Methods To answer the Legislature’s questions, we followed the same two-step procedures we have applied to other recent projects. First, we independently and systematically assessed the research literature on “what works,” and then we estimated benefits and costs. In the Appendix to this report (beginning on page 7), technical readers can find a detailed description of our methods. Here, we summarize our approach. 1. Assessing the research literature: Does evidence-based treatment of alcohol, drug, and mental illness reduce the incidence or severity of these disorders? We began by reviewing lists of evidence-based treatments that have been compiled by other researchers.4 After we reviewed all of the individual studies associated with these listed treatments, we then only included the results of “rigorous” evaluation studies in our review. To be considered rigorous, an evaluation must have included, at a minimum, a nontreatment comparison group that was well-matched to the treatment group. We used this restriction because greater confidence can be placed in causeand-effect conclusions from rigorous comparisongroup studies. Studies that use weaker research methods do not provide this level of assurance and were excluded. Thus, our judgment of what constitutes “evidence” is more restrictive than the standards used by some other researchers. In recent years, researchers have developed a set of statistical tools to facilitate systematic reviews of the evidence. The set of procedures is called “metaanalysis” and we employed that methodology in this study. Our meta-analytic review includes 206 studies (246 trials) of evidence-based treatments for persons with alcohol, drug, and mental health disorders. Most of the individual evaluation studies we examined were conducted outside of Washington State. A primary purpose of our study is to take advantage of all evaluations and, thereby, learn whether there are options that can allow policymakers in Washington to improve this state’s mental health and chemical dependency treatment system. 2. Assessing the economics: What are the benefits and costs of evidence-based treatment of alcohol, drug, and mental illness? After calculating the likely effect of an average evidence-based treatment in reducing disorders, we then estimated each option’s benefits and costs. To do this, we used the same methods we have employed in our earlier reviews of criminal justice and other prevention programs.5 We estimated the degree to which reductions in alcohol, substance abuse, and mental illness disorders improve longevity and an individual’s economic earnings, reduce health care and other costs, and reduce crime and crimerelated costs. As in our previous analyses, impacts were estimated from two different perspectives: first, we calculated benefits gained by program participants themselves; second, we estimated benefits received by taxpayers and other non-participants. An example of a participant benefit is the increased economic earnings stemming from enhanced labor productivity when a treatment reduces disorder rates. An example of a taxpayer benefit is the reduced level of taxes needed to fund hospital emergency room visits when the evidence-based treatment program reduces problematic disorders. The perspectives of both participants and taxpayers are necessary to provide a full description of fiscal and non-fiscal benefits and costs. We then estimated total potential benefits based on the number of people in Washington who could take advantage of a particular evidence-based treatment. We compiled information from a number of sources to estimate how many people in Washington have a serious alcohol, drug, or mental illness disorder, and how many could realistically be expected to benefit from an evidence-based treatment. Finally, we varied the estimates and assumptions in our analysis to gauge the overall level of uncertainty in the “bottom-line” numbers we present. 5 4 See Appendix A. See: (a) S. Aos, M. Miller, and E. Drake. (2006). Evidence-based adult corrections programs. Olympia: Washington State Institute for Public Policy; (b) S. Aos, R. Lieb, J. Mayfield, M. Miller, and A. Pennucci. (2004). Benefits and costs of prevention and early intervention programs for youth. Olympia: Washington State Institute for Public Policy; and (c) S. Aos, P. Phipps, R. Barnoski, and R. Lieb (2001). The comparative costs and benefits of programs to reduce crime. Olympia: Washington State Institute for Public Policy. 3 Findings How prevalent are alcohol, drug, and serious mental health disorders? To estimate the total benefits and costs of evidencebased treatment, we gathered national estimates of the prevalence of clinically serious alcohol, drug, and mental health disorders. We focused on serious disorders because they appear to be the most costly both to individuals with a disorder and to the rest of society.6 We focused on adults (18 years and older) to make the study compatible with current national prevalence rates and because our previous work emphasized younger people.7 In this study, we used the following prevalence rates: 9 Alcohol or Drug Disorders. About 7.6 percent of the adult (18 to 54 years old) population has a clinically significant alcohol or drug disorder.8 This is equivalent to about 1 in 13 adults. To account for the comorbidity (two conditions at the same time) between alcohol and drug disorders, we also estimated the following: • 61 percent of these people have an alcoholonly disorder • 22 percent have a drug-only disorder • 17 percent have alcohol and drug disorders 9 Serious Mental Illness. About 3.8 percent of the adult population has a serious mental illness.9 This is equivalent to about 1 in 26 adults. These serious mental illnesses were defined to include schizophrenia and other non-affective psychosis, manic depressive disorder, severe forms of major depression, and panic disorder. Does evidence-based treatment of alcohol, drug, and mental illness reduce the incidence or seriousness of these disorders? We found that the average evidence-based treatment reduces the short-term incidence or seriousness of alcohol, drug, or mental health disorders 15 to 22 percent.10 For example, if 75 percent of people with an alcohol disorder continue to have the disorder without treatment, then with an average evidence-based alcohol treatment this percentage can be lowered to 64 percent—a 15 percent improvement in disorder rates. Our analysis revealed that in the short-term, the average evidence-based treatment produces the following statistically significant decreases in the probability of these disorders: 9 Alcohol Disorders: a 15 percent reduction 9 Drug Disorders: a 22 percent reduction 9 Serious Mental Illness: an 22 percent reduction It should be emphasized that these estimates are based on studies with fairly short-term follow-up periods—often a year or less. We found few studies that evaluated effectiveness over the longer term. To account for this lack of longitudinal research, in our benefit-cost analyses we significantly reduce (technically, we “decay”) these short-term effectiveness rates, since many people speculate that the beneficial effects of treatment erode as time passes.11 What are the benefits and costs of evidencebased treatment of alcohol, drug, and mental illness? 6 See: (a) H. Harwood. (2000). Updating estimates of the economic costs of alcohol abuse in the United States: Estimates, update methods, and data. Report prepared by The Lewin Group for the National Institute on Alcohol Abuse and Alcoholism. Based on estimates, analyses, and data reported in H. Harwood, D. Fountain, and G. Livermore. (1998). The economic costs of alcohol and drug abuse in the United States, 1992. Prepared for the National Institute on Drug Abuse and the National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Dept. of Health and Human Services. NIH Publication No. 98-4327. Rockville, MD: National Institutes of Health. http://pubs.niaaa.nih.gov/ publications/economic-2000/index.htm; (b) Office of National Drug Control Policy. (2004). The economic costs of drug abuse in the United States, 1992-2002. Washington, DC: Executive Office of the President (Publication No. 207303). http://www.whitehouse drugpolicy.gov/publications/ economic_costs/economic_costs.pdf; and (c) H. Harwood, A. Ameen, G. Denmead, E. Englert, D. Fountain, and G. Livermore. (2000). The economic costs of mental illness, 1992. Prepared for the National Institute of Mental Health. http://www.lewin.com/NR/rdonlyres/ea3i6g7cjgsvls2ukpupxo7wbjlmu25vh3 nd5rldz3lwyxfab6y6e4smh2zfpcs33wmmuq2cgbp3vg/2487.pdf 7 Aos et al., Benefits and costs of prevention and early intervention programs for youth. 8 W.E. Narrow, D.S. Rae, L.N. Robins, and D.A. Regier. (2002). Revised prevalence estimates of mental disorders in the United States: Using a clinical significance criterion to reconcile 2 surveys' estimates. Archives of General Psychiatry, 59: 115-123. 9 Harwood et al., The economic costs of mental illness, Table 4.7. 4 We found that the economics of the average evidence-based treatment for people with serious alcohol, drug, or mental disorders are quite attractive. Per dollar of treatment cost, we estimate that evidence-based treatment generates about $3.77 in benefits for people in Washington. Expressed as a return on investment, this is equivalent to roughly a 56 percent rate of return. When we restrict this analysis to only those benefits that accrue to taxpayers, the benefit-tocost ratio is $2.05. 10 11 See Appendix A for details behind these estimates. Ibid. Of the total benefits to Washington, approximately: • 35 percent stem from the effect that the reduced incidence of a disorder has on the person’s economic earnings in the job market; • 50 percent are linked to fewer health care and other costs incurred; • 7 percent are due to the lowered costs of crime; and • 8 percent are for miscellaneous benefits. We also estimated the total potential impact that an evidence-based strategy could have for Washington State. This involved first estimating the number of people in Washington who have a serious disorder (described above). We then subtracted an estimate of the number of people in Washington already being treated with an evidence-based program.12 We further restricted the size of the potential treatment population by assuming that only half of those who need treatment (and are not currently being treated) would ultimately be served. Under these assumptions, we found that the total net benefits to Washington would be about $1.5 billion. From the narrower taxpayer-only perspective, the net benefits would be about $416 million. How much uncertainty exists in these estimates of benefits and costs? In any estimation of the outcomes of complex human behavior and human service delivery systems, there is uncertainty. In our analysis, we estimated the degree to which our bottom-line estimates could be influenced by this uncertainty. As described in the Technical Appendix, we performed an analysis called “Monte Carlo simulation.” We randomly varied the key factors that enter our calculations and then re-estimated the results of our analysis. We did this re-estimation process 10,000 times, each time testing the range of uncertainty in our findings. We sought to determine the probability that our estimates would produce a contrary finding. That is, we tested to see how often our positive results would turn negative— that money would be lost not gained. From the perspective of all of Washington, we found that the chance that an expansion of evidence-based treatments would actually lose money (rather than generate benefits) was less than 1 percent. From the narrower taxpayer-only perspective, we found that the chance that an evidence-based strategy would lose money is approximately 1 percent. That is, about one time out of a hundred an evidence-based strategy would end up costing taxpayers more money than it saved. Next Research Steps To complete this research project on time and on budget (the Institute received $80,000 for the study), we had to adopt several strategies to narrow the study’s scope. If the legislature decides to initiate a follow-up study, the following limitations could be addressed: 1. Expand the scope of the study to include people younger than 18. In this study, we reviewed published research evaluations of alcohol, drug, and mental health treatments. These research fields are vast. In order to make the current study manageable, we restricted our review to treatments for adults 18 years and older. We also made this restriction because most of the existing research on the prevalence and costs of alcohol, drug, and mental health disorders has been for adult populations. Additionally, we researched substance abuse programs for youth in a study we completed in 2004 on prevention programs. A subsequent study could expand the scope of the current research to identify the economics of evidence-based treatment for people 17 years and younger. 2. Expand the scope of the study to include evidence-based treatment for less serious alcohol, drug, and mental health disorders. We restricted our search for evidence-based treatments to those that focus on people with quite severe, clinically significant, levels of disorder. We did this because existing cost studies indicate that the severe forms of disorder are usually the most costly to society. A subsequent study could expand the scope to identify evidence-based treatments for less severe forms of these disorders. Because of diminishing returns, however, the returns on investment will probably not be as large as those found in this study, but this hypothesis could be tested in the subsequent study. 3. Identify specific types of evidence-based treatment. The purpose of the present study was to explore the total “market” potential of evidencebased treatment; a subsequent study could help identify specific strategies. We analyzed the economics of “prototype” evidence-based treatments for alcohol, drug, or mental health disorders. That is, we calculated the return on investment for an average evidence-based treatment. A subsequent study could focus on specific “name-brand” types of treatment for alcohol, drug, or mental health disorders and determine the economic returns associated with each. This additional detailed information could offer executive and legislative public policymakers with “line-item” information on specific evidence-based treatments. 12 For the purpose of this study, we assume that the vast majority of those currently being treated are receiving evidence-based treatment. 5 4. Conduct further research regarding the link between alcohol, drug, and mental health disorders and child abuse and neglect. This study contains only rough estimates of how alcohol, drug, and mental health disorders causally influence rates of child abuse and neglect. For example, we included estimates of how substance abuse disorders affect fetal alcohol syndrome, and we estimated how all the disorders affect the ability of a person to perform normal household activities. For the effect of these disorders on other child welfare outcomes, however, our current estimates are probably incomplete and likely underestimate the actual impact. To overcome this limitation, a subsequent study could test this linkage further and develop additional information that could be useful for public policymakers. Additional Institute Study From the Omnibus Treatment of Mental and Substance Abuse Disorders Act of 2005 Crisis Responder Pilot Evaluation The same Act that directed the study described in this report also instructed the Department of Social and Health Services to establish two pilot sites where specially trained crisis responders will investigate and have the authority to detain individuals considered “gravely disabled or presenting a likelihood of serious harm” due to mental illness, substance abuse, or both. The integration of mental health and substance abuserelated crisis investigations and the establishment of secure detoxification facilities at the pilot sites are expected to improve the efficiency of evaluation and treatment and result in better outcomes for those involuntarily detained under this new law. The pilots began operations in May 2006. The Legislature directed the Washington State Institute for Public Policy to determine if the pilots cost-effectively improve client mental health/chemical dependency evaluation, treatment, and outcomes. A preliminary report by the Institute is due to the Legislature in December 2007. The final report is to be completed by September 2008. For more information on this related project, contact Jim Mayfield at the Institute: firstname.lastname@example.org; 360-586-2783. 6 Technical Appendices Appendix A: Meta-Analytic Procedures A1: Study Selection and Coding Criteria A2: Procedures for Calculating Effect Sizes A3: Institute Adjustments to Effect Sizes for Methodological Quality, Outcome Measure Relevance, and Researcher Involvement A4: Meta-Analytic Results—Estimated Effect Sizes and Citations to Studies Used in the Analyses Appendix B: Methods and Parameters to Model the Benefits and Costs of Evidence-Based Treatment B1: B2: B3: B4: B5: B6: B7: B8: B9: B10: B11: B12: General Model Parameters Program Effectiveness Parameters Program Design Parameters Prevalence Parameters Total Potential Population to Be Treated Morbidity Parameters and Methods Lost Household Production Methods Health Care and Other Costs Mortality Parameters and Methods Crime Parameters Marginal Treatment Effect Sensitivity Analysis Exhibits: A.1: A.2: A.3: A.4: A.5: B.1: B.2: B.3: B.4: Listed Programs, Practices, and Treatments With Studies Meeting Minimum Quality Standards Meta-Analytic Results of the Effects of EBT on Disordered Alcohol Use Meta-Analytic Results of the Effects of EBT on Disordered Drug Use Meta-Analytic Results of the Effects of EBT on Mental Illness Citations of Studies Used in the Meta-Analysis The Benefits and Costs of Evidence-Based Treatment: General Model Parameters The Benefits and Costs of Evidence-Based Treatment: Annual Data Series The Benefits and Costs of Evidence-Based Treatment: Program-Specific Model Parameters Meta-Analytic Estimates of Standardized Mean Difference Effect Sizes B.4a: Citations to Studies in Exhibit B.4 B.5: The Benefits and Costs of Evidence-Based Treatment: Model Parameters Varied in the Monte Carlo Simulations Appendix A: Meta-Analytic Procedures To estimate the benefits and costs of evidence-based treatment (EBT) of alcohol, drug, and mental illness disorders, we conducted separate analyses of a number of key statistical relationships. In Appendix A, we describe the procedures we employed and the results we obtained in estimating the causal linkage for the following nine relationships: • The effect of EBT on serious alcohol disorders • The effect of EBT on serious illicit drug disorders • The effect of EBT on serious mental illness disorders • The effect of serious alcohol disorders on job market outcomes • The effect of serious illicit drug disorders on job market outcomes • The effect of serious mental illness disorders on job market outcomes • The effect of serious alcohol disorders on crime outcomes • The effect of serious illicit drug disorders on crime outcomes • The effect of serious mental illness disorders on crime outcomes To estimate these nine key relationships, we conducted reviews of the relevant research literature. In recent years, researchers have developed a set of statistical tools to facilitate systematic reviews of evaluation evidence. The set of procedures is called “meta-analysis” and we employ that methodology in this study.13 In Appendix A, we describe these general procedures, the unique adjustments we made to them, and the results of our meta-analyses. A1. Study Selection and Coding Criteria A meta-analysis is only as good as the selection and coding criteria used to conduct the study.14 Following are the key choices we made and implemented. EBT Programs Examined. Due to the broad scope of this project, we did not conduct a systematic review of all evaluations of alcohol, drug, and mental illness disorder treatments. We searched, instead, for studies associated with treatments that are considered evidence-based according to the following published sources: the United States Substance Abuse and Mental Health Services 13 We follow the meta-analytic methods described in: M.W. Lipsey, and D. Wilson. (2001). Practical meta-analysis. Thousand Oaks: Sage Publications. 14 All studies used in the meta-analysis are identified in the references beginning on page 17 of this report. Many other studies were reviewed, but did not meet standards set for this analysis. 7 Administration (SAMHSA), the University of Washington Alcohol and Drug Abuse Institute (ADAI), the Washington Institute for Mental Illness Research and Training (WIMIRT), and the Cochrane Collaboration. We did not include all programs listed by these sources, such as prevention programs for youth, the subject of a previous Washington 15 State Institute for Public Policy (Institute) analysis. We also excluded gambling, tobacco cessation, and workplace programs, and programs that exclusively target the elderly. Exhibit A.1 lists the 57 treatments and practices identified by the following sources, and for which we found studies that met our minimum quality standards. • • SAMHSA maintains a list of model, effective, and 16 promising prevention and treatment programs. For inclusion, we selected programs treating adults with alcohol, drug, or mental health disorders. ADAI publishes a list of evidence-based practices for the prevention and treatment of drug and alcohol abuse, including several programs for the treatment of individuals with co-occurring mental health and substance abuse disorders. We included only the ADAI-listed programs for adults with alcohol, drug abuse, or co-occurring disorders. • WIMIRT has published several reports identifying recommended approaches for treating or managing mental illness in vulnerable populations: children, ethnic and sexual minorities, the elderly, and those with co17 occurring disorders. We included any program listed by WIMIRT that focused on the treatment of mentally ill adults or those with co-occurring disorders. • The Cochrane Collaboration conducts and publishes systematic reviews of the effects of healthcare 18 interventions. Included in this analysis are the results of their reviews of evidence-based treatments for serious mental illness. This was our primary source of evidence for the effects of pharmacological treatments for mental illness. Study Selection. As we describe above, the process for selecting studies of EBT for alcohol, drug, and mental illness disorders was modified to limit the scope of the literature review. We used four primary means to locate studies: (a) for the meta-analysis of EBT programs, we reviewed citations provided by the organization that recommended a particular program; (b) we consulted the study lists of other systematic and narrative reviews of the research literature;19 (c) we examined the citations in the individual studies themselves; and (d) we conducted independent literature searches of research databases using search engines such as Google, Proquest, Ebsco, ERIC, and SAGE. As we will describe, the most important criteria for inclusion in our study was that an evaluation have a control or comparison group. Therefore, after first identifying all possible studies via these search methods, we attempted to determine whether the study was an outcome evaluation that had a comparison group. If a 15 Aos et al., Benefits and costs of prevention and early intervention programs for youth. 16 http://modelprograms.samhsa.gov/template_cf.cfm?page=model_list 17 http://www.spokane.wsu.edu/research%26service/WIMIRT/content/ documents/Intro%20Book.pdf 18 http://www.cochrane.org/reviews/en/topics/index.html 19 Many studies used in our review of alcohol treatment programs were identified in W.R. Miller, and P.L. Wilbourne. (2002). Mesa Grande: A methodological analysis of clinical trials of treatments for alcohol use disorders. Addiction, 97(2): 265-277. Other similar reviews are identified with an asterisk in Exhibit A.2. 8 study met these criteria, we then secured a paper copy of the study for our review. Peer-Reviewed and Other Studies. We examined all program evaluation studies we could locate with these search procedures. Many of these studies were published in peerreviewed academic journals while many others were from government reports obtained from the agencies themselves. It is important to include non-peer reviewed studies, because it has been suggested that peer-reviewed publications may be biased to show positive program effects. Therefore, our meta-analysis includes all available studies regardless of published source. Control and Comparison Group Studies. Our analysis only includes studies that had a control or comparison group. That is, we did not include studies with a single-group, prepost research design. This choice was made because it is only through rigorous comparison group studies that average treatment effects can be reliably estimated. Exclusion of Studies of Program Completers Only. We did not include a comparison study in our meta-analytic review if the treatment group was made up solely of program completers. We adopted this rule because there are too many significant unobserved self-selection factors that distinguish a program completer from a program dropout, and that these unobserved factors are likely to significantly bias estimated treatment effects. Some comparison group studies of program completers, however, also contain information on program dropouts in addition to a comparison group. In these situations, we included the study if sufficient information was provided to allow us to reconstruct an intent-to-treat group that included both completers and non-completers, or if the demonstrated rate of program non-completion was very small (e.g. under 10 percent). In these cases, the study still needed to meet the other inclusion requirements listed here. Random Assignment and Quasi-Experiments. Random assignment studies were preferred for inclusion in our review, but we also included non-randomly assigned control groups. We only included quasi-experimental studies if sufficient information was provided to demonstrate comparability between the treatment and comparison groups on important pre-existing conditions such as age, gender, and pretreatment characteristics such as prior hospitalizations. Enough Information to Calculate an Effect Size. Following the statistical procedures in Lipsey and Wilson (2001), a study had to provide the necessary information to calculate an effect size. If the necessary information was not provided, the study was not included in our review. Mean-Difference Effect Sizes. For this study, we coded mean-difference effect sizes following the procedures in Lipsey and Wilson (2001). For dichotomous measures, we used the arcsine transformation to approximate the mean difference effect size, again following Lipsey and Wilson (2001). We chose to use the mean-difference effect size rather than the odds ratio effect size because we frequently coded both dichotomous and continuous outcomes (odds ratio effect sizes could also have been used with appropriate transformations). Exhibit A.1: Listed Programs, Practices, and Treatments With Studies Meeting Minimum Quality Standards (These treatments are not necessarily recommended by the Institute) Alcohol and Drug Abuse 12-Step Facilitation Therapy (A) Behavioral Couples Therapy (A) Behavioral Self-Control Training (A) Brief Intervention (S) Brief Marijuana Dependence Counseling (A) Cognitive Behavioral Coping Skills Therapy (A) Cognitive Behavioral Therapy for Alcohol Dependence (O) Cognitive Behavioral Therapy for Substance Abuse (O) Community Reinforcement Approach (W) Contingency Management (A) Focus on Families (S) Holistic Harm Reduction (A) Individual Cognitive Behavioral Therapy (A) Individual Drug Counseling Approach to Treat Cocaine Addiction (A) Lower-Cost Contingency Management (A) Matrix Intensive Outpatient Program for Treatment of Stimulants (A) Methadone/Opiate Substitution Treatment (A) Motivational Enhancement Therapy (A) Multidimensional Family Therapy (A) Naltrexone (for Alcohol or Opiates) (A) Relapse Prevention Therapy (A) Mental Health Assertive Community Treatment (S) Behavioral Therapy for Anxiety (O) Behavioral Treatment of Panic Disorder (W) Brief Cognitive Behavioral Intervention for Amphetamine Users (A) Brief Dynamic Psychotherapy for Depression (W) Cognitive Behavior Therapy (W) Cognitive Behavior Therapy for Generalized Anxiety Disorder (W) Cognitive Therapy for Depression (W) Crisis Intervention for People With Severe Mental Illnesses (C) Electroconvulsive Therapy for Schizophrenia (C) Family Intervention (W) Interpersonal Psychotherapy (W) Light Therapy for Depression (C Motivational Interviewing (W) Multi-Family Group Intervention (W) Music Therapy for Schizophrenia (C) Pharmacotherapy for Anxiety Disorder (C) Pharmacotherapy for Bipolar Disorders (C) Pharmacotherapy for Depression (C) Pharmacotherapy for Post Traumatic Stress Disorder (C) Pharmacotherapy for Schizophrenia (C) Psychological Treatment of Post-Traumatic Stress Disorder (C) Mental Health and Substance Abuse PTSD Stress-Management Therapy (C) Anger Management for Substance Abuse and Mental Health Clients (A) Supported Employment (S) Behavioral Treatment for Substance Abuse in Schizophrenia (W) Treatment of Post Traumatic Stress (S) DBT for Substance Abusers with Borderline Personality Disorder (W) Effects of Clozapine on Substance Use Among Schizophrenics (O) Integrated Group Therapy for Bipolar and Substance Disorders (W) Integrated Program for Comorbid Schizophrenia & Substance Use (O) Integrated Treatment for Dual Disorders (W) Listed by: A = Alcohol and Drug Abuse Institute C = The Cochrane Collaboration S = Substance Abuse and Mental Health Services Administration W = Washington Institute for Mental Illness Research and Training O = Other Literature Reviews Note: While practices may be listed by multiple agencies, only one agency is shown. Unit of Analysis. In most cases, our unit of analysis for this study was an independent test of a treatment at a particular site. Some studies reported outcomes for multiple sites; we included each site as an independent observation if a unique and independent comparison group was also used at each site. For certain mental health treatments, we relied on meta-analytic reviews published by the Cochrane Collaboration. In those cases, we computed effect sizes from statistics published in the reviews and the unit of analysis was the review.20 Multivariate Results Preferred. Some studies presented two types of analyses: raw outcomes that were not adjusted for covariates such as age, gender, or pre-treatment characteristics; and those that had been adjusted with multivariate statistical methods. In these situations, we coded the multivariate outcomes. Outcomes Measures of Interest. We only recorded measures that reflected a change in symptoms, behaviors, or other outcomes closely related to the treated disorder. In mental health studies, this includes outcomes such as level of functioning, symptoms, relapse, psychometric scores, hospitalizations, and emergency room visits. Relevant substance abuse outcomes include, for example, quantity consumed, days of use, abstinence, blood or urine tests, arrests, employment, and reports of problems due to substance abuse. We did not record process and quality 20 We tested the validity of this approach by meta-analyzing the results of 16 individual studies reported in three Cochrane reviews of treatments for schizophrenia and compared the results to metaanalysis of the three reviews. The resulting standardized effect sizes differed by only 0.01. measures such as rates of treatment completion, number of counseling sessions, client satisfaction, and quality of services, etc. Choosing Among Different Outcome Measures. A single study may report a variety of outcomes. For example, one study of mental illness treatment may report psychometric scores and police contacts. A study of an alcohol abuse treatment may report the quantity of alcohol consumed per day and arrests. In such cases we recorded the outcome that most directly reflected the effect of treatment on the primary disorder: in the examples above, we would have recorded the treatment effects of psychometric scores and the quantity of alcohol consumed, respectively. Averaging Effect Sizes for Similar Outcomes. Some studies reported similar outcomes: e.g., a variety of psychometric scores in the case of a mental health treatment, or a number of different measures of substance use for an alcohol or drug treatment. In such cases, we calculated an effect size for each measure and then took a simple average. As a result, each experimental trial coded in this study is associated with a single effect size that reflects a general reduction in the severity or incidence of a given disorder. Dichotomous Measures Preferred Over Continuous Measures. Some studies included two types of measures for the same outcome: a dichotomous (yes/no) outcome and a continuous (mean number) measure. In these situations, we coded an effect size for the dichotomous measure. Our rationale for this choice is that in small or relatively small sample studies, continuous measures of treatment outcomes can be unduly influenced by a small number of outliers, while 9 dichotomous measures can avoid this problem. Of course, if a study only presented a continuous measure, we coded the continuous measure. Longest Follow-Up Periods. When a study presented outcomes with varying follow-up periods, we generally coded the effect size for the longest follow-up period. The longest follow-up period allows us to gain the most insight into the long-run benefits and costs of various treatments. Occasionally, we did not use the longest follow-up period if it was clear that a longer reported follow-up period adversely affected the attrition rate of the treatment and comparison group samples. Some Special Coding Rules for Effect Sizes. Most studies in our review had sufficient information to code exact meandifference effect sizes. Some studies, however, reported some, but not all the information required. We followed the following rules for these situations: • • Two-tail p-values. Some studies only reported p-values for significance testing of program outcomes. When we had to rely on these results, if the study reported a onetail p-value, we converted it to a two-tail test. Declaration of significance by category. Some studies reported results of statistical significance tests in terms of categories of p-values. Examples include: p<=.01, p<=.05, or non-significant at the p=.05 level. We calculated effect sizes for these categories by using the highest p-value in the category. Thus, if a study reported significance at p<=.05, we calculated the effect size at p=.05. This is the most conservative strategy. If the study simply stated a result was non-significant, we computed the effect size assuming a p-value of .50 (i.e. p=.50). A2. Procedures for Calculating Effect Sizes Me − Mc A(2): ES m = SDe2 + SDc2 2 In this formula, ESm is the estimated effect size for the difference between means from the research information; Me is the mean number of an outcome for the experimental group; Mc is the mean number of an outcome for the control group; SDe is the standard deviation of the mean number for the experimental group; and SDc is the standard deviation of the mean number for the control group. Often, research studies report the mean values needed to compute ESm in (A2), but they fail to report the standard deviations. Sometimes, however, the research will report information about statistical tests or confidence intervals that can then allow the pooled standard deviation to be estimated. These procedures are also described in Lipsey and Wilson (2001). Adjusting Effect Sizes for Small Sample Sizes Since some studies have very small sample sizes, we follow the recommendation of many meta-analysts and adjust for this. Small sample sizes have been shown to upwardly bias effect sizes, especially when samples are less than 20. Following 23 24 Hedges, Lipsey and Wilson report the “Hedges correction factor,” which we use to adjust all mean difference effect sizes (N is the total sample size of the combined treatment and comparison groups): [ 3 ⎤ ⎡ A(3): ES′m = ⎢1 − ⎥ × ES m , or , ES m ( p ) ⎣ 4N − 9 ⎦ ] Effect sizes measure the degree to which a program has been shown to change an outcome for program participants relative to a comparison group. There are several methods used by metaanalysts to calculate effect sizes, as described in Lipsey and Wilson (2001). In this analysis, we used statistical procedures to calculate the mean difference effect sizes of programs. We did not use the odds-ratio effect size because many of the outcomes measured in this study are continuously measured. Thus, the mean difference effect size was a natural choice. Computing Weighted Average Effect Sizes, Confidence Intervals, and Homogeneity Tests. Once effect sizes are calculated for each program effect, the individual measures are summed to produce a weighted average effect size for a program area. We calculate the inverse variance weight for each program effect and these weights are used to compute the average. These calculations involve three steps. First, the standard error, SEm of each mean effect size is computed with:25 Many of the outcomes we record, however, are measured as dichotomies. For these yes/no outcomes, Lipsey and Wilson (2001) show that the mean difference effect size calculation can be approximated using the arcsine transformation of the 21 difference between proportions. A(4): SEm = A(1): ESm( p ) = 2 × arcsin Pe − 2 × arcsin Pc In this formula, ESm(p) is the estimated effect size for the difference between proportions from the research information; Pe is the percentage of the population that had an outcome such as re-arrest rates for the experimental or treatment group; and Pc is the percentage of the population that was re-arrested for the control or comparison group. A second effect size calculation involves continuous data where the differences are in the means of an outcome. When an evaluation reports this type of information, we use the 22 standard mean difference effect size statistic. 21 Aos et al., Benefits and costs of prevention and early intervention programs for youth, Table B10, equation 22. 22 Ibid., Table B10, equation 1. 10 ′ )2 ne + nc ( ES m + ne nc 2(ne + nc ) In equation (A4), ne and nc are the number of participants in the experimental and control groups and ES'm is from equation (A3). Next, the inverse variance weight wm is computed for each 26 mean effect size with: A(5): wm = 23 1 SEm2 L.V. Hedges. (1981) Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6: 107-128. 24 Lipsey and Wilson, Practical meta-analysis, 49, equation 3.22. 25 Ibid., 49, equation 3.23. 26 Ibid., 49, equation 3.24. The weighted mean effect size for a group of studies in program area i is then computed with:27 A(6): ES = ∑ (w ES ′ ∑w mi mi ) mi Confidence intervals around this mean are then computed by 28 first calculating the standard error of the mean with: A(7): SE = ES 1 ∑ wmi Next, the lower, ESL, and upper limits, ESU, of the confidence interval are computed with:29 A(8): ES L = ES − z(1−α ) ( SE ES ) A(9): ESU = ES + z(1−α ) ( SE ES ) In equations (A8) and (A9), z(1-α) is the critical value for the zdistribution (1.96 for α = .05). The test for homogeneity, which provides a measure of the dispersion of the effect sizes around their mean, is given by:30 A(10): Qi = (∑ wi ESi2 ) − (∑ wi ESi ) 2 ∑ wi The Q-test is distributed as a chi-square with k-1 degrees of freedom (where k is the number of effect sizes). In Appendix A3, we describe our rationale for making these downward adjustments. In particular, we make three types of adjustments that are necessary to better estimate the results that we are more likely to achieve in real-world settings. We make adjustments for: (a) the methodological quality of each study we include in the meta-analyses; (b) the relevance or quality of the outcome measure that individual studies used; and (c) the degree to which the researcher(s) who conducted a study were invested in the program’s design. A3.a. Methodological Quality. Not all research is of equal quality, and this greatly influences the confidence that can be placed in the results of a study. Some studies are well designed and implemented, and the results can be viewed as accurate representations of whether the program itself worked. Other studies are not designed as well, and less confidence can be placed in any reported differences. In particular, studies of inferior research design cannot completely control for sample selection bias or other unobserved threats to the validity of reported research results. This does not mean that results from these studies are of no value, but it does mean that less confidence can be placed in any cause-and-effect conclusions drawn from the results. To account for the differences in the quality of research designs, we use a 5-point scale as a way to adjust the reported results. The scale is based closely on the 5-point scale developed by 32 researchers at the University of Maryland. On this 5-point scale, a rating of “5” reflects an evaluation in which the most confidence can be placed. As the evaluation ranking gets lower, less confidence can be placed in any reported differences (or lack of differences) between the program and comparison or control groups. On the 5-point scale as interpreted by the Institute, each study is rated with the following numerical ratings. Computing Random Effects Weighted Average Effect Sizes and Confidence Intervals. When the p-value on the Q-test indicates significance at values of p less than or equal to .05, a random effects model is performed to calculate the weighted average effect size. This is accomplished by first calculating the random effects variance component, v.31 A(11): v = Qi − (k − 1) ∑ wi − (∑ wsqi ∑ wi ) • A “5” is assigned to an evaluation with well-implemented random assignment of subjects to a treatment group and a control group that does not receive the treatment/program. A good random assignment study should also indicate how well the random assignment actually occurred by reporting values for pre-existing characteristics for the treatment and control groups. • A “4” is assigned to a study that employs a rigorous quasi-experimental research design with a program and matched comparison group, controlling with statistical methods for self-selection bias that might otherwise influence outcomes. These quasi-experimental methods may include estimates made with a convincing instrumental variables modeling approach, or a Heckman approach to modeling self-selection.33 A level 4 study may also be used to “downgrade” an experimental random assignment design that had problems in implementation, perhaps with significant attrition rates. • A “3” indicates a non-experimental evaluation where the program and comparison groups were reasonably well matched on pre-existing differences in key variables. There must be evidence presented in the evaluation that This random variance factor is then added to the variance of each effect size and then all inverse variance weights are recomputed, as are the other meta-analytic test statistics. A3. Institute Adjustments to Effect Sizes for Methodological Quality, Outcome Measure Relevance, and Researcher Involvement In Exhibits A.2 – A.4 we show the results of our meta-analyses calculated with the standard meta-analytic formulas described in Appendix A2. In the last columns in each exhibit, however, we list “Adjusted Effect Sizes” that we actually use in our benefit-cost analysis of each program area: alcohol, drug, and mental illness treatment. These adjusted effect sizes, which are derived from the unadjusted results, are always smaller than or equal to the unadjusted effect sizes we report in the same exhibit. 32 27 Ibid., 114. Ibid. 29 Ibid. 30 Ibid., 116. 31 Ibid., 134. 28 L. Sherman, D. Gottfredson, D. MacKenzie, J. Eck, P. Reuter, and S. Bushway. (1998). Preventing crime: What works, what doesn't, what's promising. Prepared for the National Institute of Justice. Department of Criminology and Criminal Justice, University of Maryland. Chapter 2. 33 For a discussion of these methods, see W. Rhodes, B. Pelissier, G. Gaes, W. Saylor, S. Camp, and S. Wallace. (2001). Alternative solutions to the problem of selection bias in an analysis of federal residential drug treatment programs. Evaluation Review, 25(3): 331-369. 11 indicates few, if any, significant differences were observed in these salient pre-existing variables. Alternatively, if an evaluation employs sound multivariate statistical techniques (e.g., logistic regression) to control for preexisting differences, and if the analysis is successfully completed, then a study with some differences in preexisting variables can qualify as a level 3. • A “2” involves a study with a program and matched comparison group where the two groups lack comparability on pre-existing variables and no attempt was made to control for these differences in the study. • A “1” involves a study where no comparison group is utilized. Instead, the relationship between a program and an outcome, i.e., drug use, is analyzed before and after the program. We do not use the results from program evaluations rated as a “1” on this scale, because they do not include a comparison group and, thus, no context to judge program effectiveness. We also regard evaluations with a rating of “2” as highly problematic and, as a result, do not consider their findings in the calculations of effect. In this study, we only considered evaluations that rated at least a 3 on this 5-point scale. An explicit adjustment factor is assigned to the results of individual effect sizes based on the Institute’s judgment concerning research design quality. This adjustment is critical and the only practical way to combine the results of a high quality study (e.g., a level 5 study) with those of lesser design quality (level 4 and level 3 studies). The specific adjustments made for these studies are based on our knowledge of research in other topic areas. For example, in criminal justice program evaluations, there is strong evidence that random assignment studies (i.e., level 5 studies) have, on average, smaller absolute effect sizes than weaker-designed studies.34 Thus, we use the following “default” adjustments to account for studies of different research design quality: • A level 5 study carries a factor of 1.0 (that is, there is no discounting of the study’s evaluation outcomes). • A level 4 study carries a factor of .75 (effect sizes discounted by 25 percent). • A level 3 study carries a factor of .50 (effect sizes discounted by 50 percent). • We do not include level 2 and level 1 studies in our analyses. These factors are subjective to a degree; they are based on the Institute’s general impressions of the confidence that can be placed in the predictive power of evaluations of different quality. The effect of the adjustment is to multiply the effect size for any study, ES'm, in equation (A3) by the appropriate research design factor. For example, if a study has an effect size of -.20 and it is deemed a level 4 study, then the -.20 effect size would be multiplied by .75 to produce a -.15 adjusted effect size for use in the benefit-cost analysis. A3.b. Adjusting Effect Sizes of Studies With Short-Term Follow-Up Periods. To account for the likelihood that the effects of treatment do not persist indefinitely for all subjects, we discount effect sizes, ESm, over time. The majority of studies coded report only short-term outcomes. Few of the studies provided outcomes beyond one year post-treatment and many reported outcomes only during or at the end of a treatment episode. Therefore, the unadjusted meta-analytic effect sizes reflect relatively short-term outcomes. To reflect the likelihood that the effects of a given treatment will decline over time, we built in a “decay” factor. In Appendix B, we discuss the methods by which we decay these effects. A3.c. Adjusting Effect Sizes for Research Involvement in the Program’s Design and Implementation. The purpose of the Institute’s work is to identify and evaluate programs that can make cost-beneficial improvements to Washington’s actual service delivery system. There is some evidence that programs closely controlled by researchers or program developers have better results than those that operate in “real 35 world” administrative structures. In our evaluation of a realworld implementation of a research-based juvenile justice program in Washington, we found that the actual results were considerably lower than the results obtained when the intervention was conducted by the originators of the program.36 Therefore, we make an adjustment to effect sizes, ESm, to reflect this distinction. As a parameter for all studies deemed not to be “real world” trials, the Institute discounts ES'm by .5, although this can be modified on a study-by-study basis. A4. Meta-Analytic Results—Estimated Effect Sizes and Citations to Studies Used in the Analyses Exhibits A. 2, A.3, and A.4 provide technical meta-analytic results for the effect sizes computed for this analysis. Each table provides the unadjusted and adjusted effect sizes for EBT in each of the three program areas, and lists all of the studies included in each analysis. Exhibit A.5 lists the citations for all studies used in the meta-analyses. The meta-analytic results of the effects of EBT on disordered alcohol use are displayed in Exhibit A.2. The results for disordered drug use and mental illness are displayed in Exhibits A.3 and A.4, respectively. 35 34 M.W. Lipsey. (2003). Those confounded moderators in meta-analysis: Good, bad, and ugly. The Annals of the American Academy of Political and Social Science, 587(1): 69-81. Lipsey found that, for juvenile delinquency evaluations, random assignment studies produced effect sizes only 56 percent as large as nonrandom assignment studies. 12 Ibid. Lipsey found that, for juvenile delinquency evaluations, programs in routine practice (i.e., “real world” programs) produced effect sizes only 61 percent as large as research/demonstration projects. See also: A. Petrosino, and H. Soydan. (2005). The impact of program developers as evaluators on criminal recidivism: Results from meta-analyses of experimental and quasi-experimental research. Journal of Experimental Criminology, 1(4): 435-450. 36 R. Barnoski. (2004). Outcome evaluation of Washington State's research-based programs for juvenile offenders. Olympia: Washington State Institute for Public Policy, available at <http://www.wsipp.wa.gov/rptfiles/04-01-1201.pdf>. Exhibit A.2: Meta-Analytic Results of the Effects of EBT on Disordered Alcohol Use Adjusted Effect Size Used in the Fixed Effects Model Random Effects Model BenefitCost Homogeneity Weighted Mean Effect Size & Analysis Weighted Mean Effect Size & p-value Test p-value Alcohol Treatment Effects Number of trials used in analysis: Results Before Adjusting Effect Sizes 100 Number of subjects in treatment group: 7,973 Studies Used in the Meta-Analysis Name of Study Aalto, et al. (2000) Essm -0.097 N Tx p-value p-value ES p-value ES -0.253 0.000 0.000 -0.312 0.000 -0.247 Not real Design world Score =1 N Cn 39 ES 39 5 0 ESAdj Name of Study Essm N Tx Design Not real Score world =1 N Cn ESAdj -0.097 Lhuintre, et al. (1990) -0.052 181 175 5 0 -0.052 -0.620 Aalto, et al. (2000) -0.351 37 39 5 0 -0.351 Maheswaran, et al. (1992) -0.620 21 20 5 0 Adams (1990) -0.555 29 16 3 0 -0.277 Mallams, et al. (1982) -0.666 19 16 5 0 -0.666 Allsop, et al. (1997) -0.247 15 14 5 0 -0.247 Manwell et al. (2000) -0.231 103 102 5 1 -0.115 Anderson, et al. (1992) -0.300 80 74 5 0 -0.300 Marlatt, et al. (1998) -0.251 174 174 5 0 -0.251 Anton, et al. (1999) -0.363 68 63 5 0 -0.363 Mason , et al. (1994) -0.780 7 6 5 0 -0.780 Anton, et al. (2006) -0.081 917 309 5 0 -0.081 Mason , et al. (1999) -0.290 70 35 5 0 -0.290 Anton, et al. (2006) -0.184 157 153 5 0 -0.184 McCrady, et al. (1999) 0.081 24 22 3 1 0.020 Anton, et al. (2006) -0.092 619 607 5 0 -0.092 McCrady, et al. (1999) -0.202 24 21 5 0 -0.202 Azrin (1976) -1.460 9 9 5 1 -0.730 Miller, et al. (1981) -0.350 19 16 5 0 -0.350 Babor, et al. (1992) -0.372 350 361 5 0 -0.372 Miller, et al. (1980) -0.270 19 16 4 1 -0.101 Babor, et al. (1993) -0.312 350 409 5 0 -0.312 Miller, et al. (1993) -0.618 14 14 5 1 -0.309 Bien, et al. (1993) -0.264 18 16 5 0 -0.264 Miller, et al. (2001) 0.178 28 30 5 0 0.178 Bosari, et al. (2000) -0.615 29 30 5 1 -0.308 Miller, et al. (2001) -0.040 32 33 5 0 -0.040 Bowers, et al. (1990) -0.603 15 13 5 0 -0.603 Miller, et al. (2001) 0.158 29 35 5 0 0.158 Brown (1993) -0.399 14 14 5 0 -0.399 Miller, Taylor, & West (1980) -0.201 10 10 4 0 -0.151 -0.151 Chaney, O'Leary, Marlatt (1978) -0.273 14 25 4 1 -0.102 Miller, Taylor, & West (1980) -0.201 10 10 4 0 Chick (1985) -0.496 69 64 5 0 -0.496 Monti, et al. (1990) 0.000 23 23 5 0 0.000 Chick, et al. (1988) -0.189 54 41 5 0 -0.189 Monti, et al. (1993) -0.538 7 11 5 0 -0.538 Collins, et al., (2002) 0.418 23.97 23.52 5 0 0.418 Murphy, et al. (2001) -0.183 30 24 5 0 -0.183 Collins, et al., (2002) -0.533 22.56 24.48 5 0 -0.533 Neighbors, et al. (2004) -0.326 126 126 5 0 -0.326 Donovan, et al. (1988) -0.155 20 19 5 0 -0.155 Nelson & Howell (1982-83) -0.538 16 9 3 0 -0.269 Drake, et al. (1997) -0.653 69 28 3 0 -0.326 Nilssen (1991) -0.626 212 108 5 0 -0.626 Drake, et al. (1998) a -0.033 75 68 5 0 -0.033 Obolensky (1984) -0.842 9 13 3 0 -0.626 Drake, et al. (1998) b -0.158 83 73 5 0 -0.158 O'Connell (1987) -0.074 12 11 3 0 -0.037 Drake, et al. (2000) -0.944 19 86 3 0 -0.472 Oei & Jackson (1980) -0.704 16 16 3 0 -0.352 Elvy, et al. (1988) -0.169 48 72 5 0 -0.169 Oei & Jackson (1982) -0.867 16 8 3 0 -0.434 Eriksen, Bjornstad, & Gotestam (1986) -1.139 11 12 3 1 -0.285 Oei & Jackson (1982) -0.867 16 8 3 0 -0.434 Fals-Stewart, et al. (1996) -0.174 40 40 5 1 -0.087 O'Farrell, et al. (1993) -0.578 30 29 5 0 -0.578 Feeney, et al. (2002) -0.557 50 50 3 0 -0.279 O'Malley, et al. (1992) -0.819 22 27 5 1 -0.410 Ferrell & Galassi (1981) -0.951 8 9 5 1 -0.475 Ouimette, et al. (1997) -0.076 897 1148 4 0 -0.057 Fichter, et al. (1993) -0.061 45 45 5 0 -0.061 Paille, et al. (1995) -0.172 173 177 5 0 -0.172 Fleming, et al. (2000) -0.406 392 382 5 0 -0.406 Persson, et al. (1989) -0.526 31 23 5 0 -0.526 Graeber, et al. (2003) -1.332 15 15 4 0 -0.999 Reynolds, et al. (1995) -0.449 42 36 5 0 -0.449 Handmaker, et al. (1999) -0.221 18 16 5 1 -0.111 Richmond, et al. (1995) -0.145 70 61 3 0 -0.073 Harris et al. (1990) -0.519 9 17 5 1 -0.259 Rohsenhow, Smith, & Johnson (1985) -0.232 14 20 4 0 -0.174 Heather et al. (1987) -0.028 34 38 5 1 -0.014 Romelsjo, et al. (1989) -0.147 41 42 5 0 -0.147 Heather, et al. (1996) -0.372 47 33 5 0 -0.372 Sanchez-Craig, et al. (1991) -0.101 29 67 5 0 -0.101 Hedberg, et al. (1974) -0.683 15 15 5 0 -0.683 Sanchez-Craig, et al. (1996) -0.006 74 81 5 1 -0.003 Hellerstein, et al. (1995) -0.776 23 24 5 0 -0.776 Sannibale (1989) -0.024 31 41 4 1 -0.009 Hester & Delaney (1997) -0.633 20 20 5 1 -0.317 Sass, et al. (1996) -0.498 136 136 5 0 -0.498 Hulse, et al. (2002) -0.719 47 36 4 0 -0.540 Scott (1989) -0.070 33 39 5 0 -0.070 Hunt & Azrin (1973) -1.572 8 8 3 1 -0.393 Sisson & Azrin (1986) -2.479 7 5 5 1 -1.240 James, et al. (2004) -0.260 29 29 5 0 -0.260 Smith et al. (1998) -0.470 49 32 4 0 -0.352 Jones, Kanfer, & Lanyon (1982) -0.884 24 21 4 0 -0.663 Smith, et al. (1999) -0.275 91 76 3 0 -0.138 Kelly, et al. (2000) -0.900 11 9 5 1 -0.450 Tomson, et al. (1998) -0.158 45 30 5 0 -0.158 Kivlahan et al. (1990) -0.870 15 15 5 1 -0.435 Volpicelli, et al. (1992) -0.643 35 35 5 0 -0.643 Kuchipudi, et al. (1990) -0.067 59 55 5 0 -0.067 Wallace, et al. (1988) -0.424 247 337 5 0 -0.424 Larimer, et al. (2001) -0.394 60 60 3 0 -0.197 Whitworth, et al. (1996) -0.257 74 74 4 0 -0.192 Lhuintre, et al. (1985) -0.56642 33 37 5 0 -0.566 Winters, et al (2002) -0.435 33 35 5 0 -0.435 13 Exhibit A.3: Meta-Analytic Results of the Effects of EBT on Disordered Drug Use Adjusted Effect Size Used in the Fixed Effects Model Random Effects Model BenefitCost Homogeneity Weighted Mean Effect Size & Analysis Weighted Mean Effect Size & p-value Test p-value Treatment for Disordered Drug Use Number of trials used in analysis: Results Before Adjusting Effect Sizes 44 Number of subjects in treatment group: 3,506 ES p-value p-value ES p-value ES -0.360 0.000 0.000 -0.451 0.000 -0.355 Studies Used in the Meta-Analysis Name of Study Avants, et al. (2004) Essm -0.232 N Tx 108 N Cn 112 Score 5 real ESAdj Name of Study 0 -0.232 Johnson, et al. (1992) Essm -0.491 N Tx 90 N Cn Score world =1 60 5 0 ESAdj -0.491 Azrin, et al. (1996) -0.651 37 37 3 1 -0.163 Johnson, et al. (1995) -0.641 90 60 5 0 -0.641 Azrin, et al.(1994) -0.714 15 11 4 1 -0.268 Johnson, et al. (2000) -0.487 55 55 5 0 -0.487 Baker, et al. (2001) -0.688 32 32 3 1 -0.172 Kavanagh, et al. (2004) -0.725 13 8 5 0 -0.725 Baker, et al. (2005) -0.472 74 74 4 1 -0.177 Ling, et al. (1998) -0.443 90 60 5 0 -0.443 Baker, et al. (2005) -0.494 66 74 4 1 -0.185 Margolin, et al. (2003) -0.383 45 45 5 0 -0.383 Bellack, et al. (2006) -0.680 61 49 5 1 -0.340 Marijuana Treatment Project (2004) -0.216 127 137 5 0 -0.216 Carroll, et al. (1991) -0.206 21 21 4 1 -0.077 Marijuana Treatment Project (2004) -0.610 132 137 5 0 -0.610 Carroll, et al. (1994) -0.461 52 45 4 1 -0.173 Milby, et al. (1996) -0.044 69 62 3 0 -0.022 Catalano, et al. (2002) -0.048 63 63 5 1 -0.024 Newman, et al. (1979) -0.827 50 50 5 0 -0.827 Cornish, et al. (1997) -0.577 34 17 5 0 -0.577 Petry & Martin (2002) -1.498 19 23 5 0 -1.498 Ctrits-Christoph, et al. (1999) -0.237 121 123 4 0 -0.178 Petry, et al. ( 2000) -0.772 19 23 5 0 -0.772 Dole, et al. (1969) -2.051 12 16 5 1 -1.026 Piotrowski, et al. (1999) 0.000 51 51 5 0 0.000 Drake, et al. (1997) -0.113 78 29 3 0 -0.056 Rawson, et al. (1995) -0.122 41 44 5 0 -0.122 Drake, et al. (1998)a -0.178 45 40 5 0 -0.178 Schottenfeld, et al. (1997) -0.291 30 29 5 0 -0.291 Drake, et al. (1998)b -0.124 45 40 5 0 -0.124 Silverman, et al., (1996) -0.534 15 15 4 0 -0.401 Drake, et al. (2000) -0.687 11 54 3 0 -0.344 Silverman, et al., (1998) -1.554 36 15 4 0 -1.165 Fudala, et al. (2003) -0.421 214 109 5 0 -0.421 Stephens, et al. (2000) -0.598 117 86 5 0 -0.598 Gronbladh, et al. (1989) -0.896 17 17 4 0 -0.672 Stephens, et al. (2000) -0.497 88 86 5 0 -0.497 Higgins, et al. (2000) -0.360 36 34 5 1 -0.180 Strain, et al. (1993) -0.329 84 81 5 0 -0.329 Humphreys, et al., (1999) -0.189 897 1148 4 0 -0.142 Vanichseni, et al. (1991) -0.511 120 120 5 0 -0.511 James, et al. (2004) -0.868 29 29 5 0 -0.868 Woody, et al. (1995) -0.463 57 27 3 1 -0.116 14 A.4: Meta-Analytic Results of the Effects of EBT on Mental Illness Our benefit-cost analysis focused on serious mental illness: non-affective psychosis (including schizophrenia), bipolar disorder and severe forms of panic disorder, and depression. Because studies rarely indicated the severity of subjects’ mental disorders in the studies, our analysis included all programs for depression, and we estimated effects for panic disorder based on studies for treatments of anxiety disorders. To derive a single effect size for mental illness treatments, we first calculated effect sizes for four categories of mental illness: non-affective psychosis, bipolar, anxiety, and major depressive disorders. After weighting according to prevalence among the populations with serious mental illness, we combined the separate effect sizes into a single average (see the following table). Disorder Schizophrenia (Non-affective psychosis) Bipolar disorder Anxiety disorders Major Depressive Disorder All Mental Illness Weight 0.079 0.410 0.191 0.321 1.000 Adjusted ES for BenefitCost Analysis ES Std Err -0.323 0.029 -0.382 0.048 -0.404 0.045 -0.280 0.061 -0.360 0.047 Note: Relative prevalence was based on incidence of serious major depression, serious panic disorder, and bipolar I and II reported from the National 37 38 Comorbidity Survey Replication and non-affective psychosis as reported in the National Comorbidity Survey. Adjusted Effect Size Used in the Fixed Effects Model Random Effects Model BenefitCost Homogeneity Weighted Mean Effect Size & Analysis Weighted Mean Effect Size & p-value Test p-value Results Before Adjusting Effect Sizes Treatments for Bipolar Disorder Number of trials used in analysis: Number of subjects in treatment group: 6 ES p-value p-value ES p-value ES 933 -0.386 0.000 -0.549 na na -0.382 Studies Used in the Meta-Analysis Name of Study Burgess, et al.(2001) Essm N Tx N Cn Not real Design world Score =1 -0.300 413 412 Macritchie, et al. (2003) -0.512 155 161 5 0 -0.512 Rendell, et al. (2003) -0.426 70 66 5 0 -0.426 16 Studies Used in the Meta-Analysis Browne, et al. (2002) Essm 0.045 N Tx 212 -0.300 Rendell, et al. (2003) N Cn Design Not real Score world =1 5 0 ESAdj -0.524 54 56 -0.524 Rendell, et al. (2003) -0.454 220 114 5 0 -0.454 Weiss, et al. (2000) -0.200 21 24 3 1 -0.050 Adjusted Effect Size Used in the Fixed Effects Model Random Effects Model BenefitCost Homogeneity Weighted Mean Effect Size & Analysis Weighted Mean Effect Size & p-value Test p-value Number of subjects in treatment group: 1,479 Name of Study 0 N Tx Essm Results Before Adjusting Effect Sizes Treatments for Depression Number of trials used in analysis: 5 ESAdj Name of Study N Cn ES p-value p-value ES p-value ES -0.314 0.000 0.018 -0.323 0.000 -0.315 Not real Design world =1 Score ESAdj Name of Study Essm N Tx N Cn Design Not real Score world =1 ESAdj 196 4 0 0.033 Shea, et al (1992) -0.194 59 62 5 0 -0.194 -0.008 Fava, et al. (1998) -0.513 20 20 5 0 -0.513 Shea, et al (1992) -0.008 61 62 5 0 Lima, et al. (2006) -0.434 206 179 5 0 -0.434 Simons, et al. (1986) -0.610 36 16 3 0 -0.305 Lima, et al. (2006) -0.528 143 155 5 0 -0.528 Tuunainen, et al. (2004) -0.060 39 32 5 0 -0.060 Lima, et al. (2006) -0.366 295 305 5 0 -0.366 Ward, et al., (2000) -0.185 63 67 5 0 -0.185 Moncrieff, et al. (2004) -0.325 395 355 5 0 -0.325 Wijkstra, et al. (2005) -0.430 48 101 5 0 -0.430 Reynolds, et al. (2006) -0.493 25 28 5 0 -0.493 Wijkstra, et al. (2005) -0.368 100 101 5 0 -0.368 Reynolds, et al. (2006) 0.623 25 29 5 0 -0.623 Wijkstra, et al. (2005) -0.786 22 17 5 0 -0.786 37 R.C. Kessler, W.T. Chiu, O. Demler et al. (2005), Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62(6):617-627. 38 R.C. Kessler, K.A. McGonagle, S. Zhao et al. (1994). Lifetime and 12-month prevalence of DSM-III-R Psychiatric Disorders in the United States. Archives of General Psychiatry, 51: 8-19. 15 Adjusted Effect Size Used in the Fixed Effects Model Random Effects Model BenefitCost Homogeneity Weighted Mean Effect Size & Analysis Weighted Mean Effect Size & p-value Test p-value Results Before Adjusting Effect Sizes Treatments for Anxiety Disorders Number of trials used in analysis: 31 Number of subjects in treatment group: 4,641 Studies Used in the Meta-Analysis Name of Study Essm N Tx ES p-value p-value ES p-value ES -0.256 0.000 0.000 -0.563 0.000 -0.404 Not real Design world Score =1 N Cn ESAdj Name of Study N Tx Essm N Cn Design Not real Score world =1 ESAdj Barlow, et al. (1989) -1.394 10 15 5 0 -1.394 Cordioli, et al. (2003) -1.201 23 24 5 0 -1.201 Barlow, et al. (1989) -0.858 15 15 5 0 -0.858 Dugas, et al. (2003) -1.364 25 37 5 1 -0.682 Barlow, et al. (1989) -0.938 16 15 5 0 -0.938 Durham, et al. (1994) -0.710 35 29 4 1 -0.266 Barlow, et al. (1984) -1.205 10 10 5 0 -1.205 Kapczinski (2003) -0.378 277 280 5 0 -0.378 Barlow, et al. (2000) -0.553 60 22 5 0 -0.553 Ladouceur, et al. (2000) -1.571 14 12 5 1 -0.785 Barlow, et al., (1992) -1.588 24 10 3 1 -0.397 Lindsay, et al. (1987) -1.200 10 10 5 1 -0.600 Beck, et al. (1992) -0.507 17 16 3 1 -0.127 Linehan, et al. (1999) -0.289 12 16 5 1 -0.144 Bisson & Andrew (2005) -0.375 79 77 4 0 -0.281 Marks, et al., (1993) -0.909 23 17 3 0 -0.454 Bisson & Andrew (2005) -0.426 266 187 5 0 -0.426 Mortberg, et al. (2005) -1.005 12 12 5 1 -0.502 Bisson & Andrew (2005) -1.006 44 42 5 0 -1.006 Pittler, et al. (2003) -0.201 197 183 5 0 -0.201 Blomhoff, et al. (2001) -0.256 91 88 5 0 -0.256 Stein, et al. (2000) -0.146 1872 1824 5 0 -0.146 Blomhoff, et al. (2001) -0.508 88 88 5 0 -0.508 Stein, et al (2006 ) -0.177 1270 1237 5 0 -0.177 Borkovec & Costello (1993) -0.342 18 20 4 1 -0.128 White & Keenan (1992) -0.119 26 10 3 1 -0.030 Borkovec & Mathews (1988) -0.410 10 10 5 0 -0.410 White & Keenan (1992) -0.354 31 10 3 1 -0.089 Borkovec, et al (1987) -0.367 16 14 4 0 -0.275 White & Keenan (1992) -0.318 31 10 3 1 -0.080 Butler, et al., (1991) -1.203 19 19 5 0 -1.203 Adjusted Effect Size Used in the Fixed Effects Model Random Effects Model BenefitCost Homogeneity Weighted Mean Effect Size & Analysis Weighted Mean Effect Size & p-value Test p-value Treatments for Non-Affective Psychosis (Including Schizophrenia) Number of trials used in analysis: Results Before Adjusting Effect Sizes 49 Number of subjects in treatment group: 3,926 ES p-value p-value ES p-value ES -0.370 0.000 0.000 -0.423 0.000 -0.324 Studies Used in the Meta-Analysis Name of Study Essm N Tx N Cn Score real ESAdj Name of Study Essm -0.542 N Tx N Cn Score world =1 ESAdj Aber-Wistedt, et al. (1995) -0.557 20 20 5 0 -0.557 Haddock, et al., (2006) Barrowclough, et al. (2001) -0.633 17 15 5 1 -0.317 Hoult, et al. (1984) Bigelow, et al. (1991) -0.812 15 7 3 0 -0.406 James, et al. (2004) Bond, et al. (1988) -0.460 84 83 5 0 -0.460 Joy, et al. (2004) -0.202 228 237 5 0 -0.202 Bond, et al. (1988) -0.516 84 83 5 0 -0.516 Lehman, et al. (1994) -0.201 359 302 3 0 -0.101 Bond, et al. (1990) -0.743 42 40 5 0 -0.743 Lehman, et al. (1997) -0.354 77 75 5 0 -0.354 Bond, et al. (1991) -1.098 30 10 3 1 -0.274 Lewis, et al. (2005) -0.308 40 38 5 0 -0.308 Bond, et al. (1995) -0.502 39 35 3 0 -0.251 Macias, et al. (1994) -0.802 19 18 4 0 -0.602 Bush. et al. (1990) -0.832 14 14 5 0 -0.832 Marques (2004) -0.266 208 207 5 0 -0.266 -0.541 -0.60129 15 14 5 1 -0.271 26 25 5 0 -0.541 29.000 29 5 0 -0.60129 Curtis, et al. (1996) -0.023 147 145 5 0 -0.023 McFarlane (2002) -0.514 27 14 5 0 -0.514 Chandler, et al. (1996) -0.450 115 108 5 0 -0.450 McFarlane (2002) -0.291 50 50 5 0 -0.291 Chandler, et al. (1997) -0.431 105 105 3 1 -0.108 McFarlane (2002) -0.297 34 34 5 0 -0.297 Chandler, et al. (1997) -0.431 105 105 3 1 -0.108 McFarlane, et al. (1995) -0.289 83 89 5 0 -0.289 Drake et al. (1998) -0.017 105 98 5 0 -0.017 McFarlane, et al. (2000) 0.585 37 32 5 0 0.585 Drake, et al. (1996) -0.660 39 35 3 0 -0.330 Morse, et al. (1997) -0.421 90 45 4 0 -0.316 Drake, et al. (1999) -1.166 74 76 3 1 -0.292 Mota Neto, et al. (2002) -0.590 159 83 5 0 -0.590 Dyck, et al. (2002) -0.428 55 51 5 0 -0.428 Quinlivan, et al. (1995) -0.510 30 30 5 0 -0.510 Dyck, et al. (2002) -0.150 56 150 3 0 -0.075 Shern, et al. (2000) -0.453 91 77 5 0 -0.453 El-Sayeh & Morganti (2006) -0.452 155 155 5 0 -0.452 Test, et al. (1980) -0.321 54 57 3 0 -0.160 Essock, et al. (1995) -0.503 58 50 5 0 -0.503 Test, et al. (1991) -0.680 75 47 5 0 -0.680 Fekete, et al. (1998) -0.534 58 50 3 0 -0.267 Tharyan, et al. (2005) -0.363 214 178 5 0 -0.363 Ford, et al. (1996) -0.375 47 47 3 0 -0.188 Thornley, et al. (2003) -0.229 264 248 5 0 -0.229 Gervey, et al. (1994) -1.584 17 17 3 1 -0.396 Wilson, et al. (1995) -0.511 26 33 5 0 -0.511 Goering, et al. (1988) -0.189 82 82 3 0 -0.094 Wood, et al. (1995) -0.753 32 32 3 0 -0.376 Gold, et al. (2005) -0.678 99 81 5 0 -0.678 Note: Treatments in Assertive Community Treatment were predominantly schizophrenics but included people with other serious mental illness. 16 Exhibit A.5: Citations of Studies Used in the Meta-Analysis Aalto, M., Saksanen, R., Laine, P., Forsstrom, R., Raikaa, M., Kiviluoto, M., et al. (2000). Brief intervention for female heavy drinkers in routine general practice: A 3-year randomized controlled study. Alcoholism: Clinical and Experimental Research, 24(11): 1680-1686. Åberg-Wistedt, A., Cresell, T., Lindberg, Y., Liljenberg, B., & Ösby, U. (1995). Two-year outcome of team-based intensive case management for patients with schizophrenia. Psychiatric Services, 46(12): 1263-1266. Adams, D.R., (1990). An early counseling intervention program for problem drinkers contrasting group and individual delivery formats (group treatment). Doctoral dissertation, University of British Columbia, Canada. Allsop, S., Saunders, B., Phillips, M., & Carr, A. (1997). A trial of relapse prevention with severely dependent male problem drinkers. Addiction, 92(1): 61-74. Anderson, P. & Scott, E. (1992). The effect of general practitioners advice to heavy drinking men. British Journal of Addiction, 87: 891-900. *Anton, R.F., Brady, K.T., & Moak, D.H. (1999). Pharmacotherapy. In P.J. Ott, R.E. Tarter, & R.T. Ammerman (Eds.), Sourcebook on substance abuse: Etiology, epidemiology, assessment, and treatment (pp. 303-314). Boston: Allyn and Bacon. Anton, R.F., Moak, D.H., Waid, L.R., Latham, P.K., Malcolm, R.J. & Dias, J.K. (1999). Naltrexone and cognitive behavioral therapy for the treatment of outpatient alcoholics: Results of a placebo-controlled trial. American Journal of Psychiatry, 156(11): 1758-1764. Anton, R.F., O'Malley, S.S., Ciraulo, D.A., Cisler, R.A., Couper, D., Donovan, D.M., et al. (2006). Combined pharmacotherapies and behavioral interventions for alcohol dependence: The COMBINE study - A randomized controlled trial. Journal of the American Medical Association, 295(17): 2003-2017. Avants, S.K., Margolin, A., Usubiaga, M.H. & Doebrick, C. (2004). Targeting HIV-related outcomes with intravenous drug users maintained on methadone: A randomized clinical trial of a harm reduction group therapy. Journal of Substance Abuse Treatment, 26(2): 67-78. Azrin, N.H. (1976). Improvements in the community-reinforcement approach to alcoholism. Behavior Research and Therapy, 14(5): 339-348. Azrin, N.H., Acierno, R., Kogan, E., Donohue, B., Besalel, V. & McMahon, P.T. (1996). Follow-up results of supportive versus behavioral therapy for illicit drug abuse. Behavioral Research and Therapy, 34(1): 41-46. Azrin, N.H., Donohue, B., Besalel, V.A., Kogan, E.S. & Acierno, R. (1994). Youth drug abuse treatment: A controlled outcome study. Journal of Child and Adolescent Substance Abuse 3(3): 1-16. Babor, T.F., Lauerman, R., Kranzler, H., McRee, B., Korner, P., Wiber, C., et al. (1992). Farmington, USA. In T.F. Babor & M. Grant (Eds.), Project on identification and management of alcohol-related problems report on phase II: A randomized clinical trial of brief interventions in primary care (pp. 191-209). Geneva, Switzerland: World Health Organization. Baker, A., Boggs, T.G., & Lewin, T.J. (2001). Randomized controlled trial of brief cognitive-behavioural interventions among regular users of amphetamine. Addiction, 96(9): 1279-1287. Baker, A., Lee, N.K., Claire, M., Lewin, T.J., Grant, T., Pohlman, S., et al. (2005). Brief cognitive behavioural interventions for regular amphetamine users: A step in the right direction. Addiction, 100(3): 367-378. Barlow, D.H., Cohen, A.S., Waddell, M.R., Vermilyea, B.B., Klosko, J.S., Blanchard, E.B., et al. (1984). Panic and generalized anxiety disorder: Nature and treatment. Behavior Therapy, 15: 431-449. Barlow, D.H., Craske, M.G., Cerny, J.A., & Klosko, J.S. (1989). Behavioral treatment of panic disorder. Behavior Therapy, 20: 261-282. Barlow, D.H., Gorman, J.M., Shear, M.K., & Woods, S.W. (2000). Cognitive-behavioral therapy, imipramine, or their combination for panic disorder: A randomized controlled trial. Journal of the American Medical Association, 283: 2529-2536. Barlow, D.H., Rapee, R.M., & Brown, T.A. (1992). Behavioral treatment of generalized anxiety disorder. Behavior Therapy, 23: 551-570. Barrowclough, C., Haddock, G., Tarrier, N., Lewis, S., Moring, J. & OBrien, R. (2001). Randomized controlled trial of motivational interviewing, cognitive behavior therapy, and family intervention for patients with comorbid schizophrenia and substance use disorders. American Journal of Psychiatry, 158(10): 1706-1713. Beck, A.T., Sokol, L., Clark, D.A., Berchick, R., & Wright, F. (1992). A crossover study of focused cognitive therapy for panic disorder. American Journal of Psychiatry, 149(6): 778-783. Bellack, A.S., Bennett, M.E., Gearon, J.S., Brown, C.H., & Yang, Y. (2006). A randomized clinical trial of a new behavioral treatment for drug abuse in people With severe and persistent mental illness. Archives of General Psychiatry, 63(4): 426-432. Bien, T. H., Miller, W. R., & Boroughs, J. M. (1993). Motivational interviewing with alcohol outpatients. Behavioural and Cognitive Psychotherapy, 21(4): 347-356. Bigelow, D.A., & Young, D.J. (1991). Effectiveness of a case management program. Community Mental Health Journal, 27: 115-123. Bisson, J. & Andrew, M. (2005). Psychological treatment of post-traumatic stress disorder (PTSD). The Cochrane Database of Systematic Reviews, 2: CD003388. Blomhoff, S., Haug, T.T., Hellstrom, K., Holme, I., Humble, M., Madsbu, H.P., et al. (2001). Randomised controlled general practice trial of sertaline, exposure therapy, and combined treatment in generalized social phobia. British Journal of Psychiatry, 179(1): 23-30. *Bond, G.R., Becker, D.R. & Drake, R.E. (2001). Implementing supported employment as an evidence-based practice. Psychiatric Services, 52: 313-322. Bond, G.R., Dietzen, L.L., McGrew, J.H., et al: (1995). Accelerating entry into supported employment for persons with severe psychiatric disabilities. Rehabilitation Psychology, 40: 91-111. *Bond, G.R., Drake, R.E., Mueser, K.T., & Becker, D.R. (1997). An update on supported employment for people with severe mental illness. Psychiatric Services, 48(3): 335346. Bond, G.R., Miller, L.D., Krumwied, R.D., & Ward, R.S. (1988). Assertive case management in three CMHCs: a controlled study. Hospital and Community Psychiatry, 39: 411-418. Bond, G.R., Pensec, M., Dietzen, L.L., McCafferty, D., Giemza, R., & Sipple, H.W. (1991). Intensive case management for frequent users of psychiatric hospitals in a large city: a comparison of team and individual caseloads. Psychosocial Rehabilitation Journal, 15(1): 90-98. Bond, G.R., Witheridge, T.F., Dincin, J., Wasmer, D., Webb, J., & DeGraaf-Kaser, R. (1990). Assertive community treatment for frequent users of psychiatric hospitals in a large city: a controlled study. American Journal of Community Psychology, 18(6): 865-891. Borkovec, T.D. & Costello, E. (1993). Efficacy of applied relaxation and cognitive-behavioral therapy in the treatment of generalized anxiety disorder. Journal of Consulting and Clinical Psychology, 61(4): 611-619. Borkovec, T.D. & Mathews, A.M. (1988). Treatment of nonphobic anxiety disorders: A comparison of nondirective, cognitive, and coping desensitization therapy. Journal of Consulting and Clinical Psychology, 56(6): 877-884. Borkovec, T.D., Mathews, A.M., Chambers, A., Ebrahimi, S., Lytle, R., & Nelson, R. (1987). The effects of relaxation training with cognitive therapy or nondirective therapy and the role of relaxation-induced anxiety in the treatment of generalized anxiety. Journal of Consulting and Clinical Psychology, 55(6): 883-888. Borsari, B. & Carey, K.B. (2000). Effects of a brief motivational intervention with college student drinkers. Journal of Consulting and Clinical Psychology, 68(4): 728-733. Bowers, T.G. & Al-Redha, M.R. (1990). A comparison of outcome with group/marital and standard/individual therapies with alcoholics. Journal of Studies on Alcohol, 51(4): 301-309. Brown, J. M., & Miller, W. R. (1993). Impact of motivational interviewing on participation and outcome in residential alcoholism treatment. Psychology of Addictive Behaviors, 7(4): 211-218. Browne, G., Steiner, M., Roberts, J., Gafni, A., Byrne, C., Dunn, E., et al. (2002). Sertaline and/or interpersonal psychotherapy for patients with dysthymic disorder in primary care: 6-month comparison with longitudinal 2-year follow-up of effectiveness and costs. Journal of Affective Disorders, 68: 317-330. Burgess, S., Geddes, J., Hawton, K., Townsend, E., Jamison, K., & Goodwin, G. (2001). Lithium for maintenance treatment of mood disorders. The Cochrane Database of Systematic Reviews, 3: CD003013. Bush, C.T., Langford, M.W., Rosen, P., & Gott, W. (1990). Operation outreach: Intensive case management for severely psychiatrically disabled adults. Hospital and Community Psychiatry, 41: 647-649. Butler, G., Fennell, M., Robson, P., & Gelder, M. (1991). Comparison of behavior therapy and cognitive behavior therapy in the treatment of generalized anxiety disorder. Journal of Consulting and Clinical Psychology, 59(1): 167-175. Carroll, K.M., Rounsaville, B.J. & Gawin, F.H. (1991). A comparative trial of psychotherapies for ambulatory cocaine abusers: Relapse prevention and interpersonal therapy. American Journal of Drug and Alcohol Abuse, 17(3): 229-247. Carroll, K.M., Rounsaville, B.J., Nich, C., Gordon, L.T., Wirtz, P.W. & Gawin, F.H. (1994). One-Year Follow-up of Psychotherapy and Pharmacotherapy for Cocaine Dependence: Delayed Emergence of Psychotherapy Effects. Archives of General Psychiatry, 51(12): 989-997. Catalano, R.F., Haggerty, K.P., Fleming, C.B., Brewer, D.D., & Gainey, R.R. (2002). Children of substance abusing parents: Current findings from the Focus on Families project. New York: Kluwer Academic Press/Plenum Publishers. Chandler, D., Meisel, J., Hu, T., et al. (1997). A capitated model for a cross-section of severely mentally ill clients: employment outcomes. Community Mental Health Journal, 33: 501-516. 17 Exhibit A.5: Citations of Studies Used in the Meta-Analysis, continued Chandler, D., Meisel, J., McGowen, M., Mintz, J., & Madison, K. (1996). Client outcomes in two model capitated integrated service agencies. Psychiatric Services, 47(1): 175-180. Chaney, E.F., OLeary, M.R. & Marlatt, A.G. (1978). Skill training with alcoholics. Journal of Consulting and Clinical Psychology, 46: 1092-1104. Chick, J., Lloyd, G. & Crombie, E. (1985). Counseling problem drinkers in medical wards: A controlled study. British Medical Journal, 290: 965-967. Chick, J., Ritson, B., Connaughton, J., Stewart, A., & Chick, J. (1988). Advice versus extended treatment for alcoholism: A controlled trial. British Journal of Addiction, 83(2): 159-170. Collins, S.E., Carey, K.B., & Sliwinski, M.J. (2002). Mailed personalized normative feedback as a brief intervention for at-risk college drinkers. Journal of Studies on Alcohol, 63(5): 559-567. Cordioli, A.V., Heldt, E., Bochi, D.B., Margis, R., de Sousa, M.B., Tonello, J.F., et al. (2003). Cognitive-behavioral group therapy in obsessive-compulsive disorder: a randomized clinical trial. Psychotherapy and Psychosomatics, 72(4): 211-6. Cornish, J.W., Metzger, D., Woody, G.E., Wilson, D., McLellan, A.T., Vandergrift, B., et al. (1997). Naltrexone pharmacotherapy for opioid dependent federal probationers. Journal of Substance Abuse Treatment, 14(6): 529-534. Crits-Christoph, P., Siqueland, L., Blaine, J., Frank, A., Luborsky, L. & Onken, L.S. (1999). Psychosocial treatments for cocaine dependence: National institute on drug abuse collaborative cocaine treatment study. Archives of General Psychiatry, 56(6): 493-502. Curtis, J.L., Millman, E.J., Struening, E., et al. (1996). Deaths among former psychiatric inpatients in an outreach case management program. Psychiatric Services, 47(4): 398-402. Dole, V.P., Robinson, J.W., Orraca, J., Towns, E., Searcy, P., & Caine, E. (1969). Methadone treatment of randomly selected criminal addicts. New England Journal of Medicine, 280(25): 1372-1375. Donovan, D.M. & Ito, J.R. (1988). Cognitive behavioral relapse prevention strategies and aftercare in alcoholism rehabilitation. Psychology of Addictive Behaviors, 2(2): 74. Drake, R.E., McHugo, G.J., Bebout, R.R., Becker, D.R., Harris, M., Bond, G.R., et al. (1999). A Randomized Clinical Trial of Supported Employment for Inner-city Patients With Severe Mental Disorders. Archives of General Psychiatry, 56(7): 627-633. Drake, R.E., McHugo, G.J., Becker, D.R., Anthony, W.A., & Clark, R.E. (1996). The New Hampshire study of supported employment for people with severe mental illness. Journal of Consulting and Clinical Psychology, 64(2): 391-39. Drake, R.E., McHugo, G.J., Clark, R.E., et al: (1998a). Assertive community treatment for patients with co-occurring severe mental illness and substance use disorder: a clinical trial. American Journal of Orthopsychiatry, 68(2): 201-215. Drake, R.E., Mercer-McFadden, C., Mueser, K.T., McHugo, G.J. & Bond, G.R. (1998b). Review of integrated mental health and substance abuse treatment for patients With dual disorders. Schizophrenia Bulletin, 24(4): 589-608. *Drake, R.E., Mueser, K.T., Brunette, M.F., & McHugo, G.J. (2004). A review of treatments for people with severe mental illnesses and co-occurring substance use disorders. Psychiatric Rehabilitation Journal, 27(4): 360-374. Drake, R.E., Xie, H., McHugo, G.J. & Green, A.I. (2000). The effects of clozapine on alcohol and drug use disorders among patients With schizophrenia. Schizophrenia Bulletin, 26(2): 441-449. Drake, R.E., Yovetich, N.A., Bebout, R.R., Harris, M. & McHugo, G.J. (1997). Integrated treatment for dually diagnosed homeless Adults. Journal of Nervous and Mental Disease, 185(5): 298-305. Dugas, M.J., Ladouceur, R., Leger, E., Freeston, M.H., Langlois, F., Provencher, M.D., et al. (2003). Group cognitive-behavioral therapy for generalized anxiety disorder: treatment outcome and long-term follow-up. Journal of Consulting and Clinical Psychology, 71(4): 821-825. Durham, R.C., Murphy, T. Allan, T., Richard, K., Treliving, L. R., & Fenton, G.W. (1994). Cognitive therapy, analytic psychotherapy and anxiety management training for generalized anxiety disorder. British Journal of Psychiatry, 165: 315-323. Dyck, D.G., Hendryx, M.S., Short, R.A., Voss, W.D., & McFarlane, W.R. (2002). Service use among patients with schizophrenia in psychoeducational multiple-family group treatment. Psychiatric Services, 53(6): 749-754. El-Sayeh, H.G. & Morganti, C. (2006). Aripiprazole for schizophrenia. The Cochrane Database of Systematic Reviews, 2: CD004578. Elvy, G.A., Wells, J.E. & Baird, K.A. (1988). Attempted referral as intervention for problem drinking in the general hospital. British Journal of Addiction, 83(1): 83-89. Eriksen, L., Björnstad, S., & Götestam, K. G. (1986). Social skills training in groups for alcoholics: One-year treatment outcome for groups and individuals. Addictive Behaviors, 11(3): 309-330. Essock, S.M., & Kontos, N. (1995). Implementing assertive community treatment teams. Psychiatric Services, 46(7): 679-683. Fals-Stewart, W., Birchler, G.R., & Ofarrell, T.J. (1996). Behavioral couples therapy for male substance-abusing patients: Effects on relationship adjustment and drug-using behavior. Journal of Consulting and Clinical Psychology, 64(5): 959-972. Fava, G.A., Rafanelli, C., Grandi, S., Canestrari, R., & Morphy, M.A. (1998) Six-year outcomes for cognitive behavioral treatment of residual symptoms in major depression. The American Journal of Psychiatry, 155(10): 1443-1445. Feeney, G., Young, R., Connor, J., Tucker, J., & McPherson, A. (2002). Cognitive behaviour therapy combined with the relapse-prevention medication acamprosate: are short-term treatment outcomes for alcohol dependence improved? The Australian and New Zealand Journal of Psychiatry, 36(5): 622-628. Fekete, D.M., Bond, G.R., McDonel, E.C., Salyers, M.P., Chen, A., & Miller, L. (1998). Rural assertive community treatment: a field experiment. Psychiatric Rehabilitation Journal, 21(4): 371-379. Ferrell, W. L., & Galassi, J. P. (1981). Assertion training and human relations training in the treatment of chronic alcoholics. International Journal of the Addictions, 16(5): 959-968. Fichter, M.M., & Frick, U. (1993). The key relative’s impact on treatment and course of alcoholism. European Archives of Psychiatry and Clinical Neuroscience, 243(2): 87-94. Fleming, M., Mundt, M.P., French, M.T., Manwell, L.B., Stauffacher, E.A., & Barry, K.L. (2000). Benefit-cost analysis of brief physician advice with problem drinkers in primary care settings. Medical Care, 38(1): 7-18. Ford, R., Ryan, P., & Norton, P., et al. (1996). Does intensive case management work? Clinical, social, and quality of life outcomes from a controlled study. Journal of Mental Health, 5(4): 361-368. Fudala, P.J., Bridge, T.P., Herbert, S., Williford, W.O., Chiang, C.N., Jones, K., et al. (2003). Office-based treatment of opiate addiction with a sublingual-tablet formulation of buprenorphine and naloxone. New England Journal of Medicine, 349(10): 949-958. Gervey, R. & Bedell, J.R. (1994). Supported employment in vocational rehabilitation. In J.R. Bedell (Ed.), Psychological Assessment and Treatment of Persons With Severe Mental Disorders (pp. 151-175). Washington, DC, Taylor & Francis. Goering, P.N., Farkas, M., Wasylenki, D., et al. (1988). Improved functioning for case management clients. Psychosocial Rehabilitation Journal, 12(1): 3-17. Gold, C., Heldal, T.O., Dahle, T., & Wigram, T. (2005). Music therapy for schizophrenia or schizophrenia-like illnesses. The Cochrane Database of Systematic Reviews, 2: CD004025. Graber, R.A. & Willer, W.R. (1988). Abstinence or controlled drinking goals for problem drinkers: A randomized clinical trial. Psychology of Addictive Behaviors, 2(1): 20-33. Gronbladh, L. & Gunne, L. (1989). Methadone-assisted rehabilitation of Swedish heroin addicts. Drug and Alcohol Dependence, 24(1): 31-37. Haddock, G., Barrowclough, C., Tarrier, N., Moring, J., O'Brien, R., Schofield, N., et al. (2006). Cognitive-behavioural therapy and motivational intervention for schizophrenia and substance misuse: 18-month outcomes of a randomized controlled trial. British Journal of Psychiatry, 183: 418-426. Handmaker-Sheehy, N., Miller, W.R., & Manicke, M. (1999). Findings of a pilot study of motivational interviewing with pregnant drinkers. Journal of Studies on Alcohol, 62(2): 285-287. Harris, K.B. & Miller, W.R. (1990). Behavioral self-control training for problem drinkers: Mechanisms of efficacy. Psychology of Addictive Behaviors, 4(2): 82-90. Heather, N. (1996). The public health and brief interventions for excessive alcohol consumption: The British experience. Addictive Behaviors, 21(6): 857. Heather, N., Campion, P.D., Neville, R.G. & Maccabe, D. (1987). Evaluation of a controlled drinking minimal intervention for problem drinkers in general practice (The DRAMS Scheme). Journal of the Royal College of General Practitioners, 37(301): 358-363. Hedberg, A. G., & Campbell, L. M. (1974). A comparison of four behavioral treatment approaches to alcoholism. Journal of Behavior Therapy and Experimental Psychiatry, 5(3/4): 251-256. Hellerstein, D.J., Rosenthal, R.N. & Miner, C.R. (1995). Prospective study of integrated outpatient treatment for substance-abusing schizophrenic patients. American Journal on Addictions, 4(1): 33-42. Hester, R.K. & Delaney, H.D. (1997). Behavioral self-control program for windows: Results of a controlled clinical trial. Journal of Consulting and Clinical Psychology, 65(4): 686-693. Higgins, S.T., Wong, C.J., Badter, G.J., Haug Ogden, D.E. & Dantona, R.L. (2000). Contingent reinforcement increases cocaine abstinence during outpatient treatment and 1 year of follow-up. Journal of Consulting and Clinical Psychology, 68(1): 64-72. 18 Exhibit A.5: Citations of Studies Used in the Meta-Analysis, continued Hoult, J. & Reynolds, I. (1984). Schizophrenia: A comparative trial of community orientated and hospital orientated psychiatric care. Acta Psychiatrica Scandinavica, 69(5): 359-372. Hulse, G.K. & Tait, R.J. (2002). Six-month outcomes associated with brief alcohol intervention for adult in-patients with psychiatric disorders. Drug and Alcohol Review, 21: 105-112. Humphreys, K., Huebsch, P.D., Finney, J.W., & Moos, R.H. (1999). A comparative evaluation of substance abuse treatment: V. Treatment can enhance the effectiveness of self-help groups. Alcoholism: Clinical and Experimental Research, 23(3): 558-563. Hunt, G.M. & Azrin, N.H. (1973). A community reinforcement approach to alcoholism. Behavior Research and Therapy, 11(1): 91-104. James, W., Preston, N.J., Koh, G., Spencer, C., Kisely, S.R., & Castle, D.J. (2004). A group intervention which assists patients with dual diagnosis reduce their drug use: A randomized controlled trial. Psychological Medicine, 34(6): 983-90. Johnson, R.E., Chutuape, M.A., Strain, E.C., Walsh, S.L., Stitzer, M.L. & Bigelow, G.E. (2000). A comparison of levomethadyl acetate, buprenorphine, and methadone for opioid dependence. The New England Journal of Medicine, 343(18): 1290-1297. Johnson, R.E., Eissenberg, T., Stitzer, M.L., Strain, E.C., Liebson, I.A., & Biglow, G.E. (1995). A placebo controlled clinical trial of buprenorphine as a treatment for opioid dependence. Drug and Alcohol Dependence, 40(1): 17-25. Johnson, R.E., Jaffe, J.H., & Fudala, P.J. (1992). A controlled trial of buprenorphine treatment for opioid dependence. Journal of the American Medical Association, 267(20):2750-2755. Jones, S. L., Kanfer, R., & Lanyon, R. I. (1982). Skill training with alcoholics: A clinical extension. Addictive Behaviors, 7(3): 285-290. Joy, C.B., Adams C.E., & Rice K. (2004). Crisis intervention for people with severe mental illnesses. The Cochrane Database of Systematic Reviews, 4: CD001087. Kapczinski, F., Lima, M.S., Souza, J.S., Cunha, A., & Schmitt, R. (2003). Antidepressants for generalized anxiety disorder. The Cochrane Database of Systematic Reviews, 2: CD003592. Kavanagh, D.J., Young, R., White, A., Saunders, J.B., Wallis, J., & Shockley, N. (2004). A brief motivational intervention for substance misuse in recent-onset psychosis. Drug and Alcohol Review, 23(2): 151-155. Kelly, A.B., Halford, W.K., & Young, R.M. (2000). Maritally distressed women with alcohol problems: The impact of short-term alcohol-focused intervention on drinking behavior and marital satisfaction. Addiction, 95(10): 1537-1549. Kivlahan, D.R., Marlatt, G.A., Fromme, K., Coppel, D.B., & Williams, E. (1990). Secondary prevention with college drinkers: Evaluation of an alcohol skills training program. Journal of Consulting and Clinical Psychology, 58(6): 805-810. Kristenson, H., Ohlin, H., Hulten-Nosslin, M.B., Trell, E. & Hood, B. (1983). Identification and intervention of heavy drinking in middle-aged men: Results and follow-up of 2460 months of long-term study with randomized controls. Alcoholism: Clinical and Experimental Research, 7(2): 203-209. *Kreek, M.J. & Vocci, F.J. (2002). History and current status of opioid maintenance treatments: Blending conference session. Journal of Substance Abuse Treatment, 23(2): 93-105. Kuchipudi, V., Hobein, K., Flickinger, A. & Iber, F.L. (1990). Failure of a 2-hour motivational intervention to alter recurrent drinking behavior in alcoholics with gastrointestinal disease. Journal of Studies on Alcohol, 51(4): 356-360. Ladouceur, R., Dugas, M., Freeston, M.H., Leger, E., Gagnon, F., & Thibodeau, N. (2000). Efficacy of a cognitive-behavioral treatment for generalized anxiety disorder: Evaluation in a controlled clinical trial. Journal of Consulting and Clinical Psychology, 68(6): 957-964. Larimer, M.E., Turner, A.P., Anderson, B.K., Fader, J.S., Kilmer, J.R., Palmer, R.S., et al (2001). Evaluating a brief alcohol intervention with fraternities. Journal of Studies on Alcohol, 62(3): 370-380. *Latimer, E.A. (1999). Economic impacts of assertive community treatment: A review of the literature. Canadian Journal of Psychiatry, 44(5): 443-454. Lehman, A.F., Dixon, L.B., Kernan, E., et al. (1997). A randomized trial of assertive community treatment for homeless persons with severe mental illness. Archives of General Psychiatry, 54(11): 1038. Lehman, A.F., Postrado, L.T., McNary, S.W., & Goldman, H.H. (1994). Continuity of care and client outcomes in the Robert Wood Johnson Foundation program on chronic mental illness. Milbank Quarterly, 72: 105-122. Lewis, R., Bagnall, A.M., & Leitner, M. (2005). Sertindole for schizophrenia. The Cochrane Database of Systematic Reviews, 3: CD001715. Lhuintre, J.P., Daoust, M., Moore, N. D., Chretien, P., Saligaut, C., Tran, G., et al. (1985). Ability of calcium bis acetyl homotaurine, a GABA agonist, to prevent relapse in weaned alcoholics. Lancet, 1(8436): 1014-1016. Lhuintre, J.P., Moore, N., Tran, G., Steru, L., Langrenon, S., Daoust, M., et al. (1990). Acamprosate appears to decrease alcohol intake in weaned alcoholics. Alcohol and Alcoholism, 25(6): 613-622. Lima, M.S., Moncrieff, J., & Soares, B.G.O. (2006). Drugs versus placebo for dysthymia. The Cochrane Database of Systematic Reviews, 2: CD001130. Lindsay, W.R., Gamsu, C.V., McLaughlin, E., Hood, E.M., & Espie, C.A. (1987). A controlled trial of treatments for generalized anxiety. British Journal of Clinical Psychology, 26(1): 3-15. Linehan, M.M., Schmidt, H., Dimeoff, L.A., Craft, J.C., Kanter, J. & Comtois, K.A. (1999). Dialectical behavior therapy for patients with borderline personality disorder and drug-dependence. American Journal on Addictions, 8(4): 279-292. Ling, W., Charuvastra, C., et al. (1998). Buprenorphine maintenance treatment of opiate dependence: A multicenter, randomized clinical trial. Addiction, 93(4): 475. Macias, C., Kinney, R., Farley, O.W., Jackson, R., & Vos, B. (1994). The role of case management within a community support system: partnership with psychosocial rehabilitation. Community Mental Health Journal, 30: 323-339. Macritchie, K., Geddes, J.R., Scott, J., Haslam, D., de Lima, M., Goodwin, G. (2003). Valproate for acute mood episodes in bipolar disorder. The Cochrane Database of Systematic Reviews, 1: CD004052. Maheswaran, R., Beevers, M. & Beevers, D.G. (1992). Effectiveness of advice to reduce alcohol consumption in hypertensive patients. Hypertension, 19(1): 79-84. Mallams, J. H., Godley, M. D., Hall, G. M., & Meyers, R. J. (1982). A social-systems approach to resocializing alcoholics in the community. Journal of Studies on Alcohol, 43(11): 1115-1123. Manwell, L.B., Fleming, M.F., Mundt, M.P., Staffacher, E.A., & Barry, K.L. (2000). Treatment of problem alcohol use in women of childbearing age: Results of a brief intervention trial. Alcoholism: Clinical and Experimental Research, 24(10): 1517-1524. Margolin, A., Avants, S.K., Warburton, L.A., Hawkins, K.A. & Shi, J. (2003). A randomized clinical trial of a manual-guided risk reduction intervention for HIV-positive injection drug users. Health Psychology, 22(2): 223-228. Marijuana Treatment Project Research Group. (2004). Brief treatments for cannabis dependence: Findings from a randomized multisite trial. Journal of Consulting and Clinical Psychology, 72(3): 455-466. Marks, I.M., Swinson, R.P., Basaglu, M., et al. (1993). Alprazolam and exposure alone and combined in panic disorder with agoraphobia: A controlled study in London and Toronto. British Journal of Psychiatry, 162: 776-787. Marlatt, G.A., Baer, J.S., Kivlahan, D.R., Dimeff, L.A., Larimer, M.E., Quigley, L.A., et al. (1998). Screening and brief intervention for high-risk college student drinkers: Results from a 2-year follow-up assessment. Journal of Consulting and Clinical Psychology, 66(4): 604-615. Marques, L.O., Lima, M.S., & Soares, B.G.O. (2004). Trifluoperazine for schizophrenia. The Cochrane Database of Systematic Reviews, 1: CD003545. Mason, B. J., Ritvo, E. C., Morgan, R. O., Salvato, F. R., Goldberg, G., Welch, B., et al. (1994). A double-blind, placebo-controlled pilot study to evaluate the efficacy and safety of oral nalmefene HCl for alcohol dependence. Alcoholism: Clinical and Experimental Research, 18(5): 1162-1167. Mason, B.J., Salvato, F.R., Williams, L.D., Rivto, E.C., & Culter, R.B. (1999). A double-blind, placebo controlled study of oral nalmefene for alcohol dependence. Archives of General Psychiatry, 56(8): 719-724. McCrady, B., Epsteien, E.E., & Hirsch, L.S. (1999). Maintaining change after conjoint behavioral alcohol treatment for men: Outcomes at 6 months. Addiction, 94(9): 13811396. McFarlane, W.R. (2002). Empirical studies of outcome in multifamily groups. In W.R. McFarlane (Ed.), Multifamily groups in the treatment of severe psychiatric disorders (pp. 49-70). New York: The Guilford Press. McFarlane, W.R., Dushay, R.A., Deakins, S.M., et al. (2000). Employment outcomes in family-aided assertive community treatment. American Journal of Orthopsychiatry, 70(2): 203-214. McFarlane, W.R., Lukens, E., Link, B., Dushay, R., Deakins, S.A. & Newmark, M. (1995). Multiple-family groups and psychoeducation in the treatment of schizophrenia. Archives of General Psychiatry, 52(8): 679-687. Milby, J.B., Schumacher, J.E., Raczynski, J.M., Caldwell, E., Engle, M., Michael, M., et al. (1996). Sufficient conditions for effective treatment of substance abusing homeless persons. Drug and Alcohol Dependence, 43(1): 39-47. Miller, W. R., Benefield, R. G., & Tonigan, J. S. (1993). Enhancing motivation for change in problem drinking: A controlled comparison of two therapist styles. Journal of Consulting and Clinical Psychology, 61(3): 455-461. 19 Exhibit A.5: Citations of Studies Used in the Meta-Analysis, continued Miller, W.R., Gribskov, C.J. & Mortell, R.L. (1981). Effectiveness of a self-control manual for problem drinkers with and without therapist contact. International Journal of the Addictions, 16(7): 1247-1254. Miller, W.R., Meyers, R.J., Tonigan, J.S., & Grant, K.A. (2001). Community reinforcement and traditional approaches: Findings of a controlled trial. In R.J. Meyers & W.R. Miller (Eds.), A community reinforcement approach to addition treatment (pp. 79-103). Cambridge UK: Cambridge University Press. Miller, W.R., Taylor, C.A. & West, J.C. (1980). Focused versus broad-spectrum behavior therapy for problem drinkers. Journal of Consulting and Clinical Psychology, 48(5): 590-601. *Miller, W. & Wilbourne, P. (2002). Mesa Grande: A methodological analysis of clinical trials of treatments for alcohol use disorders. Addiction, 97(2): 265-277. Moncrieff, J., Wessely, S., & Hardy, R. (2004). Active placebos versus antidepressants for depression. The Cochrane Database of Systematic Reviews, 1: CD003012. Monti, P. M., Abrams, D. B., Binkoff, J. A., Zwick, W. R., Liepman, M. R., Nirenberg, T. D., et al. (1990). Communication skills training, communication skills training with family and cognitive behavioral mood management training for alcoholics. Journal of Studies on Alcohol, 51(3): 263-270. Monti, P. M., Rohsenow, D. J., Rubonis, A. V., Niaura, R. S., Sirota, A. D., Colby, S. M., et al. (1993). Cue exposure with coping skills treatment for male alcoholics: A preliminary investigation. Journal of Consulting and Clinical Psychology, 61(6): 1011-1019. Morse, G.A., Calsyn, R.J., Klinkenberg, W.D., et al. (1997). An experimental comparison of three types of case management for homeless mentally ill persons. Psychiatric Services, 48(4): 497-503. Mörtberg, E, Karlsson, A., Fyring, C. & Sundin, Ö. (2005). Intensive cognitive-behavioral group treatment (CBGT) of social phobia: A randomized controlled study. Journal of Anxiety Disorders, in press. Mota Neto, J.I.S., Lima, M.S., & Soares, B.G.O. (2002). Amisulpride for schizophrenia. The Cochrane Database of Systematic Reviews, 2: CD001357. *Mueser, K.T., Bond, G.R., Drake, R.E., & Resnick, S.G. (1998). Models of community care for severe mental Illness: A review of research on case management. Schizophrenia Bulletin, 24(1): 37-74. Murphy, J.G., Duchnick, J.J., Vuchinich, R.E., Davison, J.W., Karg, R.S., Olson, A.M., et al. (2001). Relative efficacy of a brief motivational intervention for college student drinkers. Psychology of Addictive Behaviors, 15(4): 373-379. Neighbors, C., Larimer, M.E., & Weis, M.A. (2004). Targeting misperceptions of descriptive drinking norms: Efficacy of a computer-delivered personalized normative feedback intervention. Journal of Consulting and Clinical Psychology, 72(3): 434-447. Nelson, J.E., & Howell, R.J. (1983). Assertiveness training using rehearsal and modeling with male alcoholics. American Journal of Drug and Alcohol Abuse, 9(3): 309-323. Newman, R.G. & Whitehill, W.B. (1979). Double-blind comparison of methadone and placebo maintenance treatments of narcotic addicts in Hong Kong. Lancet, 2: 485. Nilssen, O. (1991). The Tromso study: identification of and a controlled intervention on a population of early-stage risk drinkers. Preventive Medicine, 20(4): 518-528. Obolensky, M. R. (1984). Recidivism among male DWI first offenders and intervention. Doctoral dissertation, University of Rhode Island (UMI No. 8508028). O’Connell, J.M. (1987). Effectiveness of an alcohol relapse prevention program. Doctoral dissertation, Fordham University (UMI No. 8725685). Oei, T. P. S., & Jackson, P. R. (1980). Long-term effects of group and individual social skills training with alcoholics. Addictive Behaviors, 5(2): 129-136. Oei, T. P. S., & Jackson, P. R. (1982). Social skills and cognitive behavioral approaches to the treatment of problem drinking. Journal of Studies on Alcohol, 43(5): 532-547. O’Farrell, T.J. (1993). Couples relapse prevention sessions after a behavioral marital therapy couples group program. In T.J. O'Farrell (Ed.), Treating alcohol problems: Marital and family interventions (pp. 305-326). New York: Guilford Press. O’Malley, S.S., Jaffe, A.J., Chang, G., Schottenfeld, R.S., Meyer, R.E. & Rounsaville, B. (1992). Naltrexone and coping skills therapy for alcohol dependence: A controlled study. Archives of General Psychiatry, 49(11): 881-887. Ouimette, P.C., Finney, J.W. & Moos, R.H. (1997). Twelve-step and cognitive-behavioral treatment for substance abuse: A comparison of treatment effectiveness. Journal of Consulting and Clinical Psychology, 65(2): 230-240. Paille, F. M., Guelfi, J. D., Perkins, A. C., Royer, R. J., Steru, L., & Parot, P. (1995). Doubleblind randomized multicentre trial of acamprosate in maintaining abstinence from alcohol. Alcohol and Alcoholism, 30(2): 239-247. Persson, J. & Magnusson, P. H. (1989). Early intervention in patients with excessive consumption of alcohol: A controlled study. Alcohol, 6(5): 403. Petry, N.M. & Martin, B. (2002). Low-cost contingency management for treating cocaine- and opioid-abusing methadone patients. Journal of Consulting and Clinical Psychology, 70(2): 398-405. Petry, N.M., Martin, B., Cooney, J.L. & Kranzler, H.R. (2000). Give them prizes, and they will come: Contingency management for treatment of alcohol dependence. Journal of Consulting and Clinical Psychology, 68(2): 250-257. Piotrowski, N.A., Tusel, D.J., Sees, K.L., Reilly, P.M., Banys, P., Meek, P., et al. (1999). Contingency contracting with monetary reinforcers for abstinence from multiple drugs in a methadone program. Experimental and Clinical Psychopharmacology, 7: 399-411. Pittler, M.H. & Ernst, E. (2003). Kava extract versus placebo for treating anxiety. The Cochrane Database of Systematic Reviews, 1: CD003383. Quinlivan, R., Hough, R., Crowell, A., Beach, C., Hofstetter, R., & Kenworthy, K. (1995). Service utilization and costs of care for severely mentally ill clients in an intensive case management program. Psychiatric Services, 46(4): 365-371. Rawson, R.A., Shoptaw, S.J., Obert, J.L., McCann, M.J., L Hasson, A., Marinelli-Casey, P.J., et al. (1995). An intensive outpatient approach for cocaine abuse treatment: The matrix model. Journal of Substance Abuse Treatment, 12(2): 117-127. Rendell, J.M., Gijsman, H.J., Keck, P., Goodwin, G.M., & Geddes, J.R. (2003). Olanzapine alone or in combination for acute mania. The Cochrane Database of Systematic Reviews, 1: CD004040. Reynolds, C.F., Frank, E., Perel, J.M., Imber, S.D., Cornes, C., Miller, et al. (2006). Nortriptyliine and interpersonal psychotherapy as maintenance therapies for recurrent major depression: A randomized controlled trial in patients older than 59 years. New England Journal of Medicine 354: 1130-1138 Reynolds, K. D., Coombs, D. W., Lowe, J. B., Peterson, P. L., & Gayoso, E. (1995). Evaluation of a self-help program to reduce alcohol consumption among pregnant women. International Journal of the Addictions, 30(4): 427-443. Richmond, R., Heather, N., Wodak, A., Kehoe, L., & Webster, I. (1995). Controlled evaluation of a general practice-based brief intervention for excessive drinking. Addiction, 90(1): 119-132. Rohsenow, D. J., Smith, R. E., & Johnson, S. (1985). Stress management training as a prevention program for heavy social drinkers: Cognitive, affect, drinking, and individual differences. Addictive Behaviors, 10(1): 45-54. Romelsjö, A., Andersson, L, Barrner, H., Borg, S., Granstrand, C., Hultman, O., et al. (1989). A randomized study of secondary prevention of early stage problem drinkers in primary health care. British Journal of Addiction, 84(11): 1319-1327. Sanchez-Craig, M., Davila, R., & Cooper, G. (1996). A self-help approach for high-risk drinking: Effect of an initial assessment. Journal of Consulting and Clinical Psychology, 64(4): 694-700. Sanchez-Craig, M., Spivak, K. & Davila, R. (1991). Superior outcome of females over males after brief treatment for reduction of heavy drinking: Replication and report of therapist effects. British Journal of Addiction, 86(7): 867-876. Sannibale, C. (1989). A prospective study of treatment outcome with a group of male problem drinkers. Journal of Studies on Alcohol, 50(3): 236-244. Sass, H., Soyka, M., Mann, K., Zieglgänsberger, W. (1996). Relapse prevention by acamprosate: Results from a placebo-controlled study on alcohol dependence. Archives of General Psychiatry, 53(8): 673-680. Schottenfeld, R.S., Pakes, J.R., Oliveto, A., Ziedonis, D., & Kosten, T.R. (1997). Buprenorphine vs. methadone maintenance treatment for concurrent opioid dependence and cocaine abuse. Archives of General Psychiatry, 54(8):713-20. Scott, E. & Anderson, P. (1990). Randomized controlled trial of general practitioner intervention in women with excessive alcohol consumption. Drug and Alcohol Review, 10: 313-321. Shea, M.T., Elkin, I., Imber, S.D., Sotsky, S.M., Watkins, J.T., Collins, J.F., et al. (1992) Course of depressive symptoms over follow-up: Findings from the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Archives of General Psychiatry, 49:782-787. Shern, D.L., Tsemberis, S., Anthony, W., Lovell, A.M., Richmond, L., Felton, C.J., et al. (2000). Serving street-dwelling individuals with psychiatric disabilities: Outcomes of a psychiatric rehabilitation clinical trial. American Journal of Public Health, 90(12): 1873-1878. Silverman, K., Higgins, S.T., Brooner, R.K., Montoya, I.D., Cone, E.J. & Schuster, C.R. (1996). Sustained cocaine abstinence in methadone maintenance patients through voucher-based reinforcement therapy. Archives of General Psychiatry, 53(5): 409-415. Silverman, K., Wong, C.J., Umbricht-Schneiter, A., Montoya, I.D., Schuster, C.R. & Preston, K.L. (1998). Broad beneficial effects of cocaine abstinence reinforcement among methadone patients. Journal of Consulting and Clinical Psychology, 66(5): 811-824. Simons, A.D., Murphy, G.E., Levine, J.L., & Wetzel, R.D. (1986). Cognitive therapy and pharmacotherapy for depression: Sustained improvement over one year. Archives of General Psychiatry, 43(1): 43-48. Sisson, R.W. & Azrin, N.H. (1986). Family-member involvement to initiate and promote treatment of problem drinkers. Journal of Behavior Therapy and Experimental Psychiatry, 17(1): 15-21. 20 Exhibit A.5: Citations of Studies Used in the Meta-Analysis, continued Smith, J.E., Meyers, R.J. & Delaney, H.D. (1998). The community reinforcement approach with homeless alcohol-dependent individuals. Journal of Consulting and Clinical Psychology, 66: 541-548. Smith, T.L., Volpe, F.R., Hashima, J.N., & Schukit, M.A. (1999). Impact of a stimulant-focused enhanced program on the outcome of alcohol- and/or stimulant-dependent men. Alcoholism: Clinical and Experimental Research, 23(11): 1772-1779. Stein, D.J., Ipser, J.C., & Seedat, S. (2006). Pharmacotherapy for post traumatic stress disorder (PTSD). The Cochrane Database of Systematic Reviews, 1: CD002795. Stein, D.J., Ipser, J.C., & van Balkom, A.J. (2000). Pharmacotherapy for social anxiety disorder. The Cochrane Database of Systematic Reviews, 4: CD001206. Stephens, R.S., Roffman, R.A., & Curtin, L. (2000). Comparison of extended versus brief treatments for marijuana use. Journal of Consulting and Clinical Psychology, 68(5): 898–908. Strain, E.C., Stitzer, M.L., Liebson, I.A., & Bigelow, G.E. (1993). Dose-response effects of methadone in the treatment of opioid dependence. Annals of Internal Medicine, 119: 23-27. Test, M.A. & Stein, L.I. (1980). Alternative to mental hospital treatment: III. Social cost. Archives of General Psychiatry, 37(4): 409-412. Test, M.A., Knoedler, W.H., Allness, D.J., et al. (1991). Long-term community care through an assertive continuous treatment team. In C.T. Schultz (Ed.), Advances in neuropsychiatry and psychopharmacology: schizophrenia research, Vol. 1 (pp.239-246). New York: Raven. Tharyan, P. & Adams, C.E. (2005). Electroconvulsive therapy for schizophrenia. The Cochrane Database of Systematic Reviews, 2: CD000076. Thornley, B., Rathbone, J., Adams, C.E., & Awad, G. (2003). Chlorpromazine versus placebo for schizophrenia. The Cochrane Database of Systematic Reviews, 2: CD000284. Tomson, Y., Romelsjo, A. & Aberg, H. (1998). Excessive drinking--brief intervention by a primary health care nurse. Scandinavian Journal of Primary Health Care, 16(3): 188-192. Tuunainen, A., Kripke, D.F., & Endo, T. (2004). Light therapy for non-seasonal depression. The Cochrane Database of Systematic Reviews, 2: CD004050. Vanichseni, S., Wongsuwan, B., Choopanya, K., & Wongpanich, K. (1991). A controlled trial of methadone maintenance in a population of intravenous drug users in Bangkok: Implications for prevention of HIV. International Journal of the Addictions, 26(12): 1313-20. Volpicelli, J.R., Alterman, A.I., Hayashida, M., & OBrien, C.P. (1992). Naltrexone in the treatment of alcohol dependence. Archives of General Psychiatry, 49(11): 876-880. Wallace, P., Cutler, S. & Haines, A. (1988). Randomised controlled trial of general practitioner intervention in patients with excessive alcohol consumption. British Medical Journal, 297(6649): 663-668. Ward, E., King, M., Lloyd, M., Bower, P., Sibbald, B., Farrelly, S., et al. (2000). Randomised controlled trial of non-directive counseling, cognitive-behaviour therapy, and usual general practitioner care for patients with depression I: Clinical effectiveness. British Medical Journal, 321: 1383-1388. Weiss, R.D., Griffin, M.L., Greenfield, S.F., Najavits, L.M., Wyner, D., Soto, J.A., et al. (2000). Group therapy for patients with bipolar disorder and substance dependence: Results of a pilot study. Journal of Clinical Psychiatry, 61(5): 361-367. White, J., Keenan, M., & Brooks, N. (1992). Stress control: A controlled comparative investigation of large group therapy for generalized anxiety disorder. Behavioural Psychotherapy, 20: 97-114. Whitworth, A. B., Fischer, F., Lesch, O. M., Nimmerrichter, A., Oberbauer, H., Platz, T., et al. (1996). Comparison of acamprosate and placebo in long-term treatment of alcohol dependence. Lancet, 347(9013): 1438-1442. Wijkstra, J., Lijmer, J., Balk, F., Geddes, J., & Nolen, W.A. (2005). Pharmacological treatment for psychotic depression. The Cochrane Database of Systematic Reviews, 4: CD004044. Wilson, D., Tien, G., & Eaves, D. (1995). Increasing the community tenure of mentally disordered offenders: An assertive case management program. International Journal of Law and Psychiatry, 18: 61-69. Winters, J., OFarrell, T.J., Fals-Stewart, W., Birchler, G.R. & Kelley, M.L. (2002). Behavioral couples therapy for female substance-abusing patients: Effects on substance use and relationship adjustment. Journal of Consulting and Clinical Psychology, 70(2): 344-355. Wood, K., & Anderson, J. (1995). The effect on hospital admissions of psychiatric case management involving general practitioners: preliminary results. Australian and New Zealand Journal of Psychiatry, 29(2): 223-229. Woody, G.E., McLellan, A.T., Luborsky, L. & OBrien, C.P. (1995). Psychotherapy in community methadone programs: A validation study. American Journal of Psychiatry, 152(9): 1302-1308. *Ziguras, S.J. & Stuart, G.W. (2000). A meta-analysis of the effectiveness of mental health case management over 20 years. Psychiatric Services, 51(11): 1410-1421. References marked with an asterisk (*) were not meta-analyzed but provided references to many studies used in this analysis. 21 Appendix B: Methods and Parameters to Model the Benefits and Costs of Evidence-Based Treatment To estimate the benefits and costs of evidence-based treatment (EBT) for people with alcohol, drug, and mental illness disorders, we developed an economic model. Appendix B describes the technical structure of the model as well as the data used with the model to produce the estimates for this study. The basic model takes the following form (each of the elements in the model is discussed in greater detail in this Appendix): ( ) 3 Y MTE ∗ E + HP + HC + D + C − PC ty ty ty ty ty ty ty ∗ Nt y + disrate ( 1 ) t =1 y B(1): T = ∑∑ B(2): N t = WAPOP ∗ 12MOPREVt ∗ (1 - TX t ) ∗ POTENTIALt Equation B(1) is the basic model developed to estimate the total net present value of EBT, notated as T. We estimate three prototype EBT programs: one for people with alcohol disorders, one for people with drug disorders, and one for people with mental health disorders (we account for “comorbidities” in our prevalence estimates, as discussed below). In the equation, the three prototype EBT programs are notated with a t. For each program, we estimate a series of annual cash flows that run from y to Y, where y represents the years following participation in an EBT. The subscript y equals 1 during the year that a person is administered an EBT and ends in Y—the maximum number of years over which effects are estimated. The model computes the marginal treatment effect, MTEty, for each of the three prototype EBTs in each year following treatment (the computation of MTEty is described later in this Appendix). As we discuss, we project these treatment effects to decay over time. The marginal effects are multiplied by the sum of five sources of benefits estimated in this study. These are: the value of economic production due to improvements in disorder-caused morbidity, Ety; the value of household production due to improvements in disorder-caused morbidity, HPty; the value of reduced health care and other costs due to reduced disorder rates, HCty; the value of economic and household production due to reductions in disorder-caused mortality, Dty; and the value of avoided disorder-caused crime, Cty. Each of these factors is described in this Appendix. Annual program costs, PCty, are subtracted from the annual benefits. The annual net cash flows are then discounted to present value with a discount rate, disrate. The present-valued dollars are thus based in the year in which the initial investment in an EBT would be made. A benefit-to-cost ratio, BCt, is computed for each prototype EBT by rearranging equation B(1): ( Y MTE ∗ E + HP + HC + D + C ty ty ty ty ty ty y ( 1 disrate ) + B(3): BCt = y Y PCty y y (1 + disrate) ∑ ∑ 22 ) Additionally, an internal rate of return can be computed for each EBT by using Microsoft Excel’s IRR function for the annual cash flows, CFty, given by: ( ) B(4): CFty = MTEty ∗ Ety + HPty + HCty + Dty + Cty − PCty Finally, to calculate the total net benefits for Washington, equation B(1) multiplies the per-person net present value for each prototype EBT by the number of people in Washington estimated to be in need of treatment, Nt. The computation of variable Nt is given in equation B(2) and is the product of the total number of people currently living in Washington in the age groups selected to be eligible for an EBT, WAPOP; times the 12-month prevalence of the disorder in the population, 12MOPREVt; times one minus the percent of people already treated with an EBT, TXt; times an assumption about the percentage of the remaining people in Washington with the disorder who might realistically be treated, POTENTIALt. Exhibits B.1, B.2, and B.3 display a list of the parameters used in our analytical approach; the following description refers to the information in those Exhibits. B1. General Model Parameters The model uses a number of parameters pertinent to all three evidence-based prototypes estimated in this study. Exhibit B.1 lists these parameters. The range of discount rates used in this study is shown on line 1 of Exhibit B.1. The high end of the range is a 7 percent real discount rate. This discount rate reflects the rate that has been recommended by the federal Office of Management and Budget.39 The low end of the range is a 3 percent real discount rate used by the Congressional Budget Office in a variety of analyses including its projections of the long-term financial position of Social Security.40 Our study uses a medium discount of 5 percent, the difference between the high and low rates.41 Some of the estimated benefits in this study reflect the effect of improvements in the Diagnostic and Statistical Manual of Mental Disorders (DSM) alcohol, drug, and mental illness disorders on economic outcomes. Key parameters in these projections are the level of earnings and the long-term expected rate of real (inflation-adjusted) growth in earnings. The level of earnings by age is taken from cross-sectional data from the 2005 March Supplement to the Current Population Survey (CPS), with data on earnings during 2004. The earnings are those for people with education levels between 9th grade through some college. The number of non-earners is included in the estimates so that the average earning level reflects earnings of all people at each age (earners and non42 earners). The cross-sectional estimates from the CPS are shown on Exhibit B.2 by age of person. 39 Office of Management and Budget, Circular A-94 (revised 1992). See Congressional Budget Office report: http://www.cbo.gov/ftpdocs/ 72xx/doc7289/06-14-LongTermProjections.pdf 41 For a general discussion of discount rates for applied public benefitcost analyses, see: C. Bazelon, and K. Smetters. (1999). Discounting inside the Washington D.C. Beltway. Journal of Economic Perspectives, 13(4): 213-28. See also: H. Kohyama. (2006). Selecting discount rates for budgetary purposes, Briefing Paper No. 29. http://www.law.harvard.edu/faculty/hjackson/DiscountRates_29.pdf 42 Current Population Survey data downloaded from the US Census Bureau site with the DataFerrett extraction utility: http://www.bls.census.gov/cps/cpsmain.htm 40 Exhibit B.1 The Benefits and Costs of Evidence-Based Treatment: General Model Parameters Parameter Line number 1 2 3 4 5 6 7 8 9 10 11 High Discount Rate Real annual rate of growth in earnings Fringe benefit multiple for earnings Tax rate for earnings Real annual rate of growth in health care costs Year of dollars for the analysis Year of dollars for the Current Population Survey used in the study Real cost of capital (used in the crime sub-model) Hours per week for household production, per person Dollars per hour for household production Year of dollars for the household production .070 .023 .044 - Medium .050 .013 1.423 .316 .034 2006 2004 .025 19.5 $10.08 2004 Low .030 .003 .024 - Line 2 of Exhibit B.1 shows the long-run expected growth rate in real earnings. The estimate for the medium case is taken from the Congressional Budget Office (CBO) analysis of long43 run Social Security. We included the higher rate of growth and the lower rate of growth in our sensitivity analyses, described below. household maintenance. These estimates are quite close to 49 the 21.4 hours per week calculated by Douglass et al. The average shadow wage rate, shown on line 10 of Exhibit B.1, for these three household services was taken from United State Bureau of Labor Statistics data on average wage rates 50 in Washington in 2004 for each service. Line 3 of Exhibit B.1 shows an estimate for the average fringe benefit rate for earnings. This estimate is from the Employment Cost Index as computed by the United States Bureau of Labor Statistics.44 Line 4 shows the average tax 45 rate applied to earnings. B2. Program Effectiveness Parameters Line 5 shows our assumed rate of growth in real health care costs. The medium case is taken from the current forecast for 2006 to 2015 from the US Department of Health and Human Services.46 For high and low cases, we assumed one percentage point above and below the medium rate. Line 6 of Exhibit B.1 indicates the year chosen for the overall analysis. All costs are converted to this year’s dollars with the inflation index shown in Exhibit B.2. The inflation index is taken from the Washington State Economic and Revenue Forecast Council, the official forecasting agency for Washington State government. The index is the chain-weight 47 implicit price deflator for personal consumption expenditures. Lines 9 through 11 of Exhibit B.1 indicate the estimates used to monetize the value of household production, a common procedure in cost-of-illness studies.48 We estimate 19.5 hours per week for household production. This estimate is based on an assumed 1.5 hours per day for housekeeping services, 1.0 hours per day for food preparation, and 2.0 hours per week for 43 See Congressional Budget Office data for the June 2006 report, Table W-5, at: http://www.cbo.gov/ftpdocs/72xx/doc7289/06-14SupplementalData.xls 44 United State Bureau of Labor Statistics, Employment Cost Index, March 14, 2006 release, data for December 2005: http://www.bls.gov/news.release/ecec.toc.htm 45 Tax Foundation Special Report, April 2006, Table 1, page 4: http://www.taxfoundation.org/files/sr140.pdf 46 US Department of Health and Human Services, Office of the Actuary in the Centers for Medicare & Medicaid Services. National Health Care Expenditures Projections: 2005-2015. http://www.cms.hhs.gov/National HealthExpendData/downloads/proj2005.pdf 47 Washington State Economic and Revenue Forecast Council: http://www.erfc.wa.gov/pubs/feb06pub.pdf 48 See, for example, W. Max, D. Rice, H. Sung, and M. Michel. (2004). Valuing human life: Estimating the present value of lifetime earnings, 2000. Center for Tobacco Control Research and Education. Economic Studies and Related Methods. Paper PVLE2000. http://repositories.cdlib.org/cgi/viewcontent.cgi?article=1049&context=ctcre The first section of Exhibit B.3 lists the estimates we produced for the average effectiveness of EBT for persons with serious alcohol, illicit drug, and mental illness disorders. These results, shown on lines 1 through 3 of Exhibit B.3, are the meta-analytic results discussed in Appendix B. Line 1 is the unadjusted weighted effect size of EBT for each of the three types of disorders. Line 2 is the associated standard error from the meta-analysis. Line 3 is the adjusted effect size after applying the Institute rules, described in Appendix A3, to account for the methodological quality of the evidence, outcome measurement relevance, and the degree of researcher involvement. Line 4 is an estimated standard error for the Institute-adjusted effect size. A standard error is computed for this parameter because it is used in sensitivity analyses (discussed in Appendix B12). Since we cannot estimate a standard error directly for the Institute-adjusted effect size, we employ a simple procedure to calculate a standard error for the Instituteadjusted effect size: B(5): AdjustedSE = AdjustedES ⎛ UnadjustedES ⎞ ⎟⎟ ⎜⎜ ⎝ UnadjustedSE ⎠ In this formula, we compute an estimated standard error for the Institute-adjusted effect size by dividing the Instituteadjusted effect size by the t-statistic for the unadjusted effect size (from the meta-analyses). This means we are assuming the same level of statistical significance for our adjusted effect size as that obtained from the unadjusted meta-analysis as described in Appendix A. 49 J. Douglass, G. Kenney, and T. Miller. (1990). Which estimates of household production are best? Journal of Forensic Economics, 4(1): 25-45. 50 US Bureau of Labor Statistics, November 2004 Washington Wage Data from: http://www.bls.gov/oes/current/oes_wa.htm#b39-0000 23 Exhibit B.2 The Benefits and Costs of Evidence-Based Treatment: Annual Data Series 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 Inflation Index 0.521 0.567 0.598 0.624 0.648 0.669 0.686 0.709 0.737 0.770 0.805 0.834 0.858 0.878 0.896 0.916 0.935 0.951 0.960 0.976 1.000 1.021 1.035 1.055 1.082 1.113 1.137 Age 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 Washington Population, 2006 90,790 90,133 92,505 92,067 91,768 92,829 91,519 90,951 89,859 84,783 83,095 82,259 81,105 83,687 79,971 82,154 88,366 94,869 98,231 90,956 88,589 86,958 89,355 97,011 97,353 98,843 99,616 100,711 102,892 97,464 100,774 98,177 96,511 97,627 92,805 92,303 87,140 84,198 85,219 79,737 81,019 79,625 60,323 60,948 59,924 58,056 50,275 49,947 49,296 48,336 47,086 45,567 43,810 41,846 39,708 37,434 35,059 32,620 30,153 27,690 25,262 22,897 20,617 Average annual earnings, workers and nonworkers, United States $3,174 $5,741 $7,972 $10,316 $11,527 $14,325 $15,325 $18,032 $18,144 $19,968 $20,505 $22,468 $22,530 $24,514 $23,978 $22,431 $23,354 $25,804 $27,221 $26,220 $26,894 $27,028 $27,636 $27,153 $27,214 $28,534 $28,222 $28,414 $27,974 $27,794 $28,189 $28,038 $27,896 $27,865 $28,098 $25,713 $26,649 $26,356 $23,163 $25,921 $21,941 $22,215 $23,097 $19,166 $17,390 $12,120 $11,068 $8,034 $8,775 $6,869 $7,039 $5,633 $6,577 $6,375 $3,867 $2,838 $2,027 $3,492 $2,285 $1,104 $1,844 $1,601 $976 Total number of people in households, United States 3,809,016 3,464,472 3,659,116 3,612,517 3,794,167 3,749,240 3,888,289 3,844,850 3,609,340 3,684,725 3,659,564 3,788,098 3,651,021 3,629,443 3,701,149 3,974,746 4,336,910 4,124,783 3,904,503 3,856,313 4,028,587 4,007,543 4,565,264 4,329,129 4,390,913 4,310,340 4,500,372 4,679,133 4,199,705 4,509,734 4,237,686 4,189,064 4,363,843 3,964,673 3,889,799 3,521,706 3,710,336 3,574,332 3,450,510 3,543,593 3,377,117 2,792,955 2,814,165 2,640,818 2,718,679 2,320,776 2,269,077 2,391,316 2,086,775 1,987,848 1,845,228 1,833,058 1,668,781 1,697,679 1,683,738 1,593,615 1,642,561 1,622,661 1,544,163 1,700,186 1,432,898 1,295,198 5,383,474 Total number of people in group quarters, United States 730 4,480 6,681 552 284 0 0 0 4,165 0 4,442 4,190 0 7,278 0 0 0 3,056 1,149 4,190 0 4,190 0 8,617 0 6,824 1,036 4,172 4,020 0 9,286 4,561 0 4,328 1,209 3,614 0 7,151 0 0 3,855 0 0 0 1,063 0 5,472 0 0 2,698 3,407 0 2,525 0 1,444 1,444 1,444 0 0 0 0 2,345 7,235 Total number of people in family households, United States 3,596,957 2,987,935 2,872,904 2,773,204 2,779,428 2,706,395 2,816,151 2,723,761 2,711,579 2,803,019 2,912,597 2,955,566 2,935,608 3,055,730 3,081,560 3,359,017 3,633,578 3,473,568 3,335,370 3,247,817 3,408,447 3,494,064 3,871,119 3,761,068 3,788,287 3,678,303 3,852,283 3,966,309 3,531,316 3,787,472 3,487,046 3,469,515 3,557,464 3,291,757 3,146,486 2,838,386 3,057,872 2,913,325 2,775,454 2,808,089 2,735,219 2,245,174 2,192,840 2,084,095 2,145,492 1,825,997 1,803,933 1,846,406 1,638,184 1,541,097 1,419,601 1,368,879 1,193,160 1,288,380 1,206,481 1,143,833 1,136,078 1,058,835 1,043,494 1,218,259 941,825 762,039 3,095,700 Probability of shifting household production costs upon disability or death 0.945 0.864 0.787 0.768 0.733 0.722 0.724 0.708 0.752 0.761 0.797 0.781 0.804 0.844 0.833 0.845 0.838 0.843 0.854 0.843 0.846 0.873 0.848 0.871 0.863 0.855 0.856 0.848 0.842 0.840 0.825 0.829 0.815 0.831 0.809 0.807 0.824 0.817 0.804 0.792 0.811 0.804 0.779 0.789 0.789 0.787 0.797 0.772 0.785 0.776 0.771 0.747 0.716 0.759 0.717 0.718 0.692 0.653 0.676 0.717 0.657 0.589 0.576 The inflation index is implicit price deflator for personal consumption expenditures. The Washington population numbers are from the Washington State Office of Financial Management. The average earnings data are for workers and non-workers and are from the 2005 Current Population Survey from the US Census Bureau. The household data are from the same CPS. 24 Line number Exhibit B.3 The Benefits and Costs of Evidence-Based Treatment: Program-Specific Model Parameters Evidence-Based Treatment: Adults With Alcohol, Drug, or Mental Illness Disorders See text for information about these parameters Adults with a serious DSM alcohol disorder Adults with a serious DSM drug disorder Adults with a serious DSM mental illness disorder Program Effectiveness Parameters 1 2 3 4 5 6 7 8 9 Unadjusted effect size from the meta analyses (program effect on disordered outcome) Standard error for the unadjusted effect size from the meta analyses Adjusted effect size after applying WSIPP* adjustments Estimated standard error for the WSIPP*-adjusted effect size Expected annual rate of decay in effect size Standard error Expected diminishing returns to effect size with large scale ramp up (lower expected rate of decay) (higher expected rate of decay) -.312 .027 -.247 .021 -.062 .027 .750 1.000 .500 -.451 .044 -.355 .035 -.164 .072 .750 1.000 .500 -.402 .052 -.360 .058 -.176 .089 .750 1.000 .500 39.9 13.4 18 65 $2,300 $500 2005 .000 1.0 75% 36.4 13.4 18 65 $2,300 $500 2005 .000 1.0 75% 40.4 13.4 18 65 $3,596 $782 1992 .000 1.0 75% 15.69% 5.55% 0.26% 2.94% 2.05% 0.16% 6.36% 3.80% 0.22% 11.1% 0.4% 50% 75% 25% 4,145,297 230,087 204,435 102,218 14.7% 0.9% 50% 75% 25% 4,145,297 84,955 72,497 36,248 46.2% 3.5% 50% 75% 25% 4,145,297 157,521 84,746 42,373 80 3 3 2 1 99 80 4 45 11 0 0 80 2 8 2 43 0 1992 2,125,554 333,598 107,360 0.32 2000 2,362,000 69,502 23,544 0.34 1992 2,125,554 135,189 32,381 0.24 1 -0.260 0.061 21,356 76.0% 29,715 -15.6% 1 -0.262 0.059 21,356 76.0% 29,715 -15.7% 1 -0.250 0.038 21,356 76.0% 29,715 -15.0% $44.1 1998 204,426,000 15,127,524 $4,496 10.0% 43.2% 11.2% 45.6% $15.7 2002 215,127,000 3,226,905 $6,114 10.0% 59.0% 12.7% 28.3% $46.2 1992 185,473,000 7,047,974 $13,799 10.0% 79.1% -7.4% 28.3% 0.9194 -0.0228 -0.0009 0.0000 30 0.6108 -0.0601 0.0023 0.0000 30 0.5861 -0.0177 0.0004 0.0000 30 Program Design Parameters 10 11 12 13 14 15 16 17 18 19 Average age of program participant Standard deviation of age of program participant Minimum age of program participant Maximum age of program participant Average annual program cost SD of average program cost Year of program cost estimate Annual real rate of escalation in program costs Average number of years of treatment episode, per average participant Percent of program costs paid by taxpayer Prevalence Parameters 20 21 22 Lifetime prevalence of DSM disorder in this population cohort Current (12-mo) prevalence of DSM disorder in this population cohort Standard error Total Potential Population to Be Treated 23 24 25 26 27 28 29 30 31 Proportion of target population already treated with evidence-based program Standard error Proportion of the currently unserved target population that might realistically be served high low Total current Washington population (in the age group of those to be treated) Those currently with the DSM disorder Market potential: the number not already being treated with evidence-based treatment Realistic market potential: the number realistically available for evidence-based treatment Mortality Parameters (age of death for person with disorder) 32 33 34 35 36 37 38 39 40 41 42 43 Maximum Age for Death (Normal life expectancy for control group) Distribution type for probability density Probability distribution: Parameter 1 Probability distribution: Parameter 2 Probability distribution: Parameter 3 Probability distribution: Parameter 4 Attributed Death Factor (Of those with disorder, prob death is caused by the disorder) Year of analysis Total deaths in year of analysis, United States Of the deaths that year, the number that had (ever in lifetime) a DSM condition Deaths due to disorder in the year, United States Probability of a lifetime disorder AND that the death was due to the disorder Morbidity Parameters (earnings and household production) 44 45 46 47 48 49 50 Effect size applies to: 1 (employment rate), or 2 (earnings of earners) Unadjusted ES: Economic outcomes (either employment or earnings) Earnings =f(Disorder) Standard error Average earnings (CPS 2004) includes non-earners Percent with earnings (CPS 2004) Standard deviation of average earnings (CPS 2004) earners only Percent change to average earnings, from the disorder Health Care Costs 51 52 53 54 55 56 57 58 59 Total cost (billions), United States Year of estimate Adult population for year of estimate, United States Current (12-month) number of people with a DSM disorder Annual cost per current abuser (adjusted to base year for real growth in costs) Assumed percentage (plus and minus) from the average cost Percent of costs paid by taxpayer Percent of costs paid by participant Percent of costs paid by other private payers Natural Rate of Recovery Parameters Constant 60 61 Time Time^2 62 Time^3 63 64 Cutoff age * Washington State Institute for Public Policy 25 Exhibit B.3 (Continued) The Benefits and Costs of Evidence-Based Treatment: Program-Specific Model Parameters Line number Evidence-Based Treatment: Adults With Alcohol, Drug, or Mental Illness Disorders Adults with serious Adults with serious Adults with serious DSM Alcohol DSM Drug Disorder DSM Mental Illness Disorder Disorder Crime Parameters 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 Effect Size: Crime outcomes as a function of the disorder, from meta analysis Standard error Minimum age for crime distributions Maximum age for crime distributions, =Y Maximum age for observed crime parameters, =X Scaleup =1 total convictions, Scaleup=2 for felony convictions Scaleup: estimated difference in crime at age X to age Y (=X/Y) Out of population, total percent with a crime event by age X Of those with a crime event, the average number of events per person at age X Of the total population, the average number of events per person at age X Murder offenses for this population at age X Sex offenses for this population at age X Robbery offenses for this population at age X Aggravated assault offenses for this population at age X Property offenses for this population at age X Drug offenses for this population at age X Line 5 of Exhibit B.3 lists one of several conservative assumptions we use in this analysis. It displays the annual rate of decay that we assume for the effect size shown on Line 3. For the most part, the effect size on line 3 reflects the results from a meta-analysis of individual program evaluation studies that usually have fairly short-term follow-up periods. In this benefit-cost analysis, on the other hand, we estimate the long-run benefits of EBT based on these short-term effect sizes. It can be argued that these short-term effect sizes will decay over time; that is, the effect that is observed after one year may not persist five or ten years into the future. The purpose of the estimate on line 5 is to provide a way to model the uncertainty of this potential decay. This assumed rate of decay is an important factor that determines range of uncertainty in our overall estimates. We found that effects of treatment were eroded by half in 11 years for alcohol disorders, four years for disordered drug use, and 3.5 years for mental illness. For each of the three classes of treatment (alcohol, drugs, and mental illness), we estimate a mean annual rate of decay and a standard error for the mean. These two parameters are then used in sensitivity analyses. We do this by using data from those studies in our analysis where the follow-up period is noted.51 For each broad treatment type, our regression analysis uses up to seven different functional forms to examine how length of the follow-up period influences the observed effect size. The model with the best adjusted R-square value (that is, the best fit) is chosen for each class of treatment. 51 Not all studies clearly stated the follow-up period. Our analysis included 91 studies on alcohol treatment and 40 studies on drug treatments. We estimated a single rate of decay for treatment of mental illness. Because follow-up times were often very short for mental illness, we limited this analysis to those treatments with numerous studies of varying follow-up times: chlorpromazine, Assertive Community Treatment, and non-drug therapies for depression and anxiety disorders. For some mental health treatments, where we relied on Cochrane reviews, we coded an effect size and follow-up period for each study in the review. Our analysis of effect size decay included 84 studies on mental illness. 26 .192 .099 10 80 32 1 66.5% 15.4% 2.65 .41 223 1224 1133 2766 14910 6019 3.140 .000 10 80 80 1 100.0% 100.0% 2.00 2.00 0 0 0 0 0 6019 .392 .046 10 80 32 1 66.5% 15.4% 2.65 .41 223 1224 1133 2766 14910 6019 Lines 7 through 9 on Exhibit B.3 describe another set of conservative assumptions we employ. The purpose of this study is to estimate the aggregate benefits and costs of EBT for a relatively large percentage of people with alcohol, drug, or mental health disorders in Washington. The effect size that we estimate on line 3, however, is derived mostly from individual studies of much smaller populations. Because of self-selection and diminishing returns, it can be conjectured that the average treatment effect obtained from these studies of more serious populations will not be as great if EBT programs were extended to a wider group of people with clinical disorders. It can also be argued that, as programs get larger, it becomes more difficult to maintain quality control and, therefore, a larger-scale program would yield reduced effects compared with those obtained from smaller programs. Thus, the assumptions employed on lines 7 through 9 provide a means to model this uncertainty. The assumptions are multiplicative factors that we apply to the adjusted and decayed effect sizes. For example, the base case assumption shown on line 7—a factor of .75—means that we assume the average treatment effect will only be 75 percent as large if the program were to be implemented on a large scale. In the sensitivity analyses, we allow this assumption to vary by the higher and lower assumptions shown on lines 8 and 9. B3. Program Design Parameters The second section in Exhibit B.3 lists two of the parameters we use to describe the generic EBT programs. The first set of parameters, lines 10 through 13, describes the age groups that might be eligible for the three prototype programs. These parameters are used in estimating the total size of the potential treatment populations as well as in the calculation of the estimated benefits. Using a normal distribution with a mean age (line 10) and standard deviation (line 11), and bounding the distribution by the minimum age (line 12) and maximum age (line 13), a density distribution P is estimated for the probability of program participation, such that, B(6): 1 = max ∑ Pp , p = min where the distribution P is defined to be normally distributed with a mean age and its standard deviation. Lines 14 through 19 list the assumptions we made about the cost of EBT programs. These include estimates of the average cost per treatment episode, assumptions regarding the standard deviation for these average costs, and the extent to which EBT programs would be financed by tax dollars. Rather than costing-out each of the individual EBT programs examined, we assumed that EBT is the norm for those currently receiving services. Therefore, the observed average cost per treatment episode is a reasonable approximation of the average cost per episode of an average EBT program. Of course, to the extent the current practices do not represent evidence-based approaches, we may be under-estimating the cost of EBT programs. The average costs of EBT for alcohol, drug, and mental health are derived from two sources. According to one recent report, the average cost of EBT for alcohol or drug abuse in Washington State was $2,300 per episode in 2002.52 The report did not provide separate estimates for alcohol and drug treatment, therefore, the same figure is used for both program areas. A similar episode-based cost estimate for treatment of serious mental illness was not available for Washington State. Fortunately, the same study that we used to describe health care and other costs attributable to mental illness also provided an estimate of mental health treatment costs, which in 1992 dollars, averaged $3,596 per episode.53 Updated to current dollars, we assume this to be the cost of EBT for serious mental health disorders. B4. Prevalence Parameters To determine the size of the population in Washington that has a serious disorder that could be addressed with one of the three prototype EBT programs, we reviewed the national literature on the prevalence of the disorders in the general population. There have been several national studies conducted in the last 20 years to estimate the lifetime and current prevalence of serious alcohol, drug, and mental illness in the general population. Lines 20 to 22 show the estimates from our reading of the national literature. Line 20 shows the estimated lifetime probability of having one of the disorders. This parameter is used when we model the mortality effects of the disorders, described in Appendix B9. For alcohol and drug dependence, 54 we use the lifetime prevalence rates listed in Harwood et al. The Harwood lifetime rates were taken from their analysis of the National Longitudinal Alcohol Epidemiologic Survey for adults ages 18 to 64. Harwood reports lifetime prevalence rates for males and females; we combine them into an overall average using 1992 census data on the ratio of males to females in the 18 to 64 age group. The estimate we used for a lifetime prevalence of serious mental illness, shown on line 20, was derived in the following manner. Harwood et al. (2000) provided an estimate of the 55 12-month prevalence of serious mental illness for males and females at .03 and .046, respectively, for an average rate of .038. This number accounts for comorbidity, that is, persons 52 T.M. Wickizer, A. Krupski, K. Stark, D. Mancuso, and K. Campbell. (in press). The Effect of Substance Abuse Treatment on Medicaid Expenditures among General Assistance Welfare Clients in Washington State. Milbank Quarterly. 53 Harwood et al., The economic costs of mental illness, page 3-6. 54 Harwood et al., The economic costs of alcohol and drug abuse. 55 Harwood et al., The economic costs of mental illness. with more than one serious mental illness are counted only once. We estimated lifetime prevalence by summing lifetime prevalence rates reported for the National Comorbidity Survey 56 Replication for schizophrenia, bipolar disorders, and serious forms of major depression and panic disorder. To account for comorbidity, we then multiplied by the ratio of Harwood’s 12month prevalence to the sum of 12-month prevalences for each of these disorders. Note that this rate on line 20 is for severe diagnoses which the Harwood report defines to be schizophrenia, non-affective psychosis, manic depressive disorder, severe forms of major depression and panic disorder.57 In our study, we confine our economic analyses to these severe forms of mental disorders. Line 21 of Exhibit B.3 is the estimate we use in this study of the current (i.e. 12-month) prevalence of each disorder in the general population. For serious alcohol and drug disorders, we use the estimates provided in Narrow et al. (2002) which are based on their interpretation of the clinical significance of findings from the National Comorbidity Survey and the Epidemiologic Catchment Area study.58 For serious mental illness disorders, we use the 59 estimate provided in Harwood et al. We account for the comorbidity between drug and alcohol dependency with the following calculations. Narrow et al. report a total disorder rate for any alcohol or drug disorder of 7.6 percent for the 18- to 54-year-old age group. They also report a 6.5 percent rate of alcohol disorders and a 2.4 percent rate for other drug use disorders. To account for comorbidity and avoid double counting people later in our analysis, we estimate the unique alcohol disorder rate as 5.5 percent (.055 = 0.076*(6.5/(6.5+2.4))) and the unique other drug disorder rate as 2.05 percent (.0205 = 0.076*(2.4/(6.5+2.4))). We estimate the size of the standard errors with the number of subjects in the National Comorbidity Survey (7,599). The associated standard errors are used in sensitivity analyses. B5. Total Potential Population to Be Treated We estimated two additional factors to help focus the analysis on the size of the population that could take advantage of the prototype EBT programs. First, we estimate the size of the disordered population already being treated with EBT programs in Washington. These estimates are shown on lines 23 and 24 of Exhibit B.3. For people with serious alcohol disorders and for those with serious illicit drug disorders, we analyzed the public use data set for the National Epidemiological Survey on Alcohol and Related Conditions (NESCAR). Among people indicating alcohol dependence in the past 12 months, we calculated the average percent and standard deviation that had been treated professionally for their alcohol disorder in the past 12 months. We used the same procedure for those with dependence on illicit drugs. For people with serious mental illness disorders, we relied on estimates of treatment rates by Kessler et al. based on the 60 National Comorbidity Survey. On lines 25 to 27 we also make additional restrictions on the size of the population that might be treated with EBT programs. It is never possible to completely saturate a market, so we provide factors to estimate low, medium, and 56 R. Kessler et al. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. 57 Harwood et al., The economic costs of mental illness. Table 4.7. 58 Narrow et al., Revised prevalence estimates of mental disorders. 59 Harwood et al., The economic costs of mental illness. Table 4.7. 60 R. Kessler, P. Berglund, M. Bruce, J. Koch, E. Laska, P. Leaf, R. Mandersheid, R. Rosenheck, E. Walters, and P. Wang. (2001). The prevalence and correlates of untreated serious mental illness. Health Services Research, 36(6): 987-1007. 27 high market penetration rates. These alternative rates are used in the sensitivity analysis described in Appendix B12. To compute the earnings effect of an incidence of a DSM disorder, we begin with the following equation: The factors described above are used to compute the total size of the current population in Washington that: (a) has a serious disorder, (b) is not currently being treated, and (c) might be realistically treated with a prototype EBT. Line 28 of Exhibit B.3 reports the size of the total population in Washington between the minimum and maximum age groups described on lines 12 and 13. The Washington population estimate is taken from the Washington State Office of Financial Management, and the actual population estimates 61 are shown in Exhibit B.2. To this figure, we then applied the 12-month prevalence estimate (from line 21) to produce line 29: the estimated total current number of people in Washington with the disorder. Line 30 then subtracts the estimated percentage of the population already being treated with EBT programs (from line 23). Finally, line 31 applied the base assumption about the realistic potential (from line 25) to enroll disordered people in a prototype EBT. B(8): Ea = EARNINGS a * FRINGE * INFLATION B6. Morbidity Parameters and Methods Prior studies of the costs of alcohol, drug, and mental illness disorders have found that, among people with the disorders, performance in the economic marketplace is reduced.62 To provide an independent test of this hypothesis, we conducted several meta-analyses. We sought to determine if existing research studies indicate that either an individual’s level of earnings conditional on employment, or the rate of employment alone, was significantly related to the presence of having an alcohol, drug, or mental illness disorder. We reviewed the literature on the topics and used the meta-analytic methods described in Appendix A to this report. Exhibit B.4 summarizes the results of our meta-analyses. We found that all three disorders are significantly related to the probability of employment, but not earnings conditional on employment. The effect sizes for employment from the metaanalyses are shown on line 45 of Exhibit B.3 and the associated standard errors are listed on line 46. To find the marginal effect of a disorder on average earnings levels (via the effect on employment rates), we compute the following: ES ⎞ ⎛ AE * sin(arcsin( ER ) + )^2 − AE ⎟ ⎜ ER 2 ⎠ ⎝ , B(7): EEt = AE where EE is the estimated earnings effect for each of the evidence-based treatments, t, and represents the percentage change in average earnings; AE is the average earnings of earners and non-earners taken as a whole (this estimate, shown on line 47, comes from the CPS; ER is the employment rate (shown on line 48 of Exhibit B.3, also from the CPS) and ES is the effect size of the effect of having a disorder on ER (shown on line 45, from the meta-analysis). Since the arcsine transformation is used to compute the effect size in the metaanalyses, as described in Appendix A, that effect is reversed here to return the unit change. 61 Washington State Office of Financial Management, http://www.ofm.wa.gov/pop/default.asp See: (a) Harwood, Updating estimates of the economic costs of alcohol abuse in the United States, from Table 3; (b) Office of National Drug Control Policy. The economic costs of drug abuse in the United States, from Table III-1; and (c) Harwood et al., The economic costs of mental illness, from Table 6.1. 62 28 For each age a, the total earnings of a person Ea is the annual EARNINGS taken from the Current Population Survey for that age, shown on Exhibit B.2, times the FRINGE benefit multiple, shown on line 3 of Exhibit B.1, times the INFLATION adjustment from Exhibit B.2 to inflate the CPS series (denominated in 2004 dollars) to the year chosen for this analysis (2006 dollars). The annual cash flows of lost earnings associated with having a disorder of type t is estimated with this process: P B(9): $ Ety = ∑ E p + y −1 *(1 + ER) y −1 * EEt * PPtp * −1 p In this equation, $Ety is the annual cash flow of lost earnings for a person with disorder type t in year y, where y is the number of years following participation in an EBT. The subscript y equals 1 during the year that a person is administered an EBT. Exhibit B.4 Meta-Analytic Estimates of Standardized Mean Difference Effect Sizes Results Before Adjusting Effect Sizes Number of Effect Sizes Included in the Analysis Fixed Effects Model Weighted Mean Effect Size & p-value ES Random Effects Model Homogeneity Test p-value p-value Weighted Mean Effect Size & p-value ES p-value Employment =f(alcohol disorder) 11 -.183 0.000 0.000 -.239 Wages =f(alcohol disorder) 5 .004 0.701 0.124 na Employment =f(DSM mental illness) 8 -.246 0.000 0.000 -.250 0.000 Wages =f(DSM mental illness) 7 -.140 0.000 0.000 -.213 0.000 Employment =f(drug disorder) 6 -.230 0.000 0.000 -.262 0.000 Wages =f(drug disorder) 1 .000 0.981 na na 0.000 na na Crime =f(Mental Illness) 3 .337 0.000 0.001 .392 0.000 Crime =f(Alcohol Disorder) 3 .176 0.000 0.000 .192 0.053 Studies (complete citation on next page) Used to estimate Zuvekas, Cooper, & Buchmueller, 2005 Mullahy and Sindelar, 1993 Mullahy and Sindelar, 1996 Mullahy and Sindelar, 1997 Terza, 2002 Terza, (undated) Chevrou-Severac and Jeanrenaud, 2002 Feng et al., 2001 Auld, 2002 MacDonald & Shields, 2004 Cook & Peters, 2005 Zuvekas, Cooper, & Buchmueller, 2005 Mullahy and Sindelar, 1993 Zarkin et al., 1998 Kenkel and Ribar, 1994 Bray, (2005) Harwood et al., 2000 Ettner et al., 1997 Farahati et al., 2003 Savoca, 2000 Alexandre & French, 2001 Kessler et al., 1999 Hamilton et al., 1997 Chatterji et al., 2005 Ettner et al., 1997 Marcotte, 2003 Kessler & Frank, 1997 Frank & Gertler, 1991 Bartel & Taubman, 1986 French & Zarkin, 1998 Stewart et al., 2003 DeSimone, 2002 Buchmueller and Zuvekas, 1998 Zuvekas, Cooper, & Buchmueller, 2005 Terza, (undated) Alexandre & French, 2004 French, Roebuck, Alexandre, 2001 Zuvekas, Cooper, & Buchmueller, 2005 Hodgins et al., 1996 Tiihonen, 1997 Wallace et al., 2004 Carpenter, 2003 Fergusson and Horwood, 2000 Lipsey et al., 1996 Employment =f(alcohol disorder) Employment =f(alcohol disorder) Employment =f(alcohol disorder) Employment =f(alcohol disorder) Employment =f(alcohol disorder) Employment =f(alcohol disorder) Employment =f(alcohol disorder) Employment =f(alcohol disorder) Employment =f(alcohol disorder) Employment =f(alcohol disorder) Employment =f(alcohol disorder) Wages of workers =f(alcohol disorder) Wages of workers =f(alcohol disorder) Wages of workers =f(alcohol disorder) Wages of workers =f(alcohol disorder) Wages of workers =f(alcohol disorder) Employment =f(DSM mental illness disorder) Employment =f(DSM mental illness disorder) Employment =f(DSM mental illness disorder) Employment =f(DSM mental illness disorder) Employment =f(DSM mental illness disorder) Employment =f(DSM mental illness disorder) Employment =f(DSM mental illness disorder) Employment =f(DSM mental illness disorder) Wages of workers =f(DSM mental illness disorder) Wages of workers =f(DSM mental illness disorder) Wages of workers =f(DSM mental illness disorder) Wages of workers =f(DSM mental illness disorder) Wages of workers =f(DSM mental illness disorder) Wages of workers =f(DSM mental illness disorder) Wages of workers =f(DSM mental illness disorder) Employment =f(drug disorder) Employment =f(drug disorder) Employment =f(drug disorder) Employment =f(drug disorder) Employment =f(drug disorder) Employment =f(drug disorder) Wages of workers =f(drug disorder) crime =f(mental illness) crime =f(mental illness) crime =f(mental illness) crime =f(alcohol disorder) crime =f(alcohol disorder) crime =f(alcohol disorder) 29 Exhibit B.4a Citations to Studies in Exhibit B.4 Alexandre, P. & French, T. (2001). Labor supply of poor residents in metropolitan Miami, Florida: The role of depression and the co-morbid effects of substance use. The Journal of Mental Health Policy and Economics, 4(4): 161-173. Alexandre, P.K. & French, M.T. (2004). Further evidence on the labor market effects of addiction: Chronic drug use and labor supply in metropolitan Miami. Contemporary Economic Policy, 22(3): 382-393. Auld, M.C. (2002). Robust system estimation of causal effects on binary outcomes, with application to effect of alcohol abuse on employment. Calgary, Alberta, Canada: Department of Economics, University of Calgary. http://econ.ucalgary.ca/research/200301WP.pdf. Bartel, A. & Taubman, P. (1986). Some economic and demographic consequences of mental illness. Journal of Labor Economics, 4(2): 243-256. Bray, J. W. (2005). Alcohol use, human capital, and wages. Journal of Labor Economics, 23(2): 279-312. Buchmueller, T.C. & Zuvekas, S.H. (1998). Drug use, drug abuse, and labour market outcomes. Health Economics, 7(3): 229-45. Carpenter, C. (2003). Does heavy alcohol use cause crime? Evidence from underage drunk driving laws. Unpublished paper. Chatterji, P., Alegría, M., Lu, M., & Takeuchi, D. (2005). Psychiatric disorders and labor market outcomes: Evidence from the National Latino and Asian American Study (Working Paper No. 11893). Washington, DC: National Bureau of Economic Research. Chevrou-Severac, H. & Jeanrenaud, C. (2002). The impact of alcohol abuse on employment in Switzerland. Neuchatel, Switzerland: IRER. http://perso.wanadoo.fr/ces/Pages/english/Poster17.pdf. Cook, P.J. & Peters, B.L. (2005). The myth of the drinker's bonus (Working Paper No. W11902). Washington, DC: National Bureau of Economic Research. DeSimone, J. (2002). Illegal drug use and employment. Journal of Labor Economics, 20(4): 952-977. Ettner, S.L., Frank, R.G., & Kessler, R.C. (1997). The impact of psychiatric disorders on labor market outcomes. Industrial & Labor Relations Review, 51(1): 64-81. Farahati, F., Booth, B., & Wilcox-Gők, V. (2003). Employment effects of comorbid depression and substance use. DeKalb, IL: Northern Illinois University, Department of Economics. Feng, W., Zhou, W., Butler, J., Booth, B., & French, M. (2001). The impact of problem drinking on employment. Health Economics, 10(6): 509-521. Fergusson, D.M. & Horwood, L.J. (2000). Alcohol abuse and crime: A fixed-effects regression analysis. Addiction, 95(10): 1525-1536. Frank, R. & Gertler, P. (1991). An assessment measurement error bias for estimating the effect of mental distress on income. The Journal of Human Resources, 26(1): 154164. French, M.T. & Zarkin, G.A. (1998). Mental health, absenteeism and earnings at a large manufacturing worksite. The Journal of Mental Health Policy and Economics, 1(4): 161-172. French, M.T., Roebuck, M.C., and Alexandre, P.K. (2001). Illicit drug use, employment, and labor force participation. Southern Economic Journal, 68(2): 349-368. Hamilton, V.H., Merrigan, P., & Dufresne, E. (1997). Down and out: estimating the relationship between mental health and unemployment. Health Economics, 6(4): 397-406. Harwood, H. (2000). The economic cost of mental illness, 1992. Fairfax, VA: The Lewin Group. Hodgins, S., Mednick, S.A., Brennan, P.A., Schulsinger, F., & Engberg, M. (1996). Mental disorder and crime. Evidence from a Danish birth cohort. Archives of General Psychiatry, 53(6): 489-96. Kenkel, D.S. & Ribar, D.C. (1994). Alcohol consumption and young adults' socioeconomic status. Brookings papers on economic activity: Microeconomics: 119-161. Kessler, R.C. & Frank, R. (1997). The impact of psychiatric disorders on work loss days. Psychological Medicine, 27(4): 861-873. Kessler, R.C., Barber, C., Birnbaum, H.G., Frank, R.G., Greenberg, P.E., Rose, R.M., et al. (1999). Depression in the workplace: Effects on short-term disability. Health Affairs, 18(5): 163-171. Lipsey, M.W., Wilson, D.B., Cohen, M.A., & Derzon, J.H. (1996). Is there a causal relationship between alcohol use and violence? A synthesis of evidence. In M. Galanter (Ed.), Recent Developments in Alcoholism, Volume 13: Alcoholism and Violence (pp. 245-282). New York: Plenum Press. MacDonald, Z. & Shields, M. (2004). Does problem drinking affect employment? Evidence from England. Health Economics, 13 (2): 139-155. Marcotte, D.E. & Wilcox-Gők, V. (2003). Estimating earnings losses due to mental illness: A quantile regression approach. The Journal of Mental Health Policy and Economics, 6(3): 123-134. Mullahy, J. & Sindelar, J.L. (1993) Gender differences in labor market effects of alcoholism. The American Economic Review, 81(2): 161-165. Mullahy, J. & Sindelar, J.L. (1996). Employment, unemployment, and problem drinking. Journal of Health Economics, 15(4): 409-34. Mullahy, J. & Sindelar, J.L. (1997). Women and work: Tipplers and teetotalers. Health Economics, 6(5): 533-537. Savoca, E. & Rosenheck, R. (2000). The civilian labor market experiences of Vietnam-era veterans: The influence of psychiatric disorders. The Journal of Mental Health Policy and Economics, 3(4): 199-207. Stewart, W.F., Ricci, J.A., Chee, E., Hahn, S.R., & Morganstein, D. (2003). Cost of lost productive work time among US workers with depression. Journal of the American Medical Association, 289(23): 3135- 3144. Terza, J.V. (2002). Alcohol abuse and employment: A second look. Journal of Applied Econometrics, 17(4): 393-404. Terza, J.V. (n.d.). Assessing the impact of substance abuse on employment status. Charleston, SC: Medical University of South Carolina, Department of Health Administration and Policy, Center for Health Economic and Policy Studies. http://people.musc.edu/~terza/empfina2.pdf. Tiihonen, J., Isohanni, M., Räsänen, P., Koiranen, M., & Moring, J. (1997). Specific major mental disorders and criminality: A 26-year prospective study of the 1966 northern Finland birth cohort. American Journal Psychiatry, 154(6): 840-845. Wallace, C., Mullen, P.E., & Burgess, P. (2004). Criminal offending in schizophrenia over a 25-year period marked by deinstitutionalization and increasing prevalence of comorbid substance use disorder. American Journal of Psychiatry, 161(4): 716-727. Zarkin, G.A., French, M.T., Mroz, T., & Bray, J.W. (1998). Alcohol use and wages: New results from the National Household Survey on Drug Abuse. Journal of Health Economics, 17(1): 53-68. Zuvekas, S., Cooper, P.F., Buchmueller, T.C. (2005). Health behaviors and labor market status: The impact of substance abuse (Working Paper No. 05013). Rockville, MD: US Department of Health and Human Services, Agency for Healthcare Research and Quality. http://gold.ahrq.gov. 30 The earliest age that a person might participate in an EBT is notated as p and runs to the maximum possible age P (values for p and P are shown on lines 12 and 13 of Exhibit B.3, respectively). The annual cash flows in each year following investment is the weighted sum of the product of the adjusted earnings E in year y for the age of the program participant p, times the annual real growth rate in earnings ER, times the estimated earnings effect EEt, times the probability of program participation PP at age p. This procedure produces a series of expected annual cash flows representing lost earnings following investment and weighted by the probability of program participation for the ages of the people assumed to enter the EBT. B7. Lost Household Production Methods As described above, in addition to the value of reduced or lost performance in the commercial economy, many studies of morbidity and mortality costs include estimates of the reduced or lost value of household production.63 We adopt that approach in this study. To compute the household production effect for the incidence of the DSM disorders, we begin with the following equation: B(10): H a = HOURS * $ HOUR * 52 * Pr SHIFTa * INFLATION For each age a, the annual value of household production Ha is the HOURS per week for household tasks (line 9 from Exhibit B.1, times the weighted average dollars per hour $HOUR for household tasks (line 10), times 52 weeks per year, times the probability that household tasks get shifted to someone else PrSHIFT for a person who is age a (from Exhibit B.2), times the INFLATION adjustment to bring the hourly wage (denominated in 2004 dollars) to the year chosen for this analysis (2006 dollars). Not all of the value of lost household production will be shifted to others if a person dies or is disabled as a result of having an alcohol, drug, or mental health disorder. Some people live alone and no one else is required to assume the household production if the person becomes disabled or dies as a result of the disorder. We provide an estimate for this with the variable PrSHIFTa, used in the previous equation. This variable provides an estimate of the probability that a person at age a will not be living alone and, if he or she becomes disordered, that the value of his or her household production will be shifted to someone else. We estimate this probability with national data from the same 2005 Current Population Survey (with data for 2004) described above.64 The results of this estimation are shown in Exhibit B.2 and are computed with this equation: B(11): Pr SHIFTa = FHH a ( HH a − GQa ) The probability of shifting household production PrSHIFTa in the event of a disorder is given by the total number of people in households with family members FHHa divided by the total number of people in households HHa (less those living in group quarters GQa). Values for all three variables come from the CPS. The annual cash flows of lost household production associated with having a disorder of type t is estimated with this process: 63 Max et al., Valuing human life. Current Population Survey data downloaded from the US Census Bureau site with the DataFerrett extraction utility: http://www.bls.census.gov/cps/cpsmain.htm P B(12): $ HPty = ∑ H p + y −1 *(1 + ER) y −1 * EEt * PPtp * −1 p In this equation, $HPty is the annual cash flow of shifted household production in year y, where y is the number of years following participation in an EBT. The subscript y equals 1 during the year that a person is administered an EBT. The earliest age that a person might participate in an EBT is notated as p and runs to the maximum possible age P (values for p and P are shown on lines 12 and 13 of Exhibit B.3, respectively). The annual cash flows in each year following investment is the sum of the product of household production H in year y for the age of the program participant p, times the annual real growth rate in earnings ER, times the estimated earnings effect EE, times the probability of program participation PP at age p. This procedure produces a series of expected annual cash flows representing shifted household production following investment and weighted by the probability of program participation for the ages of the people assumed to enter the EBT. B8. Health Care and Other Costs An additional set of costs of alcohol, drug, and mental health disorders covers the effect the disorders have on health care costs. We show our assumptions and estimates for this on lines 51 through 59 in Exhibit B.3. We start with the national estimates provided by Harwood in his several reports on the costs of alcohol, drug, and mental health disorders. These amount to $44 billion for alcohol disorders in 1998, $15.7 billion for drug disorders in 2002, and $46.2 billion for serious mental illness in 1992.65 On line 54, we show the adult (age 18 and over) population for the relevant years from the US Census Bureau as reported in the Statistical Abstract of the United States. On line 55, we multiply the total adult population estimates by the same 12-month prevalence percentages reported in the Harwood studies (.074 for alcohol, .015 for drug, and .038 for serious mental illness). The average costs are then computed and shown on line 55; we report on line 56 the plus and minus percentage change we use in sensitivity analyses for the average health care cost figure. Finally, on lines 57 though 59 we report the Harwood percentages for the amount of health care costs incurred by taxpayers, participants, and other private payers. The annual cash flows of health care costs associated with having a disorder of type t is estimated with this process: P B(13): $ HCty = ∑ HCCOSTt * (1 + HR) y −1 ∗ PPtp p In this equation, $HCty is the annual cash flow of health care costs in year y, where y is the number of years following participation in an EBT. The subscript y equals 1 during the year that a person is administered an EBT. Before entering this equation, the HCCOST estimate is already denominated in the dollars for the year chosen for this analysis, including the real rate of escalation in health care costs from the year of the underlying Harwood study to the base year chosen for this analysis (2006 dollars). The earliest age that a person might participate in an EBT is notated as p and runs to the maximum possible age P (values for p and P are shown on lines 12 and 13 of Exhibit B.3, respectively). The annual cash flows in each year following investment is the sum of the product of average per capita health care costs HCCOST, times the annual real growth rate in health care costs HR, times the probability of program 64 65 See footnote 6. 31 participation PP at age p. This procedure produces a series of expected annual cash flows representing health care costs following investment and weighted by the probability of program participation for the ages of people assumed to enter the EBT. B9. Mortality Parameters and Methods If the prevalence of alcohol, drug, or mental health disorders is reduced with EBT, then one form of benefits will be that people live longer and, as a result, are more productive in the marketplace. All cost-of-illness studies estimate these mortality-related effects. The mortality methods we employed in this study required three pieces of information. The first is shown on line 32 on Exhibit B.3: the normal life expectancy for the average adult today. We estimated this parameter from the Center for Disease Control for the average life expectancy of a 40-year-old, which corresponds roughly to the average age of a person in our prototype programs.66 For people who die of a disorder, we estimated the probability of death by age of death. We used data from the Washington State Vital Statistics dataset. For alcohol and drug related deaths, we counted the age of all deaths in Washington with ICD-10 death codes where a certain percentage of the deaths had been attributed to the disorder. For alcohol related deaths, we used the attribution factors for the individual diagnoses 67 contained in Max et al. For drug related deaths, we used the 68 attribution factors contained in Harwood et al. For suicide deaths, we used all deaths in Washington coded as a suicide. Using these counts of actual Washington deaths, we then estimated a probability density distribution for each disorder (alcohol, drug, and suicide). Lines 33 through 37 contain the parameters of these equations. We found that for alcohol related deaths, a Beta distribution best fit the actual Washington data; for drug related deaths, a Normal distribution fit the data; and for suicides (mental health deaths), a Weibull distribution was best. For alcohol and drug deaths, we estimated the distributions with Washington data for 2004; for suicides we used Washington data for 2003 and 2004 to increase the sample size. For each disorder, this process produces: B(14): DDa , where DDa is the probability density distribution of a person with an alcohol or drug disorder or a suicide at age a, and the distributions are defined by a Beta, Normal, or Weibull, respectively. Not everyone who has an alcohol, drug, or mental illness disorder dies of the disorder. Lines 38 through 43 of Exhibit B.3 list the parameters we used to estimate the probability that a person with a history of a disorder dies of the disorder. For the United States, Harwood estimated the total number of deaths in 1992 (for alcohol), 2000 (for drugs), and 1992 (for suicides) that were caused by having an alcohol, drug, or mental disorder, respectively. These values are shown on line 42, while line 40 shows the total number of deaths in the United States (for people 15 or older) during those years. Line 41 is the product of line 40 66 D. Hoyert, H. Kung, and B. Smith. (2005). Expectation of life by age, race, and sex: United States, final 2002 and preliminary 2003. US Department of Health and Human Services, Centers for Disease Control and Prevention, National Vital Statistics Report, 53(15), Table 6. http://www.cdc.gov/nchs/data/nvsr/nvsr53/nvsr53_15.pdf 67 Max et al., Valuing human life, Table 2. 68 Office of National Drug Control Policy. (2004).The economic costs of drug abuse in the United States, Table B-10. http://www.whitehousedrugpolicy. gov/publications/economic_costs/economic_costs.pdf 32 and line 20, the lifetime prevalence of each disorder. This provides an estimate of the number of people who died in the relevant year who had a disorder sometime in their lives. Line 43 is computed as line 42 divided by line 41; it is the attributed death factor, ADF, for each disorder. The annual cash flows of lost earnings and household production associated with having a death caused by having a disorder of type t is estimated with this process: B(15): P $ Dty = ∑ [ E p + y −1 + H p + y −1] * (1 + ER) r −1 * DDtp + y −1 K p L * ADFt * Ptp In this equation, $Dty is the cash flow of lost earnings E and household production H in year y, where y is the number of years following participation in an EBT. The subscript y equals 1 in the year that a person is administered an EBT, and runs to M—the maximum follow-up period for estimating cash flows. The earliest age that a person might participate in an EBT is notated as p and runs to the maximum possible age P (values for p and P are shown on lines 12 and 13 of Exhibit B.3, respectively). The annual cash flows in each year following investment is computed as the weighted sum of the product of the adjusted earnings E by year y for the age of the program participant p, plus the household production H by year y for the age of the program participant p, times the real growth rate in earnings ER, times the probability of a death occurring, DD, by year y for the age of the program participant, times the attributed death factor ADF for the disorder, times the probability of program participation PP for a participant of age p. This procedure produces a series of expected annual cash flows representing lost earnings and lost household production following investment and weighted by the probability death and of program participation for the ages of the people assumed to enter the EBT. B10. Crime Parameters The effect that alcohol, drug, and mental health disorders have on crime is estimated in a two-step process. First, we use meta-analyses to examine the existing research literature on the linkage between each of these disorders and crime. Second, if the meta-analyses reveal a statistically significant relationship, we then use the Institute’s cost-of-crime model to estimate the effects that the increased levels of crime have on taxpayers (who fund the criminal justice system) and crime victims (who suffer out-of-pocket costs and pain and suffering costs when they are victimized). Then, to the degree that an evidence-based treatment reduces the incidence of a disorder, the estimated costs of crime are also reduced via this linkage. In Exhibit B.4 we list the results of the meta-analyses we performed on the linkage between the disorders and crime. We only found a few studies where the research design was robust. These few studies did provide some evidence of a statistically significant relationship between alcohol disorder and crime, and between mental illness and crime. We were unable to locate studies establishing a relationship between drug disorders and crime; this is a result consistent with other inquires into this topic.69 Nonetheless, in Washington State the consumption of these substances is illegal and, therefore, 69 See, for example: H. White, & D. Gorman. (2000). Dynamics of the drug crime relationship. In G. Lafree (Ed.), Criminal Justice 2000: Volume 1: The nature of crime: continuity and change (NCJ 182408, pp. 151-218). Washington, DC: US Department of Justice, Office of Justice Programs, National Institute of Justice. http://www.ncjrs.org/criminal_justice2000/vol_1/02d.pdf. these drug crimes can result in a criminal justice system response including arrest, prosecution, and a full range of sentencing outcomes. These effects are modeled. The Institute’s model of the costs of crime has been described in detail elsewhere; the interested reader can find a full description of the routines used to calculate costs in the earlier reports.70 B11. Marginal Treatment Effect The estimated benefits of treatment are determined by the marginal effectiveness, over time, of EBT. The following equation is used to estimate the marginal treatment effect MTE for a person in an EBT treating a disorder of type t: ( ESty ) B(16): MTEty = Nty − sin(arcsin( Nty ) + )^ 2 , 2 where B(17): ESty = ESt ∗ (1 + decayratet ) y −1 ∗ scaleupt , and where B(18): Nty = NRta + NRtb1 * y + NRtb2 * y 2 + NRtb3 * y 3 . For each of the three prototype programs t, we estimate the marginal treatment effect with the parameters in these equations. The variable Nty is the “natural rate of recovery” from a disorder without treatment in year y for treatment type t, where y is a year following participation in an EBT. The subscript y equals 1 during the year that a person is administered an EBT. We estimated years from onset to “natural recovery” using data from the 2001–2002 National Epidemiologic Survey on Alcohol and Related Conditions (NESARC).71 The NESARC is a longitudinal survey conducted by the federal National Institute on Alcohol Abuse and Alcoholism. The 2001–2002 NESARC is the first wave of the survey, with a sample of 43,093 respondents representative of the US adult population 18 years of age and older. We performed separate analyses for respondents who reported ever having the following conditions: alcohol dependence, substance dependence, major depression, dysthymia, mania or hypomania, panic disorders and agoraphobia (anxiety), social phobia, specific phobia, and generalized anxiety. We analyzed the NESARC data using the generalized leastsquares estimation method that modeled the elapse (in years) between the onset of a condition and the year in which the last episode of symptoms was reported. To simulate “natural 72 recovery,” we estimated the elapsed time only for respondents who reported no treatment since onset. Each estimation model includes the following covariates: age at the interview, age at the onset of the condition, sex, and high school diploma status. In addition, the model on alcohol dependence includes the covariates of ever having substance dependence and ever having a DSM-IV diagnosis of mental illness; the model on substance dependence includes the covariates of ever having alcohol dependence and also ever having a DSM-IV diagnosis of mental illness; and the models for mental illness conditions each include the covariates of ever having alcohol dependence and substance dependence. The analyses were performed using the SAS procedure of SURVEYREG. SURVEYREG is specially designed for regression analyses involving sample survey data. The procedure allows for adjustments for complex sample designs, including sample stratification, clustering, and 73 unequal weights. The parameters shown on lines 60 through 63 in Exhibit B.3 are the parameters for a third degree polynomial for each prototype; for use in the simulation model, these are linear representations of the logistic models estimated with SAS. The determination of the effect size that is used for each year, ESty is computed with the short-run effect size, ESt, for each prototype evidence-based treatment, discussed elsewhere in this Appendix. These effect sizes were almost always obtained from studies with quite short follow-up periods, usually around a year. To account for the possibility that these short-run effect sizes might decay over the long run, we estimated decay rates, decayratet, for each prototype treatment. We describe how we obtained estimates for the decay rate in Appendix B2. In addition, also as described in Appendix B2, we multiplied the effect sizes by a factor, scaleupt, that is designed to reflect reduction in effect sizes that are likely to occur when smallscale programs are expanded significantly. B12. Sensitivity Analysis The model as described in this Appendix produces a unique result given the set of inputs listed. As we describe, however, there is a significant amount of uncertainty around many of the inputs. For most inputs to the model, we determine the range of uncertainty with the standard errors or standard deviations from relevant statistics of the underlying data for each parameter. For a few other parameters, we hypothesized low and high ranges to place bounds on our estimates of uncertainty. After we specified ranges of uncertainty on each of the inputs, we then used a simulation approach to determine how sensitive the final result is to these levels of uncertainty. To conduct the simulation, we used Palisade Corporation’s @RISK® simulation software. Using a Monte Carlo approach to the simulation, the software randomly draws from the userdesignated input variables after a particular type of probability distribution and its parameters have been specified for the input. We ran a Monte Carlo simulation for 10,000 cases. Exhibit B.5 shows input variables along with the specified probability distributions that we used in the simulation. 70 See footnote 5. http://niaaa.census.gov/ The term “recovery” refers to situations in which the last episode of symptoms had occurred no later than a year prior to the interview. It should be noted that this term is not used in the strict meaning as “cured” because in some situations the absence of symptoms before the interview could just be the “breathing” period between episodes. 71 72 73 ® SAS Institute Inc. 2004. SAS OnlineDoc 9.1.2. Cary, NC: SAS Institute Inc. 33 Exhibit B.5 The Benefits and Costs of Evidence-Based Treatment: Model Parameters Varied in the Monte Carlo Simulations Probability Distribution Type in @RISK ® See text for information about these parameters Evidence-Based Treatment: Adults With Alcohol, Drug, or Mental Illness Disorders Adults with a serious DSM alcohol disorder Adults with a serious DSM drug disorder Adults with a serious DSM mental illness disorder 1. Program Effectiveness Parameters Adjusted effect size after applying WSIPP* adjustments Estimated standard error for the WSIPP-adjusted effect size Expected annual rate of decay in effect size Standard error Expected diminishing returns to effect size with large scale ramp up (lower expected rate of decay) (higher expected rate of decay) Normal -.247 .021 -.062 .027 .750 1.000 .500 -.355 .035 -.164 .072 .750 1.000 .500 -.360 .058 -.176 .089 .750 1.000 .500 Normal $2,300 $500 $2,300 $500 $3,596 $782 Normal 5.55% 0.26% 2.05% 0.16% 3.80% 0.22% Normal 11.1% 0.4% 50% 75% 25% 14.7% 0.9% 50% 75% 25% 46.2% 3.5% 50% 75% 25% Normal -0.260 0.061 -0.262 0.059 -0.250 0.038 Triangular $4,496 10.0% $6,114 10.0% $13,799 10.0% Normal Triangular 2. Program Design Parameters Average annual program cost Standard deviation of average program cost 3. Prevalence Parameters Current (12-mo) prevalence of DSM disorder in this population cohort Standard error 4. Potential Population to be Treated Proportion of target population already treated with evidence-based program Standard error Proportion of the currently unserved target population that might realistically be served high low Triangular 5. Morbidity Parameters (earnings and household production) Employment outcomes =f(Disorder) Standard error 6. Health Care Costs Annual cost/ current abuser (adjusted to base year for real growth in costs) Assumed percentage (plus and minus) from the average cost 7. General Model Parameters Discount Rate Real annual rate of growth in earnings Real annual rate of growth in health care costs * Washington State Institute for Public Policy 34 High Discrete (equal %) Triangular Triangular .070 .023 .044 Medium .050 .013 .034 Low .030 .003 .024 For further information, contact Steve Aos at (360) 586-2740; email@example.com Document No. 06-06-3901 Washington State Institute for Public Policy The Washington State Legislature created the Washington State Institute for Public Policy in 1983. A Board of Directors—representing the legislature, the governor, and public universities—governs the Institute and guides the development of all activities. The Institute’s mission is to carry out practical research, at legislative direction, on issues of importance to Washington State.