Ban the Box, Criminal Records and Statistical Discrimination, Michigan Law, 2016
Download original document:
This text is machine-read, and may contain errors. Check the original document to verify accuracy.
LAW AND ECONOMICS RESEARCH PAPER SERIES PAPER NO. 16-012 JUNE 2016 BAN THE BOX, CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION: A FIELD EXPERIMENT AMANDA AGAN SONJA STARR THE SOCIAL SCIENCE RESEARCH NETWORK ELECTRONIC PAPER COLLECTION: HTTP://SSRN.COM/ABSTRACT=2795795 FOR MORE INFORMATION ABOUT THE PROGRAM IN LAW AND ECONOMICS VISIT: HTTP://WWW.LAW.UMICH.EDU/CENTERSANDPROGRAMS/LAWANDECONOMICS/PAGES/DEFAULT.ASPX Electronic copy available at: http://ssrn.com/abstract=2795795 Ban the Box, Criminal Records, and Statistical Discrimination: A Field Experiment Amanda Agan and Sonja Starr1 June 14, 2016 ABSTRACT “Ban-the-Box” (BTB) policies restrict employers from asking about applicants’ criminal histories on job applications and are often presented as a means of reducing unemployment among black men, who disproportionately have criminal records. However, withholding information about criminal records could risk encouraging statistical discrimination: employers may make assumptions about criminality based on the applicant’s race. To investigate this possibility as well as the effects of race and criminal records on employer callback rates, we sent approximately 15,000 fictitious online job applications to employers in New Jersey and New York City, in waves before and after each jurisdiction’s adoption of BTB policies. Our causal effect estimates are based on a triple-differences design, which exploits the fact that many businesses’ applications did not ask about records even before BTB and were thus unaffected by the law. Our results confirm that criminal records are a major barrier to employment, but they also support the concern that BTB policies encourage statistical discrimination on the basis of race. Overall, white applicants received 23% more callbacks than similar black applicants (38% more in New Jersey; 6% more in New York City; we also find that the white advantage is much larger in whiter neighborhoods). Employers that ask about criminal records are 62% more likely to call back an applicant if he has no record (45% in New Jersey; 78% in New York City)—an effect that BTB compliance necessarily eliminates. However, we find that the race gap in callbacks grows dramatically at the BTB-affected companies after the policy goes into effect. Before BTB, white applicants to BTB-affected employers received about 7% more callbacks than similar black applicants, but BTB increases this gap to 45%. 1 Princeton University and University of Michigan, respectively. The authors gratefully acknowledge generous funding from the Princeton University Industrial Relations Section, the University of Michigan Empirical Legal Studies Center, and the University of Michigan Office of Research, without which this study could not have taken place. We thank Will Dobbie, Henry Farber, Alan Krueger, Steven Levitt, Alex Mas, Emily Owens, Alex Tabarrok, David Weisbach, Crystal Yang and seminar participants at Princeton University, Rutgers University, the University of Chicago, the University of Michigan, UCLA, the University of Pennsylvania, the University of Toronto, the University of Virginia, the University of Notre Dame, the Society of Labor Economists Annual Meeting, and the American Law and Economics Association Annual Meeting for helpful comments. Finally, we thank every member of our large team of research assistants for their hard work and care, especially head RAs Louisa Eberle, Reid Murdoch, Emma Ward, and Drew Pappas, and our ArcGIS experts Linfeng Li and Grady Bridges. Electronic copy available at: http://ssrn.com/abstract=2795795 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION 1. Introduction In an effort to reduce barriers to employment for people with criminal records, more than 100 jurisdictions and 23 states have passed “Ban-the-Box” (BTB) policies (Rodriguez and Avery 2016). Although the details vary, these policies all prohibit employers from asking about criminal history on the initial job application and in job interviews; employers may still conduct criminal background checks, but only at or near the end of the employment process. Most BTB policies apply to public employers only, but seven states (including New Jersey) and a number of cities (including New York City) have now also extended these restrictions to private employers. These laws seek to increase employment opportunities for people with criminal records. They are often also presented as a strategy for reducing unemployment among black men, who in recent years have faced unemployment rates approximately double the national average (Bureau of Labor Statistics 2015).2 The theory underlying this strategy is straightforward: black men are more likely to have criminal convictions than other groups (Shannon et al. 2011), and having a criminal record is a substantial barrier to employment (Pager 2003; Holzer, Raphael, and Stoll 2006; Holzer 2007; Pager, Western, & Bonikowski 2009). Thus, a policy that increases the employment of people with records should disproportionately help minority men. This effort could have unintended consequences, however. In the absence of individual information about which applicants have criminal convictions, employers might statistically discriminate against applicants with characteristics correlated with criminal records, such as race. In this scenario, applicants with no criminal records who belong to groups with higher conviction rates, such as young black males, would be adversely affected by BTB policies. While some observational research provides support for this theory (see, for example, Finlay 2009; Freeman 2008; Holzer, Raphael, and Stoll 2006), it has never been tested experimentally. Moreover, whether statistical discrimination will occur in the context of BTB (which merely delays employer access to criminal convictions, rather than precluding it entirely) has never been tested at all. We investigate the effects of BTB laws via a field experiment. We submitted nearly 15,000 fictitious online job applications to entry-level positions before and after BTB laws went into effect 2 See for example Minnesota Department of Human Rights (2015): “The Ban the Box law can mitigate disparate impact based on race and national origin in the job applicant pool, and is one tool to help reduce these inequalities.” New York City’s public Ban the Box law was passed as part of the Young Men’s Initiative, an initiative designed to address disparities faced by young Black and Latino men (City of New York 2016). Civil rights organizations are also major supporters of Ban the Box movements (NAACP 2014, Color of Change 2015). 2 Electronic copy available at: http://ssrn.com/abstract=2795795 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION in New Jersey (March 1, 2015) and New York City (October 27, 2015). We sent these applications in pairs matched on race (black and white), which was our primary variable of interest. We also randomly varied whether our applicants had a felony conviction as well as two other characteristics that could also potentially signal criminal history to employers: whether the applicant has a GED, and whether the applicant has a one-year employment gap.3 Our study explores several key questions. First, we investigate whether employer callback rates vary by race and by felony conviction status, and whether there is an interaction between these effects. Second, we estimate how the availability of information about job applicants’ criminal records changes the racial gap in callback rates. Many employers, even absent BTB, choose not to ask about criminal convictions on employment applications, so we are able to draw cross-sectional comparisons between askers and non-askers in the pre-BTB period, as well as pre- and postcomparisons for the same employers before and after BTB. Our estimates of BTB’s effects exploit this cross-sectional and temporal variation in a triple-differences design. We estimate post-BTB changes in racial disparity after differencing out changes over the same time period among similar companies whose applications were unaffected by BTB. We also estimate the effects of having a GED and of a one-year employment gap. Finally, we assessed whether racial discrimination patterns vary based on the racial composition of the neighborhood employers are located in. Our experiment supports several key findings. First, white applicants overall received about 23% more callbacks compared to similar black applicants (a statistically significant difference of about 2.5 percentage points over a baseline of 10.6%, averaged across periods and criminal record statuses). Second, among employers that asked about criminal convictions in the pre-period, the effect of having a felony conviction is also significant and large: applicants without a felony conviction are 62% (5.2 percentage points over a baseline of 8.4%) more likely to be called back than those with a conviction, averaged across races. Third, in contrast to prior research (Pager 2003; Pager, Western, and Bonikowski 2009), we find no significant interaction between the effects of race and felony convictions. Fourth, although one might have expected that a GED (versus a high school diploma) or a 1-year gap in employment might have been disfavored or used by employers as a proxy for a criminal record, neither characteristic significantly affects callback rates. 3 We use “criminal record” and “felony conviction” interchangeably here; our experimental design varies whether employers have a felony conviction. Employers that ask about records on initial job applications overwhelmingly limit their questions to convictions (not arrests), and most limit them to felony convictions specifically. 3 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Our estimates of BTB’s effects on callback rates imply that BTB substantially increases racial disparities in employer callbacks. We find that BTB expands the black-white gap by about 4 percentage points, multiplying the gap at affected businesses by a factor of about six. In our main specification, before BTB, white applicants to BTB-affected employers received 7% more callbacks than similar black applicants, but after BTB this gap grew to 45%. This increase in racial inequality in callback rates could come from a combination of two sources. First, there could be a reduction in callbacks to black applicants with no criminal record, i.e. employers statistically discriminate against black applicants when they cannot see information about criminal history. In addition, there could be an increase in callback rates to white applicants with criminal records if employers statistically generalize that white applicants do not have records. Our results suggest some support for both of these mechanisms. Both explanations for the increasing gap involve forms of statistical discrimination, and provide reason to question the idea that BTB will reduce racial disparity in employment. When our results are broken down by jurisdiction, some interesting differences emerge. The overall effects of having a criminal record are larger in New York City than in New Jersey, where people without records receive 78% more callbacks (versus 45% in New Jersey). On the other hand, the main effects of race are much larger in New Jersey, where white applicants are 38% more likely to receive a callback (vs. a not statistically significant 6% in New York City). Further analysis suggests that this difference may be partly, but not mostly, explained by the city’s greater racial diversity. Businesses in whiter neighborhoods much more strongly favor white applicants, but even accounting for these differences, New York’s race gap in callback rates is considerably smaller. Meanwhile, the effects of BTB are fairly similar in both jurisdictions—favoring white applicants relative to black applicants—albeit operating on different pre-BTB baselines. This study makes several distinct contributions to the literature. First, this is the first empirical study of BTB’s statistical discrimination effects,4 and we hope it will inform ongoing legislative debates about BTB throughout the country. Second, removing information about criminal history on job applications allows us to use field-experimental methodology to contribute to the literature on statistical discrimination in employment, which has not generally used such methods.5 Although 4 One of the authors is currently carrying out observational research on BTB’s effects on public employers, detailed further below (Starr 2015). 5 See List (2004) for an experimental approach to statistical discrimination in another context, sports card trading. 4 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION our study is not a pure experiment (a key variable, whether the application asks about records, is not manipulated), our ability to perfectly observe and randomize all of our fictional applicants’ characteristics allows us to avoid many of the most likely threats to causal inference that affect purely observational research, and leaves us better equipped than are purely observational researchers to tease out the mechanisms underlying the effects we observe. Third, our assessment of geographic differences adds another dimension to the experimental literature on racial discrimination in employment; to our knowledge, no prior auditing study has assessed how differences in employer behavior vary based on neighborhood racial composition. Finally, we make a methodological contribution to the literature on auditing, which has for decades been a central tool for empirical research on discrimination in employment, housing, lending, and other areas. To our knowledge, this is the first study to use auditing to assess the effects of a policy, rather than to obtain a static picture of discrimination patterns. Because researchers cannot randomize the application of the policy itself, using auditing to assess policies requires combining the field-experimental approach with additional methods of causal inference— in this case, differences-in-differences analysis. We believe that combining auditing with quasiexperimental analysis of policy changes enriches the study of discrimination. 2. Background and Literature Review 2.1 Ban-the-Box Policies and their Motivations The “box” referred to in “Ban the Box” (and hereinafter in this paper) is the question on a job application form asking whether the applicant has been convicted of a crime – which is often accompanied by yes and no checkboxes. While BTB policies vary, all of them ban employers from asking such questions on application forms. The policies typically also bar employers from asking about records during an initial job interview. They do not, however, permanently bar them from performing criminal records checks. Instead, employers must delay these checks until a later stage in the hiring process: in New Jersey, that stage is anytime after the first interview, and in New York City it is after a conditional job offer is made. Some BTB laws also substantively restrict the role that criminal records can play in employers’ ultimate decisions (roughly paralleling existing federal anti-discrimination guidelines), but New Jersey’s and New York’s do not.6 6 New Jersey’s law affects only the “initial employment application process” (N.J. P.L. 2014, Ch. 32). Meanwhile, New York already had, long before the beginning of this study, a substantive restriction requiring employers to consider 5 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION BTB is often presented as an important tool for reducing racial disparity in employment, and especially for improving access to employment for black men (Pinard 2014, Southern Coalition for Social Justice 2013, Clarke 2012, and Community Catalyst 2013). Black unemployment levels are generally about twice those of whites (DeSilver 2013), so expanding black male employment is a priority for many policymakers and civil rights advocates (see, for example, NAACP 2014). This argument for BTB proceeds in several steps. First, black individuals are much more likely to have criminal records than are other groups. Brame et al (2014) find that by age 23, 49% of black men have experienced an arrest versus 38% of white men; Shannon et al. (2011) estimate that 25% of the U.S. black population has a felony conviction, compared with only 6% of the non-black population. Second, having a criminal record, especially a felony conviction, is a substantial barrier to employment (Holzer, Raphael, and Stoll 2006; Pager 2003; see Holzer 2007 for a review of studies). One can expect this employment hurdle to have a disparate impact on black men because they are more likely to have records.7 Finally, advocates argue that BTB will effectively improve access to employment for people with records. This step in the reasoning may not be so obvious, since BTB only delays rather than prevents employer access to criminal records. But BTB’s motivations are premised on a psychological claim: “Rejection is harder once a personal relationship has been formed” (Love 2011). The goal is to stop employers from making the premature judgment to throw out everyone with a record, and instead to encourage more nuanced consideration, which is believed to be more likely if employers have already met with the candidate (Pinard 2010). In short, the objective is to enable candidates with records to get their foot in the door. 2.2 The Potential for Statistical Discrimination There is, however, a plausible counterargument to the view that BTB will improve black male employment prospects. Economists have frequently suggested that in the absence of specific information about individuals (or where obtaining such information is costly), employers and other whether a conviction is job-relevant; this restriction is unchanged by BTB. N.Y. Correction Law Sec. 752. In any event, employers in all U.S. jurisdictions are subject to similar substantive restrictions at the federal level. The Equal Employment Opportunity Commission has for decades interpreted the Civil Rights Act of 1964 to bar employers from blanket bans on persons with criminal records, to avoid racially disparate impacts. According to EEOC, employers must consider “the nature and gravity of the offense or conduct; the time that has passed since the offense, conduct, and/or completion of the sentence; and the nature of the job sought” (EEOC 2012). 7 This is why EEOC interprets race discrimination law to constrain employers’ treatment of criminal records (EEOC 2012). 6 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION decision-makers are more likely to rely on statistical generalizations about groups (Phelps 1972; Arrow 1973; Aigner and Cain 1977; Fang and Moro 2011). In our context, this theory implies that if employers cannot ascertain at the outset which applicants have criminal records, they may use observable characteristics such as race to infer the probability an applicant has a criminal history, and this may trigger discriminatory treatment (Finlay 2009; Freeman 2008; Holzer, Raphael, and Stoll 2006). Thus, for example, young black men without criminal records could be hurt by BTB if employers assume that they are likely to have a record, based on assumptions about young black men generally. Of course, BTB does not permanently bar employers from obtaining record information, which could reduce the incentive to rely on demographic proxies. Still, employers may want to avoid the costs associated with interviewing and making tentative offers to candidates that they fear will ultimately be disqualified after the background check, especially if those search costs are high. The premise of the theory of statistical discrimination relies on the idea that the unobservable information is costly to obtain, not necessarily inaccessible (Phelps 1972; see also Stoll (2009) for an argument that BTB might trigger statistical discrimination). If BTB does trigger statistical discrimination against black men, it would subvert the policy objective of expanding their access to employment. Moreover, although statistical discrimination on the basis of race is sometimes defended as rational (if employers’ generalizations are accurate), it is plainly unlawful in the employment context. This prohibition reflects a policy judgment disfavoring racial generalizations and favoring expansion of workplace opportunities for historically excluded groups. Title VII of the Civil Rights Act of 1964 prohibits hiring discrimination on the basis of race as well as gender, and does not permit otherwise-illegal treatment to be based on statistical generalizations about groups, even if there is empirical support for the generalization.8 But these restrictions are famously difficult to enforce, and the fact that statistical discrimination would be an unlawful response to BTB does not mean it is impossible, or even unlikely. No prior study has yet assessed the potential statistical discrimination effect of BTB, although one of this study’s authors is currently conducting a parallel observational study focusing 8 For example, in City of Los Angeles Department of Water and Power v Manhart, 435 U.S. 702 (1978), the Supreme Court held that an employer could not rely, in formulating terms of a pension plan, on the well-founded actuarial prediction that women live longer. 7 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION on public employers.9 Outside the BTB context, several observational studies have suggested that lack of employer access to criminal records may encourage statistical discrimination (Bushway 2004; Holzer, Raphael, and Stoll 2006; Stoll 2006; and Finlay 2014). Holzer, Raphael, and Stoll (2006) and Stoll (2009) use survey data from establishments in four cities to show that employers who perform criminal records checks are more likely to hire African-Americans; the researchers interpret this finding as evidence of statistical discrimination. Bushway (2004) studies cross-state variation in accessibility of criminal records databases and finds that states with greater accessibility have smaller race gaps in employment. Finlay (2014) exploits temporal variation in states’ expansion of Internet criminal records databases and uses individual longitudinal data that includes criminal history; he finds that blacks without records have better employment outcomes under open records policies. However, Finlay (2014) also finds that the net employment effect of open records on young black men appears to be negative, suggesting that the benefits of open records to nonoffenders within that group may be outweighed by harms to offenders. Statistical discrimination has also been studied in contexts other than criminal records. For example, Wozniak (2015), relying on a similar theory, shows that legislation that allows drugtesting increases black employment, with the largest increases among low-skill black men. Autor and Scarborough (2008) find that a retail chain’s adoption of a pre-employment personality test did not hurt black employment success even though black candidates had lower scores; they interpret this as evidence that employers were statistically discriminating before they used the test. Clifford and Shoag (2016) show that bans on the use of credit checks by employers reduce black employment and employment of young people. 2.3 Auditing Research “Auditing” or “audit” studies are field experiments in which researchers randomly vary the characteristics of interest about a person with whom a subject interacts (for example, a job applicant). While some audit studies use actors for in-person communications, many use written or online communications (such as resumes and cover letters) in which the “person” in question does 9 Starr (2015, unpublished draft on file with author) uses the Current Population Survey and American Community Survey, exploiting temporal variation in the dates of cities’ and states’ adoption of BTB. Preliminary results using the CPS show a substantial increase in racial disparity in rates of being employed by local governments, but the analysis of the ACS shows no significant change. Both datasets have some limitations that might explain the differences, but it is not clear whether one or the other result is “right” (Starr 2015). In addition, we are also aware of a forthcoming working paper by Doleac and Hansen (2016) that will study the effects of BTB laws using CPS data; however, the draft was not available at the time of this posting. 8 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION not exist, so researchers can directly manipulate characteristics of interest. Such designs have been used to test employment discrimination on the basis of characteristics such as race, gender, length of unemployment spell, age, and type of postsecondary education (Neumark 1996; Bertrand and Mullainathan 2004; Lahey 2008; Oreopoulos 2011; Kroft, Lange, and Notowidigdo 2013; Deming et al. 2014; Farber et al 2015; Neumark et al 2015. In-person audits have been used by Pager (2003) and Pager, Western, and Bonikowski (2009) to explore the effects of criminal records on employment outcomes and its interaction with race, finding that criminal records have a heightened adverse effect on black applicants. For a review of auditing methods, see Riach and Rich (2002). Auditing can provide a stronger basis for causal inference than observational methods, because only the variables of interest are varied. Additionally, compared to lab experiments, audit studies provide stronger external validity, since they test real employer reactions. Despite its prominent role in discrimination research, auditing has to our knowledge never been used to study the effects of a policy on discrimination. Instead, it has been used to obtain a onetime snapshot of discrimination in a particular decision process. In our view, auditing holds considerable untapped potential as a tool of policy analysis, and we hope to demonstrate that potential. The principal challenge in auditing for policy analysis is that it is no longer a pure experiment. Applicant characteristics are randomized, but the policy variable is determined by nature, not by the researchers, and its applicability may be correlated with unobserved confounding variables (such as seasonal variations). Obtaining causal identification in this context requires combining the field-experimental method with another econometric method to filter out these potential confounds. We do so using triple-differences analysis. Because this approach involves estimating three-way interactions, it requires a larger sample than most auditing studies require, making it relatively resource intensive. However, it is otherwise quite straightforward. 3. Experimental Design We submitted online job applications on behalf of fictitious job applicants to low-skill, entry-level job openings both before and after BTB went into effect in New Jersey and New York City. New Jersey’s version of BTB, the “Opportunity to Compete Act”, was passed on August 11, 2014 and became effective March 1, 2015. We submitted applications in New Jersey in the preBTB period between January 31 and February 28, 2015 and in the post-BTB period between May 4 and June 12, 2015. New York City’s BTB law went into effect on October 27, 2015. We submitted 9 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION applications in New York City between June 10 and August 30, 2015 (the pre-BTB period) and between November 30, 2015 and March 31, 2016 (the post-BTB period). 3.1 Choosing Employers and Job Postings Our subjects were exclusively private, for-profit employers. We principally targeted chain businesses because such businesses are likely to have online job applications and to be subject to the NJ BTB policy, which exempts employers with fewer than 15 employees. We rely on two main sources for locating job openings. First, we searched snagajob.com and indeed.com, two large online job boards; snagajob.com focuses specifically on hourly employment. Second, with certain exceptions, we also directly searched the employment websites of chain businesses meeting certain size criteria in certain industries: restaurants, department stores, home centers, grocery and convenience stores, pharmacies, miscellaneous retail, service stations, and hotels/motels.10 We hired a large team of University of Michigan student research assistants to search for jobs using these methods, apply to them, and record information about the job applications. We directed them to look for jobs that were suitable for candidates with limited work experience, no post-secondary education, and no specialized skills. Such jobs are predominantly non-supervisory team-member jobs at fast food and other restaurants, grocery and convenience stores, and other retail establishments. We focus on these sectors because they almost universally use job applications (particularly online applications) rather than resumes as an initial screen of job applicants; employers that do not use applications do not have a “box” that can be banned. In addition, these sorts of jobs are likely to attract applicants with criminal records, who disproportionately tend to have relatively little work experience or post-secondary education. 10 In New Jersey, we applied to businesses with at least 30 locations and 300 employees in the state. In New York City, we applied to chains with at least 20 locations in the city, plus smaller chains if we had also applied to them in New Jersey. Employers that did not use online job applications were excluded, although the vast majority of chains meeting those size criteria do use them, as well as virtually all employers that advertise postings on Snagajob or Indeed. We also excluded a few chains due to extremely arduous online application processes (e.g., those that took our RAs more than an hour to complete). We excluded employers targeting an overwhelmingly female clientele, such as cosmetics companies. Finally, some employers required full SSNs on job applications. For ethical reasons, we wanted to avoid using potentially real SSNs, and thus assigned our applicants invalid SSNs (beginning with 9xx or 666). Some employers we initially tried to apply to had systems that automatically detected these invalid SSNs, and we excluded those businesses from further applications. It is possible that setting up such a system could be correlated with special interest in criminal records, such that excluding this pool means that our estimates of the effect of a criminal record will be lower than they would otherwise be. However, within the pool we did apply to, there was no correlation between whether employers asked for an SSN at all and whether they asked about criminal records. 10 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION 3.2 Applicant Profiles Our fictitious applicants are all male and approximately 21 to 22 years old.11 We created applicant profiles that included answers to a wide range of questions that employers could potentially ask, using the Resume Randomizer program created by Lahey and Beasley (2009). Our research assistants then filled out the applications based on those profiles. Each applicant profile included a name, a phone number, an address, an employment history, a unique email address, two references with phone numbers, information on high school diploma or GED receipt, a felony conviction status and information about the criminal charge, a formatted resume, and answers to many other routine application questions concerning job requirements, availability, and pay sought (minimum wage).12 The profiles were created in pairs, each consisting of one black and one white applicant. These pairs were assigned to the same store in the same time period. Our applicants were all similar on all but our randomly assigned treatment dimensions. In addition to race, those dimensions are: (1) Has felony criminal conviction or not a. (Conditional on conviction): convicted of property crime or drug crime (2) Has 1-year employment gap versus a 0- to 2-month gap (referred to as “no gap” below) (3) GED or High School Diploma These characteristics were randomized with equal (50%) probability. In addition to race, we chose to vary the employment gap and high school diploma status because they are also characteristics that hiring managers might perceive as correlated with criminal history.13 Race is indicated via the name of the applicant, as discussed further in Section 3.3 below. The crimes our applicants were 11 Due to legal restrictions on age discrimination, age and high school graduation year are rarely requested on job applications, so age can only loosely be inferred by the length of work history. 12 It was not possible for the applicant profiles to anticipate every question asked on the applications of all of the businesses to which we applied, especially as many applications require an extensive online personality or skills assessments. For this reason, we relied on the RAs’ judgment, but provided detailed training about what employers would likely ask and what they are generally looking for; we are confident that our RAs were capable of filling out these assessments in a satisfactory manner that would “clear the bar” and allow the applicant to be considered. 13 As of 2005, 13.6% of GEDs were issued in state and federal Prisons (Heckman and LaFontaine 2010). The relationship between GED, race, and criminal records is further addressed in the Discussion. The one-year employment gap is meant to signal potential time spent incarcerated or dealing with the criminal justice process. That an applicant may have a felony conviction and no employment gap is not implausible: of individuals charged with felonies in state courts, 62% are not detained before trial; 27% of those convicted receive no incarceration, and of those incarcerated 48% receive sentences of 1-3 months (Reaves 2013). In addition, the felonies we chose were relatively minor. 11 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION convicted of were relatively minor felonies – either property crimes (e.g., shoplifting, receiving stolen property, theft) or drug crimes (e.g., controlled substances possession). We chose 40 geographically distributed cities/towns in New Jersey and 44 neighborhoods throughout New York City’s boroughs to serve as “centers” where the applicants’ addresses would be located; each center then served as a base for application to nearby employers.14 All applicant addresses were in racially diverse, lower- to-middle-class neighborhoods. Other job applicant characteristics such as work history, address within center, high school name or GED program, and names of references were designed to have similar connotations, although they were randomly varied among a set of similar options (e.g., different high schools with similar demographic and academic profiles; employment history at different fast food restaurants) and forced to differ within pairs so as to disguise the similarity of the applications. Each applicant received a unique email account with the address format randomly varied. Phone numbers were assigned at the center/race/crime level and thus shared by multiple applicants, but in a way that almost entirely avoided using the same number more than once within any chain. For more details on profile contents and applicant characteristics, see Appendix A1. 3.3. Indicating Applicant Race Race is a central characteristic of interest in our study, and we signal race by the name of the applicant.15 To identify racially distinctive names, we used birth certificate data for babies born between 1989 and 1996 from the New Jersey Department of Health (NJDOH), which encompasses the cohort that would include our applicants. We then chose a set of first and last names that were racially distinctive (meeting threshold requirements for the percentages of babies given that name who were black or non-Hispanic white) and common (meeting threshold requirements for the total number of babies born with that name and race).16 Each applicant was then assigned a random first 14 This assignment method differed somewhat from New Jersey to New York City, due to differing geographic concerns. In New Jersey, we assigned each municipality in the state to its nearest center. For example, applicants from Princeton, NJ (one of our centers) applied to jobs in Princeton as well as in the nearby towns of East Windsor, Hightstown, Monmouth Junction, Plainsboro, Princeton Junction, and Skillman. These towns are all within 15 miles of Princeton. In New York City, because distances are much smaller generally, we prioritized distributing chain locations across centers (so that no chain received too many applications from the same neighborhood) and minimized distance within equal-distribution constraints, rather than in absolute terms. 15 This is a common strategy in auditing studies (Bertrand and Mullainathan 2004). 16 Because blacks are a much smaller fraction of the population, these thresholds varied by race: the minimum percentages were 80% for white first names, 85% for white last names, and 70% for black first and last names, while the minimum frequencies were 450 for white first names, 150 for white last names, 150 for black first names, and 100 for black last names. The white first names we used averaged 84% non-Hispanic white and 5% black, and the white last 12 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION name and random last name from the appropriate list. We expect that the combination of racially distinctive first and last names will produce a very strong racial signal: according to the birth certificate data, 96% of persons with first and last names on our “black” list are black, and 91% of persons with first and last names on our “white” list are white. A list of the names we used is provided in Appendix A2. One critique of using racially distinctive names to signify race in audit studies is that such names could also signal socioeconomic status, which employers may also believe to be correlated with productivity (Fryer and Levitt 2004). We note first that our applications provided a great deal of concrete SES-related information to employers, including complete work histories, education, current neighborhood, high school location, and wage sought. Employers thus hardly need to rely on names to draw SES inferences—whereas no other application characteristics signaled race, because those characteristics were randomized and were designed to be race-neutral. Nevertheless, to mitigate this concern we used only names falling below the socioeconomic median for whites (as measured by maternal education recorded on the birth certificate, the best available indicator), reducing the implied-SES gap between our white and black names. 17 In addition, because the names we chose were common, we avoided any perceived socioeconomic connotations that may be associated with the choice of unusual names or spellings. Although some SES gap remains, it is very similar to the overall SES gap between black and white citizens—that is, choosing distinctive names did not amplify the gap.18 Distinctively white or black names do not point to an individual being a high- or low-SES outlier within their race; in fact, such names are very common. In our birth certificate sample, 47% of black children have a racially distinct first name and 36% have a racially distinct last name (as we define distinctiveness, see footnote 17), while 35% of white children have a racially distinct first name and 65% have a racially distinct last names averaged 90% non-Hispanic white and 3% black. The black first names we used averaged 88% black and 3% non-Hispanic white; the black last names averaged 77% black and 17% non-Hispanic white. We eliminated a few first names that either were not distinctively male or that had strong associations with Islam or Judaism, so as to avoid confounding the effects of race with those of perceived gender or religion. A heavily overlapping name list would have been chosen had we classified names in the manner of Bertrand and Mullainathan (2004) or Fryer and Levitt (2004). 17 It was not possible to create a list of racially distinct names that are completely balanced on SES indicia, because virtually every distinctively white name averages higher than virtually every distinctively black name, due to socioeconomic stratification by race. 18 According to the birth certificate data, persons with first and last names that were both on our “black” lists had an average maternal education level that was nearly identical to the overall black average; persons with first and last names that were both on our “white” lists had nearly the same average maternal education as the overall white average. 13 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION name. Thus, to the extent employers make assumptions about SES based on racially distinctive names, these are assumptions that would affect a large fraction of real-world job applicants. 3.4 The Job Application Process Each RA was randomly assigned one or more of our geographic centers in which to search for jobs via the above-described methods, and applied for those jobs using profiles from that center; the profile order within and between pairs was random. While submitting the job application, they filled out a spreadsheet that indicated, among other things, which profile was used, the date and time of the submission, the name of the chain being applied to, the name of the position, address of the location, and whether the application asked about criminal history. With some time lag, a second application was submitted to each store. Most applicant profiles (approximately 59%) were sent to only one business. However, we sometimes used the same profile pairs to apply to multiple nearby locations of the same chain, as real-world applicants might do; our criteria for grouping the applications in this way differed between New Jersey and New York City, producing more grouping in New Jersey.19 The post-BTB application procedure was essentially the same, except that we began with the chains that we had already identified and applied to in the pre-period. Each specific store that we applied to at least once in the pre-period was assigned a new pair of profiles. The RAs were assigned to submit applications to these stores in an order that was designed to make the length of time between members of each pair roughly mirror what occurred in the pre-period. Stores thus received up to four applications total, one pair in each period. It was sometimes not possible to send a complete set of four applications to an establishment. The primary reason for this was that the store was hiring in one period but not the other. In addition, a few RA assignments were not completed before BTB’s effective date, leaving some applications unsent; this especially occurred in the New Jersey pre-period, our first wave of applications, which had to be completed relatively quickly. In New Jersey, we filled in these gaps 19 In New Jersey, we were concerned that the same hiring managers might cover multiple locations of chains and might become suspicious upon noticing groups of applicants coming within a short time from the same nearby town. Accordingly, we used the same applicant profiles for all locations that were assigned to a given center. In New York, our concerns were different: the centers are not towns and likely appear less distinctive to managers, and we had more available time before BTB’s effective date, so we were able to space out the timing of our applications. Thus, in New York we chose to increase power by sending each application to only one location, except for the largest five chains (in which we sent each applications to up to two or three stores). We forced addresses and phone numbers to differ within chains, such that chains would not receive multiple applications from the same ones. 14 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION in the post-period whenever possible, and identified some new opportunities on snagajob.com. In New York City, our pre-period wave represented a quite comprehensive search, so we limited the post-period wave to the same locations that we had sent at least one application to in the pre-period; there was some attrition due to unavailable jobs in the post-period. As a result, while the pre-and post-period samples are almost identical in size, the percentage of applications that are from New York City was higher in the pre-period (60% versus 52%), and moreover, the composition of chains and stores is not identical across periods. We address these concerns below. 3.5 Measuring Outcomes The main outcome of interest is whether an application receives a voicemail or email from an employer requesting that the applicant contact them or requesting an interview. We refer to this outcome as a callback (although it includes emails). For some alternative specifications, we focus on responses that specifically requested an interview. However, this outcome variable is subject to measurement error because employer messages often do not specifically mention an interview even if they are seeking to interview the applicant. Thus, our preferred specification uses the callback as the outcome. Phone calls and emails were tracked for eight weeks from the application date. In New Jersey, our pre-BTB data collection ended on April 25, 2015 (for the last applications sent);20 our post-BTB data collection ended on August 6, 2015.21 In New York City, our pre-BTB data collection ended on October 26, 2015, and our post-BTB data collection ended on May 26, 2016. 4. Summary Statistics and Main Effects of Applicant Characteristics on Employer Callbacks We submitted a total of 15,220 applications, of which 14,640 are included in our analysis sample.22 These include 6,401 applications in New Jersey and 8,239 in New York City. The 20 Note that although this is considerably after BTB went into effect, all of the applications were submitted before it went into effect, which meant that the applications did contain the criminal records question (except for businesses that voluntarily omitted the question even prior to BTB). Because our outcome of interest is the employer response to the initial application (not subsequent stages of employer decision-making, such as ultimate hiring decisions), consideration of these applicants should therefore not be affected by BTB. 21 RAs posing as the applicants responded to employer messages by leaving brief messages thanking them but stating that the applicant was no longer available. We had no further communications with the businesses and, per IRB constraints, did not collect any information about the individuals we interacted with. 22 The remaining 580 observations (3.8% of those we sent) were dropped for several reasons. First, when an entire chain was applied to only in the pre-period or only in the post-period, we had no way to code whether the application had the criminal record “box” in the other period, so the treatment variable could not be coded. Second, some stores had inconsistencies within one or both rounds as to whether the box was present. The most common reason for these inconsistencies was early precompliance with BTB (which in both jurisdictions was announced several months before it went into effect), occurring before we sent the second application but after the first. Another reason was RA mistakes in interpreting the job application form—usually answering the criminal history 15 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION summary statistics and results presented in the tables and figures below combine both jurisdictions; in Appendix A3, we replicate several of the tables and figures for New Jersey and New York separately. The applications were sent to 4,292 stores (that is, establishments) in 296 chains. We begin with summary statistics and then analyze the main effects of our randomly varied characteristics on employer callbacks. 4.1 Summary Statistics Summary statistics are presented in Table 1a, by period and overall. As expected, approximately 50% of our applications had each of our randomized characteristics of interest. However, the prevalence of our other variable of interest—whether the application asked about criminal records—was determined by nature (that is, by the chains), not by randomization. Among our pre-period applications, 36.6% had a required criminal record question (the “box”). In the postperiod, 3.6% still had the box (“noncompliers”), leaving approximately 33% of the sample as “treated” observations: employers that had the box before BTB, but not after. Overall, 1,715 applications received callbacks, a rate of 11.7% overall. This rate was slightly higher in the post-period (12.5% vs. 10.9%), and lower in NYC than in NJ (9.4% vs 14.7%; see Appendix Tables A4 and A5). Among the callbacks, about 55% specifically mentioned an interview. The overall callback rate for white applicants was 12.9%, and 10.5% for black applicants. In both periods, callback rates were much more similar across the other randomized characteristics (GED/H.S. diploma and employment gap). Although the race gaps appear fairly similar across time periods (2.1 percentage points in the pre-period and 2.8 percentage points in the post-period), they represent averages that do not differentiate treated and untreated observations, and mask large changes occurring at treated stores, as discussed below. 4.2. Effects of Applicant Characteristics on Callback Rates We begin by assessing the underlying employment patterns that BTB is principally designed to address. How much of an effect does having a criminal record have on employer callback rates? question even when they were not required to because they missed a disclaimer telling New Jersey or New York City applicants not to answer the question. In either event, when the two observations from the same store and round were in conflict, we discarded the observation that was an outlier from the overall chain norm. The effect was to drop RAmistake observations, and in the precompliance cases, to drop the later, non-box observation. Third, we also dropped some businesses (about 1% of the sample) that appeared, mysteriously but presumably due to an administrative mistake, to add the box after BTB, and therefore could not be coded as 0 or 1 on the Treated variable. We add these back in in a robustness check below, with the coding of -1. 16 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION How much does this vary by race? Table 1a did not show a breakdown of callback rates by criminal record status, because criminal record is unobserved by employers for 63% of our applications even in the pre-period, making that breakdown not very informative for the full sample. Instead, we show separate summary statistics in Table 1b limited only to pre-BTB period observations where the application had the box. Among companies with the box, callback rates are about 60% higher for applicants without criminal records (about 5.1 percentage points, over a base rate of about 8.5%). Applicants with drug convictions had similar callback rates to those with property crime convictions—perhaps surprisingly, as one might have expected employers to be particularly concerned about potential employee theft. However, all the crimes we used were of similar legal severity—relatively low-level felonies. As Table 1b further shows, for employers with the box in the pre-period, the callback rate advantage for applicants without records is slightly larger for white applicants (5.7 percentage points, or 69% higher than the base rate of 8.3%) than for black applicants (4.5 percentage points, or 52% higher than the base rate of 8.6%). Overall, when employers ask about records, we see essentially no race gap in callback rates: the white average is 11.1% and the black average is 10.9%. Figure 1 puts those numbers into perspective by comparing them to the callback rates for white and black applicants to employers without the box in the pre-period. Among these employers, white applicants have a 3.1-percentage-point (or 33%) callback rate advantage (12.5% vs. 9.4%; p<0.001). The overall callback rates at both groups of employers are essentially identical (11%), but the separation between white and black applicants is seen only at the employers who do not ask about criminal records. This is suggestive evidence for the statistical discrimination theory, although other differences between these employers could potentially underlie these cross-sectional differences; the triple-differences results below provides a stronger basis for causal inference. Table 2 provides multivariate regression estimates of the main effects of race, record, GED status, and employment gap on callback rates. These estimates closely parallel what we see in the summary statistics, which is not surprising given that all the applicant characteristics were distributed randomly. All the results shown in Table 2 are for both periods combined (unlike Table 1b and Figure 1, which were for the pre-period only), but the regression results look similar if only the pre-period observations are used. Columns 1 and 2 show the results of regressions run in the full sample of 14,640 cases. They differ in that the Column 2 regression adds chain fixed effects (with the smallest chains grouped by business category) and center fixed effects, which make little 17 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION difference. Both imply that white applicants are on average about 2.4 percentage points more likely to receive a callback from an employer, which corresponds to a statistically significant 23% increase in callbacks over the 10.5% black baseline (p<0.001). Note that the estimated criminal record effect in these regressions (about 1.5 percentage points) substantially understates the magnitude of the real criminal record effect, because in four-fifths of the sample, criminal record was not actually conveyed to the employer. Columns 3 and 4 parallel the regression in Column 2, but they are limited to observations without and with the box, respectively. (Although the time periods remain combined, the Column 4 regression’s observations are almost entirely from the pre-period, since only 3.6% of businesses retained the box after BTB.) The criminal record variable is removed from the non-box Column 3 regression because no criminal record information was conveyed. The advantage to white applicants appears only in the non-box sample, in which it is about three percentage points (Col. 3); there is no race gap at stores with the box (Col. 4). Column 4 also shows a statistically significant 5.2-percentage-point criminal record effect in the box sample (p<0.001). This represents a 63% higher callback rate for persons without records, compared to the 8.2% baseline for persons with records in this sample. Column 5, which is also limited to observations with the box, shows that this effect is similar for property crimes and drug crimes. Finally, Column 6 adds an interaction of the race and criminal record variables, within the box sample only. The negative criminal record effect is 1.5 percentage points larger for white applicants—among applicants without criminal records, whites have a slightly higher callback rate, but among applicants with criminal records, they have a slightly lower callback rate. This interaction is not statistically significant, but its sign is nonetheless interesting given that earlier, smaller auditing studies (Pager 2003; Pager, Western, and Bonikowski 2009) had found a strong interaction in the opposite direction. In every specification and sample, having a one-year employment gap and obtaining a GED rather than a high school diploma have little effect on employer responses. Point estimates for both are close to zero, and the GED coefficient varies in sign across the specifications and samples. 4.3 Alternative Specifications and Samples: Race and Criminal Record Effect In Table 3A, we show the race effect from several alternative specifications and samples. All combine “box” and “non-box” observations from both time periods, and all include chain and 18 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION center fixed effects. They are variants on the Table 2, Column 2 main effects regression, the “white” coefficient of which is reproduced in Column 1 of Table 3 for comparison purposes. In Column 2, we use interview request as the dependent variable rather than callback, which identifies observations in which a voicemail or email specifically mentioned an interview. Although the effect appears superficially smaller (1.4 percentage points), it is actually very slightly larger as a percentage of the (lower) black baseline rate: whites receive 24% more messages specifically mentioning interviews than blacks do (and 23% more callbacks). In Column 3, we alter the company fixed effect. The main specification grouped chains with fewer than 3 locations (or 12 observations) according to business type (such as fast food restaurants or clothing stores). Column 3 shows that the estimate is robust to using an ungrouped company fixed effect. In Columns 4 and 5, we show the race effect separately estimated for the New Jersey and New York City subsamples, respectively. Here we see a dramatic difference: the “white” effect is far larger in New Jersey (4.5 percentage points versus 0.7 percentage points), and is statistically insignificant in New York City. The overall callback rate is considerably higher in New Jersey (14.7% compared to 9.4%), but not nearly enough so to explain this difference: in New York City, whites receive about 8% more callbacks than equivalent black applicants, while in New Jersey they receive about 37% more. In the Appendix A3 and A4, we reproduce in full Table 2 and the other main tables and figures for New Jersey and New York separately, and we discuss the geographic differences further below. In Table 3B, we show an analogous of alternative analyses of the main effect of having a criminal record within the box sample, paralleling the estimate from Table 2, Column 4, which is reproduced in Column 1 of Table 3B. As with the “white” effect, the criminal record effect appears smaller in percentage-point terms when interview request is used as the outcome (Table 3B, Col. 2), but this effect is actually larger in relative terms. Applicants without records receive 67% more messages specifically mentioning interviews, and 61% more callbacks overall. Column 3 shows that the effect estimate is essentially unchanged by substituting the ungrouped company fixed effects. Finally, Columns 4 and 5 show that the criminal record effect is just slightly larger in percentage-point terms in New York City than in New Jersey—but in light of the city’s lower callback rate, it is much larger in relative terms. Applicants without records receive 45% more callbacks than those with records in New Jersey; in New York City, applicants without records receive 78% more callbacks. 19 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Note that clustering in all regressions is on the chain, for reasons discussed further in Section 5.5 below. Standard errors on the race and criminal record effect estimates are not substantially affected by clustering on the store or the geographic center instead (p<0.001 in all specifications). 4.4 Further Investigation of Geographic Differences in Callback Rates by Race The difference in the White effect between the New Jersey and New York City subsamples, shown in Table 3A, is quite striking, and motivates further analysis. One plausible explanation is that New York City is more racially diverse than New Jersey. Per Census data, it has a larger black population share (22%, vs. 15% in New Jersey), a smaller non-Hispanic white population share (32%, vs. 57% in New Jersey), and larger populations of other ethnicities, especially Hispanic (29%, vs. 19% in New Jersey) and Asian (14%, vs. 9% in New Jersey). New Jersey is itself a fairly diverse state, and its racial composition far more closely tracks the country as a whole, so if racial composition explains the differences in observed disparities, the New Jersey results might be more representative of broader patterns. In Table 4, we directly test whether local racial composition at a more localized level—the census block group of the business address23—influences the White effect, and whether this in turn can explain the different patterns in New York City and New Jersey. The racial composition of the neighborhood population could potentially influence employer racial discrimination in various ways. Employers could seek to appeal to local customers’ own-group preference, or perhaps to pick applicants who “fit in” based on the racial composition of current staff. Hiring managers could themselves be of different races in different neighborhoods, and this might influence their perceptions of applicants. We lack data on managers’ or staff members’ race, so we cannot differentiate these mechanisms, but we can test their cumulative effect. The regressions in Table 4 add various interactions to the main-effects regression. The center fixed effects are omitted because other geographic variables are included instead. The other variables from Table 2, Column 2 are all included in the regressions, although only the coefficients on the White variable, the geographic variables, and their interactions are shown in the table. Before incorporating the racial composition data, Column 1 of Table 4 first shows that the White x 23 When job postings were not specific to a location with an identifiable address, we used the averages for the city or town instead in New Jersey or the zip code or borough (depending on the detail given in the posting) if in New York City. 20 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION NJ interaction is significant (p<0.001) and large (3.8 percentage points), while the estimated White effect in New York is only an insignificant 0.7 percentage points, consistent with the split-sample results above. It also shows that overall callback rates for New Jersey, as noted above, were higher. Column 2 drops the NJ variables, and instead includes the non-Hispanic white population share of the census block group where the store is located, and interacts that share with White. (The racial composition variables are labeled Store CBG %White and Store CBG %Black in the tables to reflect this precise definition, but for simplicity in this text, we refer to them as PercentWhite and PercentBlack.) The interaction effect is very strong, indicating that employers in whiter neighborhoods are much more likely to discriminate based on race. Its coefficient (4.9 percentage points, p<0.01) represents the increased advantage of white applicants when one goes from an entirely nonwhite neighborhood to an entirely white neighborhood (both of which are found in our sample). The true effect, of course, may be nonlinear. Note that white neighborhoods have higher callback rates as well: the main effect of PercentWhite is 3.4 percentage points (p<0.01). Column 3 shows an analogous analysis of the effects of the black population share (PercentBlack). Its interaction with White is even larger (6 percent, p<0.001). This regression suggests that in entirely nonblack neighborhoods the White effect is large and positive (3.2 percentage points, p<0.001), while in entirely black neighborhoods, the White effect is about the same size but negative (about -2.8 percentage points). Of course, these effects do not in practice offset one another in the overall employment market, because (given the lower black population share), there are many more white (and nonblack) neighborhoods than there are black neighborhoods. The median employer neighborhood in our sample is 5% black, and only 8% of employer neighborhoods are more than half black. In Columns 4 and 5, we add back the NJ and White x NJ terms to the regressions from Columns 2 and 3 respectively, to assess whether racial composition differences can explain the White x NJ interaction. For the most part, they do not—and nor does the NJ effect explain away the racial composition effect. In each of the combined regressions, the White x NJ interaction is almost as large as it was in Column 1 (3.3 and 3.6 percentage points, respectively. In Column 4, the PercentWhite x White interaction is 3.3 percentage points, and in Column 5 the PercentBlack x White interaction is -4.9 percentage points. Column 6 shows that the White x NJ interaction persists when both sets of racial composition interactions are added to the regression. It appears that the PercentWhite*White interaction disappears--however, because PercentBlack and PercentWhite are 21 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION strongly collinear, the distinct effect of each (and their interactions) may be difficult to estimate meaningfully when both are included. The effect of local racial composition on racial discrimination patterns is important in its own right, and has not been investigated by prior auditing studies. It suggests that at least one of the mechanisms described above is at play—all forms of own-group preference. Still, it does not appear to explain most of the difference between New York and New Jersey. This is likely because, as it turns out, the racial compositions of employer neighborhoods in our New Jersey and New York samples are much less different from one another than one might have expected based on the jurisdictions’ overall demographics. For example, the median percent black for both jurisdictions is 5% (far lower than either jurisdiction’s black population share) although the mean differs (16% for New York, 11% for New Jersey). Employers in both jurisdictions, especially New York City, appear to be very disproportionately concentrated in whiter (and less black) neighborhoods. Note that we test these effects only at the census block group level, but the city’s overall greater diversity might nonetheless influence racial discrimination patterns, even if employers are not located in especially diverse neighborhoods—for example, existing staff and managers need not be drawn from the immediate neighborhood. 5. Effects of Ban-the-Box on Racial Discrimination In this section we turn to our policy-effects analysis: what is the causal effect of BTB on racial discrimination in employer callbacks? In order to answer this question we combine our field experiment with a difference-in-difference-in-differences strategy. This strategy exploits the two sources of variation in employer knowledge about criminal records before the callback: crosssectional variation in the pre-period between applications with the box and those without, and timeseries variation caused by the law change which required companies that asked about criminal records to stop doing so. 5.1 Difference-in-Difference-in-Differences Estimation Strategy One problem with comparing callback rates in two different time periods is that seasonal variation, other state- or city-level policy changes, and general economic trends could all effect callback rates in different periods, differences unrelated to the BTB policy itself. To account for this possibility, we employ a difference-in-differences-in-differences approach. This method exploits the fact that not all employers ask about criminal records even in the pre-BTB period 22 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION (indeed, the majority do not). We treat such stores as a control group, comparing whether changes in the effects of race after BTB goes into effect differ between stores that have the box in the preperiod and those that do not. This will “difference out” effects of seasonal variation or other temporal differences unrelated to BTB, leaving us with an estimate of the causal effect of the BTB policy on employer callback difference by race or other characteristics of interest. Similarly, purely cross-sectional comparisons between employers with and without the box could be confounded by unobserved differences between those employers unrelated to the presence of the box. But the triple-differences analysis will difference out those unrelated differences as well, so long as they are time-invariant over the period in question. This method implies the following general difference-in-difference-in-differences estimating equation: 𝑐𝑎𝑙𝑙𝑏𝑎𝑐𝑘 = 𝛼 + 𝛽! 𝑊ℎ𝑖𝑡𝑒 + 𝛽! 𝑃𝑜𝑠𝑡 + 𝛽! 𝑇𝑟𝑒𝑎𝑡𝑒𝑑 + 𝛽! 𝑊ℎ𝑖𝑡𝑒 𝑥 𝑃𝑜𝑠𝑡 + 𝛽! 𝑊ℎ𝑖𝑡𝑒 𝑥 𝑇𝑟𝑒𝑎𝑡𝑒𝑑 + 𝛽! 𝑃𝑜𝑠𝑡 𝑥 𝑇𝑟𝑒𝑎𝑡𝑒𝑑 (1) + 𝛽! 𝑇𝑟𝑒𝑎𝑡𝑒𝑑 𝑥 𝑊ℎ𝑖𝑡𝑒 𝑥 𝑃𝑜𝑠𝑡 + 𝜖 𝑃𝑜𝑠𝑡 is an indicator for the post-BTB period, 𝑐𝑎𝑙𝑙𝑏𝑎𝑐𝑘 is an indicator for whether the applicant received a positive-response callback from the employer, 𝑇𝑟𝑒𝑎𝑡𝑒𝑑 is an indicator for whether the criminal record question on the store’s job application form changed after BTB. Treated is coded at the individual store level. Observations from a given store are coded as not treated (𝑇𝑟𝑒𝑎𝑡𝑒𝑑 = 0) if the store never had “the box,” and also in the rarer case of stores that had the box and failed to remove it after BTB. Observations are coded as treated if the store had the box but removed it after BTB.24 In most specifications, we also add a vector of control variables that accounts for the possibility of random imbalances in other applicant or application characteristics (GED, employment gap, criminal record, and geographic center). In Equation (1) above, the main effect of interest is the triple-difference coefficient, 𝛽! , which tells us how the employer callback gap for whites versus blacks changes differentially after BTB for treated versus non-treated stores. A positive coefficient implies that BTB favors white 24 The sample used for this analysis is slightly smaller than the sample for the main-effects analysis above because we dropped a small number of observations—about 1% of the sample—for which Treated could not be coded as either 0 or 1, because the chain moved from not having the box to having it after BTB (the opposite of the expected direction of change, seemingly due to administrative mistakes). 23 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION applicants relative to black applicants, that is, that treated employers become relatively more likely to call back white applicants after the box is removed. An additional issue is that we did not apply to exactly the same set of stores or chains in the pre- and post-period—as discussed above, it was not always possible to send all four intended applications to each store. If the employers that we applied to in the post-period happened to have different patterns of discrimination from those in the pre-period (in a way that differed across treated and untreated employers), we could mistakenly interpret a compositional effect as an effect of BTB. We have two approaches for addressing these compositional differences across periods. First, in some specifications, we substitute interacted chain fixed effects instead of some of the “Treated” terms in the equation above, as follows: ! 𝑐𝑎𝑙𝑙𝑏𝑎𝑐𝑘 = 𝛼 + 𝛽! 𝑊ℎ𝑖𝑡𝑒 + 𝛽! 𝑃𝑜𝑠𝑡 + 𝛽!! 𝐶ℎ𝑎𝑖𝑛! + 𝛽! 𝑊ℎ𝑖𝑡𝑒 𝑥 𝑃𝑜𝑠𝑡 !!! ! + 𝑊ℎ𝑖𝑡𝑒 𝑥 ! 𝛽!! 𝐶ℎ𝑎𝑖𝑛! + 𝑃𝑜𝑠𝑡 𝑥 !!! (2) 𝛽!! 𝐶ℎ𝑎𝑖𝑛! !!! + 𝛽! 𝑇𝑟𝑒𝑎𝑡𝑒𝑑 𝑥 𝑊ℎ𝑖𝑡𝑒 𝑥 𝑃𝑜𝑠𝑡 + 𝜖 where 𝑖 indexes chains, and 𝐶ℎ𝑎𝑖𝑛! represents a series of dummy variables for the chains in our sample.25 Because “treated” status occasionally varies between stores (usually because some chains give franchisees a choice of application platforms, or because a chain’s BTB compliance differed between New Jersey and New York City), we assign separate 𝐶ℎ𝑎𝑖𝑛 fixed effects to treated and untreated subsets of such chains. The result is that the 𝐶ℎ𝑎𝑖𝑛 fixed effects perfectly parallel the Treated variable: Treated status follows directly from the 𝐶ℎ𝑎𝑖𝑛. The equation above substitutes the main effect of 𝑇𝑟𝑒𝑎𝑡𝑒𝑑 with 𝐶ℎ𝑎𝑖𝑛 fixed effects, and likewise substitutes 𝑇𝑟𝑒𝑎𝑡𝑒𝑑 𝑥 𝑊ℎ𝑖𝑡𝑒 𝑥 𝑃𝑜𝑠𝑡 with parallel sets of interacted fixed effects. However, it keeps the main effect of interest, the triple-differences estimate, in its easier-to-interpret form of 𝑇𝑟𝑒𝑎𝑡𝑒𝑑 𝑥 𝑊ℎ𝑖𝑡𝑒 𝑥 𝑃𝑜𝑠𝑡. This term represents the average change in racial disparity due to BTB: in effect, a weighted average of what the coefficients would be if 𝑊ℎ𝑖𝑡𝑒 and 𝑃𝑜𝑠𝑡 were instead triply interacted with 𝐶ℎ𝑎𝑖𝑛, completing the substitution. 25 The smallest chains (fewer than three locations or 12 total observations) are combined into industry-category groups; these chains represent about 9% of the sample. Original coding is used in a robustness check below. 24 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION The chain-fixed-effects specifications account for differences in composition across periods by chain, but not by individual store (or by the geographic distribution of stores). Moreover, they do not provide easy-to-interpret coefficients on the main effects of White, Treated, and Post or their two-way interactions. We thus also offer a simpler approach for confronting the compositional differences: we conduct the analysis within the subset of stores to which we did send exactly four applications: one white/black pair in each period. Fortunately, we were usually able to do so, and so this “perfect quad” sample contains 11,118 observations, or 76% of our full sample. When using the perfect quad sample, the concerns about different distributions across chains, stores, or jurisdictions disappear (and no controls for these variables are necessary), because the sample is perfectly balanced between the pre- and post-periods. The simple triple-differences analysis can thus be used, and all the coefficients are easy to interpret; the disadvantage is some loss of power. In any of these analyses, identification of 𝛽! as a causal effect relies on the assumption that, absent BTB, trends in employer callback differences by race would have been the same for treated and untreated stores (stores that had the box in the pre-period and those that did not). Unfortunately, our data are not long enough to compare pre-period trends. However, we believe the assumption is plausible. For a vast majority of stores in our sample (even those that are franchised), the job applications are standardized nationally at the chain level, with built-in variations accommodating local differences in BTB laws.26 Thus, the decision to include or not include the box on the application is made at the chain level, whereas callback decisions are made at the individual store level by store managers, or in some chains by local managers who supervise a small subset of locations. In that sense, whether a store has the box should be exogenous to the decision-makers we are studying. Moreover, there is no qualitative reason to believe that these chains differ in any way that would affect hiring trends in a racially disparate way. After all, to pose a threat to identification, hiring differences would have to be racially disparate in a way that differs over the time between our pre- and post-period applications (about four months on average). Note that not having the box does not generally reflect lack of interest in criminal records; chains with and 26 To comply with BTB laws, applications that normally have the “box” will usually ask a question similar to “Are you applying in Rhode Island, Hawaii, Massachusetts, California, or Minnesota?” If one clicks “yes,” the criminal conviction question will not appear. Alternatively, the conviction question will be preceded by instructions telling the applicant not to answer if applying in certain jurisdictions. So the treatment we are studying generally takes the form of the national chain adding New Jersey or New York City to these lists of BTB jurisdictions on the applications. 25 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION without the box, before and after BTB, routinely do back-end background checks (and their applications usually warn applicants of this fact). 5.2 Temporal Differences in Racial Disparity at Treated Stores We start descriptively with Figure 2, which compares pre- and post-BTB call back rates among treated employers—that is, those that had the box in the pre-period but then removed it to comply with BTB.27 Just as with Figure 1 (the cross-sectional comparison), Figure 2 (the temporal comparison) suggests that when companies don’t see applicants’ criminal records, they are more likely to discriminate based on race. In this sample, in the pre-period, white applicants both with and without records have a slightly higher callback rate than equivalent black applicants do: for applicants without records, the white and black rates are 13.8% and 12.7% respectively, and for those with records, the white and black rates are 8.8% and 8.4%, respectively (Figure 2). Averaging these subgroups together, the overall pre-period callback rates in this sample were 11.3% for whites and 10.5% for blacks. However, in the post-period, this quintuples in size, and white applicants receive 36% more callbacks than blacks do: the white callback rate is 15.0%, and the black callback rate is 11%. This figure does not, however, take into account potential seasonal or temporal variation between the pre- and post-period. The difference-in-difference-in-differences results below will “difference out” temporal variation in racial discrimination among employers whose applications never had the box and thus were unaffected by BTB, as discussed above. As we will see, this differencing out only strengthens the implication that BTB encourages racial discrimination. 5.3 Differences-in-Differences-in-Differences: Raw Percentages Before showing regression estimates, we start with raw percentage differences. Table 5 summarizes the changes in callback rates by race for treated and untreated stores before and after BTB went into effect. Each cell in Table 5 is itself a difference: the callback rate for black applicants minus the callback rate for white applicants. The “treated” column replicates what we already saw in Figure 2: at treated stores, the “white” advantage grew by 3.2 percentage points (from 0.7 percentage point to 4.0 percentage points) after BTB. The “not treated” column shows what happened at the same time at other stores whose applications were unaffected by BTB (mostly 27 The figure looks very similar if done only within the “perfect quad” sample. 26 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION because they did not have the box to begin with). At these stores, the “white” advantage declined very slightly, from 2.7 percentage points to 2.2 percentage points. When we further difference out the temporal differences in racial differences at untreated stores, we get a difference-in-differences-in differences figure of 3.7 percentage points. That is, the black-white gap grew by 3.7 percentage points more at the treated stores after BTB, relative to the untreated stores. This is a large increase, given that baseline callback rates are low; the average callback rate for black applicants in this sample is 10%. Below the line, we show a the tripledifferences calculation for treated and untreated observations in the perfect quad sample which is balanced on the chains and stores we applied to in the pre- and post-period 4.2 percentage points, a similarly large effect. 5.4 Triple-Differences Regressions Table 6 shows regression-adjusted triple-differences estimates across several specifications and samples. The effect of principal interest is on the top line, Post x Treated x White. Across specification, the estimates are economically large and significant (ranging from 3.6 to 4.1 percentage points, which amounts to a multifold increase in the underlying race gap). Our estimates here are somewhat less precise than the main-effects estimates discussed in Section 4, because triple-differences analyses demand much larger samples than analyses of main effects or even twoway interactions do in order to provide equivalent statistical power to estimate effects of a given size. Even so, all of these estimates are statistically significant (p<0.05), with p-values generally around 0.04. All of our regression estimates are quite similar to the basic difference-in-differencein-differences analysis in Table 5, which is unsurprising given that the applicant characteristics are randomized. Columns 1 and 2 show the simple triple-differences regression with the Treated, Post, and White variables interacted (per Equation 1), with and without controls for the other randomized applicant characteristics (GED, employment gap, and criminal record) as well as center fixed effects. Adding these controls increases the triple-differences coefficient slightly, from 3.7 to 4.1 percentage points. This analysis does not, however, account for the above-discussed differences in composition of the sample across time periods. We begin to address these in Column 3, which parallels Column 2 but substitutes interacted chain fixed effects for the Treated variable and its twoway interactions (per Equation 2). This analysis accounts for differences in the representation of the various chains in the pre- and post-period, and the main effect of interest declines slightly, to 3.6 27 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION percentage points. It bears noting that the White, Post, and Post*White estimates do not have a meaningful interpretation in this regression because the total effects of those variables are diffused among the interacted fixed effects. Column 4 then further account for differences in the individual stores represented in the preand post-period samples by limiting the analysis to the “perfect quad” sample. In this sample chains and centers are perfectly balanced across time periods and race, so there is no reason to include the chain or center fixed effects. Accordingly, we can use the simple triple differences specification, retaining from Column 2 only the controls for GED status, criminal record, and employment gap, since these might have randomly been slightly imbalanced even among the “perfect quads.” The effect estimate remains similar: 4 percentage points. In this sample, the estimated race gap at the treated stores goes from 0.7 percentage points before BTB to 4.7 points after, after differencing out changes at untreated stores. Again, to put this estimate in perspective, one must compare it to the baseline callback rate: other things equal, whites receive 6.7% more callbacks than similar black candidates do when employers are able to observe criminal records, but they receive about 45.2% more callbacks than similar black candidates when employers cannot observe records. In short, these analyses provide evidence that BTB increases racial discrimination in employer callbacks. Prior to the adoption of BTB, racial disparities are somewhat larger among the stores that do not have the box. After BTB, that difference flips. The growth in the “white” effect after BTB appears to multiply the race gap at affected stores by a factor of between five and seven; this factor varies slightly across specifications and samples, mainly because of variations in the small estimated pre-BTB race gap. In Appendix A5, we recreate the above analysis substituting GED or employment gap for White to explore whether employer responses to these characteristics, which are also correlated with a criminal record, change after BTB. The triple differences coefficients for both GED and employment gap are not significant. For the employment gap, however, the point estimates are nontrivial (around 2.5 percentage points; Table A5.2), albeit imprecise, and their signs go in the anticipated direction that statistical discrimination theory would predict: the negative effect of the employment gap increases when employers lose criminal record information. In the GED analysis, the point estimates are also negative but smaller, and very close to zero in the full-sample fixedeffect analysis (Table A5.1, Col. 3). So we cannot characterize this as even suggestive evidence of statistical discrimination on the basis of the GED. 28 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION 5.5 Alternative Specifications and Samples: Effects of BTB Our results are quite robust to alternate specifications. Table 7, Panels A and B, shows robustness checks and alternative samples corresponding to our estimates for the full sample and the “perfect quad” sample respectively. Only the triple-differences coefficient is shown. We base these variations on what we consider the main specifications for each sample, which are found in Columns 3 and 4 of Table 6. For the full sample, because of our concern about compositional differences between periods, we prefer the specification that includes the interacted chain fixed effects, and use that as the basis for the robustness checks. The triple-differences coefficient from Table 6, Column 3 is accordingly reproduced in Column 1 of Table 7A for comparison purposes. Meanwhile, the robustness checks for the “perfect quad” sample are based on the Table 5, Column 4 specification, and its triple-differences coefficient is reproduced in Column 1 of Table 7B. Columns 1 through 6 of both panels parallel one another, while Columns 7 and 8 of Panel A show additional checks that are not relevant to the “perfect quad” sample. Note at the outset that the coefficients and p-values are fairly similar for all variants except for columns 5 and 6 of each panel, which show results for New Jersey and New York City separately and are much less precise. In a few of the other specifications the p-values are above 0.05, but barely, representing only a small loss of precision or slightly reduced effect size; all pvalues are between 0.04 and 0.06, other than in the NJ-only and NYC-only regressions. Column 2 in both panels replaces the callback outcome variable with the interview variable. In percentage point terms, the estimate becomes slightly smaller (but still significant) in the full sample, and is essentially unchanged in the “perfect quad” sample. Again, however, the recorded “interview” rate was much lower (6.3% overall in the full sample, versus a “callback” rate of 11.7%)—so the effect on “interview” rates was actually quite a bit more dramatic in relative terms. That said, because we suspect that the vast majority of callbacks were in fact seeking interviews (even if they did not specifically say so), we consider the callback variable the better measure. Columns 3 and 4 in both panels alter subjective choices that we made about whether to exclude certain problematic observations. In Column 3, we add back in a group that we excluded from the main triple-differences analyses: “reverse complier” stores that had no box before BTB, but mysteriously (apparently due to administrative mistakes) added it after BTB. “Treated” cannot 29 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION be coded as 0 or 1 for these observations, but here we code it as -1, reflecting the reversal of the usual treatment direction.28 The effect size is slightly smaller in both samples and the p-value is slightly above 0.05 in the perfect-quad specification. In Column 4, we exclude a small number of observations or quads (about 0.4% of each sample) in which an RA made a mistake and answered a “box” question that she was not required to answer, or vice versa.29 Excluding them leaves both samples’ estimates virtually unchanged, though the perfect quad sample p-value again rises slightly above 0.05. Columns 5 and 6 in both panels divide the sample between New Jersey and New York City, respectively. The large reduction in sample size renders these analyses underpowered for the purpose of estimating triple differences, and thus these estimates are quite imprecise. The New Jersey point estimate is larger in percentage-point terms, but not much so in relative terms, once one accounts for New Jersey’s substantially higher callback rate (14.9% in the full sample, versus 9.4% in New York City). As a proportion of the respective samples’ callback rates, New Jersey’s fullsample point estimate is only slightly higher than New York’s, and New Jersey’s “perfect quad” point estimate is slightly lower than New York’s. In any event, because of their imprecision, one ought not to give much interpretive weight to the jurisdictional differences in the point estimates (whereas the jurisdictional differences in the main effects of race, discussed above, are clear). In Panel A, Columns 7 and 8 show two additional variants on the full sample analysis that alter the chain fixed effects and their interactions. In the main sample, the smallest chains (with under 12 observations total, or three stores) had been grouped based on business-type category (such as fast food restaurants or clothing stores). Column 7 instead uses individual chain fixed effects regardless of company size. Column 8, meanwhile, divides the chain fixed effects into New York and New Jersey subsets of each chain. Both changes add a large number of fixed-effect indicators to each regression and reduce precision slightly, but the point estimates remain similar. Clustering in all regressions shown in the tables is on the chain, because whole chains are likely susceptible to serially correlated shocks. We observed quite different callback rates by chain, 28 Note that the relationship between treatment and the passage of time is inverted for these observations, making this specification diverge from a standard triple-differences analysis. This is the main reason we excluded them. 29 The main sample had already dropped RA-error cases when they created inconsistencies in the coding of the treatment variable within stores, but kept them when the same error was made consistently within the store; we then coded the treatment variable according to how the RA interpreted the application, since that tracked the information about criminal records that the RA provided or did not provide to the employer. 30 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION as well as some chains that had distinct increases or reductions in callback rates or in job-posting availability in one or more of the four time periods in which we sent applications. The chain also encompasses the smaller units according to which the applications we sent were grouped. That is, we sometimes sent the same set of four applications to multiple locations of the same chain (especially in New Jersey, where we did so for all locations within the same center), but never to different chains. If one clusters on the geographic center instead (another dimension along which one could anticipate possible correlated shocks), the p-values for our main specifications are slightly higher in the full sample (0.054) and slightly lower in the perfect quad sample (0.024), and if one clusters on the individual store (ignoring correlations between chains), they are slightly higher in both samples (0.05 and 0.06, respectively). 6. Discussion and Conclusion Our results support BTB’s basic premise: when employers ask about them, criminal records pose an obstacle to employment. However, our findings also provide evidence of a serious apparent unintended consequence of BTB: increased racial discrimination against black men. These findings suggest a difficult dilemma for policymakers. Here, we discuss their limitations and implications further, as well as those of our results on the main effects of race. 6.1 BTB and the Effect of Criminal Records The key premise of BTB is that when employers ask about criminal records, people with records will have a much harder time getting their foot in the door. Although this seems intuitive, it can be difficult to quantify with observational research—but our field experiment provides very clear evidence of the serious obstacle to employment that criminal records pose. Applicants without records received 61% more callbacks than identical applicants without records did when employers had the box. And this is despite two facts that may have mitigated this effect. First, our applicants with records had minor records (a single conviction of a nonviolent drug or property crime, more than two years prior, with no incarceration history). Second, we applied mainly to positions that one might expect, in general, to be comparatively welcoming to people with records—for example, crew member jobs in restaurants. The practical effect of the criminal-record penalty might be offset to some degree by the fact that most employers in the sectors we studied do not have the criminal-records box even absent BTB. However, even when employers do not have the box on their applications, they are free 31 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION (absent BTB) to ask about records at an interview and to check records at any time; even with BTB, they are free to do so later in the application process. So if employers disfavor people with records, this effect may be present to some degree at later stages of the process even among non-BTB employers—stages our study does not assess. For BTB’s advocates, the good news in our findings is that employers comply with it, and thus BTB effectively eliminates criminal-record effects on employer callback rates for identical applicants. Fewer than 5% of employers retained the box in the post-period, a few months after BTB’s effective date. This means that for our applicants with records, BTB worked: those records were never conveyed to employers before the callback decision was made. Note, however, that we were unable to study the effect of BTB (or of criminal record or race) on actually getting a job, only initial employer responses. Perhaps BTB might not change employment rates after all, if firms are reluctant to hire applicants with a record even after they “get their foot in the door” (for a similar point on discrimination against the long-term unemployed, see Jarosch and Pilossoph (2015)). Still, while this is a substantial limitation, BTB is meant precisely to impact the initial stage of the hiring process, and so it is an important question whether doors do, indeed, open—and whether BTB brings about unintended consequences at the same initial stage. 6.2 Main Effects of Race Our results also confirm a clear advantage of white applicants, who receive 23% more callbacks compared to otherwise identical black applicants. This finding is consistent with those of nearly all prior auditing studies, so it should not surprise readers, although it is useful to confirm it in a newer sample and a setting (online job applications) which has hardly been studied but is central to the modern job market. Our estimate of white applicants’ advantage is somewhat less dramatic than most prior auditing studies have found, but as with the criminal record, our setting is one in which lesser race effects might have been expected. Online applications involve no personal interactions (and indeed may be initially narrowed down by software before a hiring manager ever sees them), and our applications gave no racial signals other than the name. Moreover, the job categories to which we applied are ones in which young black men are relatively well represented; one might expect black applicants to face lesser hurdles there than in fields where they would be a smaller minority. 32 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION This apparent racial discrimination could reflect a number of specific mechanisms: (1) statistical discrimination based on expectations concerning criminality (for companies that do not have the box, or in the post-BTB period); (2) statistical discrimination based on expectations concerning other productivity-related factors; (3) attempts to appeal to the discriminatory tastes of a customer base; and (4) pure taste-based discrimination unrelated to job performance expectations. A critique of auditing studies has been that they usually do not allow researchers to distinguish taste-based and statistical mechanisms of discrimination (Neumark 2011; Heckman and Siegelman 1993). Our research design offers some traction on this question, in that it helps to disentangle the first mechanism from the others, but we cannot disentangle the other three mechanisms. However, all four of these mechanisms amount to illegal racial discrimination, and all four conflict with the policy objective of expanding black male unemployment. Regardless of the specific causal pathway, then, our findings should be troubling to many policymakers, and are a reminder of the very substantial persistence of racial discrimination in hiring despite its legal prohibition. Given the prior literature, one surprise in our analysis is that the main effect of race does not pervade all segments of our sample. The advantage of white applicants is quite small when employers have the box, and it is quite small overall in New York City. Among employers with the box in New York City, the black callback rate was actually higher (10.2% versus 8.4% for whites, though this difference is not statistically significant). Moreover, our findings demonstrating a strong interaction of applicant race and neighborhood racial composition also indicate that racial discrimination is less prevalent (or may even be reversed in direction) in neighborhoods that are less white—although it also suggests larger degrees of racial discrimination in whiter neighborhoods. All of this variation suggests that racial discrimination in hiring, while prevalent, is not ubiquitous and may be avoidable—although we cannot yet fully explain why New York City is more successful than New Jersey in avoiding it, as demographic differences do not entirely explain the difference. 6.3. Effects of BTB on Racial Discrimination BTB appears to substantially increase racial discrimination against black men—indeed, by more than a factor of six in our main specifications. At BTB-affected employers, white applicants went from being 7% more likely to receive a callback than similar black applicants to being 45% more likely. This consequence is clearly unintended, as BTB is often presented as a strategy for increasing access to employment for black men. 33 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION We believe that the randomized experimental design, in combination with the tripledifferences analysis, provides a strong basis for interpreting our estimates as causal effects of BTB. The randomization means that we avoid most of the potential interpretive challenges that observational researchers encounter: our black and white applicants to all business types in all locations and periods have the same qualifications and characteristics. Any remaining threats to identification would have to come from unobserved differences that (1) affect applicants to treated and untreated businesses differently (2) in ways that differ by race and (3) this difference must differ across time periods as well. Although it is of course possible that (independent of BTB) some such difference might exist, there is no obvious candidate for what it might be. This is especially so because the time period between the pre- and post-periods is short—the two groups of quite similar businesses are unlikely to have greatly diverged from one another in their racial discrimination patterns in just a few months—and because we see approximately the same triple-differences effect in New Jersey and New York City, even though the pre- and post-periods in those two jurisdictions were seasonally nearly opposite to one another.30 We note that there are at least two plausible mechanisms that would explain this result. The first is statistical discrimination against black men: although black men with records could be helped by BTB, this effect could be swamped by negative effects for black men without records because absent the information employers treat them as if they have a high probability of having a record (Finlay (2014) concluded similarly in his research about the availability of online criminal records). Indeed, given that we gave our applicants fairly minor criminal records, it is even possible that some of our black candidates with records would have been better off revealing them (so that a more serious record was not assumed). A second mechanism focuses on BTB’s benefits for white applicants. Perhaps for some subset of employers, either black race or a criminal record are enough to push marginal candidates out of consideration. Such employers would be expected to treat white applicants with records more favorably after BTB, but their treatment of black applicants with records would not change, because black applicants without records already were not getting callbacks. The mechanism for these employers’ racial discrimination need not primarily relate to expectations about criminal records—it could be based on the other reasons identified above: pure prejudice with no statistical 30 In New Jersey we went from winter to late spring/early summer; in New York we went from summer to winter. 34 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION basis, appeals to a discriminatory customer base, or perhaps statistical discrimination on the basis of some other factor besides criminal record. This theory suggests that BTB could allow white applicants with records, in essence, to take advantage of the racial advantage that other white candidates have. It is a statistical discrimination theory as well, insofar as it requires employers to assume that white applicants likely do not have criminal records. But it suggests a more complicated story, implying that other mechanisms of discrimination may also play a role. These mechanisms are not mutually exclusive, and our results suggest that both likely contribute. At BTB-affected employers, after differencing out trends at unaffected employers, black applicants see their callback rates fall by two percentage points after BTB, while white applicants see theirs rise by two percentage points. These estimates are suggestive that both mechanisms are at work, although we lack the statistical power to disentangle them completely. (To truly tease out these pathways, we would need to add a fourth difference to our triple-differences analysis—that is, whether applicants have a record—which would require an enormous sample to do precisely.) And in any event, regardless of which explanation primarily drives our result, both suggest that BTB may not do the job that many of its advocates are hoping it will do: expanding access to employment for black men. One alternative causal theory is that BTB might affect treated businesses’ applicant pools, by encouraging more applicants with records to apply. If this is so, then even though our fictional applicants are the same in both periods, their competition is not, potentially affecting callback rates. But to explain our triple-differences estimates, changes in the competition have to affect our black and white applicants differently—and it is not obvious why this would be the case. If the mechanism involves statistical discrimination based on assumptions about records, then it is simply a variant on the theories we have already proposed. Indeed, whatever employers’ reasoning, if the theory is that BTB causes changes in the applicant pool that somehow cause employers to treat black applicants more adversely than identical whites, then it does not threaten our causal inference that BTB increases racial discrimination—it simply provides another mechanism by which it might do so. A variant of this concern is that BTB might affect untreated businesses’ applicant pools in some way (presumably reducing the number of applicants with records, as they apply to treated businesses instead) that leads them to increase callbacks of black applicants relative to whites. This possibility is more of a threat to causal identification because it would mean the control is not really 35 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION untreated. But changes to the untreated employers’ applicant pool are likely to be relatively subtle, because for many (probably most) applicants there is no necessary tradeoff between applying to treated and untreated businesses. In addition, given that the untreated employers lack the box both before and after BTB (and after BTB cannot ask about records even at interviews), it seems that many would be unlikely to notice changes in the percentage of their applicants with records, especially if those changes are not drastic. Employers would have to notice or anticipate such a change, and update their race-specific expectations and decision-making accordingly, very quickly in order to affect our results; our post-period applications were sent an average of less than three months after BTB’s effective date. Moreover, again, the change in competition would have to affect our black and white applicants differently, and it is not clear that it would. Nor is there empirical reason to suspect that it does: the estimates in Table 5 strongly suggest that the tripledifferences effect is being driven by an increase in racial disparity among treated employers, not a reduction among untreated employers. In any event, the effect of BTB on applicant pools (of either set of employer) may well be mitigated if applicants do not know what employers have the box before they are nearly done with the application (the box usually appears as one of the last screens). Some applicants with records might well gain such information before applying, but we suspect that this knowledge is at least not ubiquitous, in part due to the challenges we faced finding it. Despite considerable effort, we were unable to find resources listing employers with and without the box prior to conducting our resource-intensive data collection, and we were ourselves surprised to learn what a large share of employers did not have it. Applicants would also have to know about BTB, as well as its effective date (actual passage of BTB in both jurisdictions came months earlier, before our pre-period). There is a more direct way in which BTB might affect untreated employers, however: we identified employers as untreated based on their job applications, but BTB also governs the interview. So it is possible that it could encourage even untreated employers to statistically discriminate as well: knowing that they cannot ask records questions in the interview might make them less likely to interview candidates that they think might have records. However, if anything this possibility should mean our triple-differences estimate is downward biased, because BTB encourages statistical discrimination at both sets of employers, while we are measuring only the difference. In addition, in New Jersey employers are permitted to do background checks immediately after the interview (and even in New York City, where a conditional job offer must be 36 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION made, this could potentially occur in quite short sequence), so this concern for subsequent delay seems relatively minor—it is not a dramatic difference to find out about a record shortly after the interview rather than during it, since the time spent on the interview would already have been invested. We therefore think the best explanation for the triple-differences estimate is that BTB encourages statistical discrimination against black applicants and/or in favor of white applicants. Although such discrimination is illegal and against public policy, one could still be interested in asking: is it rational, in the sense of reflecting accurate expectations by employers about who is likely to have a criminal record? Or are employers relying on inaccurate stereotypes about black criminality? It is difficult to assess the rationality of employer decisions because there is much we do not know: for example, the costs to employers of interviewing an applicant who turns out to have a disqualifying criminal record, and on the other hand the costs of inadvertently failing to interview a candidate (due to assumptions about his record) who would have been the best choice. That said, there is good reason to believe that employers are relying on assumptions that exaggerate real-world racial differences in conviction rates. It is difficult to find useful statistics on the percent of specific populations with felony convictions – the National Longitudinal Survey of Youth 1997 (NLSY97) offers one data source, albeit with a fairly small sample size for this purpose. An initial point is that although absolute black/white differences in felony conviction rates are large (Shannon et al. 2011), they are much smaller once one conditions on other applicant characteristics that employers can observe. Indeed, this is so even once one simply limits the pool to young men with relatively limited education. Our calculations from the NLSY97 show that amongst men between the ages of 18 and 25 without any higher education degrees, 29.4% of black men had a criminal conviction between the ages of 18 and 25, whereas 24.7% of white men did. Our black and white applicants are identical on a range of other characteristics as well—work history, neighborhood, and so forth—which one would expect to narrow the gap further. And yet employers who are provided with a great deal of individualized information about our applicants appear to nonetheless be giving considerable weight to race as a predictor of criminality. One possibility is that employers engage in statistical discrimination in a far less nuanced way than rational-choice economic theory would predict—they may rely on a general impression that black rates of involvement with the criminal justice system are higher in absolute terms, without any specific sense of whether these differences persist after conditioning on the relevant set 37 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION of observed characteristics. It would not be surprising if employers made assumptions about black applicants’ likely criminality, even if those assumptions are not well founded in fact. Lab experiments on implicit biases have consistently found that most Americans make such assumptions subconsciously (see, for example, Eberhardt 2004; Nosek et al. 2007), and such mechanisms may not involve an accurate comparison of conditional probabilities. Further support for this theory comes from the contrast with our results on the GED versus high school diploma distinction; we did not find that BTB significantly increased the weight employers placed on that distinction. Nor, indeed, do employers place significant weight on this variable at all, even at non-box stores. And yet having a GED in lieu of a diploma is actually a much stronger predictor of criminal convictions than race is, conditional on the same observables. In the NYLS97, among young men with no college degrees, 43% of those with a GED have a conviction by age 25, whereas only 18% of those with a high school diploma have one. This contrast suggests that whatever employers’ cost-benefit calculus about interviewing people with records, they must either be irrationally overweighting race as a signal, underweighting education, or both. Employers also give no apparent weight (before or after BTB) to year-long employment gaps, despite the possibility that this might be associated with arrest or incarceration (or might otherwise signal that the applicant is a less appealing job prospect). 6.4. Policy Implications BTB may open doors to some applicants with records, but this gain comes at the expense of another group that faces serious employment challenges: black men. BTB is often presented as a way of increasing black male employment, but most black men do not have criminal convictions, and BTB risks harming black men without records by preventing them from signaling that fact to employers. This is a serious unintended consequence, but it is not necessarily dispositive as to BTB’s merits. Policymakers will have to evaluate how to weigh this risk versus BTB’s potential benefits, and also to consider whether there are strategies that could simultaneously be pursued that might successfully mitigate this disadvantage. Even if one simply wishes to evaluate BTB’s race-related effects (setting aside other policy concerns), the picture is somewhat complex. While in our sample BTB’s apparent effect on the race gap was fairly dramatic, an important unanswered question is how large an effect this phenomenon will have on real world job applicants. One limitation of auditing studies generally is that they do not directly provide estimates of changes in actual markets (Heckman 1998). In the real world, 38 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION applicants are not divided 50/50 between identical black male and white male candidates (and no other groups), with 50% of each group having a record. Our study suggests that BTB should be expected to substantially help applicants with records, at least at the initial callback stage, and in the real world black men have records at higher rates. This point means that even if BTB increases racial discrimination by employers, it does not necessarily follow that it will increase racial disparity in employment on balance. It could simultaneously be true that BTB helps black men with records (by eliminating record-based discrimination in callbacks), while hurting black men without records (by increasing racial discrimination), and the net effect on black male employment would depend on the size of each effect and the size of the respective groups they affect. And this calculus may vary as BTB is applied to different markets and places—employers’ treatment of both race and criminal records may vary considerably, as our comparisons of New Jersey and New York City illustrate. That said, some back-of-the-envelope calculations suggest that at least in contexts similar to the one we studied, the net effect may be to enlarge the black-white employment gap. Consider again 25-year-old men without college degrees: per the NLSY97, the black and white conviction rates are 29.4% and 24.7%, respectively. Suppose all such men were subject to changes in employer callback rates paralleling the pattern in Figure 2 (the raw pre- to post-period changes at treated employers)—a pattern that actually slightly understates the growth in racial discrimination that our triple-differences regression analyses found. Callback rates increased by 2.6 percentage points for black men with records, and declined by 1.7 percentage points for black men without records. Meanwhile, for white men with records, callback rates increased by 7.2 percentage points, and for white men without records they actually rose also, by 1.2 percentage points. (Callback rates increased at all stores in this period—an effect differenced out in the triple-differences analysis—so this preponderance of gains does not tell us anything about BTB’s effects. The relative rates are the focus of this calculation.) Applying these changes to the real-world distribution of records among young men without college degrees implies that overall black callback rates would fall by 0.4 points, while overall white callback rates would rise by 2.8 points— a net rise of 3.2 percentage points in the black-white callback rate gap (more than a quarter of the overall callback rate for the sample). This example suggests that even after offsetting the effect of eliminating criminal-recordbased discrimination, the increase in racial disparity due to BTB could be considerable. In addition 39 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION to the differential effects on white and black applicants without records, part of the reason for this is that it is white applicants with records who appear to benefit more substantially from BTB than black applicants with records do.31 Of course, a full analysis of real world effects would have to account for the fact that white and black men are not the only groups competing for jobs. We chose to focus on white and black men only because further subdividing the sample would have presented challenges in terms of statistical power. But women and men of other racial groups could be affected, and such effects could be avenues of future research. Moreover, while auditing studies point to a mechanism, observational studies can help to further explore how that mechanism plays out given the actual distribution of candidates. Policymakers might also consider whether there are other interventions that BTB could be combined with to reduce its adverse effects on black candidates. Race-based statistical discrimination in hiring is unlawful, and if the hiring discrimination laws were effectively enforced or operated as an effective deterrent, BTB could not have this unintended consequence. This, to be sure, is easier said than done, but the intuition behind BTB perhaps suggests one plausible innovation: asking employers to blind themselves to names in addition to records. The racial-disparity implications are not the only policy consideration surrounding BTB and whether our results imply that the policy is unsuccessful depends, of course, on what policymakers seek to maximize. To the extent that advocates and policymakers hoped this BTB would reduce racial inequality in employment opportunities, it appears to be doing quite the opposite. However, policymakers might reasonably endorse it on the ground that people with records are a group in acute need of a leg up, regardless of race. If jobs discourage crime, society may also have a special interest in providing that help for public safety reasons. Our study does not seek to inform every aspect of the policy debate surrounding BTB, but we do find that as a racial-disparity-reduction strategy, it appears to have unintended consequences. One complicating factor is that not every applicant in the real world has a racially distinctive name (only about half do), perhaps reducing the relative impact of the racial-discrimination effect in comparison to the record-discrimination effect. However, this point may be offset by the fact that real-world applicants may also have other signals of likely race on their job applications, such as their neighborhood of residence or high school; our fictional applications included no such signals, as everything was randomized among a set of fairly race-neutral options. 31 40 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION References Aigner, D.J. and Glen G. Cain. (1977). “Statistical Theories of Discrimination in Labor Markets”, Industrial and Labor Relations Review 30. Autor, D.H. and D. Scarborough. (2008). “Does Job Testing Harm Minority Workers? Evidence from Retail Establishments,” The Quarterly Journal of Economics 123(1): 219-277. Bertrand, M. and S. Mullainathan. (2004). “Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination”, American Economic Review 94(4): 991-1013 Brame, R., S.D. Bushway, R. Paternoster and M. G. Turner. (2014). “Demographic Patterns of Cumulative Arrest Prevalence by Ages 18 and 23,” Crime & Delinquency 60(3): 471-486. Bushway, S. (2004). “Labor Market Effects of Permitting Employer Access to Criminal History Records,” Journal of Contemporary Criminal Justice. Special Issue on Economics and Crime 20: 276-291. City of New York (Jan 2016), “Young http://www.nyc.gov/html/ymi/html/justice/justice.shtml#ban Men’s Initiative: Justice” Clarke, H. (December 20, 2012). “Protecting the Rights of Convicted Criminals: Ban the Box Act of 2012” Washington Post. Clifford, R. and Shoag, D. (2016). “No More Credit Score: Employer Credit Check Bans and Signal Substitution” Unpublished Manuscript Color of Change (November 2, 2015). “Civil Rights Group Responds to the ‘Ban the Box’ Executive Order” http://colorofchange.org/press/releases/2015/11/2/civil-rights-group-respondsban-box-executive-orde/ Community Catalyst. (December 2, 2013). Banning the Box in Minnesota—and across the United States, http://www.communitycatalyst.org/blog/banning-the-box-in-minnesota-and-across-theunited-states#.UuG1__Yo46U. Deming, D., N. Yuchtman, A. Abulafi, C. Goldin and L. Katz (September 2014). “The Value of Postsecondary Credentials in the Labor Market: An Experimental Study,” NBER WP #20528 DeSilver, D. (August 21, 2013). “Black Unemployment Rate Is Consistently Twice That of Whites”, Pew Research Center, http://www.pewresearch.org/fact-tank/2013/08/21/through-goodtimes-and-bad-black-unemployment-is-consistently-double-that-of-whites/ Equal Employment Opportunity Commission (EEOC). (2012). EEOC Enforcement Guidance 915.002. Consideration of Arrest and Conviction Records under Title VII of the Civil Rights Act of 1964. Eberhardt, J.L. et al. 2004. “Seeing Black: Race, Crime, and Visual Processing,” Journal of Personality and Social Psychology 87:876. 41 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Fang, H. and A. Moro. (2011). “Theories of Statistical Discrimination and Affirmative Action: A Survey” in Handbooks in Economics: Social Economics eds J. Benhabib, M. Jackson, and A. Bisin Farber, H., D. Silverman, and T. von Wachter. (2015, Sept 17) “Factors Determining Callbacks to Job Applications by the Unemployed: An Audit Study” Unpublished Manuscript. Accessed at: http://www.irs.princeton.edu/sites/irs/files/event/uploads/audit_hf09.pdf Finlay, K. (2009). “Effect of Employer Access to Criminal History Data on the Labor Market Outcomes of Ex-Offenders and Non-Offenders”, in Studies of Labor Market Intermediation, 89 (David H. Autor, ed.). Finlay, K. (2014). “Stigma in the Labor Market”, Unpublished Manuscript Freeman, R. (2008). “ Incarceration, Criminal Background Checks, and Employment in a Low(er) Crime Society,” Criminology & Public Policy 7: 405-412 Fryer, R.G. Jr and S.D. Levitt (2004). “The Causes and Consequences of Distinctly Black Names”, The Quarterly Journal of Economics 119(3): 767-805 Heckman, J.J. (1998) “Detecting Discrimination.” Journal of Economic Perspectives 12(2):101-116 Heckman, J.J. and P.A. LaFontaine. (2010). “The American High School Graduation Rate: Trends and Levels.” Review of Economics and Statistics 92(2): 244-262. Heckman, J., and P. Siegelman (1993). “The Urban Institute Audit Studies: Their Methods and Findings,” in ed. M. Fix and R. Struyk, Clear and Convincing Evidence: Measurement of Discrimination in America, 187-258. Holzer, H.J. (2007). “Collateral Costs: The Effects of Incarceration on the Employment and Earning of Young Workers” IZA Discussion Paper No. 3118 Holzer, H.J., S. Raphael and M.A. Stoll (2006). “Perceived Criminality, Criminal Background Checks, and the Racial Hiring Practices of Employers,” Journal of Law and Economics 49:451. Jarosch, G. and L. Pilossoph (2016). “Statistical Discrimination and Duration Dependence in the Job Finding Rate” Unpublished Manuscript Kroft, K., F. Lange and M. Notowidigdo (2013). “Duration Dependence and Labor Market Conditions: Evidence from a Field Experiment,” Quarterly Journal of Economics 128(3): 11231167 Lahey, J. (2008). “Age, Women, and Hiring: An Experimental Study,” Journal of Human Resources 43(1): 30-56. Lahey, J. and R. Beasley (2009). “Computerizing Audit Studies,” Journal of Economic Behavior and Organization 70(3): 508-514 List, J. (2004). “The Nature and Extent of Discrimination in the Marketplace: Evidence from the Field,” The Quarterly Journal of Economics 119(1): 48-89 Love, M. (2011). “Paying Their Debt to Society: Forgiveness, Redemption, and the Uniform Collateral Consequences of Conviction Act,” Howard Law Journal 54(3): 753-793 Minnesota Department of Human Rights (2015), “Ban The Box: Overview for Private Employers” http://mn.gov/mdhr/employers/banbox_overview_privemp.html Last Accessed Jan 19, 2016. 42 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION NAACP (2014, Jan). Our Accomplishments, http://www.naacp.org/pages/2106. Neumark, D. (1996), “Sex Discrimination in Restaurant Hiring: An Audit Study”, The Quarterly Journal of Economics 111(3): 915-941. Neumark, D. (2011) “Detecting Discrimination in Audit and Correspondence Studies”, The Journal of Human Resources 47(4): 1128-1157 Neumark, D., I. Burn, and P. Button (2015) “Is it Harder for Older Workers to Find Jobs? New and Improved Evidence from a Field Experiment” NBER WP 21669 Nosek, B.A. et al. (2007) “Pervasiveness and Correlates of Implicit Attitudes and Stereotypes,” European Review of Social Psychology 2007:1. Oreopoulos, P. (2011). “Why Do Skilled Immigrants Struggle in the Labor Market? A Field Experiment with Thirteen Thousand Resumes,” American Economic Journal: Economic Policy 3(4): 148-71. Pager, D. (March 2003). “The Mark of a Criminal Record,” American Journal of Sociology 108(5): 937-975. Pager, D., B. Western, & B. Bonikowski. (2009). “Discrimination in a Low-Wage Labor Market,” American Sociological Review, 74:777-799. Phelps, Edmund S. (1972). “The Statistical Theory of Racism and Sexism”, American Economic Review. 62:659. Pinard, M. (January 7, 2014). “Ban the Box in Baltimore,” Baltimore Sun. Pinard, M. (2010). Collateral Consequences of Criminal Convictions: Confronting Issues of Race and Dignity, 85 N.Y.U. L. Rev. 457. Reaves, B. (December 2013). “Felony Defendants in Large Urban Counties, 2009 – Statistical Tables” US Department of Justice Bureau of Justice Statistics Report NCJ 243777 Riach, P.A. and J. Rich (2002). “Field Experiments in Discrimination in the Market Place”, The Economic Journal 112(483): F480-F518 Rodriguez, M and B. Avery. (April 2016). “Ban The Box: U.S. Cities, Counties, and States Adopt Fair-Chance Policies to Advance Employment Opportunities for People with Past Convictions”. National Employment Law Project Guide Accessed June 9, 2016: http://www.nelp.org/publication/ban-the-box-fair-chance-hiring-state-and-local-guide/ Shannon, S., C. Uggen, M. Thompson, J. Schnittker, and M. Massoglia (2011). “Growth in the U.S. Ex-Felon and Ex-Prisoner Population, 1948-2010” Unpublished Manuscript Southern Coalition for Social Justice. (2013). Ban the Box Community Initiative Guide, http://www.southerncoalition.org/program-areas/criminal-justice/ban-the-box-communityinitiative-guide/. Starr, S. (2015). “Do Ban the Box Laws Reduce Employment Barriers for Black Men?” Unpublished Manuscript. Stoll, Michael A. (2009). Ex-Offenders, Criminal Background Checks, and Racial Consequences in the Labor Market, 1 Univ. of Chicago Legal Forum 381 (2009). Wozniak, A. (2015, July). “Discrimination and the Effects of Drug Testing on Black Employment”. The Review of Economics and Statistics 97(3): 548-566 43 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION .15 Figure 1: Callback Rates by Race, Crime, and Box: Pre-Period Applications Only 0.140 0.131 0.094 0.086 0.083 0 .05 Callback Rate .1 0.125 Box No Box Box No Box Black White Crime No Crime Notes: This figure compares callback rates within the pre-period before Ban the Box goes into effect, comparing applications with the box (application which ask about criminal records) and those without (applications that do not ask about criminal records). A callback is a personalized phone call or e-mail to the applicant requesting follow-up contact or an interview. Figure 2: Callback Rates by Race, Criminal Record, and Period: Treated Only .15 0.150 0.138 0.127 0.088 0.084 0 .05 Callback Rate .1 0.110 Pre Post Pre Black Post White Crime No Crime 44 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Notes: This figure compares callback rates within treated companies, i.e. those companies that asked the criminal record question in the pre-period, before and after Ban the Box goes into effect. A callback is a personalized phone call or e-mail to the applicant requesting follow-up contact or an interview. Table 1a: Means of Applicant and Application Characteristics and Callback Rates by Period Pre-Period Post-Period Combined Characteristics: White Crime GED Employment Gap Application has Box 0.502 0.497 0.498 0.492 0.366 0.497 0.513 0.502 0.504 0.036 0.500 0.505 0.500 0.498 0.199 Results: Callback Rate Interview Req 0.109 0.060 0.125 0.067 0.117 0.063 Callback Rate by Chars: Black White GED HSD Emp Gap No Emp Gap Observations 0.099 0.120 0.106 0.113 0.110 0.109 7246 0.111 0.139 0.127 0.122 0.126 0.124 7394 0.105 0.129 0.117 0.118 0.118 0.116 14640 Notes: Callback implies application received a personalized positive response from the employer (either via phone or e-mail). Interview request means the positive response specifically mentioned an interview. Application has box means that the application asked about criminal records. Employment (emp) gap is a 1113 month employment gap in work history, no emp gap is a 0-2 month gap. Table 1b: Callback Rates by Crime Status for Stores with the Box in the Pre-Period Callback Rate Callback Black Callback White Observations No Crime Crime Property Drug Combined 0.136 0.131 0.140 1319 0.085 0.086 0.083 1336 0.084 0.091 0.077 703 0.085 0.081 0.089 633 0.110 0.109 0.111 2655 Notes: Sample restricted to pre-period applications where the application asked about criminal records. Callback implies application received a personalized positive response from the employer. 45 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table 2: Effects of Applicant Characteristics on Callback Rates (1) 0.0244*** (0.0057) (2) 0.0239*** (0.0054) Crime -0.0161*** (0.0053) -0.0136** (0.0054) GED -0.0014 (0.0052) -0.0041 (0.0048) -0.0076 (0.0056) 0.0096 (0.0134) 0.0097 (0.0132) 0.0097 (0.0134) Emp. Gap 0.0012 (0.0048) 0.0017 (0.0046) 0.0005 (0.0050) 0.0103 (0.0101) 0.0104 (0.0100) 0.0102 (0.0101) White Pre-Period (3) 0.0297*** (0.0070) (4) -0.0010 (0.0093) (5) -0.0012 (0.0093) -0.0520*** (0.0121) -0.0444*** (0.0134) -0.0149 (0.0096) Drug Crime -0.0501*** (0.0133) Property Crime -0.0536*** (0.0143) White x Crime Constant Observations Sample Chain FE Center FE (6) 0.0065 (0.0149) -0.0149 (0.0171) 0.1132*** (0.0156) 14640 All No No -0.0069 (0.0261) 14640 All Yes Yes 0.0016 (0.0291) 11722 Non-Box Yes Yes -0.0134 (0.0538) 2918 Box Yes Yes -0.0133 (0.0539) 2918 Box Yes Yes -0.0184 (0.0537) 2918 Box Yes Yes Notes: Dependent variable is whether the application received a callback. Standard errors clustered on company in parentheses. The non-box sample includes only applications that did not ask about criminal history; the box sample includes only those applications that asked about criminal records. Company and center fixed effects are included in Columns (2) – (6) as indicated. White is as compared to black applicants, crime is as compared to no-crime, GED is as compared to a HS Diploma and Emp. Gap is a 11-13 month gap in work history as compared to a 0-2 month gap. 46 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table 3A: Robustness Checks on Main Effect of White Observations Specification (1) 0.0239*** (0.0054) 14640 Main (2) 0.0136*** (0.0045) 14640 Interview Sample All All White (3) 0.0242*** (0.0054) 14640 Ungroup Chain FE All (4) 0.0454*** (0.0097) 6401 Main (5) 0.0073 (0.0050) 8239 Main NJ-All NYC-All Notes: Dependent variable is whether the application received a callback. Standard errors clustered on company in parentheses. Column (1) reproduces the White coefficient from Column 2 of Table 2, and the remaining columns show the White coefficient from different specifications. Column (2) uses interview as the dependent variable rather callback. Column (3) uses ungrouped chain FE rather than grouped. Columns (4) and (5) separate the sample in the NJ sample and the NYC sample. Table 3B: Robustness Checks on Main Effect of Crime in the Box Sample Only Observations Specification (1) -0.0520*** (0.0121) 2918 Main (2) -0.0353*** (0.0062) 2918 Interview Sample All All Crime (3) -0.0522*** (0.0123) 2918 Ungroup Chain FE All (4) -0.0535** (0.0220) 1156 Main (5) -0.0513*** (0.0160) 1762 Main NJ-All NYC-All Notes: All regressions are conditional on the application having the box. Dependent variable is whether the application received a callback. Standard errors clustered on company in parentheses. Column (1) reproduces the Crime coefficient from Column 4 of Table 2, and the remaining columns show the Crime coefficient from different specifications. Column (2) uses interview as the dependent variable rather callback. Column (3) uses ungrouped chain FE rather than grouped. Columns (4) and (5) separate the sample in the NJ sample and the NYC sample. 47 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table 4: Local Racial Composition and the Impact of Race on Callback Rates (1) (2) (3) (4) (5) (6) White 0.00717 (0.00495) -0.00603 (0.00844) 0.0322*** (0.00664) -0.0108 (0.00856) 0.0153*** (0.00589) 0.00994 (0.0164) White x NJ 0.0380*** (0.0106) 0.0335*** (0.0103) 0.0350*** (0.0104) 0.0345*** (0.0103) NJ 0.0109 (0.0172) 0.00589 (0.0171) 0.00982 (0.0175) 0.00531 (0.0168) Store CBG %White x White Store CBG %White 0.0489*** 0.0326** 0.00770 (0.0170) (0.0164) (0.0248) 0.0342*** (0.0124) 0.0334*** (0.0111) 0.0471*** (0.0171) Store CBG %Black x White Store CBG %Black Constant -0.00246 (0.00976) 14640 All Yes No Yes -0.0173* (0.00889) 14634 All Yes No Yes -0.0597*** -0.0485*** -0.0425* (0.0154) (0.0148) (0.0229) -0.0175 (0.0146) -0.0161 (0.0156) 0.0233 (0.0233) -0.000223 (0.0107) 14635 All Yes No Yes -0.0325* (0.0177) 14634 All Yes No Yes 0.00675 (0.00588) 14635 All Yes No Yes -0.0213* (0.0113) 14634 All Yes No Yes Observations Sample Chain FE Center FE Other Controls Notes: Standard errors in parenthesis clustered on chain. Dependent variable is whether the application received a callback. All columns include controls for GED, employment gap, criminal record, and preperiod. Center or company FE included as indicated. Store CBG %White(Black) is the %White (Black) in the Census Block Group that the individual store is located (or sometimes in the town/city/borough if the address was not specified). 48 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table 5: Average Black-White Response Rate Differences by Race and Treated, Before and After BTB Goes into Effect in NJ Black - White Callback Rate, Pre Black - White Callback Rate, Post Diff Diff, Perfect Quad Sample Treated -0.008 -0.040 0.032 0.038 Not Treated -0.027 -0.022 -0.005 -0.004 Diff 0.019 -0.018 0.037 0.042 Notes: Each cell is a black-white response rate differential, measured in percentage points. The last line restricts analysis to only those stores in the “perfect quad” sample, that is, stores for which we sent two applications in the pre- and two in the post. The two outlined cells represent the raw difference-in-differences in-differences in the full sample and the perfect quad sample. 49 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table 6: Effects of Ban the Box on Racial Discrimination, Triple Difference Specification (1) Post x Treated x White 0.0371** (0.0180) (2) 0.0409** (0.0184) (3) 0.0358** (0.0180) (4) 0.0399** (0.0200) Post x White -0.00530 (0.0125) -0.00627 (0.0123) -0.00618 (0.0128) -0.00236 (0.0136) Post x Treated -0.0102 (0.0177) -0.0115 (0.0177) -0.0198 (0.0214) White x Treated -0.0187 (0.0140) -0.0213 (0.0140) -0.0175 (0.0146) Treated 0.00893 (0.0262) 0.00954 (0.0239) 0.0167 (0.0276) White 0.0268** (0.0108) 0.0281*** (0.0107) 0.106 (0.130) 0.0247** (0.0116) Post 0.0153 (0.0131) 0.0127 (0.0137) 0.340** (0.140) 0.0163 (0.0158) Crime -0.0155*** (0.00544) -0.0152*** (0.00548) -0.0174*** (0.00666) GED -0.00261 (0.00514) -0.00567 (0.00492) -0.00307 (0.00656) Employment Gap 0.000232 (0.00466) 0.00131 (0.00456) 0.00366 (0.00577) 0.108*** (0.0267) 14640 0.027 No No No Yes All -0.0101 (0.0256) 14640 0.193 Yes Yes Yes Yes All 0.0986*** (0.0216) 11188 0.003 No No No Yes Quad Constant Observations R2 Chain FE Post x Chain FE White x Chain FE Center FE Sample 0.0962*** (0.0199) 14640 0.002 No No No Yes All Notes: Standard errors in parenthesis clustered on chain. Dependent variable is whether the application received a callback. The Quad sample indicates the “perfect quad” sample of 11,118 observations where we sent exactly 4 applications, one white/black pair in each period. Fixed effects can include, chain, post x chain, white x chain, or center, and are included as indicated. 50 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table 7A: Robustness Checks: Triple Difference Specification (1) Post x Treated 0.0358** x White (0.018) Observations 14640 R2 0.193 Specification Main (2) 0.0326** Sample All All (3) 0.0328** (0.016) 14640 0.171 (0.017) 14816 0.197 Interview Main (4) 0.0361** (5) 0.0464 (6) 0.0266 (7) 0.0349* (8) 0.0348* (0.018) 14581 0.191 Main (0.037) 6401 0.216 Main (0.020) 8239 0.228 Main NJ NYC (0.018) 14640 0.236 Ungroup Chain All (0.018) 14640 0.226 Chain x NJ FE All Add Rev Drop Compliers RA Errors Notes: Standard errors clustered on chain in parenthesis. Dependent variable is whether the application received a positive call back, except in column (2) where it is whether the application received a specific request for an interview. All regressions include controls for, crime, GED, emp. gap, and fixed effects for center, chain, chain x white and chain x post. Column (1) recreates Table 6 Column (3). The remaining columns are each different modifications of this specification. Column (2) uses interview as the dependent variable, Column (3) adds in the reverse compliers, Column (4) drops instances where RA erred and answered a box question they weren’t required to answer or did not answer one they should have, Column (5) is restricted to only NJ, Column (6) is only NYC, Column (7) uses individual chain fixed effects regardless of size, and Column (8) divides chain fixed effects into NJ and NYC. Table 7B: Robustness Checks: Triple Difference Specification in Perfect Quad Sample (1) Post x Treated 0.0399** x White (0.020) Observations 11188 2 R 0.003 Specification Main Sample Quad (2) 0.0394** (3) 0.0351* (4) 0.0387* (5) 0.0500 (6) 0.0335 (0.020) 11188 0.004 Interview Quad (0.019) 11324 0.003 Main Quad + Rev. Compliers (0.020) 11128 0.003 Main Quad-Drop RA Errors (0.040) 4376 0.007 Main Quad NJ (0.021) 6812 0.003 Main Quad NYC Notes: Observations restricted to the “perfect quad” sample of 11,118 observations where we sent exactly 4 applications, one white/black pair in each period. Standard errors clustered on chain in parenthesis. Dependent variable is whether the application received a positive call back, except in column (2) where it is whether the application received a specific request for an interview. All regressions include controls for center FE, crime, GED, emp. gap. Panel A Column (1) recreates Table 6 Column (4). The remaining columns are each different modifications of this specification. Column (2) uses interview as the dependent variable, Column (3) adds in the reverse compliers, Column (4) drops instances where RA erred and answered a box question they weren’t required to answer or did not answer one they should have, Column (5) is restricted to only NJ, Column (6) is only NYC. 51 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Appendix A1. Applicant Profile Details Applicant profiles consist of all information that our RAs might need in order to fill out a given job application. In addition to the characteristics we randomly varied, many other types of information were necessary to include such as previous job titles and descriptions, home addresses, names of high schools, references, and e-mail addresses. We wanted to keep these additional characteristics as similar as possible while still introducing slight (random) variation so as not to arouse employer suspicion. (1) Work history: All job applicants have about 3.5 years of work experience: about 2 years as crew members at fast-food chains or convenience stores and about 1.5 years in manual labor jobs such as home improvement, landscaping, or moving. The fast-food chains or convenience stores were real companies that we were not applying to. Each applicant was randomly assigned a company from that list of fast-food chains or convenience stores. They were given crew member or team member positions and assigned relatively generic job duties meant to imply they held basic entry-level cashier-type positions at the establishments. The manual labor jobs were randomly assigned to be in landscaping, paving, moving, home improvement, or lawn care and were not given real company names. Company names were made up but based on names standard to the industries involved (e.g., A1 Best Landscaping, [Reference Last Name] Contracting LLC, or Newark Home Improvement Inc.). Applicants were similarly assigned generic job duties meant to imply entry-level, unskilled crew-member or assistant positions in the fictitious companies. All applicants are unemployed at the time of the job application, having ended their most recent job 2 or 3 months before the application is submitted. Descriptions of previous job duties and reasons for leaving jobs varied slightly. Applicants with employment gaps have 11 to 13 months of unemployment between the two jobs; those without employment gaps have only 0- to 2-month gaps. (2) Address and center city: Because it is likely that employers would be concerned about employees being able to travel to work, we wanted applicants to live near the jobs they 52 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION apply to. As described in the text, to achieve that, we chose 40 geographically distributed cities or towns in New Jersey and 44 in New York City to serve as centers where the applicants’ addresses would be located; each center then served as the base for applications to jobs located nearby. To choose the centers, we first narrowed down the entire list of New Jersey cities and towns as well as community districts in New York City to those that were at least 6% black, were at least 20% white, and had median annual incomes less than $100,000. We then used an optimization tool in the ArcGIS software package to select among those possibilities the 40 centers that would minimize distance to jobs; in New Jersey this was based on the distribution of postings then found (in January 2015) on snagajob.com, and in New York City it was based on the locations of employers that we located in a BusinessUSA database. In New Jersey, we assigned every municipality in the state to its nearest center, excluding only a few small towns that were more than 20 miles from any center. In New York City, we minimized distances subject to a constraint of equal distribution of chains across centers—for example, all chains with 44 or fewer locations were distributed such that no more than one location was assigned to each center, while a chain with 45 to 88 locations would be distributed with one to two locations per center, and so forth. Within each center, eight qualifying addresses were located within census blocks that were at least 10% black and 20% white and that had a median annual income less than $100,000. All addresses came from different streets, and Google Street View was consulted to ensure that the choices were appropriate residential or mixed-use blocks and that they did not notably differ from one another. Addresses were then slightly changed so as not to represent real addresses, and they were then randomly assigned to applicants. (3) High school or GED program: For diploma earners, high schools for the New Jersey study were chosen to be in New Jersey cities or towns at least 30 miles away from the center to reduce the probability that the high school could send any unobservable signals to the employer. High schools for the New York City study were divided equally between New Jersey and upstate New York schools, since similar geographic separation could not be achieved within the city. The high schools used were all at least 10% black, are at least 20% white, have at least 25,000 people, and do not have median incomes more than 53 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION $100,000. In addition, the high schools do not have median test scores above the 90th or below the 10th percentile in the state. Applicants with GEDs were randomly assigned descriptions and names of New Jersey or New York GED training programs. (4) References: Two fictitious references with phone numbers were created, representing the applicant’s supervisors for each of two previous jobs. To complement and strengthen the racial signal provided by our applicant names, the previous supervisor from the manual labor job was also given a racially distinctive name suggesting the same race as the applicant. The previous supervisor of the retail or restaurant job was given a race-neutral name. However, no employers ever called the phone numbers that we purchased and provided for the references, suggesting that little attention was likely paid to them. (5) Phone number: Each applicant was assigned a phone number based on center, race, criminal history, and time period. (Thus, each center has at least four potential phone numbers during each phase of the study; in New York City, because we were sending a larger number of distinct applications per center, we bought two numbers for each combination of characteristics and varied them randomly.) The result of that division is that no store received two applications using the same phone number. That method also helps us identify which application a voice mail belongs to, because hiring managers would not always leave all pertinent information on the voice mail. The information left, combined with the phone number being called, was sufficient to uniquely assign responses to applications. We purchased these phone numbers from www.callfire.com, which enabled us to create voicemails for our applicants using one of several available robotic voices. The wording and voice on the outgoing voice mail greeting were randomized across several options and designed to sound like a generic cell phone voice mail greeting for someone who has not recorded a personalized one. (6) E-mail address: A unique e-mail address was created for each applicant, with the format randomly varied. All e-mail addresses were created with the same domain, and the format always included the applicant’s first and last names but could also include numbers, a middle initial, periods, or underscores so as to differentiate the format across applicants to the same store. 54 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION (7) Criminal record: Applicants with felony convictions were randomly assigned either a property crime or a drug crime. Within those two categories, several potential crimes were chosen—all of them meant to imply similar levels of seriousness. In addition, many applications with the box ask the applicant to “Please explain.” For that, specific language was given as part of the profiles, with sentences randomly generated to indicate when the crime occurred, a potential expression of remorse, and a potential expression of desire to discuss the matter further in person. Each of the profiles were randomly generated using the Resume Randomizer program of Lahey and Beasley (2009). Applicant pairs were always of opposite race, and were otherwise created so that the details of the aforementioned characteristics were randomly varied among the pair. For example, both members of the pair could have high school diplomas, but never from the same high school or the same town; no two applicants in the same pair had the same address; none worked for the exact same former employers; if both had a criminal record, it did not involve the same criminal charge, and so forth. For examples of profiles, which are several pages in length, please e-mail the authors. 55 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION A2. Names Used Table A2.1: White and Black Names Used for Applicants White Names Black Names First %White Last %White First %Black Last %Black SCOTT 88.87 WEBER 94.37 TYREE 97.94 PIERRE 97.78 THOMAS 86.92 ESPOSITO 93.30 TERRELL 96.23 WASHINGTON 90.28 CODY 86.71 SCHMIDT 92.63 DAQUAN 96.04 ALSTON 88.96 RYAN 85.37 BRENNAN 92.45 JAQUAN 95.03 BYRD 85.50 NICHOLAS 84.99 MEYER 92.27 DARNELL 93.43 INGRAM 78.63 DYLAN 84.70 KANE 91.75 JAMAL 91.36 JACKSON 76.32 MATTHEW 83.97 HOFFMAN 91.38 MARQUIS 91.36 BANKS 75.68 JACOB 83.37 RYAN 89.98 JERMAINE 89.45 FIELDS 74.83 KYLE 82.93 WAGNER 89.96 DENZEL 89.27 BRYANT 74.49 TYLER 82.82 HANSEN 89.60 DWAYNE 88.89 WILLIAMS 74.22 SEAN 82.41 SNYDER 88.84 REGINALD 88.41 SIMMONS 72.45 DOUGLAS 81.93 ROMANO 88.84 TYRONE 86.75 CHARLES 72.33 SHANE 81.11 O'NEILL 88.72 MALCOLM 86.06 HAWKINS 70.81 JOHN 80.36 RUSSO 88.67 DARRYL 84.78 ROBINSON 70.70 STEPHEN 80.12 FOX 86.43 TERRANCE 84.12 JENKINS 70.50 SWEENEY 86.03 MAURICE 82.47 FRANKLIN 70.45 SULLIVAN 85.08 ISAIAH 74.06 JOSEPH 70.42 ELIJAH 72.35 Notes: The %race columns indicate the percentage of babies born in NJ between 1989 and 1996 with that first or last name that were of that race (i.e. 88.87% of babies with the first name Scott are White). 56 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION A3. Analysis Tables for NJ Only This appendix recreates Figures 1 and 2 as well as Table 1a and 1b, Table 2 and 5 for only NJ only. 0.188 0.180 0.139 0.125 0.118 0.108 0 .05 Callback Rate .1 .15 .2 Figure A3.1: Callback Rates by Race, Crime, and Box: Pre-Period NJ Applications Only Box No Box Box Black No Box White Crime No Crime Notes: Limited to only NJ applications. This figure compares callback rates within the pre-period before Ban the Box goes into effect, comparing applications with the box (application which ask about criminal records) and those without (applications that do not ask about criminal records). A callback is a personalized phone call or e-mail to the applicant requesting follow-up contact or an interview. 57 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Figure A3.2: Callback Rates by Race, Criminal Record, and Period: NJ Treated Only 0.136 0.129 0.201 0.122 0.109 0 .05 Callback Rate .1 .15 .2 0.193 Pre Post Pre Black Post White Crime No Crime Notes: Limited to only NJ applications. This figure compares callback rates within treated companies, i.e. those companies that asked the criminal record question in the pre-period, before and after Ban the Box goes into effect. A callback is a personalized phone call or e-mail to the applicant requesting follow-up contact or an interview. 58 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table A3.1a: Means of Applicant and Application Characteristics and Callback Rates by Period, NJ Only Pre-Period Post-Period Combined Characteristics: White Crime GED Employment Gap Application has Box 0.507 0.498 0.506 0.503 0.362 0.495 0.504 0.513 0.504 0.034 0.500 0.501 0.510 0.504 0.181 Results: Callback Rate Interview Req 0.147 0.081 0.146 0.076 0.147 0.078 0.125 0.170 0.139 0.156 0.145 0.150 2864 0.124 0.170 0.143 0.150 0.149 0.144 3537 0.124 0.170 0.142 0.152 0.147 0.146 6401 Callback Rate Chars: Black White GED HSD Emp Gap No Emp Gap Observations by Notes: Sample limited to NJ applications. Callback implies application received a personalized positive response from the employer (either via phone or e-mail). Interview request means the positive response specifically mentioned an interview. Application has box means that the application asked about criminal records. Employment (emp) gap is a 11-13 month employment gap in work history, no emp gap is a 0-2 month gap. Table A3.1b: Callback Rates by Crime Status for Stores with the Box in the Pre-Period, NJ Only Callback Rate Callback Black Callback White Observations No Crime Crime Property Drug Combined 0.164 0.139 0.188 507 0.113 0.108 0.118 530 0.102 0.087 0.118 293 0.127 0.139 0.118 237 0.138 0.124 0.151 1037 Notes: Sample restricted to pre-period applications in NJ where the application asked about criminal records. Callback implies application received a personalized positive response from the employer. 59 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table A3.2: Effects of Applicant Characteristics on Callback Rates NJ ONLY White (1) (2) 0.0466*** 0.0454*** (0.0100) (0.0097) (3) (4) 0.0500*** 0.0260 (0.0116) (0.0213) (5) 0.0251 (0.0210) Crime -0.0157** (0.0070) -0.0153** (0.0071) GED -0.0120 (0.0089) -0.0161** (0.0078) -0.0210** (0.0087) -0.0026 (0.0285) -0.0016 (0.0281) -0.0000 (0.0273) Employment Gap 0.0008 0.0011 0.0024 -0.0065 -0.0057 -0.0062 (0.0073) (0.0071) (0.0080) (0.0123) (0.0123) (0.0125) Pre-Period -0.0535** (0.0220) (6) 0.0515 (0.0360) -0.0280 (0.0326) -0.0034 (0.0138) Drug Crime -0.0423 (0.0305) Property Crime -0.0626** (0.0250) White x Crime Constant Observations Sample Chain FE Center FE -0.0499 (0.0368) 0.1372*** (0.0192) 6401 All No No 0.0392 (0.0380) 6401 All Yes Yes 0.0333 (0.0368) 5245 Non-Box Yes Yes 0.0137 (0.0958) 1156 Box Yes Yes 0.0128 (0.0971) 1156 Box Yes Yes 0.0021 (0.1002) 1156 Box Yes Yes Notes: This table recreates Table 2 for NJ only. Dependent variable is whether the application received a callback. Standard errors clustered on company in parentheses. The non-box sample includes only applications that did not ask about criminal history; the box sample includes only those applications that asked about criminal records. Chain and center fixed effects are included in Columns (2) – (6) as indicated. White is as compared to black applicants, crime is as compared to no-crime, GED is as compared to a HS Diploma and Emp. Gap is a 11-13 month gap in work history as compared to a 0-2 month gap. 60 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table A3.3: Effects of Ban the Box on Racial Discrimination, Triple Difference Specification NJ ONLY (1) Post x Treated x White 0.0523 (0.0380) (2) 0.0587 (0.0381) (3) 0.0464 (0.0371) (4) 0.0500 (0.0395) Post x White -0.0158 (0.0234) -0.0184 (0.0232) -0.0106 (0.0227) 0.00152 (0.0289) Post x Treated 0.0113 (0.0280) 0.00765 (0.0273) 0.00413 (0.0373) White x Treated -0.0144 (0.0307) -0.0195 (0.0307) -0.00442 (0.0314) Treated -0.00383 (0.0335) -0.00290 (0.0325) 0.00344 (0.0396) White 0.0498** (0.0206) 0.0536** (0.0205) 0.0188 (0.0348) 0.0405* (0.0204) Post -0.00447 (0.0214) -0.000530 (0.0213) 1.019*** (0.0348) -0.00828 (0.0286) Crime -0.0158** (0.00678) -0.0151** (0.00709) -0.0165** (0.00788) GED -0.0126 (0.00846) -0.0174** (0.00758) -0.0133 (0.0123) Employment Gap 0.00108 (0.00718) 0.00146 (0.00667) 0.00544 (0.0100) 0.183*** (0.0478) 6401 0.031 No No No Yes All 0.0489 (0.0360) 6401 0.216 Yes Yes Yes Yes All 0.138*** (0.0354) 4376 0.007 No No No Yes Quad Constant Observations R2 Chain FE Post x Chain FE White x Chain FE Center FE Sample 0.126*** (0.0277) 6401 0.005 No No No Yes All Notes: This table recreates Table 5 for NJ only. Standard errors in parenthesis clustered on chain. Dependent variable is whether the application received a callback. The Quad sample indicates the “perfect quad” sample of observations where we sent exactly 4 applications, one white/black pair in each period. Fixed effects can include, chain, post x chain, white x chain, or center, and are included as indicated. 61 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION A4. Analysis Tables for NYC Only This appendix recreates Figures 1 and 2 as well as Table 1a and 1b, Table 2 and 5 for only NJ only. .15 Figure A3.1: Callback Rates by Race, Crime, and Box: Pre-Period NYC Applications Only 0.126 0.088 0.073 0.073 0.058 0 .05 Callback Rate .1 0.111 Box No Box Box Black No Box White Crime No Crime Notes: Limited to only NYC applications. This figure compares callback rates within the pre-period before Ban the Box goes into effect, comparing applications with the box (application which ask about criminal records) and those without (applications that do not ask about criminal records). A callback is a personalized phone call or e-mail to the applicant requesting follow-up contact or an interview. 62 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION .15 Figure A3.2: Callback Rates by Race, Criminal Record, and Period: NYC Treated Only 0.121 0.108 0.094 0.067 0.063 0 .05 Callback Rate .1 0.102 Pre Post Pre Black Post White Crime No Crime Notes: Limited to only NYC applications. This figure compares callback rates within treated companies, i.e. those companies that asked the criminal record question in the pre-period, before and after Ban the Box goes into effect. A callback is a personalized phone call or e-mail to the applicant requesting follow-up contact or an interview. 63 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table A3.1a: Means of Applicant and Application Characteristics and Callback Rates by Period, NYC Only Pre-Period Post-Period Combined Characteristics: White Crime GED Employment Gap Application has Box 0.500 0.496 0.492 0.485 0.369 0.499 0.521 0.492 0.505 0.037 0.499 0.508 0.492 0.494 0.214 Results: Callback Rate Interview Req 0.085 0.046 0.105 0.059 0.094 0.052 0.083 0.087 0.083 0.086 0.086 0.084 4382 0.099 0.110 0.112 0.098 0.104 0.105 3857 0.090 0.098 0.097 0.092 0.095 0.094 8239 Callback Rate Chars: Black White GED HSD Emp Gap No Emp Gap Observations by Notes: Sample limited to NYC applications. Callback implies application received a personalized positive response from the employer (either via phone or e-mail). Interview request means the positive response specifically mentioned an interview. Application has box means that the application asked about criminal records. Employment (emp) gap is a 11-13 month employment gap in work history, no emp gap is a 0-2 month gap. Table A3.1b: Callback Rates by Crime Status for Stores with the Box in the Pre-Period, NYC Only Callback Rate Callback Black Callback White Observations No Crime Crime Property Drug Combined 0.118 0.126 0.111 812 0.066 0.073 0.058 806 0.071 0.093 0.046 410 0.061 0.052 0.069 396 0.092 0.099 0.085 1618 Notes: Sample restricted to pre-period applications in NYC where the application asked about criminal records. Callback implies application received a personalized positive response from the employer. 64 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table A4.2: Effects of Applicant Characteristics on Callback Rates: NYC Only (1) 0.0073 (0.0050) (2) 0.0073 (0.0050) Crime -0.0168** (0.0082) -0.0137* (0.0079) GED 0.0049 (0.0064) 0.0044 (0.0059) 0.0018 (0.0066) 0.0141 (0.0105) 0.0136 (0.0104) 0.0143 (0.0104) Employment Gap 0.0007 0.0012 -0.0033 0.0213* 0.0214* 0.0215* (0.0057) (0.0055) (0.0055) (0.0117) (0.0118) (0.0117) White Pre-Period (3) 0.0139** (0.0060) (4) -0.0182** (0.0087) (5) -0.0179** (0.0087) -0.0513*** (0.0160) -0.0571*** (0.0184) -0.0238 (0.0169) Drug Crime -0.0577*** (0.0170) Property Crime -0.0453*** (0.0164) White x Crime Constant Observations Sample Chain FE Center FE (6) -0.0239 (0.0146) 0.0115 (0.0192) 0.0961*** (0.0176) 8239 All No No 0.0192 (0.0242) 8239 All Yes Yes 0.0168 (0.0277) 6477 Non-Box Yes Yes 0.0296 (0.0575) 1762 Box Yes Yes 0.0293 (0.0572) 1762 Box Yes Yes 0.0329 (0.0567) 1762 Box Yes Yes Notes: This table recreates Table 2 for NYC only. Dependent variable is whether the application received a callback. Standard errors clustered on company in parentheses. The non-box sample includes only applications that did not ask about criminal history; the box sample includes only those applications that asked about criminal records. Chain and center fixed effects are included in Columns (2) – (6) as indicated. White is as compared to black applicants, crime is as compared to no-crime, GED is as compared to a HS Diploma and Emp. Gap is a 11-13 month gap in work history as compared to a 0-2 month gap. 65 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table A4.3: Effects of BTB on Racial Discrimination, Triple Difference Analysis: NYC ONLY (1) Post x Treated x White 0.0267 (0.0196) (2) 0.0275 (0.0198) (3) 0.0266 (0.0203) (4) 0.0335 (0.0212) Post x White -0.00191 (0.0112) -0.00165 (0.0112) -0.00402 (0.0113) -0.00380 (0.0103) Post x Treated -0.0253 (0.0344) -0.0268 (0.0342) -0.0353 (0.0360) White x Treated -0.0229* (0.0119) -0.0228* (0.0118) -0.0244* (0.0134) Treated 0.0174 (0.0289) 0.0173 (0.0283) 0.0265 (0.0292) White 0.0116 (0.00839) 0.0112 (0.00832) 0.240*** (0.0542) 0.0139 (0.0103) Post 0.0250 (0.0220) 0.0259 (0.0220) 0.192*** (0.0162) 0.0320 (0.0231) Crime -0.0166** (0.00831) -0.0143* (0.00810) -0.0175* (0.00971) GED 0.00485 (0.00628) 0.00276 (0.00622) 0.00136 (0.00737) Employment Gap -0.000253 (0.00560) 0.000362 (0.00561) 0.00234 (0.00607) 0.108*** (0.0274) 8239 0.011 No No No Yes All 0.000804 (0.0254) 8239 0.228 Yes Yes Yes Yes All 0.0739*** (0.0177) 6812 0.003 No No No Yes Quad Constant Observations R2 Chain FE Post x Chain FE White x Chain FE Center FE Sample 0.0769*** (0.0186) 8239 0.002 No No No Yes All Notes: This table recreates Table 5 for NYC only. Standard errors in parenthesis clustered on chain. Dependent variable is whether the application received a callback. The Quad sample indicates the “perfect quad” sample of observations where we sent exactly 4 applications, one white/black pair in each period. Fixed effects can include, chain, post x chain, white x chain, or center, and are included as indicated. 66 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION A5. Triple Differences with GED and Emp Gap Table A5.1: Effects of Ban the Box on GED vs High School Diploma, Triple Differences (1) -0.0108 (0.0183) (2) -0.0103 (0.0192) (3) -0.00360 (0.0190) (4) -0.0123 (0.0255) Post x GED 0.0155 (0.00971) 0.00981 (0.00962) 0.00186 (0.00969) 0.0218* (0.0127) Post x Treated 0.0137 (0.0223) 0.0142 (0.0233) 0.00619 (0.0300) Treated x GED 0.0186 (0.0155) 0.0203 (0.0150) 0.0213 (0.0219) Treated -0.00967 (0.0278) -0.0114 (0.0263) -0.00269 (0.0294) GED -0.0131 (0.00868) -0.0124 (0.00824) 0.410*** (0.131) -0.0189* (0.0110) Post 0.00478 (0.0132) 0.00469 (0.0137) 0.476*** (0.176) 0.00421 (0.0157) Crime -0.0153*** (0.00546) -0.0143*** (0.00549) -0.0174** (0.00673) Employment Gap 0.000270 (0.00466) 0.00176 (0.00475) 0.00361 (0.00583) White 0.0248*** (0.00572) 0.0236*** (0.00549) 0.0243*** (0.00613) 0.115*** (0.0280) 14640 0.027 No No No Yes All -0.0214 (0.0270) 14640 0.196 Yes Yes Yes Yes All 0.107*** (0.0244) 11188 0.003 No No No Yes Quad Post x Treated x GED Constant Observations R2 Chain FE Post x Chain FE GED x Chain FE Center FE Sample 0.116*** (0.0237) 14640 0.001 No No No Yes All Notes: This table recreates Table 5, substituting GED for White. Standard errors in parenthesis clustered on chain. Dependent variable is whether the application received a callback. The Quad sample indicates the “perfect quad” sample of observations where we sent exactly 4 applications, one white/black pair in each period. Fixed effects can include, chain, post x chain, white x chain, or center, and are included as indicated. 67 AGAN & STARR, BAN THE BOX , CRIMINAL RECORDS, AND STATISTICAL DISCRIMINATION Table A5.2: Effects of Ban the Box on Emp Gap vs No Emp Gap, Triple Differences (1) -0.0248 (0.0204) (2) -0.0262 (0.0194) (3) -0.0221 (0.0200) (4) -0.0267 (0.0231) Post x Emp Gap 0.00996 (0.0132) 0.0116 (0.0129) 0.00907 (0.0121) 0.0150 (0.0137) Post x Treated 0.0205 (0.0179) 0.0220 (0.0179) 0.0134 (0.0223) Treated x Emp Gap 0.0180 (0.0148) 0.0197 (0.0142) 0.0129 (0.0150) Treated -0.00920 (0.0297) -0.0109 (0.0274) 0.00167 (0.0300) Employment Gap -0.00549 (0.00969) -0.00775 (0.00941) 0.586*** (0.103) -0.00377 (0.00995) Post 0.00756 (0.0150) 0.00383 (0.0154) 0.633*** (0.154) 0.00764 (0.0171) Crime -0.0154*** (0.00541) -0.0150*** (0.00556) -0.0173** (0.00667) GED -0.00247 (0.00521) -0.00537 (0.00491) -0.00303 (0.00663) White 0.0247*** (0.00569) 0.0235*** (0.00529) 0.0243*** (0.00605) 0.113*** (0.0277) 14640 0.026 No No No Yes All -0.0133 (0.0251) 14640 0.194 Yes Yes Yes Yes All 0.102*** (0.0231) 11188 0.003 No No No Yes Quad Post x Treated x Emp Gap Constant Observations R2 Chain FE Post x Chain FE Emp Gap x Chain FE Center FE Sample 0.112*** (0.0233) 14640 0.001 No No No Yes All Notes: This table recreates Table 5, substituting Emp Gap for White. Standard errors in parenthesis clustered on chain. Dependent variable is whether the application received a callback. The Quad sample indicates the “perfect quad” sample of observations where we sent exactly 4 applications, one white/black pair in each period. Fixed effects can include, chain, post x chain, white x chain, or center, and are included as indicated. 68