Fbi Cheating Report Sept 2010
Download original document:
This text is machine-read, and may contain errors. Check the original document to verify accuracy.
U.S. Department of Justice Office of the Inspector General Investigation of Allegations of Cheating on the FBI’s Domestic Investigations and Operations Guide (DIOG) Exam Office of the Inspector General Oversight and Review Division September 2010 I. INTRODUCTION A. Initiation of Investigation In December 2008, the Federal Bureau of Investigation (FBI) implemented a new Domestic Investigations and Operations Guide (DIOG), which described the procedures its employees must follow when conducting domestic investigations. The DIOG replaced several older sets of guidelines that separately addressed criminal investigations, national security investigations, and foreign intelligence collection. According to testimony provided in 2008 by the FBI’s General Counsel to the Senate Select Committee on Intelligence, the DIOG is an outgrowth of the FBI’s post-September 11, 2001, transformation from primarily a law enforcement agency to a domestic intelligence agency that focuses on its national security and law enforcement missions. To help implement this transformation, in 2007 the FBI asked the Attorney General to combine several sets of guidelines that had previously governed the FBI into a single set that would reflect the FBI’s changed priorities. After lengthy review, discussions, and consultations, the Attorney General implemented the revised guidelines, entitled “Attorney General Consolidated Guidelines for FBI Domestic Operations” (Consolidated Guidelines), which took effect on December 1, 2008. According to an FBI press release, the Consolidated Guidelines “ensure that the FBI’s operating rules are consistent with the Bureau’s mission and current operational needs while at the same time protecting the privacy and civil liberties of Americans.” The DIOG, in turn, established detailed internal FBI rules and procedures that implemented the new Consolidated Guidelines. The new Guidelines created some public controversy. Some members of Congress and advocacy groups argued that the Guidelines expanded the FBI’s investigative power in ways that could potentially infringe on citizens’ privacy and civil rights, while the FBI argued that the Guidelines were a consolidation of existing authorities. In response to concerns about the proposed new Guidelines, the FBI assured Congress that the FBI would ensure that the FBI complied with the new Guidelines. For example, the FBI General Counsel testified that the “new guidelines take seriously the need to ensure compliance and provide for meaningful oversight to protect privacy rights and civil liberties.” She also testified that the guidelines would not take effect until the FBI had conducted 1 “comprehensive training to ensure that [FBI] personnel understand these new rules and will be ready to apply them in their operations.”1 In accord with this testimony, the FBI implemented comprehensive training on the DIOG for approximately 20,000 FBI agents, analysts, and technicians. These employees were required to complete 16.5 hours of live classroom-style training. Moreover, after taking this training, the employees had to take and pass (with a score of 80 or above) a written, computerized 51question exam concerning the DIOG. Written instructions given both before and during the exam specified that employees could use the DIOG and notes to take the exam, but they were not allowed to consult with other employees. In addition, Question 51 of the exam required all employees to “certify that I only consulted the DIOG, notes, or training aids but no other person while taking this exam.” As discussed below, the training on the DIOG started in the spring of 2009 and continued into the winter of 2010. FBI employees were encouraged to take the exam soon after completing the training, and the vast majority took it between May 2009 and January 2010. However, in September 2009, the FBI received an allegation that three top managers from the Washington Field Office (WFO) took the DIOG exam together and that WFO attorneys were present while the managers took the DIOG exam. Later, the FBI received four allegations about additional misconduct related to the DIOG exam. In addition, FBI officials expressed concerns to the Office of the Inspector General (OIG) that a large number of FBI employees had completed the exam extremely quickly – over 200 finished in 20 minutes or less – and that many of them received high passing scores. FBI officials who had developed the exam stated that it should take approximately 90 minutes or longer to complete the exam. In addition, the FBI reported to the OIG that it had received allegations that FBI employees, in addition to those in the WFO, had cheated on the exam by sharing answer sheets or by receiving assistance during the examination. Because of the concerns that there may have been widespread assistance given during the exam, contrary to the instructions for the exam and the FBI General Counsel Valerie Caproni, “Statement before the Senate Select Committee on Intelligence” (September 23, 2008), http://www.fbi.gov/congress/congress08/ caproni092308.htm (accessed August 4, 2010). 1 2 representations made in response to Question 51, in January 2010 the OIG opened this investigation related to the DIOG exam. B. Methodology The OIG began its investigation by examining internal FBI documents concerning the DIOG and the DIOG exam, including memoranda that described the development of the exam and e-mails that were sent to all FBI employees concerning the exam. We also examined FBI computer data showing which employees had taken the exam, how long it had taken them to pass, and what scores they had achieved. Based on our review of e-mails and the computerized exam data, we decided to conduct interviews in four FBI field offices around the country. We also investigated several of the individual complaints that had been previously filed with the FBI about alleged cheating on the exam. During our investigation we interviewed 76 employees in 9 field and regional area offices and FBI Headquarters about their conduct in taking the DIOG exam. C. Structure of the Report The report is divided into four sections. Section II provides background information regarding the new DIOG, the training given to employees about the DIOG, and the exam that followed. This section also describes the instructions that were given to employees who were taking the DIOG exam, the procedures for taking the exam, and the differences between the DIOG exam and the “typical” training exams that FBI employees take regularly. In addition, it describes the results of the FBI Inspection Division’s investigation of the allegations that senior FBI officials in the WFO had cheated on the exam. Section III presents the OIG’s findings in our investigations in the four field offices, as well as our investigations of the individual complaints. Section IV provides the OIG’s conclusions and recommendations. II. BACKGROUND A. The Domestic Investigations and Operations Guide In December 2008, the FBI issued the DIOG, whose primary purpose, according to an FBI training manual, is to standardize policy so that all investigations are conducted legally and consistently. As mentioned above, the DIOG was designed to replace numerous FBI manuals and other policy documents, which provided guidance to the FBI on conducting investigations in accordance with the law, Attorney General Guidelines, and FBI policies. 3 B. Training on the DIOG To train the FBI employees on the new DIOG, the FBI brought over 100 employees, including Chief Division Counsels (CDC) and Associate Division Counsels, to Washington, D.C., in the winter and spring of 2009 for week-long “train-the-trainer” seminars that focused on core areas of the DIOG. According to the FBI, at these seminars the trainers were informed that when conducting training they could tell FBI employees about certain specific concepts that would be covered on the exam but that they should not reveal any actual exam questions. The trainers then returned to their various offices and (aided by additional trainers who had not attended the “train-the-trainer” seminars but who had attended a 16.5-hour training session and passed the DIOG exam) gave live training seminars in the spring and summer of 2009 on the requirements of the DIOG. The seminars were mandatory for every employee involved in “criminal or national security operations, operational support or intelligence collection activities,” according to the FBI’s Office of the General Counsel (OGC). Eventually, over 20,000 employees took the seminars – virtually every FBI Special Agent and Intelligence Analyst, as well as many FBI technicians, support employees, and employees of other law enforcement agencies assigned to various FBI task forces. Generally, the full training seminar was split into four approximately 4hour sessions, each concentrating on different core parts of the DIOG, for a total of 16.5 hours of training. While the training in the various field offices was designed to be standardized, the OIG found that it in fact varied and that individual trainers used different approaches. Some trainers, for example, relied mostly on a syllabus and Power Point slides that had been distributed during the Washington, D.C., “train-the-trainer” seminars, while others developed their own syllabus and slides and relied more heavily on those. The DIOG training was somewhat different from what was described to us as “normal” FBI training. In most training sessions, employees told us instructors “train to the test,” meaning that they very explicitly gear their comments towards the questions that will ultimately appear on any exam that follows the training. For example, in other training sessions, FBI employees told the OIG, some instructors stomped a foot several times, loudly, when they were covering a question that would be on the exam. Other instructors marked their Power Point slides with attention-getting signals – such as a cartoon character – if the information on that particular slide would be on the exam. FBI employees said they knew these to be clear signals to pay attention, take notes, and remember those points, which they understood would inevitably be covered on the exam. 4 FBI employees told us that some of the DIOG instructors employed similar techniques to clue the students about an important point that would be on the DIOG exam. However, the employees told the OIG that the DIOG training was not as focused on “training to the test” as the average FBI training. In addition, some employees noted that 16 hours of mandatory classroom training was an unusually long amount of training for a computerized training exam. C. The DIOG Exam The FBI’s Corporate Policy Office (CPO), which is part of the FBI Director’s Office, developed and distributed the DIOG exam. According to CPO officials, the purpose of the exam was to ensure that each employee understood and could follow the requirements of the DIOG. The CPO does not usually develop exams for FBI employees. In most cases, exams are developed by the FBI’s Training Division. To create the DIOG exam, the CPO asked nine FBI attorneys and Special Agents to compose questions that would cover the DIOG. A second group of 10 attorneys and Special Agents (some of whom were in the original group) revised those questions and settled on a final group of 50 questions for the exam. Some of the questions required a “true” or “false” answer, while the others required the test-taker to choose from multiple-choice answers. The exam also included 15 “scenario-based” questions that required employees to evaluate hypothetical situations and identify permissible actions. According to a CPO official involved in the exam construction, and others we interviewed, the exam was designed to be more rigorous than many FBI training exams because the FBI wanted to ensure that everyone understood the important principles contained in the DIOG, as well as how to apply them. FBI employees told the OIG that they believed that some of the questions on the exam were poorly worded and difficult to understand. When the CPO later checked to see how often various questions were being answered incorrectly, it found that two or three questions stood out as more difficult than the others. 1. Exam platform Initially, the completed exam was placed on a computer platform called “Survey Dancer,” where it was taken by several hundred employees. The CPO determined that Survey Dancer was not an acceptable platform and the exam was moved to the “Virtual Academy” platform, which the FBI has used for years for employee training courses and exams. 5 In many ways, however, the exam was not a typical Virtual Academy exam. In addition to being more rigorous than most Virtual Academy exams, it was procedurally different as well. For example, most Virtual Academy exams tell employees immediately after they have answered a question whether their answer is right or wrong. The DIOG exam did not. Only after they had answered all the DIOG questions were employees told whether they had passed or failed, and the computer program never told them which answers were wrong. Similarly, on most Virtual Academy exams, employees can change an answer after selecting a response. On the DIOG exam, employees could not change their initial answer to a question. In addition, unlike most Virtual Academy tests, the answers on the DIOG test were shuffled. Although the questions remained in the same order on every test, the order of the answers changed; for example, the answer to Question 5 that was labeled “a” on one test might be labeled “b” on another. As a result, the order of the multiple choice answers on a test given to one employee likely differed from that given to a different employee. In addition, for an employee who opened the exam more than once, the order of the answers likely changed from session to session. Several employees noted to the OIG that no other Virtual Academy exam required employees to certify that they worked on their own and that they believed that they were allowed to consult with each other on other Virtual Academy exams. However, as explained more fully below, FBI employees were explicitly forbidden to do so on the DIOG exam. Employees who restarted the exam – either because they had failed it previously, had signed off before completing it, or had been forced off by computer or power failure – had to retake the entire exam. Their previous answers were not saved. Some employees told us that they took the entire exam and then pressed the wrong button at the end, resulting in the exam disappearing from their screens and forcing them to retake the entire exam before they were given credit. Several employees told the OIG that as a result of these factors and the difficulty of the questions, the DIOG exam was significantly harder and took significantly longer to complete than the typical Virtual Academy exam. 2. Consequences of failing the exam According to an e-mail sent to all employees before they took the exam, employees who did not achieve a passing score of 80 percent were required to take the exam again. Those who scored between 70 percent and 80 percent were allowed to retake the exam without any further training (unless they wanted it). Employees who scored below 70 percent were required to take remedial training before retaking the exam. 6 Many employees told the OIG that they did not remember reading this email and consequently did not know whether they had to take remedial training before attempting to pass a second time. Some employees learned it was possible to retake the exam without any remedial training after failing it and successfully signing back on. Other than the requirement to take the examination again (and possibly remedial training), there were no adverse personnel consequences to employees who failed the exam, even if they failed it several times. D. Instructions to Test-Takers 1. Written instructions Once they had completed the 16.5 hours of DIOG training, FBI employees received e-mails instructing them that they could take the exam. The e-mails stated that the exam was “open book” and that employees could use the DIOG or “other information” to help them answer the questions. However, the e-mail warned employees that they could not “ask co-workers or anyone else for assistance while taking the exam.” In addition, when they signed on to the Virtual Academy to take the DIOG exam, the first page that appeared on the screen stated that employees could “consult the DIOG or your notes; however, you may not consult with another person to answer the questions on this exam.” The last question on the exam, Question 51, required employees to “certify” that they “only consulted the DIOG, notes, or training aids but no other person while taking this exam.” 2. Verbal instructions Several trainers also told the OIG that during the DIOG training they emphasized the need for employees to take the exam alone, without consulting other employees. Most employees we interviewed said they remembered general warnings about not consulting with others but no specific discussions about permissible conduct within those constraints. Some of the trainers advised employees to expect that the exam would take about 2 hours to finish, and one CPO official said it was generally wellknown in the FBI that it would take approximately that long. 3. Gaps in instructions Although the exam answers had to be recorded on a computer linked to the FBI’s Virtual Academy site, it was technologically possible for employees to open the exam and print out each question, page-by-page, which gave them a 7 complete, written copy of all 51 questions. There was no official, broadly publicized policy, either written or verbal, regarding whether this was permissible or impermissible. We found that some employees printed the exam without asking anyone for permission to print the exam, while others asked for permission. In at least one case, a legal advisor in a field office we visited told an employee in an email that he could print out the exam.2 In another case, a CPO employee told a trainer over the telephone that the trainer could print out the exam for others in his office to use. Despite this lack of official policy, a supervisory employee in the CPO said in September 2009 – after thousands of employees had already taken the exam – that: If ever asked, the CPO would not have authorized the printing of the DIOG test questions to be shared with employees prior to them taking the DIOG test. Even though the DIOG and notes were authorized during test taking, it was never the intent of the CPO to have copies of the test questions available outside of actually taking the Virtual Academy DIOG test.3 In addition, we found that there were neither written guidelines nor verbal instructions as to how the trainers should conduct remedial training for employees who had failed the exam one or more times. Some trainers told the OIG that during their remedial training sessions they reviewed each question with employees who had failed and gave them the correct answers. Other trainers said that in the remedial training they reviewed the materials from the initial training sessions, concentrating on the questions that the employees had missed. Other trainers told us that they did not remember having any particular agenda for remedial training. Trainers (but not employees) were given access to a website that allowed them to see which employees had passed or failed the exam, as well as which questions they had missed. E. Exam Data The Virtual Academy program recorded certain data about the exam, including the date and time when each employee first opened the exam, the 2 In this report, we use the pronoun “he” to refer to both male and female witnesses. The supervisory employee’s statement was contained in an affidavit taken by the FBI’s Inspection Division during its investigation of the complaint alleging cheating by three top supervisors and a legal advisor in the WFO, which is discussed below. 3 8 date and time when the employee successfully completed the exam, and the total amount of time that the exam was open for each employee. For each employee who opened the exam and completed it in the same session, the Virtual Academy shows how much time that employee had the exam open. In addition, for each employee who opened the exam more than once (either because the employee failed it one or more times or because the employee shut it down before completing it), the Virtual Academy shows the total amount of time the employee had the exam open for the multiple sessions. The Virtual Academy program also recorded the status of the exam at the end of each day. For example, if an employee failed the exam twice in one day and passed it on the third try on the same day, the computer would record only the single success at the end of the day, but not the interim failures earlier in the day. F. Indications that Employees Took the Exam Unduly Quickly In response to concerns about possible cheating on the exam, the FBI determined that over 200 employees had completed the exam in 20 minutes or less. Concerned that the test could not be completed that quickly without cheating, supervisors in the FBI Inspection Division tried to determine how long it should take to complete the examination. They instructed an Inspection Division Supervisory Special Agent to take the exam by reading the questions and potential answers as quickly as possible and selecting answers randomly. Even using this abbreviated procedure, the Supervisory Special Agent reported that he needed 20 minutes to finish the exam, leading the Inspection Division supervisors to suspect that employees who finished the exam in 20 minutes or less may have cheated during the exam. Similarly, in September 2009 a supervisory employee of the CPO stated that: [S]omeone who took the test in under 45 minutes would raise the suspicion of the CPO. The CPO advised all employees to block out two hours for the test. Through discussions with various people, it seems as though it was taking employees 1.5 to 2 hours to complete the test. I would say it would be nearly impossible to take the test in 12 minutes and make a perfect score. I was part of the team who developed and wrote the test questions and I also conducted approximately 50 DIOG training and remedial training sessions. It took me approximately 30 minutes to take the test, without looking in the book or reference notes, and I did not get 100 percent. The DIOG test was built so that people could not speed through the test. It was developed so that people would find it necessary to think about the question, open the book, look at 9 their notes and apply the principles and concepts learned from training before answering the questions.4 G. FBI Investigation of WFO Allegations In September 2009, the FBI Inspection Division received an anonymous complaint that three top managers from the WFO – an Assistant Director in Charge (ADIC) and two Special Agents in Charge (SAC) – cheated on the DIOG exam by taking it together, and that WFO attorneys were present while they took the test. The Inspection Division conducted an investigation and found that the SACs had taken the exam together, in the same room, while discussing the questions and possible answers with a legal advisor, who was also present. While the ADIC was also in the room at the time, he did not take the exam that day. Instead, the ADIC wrote down the answers and later used them to complete the exam another day. In his defense, one SAC argued, among other things, that he did not cheat because although he had asked the legal advisor to “reference the sections in the DIOG for us to use to answer the question,” he never asked the legal advisor “what the answer was to a particular question.” The FBI Office of Professional Responsibility (OPR) rejected that argument and found that the SAC’s conduct “violated basic test-taking protocols and constituted cheating.” OPR also found that both SACs violated FBI Offense Code 2.4, prohibiting false or misleading statements, by certifying on Question 51 that they had not “consulted” with anyone during the test. OPR issued a decision suspending the two SACs for 20 days without pay and demoting them from a Senior Executive Service position to a non-supervisory GS-13 pay grade. The SACs appealed OPR’s final action to the Disciplinary Review Board. Pursuant to the FBI’s procedures, the demotions and service of the suspensions have been stayed pending a final decision on appeal. In his defense, the legal advisor asserted that he was not aware that his conduct “facilitated cheating.” The OPR concluded that the legal advisor had exhibited an “astounding lack of judgment” by “assisting Executive Management with the DIOG exam” – and violated FBI Offense Code 5.22, which prohibits, among other things, “conduct . . . which . . . seriously calls into question the judgment . . . of the employee.” Referring to Question 51, which expressly prohibited “consulting” with others, OPR wrote that “under no circumstances can your assistance, let alone the fact that the three men openly The supervisory employee’s statement was also contained in an affidavit taken by the FBI Inspection Division during its investigation of the complaint alleging cheating by a legal advisor and three top supervisors in the WFO, discussed more fully in the following section. 4 10 discussed questions, be considered anything less than consultation.” OPR issued a decision suspending the legal advisor for 10 days without pay. This decision is also under appeal. The ADIC argued that he had not cheated because the answers he wrote down for his later use constituted “notes,” which he argued were permissible under the open-book procedures of the exam. OPR rejected this claim, finding among other things that the ADIC committed misconduct when he certified on Question 51 that he had “consulted” with no other person while taking the exam. OPR’s proposal letter to the ADIC stated: This certification is straight-forward and it is not open to dispute that the [legal advisor’s] assistance, and your discussion with the SACs, constitute consultation. While the others may not have been in the room when you actually took the exam, you copied answers while the two SACs took the exam and used those answers to complete the test. The ADIC retired from the FBI after learning that the FBI proposed to discipline him, but before the FBI had reached a final decision about that discipline. After the conclusion of the Inspection Division’s investigation of the allegations about the WFO managers’ conduct on the DIOG exam, the FBI reported to us that it subsequently received allegations that employees in addition to those in the WFO may have cheated on the exam by sharing answer keys or by receiving assistance. In addition, the FBI informed us that a high number of FBI employees had completed the exam extremely quickly and received high passing scores, which appeared to be inconsistent with the design of the exam if it was taken online without an answer key. In light of these concerns, we opened this investigation in January 2010. III. OIG INVESTIGATION In this section, we first describe the results of our interviews in four field offices. We then discuss our findings regarding three individual allegations of cheating on the DIOG exam that we received from the FBI. A. Field Office Interviews The OIG chose the four field offices based on two criteria: the number of employees who finished the exam quickly or e-mail communications suggesting that one or more employees in the office may have cheated on the DIOG exam.5 We selected those criteria expecting that they would be indicative of employees cheating on the exam. However, our investigation found that these criteria were not (Cont’d.) 5 11 1. Field Office 1 We interviewed 11 employees in Field Office 1 and found evidence at this office of a significant number of FBI employees who cheated on the exam. Seven of the 11 employees we interviewed – a legal advisor, who we call Attorney James; an Assistant Special Agent in Charge (ASAC); a Supervisory Special Agent (SSA); and four special agents — acknowledged to us that they had used an answer sheet while taking the exam. Several of the seven said they also gave answer sheets to other employees in the field office. In addition, Attorney James acknowledged consulting with another agent while taking the exam. We also concluded that another one of the field office’s legal advisors, who we call Attorney Clark, attempted to impede the OIG investigation into this matter and lacked candor when questioned about those actions. a. Use of answer sheets The employees we interviewed at this field office told us that answer sheets for the exam were freely available within the field office. One agent, for example, said he received two different answer sheets, one of which contained more correct answers than the other. He also told us that answer sheets seemed to be “floating around” his entire (14-person) squad. Another agent said that he gave his answer sheet to 15 other FBI employees in that field office. The seven employees who told us that they used answer sheets worked in three different squads within the office, as well as in the legal advisor’s office and executive management of the field office. The field office employees described different types of answer sheets that were circulated. One agent told us that he saw two handwritten versions that contained both the questions and answers to the DIOG exam. He said that most of the questions and answers were in the correct order, but some were out of order and a couple of questions had the wrong answer. Another agent told us that he received a handwritten answer sheet from an agent and then he converted it to a typed answer sheet, which he gave to others. He provided us a copy of that typed answer sheet. It did not contain the exam questions, but instead consisted of a list numbered 1 through 50. necessarily related to cheating on the exam. Rather, we found that some persons who cheated took a long time to complete the exam, while others who did not cheat finished very quickly. As a result, quick exam times were not directly related to cheating. Similarly, the e-mails we found did not, for the most part, come from persons who we determined had in fact cheated. Therefore, while our investigation was intended to target field offices where there were indications of cheating, the criteria that we picked ended up being less useful in predicting cheating than we had expected. 12 Next to each number is a short phrase (such as “true” or “type 5”) giving the answer to each of the 50 exam questions. Attorney James admitted creating two answer sheets that consisted of the question number and the answer (for example, 1 – a, 2 – true, 3 – c, etc.) Attorney James told us that he gave the first answer sheet to an SSA who was also a trainer and the second answer sheet to an ASAC. Attorney James also said that he distributed the answer sheets because he felt that the DIOG exam was just another administrative hurdle that FBI employees had to surmount. Attorney James stated that if he had realized that the FBI considered the exam to be so important, he never would have given out the answers.6 According to the ASAC who received the answer sheet, after failing the exam once, he had approached Attorney James (who was one of the office’s trainers) to ask for assistance. The ASAC said that although he did not solicit it, Attorney James gave him an answer sheet and told him that he was not sure how reliable it was. The ASAC said he used the answer sheet when he subsequently retook the exam. The SSA who also received an answer sheet from Attorney James told us he was so busy at work that he missed two deadlines for taking the exam. During a conversation with Attorney James about an unrelated matter, the SSA mentioned that he had missed the deadlines for taking the exam. The SSA said that soon afterward Attorney James gave him a piece of paper containing handwritten answers to some of the questions. He told us that when he subsequently took the DIOG exam, he reviewed some of those answers. In addition to the ASAC, the SSA, and Attorney James, four other Field Office 1 agents we interviewed also acknowledged using answer sheets when taking the exam. When we asked them why they had used the answer sheets when taking the exam, FBI employees gave varying answers. The ASAC told us that once he had accepted the answer sheet from Attorney James, he was “past the point of return.” The SSA said that although he looked at the answer sheet while he took the exam, it was not very useful to him and he did not really “rely” on it. He said that in retrospect, he should have handed back the answer sheet “to remove any gray areas.” Several other field office employees claimed that they viewed the answer sheets as “notes” allowed under the DIOG exam instructions, which stated that Attorney James also told us that he used an answer sheet that he had written while sitting in Attorney Clark’s office while Attorney Clark was taking the DIOG exam, as described more fully below. 6 13 “you may consult the DIOG or your notes” while taking the exam. Others, including two agents, said that they were pressed at the time by the demands of their case work and were seeking to satisfy the DIOG requirement quickly. Another agent responded that he was being pushed to pass the test or take remedial training. Another agent questioned why it was necessary for him to pass an exam when even if he scored 100 percent he would still need to consult the DIOG on a regular basis. We interviewed five of the six Field Office 1 trainers, including Attorney James, and asked them whether they were aware of or had condoned the use of answer sheets in the field office. Three trainers said they had neither used nor distributed answer sheets. They told us that they did not know that answer sheets had been circulating in the office and that, had they known, they would have considered it to be improper and would have instructed employees to stop. Attorney James told us that he had heard rumors in 2009 about other answer sheets “floating around the office,” but that he did not consider it to be his responsibility to investigate those rumors. Another trainer, the SSA who had received an answer sheet from Attorney James, said he did not know other answer sheets were available in the office. As described earlier, one of the agents who used an answer sheet told us that he had provided an answer sheet to 15 additional employees, and another agent said he thought that everyone in his 14-person squad had received a copy of an answer sheet. We did not interview all of these additional employees or further investigate whether or how many others in this field office had used answer sheets. In addition to the use of answer sheets by Field Office 1 employees, our investigation determined that at least one of the employees we interviewed directly consulted with others while taking the exam but answered Question 51 by certifying that he had not done so. We discuss this in the next section. b. Attempt to obstruct the investigation In May 2009, Attorney James, who had been selected to be one of the DIOG trainers for the office, tried to pass the DIOG exam and failed. Soon afterward, he talked to Attorney Clark and told him that he had failed the exam. According to Attorney James, Attorney Clark said something to the effect of “let’s just get this thing done,” walked into Attorney Clark’s office and signed onto the exam on his own computer. Attorney Clark then took the exam on his computer, while Attorney James sat nearby in the office. According to Attorney James, they discussed the questions and answers as Attorney Clark was taking the test and, once the correct answer had been selected, Attorney James made a note of it on a sheet of paper. Attorney James said that he then returned to his own office and retook the exam. Attorney James also told us that toward the end of his second attempt to pass the exam, he called a friend 14 in another office to ask him about the answer to a particular question. Attorney James passed the exam on the second attempt. Despite consulting with a friend during the exam, Attorney James certified in response to Question 51 that he had not “consulted” with other persons while taking the exam. When asked why he had done this, Attorney James told us that he “just didn’t think about it.” In April 2010, the OIG began interviewing employees in Field Office 1 about the DIOG exam. According to Attorney James, shortly after learning about the OIG’s telephone calls to the field office, Attorney James and Attorney Clark started discussing what issues might be covered during any upcoming OIG interviews. Attorney James said that he did not remember their exact discussions, but that among other things Attorney James told Attorney Clark that he (Attorney James) was going to be completely candid with the OIG and would tell the OIG that he had been sitting in Attorney Clark’s office while Attorney Clark was taking his exam. Attorney James said that during these discussions he grew frustrated with Attorney Clark because Attorney Clark refused to admit that that had taken place. Attorney James said he eventually told Attorney Clark that he (Attorney James) had been in Attorney Clark’s office legitimately because it was a “remedial situation” at the time – and that therefore Attorney Clark had been permitted to have Attorney James in the room at the same time. According to Attorney James, he did not really believe that it had qualified as remedial training, but suggested that only because he wanted to couch it in a way that would allow Attorney Clark to acknowledge that they had taken the exam together. Attorney James said that during one of these discussions, Attorney Clark told Attorney James, “don’t throw me under the bus.” Attorney James told the OIG that he considered this comment to be a request for him to be less than candid with the OIG, particularly about the time that he had sat in Attorney Clark’s office while Attorney Clark took the exam. When we interviewed Attorney Clark, he said that he had allowed Attorney James to sit in his office while he took the exam because he was giving Attorney James remedial training. Attorney Clark agreed that he had told Attorney James, “don’t throw me under the bus,” but claimed that the comment was not an attempt to stifle Attorney James’s candor. Attorney Clark initially told the OIG that he made the comment nearly a year earlier, in 2009, shortly after Attorney James told him that he (Attorney James) had failed the exam, and not in 2010 when the two of them were discussing Attorney James’s upcoming interview with the OIG. Attorney Clark said he was concerned that Attorney James’s credibility as a trainer might 15 suffer if Attorney James told others that he (Attorney James) had failed the exam, so Attorney Clark said, “don’t throw me under the bus,” meaning that Attorney Clark wanted Attorney James to hide the fact that he (Attorney James) had failed the exam. Later in the interview, however, when asked if Attorney Clark might have made the comment in 2010, Attorney Clark told us that he could not remember exactly when he had made that comment. We did not find credible Attorney Clark’s statement that he said, “don’t throw me under the bus” to encourage Attorney James to hide the fact that Attorney James had failed the exam. First, the comment “don’t throw me [Attorney Clark] under the bus” does not seem logically related to the concern that Attorney James’s credibility as a trainer would suffer if others knew that Attorney James had failed the exam. Second, Attorney James was certain that Attorney Clark made the comment in 2010, as they were discussing Attorney James’s upcoming OIG interview. Assuming it was made in 2010, the DIOG training was long over and Attorney James’s credibility as a trainer no longer was an issue. We credit Attorney James’s statement that Attorney Clark was encouraging Attorney James not to reveal that Attorney James had been present in Attorney Clark’s office while Attorney Clark took the exam. We conclude that Attorney Clark attempted to impede an OIG investigation and lacked candor when we asked him about his attempt to do so. c. OIG Analysis We found cheating in this field office on the DIOG exam by a significant number of FBI employees. Of the 11 employees we interviewed, 3 supervisors and 4 agents told us that they used answer sheets for the exam. These agents worked in various squads in the office, and they indicated that answer sheets were “floating around” their squads. We found unconvincing the employees’ arguments that it was permissible to use answer sheets because they were “notes” that were explicitly allowed under the open-book procedures established for the DIOG exam. We do not believe that any FBI employee could reasonably believe that using someone else’s answers to the exam constituted using “notes” or could be viewed as complying with the test instructions, and verbal guidance, requiring them to take the test without assistance from others. Their argument is similar to that offered by the former ADIC of the WFO, who argued to FBI OPR that the list of questions and answers he created while watching others take and discuss the exam were permissible “notes,” which he later used in taking the exam himself. In rejecting that argument, FBI OPR reasonably concluded that by copying and using other people’s answers to the exam, the ADIC effectively “consulted” with others even though he received the 16 answers before he took the exam himself. The same reasoning applies to using answer sheets created by others to take the exam. Similarly, we concluded that those who distributed answer sheets for use by others acted improperly. By providing answers to others for their use in taking the exam, these employees facilitated potential cheating and violated basic test-taking protocols. In the WFO matter, FBI OPR concluded that the legal advisor who discussed the exam questions with the ADIC and other managers while the other managers took the exam had committed misconduct. We see no distinguishing difference between providing an answer sheet to an employee before that employee takes the test, as happened in Field Office 1, and providing answers to employees while some of them are taking the test, as happened in the WFO case. The argument that the use of answer sheets was somehow excusable because the employees were too busy trying to complete their other duties is also unpersuasive. The exam did take longer than the typical Virtual Academy exam, but the FBI made it clear from the start that the DIOG exam was important, that it would be different from other exams, and that employees should take it seriously and do their own work to pass the test. The employees who used answer sheets ignored those warnings. In addition, we were troubled that Attorney James consulted with a friend to help answer a question, and then falsely certified on the exam that he had not consulted with any other person in taking the exam. Attorney James also provided answer sheets for the exam to two other supervisors, who acknowledged that they used the answer sheets in taking the exam. Finally, we believe that the cheating on the exam was not confined to the limited number of employees we interviewed in this office. Despite the instructions about taking the exam on your own and the certification that the test-taker had consulted no other person when taking the test, other employees in this office had access to answer sheets when taking the exam. While we cannot say with certainty whether or how many of those employees actually used the answer sheets, based on the evidence in our investigation we believe it likely that at least some of them did. 2. Field Office 2 We also interviewed FBI employees in Field Office 2 about the DIOG test. First, and most troubling, we found four agents who took advantage of a flaw in the Virtual Academy computer program to reveal the answers to the questions as they were taking the exam. Our investigation also found widespread distribution by employees in this field office of “study guides” that contained the 50 questions on the DIOG and the portion of the DIOG that contained the answer to each question. In addition, we found that one agent created an 17 answer sheet that he sent to an employee in another field office, and another employee from this field office consulted with others while answering the questions on a printed copy of the exam. a. Use of Virtual Academy computer program to view the answers to the DIOG exam An FBI agent who works on a cyber crime squad in Washington, D.C., told the OIG that sometime in 2008 he was experimenting with Virtual Academy computer programming to see how secure it was when he noticed some “really sloppy” coding that amounted to a programming flaw. According to the agent, the flaw allowed any FBI employee taking some Virtual Academy exams to open an XML file located in the employee’s computer’s “Temporary Internet Files” folder and view all the answers to the exam.7 The agent told us that he created a computer tool that made it easier to see the DIOG exam answers by taking employees directly to the XML file that contained the answers. While the answers were available without the tool to anybody who realized where to look for them on their computer, he said the tool made it slightly easier to get to the answers. He told us that he created this tool mostly as a training device for other cyber squad employees. He said that he did not use the computer tool or the XML file to take the DIOG exam, and that he took the exam on his own without assistance from anyone else. However, the agent told us that during a computer class in March 2009 he described both the programming flaw and the computer tool that he had created to another cyber agent who worked in Field Office 2. The agent told us that a week later he sent the computer tool to the other cyber agent in Field Office 2 as an attachment to an e-mail. The agent said that he sent the computer tool only after receiving verbal assurances from the Field Office 2 agent that he would not forward the computer tool to anyone else. We reviewed the e-mails between these two agents. In one e-mail, the Washington, D.C., cyber agent mentioned the recent computer class that the two agents had attended together and explained the use of the computer tool, noting that it could be used to view the answers on most Virtual Academy exams. The agent added, at the bottom of the e-mail, the following: 7 A Virtual Academy programmer told the OIG that he and others who worked with him were aware that the answers were available in the XML file. He said that because the Virtual Academy is designed for training, and not testing, the availability of answers in the XML file had never before been a problem. In addition, he said a person would have to be particularly knowledgeable about computers to locate the file, much less use it. The Chief of the CPO told us that nobody at Virtual Academy had alerted him to the XML issue and that he had not known about it until the OIG interviewed him during this investigation. 18 “DISCLAIMER: This is only a learning tool, not to be used for official testtaking purposes.” The Field Office 2 agent who received the e-mail told us that he also took the exam on his own without using the computer tool or the XML file to look at the answers. He said, however, that after answering the questions on his own, he looked at the XML file to double-check his answers. He said that he did not believe he violated any FBI rules because he did not change any of his answers after reviewing the correct ones in the XML file. However, this field office agent forwarded the e-mail and the attached computer tool he had received to four other agents on his cyber squad in the field office. At the bottom of this forwarded e-mail, the field office agent added this comment: “Depending on how lazy you are, [this program] will make taking the tests faster.” We interviewed the four agents on the cyber squad who received this email. Three of the four agents told us that they used the computer tool or the XML file while taking the DIOG exam. One of the agents told us that he used the computer tool attached to the e-mail to check some of the answers on the exam. He said that he did not use the tool to check every answer, but that when he had trouble answering a few of the questions, he used the program to find those answers. He told us that he felt he could answer “yes” to Question 51 because, even though he had assistance, he did not receive assistance from another person. Another one of these agents told the OIG that he did not use the computer tool, but did look at the answers in the XML file. According to this agent, he failed the exam the first time he took it. He said that an Associate Division Counsel subsequently gave him remedial training by reviewing the questions he had missed and providing the correct answers for each. He also told us that by the time he retook the exam he had learned that the answers were all available on the XML file. He said that he used the answers in the XML file to help him complete the exam. He scored a 100% when he retook the exam. According to this agent, it was “common knowledge” within his office that you could see the answers to the DIOG exam by opening the XML file. He told us that knew that he was cheating at the time, but he still answered “yes” to Question 51 because he did not receive assistance from another FBI employee during the exam. The third agent said that he also looked at the answers on the XML file. According to this agent, he answered all the exam questions without help from the XML file, but then changed a few of his answers after checking them against those in the XML file. He said that he did not believe he cheated because he worked on his own to answer the questions. He also told us that 19 others on his cyber squad knew that the answers were available in the XML file. b. Study guides Several employees in Field Office 2 told us that different study guides on the DIOG exam were available in various parts of the office. While the study guides were prepared by various employees and took slightly different forms, most conformed to a basic pattern. In the study guides, the 50 DIOG exam questions were printed out and assembled in a notebook. Behind each question was a photocopied page containing the portion of the DIOG that was responsive to that particular question. To make matters easier, some of the study guides included highlighting that emphasized the particular portion of the DIOG page that contained the answer. Based on these descriptions, we concluded that while the study guides were not explicit answer sheets, they served a similar purpose.8 Of the 15 employees we interviewed in Field Office 2, 4 said they had used a study guide in answering the DIOG questions: One agent told us he used a study guide that had been left on a table in a resident agency for the use of anyone who wanted it. That agent told the OIG that he copied the study guide and returned it to the table, and then used his copy to study for and take the exam. An ASAC told us that an agent gave him a study guide, which he used to prepare for the exam. The ASAC said that when actually taking the exam he did not generally consult the study guide, but may have looked at it once or twice. An agent said that he had given the ASAC a study guide. According to the agent, he received the study guide from a friend in the FBI and then modified it by adding tabs and highlighting that made it more useful. He said he used the study guide while taking the exam. An agent told us that, in addition to using the XML files to see the answers, he also used a tabbed, marked DIOG manual that directed users to the correct answers for each question. He said he corrected some of the mistakes in the manual and passed it on to others in the office. We were unable to obtain a copy of any of these study guides. Several Field Office 2 employees told us that in connection with a move of the field office that occurred after they took the exam, they shredded unnecessary documents, including the study guides. 8 20 The Field Office 2 employees who used the study guides said that they did not consider their actions to be improper. They said that they believed the study guides qualified as “notes,” which they were permitted to use while taking the exam. Field Office 2’s chief trainer, its Chief Division Counsel, told us that he did not know that employees were using study guides. He said that he did not like the idea of using study guides and would not have encouraged their use. He also said, however, that he would not “hang someone” for using study guides because employees who used them were “going to be forced to go look at the DIOG to get the answer, so it achieved the purpose the test was designed for.” c. Answer sheets We also found that answer sheets were distributed in Field Office 2. One agent, who we call Agent Smith, told us that he received a printed copy of the exam with all the answers already filled in that had been created by a support employee at the request of another agent in the field office, who we call Agent Jones. Agent Smith said he did not rely on the answer sheet because he was not sure if it was correct. He said that he instead studied by himself and provided his own answers on the exam.9 The support employee told us that he printed out the exam and, working on his own, wrote down the correct answers to each question. After he had finished his work, he left the printed questions and handwritten answers on his desk. He said that Agent Jones asked him where his answers were and he told him. He said he did not remember telling Agent Smith about the answers and was not sure if Agent Jones had used his answers. Agent Jones said that he studied for and completed the exam using a paper copy of the exam, but that he did not receive his paper copy from the support employee. He denied that he had asked for or even knew about the support employee’s paper copy of the exam with the answers. Another field office agent (who, as discussed above, also used a study guide) told the OIG that he also created an answer sheet and sent it to a relative who worked in a different FBI field office. The OIG reviewed a copy of that answer sheet, which contained 50 complete sentences, each of which contained the correspondingly numbered exam question with the answer Agent Smith also told us that his supervisor said that there was a printed copy of the DIOG in which employees had been “highlighting what they thought was important” and his supervisor said that “everyone’s been using [it] to take their test.” 9 21 incorporated into the question. That same field office agent also told us that he had heard rumors, probably from the relative who received his answer sheet, that employees in the relative’s office had been collaborating on the exam. The CDC in Field Office 2 said that he did not know that anyone in his office had printed out either answer sheets or copies of the exam with the answers marked in. The CDC said that had he known that, he would have considered it improper and would have advised the employees to shred the answers immediately. d. Consulting with others Agent Jones said that while he was answering the exam questions on his paper copy, he consulted with other agents in the field office when he had problems answering three or four of the questions. However, on Question 51, he certified that he did not consult with anyone else. He told the OIG that he thought his answer to that question was accurate, even though he consulted with other agents, because, “I thought the whole idea of the DIOG was to generate an understanding between the agents.” e. OIG Analysis We believe that Field Office 2 agents cheated on the exam when they looked at the answers to the exam by using a computer tool or XML files. We believe that under no circumstance could this conduct be considered consistent with the test-taking instructions, and the agents did not attempt to justify it as such. In addition, we found that various field office employees, including the ASAC, used study guides created in whole or in part by others that contained highlighted portions of the DIOG attached to each of the 50 exam questions, in effect giving users the answer to each question on the exam. We believe that these “study guides,” which in fact were the 50 exam questions with the answers located behind each question, served a similar purpose as answer sheets. In addition, we found that one agent discussed the correct answers with others while working on a paper copy of the exam in preparation for entering answers on the electronic exam. The instructions accompanying the electronic exam stated that employees “may not consult another person to answer the questions on this exam.” We do not believe an employee could reasonably think it would be impermissible to consult with others while taking the exam on a computer, but permissible to consult with others while answering questions on a paper copy of the exam. This agent also certified on Question 51 that he had not consulted with other employees, when this was not true. 22 3. Field Office 3 We interviewed 10 FBI employees in Field Office 3. We learned that virtually every employee in that field office was provided with a printed paper copy of the exam, which enabled them to work on the answers on the paper copy before signing into the electronic version and completing the exam. As noted earlier, the FBI had no official publicized policy, either written or verbal, regarding whether FBI employees were permitted to print out the exam questions prior to taking the exam. In a September 2009 affidavit, a supervisory employee in the CPO said that if his office had been asked, it: would not have authorized the printing of the DIOG test questions to be shared with employees prior to them taking the DIOG test [and that] . . . it was never the intent of the CPO to have copies of the test questions available outside of actually taking the Virtual Academy DIOG test.10 However, his intent and this instruction were not formally communicated to others throughout the FBI who were taking the test. Moreover, in Field Office 3 the paper copies were distributed by Field Office 3’s training coordinator, an agent. He told us that he had received permission from a CPO employee to print out the exam and distribute it widely. The CPO employee confirmed to us that he gave the agent permission to print out the exam, although he said he believed at the time that the printed copies would be distributed to only a few employees who were having trouble accessing the computer version and not to the entire office. Similarly, an Associate Division Counsel in another field office sent an e-mail allowing one employee to print out the exam. All the Field Office 3 employees we interviewed said they had used the printed exam to work on their own and had not consulted with others about the answers – either while working with the printed exam or while taking the electronic version. Under those circumstances, we do not believe that using printed copies of the exam questions to take the test or to prepare for taking the test online was improper. We also found no evidence that Field Office 3 employees had used answer sheets, study guides, or computer programs to reveal the answers to the questions on the exam like the employees in Field Office 2. As noted above, the supervisory employee made this statement in an affidavit that he wrote when asked about the allegations of cheating on the exam by a legal advisor and managers in the WFO. 10 23 4. Field Office 4 We interviewed seven FBI employees in Field Office 4. Of those seven, four told us they had obtained printed versions of the exam that showed only the questions. They all said they worked on the answers on their own, as instructed by trainers during the pre-exam training sessions. However, we determined that one Field Office 4 agent we interviewed completed the exam and left a copy of both the questions and the answers on his desk. The agent then sent an e-mail to a secretary telling him that he was going to be out of the office, but that he had scored 100 on the test. His e-mail instructed the secretary to tell the squad supervisor and “anyone else who is interested in ‘studying’ my notes that I will leave them on my chair.” In fact, these “notes” were his copy of the exam questions and answers. The agent told us that he believed no one used his “notes” because his set of questions and answers did not appear to have been moved when he returned to the office and no one thanked him for providing them. The agent acknowledged to the OIG that his e-mail was improper and that any employee who had used his answers would have been acting improperly. None of the seven FBI employees we interviewed said they used these notes when taking the test. However, we did not interview others in Field Office 4 to determine if anyone had in fact used the agent’s “notes” or any other answer sheets. B. Other Complaints of Cheating 1. Resident Agency In December 2009, the Special Agent in Charge (SAC) of a field office received information that the Senior Supervisory Resident Agent (SSRA) in one of the field office’s resident agencies had allegedly cheated on the DIOG exam by soliciting help from a Special Agent he supervised.11 The SSRA admitted to the SAC that he had received help from the Special Agent, who we call Special Agent Johnson. In addition, the SSRA subsequently wrote a letter to the SAC stating that he had also received an answer key to the exam from an SSA within the resident agency. The OIG interviewed nine employees in the resident agency and field office and reviewed documents provided by several of them. We determined that the SSRA had asked his subordinate agent, Agent Johnson, for help on The SSRA has since been terminated by the FBI as a result of the investigation that it conducted into allegations of misconduct unrelated to those concerning the DIOG exam. 11 24 the exam and had also accepted an answer key from an SSA. In addition, the SSRA had accepted help on the exam from a third employee, a support services technician who was also subordinate to the SSRA. The SSRA told the OIG that he had failed the exam several times and had become frustrated and worried about what might happen if he could not pass it. He said he asked two employees that he supervised – both Agent Johnson and the support services technician – to help him with the exam. Both employees sat with the SSRA in his office while he re-took the exam – at two different times – and both discussed the questions and possible answers with him. Agent Johnson told the OIG that he tried to leave the SSRA’s office several times because Agent Johnson was uncomfortable helping, but that the SSRA asked him to come back in each time. The SSA told the OIG that he had failed the exam twice and was frustrated that the test did not provide feedback on which questions he answered incorrectly. The SSA was also embarrassed that he and the SSRA were the only agents in the office having trouble passing the exam. Therefore, the SSA said he took the exam a third time and decided to write down most of the questions and his answers as he went along. That way, he said, if he failed a third time he would be able to independently research the questions to determine where he had been wrong. After he passed the test and did not need the list of questions and answers, he offered them to the SSRA, who accepted them. However, the SSRA said he did not use the answer sheet when he eventually passed the exam, although he did rely on help from Agent Johnson and the support services technician. We concluded that the SSRA knew that he was not supposed to receive assistance on the exam but nevertheless asked for and received unauthorized help twice. In addition, the SSRA knowingly provided false information on the DIOG exam by certifying in answering Question 51 that he had not consulted with anyone in taking the exam when in fact he had. The SSRA admitted knowing the answer was false when he gave it, and, according to Agent Johnson, who was with the SSRA, the SSRA commented at the time he answered the question that he had not, in fact, taken the exam without help. Agent Johnson said he helped the SSRA during the exam because he was worried about the possible consequences of not helping his supervisor. However, Agent Johnson knew that the assistance he provided was not permitted under the DIOG exam protocols, and he knew that the SSRA gave a false answer to Question 51. We believe that Agent Johnson, by consulting with the SSRA while the SSRA was taking the DIOG exam, also acted improperly, in clear violation of the FBI’s exam rules. The technician also helped the SSRA, in part because the technician felt bad that his supervisor was having trouble passing. Although the technician 25 was not required to take the DIOG training or exam and consequently did not receive the same admonishments about prohibited assistance that the SSRA and Agent Johnson received, the technician was aware that the SSRA was not supposed to have help taking the exam. In addition, the technician told us that he volunteered his assistance and did not act because of any pressure from the SSRA. Finally, we found that the SSA of his own initiative offered most of the DIOG test questions and his answers to the SSRA, who was not the SSA’s supervisor. Although the SSA said he felt bad that the SSRA was having trouble passing, the SSA’s conduct violated DIOG exam protocols. Moreover, the SSA was a supervisor himself and was not acting pursuant to any request by or pressure from the SSRA. We believe that SSA also acted improperly. 2. FBI Training Division Component An anonymous complainant alleged in writing to the FBI Inspection Division in November 2009 that a supervisor of an FBI Training Division component had encouraged all his employees who had not yet taken the DIOG exam to attend a “review session,” during which they were given all of the questions and answers to the exam. The OIG investigated this complaint and interviewed 15 of the component’s current and former employees about the exam. We found that a trainer conducted a training session attended by the supervisor and six other employees. During that session the trainer read out loud each of the DIOG exam questions verbatim, in order of their appearance on the exam, and discussed the correct answers to each. One component employee told us that another employee asked during the session if these were the actual questions that would be on the exam and the trainer replied that they were not. However, the trainer later told the OIG that they in fact were the actual questions. We interviewed the supervisor, who said he took notes at the training session and relied on those notes to answer approximately half of the exam questions. He said that anyone who used notes taken during the review session and had “half a brain” would do well on the exam. The supervisor said that his notes did not “mirror” the exam and he did not feel, either during the training session or later, that the trainer had used the actual exam questions. However, four component employees told us that they noticed as they took the exam that their notes seemed very similar to the exam questions. We also reviewed notes taken by one employee during the training session and noted that they were numbered 1 through 50. Because the notes 26 were handwritten they were difficult to decipher in places, but they appeared to include both the exam questions and answers in the same order as they appeared on the exam. The employee who took those notes told us that he did not realize during the training session that the trainer was using the actual exam questions and answers. He said that soon after starting the exam and noting the equivalence between his notes and the exam questions, it became clear to him that the trainer had, in fact, given all the questions and answers, in order. Several component employees told us that notes similar to those we reviewed were passed around the office after the training session. One employee who had been out of town during the training session told us that upon his return to the office the supervisor personally handed him another employee’s notes from the review session. According to the employee who received the notes, when he subsequently took the exam he noticed that the notes seemed to paraphrase the questions on the exam, were in the same order as the questions on the exam, and provided the answers to the questions. Therefore, the evidence indicated that many component employees – not just those in the training session – were provided with notes that, in effect, listed the questions and answers to the exam. None of the employees who attended the training session said they realized before attending that they were going to be handed the answers, and, as noted above, the trainer denied during the actual training session that he was reading the questions verbatim. However, we found that some employees realized what was happening soon afterward when they started taking the exam and discovered that their notes conformed closely to the exam. The trainer told the OIG that he did, in fact, read the 50 questions and each possible answer out loud, in order, when he taught the training session. After discussing the possible answers, he stated which answer was correct and why. He said he conducted the training session the way he did because one of the component attendees had previously failed the exam. He said that during remedial sessions for employees who had failed the exam, he routinely went over each question and each answer. He also said that in retrospect, he believes he should have given two different sessions: one remedial session for the employee who had previously failed, during which he would have reviewed the exam questions individually, and a separate training session for employees who had not yet taken the exam, during which he would not have used the actual questions. We concluded that the trainer used poor judgment when he read the questions and answers verbatim in the training session. It should have been clear to the trainer from the instructions the FBI had given in connection with the training for and administration of the DIOG exam that he was supposed to 27 be training on the DIOG in general and that it would be inappropriate to give his students what in essence was an answer key to the exam. We also concluded that the supervisor who attended the session used poor judgment. Although we do not believe that he realized beforehand that the training session would include the questions and answers, we believe that he should have appreciated – either during or shortly after the session – that had in fact occurred. Other component employees realized this and we believe the supervisor should have also. More important, we believe he should have discouraged his employees from passing around notes taken during the session which, in effect, provided the exam questions and answers. 3. Counterterrorism Division Unit The OIG received information that an employee in an FBI Counterterrorism Division Unit e-mailed a printed DIOG answer sheet to two other employees in August 2009. We reviewed the e-mail, which was sent on August 7, 2009, by an intelligence analyst in the Unit to two other analysts who worked in the same Unit. The subject line of the e-mail contained a single word: “blah.” The email contained no text and included an attachment labeled “Exam questions for DIOG.doc.” The attachment was a 5-page typed document that contained 47 of the 50 questions from the DIOG, in order, with answers for each of the questions.12 Later that same day, one of the two recipients sent the sending analyst a reply e-mail that stated, “Thanks buddy.” The other recipient did not respond. We compared the e-mailed answer sheet with the official answer sheet prepared by the FBI and determined that the e-mailed answer sheet had six incorrect answers. Test data provided by the FBI show that the Unit analyst who sent the answer sheet first took the exam on July 15, 2009, and failed it at least once before passing it on October 6, 2009. The analyst who e-mailed the answer sheet told us that he created the answer sheet after he had taken the exam the first time and failed it. When he took it a second time, he said, he wrote down the questions and the answers so that if he failed a second time he could review them and determine where he had gone wrong. He said he subsequently agreed to send a colleague a copy of the DIOG questions only, but he inadvertently sent both the questions and the answers. He told us that he did not remember how the topic of sending the 12 The answer sheet did not contain the questions or answers for three questions (2, 5 and 49). 28 questions to his colleague arose. He said that he passed the exam in October, on his fourth try. We also interviewed the two Unit analysts who received the e-mail. One told us that he briefly opened the attachment when he received the e-mail in August. He said he did not look at it long enough to realize that it contained an answer sheet until several days later, when the sender came to apologize to him, telling him that he had meant to send only the questions but had instead inadvertently sent both the questions and the answers. He said that after learning this, he did not attempt to open the attachment again. The other recipient told us that he opened the attachment very briefly, did not realize what it was, and closed it without examining it further. Both said that they did not use the answer sheet to help them pass the exam and that they took the exam for the first time in October shortly after receiving a warning from their supervisors that the deadline for completing the exam was quickly approaching. We compared the answers that the three Unit analysts gave on their exams and found that each received a different score from the others and often missed different questions. The analyst who sent the sheet first opened the exam on July 15, 2009, and eventually passed it on October 6, 2009, after having spent a total of 318 minutes working on it. FBI computer data shows that he gave incorrect answers (on his passing exam) to questions 12, 19, 32, 34, 41, 45, 46, and 50 for a final score of 84%. One of the analysts who received the answer sheet opened the exam on October 7, 2009, at 9:30 a.m. and passed it after spending close to 2 hours working on it. FBI data shows that he gave incorrect answers to questions 5, 7, 13, 24, 46, 49, and 50, for a final score of 86%. The other analyst who received the answer sheet opened the exam on October 7, 2009, at 1:42 p.m. and passed it after spending 38 minutes working on it. FBI data show that he gave incorrect answers to questions 12, 32, 41, 46, and 50, for a score of 90%. We credited the statement of the person who created the answer sheet that he had done so solely to help himself in the event that he failed the exam. We also credited his statement that he sent the document to two colleagues thinking that it contained the exam questions only and not the answers. The evidence supports the statements of his two colleagues that they opened the document, did not realize what it was, and did not use it to help them complete the exam. 29 IV. CONCLUSIONS AND RECOMMENDATIONS A. Conclusions In our investigation we interviewed employees in four FBI field offices and also investigated three complaints of alleged cheating. In that limited review, we found significant abuses and cheating on the FBI DIOG exam. Many of the FBI employees we interviewed commented on perceived deficiencies in the DIOG training and exam, such as the amount of time consumed by the training and the exam, the difficulty of the test, and the exam’s wording and administration. The training was time-consuming, the exam was substantively harder than most FBI training exams, its administration was different from previous exams, and some of the questions were poorly phrased. However, we do not believe those factors excuse the cheating and improper conduct that we found that some FBI employees engaged in when taking this test. The new DIOG is a fundamental document for ensuring that FBI investigations are conducted according to law and policy, and the FBI appropriately placed a high priority on ensuring that FBI personnel learned and understood the new guidelines through training and mandatory testing. Yet, a significant number of the FBI employees we interviewed cheated on the test and did not comply with the test conditions. We agree, as some employees told us during this investigation, that the administration of and communication about the exam for FBI employees could have been better. The training was conducted by several hundred different trainers in scores of different classrooms, so different employees in different locations probably received different information. Because few of the training sessions were recorded, it is impossible to know exactly how the trainings differed, although some employees told us they remembered very little discussion about consulting with other employees while others said it was clearly covered. In addition, as some employees told us, there were some glitches with the electronic version of the exam. Some employees were frustrated, for example, that they took the entire exam and closed it, only to learn later that there was no record of their efforts and thus they had to retake the exam. Furthermore, we agree with the many employees who commented that the DIOG exam was significantly different from the typical FBI exam. It was more rigorous, did not tell employees as they selected answers whether their answers were correct or not, did not allow employees to return to a previous answer to correct it, and specifically prohibited employees from collaborating with others. 30 Despite these differences, the written test instructions and most trainers effectively communicated the core instruction that although the exam was “open book” employees were not permitted to consult with others in answering the exam questions. Yet, as detailed in this report, we found test-taking conduct that constituted cheating and abuse, such as the use of answer sheets when taking the exam; the use of study guides that in effect gave the questions on the exam with the answers; consultation with others during the exam; false certifications in response to Question 51; and, even the use of a computer programming flaw to reveal the correct answers to the exam. We found distribution or use of answer sheets, particularly in Field Office 1, but also in two of the other offices we visited. Ten FBI employees we interviewed (in Field Offices 1, 2, and 4, and a resident agency) used or distributed answer sheets on the exam, and we received information that answer sheets were widely available to others in Field Office 1. We concluded that by using other people’s answers to the exam, FBI personnel “consulted” with others in violation of test-taking protocols, even though they received the answers before taking the exam. Similarly, we concluded that by providing answers to others for their use in taking the exam, those who distributed answer sheets for use by others acted improperly. We also found distribution and use of “study guides” that contained the questions to the test and the answers to the questions highlighted. We believe that these “study guides” served a similar purpose as answer sheets. For example, in Field Office 2, four FBI employees we interviewed used guides that contained highlighted portions of the DIOG attached to each exam question, in effect giving users the answer to each question on the exam. Another employee in Field Office 2 told us that his supervisor had told him that “everyone” in his squad was taking the exam using a copy of the DIOG in which the important parts had been highlighted. One particularly disturbing means of violating FBI rules occurred when four agents took advantage of “sloppy” computer programming to look at the answers to DIOG exam questions. This was clearly improper. In addition, in one resident agency, one FBI employee admitted to consulting with others about the answers while taking the exam, and three others admitted to helping that employee. Despite consulting others for answers to the exam, the employee who received help falsely certified in response to Question 51 that he had “only consulted the DIOG, notes, or training aides but no other person while taking this exam.” We were also concerned by the fact that several supervisors and trainers were involved in the abuses regarding the exam, including a legal advisor, an 31 ASAC, and an SSA in Field Office 1, an SSRA and an SSA in a resident agency, and an ASAC in Field Office 2. In addition, we concluded that a legal advisor in Field Office 1 attempted to impede the OIG’s investigation by telling another legal advisor “don’t throw me under the bus” in connection with an upcoming OIG interview. We found that the legal advisor lacked candor about the legal advisor’s purpose in making this statement when we asked about it. We were troubled by the various justifications offered by agents and supervisors for their actions. For example, a legal advisor in Field Office 1 said he regarded the DIOG exam process as just another bureaucratic impediment, which we found to be an irresponsible attitude for someone in his position. In addition, although several agents told us they believed answer sheets constituted permissible “notes,” we do not believe that any FBI employee could reasonably believe that using someone else’s answers to the exam comported with the test instructions and verbal guidance instructing them to work alone. We believe the extent of the cheating related to this test was greater than the cases we detailed in this report, which is based on our limited investigation in only four FBI field offices, one resident agency, and two Headquarters components. We believe this for two main reasons. First, we acknowledge that our investigation was not random. Of the FBI employees that we interviewed about their conduct in taking the DIOG exam, 22 cheated or acted improperly in some manner related to the exam. We recognize that the amount of cheating that we identified in our limited interviews cannot be extrapolated to the entire population of FBI test-takers. However, while our sample was not random the factors we used to select field offices or some of the individuals we interviewed – short exam times and suspicious e-mails – were not necessarily indicators of cheating. Rather, we found that many persons with short exam times did not cheat, and many persons who did cheat took a long time to complete the exam. In addition, the e-mails we saw before investigating those field offices did not, in general, lead us directly to the persons who we identified as having cheated. Instead, some of the FBI employees who we found had cheated were not identified by either suspicious e-mails or short test times. Second, our investigation revealed evidence that other employees in the field offices we visited, and employees in other field offices, may also have used similar improper tactics in taking the test. For example, agents in Field Office 1 told us that at least 30 other FBI employees on 3 different squads in that office had access to answer sheets. An employee in Field Office 2 told us that his supervisor had told him that “everyone” in his squad was taking the exam using a copy of the DIOG in which the important parts had been highlighted. 32 Employees in other FBI field offices also told us that they had sent answer sheets to other employees, and that others knew about the XML file that contained the answers to the exam. We did not investigate all of these leads or attempt to interview agents in all of the FBI’s field offices and Headquarter components. Thus, while we cannot extrapolate the percentage of cheating we identified in the limited sample of Field Offices and FBI employees we interviewed to the entire population of FBI test-takers, we believe there was more cheating and improper conduct than we identified through our limited interviews and investigation. B. Recommendations We are referring the findings of this report to the FBI for its review and action. We believe the FBI should take appropriate disciplinary action against those employees identified by the OIG who cheated or engaged in inappropriate conduct related to the DIOG exam. Given the number of FBI field offices and the issues that this report raised, we also believe the FBI should consider steps to determine whether other FBI employees cheated on or engaged in inappropriate conduct related to the DIOG, and, if so, take appropriate action regarding those employees. We also believe the FBI should implement additional testing on the DIOG, given the problems we found in the administration and taking of this test. 1. The FBI should take appropriate disciplinary action against those employees identified by the OIG who cheated or engaged in inappropriate conduct related to the DIOG exam. As described in this report, in our limited investigation at several FBI offices, we found that many FBI employees violated various FBI rules in connection with the exam by consulting with others, falsely certifying that they had consulted with no one, using answer sheets or study guides that provided the questions and answers on the exam, or exploiting flawed computer programming to reveal the answers on the test. We recommend that the FBI take appropriate disciplinary actions against those employees. 2. The FBI should consider taking appropriate steps to determine whether other FBI employees cheated or engaged in inappropriate conduct related to the DIOG exam, and, if so, take appropriate action. In addition to the employees that we identified during our investigation, we learned that other employees may also have engaged in cheating or other 33 inappropriate actions. For example, although we did not interview them, we were given the names of several employees in Field Office 1 who were given copies of answer sheets. Employees in other field offices also told us that they had sent answer sheets to others, and that other employees knew about the XML file that contained the answers to the exam. We recommend that the FBI consider appropriate steps to determine whether other test-takers cheated or engaged in inappropriate conduct related to the DIOG exam, and, if so, take appropriate action. For example, the FBI could consider sending an inquiry to all employees who took the DIOG exam seeking voluntary disclosure of any cheating on the exam, and the FBI could then consider what actions to take based on the responses and all other information that has been gathered. 3. The FBI should draft and disseminate a new DIOG exam. Our findings, and those of the FBI Inspection Division concerning senior officials in the WFO, raise questions about the effectiveness of the FBI’s training and testing on the new DIOG. We recognize that many FBI employees took the training and the exam seriously and appropriately complied with all the requirements regarding the exam. Unfortunately for them, the actions of others within the FBI undermined the effectiveness of the exam as a whole. We therefore recommend that the FBI consider creating a new DIOG exam and requiring that FBI employees take and pass it. While this may be burdensome for some FBI employees, it will also have the benefit of reinforcing the concepts of the DIOG on a regular basis. We understand that the FBI is already revising the DIOG. We believe that another exam when the DIOG is revised would be useful and could address some of the deficiencies in the first DIOG test. C. Summary To its credit, the FBI implemented training and a rigorous exam on the important requirements of the new DIOG. Unfortunately, the actions of some FBI employees undermined those actions. In our limited investigation, we found that a significant number of FBI employees engaged in some form of improper conduct or cheating on the DIOG exam, some in clear violation of FBI directives regarding the exam. Some consulted with others while taking the exam when that was specifically forbidden by the test-taking protocols. Others used or distributed answers sheets or study guides that essentially provided the answers to the test. A few exploited a programming flaw to reveal the answers to the exam. Several supervisors, including two ASACs, two SSAs, and a legal advisor, were involved in such cheating. Almost all of those who cheated falsely certified on Question 51 (the final question of the exam) that they had not consulted with others. 34 We recommend that the FBI take action regarding those who cheated on the DIOG exam, consider other appropriate steps to determine whether other test-takers engaged in similar inappropriate conduct, and also conduct a new exam on the revised DIOG. 35