Summary

Forensic Laboratories: Many Face Challenges Beyond Accreditation to Assure the Highest Quality Services December 1998 97025 rotiduA etatS ainrofilaC S T I D U A E T A T S F O U A E R U B The first copy of each California State Auditor report is free. Additional copies are $3 each. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TDD (916) 445-0255 x 248 OR This report may also be available on the World Wide Web http://www.bsa.ca.gov/bsa/ Permission is granted to reproduce reports. December 15, 1998 97025 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As required by the California Penal Code, section 13892, the Bureau of State Audits presents its audit report concerning our assessment of California’s 19 local forensic laboratories. This report concludes that several of the local forensic laboratories have not established comprehensive quality control systems. Further, although required for accreditation, many of the laboratories do not have proficiency testing or court monitoring programs. As a result, these laboratories will have difficulty meeting certain professional standards for forensic laboratories. Additionally, many of the laboratories have cramped facilities or outdated and improperly working equipment. Laboratories could also improve efficiency by replacing outmoded equipment, installing management information systems, and providing training programs for their staff. Finally, most of the laboratories do not track information on costs of individual tests; thus, they cannot accurately assess the cost-effectiveness of the services they provide. Respectfully submitted, KURT R. SJOBERG State Auditor CONTENTS Summary 1 Introduction 5 Chapter 1 Most Local Forensic Laboratories Must Improve Their Quality Control Systems to Meet Accreditation Standards 11 Chapter 2 Laboratories Face Challenges Beyond Accreditation to Perform Optimally 19 Chapter 3 Laboratories Need to Evaluate the Effectiveness and Efficiency of Their Services 31 Chapter 4 Conclusions and Recommendations 39 Appendix Summary of ASCLD/LAB Accreditation Standards 45 Responses to the Audit Alameda County Sheriff R-1 State Auditor’s Response R-3 Contra Costa County Sheriff R-5 State Auditor’s Response R-11 El Cajon Police R-13 Huntington Beach Police R-15 State Auditor’s Response R-19 Kern County District Attorney R-21 State Auditor’s Response R-25 Los Angeles Police R-27 State Auditor’s Response R-29 Los Angeles County Sheriff R-31 Oakland Police R-33 State Auditor’s Response R-37 Sacramento County District Attorney R-39 State Auditor’s Response R-41 San Diego Police R-43 San Diego County Sheriff R-47 San Francisco Police R-49 State Auditor’s Response R-51 Ventura County Sheriff R-53 State Auditor’s Response R-57 SUMMARY RESULTS IN BRIEF A combination of municipal, county, and state-operated laboratories provides California with the majority of its Audit Highlights . . . forensic services. Forensic laboratories collect, analyze, and interpret evidence involved in the investigation and pros- Many local forensic ecution of criminal activity. County district attorneys’ offices, laboratories: county sheriffs’ offices, or city police departments operate 19 (cid:254) Lack one or more elements forensic laboratories that serve approximately 77 percent of the of a quality control State’s population in 13 counties. The State Department of system. Justice operates full-service laboratories at 11 sites that provide (cid:254) Have cramped facilities or services to the remaining counties in the State. This report outdated and improperly focuses only on the 19 local laboratories. working equipment. (cid:254) For several years, the California Association of Crime Laboratory Do not have proficiency testing or court mon- Directors (CACLD), an organization composed of directors and itoring programs. supervisors from forensic laboratories, has asserted that scarce (cid:254) financial resources and increased caseloads have hampered their Do not track meaningful information for ability to keep pace with new technologies and to guarantee management decisions. only the highest quality forensic services. CACLD believes that (cid:254) as a result, laboratories are open to attack on the credibility of Do not accurately assess their work. the cost-effectiveness of their operations and have not explored outsourcing In light of these issues, the forensic community feels it is impor- or regionalizing certain tant for laboratories to participate in the voluntary accreditation services. program of the American Society of Crime Laboratory Directors Moreover, costs to upgrade Laboratory Accreditation Board (ASCLD/LAB). To obtain accredi- and replace outdated tation, a laboratory must demonstrate that its management, equipment are significant. operations, personnel, procedures, equipment, facility, security, We estimate that more than $221 million is needed to and health and safety procedures meet established standards. construct facilities meeting The program also requires laboratories to implement proficiency recommended space testing, continuing education, and other programs that improve standards. laboratory personnel’s overall skills and services. We reviewed the 19 laboratories to assess their readiness to obtain ASCLD/LAB accreditation. We found that 13 of the laboratories have not developed or implemented one or more of the components of a quality control system. In addition, al- though required for accreditation, many of the laboratories do not have proficiency testing and court monitoring programs. Without accreditation, the laboratories cannot affirm they meet C A L I F O R N I A S T A T E A U D I T O R 1 accepted industry standards for integrity and quality assurance. We estimate that it will cost nearly $2 million annually for the laboratories to implement and maintain quality control systems. Accreditation indicates a laboratory has met certain professional standards, therefore improving the credibility of its services, but does not guarantee a laboratory operates effectively and maxi- mizes its use of available resources. Accredited laboratories may still have cramped facilities or outdated and improperly working equipment. Accredited laboratories may also lack a management information system capable of generating information in a format that allows the laboratory directors to plan or effectively manage resources, and may not provide adequate employee training programs. Thus, many of the laboratories, both those that have attained accreditation and those that have not, could improve the effi- ciency and effectiveness of their operations by constructing new facilities or redesigning existing facilities to allow staff to more efficiently and safely accomplish assigned tasks. Laboratories could also improve efficiency by replacing outmoded equipment with more efficient equipment, installing management informa- tion systems that laboratory management can use to make decisions regarding the best use of resources, and implementing training programs to develop the skills staff need to perform competently and efficiently. However, many laboratories are constrained from improving their facilities, replacing outmoded equipment, installing management information systems, and implementing training programs because they lack funding. We estimate that it will cost more than $221 million to con- struct new facilities for the laboratories that do not currently meet the standards recommended by forensic laboratory design literature. In addition, while our consultants did not estimate the cost to replace all of the outdated equipment they found at the laboratories, to provide some perspective on how significant the cost to replace equipment can be, they identified 15 items that need replacement and estimated that the cost to replace those items would approach $750,000. However, our review of the equipment lists identified many other examples of equip- ment 10 and 20 years old that may need replacement. Further, the costs to develop management information systems and implement training programs will vary with the size of the respective laboratories. Considering estimates some of the laboratories provided, the cost to develop a management information system can range from $3,000 to $8,000 per work 2 C A L I F O R N I A S T A T E A U D I T O R station. Finally, if each laboratory spends $1,000 per forensic examiner per year for training, as our consultants suggest, the annual costs to provide this training for staff at all 19 laborato- ries would exceed $600,000. We also found that the laboratories do not track information on costs of individual tests; thus, they cannot accurately assess how cost-effective their services are. They also have not explored the cost-effectiveness of outsourcing some services to private labora- tories or regionalizing certain services. For example, we found the anticipated caseloads and staffing levels of at least two of the eight laboratories planning to implement DNA testing are not sufficient to justify the investment in equipment and training. We also attempted to examine whether any of the laboratories could transfer cases to others to reduce the backlog some labora- tories experience, or whether consolidating some services would be appropriate. We were unable to draw any conclusions because workload data was either unavailable, inconsistent, or not standardized. However, some, such as the Los Angeles and San Diego county laboratories, have considered consolidating all services within their regions, though these plans have yet to take shape. RECOMMENDATIONS The Legislature will need to determine which of these deficien- cies it will appropriate funds to address. If its primary goal is to have all 19 of the local laboratories accredited, the Legislature should first appropriate funds for quality assurance in the laboratories that lack complete programs. The Legislature should distribute any additional funds equitably among all local labora- tories. Finally, the Legislature should require the laboratories to evaluate the benefits of consolidating and regionalizing their services and document the cost-effectiveness of the services they currently provide. As with any funds it appropriates, the Legislature should place constraints on the money it designates to improve the delivery of forensic services to ensure that the local laboratories use them as intended and that the money supplements, rather than replaces, local funds. C A L I F O R N I A S T A T E A U D I T O R 3 To ensure that they maintain ASCLD/LAB standards for accredi- tation and provide cost-effective forensic services, the local laboratories should do the following: • Appoint quality managers and support staff at sufficient levels to implement quality control and safety programs, including documentation and periodic audits. • Maintain a quality control program that includes proficiency testing and monitoring of court testimony. • Use qualified consultants to determine their specific facility needs and related costs. • Inventory all equipment and include a capital equipment replacement plan in their annual budgets. • Develop and implement appropriate management informa- tion systems to increase staff efficiency and improve opera- tions management. • Develop, implement, and document a formal training program for technical staff and budget for training costs. • Analyze the costs of each of their services and compare those costs to private laboratories’ charges. • Consider consolidating or regionalizing services, including DNA testing. • To aid in considering options, develop and maintain standardized and consistent workload data for the services they maintain. AGENCY COMMENTS We received comments from 13 of the 19 local forensics labora- tories. Six laboratories chose not to provide written comments to the report. In general, the laboratories agreed with our conclusions and recommendations. However, several disagreed with our conclusions regarding the cost-effectiveness of implementing DNA testing at laboratories that anticipate small caseloads. We provide our comments to these and other concerns raised by the laboratories after their respective responses. n 4 C A L I F O R N I A S T A T E A U D I T O R INTRODUCTION BACKGROUND A combination of municipal, county, and state laborato- ries provides California with the majority of its forensic services for collecting, analyzing, and interpreting evidence from criminal investigations. Three county district attorneys’ offices, nine county sheriffs’ offices, and seven city police departments operate 19 forensic laboratories. These 19 laboratories serve approximately 77 percent of the State’s popu- lation in 13 of its largest counties. See Figure 1 on the following page for the location of these laboratories. The State’s Depart- ment of Justice operates full-service laboratories at 11 sites that provide forensic services to the remaining counties; however, these laboratories are not included in the scope of this audit. The 19 local laboratories are diverse both in size and in the services they provide. For example, the staffing levels range from 5 to more than 160 employees. Similarly, some laboratories perform only limited services, such as latent fingerprint exami- nation and controlled substance analyses, whereas others are full-service laboratories and provide most services available in forensic science. These services may include DNA testing, alco- hol analysis, toxicology testing, and crime scene analysis, as well as others. If a laboratory cannot provide a particular service, typically it will contract with a private laboratory. FUNDING FOR LOCAL LABORATORIES VARIES Although legislation requires courts to collect fines from persons convicted of driving under the influence of drugs or alcohol and certain other drug offenses to offset the laboratories’ costs of analyzing samples for drugs or alcohol, these moneys constitute less than 10 percent of the laboratories’ operating costs. There- fore, most forensic laboratories depend heavily on discretionary funding at the state and local level. Laboratory directors have found that, because these funding sources are often inadequate and not conducive to long-term planning, they must look elsewhere to fund laboratory opera- tions, equipment, and facilities. Some directors have had greater C A L I F O R N I A S T A T E A U D I T O R 5 FIGURE 1 Location of the19 Local Forensic Laboratories 1 Contra Costa County Sheriff 2 Oakland Police 3 San Francisco Police 4 Alameda County Sheriff 5 San Mateo County Sheriff 6 Santa Clara District Attorney 7 Fresno County Sheriff 8 Kern County District Attorney 9 Ventura County Sheriff 10 Los Angeles Police 11 Los Angeles County Sheriff 12 Long Beach Police 13 Orange County Sheriff 14 Huntington Beach Police o 15 San Diego County Sheriff 16 San Diego Police 17 El Cajon Police 18 San Bernardino County Sheriff 1 19 Sacramento County District Attorney 2 3 4 5 6 i 7 8 9 0 q w e r t y u 6 C A L I F O R N I A S T A T E A U D I T O R success than others in obtaining additional revenues. For ex- ample, the Orange County Sheriff obtained approval to con- struct a state-of-the art laboratory using long-term financing. In addition, it uses Proposition 172—Local Public Safety Fund— moneys to fund operations and only uses general fund moneys to supplement Proposition 172 revenues when necessary. On the other hand, the Santa Clara County District Attorney’s, the Contra Costa County Sheriff’s, and the Alameda County Sheriff’s laboratories charge their municipalities for certain services to ease some of their budgetary constraints. ALTHOUGH CURRENTLY VOLUNTARY, THE IMPORTANCE OF ACHIEVING ACCREDITATION IS INCREASING For several years, the California Association of Crime Laboratory Directors (CACLD), an organization of forensic laboratory directors and supervisors, has expressed concerns that scarce financial resources and the increase in caseloads have hampered the laboratories’ ability to keep pace with new technologies. The laboratories continue to use antiquated equipment and proce- dures and are housed in inadequate and overcrowded facilities. The CACLD also believes that the lack of sufficient financial resources to ensure a laboratory provides only the highest quality of evidence, as demonstrated by the public attack on the credibility of evidence provided during the O. J. Simpson trial, has opened the way for similar discrediting of forensic results in courts. In light of these issues, the CACLD believes it is increasingly important for the laboratories to participate in the American Society of Crime Laboratory Directors Laboratory Accreditation Board’s (ASCLD/LAB) accreditation program. This is a voluntary program for laboratories that demonstrates their facilities, management, operations, and personnel meet established standards. Accreditation is part of a laboratory’s quality assur- ance program that also includes proficiency testing, continuing education, and other programs to help the laboratory improve overall services and maximize the credibility of evidence. To inspect facilities applying for accreditation, ASCLD/LAB uses an external team of trained forensic scientists from other accred- ited laboratories to conduct an evaluation and determine whether a particular laboratory meets the majority of the ASCLD/LAB accreditation standards. The Appendix presents a C A L I F O R N I A S T A T E A U D I T O R 7 summary of the ASCLD/LAB accreditation standards. Each standard is given a rating of “essential,” “important,” or “desir- able,” and the inspectors must determine if the laboratory complies with each standard. To pass an accreditation inspec- tion, a laboratory must meet 100 percent of the “essential,” 70 percent of the “important,” and 50 percent of the “desirable” standards. The inspection team prepares a final report and submits it to the ASCLD/LAB with recommendations. A majority vote of the board is required for accreditation to be granted. Other states are also addressing similar issues related to labora- tory accreditation. For example, New York passed legislation requiring the majority of its forensic laboratories to become accredited by January 1996. Furthermore, at the national level, concerned legislators drafted legislation that proposed funding for the approximately 300 state and local crime laboratories and medical examiners nationwide to apply for accreditation by the end of the year 2001. Although the national legislation was revised significantly before it passed, if it is appropriated, local laboratories will receive limited funds to upgrade certain tech- nologies, such as automated criminal history systems and fingerprint identification systems, as well as for programs lead- ing to accreditation. SCOPE AND METHODOLOGY The California Penal Code, Section 13892, mandated that the Bureau of State Audits assess the needs of California’s 19 local forensic laboratories. We focused on two areas. First, we identi- fied the changes and improvements needed for the laboratories to achieve or, for those currently accredited, to maintain accredi- tation from the ASCLD/LAB by January 1, 2004. Whenever possible, we also estimated the costs associated with the changes and improvements that we identified. Second, we attempted to analyze the local laboratories’ services to determine whether alternatives, such as consolidation of services or the use of private laboratories, were feasible. In addition, before starting our fieldwork, we consulted with an advisory committee com- posed of representatives from the CACLD, various law enforce- ment organizations, and the Department of Justice’s Bureau of Forensic Services. To identify needed changes and improvements for the local laboratories, we visited all 19 laboratories and evaluated each one using ASCLD/LAB standards. During our visits, we also 8 C A L I F O R N I A S T A T E A U D I T O R interviewed laboratory staff. To assist us in our review, we ob- tained the services of consultants with expertise in forensic science testing, laboratory operations, and the ASCLD/LAB accreditation program. The consultants reviewed the laborato- ries’ technical and scientific operations using the ASCLD/LAB accreditation standards. To determine the costs associated with needed changes and improvements, we contacted major suppliers of equipment, proficiency testing materials, and training programs for the laboratories. We also used architectural firms’ studies to estimate construction costs for new forensic laboratory facilities or to redesign existing facilities. Finally, we used a salary survey performed by the California Association of Criminalists when calculating salary costs. To analyze the services currently provided by the local laborato- ries, we interviewed each laboratory director and the chief of the State Department of Justice’s Bureau of Forensic Services to identify relationships between the local and state laboratories and other governmental or private laboratories, and to gain the directors’ perspective on consolidating services. We also re- viewed any related special studies or reports. Finally, we at- tempted to analyze and compare the laboratories’ workload to determine whether the laboratories could share services. During our fieldwork, several laboratory directors raised con- cerns regarding the Department of Health Services process for regulating and approving both procedures and the equipment laboratories and law enforcement agencies use to enforce the State’s drunk driving laws. Because this issue was not within the scope of our audit, we have not addressed these concerns in our report. However, we plan to follow up on these issues and, if necessary, provide a separate report at a later date. n C A L I F O R N I A S T A T E A U D I T O R 9 Blank page inserted for reproduction purposes only. 10 C A L I F O R N I A S T A T E A U D I T O R CHAPTER 1 Most Local Forensic Laboratories Must Improve Their Quality Control Systems to Meet Accreditation Standards CHAPTER SUMMARY A lthough forensic science has a crucial role in the admin- istration of justice, several of the local forensic labora- tories have not established comprehensive quality control systems to ensure the overall credibility and integrity of their scientific findings. Many of the laboratories have neither designated individuals as quality managers nor developed or implemented other essential elements of such a system, includ- ing quality and safety manuals, procedures and policies, or annual quality audits and reviews. Specifically, we identified 13 laboratories that lack one or more of these four elements. As a result, these laboratories will have difficulty meeting standards for forensic laboratories as established by the American Society of Crime Laboratory Directors Laboratory Accreditation Board (ASCLD/LAB). Moreover, these laboratories may lack continued confidence that the results of their work are accurate, impartial, and relevant. We estimate that it will cost nearly $2 million annually for the laboratories to implement and maintain comprehensive quality control systems. The $2 million estimate includes funds for each laboratory to hire quality managers and necessary support staff. It also includes funds to purchase proficiency tests from external providers. The laboratories administer these tests to all labora- tory forensic examiners to assess their competency in laboratory and analytical procedures. In addition to these costs, ASCLD/ LAB estimates that the laboratories together will spend more than $190,000 in inspection fees to periodically determine whether they meet accreditation standards. C A L I F O R N I A S T A T E A U D I T O R 11 A STRONG QUALITY CONTROL SYSTEM IS KEY TO ATTAINING ACCREDITATION We found that most of the local laboratories did not fully meet some ASCLD/LAB primary accreditation standards because they lack a quality control system. Of the 19 local laboratories, 8 have not designated a manager responsible for coordinating an effective quality control system. In addition, 10 of the 19 labora- tories have not developed quality manuals describing policies and procedures pertaining to all aspects of their laboratories’ operations, such as case Four Elements of a Quality Control System management, validation and verification of • Quality Manual—Consolidated statement of test procedures, handling of evidence, and the laboratory’s policies and description of laboratory protocols. Finally, 12 laboratories the elements of the quality system and do not conduct annual reviews of their quality practices. control systems, a key indicator of the overall • Quality Manager—Individual designated by top management with authority and credibility of the laboratory. obligation to ensure that the requirements of the quality system are implemented and Because of forensic science’s crucial role in the enforced. justice system, laboratories must employ • Annual Quality Audit—Tool used to verify intensive measures to ensure the overall value compliance with the quality system’s of their scientific findings. This is demon- operational requirements. • Annual Quality Review—Tool used to assure strated by ASCLD/LAB devoting nearly half of the laboratory takes all measures to provide its accreditation criteria to quality control and the highest quality service using state-of-the- safety policies and practices. For example, art forensic technologies. laboratories must have custody policies to properly store and track evidence. Further, laboratories must ensure that staff receive sufficient current training and complete periodic proficiency tests. Laboratories should also design and maintain safety pro- grams to safeguard employees from injury and health problems. For example, laboratories should require the use of safety de- vices, such as face protectors and gloves, and supply proper equipment to handle dangerous material spills. ASCLD/LAB also requires laboratories to have annual audits to evaluate and determine compliance in these activities for each accredited service. These audits include steps as simple as con- firming that the laboratory regularly maintains instruments to the more complicated task of assuring that protocols reflect the laboratory’s current practices. The director of the Orange County Sheriff’s laboratory said that these audits are valuable, but they are costly to the laboratory, particularly in staff time. While we recognize that an annual audit can be costly, ASCLD/LAB con- siders it a primary tool to confirm the quality and consistency of service. 12 C A L I F O R N I A S T A T E A U D I T O R Quality Managers Have an Essential Role in a Quality Control System Of the four crucial elements of a quality system, quality manag- ers play the most essential role because they serve as objective evaluators ensuring that the other elements of the system are fully developed and consistently applied. Presently, 11 of the 19 laboratories have designated quality managers, including all 6 accredited laboratories. However, one of the accredited laborato- ries splits the duties among a number of staff. The size and staffing of the laboratories vary widely from 5 staff in the El Cajon Police laboratory to 164 staff in the Los Angeles County Sheriff’s laboratory. Our consultants suggest the follow- ing quality control staffing levels: • For fewer than 20 staff, appoint a half-time Quality Manager’s Responsibilities quality manager. • Maintain and update the quality and safety manuals. • For 20 to 40 staff, appoint a full-time quality • Schedule and coordinate annual quality manager. control and safety audits. • Ensure maintenance of scientific • For 41 to 70 staff, appoint a full-time quality instruments and safety equipment. manager and a clerk. • Ensure that new technical procedures are valid. • For 71 to 100 staff, appoint a full-time • Investigate technical problems, propose quality manager, a clerk, and a laboratory remedial actions, and verify their technician. implementation. • Administer proficiency testing and evaluate • For more than 100 staff, appoint a full-time results. quality manager, an assistant manager, a • Administer a court monitoring program. clerk, and a laboratory technician. The graduated staffing levels address the increase in a quality manager’s tasks as the number of employees conducting case- work and testifying in court increases. The number of services the laboratory offers also affects the quality manager’s job because each service requires its own procedures and protocols and is subject to a quality control annual audit. Typically, the quality manager is responsible for coordinating these annual audits. Although 11 of the laboratories have designated quality control managers, only 1 laboratory meets our consultants’ suggested quality management staffing standards. Furthermore, even though 6 laboratories have assigned full-time quality managers, C A L I F O R N I A S T A T E A U D I T O R 13 several of these managers continue to spend time on other tasks, including casework, rather than on quality control issues. Specifically, 1 quality manager in a 47-person laboratory devotes time to cases rather than quality control because of staff vacan- cies. Other laboratories split quality control duties among Quality managers may supervisors and managers who must perform many other activi- spend time on other tasks, ties, therefore completing quality activities only as time permits. including casework, and ASCLD/LAB states that it is essential for a quality manager to therefore perform quality have organizational autonomy and to be capable of coordinat- activities only as time ing all activities to implement and maintain a comprehensive permits. quality system. Additionally, within the next two years, the ASCLD/LAB accreditation program will require a laboratory to designate a quality manager reporting to the highest level of management. The total annual cost for all 19 local laboratories to staff quality control units at the consultants’ suggested levels would be almost $1.8 million. Those laboratories already designating quality managers are currently incurring some of these costs. The annual costs of salaries and benefits per laboratory range from $38,500 for a half-time quality manager to $221,000 for the largest laboratories with a full-time quality manager, an assistant manager, a clerk, and a laboratory technician. To prepare our estimate, we used information from the California Association of Criminalists 1997-98 salary survey. Because the survey lacked data for clerical positions, we used the salary of a state employee at the office assistant level for these positions. ALTHOUGH REQUIRED FOR ACCREDITATION, SEVERAL LABORATORIES DO NOT HAVE A PROFICIENCY TESTING PROGRAM All 6 of the accredited laboratories, as well as one laboratory currently seeking accreditation, meet the ASCLD/LAB standards for proficiency testing. However, only 2 of the remaining 12 laboratories meet these standards. Proficiency testing is an integral element of a laboratory’s quality assurance program used to monitor performance and ensure the reliability of staff members’ work. Laboratories also use proficiency tests to vali- date procedures and identify needed improvements. Those laboratories not meeting the proficiency testing standards can neither assure the continued competency of their staff or reliability of their analytical results, nor attain ASCLD/LAB accreditation. 14 C A L I F O R N I A S T A T E A U D I T O R One of the key duties of a quality manager is to implement the Without proficiency laboratory’s proficiency testing program and to take corrective testing, laboratories action when necessary. All nine laboratories with proficiency cannot assure continued testing programs also have designated a quality manager. reliability of their findings. Two more laboratories are now in the process of developing proficiency testing programs; however, eight laboratories have not implemented a testing program. Laboratory directors report that insufficient financial resources, staff shortages, excessive workload, or pressure to complete casework have made it diffi- cult to fully implement these programs. All laboratory examiners must complete at least one annual proficiency test for each type of casework they perform except for those who conduct DNA analyses. DNA analysts must com- plete two proficiency tests each year, one of which must be from an approved external test provider. In addition, ASCLD/LAB requires all laboratories to participate annually in at least one external proficiency test for each forensic discipline for which they provide services. The cost of proficiency test programs will depend on the num- ber of services a laboratory provides and the number of its examiners. For example, the Orange County Sheriff’s laboratory provides several different services and must complete a profi- ciency test for each one. Each of its 65 examiners in those disciplines must also complete one proficiency test annually. Further, each of its 9 examiners who conduct DNA analysis must also complete a second test annually. Using the price lists obtained from test providers, we estimated the cost of external proficiency tests for all 19 laboratories at $119,000 per year, excluding the time staff require to complete the tests. Our cost estimates range from $240 for the smallest laboratory to more than $19,000 for one of the largest laborato- ries. We did not estimate the cost related to staff time required to complete the tests because of the many variables involved. However, laboratories still must plan for these costs. The director of one of the large laboratories estimates that the total annual cost of staff time, including salary and benefits, for proficiency testing averages $1,900 per person. A director for a smaller laboratory estimates similar average costs per person for his laboratory. C A L I F O R N I A S T A T E A U D I T O R 15 In addition to purchasing tests from outside providers, a laboratory can develop some of its own, as long as it properly designs, appropriately administers, and fairly evaluates the proficiency tests. The complexity and nature of the laboratory procedure will determine the costs and time involved to perform these steps. ASCLD/LAB REQUIRES A COURT MONITORING PROGRAM FOR ACCREDITATION Because a laboratory examiner’s testimony can be a pivotal factor in determining the verdict of a criminal trial, ASCLD/LAB requires laboratories to implement a court monitoring program. We found that 11 of the laboratories do not have court monitor- ing programs, but 4 of these 11 are currently developing theirs. We further noted that the remaining 7 laboratories also lack a quality manager. Laboratories must monitor the testimony of each examiner at least once each year to ensure it is both scientifically consistent By not monitoring the with the findings documented in the case file and understand- performance of staff able to a lay jury. Because credibility as a witness is important, testifying in court, labora- ASCLD/LAB also suggests that laboratories evaluate their staff’s tories may jeopardize their appearance, poise, and performance under cross-examination. reputation within the As a mandatory ASCLD/LAB requirement, those laboratories not judicial system. having this program cannot attain accreditation. Furthermore, by not monitoring the performance of staff testifying in court, the laboratories may jeopardize their reputation within the judicial system. Quality managers are responsible for ensuring that a court monitoring program is in place, that court moni- toring records are well organized, and that all the steps are fully documented. In contrast to some of the other accreditation elements, the costs for developing and implementing a court monitoring program should be manageable. In fact, some of the laboratories with these programs reported that they do not incur significant costs because they rely primarily on evaluation forms from court officers and have staff make a few court visits. In addition, ASCLD/LAB allows different methods for monitoring court testimony, including having a supervisor or peer observe the testimony, reviewing testimony transcripts, or having court officers complete evaluation forms. 16 C A L I F O R N I A S T A T E A U D I T O R LABORATORIES WILL INCUR COSTS INCLUDING FEES AND STAFF TIME TO OBTAIN ASCLD/LAB ACCREDITATION As we noted above, laboratories will annually incur costs of more than $1.8 million to establish and improve their quality control systems in order to attain ASCLD/LAB accreditation. In addition to these costs, the ASCLD/LAB estimates that, in total, the 19 laboratories will spend an additional $190,000 in fees every five years as the ASCLD/LAB inspects the laboratories. ASCLD/LAB bases its fees on the number of days needed for each inspection. Teams consist of two or more inspectors who work in other accredited laboratories, are familiar with the work of the laboratory under review, and have graduated from an ASCLD/LAB inspector training course. Presently, ASCLD/LAB charges $425 per inspector day to cover all expenses, including travel and per diem costs. The estimated costs and days needed for an accreditation inspection for the 19 local laboratories range from a minimum of $2,550 for 6 inspec- Laboratory inspections tor days at the smallest laboratory to $25,500 for 60 inspector could cost over $25,000 days at the largest laboratory. Currently, the ASCLD/LAB accredi- for the larger facilities. tation lasts for five years; however, the laboratories may soon pay inspection fees more frequently because ASCLD/LAB is considering shortening the accreditation to two years. To retain its accredited status for the full five years, a laboratory must demonstrate continued compliance by preparing and submitting an annual self-evaluation report. Also, ASCLD/LAB monitors the laboratories’ compliance by reviewing external proficiency testing reports. Laboratories also have other minor accreditation costs, such as staff time to give inspectors tours of the facilities and to be available to provide information and answer questions. One laboratory reported that five staff members assisted its quality manager during its ASCLD/LAB accreditation. Additionally, a laboratory will incur some costs to copy documents for each inspector and to provide manuals and records. Although these costs may not be significant, the laboratory must consider them as part of the total cost of accreditation. n C A L I F O R N I A S T A T E A U D I T O R 17 Blank page inserted for reproduction purposes only. 18 C A L I F O R N I A S T A T E A U D I T O R CHAPTER 2 Laboratories Face Challenges Beyond Accreditation to Perform Optimally CHAPTER SUMMARY W hile attaining accreditation by the American Society of Crime Laboratory Directors Laboratory Accredita- tion Board (ASCLD/LAB) indicates that a forensic laboratory has met certain professional standards, it does not guarantee that the laboratory operates effectively and maximizes its use of available resources. For example, our consultants identified 12 laboratories, 5 of which are accredited, that are overcrowded in all or some areas. Similarly, they determined that 3 of the accredited laboratories and 7 of the nonaccredited laboratories have outdated or improperly working equipment. In addition, although most laboratories have some type of manage- ment information system, we found that the systems do not always generate information in a format that allows the labora- tory directors to plan or effectively manage resources. Finally, we identified 9 laboratories that do not have documented training programs for new and continuing employees. LACK OF SUFFICIENT SPACE COULD RESULT IN EVIDENCE CONTAMINATION Our consultants found that 12 of the 19 laboratories have overcrowded facilities in all or some areas and have major deficiencies in working conditions. One of these is located in an old building in need of retrofitting. Without adequate facilities, laboratories may experience a greater risk of evidence contami- nation, compromised efficiency, adversely affected morale and productivity, and health and safety problems. Based on the suggested allowance of 1,000 square feet of space per staff mem- ber, we found that 15 laboratories lack adequate space. Seven of these need greater than a 200 percent increase in laboratory space to meet the allowance forensic laboratory design literature recommends. Even though ASCLD/LAB standards address the need for adequate space to allow staff to accomplish assigned tasks and for supplies, equipment, and records, ASCLD/LAB still C A L I F O R N I A S T A T E A U D I T O R 19 accredits laboratories with overcrowded facilities. Therefore, many of these laboratories could obtain accreditation while performing services in cramped, inadequate facilities. According to our consultants, although it is possible to under- stand that agencies are under budgetary pressure and that Many laboratories could people are doing their best to accommodate needed services, the be accredited while degree of severe overcrowding in the laboratories is of major performing services in concern. For example, the Oakland Police laboratory houses cramped, inadequate 17 staff in approximately 6,100 square feet of space in an old facilities. building. Additionally, the building was severely damaged in the 1989 Loma Prieta earthquake and now requires retrofitting. Similarly, the Huntington Beach Police laboratory is located within the city’s central police facility. The facility is more than 20 years old, and the laboratory was originally designed for half the staff that currently occupy the space. Storage space is cramped with cabinets and supplies situated inappropriately in aisles, obstructing work areas and exits. The laboratory does not have a separate, properly controlled area to dry bloody evidence, nor does it have separate examining rooms for evidence from victims and suspects. In a third example, the Los Angeles County Sheriff’s laboratory has occupied its current main facility of approximately 35,000 square feet since 1976. We observed that the laboratory in most areas is extremely cluttered and overcrowded, potentially putting evidence at risk of contamina- tion. As far back as 1994, an ASCLD/LAB inspection team noted that the labora- tory lacks storage space for supplies, equipment, and tools. In addition, the laboratory uses inappropriate space for records, reference works, and other documents. Further, the inspection team stated that the examiners do not have space for report writing and that instru- ments are crowded together and often located away from the work area of the analyst using the instru- ment. 20 C A L I F O R N I A S T A T E A U D I T O R Most Laboratories Do Not Meet the Recommended Standard of 1,000 Square Feet per Staff Member Current forensic laboratory design literature indicates that the ratios for recently constructed forensic laboratories fall between the range of 700 to 1,000 square feet per staff member. The literature’s recommended standard is 1,000 square feet. In addition, the Department of General Services recently con- tracted with an architectural consulting firm to design six new state-operated regional laboratories with an even higher average ratio of approximately 1,300 square feet per staff mem- ber. Using the standards from current literature, as shown in Table 1 below, we found that 14 of the 19 laboratories do not provide at least 700 square feet per staff member and that 15 of 19 laboratories do not provide approximately 1,000 square feet per staff member. TABLE 1 Square Footage at Each of the 19 Laboratories Total Current Square Number Facility Square Footage per Entity Operating Laboratory of Staff Footage Staff Member Alameda County Sheriff 17 9,500 559 Contra Costa County Sheriff 56 16,589 296 El Cajon Police 5 2,625 525 Fresno County Sheriff 8 6,000 750 Huntington Beach Police 11 2,700 245 Kern County District Attorney 21 24,390 1,161 Long Beach Police 18 3,245 180 Los Angeles County Sheriff 137 35,000 255 Los Angeles Police 149 36,400 244 Oakland Police 17 6,100 359 Orange County Sheriff 129 120,000 930 Sacramento County District Attorney 34 44,000 1,294 San Bernardino County Sheriff 84 24,900 296 San Diego County Sheriff 47 45,000 957 San Diego Police 64 23,200 363 San Francisco Police 16 5,250 328 San Mateo County Sheriff 21 13,000 619 Santa Clara County District Attorney 44 26,000 591 Ventura County Sheriff 41 17,784 434 Total 919 461,683 502 Note: Laboratories in bold do not provide at least 700 square feet per staff member. C A L I F O R N I A S T A T E A U D I T O R 21 Because of budgetary constraints, local governments have limited funding available to upgrade laboratory facilities; thus, the older facilities have not kept up with increasing staffing levels and have become overcrowded. For example, according to the director of the Los Angeles Sheriff’s laboratory, although the laboratory receives sufficient funding for its current operations, it has not received additional funding for personnel, laboratory space, and equipment to address all crime in the county. As shown in Table 1, at 255 square feet per staff member, the Los Angeles Sheriff’s laboratory has one of the lowest square foot ratios of the 19 laboratories. To meet the recommended space ratio of 1,000 square feet per staff member, the Los Angeles Sheriff’s laboratory requires an additional 102,000 square feet, a 291 percent increase over its current space. To mitigate the overcrowding, in March 1999, the laboratory plans to move several sections to another site as a temporary fix. According to our consultants, this move will help the space problem but could negatively affect the flow of evidence and communication between sections. Furthermore, the temporary site is only available for five to eight years. Therefore, in an attempt to address its current and future facility needs, the director of the Los Angeles County Sheriff’s laboratory has met with management of the Los Angeles Police laboratory, the Department of Justice’s California Criminalistics Institute, and the California State University, Los Angeles, to consider the joint construction of a regional crime laboratory and training facility to be located on the university’s campus. Architectural consult- ants proposed a 326,000 square foot facility at an estimated cost of $131 million. However, the participants in this project have yet to identify a funding source for the project. In contrast to the 15 laboratories with low square foot ratios, four laboratories with space ranging from 930 to 1,294 square feet per staff member have ob- tained funding to construct new facilities or to purchase and convert an existing facility to a crime laboratory. Further- more, these laboratories were constructed or designed to include space for future expansion and growth. For example, our consultants found that the facilities for the Sacramento County District Attorney’s and the Orange County Sheriff’s laboratories are spacious and 22 C A L I F O R N I A S T A T E A U D I T O R well-equipped. The new Sacramento facility includes a number of enclosed rooms that staff use to separately review the evi- dence from a suspect and a victim, and the rooms can be thor- oughly and easily cleaned before staff bring in new evidence for analysis. Additionally, the facility contains separate storage space for different types of evidence, such as controlled sub- stances and firearms, large temperature-controlled refrigerators and freezers for evidence storage, and ventilation hoods to effectively exhaust chemical and biological materials. The design of the Sacramento County laboratory minimizes the chance for evidence to become contaminated, thus preserving the integrity of evidence for use in criminal trials. The Sacramento County laboratory also contains unfinished space within its facility to accommodate future growth and is located on property that will allow for future expansion of the existing building, if needed. The Cost of Expanding Existing Laboratory Space May Be Substantial Using the standard of approximately 1,000 square feet per staff member, we estimated that the cost to construct new facilities for the 15 laboratories that do not currently meet the recom- mended standard is $221 million. We used an architectural study prepared for the Department of General Services in 1997 We estimate it will cost to compute our estimate. Based on the architectural consultant’s $221 million to provide estimated costs to construct the State’s six new regional forensic new facilities for the 15 laboratories, we calculated a cost of $322 per square foot. laboratories not meeting However, the actual costs to construct local facilities will vary the recommended space depending on location, ultimate size, and prevailing construc- standards. tion costs. For example, the San Mateo County Sheriff’s labora- tory estimates that it will cost $245 per square foot in 1999 to construct a new facility based on its analysis of the cost of crime laboratories recently built around the country. Further, laboratory costs also depend on whether a laboratory chooses to redesign and expand its current facility rather than construct a new facility. The Huntington Beach Police labora- tory, which plans to add an additional 500 square feet to its existing facility to accommodate DNA testing, estimates the project will cost $290 per square foot or $145,000. The Hunting- ton Beach Police laboratory director stated the cost per square foot of the addition may be higher than other expansion projects because of the small size of the addition. C A L I F O R N I A S T A T E A U D I T O R 23 MANY LABORATORIES USE OUTMODED EQUIPMENT THAT MAY RESULT IN LOWER LABORATORY PERFORMANCE LEVELS Our consultants determined that 10 of the 19 laboratories use outmoded equipment that they must soon replace. We also found that most laboratories have not developed long-term plans for the systematic replacement of their outmoded equip- ment. Outmoded equipment can result in high maintenance As instruments become costs, unreliable test results, and unacceptably low laboratory older, equipment failures performance. It can also limit opportunities for staff to develop are more frequent, creating their skills using modern techniques. Furthermore, unless a work disruption and laboratory has ready funds to immediately replace older equip- further delays. ment when failure occurs, it will be left with no operational equipment. As instruments become older, equipment failures become more frequent, creating work disruption and further delays. Additionally, repair companies over time discontinue servicing older equipment, and manufacturers cease making replacement parts. ASCLD/LAB standards require each labora- tory to properly calibrate its instruments and equipment; nevertheless, ASCLD/LAB will still accredit a laboratory with outmoded equipment. Our consultants observed that at least four laboratories should replace outdated gas chromatograph/mass spectrometer (GC/MS) instruments. The GC/MS is a powerful tool used to identify drug samples, arson evidence, and other materials collected at a crime scene. As an example, the Huntington Beach Police laboratory has a GC/MS that is approxi- mately 15 years old and has a broken cooler. Because the laboratory does not have the funds to replace this equipment, staff found a creative way to cool the GC/ MS using hoses rigged to a faucet. How- ever, this method could negatively affect the analysis of the evidence processed by this instrument. Similarly, the consultants also observed that at least three labo- ratories should replace their Fourier Transform Infrared Spec- trometers (FTIR). The FTIR is an expensive instrument that analyzes different substances, such as drugs, plastics, and paints. 24 C A L I F O R N I A S T A T E A U D I T O R Similar to the GC/MS, as the FTIRs become older and begin to fail, repair companies may no longer support the equipment because manufacturers no longer supply replacement parts. To provide some perspective on how significant the cost to replace equipment can be, we obtained some estimates. The estimated cost to replace the four GC/MS instruments and the three FTIRs is $280,000 and $183,000, respectively. In addition, our consultants found eight gas chromatograph (GC) instru- ments, which separate mixtures into individual components, that the laboratories should replace for a cost of $280,000. In total, to replace these fifteen instruments, the laboratories will have to spend nearly $750,000. Our consultants identified these instruments as just some of the examples of outdated equipment in the laboratories. The consultants also found many outdated microscopes that they estimate cost between $2,000 and $10,000 each to replace. Additionally, our review of laboratory equipment lists identified many other examples of equipment 10 to 20 years old that the laboratories may need to replace. Furthermore, some laborato- ries did not provide inventory lists and others provided incom- plete lists. The actual cost to replace the outmoded equipment at all 19 laboratories could reach millions of dollars. In contrast to the 10 laboratories that have outmoded equip- ment, our consultants also observed that most of the equipment at the Los Angeles Police laboratory is new. In response to the media attention focused on the O. J. Simpson case, the Los Angeles Police laboratory recently purchased new and improved equipment. Our consultants believe that many laboratories experience a “feast or famine” situation with their equipment. The laboratories receive an equipment funding windfall in response to a crisis rather than routinely Without a capital equipment purchase equipment with funds set aside based on a long-term plan, laboratories risk capital equipment plan. replacing outdated equipment all at once. Without a capital equipment plan, laboratories run the risk of facing the high cost of replacing outdated equipment all at once. To avoid this, our consultants suggest that laboratories develop a capital equipment replacement plan by first estimating the replacement costs of the current equipment inventory. Once the replacement costs are known, they recommend that each labora- tory annually budget an amount equal to between 10 percent and 15 percent of the total replacement costs for the purpose of purchasing new equipment. For example, the Alameda County C A L I F O R N I A S T A T E A U D I T O R 25 Sheriff’s laboratory has not received funds designated for the replacement of equipment for many years, and its instrumenta- tion is very old and in need of replacement. In response to this concern, the laboratory hired consultants to develop a plan to transform the existing laboratory into a full-service forensic laboratory providing timely, state-of-the-art technical informa- tion to the county justice system. In its 1996 report, the consult- ant proposed an equipment replacement budget of $1,500,000 over the next four years and an annual budget of $300,000 after 2001 to continue acquiring state-of-the-art instruments. The $300,000 represents approximately 18 percent of the laboratory’s 1998 annual budget. IMPROVED MANAGEMENT INFORMATION SYSTEMS COULD INCREASE STAFF EFFICIENCIES AND MANAGEMENT OF OPERATIONS All 19 laboratories have either manual or automated manage- ment information systems, yet these systems fail to generate the information needed to effectively manage laboratory operations. Additionally, manual systems may be inefficient because they require valuable staff time to record and manipulate the data. Although the 19 labora- ASCLD/LAB states that a management information system tories use some form of should process meaningful statistical data, such as caseload management information, distribution, the amount of time a case involves, and informa- these systems fail to tion which is helpful in budgetary planning and the allocation generate data needed to of personnel and resources. Without this data, laboratory man- make effective decisions. agement will have difficulty in determining if staff and equip- ment are fully utilized, thus limiting management’s ability to plan and manage its resources. Moreover, without workload and backlog statistics, a manager cannot quantify or justify the need for additional staff, equipment, and funds. According to the ASCLD/LAB standards, laboratories should have a management information system to assist the laboratory in accomplishing its objectives; however, ASCLD/LAB will still accredit a laboratory that does not have a management information system. At least nine of the laboratories manually track limited forensic information. For example, rather than use a central database, the Los Angeles Police laboratory’s staff manually record case data on a spreadsheet. Serology staff manually record the num- ber of serology cases completed, the number and types of analy- ses performed, related court appearances and testimony given, and the number of staff working in that section. Similarly, at the 26 C A L I F O R N I A S T A T E A U D I T O R Long Beach Police laboratory, staff maintain their own statistics and submit monthly information to one employee, who compiles it. Although a manual management information system can accurately track data, calculating statistics that management can use when making decisions is labor intensive and subject to more errors. Computerized management information systems exist and easily track and manipulate data, saving staff time and increasing a laboratory’s efficiency. In addition, a computerized system designed to generate useful reports provides manage- ment with information it can use to complete budgetary plans and to allocate personnel and resources. In addition to many of the laboratories’ failure to generate information for internal management, much of the data is not comparable among laboratories. For example, one laboratory In addition to the many may define workload in terms of the number of serology cases failures to generate received, whereas another may track the number and types of internal management tests performed on a serology case but not the number of cases information, much of the received. Without comparable workload data to gauge resource data collected is not use and to determine the cost of laboratory operations, the comparable among laboratories’ management cannot make informed decisions on laboratories. whether their costs and operations are in line with others. Further, on a larger scale, they lack information to determine whether they should regionalize services, such as DNA testing, if it would be more cost-effective to use the services of private laboratories, or whether to consolidate laboratories in the same area for greater efficiency. Several of the laboratory directors recognize the need for an automated laboratory management information system. In fact, since 1991, the Santa Clara County District Attorney’s labora- tory has used an automated system to track requests for crimi- nalistic services from user agencies. Because it bases customer charges on the information supplied by its management infor- mation system, the laboratory must accurately track and com- pile this data. Moreover, all evidence submitted to the laboratory is bar coded and is tracked on the system, along with individual case assignments and staff performance. The system also gener- ates statistical reports on case backlog, turnaround times, cases completed, activities performed, and ASCLD/LAB annual statis- tics. In addition, the staff use the system to manage all aspects of the laboratory’s day-to-day business with the intent to increase efficiency, as well as to compile statistics to support its annual budget and for national and grant-reporting purposes. C A L I F O R N I A S T A T E A U D I T O R 27 Our consultants believe that all laboratories will benefit from an integrated management information system that bar codes evidence, logs cases, tracks evidence and test results, manages case data such as time from receipt to completion, cross refer- ences customer data, and enables automated or manual report- ing. Several laboratories have already obtained cost estimates for such systems. Based on estimates provided by the Alameda County Sheriff’s, the San Diego Police’s, and the Contra Costa County Sheriff’s laboratories, the cost of a management infor- mation system, including hardware and software, can range from $3,000 to $8,000 for each workstation; the cost for a large laboratory, such as the San Diego Police laboratory, may run between $300,000 and $400,000. The purchase of a management information system is a signifi- cant initial cost, but in the long run, freeing valuable staff time from manually recording data will increase efficiency by provid- ing management with the data necessary to monitor operations and ensure full utilization of staff and equipment. MOST LABORATORIES LACK A DOCUMENTED STAFF TRAINING PROGRAM We found that 9 of the 19 laboratories do not have documented training programs for employees in each functional area, such as serology and toxicology. Additionally, 9 of the 19 laboratories do not include training costs in their budgets as an individual line item. Training programs assist in refining and developing the skills necessary to perform competently and efficiently, as well as expose staff to changing trends and discoveries in the forensic sciences. While training programs are not a mandatory require- Training programs assist ment for accreditation, without these programs employees may examiners in refining and not fully develop their forensic skills. developing their skills to perform competently and ASCLD/LAB states that the laboratories should document their efficiently. training programs or maintain records of the time and funds spent on training. According to ASCLD/LAB, through their training programs, laboratories should teach the skills required to achieve the minimum standards of competence and maintain good laboratory practices. Additionally, because laboratory examiners present and defend their findings in open court, they must develop the technical and personal skills necessary to do so competently. Since 9 of the 19 laboratories have not docu- mented their training programs, they cannot ensure they have developed staff’s technical and personal skills. 28 C A L I F O R N I A S T A T E A U D I T O R Furthermore, nine laboratories do not budget specifically for training costs as an individual line item. Instead, most of these laboratories rely heavily on the essentially free training courses the California Criminalistics Institute (institute) offers and limited FBI courses rather than pay for other training. Even though many of the examiners participate in the courses either the institute or the FBI offers, our consultants believe the exam- iners should take additional outside courses and study other approaches to casework and tests that could improve the effi- ciency of laboratory operations. The laboratory directors recog- nize the need for outside training, but they do not have the budgetary resources for other training opportunities. Because ASCLD/LAB does not guide laboratories on specific training requirements, we reviewed the budgets of the nine laboratories that do budget specifically for training to identify a reasonable budget allocation. We summarize the data in Table 2. TABLE 2 Forensic Laboratory Training Budgets Total Number Fiscal Year Training Budget of Technical 1997-98 at $1,000 per Laboratory Staff Training Budget Technical Staff Alameda County Sheriff 10 None $ 10,000 Contra Costa County Sheriff 27 $ 18,500 27,000 El Cajon Police 5 None 5,000 Fresno County Sheriff 6 None 6,000 Huntington Beach Police 7 4,750 7,000 Kern County District Attorney 12 20,000 12,000 Long Beach Police 16 7,229 16,000 Los Angeles Police 101 None 101,000 Los Angeles County Sheriff 105 107,091 105,000 Oakland Police 15 None 15,000 Orange County Sheriff 82 24,463 82,000 Sacramento County District Attorney 29 22,050 29,000 San Bernardino County Sheriff 52 7,500 52,000 San Diego Police 31 6,000 31,000 San Diego County Sheriff 28 None 28,000 San Francisco Police 9 None 9,000 San Mateo County Sheriff 17 None 17,000 Santa Clara County District Attorney 29 2,859 29,000 Ventura County Sheriff 25 None 25,000 Total $606,000 C A L I F O R N I A S T A T E A U D I T O R 29 The amount of money the 10 laboratories budgeted for training varied from $99 to $1,670 per technical staff member annually. According to our consultants, at a minimum, each laboratory should expect to spend approximately $1,000 for each technical staff member. Our consultants also noted that training costs $120 each day and the associated travel and per diem costs can quickly use up the $1,000. As shown in Table 2, using $1,000 for each technical staff member, the 19 laboratories could expect to spend approximately $606,000 annually for training costs. n 30 C A L I F O R N I A S T A T E A U D I T O R CHAPTER 3 Laboratories Need to Evaluate the Effectiveness and Efficiency of Their Services CHAPTER SUMMARY L aboratories do not collect or track information necessary to evaluate forensic services and their associated costs beyond the management information systems described in Chapter 2. Furthermore, laboratories can neither assure that they are delivering forensic services in the most efficient man- ner, nor that the forensic examiners perform enough tests to maintain competency in their respective disciplines. Moreover, some laboratories may provide services at costs greater than necessary. Without data to effectively manage, the laboratories cannot assess whether certain services are cost-effective, compe- tent and credible, and may limit their ability to provide prompt, accurate results to the law enforcement agencies they serve. FORENSIC SERVICES OFFERED VARY BY LABORATORY The 19 local laboratories offer numerous forensic services, such as identifying and grouping blood and other body fluids; exam- ining firearms; analyzing gunshot residue; analyzing hair, glass, fingerprints, and other crime scene evidence; and examining questioned documents. The last service is the examination of any type of printed, typed, or written material to determine its source, whether the document was altered, or other information surrounding its production. Some laboratories offer the full range of forensic services while others limit their services. For example, the larger laboratories in Los Angeles and Orange counties offer all of the approximately nine forensic services, while the El Cajon Police laboratory only provides two services. For services they cannot provide, local laboratories employ private laboratories. For example, the laboratory under the San Bernardino County Sheriff sends its body fluid samples to a private laboratory to determine the presence of drugs. Further- more, the local laboratories seldom request services from the C A L I F O R N I A S T A T E A U D I T O R 31 Department of Justice’s forensic laboratories because the depart- ment primarily serves the 46 counties without forensics labora- tories and its backlogs preclude it from taking on any additional work. LABORATORIES CAN NEITHER ASSURE THEIR SERVICES ARE COST-EFFECTIVE NOR IDENTIFY APPROPRIATE ALTERNATIVES Most local laboratories have not performed the necessary analy- ses to determine whether it is more cost-effective to provide a particular service in-house or to pay a private laboratory for the service. The laboratories cannot analyze their services because most do not systematically collect and track the information necessary to monitor individual tests and the associated costs. Legislation required that we review the services local forensic laboratories provide and assess alternatives to those services, such as using private laboratories or consolidating services at designated local or state laboratories. However, we could not fully assess whether alternative modes for providing services would be appropriate or cost-effective because the laboratories do not maintain pertinent workload data. Some Laboratories Plan to Implement DNA Testing Even Though It May Not Be Cost-Effective to Do So DNA technology is emerging as an invaluable tool in forensic science and will ultimately replace conventional methods of analyzing biological evidence. To assist laboratories in develop- ing or improving forensic DNA testing, the federal government The laboratory consort- approved the DNA Identification Act of 1994 (act), which ium’s proposal states that established a five-year, $40 million grant program that the each of the 17 local National Institute of Justice (NIJ) administers. laboratories is eligible for the same funding, The State Department of Justice’s (department) Bureau of Foren- regardless of caseload. sic Services and 17 of the local laboratories formed a consortium to apply for federal funds to develop DNA testing in California and to ensure that the laboratories performing this testing meet certain professional standards. The consortium’s proposal states that each laboratory is eligible for the same amount of funding, regardless of caseload. Its approach, while noble, is questionable since the complexity and cost of the testing may make it imprac- ticable for small laboratories. In fact, at least 2 of the consortium 32 C A L I F O R N I A S T A T E A U D I T O R laboratories do not anticipate enough casework to justify adding this testing to their services. Furthermore, laboratories with small caseloads and infrequent testing will not be able to assure the continued competency of the analysts performing the DNA testing. Spreading the limited funds among the State and 17 laboratories may not be maximizing the use of the federal funds. The consortium requested a total of $6.5 million over the five years of the grant and, to date, it has received $1.4 million. Anticipated Caseloads at Several Laboratories Are Too Low to Justify Implementing DNA Services In its bulletin requesting applications from state and local governments for funding, the NIJ indicated that some laborato- ries may not be able to justify DNA testing capabilities because of low caseload numbers, lack of qualified personnel or labora- tory space, or the sheer number of laboratories desiring DNA testing capabilities. Therefore, the NIJ encouraged states with more than one current or prospective DNA laboratory to work together to develop statewide DNA testing programs and to submit consortium proposals covering all affected laboratories. In response, in 1995 the department developed the consortium with 17 of the 19 local laboratories to apply for a portion of the funds. The department excluded the El Cajon and Long Beach Police laboratories because they did not plan to implement DNA testing. At the time the consortium submitted its proposal, 9 laboratories had already established some form of in-house DNA casework capabilities, while the remaining 8 planned to implement DNA testing for the first time. However, at least 2 of the 8 laboratories anticipate small caseloads and are not good candidates for DNA testing. Table 3 on the following page shows the expected caseload for the 8 laboratories that planned to implement DNA testing. According to our consultants, a minimum of 25 cases per year, or at least one case every two weeks, is essential to ensure that critical skills of the DNA analysts are maintained. Further, our Our consultants believe a consultants believe that average caseloads should range from 5 minimum of 25 cases per to 10 cases per analyst per month, or a minimum of 60 cases year is essential to annually for a laboratory with one DNA analyst. Moreover, the maintain DNA analysts’ standards of the American Society of Crime Laboratory Directors skills. Laboratory Accreditation Board (ASCLD/LAB) for DNA testing require that a laboratory sustaining a DNA unit must have a well-qualified technical leader and an effective review process. C A L I F O R N I A S T A T E A U D I T O R 33 TABLE 3 Anticipated DNA Caseload Number Number Cases/ Laboratory of Cases of Staff Staff Alameda County Sheriff 20 3 7 Huntington Beach Police 25 1 25 Kern County District Attorney 150 4 38 Sacramento County District Attorney 100 2 50 San Diego County Sheriff 256 5 51 San Francisco Police 180 2.5 72 San Mateo County Sheriff 60 2 30 Ventura County Sheriff 100 2 50 Source: Information provided by laboratory directors. Note: Laboratories in bold have caseloads significantly below the recommended level of 60 cases per analyst per year. Therefore, unless the laboratory seeks to contract with a consult- ant for the technical leader role, the standards imply there should be at least two qualified DNA analysts per laboratory. Four laboratories planning to implement DNA testing anticipate caseloads well below the level necessary to assure the DNA analysts maintain their competency. In addition to the concerns about maintaining DNA analysts’ skills, our consultants also expressed concern regarding the cost- effectiveness of implementing DNA testing at many of the laboratories. Costs to prepare a laboratory for DNA testing are considerable. Adequate space is essential. DNA standards require that, at a minimum, a facility must have at least two separate rooms specifically designated for DNA testing to prevent con- tamination of evidence. To meet these standards, some laborato- ries must spend a significant amount of funds to either remodel existing space or add additional space. Of particular concern to our consultants is the fact that some of Some of the laboratories the laboratories planning to perform DNA analysis currently planning to add DNA lack sufficient space for their other services. Yet, rather than analysis lack sufficient invest funds to improve current conditions, these laboratories space for their current intend to use both local and federal funds to implement a DNA services. service. For example, the Huntington Beach Police laboratory plans to spend approximately $145,000 to expand its facility by 500 square feet to house the new DNA section of its laboratory. 34 C A L I F O R N I A S T A T E A U D I T O R However, we question this decision for two reasons. First, ac- cording to our consultants, the laboratory’s estimate of 25 cases per year is not sufficient for the DNA analysts to maintain their skills and, secondly, it lacks sufficient space for its present ser- vices. In addition to facility costs, the cost of the equipment is also significant. According to our consultants, to establish one DNA station requires items such as freezers, pipettes, thermal cyclers, and the main DNA testing unit with software, for a total cost of approximately $120,000. Our consultants indicate that to efficiently utilize a DNA station requires four analysts and 260 cases each year; however, the anticipated caseloads and staffing levels of seven laboratories planning to implement DNA testing do not indicate they will use their DNA facilities and equipment at these levels. Three Laboratories Have Not Yet Received DNA Funds During the past three years, five of the eight laboratories have received DNA funds for equipment and other supplies. The consortium plans to provide funds, for equipment if available, to the remaining three laboratories during 1999 and 2000. Two of these laboratories, in addition to two of the five laboratories that have already received some DNA funds, have not yet started this testing. Before distributing additional funding, we believe that the State and the laboratories should consider whether other options, such as regionalization of DNA services at certain laboratories, would be more cost-effective and would optimize the critical skills of the DNA analysts rather than spreading these services among 17 local laboratories. SOME LABORATORIES HAVE CONSIDERED REGIONALIZING SERVICES Although the local laboratories have not analyzed the services they provide to determine whether other more cost-effective options exist, such as regionalizing an individual service or outsourcing to a private laboratory, some of the laboratories have considered consolidating all services within a region. For example, during the last 25 years, several groups have prepared studies recommending county regionalization of the laboratories under the San Diego Police and the San Diego County Sheriff, with the smaller El Cajon Police laboratory as a satellite office. C A L I F O R N I A S T A T E A U D I T O R 35 Similarly, during the last 20 years, a number of studies have addressed the regionalization of the forensic laboratories in Los Angeles County, including a recent study released in September 1998. Finally, the Alameda County Sheriff’s labora- tory director indicated that it had also considered regionaliza- tion with the Oakland Police’s laboratory or with the Contra Costa County Sheriff’s laboratory. Because there is no data to Although recommended by justify whether consolidating would result in greater efficiency, several studies, little less cost, or improved services, little progress has been made progress has been made towards consolidation or regionalization. towards consolidation or regionalization of In 1992, the San Diego County board of supervisors and the laboratory services. San Diego city council commissioned a $100,000 study to assess the county’s forensic science needs. The resulting study con- cluded that merging the police and sheriff laboratories, and using the El Cajon Police laboratory as one of three satellite facilities, would create a more efficient and cost-effective re- gional unit by reducing duplication and consolidating equipment, space, and supplies. These 1992 recommendations have not yet been acted upon. According to the director of the San Diego County Sheriff’s laboratory, the involved agencies have not reached an agree- ment on the structure, operation, and funding of a regional laboratory. In a continuing attempt to provide alternatives, the San Diego County Sheriff’s laboratory, in November 1996, proposed merging the overcrowded police laboratory into the current sheriff’s facility, which has sufficient space to inexpen- sively meet the current and future forensic needs of a regional laboratory. In addition, the merger would benefit the sheriff’s laboratory by providing the much-needed DNA section leader it currently lacks; however, the two laboratories have not yet come to any agreement on the merger. Similarly, since 1979, several groups have recommended merg- ing the forensic laboratories under the Los Angeles County Sheriff and the Los Angeles Police to achieve cost-effectiveness. However, the police department has argued that it would not receive the same level of service and has little to gain from consolidation. Therefore, until recently, progress towards con- solidation has been slow. During the last two years, however, the Los Angeles Sheriff and the California State University, Los Angeles, have met to discuss the construction of a new, shared forensic laboratory and training facility located on the univer- sity campus. In response to these meetings, the Los Angeles County Sheriff contracted with an architectural firm to design a 36 C A L I F O R N I A S T A T E A U D I T O R joint forensic laboratory and training facility with space for the police laboratory and the State’s forensic training institute. The consultant concluded that a regional, state-of-the-art facility avoids the unnecessary cost of duplicating certain facilities and equipment at separate locations. According to the consultant, the design of the facility incorpo- rates shared evidence storage and processing rooms, drying chambers, a firing range, gun and ammunition collections, polygraph testing rooms, analytical instrument labs, large case rooms, vehicle examination bays, chemical storage, and hazard- In addition to maximizing ous waste collection, thus saving significant building costs and facility use, regionalization improving the efficiency of the facility. A regional facility would could provide other bene- also effect other benefits, such as greater cooperation among the fits, such as interagency agencies, higher employee morale because of better working cooperation, improved conditions and state-of-the-art equipment, and better protection employee morale, and of evidence—making laboratory findings less vulnerable to legal increased credibility of test challenges. However, this project is only in its earliest phases results. and still requires negotiations concerning operational control of the facility and funding. WORKLOAD DATA IS NOT ALWAYS AVAILABLE, STANDARDIZED, OR CONSISTENT Although most laboratories have not analyzed whether their services are cost-effective, we attempted to evaluate the services using workload data. We planned to use this evaluation to consider options such as transferring cases between laboratories to equalize backlog and consolidating some services. However, we could not evaluate the services because the workload data was not always available, and when available, was not standardized or consistent. To evaluate the services, we requested the laboratories to provide us with workload data for each of the last five years. Although we received data, including the number of cases received, the number closed, case backlog at the end of each year, and turn- around times for some services, the laboratories did not maintain the same information in comparable categories. More specifically, seven laboratories did not maintain the number of cases received, five did not maintain the number completed, and eight did not maintain backlog data. In addition, only three laboratories tracked turnaround times, but only for a few of their services. As a result, we were unable to draw any conclusions about the cost-effectiveness of the laboratories’ services. C A L I F O R N I A S T A T E A U D I T O R 37 Currently, there are no standards requiring the laboratories to track specific types of workload data or to maintain the data in a Because laboratories do format that allows for comparison between laboratories. The NIJ not maintain standardized has recognized the lack of consistent and standardized workload workload data, they data as a problem at the national level. To address this concern, cannot draw conclusions NIJ provided funding to the National Institute of Standards and about the cost-effectiveness Technology, Office of Law Enforcement Standards, to assist of their services. ASCLD in developing a professional survey tool that would measure workload, backlog, and output of crime laboratories in a consistent and standardized manner. The survey tool was sent to laboratories in November 1997 so that the laboratories could begin reporting appropriate data for calendar year 1998 in early 1999. Even though the NIJ and ASCLD’s efforts are a step in the right direction, the local laboratories should take it upon themselves to maintain standardized and consistent workload data. By doing so, the laboratories could then consider whether more cost-effective options exist for some services they provide. For example, the director of one laboratory indicated that it is more cost-effective for a private laboratory to perform toxicology. A second director stated that the local laboratories could consider consolidating most trace services, which include the analysis of glass, paint, soil, and fibers from a crime scene, because the laboratories only receive a few cases a year and trace work is very expensive and time-consuming. Using more cost-effective options for some services will also allow the laboratories to increase the number of tests they complete for those services they choose to retain. As a result, the laboratories’ customers will continue to receive the same services, but with greater efficiency, and perhaps less cost. n 38 C A L I F O R N I A S T A T E A U D I T O R CHAPTER 4 Conclusions and Recommendations T he California Penal Code, Section 13892, required us to assess the 19 local forensic laboratories to identify improvements needed for the laboratories to achieve or maintain American Society of Crime Laboratory Directors Laboratory Accreditation Board (ASCLD/LAB) accreditation. We noted that even if a laboratory meets accreditation standards, it can still exhibit shortcomings, including health and safety problems, compromised efficiency, and ineffective use of labora- tory resources. Therefore, our recommendations address the changes the laboratories need not only to attain or maintain accreditation, but also to correct other conditions hindering the laboratories’ ability to operate more efficiently and effectively. As this report details, because many of the laboratories have neither designated quality managers nor implemented other essential elements of a quality control system, they cannot achieve or maintain ASCLD/LAB accreditation. We estimate that it will cost nearly $2 million annually for the laboratories to implement and maintain quality control systems, including hiring quality managers and additional support staff. As stated in Chapter 1, although 11 of the 19 laboratories have designated a quality manager, only 1 laboratory meets the staffing levels our consultants suggested. Additionally, many of these quality managers also had other responsibilities; therefore, they could not focus solely on quality assurance. Hiring indi- viduals to work exclusively in this area will help ensure the laboratories focus on implementing and maintaining appropri- ate quality control systems, including proficiency testing and court monitoring programs. The $2 million estimate also includes funds to purchase proficiency tests from external providers to administer to examiners at all 19 laboratories. In addition to these costs, the ASCLD/LAB estimates that the 19 laboratories will spend more than $190,000 in fees each time it inspects them. Laboratories could also improve the effectiveness of their opera- tions by constructing new facilities to allow staff to more efficiently and safely accomplish their analyses. We estimate that it will cost more than $221 million to construct new facili- C A L I F O R N I A S T A T E A U D I T O R 39 ties for the laboratories that do not currently meet the standards recommended by forensic laboratory design literature. We further noted that many laboratories have equipment that is between 10 and 20 years old and should be replaced. While our consultants did not estimate a total replacement cost, they did identify 15 items that need replacement at a cost approaching $750,000. The laboratories could also install management information systems to help them make decisions regarding the best use of available resources. Additionally, they could implement training programs to enhance the skills of their examiners. The costs to develop management information systems and implement training programs will vary with the size of the laboratories. Some of the laboratories estimated the cost to develop a man- agement information system can range from $3,000 to $8,000 per workstation depending on the number of workstations needed. Finally, if each laboratory spends $1,000 on training per examiner per year, as our consultants suggest, the annual costs to train staff at all 19 laboratories would reach $606,000. We also found that the laboratories do not accurately assess how cost-effective their services are because most do not track infor- mation on costs of individual tests. Neither have they explored whether outsourcing some services to private laboratories or regionalizing certain services are better alternatives. For ex- ample, at least two laboratories plan to implement DNA testing even though they will not generate enough casework to justify the investment of those funds for their laboratories. In addition, we attempted to examine whether the laboratories could transfer cases among themselves to reduce the backlog some laboratories experience, or whether consolidating some services would be appropriate. We were unable to draw any conclusions because workload data was either unavailable, inconsistent, or not standardized; however, some, such as the laboratories within Los Angeles and San Diego counties, have considered consolidating all services within a region, though these plans have yet to take shape. Because the laboratories will seek funding from the Legislature to correct the conditions we found, the Legislature will need to determine whether it will appropriate funds to correct all, or only some, of the laboratories’ deficiencies. Additionally, the Legislature will need to place certain constraints on the funds to 40 C A L I F O R N I A S T A T E A U D I T O R ensure the laboratories use them as intended and that the legislative funds supplement the budgets rather than replace local funds. Further, the Legislature should establish priorities for the use of funds. If the Legislature’s primary goal is for all of the 19 local laboratories to become accredited, then it should first appropri- ate funds to those that lack a quality assurance program. Labora- tories that receive funds for this purpose should appoint quality managers and related support staff. The quality managers should identify the minimum changes and improvements needed to allow their respective laboratories to attain accreditation. In addition, to assure continued confidence in their work, these laboratories should establish strong quality assurance programs. The quality managers and their staff should develop and imple- ment the various components of the quality assurance program, such as policy and procedure manuals, proficiency testing programs, programs to monitor court testimony, and annual quality and safety audits. To assure laboratories with established quality assurance pro- grams receive fair treatment, the Legislature should distribute any additional funds equitably so that these laboratories can improve conditions in their facilities, as well as refine their existing quality assurance programs. Finally, the Legislature should require the laboratories to document their efforts in determining the cost-effectiveness of their services and evaluate the potential benefits of consolidation or regionalization. To ensure that they meet and maintain ASCLD/LAB standards for accreditation, the local laboratories should do the following: • Appoint quality managers and support staff at sufficient levels to implement and maintain quality control and safety programs, including documentation and periodic auditing. • Implement a proficiency testing program. • Implement a court testimony monitoring program. • Work with their respective parent organization to budget the funds necessary for accreditation inspections and self- evaluations to demonstrate continued compliance with ASCLD/LAB standards. C A L I F O R N I A S T A T E A U D I T O R 41 To provide a safe environment for employees, maximize effi- ciency, and ensure evidence integrity, the laboratories should use qualified consultants to determine their specific facility needs and related costs. To ensure that their equipment is as reliable and efficient as possible, the local laboratories should do the following: • Complete an inventory of all equipment. • Develop and include in their annual budget a capital equipment replacement plan. • Replace their outmoded equipment with newer, more efficient models. To increase staff efficiencies and allow for more effective man- agement of laboratory operations, the laboratories should develop and implement management information systems appropriate for their size. To ensure the technical and personal skills staff need to most effectively and efficiently perform their work, the laboratories should do the following: • Develop, implement, and document a formal training program. • Develop and include in their annual budgets a line item for training costs. To provide services in the most cost-effective manner and at a level to maintain the critical skills of their staff, the laboratories should do the following: • Analyze the costs of each of their services and compare those costs to private laboratories’ charges. • Consider consolidating or regionalizing services, including DNA testing. • Continue to consider consolidating laboratories within a specific region. 42 C A L I F O R N I A S T A T E A U D I T O R To assist in considering options, such as whether cases may be transferred between laboratories to equalize backlog or whether consolidating some services would be appropriate, the laborato- ries should agree to develop and maintain standardized and consistent workload data for their services. Before agreeing to certain workload measures, the local laboratories should con- sider the standards included in the ASCLD survey tool to establish consistency. We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted governmental auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, KURT R. SJOBERG State Auditor Date: December 15, 1998 Staff: Elaine M. Howle, CPA, Audit Principal Denise L. Vose, CPA Christine Berthold, CPA Farra Bracht Chris Harris, CPA Harvey Hunter Virginia Anderson Johnson Ronald Sherrod C A L I F O R N I A S T A T E A U D I T O R 43 Blank page inserted for reproduction purposes only. 44 C A L I F O R N I A S T A T E A U D I T O R APPENDIX Summary of ASCLD/LAB Accreditation Standards ESSENTIAL CRITERIA IMPORTANT CRITERIA DESIRABLE CRITERIA Laboratories Must Achieve 100% Laboratories Must Achieve 70% Laboratories Must Achieve 50% Compliance Compliance Compliance LABORATORY MANAGEMENT Does the laboratory maintain written policies and procedures for the following: Handling and preserving evidence Duty hours Preparation, storage, and integrity disposition of case records Security Leave time Control of material and supplies Maintenance and calibration of equipment and instruments Job requirements and description Personnel evaluations and objectives Employee grievances Does the laboratory have a formal Is there an inventory record for written budget? laboratory equipment and instruments? Does the laboratory maintain a Does the organizational structure definition of the director’s authority group the work and personnel to and responsibilities? allow for efficiency of operations and take into account interrelations between disciplines? Are performance expectations established for staff? Is there a management information system (MIS) that provides statistical data, such as caseload distribution and case turn-around time, that is used for budgetary planning purposes? Do supervisors carefully and objectively review laboratory activities, methods, and personnel? C A L I F O R N I A S T A T E A U D I T O R 45 ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ Has the laboratory director considered and taken action to balance resource allocation? Is there evidence of regular staff meetings? ESSENTIAL CRITERIA IMPORTANT CRITERIA DESIRABLE CRITERIA Laboratories Must Achieve 100% Laboratories Must Achieve 70% Laboratories Must Achieve 50% Compliance Compliance Compliance EVIDENCE CONTROL AND QUALITY ASSURANCE Does the laboratory have a written Does the laboratory have a quality Do serology and DNA examiners or secure electronic chain-of- manager? If yes, is there a job have access to well-established custody record with all necessary description? population databases on the data that provides for complete distribution of all genetic markers tracking of all evidence? typed in the laboratory? Is all evidence marked for Does the laboratory have a quality Do serology and DNA examiners identification? control manual? have access to, and do they generate, local population databases on the distribution of all genetic markers, which are typed in the laboratory? Is evidence stored under proper Are audits of the entire laboratory seal? operation conducted annually? Is evidence protected from loss, Does the laboratory conduct an cross-transfer, contamination, annual review of its quality control and deleterious change? system? Is there a secure area for overnight Are the instruments/equipment and long-term storage of evidence? adequate for the procedures used? Are the procedures used generally Are the instruments/equipment in accepted in the field or supported proper working order? by data gathered and recorded in a scientific manner? Are the technical procedures used by the laboratory documented, and are documents available to personnel for review? Are appropriate controls and standards specified in the procedures, and are they used to ensure the validity of examination results? Is the quality of the standard samples and reagents adequate for the procedures used? Does the laboratory routinely check the reliability of its reagents? Are instruments/equipment properly calibrated? Do examiners generate—and does the laboratory maintain in a case record—notes, worksheets, etc. used by examiners to support their conclusions? 46 C A L I F O R N I A S T A T E A U D I T O R ESSENTIAL CRITERIA IMPORTANT CRITERIA DESIRABLE CRITERIA Laboratories Must Achieve 100% Laboratories Must Achieve 70% Laboratories Must Achieve 50% Compliance Compliance Compliance EVIDENCE CONTROL AND QUALITY ASSURANCE (Cont.) Does the laboratory have, use, and document peer review of the reports? Does the laboratory conduct and document administrative reviews of all reports issued? Does the laboratory monitor court testimony of examiners at least annually? If a significant technical problem is indicated, does the laboratory initiate a review and take any corrective action? Does the laboratory have a documented program of proficiency testing? HEALTH AND SAFETY Does the laboratory have a delegated health and safety manager? Does the laboratory have a health and safety manual? Is the health and safety program monitored regularly and reviewed annually? Does the laboratory have and encourage the use of safety devices, particularly those required by its health and safety manual? Does the laboratory have an adequate number of trained safety personnel? Does the laboratory have proper equipment and material available for the handling of carcinogenic, toxic, and other dangerous material spills? Are safety showers and eye wash stations present and have they been maintained? C A L I F O R N I A S T A T E A U D I T O R 47 ESSENTIAL CRITERIA IMPORTANT CRITERIA DESIRABLE CRITERIA Laboratories Must Achieve 100% Laboratories Must Achieve 70% Laboratories Must Achieve 50% Compliance Compliance Compliance HEALTH AND SAFETY (Cont.) Are sufficient exhaust hoods available to maintain a safe work environment? Are there sufficient first aid kits available and strategically located? Is appropriate space provided for safe storage of volatile, flammable, and explosive material? Are emergency exits from the laboratory accessible and free from obstruction? PERSONNEL QUALIFICATIONS Do controlled substances, trace Does the laboratory director Does the laboratory director have evidence, toxicology, and serology possess a degree in natural science, 1) at least five years forensic ex- examiners have baccalaureate or is the director supported by perience, 2) formal management degrees? scientific personnel of sufficient training,and, 3) two years rank and authority? managerial experience? Do experience and training records Does the laboratory conduct Do firearms, questioned for firearms, questioned interlaboratory or intralaboratory documents, and latent print documents, and latent print proficiency testing using the blank, examiners have baccalaureate examiners show experience re-examination, or known degrees? commensurate with the standards techniques? examinations and testimony? Do DNA examiners meet education, training, and experience required by the Technical Working Group on DNA analysis methods? Does each DNA examiner participate in at least one external proficiency test from an approved test provider? Do all examiners understand the instruments and the methods and procedures used? Do all examiners successfully complete a competency test prior to assuming casework responsibility? 48 C A L I F O R N I A S T A T E A U D I T O R ESSENTIAL CRITERIA IMPORTANT CRITERIA DESIRABLE CRITERIA Laboratories Must Achieve 100% Laboratories Must Achieve 70% Laboratories Must Achieve 50% Compliance Compliance Compliance PERSONNEL QUALIFICATIONS (Cont.) Do all examiners complete an annual proficiency test? Does the laboratory participate in proficiency testing programs conducted by approved test providers? Do the support personnel meet the requirements of their job description? Are the job descriptions for support personnel and the duties performed in agreement? TRAINING AND DEVELOPMENT Does the laboratory have an employee development program? Does the laboratory provide training programs in each functional area for new or untrained employees? Does the forensic library contain current books, journals, and other literature dealing with each functional area? Does the laboratory have a system in place to encourage review of new literature? SECURITY Is access to the operational area of the laboratory controllable and limited? Do exterior entrance/exit points Does the laboratory have an have adequate security control? intrusion alarm or use security personnel to secure the laboratory during vacant hours? Do all internal areas requiring controlled access have a lock system? Are all keys, magnetic cards, etc., Is there a fire detection system in accounted for, and is their the laboratory? distribution limited? C A L I F O R N I A S T A T E A U D I T O R 49 ESSENTIAL CRITERIA IMPORTANT CRITERIA DESIRABLE CRITERIA Laboratories Must Achieve 100% Laboratories Must Achieve 70% Laboratories Must Achieve 50% Compliance Compliance Compliance LABORATORY PHYSICAL SPACE AND DESIGN Does each employee have Does the facility provide sufficient adequate work space? space for storage of supplies, equipment, and tools? Is there adequate writing space for Are accessories stored near each examiners? instrument to facilitate the use of equipment and instruments? Is there adequate space for Do the relative locations of records? functional areas facilitate the use of equipment and instruments? Is adequate space available for each Is there general cleanliness and instrument to facilitate its apparent good housekeeping in operation? the laboratory? Does the building’s physical design permit the efficient flow of evidence from the time of its acceptance until its proper disposal? Is there adequate and proper lighting? Is the plumbing and wiring adequate? Does the laboratory have proper general ventilation? Does the laboratory have adequate cooling/heating ? 50 C A L I F O R N I A S T A T E A U D I T O R Agency’s response to the report provided as text only: Alameda County Sheriff’s Office Lake Side Plaza, 1401 Lakeside Drive, 12th Floor Oakland, California 94612 (510) 272-6866 December 2, 1998 Kurt R. Sjoberg State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Subject: REPORT REVIEW - FORENSIC LAB ENHANCEMENT PROGRAM - REF. PENAL CODE SECTION 13892 - NEEDS ASSESSMENT. Dear Mr. Sjoberg: In response to your request of November 23, 1998, my staff and I have reviewed the draft copy of the subject report entitled “Forensic Laboratories: Many Face Challenges Beyond Accreditation to Assure the Highest Quality Services” that was received on November 24, 1998. After careful review and consideration of various items, conclusions and recommendations in the report (some of which have been clarified and/or rectified by way of recent telephone conferences between Sheriff’s and Auditor’s staff) we feel that it is important to respond to statements in Chapter 3 relating to estimated DNA caseloads and implementation of forensic DNA analysis services. We take strong exception with the implication that a “caseload” threshold number in some way affects a DNA analyst’s competency to perform forensic DNA analysis, especially when “case” and “caseload” are not well defined terms nor do they specify numbers of evidence samples examined or tests performed. Our laboratory provided the State Auditor’s Office with an estimated number of 20 cases in our first year that we initially expect to be examining for DNA evidence. We fully anticipate that number of cases to increase as our user agencies become more aware of this new service. Every case that has DNA evidence will have multiple, individual samples that need to be analyzed. An estimated 20 cases, in our first year, would involve approximately 60 to 100 evidence samples to be extracted, quantitated and typed. In addition, various positive and negative quality control samples must be processed concurrently with each group of evidence samples through the same procedures. Consequently, the total number of samples to be examined from 20 cases would easily reach several hundred samples. R-1 1* In our opinion, it is inappropriate for the State Auditor’s report to link a laboratory’s “caseload” number to the worthiness of its DNA typing analysis program and its analyst’s competency. The federal government’s DNA Identification Act of 1994, authorized the creation of the DNA Advisory Board (DAB). The DAB was charged with developing standards for quality assurance for forensic laboratories and forensic analysts who perform DNA typing analysis. The quality assurance standards became effective October 1, 1998, and are detailed in a document entitled “Quality Assurance Standards for Forensic DNA Testing Laboratories”. These national standards have not specified that a forensic laboratory engaged in DNA typing analysis should have a certain “caseload” in order to provide quality results. We consider DAB and the American Society of Crime Lab Directors/ Lab Accreditation Board (ASCLD/LAB) to be the controlling authorities on quality assurance issues surrounding DNA typing. We believe that if a reputable crime laboratory meets the standards set forth by the DAB, and is ASCLD/LAB accredited, then the criminal justice system should have confidence in the quality of that laboratory’s work product, regardless of annual caseload. My staff and I are fully committed to the continuing implementation of our forensic DNA analysis program for Alameda County. Thank you for your efforts in this most important area of mutual concern. Sincerely, Signed by: Charles C. Plummer Sheriff-Coroner CCP:AES:SAS:sl *California State Auditor’s comment on this response begins on page R-3. R-2 COMMENT California State Auditor’s Comment on the Response From the Alameda County Sheriff T o provide clarity and perspective, we are commenting on the Alameda County Sheriff’s response to our audit report. The number corresponds to the number we have placed in the response. 1 Contrary to the laboratory’s contention, we do not link caseload to the worthiness of the DNA program. Rather as stated on page 32, the complexity and cost of DNA testing may make it imprac- tical for small laboratories. Furthermore, by evenly spreading the limited funds among the State and 17 laboratories, the consor- tium may not be maximizing the use of federal funds. R-3 Blank page inserted for reproduction purposes only. R-4 Agency’s response to the report provided as text only: County of Contra Costa Office of the Sheriff-Coroner Forensic Services Division 1960 Muir Road, Suite 201 Martinez, California 94553-4800 December 2, 1998 Kurt R. Sjoberg, State Auditor California State Auditor's Office Attn: Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95824 Dear Mr. Sjoberg: Enclosed are the hard copy and diskette with our comments to the draft report "Forensic Laboratories: Many Face Challenges Beyond Accreditation to Assure the Highest Quality Services" for Contra Costa County Forensic Services Division . Sincerely, WARREN E. RUPF SHERIFF Signed by: Gerald T. Mitosinka, Chief Forensic Services Division GTM:ks enclosures R-5 Page 1, Paragraph 1 1* Report should elaborate the reason(s) why an audit of the Department of Justice (DOJ) Laboratory system was not included in the report. Page 2, Paragraph 2 Contra Costa County does have several of the components of a quality control system in place: these include policies, procedures, training of technical staff prior to assignment of casework; proficiency testing, technical review of reports, etc. Page 4, Paragraph 3 Contra Costa and Solano Counties have consolidated service as evidenced by a service agreement in place providing both Counties the ability to amortize laboratory overhead and obtain necessary, but costly equipment. Contra Costa County also provides some evidence examinations, such as gunshot residue analysis, to several additional Northern California counties. 2 This report fails to evaluate the issues related to the logistics of transferring case backlog to another laboratory. This is in conflict with comments made in the section "Most Laboratories Do Not Meet the Recommended Standard of 1,000 Square Fee Per Staff Member" (page 30 of the Draft Report). Page 5, Paragraph 3 3 Legislation should require laboratories to evaluate the "productivity" of staff rather than "cost effectiveness". Agencies who continue to submit evidence to forensic laboratories where they pay fees are evidence of a "cost effective" service. An agency would not choose to pay for a product they do not consider valuable. Law enforcement agencies seek services not based solely on cost, but, in our experience, they seek timeliness of results and a quality work product at a reasonable cost. Page 7, Bullet #3 4 Contra Costa County is an example of privatized forensic services. We provide services to the Defense Bar and private citizens as well as the law enforcement community. We have analyzed our costs. Our fees for examination services are *California State Auditor’s comments on this response begin on page R-11. R-6 competitive with local private forensic laboratories. Page 7, Bullet #4 DNA examination services are now "front-line tests" and need to be close to the client agencies. Removing DNA testing from local laboratories will not develop an adequate level of local expertise to assist the law and justice community (police agencies and District Attorneys' offices) to evaluate crime scene evidence for its DNA potential prior to collection, prepare search warrants to include possible sources of DNA evidence, or to reconstruct crime scenes using information developed from DNA testing. Page 7, Last Paragraph 2 Transferring evidence to another laboratory creates a number of issues not adequately addressed by this report: The chain of custody in forensic work must be carefully documented. The standard of forensic practice dictates that unnecessary transfers of evidence are to be avoided. Expert testimony in court should be an anticipated result of any forensic analysis. Examinations performed outside the local area present difficulties in scheduling witnesses and incur travel (and perhaps per diem) expenses for either the laboratory who conducted the examination or the Prosecutor's office where the testimony will be presented. These expenses add to the overall cost of the analysis. These issues are rarely a factor when the local laboratory conducts the work and subsequently testifies. Page 9, Paragraph 2 Contra Costa County charges municipalities (all clients) for all services, not "certain services". Page 14, Paragraph 1, Last Line Contra Costa County Forensic Services Division knows its work is accurate because: The staff receives thorough training; All cases are technically reviewed for accuracy; and R-7 Many of our cases are scrutinized by private forensic consultants. The evidence that our work is impartial is demonstrated by the fact that we frequently provide examination services to the defense community. The evidence that our work is relevant is demonstrated by the fact our client agencies are willing to pay us fees for the work they submit. Page 20, Paragraph 2 5 Contra Costa County does have a proficiency testing program in place. Each analyst is tested in their subject areas prior to assuming casework responsibility. However, ongoing proficiency testing is not always conducted on an annual basis for every analyst at this time. Page 21, Paragraph 4 5 Contra Costa County does have a court monitoring program in place. Testimony feedback is solicited from prosecutors and defense attorneys. However, not all analysts are evaluated annually. (Copy of witness critique card attached.) Page 22, Paragraph 1 Contra Costa County is in the process of developing a Quality Manager position. Page 25, Paragraph 1, 6 The phase "documented training" in the last sentence be changed to "formal training". Page 30, Paragraph 2 2 This paragraph contradicts statements about consolidating services between laboratories to equalize backlog previously stated on Page 7, last paragraph. Page 39, Paragraph 2 6 Contra Costa County does document its training programs and maintain records of the time and funds spent on training for all staff members. Personnel files reflect training received by each staff member; and our budget reports reflect the cost associated with training. R-8 Page 39, Paragraph 2, Last Line Lack of training documentation does not necessarily imply management's lack of understanding of the skill level of staff. If an employee is trained, successfully completes qualifying tests, has work reviewed by peers and supervisors, and successfully passes proficiency tests, management can ensure they have developed staffs' technical and personal skills. Page 44, Paragraph 3 7 Contra Costa County has conducted the "necessary analyses" to provide particular services in-house or to pay a private laboratory for the services. Contra Costa County uses a private laboratory to conduct various toxicology examinations. Page 44, Last Paragraph Most private laboratories do not meet ASCLD accreditation standards. Sending casework to these laboratories would contradict California's goal to have evidence examinations conducted by Laboratories who meet these standards. It is expected that ASCLD will soon require for any casework being sent to another laboratory for examination, that the other laboratory must also be accredited. Currently, DNA cases that are sent to an outside laboratory for examination can only be examined by an accredited DNA laboratory. Prior to considering the use of another laboratory, the costs to perform the examination in-house verses the expenses incurred from the use of the private laboratory and related testimony expenses must be evaluated. Page 47 This report recommends that the average DNA analyst caseload should be a minimum of 60 cases annually. This recommendation does not define "caseload". While some laboratories may consider "DNA casework" to mean DNA extraction and typing alone, other laboratories include the isolation and characterization of biological fluids within the scope of DNA casework. The former approach employs a "prescreening" process conducted prior to DNA casework. This critical and time consuming step is not reflected in the auditors' recommendation. A more realistic analyst caseload, if "prescreening" is taken into account, would be 25 to 30 cases per year. R-9 Blank page inserted for reproduction purposes only. R-10 COMMENTS California State Auditor’s Comments on the Response From the Contra Costa County Sheriff T o provide clarity and perspective, we are commenting on the Contra Costa County Sheriff’s response to our audit report. The numbers correspond to the numbers we have placed in the response. 1 As stated in the Scope and Methodology on page 8, the legisla- tion mandated that we assess the needs of the 19 local forensic science laboratories. 2 As stated on page 37, we attempted to evaluate services using workload data, including backlog. However, we were unable to draw any conclusions because the data was not always available, nor, when available, was it standardized or consistent. 3 We disagree. If the legislature appropriates funds to the local laboratories, we believe those laboratories should be required to document their efforts in determining the cost-effectiveness of their services, and staff productivity contributes to cost-effective- ness. 4 Contrary to the laboratory director’s contention, while the Contra Costa County Sheriff’s laboratory may have some out- side clients, the laboratory is government-operated, therefore supported with public funds. 5 As the laboratory director stated, the laboratory does not always conduct proficiency tests or monitor analysts’ testimony annu- ally. As a result, it would fail an ASCLD/LAB inspection. 6 Per discussion with the laboratory director, we modified our text to reflect that the laboratory does have a documented training program. 7 Text modified. R-11 Blank page inserted for reproduction purposes only. R-12 Agency’s response to the report provided as text only: City of El Cajon Police Department 100 Fletcher Parkway El Cajon, California 92020 December 1, 1998 Mr. Kurt R. Sjoberg State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Mr. Sjoberg: Thank you for your review of the State's forensic laboratories. I cannot agree with some of your findings. Some examples include: 1) We maintain very accurate records of formal training received by Lab personnel. Those records, while technically housed in our Training Division, are available to the Lab Supervisor via his desk top computer; 2) Accuracy of findings, such as fingerprints, are routinely checked by other print examiners; 3) I believe the working areas for our Lab is appropriate. I do however support, what I believe to be, your overall conclusion that standardization (at least at the State level) and State support for funding that standardization is long overdue. The costs associated with forensic laboratories and the demands of expanded technology, such as DNA, are but some of the reasons why I believe regionalization of forensic laboratory services is essential. In the interim, I welcome the information in your report. I look forward to a speedy response by the legislature to help fund the changes. I must emphasize, however, that standardization must come with state funding to support the change. Sincerely, Signed by: Robert R. Moreau Chief of Police RRM/JD/gb R-13 Blank page inserted for reproduction purposes only. R-14 Agency’s response to the report provided as text only: City of Huntington Beach Police Department 2000 Main Street Huntington Beach, California 92648 December 1, 1998 Kurt Sjoberg, State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Mr. Sjoberg: We have received the draft copy of the audit report titled “Forensic Laboratories: Many Face Challenges Beyond Accreditation to Assure the Highest Quality Services”. We find ourselves in agreement with your assessment of many of the needs of crime laboratories. It is indeed important to pursue accreditation by the American Society of Crime Laboratory Directors (ASCLD). We are currently in the process of working toward meeting the standards and seeking accreditation. As you have stated, ASCLD stresses that quality is one of the most important issues within crime labs. We highly agree and feel that quality is not a singular job for one individual but rather an approach to doing business. Funding to improve and refine quality assurance programs would be very beneficial. Our situation does exist with outdated equipment. We have had a number of fiscal constraints within the last 5-10 years. These issues are due to occurrences outside of our control, but which have had adverse effects on our ability to fund the laboratory in a manner we would like. This is typified in your example of our gas chromatorgraph/mass spectrometer which was requested to be replaced five years ago. You have mentioned in the report that laboratories have not looked at cost effectiveness. You must realize that the needs of our Department go beyond the laboratory as a Unit. We consider the needs of officers and detectives in pursuing and prosecuting individuals as quickly as possible. This process means that the cost of laboratory service is second to achieving a level of service to our community that reflects cost savings in officer/detective time or soft savings which can be difficult to measure. In addition to soft savings of investigative time, there may be other savings due to defendants pleading in or before court. Additionally, we use our laboratory to assist in an investigative function that so many larger laboratories can not do because of their caseloads. Our community and citizens want and deserve a level of commitment that only we can provide expediently. R-15 The use of DNA technology is considered in the audit report. Obviously. we are one of the smaller laboratories intending to use DNA technology. DNA testing is rapidly replacing other blood typing techniques. In so doing, it places us in a position to use it for investigative and prosecutorial functions. In order to do this, we are adding dedicated space to our existing crime laboratory. The equipment costs are to come from a consortium grant process with other state laboratories. While it has been indicated that DNA equipment is expensive, it is no more expensive than other laboratory equipment. The grant is to provide funds to all California laboratories to bring them to a level of producing DNA analyses. The consumable costs, while somewhat more expensive, will be maintained within the laboratory’s budget. You state that there may be an insufficient caseload for us to pursue DNA analysis in our setting. You should note, that it is our opinion that a sufficient level of casework is that which justifies service to be done where and when our agency decides it has a need and wants an expedient response. This need is not dictated by cost or complexity, but by ability and desire. You should note that with the advent of Short Tandum Repeat (STR) DNA technology and further testing advances, the complexity of DNA analysis decreases thus making it simpler for the analyst. We take exception to your analysis of the need to have a 60 case minimum annual level to 1* warrant having DNA testing within the laboratory. In the draft audit document you use the number of 60 cases, then you also state that 25 cases is a minimum. These numbers are obviously in conflict. In choosing a number of cases, it does not take into account the actual number of samples analyzed. Specifically, in one recent homicide case, we submitted 55 samples. You imply that quality assurance and quality control are somehow related to the number of cases that an individual analyses. In reality, it is the individual’s training, ability, quality control and proficiency testing that insures correct results. Previously, in reporting our needs for DNA testing, we estimated an annual caseload of 25. Recently, we have had an increase which would upgrade our estimate to 44 cases annually. Beyond that, we would foresee an upswing in usage as the courts begin to increasingly rely on DNA evidence in addition to us receiving additional requests from detectives for investigative analyses. Some laboratories try to limit the number of samples analyzed due to backlog or cost constraints. By so doing, incomplete information may result while additional analyses can insure people are not wrongly convicted and assuredly convict the true offenders. If we do not analyze cases completely and for investigative purposes, we are doing a disservice to law enforcement, the courts and the community as a whole. This all takes us back to stating that the need is justified by our ability and desire and not the number of cases. *California State Auditor’s comment on this response begins on page R-19. R-16 In the report, our laboratory is used as an example of space needs with an indication that we should not do DNA analyses but rather use our funds for other space that is lacking. While it may be true that we have space needs, it is also true that we would not compromise any evidence or analysis. If we feel that we cannot competently analyze an item of evidence, we will be the first to admit it. In conclusion, we agree that there are many needs within crime laboratories. Funding has been less than adequate in a number circumstances. We have however, disagreed with some of the material presented in the report. It can be seen from your report, that there are varying needs within laboratories. A plan to simply disperse available funds by the population served would best serve the law enforcement community as a whole. Such a plan would allow the individual laboratories to determine how best to use the supplemental funds. We do laud the effort and appreciate any possibility to increase the funds available for laboratories especially the thought of these funds being supplemental to existing budgets. Sincerely, Signed by: _______________________ RONALD E. LOWENBERG Chief of Police R-17 Blank page inserted for reproduction purposes only. R-18 COMMENT California State Auditor’s Comment on the Response From the Huntington Beach Police T o provide clarity and perspective, we are commenting on the Huntington Beach Police’s response to our audit report. The number corresponds to the number we have placed in the response. 1 The numbers are not in conflict. Rather, as stated on pages 33 through 35, 25 cases provide minimum assurance that the analyst maintains critical skills. On the other hand, the sug- gested level of 60 annual cases reflects both the need to main- tain the analyst’s skills and maximize the use of the DNA facili- ties and equipment. R-19 Blank page inserted for reproduction purposes only. R-20 Agency’s response to the report provided as text only: Office of the District Attorney County of Kern Forensic Science Division 1431 L Street Bakersfield, California 93301 December 2, 1998 Elaine Howle Audit Principal Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: The following is in response to the draft audit report dated December 1998 received by our office on November 24, 1998. CHAPTER 1: I agree with the general philosophy presented in chapter 1 regarding quality assurance, proficiency testing and courtroom monitoring. However, the following issues need additional comment. 1. The cost of ASCLD/LAB accreditation 1* The cost of accreditation cannot be measured simply as the direct costs of inspection or the cost of proficiency testing. The critical costs for a laboratory such as our will are in additional overhead to our staff. The overhead cost associated with obtaining and maintaining accreditation has been estimated as 25 to 30 percent of staff time. Even if the quality assurance manager is funded by the state, the loss of caseload productivity for our staff will still be significant. We would soon be in the position of choosing between casework and quality assurance. 2. Internal proficiency tests 2 The suggestion that a laboratory can develop proficiency tests internally is not realistic for a laboratory of our size. While drug proficiency tests can be devised, the broad range of cases found in physical evidence would be beyond our resources. 3. Quality control manager How is the performance of the quality control manager measured? *California State Auditor’s comments on this response begin on page R-25. R-21 CHAPTER 2: The suggestions presented in Chapter 2 have merit for all 19 laboratories. Adequate space, equipment, and information management system are all desirable and necessary goals. However, the concept of “optimal performance beyond accreditation” is troubling. If the end result or goal of this exercise is to obtain a quality product from the crime lab and the standards of ASCLD/LAB do not provide sufficient criteria, how do we determine when we have succeeded? This concept leaves me with the impression of a “moving target,” a target our laboratory cannot measure. The following issues need additional comment. 1. Space 3 The space issue will be addressed in our new facility. The only issue here is to correct/update the contents of Table 1. The total number of staff is currently 21. The total size of the laboratory is 21,512 square feet of developed area. If the undeveloped area is included, the total area becomes 24,390 square feet. This calculates as 1,024 square feet per staff member. 2. Equipment Your report appears to equate “out dated” equipment with inaccuraciesand poor quality results. I think old or out dated equipment affects efficiency much more than accuracy. With a proper quality control program, this equipment can produce accurate results, although perhaps not as quickly. Replacement of old equipment is a difficult issue for our laboratory. We generally use an instrument until it is no longer supported and the parts are not available. I agree that a plan for replacement is desirable, however, funding has always been much more likely when the situation is critical. A stable source of funding is obviously needed to implement such a plan. 3. Training: The amount of funding dedicated to training is largely dependent upon the maturity of the staff or the development of a new program such as DNA. A neophyte criminalist simply requires more training than an experienced one. Since our salary structure has never attracted experienced personnel, it has been necessary for our laboratory 4 to develop staff. The $1,000 suggested for each technical staff is insufficient for an inexperienced staff or for preparing for a new program such as DNA. This equates to approximately one and one-half “essentially free CCI classes” per year. CHAPTER 3 There are several issues in this chapter that require comment. 1. Laboratories can neither assure their services are cost effective nor identify appropriate alternatives: This section seems to indicate a sense of frustration on the part of the audit team. I find it difficult to believe that laboratories have not analyzed their services to some degree. Services in our laboratory are generally provided in response to a perceived need in the law R-22 enforcement community. We avoid new services until we believe the demand justifies the addition. Unusual cases are generally out-sourced. Some individual services such as document examinations and DNA are also out-sourced. The funding for out-sourcing services comes partially or completely from the requesting agency depending on their resources. There are no local laboratories that provide the same range of services. However, alternative sources for toxicology samples have been examined. Some of the “cost effective” toxicology solutions used by one local agency have resulted in a significantly lower quality of service and a reexamination of the samples by our laboratory. The services provided by our laboratory are not simply a matter of analyzing or comparing an item of evidence. We have a unique working relationship with the officers and their departments. This relationship is an integral part of our ability to collect evidence and to provide information to the criminal justice system. Cost effectiveness is only one part of the equation for providing the tax payer with value for their dollar. 2. Implementation of DNA 5 The report indicates that our laboratory should not implement DNA testing because it is not cost effective. The basis of this conclusion was based on information from the audit consultants. The parameters given in the report for minimum caseloads were sixty cases per analyst per year or five to ten cases per month. It was further recommenced that the guidelines for allotting funds within the DNA Consortium be reexamined. The purpose of the reexamination would be to redistribute (deny Kern County) federal DNA funding. The funding would be redirected to “regional laboratories.” The issue of cost effectiveness of DNA testing for our laboratory disappears when all the facts are examined. The information presented in Table 3 was for law enforcement (forensic) casework only and should be revised to include the paternity DNA analysis we will provide for our Family Support Division. Anticipated DNA cases should also be updated to include our latest caseload. After correction and updated statistics, the corrected table should read: Laboratory Number of cases Number of Staff Cases/staff anticipated annually Forensic DNA 200 (by year 2000) Paternity 400 (present load 600 3 - 4 200 - 150 In addition, based on the experience of our technical leader, the serology program will be less expensive to operate using DNA techniques than our existing conventional genetic marker serology. This conclusion is based on the cost of personnel and assumes the DNA program is setup and running. R-23 The position suggesting denying federal funding for Kern County is unacceptable for several reasons. · Kern County has invested thousands of local tax dollars in training DNA analysts. · Kern County has invested in an area for DNA analysis in our new facility. · Kern County has hired analysts, including a technical leader. · Kern County has worked within the consortium guidelines, awaiting its turn for funding and foregoing any attempt to obtain grant money on its own. · It would be unfair to change the guidelines at this late date A great deal of emphasis was placed on implementing DNA only in regional laboratories. Additional emphasis was place on centralizing some services such as trace evidence and other low volume services. I believe our isolated location at the southern end of the San Joaquin Valley makes centralization of services difficult for our region. If consideration is also given to utilization of laboratory services, it is logical to consider Kern County a region. I don’t have the citation and perhaps it is an urban legend, but I believe that it has been documented that the further the lab is from the requesting agency, the smaller the number of cases submitted. Removing services from our laboratory to a central location would certainly extend our resources because many cases simply would not be submitted. This has certainly been our experience with the eastern portion of Kern County. In conclusion, the draft report presents many interesting issues and highlights the difficulties in managing a crime laboratory in our present environment. All of this effort will be wasted, if funds do not become available and we continue to do business as usual. Sincerely, Signed by: Vernon L. Kyle Chief Criminalist R-24 COMMENTS California State Auditor’s Comments on the Response From the Kern County District Attorney T o provide clarity and perspective, we are commenting on the Kern County District Attorney’s response to our audit report. The numbers correspond to the numbers we have placed in the response. 1 On page 17, we acknowledge that laboratories will have other accreditation costs, such as staff time and costs to provide manuals and records. 2 Contrary to the laboratory director’s assertion, we do not suggest that laboratories internally develop proficiency tests. We discuss, on pages 15 and 16, both options of purchasing external tests or developing in-house tests. 3 Table modified. 4 As stated on page 30, the suggested amount of $1,000 per staff member represents the minimum amount laboratories should budget for training. 5 Our report does not state that the Kern County District Attorney should not implement testing, nor does it suggest denying funds for Kern County. In contrast, on page 35, it states that prior to distributing additional funding, the State and local laboratories should consider whether other options, such as regionalizing DNA services, would be more cost-effective. R-25 Blank page inserted for reproduction purposes only. R-26 Agency’s response to the report provided as text only: Los Angeles Police Department P.O. Box 30158 Los Angeles, California 30030 (213) 485-3202 Ref. # 9.6 GF #98-11-31 December 1, 1998 Mr. Kurt R. Sjoberg Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Mr. Sjoberg: After thorough review and careful consideration of your report entitled "Forensic Laboratories: Many Face Challenges Beyond Accreditation to Assure the Highest Quality Services," I fully concur with the majority of points detailed in the audit. However, there are a few issues that I would like to bring to your attention to ensure clarity and request that they be included in the final audit report. 1* One of the issues that must be noted is that the audit was conducted on only the portion of Scientific Investigation Division (SID) that is actively seeking accreditation. Thus, half of SID, including the Latent Print Section with a staff of 81, was not included in the audit. Since the Latent Print Section will eventually be accredited by the American Society of Crime Laboratory Directors/Laboratory Accreditation Board, a reference to this effect would be appropriate. It would save confusion as requests to acquire funding, equipment, facilities, and personnel are generated. The other issue is the numerous references throughout the report recommending regionalization of forensic services. For the laboratories located in Los Angeles County, this is not a new concept. Consolidation of the forensic laboratories of the Los Angeles Police Department (LAPD) and the Los Angeles County Sheriff's Department (LASD) has been studied numerous times by various agencies and groups over the past 20 years. Though the concept of consolidation may make sense when considering small agencies with limited caseloads, personnel, and equipment, this is not the case in Los Angeles County. Both the LASD and LAPD's crime laboratories offer a full range of services and individually support large agencies requiring the services of a dedicated forensic laboratory. The laboratories are of similar size and serve a comparable population. The only viable recommendation offered by the numerous studies is the concept of co-locating in a joint facility, sharing only certain space and equipment that would not be in continuous use by any one agency. This concept is currently being studied and, if implemented, would provide both entities with much needed space while maintaining the autonomy and responsiveness the LAPD deserves of its forensic services. R-27 *California State Auditor’s comment on this response begins on page R-29. Mr. Kurt R. Sjoberg Page 2 9.6 Per your request, this letter is being faxed with a hard copy and your disk to follow by overnight express. Laboratory Director Michele Kestler is available at (213) 237-0044 to answer any questions regarding this matter. Very truly yours, Signed by: BERNARD C. PARKS Chief of Police Enclosure R-28 COMMENT California State Auditor’s Comment on the Response From the Los Angeles Police T o provide clarity and perspective, we are commenting on the Los Angeles Police’s response to our audit report. The number corresponds to the number we have placed in the response. 1 We excluded the latent print section because the laboratory chose not to include the section in its recent review for accredi- tation. This is consistent with our treatment of the other labora- tories in the report. R-29 Blank page inserted for reproduction purposes only. R-30 Agency’s response to the report provided as text only: County of Los Angeles Sheriff’s Department Headquarters 4700 Ramona Boulevard Monterey Park, California 91754-2169 November 30, 1998 Kurt R. Sjoberg, State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Mr. Sjoberg: We have reviewed your draft report, entitled Forensic Laboratories: Many Face Challenges Beyond Accreditation to Assure the Highest Quality of Services and agree with your conclusions and recommendations. Through an aggressive quality assurance program and ASCLD/LAB accreditation, the Los Angeles County Sheriff's Crime Laboratory has worked to minimize the possibility of cross contamination of physical evidence due to overcrowded working conditions. We believe that our quality systems address these concerns. California forensic science laboratories face many challenges. As your report points out, adequate resources are sorely needed by our public crime laboratories to address these challenges in order to deliver a quality product to the criminal justice system. Your report neglects to recognize and commend the hundreds of hard working, dedicated forensic scientists throughout the State who work under less than ideal conditions. These professionals nonetheless manage to discharge their responsibilities in an exemplary manner and deserve our appreciation. Sincerely, Signed by: JERRY L. HARPER, UNDERSHERIFF R-31 Blank page inserted for reproduction purposes only. R-32 Agency’s response to the report provided as text only: City of Oakland Police Department 455 7th Street Oakland, California 94607-3985 December 1, 1998 Mr. Kurt R. Sjoberg California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Mr. Sjoberg: RE: Request for Comments on Draft Report of Forensic Laboratories Thank you for your letter of November 23, 1998 offering our department the opportunity to review and comment on the draft report concerning the audit of local forensic laboratories. I have asked Mary Gibbons, Manager of the Criminalistics Division, to review the document and prepare comments as necessary. Her responses are attached for your consideration. Ms. Gibbons has had several telephone conversations with Ms. Ginny Johnson of your office concerning our laboratory square footage data and current staffing allocations. Ms. Johnson has indicated that corrections to those figures will be made, as discussed, and do not require written comment herein. I look forward to receiving the final report and hope that it will be helpful in securing much needed support for all local forensic science laboratories in our state. If you have any questions, please contact Ms. Gibbons at (510) 238-2108. Sincerely, Signed by: Joseph Samuels, Jr. Chief of Police attachments (1) R-33 Draft Report of Audit of Forensic Laboratories Comments of the Oakland Police Department 1* (1) Page 33, paragraph 1 and 2: The consultants recommended the replacement of GC/ MS and FTIR instruments for several laboratories. Oakland was not included in these groups of labs. As I cannot tell the basis for inclusion or exclusion, I request consideration of the following information in order to determine whether we should have been included. Both instruments were purchased in 1989, making them approximately 10 years old—old by technical equipment standards. The data station for the FTIR is obsolete now, and will have to be replaced soon as it will no longer be supported by the company. The FTIR optical bench will be supported for another two years and then will require replacement. The new bench will not be compatible with a data station replaced in 1999. We have no back-up GC/MS or FTIR. In addition, we do not have a GC/MS dedicated to fire debris analysis. The GC we have is not Y2K compliant, is over 10 years old and is considered obsolete by the manufacturer. 2 (2) Page 39, paragraph 1 and 2: the statement that “ASCLD/LAB does not require training programs for accreditation” is not quite accurate. ASCLD/LAB requires those laboratories with forensic DNA programs to comply with TWGDAM Guidelines. Further, they are poised to require our compliance with the DNA Advisory Board Standards on Quality Assurance for Forensic DNA Laboratories. Both standards require a documented continuing education program. In addition, all accredited labs must document training received prior to the assumption of casework duties. Often, the training required is of a nature that it can only be received from sources external to the laboratory. Finally, any criminalist who is certified by the American Board of Criminalistics must demonstrate continuing education to maintain their certification. (3) While OPD was not specifically mentioned in the following sections, the opinions presented therein regarding caseload measures and DNA instrument throughput and efficiency affect all laboratories. In that spirit, I offer the following comments. Page 47, paragraph 1: Average caseloads for the DNA analyst. In the opinion of the consultants, one DNA analyst should be able to complete an average of 5-10 cases per month (60-120 cases per year). What is absent from the draft report is what constitutes a “case”. Completion of 5-10 cases per month per analyst would overstate reasonable expectations if by “case” one means evidence assessment, screening of items for body fluids, identification of body fluids, selection of stains for DNA analysis, extraction, amplification, typing, interpretation and report writing. These are the component parts of a full forensic biology case from start to finish. The numbers proposed may be realistic if a case is considered to begin at DNA extraction, and accounts only for the DNA aspects of the analysis. I suspect that most laboratories conducting DNA analysis in this state do not separate the forensic biology aspects of the exam from the forensic DNA aspects, but rather view DNA analysis as a tool to be *California State Auditor’s comments on this response begin on page R-37. R-34 used within the context of a forensic case. In any event, it is critical that the conditions upon which the caseload expectation proffered are predicated and the references upon which the consultants rely for this information are available and understood so that they can be properly applied. Page 49, paragraph 1: Efficient Utilization of DNA stations: The report states that their consultants believe that efficient use of a DNA station requires four analysts and 260 cases each year. The report should contain suitable references for this information, as well. 3 (4) General comment about the needs of forensic laboratories: I am concerned that this audit has focused too narrowly on what is needed for all labs in the state to become accredited. While accreditation is a laudable goal, it can in no way be assumed that, once achieved, the needs of forensic laboratories shall have been addressed. They shall not have been addressed. Two important factors have not been considered in the report: backlog of current cases and the consequences of the emergence of forensic databases. Backlogged Cases: Our reason for existing is to provide information to the criminal justice community in a timely fashion to assist in solving crimes, identify the person or persons responsible, and exculpate the falsely accused. Backlogged cases mean that we are not providing information in a timely fashion. Accreditation status will not change that fact. Backlogged cases mean we do not have sufficient personnel resources to address our casework needs. Accreditation status will not change that fact. I had expected that the report would provide information on our collective backlogs and on the personnel resources needed to address those cases, because timely service delivery really what is at the heart of this matter. To the extent that it has not done so, our needs have not been fully elucidated. It may be the case that different laboratories count units of work differently, impeding the auditor’s ability to compare one laboratory to another. However, the mere fact that laboratories have significant backlogs (by whatever counting system they employ) is evidence that we do not have the resources necessary to meet the demand for our services. Any assessment of forensic laboratory needs must take this fact into account. Forensic Databases: The emergence of forensic databases, such as fingerprint, DNA, and firearms identification databases, means that crime laboratories are now in a position to provide leads to investigators in cases where no leads would otherwise exist. By their very existence, these databases give rise to an entirely new constellation of casework. We are now in the position of potentially providing leads on every case with computer quality latent prints. On unsolved sexual assault and rape/homicide cases where semen evidence exists, we may be able to identify the potential assailant through DNA typing and challenging of state and national offender DNA databases. This technology is applicable to old, unsolved cases and current cases, equally. These are cases which we would not have been able to work heretofore and, as such, they represent a significant increase in our effective caseload and backlog. Most of our laboratories are not staffed to integrate these R-35 cases into their existing caseloads. Yet we must do so because that is the future of forensic science work. Any assessment of forensic laboratory needs must take this sea change into account, understand its effects, and consider the resources required to meet this challenge to the fullest extent. R-36 COMMENTS California State Auditor’s Comments on the Response From the Oakland Police T o provide clarity and perspective, we are commenting on the Oakland Police’s response to our audit report. The numbers correspond to the numbers we have placed in the response. 1 Our presentation is intended to provide the reader insight into laboratories’ equipment needs. As stated on page 25, these instruments are only some examples of outmoded equipment that should be replaced. Each laboratory would need to inven- tory its particular needs and develop schedules for replacing or upgrading equipment. 2 Text modified. 3 The audit report not only discusses what is needed for accredita- tion, it also discusses ways to improve efficiency and effective- ness. Furthermore, while we attempted to evaluate workload statistics, including backlog data, as stated on page 37, the workload data was not always available, nor, when available, was it standardized or consistent. R-37 Blank page inserted for reproduction purposes only. R-38 Agency’s response to the report provided as text only: Office of the District Attorney Sacramento County 901 G Street Sacramento, California 95812-0749 December 2, 1998 Kurt R. Sjoberg State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Mr. Sjoberg: Thank you for the opportunity to review and respond to the draft copy of the report “Forensic Laboratories: Many Face Challenges Beyond Accreditation to Assure the Highest Quality Services,” prior to public distribution. Please convey my compliments to the staff of state auditors who conducted the needs assessment of the state’s nineteen local crime laboratories. They were tasked with a very difficult assignment. I also appreciate the opportunity afforded to Sacramento County to provide your auditors and consultants with the crime laboratory facility and staff in support of the auditor’s training early on in the process. Our review revealed an area of concern that I am sure is shared by many of the agency heads that received your report. Specifically, relating to Chapter 3 (Laboratories Need to Evaluate the Effectiveness and Efficiency of Their Services) and the section of the chapter entitled Some Laboratories Plan to Implement DNA Testing Even Though it May Not Be Cost Effective to Do So. The commentary makes direct reference to my agency’s forensic DNA program. Our crime laboratory established the forensic DNA program in December 1997 and our analyst has since analyzed twenty DNA cases and has testified six times on her analytical results. I feel very strongly that the decision to offer a specific forensic service, in this instance forensic DNA analysis, is the prerogative of the local jurisdiction, and is based on the investigative and prosecutorial needs identified by the local police agencies and District Attorney. I am also concerned by a recommendation from your consultants regarding an average DNA 1* caseload per analyst per month. This recommendation is presented without reference as to the source of the data. Since it is on this basis that you conclude that “the anticipated caseloads and staffing levels of seven laboratories (which includes Sacramento County) planning to implement DNA testing do not indicate they will use their DNA facilities and equipment at these levels,” I believe it is imperative that there is a reference as to this rather implausible recommendation. I doubt that serious consideration was given by your consultants to the impact on analyst productivity due to quality control and quality assurance requirements, analyst proficiency testing, the annual commitment to analyst training (as described in the DNA Advisory Board standard 5.1.3.1), casework review, participation in statewide offender database. *California State Auditor’s comments on this response begin on page R-41. R-39 1 programs, and the extensive courtroom testimony associated with forensic DNA cases. I also believe your consultants failed to take into account the considerable amount of work associated with screening evidentiary items and cutting suspected stains, work that must be performed prior to any DNA analysis. Each of these factors directly affects the number of DNA cases that can be safely and accurately analyzed in any crime laboratory environment. Our crime laboratory received over 468 sexual assault cases for examination in calendar year 1997. Of those 468 cases, 317 were submitted without an identified suspect, while the remaining 151 cases had identified a suspect. Based on previous historical trends, approximately 328 of these sexual assault cases (70%) have potential serological evidence that can benefit from forensic DNA analysis. Are these victims and their cases to fall by the wayside because our forensic DNA program does not meet the questionable recommendation of your consultants? I hardly think so. What these statistics do point out is that my agency’s crime laboratory has a desperate need for additional staff to meet the caseload demand. There are substantial costs to local jurisdictions that are forced to send their evidence to private laboratories for DNA analysis. In fiscal year 1997, my office relied on the services of private DNA laboratories to analyze seven DNA cases for a cost of $29,000 of laboratory fees and $9,000 of expert witness fees. I believe that this money is better spent in support of my crime laboratory’s forensic DNA program. 2 In light of these facts, I hope that some consideration is given to tempering the conclusion that some local crime laboratories should not develop forensic DNA programs. Further, I 1 suggest that an effort be made by your office to seek more information regarding forensic DNA programs, even to the extent of conducting a national enquiry before publishing unsubstantiated recommendations. Our local crime laboratories are an integral component of the criminal justice system. Their work is absolutely essential to the timely and successful adjudication of criminal cases. This conclusion regarding forensic DNA programs could seriously affect a local crime laboratory’s ability to provide DNA services and dampen future efforts to seek program funding. Sincerely, Signed by: JAN SCULLY DISTRICT ATTORNEY JS/bj R-40 COMMENTS California State Auditor’s Comments on the Response From the Sacramento County District Attorney T o provide clarity and perspective, we are commenting on the Sacramento County District Attorney’s response to our audit report. The numbers correspond to the numbers we have placed in the response. 1 Our conclusion is based on the expertise of our consultants, who are trained ASCLD/LAB inspectors and former laboratory direc- tors. Furthermore, one was director of a large laboratory that included DNA analysis in its caseload. This consultant has also reviewed the work of 13 DNA laboratories in the last 12 months, excluding the California laboratories. The other consultant was director of a smaller laboratory that did not do DNA analysis. However, he did perform a cost analysis and determined that the break-even caseload was a minimum of 80 samples per year. Since his laboratory’s anticipated caseload was less than 50, he chose to contract out the work. In addition, published data in the ASCLD/LAB workload surveys show a mid-range of about 5 cases per month. Finally, our consultants sought the advice of the manager of a large state system with a long history of DNA analysis at several sites. His advice was that the system would expect on the order of 12 cases per analyst per month. 2 Our conclusion does not state that local crime laboratories should not develop forensic DNA programs. Rather, we suggest that prior to distributing additional federal funds, the state and local laboratories should consider whether other options, such as regionalizing DNA services, would be more cost-effective. R-41 Blank page inserted for reproduction purposes only. R-42 Agency’s response to the report provided as text only: The City of San Diego Office of the Chief of Police 1401 Broadway San Diego, California 92101-5729 December 3, 1998 Bureau of State Audits Elaine Howle, Audit Principal 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Ms. Howle: The following responses are made to only those sections where recommendations are specifically addressed to the San Diego Police Department. The recommendations are summarized and lab responses to the audit are provided below the recommendations in small, bold print. CHAPTER 1 Quality Control Managers and Staff Are an Essential Element of A Quality Control System Suggested quality control staffing levels: 40-70 staff, appoint a full time quality control manager and a clerk (p 17). The lab currently has a budgeted staff of 64. The Laboratory has a designated, full time Quality Assurance Manager but there is no dedicated clerical support for this position. The Quality Assurance Manager is also responsible for the Laboratory safety and training programs. CHAPTER 2 Most Laboratories Do Not Meet the Recommended Standard of 1,000 Square Feet Per Staff Member Twelve [includes SDPD] of the 19 laboratories do not provide at least 700 square feet per staff and 15 [includes SDPD] of 19 laboratories do not provide approximately 1,000 square feet per staff, the literature standard (p 27-29). R-43 Laboratory Total Number of Current Facility Square Footage Per Staff Square Footage Staff Member San Diego Police 59 22,000 373 Our current staff of 64 provides 362 square feet per member under present conditions. This includes approximately 1200 sq ft of recently acquired space on the first floor. This area will house the alcohol analysis area and provide office space for 4 criminalists. A portion of the current alcohol analysis area on the main lab floor will be converted to a small library and conference room. The main headquarters building at San Diego Police Department is approximately 10 years old. The staff including the Forensic Science Section has outgrown the available space. Immediate plans call for a space utilization study to be conducted by local university students in anticipation of relocation of the Central Patrol function to another facility. It is anticipated that the Laboratory will acquire additional space at that time. Improved Management Information Systems Could Increase Staff Efficiencies and Allow for More Effective Management of Operations At least nine [including SDPD] of the laboratories manually track limited forensic information (p 36). The cost of a management information system, including hardware and software for a large laboratory, such as the San Diego Police Laboratory may run between $300,000 and $400,000 (p 38). The San Diego Police Department is in the process of installing a Department wide Management Information System at a cost of approximately $15 million. The Laboratory will be included as part of this system and approximately $300,000 has been designated for a laboratory information management system including appropriate hardware and software. The Laboratory has established a committee presently to determine the specific needs. The system is expected to be up and operational by the end of 1999. Most Laboratories Lack a Documented Staff Training Program The following table summarizes a reasonable budget allocation (p41). Laboratory Total Number of Fiscal Year 97/98 Training Budget Technical Staff Training Budget At $1,000 Per Technical Staff San Diego Police 31 $6,000 $31,000 A training budget of $40,000 was established in 1998 for laboratory personnel. This amount is in addition to POST reimbursable training classes. R-44 CHAPTER 3 Some Laboratories Have Considered Regionalizing Services Results of a study to assess the county’s forensic science needs concluded that merging the San Diego Police Department [with SDSO] would create a more efficient and cost- effective regional unit by reducing duplication and consolidating equipment, space, and supplies (p 50). Discussions of a merger between the San Diego Police Department and the San Diego County Sheriff’s Department are on-going. Our agency is not willing to compromise the level of quality service that we now provide. Any merger that would take place prior to the San Diego Sheriff’s Department reaching parity with our agency would be counter productive. Workload Data Is Not Always Available, Standardized, and Consistent Eight [including SDPD] laboratories did not maintain the number of cases received, six did not maintain the number completed, and eight [ including SDPD] did not maintain backlog data (p 52, 53). With the initiation of a laboratory information management system next year, the Laboratory will be able to efficiently track casework input and output. For the past two calendar years, monthly and annual stats have been manually prepared showing the number of cases received in each work unit. A casework backlog report has recently been instituted by the Acting Lab Director which provides backlog information in all areas within the Laboratory on a monthly basis. Signed by: GLENN BREITENSTEIN Acting Crime Lab Manager kk R-45 Blank page inserted for reproduction purposes only. R-46 Agency’s response to the report provided as text only: San Diego County Sheriff’s Office Post Office Box 429000 San Diego, California 92142-9000 December 1, 1998 Kurt R. Sjoberg, State Auditor 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Re: Response to Forensic Laboratory Audit Dear Sir, This comment is offered in response to the audit report of December, 1998 regarding "Forensic Laboratories". Chapter 2, Table 1 sites that the San Diego Sheriff's Crime Lab has 47 total staff. For clarification, the laboratory has 42 total authorized full time paid positions. The other personnel that assist the laboratory are temporary 120 day rehires and unpaid intern/ volunteers and those numbers may fluctuate occasionally. However, for the purposes of calculating required square footage per working individual the inclusion of temporary and/or part-time personnel appears appropriate. Sincerely, WILLIAM B. KOLENDER, SHERIFF Signed by: Ronald E. Barry, Manager Sheriff's Crime Laboratory REB/reb R-47 Blank page inserted for reproduction purposes only. R-48 Agency’s response to the report provided as text only: City and County of San Francisco Police Department Thomas J. Cahill Hall of Justice 850 Bryant Street San Francisco, California 94103 November 30, 1998 Kurt Sjoberg State Auditor 555 Capital Mall, Suite 300 Sacramento, California 95814 Dear Mr. Sjoberg: I am writing in response to the draft copy of your report (“Forensic Laboratories…”). I have only one minor editorial correction before responding to portions of the report that refer to the SFPD Crime Lab: Page.15, paragraph 2: “Our review revealed…because they lack a quality control system” should read: “Our review revealed…because they lack a quality system” (per inset box) Regarding the SFPD Crime Lab: Today I communicated most of these comments to Virginia Johnson of your office: 1. Page 29, TABLE 1: The most significant error as reported pertaining to the SFPD Crime 1* Lab is in this table, where the Current Facility Square Footage is listed as 13,000. Our existing facility has 5,250 square feet for 16 staff (328 square footage per staff member). I believe the auditors may have confused our new lab plan with our existing lab. The designated new lab facility has 13,400 square feet. The City and County of SF has leased a building from the Navy that must be retrofitted to meet the needs of our laboratory. Funding for the retrofit is still not available. On p. 27, we should be included in the last line for Table 1 (change to “13 of the 19 laboratories do not provide at least 700 square feet per staff”). 2. Page 39, regarding training funds: This operation is managed by our Police Academy. The training budget for the entire department (sworn and civilian) is shared by all Divisions in the Department. We get about $5,000 per year. 1 3. Page 45, DNA Consortium: There needs to be a date clarification in this section. The reader is left with the impression that the California DNA Consortium received a 5-year grant starting in 1994 when in fact, DOJ applied for the grant in 1995 and the first money was received in 1997. *California State Auditor’s comment on this response begins on page R-51. R-49 1 4. Page 46, DNA Consortium: The second paragraph should state the date of the initial application of the California DNA Consortium as 1995 so it is clearer when the 17 labs implemented or planned to implement DNA testing. We began implementation of our DNA program in March of 1997. 5. Page 47, TABLE 3: This table lists the number of cases we anticipate annually as 180. This number is conservative because of our inadequate staffing level for this operation. We would anticipate at least 250 cases per year if we had adequate staff to provide a meaningful response time to our users. 6. Page 49, DNA funds received: The statements here are accurate but perhaps a bit misleading as the amount we received this year ($40,000) is only a small portion of our operating costs. We would benefit much more from the Consortium if we could receive another DNA testing unit (such as our AB 310 Genetic Analyzer). Perhaps the sentence could read “During the past three years, six of the eight laboratories in the consortium have received some DNA funds for equipment and other supplies.” Thank you and your staff for all your hard work done on behalf of the local Crime Labs! Sincerely, Signed by: Martha Blake, Senior Criminalist R-50 COMMENT California State Auditor’s Comment on the Response From the San Francisco Police T o provide clarity and perspective, we are commenting on the San Francisco Police’s response to our audit report. The number corresponds to the number we have placed in the response. 1 Per discussion with the laboratory director, we modified our tables and text. R-51 Blank page inserted for reproduction purposes only. R-52 Agency’s response to the report provided as text only: Ventura County Sheriff’s Department 800 South Victoria Avenue Ventura, California 93009 Phone (805) 654-2380 December 3, 1998 Kurt R. Sjoberg, State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Mr. Sjoberg, First, let me thank you for the opportunity to review and comment on the draft report of the audit conducted by your office of local crime laboratories, prior to it being made public. Secondly, it is apparent that the audit was a noble and well-intended effort on the part of your office to provide the legislature with a realistic needs assessment of local laboratories 1* as it relates to ASCLD accreditation. However, after reviewing the report, I have serious concerns about how local laboratories not currently accredited are depicted as being somehow less reliable than those that are accredited. Such an inference can create opportunities for unwarranted challenges to the integrity and accuracy of the analytical results of these laboratories. While I am sure it was not intentional, the tone and manner in which the report is currently written may ultimately cause more grief than benefit for the local laboratories. Attached is our agency’s response to the draft report. I believe that these comments more fairly and accurately reflect the situation in our laboratory, as well as that of other local crime laboratories. If you have any questions or require further explanation of these comments, please refer your staff to Forensic Manager Renee Artman (805) 662-6878 or Captain Leslie Warren (805) 654-2332. Sincerely, Signed by: BOB BROOKS Sheriff *California State Auditor’s comments on this response begin on page R-57. R-53 COMMENTS ON STATE AUDIT REPORT - "Forensic Laboratories: Many Face Challenges Beyond Accreditation to Assure the Highest Quality Services" After reviewing the draft copy of "Forensic Laboratories: Many Face Challenges Beyond Accreditation to Assure the Highest Quality Services" report, we would like to make the following comments. 1 In several places, the report makes reference to the unaccredited laboratories' inability to affirm the results of their work are accurate, impartial, and relevant. This somewhat inflammatory language suggests that if a lab is not accredited then your results are automatically invalid. These comments could cause unwarranted concern from the legal defense community over the validity of the results from an unaccredited laboratory. On Page 15, you state that "Our review revealed that most of the local laboratories did not fully meet some of ASCLD/LAB primary accreditation standards because they lack a quality control system." This is a somewhat misleading statement. If a lab's quality control system differs from the recommended standards established by ASCLD, then your review contends that laboratory does not have any quality control system. The ASCLD manual states that the principle of a quality system is: "To enhance the validity of results and conclusions reported, a forensic laboratory should establish and maintain a quality system that is appropriate for the range of forensic disciplines as well as the types and numbers of examinations that are conducted. Broadly accepted procedures, equipment and materials must be used and supported by proper case records." We believe it is probably safe to say that all nineteen local laboratories have some form of quality control system which includes: quality control manuals or protocols, written policies and procedures, and quality assurance personnel. Page 19, Paragraph 2, states: "Those laboratories not meeting the proficiency testing standards can neither assure the continued competency of their staff or reliability of their analytical results, nor attain ASCLD/LAB accreditation." This statement assumes there is only one way to assure the competency of their staff. The ASCLD manual states that the principle of proficiency testing is: "Proficiency testing is an integral part of an effective quality assurance program. It is one of many measures used to monitor performance and to identify areas where improvement may be needed." Many labs have proficiency programs in place and are in the process of modifying those programs to meet ASCLD standards. Your statement 2 is very misleading, it assumes that if a lab does not entirely meet ASCLD criteria for proficiency testing, it cannot assure the competency of its staff or reliability of its results. Page 22, Paragraph 1 states: "Furthermore, we noted that 7 of 11 laboratories also lack a quality manager." This statement leads the reader to believe that in these seven labs there is no one at all responsible for quality system. Most laboratories have one or more personnel responsible for different aspects of their quality system. However, there may not be one individual whose sole responsibility is to oversee the entire quality system. Our laboratory does not necessarily agree with the "minimum of 60 cases annually for a laboratory with one DNA analyst" standard. ASCLD standards require at least two qualified DNA analysts per laboratory. In our lab, one of these analysts is also the technical leader and section supervisor and has duties above and beyond the actual R-54 analyses. In addition, turnaround time is critical in some cases. Sending cases out for analysis on such a priority basis is both cost prohibitive and cannot match the turnaround time of an in-house analysis. In conclusion, we appreciate the efforts of both the inspection teams. We understand that some of the statements made are intended to support the need for additional funding of local laboratories. However, we are most concerned that the public who will ultimately review this report understands that while ASCLD accreditation provides a documented, verifiable means to measure and quantify a laboratory's efficiency, it does not necessarily make the lab more efficient or change the validity of the results that are produced by that laboratory. R-55 Blank page inserted for reproduction purposes only. R-56 COMMENTS California State Auditor’s Comments on the Response From the Ventura County Sheriff T o provide clarity and perspective, we are commenting on the Ventura County Sheriff’s response to our audit report. The numbers correspond to the numbers we have placed in the response. 1 Contrary to the Sheriff’s statement, the report does not suggest that if a laboratory is not accredited, its work is less reliable. However, laboratories that attain accreditation from ASCLD/LAB have demonstrated that they meet certain established standards. Moreover, accreditation provides continued confidence that the results of the laboratory’s work are accurate, impartial, and relevant. 2 Our statement merely emphasizes the importance that ASCLD/ LAB places on proficiency testing to measure the capability of a laboratory’s examiners and the reliability of their analytical results. Additionally, it lends credibility to the examiners’ work when they are required to testify in court. R-57 Blank page inserted for reproduction purposes only. R-58