Summary

March 2015 California Department of State Hospitals It Could Increase the Consistency of Its Evaluations of Sex Offenders by Improving Its Assessment Protocol and Training Report 2014-125 COMMITMENT INTEGRITY LEADERSHIP The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the California State Auditor’s Office at the following address: California State Auditor 621 Capitol Mall, Suite 1200 Sacramento, California 95814 916.445.0255 or TTY 916.445.0033 OR This report is also available on our Web site at www.auditor.ca.gov. The California State Auditor is pleased to announce the availability of an online subscription service. For information on how to subscribe, visit our Web site at www.auditor.ca.gov. Alternate format reports available upon request. Permission is granted to reproduce reports. For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. For complaints of state employee misconduct, contact the California State Auditor’s Whistleblower Hotline: 1.800.952.5665. Elaine M. Howle State Auditor Doug Cordiner Chief Deputy March 12, 2015 2014-125 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the California State Auditor presents this audit report concerning the California Department of State Hospitals’ (State Hospitals) Sex Offender Commitment Program (program). The program targets a small but extremely dangerous subset of sexually violent offenders (offenders) who present a continuing threat to society because their diagnosed mental disorders predispose them to engage in sexually violent criminal behavior. State Hospitals evaluates these offenders to determine whether they meet criteria to be considered sexually violent predators (SVPs) and whether courts should consider committing such offenders to a state hospital. Our report concludes that State Hospitals’ evaluations of potential SVPs were inconsistent. Although state law requires that evaluators consider a number of factors about offenders, such as their criminal and psychosexual histories, we noted instances in which evaluators did not consider all relevant information. We noted that gaps in policies, supervision, and training may have contributed to the inconsistent evaluations. Specifically, State Hospitals’ standardized assessment protocol for conducting evaluations of potential SVPs lacks adequate detail and direction for SVP evaluators on how to perform evaluations. Further, State Hospitals’ headquarters lacks a process of supervisory review of evaluators’ work from a clinical perspective. We also noted that State Hospitals has not consistently offered training to its evaluators, and did not provide SVP evaluators with any training between August 2012 and May 2014. Also, State Hospitals could not demonstrate that its evaluators had training on a specific type of instrument used when assessing whether an individual would commit another sexual offense until it began offering such training at the end of 2014. We also noted additional areas in which State Hospitals could improve its evaluation process. Specifically, it has not documented its efforts to verify that its evaluators met the experience portion of the minimum qualifications for their positions. In addition, in March 2013, State Hospitals developed a process for assigning and tracking the workload of its evaluators and recently revised it in January 2015. Although the revised process addresses some concerns about workload assignments, it omits other elements and State Hospitals has not established a formal process for periodically reviewing its workload assignment process. Finally, State Hospitals needs to address its backlog of annual evaluations of currently committed SVPs at Coalinga State Hospital (Coalinga). When Coalinga fails to promptly perform these evaluations, it is not fulfilling one of its critical statutory obligations, leaving the State unable to report on whether the SVPs continue to pose risks to the public and whether unconditional release or release to a less restrictive environment might be an appropriate alternative. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 916.445.0255 916.327.0019 fax www.auditor.ca.gov Blank page inserted for reproduction purposes only. California State Auditor Report 2014-125 v March 2015 Contents Summary 1 Introduction 7 Chapter 1 The California Department of State Hospitals Lacks a Robust Assessment Protocol and Review Process for Its Evaluations for the Sex Offender Commitment Program 21 Recommendations 31 Chapter 2 The California Department of State Hospitals Has Not Provided Consistent Training to Ensure Its Evaluators Produce Thorough Evaluations 33 Recommendations 49 Appendix The Results of Evaluations of Sexually Violent Predators 51 Response to the Audit California Department of State Hospitals 53 California State Auditor’s Comments on the Response From the California Department of State Hospitals 59 vi California State Auditor Report 2014-125 March 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-125 1 March 2015 Summary Results in Brief Audit Highlights . . . The Legislature created the Sex Offender Commitment Program Our audit of the California Department of (program) in 1996 to target a small but extremely dangerous subset State Hospitals’ (State Hospitals) evaluation of sexually violent offenders who present a continuing threat to process for determining whether offenders society because their diagnosed mental disorders predispose meet the criteria of a sexually violent them to engage in sexually violent criminal behavior. Through predator (SVP) highlighted the following: this program, the California Department of Corrections and » It has not ensured that it conducts these Rehabilitation (Corrections) refers certain sex offenders (offenders) evaluations in a consistent manner. to the California Department of State Hospitals (State Hospitals) for psychological evaluations when those offenders are nearing » Although state law requires evaluators to their scheduled release dates. State Hospitals’ evaluators determine use a standardized assessment protocol whether the offenders meet the criteria for being a sexually violent when conducting evaluations, State predator (SVP). If State Hospitals determines that offenders meet Hospitals’ existing protocol lacks detail. the SVP criteria, it requests the county counsels to petition for the offenders’ commitments to a state hospital. If the county counsels » None of its reviews of SVP evaluations concur with the request, the counties will submit a petition to the at headquarters focus on ensuring court, which decides whether the individuals should be committed. the quality of the evaluations from a State law designates Coalinga State Hospital (Coalinga) as the clinical perspective. hospital for SVPs unless unique circumstances exist. For example, one female SVP is held at another state hospital. As of June 2014 • The quality assurance team provides approximately 930 individuals were either residing at Coalinga guidance to less experienced pending trials for commitment or were committed as SVPs. evaluators but does not provide supervisory review. Despite the critical role State Hospitals’ evaluations play in the • Coalinga State Hospital (Coalinga) has SVP commitment process, it has not ensured that it conducts not established a process to document these evaluations in a consistent manner. State law requires that its clinical reviews of evaluations. evaluators consider a number of factors about offenders, such as their criminal and psychosexual histories, when determining » It could better use data related to court whether they meet the SVP criteria. However, of the 29 evaluations outcomes to identify areas to strengthen we reviewed—23 conducted by evaluators at State Hospitals’ its evaluations. headquarters in Sacramento and six conducted by evaluators at Coalinga—we noted instances in which evaluators did not » Its training for SVP evaluators has been demonstrate that they considered all relevant information. For inconsistent—between August 2012 and example, one evaluation did not indicate that the evaluator used May 2014 it offered no training at all. a certain kind of instrument to gauge the risk that the individual would commit another sexual crime, and eight did not note that the » Coalinga has a significant backlog evaluators had reviewed a report from Corrections that identifies of annual SVP evaluations—it had any communication challenges or disabilities the individuals might 261 annual evaluations that were due have that could affect their assessments. In fact, we noted one to courts as of December 2014. instance in which differences in the documentation that evaluators indicated they reviewed led evaluators to reach very different conclusions about an individual: One evaluator noted that the individual had experienced suicidal thoughts, while the other stated that he did not have any mental health issues. 2 California State Auditor Report 2014-125 March 2015 When evaluators do not consider all relevant information, it is possible that State Hospitals may recommend that courts commit individuals who do not pose a danger to the public, or they may not recommend commitment of individuals who do. Further, when evaluators do not fully document how they reached their conclusions, they may not be able to adequately defend those conclusions if challenged in court. To avoid such situations, we would expect State Hospitals to provide its evaluators with significant guidance regarding how they should perform evaluations. State law requires evaluators to use a standardized assessment protocol when conducting evaluations. However, State Hospitals’ existing protocol lacks detail. For example, the protocol does not give guidance on specific risk assessment approaches or list specific risk assessment instruments evaluators may choose to use. In contrast, the former protocol State Hospitals used in 2007 covered approaches to risk assessment and risk assessment instruments. However, State Hospitals revised and simplified this protocol in 2008 because the Office of Administrative Law determined that certain provisions of the protocol met the definition of regulations but had not gone through the required regulatory process. Additionally, evaluators did not always consider all three criteria for determining whether offenders might be recommended for commitment; however, this decision created some efficiency. Specifically, in three evaluations we reviewed the evaluators noted that they did not diagnose a mental disorder—the second of three criteria that must be met for commitment—and therefore chose not to evaluate the third criterion, which is whether the diagnosed mental disorder makes the offenders likely to engage in sexually violent, predatory criminal behavior in the future without treatment and custody. State Hospitals has directed evaluators to complete evaluation of all three criteria regardless of the outcome of one. However, if the evaluator determines that an offender will not meet the criteria, we believe stopping the evaluations is both appropriate and efficient. Given that State Hospitals recently hired many of its evaluators and that evaluating SVPs requires highly specialized skills, we also would expect State Hospitals to have established certain quality control measures, such as supervisory reviews, to ensure that its evaluators complete adequate and consistent evaluations. However, none of State Hospitals’ reviews of SVP evaluations at headquarters focus on ensuring the quality of the evaluations from a clinical perspective. Further, in October 2013, State Hospitals established a quality assurance and training team (quality assurance team) to provide guidance to State Hospitals’ less‑experienced evaluators at headquarters; however, the quality assurance team does not provide supervisory review. At Coalinga—where evaluators conduct annual evaluations of individuals whom the State has already committed as SVPs—hospital managers stated that evaluators receive multiple levels of clinical review. However, Coalinga has not established a California State Auditor Report 2014-125 3 March 2015 process to document these reviews. Without evidence of adequate supervision and review, State Hospitals’ evaluations may fail to effectively demonstrate the need to recommend or not recommend commitment of an individual. Further, State Hospitals could better use data related to court outcomes to identify areas to strengthen its evaluations. High‑quality evaluations are important because courts use them to decide whether individuals are SVPs and should be committed to a state hospital. However, State Hospitals has not consistently tracked the disposition of SVP court cases, and the courts do not always agree with State Hospitals’ recommendations. For example, in one of the 23 evaluations we reviewed at State Hospitals’ headquarters, a court chose to release an offender even though evaluators determined that he met the SVP criteria. A November 2014 change to State Hospitals’ court scheduling process for evaluators may help State Hospitals better track case outcomes and evaluate trends for court decisions; however, it is too soon to conclude whether this new process is successful. Unless it tracks the dispositions of its SVP court cases, State Hospitals is missing an opportunity to improve its evaluation process and potentially strengthen its training and supervision of evaluators. Besides providing guidance and supervisory reviews to evaluators, providing ongoing technical training is important to ensure the competence of those conducting evaluations of potential and current SVPs. However, State Hospitals has not consistently offered training to SVP evaluators. In 2009 and 2010 State Hospitals offered its evaluators—at the time, mostly contractors—training on a variety of topics, including sex offender risk assessment tools, statistics on sexual recidivism, the effect of aging on recidivism, and the violence‑risk scale. In anticipation of hiring evaluators, State Hospitals developed its own training, which it provided in 2011 and part of 2012. However, between August 2012 and May 2014, it offered no training at all. More recently, State Hospitals began taking steps to provide more robust training to its evaluators at its headquarters, though it has yet to take similar steps for the evaluators at Coalinga. In 2014 State Hospitals’ chief psychologist and the quality assurance team developed a training plan for evaluators at headquarters. Specifically, in May 2014, State Hospitals offered comprehensive SVP training for all consulting psychologists, who currently represent 33 of 45 evaluators on staff. The training focused on the background of the SVP statutes, the various criteria under which State Hospitals evaluates potential SVPs, and a specific type of risk assessment tool. State Hospitals has a tentative plan to offer additional training but has yet to schedule it. Coalinga’s evaluators receive fewer training opportunities than the evaluators 4 California State Auditor Report 2014-125 March 2015 at headquarters. Coalinga’s forensic senior psychologist supervisor designed a training plan for fiscal year 2014–15 to help new evaluators at the hospital develop a basic understanding of state law affecting forensic evaluations, forensic report writing, and risk assessment. She indicated that Coalinga is also in the process of developing an ongoing training plan for experienced evaluators and has some trainings scheduled for 2015. Compounding the inconsistent training offered to evaluators, State Hospitals has not offered training on dynamic risk assessment instruments until recently. A dynamic risk assessment may consider factors that change slowly, such as personality disorders or sexual preference, to help predict long‑term risk, and may consider acute, rapidly changing factors, such as negative mood or intoxication, that could signal the possible timing of a reoffense. However, in two trainings on forensic assessment in 2012, State Hospitals’ instructors provided a high‑level overview of dynamic risk factors but did not provide instructions on how to use specific assessment instruments. State Hospitals’ chief psychologist stated that a dynamic risk assessment tool strengthens an evaluation by providing a higher degree of certainty when estimating the risk of a reoffense. As a result, State Hospitals provided training on dynamic risk assessment instruments in December 2014 and January 2015. Finally, Coalinga has a significant backlog of annual SVP evaluations it has not completed. State law requires State Hospitals to evaluate at least annually SVPs committed to it. However, according to Coalinga’s tracking log of overdue annual reports, it had 261 annual evaluations that were due to courts as of December 2014. According to the acting chief of forensic services at Coalinga, State Hospitals briefly required Coalinga’s evaluators to complete another type of evaluation in addition to the annual evaluations, creating additional work. Further, he stated that Coalinga has found it difficult to hire staff. When State Hospitals does not complete annual evaluations on time, it is not fulfilling its statutory obligation to consider whether an SVP is a candidate for release. Recommendations To promote efficiency, the Legislature should change state law to allow State Hospitals the flexibility to stop an evaluation once the evaluator determines that the offender does not meet one of the SVP criteria. California State Auditor Report 2014-125 5 March 2015 To improve the consistency of its evaluations, by June 2015 State Hospitals should create a written policy that requires its evaluators to include the following documentation in their evaluations: • Detail describing all the documentation they reviewed. • A description of the risk assessment instruments the evaluator used. • Acknowledgement of their review of a form from Corrections that identifies any communication challenges or disabilities the offenders might have that could affect their assessments. To promote consistency and ensure that it provides sufficient guidance to evaluators, State Hospitals should update its assessment protocol by March 2016 to include more specific instructions on how to conduct evaluations, such as what assessment instruments evaluators should use and what documents they should consider. To improve the consistency and completeness of its evaluations, by December 2015 State Hospitals should develop a plan for the formal, supervisory review of evaluations from a clinical perspective. To ensure that it has the data necessary to inform its training and supervision of evaluators, State Hospitals should identify the most efficient means for obtaining the outcomes of past trials—at least three years of past trials if possible—and should ensure that it includes such outcomes in its database by March 2016. It should use this information to provide training and supervision where they are most needed. To ensure that its evaluators have the necessary training to conduct evaluations effectively and consistently, State Hospitals should complete the development of its comprehensive training plan for all evaluators by June 2015. In addition, by September 2015 it should provide training on risk assessment instruments to all new evaluators and those who have not yet received such training. To reduce its backlog of annual evaluations at Coalinga and to reduce the number of days these evaluations are overdue, State Hospitals should continue its efforts to hire enough evaluators to meet its workload. 6 California State Auditor Report 2014-125 March 2015 Agency Comments State Hospitals generally agreed with our recommendations, described the steps it would take to implement them, and provided estimated implementation dates. California State Auditor Report 2014-125 7 March 2015 Introduction Background The Legislature created the Sex Offender Commitment Program (program) in 1996 to target a small but extremely dangerous subset of sex offenders (offenders) who present a continuing threat to society because their diagnosed mental disorders predispose them to engage in sexually violent predatory criminal behavior. State law designates these offenders as sexually violent predators (SVPs) and allows the State to commit them to a treatment facility for an indeterminate period of time. The law lists crimes that qualify as sexually violent offenses and defines predatory to mean acts against strangers, persons of casual acquaintance, or persons with whom the offender established relationships primarily for the purposes of victimization. Before the State commits offenders, state law requires that the State conduct trials to determine whether the offenders meet the criteria for an SVP—that, by reason of diagnosed mental disorders, they are likely to engage in acts of predatory sexual violence upon release. Determining whether offenders are SVPs and committing them for treatment is a civil rather than a criminal process. Changes to state law during the last decade have expanded the scope of the program. In September 2006 Senate Bill 1128 (SB 1128) became law and added more crimes to the list of sexually violent offenses that could cause offenders to qualify as SVPs. More dramatically, in November 2006, California voters passed Proposition 83, also known as Jessica’s Law. In addition to creating additional residency restrictions and requiring global positioning system monitoring for certain sex offenders, Jessica’s Law added more crimes to the list of sexually violent offenses and decreased from two to one the number of victims necessary for the SVP designation to apply. Both SB 1128 and Jessica’s Law abolished the previous two‑year term of civil commitment for SVPs and instead established a commitment term of indeterminate length. State law requires that individuals committed to the program as SVPs receive annual evaluations to consider their readiness for release. Further, as of June 2012, state law generally designated Coalinga State Hospital (Coalinga) as the state hospital for placing individuals committed as SVPs.1 The Process for Evaluating SVPs The California Department of State Hospitals (State Hospitals) and the California Department of Corrections and Rehabilitation (Corrections), including its Board of Parole Hearings (Parole Board), 1 According to a July 2014 census from State Hospitals, there was one female SVP at another state hospital. 8 California State Auditor Report 2014-125 March 2015 each play a role in identifying, evaluating, and requesting the commitment of an offender as an SVP. However, a court or jury makes the final determination of an offender’s SVP status. State law requires that Corrections and its Parole Board screen offenders based on whether they committed sexually violent predatory offenses and on reviews of their social, criminal, and institutional histories. To complete these screenings, the law requires that Corrections use a structured screening instrument developed and updated by State Hospitals in consultation with Corrections. According to state law, when Corrections determines through this screening process that offenders may be SVPs, it must refer the offenders to State Hospitals for further evaluation at least six months before their scheduled release dates. State law requires that State Hospitals evaluate Indicators That a Sex Offender Is a all offenders that Corrections refers to it. Sexually Violent Predator It specifies that for each of these offenders, State Hospitals must conduct an evaluation The California Department of State Hospitals uses the consisting of assessments by two mental health following criteria in state law to determine whether a professionals who must be practicing psychiatrists sex offender (offender) meets the criteria of a sexually or psychologists. However, in practice, State violent predator: Hospitals has an agreement with Corrections to • The offender has been convicted of a sexually conduct clinical reviews on Corrections’ behalf violent predatory offense against one or more in order to determine whether offenders merit victims, such as rape when committed with force, a full evaluation. Figure 1 illustrates the process threats, or other violence. that State Hospitals uses to determine whether • The offender suffers from a diagnosed it should recommend to the district attorneys or mental disorder. the designated county counsels responsible for - The law defines a diagnosed mental disorder as a handling SVP cases (designated counsels) that the condition affecting the emotional and volitional court should commit offenders to the program. capacity that predisposes the person to commit criminal sexual acts to a degree that the person is a State law requires State Hospitals’ evaluators menace to the health and safety of others. to determine whether the offenders that • The diagnosed mental disorder makes the person Corrections refers to it meet the criteria for likely to engage in sexually violent predatory the SVP designation. State Hospitals divides the criminal behavior in the future without treatment criteria for evaluation into three broad areas, and custody. which the text box describes in more detail. - The law defines predatory offenses as acts against Two evaluators independently review information strangers, persons of casual acquaintance, or related to each offender and attempt to interview persons with whom the offender established him or her. If both evaluators agree that the relationships primarily for the purpose offender meets the SVP criteria, State Hospitals of victimization. must request a petition for commitment. If the - Regulations require evaluators to use tests and two evaluators disagree, the law requires State instruments and to consider various risk factors to Hospitals to appoint two additional evaluators— determine the risk that an offender will commit who must meet certain professional qualifications future crimes. and cannot be employees of the State—to perform evaluations. The two additional evaluators Sources: Analysis of California Welfare and Institutions Code, Section 6600 et seq., Title 9 of the California Code of must agree that the offender meets the SVP Regulations, and a California Supreme Court decision. criteria if State Hospitals is to request a petition for commitment. California State Auditor Report 2014-125 9 March 2015 Figure 1 Process for Determining Whether an Offender Meets the Criteria of a Sexually Violent Predator Administrative Review Clinical Review Administrative staff at the California Department of A psychologist or psychiatrist State Hospitals (State Hospitals) ensure that the California conducts a review of available case NO X Department of Corrections and Rehabilitation records, a risk assessment, and a (Corrections) has forwarded relevant medical, criminal preliminary clinical diagnosis to history, and police records. Administrative staff also obtain determine whether the case may records if necessary and determine that the sex offender meet requirements in state law. (offender) is available for evaluation. YES  Evaluation Following a file review and an interview with the offender—if the offender agrees to one—two evaluators determine separately whether the offender meets the criteria as a sexually violent predator (SVP).   State Hospitals notifies State Hospitals requests a  X Corrections that the petition for commitment X X offender does not meet SVP criteria. Difference-of-Opinion Evaluation Two additional contract evaluators determine separately whether the offender meets SVP criteria.    X X X If the designated counsel for the county in which the offender was Probable Cause convicted agrees with State Hospitals’ recommendation, the counsel will file A court determines whether there NO X a petition for commitment. is probable cause to believe the offender is an SVP. YES Corrections releases the  offender at the end of his or her prison sentence. A court holds a trial to determine whether the NO X offender is an SVP. YES  Committed to Custody The State commits an SVP to the custody of State Hospitals. Sources: California Welfare and Institutions Code, Section 6600 et seq., State Hospitals’ chief psychologist in the Forensic Services Division, and State Hospitals’ records of clinical evaluations. 10 California State Auditor Report 2014-125 March 2015 Assessing the Risk of Reoffense Static and Dynamic Risk Factors Evaluators have a number of risk assessment Static risk factors are fixed or historical characteristics, such instruments at their disposal for evaluating as offender age, offense history, and sexual deviance. the risk that the offender will commit another Dynamic risk factors are characteristics that can sexually violent predatory crime in the future. change over time, such as cooperation with supervision, The tools assist evaluators in assessing the impact self-regulation, and social influences. various less changeable and more changeable Selected Risk Assessment Tools characteristics—called static and dynamic variables, respectively—have on the risk that • Static-99/Static-99R: A 10-item, score-based assessment an individual will commit another crime. The tool used for adult male offenders ages 18 and over that text box describes the types of variable risk factors addresses the risk of reoffending by examining more and identifies several risk assessment tools State static risk factors such as the offender’s age, conviction for Hospitals uses. nonsexual violence, prior sex offenses, and relationship with victims, among other items. State law established a committee—the State • Stable 2007: An evidence-based risk assessment tool that Authorized Risk Assessment Tool for Sex measures dynamic risk factors. The State Authorized Risk Offenders Review Committee (SARATSO Assessment Tool for Sex Offender Review Committee committee)—to select tools for use when (SARATSO committee) adopted the Stable 2007 in assessing whether sex offenders will likely September 2013 as the new dynamic risk assessment commit other sexual crimes. SARATSO selected instrument for California. the Static‑99R risk assessment scale as the tool • The Structured Risk Assessment/Forensic Version Light to evaluate adult males required to register as (SRA-FVL) assesses long-term vulnerabilities through a sex offenders. State law requires the SARATSO review of two domains, sexual interests and relational style, committee to determine whether the State should and a partial review of a third domain, self-management. replace or supplement the static assessment tool Sources: The Web site for the SARATSO committee and in use. Until 2013 the SARATSO committee www.static99.org, a Web site whose advisory board includes recommended supplementing the Static‑99R the creators of the Static-99. with another assessment tool—the Structured Risk Assessment/Forensic Version Light. In 2013 SARATSO selected the Stable 2007 dynamic risk assessment instrument to supplement the Static‑99R. Process for Committing Offenders as SVPs Although State Hospitals conducts evaluations to determine whether offenders meet the SVP criteria, a court or jury makes the final decision to commit the offenders. When two evaluators determine that an offender meets the SVP criteria, state law requires that State Hospitals request the designated counsel of the county in which the offender was convicted to file a petition in court to commit the offender. If the county’s designated counsel agrees with State Hospitals’ recommendation, he or she must file a petition for commitment. State law requires that a judge determine whether probable cause exists to detain an offender beyond his or her prison term. If a judge determines that there is probable cause that the offender may be an SVP, he or she will order California State Auditor Report 2014-125 11 March 2015 that the offender remain in custody in a secure facility. State law then requires a trial to determine whether the offender is an SVP. During the trial, the court may call upon State Hospitals’ evaluators to provide testimony regarding their evaluations. According to State Hospitals, as of June 2014, 354 individuals at Coalinga were awaiting trial to determine whether they would be committed or released. The acting chief of forensic services at Coalinga stated that some individuals awaiting trial receive treatment. However, State Hospitals’ chief psychologist told us that offenders’ attorneys may recommend they not seek treatment because courts may see participation in treatment as an admission that they are SVPs. Representatives of the courts may periodically request updated evaluations of potential SVPs awaiting trial. Additionally, some offenders purposely delay their trials and remain confined at a state hospital but are not technically committed as SVPs. State Hospitals’ chief psychologist stated that offenders often delay their trials because age is a factor in determining whether an individual is likely to commit another sexually violent, predatory crime, as older offenders are statistically less likely to reoffend. According to State Hospitals’ Sex Offender Commitment Program Support System, during fiscal years 2009–10 through 2013–14, courts determined that between eight and 22 individuals per year did not meet the SVP criteria and released them. In fiscal year 2013–14, for example, courts released 16 individuals, compared to the population of approximately 350 awaiting trial at Coalinga as of June 2014. Treatment and Release If a court or jury finds that an offender is an SVP, a court commits the offender to a secure facility—generally Coalinga—for an indeterminate time period. State law requires State Hospitals to offer treatment. For those SVPs refusing treatment, State Hospitals must continue to offer treatment on at least a monthly basis. According to State Hospitals, as of June 2014 the courts had committed 573 individuals as SVPs who were housed at Coalinga, and the forensic senior psychologist supervisor at Coalinga stated that 35 percent participate in treatment. Although state law does not require treatment, it allows courts to consider an SVP’s failure to participate in or complete treatment when determining whether the SVP’s condition has changed and whether the SVP is eligible for release. Figure 2 on the following page outlines the process from commitment through release. 12 California State Auditor Report 2014-125 March 2015 Figure 2 Process Through Which the State Releases Sexually Violent Predators From the Custody of the California Department of State Hospitals Committed to Custody The State commits a sexually violent predator (SVP) to the custody of the California Department of State Hospitals (State Hospitals). Annual Evaluation State Hospitals annually evaluates the SVP’s mental condition to determine whether conditional release or unconditional discharge is in his or her best interest and if the State can impose conditions that would adequately protect the community. If State Hospitals determines that conditional release or unconditional discharge is appropriate, it will authorize the SVP to petition the court. Petition for Petition for Unconditional Discharge Conditional Release Court Decision Court Decision The court determines whether the SVP should The court determines whether the SVP should be discharged. be released. If the court determines that probable cause exists to If the court determines that the petition is not frivolous, believe the person’s diagnosis has so changed that he or it will set a hearing on the petition to determine she no longer poses a danger to others, the court will set whether the SVP would pose a danger to the health a hearing on the petition. Both the designated counsel and safety of others if under supervision and treatment and the SVP have the right to demand a jury trial. in a community. The court rules in favor of The court rules against The court rules in favor of the SVP. the SVP. the SVP. The SVP is unconditionally discharged from The SVP remains in the custody of State The court places the SVP in a state-operated State Hospitals’ custody. Hospitals and cannot petition again for forensic conditional release program. The SVP is one year from the date of the ruling. placed in the community in accordance with his or her treatment and supervision plan unless good cause for not doing so is presented to the court. After one year in the conditional release program, the SVP may petition the court for an unconditional discharge, as described above. Source: California Welfare and Institutions Code, Section 6600 et seq. California State Auditor Report 2014-125 13 March 2015 State law requires State Hospitals to evaluate SVPs once a year to determine whether they might qualify for release. The law also requires that a professionally qualified person prepare the evaluation report. In addition to the evaluation performed by an evaluator at Coalinga, the SVP may retain, or request the court to appoint, an expert to perform the annual evaluation. The evaluator’s annual report must consider whether the SVP currently meets the SVP criteria and whether unconditional release or release to a less restrictive alternative than a state hospital—called a conditional release—would be in the SVP’s best interest. State Hospitals must file these annual reports with the courts in the counties that committed the SVPs. As of December 2014 Coalinga had 11 evaluators performing annual evaluations. Similar to the decision to commit an SVP, the decision to release an SVP resides with the courts. When State Hospitals finds that an SVP’s condition has changed and that he or she no longer meets the SVP criteria and unconditional discharge is appropriate, state law requires State Hospitals’ director to authorize the SVP to file a petition for unconditional discharge with the court responsible for his or her initial commitment. If the court determines that probable cause exists that an individual’s diagnosed mental disorder has changed and he or she is not a danger to others, then state law requires the court to set a hearing on the issue. Both the designated counsel and the SVP have the right to request a jury trial. When determining whether an SVP seeking unconditional discharge continues to meet the SVP criteria, state law places the burden of proof on the State to prove beyond reasonable doubt that the SVP remains a danger to others. Table 1 on the following page shows the number of offenders who had been committed as SVPs who were discharged from a state hospital, as well as the number of SVPs that State Hospitals conditionally released, during fiscal years 2009–10 through 2013–14. As part of its annual evaluations, State Hospitals may also recommend the release of an SVP to a less restrictive environment, called a conditional release. An SVP may petition a court for a conditional release with or without State Hospitals’ recommendation. If a court determines the SVP’s petition is not frivolous, the court will conduct a hearing to determine whether releasing him or her would pose a danger to the health and safety of others—that is, whether the SVP is likely to engage in sexually violent predatory criminal behavior due to a diagnosed mental disorder, if under supervision and treatment in the community. If a court determines that the SVP would not be a danger to others through treatment in the community, state law requires the court to order the SVP into a state‑operated conditional release program that includes outpatient supervision 14 California State Auditor Report 2014-125 March 2015 and treatment. The conditional release program requires SVPs to abide by various conditions. For example, in July 2014, an SVP released into the conditional release program agreed to numerous conditions, such as outpatient treatment, 24‑hour monitoring via a global positioning system, and restrictions on travel. State Hospitals has a contract with Liberty Healthcare to provide services for the conditional release program. Table 1 Individuals Discharged From California Department of State Hospitals’ Custody or Released Conditionally Fiscal Years 2009–10 Through 2013–14 OF THOSE RELEASED, INDIVIDUALS INDIVIDUALS ADMITTED DISCHARGED TO THE CONDITIONAL FISCAL YEAR OR RELEASED* RELEASE PROGRAM† 2009–10 23 4 2010–11 22 0 2011–12 17 3 2012–13 10 2 2013–14 14 2 Totals 86 11 Sources: California State Auditor’s analysis of data obtained from the California Department of State Hospitals’ (State Hospitals) Sex Offender Commitment Program Support System; information from State Hospitals’ conditional release program; and a spreadsheet of discharges, transfers, and deaths from Coalinga State Hospital. * Of the 86 individuals discharged from State Hospitals’ custody during the audit period, only nine were committed for indeterminate terms. The remaining 77 had expired two-year commitments—that is, they were committed before changes to state law in 2006, and the courts did not recommit them for an indeterminate term. Individuals may be unconditionally discharged from State Hospitals’ custody to the community when a court determines they no longer meet the criteria of a sexually violent predator (SVP). Individuals may also be discharged to parole, incarceration—for example, to a county jail—discharged to immigration for deportation, or released to the conditional release program. † SVPs released conditionally must agree to certain restrictions, such as outpatient treatment and monitoring. Progress on Implementing Recommendations From a 2011 Audit on the Program State Hospitals has fully implemented three recommendations from the California State Auditor’s (state auditor) prior audit report and has stated that it will not implement two. In July 2011 the state auditor issued a report titled Sex Offender Commitment Program: Streamlining the Process for Identifying Potential Sexually Violent Predators Would Reduce Unnecessary or Duplicative Work (Report Number 2010‑116). The report concluded that the processes at Corrections and the former California Department of Mental Health—now State Hospitals—for identifying and evaluating SVPs were not as efficient as they could be and at times California State Auditor Report 2014-125 15 March 2015 resulted in the State performing unnecessary work.2 The report made five recommendations to State Hospitals, and we discuss the status of each in the following paragraphs. The report recommended that State Hospitals expand the use of its database to capture more specific information about the offenders whom Corrections refers to it and the outcomes of the screenings and evaluations that it conducts. State Hospitals has completed database enhancements that enable it to track more specific information related to victims, offenders, offenses, clinical screening outcomes, and evaluation outcomes. State Hospitals also fully implemented a recommendation that it continue its efforts to obtain approval for a new position classification for evaluators, continue to recruit qualified individuals, and continue its efforts to train its consulting psychologists to conduct evaluations. State Hospitals received approval from the State Personnel Board for a new sexually violent predator evaluator (SVPE) position and, according to the assistant deputy director of State Hospitals’ Forensic Services Division (forensic services), has completed hiring evaluators in this classification. As of December 2014 State Hospitals had 12 SVPEs and 33 consulting psychologists on staff to conduct evaluations. Further, we recommended that State Hospitals complete and submit reports to the Legislature on its efforts to hire state employees to conduct evaluations and on the impact of Jessica’s Law on the program. As previously noted, State Hospitals completed its hiring of employees to complete evaluations. State Hospitals submitted a report to the Legislature in July 2012, which included information on the impact of Jessica’s Law. State Hospitals reported to us that it will not implement two recommendations. First, the report recommended that, to eliminate duplicative effort and increase efficiency, Corrections and State Hospitals jointly revise their structured screening instrument so that the referral process would adhere more closely to the law’s intent. As previously discussed, state law requires Corrections and its Parole Board to screen potential SVPs in accordance with a structured screening instrument developed and updated by State Hospitals in consultation with Corrections. If this screening determines that the offender is likely to be an SVP, state law requires Corrections to refer the person to State Hospitals for a full evaluation. However, our 2011 audit report found that Corrections 2 In June 2012 the California Department of Mental Health was renamed the California Department of State Hospitals. To avoid confusion, we refer to the California Department of Mental Health as State Hospitals throughout the report. 16 California State Auditor Report 2014-125 March 2015 frequently referred offenders whom State Hospitals had previously evaluated and found not to meet the SVP criteria, even though those offenders had not committed new sexual crimes. To address this recommendation, State Hospitals reported that it believes that by entering into a memorandum of understanding with Corrections in January 2011, in which Corrections delegated authority to State Hospitals to conduct a clinical review, it is in compliance with the law’s intent. State Hospitals agreed to conduct clinical review screens of offenders’ sexually violent predatory offenses and social, criminal, and institutional histories. According to the assistant deputy director of forensic services, the memorandum uses State Hospitals’ expertise in evaluating potential offenders. Finally, State Hospitals reported that it will not implement a recommendation related to reducing costs for unnecessary evaluations. Specifically, the report recommended that it should either issue a regulation or seek a statutory amendment to clarify that when resolving a difference of opinion between the two initial evaluators of an offender, it must seek the opinion of a fourth evaluator only when the third evaluator concludes that the offender meets SVP criteria. State Hospitals stated that it will not implement this recommendation because of the significant reduction in referrals it receives from Corrections and its finding that potential costs savings were insignificant. Therefore, according to the assistant deputy director of forensic services, State Hospitals plans to continue to obtain two evaluations to resolve cases in which the initial two evaluators do not agree. Scope and Methodology The Joint Legislative Audit Committee directed the state auditor to perform an audit of the policies and procedures that State Hospitals’ mental health professionals follow when evaluating sex offenders for initial commitment, recommitment, and conditional or unconditional release. California State Auditor Report 2014-125 17 March 2015 Table 2 Audit Objectives and the Methods Used to Address Them AUDIT OBJECTIVE METHOD 1 Review and evaluate the laws, rules, Reviewed relevant state laws and regulations. and regulations significant to the audit objectives. 2 Review the policies and procedures used To evaluate State Hospitals’ policies and procedures for the specified items, we did the following: by evaluators at the California Department • Reviewed current and prior versions of State Hospitals’ standardized assessment protocol and of State Hospitals (State Hospitals) when other documentation related to conducting evaluations. conducting evaluations of offenders • Reviewed a decision from the Office of Administrative Law that provisions of State Hospitals’ 2007 and sexually violent predators (SVPs). protocol were regulations that had not been approved through the Administrative Procedure Act. Specifically, determine the following: • Interviewed key staff at State Hospitals and at Coalinga State Hospital (Coalinga) to obtain their a. The amount of time that evaluators perspectives. Also reviewed State Hospitals’ available policies and procedures. are directed to spend on evaluations. b. The peer and supervisory review • Noted that neither State Hospitals nor Coalinga has formal policies regarding replacing evaluators procedures for evaluations. when evaluations are incomplete, but managers at both entities stated that a new evaluator c. The steps taken to replace an assigned to an incomplete evaluation would need to redo it entirely. evaluator when an evaluation • Interviewed key staff at State Hospitals and Coalinga to determine what peer and supervisory is incomplete. review procedures are used in the evaluation process. Also reviewed available documentation of d. Whether the policies or procedures the peer and supervisory review procedures. provide any monetary or workload To assess whether State Hospitals’ policies and procedures were consistent with best practices, we did incentives to evaluators. the following: e. Whether the policies and procedures used by State Hospitals’ evaluators • Reviewed guidance from the American Psychological Association (APA) regarding forensic are consistent with best practices, to evaluation of sex offenders and incorporated that into our work on objectives 3 and 7c. the extent that those practices can • Reviewed laws related to SVPs in Massachusetts, Washington, and South Carolina, as well as be identified. audits or other research on programs in Virginia, Florida, and Colorado. We did not identify specific practices for California to follow. 3 Review a selection of evaluations, including • Interviewed relevant staff regarding State Hospitals’ and Coalinga’s expectations for evaluations for initial commitment and for completing evaluations. conditional and unconditional release, and • Selected and reviewed 29 evaluations conducted during fiscal years 2009–10 through 2013–14, determine whether the evaluations were including 23 from State Hospitals’ evaluators at its headquarters and six from Coalinga’s evaluators. completed in accordance with state law and Our selection included initial, updated, and annual evaluations, as well as evaluations performed regulations, with State Hospitals’ policies immediately before an offender or an SVP’s release. and procedures, and with any identifiable • Assessed the extent to which the 29 evaluations adhered to State Hospitals’ protocol and best practices. As part of this review, expectations as well as APA guidance. consider the breadth of documents that State Hospitals’ evaluators consider when • Compared each of the 23 evaluations at State Hospitals’ headquarters to other evaluations of the completing an evaluation of a sex offender same individual to determine any differences in the documents the evaluators indicated they used. or SVP and the number of treating staff • For the six evaluations at Coalinga, we assessed the number of treatment staff consulted when interviewed as part of the evaluation. developing the evaluation. We have no findings in this area. 4 By year, determine the number of positive Calculated positive and negative determinations for offenders using data obtained from State determinations and the number of negative Hospitals’ Sex Offender Commitment Program Support System (SOCPSS) and present the data in determinations for both offenders and SVPs. Table 5 on page 41. See the discussion about SVPs in Objective 5. 5 Determine the following information • Calculated the number of individuals released using data from State Hospitals’ SOCPSS. by year: • Obtained a list of all SVPs who had been or were currently in State Hospitals’ conditional a. The number of SVPs that State release program. Hospitals found suitable for • In SOCPSS, State Hospitals tracks the findings of each evaluation of individuals who have been conditional release. committed to State Hospitals. However, it does not track an overall result by individual evaluated. b. The number of SVPs that State Thus, we are not able to report on the number of SVPs that State Hospitals found suitable Hospitals found suitable for for release. Instead, we report the findings of individual evaluators. The table on page 52 in unconditional release. the Appendix provides information on the conclusions of individual evaluators regarding the c. The number of SVPs released for any individuals committed to the custody of State Hospitals. reason, including a case dismissal • Interviewed key State Hospitals management regarding its efforts to track the outcomes of or a finding by State Hospitals that court cases. the individual did not meet the SVP criteria. d. The number of SVPs that State Hospitals found unsuitable for release. continued on next page . . . 18 California State Auditor Report 2014-125 March 2015 AUDIT OBJECTIVE METHOD 6 By year, determine the total number of • Using data obtained from State Hospitals’ SOCPSS, information obtained from both State Hospitals evaluators used by State Hospitals, the and Coalinga on their evaluators, and State Hospitals’ payroll data obtained from the California number of those evaluators that were State State Controller’s Office’s Uniform State Payroll System, we calculated the number of employee Hospitals’ employees, and the number of and contract evaluators State Hospitals used during fiscal years 2009–10 through 2013–14 and the cases assigned to each evaluator. number of evaluations conducted. We present this data in Table 6 on page 44. • Identified State Hospitals’ method for assigning cases to evaluators. Because State Hospitals assigns evaluators on a per-evaluation basis, rather than a per-case basis, the steps we took to address Objective 6 also pertain to Objective 7b. We also report on the process for assigning workload to evaluators in Objective 8. 7 Review the qualifications and experience • Interviewed relevant staff regarding State Hospitals’ procedures to verify that evaluators had the of the evaluators State Hospitals used. At a appropriate qualifications. minimum, consider the following: • Reviewed employment or contracting records for a selection of 15 employee and a. The number of years of relevant contract evaluators. experience for State Hospital • For the 15 selected evaluators, we determined the number of years of experience based on the employees who conducted licensure period and other documentation. evaluations as compared to the • Verified whether each of the evaluators had a current, state-issued license to practice psychology number of years of experience for or psychiatry in California. evaluators that State Hospitals contracted with. b. The number of evaluations conducted We describe the steps taken to address Objective 7b in Objective 6. by State Hospitals’ employees for each of the last five years as compared to the number of evaluations conducted by evaluators that State Hospitals contracts with over the same period. c. Whether State Hospitals provides • Reviewed training materials and schedules. adequate training to both staff and • Obtained the perspective of managers at both State Hospitals’ headquarters and Coalinga contracted evaluators. regarding training needs for evaluators. • Reviewed training plans in place at both State Hospitals and Coalinga. 8 To the extent possible, provide • Reviewed evaluator contracts and employee position descriptions for our audit period. recommendations for changes that would • Interviewed key management to understand State Hospitals’ method of assigning work to improve the policies and procedures State its evaluators. Hospitals uses to evaluate offenders and • Obtained documentation and evaluated workload assignment methodology to assess whether the SVPs and the compensation or incentives practices in place might create incentives for employees or contractors to rush work. given to evaluators. • Made recommendations to State Hospitals related to several of the audit objectives. 9 Review and assess any other issues • Interviewed key managers at State Hospitals and reviewed relevant documentation related to significant to the evaluation of offenders State Hospitals’ implementation of recommendations from a 2011 audit report by the California and SVPs. State Auditor (state auditor). • Interviewed relevant staff and obtained Coalinga’s tracking log to identify the extent of its backlog of annual evaluations for individuSals committed as SVPs. Sources: State auditor’s analysis of Joint Legislative Audit Committee audit request 2014-125, and information and documentation identified in the table column titled Method. California State Auditor Report 2014-125 19 March 2015 Methods to Assess Data Reliability In performing this audit, we obtained electronic data files extracted from the information systems listed in Table 3. The United States Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of the computer‑processed information that we use to support our findings, conclusions, or recommendations. Table 3 describes the analyses we conducted using data from these information systems, our methodology for testing them, and the issues we identified pertaining to the data. Although we recognize that these issues may impact the precision of the numbers we present, there is sufficient evidence in total to support our audit findings, conclusions, and recommendations. Table 3 Methods Used to Assess Data Reliability INFORMATION SYSTEM PURPOSE METHOD AND RESULT CONCLUSION California Department For the period July 1, 2009, • We performed data-set verification procedures and electronic Undetermined reliability for of State Hospitals through June 30, 2014, testing of key data elements and did not identify any errors. the purposes of this audit. (State Hospitals) determine whether the Although this determination • We relied on completeness testing performed as part of the individual conducting may affect the precision of State’s annual financial audit for payroll transactions between Uniform State Payroll the evaluation was a the numbers we present, January 2008 and June 2013. Because we found the payroll System (payroll system) State Hospitals’ employee there is sufficient evidence data to be complete between January 2008 and June 2013, (employee) or an evaluator in total to support our we have reasonable assurance that the payroll data for the State Hospitals’ payroll State Hospitals contracted audit findings, conclusions, period of July 2013 through June 2014 are also complete. data as maintained with (contractor) at the and recommendations. by the California State time the evaluation • We did not conduct accuracy testing on these data. Controller’s Office was performed. (state controller) for the period July 1, 2009, through June 30, 2014 State Hospitals To determine the number • We performed data-set verification procedures and electronic Sufficiently reliable for the of evaluations by type, testing of key data elements and did not identify any issues. purposes of this audit. Sex Offender outcome, and fiscal year • For a random selection of 29 evaluations, we verified that key Commitment Program and the number of cases data elements matched source documentation and did not Support System that included a difference identify any significant issues. (SOCPSS) of opinion between evaluators for offenders prior • To test the completeness of State Hospitals’ data, we State Hospitals case to commitment. haphazardly selected 29 evaluations and traced them from and evaluation data State Hospitals’ source documents back to SOCPSS. We found related to sex offenders the data to be complete. (offenders) as of September 17, 2014 continued on next page . . . 20 California State Auditor Report 2014-125 March 2015 INFORMATION SYSTEM PURPOSE METHOD AND RESULT CONCLUSION To determine by fiscal year • We performed data-set verification procedures and electronic Not sufficiently reliable for the number of offenders testing of key data elements and did not identify any issues. the purposes of this audit. awaiting trial who Although this determination • For a random selection of 46 evaluations prior to received evaluations by may affect the precision of commitment, we attempted to verify that key data elements State Hospitals. the numbers we present, matched source documentation. However, we found there is sufficient evidence To determine by fiscal year two errors in each of two fields that contain the date that in total to support our the number of cases prior to probable cause was found for an offender awaiting trial and audit findings, conclusions, commitment found by State the date that State Hospitals made its final determination for and recommendations. Hospitals to be positive or an offender prior to commitment. negative for commitment. • To test the completeness of State Hospitals’ data, we haphazardly selected 29 evaluations prior to commitment and traced them from State Hospitals’ source documents back to SOCPSS. We found the data to be complete. To determine by fiscal year • We performed data-set verification procedures and electronic Not sufficiently reliable for the number of evaluations testing of key data elements and did not identify any issues. the purposes of this audit. after commitment performed Although this determination • We performed accuracy testing for a random selection on sexually violent predators may affect the precision of of 29 evaluations performed after commitment and (SVPs) by type and outcome. the numbers we present, attempted to verify that key data elements matched there is sufficient evidence source documentation. The results of our testing identified in total to support our three errors in the field that contains the date that State audit findings, conclusions, Hospitals received an evaluation from the evaluator. and recommendations. As a result, we were unable to identify the complete universe of evaluations during our audit period that were performed after commitment. Therefore, we did not conduct completeness testing for evaluations performed after commitment. To determine by fiscal year • We performed data-set verification procedures and electronic Sufficiently reliable for the the number of offenders held testing of key data elements and did not identify any issues. purposes of this audit. due to probable cause who • We randomly selected 29 evaluations after commitment were discharged. and verified that key data elements matched source To identify SVPs who were documentation, and did not identify any issues. discharged by fiscal year. • To assess the completeness of the SOCPSS data, we haphazardly selected 29 discharge records and traced them from State Hospitals’ source documents back to SOCPSS and found the data to be complete. Determine by fiscal year • We performed data-set verification procedures and did not Not sufficiently reliable for for both State Hospitals’ identify any issues. the purposes of this audit. employees and contractors However, we determined • We performed electronic testing of key data elements and the total number the numbers we present found that the data field containing Social Security number evaluations conducted. for evaluations prior to information was blank 47 percent of the time. As a result, we commitment are accurate could not use this field to identify State Hospitals’ employees due to the additional steps who performed evaluations and took additional steps to we performed. Further, manually identify their Social Security numbers. the issues identified in • We traced the universe of employees who performed accuracy testing may evaluations during our audit period to the state controller’s impact the precision of the payroll system. We then randomly selected 29 contractors numbers we present for who performed evaluations during our audit period and evaluations performed after verified that State Hospitals had contracts with these commitment. However, evaluators during our audit period. there is sufficient evidence • As we previously described, the results of our accuracy in total to support our testing of evaluations performed after commitment showed audit findings, conclusions, that the universe of evaluations during our audit period is not and recommendations. complete. Therefore, we lack assurance that we were able to identify all evaluators who performed evaluations during our audit period. Sources: California State Auditor’s analysis of various documents, interviews, and data from the entities listed above. California State Auditor Report 2014-125 21 March 2015 Chapter 1 THE CALIFORNIA DEPARTMENT OF STATE HOSPITALS LACKS A ROBUST ASSESSMENT PROTOCOL AND REVIEW PROCESS FOR ITS EVALUATIONS FOR THE SEX OFFENDER COMMITMENT PROGRAM Chapter Summary The California Department of State Hospitals (State Hospitals) has not been consistent in its evaluations of the sex offenders (offenders) whom the California Department of Corrections and Rehabilitation (Corrections) has referred to it for possible commitment as sexually violent predators (SVPs). Specifically, our review found that State Hospitals’ evaluators did not always document that they considered all relevant information in their evaluations. State Hospitals’ current assessment protocol likely contributed to the issues we noted. For example, the protocol does not give guidance on specific risk assessment approaches or list specific risk assessment instruments evaluators may choose to use. Further, although SVP evaluations completed by evaluators at State Hospitals’ headquarters undergo several reviews, none is focused on ensuring the quality of evaluations from a clinical perspective. In October 2013 State Hospitals established a quality assurance and training team (quality assurance team) to provide guidance to State Hospitals’ less‑experienced evaluators at headquarters; however, the quality assurance team does not provide supervisory review. Given that the courts, and not State Hospitals, have the final say on whether an offender is an SVP, we would expect State Hospitals to gather and analyze data on the extent to which the courts disagree with evaluators. However, State Hospitals has not consistently tracked the disposition of court cases. As such, State Hospitals is missing an opportunity to improve its evaluation process and better inform the training and supervision of its evaluators. State Hospitals’ Evaluations of Current and Potential SVPs Have Been Inconsistent State law requires that State Hospitals evaluate offenders for possible commitment as SVPs by considering criminal history; psychosexual history; type, degree, and duration of sexual deviance; and severity of mental disorder. However, our review of 29 evaluations found that State Hospitals’ evaluators did not always document that they considered all relevant information. We reviewed 23 evaluations of current and potential SVPs completed by State Hospitals’ evaluators at its headquarters in Sacramento and 22 California State Auditor Report 2014-125 March 2015 six annual evaluations of current SVPs that it completed at Coalinga State Hospital (Coalinga). We noted instances in which evaluators did not consider all relevant documentation, address elements of offenders’ backgrounds, or use certain instruments to assess the risk of offenders committing additional crimes. When evaluators do not consider all relevant information, they may reach incorrect conclusions. Further, when they do not document the reasoning behind their conclusions, those conclusions are more likely to be challenged in court. State Hospitals’ evaluators have Our review demonstrates that State Hospitals’ evaluators have been inconsistent in the breadth of been inconsistent in the breadth of documentation they consider documentation they consider while while performing evaluations. According to State Hospitals’ chief performing evaluations. psychologist, State Hospitals expects its evaluators to review all documentation relevant to offenders they are evaluating; however, it has not formalized this expectation into a written policy or procedures manual, but rather informally communicates it to evaluators.3 Nonetheless, when we reviewed evaluations by State Hospitals’ evaluators in Sacramento, we found discrepancies in the ways that different evaluators assessed the same offenders. As discussed in the Introduction, at least two evaluators must independently evaluate whether an offender meets the criteria of an SVP. However, when we compared each of the 23 evaluations selected for review to companion evaluations performed by other evaluators, we noted differences in the documents evaluators indicated they reviewed. At times, these discrepancies led to significant differences in the evaluators’ descriptions of the offenders being evaluated. For example, in one case an evaluator listed that he reviewed several mental health records for a potential SVP that another evaluator did not list. The evaluator who listed reviewing these records noted that the offender experienced suicidal thoughts during incarceration, while the other evaluator stated that the offender did not have any mental health problems according to the offender’s records from Corrections. This type of discrepancy is concerning and could ultimately prove problematic in court. We also observed other instances in which evaluators noted that they reviewed records others did not, such as probation reports, court complaints, behavioral reports, treatment records, and psychiatric notes. In addition, the evaluations we reviewed did not always consider relevant background information. Specifically, four of the 23 evaluations did not contain sections describing that the evaluator 3 In 2015 State Hospitals’ contracts with independent evaluators require that the evaluators conduct a thorough file review, including a review of the offender’s correctional file, criminal history, arrest record, and county probation reports. However, this language did not appear in past contracts. California State Auditor Report 2014-125 23 March 2015 considered the psychosexual history of the offenders, as state law requires. According to the Center for Sex Offender Management, a psychosexual history is a detailed and thorough sexual history that includes the exploration of sexual development, attitudes, fantasies, and adjustment.4 Although two of these four evaluations contained sexual history sections and relationship history sections, they did not contain sections describing psychosexual history. According to State Hospitals’ chief psychologist, the discussion in the sexual history and relationship history sections in these two evaluations did not adequately cover psychosexual history, although they addressed some elements of it. The remaining two evaluations did not contain specific sections on sexual history. State Hospitals’ evaluators also did not always fully document their use of static and dynamic risk assessment instruments, which we describe in the Introduction. A state regulation stipulates that the evaluator, according to his or her professional judgment, must apply tests or instruments along with other static and dynamic risk factors when making the assessment. The chief psychologist told us that State Hospitals has interpreted the regulations to mean that evaluators will apply both a static and a dynamic risk assessment instrument in conducting an SVP evaluation. However, the chief psychologist acknowledged that State Hospitals has not communicated this expectation to evaluators in a written policy. We could find no documentation of the use of a dynamic risk assessment instrument for one of the 23 evaluations we reviewed, and another evaluator used scores from a previous assessment of dynamic risk factors. Further, in four of the 23 evaluations, the evaluators did not include the scoring grids for some or all of the risk assessment instruments the evaluators used, even though the scoring grids allow evaluators to demonstrate how they reached conclusions regarding risks for reoffense. In another instance, the evaluator noted that he included the scoring instruments in an addendum; however, the evaluator did not note that he was referring to an addendum to his previous evaluation of the offender. Further, State Hospitals’ evaluators did not always document State Hospitals’ evaluators did whether they took into consideration any potential barriers to not always document whether communication with the offenders they evaluated. Forensic they took into consideration psychology specialty guidelines from the American Psychological any potential barriers to Association state that when interpreting assessment results, communication with the offenders forensic practitioners consider the purpose of the assessment they evaluated. as well as the various test factors, test‑taking ability, and other characteristics of individuals being assessed that might reduce the accuracy of the evaluators’ interpretations. These communication 4 According to its Web site, the Center for Sex Offender Management is a national clearinghouse and technical assistance center that supports state and local jurisdictions in the effective management of offenders. 24 California State Auditor Report 2014-125 March 2015 barriers could include situational, personal, linguistic, and cultural differences. Corrections uses the Disability and Effective Communication System (DECS)—a statewide disability and effective communication database—as a means of allowing its staff to view disability information and make necessary accommodations for inmates and parolees in parole proceedings. According to the chief psychologist, State Hospitals expects evaluators to review the DECS report and indicate that they examined it in their reports; however, it has not written this expectation into a policy. In eight of 23 evaluations we reviewed, the evaluators did not indicate whether they considered the DECS report, an important component that if not considered could result in an inaccurate conclusion by the evaluator. When evaluators do not consider When evaluators do not consider all relevant information, they all relevant information, they risk risk drawing incorrect conclusions about whether offenders meet drawing incorrect conclusions the SVP criteria. Further, if evaluators neglect to consider an adequate about whether offenders meet the breadth of documentation or demonstrate how they reached their SVP criteria. conclusions, courts may not have accurate and complete information to reach appropriate decisions. Consequently, a court may neglect to commit someone who poses a danger to the public or choose to commit someone who does not need to be in a state hospital. In addition to not considering all relevant information, evaluators also did not always consider all three criteria for determining whether offenders might be recommended for commitment as SVPs. However, this decision created some efficiency. Specifically, the evaluators documented that they did not diagnose a mental disorder and therefore did not evaluate the third criterion for three of the 23 evaluations we reviewed. As the Introduction describes, diagnosing a mental disorder is the second of three criteria that offenders must meet to be considered for commitment as SVPs. In these cases, because the evaluators concluded that the offenders did not meet the second criterion, they chose not to assess the third criterion—whether the diagnosed mental disorder makes the offenders likely to engage in sexually violent, predatory criminal behavior in the future without treatment and custody. According to a staff legal counsel and chief psychologist, State Hospitals has directed evaluators to complete evaluations of all three criteria regardless of the outcome of one, even if that outcome means that the offender will not be considered an SVP. Nevertheless, if an evaluator determines that an offender will not meet the SVP criteria, we believe that stopping the evaluation is both sensible and efficient. State Hospitals’ Standardized Assessment Protocol Is Inadequate The inconsistencies we found in State Hospitals’ evaluations are likely due in part to the fact that its standardized assessment protocol does not provide evaluators with adequate detail and California State Auditor Report 2014-125 25 March 2015 direction on how to perform evaluations. State law requires State Hospitals to conduct its evaluations of potential SVPs in accordance State Hospitals’ existing protocol, with a standardized assessment protocol, which it must develop and which it established in regulation in update in consultation with Corrections. State Hospitals’ existing 2009, states that evaluators must protocol, which it established in regulation in 2009, states that make their assessments by applying evaluators must make their assessments by applying tests or tests or instruments along with instruments along with other static and dynamic factors according other static and dynamic factors to their professional judgment. However, the protocol provides according to their professional little additional detail to assist evaluators on how to perform the judgment; however, it provides assessments. For example, it does not describe specific risk assessment little additional detail to assist approaches or list specific risk assessment instruments evaluators may evaluators on how to perform choose to use, such as the Static‑99R or the Stable 2007. the assessments. State Hospitals’ previous protocol from 2007 was significantly more detailed. It included a discussion of approaches to risk assessment and identified the different types of risk factors and risk assessment instruments evaluators could use. However, in August 2008, State Hospitals revised its protocol and removed this type of detail in response to a ruling by the Office of Administrative Law (OAL) that certain provisions within it should have been adopted in the manner required by the Administrative Procedure Act (Act). Specifically, the OAL ruled that provisions of the protocol that contained instructions to the evaluators on how to conduct evaluations, which questions to ask, and how to submit findings met the definition of regulations; thus, those sections should have been adopted pursuant to the Act. In response, State Hospitals revised its standard assessment protocol, stripping much of its detail, and established it in regulation. However, without specific guidance regarding how to conduct evaluations, evaluators may not perform their work consistently or review all of the appropriate documents, increasing the risk that they will make erroneous assessments. When we discussed adopting a more detailed standard assessment protocol with State Hospitals, a staff legal counsel and the assistant deputy director of State Hospitals’ Forensic Services Division (forensic services) told us that State Hospitals plans to update its assessment protocol by following the Act. State Hospitals Has Provided Evaluators With Limited Supervision, but Its New Quality Assurance Team Is Taking Some Steps to Improve Quality Control Another likely cause of State Hospitals’ inconsistent evaluations is the limited supervision it has provided to its evaluators. Given its recent hiring of evaluators and the highly specialized nature of evaluating current and potential SVPs, we would expect State Hospitals to have established quality control measures, such as 26 California State Auditor Report 2014-125 March 2015 supervisory reviews, to ensure that evaluators complete adequate and consistent evaluations. However, none of State Hospitals’ reviews of SVP evaluations focus on ensuring the quality of the evaluations from a clinical perspective. According to State Hospitals’ chief psychologist, in October 2013 State Hospitals established a quality assurance team to improve opportunities for mentoring newer evaluators. Nevertheless, the quality assurance team’s role is advisory, not supervisory. Further, at Coalinga—where evaluators generally focus on conducting annual evaluations of offenders who are already committed as SVPs—managers told us that evaluators should receive several levels of supervisory review. However, because Coalinga does not require supervisors to maintain a formal record of the reviews, it cannot demonstrate that they occur. Although evaluations of potential SVPs completed at State Hospitals’ headquarters undergo several levels of review, no level of review assesses the appropriateness of clinical conclusions the evaluators draw. According to State Hospitals’ chief psychologist, a case manager performs a nonclinical review and checks evaluations for grammar and stylistic errors. A staff legal counsel at State Hospitals informed us that legal counsel also reviews certain evaluations: those that recommend commitment and those in which the two initial evaluators disagree about commitment and the two independent professionals who subsequently assess the offender also disagree. Further, she stated that legal counsel began reviewing update evaluations—evaluations that update information for prior evaluations—in January 2015. The staff legal counsel stated that the legal office checks for logic errors, continuity of thought, and consistent reasoning. The legal office also considers whether evaluations respond to the criteria necessary for legal commitment. However, it does not provide any analysis of whether evaluators correctly performed clinical elements, such as using a risk assessment instrument. If an evaluation lacks strong clinical According to State Hospitals’ chief psychologist, he signs off on elements, it may face scrutiny in some SVP evaluations, such as instances when a case manager court due to poor quality of the or State Hospitals’ legal services request a review. However, his evaluator’s analysis; ultimately, it reviews are more cursory than substantive. They critique the may fail to adequately demonstrate structure of an evaluation and ensure that its legal argument is the need to commit or not commit sound but, like the legal office’s reviews, they do not consider the an offender. quality of the clinical elements of the evaluation. If an evaluation lacks strong clinical elements, it may face scrutiny in court due to poor quality of the evaluator’s analysis; ultimately, it may fail to adequately demonstrate the need to commit or not commit an offender. California State Auditor Report 2014-125 27 March 2015 State Hospitals’ headquarters currently lacks the supervisory As of December 2014, 45 employee structure necessary to perform clinical reviews of evaluations. evaluators at State Hospitals’ As of December 2014, 45 employee evaluators at State Hospitals’ headquarters reported to the chief headquarters reported to the chief psychologist, who holds the psychologist, who holds the only only supervisory position. The chief psychologist stated that he supervisory position. is also responsible for overseeing the contract evaluators who perform SVP evaluations, and he expects there to be approximately 20 contractors for 2015. He also said that the clinical staff would like to expand case file reviews to ensure that the evaluators properly complete evaluations. However, he stated this would require an expansion of the quality assurance team and he estimates that he would need four or five veteran clinical staff on the team to fully run quality assurance of SVP evaluations. He told us that he requested the creation of such positions from the administration of forensic services in spring 2014, but the administration only approved an additional chief psychologist position in December 2014. He said that the process to fill this position will likely take several months. Although State Hospitals does not have a process to perform a clinical supervisory review of evaluations, it has taken some steps to improve the quality of the evaluations. According to the chief psychologist, he established the quality assurance team shortly after joining State Hospitals in October 2013. Led by the chief psychologist and two field trainers who are veterans in the area of psychological evaluations, the team provides guidance to State Hospitals’ consulting psychologists—a classification of evaluator that requires less experience—and assists in the development and implementation of State Hospitals’ training plan. Although the team does not perform supervisory reviews, they are available to provide feedback to evaluators on their SVP evaluations upon the request of the evaluator, State Hospitals’ management, or legal counsel. State Hospitals’ chief psychologist provided an example of an instance in which he stated an evaluator requested that a quality assurance team member review a draft evaluation prior to its completion. In the review, the quality assurance team member noted multiple instances in which the evaluator could improve the quality of the writing and the clarity of the conclusion, and he also noted places where the evaluator neglected to include necessary information. The quality assurance team is also responsible for State Hospitals’ mentorship program for new evaluators. According to the chief psychologist, in August 2014 State Hospitals started a mentorship program for new SVP evaluators to shadow more‑experienced evaluators, obtain feedback on evaluations, and receive assistance and training in courtroom testimony. State Hospitals has two classifications at headquarters responsible for conducting SVP evaluations: sexually violent predator evaluators and consulting psychologists. We discuss these classifications in more detail in Chapter 2. State Hospitals designed the mentorship program to 28 California State Auditor Report 2014-125 March 2015 assist only the consulting psychologists, because the minimum qualifications for this position do not require prior experience in the risk assessment and diagnosis of SVPs or an equivalent class of offenders. The quality assurance team is responsible for determining when new evaluators participating in the mentorship program are ready to complete evaluations on their own; according to the chief psychologist, the program usually lasts from a year to a year and a half. The chief psychologist stated that 12 consulting psychologists were participating in the mentorship program as of October 2014. The chief psychologist said he would like to expand the supervisory review function and to continue the work of the quality assurance team; however, he acknowledges that he would need additional staff to do so. Without adequate supervision and review, State Hospitals’ evaluators may not complete evaluations effectively, increasing the risk that the evaluators either will not identify offenders who meet the definition of an SVP or will erroneously conclude they do. In contrast to State Hospitals’ In contrast to State Hospitals’ process at headquarters, the process at headquarters, the managers at Coalinga told us that multiple levels of review managers at Coalinga told us that occur for the annual evaluations its evaluators perform of SVPs. multiple levels of review occur However, Coalinga has not established a process to document for the annual evaluations its these reviews. According to the acting chief of forensic services evaluators perform of SVPs, but (forensics) at Coalinga, the forensic senior psychologist supervisor there is no evidence kept of their reviews each evaluation and identifies specific problems, such as occurrence. missing risk factors, which she communicates to the evaluators. In addition, the acting chief of forensics stated that he conducts a quick read‑through of the evaluations to ensure that they make sense and are convincing. Further, he said that an analyst reviews the evaluations for grammar, spelling, and punctuation errors. However, he also told us that Coalinga does not document these various reviews. According to Coalinga’s medical director, he signs the letters that transmit evaluations to the courts. The medical director stated that he reviews the evaluations at this time to ensure that they contain correct grammar and sentence structure and that the content supports the evaluators’ recommendations. The medical director said that he does not use a checklist or follow any other guidance when reviewing these evaluations, and the only documentation to show his review is his signature on the letters to the courts. In the event that the medical director disagrees with an evaluation, he asks the evaluator to consider a modification, paying particular attention to the issue he deems to be important. If the evaluator declines to reconsider, the medical director submits a cover letter in disagreement with the evaluator’s opinion. California State Auditor Report 2014-125 29 March 2015 Coalinga has a tool for documenting supervisory review but has not formally adopted its use. In 2012 Coalinga created a checklist to use for reviewing annual evaluations to ensure that they were accurate and complete. According to Coalinga’s acting chief of forensics, he uses the checklist as a reference document when performing his reviews. Coalinga’s forensic senior psychologist supervisor, on the other hand, stated that she does not use it. Coalinga’s acting chief of forensics explained that he does not want to formalize the use of the checklist because courts could request checklists, and any errors or omissions might diminish the strength of the evaluations. However, we disagree with this reasoning since the reviews are performed before Coalinga finalizes its evaluations, so the evaluators would have a chance to correct errors. Further, using a formal checklist would assist both supervisors and evaluators in documenting that the evaluations are completed consistently and according to Coalinga’s expectations. If State Hospitals Increased Its Tracking of Court Data, It Could Strengthen Its Evaluation Process State Hospitals could better use data related to court outcomes to identify areas for strengthening its evaluations. According to its Web site, one of State Hospitals’ goals is excellence in forensic evaluation. As part of its effort to ensure that it meets this goal, we would expect it to examine the usefulness of its evaluations to those who use them—specifically, courts considering whether offenders meet the SVP criteria and should be committed. However, State Hospitals has not analyzed according to the chief psychologist, State Hospitals has not data regarding the disposition of analyzed data regarding the disposition of its cases or established its cases or established benchmarks benchmarks to evaluate the rate at which courts agree or disagree to evaluate the rate at which courts with evaluators. agree or disagree with evaluators. Courts do not always agree with State Hospitals’ evaluators regarding whether offenders should be committed as SVPs. Under state law, State Hospitals’ evaluators report on whether they believe offenders meet the SVP criteria based on their evaluations. However, a court or jury ultimately decides whether an offender will be committed to a state hospital. According to State Hospitals’ assistant deputy director of forensic services, courts periodically disagree with its evaluators’ findings. In our review of 23 evaluations, we noted one instance in which evaluators determined that an offender met the criteria as an SVP, yet the courts chose to release the offender. Given that the courts have the final say on whether offenders are SVPs, we would expect State Hospitals to gather and analyze data on court outcomes so that it can identify potential weaknesses in its processes for conducting evaluations. 30 California State Auditor Report 2014-125 March 2015 However, State Hospitals has not consistently tracked the disposition of these court cases. The assistant deputy director of forensic services explained that before November 2014, State Hospitals’ case managers tracked the disposition of court cases regarding SVP commitment by attempting to follow up with the courts and district attorneys directly. However, she indicated that this process was not always successful because court hearings were sometimes postponed or cancelled, so keeping up with the rescheduling of cases throughout the State demanded significant resources. She also stated that some counties were responsive to State Hospitals’ requests for case outcomes and frequently reported their data, whereas counties with larger caseloads were often not as responsive. Moreover, she stated that when evaluators were primarily contractors, the courts sent notices to testify on their findings to the evaluators directly. Consequently, State Hospitals was often not aware of the court schedules, making it more difficult to consistently follow up on the outcome of cases. Because State Hospitals does Because State Hospitals does not consistently track the courts’ not consistently track the courts’ dispositions for its SVP cases, it is missing an opportunity to gain dispositions for its SVP cases, it is data that could improve its evaluation process and inform its missing an opportunity to gain data training and supervision of its evaluators. State Hospitals’ chief that could improve its evaluation psychologist agreed that analyzing the dispositions of cases could process and inform its training and be beneficial. He stated that if, for example, the courts routinely supervision of its evaluators. reach conclusions that oppose evaluators’ findings, State Hospitals could try to determine the cause, such as poor report writing or testimonial skills. It could then use this analysis to focus its training to best enhance evaluators’ skills. We believe that collecting and analyzing such data would be an important element in its efforts to ensure high‑quality evaluations. State Hospitals recently changed its approach to tracking case outcomes. According to the assistant deputy director, the courts mail State Hospitals the notices for evaluators to appear in court now that most of the evaluators are employees and not contractors. As a result, case managers can better track the progress of cases through court and follow up with the district attorneys or courts for outcome results. Further, in November 2014, after our audit began, forensic services revised its process for its court scheduling. The revised process acknowledged that the previous court scheduling process was convoluted, confusing, and inefficient, with multiple instances of miscommunication between court officials, headquarters staff, and evaluators. Under the revised process, policy support staff are responsible for verifying which evaluators courts subpoena, inputting the relevant case data into the case management system, tracking key dates, and running reports for data and research. Policy support unit staff are also responsible for tracking initial and updated court appearance dates for evaluators. This new process may help State Hospitals California State Auditor Report 2014-125 31 March 2015 better track case outcomes, compile data, and evaluate trends in court decisions. However, given that State Hospitals only recently made these revisions, it is too soon to conclude that its process is effective. Recommendations Legislature To promote efficiency, the Legislature should change state law to allow State Hospitals the flexibility to stop an evaluation once the evaluator determines that the offender does not meet one of the SVP criteria. State Hospitals To improve the consistency of its evaluations, by June 2015, State Hospitals should create a written policy that requires its evaluators to include the following documentation in their evaluations: • Detail describing all the documentation they reviewed. • The offender’s psychosexual history. • A description of the risk assessment instruments the evaluator used and the scoring tool for those risk assessments. • Acknowledgement of the evaluator’s review of the DECS report. To promote consistency and ensure that it provides sufficient guidance to evaluators, State Hospitals should update its assessment protocol by March 2016 to include more specific instructions on how to conduct evaluations, such as what assessment instruments evaluators may use and what documents they should consider. State Hospitals should also develop a timeline for periodically reviewing and making any necessary updates to the assessment protocol. To comply with state law, State Hospitals should ensure that it follows the Administrative Procedures Act for future changes to its standardized assessment protocol. To improve the consistency and completeness of its evaluations, by December 2015 State Hospitals should develop a plan for the formal, supervisory review of evaluations from a clinical perspective that balances the needs of the program with its resource limitations. For example, rather than attempting to review 32 California State Auditor Report 2014-125 March 2015 every evaluation, State Hospitals could focus its review efforts on those evaluations most at risk of error or inconsistency, such as those completed by the newest evaluators. If State Hospitals adopts this or a similar approach, it should review the remaining evaluations on a sample basis. To ensure that it can demonstrate the consistency of Coalinga’s supervisory review of annual evaluations, by June 2015 State Hospitals should direct Coalinga to formally adopt its checklist for reviewing evaluations, provide the checklist to its evaluators, and include the checklist as part of its evaluation process. State Hospitals should also develop a checklist for the evaluations it performs at its headquarters and adopt it as part of its standardized assessment protocol by March 2016. To ensure that it has the data necessary to inform its training and supervision of evaluators, State Hospitals should identify the most efficient means for obtaining the outcomes of past trials—at least the outcomes of three years of past trials if possible—and should ensure that it includes such outcomes in its database by March 2016. Additionally, by June 2015 it should establish procedures to ensure that it promptly collects the outcomes from current and future trials. Finally, State Hospitals should develop procedures to analyze these data at least twice annually to identify any trends in cases in which the courts’ determinations differed from the State Hospitals evaluators’ recommendations. It should use this information to provide training and supervision where they are most needed. California State Auditor Report 2014-125 33 March 2015 Chapter 2 THE CALIFORNIA DEPARTMENT OF STATE HOSPITALS HAS NOT PROVIDED CONSISTENT TRAINING TO ENSURE ITS EVALUATORS PRODUCE THOROUGH EVALUATIONS Chapter Summary The California Department of State Hospitals (State Hospitals) has not consistently offered training to the evaluators who assess sex offenders (offenders) to determine whether the State should recommend committing them as sexually violent predators (SVPs). Although in 2009 and 2010 State Hospitals offered training on a wide variety of topics to evaluators, it did not provide them with any training between August 2012 and May 2014, a period during which it hired many new employees. Also, State Hospitals developed a training plan that began with comprehensive training for its evaluators in May 2014; however, it has yet to implement most of the plan. Consequently, until recently State Hospitals did not provide many evaluators with training on critical risk assessment tools. Ongoing training is important to ensure the competence of those conducting evaluations of current and potential SVPs. Further, State Hospitals can improve its tracking of training records. If it cannot demonstrate that its evaluators received the required training, State Hospitals might compromise the integrity of evaluations. We also noted additional areas in which State Hospitals could improve its evaluation process. Specifically, it has not documented its efforts to verify that its evaluators met the experience portion of the minimum qualifications for their positions. Further, it has only recently begun to analyze the trends in the rate at which its evaluators determine offenders meet the criteria as SVPs— what it refers to as its positive rate. In addition, in March 2013, State Hospitals developed a process for assigning and tracking the workload of its evaluators; however, evaluators expressed concerns about this process, which led to State Hospitals revising it in January 2015. Although the revised process addresses some concerns, it omits other elements. Finally, State Hospitals needs to address its backlog of annual evaluations of currently committed SVPs at Coalinga State Hospital (Coalinga). If Coalinga fails to promptly perform these evaluations, it is not fulfilling one of its critical statutory obligations, leaving the State unable to report on whether SVPs continue to pose risks to the public. 34 California State Auditor Report 2014-125 March 2015 State Hospitals Has Not Consistently Offered Training to Its SVP Evaluators State Hospitals has been inconsistent in offering training to its SVP evaluators. During 2009 and 2010, it offered training to SVP evaluators on a wide variety of topics. In anticipation of hiring employee evaluators during 2011 and 2012, State Hospitals decided to develop and implement in‑house training. This change in training approach had a greater focus on the basic principles of SVP evaluations. Nevertheless, State Hospitals did not provide any training to its evaluators between August 2012 and May 2014. Further, it did not provide current evaluators at Coalinga with any training related to performing evaluations. New management at State Hospitals has acknowledged the need for more training and recently began increasing its training efforts. Ongoing training is important to ensure the competence of those conducting evaluations of potential and current SVPs. According to state law, only practicing psychologists and psychiatrists can perform evaluations of potential SVPs. In addition, according to the American Psychological Association’s specialty guidelines for forensic psychology adopted in 2011, competence in forensic psychology can be acquired through a combination of education, training, supervised experience, and study, among other things. The specialty guidelines recommend that forensic practitioners make ongoing efforts to develop and maintain their competencies and keep abreast of developments in the fields of psychology and the law. State Hospitals offered a wide State Hospitals offered a wide range of training to SVP evaluators range of training to SVP evaluators in 2009 and 2010. Specifically, State Hospitals offered its evaluators— in 2009 and 2010, and in 2011 State at that time, mostly contractors—training on a variety of topics, Hospitals implemented its own including sex offender risk assessment tools, statistics on sexual training in-house for conducting recidivism, the effect of aging on recidivism, and the violence‑risk SVP evaluations. scale. According to the assistant deputy director of State Hospitals’ Forensic Services Division (forensic services), State Hospitals funded the cost of the trainings throughout the State, and the contract evaluators paid for their travel and expenses. However, in anticipation of hiring evaluators as employees, in 2011 State Hospitals implemented its own training in‑house for conducting SVP evaluations. Specifically, according to the assistant deputy director of forensic services, the previous acting clinical director developed training for its SVP evaluators in consultation with State Hospitals’ legal division. She stated that State Hospitals developed its own training program to benefit both the experienced contract evaluators transitioning into state service and new evaluators with less experience in forensic evaluation and court testimony. State Hospitals offered trainings during September 2011 and July 2012 on topics such as the Static‑99 California State Auditor Report 2014-125 35 March 2015 assessment (described in the Introduction), forensic report writing, legal changes that impact evaluations, and expert witness testimony preparation. From August 2012 to May 2014, however, State Hospitals’ training From August 2012 to May 2014, documentation shows that it did not provide any training for SVP State Hospitals did not provide any evaluators who worked in its headquarters. This nearly two‑year training for SVP evaluators who gap may in part be the result of a staffing issue: The clinical director worked in its headquarters. responsible for developing and implementing the training retired in April 2013. According to the assistant deputy director of forensic services, State Hospitals did not hire another clinical director— tasked in part with developing a training plan—until October 2013. Further, according to the former forensic senior psychologist supervisor at Coalinga, State Hospitals provided evaluators at Coalinga with fewer training opportunities in recent years. Specifically, he stated that before 2011, evaluators at Coalinga trained with State Hospitals’ evaluators on the Static‑99 and dynamic risk assessment tools. However, according to the former forensic senior psychologist supervisor, in 2011 Coalinga evaluators were training with State Hospitals less frequently, and by 2012 they were not part of the training offered by State Hospitals. When we asked administrators at State Hospitals and Coalinga why their evaluators stopped training together, they were unable to provide an explanation. According to Coalinga’s current forensic senior psychologist supervisor, Coalinga did not offer consistent training for its evaluators. The forensic senior psychologist supervisor stated that the lack of consistent training resulted in the evaluators producing inconsistent and at times inadequate evaluations. She said that if less‑experienced evaluators do not receive consistent training in forensic evaluations to complement their on‑the‑job experience, they cannot adequately perform their jobs and are ineffective witnesses in court. State Hospitals has recently begun taking some initial steps to implement more robust training for its evaluators at headquarters. In 2014 State Hospitals’ chief psychologist and the quality assurance and training team (quality assurance team) developed a training plan for evaluators at headquarters. Specifically, in May 2014, State Hospitals offered a comprehensive SVP training for all consulting psychologists—who, as of December 2014, represent 33 of 45 evaluators on staff at State Hospitals—on the background of the SVP statutes, the various criteria under which State Hospitals evaluates potential SVPs, and the Static‑99R risk assessment. As we discuss later in the chapter, consulting psychologists belong to one of two civil service classifications conducting evaluations at State Hospitals’ headquarters. The other classification, which requires more experience, is a sexually violent predator evaluator (SVPE). 36 California State Auditor Report 2014-125 March 2015 In December 2014 and January 2015, State Hospitals also held training about a dynamic risk assessment instrument for the current evaluators working at its headquarters. In addition, the tentative training plan included expected courses on the Stable 2007 dynamic risk assessment instrument, court testimony, and updates to risk assessment instruments. However, according to the chief psychologist, as of January 2015 State Hospitals had not scheduled any of these additional trainings. According to the chief psychologist, a second aspect of State Hospitals’ new training effort includes a mentorship program for evaluators at headquarters. The program began in August 2014 and is designed to help new consulting psychologists develop their evaluation skills by shadowing experienced evaluators. For the first year to year and a half, new evaluators will progress from performing smaller tasks to drafting written evaluations and preparing court testimony, while receiving constructive feedback from their mentors. State Hospitals’ goal is to foster the new evaluators’ development and help experienced evaluators—in their role as mentors—refine their basic skills. Coalinga’s evaluators have Coalinga’s evaluators have continued to receive fewer training continued to receive fewer training opportunities than the evaluators at headquarters. According opportunities than the evaluators to Coalinga’s forensic senior psychologist supervisor, Coalinga’s at headquarters. evaluators do not participate in State Hospitals’ mentorship program. Coalinga’s forensic senior psychologist supervisor designed a training plan for fiscal year 2014–15 to help new evaluators at the hospital develop a basic understanding of state law affecting forensic evaluations, forensic report writing, and risk assessment. She indicated that Coalinga is also in the process of developing an ongoing training plan for experienced evaluators and has some trainings scheduled for 2015. Until Recently, State Hospitals Did Not Provide Training on Dynamic Risk Assessment Instruments Compounding the inconsistent training offered to evaluators, until recently State Hospitals has not provided training on dynamic risk assessment instruments. A dynamic risk assessment instrument may consider both stable factors, such as personality disorders or sexual preference, that help predict long‑term risk; and acute, rapidly changing factors, such as a negative mood or intoxication, that signal the potential for reoffense. Although state regulation requires evaluators to apply tests or instruments and consider stable and dynamic risk factors when performing forensic evaluations, State Hospitals could not demonstrate until recently that it provided training to its evaluators on these instruments. The assistant deputy director of forensic services told us that California State Auditor Report 2014-125 37 March 2015 State Hospitals focused its training efforts during 2011 and 2012 on certain aspects of forensic evaluations, such as examining clinical and static risk factors, rather than on dynamic factors. Recently, in December 2014 and January 2015, State Hospitals provided training on dynamic risk assessment instruments. Until December 2014 State Hospitals did not provide training on Until December 2014 State how to complete dynamic risk assessment instruments. In the Hospitals did not provide training two trainings on forensic assessment in 2012, State Hospitals’ on how to complete dynamic risk instructors provided high‑level overviews of dynamic risk factors assessment instruments. but did not provide instructions on how to use specific assessment instruments, such as the Stable 2007. Our review of training materials from January 2009 through November 2014 found no other instances in which trainings addressed dynamic risk assessment instruments, even though State Hospitals was hiring evaluators from 2012 through 2014 to replace the contractors who had been performing the evaluations of potential SVPs. Although some contract evaluators who later became employees may have obtained training on dynamic risk assessment instruments on their own, new evaluators with little or no forensic experience were not provided training from State Hospitals on how to perform dynamic risk assessments. According to the assistant deputy director of forensic services, the previous clinical psychiatrist viewed certain aspects of dynamic risk assessment to be less critical to reaching conclusions during forensic evaluations. However, the chief psychologist, who started in October 2013, stated that State Hospitals acknowledged the importance of assessing dynamic risk factors when performing evaluations. State Hospitals’ chief psychologist explained that State Hospitals’ position is that sufficient evaluation of a potential SVP includes an assessment of both static and dynamic factors and that a dynamic risk assessment tool strengthens an evaluation by providing a higher degree of certainty when estimating the risk of a reoffense. Nevertheless, as of late August 2014, he estimated that about 75 percent of the consulting psychologists—one of the two civil service classifications conducting SVP evaluations at State Hospitals’ headquarters had not had adequate, updated, or any training in dynamic risk assessment instruments and variables. According to the assistant deputy director of forensic services, several consulting psychologists attended trainings on the Stable 2007 dynamic risk assessment instrument offered by external trainers between February and May 2014. Moreover, although the chief psychologist stated that he thought the SVPEs had received training on dynamic risk assessment tools, he also told us that State Hospitals has not offered update trainings to keep its evaluators current on the possible changes and new research in the field regarding the instruments. 38 California State Auditor Report 2014-125 March 2015 To ensure that its evaluators have adequate training related to dynamic risk assessment tools and other areas of knowledge specific to evaluating potential and current SVPs, State Hospitals must improve its tracking of training. According to the assistant deputy director of forensic services, State Hospitals did not track the training taken by each evaluator before 2011 because the majority of its evaluators were contractors who were experts in their field and whom it expected to stay current on training. She stated that in 2013 State Hospitals began tracking all of its evaluators’ training to ensure they were meeting the continuing education requirements necessary to maintain their licenses. However, State Hospitals’ tracking does not include an analysis of the specific types of training evaluators receive. For example, state law currently requires evaluators to use the Static‑99R to evaluate male SVPs. Therefore, evaluators must be trained on the Static‑99R instrument so they can properly use it when performing evaluations. However, because State Hospitals If the evaluators are not properly does not analyze the type of training its evaluators receive, it cannot trained on the instruments they use, demonstrate that its evaluators received the required training. If it may compromise the integrity the evaluators are not properly trained on the instruments they use, of their evaluations and result in it may compromise the integrity of their evaluations and result in challenges to their findings. challenges to their findings. Training ensures that evaluators possess the latest and best information. State Hospitals’ chief psychologist stated that to be effective, forensic evaluators must receive training every one to two years because of advances in the field of forensic psychology, changes to the way evaluations are conducted, and changes in case law that impacts evaluations, among other reasons. Without adequate training in dynamic risk assessment instruments, evaluators may use them incorrectly, increasing the likelihood of errors in estimating the risk of reoffense. In addition, evaluators who forgo tests of dynamic risk factors because they were not adequately trained may compromise their ability to fully support and defend their findings in court. State Hospitals Has Not Documented Its Efforts to Verify Its Evaluators’ Qualifications State Hospitals uses employees and some contractors to conduct evaluations of potential and current SVPs. Table 4 summarizes the minimum qualifications of the four employee classifications as well as the contractors that conduct the various evaluations at State Hospitals’ headquarters and at Coalinga. As Table 4 shows, only the SVPE position has comparable minimum qualifications to those state law requires for the contractors State Hospitals hires to complete difference‑of‑opinion evaluations. The other positions—consulting psychologist, senior psychologist specialist, and psychologist—require less experience and therefore receive less compensation and have lighter workloads. California State Auditor Report 2014-125 39 March 2015 Table 4 California Department of State Hospitals’ Evaluator Classification, Type of Evaluations Performed, Minimum Qualifications, and Number as of December 2014 NUMBER AS OF TYPES OF EVALUATIONS DECEMBER CLASSIFICATION PERFORMED LICENSE REQUIREMENT* EDUCATION* EXPERIENCE 2014 Contractor, Difference-of-opinion Valid license as a Doctorate degree Five years postdoctoral, postinternship 35 State Hospitals Initial evaluations, psychologist issued by (implied by license experience as a licensed psychologist in update evaluations, the California Board requirement) the practice of psychological evaluation initial evaluations of Psychology and risk assessment and diagnoses of sexually violent predators (SVPs). Sexually Violent Initial evaluations, Valid license as a Doctorate degree in Five years postdoctoral, postinternship 12 Predator update evaluations, psychologist issued by psychology experience as a licensed psychologist in Evaluator (SVPE) replacement the California Board the practice of psychological evaluation evaluations, of Psychology and risk assessment and diagnoses of recommitment initial SVPs or equivalent class of sex offenders evaluations (offenders). The SVPEs must have 40 hours of expert witness testimony in high-risk offender cases or SPV cases. Consulting Initial evaluations, Valid license as a Doctorate degree Two years experience in California 33 Psychologist clinical screens, psychologist issued by in psychology state civil service performing clinical update evaluations the California Board psychology duties equivalent to of Psychology those of a psychologist or clinical psychologist; or Three years of full-time postdoctoral, postinternship experience in the practice of psychology involving either training, research, consultation, or program planning in mental health services. Contractor, Annual evaluations Valid license as a Doctorate degree Five years post-licensure 7 Coalinga State psychologist (implied by license experience primarily conducting Hospital (Coalinga)† (implied by experience requirement) forensic evaluations.‡ requirement)‡ Psychologist Annual evaluations Valid license as a Completion of None. 2 (Health Facility— psychologist issued by requirements for a Clinical-Safety) the California Board doctorate degree of Psychology with specialization in clinical or child clinical psychology Senior Psychologist Annual evaluations Valid license as a Doctorate degree in One year experience in California state 2 Specialist psychologist issued by psychology service performing the duties of a the California Board psychologist (health facility) or staff of Psychology psychologist (any specialty); or Two years postdoctoral, postinternship experience in the practice of psychology involving assessment and treatment and either training, research, consultation, or program planning in mental health services. Sources: Classification bulletins from the California Department of Human Resources; California Welfare and Institutions Code, Section 6600 et. seq.; invitations for bid from the California Department of State Hospitals (State Hospitals); and employment files at State Hospitals and Coalinga, as well as interviews with the forensic senior psychologist supervisor and the acting chief of forensic services at Coalinga. * State law allows psychiatrists to conduct evaluations as well. However, the only psychiatrist conducting evaluations as of December 2014 was a contractor at Coalinga who possessed a license to practice medicine. † Coalinga contracts with professional registries that employ individuals that it uses to evaluate SVPs. ‡ During our audit period, Coalinga did not have a formal description of its minimum qualifications for its contract evaluators. According to the acting chief of forensic services, he expects contract evaluators to have five years of experience post-licensure. Starting September 2014, Coalinga contracts specify minimum qualifications for its contract evaluators that are in line with what is noted in the table. 40 California State Auditor Report 2014-125 March 2015 State Hospitals demonstrated that it verified that its evaluators met some of their positions’ minimum qualifications. We reviewed the files of 15 current evaluators at both headquarters and Coalinga— nine employees and six contractors—to determine whether State Hospitals verified that they met the minimum qualifications for their positions. Each file we reviewed contained a copy of the evaluator’s license to practice psychology—or medicine, in the case of the one psychiatrist—which we also independently verified through the State’s licensing boards. According to the California Board of Psychology, a doctorate degree is necessary for licensure as a psychologist in California. Similarly, the California Medical Board requires a doctorate degree for licensure as a psychiatrist. Therefore, although the files we reviewed did not contain evidence of doctorate degrees, the evaluators’ possession of valid licenses demonstrates that they have such degrees. However, State Hospitals could not demonstrate whether its evaluators met the experience portions of their positions’ minimum qualifications. Although it retained job applications and other information the applicants submitted, it did not document that it had verified the information related to experience, for example, by contacting references or past employers. We reviewed 15 evaluators’ personnel files—nine employees and six contractors. According to information the individuals submitted to State Hospitals, each of the six contractors had more than 10 years of experience as a licensed psychologist or psychiatrist, with four having 25 or more years experience. In contrast, only four of the nine employees had 10 years of experience or more. For 12 files we reviewed at State However, for 12 files we reviewed at State Hospitals’ headquarters, we Hospitals’ headquarters, we did not did not find any documentation that State Hospitals verified employees’ find any documentation that State or contractors’ experience. For two of the three files we reviewed at Hospitals verified employees’ or Coalinga, there was a checklist that included a section for contacting contractors’ experience. past employers. In one instance, notes in the checklist indicated that Coalinga sent letters to past employers but did not indicate whether it received answers to the letters. In the second instance, the section on the checklist was blank. Without a formal process for documenting that they verify required experience, neither State Hospitals nor Coalinga can demonstrate that it has ensured that individuals hired meet the minimum qualifications for their positions. State Hospitals Is Starting to Develop a Key Measurement for Assessing Evaluator Performance The Joint Legislative Audit Committee (audit committee) asked us to report on the number of offenders State Hospitals determined to be positive and negative for commitment. Table 5 presents the number of offender cases evaluated by State Hospitals’ evaluators and the outcome of those evaluations for fiscal years 2009–10 through 2013–14. As the table notes, during that five‑year period the rate by California State Auditor Report 2014-125 41 March 2015 which State Hospitals determined that offenders met the criteria to be an SVP remained below 8 percent. These data provide an overall picture of the number of offender cases State Hospitals determined met the criteria for commitment.5 Table 5 Final Case Outcomes of the Evaluation of Offenders Based on the California Department of State Hospitals’ Clinical Evaluations of Potential Sexually Violent Predators Fiscal Years 2009–10 Through 2013–14 TOTAL CASES CASES FOUND CASES FOUND PERCENTAGE OF CASES EVALUATED THAT POSITIVE FOR NEGATIVE FOR TOTAL CASES FOUND POSITIVE FOR INCLUDED A DIFFERENCE FISCAL YEAR COMMITMENT* COMMITMENT* EVALUATED COMMITMENT OF OPINION† 2009–10 63 1,066 1,129 5.6% 108 2010–11 122 2,014 2,136 5.7 154 2011–12 105 1,216 1,321 7.9 131 2012–13 43 791 834 5.2 78 2013–14 31 730 761 4.1 52 Source: California State Auditor’s analysis of data obtained from the California Department of State Hospitals’ (State Hospitals) Sex Offender Commitment Program Support System. * When the required number of evaluators agree that an individual meets the criteria as a sexually violent predator (SVP), State Hospitals recommends to the designated counsel of the county where the offender was convicted that the State commit the individual to a state hospital. † These cases are a subset of the total cases evaluated. State law requires that, if the first two evaluators do not agree that an individual meets the criteria of an SVP, two additional contract evaluators will conduct an evaluation. The two additional evaluators must both agree that an individual meets the criteria of an SVP for State Hospitals to recommend commitment. However, it is also important to track the rate by which individual evaluators determine that an offender meets the criteria as an SVP. Although State Hospitals has accumulated data on its evaluations for several years, it has only recently begun analyzing those data. State Hospitals’ administrators acknowledged the value of determining the rate at which State Hospitals’ evaluators initially determine that offenders meet the SVP criteria—the positive rate— and of identifying evaluators whose positive rates are unusually high or low. Specifically, the chief psychologist stated that an evaluator with a consistently low positive rate should warrant attention. Nevertheless, State Hospitals has not yet performed a comprehensive analysis to determine what constitutes a valid positive range. The chief psychologist noted that State Hospitals has performed 32,282 initial evaluations since 1996, and that 22 percent of these were positive. However, he stated that positive rates should 5 In addition to offenders, the audit committee also asked us to report on the number of SVPs found positive and negative for commitment. We report on evaluators’ conclusions regarding SVPs in the Appendix. 42 California State Auditor Report 2014-125 March 2015 be between 8 percent and 18 percent based on State Hospitals’ current analysis of the data. Our analysis of the data related to initial evaluations for the three years beginning in fiscal year 2011–12 shows that the positive rate of State Hospitals’ evaluators is 9.2 percent. The rate may be lower in recent years because of the impact of Jessica’s Law, which reduced the number of offenses needed to qualify as an SVP from two to one and increased the number of crimes considered qualifying offenses. These changes Because the rate we calculated essentially made it more likely for an offender to be evaluated for is at the bottom of the range commitment, but not necessarily meet all the criteria of an SVP. State Hospitals identified, we are Because the rate we calculated is at the bottom of the range State concerned that State Hospitals’ Hospitals identified, we are concerned that State Hospitals’ range range may not be appropriate. may not be appropriate. Further, the chief psychologist stated that in April 2014 State Hospitals began identifying consulting psychologists who have conducted a minimum number of evaluations yet have rarely concluded that offenders met the SVP criteria. Based on its analysis, the chief psychologist told us that State Hospitals identified 12 consulting psychologist evaluators who concluded that offenders met the SVP criteria less than 7 percent of the time. Six of these evaluators have never produced a positive evaluation. He also indicated that as of January 2015, several newly hired evaluators continue to produce very low positive rates that range from 0 percent to 3 percent. State Hospitals is reviewing these evaluators’ evaluations and working with them to see whether there are ways to improve the evaluation process. However, the chief psychologist indicated that retraining or mentoring for this small group of psychologists has not yet been determined. Without an appropriate range for positive evaluations, State Hospitals risks that it will not be able to adequately identify those evaluators whose positive rates deviate from the norm. State Hospitals Recently Made Changes to Its Contract Practices That Reduce Incentives for Negative Evaluations State Hospitals’ past practice of setting no minimum amount that contractors must bid to complete evaluation services may have created an incentive for evaluators to write negative evaluations. However, State Hospitals recently made changes to eliminate that possible incentive. The audit committee asked us to review whether State Hospitals’ policies or procedures provided any monetary or workload incentives to evaluators. In our review, we noted that the invitation for bids of evaluators to perform SVP evaluations may have created an incentive for some contract evaluators to write negative evaluations. Specifically, although State Hospitals California State Auditor Report 2014-125 43 March 2015 established maximum limits for contract evaluators’ bids of services, it did not set a minimum bid amount in fiscal years 2009–10 through 2010–11 for the various services contract evaluators provide—including those associated with appearing as a witness at court trials to determine whether an offender should be committed as an SVP. As a result, possibly to be more competitive on price, two of 68 contractors bid zero on court testimony time and another bid zero on court travel time and court wait time. An additional seven evaluators bid significantly below the average cost on those activities as well. Having the ability to submit zero or lower‑than‑average bids for court‑related costs may have knowingly or unknowingly created an incentive to determine that offenders did not meet the SVP criteria, because doing so decreased the likelihood that the evaluators would spend time and incur costs on court‑related activities. Furthermore, State Hospitals assigned contract evaluators up to six evaluations at a time, and as they finished evaluations, it could assign them more. Therefore, contractor evaluators who did not have to go to court or who sped through evaluations would have had more time to perform additional evaluations. However, in September 2012, State Hospitals addressed this potential problem when it issued a request for proposals (RFP) for contractors to perform SVP evaluations beginning in January 2013. Specifically, the RFP established both minimum and maximum rates for some services and set fixed rates for others. For example, it set a minimum rate for performing initial evaluations at $1,500 per case and a maximum rate of $2,500 per case. It also established an hourly rate for court testimony at $200. According to its business services chief, the contracts for 2015 are exempt from competitive bidding using a provision from the State Contracting Manual that exempts contracts solely for the purpose of obtaining expert witness testimony—the California Department of General Services approved the 2015 contracts for SVP evaluators, which included fixed rates. State Hospitals’ fixed rates will prevent evaluators from bidding low on court‑related costs, which could have created an incentive to write negative evaluations. Further, a 2008 State Personnel Board decision has significantly reduced the role of contractors in performing evaluations. The State Personnel Board found that a state law allowing state government entities to contract for work that state workers cannot perform did not justify State Hospitals’ contracting for evaluators. Historically, contract evaluators State law authorized State Hospitals to continue using contract have completed more evaluators temporarily until it could hire employees to replace evaluations for the Sex Offender them. Historically, contract evaluators have completed more Commitment Program than evaluations for the Sex Offender Commitment Program than employee evaluators have employee evaluators have completed, but that trend is shifting. completed, but that trend During fiscal years 2010–11 and 2011–12, contract evaluators is shifting. 44 California State Auditor Report 2014-125 March 2015 produced 99 percent of all evaluations. However, this pattern changed significantly beginning in fiscal year 2012–13, as State Hospitals began hiring evaluators rather than depending solely on contractors: By fiscal year 2013–14, employees performed 53 percent of all evaluations. Given that State Hospitals claimed in September 2014 that it had completed hiring employee evaluators, we expect this number to increase in the coming years. Table 6 shows the number of contract and employee evaluators and the number of evaluations each group performed during the last five fiscal years. Table 6 Evaluations Performed by the California Department of State Hospitals’ Employee and Contract Evaluators Fiscal Years 2009–10 Through 2013–14 FISCAL YEAR 2009–10 2010–11 2011–12 2012–13 2013–14 CONTRACTOR* EMPLOYEE CONTRACTOR* EMPLOYEE CONTRACTOR* EMPLOYEE CONTRACTOR* EMPLOYEE CONTRACTOR* EMPLOYEE All Evaluations Total number of 74 7 80 7 77 13 59 38 42 40 evaluators Total evaluations 3,615 101 5,746 68 3,735 55 1,315 1,507 1,168 1,336 conducted Percent of all 97% 3% 99% 1% 99% 1% 47% 53% 47% 53% evaluations Mean evaluations per individual† 49 14 72 10 49 4 22 40 28 33 Precommitment Number of initial 2,272 0 4,295 3 2,624 18 493 1,296 476 1,048 evaluations Number of other precommitment 747 0 834 0 646 0 456 69 381 100 evaluations‡ Postcommitment Number of annual 128 95 306 62 233 32 215 92 164 116 evaluations Number of other postcommitment 468 6 311 3 232 5 151 50 147 72 evaluations§ Sources: California State Auditor’s analysis of data obtained from the California Department of State Hospitals’ (State Hospitals) Sex Offender Commitment Program Support System, State Hospitals’ payroll data obtained from the California State Controller’s Office’s Uniform State Payroll System, and additional documents provided by State Hospitals. * State Hospitals contracts with individual evaluators. Coalinga State Hospital (Coalinga) retains non-civil service evaluators from registries with which it has contracts—we include these evaluators in the “contractor” category. In some cases, an individual could have been both non-civil service and civil service in the same fiscal year. In such cases, we counted the individual twice. † We present the mean evaluations per individual for informational purposes; however, the value of any conclusions drawn from this data is limited. As noted above, in cases where an individual was both a contractor and an employee in a fiscal year, we counted that individual twice. Further, not all individuals represented in the total number of evaluators were available and working the entire year, which affects the average evaluations per individual. ‡ This number includes evaluations where State Hospitals had to bring in two additional evaluators because the first two evaluators did not agree on whether individuals were sexually violent predators (SVPs). State law requires that contractors conduct these difference-of-opinion evaluations. The number also includes additional evaluations conducted at the request of attorneys or the courts. § Once a court commits an SVP, state law requires that person to receive annual evaluations. Nevertheless, attorneys or the courts may request that State Hospitals conduct additional evaluations. Also, some individuals committed prior to a 2006 law that made commitments indeterminate do not receive annual evaluations, but attorneys or the courts may request additional evaluations. California State Auditor Report 2014-125 45 March 2015 Despite State Hospitals’ statement that it has completed its hiring of evaluators, it will continue to work with contractors in the future because state law requires contract evaluators to resolve difference‑of‑opinion evaluations. According to the assistant deputy director of forensic services, State Hospitals expects to continue contracting with about 20 evaluators in 2015 to provide independent evaluations in cases where the two original employee evaluators disagree about whether offenders meet the SVP criteria. State Hospitals would likely not She stated that contractors could also perform a limited number have sufficient work to keep of initial evaluations if State Hospitals has a spike in evaluations or its contractors fully occupied, some other temporary need. Further, she stated that State Hospitals and without additional work would likely not have sufficient work to keep its contractors fully to perform, evaluators have occupied. Therefore, without additional work to perform, evaluators less financial incentive to finish have less financial incentive to finish evaluations quickly. evaluations quickly. State Hospitals Can Improve Its Efforts to Assess the Effectiveness of Its Evaluator Workload Matrix When State Hospitals reduced the number of contract evaluators and hired employee evaluators, it developed a process for assigning and tracking the workload of its evaluators. In March 2013 State Hospitals convened a group of four SVPEs and two consulting psychologists who developed a matrix that reflected the activities that affect the evaluator’s ability to complete evaluations and the associated value of each activity. The matrix lists the various evaluations and related tasks evaluators perform and includes points assigned for each evaluation or task. For example, the matrix that will be in use until March 2015 assigns one point for an initial SVP evaluation and assigns two points for court testimony provided at a jury trial. State Hospitals set matrix workload expectations of eight points per month for SVPEs and five points per month for consulting psychologists. In the event of unforeseen circumstances, such as a lengthy replacement evaluation or extended court testimony, evaluators may submit a workload adjustment form, which management must approve. State Hospitals recently revised the matrix and created one matrix for each of the two classifications of evaluators at headquarters. According to forensic services’ evaluator workload summary, in the original matrix each point was equivalent to 20 hours of work. The assistant deputy director of forensic services stated that a prior hospital administrator had calculated that the SVPEs should be able to complete an evaluation in about 20 hours, based on invoices from contractors. Therefore, State Hospitals initially set the workload of SVPEs at eight points, or 160 hours per month. She stated that because consulting psychologists are less experienced and are compensated at a level of about a third less than SVPEs, their 46 California State Auditor Report 2014-125 March 2015 workload was set at five points, or about a third less than SVPEs. However, she stated that SVPEs voiced concerns that evaluations will take longer than 20 hours due to a recent decision to require evaluators to complete an analysis of all SVP criteria even if one is negative. Based on further conversations with the evaluators, State Hospitals revised its workload matrix in January 2015, creating different matrices for SVPEs and for consulting psychologists. The new matrices give evaluators more time to perform certain aspects of their work, but omit other elements. The new matrices, which take effect in March 2015, include a workload expectation of seven points a month for SVPEs, with each point equivalent to 23 hours, or 161 hours per month. In contrast, consulting psychologists will continue with a workload expectation of five points, with a point being equivalent to 30 hours, or 150 hours per month. However, probable cause hearings are worth no points on the consulting psychologists’ matrix. According to the assistant deputy director of forensic services, based on conversations with SVPEs, a probable cause hearing is between eight and 12 hours, so giving consulting psychologists credit for one point for a probable cause hearing would mean giving them credit for 30 hours of work, more than twice what an average hearing would take. Nevertheless, the position specifications for consulting psychologists do not require the same level of experience as for SVPEs, and therefore it seems possible that preparing for and participating in a probable cause hearing could take longer for consulting psychologists than for SVPEs. Further, the matrix for SVPEs gives them one point for probable cause hearings; at 23 hours per point, this is still around twice what the expected duration of a probable cause hearing would be. It is also not clear that the matrices account for administrative or other tasks evaluators may perform. The matrices account for time spent on evaluations, court testimony, multiday training, and time off. They do not specifically account for administrative tasks, such as staff meetings and training lasting less than one day, or other job‑related activities, such as keeping up with research in the field. Further, the matrices do not specifically account for travel, stating only that State Hospitals will review travel on a case‑by‑case basis using the workload adjustment form. While State Hospitals has used While State Hospitals has used the workload adjustment form to the workload adjustment form analyze the effectiveness of its workload matrices, its analysis is to analyze the effectiveness of its limited. Specifically, between June 2012 and January 2014, State workload matrices, its analysis Hospitals assigned 59 replacement evaluations, and four evaluators is limited. submitted six workload reduction requests, of which State Hospitals approved three. The assistant deputy director stated that based on the analysis of workload adjustment requests, State Hospitals concluded that the equivalency for replacement California State Auditor Report 2014-125 47 March 2015 evaluations was appropriate. However, this analysis only reviewed one type of evaluation in isolation and not the overall effectiveness of the workload assignments in the matrix. Further, according to the assistant deputy director, few evaluators submit workload adjustments, even though management expects the evaluators to submit them to account for complex cases, additional time needed to prepare for court, or other unforeseen circumstances. She stated that management communicates the importance of completing the workload adjustment forms during its regular conference calls with the evaluators. Nevertheless, neither the policy instituting the form nor the form itself clearly indicates these expectations. Finally, State Hospitals could track evaluators’ hours to further validate the effectiveness of its workload matrix. As previously noted, revisions in the current matrices are the result of discussions with some evaluators, primarily with the more‑experienced SVPEs. According to the assistant deputy director of forensic services, evaluation of the workload matrix has been driven by concerns raised by evaluators, not because of a regular evaluation of workload. For example, State Hospitals does not track the number of hours that evaluators spend on each evaluation. According to the assistant deputy director of forensic services, State Hospitals does not expect its evaluators to complete timesheets to this level of detail. Nevertheless, tracking the actual time spent conducting the various evaluation activities over time would be useful in analyzing whether the current workload expectations are reasonable. Although State Hospitals does convene regular monthly meetings with its evaluators to discuss various topics, including workload, without meaningful periodic analysis of the evaluator matrix and the time evaluators spend on evaluations, State Hospitals risks either increasing pressure to rush evaluations or wasting resources while evaluators are idle. If State Hospitals does not give evaluators adequate time to create evaluations or to prepare for court, it may create an atmosphere that discourages evaluators from doing thorough evaluations to determine whether offenders meet the criteria of an SVP. Coalinga Has a Significant Backlog of Annual Evaluations That It Has Not Completed Coalinga has a backlog of annual evaluations of SVPs it needs to complete. State law requires State Hospitals to evaluate at According to Coalinga’s least annually SVPs committed to it. According to Coalinga’s January 2015 log of overdue annual January 2015 log of overdue annual reports, it had 261 evaluations evaluations, it had 261 evaluations that were due in court by the end of December 2014 that it had that were due in court by the end yet to complete. Coalinga’s forensic senior psychologist supervisor of December 2014 that it had yet stated that evaluators produce the oldest annual evaluations first; to complete. 48 California State Auditor Report 2014-125 March 2015 however, when Coalinga receives a request from a judge, district attorney, or defense attorney for an expedited report, such a request moves that particular evaluation to the top of the list. According to the acting chief of forensic services at Coalinga (forensics), part of the cause of its backlog is additional work State Hospitals assigned to it. Specifically, beginning in April 2011, State Hospitals’ headquarters directed evaluators at Coalinga to conduct evaluations for offenders whom the State committed before Jessica’s Law and who, therefore, only received a two‑year commitment term. Before April 2011 State Hospitals’ evaluators at headquarters conducted these evaluations, which it refers to as recommitment evaluations. According to a tracking log from Coalinga, as of October 2014, 125 offenders were awaiting trial to determine whether they should receive indeterminate commitments because they had completed their two‑year commitments. Coalinga’s acting chief of forensics stated that because evaluators at Coalinga were completing the recommitment evaluations, they were unable to complete as many annual evaluations. A State Hospitals’ legal counsel stated that there were concerns about whether the assignment of those evaluations to Coalinga’s evaluators was appropriate. Therefore, the assistant deputy director of forensic services told us that State Hospitals took back responsibility for completing recommitment evaluations in August 2013. Coalinga’s acting chief of forensics Coalinga’s acting chief of forensics stated that chronic staffing stated that chronic staffing shortages at Coalinga also have contributed to the backlog of shortages at Coalinga also have annual evaluations, a situation that Coalinga hopes to address. contributed to the backlog of He stated that more attractive incentives available for evaluator annual evaluations and does not staff at headquarters have resulted in evaluators transferring to have an estimate for when the that location. Coalinga had 11 evaluators as of December 2014, backlog will be eliminated. seven of whom were contractors. Coalinga’s medical director told us that administrators are working on a plan to alleviate the backlog that includes using contract evaluators to perform annual evaluations, continuing to hire well‑qualified evaluators, and providing incentives for evaluators to keep their employment with Coalinga by allowing them to work remotely. However, this plan is still in development, and Coalinga does not have an estimate for when the backlog will be eliminated. When State Hospitals does not ensure that it completes annual evaluations on time, it is not fulfilling one of its critical statutory obligations. Without such evaluations, the State cannot determine whether an SVP continues to pose a risk to the public and whether an unconditional release or a conditional release to a less restrictive alternative might be in the best interests of the offender and the State. California State Auditor Report 2014-125 49 March 2015 Recommendations To ensure that its evaluators, including those at Coalinga, have the necessary training to conduct evaluations effectively and consistently, State Hospitals should complete development of comprehensive training plans for all evaluators by June 2015. In addition, by September 2015 State Hospitals should provide training on the Static‑99R and dynamic risk assessment instruments to all new evaluators and those who have not yet received such training. To ensure that all its evaluators are aware of changes in forensic evaluations, State Hospitals should provide annual training on updates to risk assessment instruments. To demonstrate that it has provided appropriate training and that its employees have received that training, State Hospitals should immediately begin maintaining training records for all employee and contract evaluators. By June 2015 State Hospitals should establish a formal process for consistently documenting that it has verified that the individuals it hires as evaluators meet all the minimum qualifications for their positions. State Hospitals should ensure that staff at Coalinga follow the process established in Coalinga’s checklist for validating the past employment of employee and contract evaluators. To improve its overall effectiveness, by December 2015 State Hospitals should further analyze the rate at which its evaluators determine that offenders meet the SVP criteria. State Hospitals should focus its analysis on evaluations it performed in the most recent three fiscal years because of its transition to civil service evaluators and because changes to state law have affected how it performs evaluations. State Hospitals should establish what the normal acceptable ranges for commitment rates are and work with evaluators whose findings consistently fall outside that range. To ensure that it has an effective method for assigning and tracking evaluator workload, by September 2015 State Hospitals should establish a formal process for periodically reviewing its workload matrices. This process should include periodic assessments of how well evaluators are meeting their workload expectations and whether adjustments would be appropriate. The process should also include input from key stakeholders. State Hospitals should explore options for tracking the time evaluators spend on each evaluation activity to increase the accuracy of the workload equivalencies it includes in its workload matrix and should implement such options by September 2015. 50 California State Auditor Report 2014-125 March 2015 To reduce its backlog of annual evaluations at Coalinga and reduce the number of days these evaluations are overdue, State Hospitals should immediately determine the extent to which its evaluators who work at headquarters can provide assistance to Coalinga. To ensure that it does not develop a similar backlog in the future, State Hospitals should continue its efforts to hire evaluators sufficient to meet its workload. We conducted this audit under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives specified in the scope section of the report. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor Date: March 12, 2015 Staff: Tammy Lozano, CPA, CGFM, Audit Principal John Lewis, MPA Fahad Ali Gabrielle Gilmore Ryan Grossi, JD IT Audit Support: Michelle J. Baur, CISA, Audit Principal Ryan P. Coe, CISA, MBA Grant Volk, MA, CFE Legal Counsel: Scott A. Baxter, Sr. Staff Counsel For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. California State Auditor Report 2014-125 51 March 2015 Appendix THE RESULTS OF EVALUATIONS OF SEXUALLY VIOLENT PREDATORS The Joint Legislative Audit Committee asked us to report on the number of sexually violent predators (SVPs) that the California Department of State Hospitals (State Hospitals) found suitable or unsuitable for discharge or release. However, according to State Hospitals, it tracks the findings of individual evaluators rather than the number of SVPs found suitable or unsuitable for release. As a result, Table A on the following page reports on the conclusions of individual evaluations of SVPs. Because state law requires State Hospitals to evaluate SVPs committed to a state hospital for indeterminate terms annually, the number of annual evaluations in the table for each year generally corresponds to the number of SVPs evaluated each year. However, there is a population of individuals who were committed to State Hospitals’ custody prior to changes in state law that made commitment terms indeterminate rather than the former two‑year commitments. These individuals’ two‑year commitments have expired, and the courts have not yet held trials to determine whether these individuals are to be committed to indeterminate terms. According to a State Hospitals’ staff legal counsel, these individuals do not receive annual evaluations. Designated county counsels, defense attorneys, or the courts may request additional evaluations of these individuals, sometimes requesting multiple evaluations for the same individual. As a result, the number of other postcommitment evaluations in the table does not equate to the number of individuals evaluated. Further, because this table presents data on the conclusions of individual evaluations, it cannot be effectively compared to data on the number of SVPs released—as we present in Table 1 on page 14—because courts may disagree with the findings of State Hospitals’ evaluators. 52 California State Auditor Report 2014-125 March 2015 Table A Conclusions of Evaluations of Sexually Violent Predators Fiscal Years 2009–10 Through 2013–14 ANNUAL EVALUATIONS OTHER POSTCOMMITMENT EVALUATIONS* EVALUATOR CONCLUDES THAT THE INDIVIDUAL EVALUATOR EVALUATOR CONTINUES TO MEET CONCLUDES THAT EVALUATOR CONCLUDES EVALUATOR CONCLUDES CONCLUDES THAT EVALUATOR CONCLUDES THE CRITERIA FOR A THE INDIVIDUAL NO THAT THE INDIVIDUAL IS A THAT THE INDIVIDUAL THE INDIVIDUAL NO THAT THE INDIVIDUAL IS A FISCAL SEXUALLY VIOLENT LONGER MEETS THE POSSIBLE CANDIDATE FOR CONTINUES TO MEET THE LONGER MEETS THE POSSIBLE CANDIDATE FOR YEAR PREDATOR (SVP)† CRITERIA FOR AN SVP† CONDITIONAL RELEASE† DEFINITION OF AN SVP‡ DEFINITION OF AN SVP‡ CONDITIONAL RELEASE‡ 2009–10 223 3 1 405 69 0 2010–11 365 4 1 264 49 1 2011–12 267 0 3 197 39 4 2012–13 301 2 6 153 39 8 2013–14 275 3 7 168 48 3 Sources: California State Auditor’s analysis of data obtained from the California Department of State Hospitals’ (State Hospitals) Sex Offender Commitment Program Support System; legal counsel for State Hospitals; Coalinga’s acting chief of forensic services; and California Welfare and Institutions Code, Section 6600 et seq. * Committed individuals may receive other evaluations from State Hospitals at the request of designated county counsels, defense attorneys, or the courts. For example, a portion of Coalinga State Hospital’s (Coalinga) population was committed prior to changes in state law that made terms indeterminate and remains in Coalinga on expired, two-year committments, pending a trial to determine whether an indeterminate commitment is warranted. † These numbers represent the number of evaluations conducted and the conclusions of the individual evaluators. Because SVPs must receive annual evaluations, these numbers should generally correspond to individual SVPs; however, SVPs likely appear in multiple years. ‡ These numbers represent the number of evaluations conducted and the conclusions of the individual evaluators. For this population of SVPs, there is no ultimate State Hospitals recommendation. According to the acting chief of forensic services at Coalinga, State Hospitals transmits these evaluations to the courts that requested them and, while individual evaluators opine on the suitability of SVPs for release, State Hospitals does not make an overall conclusion. California State Auditor Report 2014-125 53 March 2015 * * California State Auditor’s comments appear on page 59. 54 California State Auditor Report 2014-125 March 2015 1 California State Auditor Report 2014-125 55 March 2015 2 56 California State Auditor Report 2014-125 March 2015 California State Auditor Report 2014-125 57 March 2015 58 California State Auditor Report 2014-125 March 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-125 59 March 2015 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM THE CALIFORNIA DEPARTMENT OF STATE HOSPITALS To provide clarity and perspective, we are commenting on the California Department of State Hospitals’ (State Hospitals) response to our audit. The numbers below correspond to the numbers we placed in the margin of State Hospitals’ response. We are concerned regarding the timeliness of State Hospitals’ 1 planned action. Rather than update its assessment protocol by March 2016 as we recommended, State Hospitals’ response indicates that it only plans to initiate the regulatory process by that date, which means that the updated protocol will not be in place until much later—typically eight to 12 months. As we state on page 25, without specific guidance regarding how to conduct evaluations, evaluators may not perform their work consistently or review all of the appropriate documents, increasing the risk that they will make erroneous assessments. We are disappointed that State Hospitals has chosen to stop short 2 of including a supervisory checklist in its formal processes. We acknowledge that using the checklist as an administrative tool is an important step in ensuring consistent evaluations. However, including such a checklist in the evaluation process and the standardized assessment protocol ensures that State Hospitals and Coalinga State Hospital can demonstrate consistency.