Summary

Department of Social Services: It Still Needs to Improve Its Oversight of County Child Welfare Services May 2000 2000-500 The first copy of each California State Auditor report is free. Additional copies are $3 each. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TDD (916) 445-0255 x 216 OR This report may also be available on the World Wide Web http://www.bsa.ca.gov/bsa/ Permission is granted to reproduce reports. C S A ALIFORNIA TATE UDITOR MARY P. NOBLE STEVEN M. HENDRICKSON ACTING STATE AUDITOR CHIEF DEPUTY STATE AUDITOR May 4, 2000 2000-500 Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: The Bureau of State Audits (bureau) presents its report entitled Department of Social Services: It Still Needs to Improve Its Oversight of County Child Welfare Services. SUMMARY T he Department of Social Services (department) has been improving its oversight of county child welfare services since the Bureau of State Audits (bureau) conducted its 1998 audit; however, the department has not fully implemented all our recommendations. The department now conducts timely county compliance reviews—designed to ensure the health and safety of children—but it is still slow to give counties written reports of their deficiencies and remiss in ensuring counties promptly submit corrective action plans (CAPs). In addition, although it now reviews the timeliness of counties’ emergency responses to allegations of abuse and neglect, the department does not always require CAPs when counties fail to respond quickly to emergencies. Further, because the department did not create a method of reviewing county administrative practices, it cannot be sure that counties are effectively managing their child welfare BUREAU OF STATE AUDITS C A L I F O R N I A S T A T E A U D I T O R 1 555 Capitol Mall, Suite 300, Sacramento, California 95814 Telephone: (916) 445-0255 Fax: (916) 327-0019 services. Finally, because it has not analyzed statewide data regarding deaths of children from abuse and neglect, the department cannot determine how its services may be failing to protect these children. As part of its January 1998 audit report on Kern County’s child protective services program, the bureau made the follow- ing three recommendations to the department to strengthen its leadership role in and improve its oversight of the State’s child welfare services: • Continue with its schedule to review each county for compliance at least once every four years until it completes the implementation of its statewide automated case manage- ment system, and every three years thereafter. • Review county emergency response systems and administrative practices as part of its comprehensive monitoring approach. • Continue to provide leadership to county welfare agencies through progressive child welfare initiatives. Because our previous report raised significant issues, the bureau decided that a follow-up audit was warranted. This report describes the department’s progress in implementing our 1998 recommendations. In addition, the previous audit ques- tioned the lack of comprehensive statewide information about the extent of children’s deaths from abuse and neglect and reported on various state laws enacted at the time to address this need. Therefore, this report discusses the department’s role in such information gathering and reviews its progress in determining how many children in California have died from abuse and neglect. The department has made some progress in implementing our recommendations. Specifically, the department conducted the on-site portion of its compliance reviews of each county by June 30, 1998, and has plans to review each county at least once every three years. However, the department is slow to finish written compliance reports and follow up on CAPs it receives from counties. These delays may extend the time a county remains out of compliance with department regulations that are supposed to ensure that children are sufficiently protected. 2 C A L I F O R N I A S T A T E A U D I T O R In addition, the department has not fully implemented our recommendation to review county emergency response systems and administrative practices as part of its comprehensive monitoring. Although the department now reviews each county’s emergency response system as part of its compliance review process, it does not always require a CAP from any county that fails to respond on time to allegations of abuse or neglect. Without formalized CAPs, the department cannot ensure counties are correcting problems that may risk the health and safety of children. Regarding our recommendation that it monitor county administrative practices, the department states that it addresses weak administrative practices, such as inad- equate training of child welfare services caseworkers or poor supervision, when discovered during its compliance reviews. However, as a matter of routine, the department does not review the administrative practices of each county. As a result, it may fail to detect other weaknesses that, if corrected, could lead to more efficient and effective management of county resources designed to safeguard children. Moreover, the department does not yet analyze existing data on children’s deaths from abuse and neglect to identify potential systemic weaknesses in child welfare services or to consider the need for legislative or regulatory changes that would reduce these fatalities. A recent law, Chapter 1012, Statutes of 1999, should improve statewide data by requiring a new reporting and tracking system for all child fatalities from abuse and neglect, but a full year’s data will not be available for analysis until January 2002. To the department’s credit, it continues to provide leadership to county child welfare services through its Structured Decision- Making Project. Initial results from this project, one of the department’s progressive child welfare initiatives, are positive. Structured decision-making tools are designed to help caseworkers make critical decisions, such as how quickly to respond to allegations of child abuse or neglect. The goal of the Structured Decision-Making Project is to better safeguard children by improving caseworker assessments of family situations and children’s protection needs. As of April 2000, twelve counties are using the tools, and three additional counties plan to implement them by fall 2000. C A L I F O R N I A S T A T E A U D I T O R 3 BACKGROUND Overview of Child Welfare Services The Welfare and Institutions Code requires the State, through the department and county welfare departments, to establish and support a public system of child welfare services. As the primary statewide intervention program for abused, neglected, and exploited children in California, child welfare services are administered by the department and operated by county welfare departments in all 58 counties. The law charges child welfare services with protecting and promoting the welfare of all children and with preventing, remedying, and helping solve problems that may cause the exploitation or delinquency of children. In addition, child welfare services prevent the unnecessary separation of children from their families, arrange to restore children back to homes from which they had previously been removed, and identify children who should be temporarily or permanently removed from their homes. The Department’s Role in Overseeing the Counties’ Child Welfare Services The Children’s Services Operations Bureau (CSOB), within the department’s Children and Family Services Division, is responsible for the statewide oversight of county child welfare services programs. The mission of CSOB is to protect children who are at risk, safeguard the rights of those who receive child welfare services, and maintain the integrity of families by helping counties uniformly implement laws and regulations governing child welfare services. CSOB accomplishes this mission by conducting county compliance reviews to identify areas needing improvement and then working with counties to develop and implement CAPs. These compliance reviews are the primary focus of this report. The CSOB is also responsible for responding to specific complaints about county child welfare services programs, conducting special case reviews, and doing special projects and studies related to child welfare services. Finally, when it identifies a need, CSOB gives counties consultation, technical assistance, and training. 4 C A L I F O R N I A S T A T E A U D I T O R Overview of the Department’s Compliance Review Process The department conducts periodic compliance reviews of child welfare services in each county. During a review, the department examines a number of program elements, including case plans, emergency responses of child welfare services caseworkers, monthly contacts with children, medical and dental examinations, and health and educational Department Guidelines for the information. After this on-site review, the depart- Compliance Reporting Process ment issues a report that details its findings and indicates areas requiring corrective action. County has 21 days (15 working days) Although no law specifies time frames for issuing to respond to the compliance reports and receiving corrective action on-site review. plans from the counties, the department has Department's draft established guidelines for completing various report due in 30 days. phases of the report, as shown in the text box. County's response to draft is due in 30 days. If the county does not respond, the draft is considered final.* At the end of its one-week to two-week on-site review, the department discusses its findings in an County has 60 days to send a corrective action exit conference with the county’s child welfare † plan to the department. services staff. As shown in the text box, the county then has 15 working days to contest the .7 1.7 2.7 4.7 department’s findings on specific cases by submit- Total Months Elapsed ting additional information to show the county was compliant. Within 30 days after this rebuttal *If the county submits comments, the department process, the department is supposed to provide the works with the county to resolve any newly identified issues. county with a draft compliance report. The county † If necessary. then has 30 days to submit any response on the draft to the department. If the county does not respond to the draft report, it becomes the final report after 30 days. Once the final report is com- plete, the county is allowed 60 days to prepare a CAP if required and submit it to the department. The time from the exit confer- ence to the submission of the county’s CAP should be about four and a half months. Once the department approves the CAP, it requires the county to submit quarterly progress reports on its corrective actions until it achieves compliance. C A L I F O R N I A S T A T E A U D I T O R 5 The Bureau’s 1998 Audit Report Criticized the Department’s Oversight of Child Welfare Services Statewide In addition to auditing Kern County, the bureau reviewed the department’s role in the statewide system of child welfare services and in January 1998 issued a report entitled Kern County: Management Weaknesses at Critical Points in Its Child Protective Services Process May Also Be Pervasive Throughout the State. The report concluded, in part, that many problems in Kern’s and other counties’ child welfare services could be attributed to the department’s shortcomings in administering child welfare services statewide. Specifically, the department had not conducted timely compliance reviews of all counties’ child welfare services programs and did not ensure that those reviews included evaluations of the counties’ emergency response and administrative practices. The report also concluded that the department did not track the statewide extent of children’s deaths from abuse and neglect. Furthermore, the report concluded that the department needed to give counties more guidance in providing child welfare services. As part of the audit, we surveyed all 58 counties in the State and found that many counties experienced problems similar to Kern County, including a lack of tools to assist case- workers in making critical decisions regarding the welfare of children and methods to monitor caseworker caseloads. Finally, our 1998 report included recommendations for the department to strengthen its leadership role and improve its oversight of the State’s child welfare services. SCOPE AND METHODOLOGY The purpose of this follow-up audit was to determine the extent to which the department has implemented the recommenda- tions included in our January 1998 report. We reviewed the department’s two-month, six-month, and one-year responses to that report and met with department staff to determine if it has taken appropriate action to address our recommendations. Specifically, we examined information contained in the department’s county review files to determine the status of the department’s compliance reviews of county child welfare ser- vices programs and to assess whether those reviews are on schedule with our recommendations. We also reviewed a sample of completed compliance review reports to see if the department 6 C A L I F O R N I A S T A T E A U D I T O R is including county emergency response systems and adminis- trative practices. In addition, we reviewed department records regarding the implementation status of the statewide Child Welfare Services/Case Management System (CMS). We also assessed the department’s use of this system to improve its guidance and oversight of county child welfare services programs. We interviewed department staff to assess whether the department is tracking statewide child fatalities caused by maltreatment and whether it uses this information to analyze trends and develop recommendations and prevention strategies. Finally, we looked at various internal documents to deter- mine the department’s efforts to develop and implement assessment tools to aid caseworkers in making critical decisions regarding the welfare of children involved in cases of alleged maltreatment. THE DEPARTMENT CONDUCTS COMPLIANCE REVIEWS AS REQUIRED BUT IS NOT PROMPTLY ENSURING CORRECTIVE ACTION The department completes the on-site portion of its compliance reviews on time. However, despite some improvement, it has not promptly issued compliance reports nor ensured that coun- ties respond to those reports by taking appropriate corrective action to improve their child welfare services. Because the department continues to find a large percentage of compliance problems that require corrective action by counties, these delays can have serious consequences, as counties may not be making timely changes that could prevent continued abuse and neglect of children. To reduce this risk to children, the department is seeking to improve its compliance review process by requesting additional staff to speed up compliance reports and the ensuing corrective action. The Department Now Conducts Compliance Reviews on Time Our January 1998 report observed that the department often did not conduct timely compliance reviews of counties. In response to that audit, and as part of the terms of a lawsuit settlement,1 1 In 1996, a nonprofit child advocacy group filed a lawsuit against the department alleging it was not conducting timely compliance reviews of the county child welfare services agencies. C A L I F O R N I A S T A T E A U D I T O R 7 the department agreed to complete compliance reviews of every county in California no later than June 30, 1998. Also, the department agreed that in the future it would review a county no less than every four years until the CMS2 becomes opera- tional in that county and not less than every three years thereaf- ter. The shorter time frame was based on the expectation that the department could complete reviews more quickly when the CMS made relevant case information readily available. Regard- less of when each county began using the CMS, the department agreed on a three-year compliance review cycle by January 1, 2000, at the latest. Our follow-up review confirmed that, begin- ning with reviews completed in fiscal year 1994-95, the depart- ment conducted an on-site review of every county by June 30, 1998. In addition, we found that if the department adheres to its latest schedule, it will meet the three-year requirement. Despite Improvement, the Department Is Still Slow to Issue Compliance Reports and Obtain Corrective Action Plans Although on-site reviews are now timely, the department still lags behind its own guidelines in giving counties written compliance reports, which then generate the counties’ formal CAPs for noted deficiencies. The department has had some success reducing the amount of time between the completion of its on-site reviews and the submission of CAPs. For all reviews for which counties have submitted CAPs, the length of time between completion of compliance reviews and receipt of county CAPs has dropped each year since fiscal year 1996-97, from more than 15 months to 8.5 months in fiscal year 1998-99. The department attributes this success to its creation of specialty units that focus solely on compliance reviews. Although this change appears to be bringing about the desired results, CAPs are still submitted on average almost 4 months later than the department’s guideline of 4.7 months after completion of compliance reviews. These delays extend the time it takes for the department to ensure that counties have taken appropriate corrective action to solve problems, some serious, identified in the reviews of their child welfare services. To the extent that these problems are significant and go uncorrected, the health and safety of children are jeopardized, and fewer positive outcomes for children and families can be anticipated. 2 CMS is a statewide computer system designed to automate the functions of county child welfare services and provide immediate access to prior abuse and neglect histories of families coming into the child welfare system in any county throughout the State. Its goal is to provide for better case management practices and provide program administrators with a statewide perspective by linking all 58 counties and the State to a common database. 8 C A L I F O R N I A S T A T E A U D I T O R Figure 1 shows the average number of months elapsed from the last day of on-site reviews to the receipt of CAPs for all reviews conducted in fiscal years 1996-97 through 1998-99. For comparison, the figure also shows the department’s guidelines for completing the various stages of the report process. 1999-2000 1998-99 1997-98 1996-97 0 5 10 15 20 C A L I F O R N I A S T A T E A U D I T O R 9 † segarevA raeY lacsiF tnemtrapeD FIGURE 1 Timelines of Total Months Required to Complete Reports and Corrective Action Plans Are Improving But Still Exceed Department Guidelines Time for the department to issue draft compliance report, measured from the last day of the on-site review. Time for the department to issue final compliance report, measured from the date it completed the draft compliance report. Time for the department to receive corrective plan from the county, measured from the date of the final compliance report. Department guidelines* 1.7 1 2 4.7 Total 8 ‡ 13 reviews 5.5 completed 13 reviews 5.7 1 1.8 8.5 Total completed 19 reviews 6.3 1.5 3 10.8 Total completed 8 18 reviews 6.9 2.7 5.5 15.1 Total completed 1 2 3 4 6 7 8 9 11 12 13 14 16 17 18 19 Months Elapsed * Source: The Department of Social Services’ Compliance Review Manual and compliance review reports. † Averages include data for reports and corrective action plans that were required and completed. In some cases, final reports and corrective action plans were not required because the counties were found to be in compliance. In other cases, the reports are not yet due. ‡ Includes the results of only four reviews. As of April 1, 2000, nine draft compliance reports are outstanding, ranging from 1 month to over 8 months since the department completed its on-site review. Three additional reviews are scheduled to be completed. As Figure 1 clearly shows, improvements have occurred in the last two stages of the timeline: the number of months it takes for the department to go from draft reviews to issuing final reports and the number of months for the counties to respond with their CAPs. In fiscal year 1998-99, it took on average about 1 month for the department to complete its final reports and about 2 months for the counties to respond with CAPs compared with about 3 months and 6 months, respectively, in fiscal year 1996-97. Although the improved time frames are consistent with department guidelines, the department still has not received three CAPs: one is currently overdue and two are not yet due because the department just recently completed the draft reports even though it completed the on-site reviews 8 to 10 months ago. If the department received the overdue CAP by May 1, including it in the depart- ment average for fiscal year 1998-99 in Figure 1 would increase the average from 1.8 to 2.2 months. Of greater concern is the fact that the department never received two CAPs related to fiscal year 1996-97 reviews, while two from fiscal year 1997-98 are still outstanding. The department explained that the two CAPs from its fiscal year 1996-97 reviews had not been completed because of lengthy delays in complet- ing its draft reports. Therefore, the department excused the counties from completing the CAPs and instead agreed to con- duct subsequent reviews to evaluate more current compliance information. The department was unable to explain its nearly two-year delay in issuing these draft reports; however, we noted that it lacked a comprehensive tracking system that might have prevented the lengthy delays. Furthermore, for the two CAPs that were not completed from its fiscal year 1997-98 reviews, the department explained that it had contacted the two counties numerous times but received no response. If these two CAPs were received by May 1, including them in Figure 1 would increase the average from 3 to 5 months. Figure 1 also reveals the department’s continuing struggle to finish initial reports within its guideline of 1.7 months. Specifi- cally, in fiscal year 1998-99, the department took on average 4 months longer than its guidelines to issue initial reports of its compliance reviews. In fact, all but one initial report took twice as long as the department guideline to complete, and six of the reports took more than 6 months. Also, long delays characterize the department’s initial reports for its fiscal year 1999-2000 reviews. Two of the four issued reports took 10 C A L I F O R N I A S T A T E A U D I T O R longer than 6 months to complete, and the department has not yet issued nine initial reports. As of April 1, three of these nine outstanding reports were more than 4 months overdue. The department’s failure to issue prompt compliance reports and to ensure that counties quickly respond with CAPs prolongs the time it takes for counties to go on record about how they plan to correct weaknesses in their child welfare services. Until the counties formalize these corrective actions, the department cannot monitor counties to ensure that appropriate changes to safeguard children’s welfare are taking place in county programs. According to department managers, the primary cause for delays in issuing its reports is the lack of adequate staff to handle the workload. In addition, we found that the department did not have a comprehensive tracking system to monitor the status of its compliance reports and CAPs, which might have prevented some of the oversights mentioned above by bringing the overdue reports to the attention of department managers. In its fiscal year 2000-01 budget, the department has requested four additional compliance review staff members to help it complete reports more quickly. The Department’s Reviews Continue to Reveal Low Rates of Compliance Among County Programs Since counties have generally fared poorly during compliance reviews, it is critical that the department completes compliance reports promptly so counties may develop and implement workable CAPs. Specifically, 96 percent of counties the depart- ment reviewed between fiscal years 1996-97 and 1999-2000 needed to take some form of corrective action because of pro- gram weaknesses found during compliance reviews. Figure 2 shows the average rate of compliance with program require- ments for counties reviewed in each fiscal year since 1996-97. The department considers a county compliant if 90 percent or more of the cases it examined during the compliance review met the requirement being evaluated. For example, if the department found that the county caseworker initiated an in-person investi- gation within the required time frame for 47 of the 50 cases reviewed (94 percent compliance), it would conclude the county complied overall with this program requirement. Anything less than 90 percent would result in a finding of noncompliance, requiring the county to develop and implement a CAP. C A L I F O R N I A S T A T E A U D I T O R 11 60% 50 53 46 40 43 42 30 20 10 0 1996-97 1997-98 1998-99 1999-2000 Fiscal Year We recognize that there are mitigating factors for these poor compliance rates. In particular, there is some variation in the severity of problems the department encounters during compli- ance reviews. For example, a county that scores only 50 percent for a compliance requirement has a more severe problem than a county that scores 85 percent, even though neither county meets the 90 percent threshold. In addition, noncompliance in some cases may be the result of a failure in documenting rather than in performing the critical action. However, the pervasive deficiencies found in county compliance reviews highlight the need for strong departmental oversight, including conducting timely reviews, issuing compliance reports quickly, and ensuring counties submit workable CAPs to protect the health and safety of children. Department Changes Should Improve the Compliance Review Process The department is making changes that should further improve its compliance review process. In March 2000, according to the chief of the Children’s Services Operations Bureau, the 12 C A L I F O R N I A S T A T E A U D I T O R egatnecreP ecnailpmoC FIGURE 2 Average County Rates of Compliance With Program Requirements Remains Low Compliance reports included: 16* 19 13 4† Source: Department of Social Services’ compliance reports. * Two reports did not include compliance rates. † The department has completed nine additional on-site reviews but has not yet prepared its initial compliance reports. It also plans to conduct three more on-site reviews before the end of the fiscal year. department developed tracking tools to help managers keep up with when compliance reports and CAP responses occur. The department also streamlined the county compliance report so it can be completed faster and give counties a clearer display of data. Further, the department requested an augmentation to its fiscal year 2000-01 budget to add four compliance review staff members to the eight existing workers responsible for conduct- ing reviews, completing reports, and reviewing and responding to CAPs. The department has recently designated a staff person to facilitate the development, review, and finalization of CAPs. Finally, the department is creating a county compliance review work group, with staff from the State and the counties, which will identify ways to make the county compliance process more efficient and effective. THE DEPARTMENT HAS NOT FULLY IMPLEMENTED OUR RECOMMENDATIONS TO IMPROVE THE QUALITY OF ITS COUNTY COMPLIANCE REVIEWS While it continues working toward a faster review timeline, the department could also improve the quality of reviews by fully including county emergency response systems and examining county administrative practices, additions we recommended in our prior report. Responding to that audit, the department now reviews county emergency response systems; however, it does not always require counties to implement corrective action when the reviews reveal deficiencies. In addition, the depart- ment has failed to implement our recommendation to include in its reviews an assessment of county administrative practices, such as whether counties are monitoring caseworker caseloads or have adequate policies and procedures. By not including such elements in its reviews, the department cannot determine whether county practices that affect the lives of children at risk are adequate. The Department Reviews Cases Requiring an Emergency Response But Does Not Follow Up on All the Problems It Uncovers Although it now examines cases from county emergency response systems during its compliance reviews, the department does not always require corrective action when it notes deficien- cies. It is important to review each county’s emergency response C A L I F O R N I A S T A T E A U D I T O R 13 system because a system that is not working properly may prevent a county from responding quickly to allegations of abuse or neglect, leaving children at risk. The State requires each county to have emergency response personnel to assess reports of child abuse and neglect and decide whether to immediately dispatch child welfare services caseworkers to visit the children. In our January 1998 audit, we found that the department was not always reviewing county emergency response services. Even when it did such reviews, the department only reviewed cases the counties opened, not those to which the counties responded, determined no further services were necessary, and then closed.3 In our follow-up audit, we found that the department now examines emergency response systems, including closed cases, as part of all its compliance reviews. During a compliance visit, the department conducts both a full case review and an emer- gency response review. In a full case review, the department reviews a sample of all cases opened and referred for some type of in-home or out-of-home service. These comprehensive reviews include an examination of case plans, the caseworker’s emergency response and monthly contacts with the child, medical and dental examinations, and health and educational information. In the emergency response review, the department selects a sample of all emergency response cases, including opened and closed cases, and focuses only on the timeliness of caseworkers’ emergency responses to allegations of child abuse and neglect. When the department notes deficiencies in a county’s emergency response during a full case review, it appropriately requires the county to develop and implement a CAP. However, if the department notes these same problems during its emergency response review, it does not require a CAP but only recommends that the county fix the problems. Therefore, any problems the department detects in its emergency response review will not result in a CAP unless those problems also appear in the full-case review. In fact, in four of the nine compliance reviews we examined, the department required no CAPs from the counties for deficiencies found during the emergency response reviews because those deficiencies were not also noted in the full-case reviews. 3 Cases are closed at the emergency response stage when the caseworker conducts a face-to-face response and determines that no further child welfare services are required. 14 C A L I F O R N I A S T A T E A U D I T O R For example, during its emergency response review of one county, the department found ten cases in which caseworkers did not initiate in-person investigations within the time frame required by department regulations. In two of the ten cases, caseworkers did not contact the children until 3 days after the allegations of abuse or neglect, although conditions required responses within 1 day. In the other eight cases, which required caseworkers to respond within 10 days, the caseworkers were from 1 to 37 days late in contacting the children, with three responses taking more than twice the number of days allowed. However, because the department did not uncover these emer- gency response deficiencies during its full-case review, it did not require the county to prepare a CAP to resolve these serious emergency response issues. The department stated that no formal corrective action was required and recommended that the county consider internal quality assurance measures to improve performance in this area. Although the department could not explain why it does not require CAPs in such cases, the chief of the Children’s Services Operations Bureau stated that the compliance review unit plans to begin requiring correc- tive actions for these cases after discussing this requirement with child welfare services staff in each county. The Department Has Not Implemented Our Recommendation to Evaluate County Administrative Practices Despite our recommendation, the department does not examine the administrative practices of child welfare services as part of its county compliance reviews. Because weak administration can hinder delivery of key program services, the department is missing opportunities to better ensure children’s health and safety. Our prior audit revealed that the department did not evaluate county administrative practices, even though many counties admitted to administrative weaknesses, such as the lack of policies and procedures on how to locally implement state regulations and the lack of a means to track worker caseloads. During our follow-up audit, we found that, although it still lacks a specific mechanism to review county administrative practices, the department does attempt to resolve administrative problems by helping counties develop CAPs for compliance review defi- ciencies. In fact, we found administrative remedies, such as providing additional training or enhancing supervisory over- sight, in six of the seven CAPs we reviewed. For example, the C A L I F O R N I A S T A T E A U D I T O R 15 department found one county was not conducting the required number of visits with a child during the first 30 days after it opened the case. In its CAP, the county said the cause of this problem was that its staff was unclear about the minimum number of visits required when a case plan is still under development. To resolve this issue, the county gave its staff instruction on regulatory requirements and enhanced the role of its supervisors in ensuring caseworkers complete the appropriate number of visits. In other words, the county improved certain administrative practices to resolve a weakness in its program. Although it is useful for the department to encourage improve- ments in county administrative practices to resolve program compliance issues, the department misses the opportunity to correct deficiencies in administrative practices sooner because it does not review them during its on-site reviews. Under our previous example, if the department had reviewed the adequacy of staff training and supervision during its on-site review, it could have brought this to the county’s attention sooner rather than waiting for the county to develop its CAP. As we noted earlier, CAPs are coming in on average 8.5 months after the department completes its on-site review. Furthermore, if the department limits itself to considering administrative practices only when they concern previously identified program problems, it may fail to detect other weaknesses that, when corrected, could further improve child welfare services. Moreover, in its responses to our prior audit, the department said that its automated CMS would improve county administra- tive practices by giving county staff the information needed to manage caseloads more effectively and efficiently. In addition, the department stated that the system would provide state and local information from which to evaluate the effectiveness of child welfare services in meeting the needs of the families and children served. We found that CMS is operational in all 58 counties; however, according to department staff and a review conducted by the federal government, county use of CMS is inconsistent, and the level of utilization varies. In addition, we found that there are no state laws or regulations requiring counties to fully use CMS. Because the counties do not use the system uniformly or consistently, its use as an effective oversight tool is limited. 16 C A L I F O R N I A S T A T E A U D I T O R THE DEPARTMENT SHOULD BEGIN ASSESSING CHILD ABUSE AND NEGLECT FATALITY DATA CURRENTLY AVAILABLE In our prior audit, we reported that the department does not know the number of children who have died from abuse and neglect in the State because counties are not required to report such deaths. The department still lacks adequate information on the statewide extent of child fatalities from abuse and neglect, but recent legislation should help the department gather more information on these tragic deaths. If the department analyzes and disseminates this data to the counties, they can use it to improve their prevention strategies. Rather than waiting until the new laws force changes, the department should assess the data currently available to see if policy or procedural changes in child welfare services are warranted. As we reported in 1998, Chapter 842, Statutes of 1997, requires state and local death review teams in collaboration with the Departments of Justice, Social Services, and Health Services to reconcile existing sources of information regarding child fatality cases to create a central database of information. Existing sources of data included in this reconciliation are the Vital Statistics database, maintained by Health Services; and the Homicide File and Child Abuse Central Index, both maintained by the Department of Justice. Health Services, which is coordi- nating this effort, completed its reconciliation of 1996 and 1997 data in October 1999 and April 2000, respectively, and expects to complete its reconciliation of 1998 data by July 2000. Chapter 1012, Statutes of 1999, requires Health Services to establish a standardized review form for a child’s death and a child abuse and neglect fatality tracking system. The law also requires Health Services to conduct annual reviews to reconcile data collected from this new tracking system with data from the other three databases described above and the department’s automated CMS. According to the manager of Health Services’ data tracking project, the new system should be operating in the 56 counties that have child death review teams by January 2001. This system, if implemented and used as intended, should improve the reporting and tracking of children’s deaths and provide better data for analyzing the adequacy of current poli- cies and procedures in preventing deaths of children who are known to the State’s child welfare services. However, it will be at least January 2002 before a full year’s data is available through the new tracking system for analysis. C A L I F O R N I A S T A T E A U D I T O R 17 Currently, the CMS includes information on the deaths of only those children previously included in the system as a result of allegations of abuse or neglect. However, Chapter 1012 now requires counties to report all cases of child deaths suspected to be related to abuse or neglect through the CMS, including those children who were not previously known to child welfare services. The department is preparing a letter to all counties advising them of this new requirement, and it plans to issue procedures during the next few months for reporting the deaths of children not already recorded in the system. Although the legislation described above promises to improve information regarding child deaths due to abuse and neglect, the department should begin assessing currently available data, including data in the CMS and the reconciled data from Health Services. Analyzing its own data along with other available information would be an important first step for the department to identify any systemic weaknesses in county child welfare services and to consider whether additional legislative or regulatory changes might prevent future deaths of children from abuse and neglect. THE DEPARTMENT’S STRUCTURED DECISION-MAKING PROJECT APPEARS TO HAVE POTENTIAL FOR STATEWIDE BENEFIT The department continues to provide leadership for statewide child welfare services by implementing its Structured Decision- Making Project. Although this pilot project is just getting started, initial indicators suggest it can benefit all child welfare services. However, the department presently does not plan to assess whether counties participating in the pilot project achieve better outcomes for children and families than counties that are not participating. Without such a comparison, the department cannot easily confirm the project’s benefits and advocate its expansion to all counties. The Structured Decision-Making Project consists of a series of tools designed to assist California’s caseworkers in making critical assessments and decisions about cases. In particular, these tools include an assessment and decision-making system for determining how quickly to investigate reports of abuse and neglect. Also provided are tools for assessing child safety at the time of investigation, the potential for future maltreatment, child and family strengths and needs, and the appropriateness 18 C A L I F O R N I A S T A T E A U D I T O R of returning children home from foster care. The project’s goal is to link risk and needs assessments to service plans and case- worker actions so that families receive appropriate services. We noted in our 1998 report that the department was establish- ing a vendor contract to pilot a structured decision-making system at 7 counties. Presently, 12 counties are using the system and 3 more are planning to implement it by fall 2000. The department is seeking approval for the funding necessary to continue the project in those 15 counties until 2004. Initial assessments of the project have been positive. For example, the department’s contractor reviewed Orange County’s emergency response system 6 months after it began using structured deci- sion-making tools. This review focused on Orange County’s risk assessment tool, which helps caseworkers to classify child welfare services cases according to the likelihood of future maltreatment. In its January 2000 report, the contractor found that the tool was functioning properly and that the caseworkers rarely disagreed with the guidance it provided. In June 2000, the department expects to receive from its contractor additional reports on the Structured Decision-Making Project in other counties. However, similar to the Orange County report, these reports will not assess whether counties using the department’s structured decision-making tools achieve better outcomes for children and families than counties that do not use them. At this point, the department is still considering the need to conduct this comparison study. In our view, such a study is critical for the department to assess the project’s effectiveness and decide whether to expand it to other counties. For example, Michigan has implemented similar structured decision-making tools and conducted a study to compare the outcomes for cases in counties that implemented the tools with those that did not. The study revealed that counties using structured decision-making tools had 27 percent fewer new referrals, 54 percent fewer new substantiated allegations of abuse against children, 40 percent fewer children removed to foster care, and 42 percent fewer child injuries requiring medical attention than counties that did not implement the tools. These impressive results suggest that when a child welfare services program can accurately classify families according to risk level, it can selectively focus resources to achieve better outcomes for children and families. C A L I F O R N I A S T A T E A U D I T O R 19 RECOMMENDATIONS To improve the effectiveness of its oversight and monitoring of counties, the department should: • Continue pursuing and implementing measures to reduce the amount of time it takes to issue compliance reports and to receive and respond to CAPs. • Require counties to develop CAPs for all emergency response deficiencies noted during compliance reviews. • Review county administrative practices during compliance reviews. To evaluate the need for policy and legislative changes, the department should assess the data on children’s fatalities from maltreatment. Such data is currently available through the CMS and other sources, such as Health Services and the Department of Justice. Further, the department needs to incorporate into its assessments any new information that becomes available as a result of recent legislative changes. To further improve statewide data regarding the extent of child deaths from abuse and neglect, the department needs to develop and disseminate procedures for counties to report all child deaths through the automated CMS as soon as possible. To improve the decision-making capabilities of caseworkers and the related outcomes for families, the department should continue evaluating the results of its Structured Decision-Making Project and should conduct an outcome evaluation to determine if this pilot project results in better outcomes for families and children. If the results continue to be favorable, it should consider pursuing a requirement for the use of structured decision-making tools statewide. 20 C A L I F O R N I A S T A T E A U D I T O R We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the scope section of this report. Respectfully submitted, MARY P. NOBLE Acting State Auditor Date: May 4, 2000 Staff: John F. Collins II, CPA Tyler Covey, CPA, CMA C A L I F O R N I A S T A T E A U D I T O R 21 Blank page inserted for reproduction purposes only. 22 C A L I F O R N I A S T A T E A U D I T O R Agency’s comments provided as text only. Department of Social Services 744 P Street Sacramento, CA 95814 April 28, 2000 Ms. Mary P. Noble Acting State Auditor Bureau of State Audits 555 Capitol Mall, Suite #300 Sacramento, California 95814 Dear Ms. Noble: Thank you for the opportunity to comment on the State Auditor Report entitled “The Department of Social Services: Still Needs to Improve Its Oversight of County Child Welfare Services.” Because our mission is to protect children at risk of abuse and neglect, we are eager to improve CWS oversight and strengthen our leadership role using every available resource. We substantially agree with the report recommendations and have provided comments on each recommendation. We are pleased that the report recognizes the department’s progress in conducting reviews as recommended by your earlier audit. In addition to completing compliance reviews of all 58 counties by June 1998, CDSS met its obligation to conduct compliance reviews in 13 counties in 1999. While we have not routinely met our self-imposed timeframes for finalizing reports, we have reduced our processing time by nearly fifty- percent from 1996/97 to 1998/99 without the benefit of any additional staff resources. We concur with your recommendation to require counties to develop corrective action plans for all closed emergency response cases that are identified as having deficiencies in timely response and will pursue this change. The report notes that many counties have been required to complete corrective action plans as a result of the compliance reviews and we recognize the importance of formalizing the county corrective action process in a timely manner. We do note however, that there has been improvement in county performance in the key areas of monthly contacts and timely medical examinations For those counties who are significantly out of compliance in major areas, we have intensified our work with those counties and have implemented more frequent reviews to ensure that their corrective action plans are effective. With the additional resources requested for FY 2000/01, we expect to be able to fully meet our goals for finalizing reports and approving county corrective action plans in a timely manner. C A L I F O R N I A S T A T E A U D I T O R 23 Ms. Mary P. Noble Page 2 We concur that consistent county administrative practice is an important component of county program management. As indicated in the response to the January 1998 audit report, we believe that the implementation of a risk assessment and structured decision model will result in improved and more consistent county practice. We recognize that the model has yet to be implemented statewide. However, our increased number of county reviews have given us the opportunity to work with counties whose compliance review deficiencies suggest the need to address administrative problems. As the audit report notes, this targeted approach to systems analysis has allowed us to impact areas of county practice linked to key regulatory 1 * requirements. We recognize that expanding the onsight review to routinely include a broad examination of administrative practices could reveal opportunities for improving county systems. This expansion would require the pursuit of staff resources. Our current focus is on improving the timelines and quality within the current scope of our reviews. We agree that the structured decision-making (SDM) project shows promise for improving the effectiveness of child welfare services (CWS) in California. While relatively new to California,, the SDM model for CWS has been in existence in jurisdictions throughout the nation since 1986 and is used in 14 states. Research in other states has shown the significant impact SDM has on outcomes for children and families. California’s SDM Project includes 15 counties that account for approximately 70% of the State’s child welfare caseload. The SDM Project includes several evaluation components that will validate the research-based risk assessment, measure inter-rater reliability, and assess the effectiveness of the process. In addition to this, the audit report recommends a comprehensive outcome study. Such a study is no doubt desirable but requires a lengthy amount of time to complete as well as adequate funding. You have recommended that the department should assess the data on children’s fatalities from maltreatment. CDSS is committed to analyzing all data in order to determine implications for policy, practice and procedures. We concur that the results of this analysis may lead to strategies to reduce the number of child fatalities and prevent these tragedies from occurring. Additionally, CDSS is implementing procedures for counties to report all child deaths through the automated Child Welfare Services/Case Management System, an effort that is scheduled for completion by Spring 2000. * California State Auditor’s comments appear on page 27. 24 C A L I F O R N I A S T A T E A U D I T O R Ms. Mary P. Noble Page 3 California’s child welfare system requires continuous effort to improve its performance. We are committed to pursing all avenues that will improve the lives of California’s children. If you have any questions or need additional information, please feel free to to call me or contact Sylvia Pizzini, Deputy Director, at (916) 657-2614. Sincerely, (Signed by: Rita Saenz) RITA SAENZ Director C A L I F O R N I A S T A T E A U D I T O R 25 Blank page inserted for reproduction purposes only. 26 C A L I F O R N I A S T A T E A U D I T O R COMMENTS California State Auditor’s Comments on the Response From the Department of Social Services T o provide clarity and perspective, we are commenting on the response to our audit report from the Department of Social Services (department). The number below corre- sponds to the number we have placed in the department’s response. 1 We are concerned that the department is discounting our recommendation to review administrative practices during its compliance reviews because it believes these reviews will require additional staff resources. In our view, reviewing administrative practices, such as whether a county has policies and procedures or a quality assurance program, would not require substantial additional time or resources. Moreover, as the department acknowledges in its response, such a review could reveal opportunities for improving overall county systems. C A L I F O R N I A S T A T E A U D I T O R 27 cc: Members of the Legislature Office of the Lieutenant Governor Milton Marks Commission on California State Government Organization and Economy Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press 28 C A L I F O R N I A S T A T E A U D I T O R