Recommendations

Follow-Up: Children in Medi-Cal The Department of Health Care Services Is Still Not Doing Enough to Ensure That Children in Medi‑Cal Receive Preventive Health Services September 2022 REPORT 2022‑502 Michael S. Tilden Acting State Auditor September 13, 2022 2022-502 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: Approximately half of California’s children receive medical care through the California Medical Assistance Program (Medi‑Cal). Nearly all of these 5.5 million children receive care through managed health care plans (health plans) to which California pays a monthly premium for each enrolled child. However, in analyzing data for nearly the past decade, we determined that less than 50 percent of the children in Medi‑Cal have received the required preventive services that would help ensure that they live healthier, more productive lives. In March 2019, my office issued report 2018‑111 regarding the Department of Health Care Services' (DHCS) oversight of the delivery of preventive services to children in Medi‑Cal. This follow‑up report presents an update on DHCS’s efforts to implement that report’s recommendations. We found that DHCS made some progress implementing that audit’s recommendations, but it has yet to fully implement eight of the 14 recommendations we made, and many of DHCS’s efforts to improve its oversight of health plans’ provision of children’s preventive services were placed on hold during the COVID‑19 pandemic. We believe that the ongoing threat of COVID‑19 and other communicable diseases provides more reason, not less, for DHCS to reinstitute and improve its oversight of health plans' provision of required preventive health care services to children. Respectfully submitted, MICHAEL S. TILDEN, CPA Acting California State Auditor 621 Capitol Mall, Suite 1200 | Sacramento, CA 95814 | 916.445.0255 | 916.327.0019 fax | www.auditor.ca.gov CALIFORNIA STATE AUDITOR iii Report 2022-502 | September 2022 Contents Recommendations 1 The State’s Medi‑Cal Program Provides Preventive Health Services That Are Vital to Children’s Well Being and Long‑Term Development 2 California Continues to Struggle to Ensure That Millions of Children in Medi‑Cal Receive Preventive Care 3 DHCS Has Not Taken Adequate Steps to Ensure That Children Receive Important Health Care 5 DHCS Has Not Done Enough to Increase the Number of Available Medi‑Cal Providers 9 Scope and Methodology 11 Response to the Audit Department of Health Care Services 14 California State Auditor’s Comments on the Response From the Department of Health Care Services 25 Selected Abbreviations Used in This Report DHCS Department of Health Care Services external reviewer External Quality Review Organization Bright Futures American Academy of Pediatrics' Bright Futures recommended schedule of care HCAI Department of Health Care Access and Information CALIFORNIA STATE AUDITOR 1 Report 2022-502 | September 2022 Recommendations To ensure that children in Medi‑Cal receive critical health services, the Department of Health Care Services (DHCS) should fully implement all of our 2019 audit's recommendations that remain outstanding: • To help increase utilization rates, DHCS should begin to monitor and identify effective incentive programs at the health plan level and share the results with all plans. • To ensure that health plans address underutilization of children’s preventive services, DHCS should require plans to use their utilization management programs to identify barriers to usage specifically for these services and hold the plans accountable to address the barriers they identify. • To better ensure the accuracy of its data and ensure that California receives all available federal Medicaid funding, DHCS should require its External Quality Review Organization (external reviewer) to perform its encounter data validation studies annually using the most recent set of data available, and it should implement recommendations from its external reviewer studies. • To mitigate health disparities for children of differing ethnic backgrounds and language needs, DHCS should revise the methodology for its external reviewer’s health disparity study to enable it to better make demographic comparisons, and it should use the findings to drive targeted interventions within health plan service areas. It should publish this study annually. • To improve its ability to ensure that children are receiving recommended preventive health services, DHCS should create an action plan to annually address its external reviewer’s recommendations relating to children’s preventive services, including any recommendations left unaddressed since the external reviewer’s fiscal year 2015–16 reports. • To ensure that eligible children and their families know about all the preventive services they are entitled to through Medi‑Cal, DHCS should include clearer and more comprehensive information about those services in its written materials and ensure annual follow‑up with any children and their families who have not used those services. • To ensure that health plan provider directories are accurate, DHCS should begin using a 95 percent confidence level and not more than a 10 percent margin of error on its statistical sampling tool and should require at least 95 percent accuracy before approving a health plan’s provider directory. In addition, DHCS should ensure that its staff adhere to its policy to retain all documentation related to its review of provider directories for at least three years. • To increase access to preventive health services for children in areas where they are needed most, DHCS should identify where more providers who see children are needed and propose to the Legislature funding increases to recruit more providers in these areas. CALIFORNIA STATE AUDITOR 2 Report 2022-502 | September 2022 The State’s Medi‑Cal Program Provides Preventive Health Services That Are Vital to Children’s Well‑Being and Long‑Term Development Figure 1 Children covered by Medicaid programs—such as Medi-Cal— are more likely to be born with medical conditions requiring quality health care. Medical providers offer preventive services... Such as: Immunization against disease Lead poisoning risk assessments Behavioral and mental health assessments Physical examinations, including measurements and sensory screenings ...resulting in Increased overall Reduced future wellness medical costs CALIFORNIA STATE AUDITOR 3 Report 2022-502 | September 2022 According to the federal Centers for Medicare & Medicaid Services, children covered by Medicaid programs, such as Medi‑Cal, are more likely to be born with low birth weights or have developmental delays, learning disorders, or other medical conditions requiring quality health care. DHCS is the single state department responsible for administering Medi‑Cal, and it reports that approximately 5.5 million—roughly half— of California’s children are enrolled in Medi‑Cal and therefore fall into this high‑risk category. The vast reach of Medi‑Cal highlights the vital importance of the preventive health screenings and other services provided by this program in ensuring the health and long‑term development of California’s children. Childhood preventive health services provide early detection, treatment, or avoidance of health problems that could negatively impact a child’s entire life. According to research, providing preventive health services to children saves lives and reduces future health care costs. Conversely, children who do not receive these services are at greater risk of medical conditions with potential long‑term impacts. For example, Medi‑Cal requires health care providers to screen young children for elevated blood lead levels— an ailment to which children younger than age 6 are particularly vulnerable and that can lead to seizures or death in extreme cases. Even low levels of lead exposure in children can lead to a reduced IQ and decreased productivity when those children become adults. Without lead screenings, a young child’s exposure to lead may go undetected and permanently diminish the quality of the child’s life. In addition, children who fall behind on vaccinations—another preventive health service provided by Medi‑Cal—are more likely to get diseases like measles, whooping cough, or other infections that may require hospital treatment. Many of these diseases are contagious and can result in localized outbreaks, endangering babies and young children. California Continues to Struggle to Ensure That Millions of Children in Medi‑Cal Receive Preventive Care DHCS has been unsuccessful in increasing the percentage of children receiving the preventive services required by Medi‑Cal. DHCS adopted the American Academy of Pediatrics’ Bright Futures recommended schedule of care (Bright Futures) to comply with federal requirements related to preventive services. The Bright Futures schedule includes services such as examinations, immunizations, and developmental screenings. In our March 2019 report, we found that less than half of the children enrolled in Medi‑Cal received the required preventive services each year from fiscal year 2013–14 through 2017–18.1 From fiscal years 2018–19 through 2020–21, DHCS continued to provide these services to less than half of the children in Medi‑Cal, leaving an average of 2.9 million children per year missing at least some preventive services. The number of children in Medi‑Cal who received preventive care decreased during the COVID‑19 pandemic. In fiscal year 2020–21, less than 42 percent of children received the required number of preventive services, which compares unfavorably to a pre‑pandemic level of almost 48 percent in fiscal year 2018–19. 1 Department of Health Care Services: Millions of Children in Medi-Cal Are Not Receiving Preventive Health Services, Report 2018-111, March 2019. CALIFORNIA STATE AUDITOR 4 Report 2022-502 | September 2022 Figure 2 50% 40 For at least the past eight years, less than 30 half of children in Medi-Cal have received the required number of services. 20 10 0 2013–14 2014–15 2015–16 2016–17 2017–18 2018–19 2019–20 2020–21 This problem was further exacerbated by the COVID-19 pandemic. An average of 2.9 million children in Medi-Cal per year did not receive all required preventive services during fiscal years 2018–19 through 2020–21. The percentage of children in Medi‑Cal receiving the required amounts of preventive care continued to be even lower among certain age groups during the most recent three fiscal years. For example, nearly three quarters of 2‑year‑olds in Medi‑Cal did not receive the required number of preventive services, which includes blood lead testing and screening for developmental disorders. According to the Centers for Disease Control and Prevention, many children have delays in language or other areas that can affect how well they do in school. However, these delays are often not identified until the children are attending school, by which time opportunities for treatment might have been missed. Further, nearly 60 percent of 1‑year‑olds did not receive the required number of preventive services, which can include immunizations against Hepatitis B, a condition that can lead to severe liver damage, and polio, an illness that can result in permanent paralysis and death. To explore additional utilization rates by year, county, age, language, and ethnicity, see our interactive graphics at https://www.auditor.ca.gov/reports/2022‑502/supplemental.html. Please refer to the section on page 1 to find the recommendations that we have made as a result of these audit findings. CALIFORNIA STATE AUDITOR 5 Report 2022-502 | September 2022 DHCS Has Not Taken Adequate Steps to Ensure That Children Receive Important Health Care Figure 3 DHCS Did Not Fully Implement 8 of the 14 Recommendations We Made In 2019 6 Recommendations Fully Implemented 7 Recommendations 1 Recommendation Will Partially Implemented Not Be Implemented Monitor and Identify Increase 1 8 Incentive Programs Access to Providers 2 Identify Barriers to Usage 3 Perform Data Validation 4 Mitigate Health Disparities 5 Address External Reviewers’ Recommendations 6 Improve Beneficiary Outreach DHCS Performed Limited Outreach. Outreach for Children Under Age Seven h Y e o a u l r t h ch s i e ld rv i i s c e e n s: titled to critical Immunizations Blood lead screenings Physical exam S c r D ee e n n i t n a g l s for concerns with: Vision Hearing Development Mental health Nutrition 7 Ensure That Provider Directories Are Accurate DHCS Approved Provider Directories Despite Finding Errors. “T is h i i n n s o s n e l u o r m n v g i b c e e e r r ” CALIFORNIA STATE AUDITOR 6 Report 2022-502 | September 2022 More than three years have passed since we issued 14 recommendations to DHCS to improve its delivery of preventive services to children in Medi‑Cal. It has fully implemented just six of those recommendations. Although DHCS made some progress toward addressing the outstanding recommendations, eight recommendations are still not fully implemented, including one that DHCS disagrees with and does not plan to implement. DHCS Chose to Limit Outreach to Medi‑Cal Families During the Pandemic DHCS is not meeting its responsibility to educate families as the federal government requires. Specifically, federal law requires DHCS to perform annual outreach to children or families of children who are eligible to receive preventive services to inform them of the benefits of preventive health care and how to obtain those services. Although DHCS directed health plans to perform a one‑time outreach campaign by June 2021 for children who had not received all of the required services, DHCS only required health plans to inform families about a limited number— not all—of children’s preventive services available to them. Citing verbal reports from health plans that providers were operating at reduced capacity during the pandemic, the chief of the Managed Care Quality and Monitoring Division (quality and monitoring chief) explained that DHCS chose to only require health plans to promote the most critical children’s preventive services: immunizations and blood lead screenings. She stated that DHCS believed that promoting additional services during the pandemic could have resulted in an influx of families requesting appointments, which would have reduced the number of appointments available for providers to treat children experiencing chronic illnesses and those in need of receiving the most critical children’s preventive services. Therefore, DHCS instructed the health plans that it was optional to discuss other children’s preventive services—such as physical exams and screenings for concerns with vision, hearing, and development—when performing outreach with children under the age of 7. The quality and monitoring chief acknowledged that DHCS needs to implement an ongoing effort to ensure that families know about the children’s preventive health care services to which they are entitled but explained that the pandemic interrupted the department’s planning for this effort. She stated that DHCS has resumed its efforts to develop a long‑range plan for conducting annual follow‑up with beneficiaries who have not used children’s preventive services. According to the deputy director of the Health Care Benefits and Eligibility Division (deputy director of benefits and eligibility), DHCS intends to issue detailed guidance over the next few months for conducting outreach to beneficiaries who have not seen their primary care provider in the past 12 months, and it intends to require all health plans to develop a robust strategy for addressing this underutilization by July 2023. CALIFORNIA STATE AUDITOR 7 Report 2022-502 | September 2022 DHCS Did Not Distribute Outreach Materials During the Pandemic Although DHCS hired a contractor to implement our recommendation to develop written materials about children’s preventive services—such as brochures, mailing notices, and content that it can post on its website and social media pages—DHCS chose not to publish the material. The quality and monitoring chief once again cited the concern that promoting children’s preventive services would potentially result in an influx of families requesting appointments that providers would not be able to accommodate during the pandemic. Further, the deputy director of benefits and eligibility stated that DHCS received the written materials from its contractor in July 2021 but the materials no longer align with DHCS’s new communication strategy for children in Medi‑Cal, so DHCS chose not to publish them. She explained that historically, DHCS’s communications for children in Medi‑Cal have focused on preventive services, which is only one component of the services DHCS is required to provide to children. For example, federal law generally requires DHCS to also provide treatment for medical conditions identified during preventive health screenings. Therefore, while the deputy director of benefits and eligibility acknowledged that a lot of effort went into developing the original written materials, she stated that DHCS wants to be more deliberate about its communication strategy for children in Medi‑Cal. She stated that DHCS will continue to work with key stakeholders to develop new materials and, to the extent feasible, will leverage portions of the original materials that are still applicable. DHCS hopes to launch the new materials by March 2023. However, because DHCS is still in the planning phase of developing these new outreach materials, we do not have assurance that this timeline is realistic. Regardless, DHCS spent time and money developing the original written materials and, by choosing not to distribute them to beneficiaries, DHCS missed an opportunity to ensure that children and their families were aware of the children’s preventive health care services to which they were entitled. DHCS Did Not Ensure That Provider Directories Were Accurate DHCS also potentially impeded families’ access to providers by not ensuring that the health care provider directories—one of the primary means by which families can locate health care providers—are accurate. We previously noted concerns with DHCS’s method for reviewing provider directories and recommended that it require at least 95 percent accuracy before approving a health plan’s provider directory. However, we evaluated a selection of eight provider directory reviews DHCS should have conducted between 2019 and 2022 and determined that DHCS could not demonstrate that it performed two of the scheduled directory reviews and that it approved the other six directories despite finding significant errors in them. CALIFORNIA STATE AUDITOR 8 Report 2022-502 | September 2022 In conducting its provider directory reviews, DHCS verifies information for a sample of providers included in the directory. For two of the six reviews it completed, DHCS was able to validate information for only about 60 percent of the providers it sampled, yet DHCS approved the directories without requiring the health plans to make any corrections. For the other four reviews, DHCS verified the information for up to 78 percent of the providers it sampled, and it ultimately required the health plans to correct all identified deficiencies before it approved their provider directories. We are concerned that significant deficiencies still existed across all six provider directories because the error rate noted among the sampled providers is an indication of additional errors in the directories. Because health plans were only required to update a portion of their provider directories, it is unlikely that any of the provider directories DHCS approved were at least 95 percent accurate. According to the chief of the Contract Oversight and Development Section (contract oversight chief), information in provider directories changes so frequently that the directories are already out‑of‑date when DHCS receives them from the health plans for a number of reasons, such as providers no longer working with a plan or changing their addresses. She said that hardcopy provider directories refer families to visit the health plans’ websites for a current list of providers because the plans are contractually required to update their websites within 30 days of the plan receiving updated provider information. However, she acknowledged that DHCS does not assess the accuracy of these web updates. Thus, DHCS’s review process is not sufficient to ensure that provider directories contain accurate information. The contract oversight chief explained that DHCS is exploring having its external reviewer verify select health plans’ provider directory information using a new approach, including the review of a larger sample size. She also stated that DHCS will continue to perform the directory reviews for two of the health plans and is considering increasing its sample size once the other reviews are transferred to the external reviewer. However, DHCS does not have a timeline for when it will transfer the verification process to the external reviewer. By not holding health plans accountable for having accurate provider directories, DHCS further compounds our concern about its ability to ensure that all families have access to a provider. DHCS Neglected to Take Many Other Actions We Recommended DHCS has not implemented many of the other recommendations we made in 2019 for increasing the number of children who receive critical health care services. These recommendations included implementing its external reviewer’s recommendations, sharing best practices between health plans for ways to develop and implement effective incentive programs, and requiring plans to identify and address barriers to children receiving preventive services. DHCS cited multiple reasons for its slow progress, including challenges associated with a staffing change and the ongoing pandemic. However, by failing to prioritize implementing our recommendations, CALIFORNIA STATE AUDITOR 9 Report 2022-502 | September 2022 DHCS has hindered its ability to ensure that children in Medi‑Cal receive critical preventive health services and has left certain children at risk of lifelong health consequences. To learn more about the other outstanding recommendations from our prior audit report, see https://www.auditor.ca.gov/reports/recommendations/2018‑111. Please refer to the section on page 1 to find the recommendations that we have made as a result of these audit findings. DHCS Has Not Done Enough to Increase the Number of Available Medi‑Cal Providers DHCS administers the Medi-Cal program ... Figure 4 ... but it incorrectly believes that it does not have authority to recruit additional Medi-Cal providers in underserved areas. As a result, some families may have to travel long distances to receive medical care. CALIFORNIA STATE AUDITOR 10 Report 2022-502 | September 2022 DHCS has not taken the steps we previously recommended to improve children’s access to providers who offer services to Medi‑Cal recipients. In our March 2019 report, we determined that one of the reasons children do not receive preventive services is that they do not have adequate access to nearby providers. We therefore recommended that DHCS identify where more Medi‑Cal providers are needed and request additional funding from the Legislature to increase the number of providers in those identified areas. However, DHCS claims that it does not have the authority to recruit providers into Medi‑Cal. To ensure that provider networks are adequate, state law establishes standards that set the allowable time and distance between beneficiaries and providers of many health care services. If a provider network is unable to meet those standards, the health plan must request an exception from DHCS. The standards typically require that a beneficiary travel no more than 10 miles, or 30 minutes, from the beneficiary’s residence to reach a provider for primary care. DHCS approved nearly 10,500 exceptions related to pediatric services in fiscal year 2020–21, according to the external reviewer. Each of these exceptions allow health plans to require beneficiaries to travel farther to reach pediatric service providers. For example, DHCS approved an exception for part of Monterey County, which allowed a plan to require beneficiaries to travel up to 58 miles, or more than 3 hours, to reach a pediatric primary care physician. Although changes in DHCS’s method for assessing compliance with these standards prevent us from comparing the number of exceptions to previous years, the large number of exceptions demonstrates a need for additional providers to ensure that children have adequate access to care. Nevertheless, DHCS has stated that it will not implement our recommendation to increase access to preventive health services for children in Medi‑Cal because it believes that addressing the shortage of health care workers in underserved areas is not within its purview. In addition, DHCS has not performed an analysis to identify the number of Medi‑Cal providers needed to ensure that all areas of the State are compliant with the time and distance requirements. DHCS claims that the responsibility for addressing the State’s shortage of health care workers lies solely with the Department of Health Care Access and Information (HCAI), formerly the Office of Statewide Health Planning and Development. We disagree. While part of HCAI’s mission is to promote an equitably distributed health care workforce, DHCS is the designated single state agency responsible for administering the Medi‑Cal program. This responsibility includes the obligation to ensure that there are enough Medi‑Cal providers to allow adequate access to children's health care services. Moreover, HCAI’s administrative deputy director agrees that state law governing HCAI does not prevent DHCS from recruiting providers into Medi‑Cal in underserved areas. Without a targeted effort by DHCS to increase the number of Medi‑Cal providers in underserved areas, children in Medi‑Cal will likely continue to face limited access to care. Despite DHCS’s assertion that it does not have authority to recruit providers, it is currently responsible for a program designed to recruit providers into Medi‑Cal. Specifically, DHCS has contracted with a nonprofit organization to administer a loan repayment program for newly practicing providers who agree that at least 30 percent of their patients will be enrolled in Medi‑Cal. The nonprofit organization CALIFORNIA STATE AUDITOR 11 Report 2022-502 | September 2022 that administers the program reported that since fiscal year 2018–19 it has annually recruited roughly 100 physicians who could provide preventive services to children. However, these efforts alone have not addressed the lack of access for children's preventive services, and DHCS does not even know how many providers it currently needs to recruit to provide adequate access to health care for children enrolled in Medi‑Cal. Unless DHCS takes immediate action to address these access issues, children may continue to miss preventive health care services, which could have a detrimental impact on their health for a lifetime. Please refer to the section on page 1 to find the recommendations that we have made as a result of these audit findings. Scope and Methodology Government Code section 8546.1(d) authorizes the California State Auditor (state auditor) to conduct follow‑up audit work on statutorily mandated or legislatively required financial and performance audits. In March 2019, the state auditor published a report titled Department of Health Care Services—Millions of Children in Medi‑Cal Are Not Receiving Preventive Health Services (2018‑111). Our follow‑up audit assessed DHCS’s implementation of the eight outstanding recommendations from that report. The table below describes our methods for evaluating DHCS’s progress. Audit Objectives and the Methods Used to Address Them AUDIT OBJECTIVE METHOD 1 Review and evaluate the laws, rules, Identified and reviewed relevant federal and state laws, rules, and regulations and regulations significant to the related to timely access to care and utilization of preventive services for audit objectives. children receiving Medi-Cal benefits. 2 Analyze the most recent three years Analyzed DHCS’s data to evaluate the use of preventive care by children of DHCS’s Medi-Cal data to determine statewide and by age, language, ethnicity, and county. Our analysis included whether utilization rates for children’s child Medi-Cal beneficiaries with full-scope benefits, which includes primary preventive health services have improved. care benefits, who were eligible for 11 or more months at a given age. If utilization rates have not improved, For infants, our analysis included beneficiaries who were eligible for eight determine what steps DHCS plans to or more months before their first birthday. We calculated utilization rates take to ensure that children receive the using the Bright Futures recommended schedule of care, with the exception preventive health services to which they of infants. According to DHCS, infants may be tracked under their mother’s are entitled. identifying number for three months and may not receive their own identifying number until 4 months of age. Therefore, we could only reasonably track the data for up to three of the seven infant services recommended by the Bright Futures schedule. As such, we considered infants who received three or more services before their first birthday to have received the recommended number of services. continued on next page . . . CALIFORNIA STATE AUDITOR 12 Report 2022-502 | September 2022 AUDIT OBJECTIVE METHOD 3 Evaluate DHCS’s progress toward improving its oversight of health plans. Specifically, determine whether: a. DHCS requires plans to use their • Interviewed DHCS staff regarding whether it requires plans to identify utilization management programs to barriers to usage and holds them accountable for addressing those identify barriers to usage for children’s barriers. preventive services and holds the • Reviewed DHCS’s Preventive Services Reports to determine whether plans accountable to address any they identified barriers to usage. identified barriers. b. DHCS implemented higher accuracy • Interviewed DHCS staff regarding whether it made improvements to its standards for health plan provider provider directory review process. directories and ensured that staff adhere • Reviewed DHCS’s policies and procedures for conducting its provider to its policy to retain all documentation directory reviews. related to its review of provider directories for at least three years. • Evaluated a selection of provider directory reviews that DHCS should have conducted dating back to 2019 to determine whether it implemented an increased sample size, ensured that provider directories were 95 percent accurate before approving them, and retained evidence of its reviews for three years. c. DHCS is adequately monitoring and • Interviewed DHCS staff regarding whether it identified effective identifying effective incentive programs incentive programs at the health plan level and shared best practices at the health plan level and sharing the across all health plans. results with all plans to help increase • Evaluated the sufficiency of DHCS’s 2021 Primary Care Incentive Tracking utilization rates. Report for informing plans of best practices they should consider implementing to improve usage of children’s preventive services. 4 Determine what progress DHCS has • Interviewed DHCS staff regarding whether it enhanced its written made toward including clearer and more communications with beneficiaries about children’s preventive services comprehensive information in its written and whether it performed annual outreach with beneficiaries who did materials about children’s preventive not use such services. health services available through Medi-Cal • Reviewed DHCS’s requirements for the health plans to conduct a and ensuring annual follow-up with any one-time outreach campaign to beneficiaries who had not used all of children and their families who have not the required children’s preventive services and assessed DHCS’s process used those services. for ensuring that health plans submitted evidence of compliance. 5 Evaluate what steps, if any, DHCS has taken to increase access to preventive health services for children in areas where they are needed most. a. If DHCS does not plan to implement • Interviewed DHCS staff regarding whether it performed an analysis the State Auditor’s recommendation to identify areas where providers are needed most. In addition, we to request funding increases from obtained DHCS’s perspective on its decision not to implement our prior the Legislature for the purpose of audit report’s recommendation. recruiting more providers in these areas, • Reviewed documentation about DHCS’s loan repayment program. determine whether DHCS has taken other steps to mitigate the problem. b. Determine whether beneficiaries are • Interviewed DHCS staff regarding its process for approving health required to travel significant distances plans’ requests for exceptions to the time and distance standards. to obtain children’s preventive • Reviewed DHCS’s reports on exceptions to the time and health services. distance standards. 6 Review and assess any other issues that are Interviewed DHCS staff and evaluated DHCS’s work products regarding its significant to the audit. efforts to implement our three prior recommendations related to the work its external reviewer performs. CALIFORNIA STATE AUDITOR 13 Report 2022-502 | September 2022 Assessment of Data Reliability The U.S. Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of the computer processed information that we use to materially support our findings, conclusions, or recommendations. In performing this audit, we relied on data obtained from DHCS related to health care procedures and patient demographics. To evaluate these data, we performed electronic testing of the data, reviewed existing information about the data, and interviewed agency officials knowledgeable about the data. As a result of our testing, we found the data to be of undetermined reliability. Although this determination may affect the precision of the numbers we present, there is sufficient evidence in total to support our findings, conclusions, and recommendations. Agency Perspective DHCS generally agreed with our recommendations and indicated that it will take action to implement them. However, it disagreed with our recommendation that it identify where more providers who see children covered by Medi‑Cal are needed and propose to the Legislature funding increases to recruit more providers in these areas. See the state auditor’s comments on the response from DHCS. We conducted this follow‑up audit in accordance with generally accepted government auditing standards and under the authority vested in the California State Auditor by Government Code section 8546.1 (d) and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives specified in the Scope and Methodology section of the report. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, MICHAEL S. TILDEN, CPA Acting California State Auditor CALIFORNIA STATE AUDITOR 14 Report 2022-502 | September 2022 State of California—Health and Human Services Agency Department of Health Care Services MICHELLE BAASS GAVIN NEWSOM DIRECTOR GOVERNOR August 26, 2022 Michael S. Tilden * Acting State Auditor California State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 DRAFT AUDIT REPORT RESPONSE Dear Mr. Tilden: The Department of Health Care Services (DHCS) is submitting the enclosed response to the California State Auditor (CSA) draft audit report number 2022-502 titled, “Department of Health Care Services Follow-up: It is Still Not Doing Enough to Ensure That Children in Medi- Cal Receive Preventive Health Services.” As we discussed with your office upon initiation of and throughout your follow-up audit, preventive health services for children in Medi-Cal is a priority for DHCS; however, DHCS and our partners have continued to be focused on appropriate response to the unprecedented COVID-19 public health emergency as well as its impending end and unwinding to ensure minimum disruption to the one in three Californians who rely on DHCS for their vital health services. Medi-Cal covers half of California’s births and more than half of California’s school- age children, and in March 2022, DHCS issued Medi-Cal’s Strategy to Support Health and Opportunity for Children and Families — a forward-looking policy agenda for children and families enrolled in Medi-Cal. Further, DHCS is in the midst of transforming Medi-Cal managed care to improve equity, quality, access, and transparency via our first-ever procurement of our Medi-Cal managed care plan contractors. Raising our expectations of plans in the Medi-Cal program, the restructured and more robust managed care contract reflects DHCS’ intention to hold all plan partners and their subcontractors more accountable. We are excited to implement our transformational policies to improve the health outcomes of millions of California’s children and families and look forward to your office’s continued oversight and support. In the above draft audit report, CSA issued eight recommendations for DHCS. DHCS agrees with CSA’s recommendations, with the exception of Recommendation 8, and has prepared Director’s Office 1501 Capitol Avenue, MS 0000 P.O. Box 997413, Sacramento, CA 95899-7413 Phone (916) 440-7400 Internet address: www.dhcs.ca.gov * California State Auditor’s comments begin on page 25. CALIFORNIA STATE AUDITOR 15 Report 2022-502 | September 2022 Mr. Michael S. Tilden Page 2 August 26, 2022 corrective action plans for implementation. In addition, DHCS noted certain inaccuracies in 1 CSA’s draft audit report. DHCS appreciates the work performed by CSA and the opportunity to respond to the draft audit report. If you have any other questions, please contact Internal Audits at (916) 445-0759. Sincerely, Michelle Baass Director Enclosure cc: See Next Page CALIFORNIA STATE AUDITOR 16 Report 2022-502 | September 2022 Mr. Michael S. Tilden Page 3 August 26, 2022 cc: Jacey Cooper Palav Babaria State Medicaid Director Deputy Director & Chief Quality Officer Chief Deputy Director Quality and Population Health Health Care Programs Management Department of Health Care Services Department of Health Care Services MS 0000 MS 0020 P.O. Box 997413 P.O. Box 997413 Sacramento, CA 95899-7413 Sacramento, CA 95899-7413 Jacey.Cooper@dhcs.ca.gov Palav.Babaria@dhcs.ca.gov Erika Sperbeck Susan Philip Chief Deputy Director Deputy Director Policy and Program Support Health Care Delivery Systems Department of Health Care Services Department of Health Care Services MS 0000 MS 4050 P.O. Box 997413 P.O. Box 997413 Sacramento, CA 95899-7413 Sacramento, CA 95899-7413 Erika.Sperbeck@dhcs.ca.gov Susan.Philip@dhcs.ca.gov Lindy Harrington Saralyn Ang-Olson Deputy Director Chief Compliance Officer Health Care Financing Office of Compliance Department of Health Care Services Department of Health Care Services MS 4050 MS 1900 P.O. Box 997413 P.O. Box 997413 Sacramento, CA 95899-7413 Sacramento, CA 95899-7413 Lindy.Harrington@dhcs.ca.gov Saralyn.Ang-Olson@dhcs.ca.gov René Mollow Wendy Griffe Deputy Director Chief Health Care Benefits and Eligibility Internal Audits Department of Health Care Services Department of Health Care Services MS 4000 MS 1900 P.O. Box 997413-7413 P.O. Box 997413 Sacramento, CA 95899 Sacramento, CA 95899-7413 Rene.Mollow@dhcs.ca.gov Wendy.Griffe@dhcs.ca.gov CALIFORNIA STATE AUDITOR 17 Report 2022-502 | September 2022 Department of Health Care Services Audit: Department of Health Care Services Follow-up: It is Still Not Doing Enough to Ensure That Children in Medi-Cal Receive Preventive Health Services Audit Entity: California State Auditor Report Number: 2022-502 (22-18) Response Type: Draft Audit Report Response Summary: The Department of Health Care Services (DHCS) noted certain inaccuracies in the California State Auditor’s (CSA) draft audit report. CSA agreed to change Items 2 2 and 3. DHCS is currently awaiting CSA feedback for Items 1 and 4. 1. On page 10, CSA stated that DHCS only required plans to inform families about 3 4 a limited number of children’s preventive services. This is incorrect. DHCS provided edits to the verbiage to state the following: “DHCS developed and shared materials for Managed Care Plans (MCP) o to disseminate to providers to increase child and adult immunizations (see attached). In addition, DHCS has issued guidance that emphasized requirements to plans regarding American Academy of Pediatrics, Bright Futures Guidelines through APL 19-101. DHCS has not issued guidance 5 that would limit the availability of children’s preventive service.” 2. On page 12, CSA stated that DHCS cited anecdotal concerns that providers were operating at reduced capacity during the pandemic. This is incorrect. CSA 6 agreed to modify the draft audit report to state “Citing anecdotal concerns verbal reports from health plans that providers were operating at reduced capacity during the pandemic…” 3. On pages 12 and 13, there are two different dates on the release of the planned outreach materials. On page 12, the last sentence references January 2023; on page 13, the last sentence references December 2022. CSA agreed to update 7 following dates in the draft audit report to state the following: Page 12: “….it intends to require all plans to develop a robust strategy for o addressing this underutilization by January July 2023” Page 13: “DHCS hopes to launch the new materials by December o 2022 March 2023” 4. On pages 15 and 16, CSA did not capture correctly information based on DHCS’ 8 interview. DHCS provided CSA edits to the verbiage in the following two paragraphs: “According to the chief of the contract Oversight and Development Section o (contract oversight chief), provider directories change so frequently…” “The contract oversight chief explained that DHCS is exploring having its o external reviewer verify….” Draft Audit Report Response | 22-18 Page 1 of 8 CALIFORNIA STATE AUDITOR 18 Report 2022-502 | September 2022 Finding 1 California continues to struggle to ensure that millions of children in Medi-Cal receive preventive care. Recommendation 1 To help increase utilization rates, DHCS should begin to monitor and identify effective incentive programs at the plan level and share the results with all plans. Agreement: Agrees with Recommendation Implementation: Will Implement Estimated Implementation Date: 7/14/2022 9 Implementation Plan: DHCS implemented an annual Supplemental Data Request (SDR) in 2021 to collect information on value-based payment (VBP) models designed to monitor and identify effective incentive programs to improve health and wellness measures for children and adolescents. The SDR allows DHCS to track the volume of incentives for child and adolescent care within the program, estimate the number of members covered under such VBP models, and share successful solutions with all plans. The 2021 report was shared with all plans in the meeting materials during the July 14, 2022, quarterly Chief Medical Officer (CMO) meeting. The 2022 report will be shared and discussed during the upcoming October 13, 2022, quarterly CMO meeting. Reports for future years will be completed and shared in a similar manner on an annual basis going forward. In addition, the 2022-23 Budget included an additional $700 million in funding for Health Equity and Practice Transformation Payments. DHCS will be launching the new incentive program in 2023, which includes specific milestones and targets to improve children’s preventive services and support catching up on key services that were missed during the COVID-19 public health emergency. Recommendation 2 To ensure that plans address underutilization of children’s preventive services, DHCS should require plans to use their utilization management programs to identify barriers to usage specifically for these services and hold the plans accountable to address the barriers they identify. Agreement: Agrees with Recommendation Implementation: Will Implement Estimated Implementation Date: 7/1/2023 Implementation Plan: DHCS will be launching strengthened requirements for health plans to monitor and address underutilization as a part of 2023 MCP contracts and the launch of the CalAIM Population Health Management program on January 1, 2023. Specifically, plans will be Draft Audit Report Response | 22-18 Page 2 of 8 CALIFORNIA STATE AUDITOR 19 Report 2022-502 | September 2022 required to regularly monitor underutilization of children’s preventive services, including stratifications by race, ethnicity and other demographics and come up with strategies to address. Plans will also be required to ensure not only do children have access to primary care but are engaged and have continuity with a primary care provider to address historic distrust in some communities, which drives health disparities as well as underutilization. DHCS will be monitoring the efficacy of MCP activities through HEDIS measures that track children’s service utilization and service delivery, as well as detailed reporting requirements for the PHM Program, which will be finalized in the upcoming months. In addition, consistent with federal regulations, DHCS will work with the External Quality Review Organization (EQRO) to perform a study to assess a plan’s methods for addressing barriers that may exist and result in underutilization of services evident in the Preventive Services Report. DHCS will utilize the results to develop next steps and will monitor progress of improvement, ensuring plans are making strides toward improvement. DHCS anticipates receiving initial monitoring reports from MCPs by Q2 2023 so will have a complete monitoring strategy implemented by 7/1/2023. Finding 2 DHCS has not taken adequate steps to ensure that children receive important health care. Recommendation 3 To better ensure the accuracy of its data and ensure that California receives all available federal Medicaid funding, DHCS should require its External Quality Review Organization (external reviewer) to perform its encounter data validation studies annually using the most recent set of data available, and it should implement recommendations from its external reviewer studies. Agreement: Agrees with Recommendation Implementation: Will Implement Estimated Implementation Date: 1/1/2023 Implementation Plan: Through close collaboration with EQRO, consistent with federal requirements, DHCS has already implemented encounter data validation (EDV) studies and followed up on EQRO requirements. The final results of the 2021 through 2022 EDV study will be published on January 1, 2023, and DHCS will implement recommendations and monitor improvement. DHCS will work with plans to ensure EQRO recommendations are examined for applicability and to track progress. Recommendation 4 To mitigate health disparities for children of differing ethnic backgrounds and language needs, DHCS should revise the methodology for its external reviewer’s health disparity study to enable it to better make demographic comparisons, and it should use the Draft Audit Report Response | 22-18 Page 3 of 8 CALIFORNIA STATE AUDITOR 20 Report 2022-502 | September 2022 findings to drive targeted interventions within plan service areas. It should publish this study annually. Agreement: Agrees with Recommendation Implementation: Will Implement Estimated Implementation Date: 1/1/2023 Implementation Plan: As a part of DHCS’ 2022 Comprehensive Quality Strategy that was submitted to CMS in January 2022, DHCS identified children’s preventive care as one of its three clinical focus areas and also launched the 50x2025 Bold Goals initiative that includes specific targets to reduce racial and ethnic disparities for well-child visits and immunizations by 50 percent by 2025. While working with the EQRO is one piece of DHCS’ strategy to achieve this goal, DHCS is also taking steps to incorporate reduction of racial and ethnic disparities in children’s preventive care to its value based payment programs (including MCP capitation rates, its forthcoming FQHC Alternative Payment Model program, and hospital-based quality incentive programs), as well as specifically supporting primary care practices to scale evidence-based practices that reduce racial and ethnic disparities as a part of its Health Equity and Practice Transformation Payment program that will launch in 2023. In addition, DHCS works closely with the EQRO to update annual health disparities studies and incorporate recommendations. DHCS will work on including a more robust demographic analyses spanning broader than racial/ethnicity categorizations and develop targeted approaches to address disparities evident in the report. Recommendation 5 To improve its ability to ensure that children are receiving recommended preventive health services, DHCS should create an action plan to annually address its external reviewer’s recommendations relating to children’s preventive services, including any recommendations left unaddressed since the external reviewer’s fiscal year 2015-16 reports. Agreement: Agrees with Recommendation Implementation: Will Implement Estimated Implementation Date: 1/1/2023 Implementation Plan: In collaboration with the EQRO, DHCS will develop an action plan to address findings/gaps discovered in the Preventive Services Report to improve services for children. Additionally, DHCS will assess previous findings and evaluate methods for improvement. Draft Audit Report Response | 22-18 Page 4 of 8 CALIFORNIA STATE AUDITOR 21 Report 2022-502 | September 2022 Recommendation 6 To ensure that eligible children and their families know about all the preventive services they are entitled to through Medi-Cal, DHCS should include clearer and more comprehensive information about those services in its written materials and ensure annual follow-up with any children and their families who have not used those services. Agreement: Agrees with Recommendation Implementation: Will Implement Estimated Implementation Date: 3/31/2023 Implementation Plan: DHCS is currently updating outreach materials to be in alignment with the Strategy to Support Health and Opportunity for Children and Families. DHCS is currently targeting to release the updated materials by March 2023 to the MCP. These updated materials are being designed to provide more in-depth information regarding the Early and Periodic Screening, Diagnostic and Treatment (EPSDT) requirements, which covers the spectrum of what Medicaid programs must provide to its covered children – ranging from preventive care to diagnostic and treatment services based on the outcomes of required screenings. These updated materials will essentially be a toolkit that target both enrolled beneficiaries and their families as well as enrolled providers. Informational brochures will be targeted to the children and families and provider trainings will be developed for enrolled providers, all focusing on the EPSDT requirement. In order to ensure our messaging is on point and resonates with our beneficiaries, DHCS will engage in focus testing with both enrolled beneficiaries and key stakeholders on the beneficiary facing materials as well as a renaming of the term EPSDT to, again, more appropriately resonate with beneficiaries, families, and providers. Included in the materials for the beneficiaries will also be a newly developed “Know Your Rights” document relative to the EPSDT requirements. Upon implementation, DHCS will direct the plans to send written materials to beneficiaries under the age of 21, including any family members, by March 2023 and annually thereafter. The materials will include information on the broad range of services that encompass EPSDT, including all preventive services and explain the importance of using the services. The materials will also explain the services are free to use. By way of background, DHCS worked with the Center for Health Literacy to develop various written member materials for either posting or distribution to beneficiaries. The materials include social media and website content, notices and brochures, and these materials describe the importance of checkups and screenings. Due to DHCS’ efforts to reevaluate carefully the materials as part of the larger, department-wide effort to ensure consistent messaging across all types of communications, the materials have not been yet published. To further emphasize the importance of required preventive health care services, including EPSDT, effective January 1, 2024, DHCS is contractually requiring all MCPs to train and educate providers on the intent and extent of EPSDT. The training must be provided on an ongoing basis, at least once every two years, and include training on Draft Audit Report Response | 22-18 Page 5 of 8 CALIFORNIA STATE AUDITOR 22 Report 2022-502 | September 2022 data collection and reporting, Population Health Management Program requirements, and health education resources, among other requirements. Recommendation 7 To ensure that plan provider directories are accurate, DHCS should begin using a 95 percent confidence level and not more than a 10 percent margin of error on its statistical sampling tool and should require at least 95 percent accuracy before approving a plan’s provider directory. In addition, DHCS should ensure that its staff adhere to its policy to retain all documentation related to its review of provider directories for at least three years. Agreement: Agrees with Recommendation Implementation: Will Implement Estimated Implementation Date: 9/30/2023 Implementation Plan: The Provider Directory Validation work through the EQRO resumed in the first quarter of 2022. The EQRO used the established process for conducting provider directory validation and provided the first and second set of quarterly results to DHCS in June and August 2022 respectively. The results are based on outbound calls to Providers that are included in the 274 file. DHCS is currently reviewing the results; however, DHCS can confirm the EQRO did use the required 95 percent confidence level and not more than a 10 percent margin of error in the statistical sampling tool. After the review 10 of the results is completed by DHCS, plans will be required to respond to any measures identified as having a confidence level lower than 95 percent to indicate how the plans will increase the success of such measures for the next quarter. DHCS has policies and procedures in place that require staff to maintain all documentation related to the review of the provider directories, and we are in the process of implementing a SharePoint site that will act as a repository for all managed care plan submissions reviewed by the Managed Care Operations Division, including all documents associated with the reviews. Finding 3 DHCS has not done enough to increase available Medi-Cal providers. Recommendation 8 To increase access to preventive health services for children in areas where they are needed most, DHCS should identify where more providers who see children are needed and propose to Legislature funding increases to recruit more providers in these areas. Agreement: Disagrees with Recommendation Implementation: Will Not Implement Draft Audit Report Response | 22-18 Page 6 of 8 CALIFORNIA STATE AUDITOR 23 Report 2022-502 | September 2022 Estimated Implementation Date: None Implementation Plan: Although DHCS agrees increasing the number of physicians practicing in California would be beneficial for all health care delivery systems and DHCS has overseen the Proposition 56 Medi-Cal Physicians and Dentists Loan Repayment Program (known as CalHealthCares), DHCS maintains that broader workforce recruitment is beyond DHCS’ 11 purview. CalHealthCares was established by Senate Bill 849 (Chapter 47, Statutes of 2018) and appropriated a one-time allocation of $220 million for the loan assistance program for recently graduated physicians and dentists. An additional $120 million was added to the program in the 2019-20 Budget, for a total of $340 million. Senate Bill 395 (Chapter 489, Statutes of 2021) provided ongoing funding for the program. Additionally, as part of the 2022-23 Budget, DHCS requested to allow for all remittance from MCPs that do not meet the minimum MLR standard of 85 percent for a reporting year be remitted to the loan repayment fund to provide additional revenues to support the program. DHCS implemented the CalHealthCares program in 2019, requiring physicians and dentists who participate in the program to commit to serving at least 30 percent Medi-Cal beneficiaries for five years. DHCS has awarded four cohorts of physicians and dentists with the latest round being issued in July 2022. For Cohort 4, DHCS focused on equity, such as languages spoken by physicians, treatment of children ages zero to three for dental practices, and the number of special needs patients seen by dentists. Also, the scoring considered experience working with underserved communities and the ability to provide culturally and linguistically competent care to Medi-Cal communities. Since 2019, DHCS has provided Physicians for a Healthy California data from the Medi-Cal managed care network certification to support CalHealthCares awardee decisions. DHCS anticipates 99 percent of beneficiaries will be enrolled in MCPs by 2024. DHCS is committed to ensuring that MCP networks are sufficient to ensure timely access to care for Medi-Cal members. DHCS has a robust process for reviewing networks, 12 certifying adequacy, and refining our process to improve monitoring of networks and timely access to care. Current monitoring activities include surveying providers to assess compliance with appointment wait time standards; monitoring access to care grievances; and network certifications of MCPs. DHCS continually evaluates the processes to improve our ability to monitor and oversee MCP compliance with network adequacy requirements. Additionally, DHCS will have an external evaluator conduct an access assessment by reviewing the various components of access, comparing MCP networks and access to care with commercial and Medicare Advantage markets. Through the assessment, DHCS aims to compare access in Medi-Cal managed care to the services for Californians who receive health coverage through the commercial or Medicare Advantage markets. This will allow DHCS to assess if any barriers to access are a reflection of systematic problems across the entire health care system—such as issues Draft Audit Report Response | 22-18 Page 7 of 8 CALIFORNIA STATE AUDITOR 24 Report 2022-502 | September 2022 related to workforce pipeline—or if access barriers are specific to Medi-Cal. The comparison will enable DHCS to potentially refine Medi-Cal’s access and network standards and inform future monitoring efforts. In addition, the assessment will track different levels of access to help determine whether issues exist at the managed care level (network adequacy), provider level (scheduling practices, responsiveness to member scheduling requests, etc.) or both. The assessment will be shared with CMS in March 2026. Lastly, the 2024 MCP Contract will obligate MCPs to provide more visibility into the payments for value, including reporting on the primary care spending as a percentage of total expenditures to help ensure sufficient investment in upstream and preventive care. The requirement will apply to all MCPs statewide across all plan model types effective January 1, 2024. DHCS has various statutory and contractual levers to be able to enforce the requirements. Draft Audit Report Response | 22-18 Page 8 of 8 CALIFORNIA STATE AUDITOR 25 Report 2022-502 | September 2022 California State Auditor's Comments on the Response From the Department of Health Care Services To provide clarity and perspective, we are commenting on DHCS’s response to our audit. The numbers below correspond to the numbers we have placed in the margin of DHCS’s response. As indicated in the comments below, we agreed to make several minor changes to 1 our report text after receiving further information and perspective from DHCS during its review of the draft report. However, there were no inaccuracies in the draft report we provided to DHCS for its response. As is our standard practice, we reached out to DHCS while it was reviewing our 2 draft report to discuss any concerns or questions it may have had about the draft report. We discussed with DHCS the four points that DHCS listed in its detailed response and, although there were no inaccuracies in the draft report, we agreed to make several minor changes to our report text based on additional information and perspective that DHCS provided. DHCS’s response uses page number references from a draft copy of our report. 3 Since we provided DHCS the draft copy, page numbers have shifted. DHCS’s response is inaccurate. Subsequent to the 2019 document DHCS references 4 and, as we discuss beginning on page 6, DHCS minimized the effectiveness of its outreach campaign that ended in June 2021 by only requiring plans to educate families about a subset of required services instead of requiring plans to inform families about all of the children’s preventive services available to them. Contrary to DHCS’s implication, we never state that it issued guidance that 5 would limit the availability of children’s preventive services. However, as we note beginning on page 6, DHCS limited its outreach regarding these services by only requiring health plans to educate families about a limited number of the required preventive services. Our audit report accurately reflects the evidence and department perspective 6 provided during the course of the audit. Although we agreed to a minor change in the report text, the fact remains that DHCS relied on verbal reports from health plans and was unable to provide empirical evidence supporting its concern. DHCS’s response is misleading: it asserts that the two estimated completion 7 dates we referenced in our draft audit report were inaccurate. However, the draft audit report we provided for DHCS’s review accurately reflected the department perspective it provided during the course of the audit. Further, on page 7, we state that because DHCS is still in the planning process, we do not have assurance that the timeline it provided was realistic. Upon reviewing the draft audit report, DHCS requested to update the prior statements it made to us about the estimated CALIFORNIA STATE AUDITOR 26 Report 2022-502 | September 2022 completion dates to reflect that it would need more time to implement our recommendation. We previously informed DHCS that we would update those completion dates, which are shown on pages 6 and 7. Contrary to DHCS’s assertion, the draft and final report text accurately reflect the 8 results of our interviews with the contract oversight chief, which she confirmed to us in writing. Although DHCS asserts in its response that it implemented our recommendation 9 on July 14, 2022, the information it shared with the health plans during this time lacked sufficient detail about identified best practices for incentive programs to be useful. Further, as indicated in its own response, DHCS plans to share additional information with the health plans in October 2022 and launch a new incentive program in 2023. Therefore, DHCS's implementation of this recommendation will not occur until sometime in 2023. DHCS is not addressing all aspects of our recommendation. Specifically, DHCS’s 10 response focuses solely on the confidence level and margin of error but does not address ensuring that health plans' provider directories are 95 percent accurate. As we note on page 8, we found that for two reviews DHCS was only able to validate information for about 60 percent of the providers it sampled, yet DHCS approved the directories without requiring the health plans to make any corrections. DHCS incorrectly links our recommendation with a broader effort to increase 11 the number of physicians practicing in California. However, the context of our March 2019 report and this follow‑up report make it clear that this recommendation relates to increasing the number of Medi‑Cal providers in underserved areas, not simply increasing the number of physicians in California. As the state agency solely responsible for administering Medi‑Cal, DHCS has the authority to implement this recommendation. Although DHCS asserts that it has a robust process for reviewing networks, we note 12 on page 10 that its process does not include an analysis that would allow it to identify the number of Medi‑Cal providers needed to ensure that all areas of the State are compliant with the time and distance requirements. Without such an analysis, DHCS is unable to make a targeted effort to increase the number of Medi‑Cal providers who provide services to children in underserved areas.