Crhd Deliverable 3 Final 070314

Evaluating the Impact of the Mental Health Services Act on Reducing Disparities in Access Deliverable 3 Final Report Recommendations for Assessing California’s Investment in Mental Health Services Principal Investigator Sergio Aguilar-Gaxiola, M.D., Ph.D. Professor of Clinical Internal Medicine Director, Center for Reducing Health Disparities Co-Principal Investigators Estella M. Geraghty, M.D., M.S., M.P.H., GISP Associate Professor of Clinical Internal Medicine Debora A. Paterniti, Ph.D. Adjunct Professor, Departments of Internal Medicine (General Medicine) and Sociology, and Center for Healthcare Policy and Research Funded by the Mental Health Services Oversight and Accountability Commission (MHSOAC) July 1, 2014 Table of Contents Introduction………………………………………………………………………………………………1 Evaluation of the Mental Health Services Act (MHSA): Assessing the Response to Mental Health Disparities…………………………………………………….…………………………………1 Framing the Key Recommendations……………………………………………………………2 Key recommendation 1: improve Client Services Information (CSI).……………………...2 Key recommendation 2: improve county reporting on population data……………………3 Key recommendation 3: improve mental health service targeting to reduce disparities…4 Key recommendation 4: improve workforce development efforts to address disparities..6 Recommendations Based on Analysis of Client and Family Member Perspectives Regarding the Impact of the MHSA on Disparities…………...…………………………………..7 Key recommendation 5: increase MHSA-funded programs aimed at access and quality care outcomes.……………………....................................................................................7 Key recommendation 6: mitigate barriers to effectively deliver mental health care in California…………………………………………………………………………………………8 Key recommendation 7: assess gaps and persistent issues for un(der)served groups…9 Recommendations Based on the Public Review and Comment of Quality Findings……..10 Key Implications……………………………………………………………………………………….12 Research Implications for Continued Evaluation to Improve Access and Outcomes..12 Practice Implications for Quality Improvement and Quality of Care Outcomes……….13 Conclusion…………….…………………………..…………………………………………………....14 References………………………………………………………………………………………….......15 i i Introduction The Mental Health Services Act (MHSA), which was enacted in 2004, provided California with the opportunity to greatly enhance support for county mental health programs. MHSA brought increased funding to improve the mental health status of individuals and families from all age groups and from diverse communities (California Department of Health Care Services, 2014). An estimated $7.4 billion has been distributed to counties between fiscal years 2006-2007 to 2011-2012 (California State Auditor, 2013). With this groundbreaking investment, California aimed to transition public mental health systems from a crisis response model to a more proactive prevention model (Lee, 2012). To monitor progress and quality improvement of this unprecedented initiative, the Mental Health Services Oversight and Accountability Commission (MHSOAC) has increasingly supported and invested in ongoing evaluation studies. Evaluating the effectiveness of an innovative Act meant to transform a large and diverse state’s county mental health systems, however, includes a number of challenges. In a recent study, Lee (2012) outlined four central challenges: (1) evaluation experts encounter limited county reporting requirements, (2) inadequate data systems, (3) a de-centralized public mental health system, and (4) a shortage of funds allocated for evaluation when seeking to investigate MHSA outcomes. Despite these challenges, a thorough evaluation of a mental health policy that distributed billions of dollars in funding over nearly a decade to all 58 California counties and two city sites is essential. In addition to the proper allocation of funds, it is important to investigate the benefits the MHSA has created for millions of diverse consumers. The California Health Interview Survey (CHIS) recently found that one in five (or 4.9 million) adults in California reported needing support for a mental health problem (Grant et al., 2011). Among the 4.9 million individuals in need of mental health care are considered to be vulnerable, underserved and inappropriately served groups whose access and treatment outcomes in relation to the MHSA deserve thorough investigation. The phrase underserved and inappropriately served will be used interchangeably with un(der)served throughout this report. Evaluation is an essential component of any health policy because it can help monitor whether aims are achieved, and can help guide future directions. Solin and Lehto (2011) contend that evaluation has become an important part in assessing the effectiveness of mental health programs in achieving positive mental health outcomes and in developing strategies to guide health promotion policy. Evaluation of the Mental Health Services Act (MHSA): Assessing the Response to Mental Health Disparities Between 2012 and 2014, the Center for Reducing Health Disparities (CRHD) sought to assess the outcomes and effectiveness of the MHSA, as well as related state and local policies and practices, with regard to reducing disparities in access to mental health services. This evaluation was guided by a mixed-methods approach, employing quantitative analyses of statewide data and qualitative Community-Based Participatory Research (CBPR) methods with underserved and inappropriately served groups. This report summarizes key evaluation findings and provides recommendations for future evaluation activities to monitor disparities in access 1 and quality of community mental health services. Recommendations focus on needs for continued evaluation and quality improvement. Framing the Key Recommendations The key recommendations featured in this report encompass key recommendations and actions that focus on two primary domains: (1) recommendations for future activities to support continued evaluation (CE) of disparities in access and outcomes for public community mental health services, and (2) recommendations for state and/or local quality improvement (QI) activities to reduce disparities based on the findings from our comprehensive MHSA evaluation. Within this report, we examined a number of recommended actions and each action was identified as CE, QI, or a combination of the two. Key recommendation 1: improve Client Services Information (CSI) The Center for Reducing Health Disparities utilized the Client Services Information (CSI) dataset to measure access to mental health systems among distinct demographic groups since the inception of MHSA, in 2005, through 2012. We analyzed trends in CSI data overall, and gender, race, and age strata. Overall, the results of our analysis indicated that there was a general trend toward increasing access to mental health services following the implementation of the MHSA until 2008. Many population subgroups saw declining access for 2009 and 2010 with an upturn in 2011. The greatest disparities in access to mental health services were seen in the adult age group (ages 26 to 59) and in older adults (over age 60). It was not possible to detect disparities by racial group since data were missing for the majority of the new clients for most years in the CSI database. Recommended actions at the county and state level to improve CSI data: 1.1 (CE domain): Increase continued evaluation efforts by investing locally and centrally to improve the CSI data system. Enhancing data collection, data management and data transfer, and additional staff support for evaluation at the county and state level is essential for continued evaluation and building upon each completed evaluation. Moreover, county- level support should include an increased investment of funds to support local data collection, data management, and data entry. Funding for at least one evaluation expert in each county would ensure that there is an ongoing focus on continued evaluation of CSI data. Local evaluation staff could monitor day-to-day and month-to-month activities, and report on successes, challenges, and lessons-learned on a regular basis, informing subsequent steps in local CSI data collection, data entry, and data management. 1.2 (QI domain): Engage in quality improvement activities by enhancing the CSI dataset to better assess quality of care outcomes and access to mental health treatment among underserved and inappropriately served groups of interest. Improvements in data collection are needed to combat a substantial challenge with regard to missing data on race and ethnicity, gender, and sexual orientation. In all counties across California, a large proportion 2 of the race and ethnicity data were missing between 2005 and 2012. Often, between one- third and one-half of data entries for client services lacked race and ethnicity data. It was not clear whether data were missing because mental health staff did not ask for the clients’ race and ethnicity or whether these data were not provided by clients when asked to identify their race and ethnicity. A key initial determination is needed to identify the factors that are contributing to inadequate data collection on race and ethnicity. It is recommended that clients be required to share their preferred race and ethnicity (e.g., “What is your primary race and ethnicity?”) during mental health service acquisition, verbally or in written form, and that mental health service staff should immediately enter these data in the CSI system 1.3 (QI domain): Include additional variables as recommended to measure access among underserved individuals by sexual orientation. Little is currently known about mental health services acquired by the lesbian, gay, bisexual, transgender, and questioning (LGBTQ) community. The CSI data system should include specific variables for gender identity and sexual orientation, and mental health service staff should be required to obtain these data from all clients during each mental health service interaction. 1.4 (CE domain): Address the substantial disparities among adults (age 26-59) and older adults (over age 60) by investing in mental health programs specific to these populations. Based on our evaluation results, these two groups, and especially older adults, were more likely to experience the greatest disparities in access to mental health services. Historically, health insurance companies and some mental health programs have been unsuccessful in prioritizing the needs of this age group. Making this group a priority and connecting their health care needs to MHSA programs and the Affordable Care Act will lead to improved mental health care for older adults. Key recommendation 2: improve county reporting on population data Overall, counties present population data that are comparable to the 2010 Decennial Census and Department of Finance (DOF) population data. While data sources varied considerably across counties, and data were presented in a variety of ways across CCPs, the overall county- specific counts and proportions for specific variables and demographic groups were similar when comparing general population data and census data. Recommended actions at the county and state level to improve quality of care outcomes: 2.1 (CE and QI domains): Establish a consistent data collection plan that counties can adopt and implement in a way that allows them to compile data with ease, and monitor progress over time. The plan should include templates for tables, figures, and text that counties can use to consistently insert their most recent data within a prescribed page limit. A consistent data collection procedure and template will facilitate: (1) increased consistency in data compilation and management across all counties, (2) increased data accuracy, and (3) a focus on provision of culturally responsive services to people from underrepresented groups (e.g., by race, age, sexual orientation and gender identity, and geographic regions) 3 2.2 (QI domain): Encourage counties to focus on reliable and consistent data sources. We recommend use of demographic and population data from the California DOF. Consistent use of DOF population data for population estimates will increase: (1) the reliability and validity of data and evaluation strategies relevant to populations being served, and (2) the effectiveness of county program design and implementation that bridges gaps in service to underrepresented groups in high priority areas, accounting for each county’s unique blend of cultural demographics. 2.3 (QI domain): Formulate a routine procedure for county-employed evaluation experts to collect and monitor local county data, using websites and document downloads that are consistently used across all counties throughout California. This will facilitate consistent collection, management, and analysis of county-level data, for multiple years, allowing for high-quality comparisons to previous years and to other counties. 2.4 (QI domain): Conduct biannual on-site reviews (fall and spring) of county-specific data collection, management, and analysis that can guide programs supported by MHSA funds. Ensure that measures are quantifiable, precise, and consistent over time. This action step can be accomplished by using a reliable data collection tool with specific measures. 2.5 (CE and QI domains): Hold biannual evaluation meetings (fall and spring) with county mental health evaluators, giving all county evaluators the opportunity to report back on evaluation successes, challenges, and lessons-learned. These biannual evaluation meetings would allow local and state-level evaluators to “take the pulse” of evaluation efforts on a regular basis, facilitating adjustments and improvements to the data collection and data management methods and systems, to ensure continued improvement in evaluation quality and consistency. Key recommendation 3: improve mental health service targeting to reduce disparities In our comprehensive evaluation, we found that counties often targeted a large number of population groups and subpopulations within their respective counties. Our evaluation results suggested that counties that targeted five or fewer priority groups, tended to focus on subpopulations with specific characteristics that placed individuals within high priority groups, most in need of mental health services, that were not based on demographic data alone. That is, these targets consisted of specific high-risk groups (e.g., “trauma-exposed individuals,” “children and youth in stressed families,” “adults with serious mental illness who are imminently at risk of institutionalization or homelessness”). Counties that practiced such focused targeting also appeared to be more effective in matching their CSS target populations to mental health programmatic developments. Other counties reported targets that focused on a large portion of the general population (e.g., “children,” “adults,” “Latinos”), rather than pinpointing specific high-risk groups. This highlights the issue as to whether targeting was effective in numerous counties, since ‘targeted’ resources 4 would be required to respond to mental health service needs in a broad portion of a county population, if not the entire county population. Recommended actions at the county and state level to improve county targeting of mental health services: 3.1 (CE and QI domains): It is recommended that counties be provided with specific and streamlined guidelines to assess local mental health population needs, in a consistent manner across all counties, and to select specific and focused CSS targets. Better targeting will lead to further improvement in the provision of mental health services, decreasing disparities in access to services among those who are disproportionately in need. The following recommendations are intended to help strengthen targeting on the county level: 3.1.1 (QI domain): Develop a targeting template for CCP requirements that assists counties in identifying three to five CSS targets that are specific (e.g., age group, racial and ethnic group, language, high-risk community), relevant, accurate, and precise for each county and city. Provide counties with examples of effective targeting as “best practice examples” in an effort to improve focus on the high-risk populations that are in greatest need of services, and as substantiated by county level data. Data gathered and entered into this template can also be used as part of the counties' annual update, strengthening outreach and recruitment activities, while fostering service provision to key target populations. 3.1.2 (CE and QI domains): Offer technical assistance to help each county, through semiannual site visits and semiannual evaluator meetings to explore gaps in gathering, managing, and analyzing CSS targeting data. Counties receiving MHSA funding to serve disadvantaged groups and overcome disparities should be required to produce specific targets that demonstrate a focused introspective assessment of needs within target groups that are most disproportionately served, with a goal of increasing services to these populations. Technical assistance can help counties achieve this goal. 3.1.3 (QI domain): Ensure that counties specifically targeted five or less target populations as an approach to adequately identify not only disparities in services, but also strategies to address and eventually reduce disparities. Specifically, when it comes to workforce education and training, it is important that the right people are hired to work with the counties’ CSS populations. We believe that counties that are more successful in matching their WET targets with their CSS target needs will be more focused on priority populations. Counties that focus on groups with specific characteristics such as, high-risk groups (e.g., “trauma-exposed individuals,” “children and youth in stressed families,” “adults with serious mental illness who are imminently at risk of institutionalization or homelessness”), will be more effective in matching their CSS target populations to programmatic developments. 5 Key recommendation 4: improve workforce development efforts to address disparities During completion of CCP reports, counties were asked to differentiate between CSS and WET targets to ensure a more accurate assessment and to discern gaps. Completeness of data for these two target categories varied across counties. Among the 24 counties that identified workforce targets, the targets frequently focused on the need to improve the diversity of mental health service staff. Many CCPs highlighted the need for hiring and training bilingual and bicultural mental health staff. Several counties (e.g., Alameda, El Dorado, Stanislaus, Sutter- Yuba) included WET targets that indicated a need for hiring and training mental health professionals with “lived experience,” meaning professionals who have experienced the effects of mental health conditions in their own lives, or in the lives of those around them. The racial composition of the mental health workforce in many counties demonstrated lower percentages of Latino staff that was needed according to the breakdown of Medical and CSS data, which portrayed higher percentages of clients within the Latino community. In some counties, there was also a disproportionately low percentage of mental health staff that were from the Asian/Pacific Islander or Native American community than was needed according to Medi-Cal and CSS data. The WET targets that were most often listed by counties reflected a need and desire to address these disparities. Recognizing the need of culturally competent staff that demonstrates a commitment to improve the delivery of services to unserved, underserved, and inappropriately served communities is critically important in rectifying disparities. Recommended actions at the county and state level to improve culturally and linguistically appropriate treatment: 4.1 (QI domain): Develop a streamlined, easy to complete, workforce targeting template to examine and monitor staff diversity and the individual needs of each county. Provide all counties with these templates, and provide best practice workforce targeting examples to improve staff diversity, bilingual capabilities, and cultural competence. Monitoring data gathered from this workforce targeting template can inform county-level annual updates in an effort to strengthen their workforce training and recruitment programs. Each county should, at a minimum, revise and update their workforce targeting worksheet on an annual basis. 4.2 (QI domain): Offer technical assistance to help each county explore gaps in gathering, managing, and analyzing workforce data through semiannual evaluator meetings/trainings and semiannual site visits. Existing templates that highlight the workforce composition on the county level can be can be streamlined and improved for clarity. Counties receiving MHSA funding to serve disadvantaged groups and overcome local disparities should be required to produce outcomes that demonstrate increases in workforce cultural competence, and evidence that disparities are decreasing. Technical assistance can help counties achieve this goal. 4.3 (QI domain): Expand the opportunities for hiring and inclusion of providers with lived experience for appropriate services, such as peer support groups. This recommendation’s core rests on the idea that lived experiences will strengthen relationships between 6 providers and consumers within targeted communities. It may be worthwhile to share best practices from counties that specifically targeted people with lived experiences (e.g. Alameda, El Dorado, Stanislaus, and Sutter-Yuba). 4.4 (CE and QI domains): Consider provision of hiring incentives to attract multicultural and multilingual staff in isolated, hard to reach counties and communities, and/or counties where disparities are particularly large. Recommendations Based on Analysis of Client and Family Member Perspectives Regarding the Impact of the MHSA on Disparities Through the use of qualitative data collection methods, including key informant interviews and focus group discussions, we gained keen insights into the viewpoints of clients and family members on the impact of the MHSA in addressing disparities in mental health access. By conducting key informant interviews with stakeholders involved in the advocacy, administration, delivery, or receipt of mental health services, and focus groups with populations who have historically been un(der)served, we were able to learn about local level experiences in accessing MHSA supported services, and obtain perspectives on multiple and diverse barriers of access to the public mental health system for un(der)served groups in California. We recommend continued use of mixed methods approaches to evaluate disparities in access to mental health services in California. Key recommendations 5: increase MHSA-funded programs aimed at access and quality care outcomes To fulfill the MHSA’s mandate to improve access to and enhance the quality of mental health care of un(der)served groups, routine implementation and systematic evaluation of novel strategies offered through MHSA-funded Innovative Programs should occur in conjunction with (a) community involvement, (b) workforce diversity and (c) consumers and family integration. Recommended actions at the county and state level to increase Innovative programs that show efficacy in engagement and treatment of underserved groups: 5.1 (CE and QI domains): Continue the systematic multi-method data collection and evaluation in order to document local lived experiences among subpopulations of consumers and providers who provide insider perspectives on successes, challenges, and potential future directions. 5.2 (QI domain): Improve access to treatment and delivery of mental health care for underserved groups, we recommend that involvement of community stakeholders continue in community assessments and program planning. While several counties have specific procedures for including underserved groups in program development, some do not. We recommend that data on “model community” outreach and involvement efforts be collected and shared across counties. Best practice models should be constructed for 7 counties based on these data and should include consideration of the scope of mental health service needs and the diversity of county-level consumers. Model communities should be engaged and evaluated for efficacy. 5.3 (CE and QI domains): Sustain the progress made by PEI and FSP programs and services by supporting on-going research. Additionally, we recommend that these programs be evaluated with specific emphasis on whether or not they have improved access and treatment outcomes for underserved groups. To accomplish this, standardization and improvement in data collection practices related to PEI and FSP programs and services must occur. Key recommendation 6: mitigate barriers to effectively deliver mental health care in California Through the qualitative component, specifically the community-based participatory research (CBPR) approach, of our MHSA evaluation, we were able to capture the views of key informants who provided suggestions to mitigate individual and organizational barriers. Individual barriers such as, language, stigma, geography, poor or no knowledge of services, and personal cost of available care; and organizational barriers, including quality and range of care, service capacity, and adequacy, constitute persistent gaps in the appropriateness and capacity of mental health services and providers in serving un(der)served groups. Recommended actions at the county and state level to reduce barriers to quality of care: 6.1 (QI domain): Expand culturally competent programs in ways that resonate with targeted un(der)served group to ensure adequate and appropriate services. To ensure appropriate practices, best practice models must be routinely identified, consistently implemented, and systematically evaluated. While MHSA funding has been allocated specifically for these recommended purposes, our report finds that efforts to improve culturally and linguistically appropriate services need to be more aggressive in order for counties and consumers to note progress. Ongoing evaluation of culturally and linguistically appropriate mental health services is needed to determine the extent to which these efforts are paying off. 6.2 (QI domain): Obtain community-informed perspectives to increase culturally competent programs and community outreach and engagement to mitigate barriers to care. Qualitative methods will allow local programs to assess whether MHSA is, in the view of participants, adequately meeting local needs. Results from such evaluation methods will also foster continued attempts to expand services to un(der)served groups while at the same time, enhancing the act’s efficacy via a cultural-and-community-informed perspective. 6.3 (QI domain): Ensure that outreach, educational, and anti-stigma campaigns be tailored to the needs of different underserved groups. While universal outreach, education, and anti- stigma campaigns can be effective, counties need to tailor their efforts to groups, including 8 Native Americans, LGBTQ, Lao Mien, Armenian, and Russian groups, among others. Outreach strategies should be consistently documented and systematically evaluated. 6.4 (CE and QI domains): Generate an inventory of successful community engaged practices (i.e., CBPR) to identify practice models that can be evaluated, tested, and replicated. After nearly a decade of funding novel programs and services, the MHSA has supported several model programs and community-informed practices. There remains a correlation between effective community engagement and the adequacy of mental health service delivery for un(der)served groups in California. . 6.5 (CE domain): Standardize methods of monitoring and evaluating the expenditure of MHSA funds, detailing funding acquisition, funding expenditures, and measurable outcomes. Our findings indicated that while counties do attempt to monitor the use of MHSA funds, practices vary widely across the state, and a standardized approach will help to curtail these inconsistencies. Key recommendation 7: assess gaps and persistent issues for un(der)served groups Increased awareness of resources and available opportunities needs to be a significant part of engaging un(der)served groups in currently available MHSA-funded services. Evaluation of MHSA promotional campaigns, Internet based tools, and mobile apps are recommended for future evaluation of MHSA efforts to reduce mental health disparities. Recommended actions at the county and state level to increase health literacy via MHSA- funded programs: 7.1 (CE and QI domains): Continue the evaluation of MHSA-funded program efforts to coordinate educational campaigns that aim to assist consumers who are navigating the public mental health system, and other community support systems to locate mental health care and obtain transportation services. 7.2 (CE domain): Conduct ongoing focus group activities to learn about participants’ technology preferences with regard to Internet searchers to learn how and where to access services. Further, focus groups with consumers and providers will be useful to assess perspectives focused on the prop 63 mobile app that helps consumers and family members search for county programs. If future public educational campaigns are developed to promote use of these technologies, mixed methods evaluation approaches should be considered to assess the reach and effectiveness of these campaigns. 7.3 (QI domain): Invest in and provide adequate training for community-based peer navigators to support persons with low health literacy and limited English proficiency to enroll in and navigate county mental health programs is recommended. A thorough and systematic evaluation of this training is also recommended to monitor successes, challenges, and room for improvement. 9 7.5 (CE and QI domains): Continue aims to improve cultural and linguistic competencies and ensure that they are guided by input from underserved consumers and their family members. County mental health programs can begin to survey their consumers, through the use of focus groups (not written surveys) to better evaluate service delivery and its contents and to identify room for improvement. Mandatory public opinion groups can help county mental health programs become more aware of consumer needs, while simultaneously involving consumers and families in assessment and treatment planning practices. Recommendations Based on the Public Review and Comment of Qualitative Findings With the assistance of the Mental Health Services Oversight and Accountability Commission (MHSOAC), we distributed our qualitative evaluation findings for public input to administrators, consumers, family members of consumers, advocates, providers, and state committees on February 15, 2014, and remained open to feedback for a period of 30 days (until March 15, 2014). We reviewed all of the feedback we received and incorporated recommendations consistent with the continued evaluation and quality improvement domains. The responses received expressed unanimous support for continued evaluation efforts to explore further the impact of the MHSA on underserved groups. Key recommendations in response to public comments: The following recommendations reflect the suggestions of the individuals who reviewed the qualitative portion of our findings and who identified concrete areas for continued evaluation.  (CE domain): Continue to invest in the hiring and training of individuals with lived experience, so that they may share their insights of appropriate and innovative means of engaging hard-to-reach groups in treatment. The inclusion of individuals with lived experience in the provision of services will offer them a voice that may influence mental health treatments and policies.  (QI domain): Document and evaluate involvement of stakeholders in programming and community assessment processes. Share findings with counties and statewide interest groups.  (CE and QI domains): Pursue efforts to increase collaboration with emergency departments responding to crises among individuals living with a severe mental illness who are not adequately connected to service providers.  (CE and QI domains): Support evaluation of model programs that have established links between public mental health clinics and other services and providers to offer a continuum of services to individuals with psychiatric emergencies.  (CE and QI domains): Invest in an evaluation of recovery outcomes among individuals living with severe mental illness, and those with co-morbid conditions (i.e., substance abuse) and the input from their family members or caregivers. 10  (CE and QI domains): Invest in an evaluation of the mental health needs and access status of individuals living with a persistent physical disability.  (CE and QI domains): All evaluation efforts should include methodology supporting the need to build trust and rapport with underserved groups. They must also include adequate sample sizes and regional representation.  (CE and QI domains): Monitor and highlight findings of evaluation efforts so that findings may serve as baselines/benchmarks for future evaluations, targeting the status and outcomes of underserved groups.  (CE and QI domains): Support and document culturally and linguistically appropriate services targeting stigma and other barriers to recovery for diverse groups and their subpopulations. Increase funding for the evaluation of community based and evidenced based interventions currently supported by MHSA funds.  (CE and QI domains): Engage in and measure the impact of culturally and linguistically appropriate health promotion strategies used to increase access to care and educate the public about services supported by the MHSA.  (CE and QI domains): Expand the scope of dissemination when sharing with the public status updates of MHSA funded services, treatment outcomes, as well as policy and funding changes so that the public may be more educated about the purpose, benefits and status of the MHSA.  (CE and QI domains): Continue to require that counties clearly describe and monitor the use of MHSA funds, so that taxpayers and consumers can determine if funds are properly invested in their populations of interests. 11 Key Implications Research Implications for Continued Evaluation to Improve Access and Outcomes Based on our findings from the review of the 2010 CCPRs, we have several recommendations for MHSOAC as well as for counties. First, we recommend that DHCS continue analyzing counties’ response to reducing mental health disparities using data from the CCPs. The use of the Department of Finance (DOF) data for county-level population estimates is recommended, as DOF appear to be the most reliable in estimating California county-level demographic variables and population changes. Use of DOF data is also typically required within state and county agencies. Second, we recommend that MHSOAC continue to strengthen communication with counties about completion and use of CCP data to assess improvements in mental health access, disparities, and delivery and utilization of culturally and linguistically competent services. Based on our disparity focused findings from review of CCP reports, we recommend that counties work to set clear objectives and realistic targets. Setting fewer than five targets is recommended in order to better prioritize the subpopulations that are experiencing the biggest gaps and the most substantial disparities in mental health services. To facilitate better targeting of mental health services to address disparities, three steps are recommend: (1) CCP requirements and submission processes should be streamlined by carefully considering and selecting data collection forms, systems, and processes, to ensure a higher CCP completion rate and adequate time to review and assess how counties are responding to disparities; (2) provision of technical assistance is recommended for county mental health agencies to provide enhanced support and guidance in the setting of realistic targets to address mental health service disparities, data identification, data management, data analysis, and data reporting; and (3) an online CCP submission system should be created to increase efficiency and promptness of reporting. This not only would increase submission consistency, but also would expedite data management and analysis, increasing data quality over time. Such improvements are of critical importance to the utility and scoring of the CCPs to analyze and assess how counties are doing in their efforts to reduce mental health disparities on an ongoing basis. Finally, the recommendations listed in this report aim to support the MHSA’s intent to increase access among underserved groups. To achieve the recommendations provided, it is essential for the state to continue evaluating MHSA-funded programs and improve its data collection and management methods. With fragmented data, evaluations cannot determine the full impact of the MHSA nor provide accurate assessments of how underserved groups benefit from the Act. With adequate data, accurate baseline statistics of treatment access among underserved groups can be determined in order to follow their progress throughout the course of the MHSA. While use of accurate data will provide indisputable facts of which groups access services, it is equally important to continue to invest in qualitative assessments of consumer and family member experiences with and views on mental health service options. Through personal accounts of access, barriers, and successes consumers offer a wealth of knowledge on how 12 they perceive current policy and whether or not it is has positively impacted their desired treatment outcomes. Practice Implications for Quality Improvement and Quality of Care Outcomes To engage the most vulnerable and underserved groups in treatment and evaluation work, culturally and linguistically appropriate outreach efforts must be conducted. According to our findings, few underserved and inappropriately served groups are aware of MHSA services that are intended to target their communities. This finding calls for the need to invest in local educational campaigns. Through development of educational campaigns of the MHSA and mental health resources, underserved groups can be empowered to seek help and to demystify enrollment processes. Given that consumers often seek services in times of crises, complicated enrollment processes become even more difficult to comprehend. By knowing where to seek services before crises emerge, crises can appear more easily manageable for a vulnerable individual. Participants repeatedly expressed lack of inclusion of underserved groups in community assessments, program planning, and MHSA-funded evaluations. It is essential to obtain the perspectives of the individuals who struggle most in accessing services in order to magnify the impact of the MHSA in treatment and recovery outcomes. The MHSA was developed to serve the most vulnerable. As a starting point, and as previously recommended by the MHSA state audit report, county mental health programs should plan and document all efforts to include stakeholders in program planning and the MHSA should further support these efforts. In several counties, we noted interest in training and hiring professionals who have “lived experiences” with regard to mental health challenges. This interest should be fully supported and encouraged in all counties. Traditionally, consumers tend to be more responsive to providers whom they perceive to be knowledgeable about his or her cultural background and lived experiences. This is considered an important indicator of effectiveness in counties appropriately matching mental health services with target populations. In addition to hiring staff with lived experience, in order to effectively combat mental health disparities, it is necessary for county programs to develop and sustain a culturally and linguistically competent mental health workforce consistent with the culture, language, gender identity and sexual orientation of the targeted population. One recommended strategy to diversify and sustain the mental health workforce and address the bilingual and bicultural shortages is for counties to explore the establishment of career pathways for immigrants who come to California from other countries with strong qualifications as mental health providers. To help counties improve their recruitment and retention of bilingual and bicultural staff it is recommended that DHCS provide technical assistance and guidance, and a forum in which county mental health programs are able to share successes, challenges, effective practices, and lessons learned in order to collectively improve mental health services across the state while consistently making efforts to reduce disparities. It may be worthwhile for DHCS to consider working with the Office of Health Equity (OHE) and the Office of Statewide Health 13 Planning & Development (OSHPD) to combine resources, and to strengthen the recruitment and retention of bilingual and bicultural staff. Hiring and retention incentives for multicultural and multilingual staff should also be considered, particularly in hard-to-reach areas, and in counties were mental health service disparities are particularly pronounced. Finally, the achievement of recommendations in this report rests in the improved transparency of the allocation and use of MHSA funds among county mental health programs and services. Some MHSA funds, particularly PEI funding, were reportedly used to support existing programs rather than new innovative services for the underserved. The development of specialized committees and policies are recommended to monitor all funding sources along with periodic progress reports accessible to consumers. Conclusion We stress the importance of further monies that will support future evaluations regarding disparities and underserved groups. A key finding from data sources used in this evaluation is that several underserved groups continue to be excluded or underrepresented in evaluation work. Further evaluation targeting larger samples of historically underserved groups is highly recommended. Given that evaluation research can help inform policymakers and stakeholders on many levels, underserved groups should be included in program evaluation design, methods, and processes, particularly when evaluation results are made public. Continued evaluation is also necessary because policies, such as the MHSA, can be gradually altered as new information about its effectiveness emerges. Continued evaluation reinforces the importance of on-going assessment to ensure the effectiveness of mental health programs (Silo & Lehto (2011) and quality care outcomes. It is important to note that some of the recommendations presented in this report echo those of previous evaluations, suggesting that the MHSA has the challenge of resolving continuing and persistent barriers to treatment access for several underserved groups. We hope that this evaluation and future evaluations not only assist in shaping MHSA programming but that future evaluations also explore reactions to evaluation outcomes. For example, it is important to investigate if the MHSA and county mental health programs welcome and make changes according to evaluation findings. Evaluations of mental health service access disparities are paid for by taxpayers, and highlight the persistent needs of the underserved, and therefore valid evaluation findings and recommendations should be considered in order to ensure quality improvements in the delivery of mental health services. 14 References California Department of Health Care Services (2014). Mental health services act (MHSA). Retrieved from http://www.dhcs.ca.gov/services/MH/Pages/MH_Prop63.aspx/ California State Auditor. (2013). Audit Report: Mental Health Services Act—The state's oversight has provided little assurance of the act's effectiveness, and some counties can improve measurement of their program performance. 212–122. Grant, D., Padilla-Frausto, I., Aydin, M., Streja, L., Aguilar-Gaxiola, S., & Caldwell, J. (2011). Adult mental health needs in California: Findings from the 2007 California Health Interview Survey. Retrieved from http://www.healthpolicy.ucla.edu/publications/Documents/PDF/MentalHealthreportnov2011. pdf Lee, D. (2012). "Mental health policy in California: the “millionaire's tax” and the Mental Health Services Oversight and Accountability Commission", Mental Health Review Journal, 17(4), 211 – 220. Solin, P. & Lehto, J. (2011). Mental health in complex health promotion policy programmes: The contribution of programme evaluations. Scandinavian Journal of Public Health, 39.3, 255- 261. 15