InitialStatewidePriorityIndicator ExecSummary

Initial Statewide ­ Priority Indicator Report ­ Draft Executive Summary � – for Stakeholder Input � UCLA Center for Healthier Children, Youth and Families ­ The following report was funded by the ­ Mental Health Services Oversight and Accountability Commission. ­ July 25, 2012 ­ Priority Indicators –Phase II, Deliverable 2E | 1 ­ Executive Summary The Mental Health Services Act Oversight and Accountability Commission (MHSOAC) charged the UCLA-EMT Evaluation Team with exploring impacts of the MHSA on California’s community mental health service system and its consumers. The current report – a draft that will be further shaped by stakeholder input – contributes to ongoing MHSA evaluation through improving monitoring of mental health consumer outcomes and community mental health system performance. The two central goals of this report are to: • Document the impact of MHSA programs and services on consumers, and assess the status of the mental health system, using existing data to compile priority performance indicators. Priority performance indicators presented in this report were initially proposed by the California Mental Health Planning Council1 and approved by the MHSOAC, and proposed by stakeholders. Priority performance indicators are key to the current evaluation; they were designed to assess impact of the MHSA in target areas that should be most changed through implementation of MHSA. • Improve and develop measurement of outcomes at the consumer and system levels through 1) exploration of possible ways to measure priority indicators given completeness and reliability of existing data; 2) provide information to guide MHSOAC development of a focused set of priority indicators appropriate for regular assessment and monitoring; 3) identify where additional information is necessary to support assessment and monitoring; and 4) establish a template for reporting at statewide and county levels. The evaluation team completed preliminary analysis of 25 priority performance indicators across eight core domains. Initial results, which may change as an outcome of the stakeholder review process, are summarized below: Consumer-level Indicators The consumer-level priority indicators presented in this report provide initial insights into several important outcomes, including: education, employment, housing, emergency care, and justice involvement. Other significant indicators supported by stakeholders (e.g., social connections) lack a database to assess reliability and practical utility. More specifically, • Analysis, supported by stakeholder feedback, revealed that existing data does not provide for comprehensive assessment consumer education outcomes (e.g., school attendance). However, rates of suspension and expulsion do provide initial evidence of educational engagement among mental health consumers who responded to satisfaction surveys. 1 California Mental Health Planning Council (January, 2010). Performance Indicators for Evaluating the Mental Health System. Priority Indicators –Phase II, Deliverable 2E | 2 ­ Additionally, existing data regarding child and TAY FSP consumers suggest that a majority “Always attends school” and “Attend school most of the time.” • Analysis of employment found that while a small proportion of FSP consumers held employment (approximately 8% across FY 2008-09 and 2009-10), most held paid employment. Overall, employment data was more complete than information relevant to other indicators, thus multiple views of employment may be elaborated in future analyses. • Indicators of housing remained effectively unchanged across FY 2008-09 and 2009-10. Rates of homeless child, TAY and older adult FSP consumers were low, relative to all mental health consumers. However, stakeholders and experts suggested the Key Event Tracking (KET) form, primarily used to collect housing information for this indicator, may not be utilized in a reliable manner. The sporadic nature of housing information was revealed in this analysis. • Analysis of justice involvement revealed less than one arrest per FSP consumer. However, more extensive information exists regarding post-arrest activities (e.g., detention, incarceration, and probation camp). Additional indicators focused on post-arrest activities may provide a more comprehensive assessment of justice involvement. • Indicators of emergency care found that on average mental health consumers visited hospitals for mental health episodes less than once annually. In contrast, consumers visited non-hospital facilities for emergency care between four and six times annually. These findings suggest that urgent care is addressed largely by these facilities and not hospitals. Findings are limited by the lack of comparable information specific to FSP consumers. • Stakeholders suggested social connection as an additional indicator of wellbeing (e.g., social support). This indicator is an important addition to consumer-level outcome monitoring, however existing data is not available to accurately assess social support from family members, non-family members, and organizations consumers deem “supportive” during a crisis and everyday life. System-Level Indicators The system level priority indicators presented in this report provide a multidimensional view of consumer access to the community mental health system, performance qualities of the system, and the structure and orientation of MHSA programs and the community mental health service system more broadly. More specifically, • Indicators of access to the community mental health system revealed services to minority consumers increased year-to-year, suggesting county mental health systems are increasing service to minority and other traditionally underserved or un-served populations. Additionally, FSP consumers’ access to a primary care physician increased year-to-year, and on average consumers and family members/caregivers indicated positive perceptions of access to the services they need. Taken together these trends suggest consumers’ access to the mental health system and the services they need are increasing. Priority Indicators –Phase II, Deliverable 2E | 3 ­ • Indicators of community mental health system performance demonstrate that ratios of FSP consumers served to planned service targets improved year-to-year, due to increased service rates and more accurate service targets established by counties as MHSA programs become more established. Additionally, consumers and family members/caregivers on average indicated positive perceptions of the consumer centered nature of their care, their wellbeing, and satisfaction with the services they received. Overall, indicators support a positive and progressing view of MHSA programs and the community mental health system. • Indicators of the structure of the community mental health system provide initial insights, revealing use of particular practices (e.g., evidence-based or promising practices, cultural appropriateness) in several counties and recovery focus (e.g., recovery, wellness, and resilience) among MHSA programs. These findings point to several important conclusions. Findings must be considered preliminary given 1) they are based on a subset of county data; 2) calculations are temporary and informed by stakeholder feedback; and 3) various interpretations of findings. With this, the following points are notable: • Many of the priority indicators evaluated suggested positively trending consumer outcomes, and progression among MHSA programs and the mental health system in line with MHSA values. • Most indicators will support more accurate assessment and monitoring to the extent the underlying data sources (e.g., Client Services Information system, Data Collection and Reporting system, Consumer Perception Surveys) become more complete and reliable. • Several indicators (e.g., Social Connection or Cultural Appropriateness of Services) will require additional data collection before they can be thoroughly evaluated. • A more streamlined set of priority indicators may be appropriate for continuous statewide monitoring of consumer outcomes and community mental health system performance. • This report represents an important intermediate step, necessary to arrive at a more focused, reliable, and instructive community mental health performance monitoring system.