Mhsa Priority Indicators: Sonoma County

Mental Health Services Act Evaluation: Report on Standardized Indicators Contract Deliverable 2F, Phase II Sonoma County UCLA Center for Healthier Children, Youth and Families EMT Associates, Inc. Submitted for review on November 30, 2012 The following report was funded by the Mental Health Services Oversight and Accountability Commission. The following report was revised in partnership with stakeholders who provided important historical context, data consultation, and revisions to ensure this report is accurate and accessible to the broadest audience possible. Feedback, collected prior to, during, and following report development, was crucial to developing this report. The UCLA-EMT Evaluation Team would like to express sincere appreciation to the research analysts, advocates, consumers and family members, agency representatives, service providers, and MHSOAC representatives who contributed invaluable insights to this document and previous versions. 2 Table of Contents Purpose ......................................................................................................................... Error! Bookmark not defined. Background ................................................................................................................. Error! Bookmark not defined. Review of Existing Data ......................................................................................... Error! Bookmark not defined. Data Sources .......................................................................................................... Error! Bookmark not defined. Priority Indicators Evaluated ..................................................................... Error! Bookmark not defined. Priority Indicators ............................................................................................... Error! Bookmark not defined. Report Organization ...................................................................................... Error! Bookmark not defined. Priority Indicator 1: Attendance ....................................................................................................................................... 5 1.1 Expulsions and Suspensions Per Year (CPS) ............................................................................................. 5 1.2 Average School Attendance Per Year (FSP) ............................................................................................... 7 Priority Indicator 2: Employment ................................................................................................................................... 9 Priority Indicator 3: Homelessness and Housing Rates ....................................................................................... 13 Priority Indicator 4: Arrest Rates .................................................................................................................................. 16 Priority Indicator 5: Demographic Profile of Consumers Served ..................................................................... 22 Priority Indicator 6: Demographic Profile of New Consumers ........................................................................... 26 Priority Indicator 7: Penetration of Mental Health Services .............................................................................. 31 Priority Indicator 8: Access to a Primary Care Physician .................................................................................... 34 Priority Indicator 9: Perceptions of Access to Services ........................................................................................ 37 Priority Indicator 10: Involuntary Status ................................................................................................................... 40 Priority Indicator 11: Consumer Perceptions of Improvement in Well-Being as a Result of Services ....................................................................................................................................................................................................... 42 Priority Indicator 12: Satisfaction with Services ..................................................................................................... 45 Appendix A – Priority Indicator Matrix ........................................................................................................................ 47 Appendix B – Preceding Reports ..................................................................................................................................... 48 Appendix C – Recoding Pre-DIG Race Data to Post-DIG Format ........................................................................ 49 3 Appendix B – Preceding Reports The evaluation team completed extensive groundwork before arriving at the conclusions contained in this report. To date, the team has documented evaluation planning in six reports based on statewide data made available online at UCLA and MHSOAC websites. Stakeholders were invited to provide feedback on draft reports by responding to questions in an accompanying guidance document. The invitation was shared online and through mass e-mail. Small stakeholder groups participated in one of two webinars, or online orientations to the report, that provided an overview to the report’s purpose and the types of feedback sought. Report title: Defining Priority Indicators Report version: Draft for stakeholder review Here, the evaluation team began to refine the core set of priority indicators proposed by the California Mental Health Planning Council to assess target outcomes of mental health consumers and the performance of the mental health system. Report title: Defining Priority Indicators Report version: Final, revised with stakeholder input In the revised report, the evaluation team illustrated how stakeholder feedback was integral to indicator development. This report incorporated changes driven by stakeholders’ comments about the comprehensiveness and appropriateness of the indicators. Report title: Compiling Data to Produce All Priority Indicators Report version: Draft for stakeholder review In this report, the evaluation team proposed how priority indicators could be calculated using existing statewide data. The report also detailed all possible data sources and specific variables or data fields that might be used to build comprehensive priority indicators. Report title: Compiling Data to Produce All Priority Indicators Report version: Final, revised with stakeholder input The initial report was revised to include information regarding measurement methods and the adequacy of existing data sources, gathered through a stakeholder feedback process similar to that used for the final Defining Priority Indicators report. Report title: Initial Statewide Priority Indicator Report Report version: Draft for stakeholder review The evaluation team used select data from fiscal years 2008-09 and 2009-101 to calculate priority indicators as outlined in the previous report (Compiling Data to Produce All Priority Indicators). Report title: Initial Statewide Priority Indicator Report Report version: Final, revised with stakeholder input The revised report incorporated several calculation updates guided by the MHSOAC. Updated calculations largely focused on outcome data collected after enrollment. The report also reflected input from consumer stakeholders about the use and accessibility of report features (illustrations, descriptions, etc.) and language. 1 The team sorted through datasets from 2005 through 2011 in search of one or more fiscal years (FY) in which data cells were largely filled where expected. Data from FY 2008-09 and 2009-10 fit this requirement. 4 Priority Indicators Analysis and Findings: Consumer Indicators Priority Indicator 1: Attendance 1.1 Expulsions and Suspensions Per Year (CPS) Indicator Summary This indicator provides descriptive information regarding the number of youth (children and TAY) expelled and suspended from school during the 2008-09 fiscal year. This indicator illustrates a subset of mental health service consumers who responded to consumer perceptions surveys. Indicator Calculation The number children and TAY who had been enrolled for at least six months and reported being expelled or suspended from school since beginning mental health services or 12 months prior to beginning such services. Note: Data is reported from Youth Satisfaction Surveys (YSS) Data Sources Consumer Perception Surveys (CPS) Data Fields: HowLong, LES12AREST, LES12PSTAREST Review of Existing Data Data sources likely to be sustained Data relevant to populations of interest Amount of missing data for child age group is approximately 5% Amount of missing data for TAY age group is approximately 0% Analytic Potential of Indicator Analysis across time possible with the exception of 09-10 data Analysis among specific service populations possible State- and county-level analysis possible with the exception of 09-10 data 5 Figure 1.1-1. Proportion of children who were suspended or expelled prior to beginning services and after receiving services for a period of 6 – 12 months during FY 2008-09 100% 90% 80% 68.4% 70% 63.2% 60% 50% No 36.8% 40% 31.6% Yes 30% 20% 10% 0% Exp/Sus 12 mon prior to servicesExp/Sus since beginning services (n = 19) (n =19) Unknown/Missing values for FY 2008-09: Exp/sus prior to services = 5% (n = 1) Exp/sus since beginning services = 5% (n = 1) Figure 1.1-2. Proportion of TAY who were suspended or expelled prior to beginning services and after receiving services for a period of 6 – 12 months during FY 2008-09 100.0% 100% 90% 85.7% 80% 70% 60% 50% No 40% Yes 30% 20% 14.3% 10% 0.0% 0% Exp/Sus 12 mon prior to servicesExp/Sus since beginning services (n = 7) (n =7) Unknown/Missing values for FY 2009-10: Exp/sus prior to services = 0% (n = 0) Exp/sus since beginning services = 0% (n = 0) 6 1.2 Average School Attendance Per Year (FSP) Indicator Summary This indicator provides descriptive information regarding the frequency for which Full Service Partnership consumers (children and TAY) attended school during the 2008-09 and 2009-10 fiscal years. Indicator Calculation The number children and TAY who attended school always, mostly, sometimes, infrequently, and never divided by the number of children for which there were data. The number of TAY who attended school always, mostly, sometimes, infrequently, and never divided by the number of TAY for which there were data. Note: Age groupings were revised such that: Child ages = 1-15 (same as previously) TAY ages = 16-18 (16-25 previously) The TAY age group was revised because education variables would be less clear for clients older than 18. Data Sources DCR (PAF - NONRES) Data Field: AttendanceCurr Review of Existing Data Data sources likely to be sustained Data relevant to populations of interest (FSPs) Amount of missing data for child age group is approximately 2% Amount of missing data for TAY age group is approximately 83% Analytic Potential of Indicator Analysis across time possible Analysis among specific service populations possible State- and county-level analysis possible 7 Figure 1.2-1 –The frequency with which children and TAY attended school during FY 2008- 09 (DCR) 100% 90% 80% 70% 60% 50.0% 50.0% 46.3% 50% Child (n = 82) 40% 31.7% TAY (n = 2) 30% 20% 14.6% 10% 4.9% 2.4% 0.0% 0.0% 0.0% 0% Always Attends Sometimes Infrequently Never attends school most attends attends attends school of the time school school school Unknown/Missing for FY 2008-09: Child = 2% (n = 2), TAY = 83.3% (n = 10) Figure 1.2-2 –The frequency with which children and TAY attended school during FY 2009- 10 (DCR) 100% 90% 80% 70% 60% 50% 39.5% 38.6% Child (n = 114) 40% 28.6% TAY (n = 7) 30% 28.6% 20% 14.3% 14.3% 14.3% 11.4% 10% 5.3% 5.3% 0% Always Attends Sometimes Infrequently Never attends school most attends attends attends school of the time school school school Unknown/Missing for FY 2009-10: Child = 1.7% (n = 2), TAY = 69.9% (n = 16) 8 Priority Indicator 2: Employment Indicator Summary This indicator provides the proportion of TAY, adults and older adults who are employed (paid and non-paid) and not employed as recorded during the most recent update (second date of service). This indicator provides descriptive information regarding clients’ employment status during their first date of service. Indicator Calculation Client & Service Information (CSI) The number of paid employed clients divided by the total number of TAY, adults, and older adults for whom there were employment data. The number of nonpaid employed clients divided by the total number of TAY, adults, and older adults for whom there were employment data. The number of paid nonemployed clients divided by the total number of TAY, adults, and older adults for whom there were employment data. Note: There were multiple periodic updates for clients within each fiscal year. These ratios provide information for those who had a second periodic update within a given fiscal year (or a second date of service). Additionally, the age groupings were revised to capture those truly eligible for employment. Those who indicated they were retired or incarcerated were excluded from calculations. Data Collection and Reporting (DCR) The number of TAY, adults, and older adults who reported paid employment divided by the total number of TAY, adults, and older adults. The number of TAY, adults, and older adults who reported nonpaid employment divided by the total number of TAY, adults, and older adults. The number of TAY, adults, and older adults who did not report any employment divided by the total number of TAY, adults, and older adults. Note for CSI and FSP data: Age groupings were revised such that TAY ages = 18-25 (previously 16-25) Older adults = 60-65 (60 and up previously) Data Sources CSI Periodic Post-dig, Data Field: Employment Status DCR (PAF - NONRES) Data Fields: Current_CompetitiveAvgHrWeek, Current_SupportedAvgHrWeek, Current_TransitionalAvgHrWeek, Current_In-HouseAvgHrWeek, Current_OtherEmploymentAvgHrWeek, Current_Non-paidAvgHrWeek Review of Existing Data Client & Service Information (CSI) Data sources likely to be sustained Data relevant to populations of interest (all consumers) Data available across multiple service years Amount of missing data for TAY age group is approximately 100% Amount of missing data for Adult age group is approximately 98% 9 Amount of missing data for Older Adult age group is approximately 100% Data Collection and Reporting (DCR) Data sources likely to be sustained Data relevant to populations of interest (FSPs) The amount of missing data for these ratios is unknown given how the employment data are collected. There is no data code option for “missing;” as a consequence, blank responses are either missing or not applicable. Analytic Potential of Indicator Client & Service Information (CSI) Analysis across time possible but very difficult Analysis among specific service populations not possible State- and county-level analysis possible Data Collection and Reporting (DCR) Analysis across time possible Analysis among specific service populations possible State- and county-level analysis possible 10 Figure 2-1 - Proportion of clients who were employed and not employed as reported during their second service date for FY 2008-09 (CSI) 100% 90% 80% 70% 60% Not employed 50% 100% Non-paid 40% Paid 30% 20% 10% 0% 0% 0% TAY (N/A) Adult (n = 8) Older adult (N/A) Unknown/Missing for FY 2008-09: TAY = 100.0% (n = 28), Adult = 98.0% (n = 317) Older adult = 100.0% (n = 16) Figure 2-2 – Proportion of clients who were employed and not employed as reported during their second service date for FY 2009–10 (CSI) 100% 90% 80% 70% 60% Not employed 50% 100% 100% Non-paid 40% Paid 30% 20% 10% 0% 0% 0% 0% 0% TAY (N/A) Adult (n = 6) Older adult (n = 1 ) Unknown/Missing for FY 2009-10: TAY = 100.0% (n = 30), Adult = 98.4% (n = 378) Older adult = 96.0% (n = 23) 11 Figure 2.3 –The proportion of FSPs who were employed during FY 2008–09 (DCR) Unknown/missing for FY 2008-09: Unknown Figure 2.4 –The proportion of FSPs who were employed during FY 2009–10 (DCR) Unknown/missing for FY 2009-10: Unknown 12 Priority Indicator 3: Homelessness and Housing Rates Indicator Summary This indicator summarizes the housing status of all mental health consumers and FSPs served during FYs 2008-09 and 2009-10. There are two parts: (a) a breakdown by most recently available housing status and (b) the percentage of consumers experiencing homelessness at any point during the year. Indicator Calculation Frequencies of the most recent housing statuses were computed for mental health and FSP consumers served in FYs 2008-09 and 2009-10. This calculation excludes consumers with no housing data within the given FYs or consumers whose most recent status was homeless. The percentages of mental health and FSP consumers who experienced homelessness at any point during the given FY were also computed. Note that a consumer who was most recently homeless would not be included in the first indicator for most recent housing status, whereas a consumer who was previously homeless and more recently reported as not homeless would be included. Data Sources Client & Service Information (CSI): H-01.0 County / City / Mental Health Plan Submitting Record; H-02.0 County Client Number; C-03.0 Date of Birth; P-01.0 Date Completed; P-09.0 Living Arrangement Data Collection and Reporting (DCR) Key Event Tracking (KET): 1.01 Global ID; 1.02 Assessment ID; 1.07 Age Group; 3.01 CountyID; 3.06 Assessment Date; 5.01 DateResidentialChange; 5.02 Current Review of Existing Data These data were taken from the Key Event Tracking (KET) updates for FSP consumers and the periodic updates for all mental health consumers, limited to the given fiscal year. Any consumer who did not have an update available during the year was not included. Data sources are likely to be sustained in the foreseeable future, providing a consistent source for tracking system performance moving forward. Taking a conservative approach, we considered cases without valid data “missing.” It should be noted that the data reporting and collection practices currently in place do not allow for a distinction between missing data from unreported changes in housing status and blank values from standard data entry practices. This is especially notable in the KET updates for FSP consumers, leading to large percentages of “missing” data. These results should be interpreted cautiously. In particular, there is the risk of systematic bias in underreporting certain housing statuses. Analytic Potential of Indicator Data across service years support analysis of the distribution and change of housing statuses, including homelessness, among consumers. Indicator Displays The first set of charts displays the most recently reported non-homeless housing statuses of consumers, by percentage, during each fiscal year. The second set displays the percentages of consumers who were reported as experiencing homelessness at any time during the fiscal year. 13 Figure 3.1 – Most recent housing status excluding homelessness, all consumers (CSI) 100% 8.1% 90% 17.0% 4.1% 18.1% 80% 6.1% 34.0% 2.1% 38.5% 36.2% 33.3% 37.5% 70% 60% 1.9% 0.0% 0.0% 0.0% 0.0% 50% 87.8% 40% 76.9% 79.9% 30% 64.1% 61.5% 63.8% 66.7% 62.5% 20% 10% 0% Child Child TAY TAY Adult Adult Older Older 08-09 09-10 08-09 09-10 08-09 09-10 Adult Adult 08-09 09-10 House or Apartment Foster Care Group Setting Unknown/Missing for FY 2008-09 = 2.6% (n = 4) for children; 0% (n = 0) for TAY; 3.0% (n = 10) for adults; and 7.7% (n = 2) for older adults Unknown/Missing for FY 2009-10 = 10.2% (n = 28) for children; 2.7% (n = 4) for TAY; 2.6 (n = 10) for adults; and 0% (n = 0) for older adults Figure 3.2 – Most recent housing status excluding homelessness, FSP consumers only (DCR) 100% 0.0% 90% 33.3% 80% 42.9% 70% 50.0% 57.3% 53.2% 71.4% 70.6% 60% 7.1% 22.2% 50% 0.0% 0.0% 100.0% 0.0% 14.3% 40% 30% 0.0% 50.0% 36.0% 40.5% 0.0% 20% 44.4% 35.7% 28.6% 29.4% 10% 6.7% 6.3% 0% 0.0% 0.0% Child Child TAY TAY Adult Adult Older Older 08-09 09-10 08-09 09-10 08-09 09-10 Adult Adult 08-09 09-10 With Family Independent Foster Care Group Setting Unknown/Missing for FY 2008-09 = 30% (n = 6) for children; 20% (n = 3) for TAY; 7.8% (n = 8) for adults; and 0% (n = 0) for older adults Unknown/Missing for FY 2009-10 = 17.6% (n =3) for children; 26.1% (n = 6) for TAY; 18.3% (n = 19) for adults; and 10.0% (n = 2) for older adults 14 Figure 3.3 – Experienced homelessness at any point during the year, all consumers (CSI) 60% 50% 40% 30% 08-09 09-10 20% 10% 2.2% 1.8% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0% Child TAY Adult Older Adult Unknown/Missing for FY 2008-09 = 2.6% (n = 4) for children; 0% (n = 0) for TAY; 3.0% (n = 10) for adults; and 7.7% (n = 2) for older adults Unknown/Missing for FY 2009-10 = 10.2% (n = 28) for children; 2.7% (n = 4) for TAY; 2.6 (n = 10) for adults; and 0% (n = 0) for older adults Figure 3.4 – Experienced homelessness at any point during the year, FSP consumers only (DCR) 60% 50.0% 50% 41.2% 40% 33.3% 30% 27.1% FY 2008-2009 22.3% FY 2009-2010 20% 10% 7.1% 5.6% 0.0% 0% Child TAY Adult Older Adult Unknown/Missing for FY 2008-09 = 30% (n =6) for children; 20% (n = 3) for TAY; 7.8% (n = 8) for adults; and 0% (n = 0) for older adults Unknown/Missing for FY 2009-10 = 17.6% (n =3) for children; 26.1% (n = 6) for TAY; 18.3% (n = 19) for adults; and 10% (n = 2) for older adults 15 Priority Indicator 4: Arrest Rates Indicator Summary This indicator provides the proportion of youth, adults, and older adults who reported being arrested 12 months prior to receiving services and the proportion of youth, adults, and older adults who reported being arrested since beginning services. For calculations involving consumer perception surveys, this indicator includes only youth, adults, and older adults who reported receiving services for 6 to 12 months. This indicator provides information regarding whether the proportion of arrested clients has increased or decreased after 6 to 12 months of service. For calculations involving Full Service Partnership consumers, this indicator tracks arrests prior to enrollment using intake data. This indicator accounts for consumers enrolled during the target fiscal years for which PAF surveys are available. Indicator Calculation Consumer Perception Surveys (CPS) The number of reported arrest 12 months prior to services divided by the total number of youth, adults, and older adults for who there was data The number of reported arrest since beginning services divided by the total number of youth, adults, and older adults for who there was data Note: Clients were surveyed multiple times during the 2008–09 fiscal year. However, only one survey administration was used to get both the proportion of clients who reported being arrested 12 months prior to beginning services and since receiving services. Age groupings are as follows: Youth, 1 – 25 years Adult, 26 – 59 years Older adult, 60 and above Data Collection and Reporting (DCR) The number of youth (children and TAY), adults, and older adults reporting arrests 12 months prior to enrollment divided by the total number of unique clients for who there was data The number of youth (children and TAY), adults, and older adults reporting arrests 12 months prior to the past 12 months divided by the total number of unique clients for who there was data Note: In rare cases where two surveys were entered for one client, only the earliest entry was used in calculations. Data Sources Consumer Perception Survey (CPS) for Youth, Adults, and Older Adults Data Fields: HowLong, les12arest, les12pstarest Data Collection and Reporting (DCR PAF NONRES): Age_Group, ArrestPast12, ArrestPrior12 Review of Existing Data Consumer Perception Surveys (CPS) Data sources likely to be sustained Data relevant to populations of interest Approximately 5.1% missing or unknown values for youth Approximately 27.1% missing or unknown values for adult Approximately 25.0% missing or unknown values for older adult 16 Data Collection and Reporting (DCR) Data sources likely to be sustained Data relevant to populations of interest On average, 0.0% missing or unknown values for children On average, 0.0% missing or unknown values for TAY On average, 0.0% missing or unknown values for adults On average, 0.0% missing or unknown values for older adults Analytic Potential of Existing Data For both data sources Analysis across time possible Analysis among specific service populations not possible State and county individual level analysis possible Note: As of the submission of this report, a new calculation has been proposed to examine arrest rates. The proposed calculation would use FSP-DCR data during consumers’ enrollment (not intake as it is presented here). An updated indicator will be available shortly. 17 Figure 4.1 – Proportion of youth who were arrested prior to beginning services and since receiving services Missing/unknown for Arrested 12 months prior to services = 6.9% (n = 2) Missing/unknown for Arrested since receiving services = 3.4% (n =1) Figure 4.2 – Proportion of adults who were arrested prior to beginning services and since receiving services Missing/unknown for Arrested 12 months prior to services = 25.7% (n = 9) Missing/unknown for Arrested since receiving services = 28.6% (n =10) 18 Figure 4.3 – Proportion of older adults who were arrested prior to beginning services and since receiving services Missing/unknown for Arrested 12 months prior to services = 25% (n = 1) Missing/unknown for Arrested since receiving services = 25% (n =1) Figure 4.4 - Proportion of children who were arrested within the past 12 months (DCR) Missing/Unknown for FY 2008-09 = 0.0% (n=0) Missing/Unknown for FY 2009-10 = 0.0% (n=0) 19 Figure 4.4 - Proportion of TAY who were arrested within the past 12 months (DCR) Missing/Unknown for FY 2008-09 = 0.0% (n=0) Missing/Unknown for FY 2009-10 = 0.0% (n=0) Figure 4.5 – Proportion of adults who were arrested within the past 12 months (DCR) Missing/Unknown for FY 2008-09 = 0.0% (n=0) Missing/Unknown for FY 2009-10 = 0.0% (n=0) 20 Figure 4.6 Proportion of older adults who were arrested within the past 12 months (DCR) Missing/Unknown for FY 2008-09 = 0.0% (n=0) Missing/Unknown for FY 2009-10 = 0.0% (n=0) 21 Priority Indicators Analysis and Findings: Community Mental Health System Indicators Priority Indicator 5: Demographic Profile of Consumers Served Indicator Summary This indicator profiles the demographics (race/ethnicity, age, and gender) of all mental health consumers and Full Service Partnership consumers served during FYs 2008-09 and 2009-10, in Sonoma County. Indicator Calculation The frequencies of all mental health consumers and FSP consumers served in FYs 2008-09 and 2009-10 were calculated overall. Additionally, the proportion of consumers represented by race/ethnicity, age, and gender categories was calculated by dividing the number of consumers within each demographic category by all consumers served. Proportions were calculated for both service populations (all consumers and FSPs) and both fiscal years examined (see Figures 5.1-5.6 below). Data Sources Client & Service Information (CSI) Data Fields: H-01.0 County / City / Mental Health Plan Submitting Record; H- 02.0 County Client Number; C-05.0 Gender; C-09.0 Ethnicity; C-10.0 Race; S-05.0 Mode of Service; S-16.0 From / Entry Date; S-17.0 Through / Exit Date; S-23.0 Date of Service. Data Collection and Reporting (DCR) Data Fields: 1.01 Global ID; 1.02 Assessment ID; 1.04 Date Partnership Status Change; 1.05 Partnership Status; 1.07 Age Group; 1.08 Assessment Type; 2.01 CSI Date of Birth; 2.02 Gender; 2.03 CSIRace1; 2.04 CSIRace2; 2.05 CSIRace3; 2.06 CSIRace4; 2.07 CSIRace5; 2.10 CSI Hispanic; 3.01 County ID; 3.05 Partnership Date; 3.06 Assessment Date. Review of Existing Data Data sources likely to be sustained Data relevant to populations of interest (all consumers and FSPs) Data available across multiple service years Less than 10% missing or unknown values Analytic Potential of Indicator Analysis across time possible Analysis among specific service populations possible (e.g., all consumers, FSPs, demographic groups) 22 Figure 5.1. Race/ethnicity of mental health consumers 100.0% 90.0% 80.0% 67.5% 67.9% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.4% 10.2% 14.6% 14.9% 10.0% 1.7% 1.8% 0.2% 0.2% 3.7% 3.4% 1.0% 0.8% 0.9% 0.8% 0.0% White Hispanic/Latino Asian Pacific Islander Black American Multirace Other Indian FY 2008-09 (4,159) FY 2009-10 (4,398) FY 2008-09 Unknown/Missing = 2.0% (n = 85); FY 2009-10 Unknown/Missing = 1.6% (n = 70) Figure 5.2. Race/ethnicity of FSP consumers 100.0% 90.0% 80.0% 66.3% 70.0% 63.2% 60.0% 50.0% 40.0% 30.0% 25.7% 22.5% 20.0% 10.0% 6.0% 5.6% 1.9% 1.3% 0.0% 0.2% 2.2% 2.7% 0.6% 0.9% 0.3% 0.4% 0.0% White Hispanic/Latino Asian Pacific Islander Black American Multirace Other Indian FY 2008-09 (315) FY 2009-10 (445) FY 2008-09 Unknown/Missing = 3.7% (n = 12); FY 2009-10 Unknown/Missing = 3.5% (n = 16) 23 Figure 5.3. Mental health consumers by age group 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 47.6% 45.8% 40.0% 25.6% 30.0% 24.1% 20.1% 20.3% 20.0% 8.1% 8.3% 10.0% 0.0% Children TAY Adults Older Adults FY 2008-09 (4,244) FY 2009-10 (4,468) Figure 5.4. FSP consumers by age group 100.0% 90.0% 80.0% 70.0% 60.0% 54.7% 50.0% 44.3% 40.0% 30.0% 25.7% 25.2% 17.7% 18.4% 20.0% 12.1% 1.8% 10.0% 0.0% Children TAY Adult Older Adult FY 2008-09 (133) FY 2009-10 (196) 24 Figure 5.5. Mental health consumers by gender 100.0% 90.0% 80.0% 70.0% 54.6% 60.0% 53.1% 46.9% 45.4% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Female Male FY 2008-09 (4,227) FY 2009-10 (4,456) FY 2008-09 Unknown/Missing = 0.4% (n = 17); FY 2009-10 Unknown/Missing = 0.3% (n = 12) Figure 5.6. FSP consumers by gender 100.0% 90.0% 80.0% 62.4% 70.0% 59.9% 60.0% 50.0% 40.1% 37.6% 40.0% 30.0% 20.0% 10.0% 0.0% Female Male FY 2008-09 (456) FY 2009-10 (322) FY 2008-09 Unknown/Missing = 1.5% (n = 5); FY 2009-10 Unknown/Missing = 1.1% (n = 5) 25 Priority Indicator 6: Demographic Profile of New Consumers Indicator Summary This indicator profiles new mental health consumers (i.e., served during FY, without service for prior six months) overall and full service partners (FSPs) served during FYs 2008-09 and 2009-10. Indicator Calculation For all mental health consumers, CSI data support calculation of new (i.e., did not receive services for 6 months prior to given FY) versus past consumers (i.e., initial services received prior to the given FY) overall and within race/ethnicity, age, and gender categories. The frequency of new consumers served was divided by all previous consumers served, in each fiscal year, to calculate the proportion of new consumers served. This same calculation was conducted within each demographic category (race/ethnicity, age, and gender), in each FY (see Figures 6.1 – 6.4 below). For FSPs, DCR data support calculation of new (i.e., did not receive services for 6 months prior to given FY) versus existing (i.e., current Full Service Partners) overall and within race/ethnicity, age, and gender categories. The frequency of new consumers served was divided by all existing consumers, in each fiscal year, to calculate the proportion of new consumers served. This same calculation was conducted within each demographic category (race/ethnicity, age, and gender), in each FY (see Figures 6.5 – 6.8, below). Data Sources Client & Service Information (CSI) Data Fields: H-01.0 County / City / Mental Health Plan Submitting Record; H- 02.0 County Client Number; C-05.0 Gender; C-09.0 Ethnicity; C-10.0 Race; S-05.0 Mode of Service; S-16.0 From / Entry Date; S-17.0 Through / Exit Date; S-23.0 Date of Service. Data Collection and Reporting (DCR) Data Fields: 1.01 Global ID; 1.02 Assessment ID; 1.04 Date Partnership Status Change; 1.05 Partnership Status; 1.07 Age Group; 1.08 Assessment Type; 2.01 CSI Date of Birth; 2.02 Gender; 2.03 CSIRace1; 2.04 CSIRace2; 2.05 CSIRace3; 2.06 CSIRace4; 2.07 CSIRace5; 2.10 CSI Hispanic; 3.01 County ID; 3.05 Partnership Date; 3.06 Assessment Date. Review of Existing Data Data sources likely to be sustained Data relevant to populations of interest (all consumers and FSPs) Data available across multiple service years Less than 10% missing or unknown values among some demographic fields Analytic Potential of Indicator Analysis across time possible Analysis among specific service populations possible (e.g., all consumers, FSPs, demographic groups) 26 All Consumers – Data Source: Client & Service Information (CSI) Figure 6.1. New and continuing mental health consumers 100.0% 90.0% 80.0% 73.8% 68.0% 70.0% 60.0% 50.0% 40.0% 32.0% 30.0% 26.2% 20.0% 10.0% 0.0% FY 2008-09 (8,144) FY 2009-10 (9,944) Previous Consumers New Consumers Figure 6.2. Race/ethnicity of new mental health consumers 100.0% 90.0% 80.0% 64.4% 70.0% 62.7% 60.0% 50.0% 40.0% 30.0% 20.0% 16.0% 13.2% 13.0% 15.5% 10.0% 1.8% 1.8% 0.2% 0.2% 3.8% 3.3% 1.2% 0.8% 1.2% 1.0% 0.0% White Hispanic/Latino Asian Pacific Islander Black American Multirace Other Indian FY 2008-09 (2,532) FY 2009-10 (2,546) FY 2008-09 Unknown/Missing = 2.7% (n = 71); FY 2009-10 Unknown/Missing = 2.2% (n = 57) 27 Figure 6.3. New mental health consumers by age group 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 42.2% 41.3% 40.0% 28.0% 29.6% 25.0% 24.5% 30.0% 20.0% 10.0% 4.8% 4.6% 0.0% Children TAY Adults Older Adults FY 2008-09 (2,603) FY 2009-10 (2,603) FY 2008-09 Unknown/Missing = 0.0% (n = 0); FY 2009-10 Unknown/Missing = 0.0% (n = 0) Figure 6.4. New mental health consumers by gender 100.0% 90.0% 80.0% 70.0% 60.0% 49.3% 47.9% 50.7% 52.1% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Female Male FY 2008-09 (2,586) FY 2009-10 (2,589) FY 2008-09 Unknown/Missing = 0.7% (n = 17); FY 2009-10 Unknown/Missing = 0.5% (n = 14) 28 FSP Consumers – Data Source: Data Collection and Reporting (DCR) Figure 6.5. New and continuing FSP consumers 100.0% 90.0% 78.9% 80.0% 70.9% 70.0% 60.0% 50.0% 40.0% 29.1% 30.0% 21.1% 20.0% 10.0% 0.0% FY 2008-09 (327) FY 2009-10 (461) Previous Consumers New Consumers Figure 6.6. Race/ethnicity of new FSP consumers 100.0% 90.0% 80.0% 66.3% 70.0% 63.2% 60.0% 50.0% 40.0% 30.0% 25.7% 22.5% 20.0% 10.0% 6.0% 5.6% 1.9% 1.3% 0.0% 0.2% 2.2% 2.7% 0.6% 0.9% 0.3% 0.4% 0.0% White Hispanic/Latino Asian Pacific Islander Black American Multirace Other Indian FY 2008-09 (315) FY 2009-10 (445) FY 2008-09 Unknown/Missing = 3.7% (n = 12); FY 2009-10 Unknown/Missing = 3.5% (n = 16) 29 Figure 6.7. New FSP consumers by age group 100.0% 90.0% 80.0% 70.0% 54.7% 60.0% 50.0% 44.3% 40.0% 30.0% 25.7% 25.2% 17.7% 18.4% 20.0% 12.1% 10.0% 1.8% 0.0% Children TAY Adults Older Adults FY 2008-09 (327) FY 2009-10 (491) Figure 6.8. New FSP consumers by gender 100.0% 90.0% 80.0% 70.0% 62.4% 59.9% 60.0% 50.0% 40.1% 37.6% 40.0% 30.0% 20.0% 10.0% 0.0% Female Male FY 2008-09 (322) FY 2009-10 (456) 30 Priority Indicator 7: Penetration of Mental Health Services Indicator Summary This indicator details rates of service access relative to estimates of need for service among residents of Sonoma County earning less than 200% of the federal poverty income level. This metric is intended to show the extent to which service access is in line with the level of need for services. Indicator Calculation The number of all mental health consumers served (i.e., at least one service received during FY) was divided by estimates of need for service (Holzer Targets) among residents of Sonoma County earning less than 200% of the federal poverty income level and among demographic category (i.e., race/ethnicity, age, and gender). (See Figures 7.1-7.4 below). Data Sources Client & Service Information (CSI) Data Fields: H-01.0 County/City/Mental Health Plan Submitting Record; H- 02.0 County Client Number; C-05.0 Gender; C-09.0 Ethnicity; C-10.0 Race; S-05.0 Mode of Service; S-16.0 From / Entry Date; S-17.0 Through / Exit Date; S-23.0 Date of Service. Estimates of need for mental health services (Holzer Targets) among Californians earning less than 200% of the federal poverty income level. Review of Existing Data Data sources likely to be sustained Data appropriate for analysis of all mental health consumers. The estimates of need for service (Holzer Targets) used are not appropriate points of comparison for FSP service levels. Data available across multiple service years Less than 10% missing or unknown values (see Appendix C for details of recoding race/ ethnicity data fields) Analytic Potential of Indicator Analysis across time possible Analysis among specific service populations possible (e.g., all consumers, FSPs, demographic groups) 31 Figure 7.1. Penetration of mental health services 100.0% 80.0% 60.0% 45.6% 47.6% 40.0% 20.0% 0.0% FY 2008-09 FY 2009-10 Consumers Served/Holzer Target FY 2008-09 (4,227/9,265) FY 2009-10 (4,456/9,365) Figure 7.2. Penetration of services by race/ethnicity 300.0% 283.5% 263.9% 250.0% 200.0% 150.0% 92.5% 100.0% 81.8% 87.5% 62.1% 66.7% 73.5% 6 9.0% 58.0% 50.0% 31.3% 29.2% 31.0% 27.0% 12.2% 12.3% 0.0% White Hispanic/Latino Asian Pacific Islander Black Amerian Indian Multirace Other FY 2008-09 FY 2009-10 White (2,808/4,845) (2,986/4,812) Hispanic/ Latino (433/3,557) (450/3,668) Black (155/211) (149/216) Asian (69/236) (77/248) Pacific Islander (8/12) (9/11) American Indian (42/134) (37/137) Multirace (607/230) (655/231) Other (37/40) (35/40) 32 Figure 7.3. Penetration of mental health services by age 70.0% 60.3% 63.6% 60.0% 50.0% 43.3% 45.0% 45.5% 45.2% 46.6% 39.6% 40.0% 30.0% 20.0% 10.0% 0.0% Children TAY Adults Older Adults FY 2008-09 FY 2009-10 FY 2008-09 FY 2009-10 Children (1,024/2,589) (1,144/2,644) TAY (854/1,416) (906/1,424) Adult (2,021/4,496) (2,048/4,503) Older Adult (345/764) (370/794) Figure 7.4. Penetration of mental health services by gender 70.0% 64.2% 59.9% 60.0% 50.0% 40.0% 36.3% 35.9% 30.0% 20.0% 10.0% 0.0% FY 2008-09 FY 2009-10 FY 2008-09 FY 2009-10 Female Male FY 2008-09 FY 2009-10 Female (1,984/5,523) (2,025/5,577) Male (2,243/3,742) (2,431/3,788) 33 Priority Indicator 8: Access to a Primary Care Physician Indicator Summary This indicator details the level of access to a primary care physician reported among FSP consumers, during FYs 2008-09 and 2009-10, in Sonoma County. Indicator Calculation The ratio of FSP consumers indicating access to a primary care physician at any point during a fiscal year to all FSP consumers served during a fiscal year was calculated (see Figure 8.1). This ratio was also calculated within demographic categories (i.e., race/ethnicity, age, and gender) for each FY (see Figures 8.2-8.4 below). Data Sources Data Collection and Reporting (DCR) Data Fields: 1.01 Global ID; 1.02 Assessment ID; 1.04 Date Partnership Status Change; 1.05 Partnership Status; 1.07 Age Group; 1.08 Assessment Type; 2.01 CSI Date of Birth; 2.02 Gender; 2.03 CSIRace1; 2.04 CSIRace2; 2.05 CSIRace3; 2.06 CSIRace4; 2.07 CSIRace5; 2.10 CSI Hispanic; 3.01 County ID; 3.05 Partnership Date; 3.06 Assessment Date; 11.01 PhysicianCurr. Review of Existing Data Data source likely to be sustained Data relevant to population of interest (FSPs). Relevant data not available to assess primary care access among all mental health consumers (e.g., CSI). Data available across multiple service years Less than 10% missing or unknown values within “PhysicianCurr” and demographic fields (see Appendix C for details of recoding race/ethnicity data fields) Analytic Potential of Indicator Analysis across time possible Analysis among specific service populations possible (e.g., all consumers, FSPs, demographic groups) 34 Figure 8.1. FSP access to a primary care physician 100.0% 86.0% 75.0% 80.0% 60.0% 40.0% 20.0% 0.0% FY 2008-09 (174) FY 2009-10 (276) Figure 8.2. FSP access to a primary care physician by race/ethnicity 100.0% 90.0% 80.0% 70.0% 67.7% 63.7% 60.0% 50.0% 40.0% 30.0% 26.4% 20.5% 20.0% 10.0% 1.5% 0.4% 0.8% 0.0% 4.8% 4.6% 1.8% 1.8% 1.5% 1.8% 0.4% 0.6% 0.0% White Hispanic/Latino Asian Pacific Islander Black Amerian Indian Multirace Other FY 2008-09 (168) FY 2009-10 (263) FY 2008-09 Unknown/Missing = 3.4% (n = 6); FY 2009-10 Unknown/Missing = 4.7% (n = 13) 35 Figure 8.3. FSP access to a primary care physician by age group 100.0% 90.0% 80.0% 70.0% 60.0% 53.4% 47.8% 50.0% 40.0% 30.0% 24.1% 20.0% 19.2% 20.1% 18.8% 14.1% 10.0% 2.3% 0.0% Children TAY Adults Older Adults FY 2008-09 (116) FY 2009-10 (276) Figure 8.4. FSP access to a primary care physician by gender 100.0% 90.0% 80.0% 70.0% 59.9% 57.2% 60.0% 50.0% 40.1% 42.8% 40.0% 30.0% 20.0% 10.0% 0.0% Female Male FY 2008-09 (172) FY 2009-10 (271) FY 2008-09 Unknown/Missing = 1.1% (n = 2); FY 2009-10 Unknown/Missing = 1.8% (n = 5) 36 Priority Indicator 9: Perceptions of Access to Services Indicator Summary This indicator provides insight into consumer and family perceptions of access to mental health services, among a sample of those currently accessing the Sonoma County mental health system. Indicator Calculation Family members/caregivers and TAY respondents’ ratings (1–Strongly Disagree to 5–Strongly Agree) of two self-report items (specified in the Data Sources section below) were averaged to calculate aggregate ratings of perceptions of access to mental health services (see Figures 9.1-9.2 and Tables 9.1-9.2 below). Aggregate ratings were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive perceptions. This calculation method is in line with previous DHCS practices. Adult and Older Adult respondents’ ratings (1-Strongly Disagree to 5-Strongly Agree) of 14 self-report items (specified under the Data Sources section below) were averaged to calculate aggregate ratings of perceptions of access to mental health services (see Figures 9.1-9.2 and Tables 9.1-9.2 below). Aggregate ratings were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive perceptions. This calculation method is in line with previous DHCS practices. Data Sources Consumer Perception Surveys Family members/caregivers and TAY self-report items analyzed (YSS/YSS-F): o The location of services was convenient for us. o Services were available at times that were convenient for us. Adult and older adult self-report items analyzed (MHSIP): o The location of services was convenient (parking, public transportation, distance, etc.). o Staff were willing to see me as often as I felt it was necessary. o Staff returned my call in 24 hours. o Services were available at times that were good for me. o I was able to get all the services I thought I needed. o I was able to see a psychiatrist when I wanted to. Note: Data collected in FYs 2008-09 and 2009-10 must be interpreted separately because a convenience sampling method was employed to gather FY 2008-09 data and a random sampling method employed to gather data in FY 2009-10. 2 Review of Existing Data Data source likely to be sustained Data relevant to population of interest (i.e., convenience or random sample of all mental health consumers) Data available across multiple service years More than 10% missing or unknown values among respondent groups Analytic Potential of Indicator Analysis across time will be possible if the sampling methodology and instrument used is employed in a consistent manner each year Analysis among specific service populations possible (e.g., all consumers, demographic groups) 2 Cowles, E. L., Harris, K., Larsen, C., and Prince, A. (2010). Assessing Representativeness of the Mental Health Services Consumer Perception Survey. 37 County analysis possible for FY 2008-09 (convenience sample), but data is not available at the county level for FY 2009-10 38 Figure 9.1 – Perceptions of access to services, FY 2008-09 Older Adults 4.34 (n=28) Adults 4.25 (n=459) TAY 4.05 (n=175) Children 4.20 (n=152) 1.00 2.00 3.00 4.00 5.00 Table 9.1. Perceptions of access to services by race/ethnicity, FY 2008-09 Family Member/ TAY Adult Older Adult Caregiver White 4.23 (n=111) 4.09 (n=117) 4.25 (n=352) 4.38 (n=26) Hispanic / Latino 4.16 (n=54) 4.07 (n=49) 4.47 (n=64) N/A Asian 3.63 (n=4) 3.86 (n=11) 3.83 (n=10) N/A Pacific Islander 4.00 (n=3) 4.17 (n=3) 4.61 (n=6) N/A Black 4.10 (n=10) 4.12 (n=13) 4.38 (n=18) N/A American Indian 4.47 (n=17) 4.02 (n=24) 4.28 (n=27) N/A Other 4.17 (n=134) 3.94 (n=44) 4.13 (n=42) 5.00 (n=1) Unknown/Missing Values: Family Member/Caregiver = 2.6% (n = 9), TAY = 6.1% (n = 17), Adult = 12.5% (n = 74), Older Adult = 3.6% (n = 1) Table 9.2. New mental health consumers by gender, FY 2008-09 Family Member/ TAY Adult Older Adult Caregiver Female 4.24 (n=48) 4.12 (n=98) 4.32 (n=201) 4.50 (n=14) Male 4.18 (n=103) 3.97 (n=75) 4.21 (n=234) 4.31 (n=13) Unknown/Missing Values: Family Member/Caregiver = 0.7% (n = 1), TAY = 1.1% (n = 2), Adult = 5.8% (n = 27), Older Adult = 3.6% (n = 1) 39 Priority Indicator 10: Involuntary Status Indicator Summary This indicator provides insight into the rates of involuntary status among all mental health consumers during FY 2008-09. Involuntary status refers to a legal designation that can be applied to individuals who are found to be a danger to themselves and/or others, and/or gravely disabled. Indicator Calculation The California Department of Health Care Services (DHCS) reports incidents of involuntary status per 10,000 consumers. Such rates are reported here (see Figure 10.1, below). Data Sources The California Department of Health Care Services provides reports of incidents of involuntary status (see http://www.dmh.ca.gov/statistics_and_data_analysis/Involuntary_Detention.asp) Review of Existing Data Data source likely to be sustained Data relevant to population of interest (all mental health consumers). Relevant data are not available to specifically assess involuntary status among FSP consumers. Data available across multiple service years Analytic Potential of Indicator Analysis across time will be possible as information from additional fiscal years becomes available from DHCS Aggregate data do not allow for analysis among specific (e.g., demographic) service populations 40 Figure 10.1. Involuntary status per 10,000 consumers, FY 2008-09 (NOTE: horizontal scale reduced for ease of viewing) Additional 14-day Intesive (suicidal) 0.0 14-day Intensive Treatment 0.0 72-Hour Evaluation and Treatment - Child 0.0 72 Hour Evaluation and Treament - Adult 0.0 0 10 20 30 40 50 60 70 80 90 100 41 Priority Indicator 11: Consumer Perceptions of Improvement in Well- Being as a Result of Services Indicator Summary This indicator provides insight into consumer and family perceptions of well-being (i.e., outcomes, functioning, and social connectedness) as a result of mental health services. Indicator Calculation Family members/caregivers and TAY respondents’ ratings (1–Strongly Disagree to 5–Strongly Agree) of 11 self- report items (specified in the Data Sources section below) were averaged to calculate aggregate ratings of perceptions of well-being as a result of mental health services (see Figures 11.1-11.2 and Tables 11.1-11.2 below). Aggregate ratings were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive perceptions. This calculation was developed to approximate domains of well-being many respondents noted in their feedback to our initial reports. Adult and older adult respondents’ ratings (1–Strongly Disagree to 5–Strongly Agree) of 14 self-report items (specified in the Data Sources section below) were averaged to calculate aggregate ratings of perceptions of well-being as a result of mental health services (see Figures 11.1-11.2 and Tables 11.1-11.2 below). Aggregate ratings were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive perceptions. This calculation was developed to approximate domains of well-being many respondents noted in their feedback to our initial reports. Data Sources Consumer Perception Surveys Family members/caregivers and TAY self-report items analyzed (YSS/YSS-F): o My child is better at handling daily life. o My child gets along better with family members. o My child gets along better with friends and other people. o My child is doing better in school and/or work. o My child is better able to cope when things go wrong. o I am satisfied with our family life right now. o My child is better able to do things he or she wants to do. o I know people who will listen and understand me when I need to talk. o I have people that I am comfortable talking with about my child's problems. o In a crisis, I would have the support I need from family or friends. o I have people with whom I can do enjoyable things. Adult and older adult self-report items analyzed (MHSIP): o I deal more effectively with daily problems. o I am better able to control my life. o I am better able to deal with crisis. o I am getting along better with my family. o I do better in social situations. o I do better in school and/or work. o I do things that are more meaningful to me. o I am better able to take care of my needs. o I am better able to handle things when they go wrong. o I am better able to do things that I want to do. o I am happy with the friendships I have. 42 o I have people with whom I can do enjoyable things. o I feel I belong in my community. o In a crisis, I would have the support I need from family or friends. Note: Data collected in FYs 2008-09 and 2009-10 must be interpreted separately because a convenience sampling method was employed to gather FY 2008-09 data and a random sampling method employed to gather data in FY 2009-10.3 Review of Existing Data Data source likely to be sustained (i.e., most items analyzed for this indicator are included in the August 2012 survey administration) Data relevant to population of interest (i.e., convenience or random sample of all mental health consumers) Data available across multiple service years Less than 10% missing or unknown values among respondent groups Analytic Potential of Indicator Analysis across time will be possible if the sampling methodology and instrument used is employed in a consistent manner each year Analysis among specific service populations possible (e.g., all consumers, demographic groups) County analysis possible for FY 2008-09 (convenience sample), but data is not available at the county level for FY 2009-10 3 Cowles, E. L., Harris, K., Larsen, C., and Prince, A. (2010). Assessing Representativeness of the Mental Health Services Consumer Perception Survey. 43 Figure 11.1. Perceptions of improvement in well-being as a result of services, FY 2008-09 Older Adults 4.16 (n=28) Adults 3.95 (n=456) TAY 3.92 (n=175) Children 3.96 (n=151) 1.00 2.00 3.00 4.00 5.00 Table 11.1. Perceptions of improvement in well-being as a result of services by race/ethnicity, FY 2008-09 Family Member/ TAY Adult Older Adult Caregiver White 3.96 (n=110) 3.94 (n=117) 3.97 (n=350) 4.13 (n=26) Hispanic / Latino 3.93 (n=55) 3.94 (n=49) 4.19 (n=64) N/A Asian 3.23 (n=4) 3.82 (n=11) 3.42 (n=10) N/A Pacific Islander 4.34 (n=3) 4.07 (n=3) 4.30 (n=6) N/A Black 3.87 (n=9) 4.18 (n=13) 3.97 (n=18) N/A American Indian 4.00 (n=17) 4.05 (n=24) 4.00 (n=27) N/A Other 3.81 (n=19) 3.81 (n=44) 3.71 (n=42) 5.00 (n=1) Unknown/Missing Values: Family Member/Caregiver = 4.0% (n = 9), TAY = 6.1% (n = 17), Adult = 2.9% (n = 71), Older Adult = 3.6% (n = 1) Table 11.2. Perceptions of improvement in well-being as a result of services by gender, FY 2008-09 Family Member/ TAY Adult Older Adult Caregiver Female 4.25 (n=48) 3.99 (n=98) 3.96 (n=200) 4.19 (n=14) Male 3.82 (n=102) 3.86 (n=75) 3.97 (n=233) 4.13 (n=13) Unknown/Missing Values: Family Member/Caregiver = 0.7% (n = 1), TAY = 1.1% (n = 2), Adult = 5.0% (n = 23), Older Adult = 3.6% (n = 1) 44 Priority Indicator 12: Satisfaction with Services Indicator Summary This indicator provides insight into consumer and family perceptions of satisfaction with mental health services. Indicator Calculation Family members/caregivers and TAY respondents’ ratings (1–Strongly Disagree to 5–Strongly Agree) of two self- report items (specified in the Data Sources section below) were averaged to calculate aggregate ratings of perceptions of access to mental health services (see Figures 12.1-12.2 and Tables 12.1-12.2 below). Aggregate ratings were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive perceptions. This calculation method is in line with previous DHCS practices. Adult and older adult respondents’ ratings (1–Strongly Disagree to 5–Strongly Agree) of 14 self-report items (specified in the Data Sources section below) were averaged to calculate aggregate ratings of perceptions of access to mental health services (see Figures 12.1-12.2 and Tables 12.1-12.2 below). Aggregate ratings were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive perceptions. This calculation method is in line with previous DHCS practices. Data Sources Consumer Perception Surveys Family members/caregivers and TAY self-report items analyzed (YSS/YSS-F): o Overall, I am satisfied with the services my child received. o The people helping my child stuck with us no matter what. o I felt my child had someone to talk to when he/she was troubled. o The services my child and/or family received were right for us. o My family got the help we wanted for my child. o My family got as much help as we needed for my child. Adult and older adult self-report items analyzed (MHSIP): o I like the services that I received here. o If I had other choices, I would still get services from this agency. o I would recommend this agency to a friend or family member. Note: Data collected in FY 2008-09 and 2009-10 must be interpreted separately because a convenience sampling method was used to gather FY 2008-09 data and random sampling was used to gather data in FY 2009-10.4 Review of Existing Data Data source likely to be sustained Data relevant to population of interest (i.e., convenience or random sample of all mental health consumers) Data available across multiple service years More than 10% missing or unknown values among respondent groups Analytic Potential of Indicator Analysis across time possible if the sampling methodology and instrument used is consistent each year Analysis among specific service populations possible (e.g., all consumers, demographic groups) 4 Cowles, E. L., Harris, K., Larsen, C., and Prince, A. (2010). Assessing Representativeness of the Mental Health Services Consumer Perception Survey. 45 Figure 12.1. Satisfaction with services, FY 2008-09 Older Adults 4.59 (n=28) Adults 4.20 (n=461) TAY 4.04 (n=176) Children 4.34 (n=151) 1.00 2.00 3.00 4.00 5.00 Table 12.1. Satisfaction with services by race/ethnicity, FY 2008-09 Family Member/ TAY Adult Older Adult Caregiver White 4.35 (n=110) 4.07 (n=118) 4.22 (n=352) 4.60 (n=26) Hispanic / Latino 4.34 (n=55) 4.12 (n=49) 4.22 (n=352) N/A Asian 3.33 (n=4) 3.83 (n=11) 3.57 (n=10) N/A Pacific Islander 4.72 (n=3) 4.00 (n=3) 4.56 (n=6) N/A Black 4.45 (n=9) 4.13 (n=13) 4.24 (n=18) N/A American Indian 4.57 (n=17) 4.08 (n=24) 4.21 (n=27) N/A Other 4.11 (n=19) 3.96 (n=44) 3.98 (n=41) 5.00 (n=1) Unknown/Missing Values: Family Member/Caregiver = 4.0% (n = 9), TAY = 6.1% (n = 17), Adult = 8.4% (n = 74), Older Adult = 3.6% (n = 1) Table 12.2. Satisfaction with services by gender, FY 2008-09 Family Member/ TAY Adult Older Adult Caregiver Female 4.49 (n=48) 4.18 (n=98) 4.37 (n=201) 4.76 (n=14) Male 4.27 (n=102) 3.91 (n=76) 4.07 (n=233) 4.45 (n=13) Unknown/Missing Values: Family Member/Caregiver = 1.3% (n = 1), TAY = 1.1% (n = 2), Adult = 5.9% (n = 27), Older Adult = 3.6% (n = 1) 46 Appendix A – Priority Indicator Matrix 47 Appendix B – Preceding Reports The evaluation team completed extensive groundwork before arriving at the conclusions contained in this report. To date, the team has documented evaluation planning in six reports based on statewide data made available online at UCLA and MHSOAC websites. Stakeholders were invited to provide feedback on draft reports by responding to questions in an accompanying guidance document. The invitation was shared online and through mass e-mail. Small stakeholder groups participated in one of two webinars, or online orientations to the report, that provided an overview to the report’s purpose and the types of feedback sought. Report title: Defining Priority Indicators Report version: Draft for stakeholder review Here, the evaluation team began to refine the core set of priority indicators proposed by the California Mental Health Planning Council to assess target outcomes of mental health consumers and the performance of the mental health system. Report title: Defining Priority Indicators Report version: Final, revised with stakeholder input In the revised report, the evaluation team illustrated how stakeholder feedback was integral to indicator development. This report incorporated changes driven by stakeholders’ comments about the comprehensiveness and appropriateness of the indicators. Report title: Compiling Data to Produce All Priority Indicators Report version: Draft for stakeholder review In this report, the evaluation team proposed how priority indicators could be calculated using existing statewide data. The report also detailed all possible data sources and specific variables or data fields that might be used to build comprehensive priority indicators. Report title: Compiling Data to Produce All Priority Indicators Report version: Final, revised with stakeholder input The initial report was revised to include information regarding measurement methods and the adequacy of existing data sources, gathered through a stakeholder feedback process similar to that used for the final Defining Priority Indicators report. Report title: Initial Statewide Priority Indicator Report Report version: Draft for stakeholder review The evaluation team used select data from fiscal years 2008-09 and 2009-105 to calculate priority indicators as outlined in the previous report (Compiling Data to Produce All Priority Indicators). Report title: Initial Statewide Priority Indicator Report Report version: Final, revised with stakeholder input The revised report incorporated several calculation updates guided by the MHSOAC. Updated calculations largely focused on outcome data collected after enrollment. The report also reflected input from consumer stakeholders about the use and accessibility of report features (illustrations, descriptions, etc.) and language. 5 The team sorted through datasets from 2005 through 2011 in search of one or more fiscal years (FY) in which data cells were largely filled where expected. Data from FY 2008-09 and 2009-10 fit this requirement. 48 Appendix C – Recoding Pre-DIG Race Data to Post-DIG Format Stakeholder feedback to previous evaluation team reports suggested inconsistency and potential inaccuracy among Race and Ethnicity data fields may be due in part to changes in the format of these fields in the CSI and DCR data systems. In 2006, DMH implemented changes to the Race and Ethnicity fields due to Uniform Data System/Data Infrastructure Grant (DIG) requirements from the federal government (see DMH Information Notice: 06-02). Although DMH provided training about these changes, Race and Ethnicity information seems to be reported inconsistently across counties. Because demographic information in the CSI system is transferred to corresponding fields in the DCR system, Race and Ethnicity information in both systems was analyzed but interpreted with caution. To ameliorate potential shortcomings of this change, the evaluation team used pre-DIG information to fill gaps in missing post-DIG Race and Ethnicity fields for analyses involving demographic information. The table below details the recoding process. Before Recode After Recode (if Post-DIG field empty) Data Post-DIG Data Pre-DIG Field Definition Definition Value Field Value Empty formerly White 1 Race White or Caucasian 1 Ethnicity / Race Empty formerly Hispanic 2 Ethnicity Yes (Hispanic or Latino) Y Ethnicity / Race Empty formerly Black or African Black 3 Race 3 Ethnicity / Race American Empty formerly American Indian or American Native 5 Race 5 Ethnicity / Race Alaska Native Empty formerly Amerasian A Race Other Asian O Ethnicity / Race Empty formerly Hawaiian Native P Race Native Hawaiian P Ethnicity / Race Empty formerly Multiple X Race Multiracial Multiracial Ethnicity / Race Empty formerly Other Asian or Pacific 4 Race Other Asian O Ethnicity / Race Islander 49