BHSOAC
Mhsa Priority Indicators: Solano County
Read the report at Solano County ↗
Mental Health Services Act Evaluation:
Report on Prioritized Indicators
Contract Deliverable 2F, Phase II
Solano County
UCLA Center for Healthier Children, Youth and Families
EMT Associates, Inc.
Submitted for review on November 30, 2012
The following report was funded by the
Mental Health Services Oversight and Accountability Commission.
The following report was revised in partnership with stakeholders who provided important
historical context, data consultation, and revisions to ensure this report is accurate and accessible
to the broadest audience possible. Feedback, collected prior to, during, and following report
development, was crucial to developing this report. The UCLA-EMT Evaluation Team would like to
express sincere appreciation to the research analysts, advocates, consumers and family members,
agency representatives, service providers, and MHSOAC representatives who contributed
invaluable insights to this document and previous versions.
2
Table of Contents
Purpose ........................................................................................................................................................................................ 4
Background ................................................................................................................................................................................ 4
Review of Existing Statewide Data Systems ................................................................................................................. 5
Data Sources ......................................................................................................................................................................... 6
Priority Indicators Evaluated .................................................................................................................................... 8
Priority Indicators .............................................................................................................................................................. 9
Report Organization ................................................................................................................................................... 10
Priority Indicator 1: Attendance ..................................................................................................................................... 11
1.1 Expulsions and Suspensions Per Year (CPS) ........................................................................................... 11
1.2 Average School Attendance Per Year (FSP) ............................................................................................. 13
Priority Indicator 2: Employment ................................................................................................................................. 15
Priority Indicator 3: Homelessness and Housing Rates ....................................................................................... 19
Priority Indicator 4: Arrest Rates .................................................................................................................................. 22
Priority Indicator 5: Demographic Profile of Consumers Served ..................................................................... 28
Priority Indicator 6: Demographic Profile of New Consumers ........................................................................... 32
Priority Indicator 7: Penetration of Mental Health Services .............................................................................. 37
Priority Indicator 8: Access to a Primary Care Physician .................................................................................... 40
Priority Indicator 9: Perceptions of Access to Services ........................................................................................ 43
Priority Indicator 10: Involuntary Status ................................................................................................................... 45
Priority Indicator 12: Satisfaction with Services ..................................................................................................... 50
Appendix A – Priority Indicator Matrix ........................................................................................................................ 52
Appendix B – Preceding Reports ..................................................................................................................................... 53
Appendix C – Recoding Pre-DIG Race Data to Post-DIG Format ........................................................................ 54
3
Purpose
The Mental Health Services Act Oversight and Accountability Commission (MHSOAC) charged the
UCLA-EMT Evaluation Team with tracking the Mental Health Services Act’s (MHSA) impact on
mental health service consumers and the community mental health service system using a set of
performance indicators. The current report – a companion to the statewide report published during
November 2012 – details the initial effort toward this goal at the county level.
Here, we provide a snapshot of consumer outcomes and community mental health system
performance within an individual county. This is a first step toward developing a set of indicators
that can help stakeholders with ongoing quality improvement. This report is the first in a series
designed to update stakeholders about mental health consumer outcomes and service system
progress. Tracking performance on particular outcomes over time, across programs, and within
counties can provide useful information to those planning, operating and monitoring services.
Indicators were designed for planning, quality improvement, and other applications that
stakeholders deem important. In this way, among many others, stakeholders can play a vital role in
a continuous quality improvement process.
Background
What are Priority Indicators and what are they intended to do?
Two concerns of public mental health system stakeholders are accountability and the ability to
conduct continuous quality improvement activities. One strategy is to use a set of indicators to
measure performance. The California Mental Health Planning Council proposed and defined a set of
performance indicators, referred to as priority indicators, designed to assess how the MHSA has
impacted mental health consumers and the mental health system in target areas that should be
most changed through MHSA implementation. Indicators will help track progress among
consumers and across community mental health systems. At the consumer level, outcomes such as
education and employment will be followed, while outcomes including mental health service
penetration and consumer demographics are examined at the broader system level. This report
examines the core set of priority indicators vetted by the MHSOAC within counties.
Previous work of the UCLA-EMT Evaluation Team leading to the statewide and present county
reports
The evaluation team began its work using the California Mental Health Planning Council’s
definitions – its collective vision of how indicators might best be measured. (These fundamental
definitions are located in Appendix A and are discussed in preceding reports available at
http://healthychild.ucla.edu/MHSA_evaluation.asp.) Priority indicator development was a joint effort
among the MHSOAC, stakeholders, and the evaluation team. The evaluation team facilitated
discussions among interested stakeholders to create the strongest, most comprehensive
representations of priority indicators that aligned with both early conceptualizations and feedback
using the data that were already collected across the state with some regularity. Where gaps
existed, the evaluation team proposed new data collection that will improve future evaluation but is
beyond the team’s current scope of work.
The evaluation team adapted advice from stakeholders, and this report examines whether these
adapted indicators provide meaningful information. Although stakeholders proposed additional
indicators, these measures have not yet been vetted by the MHSOAC to determine whether they add
useful and crucial information that aligns with the need. The MHSOAC has yet to decide whether to
4
change the previously approved priority indicators. Thus, the evaluation team explored the first
proposed priority indicators in this report, which serves as a fundamental step in the ongoing
process to refine and expand priority indicators that are not only measurable but also useful to the
range of stakeholders invested in this work.
Appendix B summarizes the MHSOAC-approved reports that 1) document the available data and
proposed calculations, and 2) lead to the initial statewide report on which this document is based.
Included in this summary is the stakeholder review process that took place throughout the
development of each report. This information can also be reviewed in the report “Initial Statewide
Priority Indicator Report,” available on the UCLA and MHSOAC websites.
The following report stems from the statewide report. In it, we document priority indicator
development at the county level during two target fiscal years – 2008-09 and 2009-10. 1
Review of Existing Statewide Data Systems
As directed by the MHSOAC, priority indicators were built upon existing data sources that are
systematically collected by California counties and reported to the California Department of Health
Care Services (DHCS).2 To build indicators for use at the state and county levels, existing data
systems were reviewed to assess their suitability for supporting outcome and performance
monitoring. Several criteria were used to evaluate the suitability of existing data sources, including:
Available – Data accessible in an analyzable format
Complete – Levels of missing information within key data fields did not prevent meaningful
analysis and interpretation
Sustained – Data sources is likely to continue to exist in the foreseeable future
Relevant – Data relevant to populations of interest (e.g., all mental health consumers and
Full Service Partnerships
Longitudinal – Data available for multiple service years
Multilevel – Data can be analyzed at multiple levels (e.g., state, county, and individual)
The application of these criteria to each key data source and important considerations and
limitations regarding each data source overall are summarized in the Data Sources table below.
These criteria were also applied to the specific data fields used to build each priority indicator that
would be used to monitor performance statewide and at the county level. Review of indicator-
specific data fields is summarized within the tables that introduce the analysis and findings of each
priority indicator (see Priority Indicator Analysis and Findings section below).
1 The evaluation team reviewed data from 2005 through 2011 in search of one fiscal year in which data cells
were largely filled where expected. Two such fiscal years were identified – FYs 2008-09 and 2009-10.
Through analysis, some proposed data sources or methods of indicator calculation, suggested in previous
reports by stakeholders and the evaluation team, were found to not be possible or to not produce meaningful
outcomes due to data limitations. Decisions made about previously proposed indicators, based on data
limitations, are summarized in the statewide report appendix.
2 Previously the Department of Mental Health (DMH); The DHCS abbreviation will be used to refer to work
completed by DMH.
5
Data Sources
Client & Service Information (CSI)
Summary: The CSI system is a repository of county, client (e.g., age, gender, preferred language, education,
employment status, living arrangement, etc.), and service information (number and length of service contact). The
data are collected from all consumers who receive mental health services, including consumers involved in the Full
Service Partnership.
Review Findings:
Available Complete Sustained Relevant Longitudinal Multilevel
Considerations and Limitations: Stakeholder feedback to previous evaluation team reports suggested that
inconsistency and potential inaccuracy among race and ethnicity data fields may be due in part to changes in the
format of these fields in the CSI data system (see DMH Information Notice: 06-02). For details regarding the Race
and Ethnicity data field changes and procedures employed by the evaluation team to improve data quality, see
Appendix C.
Additionally, the completeness of data fields used to calculate indicators varies greatly across fiscal years and
among counties (e.g., greater than 50% in some cases). Thus the representativeness and interpretability of such
data fields is in doubt. Proportions of missing or unknown information are noted within each indicator section
throughout the report.
Data Collection and Reporting (DCR) System
Summary: The DCR system houses data for consumers who are served through Full Service Partnership programs.
Data from assessments – the Partnership Assessment Form (PAF), Key Event Tracking (KET), and Quarterly
Assessment (3M) – are collected for consumers in specific age categories. The PAF reflects consumer history and
baseline information, including consumer education and/or employment, housing situation, legal issues, health
status, and substance use. The KET reflects any important changes in the consumer’s life, such as housing,
education and/or employment, and legal issues during FSP. The 3M is used to collect information on a quarterly
basis, regarding key areas such as education, health status, substance use, and legal issues.
Review Findings:
Available Complete Sustained Relevant Longitudinal Multilevel
Considerations and Limitations: Race and ethnicity information in the DCR system is imported from the CSI system
by DHCS. As such, the limitations of this information noted for the CSI system also apply here. Specifically,
stakeholder feedback to previous evaluation team reports suggested that inconsistency and potential inaccuracy
among race and ethnicity data fields may be due in part to changes in the format of these fields in the DCR data
system (see DMH Information Notice: 06-02). For details regarding race and ethnicity data field changes and
procedures employed by the evaluation team to improve data quality, see Appendix C.
Additionally, representatives from seven counties or municipalities that currently do not have data contained in
the DCR database in FYs 2008-09 or 2009-10, for several reasons (e.g., county data incompatibly formatted DHCS
database). Moving forward it will be imperative to indicator tracking of FSPs for all counties to be included in the
DCR database
Performance Outcomes and Quality Improvement (POQI) – Consumer Perception Surveys (CPS)
Summary: These consumer surveys are customized for consumer groups (e.g., family members/caregivers, youth,
adults, and older adults) receiving mental health services. Instruments are composed of widely validated measures
such as the Child Behavior Checklist, Youth Self Report, and Restrictiveness of Living Environment Scale for youth;
the Global Assessment of Functioning, Behavior and Symptom Identification Scale, and the California Quality of Life
for adults; and the Brief Symptom Inventory, Senior Outcomes Checklist 10, and Index of Independent Activities of
6
Daily Living for older adults. The data, designed to inform treatment planning and service management, are
collected from a sample of individuals with “serious, persistent” mental illness who have received services for 60
days or more and are not categorized as “medication only.”
Review Findings:
Available Complete Sustained Relevant Longitudinal
Considerations and Limitations: For FY 2008‐09 and prior years, a convenience sampling approach was used in
which county-level mental health service providers administered surveys twice a year for a two‐week period, in
early May and November. Investigation of the convenience sampling methodology revealed the resulting
information was not representative of the larger mental health service population.3 Beginning with FY 2009-10, a
random sampling methodology was employed to produce data that are more representative of the perceptions of
the mental health service population. As such, comparisons involving CPS data collected in FY 2008-09 and FY
2009-10 cannot be made.
Note – The smaller sample generated by the random sampling method employed in FY 2009-10 does not allow for
consumer perception analyses at the county levels for this fiscal year.
Note – The sampling methods that have been employed to date do not capture specific mental health service
populations, such as those in institutions for mental disease or prison.
Other Sources
Estimates of Need for Mental Health Services
To achieve a standardized rate for penetration of services across all counties, the evaluation team contracted with
Dr. Charles Holzer for statewide and county mental health service need estimates. Dr. Holzer previously developed
penetration rate estimates for the California DHCS. He estimated the proportion of persons with serious mental
illness among those whose income falls within 200% of the federal poverty level, using data from the most up-to-
date National Comorbidity Survey Replication and generated prevalence estimates for several Census years. (For
additional information regarding prevalence estimate methodology, see Dr. Holzer’s website at
http://66.140.7.155/estimation/3_Synthetic/synthetic.htm).
Review Findings:
Available Complete Relevant Longitudinal
Involuntary Status
Involuntary status information (FY 2008-09) was provided by DHCS for the following service categories: 72 hour
Evaluation and Treatment (adults, children); 14- and 30-day Intensive Treatment; 180-day Post Certification
Treatment; and Temporary and Permanent Conservatorships. Involuntary status data for FY 2009-10 were not
available from DHCS as this report was being prepared.
Review Findings:
Available Complete Sustained Relevant Longitudinal
Procedures for handling missing / unknown data
The quantity of missing or unknown data (e.g., values) was found to vary considerably across data
sources, data fields, and fiscal years. For data fields determined to be necessary for the construction
of priority performance indicators (detailed in priority indicator summary tables, see Priority
Indicator Analysis and Findings section below), if the amount of missing or unknown data was
substantial (i.e., greater than 10% of cases), the evaluation team communicated with DHCS analysts
3 Cowles, E. L., Harris, K., Larsen, C., and Prince, A. (2010). Assessing Representativeness of the Mental Health
Services Consumer Perception Survey.
7
and requested input from counties via a data verification process (detailed below) regarding
context and interpretation of such data fields. Where adequate information was received to
interpret missing or unknown values (i.e., see Appendix C), the evaluation team was able to analyze
and interpret such data fields according to current protocols specified in the data dictionaries
relevant to each data system. The proportion of missing and unknown information relevant to each
priority indicator within each county is noted throughout the report in footnotes immediately
below the relevant table or figure. Accordingly, the frequencies and percentages included in all data
displays do not include missing cases. The label “data unavailable” is inserted where data is
unavailable (data fields are blank).
Priority Indicators Evaluated4
The set of priority performance indicators evaluated in this report were arrived at through the
following processes:
The careful consideration of the California Mental Health Planning Council and approval of
the MHSOAC;5
Consideration of the MHSOAC goal of developing a comprehensive outcome and
performance monitoring system built upon existing data systems;
Consideration of consumer feedback to previous evaluation team reports regarding
proposed priority indicators (see report, “Defining Priority Indicators”);
Review of existing data sources to assess their suitability for supporting outcome and
performance monitoring through priority indicators (see Review of Existing Data, above);
and
County feedback regarding the quality and completeness of key data fields necessary to
calculate priority performance indicators
Through these evaluation processes and careful deliberation of the MHSOAC in collaboration with
the evaluation team, a set of 12 priority performance indicators was developed. These indicators
can be categorized as those intended to provide insight into the outcomes of mental health
consumers (“Consumer Indicators”) and those intended for monitoring the performance of the
community mental health system more broadly (“System Indicators”). Consumer and system
indicators, and the consumer groups they assess, are summarized in the table below.
4 Although we received strong indicator suggestions from stakeholders, this report helps vet the
appropriateness of the original set proposed by the California Mental Health Planning Council. If the MHSOAC
chooses, it may vet additional indicators, particularly those proposed by stakeholders, when revising the pool.
5 California Mental Health Planning Council (January, 2010). Performance Indicators for Evaluating the Mental
Health System.
8
Priority Indicators
CONSUMERS EVALUATED
SERVICE OLDER
CHILDREN TAY ADULTS
POPULATION ADULTS
CONSUMER INDICATORS
Indicator 1 – Average School Attendance Per Year All/FSP Consumers x x
Indicator 2 – Employed Consumers All/FSP Consumers x x x
Indicator 3 – Homelessness and Housing Rates All/FSP Consumers x x x x
Indicator 4 – Arrest Rate All/FSP Consumers x x x x
SYSTEM INDICATORS
Indicator 5 – Demographic Profile of Consumers Served All/FSP Consumers x x x x
Indicator 6 – Demographic Profile of New Consumers All/FSP Consumers x x x x
Indicator 7 – Penetration of Mental Health Services All Consumers x x x x
Indicator 8 – Access to a Primary Care Physician FSP Consumers x x x x
Indicator 9 – Perceptions of Access to Services All Consumers x x x x
Indicator 10 – Involuntary Status All Consumers x x x x
Indicator 11 – Consumer Well-Being All Consumers x x x x
Indicator 12 – Satisfaction All Consumers x x x x
Criteria used to evaluate priority indicators
Specific criteria, developed in collaboration with the MHSOAC, were established to evaluate priority
performance indicators. These criteria, outlined for consumer and system indicators below, reflect
the goals of the MHSOAC for monitoring consumer outcomes and community mental health system
performance at multiple levels (i.e., state and county) for the purposes of planning and quality
improvement. These criteria may include:
Consumer Indicator Evaluation Criteria:
Indicator can describe changes in consumer outcomes (e.g., change since initiation of
services) or describe the current status of consumers.
Indicator can provide meaningful and relevant insight into the outcomes of service
populations of interest (e.g., all mental health consumers, FSP consumers, and demographic
groups).
Indicator can provide meaningful and relevant insight into the outcomes of consumers
statewide and at the county level.
Indicator provides “actionable” insight, which stakeholders can use to identify areas for
service improvement.
System Indicator Evaluation Criteria:
Indicator can describe meaningful changes in system performance over time.
Indicator can provide meaningful and relevant insight regarding the extent and quality of
services provided to populations of interest (e.g., all mental health consumers, FSP
consumers, and demographic groups).
Indicator can provide meaningful and relevant insight into the performance of the
community mental health system at the statewide and county levels.
Indicator provides “actionable” insight, which stakeholders can use to identify areas for
improving the performance of the mental health system.
The application of consumer and system indicator evaluation criteria to each priority indicator is
detailed in the Priority Indicator Analysis and Findings section below.
9
Report Organization
The remainder of the report summarizes each indicator and its county-specific outcomes,
calculated using select data from FYs 2008-09 and 2009-10. The evaluation team presents
individual-level priority indicators (those specific to consumers), then does the same for system-
level priority indicators (pertaining to community mental health systems throughout the state). A
summary table precedes each indicator and its outcomes to orient the reader to what the indicator
measures, how it was calculated, and its usefulness. Detailed stakeholder feedback, integral to
shaping this and earlier reports, can be reviewed in “Initial Statewide Priority Indicator Report.” 6
Indicator summary tables
Each priority indicator is introduced and summarized in a concise and organized table in the
Priority Indicator Analysis and Findings section below. Indicator summary tables are organized into
the following sections:
Indicator Summary – Provides a brief definition of the indicator
Indicator Calculation – Details the computation used to produce the indicator
Data Sources – Specifies the data sources and relevant data fields (variables) used to
compute the indicator
Review of Existing Data – Review of data quality criteria (specified in the Review of Existing
Data section above) as applied to indicator-specific data fields
Analytic Potential of Indicator – Review of indicator evaluation criteria (specified in the
section on Criteria used to evaluate priority indicators above)
Note regarding indicator data displays
Each indicator is presented through one or more graphical displays of information. These displays
include bar graphs and tables of frequencies and percentages. For ease of viewing and interpreting
data displays specific to a service population, figures that display indicator information relevant to
all mental health consumers are presented in blue, and those relevant to FSP consumers are
presented in green.
Consistent with reporting practices associated with this work, counties with populations smaller
than 200,000 residents are grouped in outcomes displays. This group is referred to as “Small
Counties.” County-specific reports will be created for “small counties” on request. Analyses are
conducted where data is present, even if data is minimal. We include the label “data unavailable”
where absolutely no information is available within a county’s data field to support a calculation for
the priority indicator.
6 Stakeholder feedback was not collected for each of the county reports developed, thus is not included in the
current report. A full account of stakeholder feedback is available online at MHSOAC and UCLA sites, within
“Initial Statewide Priority Indicator Report.”
10
Priority Indicators Analysis and Findings: Consumer Indicators
Priority Indicator 1: Attendance
1.1 Expulsions and Suspensions Per Year (CPS)
Indicator Summary
This indicator provides descriptive information regarding the number of youth (children and TAY) expelled and
suspended from school during the 2008-09 fiscal year. This indicator illustrates a subset of mental health service
consumers who responded to consumer perceptions surveys.
Indicator Calculation
The number children and TAY who had been enrolled for at least six months and reported being expelled or
suspended from school since beginning mental health services or 12 months prior to beginning such services.
Note: Data is reported from Youth Satisfaction Surveys (YSS)
Data Sources
Consumer Perception Surveys (CPS) Data Fields: HowLong, LES12AREST, LES12PSTAREST
Review of Existing Data
Data sources likely to be sustained
Data relevant to populations of interest
Amount of missing data for child age group is approximately 37%
Amount of missing data for TAY age group is approximately 11%
Analytic Potential of Indicator
Analysis across time possible with the exception of 09-10 data
Analysis among specific service populations possible
State- and county-level analysis possible with the exception of 09-10 data
11
Figure 1.1-1. Proportion of children who were suspended or expelled prior to beginning
services and after receiving services for a period of 6 – 12 months during FY 2008-09
100.0%
100%
90%
80.0%
80%
70%
60%
50%
No
40%
Yes
30%
20.0%
20%
10%
0.0%
0%
Exp/Sus 12 mon prior to Exp/Sus since beginning services
services (n = 5) (n =5)
Unknown/Missing for FY 2008-09: Exp/Sus prior to services = 37.5% (n = 3)
Exp/Sus since receiving services = 37.5% (n = 3)
Figure 1.1-2. Proportion of TAY who were suspended or expelled prior to beginning services
and after receiving services for a period of 6 – 12 months during FY 2008-09
100%
90%
80%
70% 62.5% 62.5%
60%
50%
No
37.5% 37.5%
40%
Yes
30%
20%
10%
0%
Exp/Sus 12 mon prior to Exp/Sus since beginning services
services (n = 8) (n =8)
Unknown/Missing for FY 2009-10: Exp/Sus prior to services = 11.1% (n = 1)
Exp/Sus since receiving services = 11.1% (n = 1)
12
1.2 Average School Attendance Per Year (FSP)
Indicator Summary
This indicator provides descriptive information regarding the frequency for which Full Service Partnership
consumers (children and TAY) attended school during the 2008-09 and 2009-10 fiscal years.
Indicator Calculation
The number children and TAY who attended school always, mostly, sometimes, infrequently, and never
divided by the number of children for which there were data.
The number of TAY who attended school always, mostly, sometimes, infrequently, and never divided by the
number of TAY for which there were data.
Note: Age groupings were revised such that:
Child ages = 1-15 (same as previously)
TAY ages = 16-18 (16-25 previously)
The TAY age group was revised because education variables would be less clear for clients older than 18.
Data Sources
DCR (PAF - NONRES) Data Field: AttendanceCurr
Review of Existing Data
Data sources likely to be sustained
Data relevant to populations of interest (FSPs)
Amount of missing data for child age group is approximately 6%
Amount of missing data for TAY age group is approximately 31%
Analytic Potential of Indicator
Analysis across time possible
Analysis among specific service populations possible
State- and county-level analysis possible
13
Figure 1.2-1 –The frequency with which children and TAY attended school during FY 2008-
09 (DCR)
Unknown/Missing for FY 2008-09: Child = 5.6% (n = 2), TAY = 31.3% (n = 5)
Figure 1.2-2 –The frequency with which children and TAY attended school during FY 2009-
10 (DCR)
Unknown/Missing for FY 2009-10: Child = 11.1% (n = 5), TAY = 38.9% (n = 7)
14
Priority Indicator 2: Employment
Indicator Summary
This indicator provides the proportion of TAY, adults and older adults who are employed (paid and non-paid) and
not employed as recorded during the most recent update (second date of service). This indicator provides
descriptive information regarding clients’ employment status during their first date of service.
Indicator Calculation
Client & Service Information (CSI)
The number of paid employed clients divided by the total number of TAY, adults, and older adults for whom
there were employment data.
The number of nonpaid employed clients divided by the total number of TAY, adults, and older adults for
whom there were employment data.
The number of paid nonemployed clients divided by the total number of TAY, adults, and older adults for
whom there were employment data.
Note: There were multiple periodic updates for clients within each fiscal year. These ratios provide information for
those who had a second periodic update within a given fiscal year (or a second date of service). Additionally, the
age groupings were revised to capture those truly eligible for employment. Those who indicated they were retired
or incarcerated were excluded from calculations.
Data Collection and Reporting (DCR)
The number of TAY, adults, and older adults who reported paid employment divided by the total number of
TAY, adults, and older adults.
The number of TAY, adults, and older adults who reported nonpaid employment divided by the total number
of TAY, adults, and older adults.
The number of TAY, adults, and older adults who did not report any employment divided by the total number
of TAY, adults, and older adults.
Note for CSI and FSP data:
Age groupings were revised such that
TAY ages = 18-25 (previously 16-25)
Older adults = 60-65 (60 and up previously)
Data Sources
CSI Periodic Post-dig, Data Field: Employment Status
DCR (PAF - NONRES) Data Fields: Current_CompetitiveAvgHrWeek, Current_SupportedAvgHrWeek,
Current_TransitionalAvgHrWeek, Current_In-HouseAvgHrWeek, Current_OtherEmploymentAvgHrWeek,
Current_Non-paidAvgHrWeek
Review of Existing Data
Client & Service Information (CSI)
Data sources likely to be sustained
Data relevant to populations of interest (all consumers)
Data available across multiple service years
Amount of missing data for TAY age group is approximately 82%
Amount of missing data for Adult age group is approximately 83.3%
15
Amount of missing data for Older Adult age group is approximately 84.2%
Data Collection and Reporting (DCR)
Data sources likely to be sustained
Data relevant to populations of interest (FSPs)
The amount of missing data for these ratios is unknown given how the employment data are collected. There is no
data code option for “missing;” as a consequence, blank responses are either missing or not applicable.
Analytic Potential of Indicator
Client & Service Information (CSI)
Analysis across time possible but very difficult
Analysis among specific service populations not possible
State- and county-level analysis possible
Data Collection and Reporting (DCR)
Analysis across time possible
Analysis among specific service populations possible
State- and county-level analysis possible
16
Figure 2.1 - Proportion of clients who were employed and not employed as reported during
their second service date for FY 2008-09 (CSI)
Unknown/Missing for FY 2008-09: TAY = 81.7% (n = 197), Adult = 83.3% (n = 694),
Older adult = 84.2% (n = 32)
Figure 2.2 – Proportion of clients who were employed and not employed as reported during
their second service date for FY 2009–10 (CSI)
Unknown/Missing for FY 2008-09: TAY = 80% (n = 240), Adult = 87.5% (n = 1,193),
Older adult = 93.1% (n = 94)
17
Figure 2.3 –The proportion of FSPs who were employed during FY 2008–09 (DCR)
Unknown/Missing for FY 2008-09: Unknown
Figure 2.4 –The proportion of FSPs who were employed during FY 2009–10 (DCR)
Unknown/Missing for FY 2009-10: Unknown
18
Priority Indicator 3: Homelessness and Housing Rates
Indicator Summary
This indicator summarizes the housing status of all mental health consumers and FSPs served during FYs 2008-09
and 2009-10. There are two parts: (a) a breakdown by most recently available housing status and (b) the
percentage of consumers experiencing homelessness at any point during the year.
Indicator Calculation
Frequencies of the most recent housing statuses were computed for mental health and FSP consumers served in
FYs 2008-09 and 2009-10. This calculation excludes consumers with no housing data within the given FYs or
consumers whose most recent status was homeless. The percentages of mental health and FSP consumers who
experienced homelessness at any point during the given FY were also computed.
Note that a consumer who was most recently homeless would not be included in the first indicator for most recent
housing status, whereas a consumer who was previously homeless and more recently reported as not homeless
would be included.
Data Sources
Client & Service Information (CSI): H-01.0 County / City / Mental Health Plan Submitting Record; H-02.0 County
Client Number; C-03.0 Date of Birth; P-01.0 Date Completed; P-09.0 Living Arrangement
Data Collection and Reporting (DCR) Key Event Tracking (KET): 1.01 Global ID; 1.02 Assessment ID; 1.07 Age Group;
3.01 CountyID; 3.06 Assessment Date; 5.01 DateResidentialChange; 5.02 Current
Review of Existing Data
These data were taken from the Key Event Tracking (KET) updates for FSP consumers and the periodic updates for
all mental health consumers, limited to the given fiscal year. Any consumer who did not have an update available
during the year was not included. Data sources are likely to be sustained in the foreseeable future, providing a
consistent source for tracking system performance moving forward. Taking a conservative approach, we
considered cases without valid data “missing.” It should be noted that the data reporting and collection practices
currently in place do not allow for a distinction between missing data from unreported changes in housing status
and blank values from standard data entry practices. This is especially notable in the KET updates for FSP
consumers, leading to large percentages of “missing” data. These results should be interpreted cautiously. In
particular, there is the risk of systematic bias in underreporting certain housing statuses.
Analytic Potential of Indicator
Data across service years support analysis of the distribution and change of housing statuses, including
homelessness, among consumers.
Indicator Displays
The first set of charts displays the most recently reported non-homeless housing statuses of consumers, by
percentage, during each fiscal year. The second set displays the percentages of consumers who were reported as
experiencing homelessness at any time during the fiscal year.
19
Figure 3.1 – Most recent housing status excluding homelessness, all consumers (CSI)
100% 2.9% 2.8%
90% 9.7% 9.8% 1 4 0 .1 .0 % % 3 8 . . 9 9 % % 1 0 3 .0 .9 % % 1 0 6 .0 .9 % % 1 0 2 .0 .5 % % 31.5%
80%
70% 0.0%
60%
50%
40% 87.4% 87.4% 85.9% 87.3% 86.1% 83.1% 87.5%
68.5%
30%
20%
10%
0%
Child Child TAY 08-TAY 09- Adult Adult Older Older
08-09 09-10 09 10 08-09 09-10 Adult Adult
08-09 09-10
House or Apartment Foster Care Group Setting
Unknown/Missing for FY 2008-09 = 2.3% (n =18) for children; 6.7% (n = 29) for TAY; 8% (n = 71) for
adults; and 4.4% (n = 3) for older adults
Unknown/Missing for FY 2009-10 = 1.7% (n =16) for children; 4.8% (n = 24) for TAY; 6.3% (n = 93) for
adults; and 4.7% (n = 9) for older adults
Figure 3.2 – Most recent housing status excluding homelessness, FSP consumers only (DCR)
100% 0.0%
90%
35.0%
80%
47.1%
70% 62.5%
66.7% 66.7% 70.4% 0.0%
60% 75.8%
0.0%
50% 100.0%
40%
00..00%%
0.0% 60.0%
30% 0.0%
52.9%
0.0%
20% 37.5% 6.1% 22.2%
33.3% 33.3%
10% 18.2%
7.4% 5.0%
0% 00..00%% 000...000%%% 0.0%
Child Child TAY 08-TAY 09- Adult Adult Older Older
08-09 09-10 09 10 08-09 09-10 Adult Adult
08-09 09-10
With Family Independent Foster Care Group Setting
Unknown/Missing for FY 2008-09 = 66.7% (n =6) for children; 42.9% (n = 6) for TAY; 5.6% (n = 2) for
adults; and 39.3% (n = 11) for older adults
Unknown/Missing for FY 2009-10 = 81.3% (n =13) for children; 87.5% (n = 14) for TAY; 28.9% (n = 11) for
adults; and 37.5% (n = 12) for older adults
20
Figure 3.3 – Experienced homelessness at any point during the year, all consumers (CSI)
20%
18%
16%
14%
12%
10% FY 2008-2009
8% 6.8% FY 2009-2010
6%
4.4% 4.6%
4% 3.2%
1.5% 1.7%
2% 0.7%
0.1%
0%
Child TAY Adult Older Adult
Unknown/Missing for FY 2008-09 = 2.3% (n =18) for children; 6.7% (n = 29) for TAY; 8% (n = 71) for
adults; and 4.4% (n = 3) for older adults
Unknown/Missing for FY 2009-10 = 1.7% (n =16) for children; 4.8% (n = 24) for TAY; 6.3% (n = 93) for
adults; and 4.7% (n = 9) for older adults
Figure 3.4 – Experienced homelessness at any point during the year, FSP consumers only
(DCR)
20%
17.6%
18%
16%
14%
12.5%
12% 11.1%
10% 8.8% FY 2008-2009
FY 2009-2010
8%
6%
4%
2%
0.0% 0.0% 0.0% 0.0%
0%
Child TAY Adult Older Adult
Unknown/Missing for FY 2008-09 = 66.7% (n =6) for children; 42.9% (n = 6) for TAY; 5.6% (n = 2) for
adults; and 39.3% (n = 11) for older adults
Unknown/Missing for FY 2009-10 = 81.3% (n =13) for children; 87.5% (n = 14) for TAY; 28.9% (n = 11) for
adults; and 37.5% (n = 12) for older adults
21
Priority Indicator 4: Arrest Rates
Indicator Summary
This indicator provides the proportion of youth, adults, and older adults who reported being arrested 12 months
prior to receiving services and the proportion of youth, adults, and older adults who reported being arrested since
beginning services.
For calculations involving consumer perception surveys, this indicator includes only youth, adults, and older adults
who reported receiving services for 6 to 12 months. This indicator provides information regarding whether the
proportion of arrested clients has increased or decreased after 6 to 12 months of service.
For calculations involving Full Service Partnership consumers, this indicator tracks arrests prior to enrollment using
intake data. This indicator accounts for consumers enrolled during the target fiscal years for which PAF surveys are
available.
Indicator Calculation
Consumer Perception Surveys (CPS)
The number of reported arrest 12 months prior to services divided by the total number of youth, adults, and
older adults for who there was data
The number of reported arrest since beginning services divided by the total number of youth, adults, and older
adults for who there was data
Note: Clients were surveyed multiple times during the 2008 – 2009 fiscal year. However, only one survey
administration was used to get both the proportion of clients who reported being arrested 12 months prior to
beginning services and since receiving services.
Age groupings are as follows:
Youth, 1 – 25 years
Adult, 26 – 59 years
Older adult, 60 and above
Data Collection and Reporting (DCR)
The number of youth (children and TAY), adults, and older adults reporting arrests 12 months prior to
enrollment divided by the total number of unique clients for who there was data
The number of youth (children and TAY), adults, and older adults reporting arrests 12 months prior to the
past 12 months divided by the total number of unique clients for who there was data
Note: In rare cases where two surveys were entered for one client, only the earliest entry was used in calculations.
Data Sources
Consumer Perception Survey (CPS) for Youth, Adults, and Older Adults Data Fields: HowLong, les12arest,
les12pstarest
Data Collection and Reporting (DCR PAF NONRES): Age_Group, ArrestPast12, ArrestPrior12
Review of Existing Data
Consumer Perception Surveys (CPS)
Data sources likely to be sustained
Data relevant to populations of interest
Approximately 29.4% missing or unknown values for youth
Approximately 18.2% missing or unknown values for adult
Missing or unknown data are not available for older adult
22
Data Collection and Reporting (DCR)
Data sources likely to be sustained
Data relevant to populations of interest
On average, 10.8% missing or unknown values for children
On average, 5.0% missing or unknown values for TAY
On average, 0.0% missing or unknown values for adults
On average, 4.2% missing or unknown values for older adults
Analytic Potential of Existing Data
For both data sources
Analysis across time possible
Analysis among specific service populations not possible
State and county individual level analysis possible
Note: As of the submission of this report, a new calculation has been proposed to examine arrest rates.
The proposed calculation would use FSP-DCR data during consumers’ enrollment (not intake as it is
presented here). An updated indicator will be available shortly.
23
Figure 4.1 – Proportion of youth who were arrested prior to beginning services and since
receiving services
Missing/unknown for Arrested 12 months prior to services = 29.4% (n = 5)
Missing/unknown for Arrested since receiving services = 29.4% (n =5)
Figure 4.2 – Proportion of adults who were arrested prior to beginning services and since
receiving services
Missing/unknown for Arrested 12 months prior to services = 18.2% (n = 4)
Missing/unknown for Arrested since receiving services = 18.2% (n =4)
24
Figure 4.3 – Proportion of older adults who were arrested prior to beginning services and
since receiving services
Missing/unknown data for Arrested 12 months prior to services are not available
Missing/unknown data for Arrested since receiving services are not available
Figure 4.4 - Proportion of children who were arrested within the past 12 months (DCR)
Missing/Unknown for FY 2008-09 = 4.8% (n=1)
Missing/Unknown for FY 2009-10 = 16.7% (n=3)
25
Figure 4.5 - Proportion of TAY who were arrested within the past 12 months (DCR)
Missing/Unknown for FY 2008-09 = 0.0% (n=0)
Missing/Unknown for FY 2009-10 = 10.0% (n=2)
Figure 4.6 – Proportion of adults who were arrested within the past 12 months (DCR)
Missing/Unknown for FY 2008-09 = 0.0% (n=0)
Missing/Unknown for FY 2009-10 = 0.0% (n=0)
26
Figure 4.7 Proportion of older adults who were arrested within the past 12 months (DCR)
Missing/Unknown for FY 2008-09 = 8.3% (n=1)
Missing/Unknown for FY 2009-10 = 0.0% (n=0)
27
Priority Indicators Analysis and Findings: Community Mental Health
System Indicators
Priority Indicator 5: Demographic Profile of Consumers Served
Indicator Summary
This indicator profiles the demographics (race/ethnicity, age, and gender) of all mental health consumers and Full
Service Partnership consumers served during FYs 2008-09 and 2009-10. It summarizes levels of service to
California’s diverse population supported by the community mental health system.
Indicator Calculation
The frequencies of all Solano County mental health consumers and FSP consumers served in FY 2008-09 and
2009-10 were calculated overall.
Additionally, the proportion of consumers represented by race/ethnicity, age, and gender categories was
calculated by dividing the number of consumers within each demographic category by all consumers served
within Contra Costa County. Proportions were calculated for both service populations (all consumers and FSPs)
and both fiscal years examined (see Figures 5.1-5.6 below).
Data Sources
Client & Service Information (CSI) Data Fields: H-01.0 County / City / Mental Health Plan Submitting Record; H-
02.0 County Client Number; C-05.0 Gender; C-09.0 Ethnicity; C-10.0 Race; S-05.0 Mode of Service; S-16.0 From
/ Entry Date; S-17.0 Through / Exit Date; S-23.0 Date of Service.
Data Collection and Reporting (DCR) Data Fields: 1.01 Global ID; 1.02 Assessment ID; 1.04 Date Partnership
Status Change; 1.05 Partnership Status; 1.07 Age Group; 1.08 Assessment Type; 2.01 CSI Date of Birth; 2.02
Gender; 2.03 CSIRace1; 2.04 CSIRace2; 2.05 CSIRace3; 2.06 CSIRace4; 2.07 CSIRace5; 2.10 CSI Hispanic; 3.01
County ID; 3.05 Partnership Date; 3.06 Assessment Date.
Review of Existing Data
Data sources likely to be sustained
Data relevant to populations of interest (all consumers & FSPs)
Data available across multiple service years
Less than 10% missing or unknown values for CSI and DCR data (see Appendix C for details of recoding race/
ethnicity data fields)
Analytic Potential of Indicator
Analysis across time possible
Analysis among specific service populations possible
28
Figure 5.1. Race/ethnicity of mental health consumers
100.0%
90.0%
80.0%
70.0%
60.0%
50.0%
40.8% 40.4%
40.0%
27.7% 27.3%
30.0%
20.0%
13.8% 13.5% 4.7% 5.1% 10.5% 11.6%
10.0%
0.2% 0.2% 0.8% 0.5% 1.4% 1.5%
0.0%
White Hispanic/Latino Asian Pacific Islander Black American Multirace Other
Indian
FY 2008-09 (5,138) FY 2009-10 (4,587)
FY 2008-09 Unknown/Missing = 1.2% (n = 65); FY 2009-10 Unknown/Missing = 1.4% (n = 63)
Figure 5.2. Race/ethnicity of FSP consumers
100.0%
90.0%
80.0%
70.0%
60.0%
51.3%
50.0%
44.0%
40.0%
30.0% 28.7% 25.5%
20.0% 15.2%
7.8% 7.1% 6.0% 5.2%
10.0%
2.6% 0.0% 0.0% 2.6% 1.1% 1.7% 1.1%
0.0%
White Hispanic/Latino Asian Pacific Islander Black American Multirace Other
Indian
FY 2008-09 (115) FY 2009-10 (184)
FY 2008-09 Unknown/Missing = 6.5% (n = 8); FY 2009-10 Unknown/Missing = 6.1% (n = 12)
29
Figure 5.3. Mental health consumers by age group
100.0%
90.0%
80.0%
70.0%
60.0%
48.0% 46.9%
50.0%
40.0%
28.2% 30.3%
30.0%
17.8% 16.6%
20.0%
6.0% 6.3%
10.0%
0.0%
Children TAY Adults Older Adults
FY 2008-09 (5,203) FY 2009-10 (4,650)
FY 2008-09 Unknown/Missing = 0.0% (n = 0); FY 2009-10 Unknown/Missing = 0.0% (n = 0)
Figure 5.4. FSP consumers by age group
100.0%
90.0%
80.0%
70.0%
60.0%
50.0%
41.8%
40.0%
27.1% 28.6% 25.6%
30.0%
23.0% 18.8% 18.4% 16.8%
20.0%
10.0%
0.0%
Children TAY Adult Older Adult
FY 2008-09 (133) FY 2009-10 (196)
FY 2008-09 Unknown/Missing = 0.0% (n = 0); FY 2009-10 Unknown/Missing = 0.0% (n = 0)
30
Figure 5.5. Mental health consumers by gender
100.0%
90.0%
80.0%
70.0%
60.0%
47.7% 47.6% 52.3% 52.4%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
Female Male
FY 2008-09 (5,200) FY 2009-10 (4,648)
FY 2008-09 Unknown/Missing = 0.1% (n =3); FY 2009-10 Unknown/Missing = 0.0% (n = 2)
Figure 5.6. FSP consumers by gender
100.0%
90.0%
80.0%
70.0%
56.8% 55.1%
60.0%
44.9%
50.0%
43.2%
40.0%
30.0%
20.0%
10.0%
0.0%
Female Male
FY 2008-09 (125) FY 2009-10 (185)
FY 2008-09 Unknown/Missing = 6.0% (n = 8); FY 2009-10 Unknown/Missing = 5.6% (n = 11)
31
Priority Indicator 6: Demographic Profile of New Consumers
Indicator Summary
This indicator profiles new mental health consumers (i.e., served during FY, without service for prior six months)
overall and full service partners (FSPs) served during FYs 2008-09 and 2009-10.
Indicator Calculation
For all Contra Costa County mental health consumers, CSI data supports calculation of new versus past (i.e.,
initial services received prior to the given FY) consumers overall and within race/ethnicity, age, and gender
categories. The frequency of new consumers served was divided by all previous consumers served, in each
fiscal year, to calculate the proportion of new consumer served. This same calculation was conducted within
each demographic category (race/ethnicity, age, and gender), in each FY (see Figures 6.1 – 6.4, below).
For FSPs, Solano County DCR data supports calculation of new versus existing (i.e., current full service
partners) overall and within race/ethnicity, age, and gender categories. The frequency of new consumers
served was divided by all existing consumers, in each fiscal year, to calculate the proportion of new consumers
served. This same calculation was conducted within each demographic category (race/ethnicity, age, and
gender), in each FY (see Figures 6.5 – 6.8, below).
Data Sources
Client & Service Information (CSI) Data Fields: H-01.0 County / City / Mental Health Plan Submitting Record; H-
02.0 County Client Number; C-05.0 Gender; C-09.0 Ethnicity; C-10.0 Race; S-05.0 Mode of Service; S-16.0 From
/ Entry Date; S-17.0 Through / Exit Date; S-23.0 Date of Service.
Data Collection and Reporting (DCR) Data Fields: 1.01 Global ID; 1.02 Assessment ID; 1.04 Date Partnership
Status Change; 1.05 Partnership Status; 1.07 Age Group; 1.08 Assessment Type; 2.01 CSI Date of Birth; 2.02
Gender; 2.03 CSIRace1; 2.04 CSIRace2; 2.05 CSIRace3; 2.06 CSIRace4; 2.07 CSIRace5; 2.10 CSI Hispanic; 3.01
County ID; 3.05 Partnership Date; 3.06 Assessment Date.
Review of Existing Data
Data sources likely to be sustained
Data relevant to populations of interest (all consumers & FSPs)
Data available across multiple service years
More than 10% missing or unknown values for FY 2008-09 CSI data; less than 10% DCR data and FY 2009-10
CSI data (see Appendix C for details of recoding race/ ethnicity data fields)
Analytic Potential of Indicator
Analysis across time possible
Analysis among specific service populations possible
32
All Consumers – Data Source: Client & Service Information (CSI)
Figure 6.1. New and continuing mental health consumers
100.0%
90.0%
78.2%
80.0%
70.0% 65.4%
60.0%
50.0%
40.0%
34.6%
30.0%
21.8%
20.0%
10.0%
0.0%
FY 2008-09 (9,304) FY 2009-10 (10,777)
Previous Consumers New Consumers
FY 2008-09 Unknown/Missing = 13.7%; (n = 1,473) FY 2009-10 Unknown/Missing = 0.0% (n = 0)
Figure 6.2. Race/ethnicity of new mental health consumers
100.0%
90.0%
80.0%
70.0%
60.0%
50.0%
40.5% 37.5%
40.0%
26.0% 27.6%
30.0%
15.2% 15.9%
20.0%
4.5% 4.7% 11.1% 11.5%
10.0%
0.2% 0.3% 0.8% 0.3% 1.8% 2.1%
0.0%
White Hispanic/Latino Asian Pacific Islander Black American Multirace Other
Indian
FY 2008-09 (3,193) FY 2009-10 (2,326)
FY 2008-09 Unknown/Missing = 0.8%; (n = 25) FY 2009-10 Unknown/Missing = 0.9% (n = 21)
33
Figure 6.3. New mental health consumers by age group
100.0%
90.0%
80.0%
70.0%
60.0%
46.5%
50.0%
42.0%
40.0% 34.8%
28.2%
30.0%
20.2% 18.6%
20.0%
10.0% 5.0% 4.6%
0.0%
Children TAY Adults Older Adults
FY 2008-09 (3,218) FY 2009-10 (2,347)
FY 2008-09 Unknown/Missing = 0.0% (n = 0); FY 2009-10 Unknown/Missing = 0.0% (n = 0)
Figure 6.4. New mental health consumers by gender
100.0%
90.0%
80.0%
70.0%
60.0%
50.4% 51.2% 49.6% 48.8%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
Female Male
FY 2008-09 (3,217) FY 2009-10 (2,346)
FY 2008-09 Unknown/Missing = 0.0% (n = 1); FY 2009-10 Unknown/Missing = 0.0% (n = 1)
34
FSP Consumers – Data Source: Data Collection and Reporting (DCR)
Figure 6.5. New and continuing FSP consumers
100.0%
90.0%
80.0%
70.0%
54.1%
60.0%
54.1%
45.9% 45.9%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
FY 2008-09 (133) FY 2009-10 (196)
Previous Consumers New Consumers
FY 2008-09 Unknown/Missing = 0.0% (n = 0); FY 2009-10 Unknown/Missing = 0.0% (n = 0)
Figure 6.6. Race/ethnicity of new FSP consumers
100.0%
90.0%
80.0%
70.0%
60.0%
47.2%
50.0%
44.0%
40.0%
30.0% 26.4% 25.5%
15.2%
20.0%
7.2% 7.1% 6.0% 12.8%
10.0%
2.4% 0.0% 0.0% 2.4% 1.1% 1.6% 1.1%
0.0%
White Hispanic/Latino Asian Pacific Islander Black American Multirace Other
Indian
FY 2008-09 (125) FY 2009-10 (184)
FY 2008-09 Unknown/Missing = 6.0% (n = 8); FY 2009-10 Unknown/Missing = 6.1% (n =12)
35
Figure 6.7. New FSP consumers by age group
100.0%
90.0%
80.0%
70.0%
60.0%
50.0%
41.8%
40.0%
27.1% 28.6% 25.6%
30.0%
23.0% 18.8% 18.4% 16.8%
20.0%
10.0%
0.0%
Children TAY Adults Older Adults
FY 2008-09 (133) FY 2009-10 (196)
FY 2008-09 Unknown/Missing = 0.0% (n = 0); FY 2009-10 Unknown/Missing = 0.0% (n = 0)
Figure 6.8. FSP consumers by gender
100.0%
90.0%
80.0%
70.0%
56.8% 55.1%
60.0%
44.9%
50.0%
43.2%
40.0%
30.0%
20.0%
10.0%
0.0%
Female Male
FY 2008-09 (125) FY 2009-10 (185)
FY 2008-09 Unknown/Missing = 6.0% (n = 8); FY 2009-10 Unknown/Missing = 5.6% (n = 11)
36
Priority Indicator 7: Penetration of Mental Health Services
Indicator Summary
This indicator details rates of service to all consumers relative to estimates of need for service among Californians
earning less than 200% of the federal poverty income level. This metric is intended to show the extent to which
service levels are in line with the level of need.
Indicator Calculation
For all Solano County mental health consumers, the number of all mental health consumers served and among
demographic category (i.e., race/ethnicity, age, and gender) was divided by estimates of need for service
(Holzer Targets) among Solano County residents earning less than 200% of the federal poverty level overall
and among each demographic category (see Figures 7.1-7.5 below).
Data Sources
Client & Service Information (CSI) Data Fields: H-01.0 County/City/Mental Health Plan Submitting Record; H-
02.0 County Client Number; C-05.0 Gender; C-09.0 Ethnicity; C-10.0 Race; S-05.0 Mode of Service; S-16.0 From
/ Entry Date; S-17.0 Through / Exit Date; S-23.0 Date of Service.
Estimates of need for mental health services (Holzer Targets) among Californians earning less than 200% of
the federal poverty income level.
Review of Existing Data
Data sources likely to be sustained
Data appropriate for analysis of all mental health consumers. The estimates of need for service (Holzer
Targets) used are not appropriate points of comparison for FSP service levels.
Data available across multiple service years
Less than 10% missing or unknown values (see Appendix C for details of recoding race/ ethnicity data fields)
Analytic Potential of Indicator
Analysis across time possible
Analysis among specific service populations possible
37
Figure 7.1. Penetration of mental health services
100.0%
80.0%
68.6% 60.9%
60.0%
40.0%
20.0%
0.0%
FY 2008-09 FY 2009-10
Consumers Served/Holzer Target
FY 2008-09 (5,200/7,575)
FY 2009-10 (4,648/7,632)
Figure 7.2. Penetration of services by race/ethnicity
250.0%
221.9% 218.8%
200.0%
150.0%
122.6%
109.2% 107.5% 103.9%
100.0%
73.0%
64.7%
30.5% 46.1% 43.1% 38.2%
50.0%
25.9% 23.1% 22.5% 21.1%
0.0%
White Hispanic/Latino Asian Pacific Islander Black Amerian Indian Multirace Other
FY 2008-09 FY 2009-10
FY 2008-09 FY 2009-10
White (2,097/2,874) (1,853/2,865)
Hispanic (711/2,332) (617/2,384)
Black (1,425/1,162) (1,252/1,147)
Asian (241/523) (232/538)
Pacific Islander (9/39) (9/40)
American Indian (42/110) (24/114)
Multirace (542/504) (530/510)
Other (71/32) (70/32)
38
Figure 7.3. Penetration of mental health services by age
100.0%
85.5%
70.8%
80.0%
70.4% 65.1%
59.1% 56.6% 61.3% 58.5%
60.0%
40.0%
20.0%
0.0%
Children TAY Adults Older Adults
FY 2008-09 FY 2009-10
FY 2008-09 FY 2009-10
Children (1,465/2,480) (1,407/2,484)
TAY (928/1,086) (772/1,097)
Adult (2,498/3,530) (2,179/3,552)
Older Adult (312/479) (292/499)
Figure 7.4. Penetration of mental health services by gender
100.0%
88.4%
78.6%
80.0%
55.1%
60.0%
48.8%
40.0%
20.0%
0.0%
FY 2008-09 FY 2009-10 FY 2008-09 FY 2009-10
Female Male
FY 2008-09 FY 2009-10
Female (2,478/4,498) (2,214/4,536)
Male (2,722/3,079) (2,434/3,096)
39
Priority Indicator 8: Access to a Primary Care Physician
Indicator Summary
This indicator details the level of access to a primary care physician reported among FSP consumers, during FYs
2008-09 and 2009-10.
Indicator Calculation
The ratio of Solano County FSP consumers indicating access to a primary care physician at any point during a
fiscal year to all Solano County FSP consumers served during a fiscal year was calculated (see Figure 8.1). This
ratio was also calculated within demographic categories (i.e., race / ethnicity, age, and gender), for each FY
(see Figures 8.2-8.4, below).
Data Sources
Data Collection and Reporting (DCR) Data Fields: 1.01 Global ID; 1.02 Assessment ID; 1.04 Date Partnership Status
Change; 1.05 Partnership Status; 1.07 Age Group; 1.08 Assessment Type; 2.01 CSI Date of Birth; 2.02 Gender; 2.03
CSIRace1; 2.04 CSIRace2; 2.05 CSIRace3; 2.06 CSIRace4; 2.07 CSIRace5; 2.10 CSI Hispanic; 3.01 County ID; 3.05
Partnership Date; 3.06 Assessment Date; 11.01 PhysicianCurr.
Review of Existing Data
Data source likely to be sustained
Data relevant to population of interest (FSPs). Relevant data not available to assess primary care access among
all mental health consumers (e.g., CSI).
Data available across multiple service years
Less than 10% missing or unknown values within key DCR fields (see Appendix C for details of recoding
race/ethnicity data fields)
Analytic Potential of Indicator
Analysis across time possible
Analysis among specific service populations possible
40
Figure 8.1. FSP access to a primary care physician
100.0% 94.8% 93.6%
80.0%
60.0%
40.0%
20.0%
0.0%
FY 2008-09 (109) FY 2009-10 (147)
FY 2008 -09 Unknown/Missing = 0.0% (n = 0); FY 2009-10 Unknown/Missing = 0.0% (n= 0)
Figure 8.2. FSP access to a primary care physician by race/ethnicity
100.0%
80.0%
60.0%
47.6% 44.6%
40.0%
24.3%
20.0%
12.6% 13.7%
7.8% 7.2% 2.9% 7.2% 2.9% 7.2% 0.0% 2.9% 1.4% 1.9% 0.7%
0.0%
White Asian Black Multirace
FY 2008-09 (103) FY 2009-10 (139)
FY 2008 -09 Unknown/Missing = 5.5% (n = 6); FY 2009-10 Unknown/Missing = 5.4% (n= 8)
Figure 8.3. FSP access to a primary care physician by age group
100.0%
80.0%
60.0%
46.9%
40.0%
27.5% 27.5%
25.7%
20.0% 19.7% 19.3% 19.0%
14.3%
0.0%
Children TAY Adults Older Adults
FY 2008-09 (109) FY 2009-10 (147)
FY 2008 -09 Unknown/Missing = 0.0% (n = 0); FY 2009-10 Unknown/Missing = 0.0% (n= 0)
41
Figure 8.4. FSP access to a primary care physician by gender
100.0%
80.0%
55.3% 56.4%
60.0%
44.7% 43.6%
40.0%
20.0%
0.0%
Female Male
FY 2008-09 (103) FY 2009-10 (140)
FY 2008 -09 Unknown/Missing = 5.5% (n = 6); FY 2009-10 Unknown/Missing = 4.8% (n= 7)
42
Priority Indicator 9: Perceptions of Access to Services
Indicator Summary
This indicator provides insight into consumer and family perceptions of access to mental health services.
Indicator Calculation
Family members/caregivers and TAY respondents’ ratings (1–Strongly Disagree to 5–Strongly Agree) of two
self-report items (specified under the “Data Sources” section below) were averaged to calculate aggregate
ratings of perceptions of access to mental health services (see Figure 9.1 and Tables 9.1-9.2 below). Aggregate
ratings were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive perceptions.
This calculation method is in line with previous DHCS practices.
Adult and Older Adult respondents’ ratings (1-Strongly Disagree to 5-Strongly Agree) of fourteen self-report
items (specified under the “Data Sources” section below) were averaged to calculate aggregate ratings of
perceptions of access to mental health services (see Figures 9.1 and Tables 9.1-9.2 below). Aggregate ratings
were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive perceptions. This
calculation method is in line with previous DHCS practices.
Data Sources
Consumer Perception Surveys
Family members/caregivers and TAY self-report items analyzed (YSS/YSS-F):
o The location of services was convenient for us.
o Services were available at times that were convenient for us.
Adult and older adult self-report items analyzed (MHSIP):
o The location of services was convenient (parking, public transportation, distance, etc.).
o Staff were willing to see me as often as I felt it was necessary.
o Staff returned my call in 24 hours.
o Services were available at times that were good for me.
o I was able to get all the services I thought I needed.
o I was able to see a psychiatrist when I wanted to.
Review of Existing Data
Data source likely to be sustained
Data relevant to population of interest (i.e., convenience or random sample of all mental health consumers).
Data available across multiple service years
More than 10% missing or unknown values within key CPS scales
Analytic Potential of Indicator
Analysis among specific service populations possible
County analysis possible for FY 2008-09 (convenience sample), but data is not available at the county level for
FY 2009-10
43
Figure 9.1. Perceptions of access to service
Older Adults 4.05 (n=16)
Adults 4.04 (n=224)
TAY 4.04 (n=180)
Children 4.48 (n=237)
1.00 2.00 3.00 4.00 5.00
Table 9.1. Perceptions of access to services by race/ethnicity
Family Member/
TAY Adult Older Adult
Caregiver
White 4.50 (n=116) 4.07 (n=88) 4.09 (n=104) 4.06 (n=8)
Hispanic / Latino 4.56 (n=87) 4.11 (n=40) 4.01 (n=42) 3.50 (n=2)
Asian 4.22 (n=9) 3.21 (n=12) 4.28 (n=21) 4.00 (n=1)
Pacific Islander 4.64 (n=7) 3.21 (n=12) 3.82 (n=7) N/A
Black 4.42 (n=75) 4.04 (n=57) 4.05 (n=61) 3.87 (n=4)
American Indian 4.42 (n=19) 4.05 (n=20) 4.07 (n=23) 5.00 (n=1)
Other 4.65 (n=52) 4.14 (n=32) 3.95 (n=23) 4.33 (n=1)
FY 2008-09 Missing/Unknown Values: Family Member/Caregiver = 2.9% (n = 11), TAY = 5.8% (n = 16), Adult = 6.0%
(n = 18), Older Adult = 10.5% (n = 2)
Table 9.2. New mental health consumers by gender
Family Member/
TAY Adult Older Adult
Caregiver
Female 4.47 (n=89) 4.15 (n=66) 4.16 (n=95) 4.22 (n=6)
Male 4.51 (n=141) 3.95 (n=99) 3.98 (n=111) 3.77 (n=8)
FY 2008-09 Missing/Unknown Values: Family Member/Caregiver = 10.2% (n = 26), TAY = 21.4% (n = 45), Adult =
19.2% (n = 49), Older Adult = 17.6% (n = 3)
44
Priority Indicator 10: Involuntary Status
Indicator Summary
This indicator provides insight into the rates of involuntary status among all mental health consumers during FY
2008-09. Involuntary status refers to a legal designation that can be applied to individuals who are found to be a
danger to themselves and/or others, and/or gravely disabled.
Indicator Calculation
The California Department of Health Care Services (DHCS) reports incidents of involuntary status per 10,000
consumers for Solano County. Such rates are reported here (see Figure 10.1, below).
Data Sources
The California Department of Health Care Services provides reports of incidents of involuntary status (see
http://www.dmh.ca.gov/statistics_and_data_analysis/Involuntary_Detention.asp)
Review of Existing Data
Data source likely to be sustained
Data relevant to population of interest (all mental health consumers). Relevant data is not available to
specifically assess the involuntary status among FSP consumers.
Data available across multiple service years
Analytic Potential of Indicator
Aggregate data does not allow for analysis among specific (e.g., demographic) service populations
45
Figure 10.1. Involuntary status per 10,000 consumers, FY 2008-09 (NOTE: horizontal scale
reduced for ease of viewing)
Additional 14-day Intesive (suicidal) 1.8
14-day Intensive Treatment 63.3
72-Hour Evaluation and Treatment - Child 56.8
72 Hour Evaluation and Treament - Adult 0.0
0 10 20 30 40 50 60 70 80 90 100
46
Priority Indicator 11: Consumer Perceptions of Improvement in Well-
Being as a Result of Services
Indicator Summary
This indicator provides insight into consumer and family perceptions of well-being (i.e., outcomes, functioning, and
social connectedness) as a result of mental health services.
Indicator Calculation
Family members/caregivers and TAY respondents’ ratings (1–Strongly Disagree to 5–Strongly Agree) of eleven
self-report items (specified under the “Data Sources” section below) were averaged to calculate aggregate
ratings of perceptions of wellbeing as a result of mental health services (see Figure 11.1 and Tables 11.1-11.2,
below). Aggregate ratings were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate
positive perceptions. This calculation was developed to approximate domains of wellbeing many respondents
noted in their feedback to our initial reports.
Adult and Older Adult respondents’ ratings (1–Strongly Disagree to 5–Strongly Agree) of fourteen self-report
items (specified under the “Data Sources” section below) were averaged to calculate aggregate ratings of
perceptions of wellbeing as a result of mental health services (see Figure 11.1 and Tables 11.1-11.2, below).
Aggregate ratings were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive
perceptions. This calculation was developed to approximate domains of wellbeing many respondents noted in
their feedback to our initial reports.
Data Sources
Consumer Perception Surveys
Family members/caregivers and TAY self-report items analyzed (YSS/YSS-F):
o My child is better at handling daily life.
o My child gets along better with family members.
o My child gets along better with friends and other people.
o My child is doing better in school and/or work.
o My child is better able to cope when things go wrong.
o I am satisfied with our family life right now.
o My child is better able to do things he or she wants to do.
o I know people who will listen and understand me when I need to talk.
o I have people that I am comfortable talking with about my child's problems.
o In a crisis, I would have the support I need from family or friends.
o I have people with whom I can do enjoyable things.
Adult and older adult self-report items analyzed (MHSIP):
o I deal more effectively with daily problems.
o I am better able to control my life.
o I am better able to deal with crisis.
o I am getting along better with my family.
o I do better in social situations.
o I do better in school and/or work.
o I do things that are more meaningful to me.
o I am better able to take care of my needs.
o I am better able to handle things when they go wrong.
o I am better able to do things that I want to do.
o I am happy with the friendships I have.
o I have people with whom I can do enjoyable things.
47
o I feel I belong in my community.
o In a crisis, I would have the support I need from family or friends.
Review of Existing Data
Data source likely to be sustained
Data relevant to population of interest (i.e., convenience or random sample of all mental health consumers)
More than 10% missing or unknown values within key CPS scales
Analytic Potential of Indicator
Analysis among specific service populations possible
County analysis possible for FY 2008-09 (convenience sample), but data is not available at the county level for
FY 2009-10
48
Figure 11.1. Perceptions of improvement in well-being as a result of services
Older Adults 3.82 (n=16)
Adults 3.92 (n=218)
TAY 3.92 (n=182)
Children 4.02 (n=238)
1.00 2.00 3.00 4.00 5.00
Table 11.1. Perceptions of improvement in well-being as a result of services by
race/ethnicity7
Family Member/
TAY Adult Older Adult
Caregiver
White 4.01 (n=115) 3.89 (n=89) 3.96 (n=103) 3.97 (n=8)
Hispanic / Latino 4.15 (n=87) 3.95 (n=40) 3.89 (n=42) 3.50 (n=2)
Asian 3.99 (n=9) 3.92 (n=12) 4.27 (n=20) 4.00 (n=1)
Pacific Islander 3.77 (n=7) 4.37 (n=3) 3.71 (n=7) N/A
Black 3.89 (n=74) 3.95 (n=58) 4.02 (n=60) 3.32 (n=4)
American Indian 4.05 (n=19) 3.75 (n=20) 4.21 (n=23) 4.83 (n=1)
Other 4.14 (n=52) 3.92 (n=32) 3.97 (n=23) 3.75 (n=1)
FY 2008-09 Missing/Unknown Values: Family Member/Caregiver = 6.2% (n = 24), TAY = 14.8% (n = 44), Adult =
14.2% (n = 46), Older Adult = 15.0% (n = 3)
Table 11.2. Perceptions of improvement in well-being as a result of services by gender8
Family Member/
TAY Adult Older Adult
Caregiver
Female 4.07 (n=88) 3.85 (n=67) 3.91 (n=94) 3.91 (n=6)
Male 4.03 (n=141) 3.95 (n=100) 4.01 (n=109) 3.61 (n=8)
FY 2008-09 Missing/Unknown Values: Family Member/Caregiver 3.8% (n = 9), TAY 8.2% (n = 15), Adult 6.9% (n =
15), Older Adult 12.5% (n = 2)
7 Consumer perception data for FY 2009-10 cannot be disaggregated by county.
8 Consumer perception data for FY 2009-10 cannot be disaggregated by county.
49
Priority Indicator 12: Satisfaction with Services
Indicator Summary
This indicator provides insight into consumer and family perceptions of satisfaction with mental health services.
Indicator Calculation
Family members/caregivers and TAY respondents’ ratings (1–Strongly Disagree to 5–Strongly Agree) of two self-
report items (specified under the “Data Sources” section below) were averaged to calculate aggregate ratings of
perceptions of access to mental health services (see Figure 12.1 and Tables 12.1-12.2, below). Aggregate ratings
were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive perceptions. This
calculation method is in line with previous DHCS practices.
Adult and Older Adult respondents’ ratings (1–Strongly Disagree to 5–Strongly Agree) of fourteen self-report
items (specified under the “Data Sources” section below) were averaged to calculate aggregate ratings of
perceptions of access to mental health services (see Figure 12.1 and Tables 12.1-12.2, below). Aggregate ratings
were calculated for each fiscal year. Ratings of 3.5 or greater generally indicate positive perceptions. This
calculation method is in line with previous DHCS practices.
Data Sources
Consumer Perception Surveys
Family members/caregivers and TAY self-report items analyzed (YSS/YSS-F):
o Overall, I am satisfied with the services my child received.
o The people helping my child stuck with us no matter what.
o I felt my child had someone to talk to when he/she was troubled.
o The services my child and/or family received were right for us.
o My family got the help we wanted for my child.
o My family got as much help as we needed for my child.
Adult and older adult self-report items analyzed (MHSIP):
o I like the services that I received here.
o If I had other choices, I would still get services from this agency.
o I would recommend this agency to a friend or family member.
Review of Existing Data
Data source likely to be sustained
Data relevant to population of interest (i.e., convenience or random sample of all mental health consumers).
More than 10% missing or unknown values within key CPS scales
Analytic Potential of Indicator
Analysis among specific service populations possible
County analysis possible for FY 2008-09 (convenience sample), but data is not available at the county level for FY
2009-10
50
Figure 12.1. Satisfaction with services
Older Adults 4.59 (n=28)
Adults 4.20 (n=461)
TAY 4.04 (n=176)
Children 4.36 (n=241)
1.00 2.00 3.00 4.00 5.00
Table 12.1. Satisfaction with services by race/ethnicity9
Family Member/
TAY Adult Older Adult
Caregiver
White 4.38 (n=115) 4.02 (n=89) 4.17 (n=104) 4.04 (n=8)
Hispanic / Latino 4.47 (n=88) 3.96 (n=40) 4.17 (n=41) 3.50 (n=2)
Asian 4.10 (n=9) 3.42 (n=12) 4.42 (n=21) 4.00 (n=1)
Pacific Islander 4.43 (n=7) 3.57 (n=3) 4.14 (n=7) N/A
Black 4.24 (n=75) 3.96 (n=58) 4.20 (n=61) 3.50 (n=4)
American Indian 4.49 (n=20) 3.96 (n=20) 4.48 (n=22) 5.00 (n=1)
Other 4.47 (n=52) 4.07 (n=32) 4.28 (n=23) 3.33 (n=1)
FY 2008-09 Missing/Unknown Values: Family Member/Caregiver = 6.6% (n = 26), TAY = 15.1% (n = 45), Adult =
14.9% (n = 49), Older Adult 15.0% (n = 3)
Table 12.4. Satisfaction with services by gender10
Family Member/
TAY Adult Older Adult
Caregiver
Female 4.35 (n=89) 4.13 (n=67) 4.29 (n=95) 4.06 (n=6)
Male 4.38 (n=141) 3.93 (n=100) 4.11 (n=110) 3.88 (n=8)
FY 2008-09 Missing/Unknown Values: Family Member/Caregiver = 4.6% (n = 11), TAY = 8.7% (n = 16), Adult = 8.1%
(n = 18), Older Adult = 12.5% (n = 2)
9 Consumer perception data for FY 2009-10 cannot be disaggregated by county.
10 Consumer perception data for FY 2009-10 cannot be disaggregated by county.
51
Appendix A – Priority Indicator Matrix
52
Appendix B – Preceding Reports
The evaluation team completed extensive groundwork before arriving at the conclusions contained
in this report. To date, the team has documented evaluation planning in six reports based on
statewide data made available online at UCLA and MHSOAC websites. Stakeholders were invited to
provide feedback on draft reports by responding to questions in an accompanying guidance
document. The invitation was shared online and through mass e-mail. Small stakeholder groups
participated in one of two webinars, or online orientations to the report, that provided an overview
to the report’s purpose and the types of feedback sought.
Report title: Defining Priority Indicators
Report version: Draft for stakeholder review
Here, the evaluation team began to refine the core set of priority indicators proposed by the
California Mental Health Planning Council to assess target outcomes of mental health consumers
and the performance of the mental health system.
Report title: Defining Priority Indicators
Report version: Final, revised with stakeholder input
In the revised report, the evaluation team illustrated how stakeholder feedback was integral to
indicator development. This report incorporated changes driven by stakeholders’ comments about
the comprehensiveness and appropriateness of the indicators.
Report title: Compiling Data to Produce All Priority Indicators
Report version: Draft for stakeholder review
In this report, the evaluation team proposed how priority indicators could be calculated using
existing statewide data. The report also detailed all possible data sources and specific variables or
data fields that might be used to build comprehensive priority indicators.
Report title: Compiling Data to Produce All Priority Indicators
Report version: Final, revised with stakeholder input
The initial report was revised to include information regarding measurement methods and the
adequacy of existing data sources, gathered through a stakeholder feedback process similar to that
used for the final Defining Priority Indicators report.
Report title: Initial Statewide Priority Indicator Report
Report version: Draft for stakeholder review
The evaluation team used select data from fiscal years 2008-09 and 2009-1011 to calculate priority
indicators as outlined in the previous report (Compiling Data to Produce All Priority Indicators).
Report title: Initial Statewide Priority Indicator Report
Report version: Final, revised with stakeholder input
The revised report incorporated several calculation updates guided by the MHSOAC. Updated
calculations largely focused on outcome data collected after enrollment. The report also reflected
input from consumer stakeholders about the use and accessibility of report features (illustrations,
descriptions, etc.) and language.
11 The team sorted through datasets from 2005 through 2011 in search of one or more fiscal years (FY) in
which data cells were largely filled where expected. Data from FY 2008-09 and 2009-10 fit this requirement.
53
Appendix C – Recoding Pre-DIG Race Data to Post-DIG Format
Stakeholder feedback to previous evaluation team reports suggested inconsistency and potential
inaccuracy among Race and Ethnicity data fields may be due in part to changes in the format of
these fields in the CSI and DCR data systems. In 2006, DMH implemented changes to the Race and
Ethnicity fields due to Uniform Data System/Data Infrastructure Grant (DIG) requirements from the
federal government (see DMH Information Notice: 06-02). Although DMH provided training about
these changes, Race and Ethnicity information seems to be reported inconsistently across counties.
Because demographic information in the CSI system is transferred to corresponding fields in the
DCR system, Race and Ethnicity information in both systems was analyzed but interpreted with
caution. To ameliorate potential shortcomings of this change, the evaluation team used pre-DIG
information to fill gaps in missing post-DIG Race and Ethnicity fields for analyses involving
demographic information. The table below details the recoding process.
Before Recode After Recode (if Post-DIG field empty)
Data Post-DIG Data
Pre-DIG Field Definition Definition
Value Field Value
Empty formerly
White 1 Race White or Caucasian 1
Ethnicity / Race
Empty formerly
Hispanic 2 Ethnicity Yes (Hispanic or Latino) Y
Ethnicity / Race
Empty formerly Black or African
Black 3 Race 3
Ethnicity / Race American
Empty formerly American Indian or
American Native 5 Race 5
Ethnicity / Race Alaska Native
Empty formerly
Amerasian A Race Other Asian O
Ethnicity / Race
Empty formerly
Hawaiian Native P Race Native Hawaiian P
Ethnicity / Race
Empty formerly
Multiple X Race Multiracial Multiracial
Ethnicity / Race
Empty formerly Other Asian or Pacific
4 Race Other Asian O
Ethnicity / Race Islander
54