BHSOAC
Crhd Deliverable 2c Final 04 30 14
Read the report at Behavioral Health Services Oversight & Accountability Commission ↗
Evaluating the Impact of the Mental Health Services Act
(MHSA) on Reducing Disparities in Access
Deliverable 2c:
Final Analysis of the Impact of MHSA on Reducing Disparities in
Access
Principal Investigator
Sergio Aguilar-Gaxiola, M.D., Ph.D.
Professor of Clinical Internal Medicine
Director, Center for Reducing Health Disparities
Co-Principal Investigator
Debora A. Paterniti, Ph.D.
Adjunct Professor, Departments of Internal Medicine (General Medicine)
and Sociology, and Center for Healthcare Policy and Research
Funded by the Mental Health Services Oversight and Accountability Commission (MHSOAC)
April 30, 2014
Table of Contents
Executive Summary ...................................................................................................... ii
Introduction ................................................................................................................... 1
The Mental Health Services Act (MHSA) ..................................................................... 3
Achievements of the Mental Health Services Act (MHSA) ................................................. 3
California Reducing Disparities Project (CRDP) ................................................................ 4
Cultural Competence Planning (CCP) ................................................................................. 5
Collaboration Activities to Increase Stakeholder Engagement ......................................... 6
Purpose and Research Questions ...................................................................................... 7
Research Methodology ................................................................................................. 8
Study Design ........................................................................................................................ 8
Qualitative Methods and Community-Based Participatory Research (CBPR) ...................... 8
Establishment and Engagement of a Community Expert Advisory Board (CEAB) ............... 9
Systematic Document Review: Scope of Study and Topics of Interest ...............................10
Interview Structure, Content, and Participants .................................................................11
Key Informant Interviews ....................................................................................................11
Focus Group Interviews .....................................................................................................12
Data Analysis .......................................................................................................................15
Key Informant Interviews ....................................................................................................16
Focus Group Interviews .....................................................................................................16
Findings ....................................................................................................................... 18
Key Findings to Research Question 1: How has the MHSA helped to address and reduce
health disparities for un(der)served groups? ......................................................................18
Key Findings to Research Question 2: What are the most common barriers to the effective
delivery of mental health care for un(der)served groups in California? In what ways has the
MHSA helped to mitigate these barriers, and what problems still persist? ..........................21
Key Findings to Research Question 3: What are the current gaps and persistent issues for
un(der)served groups? .......................................................................................................34
Conclusion ................................................................................................................... 47
Impact of the MHSA ...........................................................................................................47
Challenges or Barriers .......................................................................................................48
Strengths and Limitations ...................................................................................................48
References ................................................................................................................... 51
Appendices
Appendix A: Diagram of Qualitative Methodology ..............................................................55
Appendix B: List of Documents and Sources Reviewed for Qualitative Analysis ................56
Appendix C: Documents Reviewed and Resulting Potential Questions for Interviews ........57
Appendix D: Key Informant Interview Script and Guide ......................................................63
Appendix E: Focus Group Interview Script and Guide ........................................................65
Appendix F: Advisory Board & Public Comments/Feedback and Changes Adopted Matrix 68
Tables
Table 1. Interviewee’s Primary Role With Mental Health Services ......................................12
Table 2. Persistent Barriers Described by Key Informants .................................................22
Table 3. Identified Groups With the Greatest Need of Outreach and Improved Treatment .34
Table 4. Common Barriers to Receiving Appropriate Mental Health Services ....................36
i
Executive Summary
The Mental Health Services Act (MHSA), also referred to in this report as “the act,” mobilized
the means for unparalleled coordination of resources to serve children, youth, adults, older
adults, and families who have mental health needs. It embodies a historic opportunity to
promote progress toward statewide goals of improving access to and enhancing the quality of
mental health care, especially for people with mental health needs who are un(der)served. A
central and longstanding goal of the MHSA has been to identify and reduce mental health
disparities in un(der)served groups. The need for a better understanding of the impact of the
MHSA on mental health disparities in un(der)served communities through the lens of MHSA
stakeholders has become more pressing due to the prevalence of barriers to care and treatment
in California, where the population is diverse and the need for mental health services is high.
The significance of this report is supported by three compelling reasons. First, this report serves
as an evaluation of the impacts of the MHSA on individuals and families, their communities, and
the service delivery system in responding to mental health disparities; it also evaluates the
quality of the outcomes of the public mental health system for un(der)served groups in
California. In this report, “mental health disparities” refers to a difference in treatment, access, or
outcomes in certain population groups that are not justified by differences in that group’s health
status, preferences, or socio-demographic composition in the general population. Second, this
report highlights what stakeholders—consumers and family members, advocates, policy
makers, and service providers and administrators—consider evidence of improvement and gaps
in mental health services and outcomes. Third, the findings from this report are intended to aid
the Mental Health Services Oversight and Accountability Commission (MHSOAC) in its efforts to
measure the effectiveness of MHSA in improving service access and treatment outcomes in
groups experiencing disparities in mental health care.
We used information obtained via a series of key informant interviews with 21 participants in
administration, advocacy, consumer care, research, and policy roles. We also convened three
focus groups that included 16 Latino immigrants, four transition-age youth from the lesbian, gay,
bisexual, transgender, and queer or questioning (LGBTQ) community, and 22 older adults and
caregivers to address three research questions. The research questions, and the key findings
from each, are as follows:
1. How has the MHSA helped to address and reduce health disparities for un(der)served
groups?
Prevention and Early Intervention (PEI) funds have been used to establish a set of
community-identified strategies to reduce mental health disparities among un(der)served
groups.
Promoting Full Service Partnerships (FSPs) as part of a “whatever it takes”
approach to help individuals with serious mental illness has begun to transform mental
health services to a recovery-focused system of care. Although FSPs have attained
productive results, their effectiveness has not been well documented or reported.
ii
Emphasis on application of peer-to-peer strategies that focused on education and
support services has led to development of statewide educational campaigns to reduce
stigma.
More progress needs to be made in (a) community involvement, (b) strategies to
increase workforce diversity, and (c) integration of consumers and family members by
implementing and evaluating new strategies via MHSA-funded innovative programs, and
later adopting those that are shown to be effective. Those approaches will help fulfill the
act’s mandate to improve access and enhance the quality of mental health care of
un(der)served groups.
2. What are the most common barriers to the effective delivery of mental health care for
un(der)served groups in California? In what ways has MHSA helped to mitigate these
barriers, and what problems still persist?
Key informants viewed individual barriers such as language, stigma, geography,
lack of knowledge of services, and cost of available care as significant factors that limit
access and availability to care and treatment.
Key informants assessed organizational barriers such as quality and range of care,
service capacity, and adequacy as persistent gaps in the appropriateness and capacity
of mental health services and providers in serving un(der)served groups.
The views of key informants suggested several ways that MHSA has and/or can do
more to mitigate individual and organizational barriers. First, the implementation of the
MHSA meant an expansion of mental health services to current consumers and to
historically hard-to-reach segments of un(der)served groups. Second, increasing
culturally competent programs in ways that resonate with the targeted un(der)served
group is vital to ensure that services are adequate and appropriate. Third, obtaining
community-informed perspectives to increase culturally competent programs and
community outreach and engagement is a step in the right direction to mitigate barriers
to care. This observation suggests that the MHSA should continue to expand services to
un(der)served groups while, at the same time, enhancing the act’s efficacy via a cultural-
and-community-informed perspective.
Shortcomings in funding allocations that impede the abilities of programs to meet
the needs of un(der)served groups must be overcome in order for MHSA to reach its full
potential in becoming a mental health transformational framework.
3. What are the current gaps and persistent issues for un(der)served groups?
An overarching theme that emerged among most focus group participants was an
evident lack of general knowledge of MHSA-funded services, treatment options, and
locations. Insufficient knowledge and awareness of services is concerning, given the
purpose and intent of the MHSA: to provide services that are not already covered by
federally sponsored programs or insurance programs.
ii i
Compounding an existing knowledge gap, low literacy and limited English
proficiency among un(der)served groups create an initial need for MHSA-funded
programs that educate people on how to navigate the mental health system and use of
other community support systems to locate services and transportation services.
Numerous un(der)served communities regularly encounter similar barriers when
seeking mental health care. For example, immigrant Latinos, LGBTQ transition-age
youth, and older adults reported the cost of care and lack of insurance as persistent
barriers to seeking and accessing care. This finding suggest that individuals from these
groups are not knowledgeable or have not been successful in finding MHSA-funded
services, which are available no cost or low cost to people who lack coverage.
The responses by focus group participants emphasized that the MHSA should
focus on (a) language proficiency, (b) cultural competency, (c) diversity and inclusion,
and (d) the integration of consumers and families when attempting to strengthen
linkages between the mental health system and un(der)served groups.
In conclusion, our findings of this report suggest that MHSA has and continues to have an
impact on reducing disparities among un(der)served groups. The findings indicate that key
informants recognize MHSA’s commitment to invest in resources that build the capacity of
un(der)served communities. For example, informants acknowledged Prevention and Early
Intervention (PEI) and Full Service Partnerships (FSPs) as two major examples that strengthen
the linkages between un(der)served groups and access to care. Furthermore, we have learned
through this study that more must be done to increase culturally competent programs and
community outreach and engagement in order to ensure that services are adequate and
appropriate to targeted un(der)served groups. This study is grounded in community-based
participatory research (CBPR), by obtaining stakeholders’ perspectives to assess the
effectiveness of the MHSA on delivery of services for un(der)served groups. As a result, the
persistent focus of MHSA on mitigating barriers to care constitutes a step in the right direction.
We encourage the continued integration of stakeholder perspectives, so that diversity in
perspectives drives the conversation about community-informed access to care and treatment.
That strategy emerged as a consistent theme throughout this report as a significant means to
help MHSA continue transforming mental health services that mirror the composition of
un(der)served groups in California. In this report, our stakeholders describe their experiences
with and perceptions related to access and appropriateness of mental health services, which
were expanded under the MHSA. We propose that this report, including the incorporation of
public responses to the draft of this report, serves as part of an ongoing dialogue between
administrators of mental health services and the people they serve.
iv
This page intentionally left blank
v
Introduction
“To provide state and local funds to adequately meet the needs of all children and adults who
can be identified and enrolled in programs under this measure. State funds shall be available to
provide services that are not already covered by federally sponsored programs or by individuals’
or families’ insurance programs.”
– Mental Health Services Act: Purpose and Intent
The Mental Health Services Act (MHSA), also referred to in this report as “the act,” mobilized
the means for unparalleled coordination of resources to serve children, youth, adults, older
adults, and families who have mental health needs. It embodies a historic opportunity to
promote progress toward statewide goals of improving access to and enhancing the quality of
mental health care, especially for people with mental health needs who are unserved or
underserved. A central and longstanding goal of the MHSA has been to identify and reduce
mental health disparities in unserved, underserved, and inappropriately-served populations
(hereafter referenced as un(der)served groups). The need for a better understanding of the
impact of the MHSA on mental health disparities in un(der)served communities through the lens
of MHSA stakeholders has become more pressing due to the prevalence of barriers to care and
treatment in California, where the population is diverse and the need for mental health services
is high. A recent study derived from the California Health Interview Survey (CHIS) revealed that
one in five (or 4.9 million) adults in California reported needing support for a mental health
problem (Grant et al., 2011). After estimating the presence of serious psychological distress, the
investigators noted that about 2 million adults in California presented with serious mental health
needs. Of these 2 million Californians, only half received mental health treatment in the past
year, one quarter received some treatment, and the remaining quarter received minimal
treatment.
Research also has established that ethnically and racially diverse groups face multiple barriers
in accessing and utilizing high-quality mental health treatment; such groups include older adults;
transitional age youth (TAY); the lesbian, gay, bisexual, transgender, queer and questioning
(LGBTQ) community; foreign-born individuals; and individuals with limited English proficiency
(Aguilar-Gaxiola et al., 2012; Sorkin, Pham, & Ngo-Metzger, 2009). Difficulty navigating mental
health systems, low numbers of ethnically diverse service providers, lack of culturally and
linguistically competent care, limited transportation, and stigma are among the barriers that
prevent vulnerable groups from accessing services and treatment (Clark et al., 2013). Access to
mental health treatment among un(der)served groups is complex and often associated with
stigma. For example, within California, Asian and African American males, foreign-born Latinos
and Asian Americans, older adults over the age of 65, and young adults 18–24 years of age
report unmet mental health needs due to stigma concerns (Clark et al., 2013).
American Indians and Alaska Natives, biracial Californians, sexual minorities, and single heads
of households presented with high levels of mental health needs. U.S.-born Latinos also were
found to present with twice the risk for mental health needs in comparison to foreign-born
Latinos. Young adults 18–24 years of age, older adults 65 and older, those with low levels of
education and limited English proficiency, foreign-born Latinos and Asians, as well as Asian
1
Americans and African Americans were found to be less likely to receive any form of treatment.
When looking at health coverage, Grant and colleagues (2011) found that individuals without
coverage and individuals with access to public health insurance were strongly associated with
higher mental health needs among all adults interviewed. Paying careful attention to the
underlying issues and needs of un(der)served groups, and avoiding the assumption that access
to health coverage alone will improve their mental health status, are vital in reducing disparities
and simultaneously improving the health of all communities in California.
The significance of this report is supported by three compelling reasons. First, this report serves
as an evaluation of the impacts of the MHSA on individuals and families, their communities, and
the service delivery system in responding to mental health disparities; it also assesses the
quality of the outcomes of the public mental health system for un(der)served groups in
California. In this report, “mental health disparities” refers to a difference in treatment, access, or
outcomes in certain population groups that are not justified by differences in that group’s health
status, preferences, or socio-demographic composition in the general population. Second, this
report highlights what stakeholders—consumers and family members, advocates, county
supervisors, and mental health service providers and administrators—consider evidence of
improvement and gaps in mental health services and outcomes. Third, the findings from this
report are intended to aid the Mental Health Services Oversight and Accountability Commission
(MHSOAC) in its efforts to measure the effectiveness of MHSA in improving service access and
treatment outcomes in groups experiencing disparities in mental health care.
2
The Mental Health Services Act (MHSA)
The Mental Health Services Act (MHSA), approved by California voters in November 2004, has
created a historic opportunity to expand programs to serve children, youth, adults, older adults,
and families who have mental health needs. Inspired by innovative models of mental health
treatment offered in California—such as those enabled and evaluated via Assembly Bill (AB)
2034, a recovery-oriented model targeting the homeless—the MHSA constitutes an unparalleled
mechanism to improve timely access to services for un(der)served groups and to reform the
fragmentation of mental health systems and services (Cashin, Sheffler, Felton, Adams, & Miller,
2008). The MHSA increased funding for county mental health programs, while promoting
progress in the improvement of services for children, transitional-age youth (TAY), older adults,
and families (California Department of Mental Health [DMH], 2011). MHSA obtains its funding
from a 1% tax imposed on California residents whose income exceeds $1 million annually. To
date, MHSA has funded an estimated $7.4 billion for counties during fiscal years 2006–2007
through 2011–2012 (California State Auditor, 2013). Lawmakers hoped that by taxing
Californians who have the highest incomes, the MHSA would obtain the revenue to expand
funds for mental health services, while protecting existing vital state services from being cut
(DMH, 2011).
The MHSA’s funding is divided into five main funding initiatives: (1) Prevention and Early
Intervention (PEI), (2) Workforce Education and Training (WET), (3) Capital Facilities and
Technological Needs (CF/TN), (4) Innovation (INN), and (5) Community Services and Supports
(CSS), which include the Full Service Partnership (FSP). These five areas furnish guidelines for
expenditure of the MHSA monies (i.e., funds distributed to county departments of mental
health). The act also outlines its nine values as a framework to increase access to mental health
care and to ensure a consumer-driven mental health system (MHSA, 2004, as cited in UCLA
Center for Healthier Children, Families, and Communities, 2011, 2013). These nine values are:
1. Client and family involvement and engagement
2. Disparities in access and outcomes
3. Cultural competency
4. Recovery/resiliency and wellness orientation
5. Integrated mental health services
6. Integration with substance abuse services and primary care
7. Community partnerships and systems collaborations
8. Stakeholder involvement throughout the public mental health system
9. Co-occurring disorder services competency
Achievements of the Mental Health Services Act (MHSA)
Since its implementation, the MHSA has achieved numerous accomplishments throughout
California. An evaluation of consumer services and supports found reductions in homelessness,
psychiatric hospitalizations, incarcerations, and physical health emergencies, along with
improvements in mental health functioning in adults and older adults, although no improvements
3
occurred in employment status or evidence of improved educational outcomes (UCLA Center
for Healthier Children, Families, and Communities, 2011). Reiter and colleagues (2012)
determined that the availability of consumer-run and consumer driven-services has increased
throughout the state. The MHSA has offered funds for development of Full Service Partnerships
(FSP), which are intensive case management services throughout the state for individuals who
have serious mental illness. The MHSA led to the development of the California Strategic Plan
on Reducing Mental Health Stigma and Discrimination, a 10-year plan to fight stigma and
discrimination related to mental health problems (Clark et al., 2013). Research also associates
the MHSA with a reduction in involuntary psychiatric holds, as a result of enhanced access to
community-based resources (Bruckner, Yoon, Brown, & Adams, 2010). Other research,
specifically exploring 12 California counties, found that the MHSA prompted an increase in
availability of evidenced-based treatments, peer support services, and culturally competent care
in 141 mental health programs in those counties (Cashin et al., 2008).
Building on past achievements, the California Reducing Disparities Project (CRDP), Cultural
Competence Planning (CCP), and collaboration to increase stakeholder engagement are
among the activities that represent MHSA’s commitment to reduce mental health disparities in
care. Given the importance of these projects to the MHSA, we briefly explore literature that
relates to these statewide projects.
California Reducing Disparities Project (CRDP)
To better understand and respond to the mental health disparities affecting diverse groups
within the state, the Mental Health Services Oversight and Accountability Commission
(MHSOAC) in 2009 authorized funding for the California Reducing Disparities Project (CRDP)
targeting five un(der)served groups: (1) African Americans; (2) Asian/Pacific islanders; (3)
Latino/as; (4) Native Americans; and (5) lesbian, gay, bisexual, transgender, queer and
questioning (LGBTQ). The aim of the CRDP was to identify community-defined solutions and
strategies, including meaningful culturally and linguistically competent programs and services to
meet the unique mental health needs of the five racial, ethnic, and cultural populations identified
for the CRDP. The CRDP is supported by the California MHSA Multicultural Coalition (CMMC).
Formed on March 2011, the CMMC aims to identify challenges in access to care among
underserved communities and to recommend solutions for the development of more culturally
and linguistically appropriate mental health systems. Composed of representatives from various
underserved communities throughout California, the CMMC offers the CRDP valuable insights
from diverse multicultural perspectives, inclusive of consumers and family members, that have
not been previously represented in mental health systems (Racial and Ethnic Mental Health
Disparities Coalition (REMHDCO, 2013).
In 2012, the population-specific reports for the five identified groups were completed and
disseminated. The CRDP is entering its implementation phase and is in the process of putting
into action the identified practices and strategies at local levels. The population-specific reports
from the five Strategic Planning Workgroups (SPWs) representing the African American, Asian
and Pacific Islander, Latino/a, LGBTQ, and Native American communities informed the
development of a comprehensive, statewide strategic plan to reduce mental health disparities.
4
Overall, the population reports developed by the five SPWs found similar barriers related to
accessibility, availability, appropriateness, affordability, and advocacy. The SPW reports also
present a framework or roadmap for each un(der)served group, with community-defined
strategies and solutions to reduce disparities, while at the same time transforming the public
mental health system. Summaries of the five SPW reports can be found in the 2014 California
Reducing Disparities Project Strategic Plan to Reduce Mental Health Disparities.
We anticipate that, when the comprehensive CRDP Strategic Plan is publicly released in the
spring of 2014, the practices and strategies that the CRDP identified will be implemented and
integrated in all MHSA-funded programs. The CRDP strategic plan calls for an evaluation
component to determine the effectiveness of the community-defined evidence in reducing
disparities listed in the SPW reports. While we acknowledge that the release of the
comprehensive CRDP strategic plan is in its initial stage, we endorse its intentions to combine
the community-identified strategies from the five SPW reports, and implement and assess the
effectiveness of these interventions.
Cultural Competence Planning (CCP)
In addition to activating the CRDP, California also is the first state to require each county to
submit Cultural Competence Plans (CCPs) delineating strategies for resolving their own
disparities (California Office of Multicultural Services, 2011).1
The goals of the CCPs were to:
1. Establish standards and requirements to create consistency in the reporting of data.
This drive toward consistency was designed to enable the former California
Department of Mental Health (DMH) to monitor improvements in the creation of more
culturally competent county mental health systems over time.
2. Improve access and the quality of care in mental health services for un(der)served
racially and ethnically diverse Medi-Cal beneficiaries.
The UC Davis Center for Reducing Health Disparities recently reviewed CCPs, the most recent
of which were submitted in 2010, and presented findings in a separate report supported by the
MHSAOAC.2 Additionally, the Mental Health Services Division of the California Department of
Health Care Services is preparing plans for revision and future assessments of the CCPs, so
that counties may continue to formulate and submit their plans to reduce mental health
disparities.
1 As defined in CCP of 1997: A set of congruent practice skills, behaviors, attitudes, and policies that come together
in a system, agency, or among consumer providers and professionals that enables that system, agency, or those
professionals and consumer providers to work effectively in cross-cultural situations (adapted from Cross et al,
1998).
2 Please refer to the Analysis of the Mental Health System Response to Reducing Disparities in MHSA Systems of
Care From Available County Care Obtained Through County Submitted Information, completed by the UC Davis
Center for Reducing Health Disparities.
5
Collaboration Activities to Increase Stakeholder Engagement
In 2012, the MHSOAC partnered with the UCLA Center for Healthier Children, Families, and
Communities to conduct a statewide participatory evaluation. Participating mental health
consumers and family members recommended evaluation of specific MHSA-funded services
with emphasis on consumer perceptions of these services (e.g., ease of access,
appropriateness of services, continuity of care, and recovery orientation). The services that the
UCLA Center evaluated included peer support services, employment support services, and
crisis intervention services. In this evaluation the UCLA research team conducted a mixed-
methods analysis that included the “lived experience” of ethnically diverse individuals,
consumers of mental health services, parents of children who have received mental health
services, and family members. Of a total of 949 individuals who were surveyed, 40 were
interviewed to explore their various experiences and perceptions of services. Findings
suggested services are accessible and individualized for most, but remain elusive for individuals
with disabilities and for homeless people. No differences occurred in employment or housing
status among those who received or did not receive services. Yet, those who did receive
employment and housing services found those services contributed to improvements in their
living situations. Access to services also positively correlated to service experiences as well as
to perceptions of personal recovery and resilience.
An evaluation that the UCLA Center for Healthier Children, Families, and Communities
conducted in 2012 found little evidence suggesting stakeholder involvement in the public mental
health system; the low level of participation impedes the progress of MHSA-funded services for
un(der)served groups. This key finding underscores the idea that the effectiveness of a
progressive public mental health system depends on stakeholder involvement and
representation of un(der)served groups. A 2011 systematic review of existing MHSA
evaluations, studies, and county documents similarly found that stakeholder involvement in
MHSA-funded services and their evaluation was minimal (UCLA Center for Healthier Children,
Families, and Communities, 2011). For example, one study mentioned involvement of
consumers in only one county’s FSP program. Two additional county reports noted consumer
involvement; one county highlighted a PEI program that supported consumer leadership roles,
while the other described consumer involvement in program planning. Seven additional counties
also described consumer involvement in peer support programs, steering committees, advocacy
groups, and as program support staff. Those findings contrast strongly with the requirement for
all counties receiving MHSA funding to develop and implement a stakeholder planning process
to guide their MHSA-funded programs. Recognizing the value of involving stakeholders in
program planning , MHSOAC is supporting an evaluation of county MHSA stakeholder planning
processes. Specifically, the MHSOAC-sponsored evaluation will assess counties’ local planning
processes. This is a step in the right direction to adequately evaluate the degree to which
counties involve stakeholders and to ensure the full participation of stakeholders in MHSA-
funded mental health services. In spite of this step, advocates for communities of color and
underserved communities argue that attention and outreach to people who have sufficient
knowledge of and connection to these communities remain inadequate.
6
A recent audit of the MHSA conducted by the California State Auditor and released in August
2013 also reported the need to improve stakeholder involvement in program planning and
services. The state audit indicated that MHSA requires counties to create plans for addressing
the mental health needs of their communities, to include stakeholders in the planning process,
and to subsequently update plans annually. While the four counties (Los Angeles, Sacramento,
San Bernardino, and Santa Clara) reviewed in the audit complied with stakeholder involvement
requirements through the inclusion of stakeholders in planning processes, the counties did not
consistently document these intentions in their MHSA plans and annual updates, which were
shared with stakeholders. Again, adequate evaluation and engagement approaches to initiate
and strengthen stakeholder involvement can lead directly to a better understanding of the needs
of un(der)served groups. This notion is interwoven with access to quality of care and treatment
that so many un(der)served groups aspire to attain. Disparities among un(der)served groups are
large and continue to impede their path to wellness if the key stakeholders are not at the table.
Purpose and Research Questions
The overall aim of this report was to apply community-based participatory research methods to
conduct an analysis of consumer3 and family member perspectives regarding impact of the
MHSA on disparities. Using a qualitative approach, we examined stakeholders’ perspectives on
the effectiveness of MHSA. Specifically, three questions guided this evaluation and subsequent
report. They are:
1. How has the MHSA helped to address and reduce health disparities for
un(der)served groups?
2. What are the most common barriers to the effective delivery of mental health care for
un(der)served groups in California? In what ways has MHSA helped to mitigate
these barriers, and what problems still persist?
3. What are the current gaps and persistent issues for un(der)served groups?
3 Although the terms “client” and “consumer” often are used interchangeably, this report consistently uses the term
“consumer” in reference to persons who are actively partaking in mental health services.
7
Research Methodology
We engaged multiple stakeholders in this evaluation. Stakeholder involvement included
participation in key informant or focus group interviews, engagement on our advisory panel, and
critiquing a draft version of this report, which was circulated for public comment for a period of
thirty days (see Appendix F for feedback and responses). This section of the report includes a
description of our project design, which explains the methods and instruments used in data
collection, the stakeholders involved in data collection and review, and the process of data
analysis that led to our findings.
Study Design
This section describes our approach to this evaluation, which consisted of four parts: (1)
qualitative and community-based participatory research (CBPR) methods, (2) systematic
document review, (3) establishment and engagement of a Community Expert Advisory Board
(CEAB), and (4) interviews with key informants and stakeholder groups. Each of these methods
is described in detail below.
Qualitative Methods and Community-Based Participatory Research (CBPR)
The purpose of qualitative methods of research is to gain an understanding of the experiences
of those engaged in mental health services. Qualitative methods consisted of interviews with
key stakeholders, including mental health administrators, advocates, and providers associated
with mental health service delivery, as well as individuals at risk of and living with serious mental
illness, their family members, and personal caregivers, who were part of historically
un(der)served groups. Interviews were employed in order to have an open-ended, directed
conversation that could explore stakeholders' “firsthand knowledge and experience” with
California mental health service systems and the impact of the MHSA. Appendix A contains a
diagram of the methodology employed in this report.
CBPR was used with qualitative methods of sampling, data collection, and analysis, in an effort
to understand the perspectives and values of persons and groups that are not adequately
represented by traditional, more quantitative models of research (Denzin & Lincoln, 2000;
Minkler & Wallerstein, 2010). CBPR involved inclusion of typically un(der)served subgroups in
the research processes of data collection, interview participation, analytic feedback and review.
Inclusion of un(der)served subgroups in the research process involved collecting data from
un(der)served groups as well as enabling persons from these groups to participate in crafting
guiding questions (i.e., those questions that helped guide data collection during the interview
process) and in interpreting responses to these questions (part of data analyses). We engaged
various stakeholders at different levels—consideration of interview topics, identification of
interviewees, review of drafted report findings for analytic clarity, and gaps in responses.
Engagement of specific stakeholders is described in detail throughout this report. Our approach
to understanding diverse stakeholder perspectives (including administrative, advocacy, service
provider, consumer, and family perspectives) on the ability of the MHSA to reduce disparities in
8
access, quality of services, and outcomes also involved a content review of current documents
relevant to understanding mental health services, consumer needs, and disparities (Minkler &
Wallerstein, 2010).
Establishment and Engagement of a Community Expert Advisory Board (CEAB)
The Community Expert Advisory Board4 was formed to give community members a means to
offer informed insight into community perspectives and needed levels of engagement. Our
CEAB was composed of four leading mental health services experts and peer advocates; they
included executive directors, advocates, and peer supporters of several un(der)served groups,
including Native Americans, older adults, young adults (TAY), and consumers with lived
experiences and their families. Members of our CEAB advocate for improved treatment and
resource support for mental health needs in Northern and Southern California.
Our CEAB assisted in identifying relevant experts for interviews based on the goal of
representing diverse populations, geographic regions, counties, and services in California. We
also aimed to ensure a diversity of viewpoints (i.e., provider, consumer, administrator, service
agency representative, mental health activist, or family members of a mental health services
consumer). We solicited participants to represent and/or articulate the experiences of
un(der)served groups, including, for example, Native Americans, people living with serious
mental illness, and LGBTQ communities. Specifically, our CEAB assisted in targeting additional
key informants beyond those initially identified by MHSOAC and our project team. After
compiling a broad-ranging list of potential key informants, our CEAB assisted in the selection of
24 key informants to represent diverse viewpoints and experiences with MHSA administration,
programs, and services.
Our research team met with the CEAB four times during an eight-month period, and solicited
comments frequently during these meetings via telephone calls as well as by electronic mail.
Our team sought advice from the CEAB regarding gaps in our list of key informants for
interviews, including their informed understandings of the relationship of the potential key
informant to the MHSA as well as the types of underserved populations, racial and ethnic
communities, and geographic areas the key informants represented. CEAB members provided
a critical eye that helped to inform the selection of a range of key informants who could speak to
the need of diverse persons and communities. Engagement with our CEAB assisted us in
refining our initial list of key informants, as well as our strategy for sampling informants and
specific focus group participants, based on the barriers that impede their access to appropriate
and effective care. Our CEAB generated especially useful suggestions that were pivotal in
identifying, recruiting, and sampling of mental health-care advocates and consumers and in
ensuring the relevance of our plan to consumers and family members. Following data collection
and preliminary analyses, members of our CEAB reviewed and critiqued a draft of our report,
and their suggestions have been incorporated into this report. After producing several iterations
of the key informant list and conducting discussions about the key informants to be interviewed
with our CEAB, we targeted 24 key informants for interview.
4 Names of CEAB members are withheld in order to protect their confidentiality.
9
Systematic Document Review: Scope of Study and Topics of Interest
To inform our exploration of stakeholder viewpoints and experiences, we conducted a
systematic review of relevant literature and documents. We concentrated on developing an
interview guide for stakeholders that would assist us in accomplishing an essential goal:
Describe the recent climate of MHSA with specific attention to varying perspectives that might
reveal insight into further questions that need to be explored. The answers to those questions
would improve understanding of the impact of the MHSA on the reduction of disparities in
access, quality of services, and outcomes.
We compiled an exhaustive list of documents based on specific recommendations by MHSOAC
as well as academic and other documents that met the following criteria:
1. Seminal academic, peer-reviewed articles related to mental health disparities
among diverse racial, ethnic, cultural, and LGBTQ groups;
2. Publications from consumer advocacy groups, such as the National Alliance on
Mental Illness (NAMI) or the California Network of Mental Health Clients (CNMHC);
3. Evaluation reports completed by MHSOAC contractors;
4. Selected County Cultural Competence Plans, which outline each county’s plans to
improve the cultural competence of MHSA-funded mental health programs;
5. Selected MHSA plans and annual updates, which contain current information on
programs funded by MHSA (e.g., California Reducing Disparities Project Populations
Reports).
Appendix B includes the specific list of documents on which our review was based.
A subset of the research team conducted the systematic review, which involved a methodical
and iterative examination of all documents that met the five criteria. As a part of their systematic
evaluation, the reviewers made special note of the following items reported in the documents:
1. Current perceptions of the MHSA;
2. MHSA-related contributions to the California mental health delivery system;
3. Recent questions posed about the mental health delivery system and specific
populations served;
4. Challenges and benefits of service delivery and quality;
5. Groups that are un(der)served in existing service delivery programs; and
6. Expressed recommendations for improved service delivery and administration of
MHSA-funded programs.
Reviewers also identified, where specified, subgroups or topics of interest that require further
exploration and explanation.
10
While we reviewed numerous documents, as described, specific documents highlighted
particular topics and subgroups of interest and repeatedly emphasized important guiding
questions for key informant and focus group interviews, which were specifically relevant to the
goals of contract Deliverable 2. The documents we reviewed, important topics presented
therein, and relevant interview questions are presented in Appendix C.
The documents, related concepts, and guiding questions listed in Appendix C provided an
unobtrusive method for exploring various stakeholder opinions. The benefits of unobtrusive
methods, such as document review, are that they are low in cost and illuminate some of the
assumptions and persistent questions of stakeholders. The questions following from these
documents helped in shaping an initial template for what eventually resulted in a refined list of
guiding questions for interviews with key informants and focus group participants (see
Appendices D and E, respectively, for the guiding questions and prompts for these interviews).
Interview Structure, Content, and Participants
We conducted two types of interviews: Key informant interviews (Gubrium & Holstein, 2001) and
focus group interviews of un(der)served subgroups (Bloor, Frankland, Thomas, & Robson,
2001). All interviewees gave informed consent before participating in the interview. Key
informants were compensated $35 each for their participation in an interview. Focus group
participants were offered light refreshments and compensated $35 each for their participation in
a single focus group interview. In addition, caregivers who took time off work to participate in
group interviews were given a $100 gift card, and those who assisted with transportation were
reimbursed for mileage. All interviews were digitally recorded and transcribed verbatim for
analysis.
Key Informant Interviews
An experienced interviewer conducted key informant interviews of 45–90 minutes over the
telephone. Interviews were guided by an interview script that contained a set of guiding
questions and prompts, which were used to explore a range of viewpoints and experiences.
Appendix D contains the script used for key informant interviews.
Key informants consisted of stakeholders in administrative service and delivery or in the
provision or receipt of care. Those identified as potential candidates for interviews served
central roles within specific communities (e.g., community leaders, religious leaders, advocates,
health educators); direct(ed) community-based agencies; or were identified as consumers,
family members, and/or persons actively engaged in community health treatment, prevention
and promotion at the grassroots level within communities of concern. Through multidisciplinary
team review, consultation with MHSOAC during biweekly meetings, and two rounds of CEAB
input and discussion, we examined our entire list of potential key informants. We then divided
the list into substrata, which defined the potential key informants’ diverse relationships to and
experiences with California mental health services and the MHSA. We then purposively
sampled from among each of the substrata to ensure diverse representation of key informants
in these substrata.
11
During the months of August through October 2013, a total of 24 key informants living in
Northern, Central, and Southern California were identified for interview. Of the 24 persons
selected, 21 agreed to participate in an interview; one potential participant could not be
interviewed during the time frame allotted for data collection, and two did not respond to a
minimum of three attempts to contact by telephone and e-mail.
Although a total of 21 key informant interviews were completed, only 19 of the 21 participants
voluntarily completed demographic questionnaires. The demographic data that the 19
interviewees submitted indicated that participants ranged from 26 to 73 years of age (with a
mean of 52 years of age); participants included 10 women, eight men, and one participant who
identified gender as queer. Among our sample, six identified as LGBTQ and the remaining
sample identified as heterosexual. Five participants identified as of Hispanic ethnicity; 14 as
non-Hispanic White. Self-reported racial identification included eight Whites, three Asian
Americans, two African Americans, one Native Hawaiian, one Native American, and two biracial
participants. Thirteen informants reported having a graduate degree, one held a college degree,
and two attended some college; the rest of the sample reported having a high school degree. All
but one key informant was employed at the time of interview. Thirteen participants reported
earning more than $100,000 annually, two earned between $50,000 and $60,000, and one
reported earning less than $20,000 in the previous year; the remaining three did not report their
income. All the participants reported having health insurance. The reliance on volunteers and
recruiting participants based on our CEAB recommendations may have led to a somewhat
larger non-Hispanic White and highly educated subpopulation. Three of the 19 reported
accessing a mental health provider in the previous year. Table 1 below summarizes key
informants’ responses when asked about their primary role with mental health services in
California.
Table 1. Interviewee’s Primary Role With Mental Health Services
Informant-identified mental health service roles N
Administrative 8
Advocacy/activism 5
Consumer or consumer care provider 3
Research 2
Policy maker 1
Total 19
Focus Group Interviews
Focus group interviews were conducted in locations familiar and convenient to participants and
lasted 70 to 90 minutes. Groups were led by a trained moderator and by a community member
who assisted with group recruitment. Interviewers used an interview script that contained a set
of guiding questions and prompts, which were used to explore a range of viewpoints and
12
experiences. Appendix E contains the script used for focus group interviews. The moderator
was skilled in using the interview script as well as in clarifying unclear concepts or terminology
for diverse participants. Trained moderators have the capacity to respond to the needs of
specific participants and to ensure that participants understand the kinds of questions being
asked. In addition, the inclusion of community members helped to ensure clarity in the
presentation of interview questions. Our team shared the interview guide and analytic plan with
our CEAB, and we revised our questions in response to the feedback we obtained. Our set of
guiding questions was then reviewed and approved by our institutional review board (IRB),
which includes community members local to the institution. Before conducting the focus group
interviews, we piloted the questions in a focus group interview with members of the general
public, and adapted minor changes to question wording based on their suggested feedback.
Focus group participants were purposefully targeted based on several factors, including
discussions and recommendations by MHSOAC, identification during key informant interviews,
critical review and discussion with our (CEAB), and factors that our literature review identified as
significant barriers to effective care. Three populations were targeted for focus group interviews,
based on their experiences of multiple and significant barriers to appropriate mental health
service. Latino Immigrants in the Central Valley (Merced County) were selected for interview
based on barriers related to geographic, language, and immigrant status. LGBTQ transitional-
age youth (Sacramento County) were selected for interview based on barriers related to age
(18–24 years)5, relation to adult parents or guardians, and gender/sexual orientation. Older
adults and caregivers (San Diego County) were chosen for focus group interview based on
potential barriers related to age, familial engagement/ surrogacy, comorbidity, and health-
related conditions. The factors around which focus group participants were selected affect
multiple groups, not simply those selected for interview. Our goal was to learn from these
groups about the multiple and diverse barriers to appropriate and effective mental health
services delivery and to use these data as starting points for exploring the barriers encountered
by other un(der)served groups who face similar obstacles.
Focus groups were conducted in three California counties—Merced, Sacramento, and San
Diego—between mid-October and late November 2013. These counties were selected from
among the counties in California due to their geographic location, size, and diversity of their
constituencies. A total of 42 persons participated in focus group interviews. Focus groups varied
from four to 22 participants.
Latino Immigrant Group (Merced County)
The Latino immigrant focus group was conducted in Spanish, the native language of the groups’
participants, and held in California’s Central Valley. Individuals were recruited through an
English as a Second Language (ESL) program that is subsidized by MHSA Innovation funds.
ESL class leaders and community health workers assisted in recruiting focus group participants
5 We did not attempt to recruit or interview youth under the age of 18 because the consent processes require
different protocols than those covered by our Institutional Review Board for this project.
13
beyond the 10 participants enrolled in the ESL class. Members of the ESL class were
encouraged to bring a family member or friend to participate in the group discussion.
The focus group had 16 participants, two of whom declined to complete a demographic
questionnaire. The 14 participants who completed a questionnaire ranged from 22 to 54 years
of age (with a mean of 34 years of age); they included five women and eight men. All but three
participants reported that they were married. Only two individuals completed 12th grade or
obtained a GED. Five participants reported being unemployed; those who stated that they were
employed reported annual incomes ranging from $10,000 to $50,000. Three participants stated
they earned $10,000–$20,000, five earned $20,000 to $30,000, and four participants reported
earning between $30,000 and $50,000 in the previous year. Only one of the 14 persons who
completed the questionnaire reported that they had health insurance. When asked about their
mental health status, six reported having “excellent” mental health, six reported having “good”
health, one reported “very good” health, and one reported “fair” mental health. Two of the
participants reported use of a mental health provider within the past year.
LGBTQ Transitional Age Youth Group (Sacramento County)
The LGBTQ transitional-age youth focus group was held at a local meeting site for LGBTQ
youth. Recruitment for this group was particularly challenging, despite intensive consultation
with LGBTQ advocates and contact with various LGBTQ service providers. Several agencies
and programs servicing LGBTQ youth were contacted for permission and assistance to recruit
potential participants from active support groups; these groups, however, were hesitant about
granting contact to these youth, considering them particularly vulnerable because of their
sexuality and mental health-related issues. We, therefore, reached participants for this group by
way of Listserv posts and distribution of flyers. These methods of recruitment also preserved the
confidentiality of LGBTQ youth associated with our contact persons and organizations.
The focus group of LGBTQ transitional-age youth consisted of four participants, all of whom
submitted demographic information. Participants ranged from 18 to 26 years of age (with a
mean of 22.5 years of age); all of the participants identified as women. Two identified as
lesbian/gay, one as bisexual, and one as pansexual. Two participants said they were each in an
unmarried committed couple relationship, one was married, and one has never been married.
All of the participants identified as White, with two of the four reporting being Latina. Three
participants reported attending college, and the formal education of the remaining member
ended with completion of the 12th grade. Participants claimed annual incomes ranging from less
than $10,000 to $40,000. Two persons noted that they had health insurance. Two described
their mental health status as “excellent”, one as “good,” and the other as “fair.” One of the
participants reported use of a mental health provider within the past year.
Older Adult and Caregiver Group (San Diego County)
The older adult and caregiver focus group was convened in an urban area of San Diego County.
Older adults and caregivers of older adults were recruited with the assistance of community
health workers, mental health program coordinators, and prevention services managers
associated with San Diego County Behavioral Health Services as well as Adult and Older Adult
14
Behavioral Health Services. We included both older adults and caregivers, because limitations
of multiple comorbid conditions limited some older adults from participating in focus group
discussions. Therefore, caregivers were encouraged to participate, so as to share the
perspectives of these consumers. The focus group was conducted in the facility of a community
organization serving multicultural older adults, supported by MHSA prevention and early
intervention funds. The community health workers, program managers, and coordinators
assisted in recruiting diverse older adults and caregivers from various communities throughout
San Diego County. The group included 22 participants, three of whom reported being caregivers
for family members who were older or experiencing a serious mental illness and were in need of
services.
All of the participants completed a demographic questionnaire. Participants ranged from 52 to
80 years of age (with a mean of 66 years of age); participants included 14 women and eight
men. Nine participants reported being married; two were divorced; six were widowed; two were
separated; and three were never married. Fifteen of the participants identified their ethnicity as
Latinos. Twelve identified their race as White; five as Asian American; two as Mexican; one as
Latino; and two did not list their race. Four participants reported never having formal schooling;
eight had less than a high school education; two completed high school and/or obtained a GED;
five had enrolled in some college courses or earned a four-year college degree; and three
attained a graduate degree. Only eight participants reported their annual income, all claiming
less than $10,000 in the past year (the others declined to state their income). Only one member
of this group reported having an “excellent” mental health status; six reported their status as
“very good”; six participants reported having a status of “good”; five described their status as
“fair”; two described their health as “poor”; and two did not report their mental health status.
Among our older adult participants, 15 said they had health insurance, and four participants
reported seeing a mental health service provider within the past year.
Data Analysis
Our data analysis included “fact checking” by our interviewees and by way of a 30-day public
comment period, which were used to assess the relative accuracy of the data collected and our
interpretation of these data. All persons who participated in an interview (key informant or focus
group) or assisted in recruitment of focus group participants were sent our analysis of the
interview data for review. Approximately 15% of persons who participated in a key informant or
focus group interview or who assisted in participant recruitment provided feedback to our
original report (Deliverable 2b). We have incorporated and responded to interviewee and public
comments made on a draft of this report, which was circulated on February 15, 2014.
We reached theoretical saturation in the themes presented in this report. Theoretical saturation
is the point at which key informants and members of focus groups became redundant in their
responses to guiding interview questions and contributed no new data, and when additional
responses might be predicted on the basis of those already provided (Morse, 2004). In
qualitative methods of analyses, saturation (redundancy in response patterns) is the point at
which a sample is considered significant in size (Morse, 2004). Although we reached saturation
in the general barriers focus group participants described, we suggested that further delving into
15
the barriers experienced by various groups will yield more nuanced understandings of how
multiple factors create or enhance barriers for diverse un(der)represented communities.
Key Informant Interviews
Analysis of key informant interviews involved an iterative and inductive process of transcript
review by multiple members of the multidisciplinary team. In our first set of meetings, we created
an outline of the key themes and ideas that we observed emerging from key informant
responses to guiding interview questions and probes (guiding questions are shown in Appendix
D). Next, we established a framework for examining those themes and ideas, based on the
examples prevalent in key informant responses (Miles & Huberman, 1994). Our team generated
a list of themes that included identification of salient (recurring) factors and associated “stories”
and their contexts, and examined these factors across all 21 key informant interviews (Denzin &
Lincoln, 2000).
To address our specific research questions, team members convened multiple meetings that
were intended to:
1. Outline what key informants described as the greatest successes and
accomplishments of the MHSA, and for which groups;
2. Develop a list of what key informants reported that they perceived to be significant
barriers to the effective delivery of mental health care, and describe specific barriers for
the most un(der)served groups;
3. Examine discussions about the groups that key informants believe have
experienced the greatest disparities in mental health services, prior to and following the
establishment of the MHSA; and
4. Ascertain what informants perceive that the enactment of the MHSA has done in an
attempt to mitigate some of the barriers that exist for the un(der)served groups in
California.
Focus Group Interviews
Analysis of focus group interviews involved an iterative and inductive process of transcript
review by multiple members of the multidisciplinary team (Miles & Huberman, 1994; Morgan &
Krueger, 1997). We used a method similar to that employed for analyzing key informant
interview data (Denzin & Lincoln, 2000; Miles & Huberman, 1994). We created an outline of the
key themes and ideas that we observed emerging from focus group members’ responses to
guiding interview questions and probes (the interview script and guiding questions are shown in
Appendix E). However, to better understand the experiences and needs of un(der)served
groups and their families, we examined the barriers related to the special needs of the focus
group population; gaps in mental health services delivery and consequences for mental health
and related quality of life; and group recommendations for accommodating community needs
(Morgan & Krueger, 1997).
16
All research team members discussed and reviewed this list of themes and contexts obtained
from interview data, and a computer-based coding system was established using a computer
software program for indexing, searching, and comparing themes that emerged from the data.6
A research assistant uploaded data from the transcripts into an online computer program so that
each of the transcripts could be systematically indexed and searched for the persistent themes,
enabling methodical exploration of the context of these themes across key informant interviews
and focus group transcripts.
6 Dedoose. Qualitative Research Data, accessed December 19, 2013, from http://www.dedoose.com/.
17
Findings
This section explores our three research questions:
1. How has the MHSA helped to address and reduce health disparities for
un(der)served groups?
2. What are the most common barriers to the effective delivery of mental health
care for un(der)served groups in California? In what ways has the MHSA helped
to mitigate these barriers, and what problems still persist?
3. What are the current gaps and persistent issues for un(der)served groups?
To examine these questions, we first describe the recurring responses (themes) that emerged
from the interviews, and we excerpted statements relevant to these themes. Following the
presentation of each research question, the recurring themes, and example interview excerpts,
we offer a brief summary of our findings to address each of the three questions highlighted
below.
None of our focus group participants had an awareness of the Mental Health Services Act
(MHSA), its ability to respond to health disparities, or most common barriers to the effective
delivery of mental health care services in California (issues that pertain to questions 1 and 2
above). As a result, we offer information that our key informants offered to address the first two
research questions. Our presentation of findings pertaining to our third research question is
based on key informant viewpoints of consumer experiences and the actual experiences
described by our focus group participants. The fact that none of our focus group participants
had an awareness of the MHSA or its impact suggests that opportunities for increasing
awareness and potential engagement of consumers that have not yet effectively been
employed.
Key Findings to Research Question 1:
How has the MHSA helped to address and reduce
health disparities for un(der)served groups?
The key informants were encouraged about the progress and contributions of the MHSA
programs in reducing disparities in mental health services for the un(der)served groups in
California. In particular, key informants applauded the Prevention and Early Intervention (PEI)
programs’ focus on targeting early onset of mental health needs through innovative and less
stigmatizing approaches among un(der)served groups. In addition to reporting on the value of
PEI programs, the key informants commented on Full Service Partnerships (FSPs) as a critical
component to increasing mental health recovery and reducing disparities. Key informants also
reported being worried about data collection that does not proportionately reflect the diversity of
people who are served by FSPs. We elaborate more on this below.
18
Prevention and Early Intervention (PEI)
PEI programs, which constitute a central component of MHSA, are important to the various
un(der)served groups, including African Americans, Asian Americans, Latinos, LGBTQ, and
Native Americans, as described in the Strategic Planning Workgroups (SPWs)7 reports,
because the programs and activities are defined by and for each community. Despite MHSA’s
success in tailoring services and treatment to the needs of un(der)served groups, the persisting
need of tracking and evaluating program-related outcomes using adequate and systematic
approaches to data collection and reporting demands more attention. Key informants agreed
that PEI could more effectively help to reduce health disparities among un(der)served groups if
consistent data collection standards and requirements were in place to increase rigorous
evaluations that would validate the utility and outcomes of PEI.
Full Service Partnerships (FSPs)
An overarching goal of FSPs is to align the MHSA funding with the recovery and resiliency of
individuals and families, emphasizing a viewpoint consistent with MHSA’s value of recovery
based on resiliency and wellness. Most key informants described FSP programs as effective in
providing “whatever it takes” to serve consumers’ needs and lead them toward recovery.
As a programmatic part of the MHSA’s Community Services and Supports component, FSPs
have helped to reduce disparities by expanding services to diverse ethnically and racially
un(der)served groups. This idea was best expressed in the words of one key informant:
“Some of the first pools of monies with the FSPs were dollars set aside that gave employment opportunities,
housing…. Family members who will call me wanting their loved ones to be engaged from ethnic
communities … and a lot of these communities of color … that wasn’t somethin’ that was open and available
to them, in a lot of counties. By the time each county had so many slots, they were largely White Americans
who got those kinda boutique-style [read: specially tailored to client needs] programs. And then, you know,
by the time the slots were filled up there was not much of those services for the Latino or API communities.
None for non-English speakers, and maybe a few for, you know, bilingual consumers.” (DS400090)
A majority of informants noted that the impact of FSPs is not adequately reported, and
consequently successes are not sufficiently tracked. In particular, the stigma that individuals or
family members experience while attempting to identify mental health issues, or that consumers
and their families might experience as they move within the mental health system, require
acknowledgement and redress. Key informants said that FSPs and PEIs both have the same
problems with data collection and evaluation, which are outlined above. Descriptions of these
problems included lack of systematic data collection and insufficiently rigorous analysis of
service delivery and consumer outcomes. Key informants expressed concern that data
collection activities do not document the diversity of participants who are served by FSPs and,
therefore, suppress understanding of the scope and impact of services provided for
un(der)served groups, most particularly those highly marginalized groups whom these services
may not reach.
7 Please refer to our description of the California Reducing Disparities Project in the Introduction.
19
Other Achievements
Interviewees reported additional achievements in peer support services, efforts to reduce stigma
toward mental health disorders, expansion of outreach activities for hard-to-reach groups, and
community involvement in service development. A type of innovative MHSA-inspired strategy to
create systems of services and support to enhance service to un(der)served groups is reflected
in the following excerpt:
“I think the biggest success was the great expansion of the full service partnership slots…. We had, you
know … maybe a thousand or something before MHSA, you know … like, whatever it takes…. Now we’ve
got about 7,000 slots. So that’s … a huge improvement.… It used to be only for adults; now we have full
service partnership slots for kids, for transition-aged youth, and older adults, as well…. Another [success]
was … that it kind of stabilized us through the fiscal downturn and without it everything would have come
apart. And the third thing is the … prevention and early intervention has gotten us the ability to try new
things, like with faith communities and primary health-care practices that we would never have been able to
do otherwise.” (DS400074)
In summary, an overall increase in the provision of promising programs and services (that our
key informants consider to be some of MHSA’s greatest accomplishments) has increased the
diversity and number of consumers served in mental health treatment. The respondents believe
that acknowledging and acting on the early identification and mental health intervention of
individuals and families may decrease serious mental illness. That is, accurate and early
detection of mental health conditions for un(der)served groups are critically important in
effective delivery of mental health care and treatment.
SUMMARY OF FINDINGS
Key informants said that the great successes of the MHSA include:
Use of Prevention and Early Intervention (PEI) funds has translated to a set of
community-identified strategies to reduce mental health disparities among un(der)served
groups.
Promoting a “whatever it takes” approach by means of Full Service Partnerships
(FSPs) to help individuals with serious mental illness has begun to transform mental health
services to a recovery-focused system of care. Although FSPs have made a contribution, it
has not been well documented or reported.
An emphasis on implementing peer-to-peer strategies that focused on education and
support services led to the development of statewide educational campaigns to reduce
stigma.
Recognizing that in order to fulfill the act’s mandate to improve access and enhance
the quality of mental health care of un(der)served groups, more needs to be done with (a)
community involvement, (b) strategies to increase workforce diversity, and (c) integration of
consumers and family members by implementing and evaluating new strategies via MHSA-
funded innovative programs and later adopting those that are shown to be effective.
20
Key Findings to Research Question 2:
What are the most common barriers to the effective delivery of mental health care
for un(der)served groups in California? In what ways has the MHSA helped to
mitigate these barriers, and what problems still persist?
While the implementation of the MHSA is a historic step toward improvement of a fragmented
mental health system and has made significant strides in California, it is not immune to
formidable obstacles in its attempts to improve mental health services in a large and
increasingly diverse state. Key informants discussed some of the most common barriers for the
effective delivery of mental health care for un(der)served groups in California. Along with
describing these barriers, they discussed persistent problems and targeted solutions that might
be used to resolve them.
Common Barriers to Effective Delivery of Mental Health Care
for Un(der)served Groups
Common and persistent barriers can be classified as barriers related to individuals and families
seeking mental health services and those hampering the systems and organizations that furnish
mental health services. Key informants identified eight impediments: (1) language, (2) cost of
available care, (3) quality and range of care, (4) stigma, (5) geography, (6) service awareness,
(7) knowledge of services, and (8) service capacity and adequacy. Key informants anecdotally
identified numerous barriers that deter un(der)served groups from receiving effective mental
health services (see Table 2).
Key Informants described the implementation and success of the MHSA, related programs, and
services as having the potential to more effectively respond to the needs of diverse consumers
and families. The importance of reaching out to diverse groups, however, validates the need for
a mental health services workforce that is ethnically, culturally, and linguistically concordant with
growing and intended consumer bases. In addition, engagement with more diverse and
un(der)served groups further highlighted the fact that some groups or a subset of members
were not accessing services or were underutilizing services available to them, thus increasing
the gap in services between those who receive adequate mental health resources and those
who do not. To rectify some of these barriers to care for un(der)served groups, key informants
encouraged an increase in community-defined best practice models, such as collaboration with
faith-based and other community-based organizations for outreach, education, and treatment.
21
Table 2. Persistent Barriers Described by Key Informants
Barrier type Interview quotes
Individual barriers
Language “When you look at the ethnic minorities, I think that they are still underserved and
there are several reasons. [N]umber one, we don’t have enough clinicians who
speak the different languages or are culturally competent to serve the minority
populations.” (UDS400088)
Stigma “The biggest barrier is still the stigma as it exists in particular communities. So that’s
why we’re trying to focus on the outreach—to reduce stigma. But as the media
demonizes mental illness, like that guy who said, ‘the guy who kept all those women
captive for years was the face of mental illness,’ I mean, whatever is wrong with him,
it’s not the face of mental illness. You know, he’s just bad.” (Unicode 003)
Geography “Individuals who are … located in the hinterlands [in more rural or agricultural areas]
may not have transportation to get to one of the [closest urban providers]. They’re
basically kind of stuck. They don’t get anything. Now, even accessing the system in
the way it’s set up in [name withheld for confidentiality] County now, to get a clinician
you know, they’re only really sort of open certain days. So if you’re having a mental
health crisis, your choices are to wait to have that crisis on the days that they are
serving this area, or to go to [specific psychiatric service facility—name withheld for
confidentiality] and 51 [psychiatric hospitals] and get 5150’d.” (DS400082)
Knowledge of
services
“Foster families shouldn’t have to hunt around to get services. They ought to be
Public awareness provided with information and training that says, ‘if you experience any of these kinds
of services of behavior, do not wait. Call these numbers immediately.’” (UDS400082)
“I think it may be related to educating the providers better. Because locally, where I
reside, there are people [clients in need of care] in FSPs who don’t know that this
Understanding program is funded by the MHSA because the providers don’t educate them around it.
the mental health … [D]o they tell them that this program is a full-service partnership and what the core
system and how values of the MHSA are? Many providers do not, and I’ve seen that firsthand.”
it works (Unicode 011)
Cost of “There are many places that have sliding scale but still their minimum, like $25.00 …
available care that’s too much. [T]hose same people either don’t qualify for Medi-Cal or … Medi-Cal
mental health services for adults if you don’t have especially—if you don’t have the
classic, you know, schizophrenia, bipolar, whatever, you don’t get anything [you
won’t get services]. Like, if you’ve got couples issues that you really need to work
out, you’re not gonna get services.“ (DS400075)8
8 MHSOAC notes that the MHSA was intended to provide services for no or low cost to individuals who don’t have
insurance and cannot otherwise receive services.
22
Organizational
barriers
Quality and “There’s two levels of care. In fact … those that get MHSA services because they’re
range of care sick enough [will] get everything. And the rest of us that were promised services are
not getting anything.… [T]he sick folks through MHSA and FSPs and severely mental
ill … the safety net is catching them. But it’s just kind of, like, people have felt that
there are gaps in … not so much access, but in the range of programs [and
services].… So that’s where that needs assessment is so important, that everything
starts with the stakeholder process. And if you don’t have racial ethnic minorities or
underserved clients on the stakeholder process, you won’t have the right needs
assessment, and you have these people saying, ‘look, here’s a need. You built this
MHSA web and it’s got this huge hole.’” (DS400071)
Service capacity
and adequacy
“However, when you look at percentages where [African Americans] are receiving
Inappropriate services … it’s jails and prisons and state hospitals, and again I think obviously due
service to discrimination, maybe not direct, I mean, you know, intentional…. I know there’s a
study by Dr. Lonnie Snowden that shows that they [African Americans] are
disproportionately over-diagnosed as schizophrenic, you know, and dangerous, and
again, just like I think probably African American boys are, you know, labeled and
thought to be disruptive … that they get put in detention, or kicked out of school more
often. So, that’s why we think you have to talk about not only unserved, but poorly
served, or inappropriately served.” (Unicode 005)
“The different communities of the Arabic-speaking world need to be reached. They
have not been.… The numbers are not small.… In the Bay Area there are many
Lack of adequate Arabic communities. In Southern California, Los Angeles, and in San Diego, there
service are sizable numbers of Arabic-speaking populations.… I would say they are
underserved. Some [of these] communities certainly have been unserved.” (Unicode
011)
Potential Avenues for Mitigating Barriers to Providing Mental Health Services
The enactment of the MHSA has illuminated persistent problems for un(der)served groups. Key
informants discussed some of these persistent problems, some of which have been brought into
clearer light in the context of mental health programs and services created or made more
accessible for un(der)served groups due to the MHSA. Key informants provided insight into
some of the factors that allow certain barriers to persist, and offered potential sources of and
solutions to these problems. Below we describe three types of obstructions to the effective
identification of needed services and their delivery to un(der)served groups, including difficulties
related to the appropriateness of engagement strategies, prioritization of services and allocation
of funds, and systems of tracking and monitoring.
Challenges Related to Appropriateness of Engagement Strategies
Clear and specific channels for appropriate service delivery. Despite the intent of the
MHSA to offer early treatment intervention and to prevent involuntary treatment, key informants
23
said that consistent mental health care services were performed most commonly in “crisis”
situations or in institutionalized circumstances (e.g., hospital emergency departments and jails)
rather than in community-based settings (such as community mental health clinics).
“In the African American community … you see a lot of the mental health encounters occurring in criminal
justice settings … for children and youth that would be juvenile hall … situations where their custody has
been removed.… Families would experience that the system has failed them.… Because you don’t have the
history, and you have individuals that might have substance abuse issues as well, they go into a jail setting
[and] they’re angry, they’re isolated. They’re misunderstood.… There’s no history in working with them.
They’re often over-medicated, or perhaps misdiagnosed.… It’s a vicious circle of not getting the broad
supports they need.“ (DS400072)
Descriptions of “inappropriate care” were common for informants who discussed institutional
settings as the primary locale for the delivery of mental health-related services. Mental health
providers, consumers, and families of people who have serious mental illness noted that
emergency departments, while not the best alternative for mental health care, are often the only
type of service available to people with serious mental illness or in crisis.
Many informants described the inappropriate use of services and untrained staff for some
un(der)served groups, including those with serious mental illness, older adults with comorbid
conditions, and African Americans. Interviewees’ examples related to the treatment of African
Americans illustrates that although un(der)served groups, like African Americans, do receive
mental health services, the treatment they receive is not entirely appropriate.
“So when you don’t effectively serve this population, they come into our system in our high-end institutional-
based services.… [W]e should have done a better job to keep them from entering these systems.… How can
we help that individual and their family come out and understand … what issues do they need to work on
[and] what their illness is about? How do we build the support? How do we use better strategies? One of the
things that I think needs to happen in the future is looking at some of the strategies we’re developing with
MHSA.…” (DS400072)
Informants perceived that potential consumers encounter difficulties in accessing services when
they do not have insurance, are undocumented, seek support-related services (such as housing
or financial support) rather than counseling services, or do not present with a mental health
diagnosis rendering their condition critical enough for attention.
“In our county, and probably many other counties, the boards of supervisors have taken positions that there
are no services going to be afforded to people who are uninsured or undocumented.… If the board, any
board in any county, took a position that really went counter to a basic mental health [care protocol] … as
professionals, we would object to that collectively.… But nobody says anything. Everybody’s fine with it …
they should be getting services through MHSA.… In fact, you’re the administrator of MHSA funds.… [Y]ou
have people who still are very high need, they’re very acute, but they’re undocumented or uninsured for
whatever reason, and they don’t get anything. You only see them when they come into crisis.” (DS400092)
This interviewee’s description of perceived denial or constraints on service provision for
undocumented persons by county administrators highlights the ambiguity and lack of direction
to counties and providers on how to proceed with services for undocumented persons. MHSA
must resolve this uncertainty and establish clear guidelines on serving undocumented
populations.
24
Planning for and development of an adequate and well-trained workforce. Key Informants
and stakeholders all requested more proactive strategies for responding to the needs of people
with serious mental illness, including planning for and development of a more adequate and
well-trained workforce in settings where people “in crisis” or living with serious mental illness
could best be served. Specific barriers to developing an adequate workforce to serve mental
health needs in California is highlighted by this stakeholder comment:
“The MHSA is assistive in many areas of administration [but] the problem persists in the hiring of individuals
with lived experience. Although there are some positions available, there is no end in sight for those of us
working on the front lines with limited time to go to training that is time consuming and often becomes
obsolete. The reason for the funding is to assist the community with more alternatives than incarceration and
other forms of restraint.” (Public commentary, February 19, 2014)
In addition, some stakeholders have noted that parameters for the use of MHSA funding and the
availability of treatment for people living with serious mental illness highlight the lack of
community-based psychiatric services, including adequate staffing or appropriate training for
primary care and other providers who eventually care for patients who suffer with mental illness.
Family members and mental health care providers noted the absence of training for personnel
who are not licensed physicians or psychiatrists.
Development of culturally competent programs. The processes of planning for and
developing a well-trained workforce in a state that has begun to engage more diverse groups
highlight a need for developing culturally competent programs and increasing delivery by
culturally concordant providers. Informants were clear that defining and developing “culturally
competent programs” takes time and involves a level of sensitivity to community outreach and
engagement that existing systems are not adequately prepared to deliver effectively. One
interviewee explained:
“You wanna create that sense of safety and … pay attention to the gender issues. You know, have a
continual understanding. Continue to understand the cultures that you’re dealing with and let ways of the
people and the values guide you in creating the programs. You don’t wanna go into a community that you do
not know and say, ‘Oh, we have parenting classes for you.’ Can you imagine anything more disrespectful?
Because that message is that you are going in to someone you don’t know and saying: ‘Here. We’ll show you
how to parent. You don’t know how to parent.’ Well, that’s the last thing you wanna ever do. You know? But
when you go in and say, ‘We have an opportunity to help the children succeed,’ and present it that way, to
succeed not only academically but also in the community, then every parent is interested in that. So there are
sensitive approaches that need to be crafted and implemented to do that.” (Unicode 011)
Key informants regard culturally competent programs as those that address the needs of the
community, including appropriate framing of the program and delivery of its contents in ways
that resonate with the targeted population, with respect and humility. Another informant spoke to
the importance of culture and cultural sensitivity in informing valuable interventions for diverse
ethnic communities:
“I think you have to learn more about the different communities. There’s been a lot of press about the
gardening project in the central part of the state that reaches the Hmong communities. That’s their way [to
achieve] wellness or having therapy without it being called therapy. So things like that I think are responsive
and sensitive to the cultures, but not a whole lot is being done about that. The Native Americans and learning
about their healing circles and naming ceremonies. For me that’s important because when they talk about
their historical trauma over the generations, what comes to mind are the historical traumas in my own culture
25
of the Palestinian occupation and how people are affected that way.… You connect by learning from other
cultures, and that information sharing is important whether we hear it at a cultural competence conference or
in a group setting, or whatever venue it might be.” (Unicode 11)
As this informant implies, programs grounded in the culture and history of the intended service
communities are more likely to appropriately and effectively serve those communities. The
importance of a culturally competent workforce cannot be overstated, and should be informed
with sensitivity to the fact that individuals in some ethnic communities may choose to seek care
outside of their communities due to the potential for stigmatization.
Community outreach and engagement. Key Informants identified community outreach and
engagement as a vital component for enhancing the efficacy of the MHSA. Strides that must be
made in order to enhance outreach and engagement include obtaining community-informed
perspectives related to assessment of health-care needs, increasing awareness of available
services and avenues for accessing services, and targeting hard-to-reach communities.
With regard to obtaining community-informed perspectives, key informants agreed that diverse
stakeholder engagement is often neglected. In particular, existing strategies for incorporating
the voices of diverse ethnic communities were described as less than ideal methods for
garnering input from those who are underserved. One informant typifies the scenarios described
by interviewees who discussed this barrier.
“These stakeholder meetings, or these big public forums, are not the most effective way to get the voices or
the opinions of underserved communities. First of all, it’s just not a forum where they’re comfortable.… [F]or
example, I’ll just use the Asian community. It’s not culturally valued to get up at a meeting and say, ‘well I
think you should give me this. You should give my community this because we deserve this.…’ People that
are newly immigrated … are outnumbered and then they are out-yelled.” (Unicode 005)
Another interviewee echoed this concern, underscoring the need to involve more consumer and
minority persons in decision-making situations to increase diversity and community
representation.
“The programs are driven by whatever the stakeholders in the community decide. And gauging the minority
folks who come to the stakeholder meeting to make decision[s] is also another difficult thing. Right? It’s very
hard to involve people who are ill and [part of a] minority group to come to the meeting to make decision[s] in
how the system could create program[s] to serve them, because they don’t have a voice. And so you ended
up having a lot more non-minorities show up to your stakeholder meetings and, by virtue of that, they
become the ones who decide which programs should be funded.” (DS400088)
Key informants presented examples of barriers that must be overcome in order to increase
awareness of and access to services for un(der)served groups. The following quote illustrates
how specific strategies for community outreach and engagement could begin to erode this
barrier:
“I think that educational workshops … would be important. Meeting in the people’s community, instead of
expecting them to go to [a] county office or town hall meeting. Maybe going to churches, or temples, or
community centers [would be a more effective approach].” (Unicode 019)
Finding adequate outreach and engagement mechanisms for marginalized and un(der)served
groups could be a first step in involving un(der)served communities in critical decision-making
26
opportunities and in the development of culturally and linguistically sensitive avenues for
treatment.
While interviewees noted that potentially effective mechanisms for community outreach and
engagement were diverse, they described traditional mechanisms for message delivery as
woefully inadequate for reaching most of the audiences and populations intended to be served
by the enactment of the MHSA—especially marginalized groups, including older adults with co-
morbid conditions that include physical disabilities, those living with serious mental illness and
their families, and the LGBTQ community. One interviewee described the following scenario as
part of a typical outreach strategy for informing and engaging underserved communities:
“In one meeting they were saying, oh, we have access to PBS. And then we were accessing 90 percent of
the population, and we are reaching however many millions. I say, excuse me, in our community we don’t
watch PBS. They would be watching the Latino channel, watching the Chinese channel, but not watching
PBS.… And then even how you do the promotion is also important. If you only do it in PBS, and people are
not going to watch it or [in the] Sacramento Bee [newspaper], people may not read it. They read their own
ethnic newspaper, listen to their ethnic radio station or TV station. And that is a fact in California.” (Unicode
019)
Some informants claimed that the most effective approach for outreach and engagement would
be to target specific communities based on the disparities that they face.
“For example, if it was shown that the Latino community had a disproportionately low penetration rate, or that
the dollars spent on people from the Latino community were low compared to [other groups], then design a
program targeted towards a particular racial or ethnic community. Right now I think, for political reasons, that
is not done as often as it should be … because the policymakers and the community advocates at the table
are not from our [underserved] communities.” (Unicode 005)
Stakeholders offering public comment about the draft of this report suggested that specific
strategies related to messaging, along with communication by health services providers, could
yield potentially effective methods for increasing awareness of MHSA-funded services and
engaging un(der)served communities.
Some stakeholders suggested that providers and organizations who receive MHSA
funding should be responsible for informing consumers about the source of funds for that care
and the values that support its funding. Lack of awareness related to resources for mental
health care can disempower communities. Greater awareness of available services and their
funding sources not only provides community members with information about the availability of
resources in their local community but may also allow them avenues for input in the type and
delivery of services. In response to findings that focus group participants were not aware of
MHSA principles and values, one stakeholder commented:
“If focus group participants had no awareness about MHSA principles and values, a
provider receiving MHSA funding bears responsibility to inform and educate recipients about core values of
wellness, resiliency, recovery and the power of hope to pave [a] pathway to quality and dignity in life. This
can do a lot for a person's engagement, success and motivation.” (Public commentary, March 6, 2014)
27
Prioritization of Services and Allocation of Funds
Nearly all key informants described the need for consistent financial support for programs to
reach their targeted populations. Funding availability, allocation, and strategies used to acquire
funding are factors that influence the success of programs targeting un(der)served groups.
Funding availability. Interviewees who identified themselves, or their role with the California
mental health services system, as related to an underserved community acknowledged that the
MHSA was one of the first programs to make funds available to meet the mental health needs of
these communities. Some interviewees elaborated that the MHSA was the first initiative to fund
grants to programs serving underserved communities; the Mien and LGBTQ communities were
named specifically.
“For the first time we’ve gotten this small grant. This is the first time that the [Mien community] is receiving
some services that’s, you know, related to mental health.… Before this we’ve just, you know, raised funds …
we do events, and we ask for contributions from people that know the community, and we don’t have rich
friends, unfortunately. Just small money here and there, and a couple of small grants. Ten thousand. Twenty
thousand. Fifty thousand. Just the small amount of money, and we’re a small shop, so, we are able to just
kind of keep things afloat.” (DS400085)
Many interviewees noted that the MHSA made funding available for the California Reducing
Disparities Project, which aims to identify the state’s most un(der)served groups and to develop
community-defined strategies for the delivery of culturally competent care in their communities.
In this regard, one informant described the specific use of MHSA funding for a stigma reduction
program targeting the African American community and people in need of housing.
“We received a large grant for doing stigma-reduction work, which is a large portion of our funding, which
comes through MHSA funding.… So, this is all of our stigma reduction work.… [It] is making a difference in
the housing sector for reducing stigma, and in the African-American community.…” (DS400083)
Frustrations related to the availability and stability of funding remain, however, in spite of these
tangible activities, as one informant summarized:
“The problem is, like, I mean, we can identify gaps and we can identify best practices through the reports, but
there isn’t any plan for money for implementation, you know?” (DS400078)
Concerns with allocation of available funds. Some key informants expressed concerns that
current MHSA funds are not reaching all un(der)served groups in need of service. Linked to
these concerns was the perception that larger, more established programs were allocated funds
more frequently than community-based organizations with stronger connections and intentions
to assist un(der)served groups.
“All these different organizations [receiving funds for services] … are predominantly White, predominantly
middle class, predominantly, you know, [with] a certain perspective. Not that they’re bad agencies,
necessarily. But they are the reason why we [LGBTQ community] continue to remain unserved and
underserved.… What tends to happen is that maybe, like, there’s one or two big agencies that always get the
funding. And then the rest of the state gets nothing.” (DS400075)
One interviewee perceived that some ethnic groups (e.g., Arab Americans) have been
overshadowed by the state’s more populous ethnic groups (i.e., Latinos and Asian Americans)
that have been targeted for services. The same interviewee proposed that problems with data
28
management (i.e., aggregation of the data or not collecting data with regard to the important
parameters that create disparities) can make un(der)served groups invisible.
“Funds seem to be funneled to the communities that have been served, such as the African-American
community, services to the Latino community, some services to the Chinese, the Filipino. And it seems to
revert back to the same groups time and time again. I don’t think that there’s a lot of attention paid to those
that need to be reached, such as the Arabic communities, and other communities like the Burmese and
Japanese. What are the needs? But in order to find out, you gotta build relationships with them. You can’t
just rely on the data because a lot of the groups that I bring up are lost in the data. Arabic people are under
the White category. How are you gonna find out about them? You gotta disaggregate not only the ethnicity
but also the heritage. You know, Arabic ethnicity is fine, but it doesn’t go far enough. You need to find out
about the Palestinian community in a county such as [county name withheld for confidentiality]. You need to
find out about the Lebanese community in [county name withheld for confidentiality], and various other
counties throughout the state. That’s heritage that’s key to really reaching the communities and finding out
more about them.” (DS400092)
A number of key informants perceived many contractors who were hired by counties to
complete MHSA-funded projects as not fully trained to conduct culturally appropriate outreach
and engagement with vulnerable populations.
“Is the same old, same old, people being given funds to go do work, and still the underserved populations are
not really being properly addressed. So for example, in the stigma discrimination, Stigma and Discrimination
Reduction Program that CalMHSA has been funding with MHSA dollars, the five authors of the [population]
reports … met with some of their contractors, and we all came away with this feeling that these contractors
don’t have a clue how to deal with our populations. So they’re getting all this money [but] they don’t know
how to work.… They’re saying they’re working with our populations, but, when we ask them specific
questions, the answers we got back—at least I certainly got back—a very broad brush, ‘sure, we’re reaching
out to the LGBTQ community.’ What does that mean?” (DS400075)
Strategies for acquiring and maintaining funding. Some informants referenced difficulties in
acquiring funding or expanding the funding currently available for services that might be
appropriately serving un(der)served groups. For example, one participant described a program
funded by Innovation funds that was found effective in reaching the underserved, but that could
not remain active due to lack of continued funds.
“For example, we have a program right now that is $1.4 million that will end June 2014. And it’s a hospital-
based program with peers and family members to transition clients to outpatient treatment in primary care
clinics as well for treatment. And it works. The evidence has demonstrated that it’s fantastic. It works in the
hospital. We’ve seen all the reductions in hospitalization, emergency psychiatric response teams, emergency
departments, et cetera. We need to replicate this in the system. Which funding is available to community
services and support for the MHSA funding? Or if I don’t have any more money, redesign existing programs
that do not have such outcomes.… There’s no [apparent] structure with the MHSA9 right now to sustain the
programs that they found to be effective under the innovation.” (DS400091)
Another participant indicated that MHSA is funding several promising pilot programs, but that no
existing resources are available to evaluate their effectiveness. Most concerning to interviewees
was the understanding that some funds have been used to maintain and sustain previously
existing programs, rather than for the development and provision of additional services.
9 The MHSOAC states that once a county demonstrates the efficacy of a program or innovative project, the program
may receive funding through CSS or PEI. Below we describe inconsistencies in monitoring practices that might
restrict counties from adequately demonstrating efficacy.
29
Interviewees often perceived this strategy as a misuse of funding, attributable to the current
economic strains affecting the state, and to substantial budget cuts that have impeded mental
health programs.
Several interviewees expressed concern about the use of PEI funding to sustain pre-existing
programs rather than the intended new PEI services. Key informants spoke about the funding
difficulties and using new funds to replace funds for existing programs, which they believe does
not reflect the intent of the MHSA to expand the number of services to the underserved. For
example, several interviewees notably expressed awareness of service providers who had
vigorously applied for prevention and early intervention funds. These interviewees added that as
part of the strategy to acquire these funds, existing programs were renamed or redesigned so
that funding might be made available through this mechanism.
“Well … this is hard to say because, you know, what other state has something like the MHSA? But I would
want to say that it almost feels like everyone is doing a pilot work. You know? Pilot study. And that the
programs need more resources—need more funds so that these programs can expand and so that the
oversight can also expand.” (DS400084)
Interviewees were skeptical, however, that the new programs that were developed sought new
consumers or truly offered the type of “new” services that PEI funds required.
“Counties were not supposed to use MHSA funds to support an existing service. But, yet somehow it was
done in a way to use this pool of money to cover other services—to cover the shortfall in the Medi-Cal or the
mental health fund. And many of the contractors for different counties are dealing with the same group of
population, so how can they change all of the sudden from a treatment population to an early intervention
population?” (DS400076)
Stakeholders offering public comment have noted that some county programs have seen an
increase in consumers served by the mental health system and the number of volunteers
trained to assist older adults with comorbid conditions. The successes of some of these new
programs under PEI, however, have been overshadowed by inconsistent efforts at tracking the
success of the programs.
Systems of tracking and monitoring. Key Informants described efforts at tracking and
monitoring MHSA-related programs as lacking in focus, inconsistent, and diffuse. They
attributed most difficulties with tracking and monitoring to county-specific differences in
interpreting and executing the intent of the MHSA.
“I don’t think we have much of an oversight on all of this MHSA. And I don’t have to tell you that. I mean, that
report came out last month talking about that. The oversight piece is really poor, has really been poorly
carried out. And it probably has to do with, you know, most county [governments] and the state are trying to
rush the money in the street for programs, and really don’t have time to really take a careful look at ‘are they
effective? How have they been done?’ We just take it at street value, but we really don’t do a really good
assessment.” (DS400088)
Key informants noted that problems with assessment have been inevitable due to the lack of
best practice models and inconsistency in the types of data collected and in formats for ensuring
reliability.
“Right now the best practice, for example, is to ask someone ‘what is your current gender identity,’ and
usually there’s a bunch of check boxes. What should be on those check boxes is also a whole other issue.
30
And then you ask ‘what sex were you assigned at birth?’ But apparently someone just told me while we were
doing our report that there was new research coming out showing that youth don’t respond well to that two-
part question for some reason. So, in one school assessment thing that they do, the youth risk something or
other, they are asking sexual orientation but they still aren’t asking gender identity because they haven’t
figured out how to do it for youths.… We need to figure out how to ask the question to get the proper data,
because bad data will only come out as bad conclusions then.” (Unicode 006)
Variation by county in the type of data collected and the practices used to collect those data
have created difficulty in assessing the impact of the MHSA for un(der)served groups and in
developing standardized practices for program management, and contract and funding
monitoring. Furthermore, inconsistent practice patterns disrupt demonstration of program
efficacy (and potential eligibility for funding that rewards innovation).
Program management. Multiple informants identified problems with program management as
barriers to understanding effective mental health services and service delivery for un(der)served
groups. Difficulty with program management created some unintended negative consequences
for interventions that some informants considered excellent interventions.
“So this wonderful program at [name of program withheld for confidentiality] … got defunded by the county.
Because here’s what the county did: The county took MHSA money and started a new program. Now Mental
Health Services Act was not there to supplant current programming. It was supposed to enrich [programs].
That’s not what happened. Across the state what happened was people found ways to circumvent that by
closing down existing programs or rebranding some as something else.” (Unicode 006)
Those programs that continued to exist following the enactment of the MHSA showed increases
in enrollment of MHSA-targeted populations, yet these increases have been evidenced mostly
in less expensive programs, or those that key informants identified as “providing fewer
resources” or lesser “quality services” unlike some of the “full-service, Cadillac programs.”
“When you look at the total numbers, did we increase access for people of color? It’d probably say yes. But
when you start breaking down those numbers, and you start looking at your most expensive programs, like
your ACT programs … I can assure that what we’re gonna find is that, in our outreach programs, in our crisis
programs, and those system-development programs within MHSA, we [the underserved] may be
represented, so when they do the total beta numbers, they may look very good. But when you start breaking
them out by financial investment and you start looking at … resources, how do communities of color fare? …
Our most expensive, highest, and best programming is not being made available to the communities that
were supposed to be prioritized. So, my conclusion is that you’ve created these MHSA programs, these
Cadillac programs that are really wonderful, 24/7 everything, and you only created them for the same people
who are already in your systems.” (DS400092)
The following excerpt summarizes the perspectives of informants who expressed frustration with
the disconnection between the MHSA’s intentions and the consequences of its enactment for
un(der)served populations.
“Really look at the system of all the programs that have been implemented since 2005 and give it an honest
criticism, or whatever, an honest look at it and say: ‘Hey, have all these program met the intent of the
MHSA?’ ... MHSA has improved lives for people who are or who have been in treatment. MHSA has not
improved life for those that are underserved because they are still underserved.” (DS400088)
Contract monitoring. Some interviewees perceived the oversight of contracts at the county
level as a main component of monitoring mental health service provision under the MHSA. A
31
few interviewees described difficulties with county-level monitoring practices, including ensuring
contactor adherence to project-approved goals, as described in this excerpt:
“So, we’ll do contract monitoring and we’ll do data reports and … we actually have program monitors who are
responsible for tracking who is being served and making sure [that service delivery is] in compliance with
what we put in their contracts. So if you get an actual caseload profile that looks different from what we put
into their contracts, then we have to work with them about remedial action to correct that or, if they’re saying
that they’ll have a certain number of bilingual staffing and they don’t have it, so they can’t fill the positions,
then we have to sort of look at what our options are in terms of getting them on the right track.” (Unicode
008)
Some concerns about differences in monitoring practices revealed suspicions about hidden
agendas or, more broadly, whether unevenness in county-level monitoring practices can ensure
that the MHSA delivers what it was intended to deliver: service to the un(der)served
communities. Added to concerns regarding consistent and appropriate monitoring, key
informants stated that when discussions of monitoring do occur, accessibility is restricted and
venues typically are not open to the public.
“They have an MHSA coordinators meeting, for example, and every county has an MHSA coordinator. …
[T]he coordinators have monthly conference calls and sometimes, I believe, they meet in person.… Members
of the public are not allowed at those calls or meetings unless expressly invited.” (Unicode 005)
Monitoring funding use. When informants were asked about how funds were monitored, and
whether they knew of evidence that the funding allocated to un(der)served communities actually
reached those communities, most indicated that they “did not know if or how funds are
monitored.” Those who discuss monitoring in any detail claimed that funding monitoring appears
to be county-specific and dependent on the priorities and practices of individual counties.
32
SUMMARY OF FINDINGS
Reference to the common barriers that impede achievement of effective delivery of mental
health care for un(der)served groups emerged as a consistent theme.
Key informants viewed individual barriers such as language, stigma, geography,
lack of knowledge of services, and cost of available care as factors that are
significantly responsible for limiting access and availability to care and treatment.
Key informants regard organizational barriers such as quality and range of care,
service capacity, and adequacy, as persistent gaps in the appropriateness and
capacity of mental health services and providers in serving un(der)served groups.
The views of key informants suggested several ways that MHSA has and/or can
do more to mitigate individual and organizational barriers. First, the implementation
of the MHSA meant an expansion of mental health services to current consumers
and to historically hard-to-reach segments of un(der)served groups. Second,
increasing culturally competent programs in ways that resonate with the targeted
un(der)served group is vital to ensure that services are adequate and appropriate.
Third, obtaining community-informed perspectives to increase culturally competent
programs and community outreach and engagement is a step in the right direction to
mitigate barriers to care. This suggests that the MHSA should continue to expand
services to un(der)served groups while at the same time, enhancing the act’s
efficacy via a cultural-and-community-informed perspective.
For MHSA to reach its full potential in becoming a mental health
transformational framework that meets the needs of un(der)served groups, existing
inequities and other shortcomings in funding allocations must be rectified.
33
Key Findings to Research Question 3:
What are the current gaps and persistent issues for un(der)served groups?
In response to the third research question, we asked the key informants to identify
un(der)served groups with the greatest need, followed by asking focus groups to discuss issues
affecting un(der)served groups.
Key Informant Identification of Underserved Groups
When asked which groups were perceived to be at greater risk of exclusion from mental health
care services, key informants identified the un(der)served groups in Table 3 as the groups with
the greatest need of outreach and improved treatment delivery.
Table 3. Identified Groups With the Greatest Need of Outreach and Improved Treatment
Number of key informants
Recommended groups
identifying group
African Americans 8
Latinos 6
Asian Americans 5
Transitional-age youth 5
Immigrants 5
LGBTQ persons 4
Older adults 4
Individuals with limited English proficiency 3
Middle Eastern and Arabic persons 3
Native Americans 2
Person living with a serious mental illness 2
Family member of a person living with a serious mental illness 1
Single parents 1
HIV-positive consumers 1
Homeless persons 1
Incarcerated persons 1
Northern Africans 1
Disabled persons 1
Poor persons 1
Russian or Slavic persons 1
Rural or geographically isolated regions 1
Risk of suicide or suicidal ideation 1
34
Importantly, the selection of un(der)served groups that we investigated in our report was based
on the un(der)served groups’ experiences of multiple and persistent barriers to care. Our report
did not investigate all groups in Table 3, which we acknowledge as a limitation and emphasize
that with adequate time and funding, more groups in Table 3 can be examined.
Our goal in conducting focus groups was to understand the ways in which specific barriers
prevent access to diverse un(der)served groups. To accomplish this, we focused the viewpoints
of key informants on the major barriers to service for un(der)served groups. We then examined
these barriers, focusing on those that were described as affecting the greatest number of
persons, and cut across diverse groups. The obstacles we found to be most significant, based
on our interviews with key informants and review of existing documents, were barriers due to
the degree of perceived stigma in accessing mental health services, immigrant status, linguistic
and cultural barriers, age-related difficulty in accessing appropriate care or shortcomings in
mechanisms for engaging appropriate care, and geographic availability of resources for mental
health care (see Table 3). Next, we identified three specific un(der)served groups, whose
experiences reflected a combination of these barriers.
Gaps and Persistent Barriers to Receiving Appropriate Mental Health Services for
Un(der)served Groups
Despite their differences, a series of common themes emerged from interviews in the three
focus groups regarding experiences with California mental health services. Very few participants
stated that they knew where and how to access necessary mental health service. The lack of
general knowledge of MHSA-funded services, treatment options, and location was evident
among most focus group participants. Their unfamiliarity with those services is concerning,
given that the purpose and intent of the MHSA is to provide services that are not already
covered by federally sponsored programs or insurance programs. Without knowledge of the
MHSA, participants were also unaware of low-cost services that the act supports. Knowledge of
MHSA funding available to serve low-income groups can empower potential consumers to seek
these services by reducing their reluctance that is based on cost or lack of insurance.
Common barriers to receiving appropriate mental health services and treatment included the
cost of care, the location of care, delayed care-seeking, family relationships, comorbid
conditions, lack of concordant care, provider or program inaccessibility, lack of awareness of
services, and insufficient outreach. Table 4 below summarizes some of the common barriers
that impede delivery of appropriate mental health services for the diverse groups with whom we
consulted with via the focus groups. Although we discuss each of these barriers to adequate
mental health care separately, obstacles for un(der)served groups are overlapping and often
intertwined with other social difficulties. In addition, the same barriers may impede members of
diverse un(der)served groups in differing ways.
35
Table 4. Common Barriers to Receiving Appropriate Mental Health Services
Latino immigrant group LGBTQ and youth group Older adult group
Recurring themes (Central Valley) (Sacramento) (San Diego)
N=16 N=4 N=22
Costs of care or lack of No insurance coverage No money for treatment Lack of insurance coverage
insurance
Deterred from seeking services Low-income (deciding between
due to cost cost of care and home expenses)
Location of services or Transportation to appointments Lack of transportation
transportation was problematic in rural areas of
the Central Valley
Needed service is provided only
in larger hospital (not local clinic)
Delayed care-seeking Postponing treatment because Postponing treatment because Could not access care prior to age
they did not receive high-quality it was not concordant with 65 (did not meet criteria for Medi-
treatment needs Cal)
Family relationships
Issues of privacy and Parents’ insurance
confidentiality related to
family Providers violate confidentiality
and involve parents (or fear of
this happening)
Responsibilities related to Not knowing how to help family Affected by the problems of
family members members in need of treatment (younger generation) family
members with mental health needs
Comorbid conditions Physical pain
Lack of concordant Services were more available for Care inappropriate to
care women transgendered persons
Adequacy of services
Cultural competence of No access to a provider for LGBTQI-focused care is part of Vicarious trauma
services and providers proper screenings to avoid crises suicide prevention
Language and literacy- Do not receive interpretive Not receiving care from well- Difficulty completing forms for
related issues service when needed trained professionals who service engagement
understand the needs of the
Low literacy LGBTQ community Low literacy
Limited English proficiency Limited English proficiency
Awareness of services No directory of services Difficulty understanding enrollment
or lack of outreach to processes
community Lack of knowledge of specialty
mental health care for family No educational campaigns for Could not find services for
members youth or adults in need of care themselves or others
Not knowing where to go for Caregivers of family members
specialized and confidential lacked knowledge of how to
LGBTQ care access services
Barriers to Receipt of Appropriate Care
Only a small number of participants claimed to understand how to navigate the mental health
system or engage county support services, such as low-income housing, transportation, care-
36
giver assistance programs, and financial supports, when they or a family member experienced a
mental health-related crisis. Stigma affected all groups in several phases of the help-seeking
process in various ways—from identifying and informing others of the need to find a provider, to
entering treatment, and remaining in treatment. Stigma is embedded in several of the decisions
reported in the focus group excerpts below. Adequate attention and sensitivity to stigma in
public policy and messaging related to issues of mental health and serious mental illness may,
in fact, provide a starting point for enhancing awareness and engaging relevant persons in
appropriate mental health services and treatment.
Lack of knowledge associated with cost of care or lack of insurance
The cost of health-care services commonly is a barrier, often driven by lack of knowledge about
free and reduced-cost health-care options. Very few participants stated that they knew where
and how to access necessary mental health service. We discerned a lack of clarity about
insurance coverage and knowledge about free and reduced health-care services associated
with the MHSA. For example, even participants without insurance who were aware that they
could access mental health services did not do so because they thought they would have to pay
for services, which they could not afford. MHSA can and should rectify widespread
misperceptions about availability of free and low-cost care. Nevertheless, the cost of care and
coverage became a barrier that varied across the three stakeholder focus groups. Some of
those in the older adult and caregiver group stated that they delayed seeking mental health
services until they were eligible for Medicare because they did not qualify for Medi-Cal
assistance programs, and that paying out-of-pocket for mental health services would require
compromises in taking care of other responsibilities related to maintaining their housing.
LGBTQ youth stated that the problems of cost were frequently linked to the type of insurance
under which their care was covered. One youth explained:
“I’ve heard of people getting denied based on income and things like that because like actual income and like
what you can afford doesn’t always match up. Like maybe your parents make money or you make money,
but it’s all going to your house and bills, and you literally have nothing left. And they’re like, well, you make
this much amount of money, so we expect you pay $100. You know what I mean. It’s like I can’t afford that.”
Participants in the Latino immigrant group discussed the complex matter of cost related to
mental health referral from a primary-care provider.
Focus group leader: “When you go to the doctor and the doctor tells you that you have depression and
recommends that you go to the counselor; do they charge you two bills?”
Focus group participant (female): “Yes, you have to pay.… It’s another cost. It’s not covered by any program.”
Focus group leader: “Do people normally go [to a counselor]?”
Focus group participant (female): “No, because they don’t have the money.”
Focus group participant (female): “They would rather try to deal with their problem themselves, and do not go [to a counselor]
because they do not have the money.”
37
Focus group leader: “This program that existed before offered a fixed price with which they [clinics] would give
you many services.”
Focus group participant (female): “Yes, they [clinics] would give you the services for what you needed. You would only pay
$30 and with that they would treat you, and if you were referred somewhere else you didn’t
have to pay extra. You just showed them a paper that was given to you and you received
the other services needed. I think that program is going to end soon since they are
removing a lot of the help.”
Those from the Latino immigrant focus group discussed their experience with mental health
care as an “add on” to care; whether or not the cost would be covered was not consistent.
Those who reported having the care covered (as a part of a primary-care visit where a mental
health referral was made) stated that they would seek care. Others in the Latino group reported
that they would delay or decline seeking care due to the additional cost.
Location of services or transportation
Many participants reported that transportation to an appropriate mental health clinic or
appropriate provider was problematic. For example, some Latino immigrants living in Merced
County stated that they sought mental health services at a large primary-care clinic in their rural
community. A majority of participants stated, however, that not all people living in the Central
Valley know about this clinic or where to go for mental health services. As the following
discussion excerpt shows, unfamiliarity with services, combined with lack of transportation,
impedes people from accessing services.
Focus group leader: “How far is it [the clinic]?”
Focus group participant (female): “15 miles.”
Focus group leader: “Can you go there [the clinic] on a bus?”
Focus group participant (female): “No.”
Focus group participant (female): “The bus arrives at 5 a.m., and if you have an appointment at 9 a.m. you have to stay over
there all day; it’s the only bus. If there was a small bus for the [name withheld to protect
confidentiality] community, the people may be happier.”
Focus group leader: “Do you think that the people who live further away know about the [primary health]
services and where to go?”
Multiple participants: “No, not all of them”.
Focus group participant (female): “Very few.”
For many older adults, transportation even to local clinics is a persistent need.
“Lack of transportation is a constant issue because, for example, she gave a ride to a couple people in this
group and if she hadn’t, they would not have been able to participate. So, it’s a constant need, so, I can
imagine, even for medical appointment, for eye appointment, for everything.”
Stakeholders offering public comments underscored that counteracting limitations in
transportation is not a simple matter but would go a long way to remediate access impediments
38
for un(der)served populations, particularly the most marginalized groups, such as people living
in rural areas, those with a serious mental illness, and older adults.
Delayed care-seeking
Despite their self-identified need for care, several participants admitted to postponing care due
to barriers preventing them from accessing or receiving proper treatment. Members of the
LGBTQ focus group reported a fear of initiating treatment because the care is perceived to be
inappropriate or unavailable for their mental health-related concerns and needs.
“Generally LGBT people are more likely to go to a place that specializes for them because they just feel
like—I personally feel more like they will understand where I’m coming from more instead of just being—you
always have that fear that you’re gonna run into that person that’s super judging you, like they don’t
understand you or they’re [on the] total opposite side of the fence from you.”
Those from the older adult group reported delaying care-seeking until the age of 65 because
they did not qualify for Medi-Cal for mental health-care coverage.
“After 65, it became easier for her to seek services because when she went to the doctor, she was receiving
more comprehensive medical care and even if it––if there was a mental health need that presented itself, the
doctor said, ‘let me refer you to a counselor, let me tell you where to go.’ But before that, it was a little bit
more pricey and she had to, you know, figure out a way to get a plan to take care of her.”
Even those who may have sought mental health-care services reported instances of delaying
care and being reluctant to return for mental health-care services due to inconsistency in the
quality of care they received, as a Latino immigrant group member explains in this excerpt:
Focus group participant (female): “The service received varies [in primary clinics]. Sometimes they treat you well and other
times they tell you [that you] have something and give you medications for it. Sometimes
the problem doesn’t go away. They are only controlling it for a while—maybe a couple
hours. When they send you to another place it’s the same. They may be able to control
what you have for a couple days, but it doesn’t go away”.
Focus group leader: “Then what happens when a person goes and just receives medication, and their issue isn’t
resolved?”
Focus group participant (female): “One doesn’t want to go back. If they return they are going to get the same thing. In time
you may have to go back because the issue persists. It’s not enough.”
Focus group leader: “When another problem arises you go back and they treat you the same?”
Focus group participant (female): “Yes, it’s the same. They give you the same thing.”
Focus group leader: “If you go to the clinic and they don’t offer to give you anything, what do you do next?”
Focus group participant (female): “You have to look for an alternative.”
Focus group leader: “What type of alternative?”
Focus group participant (female): “Go to another clinic or another place.”
These participants explained that having to find an appropriate avenue or setting for treatment is
often discouraging and does not encourage them to seek out services when they identify a
mental health-care need for themselves or a family member
39
Family relationships
Navigation of the mental health system appeared to be especially complex for focus group
participants when seeking services for family members. Several members of the Latino
immigrant and older adult focus groups reported being affected by mental health issues of
family members. A participant from the Latino immigrant group stated:
“A lot of us haven’t had the need to look for one of those places; that is why we don’t know where those
places are located. I got sent there from a clinic. It helped my daughter to go to the counselor. I think that
going to see a counselor doesn’t mean that you are crazy. Sometimes people have depression, and it
benefits those people to seek help from a counselor.”
Lack of experience accessing and navigating mental health systems, in addition to the stigma of
having a mental health-related condition, can present barriers to appropriate care. This and the
subsequent excerpts indicate how engagement of family members in prevention and outreach
activities may enhance the timeliness, appropriateness, and perceived value of the services
received.
While older adults discussed their experiences with family members, they emphasized what
they referred to as the “vicarious trauma” they have experienced with family members (with or
without mental health-related needs).
Female: “We do have a lot of problems with children, grandchildren, or ourselves. But if we needed help [mental health
treatment], we don’t want to meet if we have a problem.”
Female: “For us, we think you had to be billed crazy, you know, but no, we don’t—we’re not crazy. But we need it
sometimes and we don’t have—we don’t know when to go. We—where do we go so we can have help, because
we do have problems with our children?”
Other populations who might experience vicarious or other trauma that could create barriers to
receiving adequate mental health care include immigrant groups, many of whom have
experienced historical trauma.
LGBTQ participants described a different set of issues with their family relationships. These
issues were part of the complex intersection of their age and sexuality in addition to other
factors. One participant explained:
“If we’re talking about using health insurance … a lot of the times, people, when they need help, go to their
parents and say like, ‘hey, mom and dad, hook me up with a therapist.’ But the problem is a lot of the times,
there are strained relationships between like a particular group and like the parents. It’s not the easiest group
to say ‘hey, like at least I can ask my mom and dad because [unintelligible] and if they do, it’s like conditional
and it’s that whole, like, financial dependence. I know somebody that is—was going through a [gender]
transition that applied with Kaiser, but it was through the parents … because their parents kept like using that
against them, like ‘well, if you don’t go through with your transition, then you can’t be on our health care,’
because the parents did not support it. Kind of like I think it’s—it’s really difficult because I think a lot of
people try to go to their parents and, like, yeah, it’s not available. Like, it’s not really an option.”
Those who do use their parents’ insurance under these strained conditions fear that their
confidentiality in treatment and decision-making will be disclosed by their health-care provider to
their parents.
40
Lack of concordant care
Members of all three focus groups discussed multiple examples of problems with concordance
in the needs of their communities and the quality and type of care provided. Participants
described a lack of concordant care to meet mental health-related needs in three general areas:
adequacy of services; lack of culturally competent care; and barriers with language and literacy.
Adequacy of services. A member of the LGBTQ youth focus group described her sentiments
related to trying to access mental health services.
“I think a lot of people are still ignorant to it. I mean, you know, we—all the people now are growing up so far
in a place that doesn’t really talk about it [growing up LGTBQ]. Like no schools talk about it, you know, they
don’t go over the history of it. No one really understands it until you either know someone that’s LGBTQ or
you are and, like, you don’t know anything about it. I mean, I’ve known tons of people that are like ‘I didn’t
even know I was gay because I didn’t even really know what that meant,’ you know, and then they figure out,
‘Oh, I am. Okay, that’s what I am.’ But it’s like—it’s hard when, you know, you have people that have gone to
school and they’re—you know, they’re so advanced and they’re in this profession [mental health providers],
and they still don’t understand.”
Another participant in the same group stated:
“I think sometimes it kind of drives people from the community to not mention you are gay. If you’re going to
treatment, like, just don’t mention it. You know, like, stick to the parts you want to talk about because if you
mention anything else, it will look bad, so you avoid it and you don’t talk about your relationships unless it’s
like a past one or something. You use gender-neutral names so you don’t give away your orientation. It’s just
you try not to—and maybe they’ll just think I’m a tomboy instead of a lesbian. I think sometimes we—you
have an hour … so you avoid those little parts that you just don’t want to discuss because I think that they
think I have already figured that part out [that I am gay] and it’s like ‘no, I don’t want to talk about [it].’ I
already figured that part out. It’s the other stuff that I want to talk about.”
Participants in this group expressed the desire to discuss their sexuality in a context where it did
not distract from other mental health-related needs and care. More than a distraction from care
is the concern that the care received would not be appropriate if the larger context of care were
not considered in terms of an LGBTQ consumer’s gender or sexuality.
Focus group participant (female): “A few of them [LGBTQ acquaintances] I remember attempted suicide—and had to be
hospitalized.”
Focus group leader: “Okay.”
Focus group participant (female): “And then they didn’t wanna speak to the therapist again after that incident happened.”
Focus group leader: “Ohhhhh.”
Focus group participant (female): “And the division where we were, we had a, like, a gay man doing therapy, so we really had
limited access to other genders for therapists”.
Focus group leader: “Um-hm” [affirmative].
Focus group participant (female): “So those were some big issues.… When there are women who have been raped or
attacked by a man, it’s really hard for them to be around a man.”
Focus group leader: “Yeah.”
41
Focus group participant (female): “So, just, there are issues around men with the tr- … trans girls—that could also, you know,
be a problem when it comes to therapy.”
A member of the Latino immigrant group expressed a concern about the services provided for
men and the availability of those services, particularly for rural Latino men.
Focus group leader:
“Why do you think there are more resources for women?”
Focus group participant (male): “Because some of us just work, and women have time to go to the doctor and make
appointments.”
Focus group participant (male): “Women are always at appointments, and men like me don’t know anything.”
Focus group participant (male): “Women take care of the children and have time to go.”
Focus group leader: “How would you recommend changing the schedule to give men more access to the
resources?”
Focus group participant (male): “I would go on the days when I didn’t have to work, if I could have an appointment on those
days.”
Focus group leader: “If the clinics had flexible hours?”
Focus group participant (male): “No, I would go on days when I wouldn’t have to work.”
Work schedules with long work hours in addition to distance make access to mental health
service especially difficult for rural Latino men.
Culturally competent services. Latino Immigrants expressed a strong need for culturally and
linguistically competent mental health-care services, especially in times of crisis. A woman from
the group described the following situation with her daughter:
Focus group participant (female):
There is a lack of interpreters at hospitals, especially when you go to the emergency room.
Focus group leader: Who are your interpreters? A friend?
Focus group participant (female): Once I took my daughter to the emergency, when she was younger, and none of the
nurses spoke Spanish.
In the context of stakeholders who experience significant stigma, the need for confidentiality
from family, difficulty engaging appropriate services and programs, and a lack of interpreters or
reliance on a family member for interpretation can compromise quality of care.
LGBTQ participants described experiences with providers they regarded as untrained or
inadequately trained to understand the needs of their communities.
“I think—and that’s what a psychiatrist is supposed to do. They’re supposed to basically make you more
educated about yourself. So, I mean, it would be nice if they already knew you, you know, and what you were
going through. And to have them be educated and to go there and to not even have to like—to go there [and
discuss sexual orientation when it is not relevant] not … to even have to say that you’re gay because you’re
not even coming there for issues like that, but just to have them already know about things like that so …
they don’t have to be [LGBTQ] as long as they just—they understand. They get it, you know, they’re not
like—they’re past the point of being curious. They’ve talked to people and they’re not just reading it in a book,
you know; you have to talk to someone to understand. Like you have to get other people’s experiences with
that situation.”
42
Stakeholders who commented on the “fit” between consumers and their providers suggested
that adequate training of diverse community professionals could enhance the quality of service
for the most un(der)served groups, including marginalized ethnic groups and those with serious
mental illness, in addition to persons who participated in our focus group discussions.
LGBTQ youth did not want to have to supply the “training experience” for their mental health
providers. Many stated that the “fit” between their providers and their own mental health-related
needs was conducive to adequate or effective care.
Language and literacy. Culturally and linguistically competent care includes, at a minimum,
having interpreters available for care-related services. Beyond interpretation services,
participants in the Latino immigrant group discussed how struggles with the English language
and literacy with health-care forms have presented problems with engaging in mental health
services. Members of the Latino immigrant group discuss some examples in the following
excerpt:
Focus group participant (female): “Sometime the receptionists are very rude [receptionists in clinics]. I wonder why they put
some very unfriendly people to help.”
Focus group leader: “How are they rude to you?”
Focus group participant (male): “If you don’t fill out the paperwork correctly, they tell you to come back another day.”
Focus group leader: “They don’t help you fill out the forms if they are in English?”
Focus group participant (male): “There is times when you don’t speak English and you get the forms in English and you
can’t fill them out. They just tell you to come back again and look for someone to interpret
the forms for you.”
Focus group participant (male): “Return another day when that person isn’t there.”
Focus group participant (female): “What else can we do, we can’t stay there and argue. If that person is mad and you try to
tell them otherwise.…”
Participants from the older adult group expressed similar frustrations with form-related literacy
that sometimes prevented them from getting timely care. One participant who said she had
difficulty enrolling in services due to her limited English proficiency and low level of literacy
described her experiences:
“I just feel like the bureaucracy is tremendous, as you well know [when seeking mental health services]. So,
filling out Medi-Cal forms, filling out applications for WIC or food stamps, you know, all of that is
overwhelming, especially if you’re not familiar or English is your second language or you’re not very, you
know, adept in the language. Filling out forms like that is enough to—and then if you have some mental
issues, you know, it would be too hard.”
As this participant points out, a concomitant mental health need could further delay or impede
the process of engaging in appropriate mental health services.
LGBTQ youth report learning about mental health-related issues and services by way of the
Internet, where information quantity is far greater than information quality.
“A lot of that [finding out about mental health needs and services] happens on the Internet. I mean, like,
especially if you come from a smaller town. Like here [name of county withheld], you have like a little bit more
43
options. I grew up in a small town so it would have been—it’s a very closed-minded town so there really
weren’t any options for me and, you know, most of the—I figured out most of myself just by looking at things
on the Internet, and I actually finally found like a web page that had other people, like, sharing who they were
and I was, like, ‘this is me. This is totally me.’ So, that’s, like, that’s totally how I figured myself out, but—
yeah, yeah, in smaller towns it’s even harder.
The barrier that this participant describes regarding “small towns” is also characteristic of rural
communities, where access to services or service-related support systems is limited.
Awareness of services or lack of outreach to community
Several participants in each of the three focus groups reported difficulty finding appropriate
mental health-related care for their own needs or for those of a family member. All focus group
participants with specialized needs reported a lack of awareness and difficulty in getting care
that was appropriate to their needs or the needs of a family member.
Members of the LGBTQ youth group reported seeking information about mental health-related
definitions, needs, and services on the Internet:
Focus group leader:
“But, like, as people are transitioning and growing into adolescence, where do people go to
learn? Like you said, like, some—you know, there was one person who was, like, ‘oh, I
didn’t even know who to ask.’ But where do people go to learn, and then not just learn
about themselves and who they’re becoming and how they’re evolving, but where do they
go to learn [when they] say “I might have depression?” Or, “I might have anxiety?” Like,
what resources are out there to know?”
Focus group participant (female): “Honestly, like, it’s like the Internet.”
Those participants in both the youth and older age groups explained the impact of age as a
factor in understanding and seeking mental health-related care. A participant in the older age
group explained:
“You know, there’s issues of culture—I’m thinking it’s not just culture or your ethnicity; age groups, it’s also a
culture, no? And you know, there’s a lot of conservatism in that sometimes we’re closed in not wanting—not
necessarily not wanting, but not knowing how to branch out, where to find services. And many times we rely
on the media television to tell us about mental health services or any health issue on where to go, and it’s
obviously not doing a good job. And because, you know, for other reasons we’ve already talked about that
there’s a lot of stigma, and people just don’t know when … when an emergency presents itself, people just
don’t know where to go. And she gave an example of a suicide she had in her family, and it was very
shocking, of course.… When these crises happen in our families … we realize, oh, this is mental health.
Right? And where do we go for help?”
LGBTQ participants echoed these concerns with regard to specialty services for their
communities:
Focus group participant (female): “I don’t know where to go [for mental health services]. I don’t know how to get there. I don’t
know how to pay for it. I don’t know how to fill out these forms. You know, there’s a lot of—
a lot of, like, education component is a big part.”
Focus group leader: “What do you mean by ‘knowing where to go’?”
44
Focus group participant (female): “Knowing where to go or, you know, knowing what sort of—what is mental health? And I
think once people figure it out, they’re like ‘I need to see somebody.’ But I think that for a
long time, some people are just like, ‘oh, this is just the way I am’ and they don’t realize
that. It’s just, like, what is mental health? Like when you’re reaching out to someone. It is a
lonely process, and I don’t feel comfortable telling my parents or I don’t feel comfortable
telling the school counselor. You know what I mean?”
Those participants in the older adult focus group explained that they had difficulty understanding
how to enroll or complete the appropriate forms in order to engage in mental health services,
and they knew of no apparent resources for assisting older persons in accessing services. One
group participant stated:
“There’s a need for help in filling out forms, enrolling in medical services. For example, she got stuck in filling
out a medical form—a Medi-Cal application, and took, I’m assuming, weeks, months for her—[for] it to finally
go through. And she would even go to the place, the Medi-Cal center, and they would tell her, ‘oh, no, we
don’t offer that type of health care.’”
The interface between programs and funding exacerbates the confusion and appropriate
provision of services for some populations, as explained in the excerpt above from the older
adult group, suggesting a more seamless bureaucratic process that might allow older adults or
those with multiple co-morbidities to more easily engage mental health and other care services.
45
SUMMARY OF FINDINGS
These findings represent the focus group participants’ views about barriers to access to
care and treatment based on experiences.
An overarching theme about lack of general knowledge of MHSA-funded
services, treatment options, and location was evident among most focus group
participants. Insufficient knowledge and awareness of services is concerning, given
the purpose and intent of the MHSA: to provide services that are not already
covered by federally sponsored programs or insurance programs.
Compounding an existing knowledge gap, difficulty with low literacy and limited
English proficiency among un(der)served groups creates an initial need for MHSA-
funded programs that educate people on how to navigate the mental health system
and use of other community support systems to obtain mental health care and locate
transportation services.
Another key finding was the barriers that un(der)served communities had in
common when seeking mental health care. For example, immigrant Latinos, LGBTQ
transition-age youth, and older adults reported the cost of care and lack of insurance
as a persistent barrier to seeking and accessing care. This finding suggests that
individuals from these groups are not knowledgeable or had not been successful in
finding MHSA-funded services, which are available at little or no cost to people
without coverage.
The responses by focus group participants emphasized that the MHSA should
focus on (a) language proficiency, (b) cultural competency, (c) diversity and
inclusion, and (d) integration of consumers and families when attempting to
strengthen linkages between the mental health system and un(der)served groups.
46
Conclusion
The findings of this report suggest that MHSA has and continues to have on impact on reducing
disparities among un(der)served groups, yet that impact and future directions need to be
understood and informed, respectively, by continued stakeholder engagement that includes
members of marginalized and un(der)served groups. Confusion remains, however, about the
specific contents of some MHSA programs and a lack of systematic and sustained activities to
evaluate their impact. We will not fully understand which types of strategies work, or under what
conditions they work, if no efforts are made to develop community-based or evidence-based
standards for evaluation of the implementation of these strategies.
Impact of the MHSA
The MHSA funds the largest expansion of mental health services in the past 50 years. More
significantly, MHSA recognizes the need of a continuum of care by promoting prevention and
early intervention to ensure early identification and avert the onset of a mental disorder in
children and youth. MHSA also promotes the transformation to a recovery-focused system of
care, which is targeted to individuals living with serious mental illness who are not receiving
enough support to alleviate their risk of homelessness, incarceration, or hospitalization.
The findings of this report suggest that MHSA has and continues to have an impact on reducing
disparities among un(der)served groups. The findings indicate that key informants recognize
MHSA’s commitment to invest in resources that build the capacity of un(der)served
communities. For example, informants acknowledged Prevention and Early Intervention (PEI)
and Full Service Partnerships (FSPs) as two major examples that strengthen the linkages
between un(der)served groups and access to care. Furthermore, we have learned through this
study that MHSA must do more to increase culturally competent programs and community
outreach and engagement in order to ensure that services are adequate and appropriate for
targeted un(der)served groups. This study is grounded in community-based participatory
research (CBPR), by obtaining stakeholders’ perspectives to assess the effectiveness of the
MHSA on delivery of services for un(der)served groups. As a result, it constitutes a step in the
right direction for MHSA to continue mitigating barriers to care.
Diverse stakeholders who offered critiques and suggestions through interviews or comments on
a draft of this report during a period of public feedback observed that they have not seen many
calls for stakeholder engagement, as the UCLA report had recommended. To date, stakeholder
engagement has not been translated into an “actionable item” or meaningfully honored. We
view this report as a step toward increasing the involvement and feedback of stakeholders. As
such, we encourage the continued integration of stakeholder perspectives, so that diversity in
perspectives drives the conversation about community-informed access to care and treatment.
The importance of eliciting and using stakeholder feedback emerged as a consistent theme
throughout this report as a significant means to help MHSA continue transforming mental health
services that mirror the composition of un(der)served groups in California.
47
Challenges or Barriers
Increasing access to care for all of California’s diverse and underserved groups is a formidable
challenge. Prior to implementation of the MHSA, underserved groups faced a multitude of
impediments to accessing care and receiving culturally and linguistically appropriate services.
The MHSA has enabled California to review the progress of county mental health systems and
has highlighted persistent and remaining obstructions in treatment access. Un(der)served
groups continue to experience difficulty in accessing care due to several barriers, including
language, cost of care or lack of insurance, stigma, geographic isolation, and in obtaining
assistance in understanding mental health systems and funding sources. At the system level,
improvements must be made in culturally competent treatment, outreach for underserved
groups, and provision of care in appropriate settings through planning and development of a
well-trained mental health services workforce, encompassing a range of mental health service
providers. Large county mental health programs as well as smaller community-based
organizations continue to experience difficulty in obtaining adequate funding to respond to these
needs and to systematically monitor expenditures. Strategies for response need to also include
appropriate evaluation and monitoring of existing programs, such as PEI and FSPs, proposed to
expand their services to un(der)served populations.
While many of these barriers to treatment access and effective delivery of care existed before
implementation of the MHSA, the MHSA’s support of ongoing evaluations to explore its impact
with underserved communities is a promising first step in overcoming key barriers for
underserved groups. Through the findings of this report, which offers a voice to un(der)served
groups, we obtain a picture of the matters most pressing to various communities and how we
can begin to resolve them.
Strengths and Limitations
This report is part of a larger project that included a population-based approach on mental
health disparities for un(der)served communities that remain following the implementation of the
MHSA. Because this report focuses on the viewpoints of diverse groups of stakeholders on the
impact of the MHSA on reducing disparities for un(der)served communities, it does not focus on
the quality and quantity of county-level measurements, data or outcomes. The intent of this
report is to help to inform larger, population-based analyses with diverse stakeholder
viewpoints.
This report had some important strengths and notable limitations. Among its strengths are the
qualitative and CBPR methods used to gather data from different sources (consumers, family
members, mental health providers, administrators, advocates, and members of LGBTQ groups).
Use of the MHSOAC and CEAB as resources and collaborators to design interview protocols
and identify participants is another attribute.
Because this report used qualitative methods and a small sample, it cannot be generalized by
extension to other populations. Specifically, its reliance on volunteers and recruiting participants
using the snowball sampling or word-of-mouth method may have limited the variability of
48
participants. Although CBPR strategies are currently among the best methods for eliciting the
viewpoints of diverse and marginalized communities, it is difficult to estimate the amount of time
needed to obtain the level of trust and rapport necessary to enlist specific groups who
experience high levels of mental health stigma (e.g., LGBTQ, those who suffer from serious
mental illness, or those whose trust may have been betrayed in previous research or by
government interventions, such as African Americans or ethnically marginalized communities
who have immigrated to the U.S.) in research-related endeavors similar to this report.
Another limitation was the fact that although a small number of key informants reported also
being consumers of mental health services for themselves or a family member, we did not
report their perspective as a consumer of mental health services; we rather defined them in
terms of the primary stakeholder status with which they identified (e.g., administrator, advocate,
policy maker, researcher). Because we did not specifically ask key informants to self-report,
whether or not they were the family member of a person living with a serious mental illness or if
they had a personal history of receiving mental health services, we cannot provide precise
counts of the number of family members interviewed. In addition, several focus group
participants described experiences related to care seeking for relatives living with a serious
mental illness; some of these experiences were reported in the findings above. While we were
able to reach theoretical saturation (and redundancy in themes) related to common barriers to
receiving appropriate mental health services for un(der)served populations, a major finding of
from our focus group interviews was that the unique intersection of those barriers magnify the
impact of the barriers in a manner which may be specific to distinctive populations. For example,
although un(der)served subgroups may experience some of the same persistent barriers, the
junction of those barriers (e.g., lack of insurance coupled with the stigma of sexuality (LGTBQI);
insurance barriers and the physical disabilities of aging (Older Adults); insurance and linguistic
barriers (Latino immigrants), augment the impact of the barriers for the affected groups.
In future research efforts using CBPR, more flexible time parameters, added funding, and
expanded community-based support may result in increased representation from hard-to-reach
and highly-stigmatized communities, such as those persons living with severe mental illness,
LGBTQ, African American communities, and those persons who identify as “non-Hispanic
White” with distinct ethnicity or heritage, whose perspectives and specific barriers may not have
been adequately represented in this report but provide for investigation of how the intersection
of barriers that shared across diverse populations might differently impact particular groups.
Throughout our report, we have pointed to the impact of stigma on underserved communities,
but we have yet to understand the full extent and scope of the role stigma plays in diverse
communities on limiting access to care or diminishing quality of care or efficacy. Additional
efforts must be made to assess the way in which stigma affects diverse underserved
communities. Similarly, interviewees described the impact of multiple barriers to receipt of
effective care or access to adequate treatment. While we have highlighted some of these
intersections, future evaluations need to expand on the questions and issues we outline in this
report in order to critically detail and clarify the impact of intersecting barriers and identify ways
to remediate them. This report may serve as a point of clarification for some stakeholder
perspectives and perhaps a point for framing questions about groups that this report may not
49
have been adequately represented. This report also offers further opportunity for formulating
informed questions about California mental health services and for establishing a benchmark
against which to measure future initiatives and improvements.
50
References
Aguilar-Gaxiola, S., Loera, G., Mendez, L., & Sala, M., Latino Mental Health Concilio, and
Nakamoto, J. (2012). Community-defined solutions for Latino mental health care disparities:
California Reducing Disparities Project, Latino strategic planning workgroup population
report. Sacramento, CA: UC Davis.
Alegria, M., Canino, G., Rios, R., Vera, M., Calderon, J., Rusch, D., et al. (2002). Mental health
care for Latinos: Inequalities in use of specialty mental health services among Latinos,
African Americans, and non-Latino Whites. Psychiatric Services, 53(12), 1547–1555.
Alegria, M., Lin, J., Chin-Nan, C., Naihua, D., Cook, B., & Xioa-Li, M. (2012). The impact of
insurance coverage in diminishing racial and ethnic disparities in behavioral health services.
Health Services Research, 47(2), 1322–1344.
Alegria, M., Perez, D. J., & Williams, S. (2003). The role of public policies in reducing mental
health status disparities for people of color. Health Affairs, 22(5), 51–64.
Bloor, M., Frankland, J., Thomas, M., & Robson, K. (Eds.). (2001). Focus groups in social
research. Thousand Oaks, CA: Sage Publications, Inc.
Brach, C., & Fraserirector, I. (2000). Can cultural competency reduce racial and ethnic health
disparities? A review and conceptual model. Medical Care Research Review, 57,
Supplement, 181–217.
Bruckner, T., Yoon, J., Brown, T., & Adams, N. (2010). Involuntary civil commitments after the
implementation of California's Mental Health Services Act. Psychiatric Services, 61(10),
1006–1011.
Cabassa, L. J. (2007). Latino immigrant men's perceptions of depression and attitudes toward
help seeking. Hispanic Journal of Behavioral Sciences, 29(4), 492–509.
California Department of Mental Health (2011). Mental Health Services Act (MHSA)—‘overview’
fact sheet (Sacramento, CA).
California External Quality Review Organization. (2012). Overview of findings from CAEQRO
statewide annual report of FY10–11. APS Healthcare. Microsoft PowerPoint presentation
retrieved from http://www.caeqro.com/webx/.ee850bf
California Office of Multicultural Services. (2011). Fact sheet—county cultural competence
plans.
California State Auditor. (2013). Audit Report: Mental Health Services Act—The state's
oversight has provided little assurance of the act's effectiveness, and some counties can
improve measurement of their program performance. 212–122.
Cashin, C., Scheffler, R., Felton, M., Adams, N., & Miller, L. (2008). Transformation of the
California mental health system: Stakeholder-driven planning as a transformational
activity. Psychiatric Services, 59(10), 1107–1114.
Chow, J. C-C, Jaffee, K., & Snowden, L. (2003). Racial/ethnic disparities in the use of mental
health services in poverty areas. American Journal of Public Health, 93(5), 792–797.
Clark, W., Welch, S. N., Berry, S. H., Collentine, A. M., Collins, R., Lebron, D., & Shearer, A. L.
(2013). California’s historic effort to reduce the stigma of mental illness: The Mental Health
Services Act. American Journal of Public Health, 103(5), 786–794.
51
Denzin, N. K, & Lincoln, Y. S. (Eds.). (2000). The sage handbook of qualitative research.
Thousand Oaks, CA: Sage Publications, Inc.
Edberg, M. C., Wong, F. Y., Woo, V., Doong, T. (2003). Elimination of health disparities in
racial/ethnic minority communities: Developing data indicators to assess the progress of
community-based efforts. Evaluation and Program Planning, 26(1), 11–19.
Gee, G. C., Ryan, A., Laflamme, D. J., & Holt, J. (2006). Self-reported discrimination and mental
health status among African descendants, Mexican Americans, and other Latinos in the
New Hampshire REACH 2010 initiative: The added dimension of immigration. American
Journal of Public Health, 96(10), 1821–1828.
Grant, D., Padilla-Frausto, I., Aydin, M., Streja, L., Aguilar-Gaxiola, S., & Caldwell, J. (2011).
Adult mental health needs in California: Findings from the 2007 California Health Interview
Survey. Retrieved from
http://www.healthpolicy.ucla.edu/publications/Documents/PDF/MentalHealthreportnov2011.
pdf
Gubrium, J. F., & Holstein, J. A. (Eds.). (2002). Handbook of interview research: Context and
method. Thousand Oaks, CA: Sage Publications, Inc.
LeCook, B., McGuire, T. G., & Zaslavsky, A. (2009). Measuring racial/ethnic disparities in health
care: Methods and practical issues. Health Services Research, 66, 23–48.
Lofland, J., Snow, D. A., Anderson, L., Lofland, L. H. (2005). Analyzing social settings: A guide
to qualitative observation and analysis (4th ed.). Cengage Learning.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook.
Thousand Oaks, CA: Sage Publications, Inc.
Minkler, M., & Wallerstein, N. (Eds.). (2010). Community-based participatory research for
health: From process to outcomes. San Francisco, CA: Wiley.
Miranda, J., & Cooper, L. A. (2004). Disparities in care for depression among primary care
patients. Journal of General Internal Medicine, 19(2), 120–126.
Miranda, J., Lawson, W., & Escobar, J. (2002). Ethnic minorities. Mental Health Services
Research, 4(4), 231–237.
Morgan, D. L., & Krueger, R. A. (1994). Focus group kit. Thousand Oaks, CA: Sage
Publications, Inc.
Morse, J. (2004). Theoretical saturation. In M. Lewis-Beck, A. Bryman, & T. Liao (Eds.),
Encyclopedia of social science research methods (pp. 1123–1124). Thousand Oaks, CA:
Sage Publications, Inc.
Nadeem, E., Lange, J. M., Edge, D., et al. (2007). Does stigma keep poor young immigrant and
US-born Black and Latina women from seeking mental health care? Psychiatric Services,
58(15), 47–1554.
Pescosolido, B. A. (1992). Beyond rational choice: The social dynamics of how people seek
help. The American Journal of Sociology, 97(4), 1096–1138.
Pescosolido, B. A. (2006). Of pride and prejudice: The role of sociology and social networks in
integrating the health sciences. Journal of Health and Social Behavior,47(3), 189–208.
52
Pescosolido, B. A., McLeod, J. D., & Avison, W. R. (2007). Through the looking glass: The
fortunes of sociology of mental health. In W. R. Avison, J. D. McLeod, & B.A. Pescosolido
(Eds). Mental health, social mirror (pp. 3–32). New York: Springer.
Pope, C., & Mays, N. (Eds.). (2000). Qualitative research in health care. Malden,
Massachusetts: BMJ.
Racial and Ethnic Mental Health Disparities Coalition (2013). CMMC overview. Retrieved from
http://remhdco.org/cmmc/cmmc-overview/
Reiter, M., Sinz, S., & SenGupta, S. (2012). Overview of findings from CAEQRO statewide
annual report of FY10–11. PowerPoint presentation delivered to the California Department
of Health Care Services.
Snowden, L. (2005). Racial, cultural and ethnic disparities in health and mental health: Toward
theory and research at community levels. American Journal of Psychology, 35(1/2), 1–8.
Snowden, L., Masland, M., Ma, Y., & Ciemens, E. (2006). Strategies to improve minority access
to public mental health services in California: Description and preliminary evaluation. Journal
of Community Psychology, 34(2), 225–235.
Sorkin, D. H., Pham, E., & Ngo‐Metzger, Q. (2009). Racial and ethnic differences in the mental
health needs and access to care of older adults in California. Journal of the American
Geriatrics Society, 57(12), 2311–2317.
UCLA Center for Healthier Children, Families, & Communities. (2011). Evaluation brief
summary and synthesis of findings on MHSA values. Retrieved from
http://www.mhsoac.ca.gov/Evaluations/docs/Del3B_Phase-II_MHSA-ValuesReport.pdf
UCLA Center for Healthier Children, Families, & Communities. (2011). Evaluation brief
summary and synthesis of findings on CSS consumer outcomes. Retrieved from
http://mhsoac.ca.gov/Meetings/docs/Meetings/2011/May/OAC_052611_Tab8_UCLAEvalBri
ef.pdf
UCLA Center for Healthier Children, Families, & Communities. (2013). California’s investment in
the public mental health system: Prop 63 allocation and expenditures (FY 06–07 through
09–10). Retrieved from
http://www.mhsoac.ca.gov/meetings/docs/Meetings/2013/FinancialOversight_102313_Tab5
_UCLAReport_CAInvestmentInPublicMHSystem.pdf
Van Voorhees, B. W., Walters, A. E., Prochaska, M, & Quinn, M. T. (2007). Reducing health
disparities in depressive disorders outcomes between non-Hispanic Whites and ethnic
minorities a call for pragmatic strategies over the life course. Medical Care Research and
Review, 64(5 Suppl.), 157S–194S.
Vega, W. A., & Lopez, S. R. (2001). Priority issues in Latino mental health services research.
Mental Health Services Research. Special Issue: Barriers to Providing Effective Mental
Health Services to Racial and Ethnic Minorities in the United States, 3(4), 189–200.
Wallerstein, N. B., & Duran, B. (2006). Using community-based participatory research to
address health disparities. Health Promotion Practice, 7(3), 312–323.
Wells, K., Klap, R., Koike, A., & Sherbourne, C. (2001). Ethnic disparities in unmet need for
alcoholism, drug abuse, and mental health care. American Journal of Psychiatry, 158(12),
2027–2032.
53
White, K., Haas, J. S., & Williams, D. R. (2012). Elucidating the role of place in health care
disparities: The example of racial/ethnic residential segregation. Health Services Research,
47(2), 1278–1299.
Williams, D. R., & Rucker, T. D. (2000, Summer). Understanding and addressing racial
disparities in healthcare. Health Care Financing Review, 21(4), 75–90.
Yoo, J., & Ward, K. J. (2012). MHSA statewide participatory evaluation initial report phase III
deliverable 2a-2 and 2b-2. UCLA Center for Healthier Children, Families, & Communities.
Retrieved from
http://www.mhsoac.ca.gov/Evaluations/docs/Eval_MHSAStatewidePartEval_InitialReport_P
haseIIIDeliv2a-2And2b-2_103112.pdf
54
Appendix A: Diagram of Qualitative Methodology
UCD IRB review
Contact referral sources and key informants for interview
and approval
Sample referral source list for Define groups a nd develop list of Describe findings related to topic
additional key informant interviews potential participants for focus areas elicited by responses to
group interview s guiding interview questions
Analyze key informant interviews for
Categorize an d purposefully
insight into focus group composition
sample focus group participants
and guiding questions
Contact focus group participants and
Revise focus group questions
conduct focus groups
Analyze focus group interview data
Examples of stakeholder experiences of Develop a list of Describe findings related
consumers and families from un(der)served barriers to receiving to topic areas elicited by
and inappropriately-serviced communities adequate or responses to guiding
appropriate care interview questions
Summarize interview findings and develop recommendations
5 5
Appendix B: List of Documents and Sources
Reviewed for Qualitative Analysis
Pertinent Reports
Aron, L., Honberg, R., Duckworth, K., et al. (2009). Grading the states 2009: A report on
America’s health care system for adults with serious mental illness. Arlington, VA: National
Alliance on Mental Illness. Retrieved from
http://www.nami.org/gtsTemplate09.cfm?Section=Grading_the_States_2009&Template=/Co
ntentManagement/ContentDisplay.cfm&ContentID=75459
California External Quality Review Organization. (2011, October). Statewide report prepared for
the California Department of Mental Health, Volume I of III. APS Healthcare. Retrieved from
http://www.caeqro.com/webx/.ee85071
California External Quality Review Organization. (2011). Statewide report prepared for the
California Department of Mental Health, Volume II of III. APS Healthcare. Retrieved from
http://www.caeqro.com/webx/.ee85073
California Network of Mental Health Clients. (n.d.). Comments on the State Department of
Mental Health (DMH) draft proposed mental health services act (MHSA) capital facilities
guidelines. Retrieved from http://www.californiaclients.org/pdf/06 13
CNMHCCommentsonDMHDraftCapitalFacilitiesGuidelines.pdf
California Network of Mental Health Clients. (2008, April). Recommendations to OAC. Retrieved
from http://www.californiaclients.org/pdf/CNMHC%20April%20Memo%20to%20OAC.pdf
Center for Reducing Health Disparities. (2009). Mental health disparities summit final report.
University of California, Davis, Health System.
Meisel, J. (2012, September). MHSOAC evaluation master plan [PowerPoint presentation].
Pescosolido, B. A. (2006). Of pride and prejudice: The role of sociology and social networks in
integrating the health sciences. Journal of Health and Social Behavior,47(3), 189–208.
Saumitra, S. Zhang, H., & Baler, S. (2008, November). Measuring disparities using mental
health performance indicators: The California external quality review experience
[PowerPoint slides].
Scheffler, R. M., Felton, M., Brown, T. T., Chung, J., & Choi, S-S. (2010). Evidence on the
effectiveness of full service partnership programs in California’s public mental health system.
Nicholas C. Petris Center on Health Care Markets and Consumer Welfare, University of
California, Berkeley, School of Public Health.
Technical Assistance Collaborative & Human Services Research Institute. (2012). California
Mental Health Substance Use Needs Assessment. Submitted to the California Department
of Health Care Services.
Yoo, J., & Ward, K. J. (2012, September). MHSA statewide participatory evaluation initial report:
Phase III deliverable 2a-2 and 2b-b. UCLA Center for Healthier Children, Families &
Communities.
http://www.mhsoac.ca.gov/Evaluations/docs/Eval_MHSAStatewidePartEval_InitialReport_P
haseIIIDeliv2a-2And2b-2_103112.pdf
56
Appendix C: Documents Reviewed and
Resulting Potential Questions for Interviews
Contributing Relevant Guiding questions for interviews
documents constructs of
Interest
The Network- Perceived treatment How do individuals evaluate need for treatment within age, gender, and
Episode Model need ethnically diverse communities? How do these individuals gather mental
(Pescosolido, health information?
2006).
Appropriateness of How do these individuals perceive the sociocultural fit of the available
treatment options? How do mental health consumers and their family caregivers
compensate for limited services?
Access and Are there typical or regular informational sources of treatment for age,
resources gender, and ethnically diverse communities? What resources do
individuals turn to? [Inside and outside networks]
Organizational How do culture and climate within treatment organizations shape
culture treatment options, bonds between providers and ethnically diverse
consumers, and outcomes?
Connections among How do the dynamics of organizing and financing treatment systems
organizations and alter network ties in treatment sites, among providers, and between
people providers and consumers?
57
Contributing Relevant Guiding questions for interviews
documents constructs of
Interest
Center for Needs of special In what ways does MHSA assess the needs of ethnic communities?
Reducing communities
Health What are the factors that are considered most important in the
assessment of needs in ethnic communities? Are these factors
Disparities.
considered universally important across all communities?
(2009). Mental
Health
Disparities Alternative services What happens to ethnically diverse individuals who are denied MHSA
Summit. services? Where do these individuals go? Can you describe whether or
not they enter the system at another point with different
needs/conditions?
Special communities How has the MHSA included the Native American community in its
in need development of services?
How has the MHSA included the LGBTQ community in its development
of services?
California Achievements and What are the greatest successes and accomplishments of MHSA with
External Quality benefits of regard to unserved, underserved, or inappropriately served age, gender,
organization ethnically diverse groups? [Probe as to who or what groups benefitted.]
Review
What are the perceived gaps or issues yet to be addressed, including
Organization
barriers to resolving these issues?
(2011).
Statewide
report prepared New practice Which practice patterns (and outcomes) demonstrate innovative ways of
for California strategies and addressing the problems of ethnic and otherwise marginalized
Department of implementation communities? Where have these practices been implemented most
successfully? What have been the barriers to implementation in other
Mental Health.
communities experiencing disparities in mental health care access,
Volume II of III.
service delivery and outcomes?
58
Contributing Relevant Guiding questions for interviews
documents constructs of
Interest
California Barriers to effective What are some of the barriers to the effective delivery of mental health
External Quality delivery care for the populations you serve?
Review
Organization.
Important cultural What do you see to be the most important advances in terms of meeting
(2011,
advances the cultural needs of the community(ies) you serve?
October).
California
Mental Health Culturally competent What do you consider to be culturally competent mental health care?
Planning care and service How is this type of care planned for and/or delivered?
Council Cultural
Competence
Committee.
NAMI State Funding and Tell us about how MHSA funds are disbursed and managed in order to
Report Card for disbursement ensure service delivery to underserved communities.
the State of
California.
Response to needs How does MHSA respond to the needs of mentally ill, homeless, and
(2009).
of special elderly people?
populations
Service safety nets What kinds of services or safety nets are available for people who are
experiencing multiple social and psychological problems? Are the
services they receive sufficiently meeting their needs?
Persistent barriers What sorts of barriers are persistent in the provision or accessibility of
these services?
59
Contributing Relevant Guiding questions for interviews
documents constructs of
Interest
Scheffler, R. Potential risks and What do you see to be the (most severe) consequences for populations
M., Felton, M., negative outcomes accessing MH services? What groups do you see to be at the greatest
risk for some of the most severe negative outcomes?
Brown, T. T.,
Chung, J., &
Choi, S-S.
Mitigating negative Can you describe the ways in which the MHSA has helped to mitigate
(2010).
consequences of some of the unintended negative consequences for the unserved and
Evidence on underserviced underserved who seek care? [Probe for the ways in which the MHSA
the populations has done so for specific (marginalized) groups.]
effectiveness of
full service
Improvements for Are there improvements that have been made in specific populations
partnership
underserved that have provided lessons for how some of these less desirable
programs in
populations outcomes might be best addressed?
California’s
public mental
health system.
Nicholas C.
Petris Center
on Health Care
Markets and
Consumer
Welfare,
University of
California,
Berkeley,
School of
Public Health.
60
Contributing Relevant Guiding questions for interviews
documents constructs of
Interest
Technical Resources for How has the MHSA provided the resources or avenues for monitoring
Assistance monitoring service and responding to gaps in services? What innovative strategies have
gaps been used? What barriers to effective monitoring still exist?
Collaborative &
Human
Services
Additional barriers to Can you discuss some of the factors external to the system or to
Research
care individual ability to engage in types of care that affect mental health
Institute. outcomes? [Probe here for specific subgroups and those with worse
(2012). outcomes.]
California
Mental Health
Evidenced-based What are your views on existing evidenced-based practices? How are
Substance Use
practice they responding to the ethnically underserved communities across
Needs
gender and age span?
Assessment.
Submitted to
the California
Department of
Health Care
Services.
Yoo, J., & Insights of groups Describe the ways in which the opinions and experiences of mentally ill
Ward, K. J. served on service people have been assessed with regard to the delivery of services,
delivery service types, and their integration. What insights have these groups
(2012). MHSA
offered? How have these insights been translated into practice patterns?
statewide
participatory
evaluation initial
In what types of communities have the use of peer or family support-
report phase III, Peer- or family- related interventions been the most effective? What have been the
deliverable 2a-2 based intervention constructive effects of this type of engagement/intervention? In what
and 2b-2. and impacts ways does this type of engagement/intervention need improvement?
What insights do you have about how this might be improved?
UCLA Center
for Healthier
Children,
Families, &
Communities
61
Contributing Relevant Guiding questions for interviews
documents constructs of
Interest
Additional Service availability What is your perspective on the availability of MHSA services throughout
questions, and sufficiency the state of California? Are services sufficient or inadequate?
arising from
team
Do any groups still lack access to MHSA programs? What prevents
discussion and
Groups lacking MHSA from reaching them?
review.
access
How do you view age, gender, and ethnically diverse consumer
Consumer engagement with MHSA services? How do you define treatment
engagement retention? In your perspective, how has treatment retention changed
since the establishment of MHSA?
How have consumers, contractors, and providers responded to efforts to
Cultural competence improve cultural competence among MHSA-funded programs? What
types of critiques do providers and consumers give?
In your opinion, have services become more culturally competent? In
what ways? What impediments to improving cultural competence
remain?
Can you identify a model program or practice that you believe has
Model practices
achieved what you conceive to be culturally competent care?
62
Appendix D: Key Informant Interview Script and Guide
Study goal: We’re here today to talk with you about your experiences with California mental
health services. We are particularly interested in your perspective about mental health services
since the Mental Health Services Act of 2005—specific benefits related to care and service for
un(der) or inappropriately served persons or groups and barriers to adequate and appropriate
care. We will use what you tell us, along with the feedback we receive from other stakeholders,
to evaluate mental health services for un(der) or inappropriately served persons or groups, and
to make recommendations for addressing existing needs. We will be interviewing approximately
50 stakeholders in total. Approximately 20 stakeholders will be interviewed individually, and an
additional 30 stakeholders, principally mental health care consumers, will participate in a focus
group interview.
Participants’ role: You are the expert in this discussion. We want to hear about your
experiences and how your participation in the California Mental Health System [state the
stakeholder role here: administrator, provider, consumer, family member of consumer, activist]
has impacted you. Not everyone is going to have the same experience. We want to hear about
a variety of experiences. We would like you to consider yourself someone who is representing
others who might be like you and share your perspective—people who are your same age,
gender, race, or who come from your neighborhood.
Rules of participation: We will be audio recording this conversation and taking notes, so that
we can be sure to capture and accurately represent what you have to say. We need to make
sure that we hear what you have to say, so we ask that you speak loudly and clearly.
Process: We have quite a few questions to ask you, and we would like to get to all of them.
There may be times when I will have to move the conversation along. You have note pads and
a pen in front of you to keep track of anything you would like to say that you might not have a
chance to say [these items were provided to interviewees]. Please jot it down, and at the end of
the interview, we’ll try to get back to what you have noted.
Topics: We have several topics that we would like to discuss with you, including how the MHSA
has changed your experience with the California mental health system. There are four topics
that we will discuss related to your understandings of and experiences with mental health care.
We’ll begin by talking about how you or un(der) - or inappropriately-served groups might come
to seek care. We’ll then talk about where you or they might go to get help, and your
perspectives on the kinds of services offered. As we discuss each of these topics, we will
explore factors that have been particularly helpful in care seeking and receipt of care, and those
barriers you think might need to be addressed in order to improve mental health care in
California.
Do you have any questions before we begin? [Address questions and then begin the interview.]
63
Guiding questions (and follow-up probes) for key informant interviews
Tell us about your role and experience with California mental health services.
What are some of the barriers to the effective delivery of mental health care for the populations you serve? Can
you describe the ways in which the MHSA has helped to mitigate some of the unintended negative consequences
for the underserved who seek care? [Probe for the ways in which the MHSA has done so for specific
(marginalized) groups.]
In what ways does MHSA assess the needs of ethnic communities? What are the factors that are considered
most important in the assessment of needs in ethnic communities? Are these factors considered universally
important across all communities? How has the MHSA included the Native American community in its
development of services? How has the MHSA included the LGBTQ community in its development of services?
Tell us about how MHSA funds are dispersed and managed in order to ensure service delivery to underserved
communities.
What happens to individuals who are denied MHSA services? Where do these individuals go? Can you describe
whether or not they enter the system at another point with different needs or conditions?
What are the greatest successes and accomplishments of MHSA with regard to unserved, underserved or
inappropriately served groups? [Probe as to who or what groups benefitted.] What are the gaps or issues yet to
be addressed, and what have been some of the barriers to addressing these issues?
Which practice patterns (and outcomes) demonstrate innovative ways of addressing the problems of ethnic and
otherwise marginalized communities? Where have these practices been implemented most successfully? What
have been the barriers to implementation in other communities experiencing disparities in mental health care
access, service delivery, and outcomes?
What do you see to be the (most severe) consequences for populations accessing MH services? What groups do
you see to be at the greatest risk for some of the most severe negative outcomes?
How has the MHSA provided the resources or avenues for monitoring and addressing gaps in services? What
barriers to effective monitoring still exist?
Is there anything that we didn’t ask about that you would like to say, or perhaps something you would like to add
to your comments? Thank you.
64
Appendix E: Focus Group Interview Script and Guide
Study goal: We’re here today to talk with you about your experiences with California mental
health services. We are particularly interested in your perspective about mental health services
since the Mental Health Services Act of 2005—specific benefits related to care and service for
un(der) or inappropriately served persons or groups and barriers to adequate and appropriate
care. We will use what you tell us, along with the feedback we receive from other stakeholders,
to evaluate mental health services for un(der) or inappropriately served persons or groups and
to make recommendations for addressing existing needs. We will be interviewing approximately
50 stakeholders in total. Approximately 20 stakeholders will be interviewed individually, and an
additional 30 stakeholders, like yourselves, will participate in a focus group interview.
Participants’ role: Thank you for participating in this focus group. You are the expert in this
discussion. We want to hear about your experiences and how your participation in the California
mental health system [state the stakeholder role here: consumer, family member of consumer]
has impacted you. Not everyone is going to have the same experience. We want to hear about
a variety of experiences. We would like you to consider yourself someone who is representing
others who might be like you and share your perspective—people who are your same age,
gender, race, or who come from your neighborhood.
Rules of participation: We will be audio recording this conversation and taking notes, so that
we can be sure to capture and accurately represent what you have to say. We need to make
sure that we hear what you have to say, so we ask that you speak loudly and clearly.
We will use a digital recorder to record your responses and will give the digital recording a code.
The digital recording will not contain your identifying information. We will then transcribe the
recording. All recordings will be stored in a locked cabinet in a secured office accessible only to
the researchers of this study. Information gathered from these focus groups will be used to
complete this evaluation and for future publications. The results of this study will be made public
by the Mental Health Services Oversight and Accountability Commission. However, we will not
publish any information that could be used to identify an individual subject.
This focus group should take about 1.5 to 2 hours of your time. Please note that this focus
group is completely voluntary. You can decline to participate in this focus group, not respond to
any question you do not feel comfortable answering, or you may end your participation at any
time. The risks related to your participation in this focus group are minimal. However, it is
possible that discussing the receipt of mental health services may cause you minimal
discomfort.
Process: We have quite a few questions to ask you, and we would like to get to all of them.
There may be times when I will have to move the conversation along. You have note pads and
a pen in front of you to keep track of anything you would like to say that you might not have a
chance to say [these items were provided to interviewees]. Please jot it down, and at the end of
the interview, we’ll try to get back to what you have noted.
Topics: We have several topics that we would like to discuss with you, including how the MHSA
has changed your experience with the California mental health system. There are four topics
65
that we will discuss related to your understandings of and experiences with mental health care.
We’ll begin by talking about how you or your family member or friend might come to seek care.
We’ll then talk about where you or they might go to get help, and your perspectives on the kinds
of services offered.
As we discuss each of these topics, we will explore factors that have been particularly helpful in
care seeking and receipt of care, and those barriers you think might need to be addressed in
order to improve mental health care in California.
Do you have any questions before we begin? [Address questions and then begin the interview.]
66
Guiding questions for focus group interviews
Please tell us a bit about yourselves. How are you connected to the California mental health services system?
How do you assess your own needs for mental health care and treatment? How do you gather your mental
health-related information? Where do you get it?
Can you describe where you regularly go for mental health treatment? What other options for treatment do you
have?
How do you perceive the “fit” of the options available to you and the kinds of needs you have? [Ask the
interviewee to discuss specific examples.] What kinds of things do you do when you can’t get treatment for your
mental health issues or when services are limited?
Can you discuss some of the factors that limit your personal ability to engage in the care that is available? What
do these limitations mean for your mental health? [Probe here for specific examples.]
In your opinion, how does the kind of care you receive through the MHSA respond to the needs of mentally ill,
homeless, and elderly people?
What kinds of services are available for people impacted by multiple issues? [Try to give examples of the sort that
were brought up during the interview.] Are the services you receive meeting your needs?
In what ways do they meet your needs? In what ways do they fall short?
What sorts of barriers present themselves? [Probe for barriers specific to the focus group: ask immigrants about
language barriers and accessibility of services for immigrants in rural areas; ask about elderly people who have
comorbid conditions, age-related issues, and caregivers.]
Where have you seen the biggest impact of peer or family support-related interventions? Can you describe that
impact? In what ways might this type of program need improvement? What insights do you have as to how this
might be improved?
Has anyone ever asked you about your own experience in the mental health services system? [If so, who and
what did they ask; when did they ask?] If you have been asked about your experience, what have been the
insights that you feel you have been able to offer? Have you seen your suggestions or comments used in any
way?
Is there anything that we didn’t ask about that you would like to say, or perhaps something you would like to add
to your comments? Thank you.
67
Appendix F: Advisory Board & Public Comments/Feedback and Changes Adopted Matrix
Deliverable 2c
Report Sections Comments/Feedback Received Changes Adopted
I. Executive Comment (Advocate): The following is a troubling finding that Response: Thank you for your comments. In this section, we
Summary needs continual attention: "The UCLA Center for Healthier have attempted to clarify some of the issues regarding stakeholder
Section Children, Families and Communities (2012) evaluation found engagement and how it informs our report by introducing the
little evidence suggesting stakeholder involvement in the public following statement:
mental health system, which impedes the progress of MHSA-
funded services for un(der)served groups. This key finding This study is grounded in community-based participatory research
underscores the idea that the effectiveness of a progressive (CBPR), by obtaining stakeholders’ perspectives to assess the
public mental health system depends on stakeholder
effectiveness of the MHSA on delivery of services for un(der)served
involvement and the importance of representation from
groups. As a result, the persistent focus of MHSA on mitigating
un(der)served groups. A 2011 systematic review of existing
barriers to care constitutes a step in the right direction. We
MHSA evaluations, studies, and county documents found the
utility of stakeholder involvement on MHSA-funded services and encourage the continued integration of stakeholder perspectives,
their evaluation to be minimal (UCLA Center for Healthier so that diversity in perspectives drives the conversation about
Children, Families and Communities, 2011)." community-informed access to care and treatment. That strategy
emerged as a consistent theme throughout this report as a
Comment (Advocate/Consumer): What was the criteria in this
significant means to help MHSA continue transforming mental
report? To what extent was blended stakeholder, i.e. consumer
health services that mirror the composition of un(der)served groups
family member feedback used in the conclusions?
in California. In this report, our stakeholders describe their
With this much money going out shouldn’t we be hearing some experiences with and perceptions related to access and
astounding results from thousands moving through recovery, appropriateness of mental health services, which were expanded
served directly by programs that increase the wellness and the under the MHSA. We propose that this report, including the
health of individuals and communities through taking action for incorporation of public responses to the draft of this report, serves
social change...strategies to restructure mental health systems
as part of an ongoing dialogue between administrators of mental
to be recovery-oriented and community-based; to protect our
health services and the people they serve.
civil and human rights...to build more accepting, inclusive, and
diverse organizations and communities...maximize our individual
and collective strengths, potential, and creativity to make
wellness and social justice a reality for all. Are following
outcomes a result of or addressed in the report?
Are outcome assessments offered by consumers about their
reaction to health care a direct service?
68
Comment (Administrator): What about documentation on
consumer/stakeholder level of involvement: sporadic vs on-
going ; providing input, member of advisory, or involvement is
on-going and participates in collaborative a decision making
process?
Deliverable 2c
Report Sections Comments/Feedback Received Changes Adopted
II. Introduction Response: Thank you for pointing this out. We have added a
Comment (Advocate): One of the important groups furthering
footnote to acknowledge these efforts.
the reduction of disparities is the California MHSA Multicultural
Coalition (CMMC). This Coalition representing communities from The CRDP is supported by the California MHSA Multicultural
under represented and underserved constituencies (i.e., Coalition (CMMC). Formed on March 2011, the CMMC aims to
Deaf/Hard of Hearing, Middle Eastern Arab communities, identify challenges in access to care among underserved
Armenian, Russian-Speaking, Islamic populations, aging and communities and to recommend solutions for the development of
more culturally and linguistically appropriate mental health systems.
older adults, and other groups) plays a vital role in advocacy and
Composed of representatives from various underserved
in analysis of client, family member, and lived experience
communities throughout California, the CMMC offers the CRDP
perspectives regarding impact of the MHSA on mental health
valuable insights from diverse multicultural perspectives, inclusive
disparities. of consumers and family members, that have not been previously
represented in mental health systems (Racial and Ethnic Mental
Health Disparities Coalition (REMHDCO, 2013).
Comment (Mental Health Organization): Omission of the
California MHSA Multicultural Coalition (CMMC) in the section
describing the California Reducing Disparities Project (CRDP).
(Page 4) The CMMC is an integral part of the CRDP. In addition
to the five groups selected for special population reports, the
CMMC represents additional underserved communities in
addition to these five groups (the Middle Eastern communities,
the Deaf and Hard of Hearing community, the Muslim
community, the Older Adult community, the Armenian
community, etc.). The CMMC has produced reports and
documents that include information on the needs of these
69
communities and policy issues that impact reducing disparities
under the MHSA for all underserved communities.
Comment (Commissioner): Previous efforts to engage Response: We acknowledge these comments, which suggest that
stakeholders have not been met with active uptake of recommendations from the UCLA report related to stakeholder
recommendations or have not made recommendations that engagement were not translated into actionable items or have not
have been, according to public comment, actionable policy been meaningfully honored. While we do not directly address this
measures for the most marginalized of these groups. issue in the Introduction to our report, we provide a statement in the
Conclusion to our report, which draws attention to this point raised
Comment (Advocate): Annual updates with stakeholder input is by several of our public comments.
not reviewed carefully enough or, maybe minimally, by local
Boards nor honored as meaningful input by leadership. Including In the conclusion to our report, we added the following:
it as written form with seemingly glib responses in the update
discounts and often negates stakeholder involvement. Diverse stakeholders who offered critiques and suggestions
through interviews or comments on a draft of this report during a
Comment (Administrator): What about documentation on period of public feedback observed that they have not seen many
consumer/stakeholder level of involvement: sporadic vs on- calls for stakeholder engagement, as the UCLA report had
going ; providing input, member of advisory, or involvement is recommended. To date, stakeholder engagement has not been
on-going and participates in collaborative a decision making translated into an “actionable item” or meaningfully honored. We
process? view this report as a step toward increasing the involvement and
feedback of stakeholders. As such, we encourage the continued
integration of stakeholder perspectives, so that diversity in
perspectives drives the conversation about community-informed
access to care and treatment. The importance of eliciting and using
stakeholder feedback emerged as a consistent theme throughout
this report as a significant means to help MHSA continue
transforming mental health services that mirror the composition of
un(der)served groups in California.
Comment (Mental Health Organization): Lack of analysis in Response: Thank you for pointing out this oversight. We have
regards to communities of color and other underserved added the following italicized statement to our discussion of
communities in the section regarding Stakeholder Engagement. stakeholder engagement by the MHOAC:
This section describes evaluation efforts by the MHSOAC in
regards to stakeholder engagement but does not discuss that Specifically, the MHSOAC-sponsored evaluation will assess
lack of attention given to obtaining information from people with counties’ local planning processes. This is a step in the right
knowledge and connection to racial and ethnic communities.
direction to adequately evaluate the degree to which counties
For example, the most recent evaluation of the stakeholder
involve stakeholders and to ensure the full participation of
process was planned by a collaboration of organizations, none
stakeholders in MHSA-funded mental health services. In spite of
of which primarily served racial, ethnic, and cultural
70
communities. A great number of focus groups were to be this step, advocates for communities of color and underserved
conducted (58), but none of the focus groups targeted specific communities argue that attention and outreach to people who have
racial, ethnic or cultural communities as this study on
sufficient knowledge of and connection to these communities
underserved communities.
remain inadequate.
Deliverable 2c
Report Sections Comments/Feedback Received Changes Adopted
III. Research Comment (Advocate/Consumer): What was the criteria in this Response: Thank you for your question and the opportunity to
Methodology report? To what extent was blended stakeholder, i.e. consumer clarify throughout our report. We have added various sentences to
Section family member feedback used in the conclusions?
our Research Methodology section in order to clarify stakeholder
involvement and the purpose of using qualitative methods. One of
those specific comments is highlighted below:
We engaged multiple stakeholders in this evaluation. Stakeholder
involvement included participation in key informant or focus group
interviews, engagement on our advisory panel, and critiquing a
draft version of this report, which was circulated for public comment
for a period of thirty days (see Appendix F for feedback and
responses).
Comment (Advocate/Consumer): What was the criteria in this Response: Thank you for your question and the opportunity to
report? To what extent was blended stakeholder, i.e. consumer clarify throughout our report. Qualitative and quantitative analysis
family member feedback used in the conclusions?
take different approaches to sampling and observation. This
Deliverable reports the qualitative findings of our evaluation.
Comment (Advocate/Consumer): Offhand I believe qualitative
Quantitative (population-based) findings and methods are reported
statistics, not measuring the survey numbers, not quantitative
in a separate deliverable,
methods were applied to conclude that Reducing Disparities
targets were being met. It is helpful to know what other agency
We have added various sentences to our Research Methodology
responses, (see Orange County) are to this report which
section in order to clarify stakeholder involvement and the purpose
appears to completely miss the point in its unbalanced
of using qualitative methods. One of those specific comments is
conclusion.
highlighted below:
71
Comment (Mental Health Organization): Even though the The purpose of qualitative methods of research is to gain an
evidence to be acquired is largely qualitative in nature, it is understanding of the experiences of those engaged in mental
important that the sampling strategies have some reasonable
health services. Qualitative methods consisted of interviews with
quantitative basis in order to be taken as evidence to support
key stakeholders, including mental health administrators,
hypotheses or objectives.
advocates, and providers associated with mental health service
delivery, as well as individuals at risk of and living with serious
mental illness, their family members, and personal caregivers, who
were part of historically un(der)served groups. Interviews were
employed in order to have an open-ended, directed conversation
that could explore stakeholders' “firsthand knowledge and
experience” with California mental health service systems and the
impact of the MHSA.
Comment (Advocate/Consumer): What were the criteria in this Response: Thank you for your question and the opportunity to
report? To what extent was blended stakeholder, i.e. consumer clarify throughout our report. We have added various sentences to
family member feedback used in the conclusions?
our Research Methodology section in order to clarify stakeholder
Comment (Advocate/Consumer): Offhand I believe qualitative involvement and the purpose of using qualitative methods. One of
statistics, not measuring the survey numbers, not quantitative those specific comments is highlighted below:
methods were applied to conclude that Reducing Disparities
targets were being met. It is helpful to know what other agency We engaged various stakeholders at different levels—consideration
responses, (see Orange County) are to this report which of interview topics, identification of interviewees, review of drafted
appears to completely miss the point in its unbalanced report findings for analytic clarity and gaps in responses.
conclusion.
Comment (Administrator): What about documentation on
consumer/stakeholder level of involvement: sporadic vs on-
going; providing input, member of advisory, or involvement is
on-going and participates in collaborative a decision making
process?
Comment (Advocate): Among its strengths are the qualitative Response: Thank you for catching this oversight. We have
and CBPR methods used to gather data from different sources included advocates among our list of sources.
(consumers, family members, mental health providers,
administrators, and members of LGBTQ groups). Advocates can
Our approach to understanding diverse stakeholder perspectives
be included in this list.
72
(including administrative, advocacy, service provider, consumer,
and family perspectives) on the ability of the MHSA to reduce
disparities in access, quality of services, and outcomes also
involved a content review of current documents relevant to
understanding mental health services, consumer needs, and
disparities (Minkler & Wallerstein, 2010).
Comment (Mental Health Organization): Lack of details and Response: Thank you for allowing us the opportunity to clarify this
transparency with regard to the efforts of the CEAB (Community process. We now realize that our attempt to explain the process as
Expert Advisory Board). As the report states, “The CEAB met
simply as possible has made the process less transparent. We
four times during an 8-month period and solicited frequent
hope that our attempt to clarify has assisted in making the process
feedback . . . . Engagement with our CEAB assisted us in
a bit more clear.
refining our initial list of Key Informants as well as our strategy
for sampling informants and specific focus group participants
based on the barriers they face to appropriate and effective Our research team met with the CEAB four times during an eight-
care.” (Page 9) This appears to be a vital element in this report, month period, and solicited comments frequently during these
but this process lacks transparency. meetings via telephone calls as well as by electronic mail. Our team
sought advice from the CEAB regarding gaps in our list of key
What was the feedback? Can we learn more from this process
informants for interviews, including their informed understandings of
that will help explain why certain key informants and focus
the relationship of the potential key informant to the MHSA as well
groups were chosen?
as the types of underserved populations, racial and ethnic
Comment (Mental Health Organization): “I’m struck by the communities, and geographic areas the key informants
small size of the samples, and the use of an ESL class represented. CEAB members provided a critical eye that helped to
subsidized by MHSA funds to use as a focus group. It appears inform the selection of a range of key informants who could speak
that they deliberately used groups that weren’t involved in MH to the need of diverse persons and communities. Engagement with
services. Using a group of four gay and lesbian TAY as a focus
our CEAB assisted us in refining our initial list of key informants, as
group is too small to be meaningful. They don’t explain how
well as our strategy for sampling informants and specific focus
they got the 21 key informants.“
group participants, based on the barriers that impede their access
to appropriate and effective care.
.
73
Comment (Advocate): The guiding questions for the focus Response: Thank you for making this point. We have noted this in
groups are insightful; I'm wondering if they were clearly our discussion of interview methods.
understood by all participants (e.g., is "assess" clear? does
"engage" mean something different to cultural groups in their
The moderator was skilled in using the interview script as well as in
native language? are "interventions" understood or what does
clarifying unclear concepts or terminology for diverse participants.
an intervention look like to an immigrant? Is "mental health
Trained moderators have the capacity to respond to the needs of
services system" clear or is there awareness about a system
specific participants and to ensure that participants understand the
even, esp. when there's no familiarity about MHSA?) and
kinds of questions being asked. In addition, the inclusion of
whether they can be simplified even though a facilitator will
community members helped to ensure clarity in the presentation of
explain these terms well.
interview questions.
Comment (Mental Health Organization): We are told that the Response: We appreciate the fact that this information would be
CEAB “aimed to ensure a diversity of viewpoints . . . that useful and provide greater validation of our findings. However, our
represent and/or can speak to the experiences of un(der)served IRB protocol prohibits disclosure of any information that might
groups including, for example, Native Americans, the severely disclose the identity of a key informant. For those serving certain
mentally ill, and LGBTQ communities.” (Page 9) vulnerable populations or in smaller communities, such information
could potentially jeopardize the confidentiality of the interviewee.
However, the report gives no information as to what these aims Therefore, we have withheld such information.
produced and what populations the subsequent key informants
primarily represent. Such answers would provide key insights
into understanding the barriers and issues that informants
identified.
Comment (Mental Health Organization): Lack of attention to
gender and family issues. Key informants mention gender and
family issues only once on page 25. Thus, it would be pivotal to
know if key informants work primarily with women, men, families,
etc.
Comment (Mental Health Organization): …the report states Response: Thank you for asking for clarification regarding the
that “Three populations were targeted for focus group interviews generalizability of our findings. We address these comments
based on their experiences of multiple and significant barriers to
related to generalizability in two sections of our report. First, in
appropriate mental health services” (Page 13) What does the
describing our goals in selecting focus group participants, and
term “significant” encompass in this context?
second, in our discussion of limitations of our report. The additions
we have made to these sections of the report are noted below.
74
Comment (Administrator): this includes age, condition and
culture/linguistic To the methods discussion, we added:
Comment (Advocate/Consumer): This could be compared to Our goal was to learn from these groups about the multiple and
the number and demo characteristics of informants diverse barriers to appropriate and effective mental health services
delivery and to use these data as starting points for exploring the
Comment (Provider): A big problem I see for this report is the barriers encountered by other un(der)served groups who face
credibility for readers across the state. The amount of people similar obstacles.
interviewed and the limited regions surveyed may cause those
to not relate since it is based on such a small population. The To the limitations of our study, we added:
interviewee amount was minimal and central to certain areas,
not truly representative of the state and counties. If I were a Because this report used qualitative methods and a small sample,
reader I may say, "well that's nice to read 30 individuals views in it cannot be generalized by extension to other populations.
3 counties, but it doesn't help our area with the disparities here Specifically, its reliance on volunteers and recruiting participants
because of our populations, cultures, groups, etc... are different, using the snowball sampling or word-of-mouth method may have
limited the variability of participants. Although CBPR strategies
unique, reached, etc.."
are currently among the best methods for eliciting the viewpoints
of diverse and marginalized communities, it is difficult to estimate
Comment (Mental Health Organization): …the choice of
the amount of time needed to obtain the level of trust and rapport
counties to sample seems non-representative. The three
necessary to enlist specific groups who experience high levels of
counties selected were all large, urban counties. No rural or mental health stigma (e.g., LGBTQ, those who suffer from serious
medium-sized counties were included, which seems an mental illness, or those whose trust may have been betrayed in
insufficient sample size for a report of this type. previous research or by government interventions, such as
African Americans or ethnically marginalized communities who
Comment (Advocate/Consumer): Offhand I believe qualitative have immigrated to the U.S.) in research-related endeavors
similar to this report.
statistics, not measuring the survey numbers, not quantitative
methods were applied to conclude that Reducing Disparities
Another limitation was the fact that although a small number of
targets were being met. It is helpful to know what other agency
key informants reported also being consumers of mental health
responses, (see Orange County) are to this report which services for themselves or a family member, we did not report
appears to completely miss the point in its unbalanced their perspective as a consumer of mental health services; we
conclusion. rather defined them in terms of the primary stakeholder status
with which they identified (e.g., administrator, advocate, policy
75
Comment (Administrator): You list three “un(der)served maker, researcher). Because we did not specifically ask key
groups” in your report but left out the most important informants to self-report, whether or not they were the family
member of a person living with a serious mental illness or if they
underserved group – those with a serious mental illness;
had a personal history of receiving mental health services, we
schizophrenia, bipolar disorder or major depression.
cannot provide precise counts of the number of family members
interviewed. In addition, several focus group participants
Comment (Mental Health Organization): Lack of diversity in
described experiences related to care seeking for relatives living
the LBGTQ focus group. Only four members were included in with a serious mental illness; some of these experiences were
the LGBTQ focus group and “all identified as white, with two of reported in the findings above. While we were able to reach
the four reporting being Latina.” theoretical saturation (and redundancy in themes) related to
common barriers to receiving appropriate mental health services
for un(der)served populations, a major finding of from our focus
Comment (Mental Health Organization): The choice of
group interviews was that the unique intersection of those barriers
persons to speak for underserved groups seems not to have
magnify the impact of the barriers in a manner which may be
been large enough to be representative of those groups.
specific to distinctive populations. For example, although
un(der)served subgroups may experience some of the same
persistent barriers, the junction of those barriers (e.g., lack of
insurance coupled with the stigma of sexuality (LGTBQI);
insurance barriers and the physical disabilities of aging (Older
Adults); insurance and linguistic barriers (Latino immigrants),
augment the impact of the barriers for the affected groups.
Comment (Administrator): Also, if the premise of holding a Response: Thank you for allowing us to clarify. We have added
focus group for Older Adults was because they have little the following:
access to care then why wasn’t the focus group held with Older
Adults themselves as opposed to the caregivers?
We included both older adults and caregivers, because limitations
of multiple comorbid conditions limited some older adults from
participating in focus group discussions. Therefore, caregivers were
encouraged to participate, so as to share the perspectives of these
consumers.
Comment (Advocate/Consumer): What were the criteria in this Response: Thank you for your question and the opportunity to
report? To what extent was blended stakeholder, i.e. consumer clarify throughout our report. We have added various sentences to
family member feedback used in the conclusions?
our Research Methodology section in order to clarify stakeholder
76
Comment (Administrator): What about documentation on involvement and the purpose of using qualitative methods. One of
consumer/stakeholder level of involvement: sporadic vs on- those specific comments is highlighted below:
going; providing input, member of advisory, or involvement is
on-going and participates in collaborative a decision making
Our data analysis included “fact checking” by our interviewees and
process?
by way of a 30-day public comment period, which were used to
assess the relative accuracy of the data collected and our
interpretation of these data. All persons who participated in an
interview (key informant or focus group) or assisted in recruitment
of focus group participants were sent our analysis of the interview
data for review. Approximately 15% of persons who participated in
a key informant or focus group interview or who assisted in
participant recruitment provided feedback to our original report
(Deliverable 2b). We have incorporated and responded to
interviewee and public comments made on a draft of this report,
which was circulated on February 15, 2014.
Deliverable 2c
Report Sections Comments/Feedback Received Changes Adopted
IV. Findings Comment (Mental Health Organization): “I would like to state Response: Thank you.
Section that I thought the inclusion of "actual comments" made by
consumers and/or family members that were included in the
report was especially meaningful. They were direct statements
rather than "the family/and or consumer said" - it is nice to see
the recognition of the speaker included in the quotes.”
Comment (Advocate/Consumer): I wanted to remind everyone Response: Thank you for noting this. We have included a
that it’s far more complicated -- historically, emotionally and statement in our report that foreshadows our conclusions in this
ethically. How do we maintain compassion for those of us report and pending recommendations.
charged with this kind of study and work to support the
uncovering and goals by asking the right questions to gain more None of our focus group participants had an awareness of the
clarity or insight and see this as a collective community effort? Mental Health Services Act (MHSA), its ability to respond to health
disparities, or most common barriers to the effective delivery of
77
Comment (Family Member/Advocate): An overarching theme mental health care services in California (issues that pertain to
that emerged was lack of general knowledge of MHSA-funded questions 1 and 2 above). As a result, we offer information that our
services, treatment options, and location was evident among key informants offered to address the first two research questions.
most focus group participants. The lack of knowledge and Our presentation of findings pertaining to our third research
awareness of services is concerning given the purpose and question is based on key informant viewpoints of consumer
intent of the MHSA, to provide services that are not already experiences and the actual experiences described by our focus
covered by federally sponsored programs or insurance group participants. The fact that none of our focus group
programs. participants had an awareness of the MHSA or its impact suggests
that opportunities for increasing awareness and potential
Comment (Commissioner): One finding seems clear re: the engagement of consumers that have not yet effectively been
lack of awareness of MHSA programs. That seems actionable employed.
and under our control.
In our limitations section we state:
Comment (Advocate): Stigma experienced by individuals as
they move within the system is a barrier that needs to be Because this report used qualitative methods and a small sample, it
constantly addressed. Although FSPs have had an impact, it cannot be generalized by extension to other populations.
has not been well documented or reported -- and for Specifically, its reliance on volunteers and recruiting participants
underserved/under represented, how is public stigma reduction using the snowball sampling or word-of-mouth method may have
in the general community measured? limited the variability of participants. Although CBPR strategies are
currently among the best methods for eliciting the viewpoints of
diverse and marginalized communities, it is difficult to estimate the
amount of time needed to obtain the level of trust and rapport
necessary to enlist specific groups who experience high levels of
mental health stigma (e.g., LGBTQ, those who suffer from serious
mental illness, or those whose trust may have been betrayed in
previous research or by government interventions, such as African
Americans or ethnically marginalized communities who have
immigrated to the U.S.) in research-related endeavors similar to this
report.
Another limitation was the fact that although a small number of key
informants reported also being consumers of mental health
services for themselves or a family member, we did not report their
perspective as a consumer of mental health services; we rather
defined them in terms of the primary stakeholder status with which
they identified (e.g., administrator, advocate, policy maker,
researcher). Because we did not specifically ask key informants to
self-report, whether or not they were the family member of a person
living with a serious mental illness or if they had a personal history
78
of receiving mental health services, we cannot provide precise
counts of the number of family members interviewed. In addition,
several focus group participants described experiences related to
care seeking for relatives living with a serious mental illness; some
of these experiences were reported in the findings above. While we
were able to reach theoretical saturation (and redundancy in
themes) related to common barriers to receiving appropriate mental
health services for un(der)served populations, a major finding of
from our focus group interviews was that the unique intersection of
those barriers magnify the impact of the barriers in a manner which
may be specific to distinctive populations. For example, although
un(der)served subgroups may experience some of the same
persistent barriers, the junction of those barriers (e.g., lack of
insurance coupled with the stigma of sexuality (LGTBQI); insurance
barriers and the physical disabilities of aging (Older Adults);
insurance and linguistic barriers (Latino immigrants), augment the
impact of the barriers for the affected groups.
Comment (Advocate): Stigma experienced by individuals as Response (Research question 1): We appreciate your comments
they move within the system is a barrier that needs to be regarding the difficulty in identifying mental health issues as well as
constantly addressed. Although FSPs have had an impact, it in being a family member attempting to engage a loved one in the
has not been well documented or reported -- and for mental health system. We also acknowledge comments related to
underserved/under represented, how is public stigma reduction stigma that may emerge (continually) as consumers and family
in the general community measured? members move through the system. We have included a statement
in our report that foreshadows our conclusions in this report and
Comment (Mental Health Organization): Lack of attention to pending recommendations.
stigma and the diversity of stigma among the focus groups. The
focus groups also had a limited frame of questions with regard In particular, the stigma that individuals or family members
to uncovering the diversity of stigma. experience while attempting to identify mental health issues, or that
79
Comment (Advocate/Consumer): .I initially felt sad as I read consumers and their families might experience as they move within
the comments because it seemed people were looking down the mental health system, require acknowledgement and redress.
from the sky upon the MH system and assuming they knew the
"best assessment method, use of funds or "fix" for this problem. In our limitations section we state:
I wanted to remind everyone that it’s far more complicated --
historically, emotionally and ethically. How do we maintain Because this report used qualitative methods and a small sample, it
compassion for those of us charged with this kind of study and cannot be generalized by extension to other populations.
work to support the uncovering and goals by asking the right Specifically, its reliance on volunteers and recruiting participants
questions to gain more clarity or insight and see this as a using the snowball sampling or word-of-mouth method may have
collective community effort? I am merely trying to give more limited the variability of participants. Although CBPR strategies are
context to why the study is important -- to hopefully capture the currently among the best methods for eliciting the viewpoints of
feelings behind the numbers and/or the depth/ contextual factors diverse and marginalized communities, it is difficult to estimate the
of disparities that exist. amount of time needed to obtain the level of trust and rapport
necessary to enlist specific groups who experience high levels of
mental health stigma (e.g., LGBTQ, those who suffer from serious
Comment (Administrator): I enjoyed reading your report and
mental illness, or those whose trust may have been betrayed in
found that the findings seem to indicate that after the first years previous research or by government interventions, such as African
of MHSA, the funding is starting to achieve its goals, but that Americans or ethnically marginalized communities who have
there is more needed to reach out and educate those on the immigrated to the U.S.) in research-related endeavors similar to this
report.
services throughout the state and increase access for each
group to receive services by designing specific teams for their
Another limitation was the fact that although a small number of key
populations.
informants reported also being consumers of mental health
services for themselves or a family member, we did not report their
Comment (Administrator): Person living with a serious
perspective as a consumer of mental health services; we rather
mental illness defined them in terms of the primary stakeholder status with which
they identified (e.g., administrator, advocate, policy maker,
Comment (Administrator): Family member of a person living researcher). Because we did not specifically ask key informants to
with a serious mental illness self-report, whether or not they were the family member of a person
living with a serious mental illness or if they had a personal history
Comment (Provider): The many secrets that exist in our lives of receiving mental health services, we cannot provide precise
often make us become ill it does not help when there are more counts of the number of family members interviewed. In addition,
several focus group participants described experiences related to
secrets involved about our mental health. Our individual needs
care seeking for relatives living with a serious mental illness; some
are often half met and the rest is often given a brief view and not
of these experiences were reported in the findings above. While we
taken into account.
were able to reach theoretical saturation (and redundancy in
themes) related to common barriers to receiving appropriate mental
health services for un(der)served populations, a major finding of
from our focus group interviews was that the unique intersection of
80
those barriers magnify the impact of the barriers in a manner which
may be specific to distinctive populations. For example, although
un(der)served subgroups may experience some of the same
persistent barriers, the junction of those barriers (e.g., lack of
insurance coupled with the stigma of sexuality (LGTBQI); insurance
barriers and the physical disabilities of aging (Older Adults);
insurance and linguistic barriers (Latino immigrants), augment the
impact of the barriers for the affected groups.
In future research efforts using CBPR, more flexible time
parameters, added funding, and expanded community-based
support may result in increased representation from hard-to-reach
and highly-stigmatized communities, such as those persons living
with severe mental illness, LGBTQ, African American communities,
and those persons who identify as “non-Hispanic White” with
distinct ethnicity or heritage, whose perspectives and specific
barriers may not have been adequately represented in this report
but provide for investigation of how the intersection of barriers that
shared across diverse populations might differently impact
particular groups.
Throughout our report, we have pointed to the impact of stigma on
underserved communities, but we have yet to understand the full
extent and scope of the role stigma plays in diverse communities
on limiting access to care or diminishing quality of care or efficacy.
Additional efforts must be made to assess the way in which stigma
affects diverse underserved communities. Similarly, interviewees
described the impact of multiple barriers to receipt of effective care
or access to adequate treatment. While we have highlighted some
of these intersections, future evaluations need to expand on the
questions and issues we outline in this report in order to critically
detail and clarify the impact of intersecting barriers and identify
ways to remediate them. This report may serve as a point of
clarification for some stakeholder perspectives and perhaps a point
for framing questions about groups that this report may not have
been adequately represented. This report also offers further
opportunity for formulating informed questions about California
mental health services and for establishing a benchmark against
which to measure future initiatives and improvements.
81
Comment (Administrator): One barrier I typically find is that Response (Research question 1): Thank you for your comment.
even with education to the community, it is difficult to deliver We have included a statement in our report that foreshadows our
services to those with the most need with MHSA because of the conclusions in this report and pending recommendations.
limits on inpatient funding. The clients will come into inpatient
service providers and then leave with no follow up or continued Key informants expressed concern that data collection activities do
services within the MHSA continuum of care. not document the diversity of participants who are served by FSPs
and, therefore, suppress understanding of the scope and impact of
Comment (Family Member/Advocate): You state in your services provided for un(der)served groups, most particularly those
report that “The Mental Health Services Act (MHSA or Act) highly marginalized groups whom these services may not reach.
provides an unparalleled opportunity to serve children, youth,
adults, older adults, and families with mental health needs.” In our limitations section we state:
Unfortunately, this money, which was intended for the most
seriously mentally ill somehow never seems to quite get to this Because this report used qualitative methods and a small sample, it
population, probably because the money is being wasted on cannot be generalized by extension to other populations.
meaningless studies such as this one, along with programs for Specifically, its reliance on volunteers and recruiting participants
the worried well.
using the snowball sampling or word-of-mouth method may have
limited the variability of participants. Although CBPR strategies are
Comment (Administrator): Person living with a serious mental currently among the best methods for eliciting the viewpoints of
illness
diverse and marginalized communities, it is difficult to estimate the
amount of time needed to obtain the level of trust and rapport
necessary to enlist specific groups who experience high levels of
mental health stigma (e.g., LGBTQ, those who suffer from serious
mental illness, or those whose trust may have been betrayed in
previous research or by government interventions, such as African
Americans or ethnically marginalized communities who have
immigrated to the U.S.) in research-related endeavors similar to this
report.
Another limitation was the fact that although a small number of key
informants reported also being consumers of mental health
services for themselves or a family member, we did not report their
perspective as a consumer of mental health services; we rather
defined them in terms of the primary stakeholder status with which
they identified (e.g., administrator, advocate, policy maker,
82
researcher). Because we did not specifically ask key informants to
self-report, whether or not they were the family member of a person
living with a serious mental illness or if they had a personal history
of receiving mental health services, we cannot provide precise
counts of the number of family members interviewed. In addition,
several focus group participants described experiences related to
care seeking for relatives living with a serious mental illness; some
of these experiences were reported in the findings above. While we
were able to reach theoretical saturation (and redundancy in
themes) related to common barriers to receiving appropriate mental
health services for un(der)served populations, a major finding of
from our focus group interviews was that the unique intersection of
those barriers magnify the impact of the barriers in a manner which
may be specific to distinctive populations. For example, although
un(der)served subgroups may experience some of the same
persistent barriers, the junction of those barriers (e.g., lack of
insurance coupled with the stigma of sexuality (LGTBQI); insurance
barriers and the physical disabilities of aging (Older Adults);
insurance and linguistic barriers (Latino immigrants), augment the
impact of the barriers for the affected groups.
In future research efforts using CBPR, more flexible time
parameters, added funding, and expanded community-based
support may result in increased representation from hard-to-reach
and highly-stigmatized communities, such as those persons living
with severe mental illness, LGBTQ, African American communities,
and those persons who identify as “non-Hispanic White” with
distinct ethnicity or heritage, whose perspectives and specific
barriers may not have been adequately represented in this report
but provide for investigation of how the intersection of barriers that
shared across diverse populations might differently impact
particular groups.
Throughout our report, we have pointed to the impact of stigma on
underserved communities, but we have yet to understand the full
83
extent and scope of the role stigma plays in diverse communities
on limiting access to care or diminishing quality of care or efficacy.
Additional efforts must be made to assess the way in which stigma
affects diverse underserved communities. Similarly, interviewees
described the impact of multiple barriers to receipt of effective care
or access to adequate treatment. While we have highlighted some
of these intersections, future evaluations need to expand on the
questions and issues we outline in this report in order to critically
detail and clarify the impact of intersecting barriers and identify
ways to remediate them. This report may serve as a point of
clarification for some stakeholder perspectives and perhaps a point
for framing questions about groups that this report may not have
been adequately represented. This report also offers further
opportunity for formulating informed questions about California
mental health services and for establishing a benchmark against
which to measure future initiatives and improvements.
Comment (Provider): Our medical emergency department is Response (Research question 2): Thank you for helping to clarify
impacted with psychiatric patients who are denied services some of the issues described by our Key Informants regarding use
supported by MHSA funds because they on an involuntary hold. of emergency services by those impacted by serious mental health
They are too acutely ill to receive psychiatric services from the issues or who are “in crisis.” We have added some detail to our
Agencies that are the recipients of these funds. By default these report to clarify our discussion.
acutely ill psychiatric patients become the responsibility of
medical community, not psychiatrists or anyone with psychiatric Descriptions of “inappropriate care” were common for informants
experience. They are held without treatment in our medical who discussed institutional settings as the primary locale for the
emergency rooms. This is the story of the misuse of these delivery of mental health-related services. Mental health providers,
resources and the lack of psychiatric services for the mentally ill consumers, and families of people who have serious mental illness
in the majority of communities. noted that emergency departments, while not the best alternative
for mental health care, are often the only type of service available
to people with serious mental illness or in crisis.
Comment (Provider): We are facing a very real crisis in Response (Research question 2): Thank you for helping to clarify
California and that is a shortage of psychiatrists. Every day some of the issues described by our Key Informants regarding use
of emergency services by those impacted by serious mental health
individuals come to our emergency departments in crisis. The
issues or who are “in crisis.” We have added some detail to our
only way that they will be able to see a psychiatrist within a
report to clarify our discussion.
reasonable period of time is to be placed on a 5150 hold. Once
this happens they are no longer eligible for services funded by Many informants described the inappropriate use of services and
untrained staff for some un(der)served groups, including those with
84
the MHSA. serious mental illness, older adults with comorbid conditions, and
African Americans. Interviewees’ examples related to the treatment
of African Americans illustrates that although un(der)served groups,
like African Americans, do receive mental health services, the
treatment they receive is not entirely appropriate.
Comment (Administrator): This holds also true for older
adults…older adults tend to access mental health services in
ER, hospital and or in Primary Care.
Comment (Administrator): What about workforce Response (Research question 2): Thank you for bringing our
issues…wondering if these issues where discussed by the attention to this very important point. We have added a brief
section about workforce planning and development based on some
various informants? There is no way to deliver the type of
of our interview data and the valuable public comments provided by
services that will help reduce disparities without a well trained
stakeholders.
workforce.
Planning for and development of an adequate and well-trained
Comment (Advocate/Consumer): This is why there was an
workforce. Key Informants and stakeholders all requested more
MHSA audit on intent of purpose, misappropriation of funding, proactive strategies for responding to the needs of people with
better outcomes of reducing disparities, integrating and blending serious mental illness, including planning for and development of a
consumers in the workforce as provided by MHSA design- this is more adequate and well-trained workforce in settings where people
“in crisis” or living with serious mental illness could best be served.
how law was legislated. Surprising results when MHSA helps
Specific barriers to developing an adequate workforce to serve
with housing, job, self-determination. With encouragement and
mental health needs in California is highlighted by this stakeholder
tools even seriously ill population can integrate and recover.
comment:
“The MHSA is assistive in many areas of
administration [but] the problem persists in the hiring
85
Comment (Provider): The MHSA is assistive in many areas of of individuals with lived experience. Although there
administration the problem persists in the hiring of individuals are some positions available, there is no end in sight
for those of us working on the front lines with limited
with lived experience and although there are some positions
time to go to training that is time consuming and
available there is no end in sight for those of us working on the
often becomes obsolete. The reason for the funding
front lines with limited time to go to training, that is time
is to assist the community with more alternatives
consuming and often becomes obsolete. The reason for the than incarceration and other forms of restraint.”
funding is to assist the community with more alternatives than (Public commentary, February 19, 2014)
incarceration and other forms of restraint. Further obstacles are
identified when the conferences and trainings are specified for In addition, some stakeholders have noted that parameters for the
use of MHSA funding and the availability of treatment for people
doctors only. The many secrets that exist in our lives often
living with serious mental illness highlight the lack of community-
make us become ill it does not help when there are more
based psychiatric services, including adequate staffing or
secrets involved about our mental health. Our individual needs
appropriate training for primary care and other providers who
are often half met and the rest is often given a brief view and eventually care for patients who suffer with mental illness. Family
not taken into account. More is better but not always quality members and mental health care providers noted the absence of
with the overloading of our staff and colleagues. Sounds like I training for personnel who are not licensed physicians or
psychiatrists.
am ranting about the many areas of concern and it is just that a
concern for the many lives it touches when we confront the
well-being of our community. Would like to thank you for efforts
in this and many other areas you have helped out.
Comment (Provider): Funding from the MHSA would serve
those in crisis most efficiently if it were to go to relieve our
medical emergency departments from the burden of holding
these individuals in an environment that is uniquely ill-suited for
the initiation of treatment. If MHSA funds were accessible to
serve patients in all phases of their illness I could support of
those efforts.
Comment (Advocate): Consideration should be given that Response (Research question 2): Thank you for making this
some individuals of ethnic backgrounds may not want to get important point. We have included a comment in our report
care from someone from their own community because of word regarding sensitivity to issue.
getting out in their community circles; they'd rather seek care
from someone outside of their cultural circle yet who has The importance of a culturally competent workforce cannot be
awareness about their cultural background, history of heritage, overstated, and should be informed with sensitivity to the fact that
86
and sensitivity about generational values and roles. individuals in some ethnic communities may choose to seek care
outside of their communities due to the potential for stigmatization.
Comment (Advocate): Another barrier is messaging, how the Response (Research question 2): Thank you for this comment.
message about mental health is delivered to cultural, We have noted in our report that messaging is an important
underserved communities (e.g., CHAA in Alameda County: component relating to awareness and potential engagement. We
Community Health for Asian Americans does not have "mental" have added the following statement:
in its name - this matters in how people view CHAA and its
approach to community, so the name isn't laden with stigma). Stakeholders offering public comment about the draft of this report
Vigilance around messaging about mental health is important, suggested that specific strategies related to messaging, along with
paying attention so that the language/terms do not alienate nor communication by health services providers, could yield potentially
stigmatize. effective methods for increasing awareness of MHSA-funded
services and engaging un(der)served communities.
Comment (Advocate): If focus group participants had no
awareness about MHSA principles and values, a provider Some stakeholders suggested that providers and organizations
receiving MHSA funding bears responsibility to inform and who receive MHSA funding should be responsible for informing
educate recipients about core values of wellness, resiliency, consumers about the source of funds for that care and the values
recovery and the power of hope to pave pathway to quality and that support its funding. Lack of awareness related to resources for
dignity in life. This can do a lot for a person's engagement, mental health care can disempower communities. Greater
success and motivation. awareness of available services and their funding sources not only
provides community members with information about the availability
of resources in their local community but may also allow them
avenues for input in the type and delivery of services. In response
to findings that focus group participants were not aware of MHSA
principles and values, one stakeholder commented:
“If focus group participants had no awareness about
MHSA principles and values, a provider receiving
MHSA funding bears responsibility to inform and
educate recipients about core values of wellness,
resiliency, recovery and the power of hope to pave
[a] pathway to quality and dignity in life. This can do
a lot for a person's engagement, success and
motivation.” (Public commentary, March 6, 2014)
87
Comment (Advocate/Provider): Well, as a senior peer Response (Research question 2): We appreciate these
counselor in Sonoma County our program (funded under comments. We note that they reinforce the value of PEI programs,
Prevention and early intervention by MHSA) has doubled both in and they also underscore problems related to inconsistent tracking
the number of seniors we see and in the number of volunteers of these types of programs. We added the following statement to
who are trained for this important work. County and other social our analysis:
workers say that the impact we have helping seniors with the
vicissitudes of aging including depression, drug and/or alcohol Stakeholders offering public comment have noted that some county
counseling, and knowledge of community resources makes our programs have seen an increase in consumers served by the
seniors more hopeful and positive and able to conduct their lives mental health system and the number of volunteers trained to
in a more hopeful manner with greater independence. We get assist older adults with comorbid conditions. The successes of
extremely positive feedback from exit surveys. I can't give you some of these new programs under PEI, however, have been
names and numbers but I am sure even those would not satisfy overshadowed by inconsistent efforts at tracking the success of the
your cynicism. programs.
88
Comment (Administrator): A FEW EXAMPLES OF WHAT WE
DO NOT KNOW AFTER STEINBERG REVIEW:
Are any additional services in the core Systems of Care
funded by MHSA, as intended by voters and explained
by the Legislative Analyst in 2004 ballot?
Are counties funding MHSA PEI programs for people
with serious mental illnesses to prevent illnesses from
becoming more severe and disabling?
How much money are counties spending on Systems of
Care and PEI direct services and how much on
overhead, administrators, conferences, consultants,
studies, and other products?
What Systems of Care services are most in demand by
consumers and families, and what is county capacity to
meet that need?
How has the county integrated MHSA components and
core programs, funded improvements in cultural
competency and co-occurring disorder services,
reduced waiting lists for children’s wraparound
programs?
Has the county developed self-help programs, reduced
caseloads for service coordinators and physicians,
added individual or group counseling options?
Comment (Administrator): this includes age, condition and Response (Research question 3): We appreciate concerns
culture/linguistic expressed related to the generalizability of our findings and have
provided more specific rationale for our selection of focus group
Comment (Administrator): This could be compared to the participants. In addition, we have added more detail to our
number and demo characteristics of informants discussion of the limitations of this report. We have added the
following description to those selected for focus group participation:
89
Comment (Administrator): A big problem I see for this report Our goal in conducting focus groups was to understand the ways in
is the credibility for readers across the state. The amount of which specific barriers prevent access to diverse un(der)served
people interviewed and the limited regions surveyed may cause groups. To accomplish this, we focused the viewpoints of key
those to not relate since it is based on such a small population. informants on the major barriers to service for un(der)served
The interviewee amount was minimal and central to certain groups. We then examined these barriers, focusing on those that
areas, not truly representative of the state and counties. If I were were described as affecting the greatest number of persons, and
a reader I may say, "well that's nice to read 30 individuals views cut across diverse groups. The obstacles we found to be most
in 3 counties, but it doesn't help our area with the disparities significant, based on our interviews with key informants and review
here because of our populations, cultures, groups, etc... are of existing documents, were barriers due to the degree of perceived
different, unique, reached, etc.." stigma in accessing mental health services, immigrant status,
linguistic and cultural barriers, age-related difficulty in accessing
Comment (Advocate/Consumer): Offhand I believe qualitative appropriate care or shortcomings in mechanisms for engaging
statistics, not measuring the survey numbers, not quantitative appropriate care, and geographic availability of resources for
methods were applied to conclude that Reducing Disparities mental health care (see Table 3). Next, we identified three specific
targets were being met. It is helpful to know what other agency un(der)served groups, whose experiences reflected a combination
responses, (see Orange County) are to this report which of these barriers.
appears to completely miss the point in its unbalanced
conclusion.
Comment (Administrator): You list three “un(der)served
groups” in your report but left out the most important
underserved group – those with a serious mental illness;
schizophrenia, bipolar disorder or major depression.
Comment (Advocate): Another barrier is messaging, how the Response (Research question 3): Thank you for this comment.
message about mental health is delivered to cultural, We have added the following statement to draw attention to the
underserved communities (e.g., CHAA in Alameda County: importance of messaging in awareness and engagement:
Community Health for Asian Americans does not have "mental"
in its name - this matters in how people view CHAA and its Adequate attention and sensitivity to stigma in public policy and
approach to community, so the name isn't laden with stigma). messaging related to issues of mental health and serious mental
Vigilance around messaging about mental health is important, illness may, in fact, provide a starting point for enhancing
paying attention so that the language/terms do not alienate nor awareness and engaging relevant persons in appropriate mental
stigmatize. health services and treatment.
90
Comment (Administrator): there is almost no Response (Research question 3): Thank you for bringing up this
acknowledgement of disabilities apart from mental health or issue. There were some discussions regarding co-morbid
drug and alcohol abuse disabilities. You and I both know that conditions in our Older Adult group, and we have added a
co-morbidity when someone has a disability is high – it’s more statement related to transportation and these issues, to draw
common to have mental health *and* physical, sensory, etc. attention to the importance of this barrier and access to care for
than not. Access to services is critical and hard to come by. those facing multiple conditions (the statement we added is
One of my early work assignments in 1996 was facilitating a provided below). In addition, we will consider this in our discussion
support group for heroin addicts who were persons living with of recommendations (currently under development).
spinal cord injuries. Most of them had high level injuries limiting
the use even of their hands. These people had a hard time Stakeholders offering public comments underscored that
finding treatment programs that could accommodate their counteracting limitations in transportation is not a simple matter but
needs, increasing their relapses. People with physical would go a long way to remediate access impediments for
disabilities are statistically more likely to experience clinical un(der)served populations, particularly the most marginalized
depression than those without disabilities. People who are Deaf groups, such as people living in rural areas, those with a serious
have trouble getting sign language interpreters in any kind of mental illness, and older adults.
medical or treatment setting. In the evaluation’s discussion of
language and access issues, this conversation was glaringly
absent and reflects a narrow perspective that does not serve the
public well.
Comment (Advocate): Effectiveness of a progressive public Response (Research question 3): Thank you for this comment; it
mental health system depends on stakeholder involvement and has helped us to frame some of the issues expressed in our
the importance of representation from un(der)served groups - immigrant group and older adult group. In addition, it is an
this calls for awareness of and respect for a community/culture's important issue that merits consideration for our pending
history (i.e., historical trauma, displacement, etc.) and values, recommendations. The statement we added to elaborate on the
including gender/generational roles. experiences described by our focus group participants is as follows:
Other populations who might experience vicarious or other trauma
that could create barriers to receiving adequate mental health care
include immigrant groups, many of whom have experienced
historical trauma.
Comments (Provider): More is better but not always quality Response (Research question 3): We have included the following
with the overloading of our staff and colleagues. comment as part of our attention to the workforce and competency
issue:
Stakeholders who commented on the “fit” between consumers and
their providers suggested that adequate training of diverse
community professionals could enhance the quality of service for
91
the most un(der)served groups, including marginalized ethnic
groups and those with serious mental illness, in addition to persons
who participated in our focus group discussions.
Deliverable 2c
Report Sections Comments/Feedback Received Changes Adopted
V. Conclusion Comment (Family Member/Advocate): Emphasize the overall Response: Thank you for your insights. We have added two
Section lack of effort so far to develop either community-based or sentences to underscore these points, and will aim toward making
evidence-based standards for evaluation of the implementation recommendations in a later report that draw attention to this
of these strategies. How will we ever know if the strategies suggestion.
work, or under what conditions they work, if no efforts are made
to develop standards and evaluate the programs utilizing the Confusion remains, however, about the specific contents of some
strategies? You address the issue of poor data collection on MHSA programs and a lack of systematic and sustained activities
WHO receives FSP services, but it may be even more critical to to evaluate their impact. We will not fully understand which types of
understand WHAT these FSP programs consist of, and how strategies work, or under what conditions they work, if no efforts are
effective they are at providing those services to a given made to develop community-based or evidence-based standards
audience for evaluation of the implementation of these strategies.
Comment (Commissioner): Previous efforts to engage Response: Thank you for these comments, we have attempted to
stakeholders have not been met with active uptake of acknowledge them in our conclusions by adding the following:
recommendations or have not made recommendations that
have been, according to public comment, actionable policy The findings of this report suggest that MHSA has and continues to
measures for the most marginalized of these groups. have an impact on reducing disparities among un(der)served
groups. The findings indicate that key informants recognize
Comment (Advocate): Annual updates with stakeholder input is MHSA’s commitment to invest in resources that build the capacity
not reviewed carefully enough or, maybe minimally, by local of un(der)served communities. For example, informants
Boards nor honored as meaningful input by leadership. Including acknowledged Prevention and Early Intervention (PEI) and Full
it as written form with seemingly glib responses in the update Service Partnerships (FSPs) as two major examples that
discounts and often negates stakeholder involvement. strengthen the linkages between un(der)served groups and access
to care. Furthermore, we have learned through this study that
MHSA must do more to increase culturally competent programs
and community outreach and engagement in order to ensure that
services are adequate and appropriate for targeted un(der)served
groups.
92
Comment (Provider): The MHSA is assistive in many areas of Response: Thank you for these comments, we have attempted to
administration the problem persists in the hiring of individuals acknowledge them in our conclusions by adding the following:
with lived experience and although there are some positions
available there is no end in sight for those of us working on the MHSA also promotes the transformation to a recovery-focused
front lines with limited time to go to training that is time system of care, which is targeted to individuals living with serious
consuming and often becomes obsolete. The reason for the mental illness who are not receiving enough support to alleviate
funding is to assist the community with more alternatives than their risk of homelessness, incarceration, or hospitalization.
incarceration and other forms of restraint. Further obstacles are
identified when the conferences and trainings are specified for
doctors only. The many secrets that exist in our lives often make
us become ill it does not help when there are more secrets
involved about our mental health. Our individual needs are often
half met and the rest is often given a brief view and not taken
into account. More is better but not always quality with the
overloading of our staff and colleagues. Sounds like I am ranting
about the many areas of concern and it is just that a concern for
the many lives it touches when we confront the well-being of our
community. Would like to thank you for efforts in this and many
other areas you have helped out.
Comment (Provider): Funding from the MHSA would serve
those in crisis most efficiently if it were to go to relieve our
medical emergency departments from the burden of holding
these individuals in an environment that is uniquely ill-suited for
the initiation of treatment. If MHSA funds were accessible to
serve patients in all phases of their illness I could support of
those efforts.
Comment (Advocate/Consumer): This is why there was an
MHSA audit on intent of purpose, misappropriation of funding,
better outcomes of reducing disparities, integrating and blending
consumers in the workforce as provided by MHSA design- this is
how law was legislated. Surprising results when MHSA helps
with housing, job, self-determination. With encouragement and
tools even seriously ill population can integrate and recover.
93
Comment (Administrator): …given the qualitative nature of Response: We appreciate your comments. We have added a
this study carving out a significant amount of time to build sentence to clarify the generalizability of our findings and to make
relationships, community, and trust is essential and critical in note of this.
working with un(der)served populations. This is especially true if
they have a historical mistrust of others that may come from Although CBPR strategies are currently among the best methods
government and academia due to racism and discrimination. Not for eliciting the viewpoints of diverse and marginalized
taking the time to build relationships and trust with the communities, it is difficult to estimate the amount of time needed to
community may explain why there were only four LGBTQ TAY obtain the level of trust and rapport necessary to enlist specific
participants who all identified as white (pg. 14). Perhaps I would groups who experience high levels of mental health stigma (e.g.,
suggest making mention of the need to build relationships and LGBTQ, those who suffer from serious mental illness, or those
trust in the discussion part of the report...pages 45-46 whose trust may have been betrayed in previous research or by
government interventions, such as African Americans or ethnically
marginalized communities who have immigrated to the U.S.) in
research-related endeavors similar to this report … In future
Comment (Administrator): I’ll like to note that the Asian/Pacific research efforts using CBPR, more flexible time parameters, added
Islander (API) population are underserved in many counties funding, and expanded community-based support may result in
increased representation from hard-to-reach and highly-stigmatized
across the State and it is very critical they should have been
communities, such as those persons living with severe mental
included in informing the report. Also, if the premise of holding a
illness, LGBTQ, African American communities, and those persons
focus group for Older Adults was because they have little
who identify as “non-Hispanic White” with distinct ethnicity or
access to care then why wasn’t the focus group held with Older heritage, whose perspectives and specific barriers may not have
Adults themselves as opposed to the caregivers? been adequately represented in this report but provide for
investigation of how the intersection of barriers that shared across
diverse populations might differently impact particular groups.
94
Comment (Administrator): This report is very valuable Response: Thank you for your valuable and insightful comments.
considering the current political climate around MHSA services. We have added the following sentence to clarify:
Overall, this report does an impressive good job of highlighting
Although CBPR strategies are currently among the best methods
the good work that MHSA does in reducing disparities.
for eliciting the viewpoints of diverse and marginalized
communities, it is difficult to estimate the amount of time needed to
My first comments are about the emphasis placed on CBPR as
obtain the level of trust and rapport necessary to enlist specific
an evaluation method in this report. My understanding of CBPR
groups who experience high levels of mental health stigma (e.g.,
methods may differ than the authors of the report. As I
LGBTQ, those who suffer from serious mental illness, or those
understand it, community members have a very active and
whose trust may have been betrayed in previous research or by
leading role throughout the evaluation process, including
government interventions, such as African Americans or ethnically
inception, design, implementation, evaluation, & dissemination.
marginalized communities who have immigrated to the U.S.) in
Therefore, CBPR would have to be much more than interviewing
research-related endeavors similar to this report … In future
community members and having them provide input on the
research efforts using CBPR, more flexible time parameters, added
selection of key stakeholders and content of the interview guide.
funding, and expanded community-based support may result in
Accordingly, a CBPR evaluation project would hire and train
increased representation from hard-to-reach and highly-stigmatized
consumers from the target population and have them lead, to
communities, such as those persons living with severe mental
the fullest extent possible, the evaluation process. Consumers
illness, LGBTQ, African American communities, and those persons
have a much more significant role in the research process.
who identify as “non-Hispanic White” with distinct ethnicity or
heritage, whose perspectives and specific barriers may not have
been adequately represented in this report but provide for
investigation of how the intersection of barriers that shared across
diverse populations might differently impact particular groups.
95
Comment (Commissioner): Previous efforts to engage Response: We acknowledge these comments, which suggest that
stakeholders have not been met with active uptake of recommendations from the UCLA report related to stakeholder
recommendations or have not made recommendations that engagement were not translated into actionable items or have not
have been, according to public comment, actionable policy been meaningfully honored. In the conclusion to our report, we
measures for the most marginalized of these groups. added the following:
Diverse stakeholders who offered critiques and suggestions
through interviews or comments on a draft of this report during a
period of public feedback observed that they have not seen many
calls for stakeholder engagement, as the UCLA report had
recommended. To date, stakeholder engagement has not been
translated into an “actionable item” or meaningfully honored. We
view this report as a step toward increasing the involvement and
feedback of stakeholders. As such, we encourage the continued
integration of stakeholder perspectives, so that diversity in
perspectives drives the conversation about community-informed
access to care and treatment. The importance of eliciting and using
Comment (Advocate): Annual updates with stakeholder input is stakeholder feedback emerged as a consistent theme throughout
not reviewed carefully enough or, maybe minimally, by local this report as a significant means to help MHSA continue
Boards nor honored as meaningful input by leadership. Including transforming mental health services that mirror the composition of
it as written form with seemingly glib responses in the update un(der)served groups in California.
discounts and often negates stakeholder involvement.
Comment (Administrator): What about documentation on
consumer/stakeholder level of involvement: sporadic vs on-
going ; providing input, member of advisory, or involvement is
on-going and participates in collaborative a decision making
process?
96
Comment (Administrator): What about workforce Response: Thank you for these comments, we have attempted to
issues…wondering if these issues where discussed by the acknowledge them in our conclusions by adding the following:
various informants? There is no way to deliver the type of
services that will help reduce disparities without a well trained At the system level, improvements must be made in culturally
workforce. competent treatment, outreach for underserved groups, and
provision of care in appropriate settings through planning and
development of a well-trained mental health services workforce,
encompassing a range of mental health service providers. Large
county mental health programs as well as smaller community-
based organizations continue to experience difficulty in obtaining
adequate funding to respond to these needs and to systematically
monitor expenditures.
Comment (Advocate): Stigma experienced by individuals as Response: Thank you for these comments, we have attempted to
they move within the system is a barrier that needs to be acknowledge them in our conclusions by adding the following:
constantly addressed. Although FSPs have had an impact, it
has not been well documented or reported -- and for Strategies for response need to also include appropriate evaluation
underserved/under represented, how is public stigma reduction and monitoring of existing programs, such as PEI and FSPs,
in the general community measured? proposed to expand their services to un(der)served populations.
Comment (Administrator): A big problem I see for this report is Response: Thank you for these comments, we have attempted to
the credibility for readers across the state. The amount of people acknowledge them in our conclusions by adding the following:
interviewed and the limited regions surveyed may cause those
Because this report used qualitative methods and a small sample, it
to not relate since it is based on such a small population. The
cannot be generalized by extension to other populations.
interviewee amount was minimal and central to certain areas,
Specifically, its reliance on volunteers and recruiting participants
not truly representative of the state and counties. If I were a using the snowball sampling or word-of-mouth method may have
reader I may say, "well that's nice to read 30 individuals views in limited the variability of participants. Although CBPR strategies are
3 counties, but it doesn't help our area with the disparities here currently among the best methods for eliciting the viewpoints of
diverse and marginalized communities, it is difficult to estimate the
because of our populations, cultures, groups, etc... are different,
amount of time needed to obtain the level of trust and rapport
unique, reached, etc.."
necessary to enlist specific groups who experience high levels of
mental health stigma (e.g., LGBTQ, those who suffer from serious
Comment (Administrator): Facilitating three (3) focus groups mental illness, or those whose trust may have been betrayed in
with a total of forty (40) individuals to inform the impact of a previous research or by government interventions, such as African
Statewide policy/act is not sufficient enough to adequately depict Americans or ethnically marginalized communities who have
immigrated to the U.S.) in research-related endeavors similar to this
the true impact of MHSA on the un(der)served population.
report.
97
Comment (Administrator): I’ll like to note that the Asian/Pacific
Islander (API) population are underserved in many counties Another limitation was the fact that although a small number of key
informants reported also being consumers of mental health
across the State and it is very critical they should have been
services for themselves or a family member, we did not report their
included in informing the report.
perspective as a consumer of mental health services; we rather
defined them in terms of the primary stakeholder status with which
Comment (Mental Health Organization): A surprisingly small
they identified (e.g., administrator, advocate, policy maker,
N = 4 for the LGBT/Q group, all of whom were female, seems researcher). Because we did not specifically ask key informants to
non-representative. Also, numbers for persons of African- self-report, whether or not they were the family member of a person
American, American Native, or Asian/Pacific Islander living with a serious mental illness or if they had a personal history
of receiving mental health services, we cannot provide precise
race/ethnicity seem to be insufficient for a well-designed study.
counts of the number of family members interviewed. In addition,
Although there were some persons of Hispanic ethnicity, we
several focus group participants described experiences related to
question whether there were adequate numbers to be
care seeking for relatives living with a serious mental illness; some
representative of this group, as the total persons interviewed in of these experiences were reported in the findings above. While we
the study for Objective #3 remains quite small. Numbers of were able to reach theoretical saturation (and redundancy in
persons in each age category of interviewees seems to be not themes) related to common barriers to receiving appropriate mental
adequate. health services for un(der)served populations, a major finding of
from our focus group interviews was that the unique intersection of
those barriers magnify the impact of the barriers in a manner which
may be specific to distinctive populations. For example, although
un(der)served subgroups may experience some of the same
persistent barriers, the junction of those barriers (e.g., lack of
insurance coupled with the stigma of sexuality (LGTBQI); insurance
barriers and the physical disabilities of aging (Older Adults);
insurance and linguistic barriers (Latino immigrants), augment the
impact of the barriers for the affected groups.
Comment (Mental Health Organization): The intersection of Response: Thank you for your insight. We want to be careful not
multiple barriers. The LGBTQ focus group is described as to over generalize our findings, yet we acknowledge the
identifying issues that represent the “complex intersection of significance of this finding for each of three groups we interviewed.
their age and sexuality in addition to other factors.” But what We have added the following to our discussion of limitations:
about the intersection of age, race, gender and other factors
Although CBPR strategies are currently among the best methods
with regard to the other focus groups? Barriers specific to Latino for eliciting the viewpoints of diverse and marginalized
men are mentioned briefly on page forty. Moreover, the older communities, it is difficult to estimate the amount of time needed to
adult focus group is the most racially diverse of all the groups, obtain the level of trust and rapport necessary to enlist specific
but this diversity is not represented in this report. There is only groups who experience high levels of mental health stigma (e.g.,
LGBTQ, those who suffer from serious mental illness, or those
one mention of low-English proficiency in the older adult focus
whose trust may have been betrayed in previous research or by
9 8
group. government interventions, such as African Americans or ethnically
marginalized communities who have immigrated to the U.S.) in
research-related endeavors similar to this report … In future
research efforts using CBPR, more flexible time parameters, added
funding, and expanded community-based support may result in
increased representation from hard-to-reach and highly-stigmatized
communities, such as those persons living with severe mental
illness, LGBTQ, African American communities, and those persons
who identify as “non-Hispanic White” with distinct ethnicity or
heritage, whose perspectives and specific barriers may not have
been adequately represented in this report but provide for
investigation of how the intersection of barriers that shared across
diverse populations might differently impact particular groups.
Comment (Mental Health Organization): “Another thought: Response: Thank you for your inquiry into our recommendations.
Will (the authors) come up with a separate section on Our deliverables for the contract are presented in separate
recommendations?” documents, so that the quantitative (population-based) analyses
and final recommendations, which are based on both quantitative
The report suggests that both these statements are true: that
(population-based evaluation) and qualitative (stakeholder
MHSA funded programs are successful in reducing disparities
viewpoints and feedback) assessments, are presented as separate
through increased access of un(der)served communities to Full
deliverables in separate documents. Our recommendations are
Service Partnerships and expanded Prevention services; and
forthcoming and will be reported in Deliverable 3.
that disparities in access persist because of unaddressed
barriers such as language/cultural issues, costs, transportation,
and other difficulties. The report does not outline a strategy to
use MHSA program funds to mitigate these difficulties.
Comment (Mental Health Organization): We observe that the Response: We thank you for this comment and your insight. We
State of California invested years of effort and millions of dollars also view this report as a starting point for formulating better
of Prevention and Early Intervention funds in the Reducing questions about California mental health services and as a
Disparities Project (CRDP). Working with community groups benchmark against which to measure future initiatives and
throughout the State, the CRDP produced population reports improvements.
that provided an exhaustive discussion of the challenges facing
5 racial and cultural communities. Each of these reports
provided community defined solutions that would address the
many barriers to full access by these underserved communities
99
to public mental health services. The State will not have an
opportunity to implement any of these recommendations until
the full Strategic Plan to Reduce Disparities is finalized. So
while this evaluation is not as comprehensive as needed, the
Council recommends that it could be used as a benchmark to
measure future improvements resulting from the
implementation of CRDP strategies to reduce disparities in
these underserved communities.
Deliverable 2c General Comments/Feedback Received Changes Adopted
Response: Thank you for these comments. We utilized the
Comment (Mental Health Organization): “I briefly looked at the
services of a professional editor and shared these concerns with
report and began to wonder if this report is for legislators and
him. The editor has made the following changes:
professionals who are pretty savvy [about] the mental health
Revised technical jargon, when appropriate.
system. I understand that this report was asked by OAC in 2012
Has used the suggested terms (serious mental illness,
and that before it is sent to OAC, it is to go through a “public
person living with a serious mental illness, consumer, and
review”. Is this “public” intended to be a professional group that
family member of a person living with a mental illness)
is knowledgeable on mental health systems? My point is this:
throughout the document.
what is the grade level of this report? Will the “public” be able to
understand it? I find it scholarly, thanks to the authors, but I fear He also distinguished between consumers and family
members where appropriate.
that comprehension might be minimal.” “This is a HUGE issue
and complaint - this is written in academic language and folks
need an interpreter to understand what is being said and/or
written.”
Comment (Administrator): Are you anticipating “consumers of
mental health services” to read the report and respond? If so,
can it be translated into simple, everyday language in a shorter
number of pages?
Comment (Administrator): Would like to see a more proactive
word used than “effort” in providing a diverse workforce.
10 0
Comment (Administrator): Please do not join the terms
consumers and family member by saying that you interviewed
clients and/or family members, would like to see this separated
as consumers AND family member.
Comment (Administrator): Want to make sure the same
verbiage is being used throughout the document: either client or
consumer.
Comment (Administrator): Please use Person living with a
serious mental illness rather than (Severe mental illness when
appropriate) and Family member of a person living with a
serious mental illness.
10 1