BHSOAC
Eval MHSAStatewidePartEval InitialReport PhaseIIIDeliv2a 2And2b 2 103112
Read the report at Behavioral Health Services Oversight & Accountability Commission ↗
Mental Health Services Act (MHSA) Expanded Statewide Evaluation
MHSA STATEWIDE PARTICIPATORY
EVALUATION INITIAL REPORT
Phase III Deliverable 2a-2 and 2b-2
Submitted by:
UCLA Center for Healthier Children, Families & Communities
Prepared by:
Jane Yoo, PhD, MSW and Kristin J. Ward, PhD
Co-Principal Investigators
September 2012
This work was made possible through a contract with the
Mental Health Services Oversight & Accountability Commission (MHSOAC)
MHSA Statewide Participatory Evaluation
Evaluation Team
Jane Yoo, PhD, MSW
Kristin J. Ward, PhD
Todd Sosna, PhD
Todd M. Franke, PhD, MSW
Christina A. Christie, PhD
Ashaki Jackson, PhD
Jennifer Ho, EdM
Timothy Ho, MA
Laura Valles, MSW
Phina Li, MS
Colleen Janczewski, MSW
Celina Lee Chao, MA
Emily Chan
Participatory Evaluation Partners
Kamila Baker
Delphine Brody
Viviana Criado, MPA
Kathleen Grove Derby, MA
Steve Leoni
Hurley Merical and John Merical, BA
Raja Mitry
Luz Parra
Cici Romero
Doug Stephens
Author Acknowledgments
The MHSA statewide participatory evaluation was a large undertaking made possible by the
involvement of many individuals and organizations. We would like to first acknowledge the
individuals who participated in the statewide planning process for helping to map the course
for this evaluation. The planning process laid the foundation for building the valuable
relationship between the UCLA evaluation team and Participatory Evaluation Partners (PEPs).
We wholeheartedly thank each evaluation partner for her/his contributions to the participatory
process and acknowledge that the evaluation was stronger because of their dedication and
expertise.
The evaluation benefited from a number of individuals from provider agencies, county mental
health departments, and advocacy groups. They provided critical feedback on the survey, and
played an important role in recruiting participants for the planning process, survey, and
interviews.
Participatory evaluations present an important opportunity to involve individuals with lived
experience and their family members in the evaluation of services that impact their lives. We
appreciate the Mental Health Services Oversight and Accountability Commission for
incorporating the MHSA value of stakeholder involvement into their evaluation endeavors.
This evaluation study was a team effort. We are grateful to our colleagues at UCLA and Clarus
Research for helping to spearhead this first-ever statewide MHSA participatory evaluation.
We acknowledge that this study would not have been possible without the participation of
individuals with lived experience, their family members, and others who took the time to make
their voices heard through the survey or an interview. We thank all the study participants for
sharing their experiences with mental health services, and we are especially indebted to
interview participants for sharing their deeply personal stories with us. We are truly humbled
by their life experiences and heartened by their perseverance as they continue their recovery
journeys.
Table of Contents
Table of Contents ................................................................................................................................................ i
Table of Figures ................................................................................................................................................... v
Table of Tables .................................................................................................................................................. vii
EXECUTIVE SUMMARY ................................................................................................................................... ES-1
INTRODUCTION.................................................................................................................................................. 1
Description of Deliverables ....................................................................................................................................... 1
Overview of Study ..................................................................................................................................................... 2
Study Focus and Methods .................................................................................................................................... 2
Study Questions ................................................................................................................................................... 3
Report Organization ............................................................................................................................................. 4
PARTICIPATORY EVALUATION PROCESS ............................................................................................................. 5
Participatory Approach ............................................................................................................................................. 5
Participant Selection, Roles, and Responsibilities ................................................................................................ 5
Participatory Evaluation Partners and Their Roles in Evaluation Activities .............................................................. 6
METHODS .......................................................................................................................................................... 7
Survey Methods ........................................................................................................................................................ 7
Sample .................................................................................................................................................................. 7
Study Indicators ................................................................................................................................................... 7
Mental Health Services Act: Statewide Survey of Client Experience (SSCE) ........................................................ 8
Survey Distribution ............................................................................................................................................. 10
Analysis of Survey Data ...................................................................................................................................... 11
Interview Methods .................................................................................................................................................. 12
Sample ................................................................................................................................................................ 12
Interview Protocol .............................................................................................................................................. 13
Interview Participant Recruitment ..................................................................................................................... 13
Conducting Interviews ....................................................................................................................................... 14
Analysis of Interview Data .................................................................................................................................. 14
Human Subjects Review .......................................................................................................................................... 16
SURVEY RESULTS ............................................................................................................................................... 17
Peer Support Services .............................................................................................................................................. 17
Characteristics of Individuals Who Received Peer Support Services ................................................................. 17
Types of Peer Support Services Received .......................................................................................................... 27
Access to Peer Support Services ........................................................................................................................ 28
Recovery Orientation of Peer Support Services ................................................................................................. 30
Peer Support Services Impact: Employment, Housing, and Personal Recovery and Wellness .......................... 32
Employment Support Service .................................................................................................................................. 37
Characteristics of Individuals Who Received Employment Support Services .................................................... 37
Types of Employment Support Services Received ............................................................................................. 47
Access to Employment Support Services ........................................................................................................... 48
Recovery Orientation of Employment Support Services .................................................................................... 51
Table of Contents i
Employment Support Services Impact: Employment, Housing, and Personal Recovery and Wellness ............. 54
Crisis Intervention Services ...................................................................................................................................... 58
Characteristics of Individuals Who Received Crisis Services .............................................................................. 58
Types of Services Received ................................................................................................................................ 71
Access to Crisis Services ..................................................................................................................................... 75
Continuity of Care: Routine Mental Health Services Before and After the Crisis .............................................. 78
Recovery Orientation of Crisis Services .............................................................................................................. 78
Crisis Services Impact: Employment, Housing, and Personal Recovery/Resilience and Wellness ..................... 81
INTERVIEW RESULTS ......................................................................................................................................... 85
Respondent Characteristics ..................................................................................................................................... 85
Peer Support Interview Respondents ................................................................................................................ 85
Employment Support Interview Respondents ................................................................................................... 85
Crisis Intervention Interview Respondents ........................................................................................................ 86
Services Received .................................................................................................................................................... 87
Overlapping Services Received by Peer Support Respondents .......................................................................... 87
Overlapping Services Received by Employment Support Respondents............................................................. 88
Overlapping Services Received by Crisis Intervention Respondents ................................................................. 88
Recovery/Resilience Orientation of Services ........................................................................................................... 88
Access to Services .............................................................................................................................................. 89
Promote Engagement and Continuous Care ...................................................................................................... 94
Family Involvement .......................................................................................................................................... 101
Right to Self-Determination ............................................................................................................................. 101
Cultural Appropriateness ................................................................................................................................. 102
Recovery/Resilience .............................................................................................................................................. 103
Barriers and Risks to Recovery ......................................................................................................................... 104
Positive Self-Perception and Self-Determination ............................................................................................. 104
Proactive Coping .............................................................................................................................................. 106
Purposeful Pursuits .......................................................................................................................................... 112
Connecting and Forming Relationships ............................................................................................................ 115
Hopefulness for the Future .............................................................................................................................. 116
SUMMARY OF SURVEY FINDINGS .................................................................................................................... 117
Peer Support Services ............................................................................................................................................ 117
Characteristics of Individuals Who Received Peer Support Services ............................................................... 117
Types of Peer Support Services Received ........................................................................................................ 117
Access to Peer Support Services ...................................................................................................................... 117
Recovery Orientation of Peer Support Services ............................................................................................... 118
Peer Support Services Impact: Employment, Housing, and Personal Recovery and Wellness ........................ 118
Employment Support Services ............................................................................................................................... 119
Characteristics of Individuals Who Received Employment Support Services .................................................. 119
Types of Employment Support Services Received ........................................................................................... 119
Access to Employment Support Services ......................................................................................................... 119
Recovery Orientation of Employment Support Services .................................................................................. 120
Employment Support Services Impact: Employment, Housing, and Personal Recovery and Wellness ........... 121
Crisis Intervention Services .................................................................................................................................... 121
Characteristics of Individuals Who Received Crisis Services ............................................................................ 121
Types of Crisis Intervention Services Received ................................................................................................ 122
Access to Crisis Intervention Services .............................................................................................................. 122
Continuity of Care ............................................................................................................................................ 122
Recovery Orientation of Crisis Intervention Services ....................................................................................... 122
Table of Contents ii
Crisis Intervention Services Impact: Employment, Housing, and Personal Recovery/Resilience and Wellness
......................................................................................................................................................................... 123
SUMMARY OF INTERVIEW FINDINGS ............................................................................................................... 124
Characteristics of Interview Respondents ............................................................................................................. 124
Services Received .................................................................................................................................................. 124
Recovery/Resilience Orientation of Services ......................................................................................................... 124
Personal Recovery/Resilience and Wellness ......................................................................................................... 125
INTEGRATED SURVEY AND INTERVIEW FINDINGS AND DISCUSSION ................................................................. 127
Sample Representation and Generalizing Study Findings ..................................................................................... 127
Consumer Experiences with Services ..................................................................................................................... 128
Access and Barriers to Access .......................................................................................................................... 128
Overlapping Services ........................................................................................................................................ 129
Appropriateness of Services ............................................................................................................................. 130
Continuity of Care ............................................................................................................................................ 130
Recovery Orientation of Services ..................................................................................................................... 131
Service Impact ....................................................................................................................................................... 131
Employment and Housing ................................................................................................................................ 131
Personal Recovery/Resilience and Wellness .................................................................................................... 132
Outcome Findings and Implications for Measurement ................................................................................... 132
Value Added Services ....................................................................................................................................... 134
Study Limitations and Strengths ........................................................................................................................... 135
Limitations to the Study ................................................................................................................................... 135
Study Strengths ................................................................................................................................................ 136
APPENDIX A – Sample Description of Survey Respondents ............................................................................... 138
Survey Response.................................................................................................................................................... 138
Respondent Demographics ................................................................................................................................... 141
Respondent Representation by Race/Ethnicity, Gender, and Age ........................................................................ 149
Respondent Past, Current, and Desired Employment/Education .......................................................................... 150
Respondent Past, Current, and Desired Living Situation ....................................................................................... 151
Seriousness of Mental Health Concern ................................................................................................................. 154
Mental Health Services Received .......................................................................................................................... 155
APPENDIX B – Standardized Measures ............................................................................................................. 157
Description of Measures ....................................................................................................................................... 157
Recovery Process Inventory ............................................................................................................................. 157
Strengths and Difficulties Questionnaire ......................................................................................................... 157
Recovery Oriented Systems Indicators Measure ............................................................................................. 158
Reliability Analysis of Measures Using Survey Data ............................................................................................. 159
Recovery Process Inventory ............................................................................................................................. 159
Strengths and Difficulties Questionnaire ......................................................................................................... 160
Recovery Oriented System Indicators .............................................................................................................. 160
APPENDIX C – Sample Description of Interview Respondents ........................................................................... 161
Table of Contents iii
Sample Description ............................................................................................................................................... 161
Demographics of Peer Support Services Respondents .................................................................................... 161
Demographics of Employment Support Services Respondents ....................................................................... 161
Demographics of Crisis Intervention Respondents .......................................................................................... 161
Table of Contents iv
Table of Figures
Figure 1 – Peer Support Services Received/Not Received ................................................................................... 17
Figure 2 – Peer Support Services Wanted/Not Wanted ...................................................................................... 18
Figure 3 – Length of Peer Support Services Received .......................................................................................... 18
Figure 4 – Peer Support Services Received/Not Received by Age Group ............................................................. 19
Figure 5 – Peer Support Services Received/Not Received by Gender .................................................................. 20
Figure 6 – Peer Support Services Received/Not Received by Race/Ethnicity ....................................................... 21
Figure 7 – Peer Support Services Received/Not Received by Region ................................................................... 22
Figure 8 – Peer Support Services Received/Not Received by County Size ............................................................ 22
Figure 9 – Peer Support Services Received/Not Received by Annual Income ...................................................... 23
Figure 10 – Peer Support Services Received/Not Received by Educational Attainment ....................................... 24
Figure 11 – Peer Support Services Received/Not Received by Seriousness of Mental Health Concern ................. 25
Figure 12 – Peer Support Services Received/Not Received by Unserved/Underserved Group ............................. 26
Figure 13 – Peer Support Services Received ....................................................................................................... 27
Figure 14 – Difficulties Getting Peer Support Services: Respondents Who Received Peer Support Services ......... 28
Figure 15 – Reasons Not Able to Access or Receive Peer Support Services: Respondents Who Did Not Receive
Peer Support Services but Wanted Them .................................................................................................. 29
Figure 16 – Employment Support Services Wanted/Not Wanted ........................................................................ 37
Figure 17 – Employment Support Services Wanted/Not Wanted ........................................................................ 38
Figure 18 - Length of Employment Support Services Received ............................................................................ 38
Figure 19 – Employment Support Services Received/Not Received by Age Group ............................................... 39
Figure 20 - Employment Support Services Received/Not Received by Gender .................................................... 40
Figure 21 - Employment Support Services Received/Not Received by Race ........................................................ 41
Figure 22 - Employment Support Services Received/Not Received by Region ..................................................... 42
Figure 23 – Employment Support Services Received/Not Received by County Size ............................................. 42
Figure 24 – Employment Support Services Received/Not Received by Annual Income ........................................ 43
Figure 25 – Employment Support Services Received/Not Received by Educational Attainment .......................... 44
Figure 26 – Employment Support Services Received/Not Received by Seriousness of Mental Health Concern .... 45
Figure 27 – Employment Support Services Received/Not Received by Unserved/Underserved Group ................ 46
Figure 28 - Employment Support Services Received ........................................................................................... 47
Figure 29 – Difficulties Receiving Employment Support Services: Respondents Who Received Employment
Support Services ...................................................................................................................................... 48
Figure 30 – Reasons for Not Receiving Needed or Wanted Employment Support Services: Respondents Who Did
Not Receive Employment Support Services ............................................................................................... 49
Figure 31 – Crisis Experienced and Number of Crises Experienced in Past Year ................................................... 58
Figure 32 – Crisis Services Received/Not Received ............................................................................................. 59
Figure 33 – Crisis Services Wanted/Not Wanted ................................................................................................ 59
Figure 34 – Crisis Services Received/Not Received by Age Group ....................................................................... 60
Figure 35 – Crisis Services Received/Not Received by Gender ............................................................................ 61
Figure 36 – Crisis Services Received/Not Received by Race/Ethnicity ................................................................. 62
Figure 37 – Crisis Services Received/Not Received by Preferred Language .......................................................... 62
Figure 38 – Crisis Services Received/Not Received by Region ............................................................................. 63
Figure 39 – Crisis Services Received/Not Received by County Size ...................................................................... 63
Figure 40 – Crisis Services Received/Not Received by Annual Income................................................................. 64
Figure 41 – Crisis Services Received/Not Received by Educational Attainment ................................................... 65
Figure 42 – Crisis Services Received/Not Received by Seriousness of Mental Health Concern ............................. 66
Figure 43 – Crisis Services Received/Not Received by Number of Crises in Past Year .......................................... 67
Figure 44 – Crisis Services Received/Not Received by Years of Receiving Mental Health Services ....................... 68
Figure 45 – Crisis Services Received/Not Received by Unserved or Underserved Group ...................................... 69
Figure 46 – Nature of Mental Health Crisis: Comparison between Respondent Groups....................................... 70
Figure 47 – Mental Health Services Received for Crisis ....................................................................................... 71
Figure 48 – Where Mental Health Services for Crisis Were Received .................................................................. 74
Table of Contents v
Figure 49 – Mental Health Services for Crisis Needed or Wanted for Crisis: Respondents Who Received Crisis
Services ................................................................................................................................................... 75
Figure 50 – Mental Health Services Needed or Wanted for Crisis: Respondents Who Did Not Receive Crisis
Services ................................................................................................................................................... 76
Figure 51 – Reasons for Not Receiving Crisis Services: Respondents Who Did Not Receive Crisis Services but
Wanted Them .......................................................................................................................................... 77
Figure 52 – Survey Type................................................................................................................................... 138
Figure 53 – Survey Type by Language............................................................................................................... 139
Figure 54 – Person Completing Survey ............................................................................................................. 140
Figure 55 – Age Groups ................................................................................................................................... 141
Figure 56 – Gender .......................................................................................................................................... 141
Figure 57 – Race/Ethnicity ............................................................................................................................... 142
Figure 58 – Regional Counties .......................................................................................................................... 143
Figure 59 – County Size ................................................................................................................................... 143
Figure 60 – Respondent County of Residence .................................................................................................. 144
Figure 61 – Member of Unserved or Underserved Group(s) ............................................................................. 145
Figure 62 – Highest Level of Education ............................................................................................................. 146
Figure 63 – Individual Annual Income .............................................................................................................. 148
Figure 64 – Seriousness of Mental Health Concerns ......................................................................................... 154
Figure 65 – Mental Health Services Received (Within Past Year) ...................................................................... 155
Figure 66 – Length of Mental Health Services Received .................................................................................... 156
Table of Contents vi
Table of Tables
Table 1 – Reasons for Not Receiving Peer Support Services: Comparison Between Belonging or Not Belonging to
Unserved/Underserved Group ................................................................................................................. 30
Table 2 – ROSI Results: Comparison Between Respondent Groups ..................................................................... 31
Table 3 – Perceived Access and Appropriateness of Peer Support Services* ....................................................... 32
Table 4 – Perceived Impact of Peer Support Services* ....................................................................................... 33
Table 5 – Current and Desired Employment or School Status: Comparison Between Respondent Groups ........... 34
Table 6 – Current and Desired Living Situation: Comparison Between Respondent Groups ................................. 34
Table 7 – RPI Results: Comparison Between Respondent Groups ....................................................................... 35
Table 8 – Stigma: Comparison by Regions .......................................................................................................... 50
Table 9 – Did Not Know How to Apply: Comparison by Seriousness of Concern .................................................. 50
Table 10 – Placed on Waiting List: Comparison Between Belonging or Not Belonging to an
Unserved/Underserved Group ................................................................................................................. 50
Table 11 – Services Not Available in Preferred Language: Comparison by Preferred Language ............................ 51
Table 12 – ROSI Results: Comparison Between Respondent Groups ................................................................... 52
Table 13 – ROSI Results: Comparison Between Respondents Who Received Peer Support Services and Those
Who Did Not ............................................................................................................................................ 52
Table 14 – Perceived Access and Appropriateness of Employment Support Services* ........................................ 53
Table 15 – Overall Experience with Employment Support Services (N = 156) ...................................................... 54
Table 16 – Current and Desired Employment or School Status: Comparison Between Respondent Groups ......... 55
Table 17 – Current and Desired Living Situation: Comparison Between Respondent Groups ............................... 55
Table 18 – RPI Results: Comparison Between Respondent Groups ..................................................................... 56
Table 19 – Hotline/Warmline and Safety Plan: Comparison by County Size ........................................................ 72
Table 20 – Hotline Warmline: Comparison by Regional Counties ........................................................................ 72
Table 21 – Psychiatrist/Medication and Crisis Residential: Comparison Between Belonging or Not Belonging to
Unserved/Underserved Group ................................................................................................................. 73
Table 22 – Routine Mental Health Services: Comparison Between Respondent Groups1 .................................... 78
Table 23 – ROSI Results: Comparison Between Respondents Who Received Crisis Services and Those Who Did
Not But Wanted Them ............................................................................................................................. 79
Table 24 – ROSI Results for Respondents Who Received Crisis Services: Comparison Between Respondents Who
Received Peer Support Services and Those Who Did Not .......................................................................... 80
Table 25 – ROSI Results for Respondents Who Received Crisis Services: Comparison Between Respondents Who
Received Employment Support Services and Those Who Did Not .............................................................. 80
Table 26 – ROSI Results for Respondents Who Received Crisis Services: Comparison Between Respondents Who
Received Crisis, Peer, and Employment Support Services and Those Who Did Not .................................... 80
Table 27 – Employment Change: Comparison Between Respondent Groups ...................................................... 81
Table 28 – Housing Change: Comparison Between Respondent Groups ............................................................. 81
Table 29 – SDQ Results: Comparison Between Groups ....................................................................................... 82
Table 30 – Psychiatric Hospitalization: Comparison Between Respondent Groups .............................................. 83
Table 31 – RPI Results: Comparison Between Respondent Groups ..................................................................... 83
Table 32 – Age Groups by Gender .................................................................................................................... 142
Table 33 – Highest Level of Education by Age Group ........................................................................................ 147
Table 34 – Representation of Survey Respondents to Statewide Clients by Race/Ethnicity, Gender, and Age.... 149
Table 35 – Past and Current Employment or School Status............................................................................... 150
Table 36 – Desired Employment or School Status ............................................................................................ 151
Table 37 – Current Housing Situation ............................................................................................................... 152
Table 38 – Desired Housing Situation............................................................................................................... 153
Table 39 – Internal Consistency Estimates of the Recovery Process Inventory .................................................. 159
Table 40 – Internal Consistency Estimates of the Strengths and Difficulties Questionnaire ............................... 160
Table 41 – Internal Consistency Estimates of the Recovery Oriented System Indicators .................................... 160
Table of Contents vii
EXECUTIVE SUMMARY
Introduction
A statewide participatory evaluation of the Mental Health Services Act (MHSA) was designed
and implemented according to a participatory planning process led by the University of
California Los Angeles (UCLA) Center for Healthier Children, Families and Communities. As part
of this planning process, mental health consumers and their family members identified two
service areas within General Systems Development (GSD) funding and one service area within
the larger MHSA funding to be the focus of the evaluation. They also identified a set of
indicators and recommended a survey of and interviews with consumers and family members
as the preferred study methods. This report details the resulting study and findings.
Three service areas were selected for the evaluation: peer support services, employment
support services, and crisis intervention services. Peer support services, including parent/family
partnership supports, refer to any services, supports, guidance, advocacy, mentoring, or
assistance provided by an individual who has lived experience with mental health services.
These services may be provided as part of a clubhouse, wellness or recovery center, consumer
or family led organization, or other similar program. Employment support services refer to any
program or activity specifically intended to assist with preparing for or obtaining employment,
whether full time, part time, or voluntary. Crisis intervention services refer to any mental health
program or activity that helps an individual deal with a serious and unexpected situation or a
worsening mental health condition. Crisis services are commonly intended to help the
individual avoid the need for treatment in a psychiatric hospital.
The overarching goal of this evaluation was to study the impact of these services on
employment, housing, and recovery/resilience/wellness, as well as gain an understanding of
consumer perceptions of access to services, appropriateness of services, continuity of care, and
recovery/resilience orientation of services.
The overarching study questions for the evaluation of these three service areas were:
1. What were the characteristics of individuals who received services?
2. What types of services were received?
3. What were individuals’ perceptions of access to services?
4. Was there continuity of care for individuals who received crisis services before and after
the crisis?
5. To what extent did services exemplify a recovery/resilience orientation?
6. Was there a change in employment, housing, and recovery/resilience/wellness after
receiving services?
The participatory evaluation study was developed and conducted utilizing an extensive
participatory process that relied upon the lived experience of individuals, consumers of mental
health services, parents of children who have received services, and family members to focus
and shape all study activities and to help insure that the evaluation methods are credible and
Executive Summary ES-1
the results are accurate, meaningful, and actionable. The study was conducted in collaboration
with a group of Participatory Evaluation Partners (PEPs or “evaluation partners”). The PEPs, all
of whom are persons with lived experience and/or family members, worked closely with the
UCLA evaluation team to carry out each step of the evaluation study.
Study Methods
Based on recommendations from the participatory planning process, a mixed-methods
evaluation employing a statewide survey and interviews was conducted. The purpose of the
survey was to collect a breadth of information to answer all the study questions separately for
each service area across numerous respondents. The purpose of the interviews was to collect
in-depth information from a relatively small group of respondents across service areas to help
enhance the interpretation and understanding of particular study questions.
Measures
The survey, titled the Mental Health Services Act: Statewide Survey of Client Experience (SSCE),
was developed in collaboration with the evaluation partners. It was designed to collect
information about the characteristics of individuals who received mental health services, as
well as the types of services received. In addition, the survey addressed seven indicators: (1)
consumer perception of access to services; (2) continuity of care; (3) recovery/resilience
orientation of services; (4) appropriateness of services; (5) employment situation; (6) housing
situation; and (7) consumer recovery/resilience and wellness. Three standardized scales were
incorporated into the SSCE. Recovery orientation of services was measured using the Recovery
Oriented Systems Indicators (ROSI). Personal recovery was measured for adults using the
Recovery Process Inventory (RPI), while resilience in children was measured using the Strengths
and Difficulties Questionnaire (SDQ). The remaining indicators were measured using items
developed by the PEPs and the UCLA evaluation team.
Interviews were guided by a semi-structured, open-ended interview protocol designed
collaboratively with the evaluation partners. Interview questions were intended to identify
themes concerning respondent perceptions of the recovery/resilience orientation of services
and personal recovery/resilience and wellness.
Data Collection
The survey was designed for completion by people with lived experience (or by family members
or consumer representatives completing the survey on their behalf). The survey was available
online in English and Spanish. Paper-and-pencil surveys were available in English, Spanish, and
Traditional Chinese.
Interviews were conducted with a diverse group of clients and family members from across the
state to understand their perceptions of and experiences with at least one of the three service
areas. Most interviews were conducted in person (or by telephone when necessary) in English,
Spanish, and Chinese.
Executive Summary ES-2
Survey Samples
Responses to the survey were stronger than expected, with a total of 949 completed surveys,
exceeding the study goal of 750. Respondents were diverse, representing all regions of the
state, urban and rural communities, all four MHSA age categories (children, transition age
youth, adults, and older adults), and genders. In addition, there was representation from a
broad range of racial/ethnic groups and individuals speaking Spanish and English. Importantly,
and consistent with the study intent, there was strong participation by traditionally unserved
and underserved populations (e.g., individuals with physical disabilities; individuals who are
homeless; individuals from unserved/underserved ethnic groups; and individuals who are
lesbian, gay, and transgendered).
For each service area, the survey provided a sample of respondents who received services and a
comparative sample of respondents who did not receive services despite needing or wanting
them. A total of 328 survey respondents (42.8% of all respondents) reported that they received
peer support services. An additional 120 respondents reported that they did not receive peer
support services but needed or wanted them. A total of 156 survey respondents (25.0% of all
respondents) reported that they received employment support services. An additional 107
respondents reported that they did not receive employment support services but needed or
wanted them. Finally, a total of 231 survey respondents (68.9% of all respondents who
reported experiencing a crisis in the past year) reported that they received crisis services. An
additional 92 respondents reported that they did not receive crisis services after experiencing a
crisis despite needing or wanting them.
Interview Sample
Altogether, 40 interviews were conducted across the state, thus meeting the study target. As
was the case for survey respondents, interview respondents were diverse, representing all
regions of the state, urban and rural communities, all four MHSA age categories, and genders.
There was good racial/ethnic representation across interview respondents, and there was
strong participation by individuals belonging to traditionally unserved and underserved
populations.
Sample Representation and Generalizing Study Findings
The survey sample as a whole represents the population that the study intended to target –
that is, clients who have had experience with a wide array of public mental health services. The
strong representation of traditionally unserved and underserved individuals in both the survey
and interview samples was desired at the outset – both because the target population is an
MHSA focus, and because, through the participatory planning process, stakeholders
emphasized the importance of representation from traditionally unserved and underserved
groups.
Thirty-eight (38) of the 58 counties in California were represented in the survey. Although not
all counties across the state participated in the study, there was, overall, representative
participation from small and large counties across all regions of the state. The study findings are
Executive Summary ES-3
generalizable to the state based on comparisons of survey respondents to mental health clients
across the state in terms of age, race/ethnicity, and gender.1 More importantly, the study
respondents represent the populations targeted by GSD and the larger MHSA funding.
Summary and Discussion of Findings
This summary and discussion of findings is organized according to the overarching study
questions. Findings from both the survey and interviews are integrated in this summary. The
first section presents a summary of the characteristics of individuals who received peer
support, employment support, or crisis intervention services. Second, the most frequently
identified types of services received within all three service areas are presented, along with a
discussion of overlapping services. In the third section, findings on consumer experiences with
services, including access to services, continuity of care, and recovery/resilience orientation of
services are summarized. The fourth section is a summary of findings on service impact,
including employment and housing outcome findings for the three service areas, as well as
findings on personal recovery/resilience/wellness and psychiatric hospitalization. This section
also includes a discussion of measurement implications for employment and housing outcomes.
Table ES-1 provides a summary of survey findings on seven indicators.
Characteristics of Individuals Who Received Services
For each service area, the two groups of respondents (those who received services compared to
those who did not despite needing or wanting them) were, overall, similar in characteristics and
demographics, including age group, race/ethnicity, gender, education, income, seriousness of
mental health concern, and residence by regional counties. Because there is extensive
information on characteristics of individuals who received services (as well as those who did
not), the reader is directed to the results section of the full report for more detailed
information.
Types of Services Received
The two most common peer support services reported were one-on-one counseling or support
from a peer or parent/family partner and support group. Respondents who received peer
support services received, on average, two types of peer support services. The most common
employment support services reported were help preparing a résumé, help preparing for an
interview, job placement services, vocational training, and job coaching or employment
counseling. Respondents who received employment support services received, on average,
three types of employment support services. Finally, the two most common crisis services
reported were receiving counseling and seeing a psychiatrist/having medication adjusted.
Other crisis services included a safety plan to address the crisis and hotline or warmline to talk
to someone. Respondents who received crisis services received, on average, two crisis services
for the crisis occurring in the past year.
1
The data on these demographics come from the 2007-08 fiscal year report that contains CSI data provided to the
state as of June 2010. This is the latest report published on the California Department of Mental Health
Department website: www.dmh.ca.gov.
Executive Summary ES-4
Overlapping Services
The phenomenon of receiving overlapping services (i.e., receiving multiple types of services
within and across service areas) was evident from the study findings and particularly
pronounced in the interviews. On the whole, interview respondents emphasized that they were
utilizing a range of services and supports as part of care that was tailored to their individualized
needs and goals, as well as part of a proactive strategy to manage and cope with their mental
health. They explained that a deeper engagement in services allowed greater opportunity for
developing important relationships, pursuing meaningful activities, and fostering an improved
self-image and sense of hope. This phenomenon also was evident in the survey findings that
showed ratings of both services and personal recovery/resilience were significantly more
positive when respondents received overlapping services.
Table ES-1 – Summary of Survey Findings on Seven Study Indicators
Peer Support Employment Crisis
Indicators Services Support Services Intervention
Services
Access to Services: What percentage of
respondents who received services reported 10.0% 21.1% 21.1%
difficulties accessing services?
Appropriateness of Services: What percentage
of respondents who received services agreed
that:
Services fit their cultural and life
76.8% 56.7% N/A
experiences?
The physical spaces where services were
78.0% 72.2% N/A
received were inviting and dignified?
The services they received were what
76.7% 68.3% N/A
they wanted?
Continuity of Care: Was there a difference Yes, statistically
between respondents who received crisis services N/A N/A significant
and those who did not (but wanted them) in differences in favor
terms of receiving routine mental health services of respondents
before and after the most recent crisis? who received crisis
services
Recovery Oriented Services: Was there a Yes, statistically Yes, statistically Yes, statistically
difference between respondents who received significant significant significant
services and those who did not (but wanted differences in favor differences in favor differences in favor
them) in the perception of services as recovery of respondents of respondents of respondents
oriented? who received who received who received
services services services
(Continued)
Executive Summary ES-5
Table ES-1 – Summary of Survey Findings on Seven Study Indicators (Continued)
Peer Support Employment Crisis
Indicators Services Support Services Intervention
Services
Employment: Was there a difference between No statistically No statistically No statistically
respondents who received services and those significant significant significant
who did not (but wanted them) in employment differences differences differences
situation?
What percentage of respondents who received
services agreed that:
Services helped improve their 52.7% 67.2% N/A
employment situation?
Housing: Was there a difference between No statistically No statistically No statistically
respondents who received services and those significant significant significant
who did not (but wanted them) in housing differences differences differences
situation?
What percentage of respondents who received
services agreed that:
Services helped improve their living 71.7% 64.3% N/A
situation?
Recovery/Resilience and Wellness: Was there a Yes, statistically Yes, statistically Yes, statistically
difference between respondents who received significant significant significant
services and those who did not (but wanted differences in favor differences in favor differences in favor
them) in perceived personal recovery/resilience of respondents of respondents of respondents
and wellness? who received who received who received
services services services
Was there a difference between respondents who No statistically
received crisis services and those who did not N/A N/A significant
(but wanted them) in psychiatric hospitalization? differences
What percentage of respondents who received
services agreed that:
Services helped them feel better? 81.3% N/A N/A
Services helped with their recovery? 76.9% N/A N/A
Consumer Experiences with Services
Access to Services
Both survey and interview respondents reported high levels of access to services across the
three service areas. The majority of survey respondents who received peer support,
employment support, or crisis services reported no difficulties with accessing these services. In
addition, the most common peer support services (i.e., one-on-one counseling and support
from a peer or parent/family partner) were not associated with any particular respondent
Executive Summary ES-6
characteristics; thus, peer support services in general appear to be received and utilized
indiscriminately.
Overall, for respondents who did not receive services despite wanting them, stigma of mental
health services and lack of information or knowledge about services were identified as key
barriers to accessing these services. These are common barriers that have been identified in
other studies on personal recovery.2 Furthermore, for those survey respondents who did not
receive peer support services, respondents who belonged to an unserved or underserved group
were more likely to report feeling uncomfortable or unwelcomed, having access challenges in
terms of location and time, and disliking the services. Most of these respondents identified
themselves as physically disabled and/or homeless.
Continuity of Care
An analysis of continuity of care was conducted for crisis services only.3 Respondents who
received crisis services were more likely to have routine mental health services before and after
the crisis compared to those who did not receive crisis services. When routine mental health
services were not in place during a crisis, follow up services were less likely to be received,
potentially placing greater risk for a more serious crisis in the future.
Recovery/Resilience Orientation of Services
Respondents who received peer support, employment support, and crisis intervention services
reported significantly more positive experiences with mental health services in general than
respondents who did not receive these services despite wanting them. Respondents who
received services rated their experiences of mental health services as being more person-
centered, more holistic in meeting other needs such as housing, more oriented toward
employment or school stability and/or advancement, more focused on basic needs such as
income and transportation, having less service inadequacies, and/or being less oriented toward
mistreatment.
In addition, several of the main themes that emerged from the interview data, as well as survey
findings from peer support and employment support services, converge to support an overall
finding that mental health services received by study participants were appropriate on many
fronts. Most interview respondents attested to the fact that services they received adopted a
philosophy that recovery is possible, provided individualized care, and/or supported their right
to self-determination. Most also agreed that services received respected their cultural
background. Likewise, three-fourths or more of survey respondents who received peer support
or employment support services agreed that services were appropriately tailored to their needs
and wants. However, while more than three-fourths of peer support service recipients agreed
2
Smith, M.K. (2000). Recovery from a severe psychiatric disability: Findings of a qualitative study. Psychiatric
Rehabilitation Journal, (24)2, 149-158.
3
This indicator was specifically intended for the analysis of crisis services, because continuity of care was
conceptualized as receiving routine mental health services before and after the most recent crisis within the past
year.
Executive Summary ES-7
that services fit their cultural and life experiences, less than two-thirds of employment support
recipients did, suggesting that there is room for improvement in this area.
Service Impact
Employment and Housing
Based on reports of current and desired employment and housing situations, as well as reports
of changes in employment and housing, there were no significant differences between
respondents who received services (peer support, employment support, or crisis services) and
those who did not in terms of employment and housing. However, based on respondent ratings
of experiences with peer support and employment support services, about two-thirds of
respondents who received these services agreed that the services had a positive impact on
their living situation. In addition, about two-thirds of respondents who received employment
support services agreed that the services helped improve their employment situation. Just over
half of respondents who received peer support services agreed that the services helped
improve their employment situation. These findings were supported by examples from
interview respondents who reported improvement in and/or satisfaction with their housing
and/or school or employment situations and credited the mental health supports they received
as helping to enable these positive changes.
Personal Recovery/Resilience and Wellness
Strong evidence of improvement in personal recovery/resilience and wellness after receiving
mental health services emerged from both survey and interview respondents. In all three
service areas, respondents who received services had a more positive perception of personal
recovery/resilience and wellness compared to those who did not receive services despite
wanting them. Specifically, children who received crisis services had significantly fewer peer
problems and significantly greater prosocial behaviors in comparison to a small sample of
children who did not receive crisis services despite needing them.
Furthermore, respondents over the age of 18 who received peer support, employment support,
or crisis intervention services reported significantly more positive perceptions of personal
recovery than respondents who did not receive these services despite wanting them.
Respondents who received services perceived less anguish, felt more connected to others, were
more confident about life, felt more surrounded by people who care, perceived greater housing
stability, and/or were more hopeful compared to those who did not receive services. However,
there were no significant differences in psychiatric hospitalization between respondents who
received crisis services and those who did not despite wanting or needing them.4
From the qualitative interviews emerged numerous stories of personal recovery/resilience,
which respondents credited, at least in part, to the recent mental health services they received.
4
Psychiatric hospitalization is another indicator of recovery and wellness and was measured for the analysis of
crisis services only and included adults over the age of 18.
Executive Summary ES-8
Five recovery themes emerged that encompass the perceptions respondents had about how
and what they do to live full and meaningful lives. Despite daily stressors and other barriers to
recovery reported by respondents, they felt hopeful for the future. In addition, they held a
positive view of themselves, which is related to growing confidence and self-determination.
They proactively managed their mental health concerns in a variety of ways, including utilizing
the mental health services at their disposal and activating a strong safety net of supports.
Moreover, most of the respondents interviewed devoted time and energy pursuing meaningful
activities such as spirituality, vocational interests, and “giving back” to help others. Last, they
developed and reinforced positive relationships and connections instrumental to recovery.
Outcome Findings and Implications for Measurement
The lack of significant differences in employment and housing outcomes between respondents
who received services and those who did not may be attributed, in part, to the extent to which
they are appropriate to measure given the types of services evaluated. For instance, receiving
peer support services was not associated with concrete changes in employment or housing, but
it was strongly associated with intrinsic changes that promote personal recovery and wellness.
In support of these findings, respondent ratings on perceived impact of peer support services
on employment were noticeably lower than ratings on other outcomes such as helping them
feel better and helping their recovery. Moreover, the lack of association between receiving
peer support services and concrete changes is consistent with the nature of peer support
services, which are intended to provide more intrinsic support (such as surrounding clients with
people who have similar experiences and people who care) than concrete support (such as
direct employment support). Therefore, measuring personal recovery/resilience in addition to
concrete changes (e.g., employment status and housing situation) as an outcome of services is
appropriate and meaningful.
Conclusion
Altogether, a system oriented toward recovery/resilience must be accessible, facilitate access
to a variety of overlapping services that help make recovery sustainable, and provide
appropriate services that support the individual’s goals and efforts. Overall, the study findings
suggest that services across the three service areas are accessible. In particular, peer support
services appear to be readily accessible to a broad base of individuals. However, there are
access issues that remain to be addressed, especially for certain populations of individuals who
have traditionally been underserved (e.g., individuals with physical disabilities and individuals
who are homeless). In addition, study findings confirm that access to a variety of supportive
services is being achieved, and recipients perceive services as appropriately individualized,
encouraging, and respectful of their wishes and goals. These elements of recovery oriented
services converge to promote continuity of care that has important implications for personal
recovery/resilience and wellness.
There were no significant differences between respondents who received services and those
who did not in terms of employment and housing; however, ratings of perceived impact on
these outcomes by respondents who received peer support and employment support services
Executive Summary ES-9
indicated that many respondents believed the services were helpful to their employment and
housing situations. Importantly, there were significant differences between respondents in
terms of service experience and personal recovery/resilience and wellness for all three service
areas. Respondents who received services – compared to those who did not – perceived mental
health services to be more recovery oriented; they had a more positive perception of personal
recovery/resilience; their positive service experience was related to a more positive perception
of personal recovery/resilience and wellness; and these positive perceptions were equally
perceived regardless of their characteristics (e.g., race/ethnicity and gender). The survey results
converged with the interviews from which numerous stories of personal recovery/resilience
and hope emerged.
Executive Summary ES-10
INTRODUCTION
Description of Deliverables
As part of the statewide evaluation of the Mental Health Services Act (MHSA or “the Act”), the
University of California Los Angeles (UCLA) at the Center for Healthier Children, Families and
Communities has been charged with carrying out a pair of participatory evaluation studies.
These studies are funded by the Mental Health Services Oversight and Accountability
Commission (MHSOAC). The UCLA evaluation team’s specific charge, per contract language, is
as follows:
Deliverable 2, Phase III – Using participatory research with individuals living with mental illness,
their family members and personal caregivers, ensuring participation of traditionally unserved
and underserved communities across the life span:
a1. Determine the impact of at least one type of service/strategy1 funded with General
System Development (GSD) funding category on at least one outcome prioritized from
the MHSA/System of Care statutes at the individual/client level.2
b1. Determine the impact of involvement of individuals living with mental illness, their
families and personal caregivers in the public mental health system on at least one
outcome prioritized from the MHSA/System of Care statutes.3
The participatory planning process, as noted in MHSOAC RFP 10-70134-000 (page 12), states
“...all aspects of the research shall be developed through a partnership between researchers
and individuals living with mental illness, their family members and personal caregivers,
ensuring participation of traditionally unserved and underserved communities across the life
span. This collaborative process determines priorities for Deliverable 2 regarding what is to be
studied, where, when and how it is to be studied. All partners contribute their expertise to
enhance understanding of the research question, design, implementation and interpretation of
results.”
Consistent with this direction, the principal goals of the participatory planning process carried
out by the UCLA evaluation team were to determine:
1. The programs or activities to be the focus of two evaluation studies, one that focuses on
at least one GSD funded service/strategy and the second that focuses on at least one
MHSA service/strategy (within the broader MHSA funding) that involves consumers,
1 The terms “programs”, “strategies”, and “services” are used interchangeably throughout this document.
2 The MHSOAC’s Initial Statewide Evaluation of the MHSA is expected to provide a summary of GSD activities and
expenditures that can be used as base information for this analysis.
3 This refers to any service or strategy (under any MHSA funding stream) that involves consumers, their families,
and caregivers in the public mental health system.
Introduction 1
family members, and caregivers in the public mental health system.
2. The outcomes to be investigated in each of the studies.
3. The methods to be used in conducting each of the studies.
Overview of Study
Two separate and distinct participatory evaluation studies – the first to investigate two GSD
funded programs, and the second to investigate an MHSA funded program – were proposed.
The study proposals were developed based on an extensive participatory planning process and
were reviewed and approved by the MHSOAC in January 2012.4
Study Focus and Methods
According to results from the participatory planning process, the evaluation proposals focused
on peer support services, employment support services, and crisis intervention services. In the
planning process both peer support and crisis intervention services were the top choices for
services to be evaluated under GSD. Employment support was the top choice to be evaluated
under MHSA. Seven indicators were selected for examination across all services (for more
information, see the methods section). Moreover, participant feedback during the planning
process indicated that the methods to be employed for both studies should include survey and
interview data.
Given that the indicators and study methods prioritized during the participatory planning
process significantly overlapped for the studies of the three services/strategies, a synchronized
mixed-methods evaluation design involving a comprehensive statewide survey and a set of 40
in-depth interviews was implemented. It was determined that the results from the two
proposed studies would best be combined and presented in one report. To minimize confusion,
we refer to one evaluation study throughout the remainder of the report.
A survey designed for completion by people with lived experiences (or by family members or
consumer representatives completing the survey on their behalf) was implemented statewide.
All MHSA age groups (children, transition age youth, adults, and older adults) were invited to
participate in the survey. Throughout the report, the term “respondents” is used to describe all
persons represented in the survey, regardless of who completed the survey. The survey focused
on the experiences of clients in regards to the three service areas selected for study. Likewise,
qualitative interviews were conducted with a diverse group of clients and family members from
across the state to understand their perceptions of and experiences with at least one of the
three service areas. The survey allowed for collecting a breadth of information to answer the
study questions separately for each service area across numerous respondents. Conversely, the
interviews allowed for collecting in-depth information from a relatively small group of
respondents across service areas to help enhance the interpretation and understanding of
4
For a description of the planning process and the resulting study proposals, contact the MHSOAC.
Introduction 2
particular study questions addressed. Therefore, the mixed methods complement rather than
duplicate one another in terms of answering the study questions.
The study was conducted in collaboration with a group of Participatory Evaluation Partners
(PEPs or “evaluation partners”), consisting of a subset of individuals who participated in the
planning process.5 The PEPs, all of whom are persons with lived experience and/or family
members, worked collaboratively with the UCLA evaluation team to carry out each step of the
evaluation study. Their roles were to help: (1) develop the survey instrument and interview
protocol; (2) recruit consumers and family members to complete the survey and participate in
interviews; (3) co-conduct qualitative interviews; (4) analyze and interpret evaluation findings;
and (5) review and provide feedback on the report.
Study Questions
The evaluation study investigated three service areas that are defined below.
Peer support services, including parent/family partnership supports, refer to any services,
supports, guidance, advocacy, mentoring, or assistance provided by an individual who has lived
experience with mental health services. These services may be provided as part of a clubhouse,
wellness or recovery center, consumer or family led organization, or other similar program.
Employment support services refer to any program or activity specifically intended to assist
with preparing for or obtaining employment, whether full time, part time, or voluntary.
Crisis intervention services refer to any mental health program or activity that helps an
individual deal with a serious and unexpected situation or a worsening mental health condition.
Crisis services are commonly intended to help the individual avoid the need for treatment in a
psychiatric hospital. The study’s intent is to assess crisis services as defined here. However, the
information gathered on crisis services includes psychiatric hospitalization as a service because
for some clients, psychiatric hospitalization is a part of their service continuum that also
includes routine mental health services intended to avert psychiatric hospitalization.
The overarching study questions were:
1. What were the characteristics of individuals who received services?
2. What types of services were received?
3. What were individuals’ perceptions of access to services?
4. Was there continuity of care for individuals who received crisis services before and after
the crisis?
5. To what extent did services exemplify a recovery/resilience orientation?
6. Was there a change in employment, housing, and recovery/resilience/wellness after
receiving services?
5
Ibid 4.
Introduction 3
Report Organization
The next section of the report describes the participatory evaluation process, including the
roles and responsibilities of the evaluation partners. It is followed by a description of the survey
and interview methods, respectively. Next, the survey results are presented for peer support
services, employment support services, and crisis intervention services. The interview results
are then described along with a number of profiles highlighting the stories of interview
respondents. Following the presentation of the results are brief summaries of the survey and
interview findings. An integrated discussion of the survey and interview findings concludes the
report.
Introduction 4
PARTICIPATORY EVALUATION PROCESS
Participatory Approach
Participatory evaluation, which entails developing and carrying out evaluation efforts in
partnership with stakeholders, is inherently compatible with and an extension of MHSA values.
This approach to evaluation holds tremendous promise for focusing efforts on fresh and
relevant topics and encouraging the use of research methods that inform actionable program
and system improvement activities.
The participatory evaluation study was developed and conducted utilizing an extensive
participatory process that relied upon the lived experience of individuals, consumers of
services, parents of children who have received services, and family members to focus and
shape all study activities and to help ensure that the evaluation methods are credible and the
results are accurate, meaningful, and actionable. Early in the participatory process, the
evaluation team was made aware of the different terms used to refer to individuals who want
or have received mental health services. Three particular terms – consumer, client, and
individuals with lived experience – were used as part of the participatory planning process, as
well as the evaluation, which uses these terms interchangeably.
Given the range of approaches to and within participatory forms of evaluation and research, it
was expected that MHSA stakeholders may have different views and expectations about how
the evaluation deliverables of the participatory evaluation component would be carried out.
Therefore, our approach was clearly delineated so that interested and involved parties
understood the nature of the project, its strengths and limitations, and their individual roles
and responsibilities.
Participant Selection, Roles, and Responsibilities
We used Cousins and Whitmore’s10 identified dimensions of participation and collaboration in
evaluative inquiry to describe who would be the main partners in our participatory approach,
the depth of their participation, and where decision making power rested.
Control over decision making in our team was balanced between the UCLA evaluation team and
stakeholders. The UCLA evaluation team guided the process in response to participant priorities
but needed to attend to practical constraints such as timelines, resources, methodological
issues, and data validity and reliability. The UCLA evaluation team mediated and helped to
facilitate the decision-making process when there were differences of opinion or disagreement
among participant stakeholders.
10
Cousins, J.B., & Whitmore, E. (1998). Framing participatory evaluation. In E. Whitmore (ed.), Understanding and
practicing participatory evaluation. New Directions for Evaluation, (80), pp. 5-23.
Participatory Evaluation Process 5
The main group of stakeholders invited to partner in conducting the evaluation was comprised
of people with lived experience and/or family members. During our planning process,
participation was very broad and inclusive, so as to gather input from as many consumers and
representatives as possible on the programs of interest, indicators, and methods. For practical
reasons, a core group was needed as we moved forward to the implementation phase of the
evaluation for continuity of participation, decision-making, and maintaining project progress.
Therefore, a group of evaluation partners was convened.
In collaborative or participatory inquiry, the depth of participation from stakeholders – as with
everything else – can vary widely from participant stakeholders only playing a “limited”
consultative role at the planning and/or interpretive stages of the study to having “extensive”
depth of participation in all aspects of the evaluation.11 In our evaluations, the PEPs were
invited to participate extensively in all aspects of the study from planning to implementation
and dissemination.
Participatory Evaluation Partners and Their Roles in Evaluation Activities
As mentioned, the evaluation study was carried out in partnership with a core team of PEPs
who were primarily recruited from the larger pool of evaluation planning participants who
expressed interest in being included in the actual implementation of the evaluation.
We purposefully recruited evaluation partners who are closely affiliated with advocacy or
service organizations either as advocates, volunteers, or professional staff. We also recruited to
include representation from across all four regions of the state (Southern, Bay Area, Central,
and Northern), all MHSA age groups, as well as an array of demographic, unserved, and
underserved groups including ethnic groups, veterans, and members of the lesbian, gay,
bisexual, transgendered, queer or questioning, and intersex (LGBTQI) community.
The PEP team originally consisted of 13 individuals, all of whom are people with lived
experience, family members, or both.12 The evaluation partners participated in all levels of the
evaluation study. Participation was facilitated through weekly two-hour webinars/conference
calls involving evaluation partners and evaluators. Primary roles and responsibilities of
evaluation partners included review of the study proposals; training in key areas to facilitate full
participation; design of the statewide survey; development of the interview protocol; survey
and interview recruitment efforts; co-conducting interviews; data analysis and interpretation;
and report review and feedback. A formal training was conducted on human subjects
protection, and tutorials were provided on participatory evaluation, survey and interview
development, conducting interviews, and data analysis. The evaluation partners each received
a $500 honorarium for their participation in and contributions to the evaluation.
11 Cousins, J.B., & Earl, L.M. (Eds.) (1995). Participatory evaluation in education: Studies in evaluation use and
organizational learning. London: Falmer.
12 Three of the individuals ultimately withdrew their participation due to other commitments.
Participatory Evaluation Process 6
METHODS
A mixed-methods evaluation was conducted using a cross-sectional design with qualitative in-
depth interviews and a statewide survey. The purpose of the survey was to collect a breadth of
information to answer all the study questions separately for each service area across numerous
respondents. The purpose of the interviews was to collect in-depth information from a
relatively small group of respondents across service areas to help enhance the interpretation
and understanding of particular study questions.
Survey Methods
Sample
The survey provided a nonprobability sample of respondents who received services and a
comparative sample of respondents who did not receive services despite needing or wanting
them. Responses to the Statewide Survey of Client Experience (SSCE) were stronger than
expected, with a total of 949 completed surveys, exceeding the study goal of 750. Respondents
were diverse, representing all regions of the state, urban and rural communities, all four MHSA
age categories, and genders. In addition, there was representation from a broad range of
racial/ethnic groups and individuals speaking Spanish and English. Importantly and consistent
with the study intent, there was strong participation by traditionally unserved and underserved
populations (e.g., individuals with physical disabilities; individuals who are homeless; individuals
from unserved/underserved ethnic groups; and individuals who are lesbian, gay, and
transgendered). Moreover, survey respondents were similar to clients across the state. In
comparing statewide penetration rates by race/ethnicity, gender, and age, survey respondents
were generally representative of clients using mental health services. (See Appendix A for a full
description of the survey sample.)
Study Indicators
The indicators selected for examination during the participatory planning process included: (1)
consumer perception of access to services; (2) continuity of care; (3) recovery orientation of
services; (4) appropriateness of services; (5) employment situation; (6) housing situation; and
(7) consumer recovery/resilience and wellness. Perception of access to services was measured
by asking survey respondents to identify challenges accessing services that they received as
well as those they wanted but did not receive. Continuity of care was measured by asking
survey respondents who received crisis services, as well as those who did not despite wanting
them, whether or not they received routine mental health services before and after their most
recent crisis within the past year.13 Recovery orientation of services was measured using a
13
This indicator was specifically intended for the analysis of crisis services because continuity of care was
conceptualized as receiving routine mental health services before and after the most recent crisis within the past
year.
Methods 7
standardized scale (see below). Appropriateness of services was measured by asking survey
respondents to rate the extent to which the services they received fit their unique cultural and
life experiences, felt inviting and dignified, met their personal goals, and were the services that
they wanted. For peer support and employment support services, employment situation was
measured by comparing respondents’ current employment status with their desired
employment status; and housing situation was measured by comparing respondents’ current
living situation with their desired living situation. For crisis intervention services, employment
and living situation were measured by respondent reports of employment and living situation
changes after their most recent crisis within the past year. Finally, recovery/resilience and
wellness were measured differently for adults (age 18 and over) and children by two separate
standardized scales (see below). In addition to the scale used with adults, for crisis intervention
services only, adult respondents were asked to report whether or not they went to a psychiatric
hospital after their most recent crisis within the past year.
Mental Health Services Act: Statewide Survey of Client Experience (SSCE)
The Mental Health Services Act: Statewide Survey of Client Experience (SSCE) was developed
specifically for the participatory evaluation with direction from the PEPs. The survey was
designed to be completed by adults with lived experience, parents of a minor child, or a
consumer representative who is 18 years or older. The survey was not intended to be
completed by a minor child. The survey consisted of six sections of questions, as follows:
The first section focused on demographic information about the client/recipient of services. In
addition, there was one set of items asking about participation in school/employment and
another set of items on housing situation. These items were designed to provide an indication
of change over the course of the prior year, as well as correspondence between one’s current
employment and housing situation relative to one’s goals.
The second survey section focused on personal recovery/resilience and wellness. In order to
increase applicability of the findings, standardized measures were incorporated into the SSCE.
(See Appendix B for details on the standardized measures, as well as a reliability analysis of the
measures using data from the survey.) For adults, the Recovery Process Inventory (RPI) was
selected. The RPI is an instrument that measures recovery and the factors related to feeling
recovered. There are six scales of the RPI: anguish, connected to others, confidence and
purpose, others’ care/help, living situation, and hopeful/cares for self. With the exception of
the scale of anguish, a higher mean score denotes a more positive perception of recovery.
Because the items in the RPI are scored from 1 to 5 (with a higher number representing greater
agreement), a mean score above 3.0, for example, represents agreement or strong agreement
with items for each scale. Each of the six scales and their respective scores is analyzed
separately. For children, the Strengths and Difficulties Questionnaire (SDQ) was included. The
SDQ is an internationally used instrument to measure 25 attributes (some positive and some
negative) of children. There are five scales to the SDQ: emotional symptoms, conduct problems,
hyperactivity/inattention, peer relationship problems, and prosocial behavior. With the
exception of the prosocial behaviors scale, a higher mean score denotes greater difficulty with a
set of attributes (or in other words, less strength in the set of attributes). The first four scales
Methods 8
are added together to generate a total difficulties score, but each of the scales is also analyzed
separately.
Section three focused on the types of mental health services that had been received over the
past year. In addition, the Recovery Oriented Systems Indicators (ROSI), a standardized
measure of the extent to which services were consistent with recovery principles and practices,
was incorporated. The ROSI is be completed by adult recipients of services; it measures the
extent to which clients perceive mental health services to be recovery and wellness oriented.
There are six scales of the ROSI: person-centered focus and directed decision-making; holistic
focus; moving on up; basic material resources; system potholes; and mistreatment. Each of the
six scales and their respective scores is analyzed separately. A higher mean score denotes a
more positive recovery orientation for the first four scales listed. A lower mean score denotes a
more positive recovery orientation for the last two scales listed. Because the items in the ROSI
are scored from 1 to 4 (with a higher number representing greater agreement), a mean score
above 3.0, for example, represents agreement or strong agreement with items for each scale.
Sections four and five focused on employment services and peer support services, respectively.
In both of these sections, individuals who had received services were asked about the types of
services received, ease of access to those services, any difficulties in obtaining the services, and
the impact of services. In addition, both sections included items about barriers to accessing
services experienced by individuals who wanted but did not receive services in the respective
service areas.
The sixth and final survey section focused on crisis events that had been experienced, the
nature of the crisis services that were received, or barriers to receiving crisis services if services
were wanted but not received. In addition, the impacts of the crises, in terms of subsequent
psychiatric hospitalization or changes in housing or employment were investigated. Finally,
items were included concerning the extent to which routine mental health services were being
received prior to and/or following the crisis.
Survey items were selected based on extensive input from the PEPs. Initially, items were
generated based on their responsiveness to the seven indicators for the study. The evaluation
partners reviewed and provided recommendations for individual items across successive drafts.
Numerous measures of individual recovery, as well as service system adherence to recovery
principles, also were reviewed. The RPI, SDQ, and ROSI were ultimately chosen by the PEPs to
be incorporated into the SSCE. Furthermore, data experts from select counties provided input
on the survey, especially providing input on answer options for services typically provided by
counties.
Online and paper-and-pencil versions of the survey were prepared. The online survey was
available in English and Spanish. Paper-and-pencil surveys were available in English, Spanish,
and Traditional Chinese. The survey took about 30 minutes to complete. Survey respondents
were invited to enter a raffle to win a $50 gift card in appreciation for their participation in the
survey. Once all surveys had been submitted and entered into a database, raffle winners were
randomly selected. They were notified by telephone or Email, and mailed the gift card to the
Methods 9
address they provided. Approximately one in every 75 survey respondents (or 13 altogether)
received a gift card.
Survey Distribution
People with lived experience, family members, and representatives of consumers were
recruited to complete the survey using a multipronged statewide recruitment strategy. Our goal
was to collect a minimum of 750 surveys from a highly diverse group of individuals. Specifically,
the recruitment effort was intended to gather survey respondents from all regions of the state
and from communities that varied in population density (i.e., urban, suburban, and rural). We
strived to recruit respondents of diverse genders, from diverse racial/ethnic groups, and
individuals who are affiliated with groups that are often unserved/underserved or otherwise
disenfranchised or overlooked in studies of mental health services (e.g., individuals who have
been homeless, incarcerated, or who are from LGBTQI communities).
One recruitment strategy was to solicit assistance from community-based providers, county
mental health departments, and advocacy groups to distribute study notices and materials. In
addition, PEPs used their connections with local and statewide consumer and advocacy groups
to similarly distribute study notices and materials, with a special focus on historically hard to
reach populations.
Initially, notices were sent to all county mental health departments, various providers, and
advocacy groups in advance of the survey release in order to solicit interest and request
assistance and support distributing the survey. More specifically, these notices were emailed to
all county mental health directors by the California Mental Health Directors Association, to
county MHSA coordinators, and California Institute for Mental Health (CiMH). These notices
also were emailed to hundreds of providers by the California Council of Community Mental
Health Agencies (CCCMHA), the National Alliance on Mental Illness, California (NAMI
California), United Advocates for Children and Families, and the Client Network. The notices
included information about the survey and asked the recipient to consider posting notices
inviting people with lived experience, family members, and consumer representatives to
complete the survey. In addition, the recipient was asked to consider making paper-and-pencil
surveys available to the people they serve and to facilitate return of the surveys to the
evaluators.
County mental health departments, private providers, and advocacy groups who agreed in
advance to assist with the study were sent survey fliers to be posted and/or distributed, as well
as paper-and-pencil surveys and self-addressed/stamped return envelopes. In addition, in order
to make the surveys more readily available and to accommodate participation by large
providers with multiple sites, counties and agencies were provided PDF versions of the survey.
The survey could be printed and shared within and outside of the recipient’s organization in a
variety of locations.
Subsequently, a notice inviting counties, agencies, advocacy groups, and other interested
parties to support and participate in the study was distributed by the MHSOAC and posted on
Methods 10
their website. Follow up notices inviting counties and providers to support the survey were
distributed by the UCLA evaluation team during the open window for completing the survey.
The survey could be accessed and completed online in English or Spanish through a simple web
address. In addition, surveys could be completed using the paper-and-pencil versions available
in English, Spanish, and Traditional Chinese. Paper-and-pencil versions could be obtained from
participating county mental health and private providers, advocacy and consumer groups, or
through email distribution of the PDF version. Respondents could return paper versions of the
survey to the evaluators individually by regular mail, or by returning a survey to one of the host
agencies who in turn returned surveys in bulk.
Analysis of Survey Data
The survey data, which were primarily quantitative, were analyzed using the Statistical Package
for the Social Sciences (SPSS) version 20.0. Qualitative data from open-ended questions were
content analyzed and reported as quantitative data. When responses to open-ended questions
were the same as close-ended response options, they were recoded as such. When open-ended
responses did not match any close-ended response options, they were grouped into like
categories and coded to be counted as additional response options.
Each variable in the survey data set was first analyzed using univariate descriptive statistics
such as frequencies, percentages, and means. Bivariate statistics were also conducted to
compare two variables at a time. Specifically, the chi-square test was used to examine
associations between two variables based on proportional distributions (e.g., association
between gender and whether or not a service was received). The independent samples t-test
was used to examine mean differences (e.g., differences between two groups in the scores of a
scale measuring perception of recovery). The Pearson’s r correlation was used to examine the
relationship or correlation between two variables (e.g., correlation between length of service
and a scale measuring perception of recovery). Moreover, multivariate statistics were
conducted to examine multiple variables at a time. In particular, logistic regression was used to
test whether respondent characteristics predict group membership in, for example, the group
that received services or the group that did not receive services. The evaluation partners were
involved in the analysis of survey data by reviewing tables of analysis outputs. As part of this
process, PEPs suggested different ways to analyze or present the data, as well as suggested
additional analyses.
When statistical tests were conducted comparing two or more groups (e.g., respondents who
received crisis services and those who did not), the test result was reported as “statistically
significant” or “not statistically significant” based on a probability level of .05 (reported as “p” in
the report). The probability level was set at 0.05 for this study, meaning that a result was not
reported as statistically significant if the probability value (or p-value) was higher than 0.05.
This is a typical cut-off point for studies of this nature and indicates that a relationship occurred
due to chance no more than five times out of 100 randomized trials. A p-value that is lower
than 0.05 gives us even greater confidence that the observed relationship (e.g., differences in
the length of services between males and females) was less due to chance and that the
Methods 11
observed relationship was actually “true”. Probability values lower than 0.05 (e.g., p < .01) are
also reported in the tables and graphs.
Although a total of 949 surveys were returned, each analysis did not include a sample of 949
respondents. There were missing data across all the analyses due to several reasons, including
information that respondents did not provide in completing the survey.14 If there were more
than two missing answers on subscales within the RPI, SDQ, and ROSI, they were not included
in the analysis to ensure a more complete and accurate data set. In the report, the sample size
is reported as “N”. When applicable, the tables include the sample size for each group or
subgroup. It is important to note that the sample sizes vary from analysis to analysis, depending
on the data that were available. Furthermore, as more categories of respondents were
analyzed at once, the sample sizes diminished for each category. When the sample sizes were
unstable, the analysis was not reported.
When conceptually appropriate, variables were collapsed to form fewer categories or
groupings. For example, there were multiple descriptions of belonging to an unserved or
underserved group (e.g., being physically disabled or homeless). However, these descriptions
were collapsed into two groupings as either unserved/underserved or not
unserved/underserved.
Interview Methods
Sample
Purposeful sampling was used to select interview participants. Purposeful samples are not
meant to be representative samples. Rather, they are intended to elicit information from
respondents with particular set of characteristics to illuminate specific questions of interest to
an evaluation. Altogether, 40 interviews were conducted across the state. There were 22 male
respondents and 18 female respondents. Five of the interviews focused on children; 10
interviews were conducted with transition age youth. Thirteen (13) interview respondents were
adults, and 12 were older adults. There was good racial/ethnic representation across interview
respondents, including 13 Caucasians, 10 African Americans, eight Latinos, four Asians, and five
respondents of mixed race/ethnicity (i.e., two respondents who are Native American/Latino,
one Native American/African American, one African American/Caucasian, and one
Filipino/Irish). Eighteen (18) of the interviews were conducted in the Southern region, 12 in the
Bay Area, six in the Central region, and four in the Northern region. (See Appendix C for a full
description of the interview sample.)
14
Data accuracy checks were conducted to ensure that the most accurate data sets were analyzed for the three
service areas. For example, multiple filters were applied to select only those responses applicable to particular sets
of analyses.
Methods 12
Interview Protocol
A semi-structured, open-ended interview protocol was designed in collaboration with the
evaluation partners. The protocol included a set of questions for each of the three service areas
of interest to the study: peer support services, employment support services, and crisis
intervention services. The basic content of the primary interview questions was standardized
across the three service areas in order to increase comparability and facilitate cross-case
analysis. However, the semi-structured nature of the protocol was intended to allow the
interviewer ultimate flexibility to customize the sequencing, as well as the direction and depth
of probing, for each individual interview respondent.
To develop the interview protocol, the evaluators first presented a tutorial on qualitative
interviews and facilitated an exercise whereby the PEPs brainstormed ideas about potential
interview topics they wanted to explore during the interviews relative to the overarching study
questions. Based on this information, the evaluators then drafted a basic set of questions for
each service area. The draft protocol was then shared with the PEPs, and an ongoing, iterative
process of review, discussion, and revision ensued. Discussion with and feedback from the
evaluation partners resulted in three sets of revised questions with probes added to enhance
the interviewers’ ability to elicit more specific and detailed information about respondents’
experiences with mental health services.
Like the survey, the interview guide included questions pertaining to the respondents’ history
and reasons why services were received; the type(s) of services received; and the experience of
accessing services. The interview also included questions about personal recovery/resilience
and wellness, allowing the interview respondent to explain in depth whether, how, and why
services helped them in their daily lives.
Interview Participant Recruitment
Interviews with people with lived experience or their family members were designed to
supplement quantitative data gathered through the survey with personal “first-hand” accounts
of service experiences and impacts. The study intent was to conduct 40 interviews representing
a broad range of individuals, service systems, and experiences, with a special focus on including
individuals who historically have been disenfranchised.
To accomplish this goal of achieving broad representation, interview recruitment targets were
established to guide recruitment efforts. Interview recruitment targets included participation
from females and males, all four MHSA age groups, all regions of the state, and personal
experiences with the three service areas that were the focus of the study. In addition,
recruitment efforts focused on including representation from diverse racial/ethnic groups and
individuals who are traditionally unserved or underserved – for example, veterans; individuals
from the LGTBQI community; individuals who have hearing, vision, or other physical
disabilities; and individuals who have a history of being homeless or incarcerated.
Methods 13
Recruitment of interview respondents was largely facilitated by PEPs. Using the recruitment
targets as a guide, they reached out to local and statewide groups with which they were
affiliated to identify individuals who were interested in participating in an interview. Once
interested individuals had been identified and given their permission to be contacted by a
member of the UCLA evaluation team, an evaluator called them to confirm their interest in and
fit with the study. During these follow up calls, the individual was reminded that participation in
an interview would be completely voluntary, an overview of the study was shared, the service
area focus was discussed, and the process for conducting interviews was explained. When an
individual was a good fit for the study and remained interested in participating, an interview
was scheduled to be completed in-person at a location of her or his choice or by telephone.
Prospective respondents also were given the option of having an evaluation partner co-conduct
the interview.
In addition to this general process of recruitment, more focused recruitment efforts were
completed by the evaluators in an effort to ensure that the diversity targets were being met.
For example, specific efforts were made to identify Chinese-speaking and Spanish-speaking
consumers or family members. In these cases, evaluators contacted county mental health
departments and community-based providers to solicit their assistance in identifying individuals
interested in completing an interview.
Conducting Interviews
All interviews were conducted by at least one member of the UCLA evaluation team. In most
cases, a scribe was also present to assist with the consenting process and note taking. The
evaluation partners sometimes co-conducted interviews based on the interview respondent’s
preferences and PEP availability. Interviews were guided by the protocol but followed the
respondent’s lead and were supportive of elaboration and detail in responses. The interview
tone was casual and comfortable, lasting one to two hours in duration.
Interviews were most often completed in person (typically at the individual’s residence or at
the offices of a local provider). In less than a quarter of instances, interviews were conducted
over the phone. All interviews were audio recorded, with permission, and later transcribed for
analysis. Participants received a $30 gift card for their time and assistance.
Analysis of Interview Data
The foundation for the qualitative cross-case analysis is thematic analysis, a conventional
method in qualitative research that involves reading transcripts to identify ideas and to
categorize meaningful patterns in the data.15 Evaluators also borrowed specific coding practices
– open and axial coding – from the grounded theory approach to analysis developed by Strauss
15 Creswell, J. (1994). Research design: Qualitative and quantitative approaches. London: Sage.
Miles, M. & Huberman, A. (1994) Qualitative data analysis: An expanded sourcebook (2nd edition). London: Sage.
Methods 14
and Corbin.16 Thematic analysis and grounded theory are similar analytic frameworks in that
they both endeavor to represent a view of reality through systematically and progressively
identifying and integrating concepts into higher level themes. Both analytical approaches are
initially inductive, allowing concepts and patterns to emerge from the data; and later deductive,
in the practice of going back to the data to verify patterns and themes. However, thematic
analysis falls short of developing theoretical hypotheses which is the convention in grounded
theory analysis. Rather, thematic analysis produces a description and interpretation of
individual subjective experiences and oftentimes is more fitting in the context of program
evaluation.
Analysis began by systematically reducing the data during open coding procedures that
fragment data into separate units. During this process, data are scrutinized line-by-line and
individual excerpts are coded so that the excerpts, which become the source of the ultimate
themes, remain rooted in the data. This first level of open coding involved the initial
identification of concepts and was conducted by five members of the UCLA evaluation team,
who we referred to as “first-level coders”. The coders utilized both a small set of a priori codes
based on the evaluation questions of interest, as well as grounded codes that emerged from
the data. After a number of transcripts had been initially coded, the codes, or concepts, were
shared with PEPs in the form of a draft coding scheme. The evaluation partners were
encouraged to review the coding scheme and then read interview transcripts and provide
feedback as to whether important concepts from the interview transcript were missing from
the coding scheme or somehow mislabeled or not labeled in a clear way. The PEP’s review of
the coding scheme was not intended to be a reliability check; rather, it was an opportunity for
them to have input and to help establish content validity of the coding scheme.
All interview transcripts received a first level of coding and were then reviewed by “second-
level cross-checkers” who conducted a consistency check for coder agreement as they
uploaded the data into Dedoose, an online mixed methods data analysis tool. To do so, the
cross-checkers reviewed the coded transcripts and identified areas of disagreement where they
would have applied more, fewer, or different codes than the first-level coders. Where
discrepancies were found, the two cross-checkers discussed and reconciled differences with
one another until consensus was reached. This served as an important quality review process to
reduce bias and increase the consistency of how codes were applied across interview
transcripts.
Next, the two second-level coders continued the open coding process, which involved grouping
concepts into categories, clarifying dimensions of categories, and identifying patterns. Coded
excerpts were then clustered into initial themes that were refined and verified by returning to
the data during the process of axial coding. Relationships between and among themes were
studied; when warranted, themes were collapsed or merged and appropriate ordering was
determined for presentation in the report.
16 Strauss, A. & Corbin, J. (1998) Basics of qualitative research: Techniques for developing grounded theory (2nd
edition). London: Sage.
Methods 15
Overall, interview data were analyzed to illuminate key patterns in the data related to the
changes that respondents described in their lives since receiving MHSA services
(recovery/resilience themes), as well as some of the factors that facilitate or impede that
change (recovery oriented services themes). Because of the overlap of services received by
individuals in our interview sample, a cross-case analysis was employed (i.e., data were
analyzed across all 40 respondents instead of looking separately at those who received peer
support, employment support, and crisis intervention services). Several key themes (defined as
those that were identified by at least 50.0% of respondents) pertaining to the recovery
orientation of services and the personal recovery/resilience experiences of service recipients
emerged across the interviews. In the qualitative findings section, thematic content is
summarized and illustrated using examples and experiences in the voice of respondents.
Themes are further illuminated through a number of recovery/resilience profiles featured
throughout the findings section. These profiles help to demonstrate the overlap with and
interplay between the themes of recovery oriented services and personal recovery/resilience.
Human Subjects Review
The participatory evaluation study received approval from the Institutional Review Board (IRB)
at UCLA. The IRB oversees researchers’ ethical practices where human participants are
involved. The IRB closely reviews all research materials and methods prior to the study to
ensure that participants are safeguarded from undue harm and made fully aware of their roles
in the study. This includes how participants will be involved in the study, its duration,
participation risks and benefits, response confidentiality or anonymity, and any compensation.
As part of obtaining IRB approval, the UCLA evaluation team described how they met each of
these requirements and specified how the team would inform participants that involvement
was voluntary—that they could, at any time, withdraw from the study without penalty.
The UCLA evaluation team members and PEPs – each trained in human subjects research –
explained the study and its voluntary nature with prospective participants during interview
recruitment. Evaluation team members who were responsible for leading interviews walked
through detailed consent forms with interview participants at the beginning of interviews. A
signature on consent forms indicated an individual’s willingness to be part of the study to the
extent that he or she was comfortable. Signatures were obtained from all interview
respondents at the beginning of the interview.
Methods 16
SURVEY RESULTS
Peer Support Services
Characteristics of Individuals Who Received Peer Support Services
The characteristics of individuals (or “respondents”) who received peer support services are
described and are compared with survey respondents who did not receive peer support
services despite needing or wanting them.
A total of 767 respondents reported whether or not they received peer support services in
the past year. As presented in Figure 1, 328 (or 42.8%) of these 767 respondents reported
having received peer support services. Furthermore, as presented in Figure 2, 120
respondents (or 28.3%) who did not receive peer support services reported needing or
wanting these services.
Figure 1 – Peer Support Services Received/Not Received
100%
75%
57.2%
50% 42.8%
25%
0%
Yes, received peer support services No, did not receive peer support
(n=328) services (n=439)
N = 767
Survey Results: Peer Support Services 17
Figure 2 – Peer Support Services Wanted/Not Wanted
100%
71.7%
75%
50%
28.3%
25%
0%
Did not receive peer support services but Did not receive peer support services and
wanted or needed them (n=120) did not want or need them (n=304)
N = 424
(There are 15 missing responses from the sample of 439 respondents who reported that they did not receive peer support
services.)
Of the 328 respondents who received peer support services, 284 reported the number of
months they received peer support services in the past year. As shown in Figure 3, 31.3% of
respondents reported receiving less than six months of peer support services. The
remaining 68.7% reported receiving six months or more of peer support services, while a
sizeable 44.0% of these respondents reported having received 12 months of peer support
services. On average, respondents reported receiving about eight months of peer support
services (standard deviation of 4.1 months). Altogether, there was variation among the
respondents in terms of length of service, but the majority had received peer support
services for more than six months in the past year.
Figure 3 – Length of Peer Support Services Received
Less than 6
months
89 (31.3%)
6 months or
more
195 (68.7%)
N = 284
Survey Results: Peer Support Services 18
The following tables provide the distribution of two sets of respondents. We often refer to
these two sets of respondents as “groups”. One group includes the respondents who
received peer support services. The second group includes the respondents who did not
receive peer support services despite needing or wanting them. In the tables below, we
present the distribution of these two groups side by side to compare whether there is a
statistically significant association (using the chi-square test) between respondent
characteristics/demographics and whether or not they received peer support services.
Starting with an analysis on age, there was no association between age group and whether
or not peer support services were received (p > .05). Figure 4 details that more adults were
represented in both groups because there was overrepresentation of adult survey
respondents. However, between the two groups, there was equal distribution of children
(16.4% and 20.2%), transition age youth (14.8% and 20.2%), adults (55.1% and 49.5%), and
older adults (13.8% and 10.1%).
Figure 4 – Peer Support Services Received/Not Received by Age Group
100%
Yes, received peer
75% support services
(n=305)
55.1%
49.5%
50%
No, did not receive
peer support services
but wanted them
25% 20.2% 20.2% (n=109)
16.4%
14.8% 13.8%
10.1%
0%
Child (n=72) TAY (n=67) Adult (n=222) Older Adult (n=53)
N = 414
Survey Results: Peer Support Services 19
There was no association between gender and whether or not peer support services were
received (p > .05). As shown in Figure 5, similar proportions of females and males reported
receiving peer support services (55.4% compared to 43.0%, respectively) and not receiving
services but wanting them (51.7% and 48.3%, respectively). A small group of respondents
who identified themselves as transgendered (1.6%) received peer support services; none
reported wanting peer support services but not receiving them.
Figure 5 – Peer Support Services Received/Not Received by Gender
100%
75%
55.4% Yes, received peer
51.7%
48.3% support services (n=321)
50%
43.0%
No, did not receive peer
support services but
25% wanted them (n=118)
1.6% 0.0%
0%
Female (n=239) Male (n=195) Transgendered (n=5)
N = 439
Survey Results: Peer Support Services 20
There was no association between race/ethnicity and whether or not peer support services
were received (p > .05). The findings are reported in Figure 6. About 10.0% each of African
American and “other race” respondents were represented in both groups. Larger
proportions reported by Caucasian and Latino respondents (about 35.0% each) also were
similar in both groups. A smaller proportion of Asian and Pacific Islander respondents (3.4%)
reported receiving peer support services, while double the proportion of Asian and Pacific
Islander respondents (7.6%) reported not receiving the services despite wanting them. This
contrast is the most noticeable, although altogether there was no statistical association.
Figure 6 – Peer Support Services Received/Not Received by Race/Ethnicity
50%
38.3%
40%
37.0%
35.3% 34.9% Yes, received peer
support services
(n=321)
30%
No, did not receive
20%
peer support
12.8% services but wanted
10.6% 10.9%
9.2% them (n=119)
10%
7.6%
3.4%
0%
African Asian and Caucasian Latino (n=156) Other* (n=54)
American Pacific Islander (n=165)
(n=45) (n=20)
N = 440
* “Other race” includes Native American, mixed race, and other ethnic groups not identified within the racial groups
provided as answer options in the survey.
Survey Results: Peer Support Services 21
Because of the larger representation of respondents in the Southern region, it was expected
that most respondents receiving peer support services (and even those not receiving them)
would reside in the Southern region. The distribution of respondents in both groups is
presented in Figure 7. There was no association between regional counties and whether or
not peer support services were received (p > .05). For the most part, the proportions across
regional counties were similar for both groups. For example, 57.0% of respondents who
received peer support services resided in the Southern region and 56.6% of those who did
not receive these services resided there.
Figure 7 – Peer Support Services Received/Not Received by Region
100%
Yes, received
peer support
75% services (n=321)
57.0%
56.6%
No, did not
50%
receive peer
support services
but wanted them
20.4%
25%
15.3% 15.9% (n=113)
11.8% 14.2%
8.8%
0%
Southern Central (n=72) Bay Area Northern
(n=247) (n=48) (n=67)
N = 434
There was no association between county size and whether or not peer support services
were received (p > .05). As shown in Figure 8, less than 10.0% of respondents from each
group were from small counties. In contrast, the vast majority of respondents from each
group resided in larger counties (93.8% for those who received peer support services and
89.7% who did not but wanted them).
Figure 8 – Peer Support Services Received/Not Received by County Size
100% 93.8%
89.7%
Yes, received
peer support
75% services (n=323)
50% No, did not
receive peer
support services
25% but wanted
10.3%
6.2% them (n=120)
0%
Small county (n=32) Larger county (n=407)
N = 443
Survey Results: Peer Support Services 22
There was no association between annual income and whether or not peer support services
were received (p > .05). As shown in Figure 9, most respondents from both groups earned
less than $15,000 annually (53.1% and 61.1%). For respondents who received peer support
services, 19.1% had no individual income compared to 15.6% for respondents who did not
receive peer support services but wanted them.
Figure 9 – Peer Support Services Received/Not Received by Annual Income
100%
75% Yes, received
peer support
61.1%
services (n=277)
53.1%
50%
No, did not
receive peer
25% 19.1% support services
16.3%
15.6% 13.3% but wanted
them (n=90)
5.6%
4.7% 3.2% 2.2% 3.6% 2.2%
0%
No individual Less than $15,000 - $30,001 - $45,001 - Over $60,000
income (n=67) $15,000 (n=202) $30,000 (n=57) $45,000 (n=18) $60,000 (n=11) (n=12)
N = 367
Survey Results: Peer Support Services 23
The distribution of respondents by educational attainment was similar for both groups. As
shown in Figure 10, for respondents who received peer support services, 23.2% had less
than a high school education. In contrast, for respondents who did not receive peer support
services but wanted them, 33.1% had less than a high school education. Therefore, a slightly
higher proportion of respondents who received peer support services had less than a high
school education than their counterparts who did not receive peer support services.
However, taking into consideration all the education levels, there was no association
between educational attainment and whether or not peer support services were received (p
> .05).
Figure 10 – Peer Support Services Received/Not Received by Educational Attainment
50%
40%
Yes, received peer
33.1%
support services
29.7%
30% (n=310)
27.1%
26.1%
23.2%
21.7% No, did not
20.0%
20% 19.1% receive peer
support services
but wanted them
(n=115)
10%
0%
Less than High High School Vocational/Some College Grad or
School (n=110) Grad/GED (n=122) College (n=109) Above (n=84)
N = 425
Survey Results: Peer Support Services 24
There was no association between the seriousness of respondents’ mental health concern
and whether or not peer support services were received (p > .05). As shown in Figure 11,
less than 10.0% of respondents from both groups perceived their mental health concern to
be mild. Approximately 20.0% of respondents from both groups perceived their concern to
be moderate. In both groups, about one-third perceived a serious concern (36.7% and
33.0%) and another one-third perceived an extremely serious concern (36.7% and 35.6%).
All in all, the majority of respondents from both groups perceived their mental health
concern to be serious or extremely serious.
Figure 11 – Peer Support Services Received/Not Received by Seriousness of Mental Health Concern
50%
40% 36.7% 36.7%
35.6% Yes, received peer
33.0%
support services
30% (n=308)
22.9%
20% 17.5% No, did not receive
peer support services
9.2% 8.5% but wanted them
10%
(n=118)
0%
Mild concern Moderate concern Serious concern Extremely serious
(n=38) (n=81) (n=152) concern (n=155)
N = 426
There was an association between belonging to an unserved or underserved group and
whether or not peer support services were received (p < .05). (See Appendix A for details on
the different characteristics of individuals belonging to an unserved or underserved group.)
Therefore, one characteristic differentiated respondents who received peer support
services from those who did not but wanted them. As presented in Figure 12, among
respondents who received peer support services, 50.0% belonged to an unserved or
underserved group and the remaining 50.0% did not. In contrast, among respondents who
did not receive peer support services but wanted them, 30.8% belonged to an unserved or
underserved group and the remaining 69.2% did not. Therefore, more unserved or
underserved respondents (e.g., homeless and physically disabled) were represented in the
group that received peer support services, while more respondents who are not unserved
or underserved were represented in the group that did not receive peer support services. It
is important to understand that these distributions refer to sample representation and do
not necessarily indicate the extent to which unserved or underserved respondents had
access to services.
In order to assess the potential effect of this association on the results, we examined
whether respondent characteristics predict membership in the unserved or underserved
Survey Results: Peer Support Services 25
group. Using logistic regression, we found that the seriousness of the mental health concern
and whether or not crisis services were received significantly “discriminated” respondents
who belonged to an unserved or underserved group (p < .05).17 In other words, more
respondents who did not receive peer services and who did not belong to an unserved or
underserved group rated their mental health condition to be more moderate (27.7%) than
their counterparts (12.9%). Based on this self-rating and by virtue of its
underrepresentation of unserved and underserved respondents, the group of respondents
who did not receive peer services had more individual members who have moderate
concerns with their mental health. Conversely, the group of respondents who received peer
support services had more individual members who have more serious concerns with their
mental health.
Figure 12 – Peer Support Services Received/Not Received by Unserved/Underserved Group
100%
75%
69.2%
50.0% 50.0%
50% Yes, received peer support
services (n=328)
30.8%
25%
No, did not receive peer
support services but
wanted them (n=120)
0%
Unserved/Underserved (n=237) Not unserved/underserved
(n=211)
N = 448
Χ2 = 4.14 (1), p < .05
17 The model test result including 426 respondents was Χ2 = 12.69 (4), p < .05. The results of the predictor
variable of seriousness of mental health concern are: B = 0.82, S.E. = 0.28, Wald’s Χ2 = 8.46 (1), p < .01, and
odds ratio = 0.44. The results of the predictor variable of whether or not peer support services were received
are: B = 0.44, S.E. = 0.22, Wald’s Χ2 = 3.81 (1), p < .05, and odds ratio = 0.65.
Survey Results: Peer Support Services 26
Types of Peer Support Services Received
Respondents who reported receiving peer support services were asked in the survey to
identify the types of services received. Figure 13 lists the services and responses. It is
important to note that more than one type of service could be selected. Therefore, the
percentages are based on total responses rather than the number of respondents in order
to assess the relative commonality of peer support services received by respondents.
The two most common peer support services reported were one-on-one counseling or
support from a peer or parent/family partner (31.2%) and support group (28.6%). Wellness
centers (11.8%) and drop-in centers (9.8%) were each reported in about 10.0% of
responses, but clubhouses were reported less frequently (3.7%).18 Moreover, peer support
for specific types of services was reported for substance abuse (7.4%) and employment
services (6.3%). On average, respondents reported receiving two types of peer support
services in the past year (standard deviation of 1.4 services). When the most common peer
support service of one-on-one counseling or support from a peer or parent/family partner
was further examined for possible associations between respondent characteristics (e.g.,
age, race/ethnicity, and gender) and whether or not this service was received, none were
found.
Figure 13 – Peer Support Services Received
One-on-one counseling or support
194 31.2%
from a peer or parent/family partner
Support group 178 28.6%
Wellness Center 74 11.8%
Drop-in Center 61 9.8%
Support or counseling in reference to
7.4%
substance abuse 46
Support or counseling in reference to
6.3%
employment services 39
Clubhouse 3.7%
23
Other* 1.2%
7
0% 25% 50%
N = 328, Total Responses = 622
*Other includes National Alliance for Mental Illness (NAMI) (n=5) and religious/church/spiritual group or center (n=2).
18
Wellness centers, drop-in centers, and clubhouses are typically facilities that provide a place for consumers
to meet and receive/give support to one another, as well as receive support and services from mental health
professionals. There may be overlaps in the types of support and services provided.
Survey Results: Peer Support Services 27
Access to Peer Support Services
Respondents who received peer support services were asked in the survey to report
whether or not they experienced any difficulty getting peer support services. Figure 14
presents the difficulties reported by 10.0% of respondents (or 34 out of 338) who received
these services. More than one difficulty could be selected; therefore, the percentages are
based on total responses rather than the number of respondents. The two most common
difficulties reported were “I was put on a waiting list or experienced other delays” (26.5%)
and “I tried to get these services several times before I was able to” (24.5%). Other
difficulties reported were “The services were not offered during times that were convenient
to me” (13.7%), “The services were not available in a convenient location for me” (12.7%),
and “I did not feel comfortable or welcomed” (11.8%). There were only a handful of
respondents who reported difficulties accessing peer support services due to services not
having available staff of their cultural background (4.9%) and not being available in their
primary language (2.0%).
Figure 14 – Difficulties Getting Peer Support Services: Respondents Who Received Peer Support Services
I was put on a waiting list or
27 26.5%
experienced other delays
I tried to get these services several
25 24.5%
times before I was able to
The services were not offered during
14 13.7%
times that were convenient to me
The services were not available in a
13 12.7%
convenient location for me
I did not feel comfortable or
12 11.8%
welcomed.
The services were not offered by staff
5 4.9%
who share my cultural background
The services were not offered in my
2 2.0%
primary language
Other* 4 3.9%
0% 25% 50%
N = 34, Total Responses = 102
* Other difficulties include: not knowing how to access the services; not feeling like the respondent’s crisis mattered;
negative feelings about the mental health system; and services not available through health insurance provider.
Survey Results: Peer Support Services 28
Respondents who did not receive peer support services but wanted them also were asked
in the survey to report the reasons they were not able to receive peer support services.
More than one reason could be selected; therefore, the percentages are based on total
responses rather than the number of respondents. As presented in Figure 15, the most
common reason reported was “I did not know how to apply for services” (24.3%), followed
by “stigma” (e.g., fear or feeling embarrassed) (16.5%) and “I could not pay for the services”
(11.2%).
Figure 15 – Reasons Not Able to Access or Receive Peer Support Services: Respondents Who Did Not Receive
Peer Support Services but Wanted Them
I did not know how to apply for services 41 24.3%
Stigma 28 16.5%
I could not pay for the services 19 11.2%
I did not feel comfortable or welcome 14 8.3%
I was told that I did not qualify 14 8.3%
Services were not available in a location
13 7.8%
that was convenient to me
Services were not available during times
9 5.3%
that were convenient to me
I did not like the services that were
9 5.3%
offered
Services that are sensitive to my cultural
6 3.5%
orientation were not available to me
I was placed on a waiting list 6 3.5%
Services were not available in my
2 1.2%
language
Other* 8 4.8%
0% 25% 50%
N = 120, Total Responses = 169
*Other reasons include: services were not offered (n=5) and one response each for could not arrange a meeting to get
services, did not feel supported enough to obtain services, and did not feel ready to seek services.
Survey Results: Peer Support Services 29
The reasons for not being able to access or receive peer support services were further
examined against respondent characteristics. There was an association between access
barriers and whether or not respondents belonged to an unserved or underserved group (p
< .05). As shown in Table 1, unserved or underserved respondents were more likely to
report that they did not access or receive peer support services because they did not feel
comfortable or welcomed (22.0%), service location was not convenient (22.0%), service
times were not convenient (15.0%), and services were not offered (15.3%).
Table 1 – Reasons for Not Receiving Peer Support Services: Comparison Between
Belonging or Not Belonging to Unserved/Underserved Group
Unserved or Not Unserved or
Underserved Underserved
I did not feel comfortable or 13 (22.0%) 0 (0.0%)
welcomed.1
Services were not available in a location 13 (22.0%) 0 (0.0%)
that was convenient.2
Services were not available during times 9 (15.0%) 0 (0.0%)
that were convenient.3
I did not like the services that were 9 (15.0%) 0 (0.0%)
offered.4
TOTAL N = 60* N = 39*
* The numbers do not add up to the sample size for each group because only the frequency of responses for selecting each
reason is reported in the table.
1 Χ2 = 13.15 (1), p < 0.01; 2 Χ2 = 9.91 (1), p < 0.01; 3 Χ2 = 6.43 (1), p < 0.01; 4 Χ2 = 9.91 (1), p < 0.01
Recovery Orientation of Peer Support Services
The Recovery Oriented System Indicators (ROSI) scores were analyzed for respondents 18
years or older only because the instrument was not intended for children. We analyzed
whether respondents perceived the orientation of mental health services differently if they
received or did not receive peer support services. (Note that respondents who did not
receive peer support services completed the ROSI based on experiences with other mental
health services received currently or in the past.) Using the independent samples t-test, we
found statistically significant differences between the two groups for all six ROSI scales (p <
.05). The findings are presented in Table 2. Respondents who received peer support services
perceived a more positive recovery orientation of services in all six areas: person-centered
focus and directed decision-making, holistic focus, moving on up, basic material resources,
system potholes, and mistreatment. In other words, respondents who received peer
support services reported, on average, that the mental health services they experienced
were more person-centered, more holistic in meeting other needs such as housing, more
oriented toward employment or school stability and/or advancement, more focused on
basic needs such as income and transportation, had less service inadequacies, and were less
oriented toward mistreatment.
Survey Results: Peer Support Services 30
Table 2 – ROSI Results: Comparison Between Respondent Groups
Yes, received peer No, did not receive peer support
support services services but wanted them
Sample Size, Mean (SD) Sample Size, Mean (SD)
Person-centered focus and n = 181, 3.36 (0.66) n = 60, 2.73 (0.72)
directed decision-making1
Holistic focus2 n = 188, 3.16 (0.53) n = 66, 2.77 (0.61)
Moving on up3 n = 189, 2.95 (0.76) n = 66, 2.21 (0.91)
Basic material resources4 n = 235, 2.80 (0.77) n = 76, 2.55 (0.76)
System potholes5 n = 204, 2.03 (0.65) n = 65, 2.48 (0.60)
Mistreatment6 n = 181, 1.56 (0.67) n = 60, 1.92 (0.76)
1 t(239) = 6.28, p < .001; 2 t(252) = 4.89, p < .001; 3 t(235) = 5.91, p < .001; 4 t(309) = 2.60, p < .01; 5 t(267) = 4.99, p <.001;
6 t(239) = 3.48, p < .001
We also tested whether respondents who received employment support services along with
peer support services perceived the recovery orientation of services differently from those
who did not receive employment services. There were no statistically significant differences
in the ROSI scales between these respondents (p > .05). Altogether, the findings indicate
that receiving peer support services is associated with having a more recovery oriented
service experience but that the additional utilization of employment support services does
not change that experience for better or worse.
Furthermore, a Pearson’s r correlation test was used to examine the relationship between
the perception of services as recovery oriented and the number of peer support services
received for respondents who received peer support services. Two of the ROSI scales were
significantly and positively correlated with the number of peer support services received (p
< .05). Respondents who received more types of peer support services perceived that the
mental health services they had received were more holistic (r = 0.19) and more oriented
toward employment or school stability and/or advancement (r = 0.16). Because the
correlations were not particularly strong (i.e., the r is at the lower end of a range of 0 to
1.0), it is with caution that any conclusions are drawn from these findings. The same
statistical test examining length of peer support services did not yield a significant
correlation.
Further analysis of access to and appropriateness of services was conducted using ratings
that were provided by respondents to describe their experience with peer support services.
Results for ratings on three statements about the recovery orientation of peer support
services are presented in Table 3. There was agreement by most respondents that the peer
support services they received were inviting and dignified (78.0% who strongly agreed or
agreed). Most respondents also agreed that the services fit their unique culture and life
Survey Results: Peer Support Services 31
experiences (76.8%) and the services they received were what they wanted (76.7%). Among
these respondents, there were no associations between respondent characteristics and
whether or not there was agreement or disagreement with each of the ratings (p > .05). In
other words, no particular respondent characteristic appeared to be associated with
respondent ratings of their experience with peer support services.
Table 3 – Perceived Access and Appropriateness of Peer Support Services*
Strongly Agree Neutral Disagree Strongly
Agree Disagree
The peer support services I 102 (33.8%) 130 (43.0%) 52 (17.2%) 12 (4.0%) 6 (2.0%)
received fit my unique culture
and life experiences.
The physical space where I 104 (34.2%) 133 (43.8%) 41 (13.5%) 17 (5.6%) 9 (3.0%)
received peer support services
felt inviting and dignified.
The peer support services I 103 (33.9%) 130 (42.8%) 50 (16.4%) 13 (4.3%) 8 (2.6%)
received were what I wanted.
*N = 314
Peer Support Services Impact: Employment, Housing, and Personal Recovery and Wellness
Overall Perceived Impact of Peer Support Services
Respondents who received peer support services were asked to rate the extent to which
the services they received impacted different aspects of their life and recovery. (Parents of
children represented in the survey would have provided ratings based on their experience
with parent/family partnership supports). The results are presented in Table 4. For the most
part, respondents either strongly agreed or agreed with each statement in the scale. There
was agreement by a large majority of respondents (81.3% who strongly agreed or agreed)
that peer support services made them feel better. This was also the case when asked about
services helping with recovery (76.9%) and removing stigma of receiving mental health
services (71.9%). In terms of concrete services, most respondents agreed (71.7%) that peer
support services helped improve their living situation, but a noticeable decline in agreement
was seen in the response regarding employment. That is, 52.7% strongly agreed or agreed
that peer support services improved their employment situation. The remaining proportions
of respondents for each statement were either neutral in their response or
disagreed/strongly disagreed.
Survey Results: Peer Support Services 32
Table 4 – Perceived Impact of Peer Support Services*
Strongly Agree Neutral Disagree Strongly
Agree Disagree
The peer support services I 115 (37.7%) 133 (43.6%) 41 (13.4%) 8 (2.6%) 8 (2.6%)
received helped me feel better.
The peer support services I 105 (37.4%) 111 (39.5%) 48 (17.1%) 9 (3.2%) 8 (2.8%)
received helped with my
recovery.
Peer support services helped 92 (34.1%) 102 (37.8%) 53 (19.6%) 13 (4.8%) 10 (3.7%)
remove of the stigma (fear or
shame) of receiving mental
health services.
The peer support services I 90 (31.5%) 115 (40.2%) 50 (17.5%) 21 (7.3%) 10 (3.5%)
received helped improve my
living situation.
The peer support services I 54 (24.1%) 64 (28.6%) 58 (25.9%) 29 (12.9%) 19 (8.5%)
received helped improve my
employment situation.
*N = 314
Employment
Overall, there was no association between current and desired employment or school
status and whether or not peer support services were received (p > .05). Table 5 presents
three major combinations of current and desired employment or school status. The
proportion of respondents who reported being currently employed or attending school and
desired this status was similar between respondents who received peer support services
(26.5%) and those who did not receive peer support services despite wanting them (25.8%).
Also, the proportion of respondents who reported being currently unemployed but desiring
work was similar between respondents who received peer support services (9.5%) and
those who did not (8.3%). While there was no statistical association and the sample size was
small, a noticeably higher proportion of respondents who did not receive peer support
services were disabled currently but desired to work, attend school, attend a work training
program, or be a paid artist (9.1%) compared to their counterparts who received peer
support services (5.4%). In all other possible combinations of current and desired
employment or school status, the proportions between the two groups were similar.
Survey Results: Peer Support Services 33
Table 5 – Current and Desired Employment or School Status: Comparison Between Respondent
Groups
Yes, received peer No, did not receive peer support
support services services but wanted them
Employed or attending school: 87 (26.5%) 31 (25.8%)
currently and desired
Unemployed currently but desires to 31 (9.5%) 10 (8.3%)
work
Disabled currently but desires to 18 (5.4%) 11 (9.1%)
work, attend school, be in a work
training program, or be a paid artist
TOTAL N = 328* N = 120*
* The numbers do not add up to the sample size for each group because only the frequency of responses for the selected
employment or school status is reported in the table.
Housing
Overall, there was no association between current and desired living situation and whether
or not peer support services were received (p > .05). Table 6 presents two major
combinations of current and desired living situation. The proportion of respondents who
reported living independently and desiring it was similar between respondents who
received peer support services (41.5%) and those who did not receive peer support services
despite wanting them (43.6%). Also, the proportion of respondents who reported not living
independently (e.g., with friends/family, in a group home, or homeless) but desiring to live
independently was similar between respondents who received peer support services
(25.3%) and those who did not (23.2%). In all other possible combinations of current and
desired living situation, the proportions between the two groups were similar.
Table 6 – Current and Desired Living Situation: Comparison Between Respondent Groups
Yes, received peer No, did not receive peer support
support services services but wanted them
Lives independently: currently and desired 130 (41.5%) 51 (43.6%)
Does not live independently, but desires to 80 (25.3%) 27 (23.2%)
live independently
TOTAL N = 313* N = 117*
* The numbers do not add up to the sample size for each group because only the frequency of responses for the selected
living situation is reported in the table.
Survey Results: Peer Support Services 34
Personal Recovery and Wellness
The Recovery Process Inventory (RPI) scores were analyzed for respondents 18 years or
older; therefore, results on personal recovery and wellness using the RPI do not include
children. The findings are presented in Table 7. Comparing respondents who received peer
support services and those who did not despite wanting them, we found statistically
significant differences in all six scales between the two groups (p < .05). All the differences
were in favor of respondents who received peer support services. In contrast to
respondents who did not receive peer support services but wanted them, respondents who
received peer support services perceived or felt less anguish and isolation, more connected
to others, more confident about life, more surrounded by people who care, more housing
stability, and more hopeful.
Table 7 – RPI Results: Comparison Between Respondent Groups
Yes, received peer No, did not receive peer support
support services services but wanted them
Sample Size, Mean (SD) Sample Size, Mean (SD)
Anguish1 n = 250, 2.52 (0.78) n = 86, 2.80 (0.73)
Connected to Others2 n = 248, 3.59 (0.83) n = 83, 3.23 (0.88)
Confidence and Purpose3 n = 254, 3.70 (0.88) n = 86, 3.23 (0.98)
Others’ Care/Help4 n = 243, 3.56 (0.76) n = 83, 3.16 (0.69)
Living Situation5 n = 251, 3.72 (1.07) n = 83, 3.29 (1.10)
Hopeful/Cares for Self6 n = 247, 4.16 (0.69) n = 85, 3.94 (0.88)
1 t(334) = 2.84, p < .01; 2 t(329) = 3.31, p < .001; 3 t(338) = 4.10, p < .001; 4 t(324) = 4.19, p < .001;
5 t(332) = 3.17, p < .01; 6 t(330) = 2.30, p < .05
A correlational analysis of the RPI scales and the ROSI scales further supported these
findings. For respondents who received peer support services, there were many significant
correlations between the RPI scales and the ROSI scales (p < .05).19 For example, there was
a positive relationship between feeling more anguish and perceiving more system potholes;
there was a positive relationship between feeling more connected to others and perceiving
services to be more person-centered; and there was a positive relationship between feeling
more confident and perceiving greater employment or school advancement.
For respondents who did not receive peer support services despite wanting them, there
were fewer significant correlations between perceptions of personal recovery and
perceptions of the recovery orientation of services. For example, there were no
19
The Pearson’s r ranged from 0.18 to 0.57 for significant correlations for respondents who received peer
support services (n = 169-261 depending on the analysis). The Pearson’s r ranged from 0.33 to 0.51 for
significant correlations for respondents who did not receive peer support services but wanted them (n = 54-71
depending on the analysis).
Survey Results: Peer Support Services 35
relationships between feeling connected to others and any ROSI scale, whereas for
respondents who received peer support services, feeling connected to others was
correlated with four ROSI scales (person-centered focus, system potholes, moving on up,
and basic materials). Altogether, these findings suggest that more positive experiences with
peer support services and other mental health services are related to a more positive
perception of personal recovery.
Furthermore, for respondents who received peer support services, the number of services
and the length of services were significantly correlated with scales on the RPI (p < .05). First,
respondents who received more types of peer support services were more hopeful (r =
0.15). Second, respondents who received peer support services for a longer period of time
perceived that there were fewer people surrounding them who cared (r = 0.13). A closer
examination revealed that this inverse relationship was particularly evident among
respondents who received more than six months of peer support services. Because the
correlations were not particularly strong (i.e., the r is at the lower end of a range of 0 to
1.0), it is with caution that any conclusions are drawn from these findings.
We further analyzed whether characteristic differences between respondents who received
peer support services and those who did not accounted for the results we found above.
There were variations within both groups of respondents in terms of age, race/ethnicity,
educational attainment, and seriousness of the mental health concern. These variations
were similar across the groups. For example, in both groups, respondents who described
their mental health concern to be extremely serious felt more anguish than respondents
who described their concern to be mild. Based on these findings, we analyzed whether
these characteristics, including respondent perceptions of the recovery process, were
predictors of membership in one group or another using logistic regression. Only one of the
scales – namely, others’ care/help – significantly “discriminated” respondents who received
peer support services from those who did not receive these services (p < .05).20 As
previously reported, respondents who received peer support services (compared to those
who did not), on average, had a more positive perception of personal recovery in terms of
feeling that more understanding people were around and that others care when they
themselves do not.
20 The model test result including 270 respondents was Χ2 = 35.77 (27), p < .05. The results of the predictor
(“others’ care/help” scale on the RPI) are: B = 0.54, S.E. = 0.27, Wald’s Χ2 = 3.94 (1), p < .05, and odds ratio =
0.58.
Survey Results: Peer Support Services 36
Employment Support Service
Characteristics of Individuals Who Received Employment Support Services
The characteristics of individuals (or “respondents”) who received employment support
services are described and are compared with survey respondents who did not receive
employment support services despite needing or wanting them.
A total of 624 adult respondents reported whether or not they received employment
support services in the past year.21 As displayed in Figure 16, 156 (or 25.0%) of these 624
respondents reported having received employment support services. Furthermore, as
presented in Figure 17, 107 respondents (or 25.8%) who did not receive employment
support services reported needing or wanting them.
Figure 16 – Employment Support Services Wanted/Not Wanted
100%
75.0%
75%
50%
25.0%
25%
0%
Yes, received employment support No, did not receive employment
services (n=156) support services (n=468)
N = 624
21 Because employment support services are typically offered to adults, the analyses on employment were
conducted for transition age youth, adults, and older adults only.
Survey Results: Employment Support Services 37
Figure 17 – Employment Support Services Wanted/Not Wanted
100%
74.2%
75%
50%
25.8%
25%
0%
Did not receive employment support services Did not receive employment support services
but wanted or needed them (n=107) and did not want or need them (n=307)
N = 414
(There are 54 missing responses from the sample of 414 respondents who reported that they did not receive employment
support services.)
Of the 156 respondents who received employment support services, 127 reported the
number of months they received employment support services in the past year. As shown in
Figure 18, 67.7% reported receiving less than six months of employment support services.
The remaining 32.3% reported receiving six months or more of employment support
services. On average, respondents reported receiving less than six months of employment
support services (standard deviation of 4.2 months). Altogether, the majority had received
employment support services for less than six months in the past year.
Figure 18 - Length of Employment Support Services Received
6 months
or more
41 (32.3%)
Less than 6
months
86 (67.7%)
N = 127
Survey Results: Employment Support Services 38
The following tables provide the distribution of two sets of respondents. We often refer to
these two sets of respondents as “groups”. One group includes the respondents who
received employment support services. The second group includes the respondents who did
not receive employment support services despite needing or wanting them. In the tables
below, we present the distribution of these two groups side by side to compare whether
there is a statistically significant association (using a chi-square test) between respondent
characteristics/demographics and whether or not they received employment support
services.
Starting with an analysis on age, there was no association between age group and whether
or not employment support services were received (p > .05). As shown in Figure 19, the
distributions of age across both groups were comparable, with approximately 68.0% of
adults in both groups representing the largest age group for this sample.
Figure 19 – Employment Support Services Received/Not Received by Age Group
100%
75%
67.8% 68.0%
Yes, received
employment support
50% services (n=143)
No, did not receive
27.2%
employment support
25% 21.0% services but wanted
them (n=103)
11.2%
4.9%
0%
TAY (n=58) Adult (n=167) Older Adult (n=21)
N = 246
Survey Results: Employment Support Services 39
There was no association between gender and whether or not employment support
services were received (p > .05). As presented in Figure 20, 52.3% were female and 46.4%
were male among respondents who received employment support services. This is
compared to 45.3% of females and 52.8% respondents who did not receive employment
support services. Therefore, a slightly higher proportion of females were represented in the
group that received employment support services; however, the differences were not
pronounced enough to indicate a statistical association.
Figure 20 - Employment Support Services Received/Not Received by Gender
100%
75%
Yes, received
52.3% 52.8% employment support
50% 45.3% 46.4% services (n=156)
No, did not receive
employment support
services but wanted
25%
them (106)
1.3% 1.9%
0%
Female (n=128) Male (n=127) Transgendered (n=7)
N = 262
Survey Results: Employment Support Services 40
There was a significant association between race/ethnicity and whether or not employment
support services were received (p < .05). For example, as presented in Figure 21, 20.3% of
respondents who received employment support services were African American compared
to 7.5% who did not receive these services. This was the starkest difference, suggesting that
representation of African American respondents was higher in the group that received
employment support services than the group that did not receive such services despite
wanting them. Due to the relatively small sample size of African American respondents in
the group that did not receive employment support services, further analysis to assess the
potential effects of this association of the results was not conducted.
Figure 21 - Employment Support Services Received/Not Received by Race
100%
Yes, received
75% employment
support services
(n=153)
50% 43.9%
37.3% No, did not receive
30.8% employment
24.8%
25% 20.3% support services
13.7% but wanted them
10.3%
7.5% 7.5%
3.9% (107)
0%
African Asian and Caucasian Latino (n=71) Other* (n=32)
American (n=39) Pacific Islander (n=104)
(n=14)
N = 260
Χ2 = 10.478 (4), p < .05
* “Other” includes Native American, mixed race, and other ethnic groups not identified within the racial groups provided
as answer options in the survey.
Survey Results: Employment Support Services 41
Because of the larger representation of respondents in the Southern region, it was expected
that most respondents receiving employment support services (and even those not
receiving them) would reside in the Southern region. The distribution of respondents in
both groups is presented in Figure 22. There was no association between regional counties
and whether or not employment support services were received (p > .05). However, slight
proportional differences between the two groups were noticeable. For example, while
63.3% of respondents who received employment support services resided in the Southern
region, 51.0% of those who did not receive these services resided there. Consequently,
slightly larger proportions of respondents in the Central region and Bay Area did not receive
employment support services.
Figure 22 - Employment Support Services Received/Not Received by Region
100%
Yes, received
employment
75% support services
63.3%
(n=150)
51.0%
50%
No, did not
receive
25% 14.7% 19.2% 17.3% 12.5% employment
10.7% 11.3% support services
but wanted them
0%
(n=104)
Southern (n=148) Central (n=42) Bay Area (n=34) Northern (n=30)
N = 254
There was no association between county size and whether or not employment support
services were received (p > .05). As shown in Figure 23, less than 5.0% of respondents from
each group were from small counties. In contrast, the vast majority of respondents from
each group resided in larger counties (95.3% for those who received employment support
services and 95.8% who did not but wanted them).
Figure 23 – Employment Support Services Received/Not Received by County Size
100% 95.3% 95.8%
Yes, received
employment
75% support services
(n=150)
50%
No, did not
receive
25% employment
support services
4.7% 4.2%
but wanted them
0% (n=118)
Small county (n=12) Larger county (n=256)
N = 268
Survey Results: Employment Support Services 42
There was no association between annual income and whether or not employment support
services were received (p > .05). As shown in Figure 24, most respondents in both groups
earned less than $15,000 annually (56.3% and 58.3%). For respondents who received
employment support services, 18.5% had no individual income. For respondents who did
not receive employment support services but wanted them, a larger proportion of 28.2%
had no individual income.
Figure 24 – Employment Support Services Received/Not Received by Annual Income
100%
Yes, received
75% employment
58.3% support services
56.3% (n=151)
50%
28.2%
No, did not
25% 18.5% receive
11.9%
8.7% 6.6% 3.3% 3.3% employment
1.0% 2.9% 1.0% support services
0% but wanted them
No individual Less than $15,000 - $30,001 - $45,001 - Over $60,000 (n=103)
income (n=57) $15,000 (n=145) $30,000 (n=27) $45,000 (n=11) $60,000 (n=8) (n=6)
N = 254
Survey Results: Employment Support Services 43
There was no association between educational attainment and whether or not employment
support services were received (p > .05). However, the slightly different proportions
between the two groups are noteworthy. As shown in Figure 25, for respondents who
received employment support services, 11.8% had less than a high school education while
27.0% of respondents were college graduates. In contrast, for respondents who did not
receive employment support services but had wanted them, 16.2% did not have a high
school education while 18.1% of respondents were college graduates. Therefore, a slightly
larger proportion of respondents who received employment support services reported
having graduated from college or graduate school.
Figure 25 – Employment Support Services Received/Not Received by Educational Attainment
50%
40%
33.3%
32.4% 32.2%
28.9%
30%
27.0%
Yes, received
employment support
services (n=152)
20% 18.1%
16.2%
11.8% No, did not receive
10% employment support
services but wanted
them (n=105)
0%
Less than High High School Vocational/Some College Grad or
School (n=35) Grad/GED (n=78) College (n=84) Above (n=60)
N = 257
Survey Results: Employment Support Services 44
There was no association between the seriousness of respondents’ mental health concern
and whether or not employment support services were received (p > .05). As shown in
Figure 26, over 60.0% of respondents in both groups described their concern to be serious
or extremely serious, indicating that many respondents who received employment support
services, as well as those who did not despite wanting them, had serious concerns with
their mental health while receiving and seeking employment support services.
Figure 26 – Employment Support Services Received/Not Received by Seriousness of Mental Health Concern
50%
40% Yes, received
34.2% 34.2% employment
32.7% 32.7%
support services
30% (n=146)
23.8%
18.5% No, did not receive
20%
employment
13.0%
support services but
10.9%
wanted them
10%
(n=101)
0%
A mild concern A moderate concern A serious concern An extremely serious
(n=30) (n=51) (n=83) concern (n=83)
N = 247
Survey Results: Employment Support Services 45
There was no association between belonging to an unserved or underserved group and
whether or not employment support services were received (p > .05). (See Appendix A for
details on the different characteristics of individuals belonging to an unserved or
underserved group.) As shown in Figure 27, regardless of whether or not respondents
received employment support services, larger proportions of respondents identified
themselves as unserved or underserved. Between groups, there was no statistical
association but the proportions were slightly different in that 61.5% of respondents who
received employment support services identified as unserved or underserved, while a
higher proportion of 70.1% of respondent who did not receive these services identified as
unserved or underserved.
As with the findings on peer support services, it is important to understand that these
distributions refer to sample representation and do not necessarily indicate the extent to
which unserved or underserved respondents had access to services. In the overall survey
sample, 52.0% of respondents belonged to an unserved or underserved group. Therefore,
for both groups, the distribution of respondents is such that a larger proportion of unserved
or underserved respondents (e.g., physically disabled) were represented in the analyses on
employment support services.
Figure 27 – Employment Support Services Received/Not Received by Unserved/Underserved Group
100%
75% 70.1%
61.5%
Yes, received
employment support
50%
services
38.5%
29.9%
No, did not receive
25% employment support
services but wanted
them
0%
Unserved/underserved (n=171) Not Unserved/underserved (n=92)
N = 263
Survey Results: Employment Support Services 46
Types of Employment Support Services Received
Respondents who received employment support services were asked in the survey to
identify the types of services they received in the past year. Figure 28 lists the services and
responses. It is important to note that more than one type of service could be selected.
Therefore, the percentages are based on total responses rather than the number of
respondents in order to assess the relative commonality of employment support services
received by respondents. The two most common employment support services reported
were help preparing a résumé (18.5%) and help preparing for an interview (16.3%). Other
services reported were: job placement services (14.7%), vocational training (12.7%), job
coaching or employment counseling (12.3%), diagnostic services or vocational evaluation
(9.1%), peer employment supports (8.2%), and supported or sheltered employment (4.6%).
On average, respondents reported receiving three types of employment support services in
the past year (standard deviation of 2.1 services).22
Figure 28 - Employment Support Services Received
Help preparing a resume 77 18.5%
Help preparing for an interview 68 16.3%
Job placement services 61 14.7%
Vocational or employment training 53 12.7%
Job coaching or employment counseling 51 12.3%
Diagnostic services or vocational evaluation 38 9.1%
Peer employment supports 34 8.2%
Supported or sheltered employment 19 4.6%
Other* 15 3.6%
0% 10% 20% 30%
N = 156, Total Responses = 416
* Other responses include volunteering (n=3), Job training Employment Assistance Program (EAP), English as Second
Language (ESL), internships, and help with social security or other benefits.
22
The types of employment support services were further analyzed to identify whether certain characteristics
of respondents were associated with having received one of more of these services. A number of significant
associations were found (p < .05); however, they were not reported because the sample sizes were too
unstable to draw any conclusions from the findings.
Survey Results: Employment Support Services 47
Access to Employment Support Services
Respondents who received employment support services were asked in the survey to report
whether or not they experienced any difficulties in getting employment support services.
Figure 29 presents the difficulties reported by 21.1% of respondents (or 28 out of 133) who
received these services. More than one difficulty could be selected; therefore, the
percentages are based on total responses rather than the number of respondents. Three
difficulties were equally reported: being put on a wait list (17.2%), services not being
available in a convenient location (17.2%), and services not being provided during
convenient times (17.2%). Other difficulties reported included having to try several times to
receive services before being able to (15.5%) and not feeling comfortable or welcomed
(13.8%). A handful of respondents reported difficulties accessing employment support
services due to services not being offered by staff members who share their cultural
background (6.9%) and not being available in their primary language (3.4%).
Figure 29 – Difficulties Receiving Employment Support Services: Respondents Who Received Employment
Support Services
The services were not offered during
10 17.2%
times that were convenient to me
The services were not available in a
10 17.2%
convenient location for me
I was put on a waiting list or experienced
10 17.2%
other delays
I tried to get these services several times
9 15.5%
before I was able to
I did not feel comfortable of welcomed 8 13.8%
The services were not offered by staff
4 6.9%
who share my cultural background
The services were not offered in my
2 3.4%
primary language
Other* 5 8.6%
0% 10% 20% 30%
N = 28, Total Responses = 58
* Other responses include stolen identity, discontinuation of services, being taken advantage of, had minimal contact with
staff, and staff not recognizing need for help due to education.
Survey Results: Employment Support Services 48
Respondents who did not receive employment support services but wanted them also were
asked in the survey to report the reasons they were not able to receive employment
support services. More than one reason could be selected; therefore, the percentages are
based on total responses rather than the number of respondents. As presented in Figure
30, at 24.6%, not knowing how to apply for services was the most common reason reported
for not receiving needed or wanted employment support services. Other common reasons
included stigma (e.g., fear or embarrassment) associated with receiving services (13.8%),
being told that the individual did not qualify (9.4%), and not feeling comfortable or
welcomed (8.9%).
Figure 30 – Reasons for Not Receiving Needed or Wanted Employment Support Services: Respondents Who
Did Not Receive Employment Support Services
I did not know how to apply for services 51 25.5%
Stigma 28 14.0%
I was told that I did not qualify 20 10.0%
Services were not available in a location
19 9.5%
that was convenient to me
I did not feel comfortable or welcomed 19 9.5%
I could not pay for the services 12 6.0%
Services were not available during times
10 5.0%
that were convenient to me
I was placed on a waiting list 10 5.0%
I did not like the services that were
9 4.5%
offered
Services were not available in my
6 3.0%
language
Services that are sensitive to my cultural
5 2.5%
orientation were not available to me
Other* 11 5.5%
0% 10% 20% 30%
N = 108, Total Responses = 200
* Other reasons include being incarcerated, suffering from physical illness, attending school, no follow-up after job loss,
being a parent, applying for SSI.
Survey Results: Employment Support Services 49
The reasons for not being able to access or receive employment support services were
further examined against respondent characteristics. All the associations presented below
were statistically significant (p < .05). As shown in Table 8, larger proportions of
respondents in the Northern region (53.8%) and Central region (35.0%) reported stigma as a
reason for not receiving employment support services.
Table 8 – Stigma: Comparison by Regions
Southern Central Bay Area Northern
Yes, stigma 9 (17.0%) 7 (35.0%) 5 (27.8%) 7 (53.8%)
No 44 (83.0%) 13 (65.0%) 13 (72.2%) 6 (46.2%)
TOTAL 53 (100.0%) 20 (100.0%) 18 (100.0%) 13 (100.0%)
Χ2 = 8.12 (3), p < .05
As shown in Table 9, a larger proportion of respondents who described their mental health
concern as serious (60.6%) or extremely serious (57.6%) reported that they did not know
how to apply for services.
Table 9 – Did Not Know How to Apply: Comparison by Seriousness of Concern
Mild Concern Moderate Serious Extremely
Concern Concern Serious Concern
Yes, did not know how to apply 3 (27.3%) 5 (20.8%) 20 (60.6%) 19 (57.6%)
No 8 (72.7%) 19 (19.2%) 13 (39.4%) 14 (42.4%)
TOTAL 11 (100.0%) 24 (100.0%) 33 (100.0%) 33 (100.0%)
Χ2 = 12.26 (3), p < .01
As shown in Table 10, a larger proportion of respondents who belonged to an unserved or
underserved group (13.3%) reported having been put on a waiting list.
Table 10 – Placed on Waiting List: Comparison Between Belonging or
Not Belonging to an Unserved/Underserved Group
Unserved or Not Unserved or
Underserved Underserved
Yes, placed on waiting list 10 (13.3%) 0 (0.0%)
No 65 (86.7%) 32 (100.0%)
TOTAL 75 (100.0%) 32 (100.0%)
Χ2 = 4.71 (1), p ≤ .05
Survey Results: Employment Support Services 50
Finally, as shown in Table 11, a larger proportion of respondents whose preferred language
is Spanish (25.0%) reported that services were not available in their language (compared to
only 2.3% of respondents whose preferred language is English).
Table 11 – Services Not Available in Preferred Language:
Comparison by Preferred Language
English Spanish
Yes, services not available 2 (2.3%) 4 (25.0%)
in preferred language
No 86 (97.7%) 12 (75.0%)
TOTAL 88 (100.0%) 16 (100.0%)
Χ2 = 13.22 (1), p ≤ .01
Recovery Orientation of Employment Support Services
The Recovery Oriented System Indicators (ROSI) scores were analyzed for respondents 18
years or older only because the instrument was not intended for children. Furthermore,
employment support services are not intended for children. We analyzed whether
respondents perceived the orientation of mental health services differently if they received
or did not receive employment support services. (Note that respondents who did not
receive employment support services completed the ROSI based on experiences with other
mental health services received currently or in the past.) Using the independent samples t-
test, we found significant differences between the two groups on five of the six ROSI scales
(p < .05). As shown in Table 12, respondents who received employment support services
had higher average scores on the following scales: person-centered focus and directed
decision-making, holistic focus, moving on up, and basic material resources. Also,
respondents who received employment support services had significantly lower scores on
the system potholes scale. Therefore, respondents who received employment support
services reported, on average, that the mental health services they experienced were more
person-centered, more holistic in meeting other needs such as housing, more oriented
toward employment or school stability and/or advancement, more focused on basic needs
such as income and transportation, and had less service inadequacies.
Survey Results: Employment Support Services 51
Table 12 – ROSI Results: Comparison Between Respondent Groups
Yes, received employment No, did not receive employment
support services support services but wanted them
Sample Size, Mean (SD) Sample Size, Mean (SD)
Person-centered focus and n = 113, 3.27 (0.65) n = 69, 2.82 (0.80)
directed decision-making1
Holistic focus2 n = 122, 3.13 (0.50) n = 76, 2.78 (0.61)
Moving on up3 n = 129, 2.91 (0.71) n = 74, 2.13 (0.76)
Basic material resources4 n = 141, 2.75 (0.75) n = 94, 2.42 (0.72)
System potholes5 n = 125, 2.07 (0.62) n = 75, 2.48 (0.69)
Mistreatment n = 113, 1.57 (0.69) n = 63, 1.74 (0.60)
1 t(180) = 4.17, p < .001; 2 t(196) = 4.42, p < .001; 3 t(201) = 7.38, p < .001; 4 t(233) = 3.40, p < .001;
5 t(198) = -4.36, p < .001
We also tested whether respondents who received employment support services plus peer
support services perceived the recovery orientation of services differently from those who
did not receive peer support services. There were statistically significant differences in three
ROSI scales between these respondents (p < .05). As shown in Table 13, those respondents
who received both sets of services perceived that services were more person-centered,
more holistic in meeting other needs such as housing, and more oriented toward
employment or school stability and/or advancement. The respondents who only received
employment support services reported, on average, that services received were less
recovery oriented in these areas.
Table 13 – ROSI Results: Comparison Between Respondents Who Received Peer Support
Services and Those Who Did Not
Yes, also received peer No, only received employment
support services and not peer support services
Sample Size, Mean (SD) Sample Size, Mean (SD)
Person-centered focus and n = 58, 3.38 (0.64) n = 13, 2.81 (0.74)
directed decision-making1
Holistic focus2 n = 63, 3.20 (0.47) n = 13, 2.89 (0.60)
Moving on up3 n = 70, 3.07 (0.65) n = 13, 2.42 (1.01)
1 t(69) = 2.82, p < .01; 2 t(74) = 2.02, p < .05; 3 t(81) = 3.00, p < .01
Furthermore, a Pearson’s r correlation test was used to examine the relationship between
the perception of services as recovery oriented and the number of employment support
services received for respondents who received employment support services. There was
Survey Results: Employment Support Services 52
no significant correlation between these two variables (p > .05). The same statistical test
was used to examine length of employment support services; the test did not yield any
statistically significant correlations (p > .05).
Further analysis of access to and appropriateness of services was conducted using ratings
provided by respondents to describe their experience with employment support services.
Results for ratings on six statements about the recovery orientation of peer support services
are presented in Table 14. The majority of respondents agreed/strongly agreed that staff
had worked hard to achieve their personal employment goals (69.3%), services fit with the
respondents cultural and life experiences (56.7%), staff respected respondents’ decisions
about employment goals (74.7%), the physical space where employment supports were
received was inviting and dignified (72.2%), staff included people that were important to
respondents in achieving employment goals (63.3%), and employment supports received
were what the respondent wanted (68.3%).
Among these respondents, there was one association between gender and whether or not
there was agreement or disagreement with the ratings (p < .05). A larger proportion of male
respondents agreed that the employment support services received fit their unique culture
and life experiences. Of the 70 male respondents who received employment support
services, 65.7% agreed or strongly agreed with the statement. In contrast, of the 63 female
respondents who received employment support services, 47.6% agreed or strongly agreed
with the statement.
Table 14 – Perceived Access and Appropriateness of Employment Support Services*
Strongly Agree Neutral Disagree Strongly
Agree Disagree
Staff worked hard to help me fulfill 38 (26.6%) 61 (42.7%) 33 (23.1%) 7 (4.9%) 4 (2.8%)
my personal employment goals.
The employment support services I 30 (22.1%) 47 (34.6%) 44 (32.4%) 9 (6.6%) 6 (4.4%)
received fit my unique culture and
life experiences.
Staff listened to me and respected 44 (31.0%) 62 (43.7%) 28 (19.7%) 3 (2.1%) 5 (3.5%)
my decisions about my
employment goals.
The physical space where I received 41 (29.9%) 58 (42.3%) 27 (19.7%) 7 (5.1%) 4 (2.9%)
employment support services felt
inviting and dignified.
Staff helped me include people who 37 (28.2%) 46 (35.1%) 36 (27.5%) 10 (7.6%) 2 (1.5%)
are important to me in my efforts
to achieve my employment goals.
The employment support services 33 (23.7%) 62 (44.6%) 30 (21.6%) 10 (7.2%) 4 (2.9%)
that I received were what I wanted.
*N = 156
Survey Results: Employment Support Services 53
Employment Support Services Impact: Employment, Housing, and Personal Recovery and
Wellness
Overall Perceived Impact of Employment Support Services
Respondents who received employment support services were asked to rate the extent to
which the services they received impacted their everyday lives in terms of living situation,
employment situation, and wellbeing. As shown in Table 15, the majority of respondents
either agreed or strongly agreed that the employment supports received improved their
situation in each instance. Most respondents agreed that the employment support services
received improved their living situation (64.3%) and improved their employment situation
(67.2%). The strongest agreement among respondents was in improvements to general
wellbeing (73.3%).
Table 15 – Overall Experience with Employment Support Services (N = 156)
Strongly Agree Neutral Disagree Strongly
Agree Disagree
The employment support services 27 (20.9%) 56 (43.4%) 28 (21.7%) 14 (10.9%) 4 (3.1%)
that I received helped improve my
living situation.
The employment support services 35 (26.7%) 53 (40.5%) 27 (20.6%) 13 (9.9%) 3 (1.9%)
that I received helped improve my
employment situation.
The employment support services 38 (28.1%) 61 (45.2%) 25 (18.5%) 6 (4.4%) 5 (3.7%)
I received helped improve my
current wellbeing.
Employment
Overall, there was no association between current and desired employment or school
status and whether or not employment support services were received (p > .05). Table 16
presents three major combinations of current and desired employment or school status.
The proportion of respondents who reported being currently employed or attending school
and desired this status was slightly different between respondents who received
employment support services (30.1%) and those who did not receive employment support
services despite wanting the services (21.0%). Although these proportions appear to be in
favor of respondents who received employment support services, there was no statistical
association. The proportion of respondents who reported being currently unemployed but
desiring work was similar between respondents who received employment support services
(22.0%) and those who did not (21.9%). Additionally, the proportion of respondents who
reported being disabled but desiring to work, attend school, be in a work training program,
Survey Results: Employment Support Services 54
or be a paid artist was similar between respondents who received employment support
services (8.8%) and those who did not (11.4%). In all other possible combinations of current
and desired employment or school status, the proportions between the two groups were
similar.
Table 16 – Current and Desired Employment or School Status: Comparison Between Respondent Groups
Yes, received employment No, did not receive employment
support services support services but wanted them
Employed or attending school: 41 (30.1%) 22 (21.0%)
currently and desired
Unemployed currently but desires to 30 (22.0%) 23 (21.9%)
work
Disabled currently but desires to 12 (8.8%) 12 (11.4%)
work, attend school, be in a work
training program, or be a paid artist
TOTAL N = 136* N = 105*
* The numbers do not add up to the sample size for each group because only the frequency of responses for the selected
employment or school status is reported in the table.
Housing
Overall, there was no association between current and desired living situation and whether
or not employment support services were received (p > .05). Table 17 presents two major
combinations of current and desired living situation. The proportion of respondents who
reported living independently and desiring it was similar between respondents who
received employment support services (36.4%) and those who did not receive employment
support services despite wanting them (35.5%). Also, the proportion of respondents who
reported not living independently (e.g., with friends/family, in a group home, or homeless)
but desiring to live independently was similar between respondents who received
employment support services (17.1%) and those who did not (17.8%). In all other possible
combinations of current and desired living situation, the proportions between the two
groups were similar.
Table 17 – Current and Desired Living Situation: Comparison Between Respondent Groups
Yes, received employment No, did not receive employment
support services support services but wanted them
Lives independently: currently and 51 (36.4%) 38 (35.5%)
desired
Does not live independently, but 24 (17.1%) 19 (17.8%)
desires to live independently
TOTAL N = 140* N = 107*
* The numbers do not add up to the sample size for each group because only the frequency of responses for the selected
living situation is reported in the table.
Survey Results: Employment Support Services 55
Personal Recovery and Wellness
The Recovery Process Inventory (RPI) scores were analyzed for respondents 18 years or
older; therefore, results on personal recovery and wellness using the RPI do not include
children. Comparing respondents who received employment support services and those
who did not despite wanting them, we found statistically significant differences in five of
the six scales between the two groups (p < .05). The findings are presented in Table 18. All
the differences were in favor of respondents who received employment support services. In
contrast to respondents who did not receive employment support services but wanted
them, respondents who received employment support services perceived or felt less
anguish and isolation, more connected to others, more confident about life, more
surrounded by people who care, and more housing stability.
Table 18 – RPI Results: Comparison Between Respondent Groups
Yes, received employment No, did not receive
supports services employment support services
Sample Size, Mean (SD) Sample Size, Mean (SD)
Anguish1 n = 147, 2.51 (0.77) n = 102, 2.77 (0.86)
Connected to Others2 n = 145, 3.58 (0.86) n = 99, 3.21 (0.89)
Confidence and Purpose3 n = 148, 3.74 (0.84) n = 102, 3.37 (1.07)
Others’ Care/Help4 n = 146, 3.59 (0.81) n = 99, 3.25 (0.80)
Living Situation5 n = 147, 3.72 (1.05) n = 103, 3.27 (1.09)
Hopeful/Cares for Self n = 144, 4.08 (0.74) n = 101, 4.05 (0.75)
1 t(247) = -2.47, p < .05; 2 t(242) = 3.34, p < .001; 3 t(248) = 3.13, p < .01; 4 t(243) = 3.19, p < .01;
5 t(248) = 3.29, p < .001
A correlational analysis of the RPI scales and the ROSI scales further supported these
findings. For respondents who received employment support services, there were many
significant correlations between the RPI scales and the ROSI scales (p < .05).23 For example,
there was a positive relationship between feeling more anguish and perceiving more system
potholes; there was a positive relationship between feeling more connected to others and
perceiving services to be more holistic; and there was a positive relationship between
feeling more confident and perceiving greater employment or school advancement.
For respondents who did not receive employment support services despite wanting them,
there were fewer significant correlations between perceptions of personal recovery and
perceptions of the recovery orientation of services. Only one correlation was found:
respondents who perceived a more holistic orientation of services also tended to perceive
greater hopefulness in terms of personal recovery. In contrast, for respondents who
23
The Pearson’s r ranged from 0.18 to 0.67 for significant correlations for respondents who received peer
support services (n = 107-133 depending on the analysis). The Pearson’s r ranged from 0.25 to 0.64 for
significant correlations for respondents who did not receive peer support services but wanted them (n = 68-
100 depending on the analysis).
Survey Results: Employment Support Services 56
received employment support services, services perceived to be more holistic, more
focused on employment and school stability and/or advancement, and more focused on
basic material resources were all correlated with greater hopefulness in terms of personal
recovery. Altogether, these findings suggest that more positive experiences with
employment support services and other mental health services are related to a more
positive perception of personal recovery.
Furthermore, a correlation test was used to examine the relationship between perceived
personal recovery and the number of employment support services received for
respondents who received employment support services. There was no significant
correlation between these two variables (p > .05). The same statistical test was used to
examine length of employment support services; the test did not yield any statistically
significant correlations (p > .05).
We further analyzed whether characteristic differences between respondents who received
employment support services and those who did not accounted for the results we found
above. There were variations within both groups of respondents in terms of age,
race/ethnicity, and seriousness of the mental health concern. These variations were similar
across the groups. For example, in both groups, respondents who described their mental
health concern to be extremely serious felt less confident than respondents who described
their concern to be mild. Based on these findings, we analyzed whether these
characteristics, including respondent perceptions of the recovery process, were predictors
of membership in one group or another using logistic regression. Only one of the scales –
namely, connected to others – significantly “discriminated” respondents who received
employment support services from those who did not receive these services (p < .05).24 As
previously reported, respondents who received employment support services (compared to
those who did not), on average, had a more positive perception of personal recovery in
terms of feeling more connected to others such as friends.
24 The model test result including 210 respondents was Χ2 = 32.99 (215), p < .05. The results of the predictor
(“connected to others” scale on the RPI) are: B = 0.53, S.E. = 0.26, Wald’s Χ2 = 4.19 (1), p < .05, and odds ratio =
0.59.
Survey Results: Employment Support Services 57
Crisis Intervention Services
Characteristics of Individuals Who Received Crisis Services
The characteristics of individuals (or “respondents”) who received crisis services are
described and are compared to survey respondents who did not receive crisis services
despite needing or wanting them.
A total of 790 respondents reported whether or not they experienced a crisis in the past
year. As presented in Figure 31, 352 (or 45.0%) of these 790 respondents reported having
experienced a crisis in the past year. Of these 352 respondents, 40.0% reported having
experienced one crisis, another 25.0% reported having experienced two crises, and the
remaining 35.0% reported having experienced three or more crises in the past year.
Figure 31 – Crisis Experienced and Number of Crises Experienced in Past Year
100%
75%
55.0%
50% 45.0%
40.0%
35.0%
25.0%
25%
0%
Yes (n=352) No (n=438) 1 crisis (n=143) 2 crises (n=87) 3 or more crises
(n=122)
Experienced a crisis in past year Number of crises experienced in past year
N = 790
Survey Results: Crisis Intervention Services 58
Crisis Services Received or Not Received: Comparison between Respondents
Of the 352 respondents who experienced one or more crises in the past year, 335
responded to the survey question about whether or not they received crisis services after
the crisis. As presented in Figure 32, of these 335 respondents, 231 or 68.9% reported
having received mental health services for the crisis occurring in the past year. The
remaining 104 or 31.1% reported not having received mental health services after the crisis
occurring in the past year. Furthermore, as presented in Figure 33, of these 104
respondents who did not receive crisis services, 92 or 88.5% reported needing or wanting
one or more crisis services but not getting them.
Figure 32 – Crisis Services Received/Not Received
100%
75% 68.9%
50%
31.1%
25%
0%
Yes, received crisis services No, did not receive crisis
(n=231) services (n=104)
N = 335
Figure 33 – Crisis Services Wanted/Not Wanted
100%
88.5%
75%
50%
25%
11.5%
0%
Did not receive crisis Did not receive crisis
services but wanted or services and did not need or
needed them (n=92) want them (n=12)
N = 104
Survey Results: Crisis Intervention Services 59
The following tables provide the distribution of two sets respondents. We often refer to
these two sets of respondents as “groups”. One group includes the respondents who
experienced a crisis and received crisis services in the past year. The second group includes
the respondents who experienced a crisis but did not receive crisis services despite needing
or wanting them. In the tables below, we present the distribution of these two groups side
by side to compare whether there is a statistically significant association (using a chi-square
test) between respondent characteristics/demographics and whether or not they received
peer support services.
First, there was no association between age group and whether or not crisis services were
received (p > .05). Figure 34 details that more adults were represented in both groups
because there was overrepresentation of adult survey respondents. However, between the
two groups, there was equal distribution of transition age youth (20.8% and 17.9%) and
older adults (7.2% and 11.9%). Although statistically there was no association, a noticeably
larger proportion of children were represented in the group that received crisis services
(15.8% compared to 7.1%).
Figure 34 – Crisis Services Received/Not Received by Age Group
100%
75%
63.1%
56.1% Yes, received crisis
services (n=221)
50%
No, did not receive
crisis services but
wanted them (n=84)
25% 20.8%
17.9%
15.8%
11.9%
7.1% 7.2%
0%
Child (n=41) TAY (n=61) Adult (n=177) Older Adult (n=26)
N = 305
Survey Results: Crisis Intervention Services 60
There was no association between gender and whether or not crisis services were received
(p > .05). As shown in Figure 35, similar proportions of female, male, and transgendered
respondents reported receiving crisis services. However, for each group, there were more
females (53.0% and 57.1%) than males (45.7% and 41.8%), and only a small number of
transgendered respondents (1.3% and 1.1%).
Figure 35 – Crisis Services Received/Not Received by Gender
100%
75%
57.1%
53.0% Yes, received crisis
50% 45.7% services (n=230)
41.8%
No, did not receive
crisis services but
25%
wanted them (n=91)
1.3% 1.1%
0%
Female (n=174) Male (n=143) Transgendered (n=4)
N = 321
Survey Results: Crisis Intervention Services 61
There was no association between race/ethnicity and whether or not crisis services were
received (p > .05). The findings are reported in Figure 36. In both groups, the largest
proportions of respondents were Caucasian (36.8% and 45.7%) and Latino (31.6% and
27.2%). African American (15.2% and 9.8%) and “other race” respondents (13.0% and
12.0%) were equally represented in both groups, and Asian and Pacific Islander respondents
in both groups represented the smallest group (3.5% and 5.4%). Moreover, as presented in
Figure 37, the majority of respondents in both groups preferred English as their primary
language (91.2% and 90.1%). A small proportion of respondents in both groups preferred
Spanish (8.8% and 9.9%).
Figure 36 – Crisis Services Received/Not Received by Race/Ethnicity
100%
75%
Yes, received crisis
services (n=231)
50% 45.7%
36.8%
31.6% No, did not
27.2%
receive crisis
25%
15.2% 13.0% services but
12.0%
9.8%
3.5% 5.4% wanted them
(n=92)
0%
African AmericanAsian and Pacific Caucasian Latino (n=98) Other* (n=41)
(n=44) Islander (n=13) (n=127)
N = 323
* “Other” race includes Native American, mixed race, and other ethnic groups not identified within the racial groups
provided as answer options in the survey.
Figure 37 – Crisis Services Received/Not Received by Preferred Language
100%
91.2% 90.1%
Yes, received
75% crisis services
(n=229)
50%
No, did not
receive crisis
25%
services but
8.8% 9.9%
wanted them
(n=91)
0%
English (n=291) Spanish (n=29)
N = 320
Survey Results: Crisis Intervention Services 62
There was no association between regional counties (as well as county size) and whether or
not crisis services were received (p > .05). As presented in Figures 38 and 39, most
respondents in both groups resided in the Southern region (55.1% and 59.6%) and in larger
counties (90.5% and 92.4%), where many are located in Southern California.
Figure 38 – Crisis Services Received/Not Received by Region
100%
75%
Yes, received
59.6%
55.1% crisis services
(n=227)
50%
No, did not
receive crisis
22.0% services but
25%
wanted them
13.5% 13.5% 13.5%
10.6% 12.3% (n=89)
0%
Southern (n=178) Central (n=62) Bay Area (n=36) Northern (n=40)
N = 316
Figure 39 – Crisis Services Received/Not Received by County Size
100%
92.4%
90.5%
75%
Yes, received
crisis services
(n=229)
50%
No, did not
receive crisis
25%
services but
wanted them
9.5%
7.6% (n=91)
0%
Small county (n=31) Larger county (n=289)
N = 320
Survey Results: Crisis Intervention Services 63
There was no association between annual income and whether or not crisis services were
received (p > .05). As shown in Figure 40, about three-quarters of respondents in both
groups reported either no income or income less than $15K. The remaining quarter of
respondents reported higher incomes from $15K to $60K and up.
Figure 40 – Crisis Services Received/Not Received by Annual Income
100%
75%
Yes, received
56.4% crisis services
54.1% (n=188)
50%
No, did not
22.9%
receive crisis
25% 22.3%
services but
14.9%
11.8% wanted them
4.7% 4.7% (n=85)
3.2% 1.6% 1.0% 2.4%
0%
No individual Less than $15,000 - $30,001 - $45,001 - Over $60,000
income (n=62) $15,000 (n=152) $30,000 (n=38) $45,000 (n=10) $60,000 (n=7) (n=4)
N = 273
Survey Results: Crisis Intervention Services 64
The distribution of respondents by educational attainment was similar for both groups. As
shown in Figure 41, more than a quarter of respondents in each group reported having less
than a high school education (29.7% and 23.3%). Another quarter of respondents reported
graduating from high school or obtaining a General Equivalency Diploma (GED) (25.6% and
25.6%). About the same proportion of respondents reported receiving vocational training or
attending some college (30.1% and 24.4%). Finally, a smaller proportion of respondents
who received crisis services (14.7%) reported completing college or graduate school
compared to respondents who did not receive crisis services (26.7%).
Figure 41 – Crisis Services Received/Not Received by Educational Attainment
50%
40%
Yes, received
crisis services
29.7% 30.1%
(n=219)
30%
25.6% 26.7%
25.0%
23.3% 24.4%
No, did not
20%
receive crisis
14.7%
services but
wanted them
10% (n=90)
0%
Less than High School High School Vocational/Some College Graduate or
(n=86) Graduate/GED (n=79) College (n=88) Above (n=56)
N = 309
Survey Results: Crisis Intervention Services 65
There was no association between the seriousness of respondents’ mental health concern
and whether or not crisis services were received (p > .05). As shown in Figure 42 and as seen
in the sample description on respondents who experienced a crisis, the majority of
respondents in both groups described their concern to be serious (34.5% and 30.2%) or
extremely serious (48.2% and 47.7%).
Figure 42 – Crisis Services Received/Not Received by Seriousness of Mental Health Concern
50% 48.2% 47.7%
40%
34.5% Yes, received
30.2% crisis services
30% (n=226)
20%
No, did not
13.7%
14.0%
receive crisis
services but
10% 8.1%
wanted them
3.5% (n=86)
0%
A mild concern A moderate concern A serious concern An extremely
(n=15) (n=43) (n=104) serious concern
(n=150)
N = 312
Survey Results: Crisis Intervention Services 66
There was no association between the number of crises experienced in the past year and
whether or not crisis services were received (p > .05). As shown in Figure 43, an equal
proportion of respondents in both groups reported experiencing only one crisis in the past
year (41.6% and 40.2%, respectively). Although there was no statistical association, a
slightly larger proportion of respondents who did not receive crisis services (32.6%
compared to 22.5%) reported two crises, and conversely, a slightly larger proportion of
respondents who received crisis services reported three or more crises (35.9% compared to
27.2%).
Figure 43 – Crisis Services Received/Not Received by Number of Crises in Past Year
50%
41.6%
40.2%
40%
35.9%
32.6%
Yes, received crisis
30% 27.2% services (n=231)
22.5%
20%
No, did not receive
crisis services but
wanted them
10% (n=92)
0%
1 time (n=133) 2 times (n=82) 3 or more times (n=108)
N = 323
Survey Results: Crisis Intervention Services 67
There was no association between years of receiving mental health services and whether or
not crisis services were received (p > .05). As shown in Figure 44, an equal proportion of
respondents in both groups reported receiving mental health services for less than one year
(15.6% and 12.6%, respectively). Within both groups, the majority of respondents reported
receiving more than one year of services, and close to half of all respondents in both groups
reported receiving more than five years of services.
Figure 44 – Crisis Services Received/Not Received by Years of Receiving Mental Health Services
49.4%
50%
46.2%
40%
Yes, received
crisis services
30% (n=225)
22.2%
20.7% No, did not
20% receive crisis
17.2%
15.6% 16.0% services but
12.6% wanted them
(n=87)
10%
0%
Less than 1 year 1 to 2 years (n=65) 3 to 5 years (n=54) More than 5 years
(n=46) (n=147)
N = 312
There was an association between belonging to an unserved or underserved group and
whether or not crisis services were received (p < .05). (See Appendix A for details on the
different characteristics of individuals belonging to an unserved or underserved group.)
Therefore, one characteristic differentiated respondents who received crisis services from
those who did not but wanted them. As shown in Figure 45, among respondents who
received crisis services, 42.0% identified themselves as belonging to an unserved or
underserved group. In contrast, 25.0% of respondents who did not receive crisis services
(but wanted them) identified themselves in this way. Therefore, more unserved or
underserved respondents (e.g., homeless and physically disabled) were represented in the
group that received crisis services, while more respondents who are not unserved or
underserved were represented in the group that did not receive crisis services. It is
important to understand that these distributions refer to sample representation and do not
necessarily indicate the extent to which unserved or underserved respondents had access
to services.
In order to assess the potential effect of this association on the results, we examined
whether respondent characteristics predict membership in the unserved or underserved
Survey Results: Crisis Intervention Services 68
group. Using logistic regression, we found that the seriousness of the mental health concern
and whether or not crisis services were received significantly “discriminated” respondents
who belonged to an unserved or underserved group (p < .05).25 In other words, more
respondents who received crisis services and who belonged to an unserved or underserved
group rated their mental health condition to be extremely serious (54.5%) than their
counterparts (39.4%). Based on this self-rating and by virtue of its overrepresentation of
unserved and underserved respondents, the group of respondents who received crisis
services had more individual members who have extremely serious mental health concerns.
Figure 45 – Crisis Services Received/Not Received by Unserved or Underserved Group
100%
75.0%
75%
58.0%
50% Yes, received crisis
42.0%
services (n=231)
25.0%
25% No, did not receive
crisis services but
wanted them (n=92)
0%
Unserved or underserved (n=120) Not unserved or underserved
(n=203)
N = 323
Χ2 = 8.13 (1), p < 0.01
Nature of Mental Health Crisis: Comparison between Respondents
There was no association between the nature of the mental health crisis and whether or not
respondents received crisis services (p > .05). Figure 46 lists the responses provided by both
sets of respondents. More than one description of the crisis could be selected; therefore,
the percentages are based on total responses rather than the number of respondents. The
two most common descriptions provided by respondents who received crisis services were
“I was experiencing significant sadness” (15.7%) and “I was having trouble sleeping, which
affected my thoughts, mood, and/or behavior” (15.7%). Other common descriptions were:
“My emotions were hard to manage and jeopardizing my ability to take care of myself”
(14.2%); “I was very confused” (13.9%); “I was thinking about harming or killing myself”
(12.4%); and “My thinking was disorganized and I was having a hard time taking care of
25 The model test result including 312 respondents was Χ2 = 20.74 (5), p < .001. The results of the predictor
variable of seriousness of mental health concern are: B = 0.66, S.E. = 0.27, Wald’s Χ2 = 5.89 (1), p < .05, and
odds ratio = 0.52. The results of the predictor variable of whether or not crisis services were received are: B =
0.85, S.E. = 0.24, Wald’s Χ2 = 8.43 (1), p < .01, and odds ratio = 0.43.
Survey Results: Crisis Intervention Services 69
myself” (12.4%). Twenty (20) respondents also provided other descriptions of the nature of
their mental health crisis. For example, six respondents described difficult life circumstances
such as domestic violence and family problems, and 12 respondents reported a mental
illness such as schizophrenia and anxiety disorder.
The most common descriptions provided by respondents who did not receive crisis services
but wanted them were “I was having trouble sleeping, which affected my thoughts, mood,
and/or behavior” (18.5%); “My emotions were hard to manage and jeopardizing my ability
to take care of myself” (16.2%); “My thinking was disorganized and I was having a hard time
taking care of myself” (15.7%); “I was experiencing significant sadness” (14.2%); “I was
thinking about harming or killing myself” (11.6%); and “I was very confused” (11.6%). Eight
respondents also provided other descriptions of the nature of their mental health crisis. For
example, one described a serious medical condition; two described difficult life
circumstances, including child welfare system involvement; and five reported a mental
illness or substance abuse.
Figure 46 – Nature of Mental Health Crisis: Comparison between Respondent Groups
I was having trouble sleeping, which affected 15.7%
my thoughts, mood, and/or behavior (n=183) 18.5%
15.7%
I was experiencing significant sadness (n=170)
14.2%
My emotions were hard to manage and
14.3%
jeopardizing my ability to take care of myself
16.2%
(n=165)
13.9%
I was very confused (n=148)
11.6%
My thinking was disorganized and I was
12.4%
having a hard time taking care of myself
15.7%
(n=148)
I was thinking about harming or killing myself 12.4%
(n=135) 11.6%
7.5%
I was experiencing extreme mania (n=80)
6.3%
I was thinking about harming another person 5.6%
(n=55) 3.3%
2.5%
Other* (n=28)
2.6%
0% 10% 20% 30%
Yes, received crisis services No, did not receive crisis services but wanted them
N = 323, Total Responses = 1,112
* Other descriptions are provided in the narrative (e.g., difficult life circumstance).
Survey Results: Crisis Intervention Services 70
Types of Services Received
Respondents who received crisis services were asked in the survey to identify the types of
mental health services they received for the crisis occurring in the past year. It is important
to note that more than one type of service could be selected. Therefore, the percentages
are based on total responses rather than the number of respondents in order to assess the
relative commonality of crisis services received by respondents. As shown in Figure 47, the
most common services were counseling (26.2%) and seeing a psychiatrist or having
medication adjusted (23.0%). Other services were not as common, but over ten percent of
responses included these services: safety plan to help address the crisis (13.8%) and hotline
or warmline to talk to someone (11.1%). The mean number of services identified was 2.6
(standard deviation of 1.5; range of 1 to 8). This means that respondents reported, on
average, having received a little over two services for the crisis occurring in the past year.
Figure 47 – Mental Health Services Received for Crisis
Counseling 148 26.2%
I saw a psychiatrist or medications were adjusted 130 23.0%
A safety plan was developed to help me address the
78 13.8%
crisis
Hotline or warmline to talk to someone 63 11.1%
Day treatment/partial hospitalization 54 9.6%
Assistance as part of a Full Service Partnership 36 6.4%
Crisis residential 31 5.5%
Psychiatric hospitalization/psychiatric ward 23 4.1%
Alcohol treatment 2 0.3%
0% 10% 20% 30%
N = 231, Total Responses = 565
Survey Results: Crisis Intervention Services 71
The types of crisis services were further analyzed to identify whether certain characteristics
of respondents were associated with having received one or more of these services. The
following characteristics were analyzed: age group, gender, race/ethnicity, preferred
language, education, income, member of an unserved or underserved group, seriousness of
mental health concern, size of county residence, and region of county residence. We found
a number of associations. These findings are presented in Tables 19 through 21, and all
findings are statistically significant (p < .05).
A larger proportion of respondents in smaller counties reported receiving hotline/warmline
services (66.7% compared to 23.9%), as well as a safety plan (61.1% compared to 31.5%). In
contrast, a smaller proportion of respondents in the Southern region, which represents a
number of larger counties in this study, had a significantly smaller proportion of
respondents who received hotline or warmline services (16.8% compared to 46.4% for the
Northern region). Furthermore, a larger proportion of respondents who identified as
belonging to an unserved or underserved group reported having received
psychiatric/medication (76.7%) and crisis residential services (17.2%) than those who did
not identify themselves in this way (43.3% and 8.2%, respectively).
Table 19 – Hotline/Warmline and Safety Plan: Comparison by County Size
Small County Larger County
Hotline or Warmline1
Yes 12 (52.2%) 51 (24.9%)
No 11 (47.8%) 154 (75.1%)
TOTAL N = 23 N = 205
Safety Plan2
Yes 12 (52.2%) 66 (32.0%)
No 11 (47.8%) 140 (68.0%)
TOTAL N = 23 N = 206
1 Χ2 = 7.71 (1), p < .01
2 Χ2 = 3.73 (1), p < .05
Table 20 – Hotline Warmline: Comparison by Regional Counties
Southern Central Bay Area Northern
Yes, hotline or warmline 21 (16.8%) 20 (40.0%) 9 (37.5%) 13 (46.4%)
No 103 (82.4%) 30 (60.0%) 15 (62.5%) 15 (53.6%)
TOTAL N = 124 N = 50 N = 24 N = 28
Χ2 = 17.78 (3), p < .01
Survey Results: Crisis Intervention Services 72
Table 21 – Psychiatrist/Medication and Crisis Residential: Comparison
Between Belonging or Not Belonging to Unserved/Underserved Group
Unserved or Not Unserved or
Underserved Underserved
Psychiatric/Medication1
Yes 88 (65.7%) 42 (43.3%)
No 46 (34.3%) 55 (56.7%)
TOTAL N = 134 N = 97
Crisis Residential2
Yes 23 (17.2%) 8 (8.2%)
No 111 (82.8%) 89 (91.8%)
TOTAL N = 134 N = 97
1 Χ2 = 11.45 (1), p < .001
2 Χ2 = 3.85 (1), p < .05
Survey Results: Crisis Intervention Services 73
In addition to asking respondents to identify the types of mental health services they
received, respondents were asked to identify where mental health services were received
for the crisis experienced in the past year. In Figure 48, the service settings are listed
starting with the most common. More than one setting could be selected; therefore, the
percentages are based on total responses rather than the number of respondents. These
findings are consistent with the findings above – for example, the two most common
mental health services received were counseling and seeing a psychiatrist or having
medication adjusted. Accordingly, the two most common service settings were the
counseling office (18.8%) and psychiatric hospital/facility (17.0%). Other common settings
were the hospital emergency room (14.0%), on the telephone (13.0%), and at home
(10.9%). Moreover, although receiving services in a psychiatric hospital or facility was
relatively common (suggesting that psychiatric hospitalization was common), the multiple
responses that are possible indicate that psychiatric hospitalization was not exclusively
received. In fact, earlier findings on routine mental health services indicate that the majority
of respondents who received crisis services (70.3%) received other services outside of the
psychiatric hospital. It is unknown whether the psychiatric hospital was the first point of
contact for these respondents because the study examines a one-year period and almost
three-quarter of the respondents who received crisis services have been receiving mental
health services for more than one year.
Figure 48 – Where Mental Health Services for Crisis Were Received
Counseling office 90 18.8%
Psychiatric hospital/facility 81 17.0%
Hospital emergency room 67 14.0%
On the telephone 62 13.0%
At home 52 10.9%
Drop in center 33 7.0%
Crisis residential facility 28 5.8%
Wellness Center 26 5.4%
Mobile crisis team 19 4.0%
Other* 20 4.2%
0% 10% 20% 30%
N = 231, Total Responses = 478
* Other settings were identified: doctor’s office (n=6); jail (n=4); school setting (n=3); drug treatment facility (n=2);
residential treatment (n=2); one response each for employment center, with family/friends, and group home.
Survey Results: Crisis Intervention Services 74
Access to Crisis Services
Respondents who received crisis services were asked to identify if there were mental health
services they needed or wanted but did not receive for the crisis occurring in the past year.
Figure 49 presents a range of services reported by 21.1% of respondents (or 49 out of 231)
who received crisis services. More than one service could be selected; therefore, the
percentages are based on total responses rather than the number of respondents. The most
common service identified was counseling (22.8%), followed by medication (15.8%), other
mental health interventions (14.1%), housing (12.3%), and building a social network
(12.3%).
Figure 49 – Mental Health Services for Crisis Needed or Wanted for Crisis: Respondents Who Received Crisis
Services
Counseling 13 22.8%
Medication 9 15.8%
Other mental health interventions,
8 14.1%
including staff
Social Network 7 12.3%
Housing 7 12.3%
Hospital 5 8.8%
Hotline 3 5.3%
Aftercare or respite care 2 3.5%
Primary care provider 1 1.7%
Basic Needs 1 1.7%
Employment 1 1.7%
0% 10% 20% 30%
N = 49, Total Responses = 57
Survey Results: Crisis Intervention Services 75
Respondents who did not receive crisis services were asked in the survey to identify mental
health services they needed or wanted for the crisis occurring in the past year. The findings,
presented in Figure 50, are consistent with the findings above. More than one service could
be selected; therefore, the percentages are based on total responses rather than the
number of respondents. Among respondents who did not receive services, the two most
common services wanted or needed were counseling (33.8%) and psychiatrist/medication
adjustment (23.3%), followed by hotline/warmline (11.9%) and safety plan (11.9%). The
mean number of services they needed or wanted was 2.2 (standard deviation of 1.4; range
of 1 to 7). This means that respondents reported, on average, wanting or needing about
two services.
Figure 50 – Mental Health Services Needed or Wanted for Crisis: Respondents Who Did Not Receive Crisis
Services
Counseling 65 33.8%
To see a psychiatrist or to have
45 23.3%
medications adjusted
To develop a safety plan to help me
23 11.9%
address the crisis
Hotline or warmline to talk to someone 23 11.9%
Assistance as part of a Full Service
13 6.7%
Partnership
Day treatment/partial hospitalization 10 5.2%
Crisis residential 8 4.1%
Other* 6 3.1%
0% 10% 20% 30% 40% 50%
N = 92, Total Responses = 193
* Other includes hospital (n=2), building a support network (n=2), and one response each for case manager and
employment.
Survey Results: Crisis Intervention Services 76
The same group of respondents who did not receive mental health services for the crisis
occurring in the past year reported the reasons for not receiving services that they needed
or wanted.26 More than one reason could be selected; therefore, the percentages are based
on total responses rather than the number of respondents. As shown in Figure 51, at 18.3%,
stigma (e.g., feeling embarrassed or fearful) was the most common reason reported for not
receiving needed or wanted crisis services. Other common reasons were not being able to
pay for the services (15.5%), being told that s/he did not qualify (12.0%), and not feeling
comfortable or welcome (10.6%).
Figure 51 – Reasons for Not Receiving Crisis Services: Respondents Who Did Not Receive Crisis Services but
Wanted Them
Stigma 26 18.3%
I could not pay for the services 22 15.5%
I was told that I did not qualify 17 12.0%
I did not know how to apply for services 15 10.6%
I did not feel comfortable or welcome 15 10.6%
Services were not available in a location that
12 8.5%
was convenient to me
Services that are sensitive to my cultural
10 7.0%
orientation were not available to me
Services were not available during times that
8 5.6%
were convenient to me
I did not like the services that were offered 6 4.2%
I was placed on a waiting list 2 1.4%
Services were not available in my language 2 1.4%
Other* 7 4.9%
0% 10% 20% 30%
N = 92, Total Responses = 142
*Other includes services not available or appropriate (n=2) and one response each for outreach due to drug problems, lack
of confidentiality, staff not believing in client, closing case, and services not available for children in crisis.
26
Due to the small sample size, further analysis was not conducted to examine whether respondent
characteristics are associated with the reasons for not being able to access or receive crisis services.
Survey Results: Crisis Intervention Services 77
Continuity of Care: Routine Mental Health Services Before and After the Crisis
There was an association between receiving routine mental health services and whether or
not respondents received crisis services (p < .05). As presented in Table 22, 65.0% of
respondents who received crisis services reported receiving routine mental health services
before and after the crisis. In contrast, 48.9% of respondents who did not receive crisis
services reported receiving routine mental health services before and after the crisis.
Furthermore, only 8.4% of respondents who received crisis services did not report receiving
routine mental health services before and after the crisis. This proportion is significantly
lower than the 28.9% of respondents who did not receive crisis services or any routine
mental health services before and after the crisis.27
Table 22 – Routine Mental Health Services: Comparison Between Respondent Groups1
Yes, received crisis No, did not receive crisis
services services but wanted them
Received routine mental health services 146 (65.0%) 44 (48.9%)
before the crisis and continued routine
services after the crisis.
Receive mental health services before the 12 (5.3%) 3 (3.3%)
crisis but did not continue routine mental
health services after the crisis.
Did not receive routine mental health 48 (21.3%) 17 (18.9%)
services before the crisis but began routine
services after the crisis.
Did not receive routine mental health 19 (8.4%) 26 (28.9%)
services before the crisis and did not
receive routine mental health services after
the crisis.
TOTAL N = 225 (100.0%) N = 90 (100.0%)
1 Χ2 = 22.26 (3), p < .001
Recovery Orientation of Crisis Services
The Recovery Oriented System Indicators (ROSI) scores were analyzed for respondents 18
years or older only because the instrument was not intended for children. We analyzed
whether respondents perceived the orientation of mental health services differently if they
received or did not receive crisis services. (Note that respondents who did not receive crisis
27
Because of small sample sizes (at times as low as n=2 depending on the analysis), further analysis was not
conducted to examine whether respondent characteristics are associated with having received routine mental
health services before or after the crisis.
Survey Results: Crisis Intervention Services 78
services completed the ROSI based on experiences with other mental health services
received currently or in the past.) Using an independent samples t-test, we found
statistically significant differences between respondents who received crisis services and
those who did not (but wanted them) in four of the six ROSI scales (p ≤ .05). The findings are
presented in Table 23. Respondents who received crisis services perceived a more positive
recovery orientation in four areas: person-centered focus and directed decision-making,
holistic focus, systems potholes, and mistreatment. In other words, compared to
respondents who did not receive crisis services despite wanting them, respondents who
received crisis services reported, on average, that the mental health services they
experienced were more person-centered, more holistic in meeting other needs such as
housing, had less service inadequacies, and were less oriented toward mistreatment.
Table 23 – ROSI Results: Comparison Between Respondents Who Received Crisis Services and
Those Who Did Not But Wanted Them
Yes, received crisis No, did not receive crisis
services services but wanted them
Sample Size, Mean (SD) Sample Size, Mean (SD)
Person-centered focus and n = 141, 3.21 (0.73) n = 52, 2.95 (0.83)
directed decision-making1
Holistic focus2 n = 148, 3.01 (0.53) n = 56, 2.72 (0.69)
Moving on up n = 136, 2.68 (0.87) n = 62, 2.47 (0.87)
Basic material resources n = 179, 2.60 (0.77) n = 71, 2.53 (0.84)
System potholes3 n = 151, 2.24 (0.64) n = 57, 2.44 (0.63)
Mistreatment4 n = 138, 1.60 (0.70) n = 53, 1.86 (0.76)
1 t(191) = 2.08, p < .05; 2 t(202) = 3.22, p < .001; 3 t(206) = 2.02, p < .05; 4 t(189) = 2.25, p < .05
We also tested whether respondents who received crisis services plus peer support services
perceived the recovery orientation of services differently from those who did not receive
peer support services. As presented in Table 24, respondents who received crisis services
and peer support services had, on average, statistically higher ROSI scores on the subscales
of holistic focus and moving on up (p < .05). That is, these respondents perceived their
service experience to be more recovery oriented with respect to meeting their needs more
holistically and stabilizing and/or advancing employment or school compared to
respondents who did not receive the additional peer support. This was also the case for
respondents who received crisis services and employment support services. As shown in
Table 25, these respondents perceived their employment or school situation to be more
stable and/or advanced than those who did not receive the additional employment support.
The mean score differences for these two scales were significantly different (p < .05).
Finally, as shown in Table 26, respondents who received all three services (crisis, peer, and
Survey Results: Crisis Intervention Services 79
employment) perceived services to be more holistic, more focused on employment or
school stability and/or advancement, and less inadequate as part of a service system.
Table 24 – ROSI Results for Respondents Who Received Crisis Services: Comparison
Between Respondents Who Received Peer Support Services and Those Who Did Not
Yes, also received peer No, only received crisis and
support services not peer support services
Sample Size, Mean (SD) Sample Size, Mean (SD)
Person-centered focus and n = 65, 3.28 (0.72) n = 20, 2.86 (0.74)
directed decision-making1
Moving on up2 n = 68, 2.83 (0.81) n = 21, 2.17 (0.93)
1 t(83) = 2.23, p < .05; 2 t(87) = 3.20, p < .01
Table 25 – ROSI Results for Respondents Who Received Crisis Services: Comparison Between
Respondents Who Received Employment Support Services and Those Who Did Not
Yes, also received No, only received crisis and not
employment support services employment support services
Sample Size, Mean (SD) Sample Size, Mean (SD)
Person-centered focus and n = 39, 3.34 (0.66) n = 27, 2.95 (0.72)
directed decision-making1
Moving on up2 n = 44, 2.99 (0.70) n = 27, 2.29 (0.82)
1 t(64) = 2.28, p < .05; 2 t(69) = 3.85, p < .001
Table 26 – ROSI Results for Respondents Who Received Crisis Services: Comparison Between
Respondents Who Received Crisis, Peer, and Employment Support Services and Those Who Did Not
Yes, received all three services No, only received crisis services
Sample Size, Mean (SD) Sample Size, Mean (SD)
Holistic focus1 n = 26, 3.20 (0.56) n = 122, 2.97 (0.52)
Moving on up2 n = 27, 3.12 (0.65) n = 109, 2.57 (0.89)
System potholes3 n = 23, 1.93 (0.71) n = 125, 2.30 (0.61)
1 t(139) = 2.07, p < .05; 2 t(134) = 3.67, p < .01; 3 t(149) = 2.78, p < .01
Furthermore, a Pearson’s r correlation test was used to examine the relationship between
the perception of services as recovery oriented and the number of crisis services received
for respondents who received crisis services. Three of the ROSI scales were significantly and
positively correlated with the number of crisis services received (p < .05). Respondents who
received more types of crisis services perceived that the mental health services they had
received were more person-centered (r = 0.22), holistic (r = 0.19), and oriented toward
employment or school stability and/or advancement (r = 0.21). Because the correlations
were not particularly strong (i.e., the r is at the lower end of a range of 0 to 1.0), it is with
caution that any conclusions are drawn from these findings.
Survey Results: Crisis Intervention Services 80
Crisis Services Impact: Employment, Housing, and Personal Recovery/Resilience and
Wellness
Employment
Based on a small sample of respondents (18 years or older) who provided information on
employment changes, there was no association between a change in employment status
after the crisis and whether or not crisis services were received (p > .05). As presented in
Table 27, almost half of respondents in each group reported an employment change after
the crisis. Between groups, the proportions of respondents who reported a change were
equal (48.7% and 46.9%).
Table 27 – Employment Change: Comparison Between Respondent Groups
Yes, received crisis No, did not receive crisis
services services but wanted them
Employment change after crisis 19 (48.7%) 15 (46.9%)
No employment change after crisis 20 (51.3%) 17 (53.1%)
TOTAL N = 39 (100.0%) N = 32 (100.0%)
Housing
Based on a small sample of respondents who provided information on housing changes,
there was no association between a change in living situation after the crisis and whether or
not crisis services were received (p > .05). As presented in Table 28, 62.1% of respondents
who received crisis services reported a housing change compared to 46.7% of respondents
who did not receive crisis services. Although a larger proportion of respondents who
received crisis services reported a housing change, there was no statistical association.
Table 28 – Housing Change: Comparison Between Respondent Groups
Yes, received crisis No, did not receive crisis
services services but wanted them
Housing change after crisis 41 (62.1%) 14 (46.7%)
No housing change after crisis 25 (37.9%) 16 (53.3%)
TOTAL N = 66 (100.0%) N = 30 (100.0%)
Survey Results: Crisis Intervention Services 81
Resilience and Wellness: Children
The Strengths and Difficulties Questions (SDQ) scores were analyzed for children only by
comparing respondents who received crisis services and those who did not despite wanting
them. Because so few children in our analysis did not receive crisis services, the sample size
for this group is very small. Therefore, the findings are viewed with caution. Given the
timing of when the survey was completed, the results of the SDQ reflect children’s strengths
and difficulties after the last crisis. Further, the SDQ scores represent the perspective of the
caregiver who completed the survey. Based on normed data from across the country, the
mean scores for this study’s sample indicate that the children represented in this study, on
average, have substantially more difficulties. This is expected given the population targeted
for the study.28
As shown in Table 29, the mean scores for two scales plus the total difficulties score was
significantly different in favor of children who received crisis services (p < .05). That is,
children who received crisis services were reported to have fewer peer problems (3.72
compared to 5.75) and greater prosocial behaviors (6.67 compared to 4.50).
Table 29 – SDQ Results: Comparison Between Groups
Yes, received services No, did not receive services
(N = 32) but wanted them (N = 8)
Mean (SD) Mean (SD)
Emotional symptoms 4.90 (2.82) 6.00 (3.21)
Conduct problems 4.66 (2.47) 6.37 (2.67)
Hyperactivity/inattention 6.59 (2.65) 8.25 (1.67)
Peer problems1 3.72 (2.00) 5.75 (2.71)
Prosocial behaviors2 6.67 (2.51) 4.50 (3.38)
Total Difficulties3 19.87 (7.03) 26.37 (6.67)
1 t(38) = 2.39, p < .05; 2 t(38) = 2.06, p < .05; 3 t(38) = 2.36, p < .05
28
See www.sdqinfo.com for normed data.
Survey Results: Crisis Intervention Services 82
Personal Recovery and Wellness: Adults
Reports on psychiatric hospitalization were analyzed between respondents who received
crisis services and those who did not (but wanted them). Although the distribution between
groups was noteworthy, there was no association between psychiatric hospitalization and
whether or not crisis services were received (p > .05). As shown in Table 30, 20.3% of
respondents who received crisis services reported being hospitalized after the crisis
compared to 33.3% of respondents who did not receive crisis services.
Table 30 – Psychiatric Hospitalization: Comparison Between Respondent Groups
Yes, received crisis No, did not receive crisis
services services but wanted them
Psychiatric hospitalization after 24 (20.3%) 9 (33.3%)
crisis
No psychiatric hospitalization 94 (79.7%) 18 (66.7%)
after crisis
TOTAL N = 118 (100.0%) N = 27 (100.0%)
The Recovery Process Inventory (RPI) scores were analyzed for respondents 18 years or
older. The findings are presented in Table 31. Comparing respondents who received crisis
services and those who did not despite wanting them, we found statistically significant
differences in five scales between the two groups (p < .05). All the differences were in favor
of respondents who received crisis services. In contrast to respondents who did not receive
crisis services but wanted them, respondents who received crisis services perceived or felt
more connected to others, more confident about life, more surrounded by people who care,
more housing stability, and more hopeful.
Table 31 – RPI Results: Comparison Between Respondent Groups
Yes, received crisis No, did not receive crisis
services services but wanted them
Sample Size, Mean (SD) Sample Size, Mean (SD)
Anguish n = 69, 2.67 (0.76) n = 64, 2.89 (0.89)
Connected to Others1 n = 69, 3.47 (0.88) n = 60, 2.97 (0.87)
Confidence and Purpose2 n = 70, 3.70 (0.83) n = 64, 3.05 (1.10)
Others’ Care/Help3 n = 69, 3.57 (0.88) n = 62, 3.10 (0.89)
Living Situation4 n = 70, 3.53 (1.08) n = 65, 3.14 (1.16)
Hopeful/Cares for Self5 n = 69, 4.11 (0.76) n = 63, 3.82 (0.96)
1 t(127) = 3.24, p < .01; 2 t(132) = 3.88, p < .001; 3 t(129) = 3.10, p < .01; 4 t(133) = 2.06, p < .05; 5 t(130) = 1.93, p < .05
Survey Results: Crisis Intervention Services 83
A correlational analysis of the RPI scales and the ROSI scales resulted in similar findings for
both respondent groups. For respondents who received crisis services, all scales of the RPI
were correlated with all of the ROSI scales except the scale of mistreatment (p < .05).29 For
respondents who did not receive crisis services, there were fewer significant correlations;
however, the differences were not drastic. For both sets of respondents, the strongest
correlation was seen between the RPI scale of living situation and the ROSI scale of basic
material resources (r = .60 for respondents who received crisis services and r = .70 for
respondents who did not receive crisis services but wanted them). In other words, for both
groups, when services were perceived to be more focused on basic material resources,
respondents’ perception of their living situation was more positive.
Furthermore, a correlation test was used to examine the relationship between perceived
personal recovery and the number of crisis services received for respondents who received
crisis services. There was no significant correlation between these two variables (p > .05).30
29
The Pearson’s r ranged from 0.22 to 0.60 for significant correlations for respondents who received crisis
services (n = 129-142 depending on the analysis). The Pearson’s r ranged from 0.26 to 0.70 for significant
correlations for respondents who did not receive crisis services but wanted them (n = 54-77 depending on the
analysis).
30
Further analysis using multivariate statistics was not conducted due to small sample sizes for respondents
who did not receive crisis services but wanted them.
Survey Results: Crisis Intervention Services 84
INTERVIEW RESULTS
Respondent Characteristics
Altogether, 40 interviews were conducted across the state. Demographic details about the
interview participants are described in more detail in Appendix C. In this section, the
characteristics of interview respondents, including those characteristics that are congruent
with our focus on unserved and underserved groups, are described for each of the three
service areas.
Peer Support Interview Respondents
Several of the nine respondents interviewed primarily about their experiences with peer
support services belonged to one or more unserved or underserved groups. Two of the
respondents were immigrants from Mexico, one of whom was a monolingual Spanish
speaker. Three respondents had a history of incarceration, and two had experienced
episodes of homelessness. One of the peer support interview respondents had a visual
impairment, one identified as transgendered, and one respondent was a veteran.
Respondents interviewed about peer support services discussed having serious mental
health concerns now and in the past (one-third described a long history of mental health
concerns and interventions). Two said that they had been in a psychiatric hospital. One had
attempted suicide. Over half of the sample (5) said they had experienced depression. Three
respondents had been diagnosed with schizophrenia, and two said they had bipolar
disorder. Two respondents experienced co-occurring mental health and substance abuse
issues. One was diagnosed with post-traumatic stress disorder (PTSD) and another with an
anxiety disorder. Note that these counts are not mutually exclusive.
Employment Support Interview Respondents
Several respondents interviewed about employment support services belonged to one or
more unserved or underserved groups. More specifically, three respondents had a physical
disability – one was visually impaired, one had a hearing impairment, and the third was
physically disabled as a result of a work injury. Three of the respondents had been
incarcerated, two had experienced homelessness, and one was a veteran. These counts are
not mutually exclusive.
A majority of the people we spoke with regarding employment support services said they
had co-occurring mental health and substance abuse issues, and half said they had
experienced depression. One respondent had bipolar disorder, and one was diagnosed as
having a psychotic disorder. Four of these respondents had experienced psychiatric
hospitalization, and one said that he had attempted suicide. Note that these counts are not
mutually exclusive.
Interview Results 85
The employment histories and education backgrounds of employment support interview
respondents were rich and varied. In terms of education background, two respondents had
a degree or certificate and experience in nursing, one had a degree from a two-year college,
and two of the interview respondents dropped out of college due to their mental health
concerns. With respect to employment history, one respondent described herself as a
“welfare-to-work mother” who ultimately completed a four year degree. Another had
worked repairing cars after a stint in the armed forces. A third respondent was once
employed in the engineering field. Other jobs held by interview respondents included farm
laborer, pipe fitter, furniture mover, and grocery store clerk.
As described above, the interview respondents in this sample had managed successfully in
the workforce at one point. They came to utilize employment support services after a
period (or periods) of time when their mental health situations prevented them from
successfully managing job or school responsibilities. At least two of the respondents had
breakdowns during a time of employment and lost their jobs as a result. One of these
respondents became homeless, while the second was able to live at home with her children
and extended family. Another had a lucrative and secure job with the pipe fitter’s union, but
the stress of long hours during one project led to a relapse in substance use. He failed to
show up for his job and was fired. Three deaths in the family over a short span of a couple
years led another respondent to lose his sobriety, and he found himself incarcerated, out of
work, and homeless. It was under circumstances such as these that our respondents sought
employment support services.
Crisis Intervention Interview Respondents
Several unserved and underserved groups targeted for this study were represented among
the 22 interview respondents who discussed their experiences with crisis services. For
example, at least seven of these interview respondents had a history of homelessness,
three stated that they had a history of incarceration, one identified as gay, and one as
transgendered. One respondent was a veteran, one had a physical disability, one had a
hearing impairment, and another had a developmental disability. One was a monolingual
Spanish speaker, and two were monolingual Chinese speakers. These counts are not
mutually exclusive.
Respondents who received crisis intervention services described serious mental health
concerns in their past and present lives. Nearly half of those we spoke with disclosed that
they had contemplated or attempted suicide; as many said that they had a history of being
on psychotropic medication(s). Nine revealed they had been in a psychiatric hospital at least
once. More than one-third of those interviewed said they experienced depression. Another
third had child behavior problems (e.g., oppositional defiant disorder [ODD], attention
deficit disorder [ADD] and/or attention deficit hyperactivity disorder [ADHD]). Four
interview respondents were diagnosed with bipolar disorder. Another four had been
diagnosed with schizophrenia. An equal number described co-occurring substance abuse
and mental health concerns. Several interview respondents described having PTSD and/or
an anxiety disorder.
Interview Results 86
Moreover, respondents explained that those mental health issues have been persistent. A
majority of those we spoke with about crisis services described a long history of mental
health concerns and intervention in their lives. For example, the parents in our sample who
were interviewed about their children’s experiences with mental health services described
child behavior problems – and efforts to seek services in the school and/or mental health
systems – beginning as early as when the child was five years old and continuing after
graduation from high school. The need for services became apparent when a child exhibited
problematic oppositional, concentration, and/or control behaviors, tried to hurt her- or
himself, or tried to hurt someone else. The transition age youth respondents we spoke with
also remembered experiencing mental health issues from an early age. Two said their
concerns stemmed from their mothers abusing drugs while pregnant and one college-aged
youth said her first suicide attempt was in middle school. Several of the adults and older
adults that reported on crisis services also remembered experiencing mental health
concerns from a young age. One older adult respondent in our crisis services interview
sample said she had been in and out of psychiatric hospitals since 1968. Another described
being abandoned by her parents as a child in Central America and being shuttled among
different relatives until she immigrated to the United States in her early 20s. Now in her
60s, she remembers experiencing pain, stress, and depression since childhood. A third older
adult was a veteran who experienced PTSD after returning from the Vietnam War.
Respondents who were interviewed about their experiences with crisis services were asked
to describe the most recent crisis for which they sought services. Parents in the sample
either sought services for one of their children who had hurt or threatened to hurt a sibling
(e.g., one “burned his sister with a knife”) or because their child was hurting her- or himself
(e.g., cutting). A handful of others experienced the death of family members or close friends
that sunk them into a deep depression. An adult respondent left an abusive employment
situation as a live-in health aid and moved in with a boyfriend who rented his apartment to
“druggies.” At one point, she was beaten, burned, raped, and “shot up with drugs for days”.
She finally got away and was hospitalized for her physical injuries. When released, she
found herself homeless, suffering from depression and PTSD, and attempting to commit
suicide. An older adult respondent lost his job – and with it his important “extended family”
of work colleagues. He became suicidal. Lastly, a transition age youth respondent left her
rented apartment to go live with and care for her ailing grandmother. When familiar (and
harmful) family dynamics re-surfaced and she was unable to safely remain with her
grandmother, she found herself homeless and in crisis.
Services Received
Overlapping Services Received by Peer Support Respondents
Most of those who received peer support services also received other types of mental
health supports. For instance, seven of the nine respondents in this sample received
professional counseling, and five described receiving dedicated staff support. Nearly half
received employment support services. Three reported receiving housing supports, and
Interview Results 87
three received home visits. Two of the interview respondents were Full Service Partnership
(FSP) participants, and two mentioned that they received services to help meet their
physical health care needs. Hotline/warmline services, independent living, school support,
and wraparound were services received by one individual apiece. All but one respondent
received services in addition to peer support. For the eight respondents who received
additional services, the overlap ranged from receiving one to nine services in addition to
peer support. Half of those respondents received four or more additional services.
Overlapping Services Received by Employment Support Respondents
In addition to the employment support services they received, most respondents accessed
additional mental health services. Six of the nine people we spoke with received peer
support in addition to employment support, five received professional counseling, and five
received psychiatric services and medication. Two respondents received substance abuse
services. One respondent was an FSP participant who also received housing services. The
amount of overlap of services ranged from receiving two to five services along with peer
support, with over half of respondents receiving more than four additional services.
Overlapping Services Received by Crisis Intervention Respondents
Interview respondents received a range of overlapping crisis services. The service received
most frequently – but only by the slightest margin – was counseling (talk therapy) from a
professional. In fact, three-fourths of interview respondents said they received counseling
as part of their package of crisis intervention services. Almost as many saw a psychiatrist
and received medication, as well as received staff support (e.g., a case manager, personal
service coordinator, or the like). Nearly half of interview respondents received housing
services. Nine of the respondents received home visits as part of their intervention services,
and an equal number received peer support services. Eight respondents received school
support services. Four of the people we spoke with about crisis intervention also reported
receiving employment support services. Physical health services, substance abuse services,
and services from a hotline or warmline were mentioned by three respondents each. Two
child respondents received wraparound services. The range of overlapping services among
respondents was between two and 10. Half of respondents received a combination of five
or more services to address their mental health needs.
Recovery/Resilience Orientation of Services
In this section and the next, we present key themes that emerged from descriptions of
respondents’ personal recovery/resilience experiences, including whether and how services
supported their recovery and wellness. Again, due to the overlapping nature of the services
received by our interview respondents, the thematic findings presented below are based on
a cross-case analysis of all 40 respondents across the three service areas. As mentioned
earlier, in order for a theme to be presented, more than half of all respondents had to have
discussed it during their interview. Therefore, the themes discussed below are considered
Interview Results 88
to apply across service areas. From time to time, however, a particular service area may be
highlighted during the presentation of a theme if the theme (or one of its subthemes) was
particularly strong for that set of interview respondents.
Overall, interview respondents characterized the recent mental health services they
received as demonstrating recovery/resilience and wellness oriented principles. First, a
majority of those we spoke with found the respective services they received to be
accessible. They also reported service experiences that promoted engagement and
continuity of care; encouraged family involvement; respected their right to self-
determination; and were culturally respectful and appropriate. These five key themes are
further explicated below.
Access to Services
Nearly 60 percent of all interview respondents said that they were referred to services by a
mental health service agency or professional, whereas a quarter of all respondents cited a
peer as a referral source. Counts were not mutually exclusive. Nearly 80 percent of peer
support interview respondents were referred to those services by a peer. That peers are
referring one another to peer support activities seems to be a testament to the helpfulness
of those types of services.
Accessibility is an important element of a recovery oriented system of care.31 Overall,
interview respondents across the three service areas did not report encountering many
access barriers. From respondent reports, it appears that self-advocacy and self-
determination has played an important role in accessing services for over two-thirds (68%)
of the respondents we interviewed. The relatively small number of respondents who had a
complaint about service access mentioned that – although they had accessed and were
receiving services already – they wanted more. Detailed findings related to access are
described in the following paragraphs.
When asked directly whether there were services (peer support or otherwise) that they
wanted but did not receive, six of the nine peer support interview respondents said no. Most
of the peer support respondents noted that one reason for this is the nature of peer
support services versus professional services. For example, where availability of
professional services such as counseling and case management can be restricted by cost
and the relatively limited availability of doctors and staff, peer support services are more
open, available, and flexible in nature. One adult respondent who had been receiving peer
support services for approximately six years, commented, “With peer support, [access] has
never been an issue.” When pressed as to whether other services have been more difficult
to access, he continued by saying, “No, but a lot of it had to do with my own advocacy… I
usually ask for what I need. That’s one of the things that I’ve learned…through [peer
support at] NAMI [National Alliance on Mental Illness].” Additional respondents who
31
Davidson, L., Tondora, J., O’Connell, M.J., Kirk, T., Rockholz, P., & Evans, A.C. (2007). Creating a recovery-
oriented system of behavioral health care: Moving from concept to reality. Psychiatric Rehabilitation Journal,
31(1), 23-31.
Interview Results 89
received peer support services commented that self-advocacy and self-determination play a
key role in being able to access needed and wanted services across the board. They agreed
that the knowledge and confidence for self-advocacy can be cultivated from participation in
peer support activities. The small handful of respondents who noted any issues accessing
peer support services wished that there were more groups or classes to attend or said that
they desired more social outings.
As was the case among respondents who received peer support services, those who
received employment support services did not report encountering problems accessing
services. Only one of the respondents – an older adult who recently completed a mental
health and substance abuse program – complained about access to employment supports.
He was disappointed because he had not qualified for a training program of interest to him.
He explained that this particular training program only accepted people who are homeless,
and he had recently received housing.
When asked directly whether there were services that they wanted but did not receive in
relation to their most recent crisis, a majority of crisis services interview respondents replied
that there were not. Only one respondent, a male transition age youth, said that he did not
access the services he needed and wanted because of stigma. This respondent explained
that he did not want his boss to know he has mental health concerns, so he had not
requested time off to get services. He cited cost as another barrier to receiving the services
he wanted. A majority of crisis intervention respondents, however, simply stated that they
could not think of any services they needed or wanted but did not receive, and they did not
expand further. Others took the opportunity to praise their case managers or other mental
health professionals who helped them navigate the system and meet their mental health
needs. For example, an older adult named Daniel who had isolated himself during a period
of unemployment and became suicidal, explained how his case manager “did some work
and found…a community that could help me. She was a grand, great, incredible resource for
opening up options.” (To learn more about Daniel’s story, read his recovery profile below.)
Interview Results 90
“Daniel”
At the age of 65, Daniel realized that he had spent 60 years not telling the truth. “I lived my life bumping
along, not really ever doing what I wanted,” he reflects. In his youth, having been on track to join the
Catholic Church as a priest, his decision not to become ordained forced him to admit to his mother and
father that he was homosexual. He recounts, “That was the end of any relationship with my mother I
could have, and that was when my dad told me to take my life.” Instead, Daniel moved forward without
the support of his parents, and eventually entered into a career working with patients in a psychiatric
hospital. “But all the guilt I never dealt with – I just shoved it aside,” he says. For decades, Daniel
internalized his overwhelming anger at the cataclysm between the church, his family, and his sexual
identity. He convinced himself that, even as a homosexual, he couldn’t be a “bad guy” if he remained
celibate. Anger steadily became depression, and when Daniel eventually lost his job at the hospital – and
with it, his extended work family – it initiated a crisis.
Along with his job and co-workers, Daniel also lost his medical insurance. His medication ran out, and for
a year and a half, he sat in his apartment surviving on his savings and investments. An avid reader, Daniel
stopped reading; he stopped going to church; he stopped sleeping. “I just stopped everything,” he
summarizes. Although his sister would call once a month, Daniel was adept at putting on a good face.
“I’m a great liar – I’ve done that all my life.” Toward the end, for a period of three and a half months,
Daniel saw nothing but his bedroom, his bathroom, and the front door through which he accepted
deliveries. As he drew closer to the last of his savings he began to suffer from panic attacks; envisioning
no place else to go, he ultimately contemplated suicide. His plan was to drive his car out into the desert,
sit on a rock, and wait in hopes that no one would find him. “I knew intellectually it would be a difficult
way to die,” he notes, “but I thought it was better than the pain of continued existence. But God said no,
and it worked out in a much different way.”
Luckily, Daniel’s sister heard something in his voice over the phone one day that she did not like, and she
decided to pay him a visit. She was then able to find a psychologist who directed Daniel to county
services. Reassured that he would not be committed into a psychiatric hospital, Daniel “surrendered to
what was” and was next referred to Older Adult Services where he met his current social worker and
psychiatrist. Daniel was placed back on medication and met with his social worker every two weeks.
Their meetings gradually became monthly, and she introduced him to a therapist in an outreach program
offering mental health services specifically for the gay community. Daniel was relieved by the non-
threatening, judgment-free attention of his therapist. “I found that I could start telling the truth for the
first time in my life, and I didn’t feel bad about it.”
Daniel describes this past year as being very productive. As a result of therapy, medication, and following
his homework assignments (such as journaling), he feels that he has accomplished a great deal of
personal growth and is beginning to think about the possibilities his future holds. Recently, a former co-
worker asked Daniel to fill an open position she thought he would fit well. Although officially retired, and
not entirely sure whether he was ready for work, Daniel decided he was now emotionally stable enough
to accept. With this gain in financial stability, Daniel is next looking forward to managing his credit card
debt. He also appreciates being back in the service of others, particularly as many of the people he now
works with are diagnosed as depressed. He anticipates volunteering for the same outreach services from
which he has been receiving therapy as a way of “paying back the great debt” he owes to them.
“Literally, they saved my life,” he emphasizes. “It was a safety net that I didn’t know we had. Thank God
someone else did. Without services, I really would have done away with myself.” Indeed, Daniel is
hopeful for his future. “Whatever time I am given now, I’m going to use it,” he insists. “I’m coming more
and more to my truth and the ‘me’ I hope to be once I’m done. What’s the phrase? God’s not finished
with me yet!”
Interview Results 91
In particular, the FSP participants we spoke with perceived no service barriers. An adult
male FSP partner who has experienced bouts of homelessness and incarceration remarked,
“They help you when you ask for it or when they see that you really, really need it but don’t
want to ask for it. If they see you need it, they give it to you anyway.” A female transition
age youth FSP participant agreed, “They do anything – like taking you to the grocery store.
I’m really going to miss this program, because I’m getting ready to age out.”
A theme about the role that self-advocacy (or advocacy of a parent on behalf of a child)
plays in helping to access appropriate and useful services emerged from conversations with
crisis services respondents, as it had with peer support respondents. Nearly half of
respondents who had received crisis services described situations where they advocated
strongly for services that they or their children needed, sometimes after being stuck in
ineffective or inappropriate programs. Examples of self-advocacy were shared by
respondents from all age groups. Two interview respondents (one older adult and one
adult) described situations where they were accepted into programs and were receiving
mental health services; however, those services were located in areas that were not
conducive to their recovery. The first was an older adult veteran who had PTSD, and the
program was located in an area of the city that was loud and where gunshots could
sometimes be heard. The second was an adult woman with co-occurring substance abuse
and mental health issues. The program she was first accepted into was located in an area
where she used to do drugs; she feared she would not be able to successfully complete the
program in that environment. Both of these respondents successfully advocated for
themselves and were able to access services in other locations, thereby removing
themselves from stressors that could compromise their recovery and wellness.
It was particularly striking that every parent who was interviewed described an important
act or acts of advocacy that ultimately led to more desirable and effective services for their
child and/or family. For one set of parents, advocacy began when their male child was in
early grade school, and they had to fight with the school system to get proper assessments
and an Individual Education Plan (IEP) for him. After an incident several years later that
required the removal of their son from his school, these parents found themselves
advocating with the district attorney’s office so that their son would go to “a place that
would take care of his mental health needs, as well as build character and provide
education” rather than to “work camp environment”, which was where he was headed.
During that time, they advocated and worked with the county department of mental health
to find residential treatment options. However, they later watched their son struggle in that
residential placement because it did not encourage family involvement. In addition,
employed staff persons were sometimes inappropriate in their physical discipline, and the
program applied inconsistent guidelines pertaining to when and why restrictions were
increased and decreased. Once again, these parents advocated on behalf of their son and,
after additional research, successfully managed to have him moved to a more wellness
oriented residential program. In general, parents who were interviewed described
advocating for appropriate services for their children as a part time job, as they sought out
people from different service delivery systems who could help and found other supportive
resources including seminars, workshops, and conferences.
Interview Results 92
This need for self-advocacy in order access services is related to gaps and areas for
improvement identified by over half of respondents who were interviewed about their
experiences with crisis services. In particular, several respondents perceived a gap in readily
accessible information about available services. Among this subsample of respondents, this
was a complaint most common among those whose school-age children needed mental
health services. One father described his frustration about the lack of information regarding
IEP services and postulated that part of the school’s reluctance to provide transparent
information resulted from not wanting to pay for services.
“The criticism, again, [is about] the IEP experience, because the more
accommodations you ask for – and the more you know that you’re entitled to – the
more it’s going to cost the school district. It’s like a lot of parents have kids with
special needs, they don’t know what to ask for. That’s why Wraparound is around.
That's why parent peer support, evidence based practice, you have all these terms
out, but it’s about information so that you can make the best decision for your child.
That's the hard thing. A lot of times information is THERE, but it’s not in a consumer-
friendly form … So how do you know how to take advantage of what it is?”
Parents also noted gaps in information from the school about additional mental health
resources available for their children, including information about peer support groups or
other community services and activities for children and youth with special needs. One
mother explained that she did not receive any information from the school system or
residential facility when her child turned 18 years of age. She found out about a transition
age youth program she would consider enrolling him in for independent living skills almost
by accident:
“The interesting thing is the only reason I know that [the TAY program] exists is
because I took the peer advocacy training a couple of months ago, and one of the
girls, who was nineteen who was also taking the training... I was talking to her about
it and that's how I found out about it, so at least… now I’m aware that that’s there.”
She also described lack of information about or access to transitional housing for the period
of time in between when a psychiatrist said it was no longer safe for her child to be in the
home and the time when a residential facility had been selected. Another mother regretted
that school-based services (e.g., therapy) are less accessible during summer months when
school is not in session. She contended that this interruption in the continuity of care
potentially threatened improvement in her son’s functioning and wellness.
The primary concern for adults and older adults with respect to service access was the
amount of time that they get to spend with the mental health professionals who work with
them. Several respondents observed that the case loads of psychiatrists, counselors,
personal services coordinators, case managers, and other mental health staff are too high.
One adult female respondent who had been living on the streets for a time prior to services
had this to say:
Interview Results 93
“I don’t want [doctors] pushing you out like you’re a number. I think they try to push
you along – say what you gotta say and then push you out. Like how we’re talking
now [during this interview], I couldn’t have talked that long [with my doctor] because
they got someone coming in 15 minutes. I only have 15 minutes with them.”
Another adult female explained:
“I wish psychiatrists would sit and take a little bit longer with you. Cause you just go
and tell them how you feel, a little about how you feel, and they just diagnose you
and give you the prescription. You know what I mean? And I just feel like if they could
take a little more time to … talk to you and see what’s really going on.”
A third adult female respondent with a long history of battling depression described a
frustrating experience trying to get the time and attention she wanted with mental health
professionals in a new FSP program. She surmised that the problem was due to caseloads
being too high:
“When we moved in originally based on the idea that it would be fully staffed all the
time and … having all these classes and groups available to us. It was such a new
project that that didn’t happen. … It just ended up being that when I did need
someone, the personal services coordinator that I had for that time, I would call him
and he’d tell me we would set up a therapy appointment. I’d be ready, and I’d
confirm the time with him, but there was always another crisis [for someone else in
the program] that happened that intervened in my receiving my services. … It’s a
good system they have, but they have a caseload now that a client can’t just pop in
and talk to their coordinator. You have to set an appointment now, which is hard,
because you can’t always predict when you are going to need to talk to someone or
need support.”
Promote Engagement and Continuous Care
Among the five main themes related to the recovery/resilience orientation of services, none
was more prominent than that of promoting engagement and continuity of care. Nearly 70
percent of respondents said they experienced services that were engaging and promoted
continuous care. Across the three service areas, respondents discussed this domain of
recovery oriented services more than any other, though it was most frequently mentioned
by respondents whose interviews focused on peer support.
Providing a Welcoming Environment
Many interview respondents expressed that they were immediately open to continuing
services due to the welcoming environment they encountered in peer support groups and
at places like wellness centers and other program sites. For example, a transgendered
transition age youth determined immediately that he felt comfortable and would like to
continue receiving services when he first visited a LGBTQI youth center:
Interview Results 94
“From the beginning, I thought it was just such a great space, and I thought I would
come back. I liked this place from the start…It was totally welcoming. It’s very
welcoming to everyone. They are happy to have you there. Every staff member is
more than happy to talk with you.”
Respondents used words like “homey”, “friendly”, and “casual” to describe program
environments like wellness centers and “nice” and “happy” were words repeatedly used to
describe staff and other people they encountered there. Some said they felt “respected”,
“accepted”, and “safe” in these environments. Others contrasted the home-like feel of their
program spaces with less desirable clinical environments and said, “It’s more like family.”
Valuing Peer Expertise and Involvement
Recovery/resilience oriented services value the expertise of different types of individuals
beyond clinical professionals, including peers who volunteer or who are hired, and different
types of services, including peer support services.32,33 Interview respondents highly valued
their participation in peer support activities and reported being engaged and inspired by
(former) consumers employed as staff. Our respondents found commonalities with and
could relate to these individuals – and vice versa. One respondent’s co-occurring disorder
resulted in her losing her children to the child welfare system and becoming homeless. She
found a residential treatment program and is now trying hard to get her children back. She
described the program she participates in as one that promotes peer expertise and
involvement, and she explained why she has found this helpful:
“I don’t feel like a patient. I feel like, I don’t know, like I trust them, you know? I trust
them and I respect their opinions…They’ve done this before. They actually graduated
from this program, and I just felt all of them have something different to teach me.
You know what I mean? I learned something from all three of them.”
An older adult respondent described the services she received as valuing peer expertise and
involvement when she expressed that the best thing about the peer support services she
received was relating to someone and feeling like they really “get it”. In the short passage
below, she describes her relationship with a favorite counselor:
“They need more people like [Marcy]. More understanding people [like her]. You
know, I have not asked her what she has been through, but I get the impression that
she or someone close to her has had some of the same issues that she talks to me
about. … It was just like talking to a friend; she was very on a personal side. She was
not a stuffed shirt type.”
32
Ibid 31.
33
Jacobson, N. & Curtis, L. (2000). Recovery as policy in mental health services: Strategies emerging from the
states. Psychosocial Rehabilitation Journal, (23)4, 333-347.
Interview Results 95
Delivering Person-Centered and Individualized Care
Recovery/resilience oriented services also value the role of professional helpers but stress
that they must successfully engage service recipients and provide person-centered,
individualized care.34 Interview respondents generally felt that their recent experiences with
mental health services and professionals were individualized and that their professional
helpers actively endeavored to know, respect, and relate to them as individuals as opposed
to treating them like a diagnosis. In the most successful situations, even the respondents
who described themselves as being “paranoid” that people were out to hurt them and
mistrustful of the system found professional helpers that they opened up to and, over time,
came to trust. One transition age youth respondent’s misdiagnoses as a child and related
conflicts with family resulted in her spending time in the juvenile justice system and
experiencing periods of homelessness. Kenya told us about several negative experiences
with mental health services early in her life, but she has begun to find her footing and build
a foundation for recovery during her time as an FSP participant. Kenya attributed her
willingness to engage in and continue services to dedicated staff that offered constant
support and patiently tried to get to know her as an individual:
“In the beginning, it’s hard to connect with someone who hasn’t walked the same
lifestyle as you. It is hard. You sit there and look at them like, you don’t know
nothing. But they try. They really try. They ask you to try to help them understand. I
had a difficult time at the beginning to connect with someone and feel safe. Over the
years, I’ve understood they are really here to learn and to help me… That’s what
eased me up a little bit. At the beginning, I didn’t feel they understood. … The time
they invest, it speaks volumes. If they didn’t have all that time to invest, it probably
wouldn’t be working.”
(To learn more about Kenya’s story, read her recovery profile below.)
Another transition age youth respondent who spent time in residential care contrasted his
most recent experience concerning a mental health care professional who treated him like
an individual with earlier experiences that were not as positive or productive:
“My [recent] therapist was a guy by the name of [Aaron]. He took time to sit down
and know me as a person. I don’t think once he really sat there and said this is your
problem; this is how you fix it; take these meds; get away from me. Every therapist
I’ve had has told me take these meds, don’t do this, this is your problem, and this is
how you fix it, and if you don’t fix it you’re not a normal human being. Any therapist
who says that to their patient will get nowhere because their patient does not want
to hear those things, first of all, even if they could be true or they are true. Secondly,
you need to know the person. … To help the person with their problem, you have to
see how their personality intertwines with the mental illness.”
34
Ibid 31 and 33.
Interview Results 96
“Kenya”
Kenya has received mental health services since the age of 10, when she was misdiagnosed with
depression and a tendency to “act out.” Throughout her upbringing in foster care, Kenya went to family
therapy and was later enrolled in services for troubled teens. Because she did not take her medication
consistently, Kenya would experience bouts of depression and outbursts. She frequently ended up in
juvenile hall as a way of getting back on medication. When she became ineligible for teen services at age
18, probation services helped her to find her own apartment. During that time, she was working and
taking courses to become a registered nurse. Kenya eventually chose to give up her apartment to go care
for her ailing grandmother. Returning to her family environment, she found herself back in a negative
pattern. “There’s no support from my family. For a long time, I was just a ‘loony bin’ to them,” she
explains. Needing to remove herself from that “stressful” situation, Kenya moved out and found herself
homeless at 21 years old. With no access to transportation, she dropped out of nursing school.
Kenya was off medication by the time she found a shelter program that offered transitional housing and
connected her with an FSP. At that point, she was diagnosed as having bipolar disorder, received
therapy, and was put on medication. She was living in group housing – a challenging situation for her –
and was put on a waiting list for her own apartment. As she waited and started to feel better, the FSP
helped her pay for school and provided transportation. Kenya became a certified nursing assistant (CNA)
and began working off and on for two years. However, she went off of medication again and found it
difficult to maintain employment due to high levels of stress, depression, and her inability to cope. For a
couple years she was working here and there and losing jobs. “The depression got the best part of me
and I couldn’t get out of the house,” she describes. “I didn’t feel safe being around people and didn’t feel
like people were safe around me. I was getting into fights and arguments everywhere.”
Through these personal challenges, Kenya has seen her therapist on a weekly basis. Her psychiatrist is
available every day of the week and on-call services are available at any time of the day. Kenya reports
that she has the phone numbers of all the PSC’s programmed into her cell phone. Her therapist will come
to her home if she feels unable to leave the house and will call if she hasn’t visited the center. Kenya also
receives reminders to take her medication, which she now does. She emphasizes, however, that this time
her decisions to take advantage of services and medication are her own. “The only thing why I’ve been in
this program so long – not just the housing – is that I never felt forced,” she states. “When I was younger,
everything was forced – therapy and medication. When I got [to the FSP] it was like, if you’re not ready,
then we can’t make you. Nothing was ever forced. They waited – every time I started medication and got
off – they waited and they were still there for me when I came back. No matter what.”
Despite what she refers to as a “down stupor” over the past nine months, the individualized care and the
combination of mental health services Kenya has received have helped her to make some important
strides in her outlook on life. “I realize that everyone is not my enemy… It makes a difference when you
talk to a therapist and they show you that you don’t have to be on the defensive with everyone. Not
everyone is going to try to hurt you.” While she admits that no program is perfect, Kenya attributes her
well-being to her consistent receipt of services through the FSP. “If I wasn’t receiving services here,” she
maintains, “I would be in jail or be killed somewhere. My attitude was where I didn’t care about
anything, and through this program I just found some people who are really there for you. And I’ve not
had that my entire life from my friends or family. To find it in a program is unbelievable.”
Approaching the age of 26, Kenya will soon be too old to remain with her FSP. She is currently working
with her case manager to make a transition into an adult program. Although she is concerned about how
well she will function without such attentive support and whether she will be able to establish trusting
relationships with her new psychiatrist and other staff, Kenya remains, on the whole, positive about the
changes ahead. She reiterates, “I’m learning that people aren’t out to hurt me.”
Interview Results 97
Many respondents further explained the importance of being known and treated as an
individual and reported that this was their recent experience in receiving mental health
services. One adult respondent described his experience in this way:
“I used to be paranoid, like they were recording me to the FBI. But I am getting
better. I’m learning to trust more. People aren’t out to sabotage me or hurt me. I’m
getting better about that. … [Staff] helped bring me through the things I went
through [because] they know me as a person. They know my moods and where I’m
coming from. … Here [at this program] they know you. You’re not just a number.
They know you as a person. … They are looking out for my safety and theirs. It helps
me feel safety. I can be at ease when I come here.”
Adopting a Philosophy That Recovery Is Possible
Interview respondents were engaged and inspired on their road to recovery by peer
support, employment support, and crisis services that adopted the philosophy that recovery
is possible. These services and service providers were strengths-based and encouraging. At
times, this service philosophy offered those who were receiving services – and perhaps had
been receiving them for quite some time – a new perspective. For instance, one adult FSP
participant described a time when she wanted to go to the hospital after a family member
committed suicide. However, because she was well-known to her personal services
coordinator and other staff within a program that adopted a recovery oriented philosophy,
she safely received encouragement and support instead of hospitalization. The quote below
demonstrates how this message of hope and recovery inspired her:
“[I wanted to go to the hospital], because [people] would speak about going to the
hospital as “a vacation”. So I thought maybe that’s what I needed. But… [my
personal services coordinator] saw that that wouldn’t have been the best for me in
the end. … They encouraged me that I didn’t want that, that I could do better. … As
mental health patients, we tend to do what other people tell us to do. We act like we
have a handicap. But to know that you are in recovery – that was a good term that
they would talk about with us – they would tell us that we would be better, that we
could go out and function in society. It’s huge. It’s encouragement. It plays on your
mind, knowing you can do this. Because personally, I had so much of the opposite –
of people telling me that I CAN’T do anything. So I think that that was very, very
important having that. Having them tell you you’re not going to always be sick. That
you aren’t always going to have the label of being a mental health patient for the
rest of your life. … I remember them telling me it wasn’t permanent housing, it was a
stepping stone. I remember hearing that: This is not permanent! There is a goal of
going out and ‘being normal’ and living in the community.”
Interview Results 98
Many respondents were quick to state their preference for providers and services that
believe in and promote recovery. Rebecca, an adult with bipolar disorder doing her best to
rebuild her life after being disowned by her family, contrasted the positive experiences she
had with peer and employment support services with the negative relationship she had
with her psychiatrist:
“I didn’t feel supported by the psychiatrist – he was of the school where you take a
pill and you don’t ever get better – you’re going to be like that for the rest of your
life. But the job rehabilitation had the philosophy that you could get better.”
(To learn more about Rebecca’s story, read her recovery profile below.)
Providing Multiple Types of Services and Supports
Recovery/resilience oriented care supports the individual’s goals and efforts to lead full and
productive lives in the broader community. This requires continuity of care, which involves
facilitating access to a variety of supportive services, including employment support and
housing, which help make recovery sustainable.35 As mentioned above, a majority of the
interview respondents we spoke with received multiple types of services and supports in
varying combinations: for example, counseling, staff support, psychiatric care and
medication, peer support, employment or school support, hotline and warmline support,
housing, physical health care services, and/or substance abuse services. A sizeable
proportion (40%) of interview respondents across service areas mentioned that they had
received employment supports in combination with other services. As well, approximately
one third of interview respondents reported that they obtained housing or improved their
housing situation through MHSA supported services.
35
Ibid 31.
Interview Results 99
“Rebecca”
Rebecca estimates that she had suffered from major depression for about 20 years before she was
diagnosed with bipolar disorder. She recognizes now that when she was a stay-at-home mom, she was
co-dependent on her husband and children. These are both reasons why she fell into crisis when her
husband initiated divorce. Feeling devastated that she had “lost everything”, Rebecca remembers this as
a time when she needed love and support. Instead, her family expected her to “be stronger” and to pull
herself “up by the bootstraps.” Rebecca became suicidal. “My family wanted to lock me up and get help,”
she recounts. Her family called upon Emergency Treatment Services, and then Rebecca discovered that
they had filed a restraining order against her. For the next year or so, Rebecca was homeless, in and out
of jail, and in and out of the hospital. Her calls home went unanswered.
Rebecca was accepted into an FSP program following her last hospitalization. She received her bus pass,
food stamps, and basic hygiene supplies and then attended peer support groups and job training once a
week. Rebecca identifies therapy as an essential resource, and continues to reach out to her therapist
when stressful issues resurface in her life. But peer support stands out to Rebecca as a tremendous
support system in her recovery. She explains, “Peer support helped me because there was
understanding. When I would share, there was immediate connection and understanding, so I didn’t feel
so bad or so different. It was only when I was in the peer support that I felt like I was normal. In the real
world, I felt abnormal and that I didn’t belong. And in peer support I felt normal, and I was happy. I was
working on my issues, and that’s how it helped me the most. It made me feel like normal.”
In addition to peer support, Rebecca has found proactive coping and self-determination to be important
tools in her recovery. She has found it especially necessary to be proactive in her interactions with a
psychiatrist who does not embrace the philosophy that recovery is possible. “I find that education and
learning about my diagnosis and medication is the best thing I can do for myself and not rely on doctors,”
she insists. Although she would research her own medications and suggest treatment alternatives that
would avoid side effects and addiction, Rebecca felt her voice was never heard. “It is so hard to advocate
for yourself when you have these side effects and the doctor doesn’t believe you, or they think you are
just ‘med-seeking,’’’ she complains. “So when I go into a [peer] group and we talk about frustrations with
a doctor, I feel so much better, because I don’t feel like I’m the only one he treats like this.”
About a year ago, Rebecca went to peer employment training and then began volunteering for NAMI. “I
was not stable until after I went to the peer employment training and began working…and running
groups. Then I found a purpose,” she recounts. Rebecca showed up at the NAMI office every day as a
way of getting herself out of the house. “And that’s when I started getting better,” she says. “You have to
keep your mind occupied and feel like you did something meaningful each day.” Rebecca was ultimately
hired by NAMI and she now shares her life story with the community, finding that it boosts her self-
perception and confidence. “I’m finding meaning and purpose in telling my story, educating people about
mental illness, reducing the stigma, and letting people know that people can recover,” she comments.
“I’m getting a good response people tell me how honest and brave I am, and I feel like I’m helping.”
Despite struggles with her psychiatrist, Rebecca believes that the services offered through FSP “saved my
life.” While she continues to try to mend her relationships with her family and is in contact with them at
a minimal level, one of the most important things right now is that Rebecca knows that she is not alone
and finds purpose in giving back as a person with lived experience. “I just think that the peer
employment training that I went through to become a peer support specialist changed my life. I think
that peer support is the way to go because you can relate to somebody with the experiences you’ve gone
through to help somebody else. You can help someone advocate for themselves, for example. You can
help them feel normal and to feel loved and that there’s hope and things will get better because I am
evidence that things do get better.”
Interview Results 100
Family Involvement
Recovery does not happen in isolation. Recovery/resilience oriented services encourage the
involvement of family and other natural supports in the process of treatment and
recovery.36 Interview respondents discussed the extent to which the services they received
involved family support. Respondents who received crisis services most frequently
described the involvement of family in counseling sessions, Wraparound services, and/or
visits at residential facilities. Parents with children or transition age youth in residential care
felt better served by their recent experience with programs that encouraged family contact
and visits than by previous services that, for example, used family visitation and weekend
home passes as “sticks and carrots” in an attempt to control youth behavior in residential
placement. When this type of practice occurs, it can become more difficult for youth to
reintegrate into their families and communities after placement in residential care. One
child’s parents experienced this practice of withholding visitation privileges when their son
was first placed in an out-of-state program. They were much more satisfied with the
services they later found in California that facilitated a high level of family involvement.
“Well, I think the fact that what really worked was… being able to do home visits. … I
think the access to us, coming home twice a month on weekends – sometimes
staying longer if it was a three-day weekend. It helped him to re-assimilate.”
Interviewees who received peer support services tended to mention family involvement
more as an aspect of the professional counseling they were receiving as opposed to the
peer support services they received. Those who received employment support services had
the fewest examples of services that encouraged family involvement. A handful of
respondents said that despite the fact that the services they received encouraged it, their
families did not want to be involved in their treatment and recovery.
Right to Self-Determination
Recovery/resilience oriented services support the preferences and goals of people living
with mental health concerns.37 This theme, too, emerged across interview respondents and
service areas but was strongest among those who received employment support services.
Those who spoke about the right to self-determination as a meaningful aspect of their
service experience underscored the importance of being afforded the opportunity and
“independence” to exercise “choice” in both their treatment and life decisions. These
respondents appreciated receiving “suggestions” from staff, but they did not want to be
“forced” – either literally (e.g., one transition age youth respondent did not want staff to
“physically touch me or tackle me”) or figuratively. Respondents wanted to be afforded the
leeway and dignity to make their own decisions and their own mistakes; they simply wanted
the option to take responsibility for themselves.
36
Ibid 31 and 33.
37
Ibid 31 and 33.
Interview Results 101
For instance, one adult veteran in our interview sample had battled co-occurring substance
abuse and mental health issues for a number of years. Over time, he has had periods of
employment and sobriety, and he has also endured unemployment, homelessness, and
incarceration. During his recent experience receiving employment support and other
services, he was assigned an enthusiastic career counselor who was eager to place him in a
job, regardless of his preferences. In his words:
“With [the employment support program] I had a little problem with my career guy.
He was so gung-ho, because I have a couple trades under my belt. I’ve worked for a
car dealership doing frontend alignments. I used to work for a tire company and
changed truck tires on the road for 20 years. But I left that, and I wanted to drive
truck – that was my passion. Well, the career guy was adamant about wanting to get
me any type of job at first, and I didn’t want to go back to changing tires and that
stuff. So we butted heads there. I stopped going to [the employment support
program] for a minute and talked to my case manager about it. He got involved in it,
and then the [program] boss said that if you don’t want one of those jobs, that’s fine.
We’ll strictly try to get you truck driving jobs.”
In this example, the career counselor may have had the best intentions but not the best
training; he was not respecting the respondent’s right to self-determination. This pushed
the respondent away from the services he wanted and needed. With advocacy on the
respondent’s part, and that of his case manager, the issue was revisited. The respondent’s
right to self-determination was respected, and he made it through truck driving school.
Currently, he enjoys his dream job as a truck driver.
An additional example comes from an adult respondent who recounted a time of crisis
when FSP staff put the philosophy of self-determination into practice. One can discern the
importance that the respondent places on this interaction:
“They listened to me and let me vent. Not vent in a threatening way, but they let me
talk, and they listened to me. They weren’t too quick to give me a suggestion, but
they helped address the management in my building and what they were going to do
to help me sustain my housing. Because my mom backed up off me, my own mom.
My mom wanted to take over my finances, but I didn’t want to do that because I’d
have been like a child going back there. I didn’t want that. My mom would treat me
like a child. So I let [the program] suggest to me substitute pay services with [the]
County. Then I worked it out. [Staff] suggested this to me, but they weren’t going to
force me. They said it was up to me, but they just helped me figure it all out.”
Cultural Appropriateness
Recovery/resilience oriented services should deliver culturally respectful and responsive
interventions.38 During each interview, respondents were pointedly asked, “Did the people
38
Ibid 31.
Interview Results 102
who provided you with crisis services respect you and your cultural background?” A majority
of interview respondents perceived their service experience to be culturally appropriate.
Relatively few respondents expounded on the reasons why. Those who did specifically
mentioned feeling “no judgment” about and “acceptance” of their differences from
providers.
Three of the four monolingual Spanish and Chinese speaking individuals in our sample
reported that their case workers come from a similar cultural background and speak the
same language; they found this “comfortable” and “helpful”. An older adult monolingual
Spanish speaker explained that the counselor who visits her in her home only speaks English
but always brings a translator. In her words, “Of course it’s not the same having a translator,
but it was fine to have a translator. It was fine. For me, I understood it fine.”
Recovery/Resilience
During the interviews, service recipients talked about the differences they believe mental
health services have made in their lives. Some respondents spoke about changes in their
clinical recovery (i.e., improved functioning, reduced symptoms); more often, they shared
their feelings about and examples of where they are on their individual journeys of personal
recovery/resilience. According to the stories shared by interview respondents, recovery and
hope can be a reality with opportunity and support from the service system and others. In
the following pages, findings on recovery/resilience are grouped by five important themes:
Positive Self-Perception and Self-Determination; Proactive Coping; Purposeful Pursuits;
Connecting and Forming Relationships; and Hopefulness for the Future. These themes are
described in detail below.
Personal recovery is rarely a linear process.39,40,41 Accordingly, interview respondents
shared information about their struggles as well as their successes. Prior to presenting the
recovery themes, we briefly present some of the roadblocks and daily stressors described
by respondents that have challenged them and may continue to do so as they travel their
recovery paths. Recognition of ongoing challenges brings to focus the importance of a
recovery/resilience oriented system that promotes continuity of care. At the same time, it
renders even more impressive the stories of recovery/resilience presented throughout this
report.
39
Ridgway, P. (2001). Restorying psychiatric disability: Learning from first person recovery narratives.
Psychiatric Rehabilitation Journal, (24)4, 335-343.
40
Smith, M.K. (2000). Recovery from a severe psychiatric disability: Findings of a qualitative study. Psychiatric
Rehabilitation Journal, (24)2, 149-158.
41
Sullivan, W.P. (1994). A long and winding road: The process of recovery from severe mental illness.
Innovations & Research, (3)3, 19-27.
Interview Results 103
Barriers and Risks to Recovery
In terms of system or service level barriers, fewer than half of respondents discussed a lack
of or inappropriate services as a barrier they had recently faced. Within this category were
the barriers to access described in a previous section of the report. In addition, a quarter of
interview respondents said that they have had problems recently accessing housing services
or have experienced housing instability. The stress caused by lack of housing and/or housing
instability can be a stumbling block in recovery.
Slightly more than one third of interview respondents noted recent or ongoing challenges
with respect to functioning, with the majority of functioning issues described as ongoing
depressive symptoms and continued child behavior problems.
Approximately half of the interview respondents described at least one stressor in their life.
The most often identified stressor was that of family dynamics, including parents with
children who need a lot of support. Other stressors mentioned by a handful of individuals
each included physical/medical issues, financial stress, work or school stress, and risky or
stressful environments (e.g., living in a high drug-use neighborhood). Respondents admitted
that, at times, they get in their own way when it comes to recovery. Risk taking behaviors
were reported by less than a quarter of those we interviewed; these included substance
use, breaking the law, or going off medication.
Altogether, these are the mental health stressors for which respondents continue to utilize
the safety net of services and supports from the mental health system. For a good portion
of the respondents we spoke with, these mental health services (and the supportive
connections with professionals and peers that are gained through them) are critical because
they tend to lack natural supports, especially the support of family. A good portion of
respondents explained that they were disconnected from their families and could not
necessarily rely on them as a source of support.
Positive Self-Perception and Self-Determination
When asked how the mental health services received have helped them in their daily lives,
a large majority (85%) of those we interviewed across service areas said they had generated
or regenerated a positive self-perception with the assistance of the support they received.
These respondents explained that with a greater sense of self-worth often comes the
confidence and determination to advocate for – and to do – what they need and want both
in terms of their mental health services and living their lives.
Interview Results 104
Those who have mental health concerns can lose their former sense of self or self-worth,
and through the process of recovery they begin to reclaim it.42,43,44 Interview respondents
described supportive peers, case workers, counselors, job trainers, and others who
reminded them of their strengths, complimented them, and encouraged them in their
personal journeys of recovery. For many, positive self-images were recaptured and
reinforced. One adult respondent explained:
“It’s good to be reminded that I am an intelligent and capable person. I am a good
person. And I’ve always had the ability, but some things came in my life that
disrupted that ability a lot.”
Several of those who received employment supports specifically discussed how training
opportunities improved their self-perception and confidence. Another adult respondent
shared the following example:
“I went to peer employment training. It was a two week training that taught you to
be a peer support person. And that was really great. It made me feel like I still had
value and that my experiences can be used to help others. It helped me get through
some of my trauma.”
An older adult who received employment support services saw herself more positively after
she was able to successfully complete a general training meant to prepare individuals for
the world of work. She was hesitant to sign up for the training due to self-doubt and fears
that she could not handle it. Completing the training boosted her self-image as she
demonstrated to herself that she was capable of managing in the world of work, and that
was the “best thing” about services as far as she was concerned:
“Just knowing that I could get up at a certain time, commute, stay in the classroom
for X amount of hours, and get acquainted with new people. That I accomplished
that [was the best part]!”
Finding supportive peer communities and professional services gave many respondents the
strength to embrace their identities, experience self-growth, and gain the self-confidence to
do things they did not think possible. For instance, an adult respondent offered one reason
why peer support services were helpful to him:
“Recognizing that I have something of value... Telling [myself] I am of value – I think
that’s significant for anyone, not just people with mental illness.”
42
Davidson, L., & Strauss, J.S. (1992). Sense of self in recovery from severe mental illness. British Journal of
Medical Psychology, (65), 131-145.
43
Mezzina, R., Davidson, L., Borg, M., Marin, I., Topor, A., & Sells, D. (2006). The social nature of recovery:
Discussion and implications for practice. American Journal of Psychiatric Rehabilitation, (9)1, 63-80.
44
Sells, D.J., Stayner, D.A., & Davidson, L. (2004). Recovering the self in schizophrenia: An integrative review of
qualitative studies. Psychiatric Quarterly, (75)1, 87-97.
Interview Results 105
In another example, a transition age youth respondent boasted a healthy self-image after
two years of meeting with his peer group and attending weekly individual sessions with his
counselor. Bursting at the seams with pride, this respondent explained that he is now
determined to embrace every opportunity to speak about and advocate for himself and his
community:
“My goal was to be out there and to be proud. Ignorance causes a lot of stigma, so I
feel like me being out and proud and educating people, I feel like I can get rid of that
stigma. So that’s the main thing for me – any kind of involvement is just like erasing
that stigma. … I wouldn’t have been able to do that two years ago. The
trans[gender] group has been awesome in every aspect. I’m just ecstatic about it. I
have learned a lot and it’s made me stronger in a sense. … I get support on both
planes – deep down [one-on-one counselor] and at the surface level [peer group].
And on both levels it makes an awesome person – a.k.a. me!”
Like the respondent above, others said they found themselves more open to opportunities
and more determined to live meaningful and fulfilled lives as their self-image improved. The
following quote comes from an adult respondent who has found the determination to
explore a personal relationship as he experiences a more positive self-perception.
“I’m loving myself more, and I’m growing. I’m trying to be a Christian man now. Now
I’m opening up to someone and getting to know them and myself – not giving up,
having to deal with conflicts and issues. I’ve been used to doing things my way and
running away and resorting to drugs and alcohol to numb myself – thinking things
are the end of the world and that no one would love me or I was ugly or I could never
get a woman. I think it’s coming to pass now slowly… I think they say things happen
so you can grow, and I think I’m growing.”
Through the mental health services and supports he has received, another adult respondent
explained that his life has steadily improved. As his self-perception improves and his self-
confidence grows, so too does his determination. He has been working part time for the last
four years and is going to school part time as well. He is on his way to becoming financially
independent.
“I didn’t like the idea of being on SSI for the rest of my life. That’s what motivates
me. … I’m getting to the place where I’m becoming completely independent. The goal
is to have a full time job here or somewhere else and get off of SSI. That’s what gets
me out of bed, so to speak.”
Proactive Coping
Whereas the theme above illustrates one aspect of how study participants feel better as a
result of mental health services (e.g., more positive sense of self, more determined, more
confident), proactive coping is an aspect of recovery pertaining to the strategies people use
Interview Results 106
to manage their mental health issues.45 A majority (70%) of interview respondents
explained that the services and supports they received have helped them learn about their
mental health and accept the challenges that may confront them for the rest of their lives.
However, this acceptance is not passive; rather, it involves being self-aware, anticipatory,
and prepared to take measures to avoid or intervene when necessary. Interview
respondents who learned or decided to practice proactive coping were purposefully pacing
themselves, actively avoiding stressors, and unwaveringly utilizing the services and supports
accessible to them.
Self-Pacing
While examples of self-pacing emerged across service areas, they were relatively abundant
from individuals who are working, training for work, or wanting to work. An older adult who
received employment support services described the helpfulness of self-pacing in an
employment context:
“Actually, I think just the time. Time allowed me to adjust and to grow. And I didn’t
have anyone with a pitchfork behind me pushing me out. Just over time, gradual and
natural, I became more and more capable. … For one thing, it allowed me to start
slow, and at my own comfort level. … I think there is actual healing that had to
happen. And with that, I mean, I have spent a lot of my life, these last few years, just
sitting and letting things settle. I work a four-day work week here. But, I have three
days off. Those three days off allows me to catch up on my sleep, if I have been
pushing myself. I am able to take care of myself.”
Mai, an adult respondent who recently felt well enough to pursue employment support
services, discussed the importance of self-pacing in the context of her consumer internship:
“They don’t pressure me on the hours. Like yesterday, I had a group, but I had pain
and so other people stepped up for me to take care of the group. … So they support
me every week… My supervisor meets me every week and asks how I’m doing, if I’m
stressed, if anyone is pushing me too hard.”
(To learn more about Mai’s story, read her recovery profile below.)
45
Ibid 39, 40, 41,43.
Interview Results 107
“Mai”
Mai immigrated to the United States from Vietnam many years ago and went to college to be a health
care professional. When an injury eventually forced her to leave her job at a hospital, Mai worked three
jobs at a time to support her children. After trying repeatedly to start one unsuccessful business after
another, Mai remembers that she felt very scattered and could not focus or concentrate. At that time,
Mai did not realize that her lack of ability to concentrate was the symptom of an illness. She recalls that
“nothing was a success, and then I get depressed more” until “one day I started hearing voices and it told
me I’m not worth it. I heard voices telling me to go die.” Because of her previous work in the healthcare
field, Mai then realized she might need help. Before she was able to receive it, however, she attempted
suicide and ended up in a psychiatric hospital. The voices in her head continued.
At the hospital, Mai was referred to services at an organization that serves predominantly Asians, and
she has been receiving a plethora of services there for several years. Mai has received psychiatric
counseling and medication, therapy (individual and family), and peer support over the years. She has
been particularly appreciative of the individualized care she has received and grateful for the agency’s
approach of encouraging family involvement. “They had my daughter and son involved. They came to see
the house to see what’s going on with the house. They are flexible and understanding. They look at the
whole. They find as many supports as they can – they come and see my mom, they see my sister…My
counselor knows everything about me and my everyday activities and daily living so she can make my
plan better… We make the plan together.”
It was just earlier this year that Mai felt well enough to try to work again, so she has recently begun to
receive employment support services through the agency. Mai has secured a consumer internship with
the agency – meaning that she is helping with various peer support groups at the agency, including
sewing and jewelry making groups. She also helps with independent living skills by teaching people how
to take the bus and taking them to the market so they can socialize. Mai has confidence in her ability to
help other people, and she says that she will “bring all my effort” to do so. “I know that I don’t have any
problem because from my own self experiences, I can help them. That is my wish.” She takes great pride
in the fact that “a lot of people want to come to my group, participate, get out and do things. I’m happy
about that. … I just like to help, and the people like me here. They say if I’m not here they won’t come to
the group.”
Mai is also beginning to receive other employment supports like computer training, assistance with
resume writing, and help with job applications. She thinks she wants to apply to be a peer mentor at the
agency or a peer advocate who will help people who do not speak English well access services. However,
because Mai is still unsure what type of job she wants, she is going to take an aptitude test offered as
part of employment support services. She harbors a lot of hope for the future and confides with a grin
that there is a part of her that is excited to see if she will qualify to train to become a pharmacy
technician. With the world of work reopening for Mai, there is a lot going on and it might be slightly
overwhelming. When asked about that, however, Mai explains that the agency does not pressure her
about the hours she works, and she gives examples of people stepping up for her to take care of groups
when she does not feel well enough to lead. “They support me every week. My supervisor meets me
every week and asks how I’m doing, if I’m stressed, if anyone is pushing me too hard.”
At the end of the interview, after describing the employment support services she has received and how
she is feeling about things, Mai reflects: “It’s a lot better this year from last year. I have a purpose in life
and feel better now than I did last year.”
Interview Results 108
Recognizing Symptoms and Avoiding Stressors
Part of proactive coping involves recognizing symptoms, avoiding stressors, and being
prepared with strategies to counteract them. James’ mom described the improvements she
has observed in his ability to manage his emotional outbursts around members of his
family. She credited the improvement to work that was done with the whole family through
wraparound services:
“I have seen him make big strides. When he’s about to lose it with me, he’ll stop,
take a breath, and go out and take a walk. Or, if he’s going to lose it with [his
brother], he’ll put his music on, go in his room, and shut his door. When he’s with his
dad, it happens a lot. But [James is] the one who has to stop the escalation and do
something different.”
(To learn more about James’ story, read his recovery profile below.)
In another instance, an older adult veteran who has been incarcerated and homeless
described the importance of monitoring his feelings and moods. He explained why it is
critical for him to recognize when he begins to “slip” and described a simple strategy he
employs to try to relax:
“Something bad happened in this program, and some guy jumped on another. And
when he got arrested and they took him away, I started crying and thinking thank
god it’s not me this time. I watch and learn. Because it’s not them, it’s their choices.
So I make better choices. When I start slipping, I’m the first one to catch myself.
Because I know what I don’t want [i.e., to be in prison or homeless]. I can stay
positive. Sometimes when I don’t know what to do, I just lie down. And time heals. I
wake up, and something has changed. I just get out of my own way.”
Stressors can also be in the environment, like hanging out in areas where there is high drug
and alcohol use, or they can be people who have been negative influences in one’s life. One
adult respondent insisted that since receiving services he has “transitioned myself away
from lots of people who I thought were my friends but who weren’t.” Another said she now
recognizes the need to “stay connected with positive people that are clean and sober” once
she leaves her residential treatment program. To do so, she knows she will need help with
housing services so that she does not end up back on the streets, because “everybody that I
know, other than my parents, uses…drugs or drinks alcohol.” It was important that she
recognized the necessity of continued use of services and supports as part of her proactive
coping strategy.
Interview Results 109
“James”
Since the first grade, James has been in and out of therapy. In the third grade, a parent-teacher
conference revealed that James could not stay physically seated for more than seven minutes; by the
fourth grade, James had been diagnosed with ADHD and prescribed medication. By the time James was
in middle school, he had been diagnosed with bipolar disorder and began to rage at home – sometimes
even blacking out. As his rages began to escalate, his mother and father – out of concern for safety –
were eventually forced to call the police, which concluded with James’ first hospitalization. After that,
although he has two brothers, James was unable to be left alone with either of them. His second
hospitalization was the result of a rage that involved him going after his brother with a power drill.
At the end of this second 72-hour hospital hold, the psychiatrist informed James’ family that he could not
return home. Without any placement services or plans for transitional housing, the hospital advised
James’ parents to sell their house and send him into private care. An impossible suggestion, the family’s
natural supports became invaluable as friends offered to take James in for three months. Through their
marriage separation during that time, James’ mother and father endured such a grueling schedule of
shuttling each of their three sons to separate schools every day that James’ mother could only work part-
time. In addition to this exhausting routine, she was in regular contact with the school district to initiate
James’ hard-won Individualized Education Plan (IEP), and later, to request schools she had researched for
James’ residence. With the help of an educational advocate and, eventually, a lawyer, James was first
transferred into a non-public school and finally into residential care. As a consequence of their legal
guidance, Wraparound services were included in James’ IEP for when residential was complete.
“And thank goodness,” sighs James’ mother. When James returned from two years of residential
placement, she received phone calls and a visit from their Wraparound team. “A team of four people
showed up for the introduction,” she states, as she explains her surprise at the team’s services. Over the
next 15 months, James and his parents would meet with the Wrap team every one to two weeks. In
addition, James would meet with his therapist once a week, as well as with his youth specialist, totaling
up to three weekly therapy sessions. James’ mother would meet with a parent partner on a weekly basis,
and family therapy sessions – which included James’ father and brothers – would take place once a
month. After that point, when James would rage, his mother would call upon their Wraparound team
members until, gradually, she was directed to the “crisis line” if help was needed after hours. “We never
had to call the police after the Wrap team,” she emphatically states.
With therapy, all of the brothers learned how to “detach and disengage” when they felt themselves
winding each other up and, as a result, they can now be left alone with each other. James has developed
coping mechanisms that help him deal with his rage and has learned how to put a stop to his escalations
by removing himself from stressful situations and redirecting his attention to other activities. He is able
to manage his medication, has become increasingly aware of when he is “cycling up,” and takes
responsibility for mitigating his emotional buildup before he reaches a “manic stage.” James became a
leader and role model in his high school and was known to break up fights and help other students
develop their own coping skills as he cooled them off in walks around campus. He was asked to visit his
former residential treatment facility as a guest speaker for the next cohort’s “move-up ceremony.”
Looking toward the future, James has made contact with his birth family in another state. He has made
plans to rent a room there with one of his birth sisters and her three children (some of whom share
James’ diagnoses) and to attend community college. James’ mother expects this to be a challenge for
him, particularly given his potential lack of independent living skills and an environment where there are
likely to be more stressors and fewer natural supports. Nevertheless, she is confident in James’
willingness to try and his ability to stay on his medication. Furthermore, she fully understands that this is
his goal, his decision, and his life. “He has got to do this for his own peace of mind.”
Interview Results 110
Utilizing Services and Supports
Recognizing the need to utilize services and supports in a consistent, strategic manner is a
vital part of proactive coping. Most interview respondents reported that ongoing use of a
range and combination of services – from actively managing medications to utilizing
professional and peer supports – is a purposeful part of their recovery strategy.
An adult respondent who received a range of services including employment supports
revealed her personal philosophy on the matter: It is okay to need help.
“I can be really out of control. I have a felony assault with a deadly weapon, you
know? I get locked up. I go to jail. So I stay with my support. … I have a lot of issues. I
have not come this far by myself. I know that I need as much support as humanly
possible because, you know, today’s a good day, but I do not know what tomorrow is
like… I realize how important it is to ask for help, when you need it, and it’s okay
when you ask for help.”
Another respondent detailed the laundry list of services and supports that she utilizes,
including professional counseling, peer support (from three different agencies), drug and
alcohol treatment, and school/employment support. In particular, she highlighted the
importance, for her, of actively managing her medication:
“My medication helps because when I hear the voices in my head that tell me I’m no
good and then I take my meds and I don’t hear them anymore. … Oh yeah. Without
them I know I would be dead right now. I would have killed myself.”
A third example is provided in the account of an adult respondent who described two
instances when he recognized the importance of “protecting” his recovery by reaching out
for help by contacting a crisis hotline:
“There have been a couple instances where I’ve had to call because one of the clients
[in a building where I work] was symptomatic with their diagnosis. And I started
dating one of the residents there, because they don’t have rules about that, but we
broke up. Then she started hanging out with this guy around there, and he is trouble.
I was worried about her. I started thinking about doing something to him … I’ve got
two strikes, and if I get three on this one, I’m going to make it count. So I called crisis
to say either he’s going to do something to me, or I’m going to do something to
him… So that’s an example of me using crisis services. … I was [also] involved with
one girl and she was very violent. She threatened to stab me, so I called the crisis
team and tried to get help. I don’t want to go back to prison. My life is okay now, and
I finally have a second chance.”
Interview Results 111
Purposeful Pursuits
An important element of recovery involves actively engaging and finding meaning and
purpose in life.46 Most of the interview respondents in our study credited the mental health
supports and services they received with improving their lives. A majority (70%) of interview
participants who delineated this improvement as including increased interest and
engagement in meaningful activities and pursuits such as spirituality, vocational interests,
and “giving back” to help others. One of the veterans in our sample explained how his
spirituality helps to ground him in his recovery:
“Once I got to be 16-17, I wanted to rebel from going to church every Sunday. But
now that I’m grown up, I’m glad my parents made me go through it, because it is a
foundation that I can always go back to. I like to go listen to sermons. It’s fulfilling to
me. I come out of there inspired, you know? … I’m glad I went through that religious
upbringing, so when things start happening to me, I start praying and thinking about
all that.”
In another example, an older adult who received employment support services explained
how vital work has been to her recovery journey:
“I don’t think that I would have recovered [if I hadn’t received employment support
and become employed]. Because prior to [that], my days consisted of drinking coffee,
smoking, and pacing. I don’t think I would have gotten past that, or maybe I would
have graduated to just sitting. [The work] was healing… I didn’t have a lot of
interests at that time. It took me a long time to heal, to even be able to develop
interests. And a lot of my interests right now are surrounded by my job.”
Many respondents expressed a debt of gratitude to caring peers and dedicated
professionals, and they vowed that they would “give back” to help others. Several were
already doing so through their jobs or volunteer activities, and some were pursuing helping
professions in their educational pursuits. (To learn more about Jason’s story, read his
recovery profile below.) In fact, a strong pattern emerged in that regard, as demonstrated
in the following series of quotes from several different respondents.
“NAMI has given me a lot, to be honest. And mental health services in this county in
general. You know, you’ve been put here because people cared about you, and so I
felt a need to get involved and give back. And I enjoy what I do.”
“I was not stable until this past year until after I went to the peer employment
training and began working answering telephone and filing for the case workers,
running groups… Then I found a purpose. That’s also when I got my Shelter Plus care.
I didn’t like living alone – I was depressed. So it helped me to get out and volunteer
every day because it gave me purpose and meaning. And that’s when I started
getting better.”
46
Ibid 39, 40, 41, 43.
Interview Results 112
“If it wasn’t for Behavioral Health, I don’t even know if I’d be here right now. It was
that bad. But being able to come in [to work] and see everyone’s faces every day.
Just coming to work, period. That’s what my number one thing is – waking up and
coming to work. Giving back to the community. Behavioral Health has really helped
me and enriched my well-being so much, that I love giving back to fellow
consumers.“
“Starting NAMI [in my community] is still a goal of mine. … I want to help as many
people as I humanly can.”
“I am doing stuff helping out at the drop-in center. That’s how I hold on to my
recovery is to go there and help others.”
Interview Results 113
“Jason”
Jason has struggled to complete college and launch into a nursing career as a result of his depressive
disorder. Although he managed to obtain his nursing certificate and was working in a hospice for a time,
three attempts at finishing a college degree program were thwarted due to complications with mental
illness. Jason found he was unable to continue with work because of his disability, despite his passion for
nursing and caring for the elderly.
After several hospitalizations and some time spent on a waiting list, Jason eventually started receiving
case management services from an agency. While receiving case management, he learned from peers at
the agency about employment support opportunities. According to Jason, the agency has a policy that
clients must take the initiative to participate in the employment support program. So he made his own
pro-active inquiry and appreciated that the program “gives you a chance to choose an area you’d like to
volunteer in to earn a stipend and prepare you for future employment.” Although he was afforded a
variety of volunteering options, such as working at an animal shelter or with LGBTQI youth, Jason
gravitated toward an opportunity to work with seniors. After a brief trial period, Jason decided that he
liked the position and has since continued to work there.
The volunteer job seems to have been an excellent match for Jason. “I really enjoy it,” he says and
further estimates, “It’s probably one of my favorite things in the week.” Jason admittedly continues to
struggle through periods of depression, such as the one he had been experiencing around the time of the
interview. However, he sees his volunteer work as a strategy to manage his symptoms. “Just getting out
of the house, and going and volunteering, and getting out of myself really helps… I feel a lot better about
myself doing something nice, rather than being at home, being depressed, thinking about how bad my
life is. If I’m out engaging with other people… helping them, it distracts myself from everything that I’m
going through.”
For Jason, giving back to a community of seniors is a significant pursuit. He feels so strongly about his
own personal growth through giving back to others that he even facilitates a senior group called,
“Growing by Giving Back.” Fostering a reconnection with nursing as a meaningful life activity, Jason feels
that his volunteer experiences have contributed to his own job skill development and continuing career
interests. “I believe I’ve gained a lot of job skills that have helped me,” he says. “It’s really nice getting a
compliment from a counselor who has a Master’s saying, ‘Oh you’ll be a counselor one day, I can just
tell.’”
In his early 20s, Jason’s future plans include returning to college and completing a degree. The more he
works with seniors, the more he thinks he’d like to focus on becoming a counselor, or possibly focus on
gerontology. He currently works with his own counselors to explore options for study and to prepare to
succeed in school when the time comes. But pacing is important, Jason emphasizes, so at the moment
the primary focus is on volunteering through the program and avoiding high levels of stress. For now,
Jason is content and excited about working and connecting with a wonderful group of people. “They’re
just so great. They all remember me and ask, ‘Oh, how are you doing, Jason?’ And it just makes my day
going in and seeing them. They get excited to see me, ask me about how my week was, talk about their
kids, talk about the news... They like to hear about my life.”
Interview Results 114
Connecting and Forming Relationships
Recovery is not a journey walked alone.47,48 For most (65%) of our interview respondents,
the mental health services they received were instrumental in facilitating vital relationships
and connections ranging from one-on-one relationships with professionals and personal
friendships with peers to improving family relationships and dynamics. In addition, the
broader sense of connection to a community of others who share similar experiences was
said to offer perspective and to provide comfort that one is not alone. Respondents said
that through the services they received, they found “social lives” and “confidants”; some
gained “a sense of family” and others “perspective.” They “joked” with peers at the expense
of themselves and their doctors. They “bonded” and they “traded ideas” and strategies
about coping and services that worked for them. They felt “normal” and “comfortable” and
“safe” in these relationships. Many simply felt “understood,” as exemplified by the
following quotes from an adult and a transition age youth respondent, respectively:
“I liked the fact I could talk to my friend about my pills and how they made me feel
and that I slept over 10 hours a day and gained over 150 pounds. It’s good to have
someone to talk to about that and have them give you honest feedback, as opposed
to a counselor. That kind of connection is important and helpful.”
“It’s a sense that you know that everyone in the room is in the same boat as you.
Being LGBTQ – under that umbrella – it’s already separate from everyone else, but
when you’re in the trans group, it’s like the minority of the minority… The
transgender community is the minority under the umbrella. It’s nice to go in the
room and there are people who are just like you. Some people might have it better
with their family or more horrible with their family than you did…. They understand
your feelings. I could tell you about [what I’m going through], but you’ve never
experienced it. It’s easier to tell a trans person, and they’ll say, ‘I know exactly how
you’re feeling.’ And they do.”
Another adult respondent summed it up in this manner:
“Knowing that I’m not alone. I felt alone. But taking the Peer to Peer class, I didn’t
feel like I was alone. Being in Connections group, I was able to feel normal. I was
normal. In the real world, you don’t feel normal, you feel like there’s something
wrong with you. But when you are in a peer support group, you feel normal. Other
people have experienced these same feelings and sometimes they have it even worse
than you do. It gives you perspective.”
47
Davidson, L., Borg, M., Marin, I., Topor, A., Mezzina, R., & Sells, D. (2005). Processes of recovery in serious
mental illness: Findings from a multinational study. American Journal of Psychiatric Rehabilitation, (8), 177-
201.
48
Ibid 31, 39, 43.
Interview Results 115
For an older adult respondent, the bottom line was that the connection he made through
services was a vital resource he never had before:
“Ultimately what is important is this conversation and connection. Now I have people
that I can ask for help…I’ve never had that before.”
Hopefulness for the Future
Intrinsic to the nature of all of the themes described above – capturing a positive self-
perception, practicing proactive coping skills, pursuing purposeful activities, and grounding
oneself in meaningful relationships – is a hopefulness and anticipation for a positive and
fulfilling future.
Interview Results 116
SUMMARY OF SURVEY FINDINGS
Peer Support Services
Characteristics of Individuals Who Received Peer Support Services
A total of 338 respondents (or 42.8% of all survey respondents) reported having received
peer support services. These respondents were diverse in terms of numerous characteristics
and demographics. For example, respondents from all age groups reported receiving peer
support services. The largest proportion was adults (55.1%), followed by children/parents
(16.4%), transition age youth (14.8%), and older adults (13.8%). Slightly more females
(55.4%) than males (43.0%) reported receiving peer support services. There was
racial/ethnic diversity among respondents who received peer support services. The largest
proportions of respondents were Caucasian (38.3%) and Latino (34.9%). About 10.0% were
African American and 3.4% were Asian or Pacific Islander. “Other race” respondents,
including Native Americans and Alaskan Natives, represented 12.8% of all respondents who
received services.
A separate 120 respondents (or 28.3% of respondents who did not receive peer support
services) reported that they needed or wanted peer support services but did not get them.
Those who received peer support services were compared to respondents who did not
receive peer support services. Both groups of respondents were similar in characteristics
and demographics except for identification as unserved or underserved. Half (50.0%) of the
respondents who received peer support services belonged to an unserved or underserved
group. In contrast, 30.8% of respondents who did not receive peer support services
belonged to an unserved or underserved group.
Types of Peer Support Services Received
The two most common peer support services reported were one-on-one counseling or
support from a peer or parent/family partner (31.2%) and support group (28.6%).
Respondents who received peer support services received, on average, two types of peer
support services. When one-on-one counseling or support was further examined for
possible associations between respondent characteristics (e.g., age, race/ethnicity, and
gender) and whether or not this service was received, none were found.
Access to Peer Support Services
Of those respondents who received peer support services, only 10.0% reported difficulties
getting peer support services. The two most common difficulties reported were delays in
getting services (26.5%) and having to try several times before actually getting services
(24.5%).
Summary of Survey Findings 117
Respondents who did not receive peer support services despite wanting them reported a
number of reasons for not being able to receive these services. The two most common
reasons reported were that they did not know how to apply for services (24.3%) and stigma
(e.g., fear or embarrassment) (16.5%). Additionally, more respondents who identified as
unserved or underserved (and did not receive peer support services) reported feeling
uncomfortable or unwelcomed, having access challenges in terms of location and time, and
disliking the services than their counterparts who did not identify as unserved or
underserved.
Recovery Orientation of Peer Support Services
Respondents who received peer support services reported significantly more positive
experiences with mental health services in general than respondents who did not receive
peer support services despite wanting them. Respondents over the age of 18 completed the
ROSI scale. In comparison to respondents who did not receive peer support services, those
who did receive peer support services reported, on average, that the mental health services
they experienced were more person-centered, more holistic in meeting other needs such as
housing, more oriented toward employment or school stability and/or advancement, more
focused on basic needs such as income and transportation, had less service inadequacies,
and were less oriented toward mistreatment.
These findings are consistent with respondents’ ratings of their experience with peer
support services. Of those respondents who received peer support services, 76.8% agreed
that services fit their cultural and life experiences, 78.0% agreed that the physical space
where they received services was inviting and dignified, and 76.7% agreed that the types of
peer supports services they received were what they wanted.
Peer Support Services Impact: Employment, Housing, and Personal Recovery and Wellness
There were no associations in employment/school status or living situation between
respondents who received peer support services and those who did not receive these
services despite wanting them. There were, however, significant differences between the
two groups in perceived person recovery and wellness. Respondents over the age of 18 the
RPI scale. In comparison to respondents who did not receive peer support services, those
who did receive peer support services had more favorable results, perceiving or feeling less
anguish and isolation, more connected to others, more confident about life, more
surrounded by people who care, more housing stability, and more hopeful. Moreover, more
positive experiences with peer support services (as well as other mental health services)
were related to a more positive perception of personal recovery and wellness, as evidenced
by the correlations between the RPI and ROSI for both groups. These findings were further
supported by respondents’ ratings of perceived impact of peer support services on different
aspects of their life. Of those respondents who received peer support services, 81.3%
agreed that services helped them feel better, 76.9% agreed that services helped with their
recovery, 71.9% agreed that services helped remove the stigma of receiving mental health
Summary of Survey Findings 118
services, 71.7% agreed that services helped improve their living situation, and 52.7% agreed
that services helped improve their employment situation.
Employment Support Services
Characteristics of Individuals Who Received Employment Support Services
A total of 156 respondents (or 25.0% of all survey respondents) reported having received
employment support services. These respondents were diverse in terms of numerous
characteristics and demographics. For example, respondents from the three age groups (of
respondents over 18) reported receiving employment support services. As expected, the
largest proportion was adults (67.8%), followed by transition age youth (21.0%), and older
adults (11.2%). Slightly more females (52.3%) than males (46.4%) reported receiving
employment support services. There was racial/ethnic diversity among respondents who
received employment support services. The largest proportions of respondents were
Caucasian (37.3%), Latino (24.8%), and African American (20.3%). Close to 4.0% were Asian
or Pacific Islander and almost 14.0% were “other race” respondents, including Native
Americans and Alaskan Natives.
A separate 107 respondents (or 25.8% of respondents who did not receive employment
support services) reported that they needed or wanted employment support services but
did not get them. Those who received employment support services were compared to
respondents who did not receive employment support services. Both groups of respondents
were similar in characteristics and demographics except for the distribution of racial/ethnic
groups. That is, 20.3% of respondents who received employment support services were
African American, whereas only 7.5% of respondents who did not receive employment
support services but needed or wanted them were African American.
Types of Employment Support Services Received
The most common employment support services reported were help preparing a résumé
(18.5%), help preparing for an interview (16.3%), job placement services (14.7%), vocational
training (12.7%), and job coaching or employment counseling (12.3%). Respondents who
received employment support services received, on average, three types of employment
support services.
Access to Employment Support Services
Of those respondents who received employment support services, 21.1% reported
difficulties in getting employment support services. The most common difficulties reported
were being put on a waiting list (17.2%), services not being offered in a convenient location
(17.2%), and services not being offered during a convenient time (17.2%).
Summary of Survey Findings 119
Respondents who did not receive employment support services despite wanting them
reported the following reasons for not receiving employment support services: not knowing
how to apply for services (24.6%) and stigma associated with receiving services that
discouraged pursuit of these services (13.8%). Significantly larger proportions of
respondents residing in the Central and Northern regions reported stigma as the reason for
not receiving employment support services. Also, a significantly larger proportion of
respondents with a serious or extremely serious mental health concern did not know how
to apply for services, and more respondents who identified as unserved or underserved
reported being placed on a waiting list. Finally, a significantly larger proportion of Spanish
speakers (compared to English speakers) did not receive services because services were not
offered in their primary language.
Recovery Orientation of Employment Support Services
Respondents who received employment support services reported significantly more
positive experiences with mental health services in general than respondents who did not
receive employment support services despite wanting them. Respondents over the age of
18 completed the ROSI scales. In comparison to respondents who did not receive
employment support services, those who did receive employment support services
perceived their support experience as more person-centered, more holistic, more oriented
towards employment or school stability and/or advancement, and more focused on basic
needs such as housing and transportation. These respondents also perceived fewer service
inadequacies in their treatment.
In some cases, respondents who received employment support services also received peer
support services. These respondents, compared to those who received only employment
support services, perceived services to be more person-centered, more holistic in meeting
other needs such as housing, and more oriented toward employment or school stability
and/or advancement.
The positive perception of services as recovery oriented is consistent with respondents’
ratings of their experience with peer support services. Of those respondents who received
employment support services, 69.3% agreed that staff had worked hard to achieve their
personal employment goals, 56.7% agreed that services fit with the respondents’ cultural
and life experiences, 74.7% agreed that staff respected respondents’ decisions about
employment goals, 72.2% agreed that the physical space where employment supports were
received was inviting and dignified, 63.3% agreed that staff included people that were
important to respondents in achieving employment goals, and 68.3% agreed that
employment supports received were what the respondent wanted. At 56.7%, overall
agreement about services fitting cultural and life experience was not as strong as ratings in
other areas. A significantly smaller proportion of female respondents compared to male
respondents reported that employment support services fit their unique cultural and life
experiences.
Summary of Survey Findings 120
Employment Support Services Impact: Employment, Housing, and Personal Recovery and
Wellness
There were no associations in employment/school status or living situation between
respondents who received employment support services and those who did not receive
these services despite wanting them. There were, however, significant differences between
the two groups in perceived personal recovery. Respondents over the age of 18 completed
the RPI scales. In comparison to respondents who did not receive employment support
services, those respondents who did receive employment support services had more
favorable results, perceiving or feeling less anguish and isolation, more connected to others,
more confident about life, more surrounded by people who care, and more housing
stability. In addition, more positive experiences with employment support services (as well
as other mental health services) were related to a more positive perception of personal
recovery, as evidenced by the correlations between the RPI and ROSI for both groups.
These findings were further supported by respondents’ ratings of perceived impact of
employment support services on different aspects of their life. Of those respondents who
received employment support services, 64.3% agreed that services helped improve their
living situation, 67.2% agreed that services helped improve their employment situation, and
73.3% agreed that services helped improve their current wellbeing.
Crisis Intervention Services
Characteristics of Individuals Who Received Crisis Services
A total of 231 respondents (or almost 70.0% of 352 respondents who experienced a crisis in
the past year) reported having received crisis services. These respondents were diverse in
terms of numerous characteristics and demographics. For example, respondents from all
age groups reported receiving crisis services. The largest proportion was adults (56.1%),
followed by transition age youth (20.8%), children/parents (15.8%), and older adults (7.2%).
Slightly more females (53.0%) than males (45.7%) reported receiving crisis services. There
was racial/ethnic diversity among respondents who received crisis services. The largest
proportions of respondents were Caucasian (36.8%) and Latino (31.6%). About 15.0% were
African American and 3.5% were Asian or Pacific Islander. “Other race” respondents,
including Native Americans and Alaskan Natives, represented 13.0% of all respondents who
received crisis services.
A separate 104 respondents (or 31.1% of respondents who did not receive crisis services)
reported that they needed or wanted crisis services but did not get them. Those who
received crisis services were compared to respondents who did not receive crisis services.
Both groups of respondents were similar in characteristics and demographics except for
identification as unserved or underserved: 42.0% of respondents who received crisis
services belonged to an unserved or underserved group, whereas 25.0% of respondents
who did not receive crisis services belonged to an unserved or underserved group
Summary of Survey Findings 121
Types of Crisis Intervention Services Received
The two most common crisis services reported were counseling (26.2%) and seeing a
psychiatrist or having medication adjusted (23.0%). Other services included a safety plan to
address the crisis (13.8%) and hotline or warmline supports to talk to someone (11.1%).
Respondents who received crisis services received, on average, two crisis services for the
crisis occurring in the past year.
Although the sample size for respondents in small counties was small, there was a
noteworthy association between county size and types of services received. A larger
proportion of respondents residing in small counties reported receiving hotline or warmline
services (52.2%), as well as safety plans (52.2%), than respondents in larger counties (24.9%
and 32.0%, respectively). In other words, fewer respondents residing in smaller counties
reported receiving the two most common crisis services (i.e., counseling and
psychiatrist/medication) reported by all respondents who received crisis services.
Access to Crisis Intervention Services
Of those respondents who received crisis services, 21.1% reported not receiving services
they wanted after the crisis. The most common services that they wanted were counseling
(22.8%) and medication (15.8%).
Respondents who did not receive crisis services despite wanting them also reported
counseling (33.8%) and medication (23.3%) as the two most common services desired. For
this same group of respondents, the stigma of pursuing services (18.3%) and not being able
to pay for services (15.5%) were the two most common reasons reported for not receiving
needed or wanted crisis services.
Continuity of Care
Of those respondents who received crisis services, 65.0% reported receiving routine mental
health services before and after the crisis. In contrast, of those respondents who did not
receive crisis services, 48.9% reported receiving routine mental health services before and
after the crisis. At the same time, only 8.4% of respondents who received crisis services did
not report receiving routine mental health services before and after the crisis. This
proportion is significantly lower than the 28.9% of respondents who did not receive crisis
services or any routine mental health services before and after the crisis.
Recovery Orientation of Crisis Intervention Services
Respondents who received crisis services reported significantly more positive experiences
with mental health services in general than respondents who did not receive crisis services
despite wanting them. Respondents over the age of 18 completed the ROSI scales. I
comparison to respondents who did not receive crisis services, those who did receive crisis
services perceived their support experience to be more person-centered, more holistic in
Summary of Survey Findings 122
meeting other needs such as housing, less inadequate, and less oriented toward
mistreatment.
In some cases, respondents who received crisis services also received peer support services.
These respondents, compared to those who received only crisis services, perceived services
to be more person-centered and more oriented toward employment or school stability
and/or advancement.
In other cases, respondents who received crisis services also received peer support services
and employment support services. These respondents, compared to those who received
only crisis services, perceived services to be more holistic, more oriented toward
employment or school stability and/or advancement, and less inadequate.
Crisis Intervention Services Impact: Employment, Housing, and Personal
Recovery/Resilience and Wellness
There were no associations in employment/school status or living situation between
respondents who received crisis services and those who did not receives these services
despite wanting them. Also, there was no association between psychiatric hospitalization
and whether or not crisis services were received.
There were, however, significant differences between the two groups in perceived change
both in terms of behavior for children and personal recovery for adults. Children who
received crisis services had more favorable results according to SDQ scores. In comparison
to a small sample of children who did not receive crisis services despite needing them,
children who received crisis services had fewer peer problems and greater prosocial
behaviors.
Adult respondents who received crisis services had more favorable results according to the
RPI scores, perceiving or feeling more connected to others, more confident about life, more
surrounded by people who care, more housing stability, and more hopeful.
Summary of Survey Findings 123
SUMMARY OF INTERVIEW FINDINGS
Characteristics of Interview Respondents
Altogether, 40 interviews were conducted across the state. There were 22 male
respondents and 18 female respondents. Five of the interviews focused on children; 10
interviews were conducted with transition age youth. Thirteen (13) interview respondents
were adults, and 12 were older adults. There was good racial/ethnic representation across
interview respondents, including 13 Caucasians, 10 African Americans, eight Latinos, four
Asians, and five respondents of mixed race/ethnicity (i.e., two respondents who are Native
American/Latino, one Native American/African American, one African American/Caucasian,
and one Filipino/Irish). Eighteen (18) of the interviews were conducted in the Southern
region, 12 in the Bay Area, six in the Central region, and four in the Northern region.
The interview respondents in our sample had serious and often long-standing mental health
concerns. In addition, there was strong representation from traditionally unserved and
underserved groups, including veterans; people with physical disabilities (including
respondents who have visual and hearing impairments); individuals who identified as
LGBTQI; non-English speakers; and people who had experienced homelessness and/or
incarceration. After subtracting children from the interview sample, nearly 80.0% of the
remaining older adult, adult, and transition age youth respondents represented at least one
unserved or underserved group. Almost one-third of those belonged to at least two
unserved or underserved groups.
Services Received
The most notable finding with respect to services received by interview respondents was
the amount of overlap that characterized service receipt. All but one interview respondent
reported receiving more than one type of service. Half of interview respondents received a
combination of four or more mental health services.
Recovery/Resilience Orientation of Services
Overall, interview respondents characterized the recent mental health services they
received as demonstrating recovery/resilience and wellness principles. Five themes
emerged from the data with respect to the recovery/resilience orientation of services. First,
a majority of interview respondents found the respective services they received to be
accessible. Several interview respondents – and in particular, parents who were describing
experiences accessing services on behalf of their children – noted the important role that
self-advocacy had played in helping to access appropriate and useful services. The relatively
small number of respondents who had a complaint about service access mentioned that,
although they had accessed and were receiving services already, they wanted more of the
Summary of Interview Findings 124
same types of service (e.g., more time spent with their counselor or psychiatrist, or more
peer support groups offered).
Second, interview respondents perceived a service system that promoted engagement and
continuous care. On the whole, they described a welcoming environment at program sites,
where they were made to feel “respected” and “safe”. They were further engaged through
encounters with peers in these environments (including those who were employed as staff),
and by professional staff who related to them as individuals as opposed to treating them
like a diagnosis. Several interview respondents were inspired by what they experienced as
services that adopted a philosophy that recovery is possible.
Family involvement emerged as a third theme related to recovery/resilience orientation of
services. Respondents whose interviews focused on crisis services discussed family
involvement most frequently, with parents of children providing many examples of the
importance of Wraparound services and family counseling. Older adults, adults, and
transition age youth who received crisis services were more likely to report that although
programs may promote family involvement, it did not necessarily mean that their families
wanted to be involved. Peer support and employment support respondents offered few
examples of family being involved in those types of services.
Fourth, many respondents said they experienced services that supported their preferences
and goals – in other words, services that respect a person’s right to self-determination. This
theme was particularly strong among respondents receiving employment support services,
meaning that the employment services they encountered offered a good deal of choice and
allowed them to pursue opportunities that were of interest to them.
Last, most interview respondents said they received services that respected their cultural
background. Interview respondents did not talk at length about nor provide many specific
examples related to cultural appropriateness but said that they felt their “differences” were
accepted, not judged, by providers.
Personal Recovery/Resilience and Wellness
Despite continuing to face stressors and other risks to recovery, interview respondents
were generally enthusiastic about the impact that services have had on their daily lives. Five
themes emerged from the interview data that illustrate the mainly positive perspectives
that respondents have regarding their recovery, including the ways they are building and
living purposeful lives.
First, on the whole, interview respondents held a feeling of hopefulness for the future.
Next, through supportive peers and dedicated caseworkers, counselors, and job trainers
who reminded them of their strengths and encouraged them to pursue their goals,
respondents said they have generated or regenerated a positive self-perception and sense
of self-worth. Along with it, many reported that their confidence and self-determination to
advocate for themselves had increased.
Summary of Interview Findings 125
Third, most interview respondents explained that the mental health services and supports
they received have inspired them to learn more about their mental health concerns and
diagnoses and to develop proactive strategies to manage and/or cope with them. These
respondents explained how they have purposefully paced themselves, especially in the
context of their employment or volunteer activities. They have practiced self-awareness and
have actively avoided stressors, such as environments, situations, and people they may not
be able to manage. Additionally, they have utilized the plethora of services and supports
accessible to them and have not been shy about asking for help.
In addition, most respondents reported spending their time and energies pursuing
meaningful activities such as spirituality, vocational interests, and “giving back” to their
peers and communities. Oftentimes, this type of pursuit was one of the levers that
stimulated or reinforced both an increase in positive self-perception and the sense of
relationship and connection that help to undergird recovery. Finally, with respect to this last
theme, connecting and forming relationships, most interview respondents found the mental
health services they received to be instrumental. These individual relationships and sense of
community have fostered a sense of belonging and comfort that one is not alone.
Summary of Interview Findings 126
INTEGRATED SURVEY AND INTERVIEW FINDINGS AND DISCUSSION
Sample Representation and Generalizing Study Findings
Responses to the survey were stronger than expected, with a total of 949 completed
surveys, exceeding the study goal of 750. The survey sample as a whole represents the
population that the study intended to target – that is, clients who have had experience with
a wide array of public mental health services. Because of the study’s focus on the three
service areas of peer support, employment support, and crisis intervention, the primary
target included clients who either received one or more of these services, or did not receive
one or more of these services despite wanting them. Although the sample sizes for
employment support and crisis services were not as large as the sample size for peer
support, the sizeable number of survey respondents represented in each analysis of the
three service areas is reflective of the study’s success with outreach and recruitment. In
addition to the diversity of respondents in terms of various demographics and
characteristics, there was strong representation of traditionally unserved and underserved
individuals. This strong representation was desired at the outset – both because the target
population is an MHSA focus, and because, through the participatory planning process,
stakeholders emphasized the importance of representation from traditionally unserved and
underserved groups. The study achieved this with the significant assistance of the PEPs
(who had local and statewide reach), as well as many counties and agencies/organizations
that participated in the study.
The study successfully recruited a purposeful sample of 40 interview respondents. This was
the target number for interviews made possible through the assistance of PEPs and by
making targeted requests to county mental health departments and community-based
providers when necessary. Purposeful samples are not meant to be representative samples.
Rather, they are intended to elicit information from respondents with particular set of
characteristics to illuminate specific questions of interest to an evaluation. The
demographic and other characteristics of interview respondents closely match the
population of unserved and underserved groups of individuals that MHSA is trying to reach.
Thirty-eight (38) of the 58 counties in California were represented in the survey. Although
not all counties across the state participated in the study, there was, overall, representative
participation from small and large counties across all regions of the state. The study findings
are generalizable to the state based on comparisons of survey respondents to mental health
clients across the state in terms of age, race/ethnicity, and gender.49 More importantly, the
study respondents represent the populations targeted by GSD and the larger MHSA funding.
49
The data on these demographics come from the 2007-08 fiscal year report that contains CSI data provided
to the state as of June 2010. This is the latest report published on the California Department of Mental Health
Department website: www.dmh.ca.gov.
Integrated Survey and Interview Findings and Discussion 127
Consumer Experiences with Services
A system oriented toward recovery/resilience must be accessible. In addition, part of what
it means to have a recovery oriented service system is to provide services that support the
individual’s goals and efforts to lead full and productive lives in the broader community.
This involves facilitating access to a variety of supportive services that help make recovery
sustainable.
Overall, the study findings suggest that services across the three service areas are
accessible. In particular, peer support services appear to be readily accessible to a broad
base of individuals. However, there are access issues that remain to be addressed,
especially for certain populations of individuals who have traditionally been underserved
(e.g., individuals with physical disabilities and individuals who are homeless). In addition,
study findings confirm that access to a variety of supportive services is being achieved, and
recipients perceive services as appropriately individualized, encouraging, and respectful of
their wishes and goals. While services are perceived largely as respecting the individual’s
cultural background, there is room for improvement in this area. Altogether, these
elements of recovery/resilience oriented services converge to promote continuity of care
that has important implications for personal recovery.
Access and Barriers to Access
Overall, both survey and interview respondents reported high levels of access to services
across the three study areas. The majority of survey respondents who received peer
support, employment support, or crisis services reported no difficulties with accessing these
services. The proportions of respondents who did report difficulties were relatively small:
10.0% for peer support services, 21.1% for employment support services, and 21.1% for
crisis services. Comparing across the three service areas, there appears to be greater access
to peer support services than employment support or crisis services. This survey finding is
consistent with sentiments expressed by interview participants that peer support services
are typically readily available and accessible, certainly in comparison with professional
services like counseling and case management where staff caseloads are perceived has high.
In addition, given that the most common peer support services (i.e., one-on-one counseling
and support from a peer or parent/family partner) were not associated with any particular
respondent characteristics, peer support services in general appear to be received and
utilized indiscriminately.
For those survey respondents who did not receive peer support, employment support, or
crisis services despite wanting them, the most common reasons for not receiving them were
similar across the groups. For respondents who did not receive peer support or
employment support services, the two most common reasons were: not knowing how to
apply for services and stigma (e.g., fear or embarrassment). Stigma was the most common
barrier, followed by not being able to pay for services, for respondents who did not receive
crisis services but wanted them. Therefore, stigma of mental health services and lack of
information or knowledge about services, in particular, are key barriers to accessing these
Integrated Survey and Interview Findings and Discussion 128
services. These are common barriers that have been identified in other studies on personal
recovery.50
Furthermore, for those survey respondents who did not receive peer support services,
respondents who belonged to an unserved or underserved group were more likely to report
feeling uncomfortable or unwelcomed, having access challenges in terms of location and
time, and disliking the services. Most of these respondents identified themselves as
physically disabled and/or homeless, suggesting that respondents facing these challenges
especially experienced these barriers to accessing peer support services. This finding, as it
particularly relates to respondents with physical disabilities, is consistent with other study
findings that document the barriers to accessing health care, including mental health
services, among individuals with disabilities.51
Overlapping Services
The phenomenon of receiving overlapping services (i.e., receiving multiple types of services
within and across service areas) was evident from the study findings and particularly
pronounced in the interviews. From a recovery/resilience perspective, there were various
reasons why overlapping services are important. At a very basic level, receiving multiple
types of services indicated that services are generally available and that they were being
accessed. Second, the receipt of overlapping services suggested that clients are being
provided with care that is individualized and tailored to their needs and personal goals. This
is a critical component of recovery oriented services, which should go beyond simply
supporting a client’s clinical recovery with, for example, medication to support the pursuit
of opportunities for leading a full and productive life in the community.52 Third, clients who
received a number of different services were – or had the opportunity to be – engaged in
their mental health recovery on a more active and/or deeper level. For instance, interview
respondents stressed the importance of utilizing a full range of services and supports as part
of a proactive strategy to cope with and manage their mental health. Furthermore, the
more services an individual received, the greater the opportunities became for developing
important relationships and connections, pursuing meaningful activities, and reaping the
associated benefits of improved self-perception and increased self-determination. The
result was a stronger safety net and a wider circle of support. In fact, this positively
reinforcing cycle of events was evidenced in the stories of hope and recovery shared by
many of our interview respondents. This was also evident in the survey findings that ratings
of both services (ROSI) and personal recovery and wellness (RPI) were significantly more
positive when respondents received overlapping services.
50
Ibid 40.
51
National Council on Disability (2009). The Current State of Health Care for People with Disabilities. National
Council on Disability: Washington, DC.
52
Ibid 31.
Integrated Survey and Interview Findings and Discussion 129
Appropriateness of Services
Recovery oriented services are meant to offer appropriate care on a variety of levels.
Several of the main themes that emerged from the interview data, as well as survey findings
from peer support and employment support services, converge to support an overall finding
that mental health services received by study participants were appropriate on many fronts.
Many interview respondents described feeling welcomed and comfortable in the locations
where they received services, while approximately three-fourths of survey respondents who
received peer support and employment support services agreed that the physical space
where they received services was inviting and dignified. Further, most interview
respondents attested to the fact that services they received adopted a philosophy that
recovery is possible, provided individualized care, and supported their right to self-
determination. At the same time, almost three-quarters of survey respondents receiving
employment support services felt that the employment supports they received were what
they wanted, agreed that staff had respected their personal decisions about employment
goals, and believed staff worked hard to help them achieve those goals. More than three-
fourths of peer support recipients agreed that the types of peer supports they received
were what they wanted.
Most interview respondents reported that the services they received respected their
cultural background. Likewise, more than three-quarters of survey respondents who
received peer support services agreed that services fit their cultural and life experiences.
For survey respondents who received employment support services, agreement that
services fit their cultural and life experiences was not as high, with less than two-thirds
agreement. In addition, a smaller proportion of female respondents compared to male
respondents reported that employment support services fit their unique cultural and life
experiences.
There is room for improvement in providing services that are appropriate in terms of what
individuals want and need. Also, there is room for improvement in providing services that
are sensitive and aware of culture, gender, and life experiences. However, overall, services
were perceived as appropriate on many different levels by study respondents. The
experience of being engaged in services that encourage recovery/resilience while respecting
one’s individuality and right to self-determination promoted participation in continued care.
Ultimately, with respect to the three service areas studied, participation in care not only
contributed to a more positive service experience, but it also promoted personal
recovery/resilience and wellness.
Continuity of Care
As part of the discussions above on overlapping services and appropriateness of services,
we discussed the phenomenon of how once a client is in the system with services in place
that are individualized, encouraging, and respectful, they are building a circle of support and
a safety net. As discovered in the interviews, many respondents detailed being involved in
multiple services, often times for several years. Although they had crises during these times,
Integrated Survey and Interview Findings and Discussion 130
they were able to utilize the services and supports they had in place and to do so more
immediately, helping to avert the need for hospitalization or a change in housing situation,
for example.
This phenomenon was also evident in the survey findings on continuity of care, which was
analyzed for crisis services only. Respondents who received crisis services were more likely
to have routine mental health services before and after the crisis compared to those who
did not receive crisis services. When routine mental health services were not in place during
a crisis, follow up services were less likely to be received, potentially placing greater risk for
a more serious crisis in the future. Altogether, these findings indicate that having the safety
net in place helps to ensure continuity of care that may ultimately have implications for
client outcomes. In fact, survey respondents who received crisis services and had greater
continuity of care, on average, had a more positive service experience, as well as a more
positive sense of personal recovery and wellness.
Recovery Orientation of Services
Respondents who received peer support, employment support, and crisis intervention
services reported significantly more positive experiences with mental health services in
general than respondents who did not receive these services despite wanting them.
Respondents who received services rated their experiences of mental health services as
being more person-centered, more holistic in meeting other needs such as housing, more
oriented toward employment or school stability and/or advancement, more focused on
basic needs such as income and transportation, having less service inadequacies, and/or
being less oriented toward mistreatment.
Service Impact
Employment and Housing
Based on reports of current and desired employment and housing situations, as well as
reports of changes in employment and housing, there were no significant differences
between respondents who received services (peer support, employment support, or crisis
services) and those who did not in terms of employment and housing. However, based on
respondent ratings of experiences with peer support and employment support services,
close to three-quarters of respondents who received these services agreed that the services
had a positive impact on their housing situation. In addition, close to three-quarters of
respondent who received employment support services agreed that the services helped
improve their employment situation. Just over half of respondents who received peer
support services agreed that the services helped improve their employment situation.
These findings were supported by examples from interview respondents who reported
improvement in and/or satisfaction with their housing and/or school or employment
situations and credited the mental health supports they received as helping to enable these
positive changes.
Integrated Survey and Interview Findings and Discussion 131
Personal Recovery/Resilience and Wellness
There were significant differences between respondents who received services and those
who did not in perceived change both in terms of behavior for children and personal
recovery for adults. Children who received crisis services had more favorable results
according to SDQ scores. In comparison to a small sample of children who did not receive
crisis services despite needing them, children who received crisis services had fewer peer
problems and greater prosocial behaviors.
Respondents over the age of 18 who received peer support, employment support, or crisis
services reported significantly more positive perceptions of personal recovery than
respondents who did not receive these services despite wanting them. Respondents who
received services perceived less anguish, felt more connected to others, were more
confident about life, felt more surrounded by people who care, perceived greater housing
stability, and/or were more hopeful compared to those who did not receive services.
However, there were no significant differences in psychiatric hospitalization between
respondents who received crisis services and those who did not despite wanting or needing
them.
From the qualitative interviews emerged numerous stories of personal recovery/resilience,
which respondents credited, at least in part, to the recent mental health services they
received. Five recovery themes emerged that encompass the perceptions respondents have
about how and what they do to live full and meaningful lives. Despite daily stressors and
other barriers to recovery reported by respondents, they felt hopeful for the future. In
addition, they held a positive view of themselves, which is related to growing confidence
and self-determination. They were proactively managing their mental health concerns in a
variety of ways, including utilizing the mental health services at their disposal and activating
a strong safety net of supports. Moreover, most of the respondents we interviewed
devoted time and energy pursuing meaningful activities such as spirituality, vocational
interests, and “giving back” to help others. Last, they were developing and reinforcing
positive relationships and connections instrumental to recovery.
These themes were allowed to emerge directly from the data. That is, they were not
imposed on the data based on a framework of recovery from the literature. Nonetheless,
after the themes had been developed from the data, a look to the literature confirmed a
strong convergence of our emergent themes with common elements of recovery reported
there.53 That our evaluation interviews resulted in trustworthy themes lends an added
measure of confidence to the finding that respondents were doing well in their personal
recovery/resilience journeys.
Outcome Findings and Implications for Measurement
Receiving peer support services was not associated with concrete changes in employment
or housing, but it was strongly associated with intrinsic changes that promote personal
53
For example: Ibid 39, 40, 41, 44, 47.
Integrated Survey and Interview Findings and Discussion 132
recovery and wellness. This finding is consistent with the nature of peer support services,
which are intended to provide more intrinsic support (such as surrounding clients with
people who have similar experiences and people who care) than concrete support (such as
direct employment support). The finding also is consistent with respondent ratings on their
experience with peer support services received: at 52.7%, there was only moderate
agreement that the peer support services that they received improved their employment
situation. Therefore, measuring personal recovery (in addition to concrete changes) as an
outcome of peer support services is appropriate and meaningful.
These findings for peer support services were echoed in the findings on employment
support services. While these services are typically designed to promote concrete changes
in skills and employment, they do promote intrinsic changes, as evidenced by the more
positive perceptions of personal recovery among respondents who received employment
support services. In fact, respondent ratings on their experience with employment support
services indicated that there was stronger agreement that employment support services
improved their current wellbeing more so than their living situation or employment
situation. Furthermore, all the interview respondents reported either working and/or
volunteering, and they credited the mental health supports they received as helping to
enable this. However, self-pacing was an important concept in their lives with respect to
employment, suggesting that immediate change toward fulltime employment, for example,
was neither achieved nor, more importantly, desired. As part of the recovery process, self-
pacing is an important component of proactively managing and coping with stressors and
other mental health concerns.54 This has implications for how employment outcomes are
measured. Although concrete changes are an important part of measuring the impact of
employment services, especially if the services are focused on job placement, they alone do
not capture the intrinsic changes that are possible. If finding and keeping a job are short-
term and long-term goals, the effects of personal recovery – whether it is hope, confidence,
or a sense of purpose – are critical to measure as recovery assets that help individuals to
achieve those goals.
The findings for crisis services were similar to the findings for peer support and employment
support services. There were no differences between respondents who received crisis
services and those who did not in employment, housing, or psychiatric hospitalization.
However, there were differences in personal recovery (for adults) and resilience (for
children), suggesting that crisis services address the process of recovery/resilience and
wellness, including how a person feels about oneself and one’s life. Like peer support
services, crisis services are not necessarily intended to change employment or housing
situations directly, but they are intended to promote recovery/resilience and wellness to
effectively address crisis events. This has implications for how the outcomes of crisis
services are measured. The concept of personal recovery/resilience is an important
measure as an intermediary outcome that potentially facilitates long-term stability.
Whether or not an individual is hospitalized for a psychiatric crisis is one measure of the
effectiveness of crisis services; however, it alone does not do justice to the process that
individuals may undergo as part of receiving support through counseling, for example. In
54
Ibid 39, 40, 41.
Integrated Survey and Interview Findings and Discussion 133
this study, respondents who received crisis services did not have significantly lower rates of
psychiatric hospitalization. Yet they had a significantly more positive perception of personal
recovery and wellness. Though these findings may seem contrary, they reflect the reality
that the relationship between personal recovery and psychiatric hospitalization is not
necessarily linear because recovery is not by nature linear.55
Value Added Services
Survey respondents who received employment support services plus peer support services
had a more positive perception of services as recovery oriented compared to those who
only received employment support services. Moreover, survey respondents who received
crisis services plus peer support services also had a more positive perception of services as
recovery oriented compared to those who received only crisis services. This finding speaks
not only to the benefits of overlapping services, it also indicates that there is added value
specifically from receiving peer support services in relation to a more positive perception of
services as recovery oriented, which is associated with a more positive perception of
personal recovery/resilience and wellness.
These relationships between receiving peer support services and having a more positive
perception of service experiences and personal recovery/resilience are reflective of the
characteristics inherent in peer support services and consistent with other research that
suggests participation in peer support is related to ratings of recovery orientation and can
enhance wellness.56 Interview findings illustrated, for instance, that peer-led or peer-
focused services were highly likely to operate under a philosophy that recovery is possible.
They provided recipients with an improved sense of connectedness, belonging, and
understanding. As well, peer support services provided meaningful access to peer mentors
who offered inspirational examples of recovery and hope, and these services fostered
relationships with peers who offered daily support such as exchanging strategies for coping
and referring one another to helpful services. Interviews were ripe with examples of peer
support recipients who were inspired to “give back” – which essentially means becoming
part of and perpetuating the system of care.
Our analysis of peer support services appears especially meaningful in the contexts of both
promoting and sustaining MHSA services. Taking into consideration the survey and
interview findings for this study, peer support services, as part of the larger continuum of
care, appear to have great potential in impacting individuals with lived experience. In many
respects, peer support services present a win-win situation for clients and funders, as they
appear to be a solid investment for both. Peer support services involve clients; they do not
rely primarily on professionals; they are readily accessible; and they are less expensive.
Simultaneously, they are powerful at promoting personal recovery/resilience, as they
continually reinforce positive outcomes through improving perceptions of one’s own
confidence, wellness, and hope.
55
Ibid 39, 40, 41.
56
Resnick, S.G., & Rosenheck, R.A. (2008). Integrating peer-provided services: A quasi-experimental study of
recovery orientation, confidence, and empowerment. Psychiatric Services, (59)11, 1307-1314.
Integrated Survey and Interview Findings and Discussion 134
Study Limitations and Strengths
Limitations to the Study
For the current study, its statewide reach and focus on three different service areas and all
four MHSA age groups were factors that limited the specificity of what could be said about
each service area and each target age group. Taking into consideration the diversity of
counties, including the broad array of services they provide, the study findings could neither
identify nor assess specific services or programs within these service areas. Rather, the
study refers to the three service areas more generically in terms of the typical services they
offer. This is in line with the intent of the study to examine the overall impact of these
MHSA services rather than to examine, for example, whether particular components or
service traits within each service area are more effective than others. Additionally, the
different experiences across the life span were not thoroughly captured, in particular for
children whose developmental experiences are unique from adults over the age of 18.
The study design was largely informed by a participatory planning process that identified
interviews and a survey as the preferred ways to collect data. This mixed methods approach
is a strength of the study (discussed below). However, as with any study design, there are
inherent limitations to employing interviews and surveys. Interviews are naturally limited in
that they are time and resource intensive and are therefore typically conducted with small
purposeful samples that are not meant to be generalized but rather used to understand
specific information about context and experience that can illuminate certain evaluation
questions in a way that survey methods cannot. While interview respondents well
represented the population of interest for the evaluation (i.e., they were members of
traditionally unserved and underserved populations), those who were identified to
participate in an interview were likely deemed “well” enough by the providers who serve
them to give a productive interview. In that sense, the information shared by some
respondents in our sample may have been less likely to lean toward a negative view of
services and more likely to be colored with positive examples. Nonetheless, interview
responses provided valuable data about the impact of MHSA services, including information
about personal recovery/resilience and wellness that converged with and enhanced survey
findings on that outcome indicator.
Additionally, an inherent survey limitation is that it is restricted in what it can measure and
when. Because of the retrospective nature of the survey, the survey timeframe was set at
one year. Information reported by respondents was hence time limited, although it is likely
that respondents also reported on their experiences beyond one year’s time. Therefore, the
exact timeframe of services received is unknown.
There were other measurement limitations. First, the survey was limited in the way
recovery orientation of services was measured. The service experience of children under
the age of 18 was not measured, because the ROSI was not intended for children. The ROSI
was completed by survey respondents regardless of whether or not they received any of the
three types of services. Therefore, there was a built-in comparison group for analysis;
however, because respondents completed the ROSI by rating their overall service
Integrated Survey and Interview Findings and Discussion 135
experience (as opposed to rating one specific service area such as employment support
services, for example), the comparative sets of services were not constant across all
respondents.
Although the survey sample size was large and the diversity of respondents covering the
three service areas in one study was highly beneficial from a study design and cost
perspective, it ultimately meant smaller sample sizes and less specificity in terms of what
could be said about each particular service area. Survey respondents were first categorized
by service area and then further divided according to whether they received the service or
wanted the service but did not receive it. Consequently, subgroup sample sizes were at
times too small to confidently analyze and report findings. In addition, it was not possible to
analyze racial/ethnic groups in a way that captured the tremendous diversity of survey
respondents. This is a typical challenge in quantitative data analysis.
Study Strengths
This study, with its statewide reach and its focus on three service areas and seven outcome
indicators, was ambitious given its design, timeframe, and resources. However, much was
accomplished in a time- and cost-effective manner. There was good participation from
counties across the state, and survey respondents were generally representative of mental
health recipients across the state. Further, there was strong representation of unserved and
underserved groups, which matched the intention of the study and is congruent with the
target population of the MHSA.
The participatory nature of this evaluation, including the personal dedication of several of
the evaluation partners, was one of its greatest strengths. From beginning to end, the UCLA
evaluation team worked collaboratively with the evaluation partners to meet tight
timelines. At all points throughout the study process, from survey development to report
writing, input from the evaluation partners enriched the study. The evaluation partners
were instrumental in survey development as well as recruitment for survey and interview
respondents. In particular, PEPs were highly involved in the selection of standardized
measures for the survey. Their recommendation to use the RPI, which measures personal
recovery versus wellbeing or functioning, benefited the study by capturing personal
recovery in ways that are consistent with the principles and values of the MHSA.
Participatory evaluations are generally thought to require a lengthier time period for
development and implementation than conventional evaluations. This is due to the nature
of the time it takes to facilitate the meaningful engagement and active involvement of a
group of PEPs who may or may not be familiar with evaluation research and therefore need
some skill development and training to maximize their input. More importantly, however, a
lengthier timeframe is necessary because evaluation partners are limited in the time they
have to devote to the evaluation project. Thus, the timeframe for a study of this nature,
scope, and scale was exceptionally short.
Integrated Survey and Interview Findings and Discussion 136
The mixed methods approach was an additional study strength. Because there are specific
weaknesses and strengths inherent in any one data collection method, blending interview
and survey methods helped to compensate for single-method deficiencies and increased
the rigor of our evaluation design.
Although the study used a survey, the survey was designed to capture built-in comparison
groups – that is, respondents who received services in a given service area, and respondents
who did not receive services in a given service area despite wanting them. Respondents
who received services in each service area were found to be similar to respondents for each
service area who did not receive the service despite wanting them. Thus, the ability to
analyze each of the three service areas using unbiased comparison groups was a study
strength and lends greater confidence in the study findings.
The survey was lengthy, but the length was necessary given its intent to capture
information on three different service areas and from those who received services as well
as those who did not receive services despite wanting them. It had multiple outcome
measures and different types of measures, including standardized instruments. Although
there was the potential for respondent burnout and missing data, the tradeoff was that the
survey allowed a tremendous amount of information to be gathered and analyzed. The
resulting findings provided substantive information that has implications for MHSA policy
decisions.
Integrated Survey and Interview Findings and Discussion 137
APPENDIX A – Sample Description of Survey Respondents
Survey Response
The Statewide Survey of Client Experience (SSCE) was distributed throughout the State of
California in both an online format and a paper-and-pencil format. As shown in Figure 52 a
total of 946 surveys were returned, exceeding the study goal of 750. A total of 661 surveys
(or 69.9%) were submitted in hard copy and 285 (or 30.1%) were submitted online.
Figure 52 – Survey Type
Online Survey
285 (30.1%)
Paper Survey
661 (69.9%)
N = 946
Appendix A: Sample Description of Survey Respondents 138
As shown in Figure 53, the majority of surveys were completed in English, representing
87.0% of surveys in both online and paper-and-pencil formats. The remaining surveys were
completed in Spanish, representing 12.0% of all surveys.57
Figure 53 – Survey Type by Language
Spanish Online
19 (2.0%)
English Online
266 (28.0%)
English Paper
560 (59.0%)
Spanish Paper
99 (10.0%)
Paper survey: n = 661
Online survey: n = 285
N = 946
57
The paper-and-pencil version of the survey was also available in traditional Chinese. Although researchers
reached out to Asian specific mental health providers in the Bay Area and Southern California to assist with
recruitment of monolingual Chinese speaking consumers and family members, only two surveys were
completed in traditional Chinese.
Appendix A: Sample Description of Survey Respondents 139
The majority of respondents identified themselves as clients (61.9%). As presented in Figure
54, family members constituted nearly one-third of all returned surveys (31.0%), and client
representatives accounted for 7.1% of the respondent population. Most children
represented in the survey had the survey completed on their behalf by a family member.
Only four children had the survey completed on their behalf by a non-family representative.
A family member or representative completed the survey on behalf of one-quarter of
transition age youth, one-third of adults, and slightly less than one-quarter of older adults.58
Figure 54 – Person Completing Survey
Family Member
272 (31.0%)
Client
543 (61.9%)
Client
Representative
62 (7.1%)
N = 877
58
Given the representation of blind or visually impaired individuals in the survey, we surmise that family
members or non-family members assisted these individuals with the completion of the survey.
Appendix A: Sample Description of Survey Respondents 140
Respondent Demographics
As presented in Figure 55, over half of the respondents (53.3%) were adults. The remaining
respondents were transition age youth (TAY) (18.7%), parents of child clients (15.4%), and
older adults (12.6%).59
Figure 55 – Age Groups
Older Adult
Adult (60+ years)
(26-59 years) 109 (12.6%)
460 (53.3%)
Child
(17 years or
TAY younger)
(18-25 years) 133 (15.4%)
161 (18.7%)
N = 863
As presented in Figure 56, 53.4% of respondents were female and 45.3% were male. One
percent of respondents were transgendered, and less than one percent identified as gender
fluid.
Figure 56 – Gender
Trans- Gender
gendered fluid
8 (1.0%) 3 (0.3%)
Male
389
(45.3%)
Female
458
(53.4%)
N = 858
59
In MHSA, 16 and 17 year olds are considered transition age youth. For this study, however, the age group
for transition age youth was defined as 18-25 year olds because children were defined as younger than 18
years of age. This definition was necessary to be consistent with the study’s Institutional Review Board
restriction to include minors by requiring that adults 18 years or older complete the survey for their children.
Appendix A: Sample Description of Survey Respondents 141
Information on gender by age group is presented in Table 32. In the case of children and
transition age youth, the majority of respondents were male (60.9% and 57.5%,
respectively), while the majority of adult and older adult respondents were female (59.1%
and 67.9%, respectively).
Table 32 – Age Groups by Gender
Child TAY Adult Older Adult
Female 50 (37.6%) 66 (40.7%) 270 (59.1%) 72 (67.9%)
Male 81 (60.9%) 92 (57.8%) 182 (39.8%) 34 (32.1%)
Transgendered 1 (0.75%) 2 (.01%) 5 (1.1%) 0 (0%)
Gender fluid 1 (0.75%) 2 (.01%) 0 (0%) 0 (0%)
TOTAL n=133 n=162 n=457 n=106
The distribution of respondents by racial/ethnic group is presented in Figure 57. The largest
number of respondents identified themselves as Caucasian at 40.9%, followed by Latino
(36.0%), African American (11.9%), Native American or Alaskan Native (4.7%), Asian (4.0%),
Pacific Islander (1.3%), and “other race” (1.2%). Additional descriptions provided by
respondents are listed in a footnote.60
Figure 57 – Race/Ethnicity
Native
American or
Alaskan
Latino
Native
365 (36.0%)
48 (4.7%) Pacific
Islander
13 (1.3%)
Other Race*
Caucasian
12 (1.2%)
415 (40.9%)
Asian
African
41 (4.0%)
American
121 (11.9%)
Total Responses = 1,015
N = 917
* Includes 78 who identified multiple races
60
Optional descriptions include Afghani, Belize, Chilean, Chinese, Colombian, Creole, Danish, Egyptian, El
Salvadorian, English, Filipino, French, German, Guatemalan, Haitian, Hawaiian, Hmong, Icelandic, Iranian, Irish,
Italian, Japanese, Korean, Laotian, Mexican, Native American tribes (Apache, Blackfoot, Cherokee, Choctaw,
Navajo, Tarascan Indian, and others), Nicaraguan, Pakistani, Portuguese, Puerto Rican, Russian, Spanish,
Swedish, Taiwanese, and Vietnamese.
Appendix A: Sample Description of Survey Respondents 142
As presented in Figure 58, the majority of respondents reported residing in Southern
California (54.0%), followed by the Central region (18.7%), Northern region (13.6%), and Bay
Area (10.3%).
Figure 58 – Regional Counties
Central
177 (18.7%)
Bay Area
97 (10.3%)
Southern
511 (54.0%) Northern
129 (13.6%)
N = 914
Surveys were received from individuals residing in both urban and rural counties, with 4.3%
being received from small counties with a population of less than 200,000, as shown in
Figure 59.61
Figure 59 – County Size
Small county
57 (6.2%)
Larger county
857 (93.8%)
N = 914
Surveys were received from individuals who reside throughout the state, with
representation from 36 of California’s 58 counties, as presented in Figure 60. The counties
with the largest representation are in the Southern county of Los Angeles, the Central
61
Only a formal definition of a small county is used statewide; therefore, analyses by county size were
conducted separating small counties from larger counties.
Appendix A: Sample Description of Survey Respondents 143
county of Tulare, and the Northern county of Butte.62 Altogether, the respondents
represent a wide distribution of counties throughout the state. In counties such as Contra
Costa, Colusa, El Dorado, Marin, Napa, San Mateo, and San Bernardino, county
representation within the respondent sample is similar to their proportional representation
of California’s population.63
Figure 60 – Respondent County of Residence
0% 5% 10% 15% 20% 25% 30% 35% 40%
Alameda 1.2%
Amador 0.2%
Butte 10.4%
Calaveras 1.8%
Colusa 0.1%
Contra Costa 3.2%
El Dorado 0.2%
Fresno 1.1%
Humboldt 0.2%
Kern 0.4%
Lassen 0.1%
Los Angeles 32.6%
Marin 0.2%
Napa 0.1%
Nevada 2.2%
Orange 3.9%
Placer 0.8%
Riverside 2.8%
Sacramento 2.1%
San Bernardino 4.9%
San Diego 2.8%
San Francisco 0.2%
San Joaquin 0.3%
San Luis Obispo 0.7%
San Mateo 1.5%
Santa Barbara 3.6%
Santa Clara 1.2%
Santa Cruz 0.3%
Shasta 1.1%
Solano 0.2%
Sonoma 2.4%
Stanislaus 0.2%
Sutter 0.1%
Tulare 12.5%
Ventura 4.0%
Yolo 0.1%
N = 19
62
Compared to California’s overall population, there is overrepresentation of these counties in comparison to
the proportion of California residents living within them. U.S. Census Bureau:
http://quickfacts.census.gov/qfd/states/06/06111.html
63
San Francisco County did not participate in the survey due to a number of competing countywide surveys
administered at the same time. The two respondents from this county reported in the table participated in the
survey as part of their participation in the interviews.
Appendix A: Sample Description of Survey Respondents 144
Survey-takers were asked if they identified as a member of an unserved or underserved
group(s).64 As detailed in Figure 61, fifty-two percent (52.0%) of respondents indicated that
they were a member of at least one unserved or underserved group, with some
respondents indicating that they were affiliated with more than one group. The largest
group comprised of respondents who are physically disabled (23.5%).65
Figure 61 – Member of Unserved or Underserved Group(s)
Parolee
24 (4.3%)
Veteran
30 (5.3%)
Physically
disabled
132 (23.5%) Blind or visually
impaired
36 (6.4%)
Other, including
ethnic, cultural, Deaf or hard of
or religious group hearing
129 (23.0%) 37 (6.6%)
Homeless
LGBTQI
118 (21.0%)
55 (9.8%)
Total Responses = 561
N = 489
64
This group membership was self-identified by respondents. While many ethnic groups are considered
unserved or underserved, respondents who were racial/ethnic minorities were not automatically put in this
group.
65
A large number of responses fell under the “other” category, including ethnic, cultural, or religious groups.
Respondents who provided further information under this category listed adopted and foster children, single
parents, and low income, probation, senior citizen, and individuals on probation; religious groups including
Baha’i, El Shaddai, Mormon, Muslim, Jehovah’s Witness, Jewish, Muslim, Wiccan, and atheist; and
ethnic/cultural groups including Hmong and Native American.
Appendix A: Sample Description of Survey Respondents 145
Information for transition age youth, adult, and older adult respondents’ education is
provided in Figure 62. A relatively large proportion of respondents (20.7%) did not
complete high school. About 30% obtained a General Education Development (GED)
certificate or completed high school, 4.2% completed technical or vocational training, 22.6%
attended some college, 15.1% completed college, and 5.5% obtained a graduate degree.
Figure 62 – Highest Level of Education
Completed high Completed
school technical/voca-
171 (23.4%) tional training
31 (4.2%)
Earned a graduate
Attended college degree
but did not earn a 40 (5.5%)
college degree
Completed GED
165 (22.6%)
Certificate
46 (6.3%)
Earned a college
degree
110 (15.1%)
Did not complete N = 714
high school
151 (20.7%)
Appendix A: Sample Description of Survey Respondents 146
As can be seen in Table 33, a sizeable number of transition age youth respondents are likely
still in high school, with relatively few completing any post-secondary school education. A
smaller proportion of older adults, as compared to adults, reported obtaining a General
Education Development (GED) certificate or completing high school as their highest level of
education (18.3% compared to 30.2%, respectively). Conversely, a larger proportion of older
adults, as compared to adults, reported having completed college and obtaining a graduate
degree (38.5% compared to 23.3%, respectively).
Table 33 – Highest Level of Education by Age Group
TAY Adult Older Adult
54 (33.8%) 85 (18.9%) 12 (11.5%)
Did not complete high school
Completed GED (General Education 5 (3.1%) 40 (8.9%) 1 (1.0%)
Development) Certificate
57 (35.6%) 96 (21.3%) 18 (17.3%)
Completed high school
5 (3.1%) 21 (4.7%) 5 (4.8%)
Completed technical/vocational training
Attended college but did not earn a 34 (21.2%) 103 (22.9%) 28 (26.9%)
college degree
5 (3.1%) 78 (17.3%) 27 (26.0%)
Earned a college degree
0 (0%) 27 (6.0%) 13 (12.5%)
Earned a graduate degree
TOTAL n = 160 n = 450 n = 104
Appendix A: Sample Description of Survey Respondents 147
Information on income, as presented in Figure 63, shows that many individuals (excluding
respondents under the age of 18) have very limited financial resources, with nearly three-
quarters (74.3%) of respondents reporting either no income or an annual income of less
than $15,000.
Figure 63 – Individual Annual Income
$15,000 - $30,000
124 (15.9%)
$30,001 - $45,000
36 (4.6%)
$45,001 - $60,000
22 (2.8%)
Less than $15,000 Over $60,001
417(53.6%) 18 (2.3%)
No individual
income
161 (20.7%)
Total Responses = 778*
* 52 respondents did not reply to this item, and the question of income did not apply to 116
individual respondents under the age of 18 years.
Appendix A: Sample Description of Survey Respondents 148
Respondent Representation by Race/Ethnicity, Gender, and Age
In comparing statewide penetration rates based on data from the Client and Services
Information System (CSI), the survey respondents were generally representative of the state
in terms of race/ethnicity, gender, and age.66 As shown in Table 34, the proportions of
African Americans (11.9% versus 12.5% statewide), Asians and Pacific Islanders (5.3% versus
5.7% statewide), and Caucasians (40.0% versus 38.1% statewide) were similar to the state.
Latinos (36.0% versus 28.1% statewide) and Native Americans and Alaskan Natives (4.7%
versus 0.8%) were slightly over represented in the survey, whereas “other race”
respondents were under represented. By gender, representation of females (53.4% versus
49.2% statewide) and males (45.3% versus 50.8%) was similar to the state. Finally, because
age groupings were defined differently between the survey and the CSI, the best estimate
was derived from combining children and transition age youth into one age group and
adults and older adults into another. Generally, the proportions of children and transition
age youth (34.1% versus 41.0% statewide) as well as adults and older adults (65.9% versus
59.0%) respondents were similar to that of the state. The overlap in age definition between
transition age youth and adults (by one year) could account for the small differences that
indicate there are slightly more adults or older adults represented in the survey.
Table 34 – Representation of Survey Respondents to Statewide Clients
by Race/Ethnicity, Gender, and Age
Survey Statewide67
Race/ethnicity
African American 11.9% 12.5%
Asian and Pacific Islander 5.3% 5.7%
Caucasian 40.9% 38.1%
Latino 36.0% 28.1%
Native American and 4.7% 0.8%
AOltahsekra n Native 1.2% 14.8%
Gender
Female 53.4% 49.2%
Male 45.3% 50.8%
Age Group
Children and TAY 34.1% 41.0%
Adults and Older Adults 65.9% 59.0%
66
The data on these demographics come from the 2007-08 fiscal year report that contains CSI data provided
to the state as of June 2010. This is the latest report published on the California Department of Mental Health
Department website: www.dmh.ca.gov.
67
Ibid 66.
Appendix A: Sample Description of Survey Respondents 149
Respondent Past, Current, and Desired Employment/Education
Information about employment (for respondents over the age of 18) and involvement in
school (for both children and respondents over the age of 18) was gathered in reference to
two points in time: one-year ago and today. In addition, respondents indicated their
desired, as opposed to actual, levels of employment and participation in school. More than
one response could be selected; therefore, the percentages are based on total responses
rather than the number of respondents.
Reports on past (one-year ago) and current employment and school involvement are
presented in Table 35. Overall, employment and school involvement reported for one-year
ago was similar to the current status. Being disabled and not able to work was reported in
21.1% of all responses in the past and 19.3% of all responses currently. Reporting on the
past, 10.6% of respondents indicated that they were unemployed or retired; 7.2% reported
that they are a stay-at-home parent; 19.5% had full or part-time employment; or 20.1%
were attending school. These reports were similar to current reports: 11.4% reported that
they are unemployed or retired; 7.7% reported they are a stay-at-home parent; 17.7% have
full or part-time employment; or 18.8% are attending school.
Furthermore, being unemployed and seeking work was reported in 9.5% of all responses in
the past and 11.2% of all responses currently. Reporting on the past, 2.4% of respondents
indicated that they participated in work training programs; 8.9% participated in volunteer
activities; or less than 1.0% were paid artists. These reports were similar to current reports:
2.8% reported that they are participating in work training programs; 10.4% are participating
in volunteer activities; or less than 1.0% are paid artists.
Table 35 – Past and Current Employment or School Status
Employment or School Status Past Current
Disabled and not able to work 242 (21.1%) 217 (19.3%)
Unemployed and not seeking employment 72 (6.3%) 70 (6.2%)
Retired 49 (4.3%) 59 (5.2%)
Stay-at-home parent 83 (7.2%) 86 (7.7%)
In school 231 (20.1%) 211 (18.8%)
Working part-time (less than 35 hours per week) 122 (10.6%) 122 (10.9%)
Working full-time (35 or more hours per week) 102 (8.9%) 76 (6.8%)
Work-training program 27 (2.4%) 32 (2.8%)
Unemployed and seeking employment 109 (9.5%) 126 (11.2%)
Paid artist 9 (0.8%) 8 (0.7%)
Volunteering 102 (8.9%) 117 (10.4%)
Total Responses = Total Responses =
TOTAL
1,148 1,124
Appendix A: Sample Description of Survey Respondents 150
Results on desired employment and school participation indicated a greater interest in
being employed. As shown in Table 36, desiring to be unemployed or retired was reported
in 10.9% of all responses. However, a larger proportion of responses indicated a desire for
full or part-time employment (29.0%) or wanting to attend school (23.9%). These responses
are higher than the current employment and school status reported.
An interest in seeking work was reported by 11.8% of all responses on desired employment
and school status. Respondents also reported that they want to be a stay-at-home parent
(4.8%), want to participate in work training programs (5.8%); participate in volunteer
activities (10.1%); or be a paid artist (3.6%), which is higher than reported for the current
status.
Table 36 – Desired Employment or School Status
I desire to not be employed 83 (6.6%)
I desire to be retired 55 (4.3%)
I desire to be a stay-at-home parent 61 (4.8%)
I desire to be in school 302 (23.9%)
I desire to work part-time (less than 35 hours per week) 166 (13.1%)
I desire to work full-time (35 or more hours per week) 201 (15.9%)
I desire to be in a work-training program 74 (5.8%)
I desire to be seeking employment 149 (11.8%)
I desire to be a paid artist 46 (3.6%)
I desire to volunteer 128 (10.1%)
TOTAL Total Responses = 1,265
Respondent Past, Current, and Desired Living Situation
Information about housing status (for children and respondents over the age of 18) was also
gathered in reference to two points in time: one-year ago and today. In addition,
respondents indicated their desired, as opposed to actual, housing situation. Unlike
information on employment and school status, only one response was possible to describe
past and current housing status.
Results on past (one-year ago) and current housing status are presented in Table 37.
Overall, living situation reported for one-year ago was similar to the current status. Living
independently in their own home or apartment was reported by 37.0% of respondents in
the past and 37.5% currently. Reporting on the past, an additional 12.0% reported living in
single room occupancy, supportive housing, or a board and care; and 34.9% reported
residing with their parents/family, friends, or in relative foster care. These reports were
Appendix A: Sample Description of Survey Respondents 151
similar to current reports: 11.0% reported living in single room occupancy, supportive
housing, or a board and care, and 32.4% reported residing with their parents/family,
friends, or in relative foster care.
Furthermore, reporting on the past, 1.5% reported living in non-relative foster care; 2.3%
reported residing in a group home or skilled nursing home; 7.6% reported being homeless
or residing in a shelter or temporary housing; and 1.6% reported being in a psychiatric
hospital or incarcerated. These reports were similar to current reports: 1.2% reported living
in non-relative foster care; 2.7% reported residing in a group home or skilled nursing home;
6.6% reported being homeless or residing in a shelter or temporary housing; and less than
1.0% reported being in a psychiatric hospital or incarcerated.
Table 37 – Current Housing Situation
Living Situation Past Current
I live independently in my own house or 331 (37.0%) 355 (37.5%)
aI lpivaert imn ean Sti ngle Room Occupancy 16 (1.8%) 12 (1.3%)
I live in supportive housing 61 (6.8%) 67 (7.1%)
I live in a board and care 30 (3.4%) 25 (2.6%)
I live with my parents/family 272 (30.4%) 267 (28.2%)
I live with friends 40 (4.5%) 37 (3.9%)
I live in foster care with a relative 3 (0.3%) 2 (0.2%)
I live in foster care with a non-relative 14 (1.5%) 11 (1.2%)
I live in a group home or residential treatment 21 (2.3%) 25 (2.6%)
f I a li c v i e li t i y n a skilled nursing home 1 (0.1%) 1 (0.1%)
I live in a shelter/temporary housing 11 (1.2%) 27 (2.9%)
Homeless 57 (6.4%) 35 (3.7%)
In a psychiatric hospital 8 (0.9%) 6 (0.6%)
Incarcerated/in prison 6 (0.7%) 2 (0.2%)
I do not know 2 (0.2%) N/A
Other* 21 (2.3%) 21 (2.2%)
TOTAL N = 894 N = 893
* Other housing situations include: facing eviction, living in a hotel, having a roommate, and runaway.
Appendix A: Sample Description of Survey Respondents 152
Reports of desired housing indicated a much greater interest in living independently. As
presented in Table 38, almost two-thirds (64.5%) of respondents reported wanting to live
independently in their own home or apartment. An additional 6.0% reported wanting to live
in single room occupancy, supportive housing, or a board and care. Furthermore, 17.6%
reported wanting to reside with their parents/family, friends, or in relative foster care.
A very small number of respondents reported wanting to live in non-relative foster care
(0.5%), or a group home or skilled nursing home (0.4%). An even smaller number reported
wanting to live in a shelter/temporary housing or a psychiatric hospital (0.3%).
Table 38 – Desired Housing Situation
To live independently in my own house or 610 (64.5%)
a T p o a li r v t e m i e n n a t Single Room Occupancy 8 (0.8%)
To live in supportive housing 39 (4.1%)
To live in a board and care 10 (1.1%)
To live with my parents/family 146 (15.4%)
To live with friends 20 (2.1%)
To live in foster care with a relative 1 (0.1%)
To live in foster care with a non-relative 6 (0.6%)
To live in a group home or residential treatment 3 (0.3%)
f T a o c i l l i i v t e y in a skilled nursing home 1 (0.1%)
To live in a shelter/temporary housing 2 (0.2%)
To live a psychiatric hospital 1 (0.1%)
I do not know 23 (2.4%)
Other* 16 (1.7%)
TOTAL N = 886
* Other desired housing situations include: living in a camper, being a nomad, living in a retired home, and
having a roommate.
Appendix A: Sample Description of Survey Respondents 153
Seriousness of Mental Health Concern
Respondents were asked about the seriousness of their mental health concerns in reference
to receiving services over the past year. As presented in Figure 64, 69.3% of respondents
described their concerns to be serious or extremely serious. Another 30.7% described their
concerns to be moderate or mild.
Figure 64 – Seriousness of Mental Health Concerns
An extremely
serious concern
A serious 281 (32.4%)
concern
320 (36.9%) A mild concern
97 (11.2%)
A moderate
concern
169 (19.5%)
N = 867
Appendix A: Sample Description of Survey Respondents 154
Mental Health Services Received
Respondents were asked about the types of services they received over the past year. The
services are listed in Figure 65. Respondents could select more than one response; on
average, respondents selected nearly three types of services received over the past year.
However, the percentages presented in the figure are percentages of respondents who
separately reported receiving each service, and are based on the total survey sample size.
Counseling/psychotherapy was reported by the largest number of respondents (58.5%),
followed by case management (40.1%), medication management/support (35.8%), and peer
support services (27.0%). Other services reported were Full Service Partnership programs
(18.4%), housing/residential services and supports (18.6%), crisis services (13.7%),
employment support services (12.8%), psychiatric hospitalization (11.9%), and alcohol/drug
abuse treatment (11.1%). Fewer than 10.0% of respondents reported receiving each of the
following services: hotline or warmline supports, parenting classes, residential
treatment/crisis residential, day treatment/partial hospitalization, and clubhouses.
Figure 65 – Mental Health Services Received (Within Past Year)
Counseling/Psychotherapy 58.5%
Case management 40.1%
Medication management/Medication support 35.8%
Peer supports/Parent and family partnerships 27.0%
Full Service Partnership (FSP) 18.4%
Housing/Residential services and supports 18.6%
Crisis intervention services 13.7%
Employment support services/Vocational services 12.8%
Psychiatric hospitalization 11.9%
Alcohol/Drug abuse treatment 11.1%
Hotline or warmline to talk to someone 9.2%
Parenting class 7.7%
Residential treatment/Crisis residential 6.8%
Day treatment/Partial hospitalization 6.6%
Clubhouse 6.7%
Other* 7.9%
0% 10% 20% 30% 40% 50% 60% 70%
N = 946, Total Responses = 2,738
* Other services received include: Wraparound services, in-home respite care, mental health awareness
groups, and specific evidence based services.
Appendix A: Sample Description of Survey Respondents 155
As noted in Figure 66, over one-third of respondents (37.6%) reported having had received
services for more than five years. Another 14.0% reported having had received services for
three to five years, 16.3% reported one to two years, and 18.0% reported less than one year
of receiving mental health services.
Figure 66 – Length of Mental Health Services Received
1 to 2 years
154 (19.1%)
3 to 5 years
Less than 1 year
132 (16.2%)
170 (20.9%)
More than 5 years
356 (43.8%)
N = 812
Appendix A: Sample Description of Survey Respondents 156
APPENDIX B – Standardized Measures
Description of Measures
Recovery Process Inventory
Developers of the 22-item Recovery Process Inventory (RPI) sought to measure recovery,
which they defined as living a fulfilling and productive life “despite the limitations of one’s
mental illness.”68 The RPI was designed to capture what earlier recovery instruments
measured in addition to new constructs identified during consumer focus groups – health,
well-being, mental health triggers, mental health management, and self-efficacy (acquiring
employment and housing).
A draft of the RPI was implemented by a group of seven interviewers (trained consumers)
who read aloud the inventory to 459 consumer stakeholders throughout South Carolina
mental health centers. The Mental Health Statistics Improvement Program (MHSIP) Adult
Consumer Perception Survey was also included during the interview for data comparison
purposes. (Both the RPI and MHSIP share three topic areas that should have similar
outcome scores among consumers: service quality, appropriateness, and perceived
outcomes.) The final 22-item inventory includes six dimensions of recovery defined by the
developers and consumers – “anguish, connected to others, confidence and purpose,
others’ care and help, good living situation, and hopeful/cares for self”. Respondents are
requested to read each item and provide their response on a 5-point Likert scale comprised
of the following options: I strongly agree; I agree; I am neutral; I disagree; and I strongly
disagree.
All dimensions, except for Others’ Care, had strong internal consistency (α ≥ .71 for each).
Also, all dimensions were somewhat correlated (.26 ≤ r ≤ .55), meaning that they each
measured different constructs without substantial overlap. RPI outcomes were moderately
correlated with MHSIP outcomes where expected; participants’ positive responses to the
RPI aligned with positive responses to items about service quality and appropriateness and
perceived outcomes in the MHSIP. However, the correlations suggest that the scales do not
measure the same concepts.
Strengths and Difficulties Questionnaire
The Strengths and Difficulties Questionnaire (SDQ) was developed as a behavioral screening
questionnaire for 3-16 year olds. In addition to screening, however, it has since been used in
clinical assessments; epidemiological, developmental, genetic, social, clinical, and
educational studies; and for the purpose of evaluating the outcomes of specific emotional
68
Jerrell, J.M., Cousins, V.C., Roberts, K.M. (2006). Psychometrics of the Recovery Process Inventory. Journal
of Behavioral Health Services & Research, 33(4), 464.
Appendix B: Standardized Measures 157
and behavioral health interventions. The SDQ has abundant evidence of strong reliability
and validity.69
The 25 items on the SDQ are divided evenly between five subscales: emotional symptoms,
conduct problems, hyperactivity/inattention, peer relationship problems; and prosocial
behaviors. In the self-report version, respondents are requested to read each of the 25
items and mark their responses on a 3-point Likert scale: not true, somewhat true, and
certainly true. Each response is assigned a score of 1 to 3 points, which are subtotaled for
each subscale. The first four scales (comprised of 20 items) are added together to generate
a “total difficulties” score.
Recovery Oriented Systems Indicators Measure
The Recovery Oriented Systems Indicators Measure (ROSI) is developed from and grounded
in the lived experiences of adults with serious and persistent mental illness. It is comprised
of two parts: the Consumer Self-Report Survey and the Administrative Data Profile. The
consumer survey consists of 42 items designed to assess the recovery orientation of
community mental health systems for adults with serious and prolonged psychiatric
disorders. Section one of the ROSI requests that respondents read 16 statements and
indicate their responses on a 5-point Likert scale including the following options: does not
apply to me; strongly agree; agree; disagree; and strongly disagree. Section two of the ROSI
includes 26 items corresponding to the 5-point Likert scale in section one: does not apply to
me; almost always/always; often; sometimes; never/rarely. Section 3 consists of one open-
ended response item, which was not included for use in the SSCE.
Initial refinement of the ROSI involved administering the survey to a diverse cross section of
219 consumers in seven states.70 A reliability coefficient was computed for the final set of
42 items, resulting in a Chronbach’s alpha of 0.95—an indication of the instrument’s strong
reliability. Respondents also rated items in terms of their importance, and all final items
were rated relatively high in their ability to reflect the lived experience of consumers.71 A
factor analysis of the results evidenced eight underlying domains for the 42 items; however,
subsequent analysis of a larger sample across several states resulted in six scales: person-
centered focus and directed decision-making; holistic focus; moving on up; basic material
resources; system potholes; and mistreatment.72 Final items were selected based on the
response scale distribution and whether the direction of responses indicated support of
69
YouthinMind (January 1, 2012). What is the SDQ? Retrieved from http://sdqinfo.com/a0.html.
70
National Technical Assistance Center for State Mental Health Planning. (2004). National Research Project for
the Development of Recovery Facilitating System Performance Indicators - ROSI Measures, 2. Alexandria, VA:
Onken, S.J., Dumont, J.M., Ridgway, P., Dornan, D.H., Ralph, R.O.
71
National Technical Assistance Center for State Mental Health Planning. (2006). Mental Health Recovery:
What Helps and What Hinders? Phase II Technical Report: Development of the Recovery Oriented System
Indicators (ROSI) Measures to Advance Mental Health System Transformation, 39. Alexandria, VA: Onken, S.J.,
Dumont, J.M., Ridgway, P., Dornan, D.H., Ralph, R.O.
72
These six scales are based on the latest known analysis from the scale developers. The six scales were
identified by the scale developers in unpublished documents shared with the UCLA evaluation team.
Appendix B: Standardized Measures 158
recovery; whether they maintained themes of recovery; whether they were unique in their
meaning, content, or interpretation; whether they were clearly worded; whether they were
deemed of the highest priority by consumers; and whether they pertained to specific
demographic variables and subpopulations. As such, the ROSI is viewed to successfully
bridge the gap between the principles of recovery and self-help, and the real-world
application of these principles in the everyday work of staff and service systems.73
Reliability Analysis of Measures Using Survey Data
A reliability analysis on the items of each instrument was used to examine the internal
consistency of the scales. It is important to note that these results are based on the overall
sample in order to provide a general assessment of the reliability of these measures. In
order to assess the internal consistency of each scale, coefficient alpha was obtained for
each scale. This measure examines the interrelatedness of items within each scale and the
extent to which scale items are being influenced by error. Coefficient alpha is bounded
between 0 and 1. Higher scores indicate higher reliability of items within a scale. Generally,
a coefficient alpha that is greater than 0.60 is accepted as a satisfactory level of internal
consistency. Subscales with lower levels of internal consistency may be more difficult to find
significant differences.
Recovery Process Inventory
Coefficient alpha was estimated for the RPI using 813 survey respondents but only a subset
was used for the current analysis due to missing responses for items. Table 39 lists the
sample size, number of items, and estimated coefficient alphas for each of the six subscales
of the RPI. The Anguish subscale and the Confidence and Purpose subscale are above the
acceptable level of reliability with an estimated coefficient alpha of 0.800. In contrast, the
Other’s Care/Help has a low coefficient alpha of 0.357. This might be due in part to the low
number of items comprising this subscale. However, the Living Situation and Hopeful/Cares
for Others subscales have only two items each, but have adequate levels of internal
consistency. Due to the low level of internal consistency in the Others’ Care/Help subscale,
inferences made from this subscale are cautioned.
Table 39 – Internal Consistency Estimates of the Recovery Process Inventory
Scale N Number of Items Coefficient Alpha
Anguish 630 8 .802
Connected to Others 686 3 .608
Confidence and Purpose 648 4 .800
Others’ Care/Help 651 3 .357
Living Situation 670 2 .738
Hopeful/Cares for Others 659 2 .720
73
Ibid 71.
Appendix B: Standardized Measures 159
Strengths and Difficulties Questionnaire
The internal consistency of the SDQ was analyzed for the 133 children for whom the survey
was completed. Table 40 below summarizes the obtained coefficient alphas for each
subscale, with the Total Difficulties scale omitted. Overall, the scales derived from the SDQ
show an adequate level of reliability, with four of the five SDQ subscales in the 0.720 to
0.760 range. The Peer Problems subscale had the lowest level of internal consistency of the
SDQ subscales, with an estimated coefficient alpha of 0.614. These results suggest a
reasonable amount of scale measurement that is not associated with error.
Table 40 – Internal Consistency Estimates of the Strengths and
Difficulties Questionnaire
Scale N Coefficient Alpha
Emotional Symptoms 115 .720
Conduct Problems 113 .749
Hyperactivity/Inattention 112 .734
Peer Problems 116 .614
Prosocial 117 .760
Recovery Oriented System Indicators
Estimated coefficient alphas were obtained for the ROSI with data from 813 adult survey
respondents. However, due to missing responses to survey items, the reliability analyses
were conducted on a smaller subset of adult respondents.
The total sample size, number of items, and coefficient alpha for the ROSI subscales are
listed in Table 41. There is a wide range of estimated coefficient alpha values obtained for
each of the subscales, ranging from 0.600 for the Mistreatment subscale to .943 for the
Person Centered Focus and Decision Making Subscale. All scales reached an adequate level
of reliability greater than or equal to 0.60.
Table 41 – Internal Consistency Estimates of the Recovery Oriented System Indicators
Scale N Number of Items Coefficient Alpha
Person Centered Focus and Decision Making 291 13 .943
System Potholes 396 8 .851
Holistic Focus 319 6 .712
Moving On Up 244 4 .669
Mistreatment 424 3 .600
Basic Material Resources 456 4 .694
Appendix B: Standardized Measures 160
APPENDIX C – Sample Description of Interview Respondents
Sample Description
Altogether, 40 interviews were conducted across the state. Eighteen (18) of the interviews
were conducted in the southern region, 12 in the Bay Area region, 6 in the central region,
and 4 in the northern region. Demographic details about the interview participants are
described below according to the service area that was the focus of each interview.
Demographics of Peer Support Services Respondents
Nine interviews were conducted with a primary focus on peer support services. Five of
these interviews were conducted with females and four were conducted with males. Four
of the peer support interviews were conducted with Caucasian respondents, four with
Latino(a) respondents, and one with a respondent of mixed race/ethnicity (i.e.,
Latino/Native American). Almost half (4) of the interviews that focused primarily on peer
support services were conducted with older adults. Three interviews were conducted with
adults and two with transition age youth.
Demographics of Employment Support Services Respondents
Nine interviews were conducted with a primary focus on employment support services. Of
these, four were conducted with adults, four with older adults, and one with a transition
age youth respondent. Four of the respondents were Caucasian and three were African
American. One interview respondent identified as Asian and one as Latino.
Demographics of Crisis Intervention Respondents
Twenty-two (22) interviews were conducted with a primary focus on crisis intervention
services. Thirteen (13) of these interviews were conducted with males, and nine were
conducted with females. Nearly one-third (7) of the interview respondents were African
American, and almost a quarter (5) were Caucasian. Four respondents had a mixed ethnic
background (i.e., African American/Caucasian, African American/Native American,
Latino/Native American, and Filipino/Irish). Three interview respondents were Asian, and
three were Latina. The distribution of interview respondents across the age groups was five
interviews with parents of children, seven interviews with transition age youth, six
interviews with adults, and four interviews with older adults.
Appendix C: Sample Description of Interview Respondents 161