Recommendations

Calbright College It Must Take Immediate Corrective Action to Accomplish Its Mission to Provide Underserved Californians With Access to Higher Education May 2021 REPORT 2020‑104 CALIFORNIA STATE AUDITOR 621 Capitol Mall, Suite 1200 | Sacramento | CA | 95814 916.445.0255 | TTY 916.445.0033 For complaints of state employee misconduct, contact us through the Whistleblower Hotline: 1.800.952.5665 Don’t want to miss any of our reports? Subscribe to our email list at auditor.ca.gov For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255 This report is also available online at www.auditor.ca.gov | Alternative format reports available upon request | Permission is granted to reproduce reports Elaine M. Howle State Auditor May 11, 2021 2020-104 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As directed by the Joint Legislative Audit Committee, my office conducted an audit of Calbright College (Calbright) to assess its progress in establishing the college and its ability to achieve its mission of providing self-paced educational opportunities to California adults who face barriers to attending traditional colleges. We determined that Calbright’s potential value to the State is significant. However, it is behind in accomplishing key milestones and must act quickly to demonstrate its ability to achieve its mission. A primary reason why Calbright’s progress is not on track is that its former executive team failed to develop and execute effective strategies for launching the college. It also used ineffective and inappropriate hiring and contracting processes that failed to ensure that the right team was in place to accomplish its goals. Because of these missteps, Calbright has struggled to adequately enroll the students it was intended to serve, took longer than it should have to develop a student support system, and did not adequately partner with employers in the development of its educational programs, thereby hindering its ability to assist its students in obtaining jobs. If it succeeds in recovering from these missteps, Calbright could fulfill its purpose and help address barriers that many Californians face to complete a postsecondary education to improve their economic mobility. Calbright’s new leadership has taken some initial steps to address the deficiencies we observed. The college must now make significant progress in enrolling, graduating, and helping to secure jobs for its students by following the recommendations we make in this report. If Calbright does not demonstrate meaningful implementation of our recommendations by the end of 2022, we recommend that the Legislature eliminate the college as an independent entity and explore other options for providing self-paced educational programs to California adults. Respectfully submitted, ELAINE M. HOWLE, CPA California State Auditor 621 Capitol Mall, Suite 1200 | Sacramento, CA 95814 | 916.445.0255 | 916.327.0019 fax | www.auditor.ca.gov iv California State Auditor Report 2020-104 May 2021 Blank page inserted for reproduction purposes only. California State Auditor Report 2020-104 v May 2021 Contents Summary 1 Introduction 5 Chapter 1 Calbright’s Early Missteps Have Hindered Its Ability to Achieve Its Mission 11 Recommendations 28 Chapter 2 Calbright Must Take Immediate Action to Enroll, Educate, and Help Secure Jobs for Its Target Student Population 31 Recommendations 49 Appendix A Results of Our Online Survey of Calbright Students 53 Appendix B Scope and Methodology 55 Response to the Audit Calbright College 59 California State Auditor’s Comments on the Response From Calbright College 67 vi California State Auditor Report 2020-104 May 2021 Blank page inserted for reproduction purposes only. California State Auditor Report 2020-104 1 May 2021 SUMMARY Results in Brief Audit Highlights . . . Individuals with higher levels of education tend to receive Our audit of Calbright highlighted higher wages, and reporting by the Public Policy Institute of the following: California (PPIC) indicates that by 2030 about two-thirds of the jobs » Its former executive team’s poor in the State will depend on at least some postsecondary education. management contributed to ineffectively However, significant barriers make it difficult for many Californians setting up the college. to attend postsecondary education programs, including work and family responsibilities. Competency-based education, in which • There is no detailed strategy for students progress at their own pace rather than according to a spending the more than $175 million traditional academic calendar, could help these adults complete a that it expects to receive in postsecondary education and improve their economic mobility. state funding. • It failed to follow sound hiring In recognition of a deficit in competency-based education in practices, resulting in a substantial the community college system, in 2018 the Legislature created an lack of needed experience across online community college, now named Calbright College (Calbright), key positions. to provide high-quality, affordable, and self-paced educational programs. The Legislature identified a target student population that • It delayed setting up a student it intended Calbright to serve, including adults between the ages of support system. 25 and 34 with high school diplomas who lack college degrees and who might otherwise struggle to obtain an education because of their » Calbright’s methods for ensuring that work and family responsibilities. The Legislature specified milestones students succeed are inadequate—most that were critical to effectively setting up the college and assigned students have either dropped out or deadlines for achieving them. However, Calbright is significantly stopped progressing. behind in meeting those milestones and risks failing to achieve its » It has not developed a process for helping mission unless it undertakes fundamental planning and strategizing students obtain well‑paying jobs. activities that it has yet to adequately perform. » Although Calbright has made recent Calbright’s struggles to fulfill its mission can be attributed, in part, improvements, it has yet to develop a to its former executive team’s poor management of the setup of clear and robust strategy to accomplish the college. Most importantly, this team did not develop a detailed its mission. strategy for how and when Calbright would spend the more than $175 million in state funding it expects to receive through June 2025 to accomplish key milestones. In the absence of such a plan, the purpose of its spending to date is unclear, and neither the Legislature nor the public can effectively assess its progress. In addition, although Calbright entered a contract with the Foundation for California Community Colleges (Foundation) to assist in developing key administrative functions such as payroll and accounting, it did not clearly define the work that it expected the Foundation to perform nor adequately plan the transition to managing its own key business operations. As a result, Calbright has paid the Foundation more than $4 million to date but still cannot independently manage functions such as its own payroll and accounting. 2 California State Auditor Report 2020-104 May 2021 Although hiring staff qualified to set up the college is critical to Calbright’s success, its former executive team also failed to follow sound hiring practices. Nine of the 14 hiring decisions we reviewed were problematic because Calbright either did not conduct competitive hiring processes or gave preferential treatment to certain candidates. Ultimately, Calbright’s staffing decisions resulted in a substantial lack of public education experience or even broader public sector experience across key positions, which likely hindered Calbright’s success at achieving its goals. The former executive team’s poor management negatively affected the college’s progress in setting up its educational programs and serving its target student population. For instance, it failed to ensure that one of the three initial educational programs it chose to launch in October 2019 would benefit the target students. Specifically, it decided to offer a cybersecurity program even though most cybersecurity jobs require at least a bachelor’s degree, which its intended students typically do not have. Additionally, Calbright has generally struggled to enroll its target student population in its programs, likely in part because of its program choices and in part because it has not yet developed and implemented an effective plan for reaching out to such individuals. In addition, Calbright has not established adequate methods for ensuring that students receive the support they need to succeed. The majority of its students have either dropped out or stopped making progress in their studies. Although a number of factors may have contributed to these outcomes, Calbright cannot demonstrate that it provided the support the students needed to succeed because under its former executive team, it was slow to develop a process for doing so. Further, although helping its students obtain well-paying jobs is central to its mission, Calbright has not adequately collaborated with employers to ensure that its graduates are prepared for and can get jobs. In short, Calbright is not yet adequately achieving its core purpose: enrolling adult students who cannot otherwise obtain postsecondary education, guiding them through completion of a program that provides industry-valued credentials, and helping them secure employment. Calbright has recently made certain improvements to its operations but must do more to position itself to effectively achieve its mission. Most of its initial executive team left in 2020, and Calbright is now under new leadership that has begun taking positive steps toward correcting the deficiencies we identified, such as through developing stronger hiring practices. However, Calbright has yet to develop a clear and robust strategy for how it will accomplish its mission. It has not set adequate goals for what it hopes to accomplish, determined the steps necessary to achieve those goals, or established a timeline for when it will accomplish California State Auditor Report 2020-104 3 May 2021 them. The need remains for flexible educational opportunities for California adults who face barriers to attending traditional community colleges, and if successful, the competency-based education model that Calbright offers could provide those opportunities. However, Calbright must make immediate, substantial progress in accomplishing the setup of the college and in enrolling, graduating, and securing jobs for the California adults it is intended to serve. To ensure the State receives appropriate value from its investment, we therefore recommend that the Legislature eliminate the college if Calbright cannot demonstrate meaningful progress implementing these reforms—which we detail later in the report—by December 2022. Summary of Recommendations Legislature The Legislature should eliminate Calbright as an independent community college district if it does not demonstrate substantive improvements in its ability to accomplish its mission by implementing the recommendations in this report by December 2022. If it eliminates Calbright, the Legislature should explore other options for providing competency-based education for California adults who face barriers to traditional postsecondary education. Calbright Calbright should immediately begin developing an implementation plan that outlines the specific steps necessary for it to accomplish its mission, and it should complete the planning process and begin following the plan by November 2021. The plan should include a specific timeline for performing each of the steps it identifies, as well as the estimated costs. By November 2021, Calbright should develop and implement specific strategies for the following: • Developing educational programs that can benefit its target student population. • Enrolling its target student population. • Ensuring that its students receive the support they need to graduate. 4 California State Auditor Report 2020-104 May 2021 • Working with employers to ensure that its students are prepared for and can secure jobs after completing a certification through one of its programs. By November 2021, Calbright should finalize its development of hiring processes, policies, and procedures that align with requirements in state law and regulation. Agency Comments Calbright agreed with our recommendations and acknowledged that it needs to make improvements to address the deficiencies that we identified. California State Auditor Report 2020-104 5 May 2021 INTRODUCTION Background Individuals with higher levels of education tend to receive higher wages, and reporting by the Public Policy Institute of California (PPIC) indicates that by 2030 about two-thirds of the jobs in the State will depend on at least some postsecondary education.1 Consequently, Californians who do not complete a postsecondary education program may struggle to obtain jobs that pay well. More broadly, the State also benefits when more Californians complete postsecondary education programs. The PPIC cites research finding that increased education is associated with lower rates of unemployment, poverty, and incarceration; reduced demand for public assistance programs; and greater tax revenue and civic engagement. More than 11 million Californians who are age 25 or over-or 41 percent of individuals in the State in that age group-have a high school diploma or its equivalent but lack a college degree. Although it is likely some of these individuals have completed some postsecondary education, these data suggest that a large number of Californians could benefit from obtaining a postsecondary education. However, a variety of barriers may make enrolling at a traditional college, university, or vocational school difficult for many Californians. Some individuals may be unable to afford the cost of postsecondary education. For Californians who are employed, work schedules and responsibilities may prevent them from devoting time to a traditional education. Some individuals may also be too busy caring for family members to dedicate time each week to coursework and studying. These barriers can be more pronounced for individuals from low-income backgrounds, who often cannot afford childcare or other assistance and who may be unable to rely on family members to support them financially while in school. The Creation of Calbright College In May 2017, then-Governor Edmund G. Brown Jr. directed the Office of the Chancellor of the California Community Colleges (Chancellor’s Office) to submit to him a plan to establish a new community college that exclusively offered fully online programs. In his letter, the Governor stated it was time for the community college system to increase the availability of online programs in order to make college more accessible and affordable. To assist in developing a plan, the Chancellor’s Office—with assistance 1 The Public Policy Institute of California is a nonpartisan think tank focused on California public policy. 6 California State Auditor Report 2020-104 May 2021 from the National Center for Higher Education Management Systems—convened a workgroup of community college employees and stakeholders.2 The workgroup determined that California has a strong supply of well-paying jobs that require education beyond high school but less than a college degree, particularly in sectors such as health care, manufacturing, and finance. The workgroup also indicated that most occupations favored by workers without college degrees tend to require a certification or government-issued license. Although the State already offered online, certification-oriented learning options through its community college system, these programs tended to follow a fixed academic calendar, which could exclude prospective students who have work or family responsibilities that limit their ability to keep up with a fixed schedule. The workgroup found that instead of using a public education option, the majority of Californians who obtain certifications do so through programs managed by for-profit organizations. According to the Department of Finance (Finance), programs managed by for-profit organizations and out-of-state institutions can cost up to nine times more per unit than a community college. Further, Finance noted that the programs may leave Californians with poor employment outcomes and substantial debt, which students may incur whether or not they successfully obtain a certification. After the workgroup completed its analysis, Finance and the Chancellor’s Office developed a proposal for the creation of a new, fully online community college to help those Californians who are underserved by the State’s postsecondary education system. In response, in 2018 the Legislature created the California Online Community College, later named Calbright College (Calbright). Based on the recommendation of the workgroup, the Legislature established a seven-year period—from July 2018 through June 2025—to accomplish the setup of the college. It also provided an initial budget of $100 million in one-time start-up funds, and another $20 million per year during Calbright’s first two years. Calbright indicated that it anticipated receiving an additional $20 million annually for the remainder of the seven-year implementation period, for a total of $240 million. 2 The National Center for Higher Education Management Systems is a nonprofit organization specializing in higher education strategy. California State Auditor Report 2020-104 7 May 2021 Calbright’s Intended Benefits According to the Legislature, it created Calbright because working adults should have access to high-quality, affordable, and flexible opportunities to pursue postsecondary education. The Legislature stated that the existing higher education infrastructure needed to be augmented with learning options for working learners to ensure the future economic resiliency of California’s communities. It further stated that a guiding principle for Calbright was addressing the barriers that prevent working adults from accessing postsecondary education. As we note earlier, California’s other online community college programs generally require students to follow a preset academic calendar. By contrast, Calbright follows an instructional model known as competency-based education that allows students to enroll at any time and to progress at their own pace, and as of April 2021 its programs and course materials were available to students at no cost. In enacting the state law establishing Calbright, the Legislature made numerous findings that emphasized the benefits that Calbright could provide to Californians from particular backgrounds. For instance, the Legislature found that 2.5 million Californians between the ages of 25 and 34 had not completed a postsecondary education program, 80 percent of these individuals were working, and nearly half of these working individuals were Hispanic. It also found that flexible education options could help Californians who lose their jobs during a recession, especially women, who as a Calbright’s Target Population group have regained employment more slowly after • Individuals between the ages of 25 and 34. the Great Recession than men. In its first milestone update to the Legislature in August 2019, Calbright • Adults who completed high school and do not have a college degree. defined its target population as Californians from ages 25 to 34 who fell into several of the groups in • Working adults. the text box, including women, veterans, and those • Hispanic or Latino individuals. with only a high school diploma or equivalent. • Individuals displaced by job loss triggered by a recession. More recently, in its March 2021 strategic vision • Women. report, Calbright defined focus populations that it intends to predominately serve. Included again • Immigrants. were adults age 25 years or older without a college • Military veterans. degree, as well as Latino, African American, and • Incarcerated or formerly incarcerated individuals. Indigenous persons. We refer to the various groups Source: Analysis of state law, Calbright’s August 2019 milestone named in state law and Calbright’s strategic document, and March 2021 strategic vision report. documents—which we list in the text box— as Calbright’s target population. State law requires Calbright to provide unique programs—called program pathways—that do not duplicate those offered at other community colleges. As of March 2021, Calbright offered three 8 California State Auditor Report 2020-104 May 2021 program pathways leading to certification: medical coding, information technology (IT) support, and cybersecurity. Other community colleges also offer statewide, fully online versions of all three program pathways. However, none of the other colleges’ program pathways are self-paced, meaning that they do not fully address the barriers that may prevent some students from attending college. Therefore, we conclude that Calbright’s pathways are not duplicative when compared to the other programs we reviewed. As the text box shows, multiple students indicated in response to a survey that Calbright Student Comments on Calbright’s conducted in June 2020 that its self-paced Self‑Paced Programming structure was beneficial. Calbright opened these “I enjoy being able to learn on my own time since I am a first three program pathways for enrollment in stay-at-home mom. I do not have much down time until October 2019. By October 2020, Calbright had my little one is asleep, so it’s great to have found a college enrolled about 470 students, of whom 12 had that offers this type of schooling.” completed a pathway—essentially finishing their time with Calbright. We discuss the three * * * pathways in greater detail in Chapter 2. “I’m grateful for the self-paced nature of the course . . . especially during this time! The ability to relieve myself of the pressure to complete, when I need that relief, is priceless.” Calbright’s Oversight Structure * * * The Legislature established several forms “My work hours have increased 50 percent and my schedule of oversight to ensure Calbright’s effective was erratic and I have less time to devote to study and I’m more tired when I do, which does make it more difficult. start-up, including requiring it to meet specific My schedule recently stabilized, at least temporarily, but my milestones in its first seven years. Some of these free time is so limited, it’s hard to keep up. But I really want milestones relate to organizational management to continue.” and administration. For example, the Legislature required that, by July 2019, Calbright * * * validate a business plan, develop a seven-year “Self-paced and self-directed is amazing. I’ve never felt implementation plan, and develop internal policies not-worried about school before. It’s really nice to have and procedures for business and personnel a program I can ignore for a few weeks when the world matters. Other milestones relate to the student throws me a curveball!” experience. For example, the Legislature required Source: Calbright’s June 2020 survey of students. Calbright to create three program pathways by July 2019 and to plan to begin enrolling students by the last quarter of 2019. State law requires Calbright to periodically report to the Legislature and Finance its progress in implementing these milestones. We provide a list of key milestones in Chapter 1, and throughout the report, we discuss Calbright’s progress in meeting them. The Legislature also placed Calbright under the administration of the California Community Colleges Board of Governors (Board of Governors), whose members are almost all appointed by the Governor. California community colleges belong to districts, and each is supervised by a board of trustees that oversees the district’s educational, operational, and financial policies. State law requires the Board of Governors—which is primarily responsible for setting California State Auditor Report 2020-104 9 May 2021 policy and providing guidance for the entire California community college system—to serve as Calbright’s board of trustees. Generally, state law allows community college boards of trustees broad authority to act where there is no specific provision in state law to the contrary, including employment and personnel matters. As the text box notes, state law requires the Board of Governors to perform certain oversight functions Selected Responsibilities That State Law for Calbright. Calbright’s chief executive officer Assigns to the Board of Governors (CEO) serves at the pleasure of the Board • Develop comprehensive plans for the short- and of Governors. long-term growth and development of Calbright’s academic programs. • Consult Calbright in the development and review of The Current Status of Calbright policy proposals. • Approve employment practices, salaries, and benefits As of March 2021, Calbright employed about for all Calbright employees. 60 staff members, including five executive staff, • Approve Calbright’s annual budget. five instructors, and support staff. Calbright • Approve contracts for goods and services, or delegate experienced significant staff turnover in 2020. Its that authority for contracts below a certain threshold. initial CEO resigned in January 2020, and most of its other executives left later that year. It has since Source: State law. replaced most of those staff or eliminated their job positions. Its current CEO assumed the position in February 2020 on an interim basis, which the board made permanent effective August 2020. As authorized by its establishing legislation, Calbright has contracted with the Foundation for California Community Colleges (Foundation) to assist with several key start-up functions. Established in 1998, the Foundation is a nonprofit organization that provides operational, programmatic, and financial support to the State’s community college system. In September 2018, Calbright entered into a contract for the Foundation to provide services related to establishing key administrative functions—such as accounting, payroll, and procurement—and, in the interim, to help manage some of those functions on behalf of the college. As we describe more fully in Chapter 1, Calbright had paid the Foundation $4.4 million as of January 2021, and it continues to rely on the Foundation for many of its administrative functions. However, Calbright’s future is in doubt. In 2020 the Legislature considered a bill that would have required the Board of Governors to close Calbright by the end of that year. At the time, the State anticipated a multibillion dollar budget deficit due to the impact of COVID-19. In a review of higher education budgetary options, the Legislative Analyst’s Office (LAO)—the Legislature’s nonpartisan fiscal and policy advisor—recommended that the Legislature shut down Calbright in an effort to better target spending reductions. The LAO cited concerns regarding Calbright’s high cost per student, lack of accreditation, and duplication of programs existing at other colleges. 10 California State Auditor Report 2020-104 May 2021 However, instead of closing Calbright, the Legislature reduced its one-time start-up funding by $40 million and reduced its appropriation for that year by $5 million. As a result of this reduction, Calbright now expects that its annual appropriation will be $15 million each year, resulting in a total of $175 million over the course of its seven-year implementation period. Because of concerns similar to those raised by the LAO, the Joint Legislative Audit Committee (Audit Committee) approved this audit of Calbright. As of March 2021, the Legislature was again considering legislation to terminate Calbright. California State Auditor Report 2020-104 11 May 2021 Chapter 1 CALBRIGHT’S EARLY MISSTEPS HAVE HINDERED ITS ABILITY TO ACHIEVE ITS MISSION Chapter Summary Calbright’s former executive team mismanaged the college’s setup, resulting in significant planning deficiencies and inadvisable choices related to hiring and contracting. For example, it did not develop an adequate plan for how and when it would spend the $175 million of funding Calbright expects to receive. Until the current executive team develops such a plan, it will risk mismanaging its budget, failing to accomplish its goals, or both. Further, the Board of Governors and the Legislature will lack clear benchmarks for assessing the reasonableness of its spending. Calbright’s former executive team did not ensure that Calbright’s operations were always appropriate. Specifically, they circumvented best practices and requirements for hiring and contracting. Nine of the 14 hiring decisions from 2019 we reviewed were problematic, lacking key components of competitive hiring or showing explicit evidence of favoritism. Further, many of the individuals Calbright relied on to launch the college did not have previous public sector or education experience, a factor that likely led to some of Calbright’s early struggles. The Board of Governors also approved salaries for many of those individuals that were well above the next highest salaries in the community college system for comparable positions. Additionally, although Calbright relied heavily on contracted individuals to perform work to establish the college, it did not develop effective processes for entering into or managing their contracts. As a result, some contract awards appear to have been motivated by personal or professional connections. In another issue, Calbright contracts did not always contain adequate scope of work descriptions to allow for an assessment of the contractor’s performance. These deficiencies in planning and operations undermined Calbright’s progress in effectively serving its students. However, following the departure of almost all of its former executive team in 2020, Calbright has begun to make improvements. Specifically, Calbright has begun developing hiring procedures and it has reduced most of its executives’ salaries. To safeguard public funds and to ensure its ability to successfully achieve the Legislature’s intent in creating an online community college, Calbright will need to continue to implement key reforms. 12 California State Auditor Report 2020-104 May 2021 Calbright Needs a Specific Plan Describing How It Will Use Its Funding to Accomplish Its Goals and Establish Its Operations The Legislature has allocated Calbright significant funding to set up the college and accomplish key milestones so it is critical that Calbright have a specific, transparent plan for how it will use its funding to accomplish those goals while remaining within budget. However, it has yet to develop such a plan. As a result, the purpose of its spending to date is unclear, and both the public and the Legislature are unable to effectively assess its progress. In addition, although Calbright contracted with the Foundation to assist in developing key administrative functions, it did not clearly define the work that it expected the Foundation to perform, nor did it adequately plan for transitioning from relying on the Foundation to managing its key business operations. As a result, it has paid the Foundation more than $4 million but still lacks the ability to independently manage core functions such as payroll and accounting. Calbright Must Develop a Specific Plan for Spending the $175 Million in Public Funds That It Expects to Receive Community colleges, like all public entities, have an obligation to employ sound fiscal management practices to ensure fiscal accountability and to use state resources efficiently and effectively. The Legislature allocated $115 million to Calbright as of April 2021, and Calbright anticipates that the Legislature will allocate a total of $175 million in funding through June 2025. When the Legislature established Calbright, it specified in state law key milestones that it expected the college to meet through the use of its funding. Figure 1 lists these milestones and includes our assessment of Calbright’s progress toward each. We also discuss key milestones in more detail throughout this report. Among other milestones, state law requires that by July 2019, Calbright develop an implementation plan for its first seven years, referred to as the start-up period. State law also allows Calbright to spend some of its start-up funding on the development of a seven-year business plan with key milestones, indicators, and outcomes. In light of the requirements and direction in state law to develop implementation and business plans, we expected that Calbright would have incorporated into one of these plans a detailed strategy for how and when it would spend its public funding (spending plan). Such a spending plan would necessarily include cost estimates for major tasks, such as designing an online system for students and obtaining accreditation, and a timeline for when the college planned to make those expenditures. These details are key to ensuring that Calbright effectively manages its resources and they are necessary for the Board of Governors, the Legislature, and the public to measure its progress and hold it accountable. California State Auditor Report 2020-104 13 May 2021 Figure 1 Calbright’s Progress in Meeting Milestones Established in State Law Plan to begin enrolling students by the last quarter of 2019. Completed Develop an accreditation plan. Completed Develop three program pathways designed in Not adequately partnership with employers and industry groups. completed Map the student experience from start to finish, Not adequately including entry into jobs. completed Develop a seven-year implementation plan and Not adequately validate a business plan. completed Develop internal business processes and Not adequately personnel policies. completed Not adequately Create a statewide outreach plan. completed Establish a process for recognition of prior learning. Not adequately completed Apply for accreditation. On track Incorporate student feedback to improve programs and support services. Develop three additional program pathways. On track ! Publish a comprehensive status report on the college’s activities and students’ outcomes, including graduates’ Not on track employment and earning gains. Design and validate at least 10 additional program pathways. — Incorporate student feedback to improve programs and — support services. Incorporate student feedback to improve programs and — support services. senildaeD tsaP senildaeD gnimocpU senildaeD erutuF MILESTONE STATUS DEADLINE July 1, 2019 Define the duties for instructional support, program Completed development, and other student experience activities. Establish goals for student outcomes. Not adequately completed DEADLINE July 1, 2021 On track DEADLINE August 1, 2021 DEADLINE July 1, 2023 DEADLINE July 1, 2025 Source: Analysis of state law, enrollment data, and Calbright’s planning documents. Note: Duplicate milestones denoted with italics. 14 California State Auditor Report 2020-104 May 2021 Nonetheless, as Figure 2 shows, Calbright has yet to develop an adequate spending plan containing such information. In a mandatory report to the Legislature in August 2019, Calbright summarized its planned expenses for the seven-year period. However, the report did not clearly explain how Calbright’s planned spending related to its goals. For example, the report indicated Calbright would spend $22 million—or 44 percent of its fiscal year 2019–20 expenses—on technology and physical capital outlay, yet the report did not describe specifically how it would spend this funding. The documentation supporting the report also lacked specificity. For example, that documentation reports that $5 million of the $22 million in expenses were for “external technology development services/contractors” but it did not provide any additional detail. Similarly, the report stated that Calbright would spend $34 million in fiscal year 2020–21 on “operating expenses”—which accounted for over half of its planned expenditures in that fiscal year. However, the supporting documentation did not describe with any greater specificity how it intended to spend that funding. We expected Calbright to have more clearly documented how it was going to spend those funds so that it could be best positioned to prudently manage over half of its planned spending. Furthermore, Calbright has not updated the seven-year spending projections it included in its 2019 report to account for the significant developments that have since occurred. Most importantly, in June 2020 the Legislature reduced Calbright’s one-time start-up funding by $40 million and also reduced its appropriation for that year by $5 million. However, as of February 2021, Calbright had yet to revise its spending projections or implementation strategy to reflect its reduced funding. Without an up-to-date, specific strategy for how it will use the funding it has been allocated, neither Calbright nor the Board of Governors can demonstrate that it is spending its funding wisely or that it is on track to accomplish its objectives while remaining within budget. The absence of any specific spending strategy makes the purpose and benefits of Calbright’s spending to date uncertain. Calbright had spent about $28 million as of January 2021, which is about $91 million less than it projected it would spend by June 2021. However, in the absence of an effective spending plan, it is unclear whether Calbright’s reduced spending is the result of prudent savings or an indicator that it is behind on achieving its mission. Best practices indicate that an organization needs to know not only how much it is spending, but also what services it is providing in return, how it is ensuring that it spends its funds effectively, and how that spending is furthering its mission. Although Calbright periodically notifies the Board of Governors of its recent purchases and contracting decisions and annually presents its budget, it cannot fully demonstrate that its spending decisions are reasonable without a detailed spending plan that contextualizes its spending. California State Auditor Report 2020-104 15 May 2021 Figure 2 The Former Executive Team Failed to Follow Fundamental Practices for Achieving Organizational Goals Calbright’s former executive team did not develop strategies that best practices say are critical for an organization to achieve its goals: Specific tasks needed to accomplish Translating its mission the start-up of the college into goals and Measurable goals for achieving actionable strategies start-up tasks Clear due dates for major objectives Planning how it would spend its $175 million Detailed projections of costs Plans for remaining on budget This lack of effective leadership undermined Calbright’s progress in setting up the college. Source: Review of Calbright’s planning documentation and best practices for implementation planning. Although some members of the Board of Governors criticized the minimal level of detail in Calbright’s August 2019 report, it did not require Calbright to submit a second, improved report for its review. During the July 2019 board meeting at which Calbright presented the 2019 report, some board members stated that it lacked adequate detail in certain areas. In particular, one board member indicated that the report was too high-level and urged Calbright to add greater detail about what it was planning to do and how it would accomplish its goals, including a clearer explanation of how it would use its budget. Calbright’s then-CEO stated that Calbright needed to do a much better job describing the specifics, and she said she would appreciate the chance to present additional information to the board again. Nonetheless, the Board of Governors did not formally request that Calbright develop a new plan for its review and Calbright has not done so. 16 California State Auditor Report 2020-104 May 2021 When we asked about the deficiencies in Calbright’s implementation and spending plan, Calbright’s senior vice president of strategic initiatives acknowledged that the former executive team did not outline target goals or identify activities that Calbright would perform to achieve its objectives. Calbright’s new chief finance and administrative officer, who took his position in February 2021, stated that Calbright plans to put into place a detailed spending plan that is reasonable and appropriate. Although Calbright has not yet developed a draft of this plan, he indicated that it would be feasible for it to include detailed projections for Calbright’s next several years of spending, and feasible to finalize that plan by November 2021. Until Calbright adopts a new implementation plan that incorporates a robust up-to-date spending plan, it is likely to continue to struggle to achieve its goals. Further, until it provides regular updates regarding how its spending has advanced the purposes of its mission, the Board of Governors and the Legislature will lack the information necessary to assess its progress and the reasonableness of its spending. Calbright Must Develop a Plan and Timeline to End Its Reliance on the Foundation for Key Administrative Functions Calbright has not completed development of key administrative functions that state law requires. State law required Calbright to develop internal business processes and personnel policies by July 1, 2019. As we discuss in the Introduction, state law allows Calbright to contract with the Foundation to provide administrative support for the college’s start-up functions. Calbright entered into a contract with the Foundation in September 2018 that calls for the Foundation to help Calbright Key Activities That Calbright Contracted With complete several key administrative activities, as the Foundation for Help in Completing the text box describes, to enable the college to develop internal business processes and become • Development of an expenditure plan, budget, and self-sufficient. timeline for first year start-up activities. • Development and execution of a plan to transition all However, Calbright did not clearly identify the Calbright business processes to Calbright. work that it expected the Foundation to perform, • Establishment of a separate, stand-alone back office nor did it establish a timeline for when it would operating structure for Calbright. no longer require the Foundation’s support. • Development of Calbright’s payroll services. Calbright’s contract does not clearly identify • Provision of financial management services. the specific responsibility of the college and of the Foundation. Instead, the contract states that • Assistance in developing procedures, criteria, and policies for fiscal management. Calbright and the Foundation would hold an initial meeting—within 10 days of the effective • Provision of procurement and contract management services. date of the contract—to prepare a work plan that included staff assignments, a timeline, and Source: Calbright’s contract with the Foundation. an initial budget. It further says that subsequent meetings would occur at least every three months California State Auditor Report 2020-104 17 May 2021 focused on updates to the work plan, budget, and staff assignments. Nonetheless, Calbright could not demonstrate that it completed these critical initial activities. Calbright’s current CEO provided evidence that some meetings with the Foundation occurred beginning in late 2019. However, referring to earlier in Calbright’s history, she indicated that Calbright and the Foundation followed an “organic approach” to the coordination of services instead of creating a formal work plan, as required by the contract, to ensure that Calbright efficiently transitioned to self-sufficiency. She further stated that Calbright has not since developed that work plan because it has been focused on the programmatic needs of its students and on the changing conditions in the labor market brought on by the COVID-19 pandemic. As a result, Calbright lacks clear criteria for evaluating the support for which it has been paying the Foundation or for assessing its progress towards self-sufficiency in key areas. Moreover, Calbright still lacks fully developed business processes. Specifically, more than two years after the effective date of the contract, Calbright has yet to complete key activities necessary for it to become self-sufficient. For instance, as of March 2021 Calbright continued to rely on Foundation staff to manage critical aspects of its payroll and accounting processes. In the absence of a clear work plan and timeline for the completion of these activities, Calbright cannot effectively demonstrate whether its continued reliance on the Foundation is expected or signals that it is behind in establishing its own operations. Further highlighting Calbright’s need to more deliberately plan to be independent from the Foundation are the additional costs it has paid for the Foundation’s services. In the early months of the college’s existence, the Foundation provided services that were essential for Calbright to operate and that Calbright was likely unable to perform on its own. However, in its contract, Calbright agreed to pay the Foundation an indirect cost fee equal to 10 percent of all direct costs for some of those services. For example, until July 2020, the Foundation paid Calbright’s employees on Calbright’s behalf, a service for which it charged the college an indirect cost fee equal to 10 percent of all of the salaries and benefits Although Calbright now pays it administered—about $655,000. Although Calbright now pays its its staff directly, the Foundation staff directly—eliminating the largest source of indirect costs—the continues to charge Calbright a Foundation continues to charge Calbright a 10 percent indirect cost 10 percent indirect cost fee for fee for other services. As of January 2021, Calbright had paid the other services. Foundation about $4.4 million—20 percent of which represented indirect cost fees. Calbright must establish clear expectations for the work that the Foundation will perform, including a timeline for becoming self-sufficient. Without a timeline for ending its reliance on the 18 California State Auditor Report 2020-104 May 2021 Foundation, Calbright risks remaining unable to support its own business operations and continuing to pay additional costs for services that it could perform itself. In early April 2021, when we asked Calbright’s current CEO for perspective, she stated that Calbright recognizes the importance of having a written work plan and fully intends to adopt such a plan in the next few months as it expands staffing capacity in the administrative and fiscal functions of Calbright. Specifically, she noted that Calbright recently hired a chief financial and administrative officer and obtained approval from the Board of Governors to hire additional staff who can help guide it through the transition going forward. Under Its Former Executive Team, Calbright Made Problematic Hiring Decisions and Paid Excessive Salaries Calbright’s former executive team failed to consistently use hiring processes designed to ensure fairness and that it was hiring the most qualified individuals to fill positions. Calbright hired multiple staff and contractors to start up the college. However, in our review we found that many of those individuals lacked experience in education or the public sector, which undermined Calbright’s ability to effectively start its operations. The former executive team—with the approval of the Board of Governors—also set excessive salaries for themselves, which in most cases made them the highest paid individuals among comparable positions in the community college system. Calbright’s new executive team has begun implementing new hiring processes and has reduced executive salaries. Calbright must continue these efforts to ensure fairness in its hiring process and the appropriate use of public funds. Calbright’s Former Executive Team Avoided Competitive Hiring Processes to the Detriment of the College Although Calbright’s hiring of qualified staff was crucial to developing its infrastructure and supporting its students, its former executive team made problematic hiring decisions and sometimes ignored competitive hiring processes, thereby failing to follow an appropriate process for building a team that could lead the college to success. The Legislature has declared its intent that all qualified individuals have a full and fair opportunity to compete for hiring in community college employment. Although Calbright, like all community college districts, generally has broad authority in employment matters where there is no specific provision in state law to the contrary, it must still adhere to sound management principles to ensure the fiscal stability of the district through the responsible stewardship of available resources, including following personnel practices that make the most effective use of available California State Auditor Report 2020-104 19 May 2021 human resources. Thus, we expected Calbright to follow an appropriate hiring process, such as that outlined by the California Department of Human Resources. To ensure that agencies hire effectively, the California Department of Human Resources guides agencies to identify the tasks necessary to accomplish their goals, the staff positions they need Key Steps in a Competitive Hiring Process to fill to perform those tasks, and the qualifications • Advertise an open position. that staff in those positions should have. In addition, • Consider all applicants. state hiring practices provide for a competitive hiring process—including the key steps in the • Evaluate and score applicants using consistent processes. text box—to ensure that public entities hire only • Conduct interviews using job-related criteria. the most qualified candidates, and to protect the • Select the most qualified available applicant for fairness of public hiring processes by preventing each position. favoritism. Finally, in January 2019 the Board Source: State law and regulation. of Governors established a policy requiring its approval of all full-time appointments to Calbright. Nonetheless, we found that the hiring decisions that Calbright’s former executive team made frequently diverged from sound hiring practices. When we reviewed 14 hiring decisions that the former executive team made in 2019 as it began to establish its organization, we found nine staff positions wherein the hiring process was problematic, including four of its leadership positions. For each of these nine problematic decisions, Calbright could not demonstrate that it conducted a full and fair competitive hiring process. In some of these nine cases, Calbright did not perform key steps in a competitive hiring process, such as advertising the position to solicit applicants or interviewing candidates. For example, Calbright’s former executive team created a position that it tailored to fit an individual that they wanted to hire. Calbright identified the need for an executive to oversee the division responsible for establishing employer partnerships. In April 2019, Calbright’s recruiter—an external contractor Calbright hired to recruit executive-level staff—suggested an individual for that role. According to email records, the individual did not want that executive role, so the Calbright executive team directed the executive recruiter to create a new position tailored to the individual’s preferred role and skill set; the new position did not include managing the division that Calbright had intended the executive position to oversee and instead focused on soliciting new employer partnerships for the college. Calbright’s recruiter then created the new position of senior vice president of partnerships, which Calbright had not previously planned to create. Although Calbright advertised the new position, we found no clear evidence demonstrating that Calbright interviewed any other candidates—or even that the individual herself interviewed for the position. Regardless, Calbright hired her into that new position in June 2019 and did not fill the original executive role—leaving Calbright with no one to manage the team of partnership staff that it had originally believed it needed. 20 California State Auditor Report 2020-104 May 2021 In others of the nine cases, even In others of the nine cases, even when Calbright did perform key when Calbright did perform key competitive hiring process steps, it undermined the purpose of the competitive hiring process steps, process by showing favoritism. For example, in Calbright’s initial it undermined the purpose of the months of operation, it relied on contractors to perform start-up process by showing favoritism. activities. Essentially, these contractors served in an acting capacity before Calbright posted a formal job opening. In some cases, evidence suggests that Calbright gave preferential treatment to candidates that it had first hired as its contractors—including candidates who received their contracts because of a past personal or professional connection to a member of the former executive team, which we discuss later in this chapter. In one such situation, Calbright executive leadership awarded a no-bid contract for services to a consultant in early 2019. Calbright then posted a full-time position for the same services about one month later, and although Calbright accepted applications from other candidates, the college had already negotiated and agreed to a salary amount with the original consultant and considered him as the finalist before conducting all the other candidate interviews. Although Calbright technically performed the actions necessary in a competitive hiring process, it planned to offer the position to the consultant rather than meaningfully considering all other candidates it planned to interview. Calbright’s choices for key team members also likely hindered its success in achieving the setup goals for the college and positioning it to achieve its mission. By December 2019, Calbright had hired 31 full-time employees and had also relied on key individuals as contractors to establish the infrastructure that would ensure that the students enrolling at the college would succeed. Among the 14 hiring decisions we reviewed and those of two key contractors, we found that almost all of the hired individuals lacked any experience in public education or, more broadly, in state or local government. Their experience had been predominantly in the private sector, such as in sales, healthcare, or social media. Although private sector candidates can be substantially qualified for positions in the public sector, the broader lack of public sector and public education experience across Calbright’s initial leadership team likely contributed to some of the significant problems that we describe in this report. For example, early in the setup of the college, Calbright contracted with one individual to create several functions related to serving its students, including leading the development of student support services, designing and implementing tutoring programs, and collaborating in the development of a program to transition students from learning programs to employment. However, that individual had no experience in state government, community college, or public educational institutions, and his background was primarily in sales and IT. By itself, the decision to contract with that individual to lead the development of these functions may not have been problematic. But his lack of experience in public education, coupled with the almost total California State Auditor Report 2020-104 21 May 2021 lack of public education experience across Calbright’s former leadership team, was likely a significant factor in the deficiencies that we found in Calbright’s ability to serve its students. All of the inappropriate hiring that we observed occurred under the leadership of the former executive team. Although five of the individuals Calbright inappropriately hired remain in their positions, four have left. Calbright’s new executive team has recently made improvements to its hiring process. In August 2020, Calbright hired an executive director of human resources (human resources director), who began developing formal policies and procedures for hiring that are consistent with state law and regulations. He explained that Calbright is in the process of formalizing approximately six policies of its plan to develop almost 100 human resources policies in total. The more recent hiring decisions that we reviewed have complied with key best practices that Calbright had not adhered to in the earlier problematic hires. To ensure that its future hiring efforts align with requirements and guidance, Calbright must finish developing formal policies, procedures, and processes for recruiting and hiring qualified staff. Calbright Paid Its Former Executive Team Salaries That Far Exceeded Those at Comparable Community Colleges State law authorizes community colleges to establish their own terms of employment, including salaries, with approval of its board of trustees—in Calbright’s case, the Board of Governors. As a community college, Calbright has a responsibility to employ sound management practices to ensure fiscal stability and to use its funding effectively and efficiently, which includes paying its employees reasonable salaries. One method of validating that its salaries are reasonable is ensuring that they are in line with salaries for comparable positions at other community college districts. However, Calbright paid some of its former executive team members salaries that far exceeded those at other community colleges. In fact, as Table 1 shows, four of the six Four of the six Calbright executives Calbright executives we tested earned more than any other person in we tested earned more than any a comparable position in the entire community college system. The other person in a comparable Board of Governors approved these high salaries despite the fact that position in the entire community Calbright did not present an analysis that justified the need for such college system. high salaries. In the absence of such an analysis, the president of the Board of Governors stated that the Board approved these salaries because Calbright argued that the people they were hiring were coming from the private sector and taking substantial pay cuts and because Calbright is a novel start-up organization. We do not question the difficulty of creating a new organization. However, as described earlier in this chapter, Calbright did not always hire people using a fair and competitive hiring process. That fact leaves open the possibility that it did not need to attract individuals from the private sector with higher- than-usual salaries to accomplish its mission. 22 California State Auditor Report 2020-104 May 2021 Table 1 Calbright Paid Its Former Executive Team Salaries That Were Substantially Higher Than Those Paid to Executives at Other California Community Colleges MEDIAN COMMUNITY HIGHEST COMMUNITY CALBRIGHT DIFFERENCE FROM COLLEGE SALARY COLLEGE SALARY SALARY* HIGHEST SALARY Chief Executive Officer $260,000 $352,000 $398,000 + $46,000 Chief Learning Officer 198,000 264,000 290,000 + 26,000 Chief Finance and Administrative Officer 188,000 250,000 235,000 – 15,000 Chief Culture and People Officer 173,000 258,000 250,000 – 8,000 Chief Success Officer 195,000 258,000 270,000 + 12,000 Chief Technology Officer 152,000 227,000 270,000 + 43,000 Source: Calbright payroll data and Association of California Community Colleges data on executive salaries at single-district community colleges. * All of the executives in these positions except for the chief technology officer left Calbright in 2020. Calbright has yet to fully address its excessive salaries. Most of its executive staff left in 2020, and the former president of the Board of Governors has since publicly agreed that the salary it paid to the former CEO was excessive. The salaries Calbright pays its new executives are comparable to those earned at other community colleges. However, Calbright’s original chief technology officer is still in his position, and his annual salary of $270,000 is nearly twice the median salary paid to his counterparts at other community colleges and about $43,000 higher than the next highest paid individual in a comparable position in the California community college system. According to Calbright’s new human resources director, the college has not yet established a pay schedule to ensure that its salaries for new employees will be reasonably consistent with salaries for comparable positions in the California community college system. The current CEO stated Calbright is relying upon Association of California Community College Administrators data to anchor its salary offers relative to comparable community college system positions. When we asked why Calbright has not yet established a salary schedule, the current CEO stated that Calbright is prioritizing negotiations with its faculty bargaining unit to determine instructional salary schedules, and that once the board approves its instructional schedules, it will adopt a formalized administrative salary schedule. Until it formalizes a salary schedule, Calbright cannot adequately ensure that it is compensating its staff in a fair and reasonable manner and thus is safeguarding public funds. California State Auditor Report 2020-104 23 May 2021 Calbright’s Contracting Processes Lack Sufficient Safeguards to Protect Public Funds and Prevent Favoritism During its first two years, Calbright relied heavily on contracts with consultants and other vendors to staff the college and conduct start-up activities. However, it did not first establish important safeguards to ensure that its contracting process aligned with state law and regulation. Its deficient contracting processes resulted in problems with its contracts, including overly vague descriptions of the work it expected from contractors. Additionally, its former executive team failed to follow competitive bidding processes to ensure that Calbright obtained the best value for public funds, even when state law generally required it to do so. In some cases, contract awards appeared to have been influenced by professional or personal connections between the contractor and Calbright staff. Calbright Must Establish Stronger Oversight Over Its Contracting Processes Since its inception, Calbright has entered a significant number of contracts to obtain staff and technological systems for starting and operating the college, among other things. Calbright used contractors to perform a significant amount of the initial work to set up the college, and as of December 2020, Calbright had As of December 2020, Calbright entered into contracts for goods and services totaling about had entered into contracts for $30 million. In light of its heavy reliance on contractors, the goods and services totaling about strength of Calbright’s contract management is an important lens $30 million. through which to assess its performance to date. Sufficient contract oversight and management necessarily includes developing policies and procedures that align with state law and regulation, verifying that contractors are adequately qualified and that their costs are reasonable, and hiring staff who have knowledge and experience in public sector or community college contracting practices to oversee the contracting process. Despite the degree to which its success has depended on adequate contractor performance, Calbright has not ensured that it has critical contract oversight and management systems. For example, Calbright staff attested that its undocumented process relies on the contract requester to validate key issues of cost and qualifications. Managers within its departments identify vendors and then submit requests for contracts with those vendors. Although a director and an executive at Calbright must approve each contract, Calbright’s director of finance and its former chief financial and administrative officer confirmed that those reviewers have not assessed whether the contractors are qualified. Further, Calbright’s business operations manager explained that Calbright does not require staff to provide evidence of vendor qualifications when asking for contract approval. The former chief financial and administrative 24 California State Auditor Report 2020-104 May 2021 officer stated that his primary concern when approving new contracts was whether the contract amount exceeded the competitive bidding threshold and ensuring both that the contract language did not create any unnecessary obligations or liabilities for Calbright and that there were no legal or compliance issues with the contract. Similarly, the contract request form that Calbright management reviews as a part of the contract approval process does not require staff to indicate how they assessed the reasonableness of contractor pricing, leaving management with no direct assurance that staff have considered whether the same services are available at a lower price. Without effective processes for verifying that contractors are sufficiently qualified and that their costs are reasonable, Calbright risks spending more than necessary for goods and services or not receiving the goods and services it needs. In the early stages of its launch, it may have been possible for Calbright to adequately compensate for its lack of policies and procedures by employing staff with sufficient contracting experience to guide its efforts. However, one of the two employees Calbright enlisted to manage its contracting—its business operations manager, who was still in that position as of March 2021—had no prior experience in public sector or community college contracting. Nonetheless, she reported to us that one of her key responsibilities is drafting Calbright’s contracts and routing them for signature. The other employee Calbright retained to manage its contracting was an acting procurement director who possesses extensive experience in procurement and contract management in the public sector. However, the college contracted with her to manage only those contracts that were required to go through a competitive bidding process when the executive team so requested and not to oversee contracts not required to go through a competitive bidding process. The majority of Calbright’s contracts fall below or at the competitive bidding threshold, and therefore the acting procurement director had little involvement in most of Key Requirements for State Contracts Calbright’s contracts, hindering her ability to ensure consistent compliance with relevant state laws or • Clear and concise descriptions of the work to be adherence to best practices. In April 2021, after we performed by the contractor. expressed our concerns about its lack of experienced • Clear, measurable deliverables. procurement staff, Calbright indicated that it plans to • Specific due dates for completion of the work. hire a permanent procurement coordinator and provided us a draft job description for that position. • The maximum amount to be paid to the contractor. Source: State Contracting Manual. Calbright’s inadequate contracting safeguards have resulted in problems with some of its contracts. The State Contracting Manual requires that state contracts contain certain elements to ensure proper and efficient use of funds, some of which we list in the text box. As a community college district, Calbright is not required to comply with those California State Auditor Report 2020-104 25 May 2021 requirements. But because the State Contracting Manual provides guidelines, or safeguards, to promote sound business decisions and practices in securing necessary services, it is a source of contracting best practices that Calbright should reasonably follow. However, Calbright has not ensured that the contracts that it has entered include these safeguards. For instance, we found that 12 of Calbright’s contracts had significantly vague descriptions of the work that it expected from the contractors. In one of these cases, the contract for a consultant acting in a key leadership position had a scope of work consisting of a single sentence—“Role is to help document overall strategy for Success Organization.” This scope of work included no specific, measurable deliverables and no timeline for completion of the consultant’s work. In the absence of specific expectations for the services that a contractor will provide and a specific time frame for when the contractor will provide those services, Calbright cannot effectively ensure that the contractor will satisfactorily provide those services, increasing the risk that it will pay for work that does not meet its needs. Because Calbright entered the contracts we describe above in 2019, we also reviewed three contracts that Calbright entered in 2020 specifically to see if they included clearer and more specific scopes of work. Those three contracts included more specific scopes of work that will allow it to better ensure that it receives the goods and services that it needs. However, we are still concerned that We are still concerned that Calbright’s lack of policies, procedures, and adequate staffing risks Calbright’s lack of policies, mismanagement of state funds. Calbright’s former chief finance and procedures, and adequate administration officer stated that during the approximately one year staffing risks mismanagement that he worked at Calbright, developing policies and procedures of state funds. for procurement was originally a priority but that it became overshadowed by other priorities. Calbright’s current director of finance, who joined Calbright in April 2020, stated that she has not developed such policies and procedures because she has been busy addressing other priorities. She stated that she plans to do so under the direction of Calbright’s new chief finance and administration officer, who joined Calbright in February 2021. Calbright contracted for staff support services in February 2021 and although the contract does not specifically state that the contractor will assist in the development of procurement policies and procedures, Calbright’s new chief finance and administration officer asserted that providing such assistance would be a focus of that contractor in the near term. Given the significant problems that we have identified with the college’s contracting practices—more of which we describe in the following section—it is critical that Calbright prioritize establishing strong oversight of its contracting, including developing processes, policies, and procedures that ensure that its contracts comply with state law and provide appropriate value. 26 California State Auditor Report 2020-104 May 2021 Calbright’s Weak Contracting Processes Led to Inappropriate Contracting Activity The weaknesses in Calbright’s contracting processes that we describe in the previous section contributed to its failure to follow contracting requirements and best practices. State law establishes public contracting requirements that are designed to eliminate favoritism, fraud, and corruption and to ensure that public Community College Competitive entities obtain the best value for public funds. For example, Bidding Thresholds by Calendar Year state law generally requires public entities to solicit bids from multiple vendors. There are some exceptions to this 2018: $90,200 requirement, including an exception that community colleges 2019: $92,600 may execute contracts without competitive bidding if the amount is below a certain threshold. State law also requires 2020: $95,200 the Board of Governors to adjust this threshold annually Source: Chancellor’s Office annual memoranda based on economic indicators, as the text box shows. establishing competitive bidding thresholds. State law gives the authority to enter into contracts to a community college’s board of trustees, but permits the boards to delegate that authority. The Board of Governors, in its role as Calbright’s board of trustees, gave Calbright’s CEO position the authority to approve contracts worth up to $100,000. However, as we noted in the previous section, Calbright lacks procurement policies and adequate oversight over its contracting activity. As a result, Calbright’s procurement activity has not always aligned with best practices and requirements. Calbright did not always follow contracting requirements or best practices by using a competitive process or documenting that it appropriately used an exemption from competition when selecting contractors. Specifically, of the 20 contracts that Calbright entered between July 2018 and December 2020 that exceeded the competitive bidding thresholds and would have required competitive bidding, Calbright awarded 11 consulting contracts totaling a combined $1.3 million without soliciting competitive bids. Each of these contracts exceeded the required threshold for competitive bidding, although only two of the contracts exceeded that threshold by more than $2,500. Nonetheless, Calbright was generally required to solicit bids before awarding the contracts. Instead of soliciting bids, Calbright included a reference in the contract to an exception in state law that allows community colleges to forego the competitive bidding process if a contract is for specialized services with a consultant who is specially trained, experienced, and competent to perform the services required. However, Calbright could not provide documentation that it sufficiently evaluated the contractors in question to ensure that they possessed the specialized training or experience necessary to appropriately apply the exception. In other words, for each of these California State Auditor Report 2020-104 27 May 2021 11 contracts Calbright could not demonstrate that it had assessed that the competitive bidding exceptions applied and that it had determined the consultants were specialized experts. Therefore, we question what special expertise Calbright determined they possessed that warranted the use of this exception to competitive bidding. The former chief financial and administrative officer indicated that Calbright executives used short-term, noncompetitively bid contracts for staff as the quickest way to staff up the college. Such an approach was likely reasonable for quickly staffing the college and for wanting to avoid a lengthy competitive bidding process. However, that explanation still does not resolve why Calbright did not document how it assessed each contractor’s qualifications. In the absence of a formal contracting process, Calbright’s decision to contract with certain consultants appears to have been motivated in part by the professional or personal relationship between Calbright staff and the consultants. We found nine contracts, We found nine contracts, totaling totaling $1.1 million in value, wherein a key reason Calbright $1.1 million in value, wherein a key identified and hired the consultant was that they were known to the reason Calbright identified and staff. Five of these contracts are among the 11 we describe above as hired the consultant was that they contracts that Calbright did not competitively bid and another four were known to the staff. were valued at or below the competitive bidding threshold. Similar to the previously described contracts, there was no evidence for these four contracts that Calbright ever considered other contractors or evaluated whether the contract pricing was reasonable. All but one of these contracts had a maximum compensation amount below $100,000 and therefore were not subject to approval by the Board of Governors. The contract that did require board approval contained itemized costs totaling about $376,000 for executive recruiting services, and the contractor was a previous associate of the former CEO and the former chief operating officer. Despite the significant dollar amount of the contract and the lack of competitive bidding, the Board of Governors approved the contract. Calbright also paid four contractors more than the original contracted amount by shifting the contractors to additional agreements with the Foundation. As we discuss in the Introduction, Calbright has contracted with the Foundation to assist with key start-up functions. Calbright staff sometimes had the Foundation enter into contracts with Calbright’s consultants, and the Foundation would then include the payments that it made for those contracts in the monthly invoices that it sent to Calbright under its own contract and apply an additional 10 percent indirect cost charge. In May 2019 Calbright transferred four consultants to the Foundation through this means. All four of these consultants were among the nine contracts that Calbright had not competitively bid or informally assessed qualifications and cost. In three of the four cases, Calbright 28 California State Auditor Report 2020-104 May 2021 had exhausted or was about to exhaust the value of the original contract right before transferring the contract to the Foundation. The value of the four consultants’ new contracts with the Foundation totaled $345,000, bringing the combined value of these four consultants’ contracts with Calbright and the Foundation to more than $725,000. Because Calbright had not accurately anticipated its need for these services, it paid for them using this two-contract approach rather than competitively bidding for the contracts or presenting them to the Board of Governors for approval. There was overlap between the scope of work for some of these contractors and the work that Calbright had contracted with the Foundation to perform. However, regardless of whether Calbright’s consultants—whose contracts had not been competitively sourced—were fulfilling similar functions, transferring these agreements meant that Calbright could continue to pay consultants for services without being directly accountable for the payments. Finally, one of the four contracts—for executive coaching services—was not clearly within the scope of its agreement with the Foundation because that agreement did not call for the Foundation to provide executive coaching services. The Foundation’s general counsel asserted that before entering these contracts, Foundation executives requested that Calbright seek approval from the Board of Governors. Calbright’s legal counsel asserted that she received verbal approval for two of the four transfers from the board’s president. However, she could not provide evidence that she had sought or received such approval for the remaining two. Recommendations To provide greater accountability regarding its spending and to ensure that it effectively uses the public funds it receives to accomplish the goals for which it was created, Calbright should do the following: • By November 2021, incorporate into its implementation plan a spending plan that details how and when it expects to spend the funds the Legislature allocates to it. At a minimum, the spending plan should identify the estimated costs to accomplish the tasks set forth in its implementation plan and a timeline for when it expects to incur those costs. The spending plan should also describe Calbright’s strategy for staying within its budget while completing necessary start-up activities and achieving its milestones. California State Auditor Report 2020-104 29 May 2021 • Calbright should annually review the spending plan and make adjustments as necessary. It should also annually report to the Board of Governors on its spending to date and explain how its spending has furthered its progress in achieving its mission. To ensure that its hiring process is fair and results in the hiring of well-qualified staff, Calbright should, by November 2021, finalize its development of human resources policies and procedures for recruitment and hiring that comply with state law and regulation. To ensure that it uses state resources responsibly, Calbright should do the following: • Immediately commence the process of hiring a procurement director with significant experience in public sector procurement. • By July 2021, complete the development of a fully functioning procurement process that aligns with state law, regulations, and to the extent practicable, the State Contracting Manual. That process should include policies and procedures that ensure that all of Calbright’s contracts provide clear expectations of the work that contractors will perform. It should further include strong contract management processes for ensuring that contractors perform that work satisfactorily before receiving payment. • By August 2021, provide training on relevant laws, policies, and procedures related to procurement to all staff involved in approving or managing contracts. To ensure that the compensation it provides its employees is reasonable, Calbright should establish a pay schedule for all employees by November 2021 that does the following: • Includes salary, benefits, and all other forms of compensation. • Establishes compensation packages that are comparable to those for similar positions within the community college system. 30 California State Auditor Report 2020-104 May 2021 Blank page inserted for reproduction purposes only. California State Auditor Report 2020-104 31 May 2021 Chapter 2 CALBRIGHT MUST TAKE IMMEDIATE ACTION TO ENROLL, EDUCATE, AND HELP SECURE JOBS FOR ITS TARGET STUDENT POPULATION Chapter Summary The Legislature created Calbright to provide working adults with access to flexible postsecondary education that will position them to obtain well-paying jobs. However, as Figure 3 shows, Calbright has failed to take critical steps necessary to achieve that mission. It has yet to adopt sufficient processes for selecting the educational programs it offers, and it has not worked with employers to ensure that the programs it selects adequately prepare its target student population to obtain jobs. One program, its cybersecurity program—one of the initial three programs it offered to students—largely benefits those who already possess a bachelor’s degree and Calbright could not demonstrate how it chose that program. Calbright further lacks clear strategies for reaching its target student population, and its enrollment of some of these demographic groups lags behind that of other community colleges. In addition, Calbright has neither established a plan for helping its students obtain jobs after graduation nor has it tracked whether its programs are effective in helping graduates secure jobs. Few students have graduated, and the majority of those who have enrolled in Calbright programs have either dropped out or failed to progress. This is likely due in part to delays in Calbright’s establishment of a system for providing support to its students and to its continued lack of a method for monitoring to ensure that students receive support. Calbright Has Not Ensured That All Three of Its Educational Programs Can Meet the Needs of Its Target Student Population To fulfill its purpose, Calbright must offer educational programs that enable working adults who lack access to a traditional college education to obtain well-paying jobs after graduation. The educational programs that Calbright selects should provide job market value for its target student population, meaning that the programs should lead to credentials that employers will value, and it should prepare graduates for jobs with good pay in high-demand fields. Because Calbright was established to serve students who do not have college degrees, those credentials should qualify its graduates for jobs that do not require a college degree. In fact, state law required that Calbright develop three programs within its first three years that would exclusively serve students who were not accessing postsecondary education or who did not have an industry valued credential. 32 California State Auditor Report 2020-104 May 2021 Figure 3 Calbright Will Not Fulfill Its Mission Unless It Improves In Three Key Areas 1 | Enroll more of its target population Calbright collects insufficient data to know whether its students are: Low-income ? ? ? ? ? Veterans Immigrants Having difficulty accessing a traditional community college But the data it collects show that it is not adequately reaching its target student population. Calbright 93% CA Community College System 73% 54% 46% 46% 32% HISPANIC WOMEN NO BACHELOR’S OR LATINO DEGREE 2 | Increase student success Out of 904 enrolled students . . . 384 87 Only 12 STUDENTS STUDENTS HAVE BEEN STUDENTS DROPPED OUT INACTIVE FOR AT LEAST 90 DAYS GRADUATED 3 | Provide path to employment Calbright has not adequately: Worked with employers to ensure that its educational programs prepare students for jobs. Monitored whether its students get jobs in their chosen field after they graduate. Source: Analysis of Calbright data as of October 2020 from its case management and learner management systems, and review of Calbright records of its efforts to develop partnerships with employers. California State Auditor Report 2020-104 33 May 2021 However, Calbright’s decisions regarding the programs it offers have not adequately reflected an awareness of the needs of its target population. Each of Calbright’s three initial educational programs— medical coding, IT support, and cybersecurity—leads to credentials for occupations with good wages and a growing number of job openings. Nonetheless, as Figure 4 shows, the cybersecurity program is unlikely to help individuals from a significant segment of Calbright’s target population obtain employment because jobs in cybersecurity generally require a bachelor’s degree or significant experience in IT. In fact, an Employment Development Department (EDD) analysis from February 2020 found that 87 percent of projected job openings in cybersecurity require a bachelor’s degree or higher. The cybersecurity credential earned by Calbright students is likely best suited for established IT professionals trying to advance their careers, which may explain why almost half of students enrolled in the cybersecurity program already have a bachelor’s degree. Calbright’s interim vice president of learning and instruction Calbright’s interim vice president could not provide documentation demonstrating why the college of learning and instruction could selected the cybersecurity pathway but stated that it did so at the not provide documentation direction of Calbright’s former CEO. The interim vice president demonstrating why the college for workforce, strategy, and innovation stated that Calbright has selected the cybersecurity pathway done an analysis showing that its cybersecurity program can help but stated that it did so at the students obtain jobs, including those without bachelor’s degrees. direction of Calbright’s former CEO. However, the analyses Calbright provided do not adequately support this claim. Calbright’s current CEO additionally noted that state law also charges Calbright with identifying opportunities for stackable credentials—a sequence of credentials that an individual can accumulate and that move him or her along a career pathway or up a career ladder. The CEO expressed that the cybersecurity pathway aligned with this portion of Calbright’s founding statute. However, Calbright had no documentation showing that among its target population this stackable credential was preferred and should have been chosen above other stackable credentials. Therefore, although the nature of the program may fit within the broader scope of Calbright’s mission, we question how Calbright determined it was the most appropriate offering to provide when it opened the college. As we describe in the Introduction, the Board of Governors is responsible for developing plans for the growth and development of Calbright’s educational programs. Given the importance of selecting effective educational program offerings for Calbright to fulfill its purpose, we expected that the Board of Governors would approve only those educational programs that would benefit Calbright’s target population by effectively preparing them for jobs. However, the board’s approval of the cybersecurity program suggests that it was too deferential to Calbright’s staff and thus it harmed Calbright’s ability to most effectively serve its target student population. 34 California State Auditor Report 2020-104 May 2021 Figure 4 Calbright Has Not Ensured That Its Educational Programs Are Available and Beneficial to Its Target Student Population IT Support AVAILABILITY: Enrolling new students JOB PROSPECTS: Good Cybersecurity AVAILABILITY: Enrolling new students JOB PROSPECTS: Most jobs require a bachelor’s degree or significant experience in IT Medical Coding AVAILABILITY: Closed to new students as of July 2020 JOB PROSPECTS: Good Source: Analysis of Calbright’s three educational programs and EDD’s February 2020 analysis of the job marketability of Calbright’s educational programs. In contrast, EDD’s job market analysis shows that Calbright’s medical coding program offers job market value to its students and that students can get jobs in medical coding without a degree. However, Calbright limited the number of students who can participate in that program to only 80 students, compared to 300 students in each of its other two programs. High demand caused these 80 seats to be almost completely filled by July 2020, while only 17 percent of the 300 spaces in the cybersecurity program were filled by the end of Calbright’s first year of operation. Instead of making more seats available in the medical coding program, Calbright stopped accepting new students. That decision likely harmed its ability to reach part of its target population. The majority of students who enrolled in the medical coding program were women—80 percent, compared to 40 percent in the IT program and 30 percent in the cybersecurity program. Calbright’s interim vice president of learning and instruction indicated one reason it closed the medical coding program was the high cost of offering it. As of March 2021, Calbright was free to students, and it pays fees for instructional materials, textbooks, and California State Auditor Report 2020-104 35 May 2021 examinations for each student it enrolls. These fees totaled more than $1,800 per student for the medical coding program, which is much higher than the per-student cost of about $420 for the IT support program and $380 for the cybersecurity program. However, before selecting its programs Calbright should have determined whether cost would prohibit a program’s expansion. It could then have either planned to invest more funding in that program or selected a more cost-effective alternative depending on its assessment of the program’s likely appeal and potential benefit to its target population. Calbright’s ineffective choices regarding its educational programs reflect its failure to adequately Key Issues Calbright Should Have Considered plan. As the text box shows, there are several key When Selecting Its Educational Programs considerations for selecting educational programs. • Current and future job openings. We expected that Calbright would have developed a process for selecting the educational programs to • Graduates’ ability to earn higher wages. offer. As of January 2021, Calbright’s current CEO • Graduates’ ability to obtain a job without a confirmed that it had not yet documented a process college degree. for selecting new educational programs. Although • Demand for the program among target student Calbright provided some documentation that it had population. considered projections for job openings and wages • Feedback from employers on the skills they desire. when selecting its first three program offerings, as of • Ability to offer the course fully online and self-paced. March 2021, Calbright staff could provide little other evidence of how or why it chose them. • Cost of administering the program. Source: Analysis of state law. Additionally, Calbright could not show that it met a key requirement in state law: that it worked with employers and industry groups to design its first three programs. Calbright repeatedly indicated in both its August 2019 and July 2020 reports to the Legislature that it had or would build partnerships with employers so that employer needs could inform the development of its educational programs and services. While the interim vice president of learning and instruction stated that Calbright had multiple informal discussions with employers when designing its programs, she could only provide documentation of one meeting related to the medical coding pathway. A single meeting falls short of the collaboration that the Legislature required and represents a missed opportunity to more fully inform the selection of pathways. Although not expressly required by Calbright’s founding statute, it would also be in Calbright’s best interest to establish and maintain ongoing partnerships with employers. However, the interim vice president of learning and instruction indicated that it has not developed any long-term partnerships with employers for its existing three programs. She did provide some evidence that a small number of representatives from certain companies had spoken at 36 California State Auditor Report 2020-104 May 2021 informational meetings with Calbright students and stated that such meetings would help the college to begin developing relationships with those companies. Calbright has taken some steps toward developing relationships with employers to develop its new programs. The Timeline for Calbright’s Development of 16 Programs State law establishes a timeline for Calbright’s development of a minimum number of programs, as the text box shows. July 2019: 3 new programs Calbright selected its first three program offerings by the July 2021: 3 new programs July 2019 deadline, and it is required to develop an additional July 2023: 10 new programs three programs by July 2021. As of March 2021, Calbright had developed two new programs. One is in customer Source: State law. relationship management platform administration, which prepares students for a credential in the use of a specific software. For this program, Calbright established one employer partnership in January 2021 with a technology company that may provide job opportunities to students who enroll in this new pathway. The second program that Calbright developed is the first in what Calbright describes as a planned series related to trainings for healthcare workers on certain skills, such as interpersonal skills. However, that program does not actually lead to an industry-recognized credential, which presents concerns about how closely aligned it is with Calbright’s purpose. To provide its students with valuable educational opportunities, Calbright must adopt an effective process for developing new programs that will benefit and provide job market value to its target population. Calbright’s Inadequate Outreach Strategies Have Hindered Its Ability to Enroll the Students It Was Created to Serve Calbright can only achieve its mission if it can successfully enroll individuals from its target student Calbright does not collect data to know whether its students are . . . population. As we describe in the Introduction, this population consists of working adults aged 25 to 34 who • Having difficulty accessing traditional do not have college degrees as well as other specific community colleges groups the law identifies as being able to benefit from • Employed* Calbright’s programs and which Calbright identified • Responsible for the care of dependent in strategic documents are priority groups. Groups in children or adults Calbright’s target population include women, Hispanic • Low-income or Latino individuals, veterans, immigrants, and the • Veterans formerly incarcerated. • Immigrants • Incarcerated or formerly incarcerated However, Calbright does not know how successful it has been in enrolling its target student population because it Source: Interviews with Calbright staff. * Calbright began collecting employment does not collect sufficient data to make such an information from new students in February 2021. assessment. The text box shows key pieces of information that Calbright needs in order to determine whether its actual student population aligns with its target student California State Auditor Report 2020-104 37 May 2021 population. Calbright’s vice president of strategic initiatives explained that it does not collect these data because the California Community Colleges application that Calbright uses does not ask applicants to provide it. However, we are skeptical of this explanation as the California Community College application allows schools to add supplemental questions to the application. During our audit, in February 2021, Calbright added two supplemental questions to its application: one asking students how many jobs they have, and one asking students how many hours per week they currently work. However, Calbright has not begun collecting the other critical pieces of information the text box lists. Because it has not collected this information, Calbright cannot assess whether it is reaching its target student population. Further, the absence of data impairs both the public and the Board of Governors from holding Calbright accountable for serving the students for whom it was created. Calbright’s actual success in enrolling its target population is mixed. To review the student population, Findings From Our Survey we combined the limited information that Calbright of Calbright Students does collect with our own survey of Calbright • 65% were employed. students.3 Data from our survey—which is included in the text box—suggests that Calbright has successfully • 46% were responsible for the care of dependent enrolled students from certain groups within its target children or adults. population, including those who work or have low • 58% had a household income of below $60,000. incomes. The data that Calbright collects showed that • Fewer than 10% were veterans. in its first year of enrollment, more than 90 percent • 25% were born outside the U.S. of its students were persons 25 years of age or older. • Fewer than 10% were incarcerated or formerly Calbright has also done a substantially better job of incarcerated. enrolling African American students than has the rest of the community college system. Although this Source: Analysis of survey data. group is not specifically named in state law as part of Calbright’s target student population, Calbright’s success in this area is nonetheless notable. However, the demographic information Calbright collects indicates that it is failing to reach other key groups from its target student population. One of those key groups is students who do not have a college degree; in fact, a significant percentage of students enrolled in its first year—more than 25 percent—had already earned at least a bachelor’s degree. Further, although the Legislature found that women or Hispanic individuals were two major groups that would benefit from the flexible educational opportunities that Calbright is intended to provide, the proportions of students that it enrolled in its first year who were women and Hispanic or 3 We surveyed the 492 students who were enrolled at Calbright as of December 30, 2020. Of those surveyed, 95 students responded. 38 California State Auditor Report 2020-104 May 2021 Latino were significantly lower than the proportions enrolled by the rest of the California Community College system. Calbright’s senior vice president of strategic initiatives acknowledged that the college’s outreach efforts were not very targeted during its initial enrollment period and that Calbright has since conducted marketing research and begun doing more targeted advertising. She also stated that Calbright has made adjustments to its marketing to be more female focused, such as including images of women in IT roles. Calbright’s failure to enroll its target population is likely due in large part to its failure to develop and implement an effective outreach plan. To be effective, such a plan should include specific strategies for reaching the different groups within Calbright’s target population as well as a process for monitoring whether it is successfully reaching those groups. State law required Calbright to develop a statewide outreach plan by July 2019. Although Calbright included a section it called its outreach plan in its August 2019 report to the Legislature, that plan is flawed. The plan provides a list of advertising options Calbright asserted that it would pursue, but it does not identify how these numerous approaches will work together to achieve specific outreach goals. It also does not articulate detailed strategies for reaching Calbright’s target population or describe how Calbright will determine whether its outreach efforts are resulting in students from its target population applying to and enrolling at the college. For example, state law specifies that Calbright’s outreach plan should include partnering with community-based organizations and immigrant groups to conduct outreach. However, Calbright’s outreach plan is inadequate in this area. Its plan for working with these groups mostly consists of broad descriptions of the types of organizations it intends to partner with, such as “organizations working with communities of color” and “organizations working to close equity and accessibility gaps for women and those with disabilities.” Calbright then stated in the plan that it would work with these organizations to organize information sessions and distribute materials. A plan this broad, Planned Outreach Strategies when compared to a strategy for partnering with That Calbright Had Not Implemented specific organizations to achieve measurable goals, increases the risk that Calbright’s outreach will fail to • Working with employers that have employees who reach the potential students it was meant to reach. could benefit from Calbright. • Collaborating with community-based organizations. The shortcomings of the outreach plan have been Source: Analysis of Calbright’s outreach plan and activities exacerbated by Calbright’s failure to take the actions as of December 2020. it describes in the plan that likely would have been beneficial. The text box lists these unimplemented activities. Calbright’s director of outreach confirmed that Calbright has not actually implemented much of the outreach plan and has done little to create partnerships with community-based organizations for two reasons. First, the current outreach team— which did not develop the plan—disagrees with some of its elements. California State Auditor Report 2020-104 39 May 2021 For example, he explained that certain forms of outreach, such as outdoor and transit ads, are expensive and not targeted to the population Calbright was created to serve. Second, Calbright’s outreach approach has been limited by the COVID-19 pandemic; as an example, it could not conduct marketing events to build relationships with community-based organizations. According to the director of outreach, the college has not developed an updated outreach plan because the executive position responsible for doing so, the vice president of external affairs, marketing, and communications, had been vacant since February 2020. Calbright filled the position in January 2021. He stated that Calbright’s statewide approach to outreach has consisted of digital advertising, and that its outreach efforts have been reactive to the global pandemic and the college’s evolving needs for student enrollment rather than reflective of a long-term plan. Despite the challenges Calbright may face in implementing an updated outreach plan that aligns with best practices, it is vital that it do so. If it does not, it risks enrolling students it was never meant to serve while leaving those it was supposed to benefit unaware of or The senior vice president of uninterested in its programs. The senior vice president of strategic strategic initiatives agreed that initiatives agreed that Calbright’s outreach efforts need more Calbright’s outreach efforts need attention and focus. more attention and focus. Calbright Lacks an Effective System for Ensuring That Its Students Receive the Support They Need to Progress In addition to financial, family, and work barriers to accessing education that arise from the personal circumstances of adult students, research shows that they may have difficulty navigating the educational system and that they may struggle with emotional hurdles, such as fear of failure. Taking courses online also presents challenges for these students. A January 2019 survey of students from 10 community colleges across the country found that two of the primary obstacles students faced when taking online classes were difficulty learning the material on their own and lack of interaction with faculty. The American Council on Education and the PPIC report that to assist both adult students and students in online courses, colleges must ensure strong connections and frequent interaction between students and faculty.4 State law establishes that Calbright’s guiding principles include establishing innovative student support methodologies and addressing barriers its students face to accessing postsecondary education. 4 The American Council on Education is a national organization representing more than 1,700 higher education institutions. 40 California State Auditor Report 2020-104 May 2021 Nonetheless, in its first year of operation, Calbright lacked a robust system of support services for its students. State law required it to define its duties for instructional support and map key aspects, including the instructional experience, by July 2019. We expected these to include a detailed plan for how to support its students in preparation for enrolling its first students later that year, but Calbright did not develop such a plan before opening enrollment. Calbright began enrolling students in October 2019 and hired several instructors and academic counselors. However, for the following year, although Calbright had limited expectations and guidance for when or how often its instructors and academic counselors should interact with students to provide support, it lacked any method for identifying when students were struggling or for centrally tracking the support that instructors and counselors provided to them. In the absence of a well-designed In the absence of a well-designed support system, many Calbright support system, many Calbright students have not progressed. In our January 2021 survey of enrolled students have not progressed. students, the majority reported that they were satisfied with the support that the college’s instructors and counselors had provided. However, less than 20 percent of Calbright’s enrolled students responded to our survey, and the students’ actual progress is a more accurate measure of Calbright’s success at supporting its students. After Calbright’s first year of instruction, the majority of students who had enrolled had either dropped out or stopped progressing in their programs. Although a variety of reasons could explain why students had dropped out or become inactive, Calbright cannot adequately demonstrate that its instructors or counselors offered and provided support to those students to help them progress. We expected that Calbright would be tracking communications between its instructors and counselors and its students to better ensure that the students are receiving the support they need, especially when they fail to progress. Calbright has many channels through which instructors and counselors may contact students, including through its learner management system, phone calls, text messages, and virtual meetings. However, the chief technology officer acknowledged that Calbright has no fully centralized method of tracking when instructors and counselors interact with students across all of these channels. He stated that Calbright does not centrally track certain channels of communication unless instructors and counselors manually record those interactions within its case management system. For example, if a counselor does not manually record in the case management system that he or she spoke to a student on the phone and does not schedule the call using Calbright’s meeting invite software, Calbright has no record of that interaction. The records of student interactions with instructors and counselors that Calbright does maintain indicate that the level of support that students have received from instructors and counselors has California State Auditor Report 2020-104 41 May 2021 varied. We reviewed the available records of the interactions that a selection of 10 students had with instructors and counselors through Calbright’s system for case management from October 2019 through the end of September 2020 and through its system for learner management from October 2019 through November 2020. These records included messages sent through Calbright’s systems for case management and learner management and records of phone calls, texts, and virtual meetings recorded in those systems. The records showed that the number of interactions that counselors and instructors had with these students differed. For example, one student enrolled in July 2020 interacted with Calbright staff multiple times in the first three weeks of her enrollment. In contrast, two of the 10 students had no direct interaction with a counselor or instructor over the approximately nine months that they were each enrolled, and Calbright dropped them for inactivity. The absence of any record of attempted outreach by instructors or counselors to those students indicates that it dropped them without attempting to encourage their continued participation in the college. The number of times that an instructor or counselor interacts with a student is not the only measure of the adequacy of support that a student receives nor will every student require frequent interactions with instructors or counselors. But given that best practices for educating Calbright’s target student population state that these interactions should be frequent and the connections to faculty should be strong, Calbright should ensure that it provides that kind of support to its students, especially when students do not progress, as many of Calbright’s students have not. In October 2020, Calbright completed a plan describing the process that it would use to provide support to its students. The plan describes an onboarding process that includes frequent interactions with counselors. It also includes triggers for when Calbright will reach out and provide support to students if they fall behind in their program. However, If Calbright does not monitor its the plan still does not establish a monitoring process to better ensure staff’s adherence to its plan for that students actually receive the support that the plan prescribes. supporting students, it will lack If Calbright does not monitor its staff’s adherence to the plan, it will assurance that it is supporting them lack assurance that it is supporting students in the way it intends. in the way it intends. In addition, tracking instructors’ and counselors’ interactions with students would enable Calbright to evaluate the effectiveness of the support that it provides, both for the benefit of its own program improvements and the broader community college system. State law establishes expectations that Calbright develop and implement innovative student support methodologies, including leveraging student data to support students’ educational and career goals. State law further encourages Calbright to enhance systemwide student success efforts by sharing its best practices with the rest of the community college system. Calbright’s learner management system 42 California State Auditor Report 2020-104 May 2021 records students’ activity in their educational programs, the progress they make, and their success or failure in passing their assessments. If Calbright partnered these data with existing data on its student support activities from its case management system, it could evaluate whether the types of support it provides help students to make progress in their programs, and it could share its results to benefit the broader community college system. By developing a plan for providing student support that lacks a means of ensuring that the support works, Calbright risks failing to identify when its methods are ineffective. As a result, its students may not receive the support they really need to succeed and complete their programs. It also calls into question Calbright’s ability to support the broader community college system through the identification of effective online student support strategies for adult students as the Legislature intended. Calbright Should Not Require Students to Take Coursework on Subjects They Have Already Mastered Best practices state that to assist adult students in completing their education, colleges should give them academic credit for previous learning or experience relevant to the program in which they are Providing credit for prior learning is enrolled. Providing credit for this learning or experience is referred to an important means of facilitating as credit for prior learning and generally involves a college conducting increased education, especially an assessment of each student’s existing knowledge and skills. for adult learners coming from Students may earn such credit for knowledge and skills attained the workplace or military. outside of college through, for example, prior apprenticeships, military service, or relevant professional experience. According to the American Council on Education, credit for prior learning is an important means of facilitating increased education, especially for adult learners coming from the workplace or military. In 2019 the Success Center for California Community Colleges reported that students who earn credit for prior learning are more likely to finish their programs.5 State regulations require each community college district to adopt policies related to recognition of prior learning, and the state law establishing Calbright specifically required it to establish a process for recognition of prior learning by July 2019. Although Calbright does not yet offer formal academic credit, the principles of credit for prior learning can still apply in its present operations. For example, it could still assess students’ prior experiences to determine whether they need to complete all of the coursework in a given program or if the students’ prior experience demonstrates competency in certain areas. 5 The Success Center for California Community Colleges is a division of the Foundation that assists the Chancellor’s Office in the implementation of systemwide student success initiatives. California State Auditor Report 2020-104 43 May 2021 Nonetheless, Calbright has not established such a process. In its August 2019 report to the Legislature, Calbright provided a high-level description of a process for giving students credit for prior learning that it claimed it developed using recommendations from the American Council on Education. Calbright also implied that it had already established the process it described. However, in January 2021, more than one year after its deadline to do so, Calbright’s vice president of strategic initiatives stated that it did not have a plan to implement recognition of prior learning. She also stated that Calbright plans to develop such a plan after it achieves accreditation, which, according to Calbright’s accreditation plan, it hopes to achieve in February 2022. Calbright’s failure to recognize prior learning is particularly harmful to its mission and the population that the Legislature intended it to serve. By not implementing a plan for recognition of prior learning, Calbright may have unnecessarily added to the amount of time its students must spend to finish its programs, which is contrary to its goal of offering short, self-paced, competency-based programs leading to upward mobility in the workplace. For example, until just recently, Calbright has required its students to take a course covering certain basic skills, such as literacy, basic writing, and math without first assessing whether its students were already proficient in those skills. Calbright’s target population consists of adults who have work and family responsibilities that make devoting time to their studies difficult. By causing such students to take coursework on subjects in which they are already Students’ Comments Regarding the proficient, Calbright may have caused students to Requirement to Take Basic Coursework become discouraged and even to drop out. “I would prefer getting directly to the course rather than being required to do basic skills. Maybe Calbright could In June 2020, Calbright completed a survey of its consider an assessment to bypass any repetition of skills.” students that reflects this discouragement. In the * * * survey, 70 percent of respondents reported either that their program was taking longer than expected or “The pre-coursework is much too long and elementary. that they were not pleased with their progress. Some I don’t need to be taught what email is or how to schedule my time. This coursework should not be mandatory if students expressed frustration with the requirement someone has proficiencies.” to take basic coursework and with the inability to test out of those courses. The text box contains examples * * * of these students’ comments. Calbright’s failure to help “It would be great and more efficient for students like me students more quickly progress through its programs by to have an option to waive or test out of the competency granting credit for their prior training and experience is classes. This way, no precious time is spent if not necessary inconsistent with its charge to remove the barriers that while those hours can be spent on the actual program.” prevent these students from obtaining an education that Source: Calbright’s June 2020 survey of students. can help them to obtain higher-paying jobs. In March 2021, Calbright’s board approved a revised version of the basic skills course that altered its content to focus more on career readiness skills. Specifically, Calbright removed the content related to literacy, writing, and math. Instead, Calbright’s required course now covers skills such as resume development and networking. 44 California State Auditor Report 2020-104 May 2021 Nonetheless, until it adopts a model of full recognition of prior learning, Calbright will fall short of the best practices we identified and of the mandate it has in state law. Calbright Has Not Adequately Developed Partnerships With Employers to Help Its Students Obtain Good Jobs In the midst of the turnover of its executive team in 2020 and the resulting shift in its strategies, Calbright has failed to produce a clear path toward employment for its students. State law requires that one of Calbright’s guiding principles must be offering educational programs that lead to employment, earning gains, or upward mobility for students, not just to degrees and certificates. To achieve that goal, state law requires that Calbright establish educational programs based in part on the advice of employers. State law further required that by July 2019, Calbright map its students’ experience in multiple areas, including entry into internships and jobs. These requirements clearly demonstrate how central jobs are to Calbright’s overall mission. Calbright itself recognized the importance of jobs when it told the Legislature in August 2019 that it would place 300 to 400 graduates into paid apprenticeships or jobs in its first year. However, it did not reach that goal; in fact, at the end of its first year, only 12 students had graduated and Calbright does not know the employment status of all of these graduates. Calbright has not established Moreover, even if Calbright had graduated more students, it has the relationships with employers not established the relationships with employers necessary to necessary to ensure its graduates ensure that these graduates obtained employment upon graduation. obtained employment upon Further, Calbright could not show us that it had even pursued such graduation. relationships. In its August 2020 report to the Legislature, Calbright stated that its original model for operation relied on direct placement of students with employers, but that that model was unworkable and based in part on untested assumptions. Calbright’s current CEO explained that its former executive team developed the college’s original job placement approach and that approach is not effective under the current labor market conditions. However, because Calbright could not provide documentation of its efforts to pursue this strategy, we question the degree to which Calbright attempted to engage with employers to help secure job placements. Calbright’s August 2020 report also noted problems with its original job placement model and indicated that it would be shifting to focus on career readiness and implementing a career services model that prepares graduates to find and apply for jobs. Calbright has taken some steps toward providing career services for its students. According to its director of partnership development, its career California State Auditor Report 2020-104 45 May 2021 services officially launched in November 2020. These services include one-on-one coaching and having representatives from Career Services Calbright Offers companies speak to students in webinars, as the • Informational handouts about job searching. text box shows. • Professional skill-building workshops. These services will likely benefit Calbright’s students and it • One-on-one career coaching. should continue to offer them. That said, career services are • Webinars with company representatives. insufficient substitutes for establishing strong relationships Source: Analysis of Calbright career services. with employers who could or would hire Calbright graduates. Calbright’s current CEO indicated that its lack of partnerships are due to uncertainty following the COVID-19 pandemic and problems with its original job placement model. However, this deficiency likely also stems from the fact that as of December 2019— more than one year into its existence—Calbright had not hired most of the positions it planned to fill with responsibility for developing these partnerships. Although Calbright has now begun developing a general process for establishing relationships with employers, as of March 2021, it still had no formal plan. Until it has a formal plan for taking into account input from employers or industry groups about which occupations it should develop programs for and the key knowledge, skills, and abilities that its program graduates must possess, Calbright risks failing to fulfill its core mission of helping students improve their economic mobility. Finally, Calbright does not have a sufficient mechanism for determining whether the employment programs and services it provides its students are successful. Such a mechanism necessarily includes a method for collecting general information from employers who hire its graduates about their preparedness for their jobs. In the absence of relationships with employers, Calbright lacks a ready means to obtain this feedback. Measuring the success of its programs should also include tracking whether Calbright graduates achieved positive outcomes, including jobs in the relevant field and higher wages. However, Calbright has not taken sufficient steps to monitor how many of its graduates—24 as of December 2020—have actually obtained jobs in a field related to the program they completed. Calbright did perform a survey of its 24 alumni in January 2021 to gather data on their employment outcomes. It received 11 responses and found that one, a graduate of the cybersecurity program, had obtained a job in a related field. In order for Calbright to assess the effectiveness of its educational programs and career services, it must formalize a process to collect information on whether graduates are able to secure jobs with the credentials they earn, whether those jobs provide the graduates with higher wages, and whether employers are satisfied with how the college prepared the graduates for success in their new positions. 46 California State Auditor Report 2020-104 May 2021 Calbright Must Take Action to Demonstrate Its Ability to Accomplish Its Mission Despite the missteps early in Calbright’s launch, the core elements that led the Legislature to create it have not changed. As we note in the Introduction, millions of Californians over the age of 25 lack a college degree. In 2018, Finance and the Chancellor’s Office reported that these Californians are underserved by the State’s existing educational programs. The State can benefit from providing these individuals with a postsecondary education program that offers the flexibility necessary to meet their needs. In fact, while Calbright has much to do to succeed in its mission, several of the students who answered our surveys were appreciative of the college. Further, according to the California EDGE Coalition, the competency-based education model—which Calbright offers—could help the State move low-wage workers out of poverty while helping fill the projected gap in the supply of skilled workers in the labor market.6 Calbright would take a significant step toward better serving Californians if it revised its implementation plan— Components of an Effective which it published in August 2019—to more clearly reflect Implementation Plan how it will achieve its mission. The text box lists some of • Establishes goals for implementation. the key components of a well-designed implementation • Details critical steps to accomplish those goals. plan. An effective implementation plan translates an organization’s mission into goals and actionable strategies. • Documents the resources needed to achieve each step. However, Calbright’s plan does not always identify specific deliverables, measurable goals, or clear due dates for certain • Identifies deliverables for project milestones. major objectives, such as establishing a student outreach • Assigns due dates for project milestones. strategy. Further, Calbright’s current plan does not contain a • Assigns staff responsible for completion of detailed strategy for how and when it will spend its funding, project milestones. as we note in Chapter 1. The absence of an implementation • Establishes criteria to measure success. plan with well-supported goals and strategies has almost • Accounts for major changes in circumstances certainly contributed to Calbright’s missteps and the through periodic updates. challenges that we detail throughout the report. A revised plan would allow Calbright, the Board of Governors, and Source: Analysis of best practices for implementation planning and project management. the Legislature to monitor the college’s progress and would increase accountability to keep Calbright on track toward meeting its goals. Calbright’s current leadership has stated its commitment to revising the college’s goals and strategy. It is currently developing a new strategic plan that its senior vice present for strategic initiatives asserted will include the types of features and details we noted were missing from Calbright’s initial implementation plan. As part of this process, in March 2021 Calbright released a strategic vision report 6 The California EDGE Coalition is an association of business, labor, education, workforce, and social justice organizations that promotes opportunities for skill development and postsecondary educational attainment. California State Auditor Report 2020-104 47 May 2021 stating its goals for the next three years for certain major objectives such as student enrollment and labor market outcomes. Although the report is a step in the right direction, we found that Calbright did not present adequate support for its goals. For example, Calbright described “positive labor market outcomes” for its graduates but did not specify how it would measure those outcomes, and Calbright based its enrollment goals solely on the number of students that one other college that offers competency-based education enrolled in its initial years of operation. To better demonstrate that it has set reasonable goals, Calbright needs to benchmark them against multiple colleges or other points of comparison. Further, Calbright will need to move quickly to demonstrate that it can fulfill its original promise. Its continued existence has been discussed in recent legislative sessions. Within the past year, the Legislature reduced the State’s financial commitment to Calbright, and it is currently considering a bill that would render the college inoperative at the end of fiscal year 2022–23. The seven-year timeline that state law establishes for Calbright’s development is effectively made up of two phases. The first phase, which contains the milestones due by July 2019 and July 2021, is focused on Calbright’s formation, including its establishment of business processes and processes for serving its students, and its application for accreditation. The second phase, which contains the milestones due by July 2023 and July 2025, is focused on Calbright’s growth, including adding additional program pathways and enrolling more students. As of the date of this report, Calbright is nearing the end of the formation phase but it is significantly behind on its formation milestones, and thus far, has accomplished minimal results in comparison to the funding it has spent. Specifically, despite having spent about $28 million, it has not successfully enrolled its target student population, has graduated fewer than 30 students, has not secured a path to employment for its students, and has not Calbright must show that completed the setup of basic operational functions. Calbright must its actual success can match its show that its actual success can match its potential benefit and that it potential benefit and that it is is worth the significant investment of public funds that the State has worth the significant investment made. To do so, it must complete the setup of the college by adequately of public funds that the State accomplishing its formation milestones, and prove that it is ready for has made. growth by July 2023 in line with the expectations set forth in state law. Figure 5 presents a set of our recommendations to Calbright that we believe are critical for it to demonstrate to the Legislature that it should continue to receive funding. If Calbright cannot demonstrate meaningful progress in these areas by December 31, 2022—six months ahead of its first growth milestone, then the Legislature should repeal the law establishing the college as an independent district and explore other options for offering self-paced, competency-based education. 48 California State Auditor Report 2020-104 May 2021 Figure 5 Recommendations Calbright Should Implement to Demonstrate Its Benefit to California Protect Public Funds By July 2021, Calbright should: Complete development of a fully functioning procurement process that aligns with state law and regulations. By November 2021, Calbright should: Formalize and implement policies, procedures, and processes for hiring that align with state law and regulations. Establish a salary schedule to ensure that staff and faculty salaries are reasonable and consistent with salaries at the other California community colleges. Develop a timeline for ending its reliance on the Foundation for business operations. Develop an Effective Strategy for Success By November 2021, Calbright should: Finalize an implementation plan that sets goals for student success and for completing the setup of the college and defines the actions it must take to achieve those goals and a timeline for completing them. Complete a plan that details how and when it expects to spend its funding, including a plan to remain on budget. Develop and implement a formal process for selecting effective educational programs. Perform Duties Critical to Enroll, Graduate, and Secure Jobs for Its Students By November 2021, Calbright should: Create and implement a strategy for reaching its target student population. Establish a system for monitoring to ensure that students receive the support they need to graduate. Develop and implement a plan for assisting its students in obtaining jobs, including a path toward securing job placements, and begin collecting data on student employment outcomes. Demonstrate Its Ability to Achieve Its Mission By July 2022, Calbright should demonstrate: It has consistently followed its implementation plan. It has effectively reached its target population. It has succeeded in assisting its graduates to obtain employment, higher wages, or career advancement. California State Auditor Report 2020-104 49 May 2021 Recommendations Legislature To ensure that Calbright provides educational and economic opportunities to Californians and is accountable for its performance, the Legislature should do the following: • Require Calbright to demonstrate substantive compliance with our audit recommendations. • Require the California State Auditor (State Auditor) to provide an update to the Legislature by no later than December 2022 about Calbright’s progress in implementing those recommendations. • Adopt a sunset provision that would eliminate Calbright as an independent community college district if the State Auditor determines that Calbright has not demonstrated substantive compliance with those recommendations by December 2022. • If it eliminates Calbright, the Legislature should explore other options for providing competency-based education for California adults who face barriers to traditional postsecondary education. Calbright To adequately address its foundational purpose for existing, Calbright should immediately develop a robust implementation plan that aligns with best practices and translates its mission into actionable goals and strategies. It should complete that plan and begin implementing it by November 2021. At a minimum, Calbright should include in its implementation plan all of the following: • Its goals, which should include both its goals for completing the setup of the college and its student outcome goals. It should develop its student outcome goals based, at a minimum, on a comparison of the student outcomes for multiple other reasonably comparable educational programs. • The major steps necessary to achieve its goals. • The estimated resources and specific deliverables that each step will require. • The due dates and assigned staff for each deliverable or major step. • The criteria it will use for measuring its success and monitoring its progress. • A strategy and timeline for ending its reliance on the Foundation. 50 California State Auditor Report 2020-104 May 2021 After completing its implementation plan, Calbright should review the plan at least every six months, and revise and update it as needed to account for major changes relevant to the college’s implementation. By July 2022, Calbright should demonstrate that it has made consistent progress in implementing its plan. To improve its accountability for its actions toward fulfilling its mission, Calbright should annually report to the Legislature its progress related to each step in its implementation plan. To effectively reach and enroll the students the Legislature intended it to serve, Calbright should, by November 2021, do the following: • Develop and implement a specific plan for conducting outreach to individuals within its target student population. The plan should reflect its current outreach strategies and long-term goals, including strategies for reaching each group within its target population. • Establish methods for measuring whether it has successfully enrolled its target student population. These methods should include collecting and reviewing the information necessary to ensure that it is reaching its target student population, including data on student income level, veteran status, employment status, incarceration history, and reasons for enrolling in Calbright instead of a traditional community college. By July 2022, Calbright should demonstrate that its efforts have been effective at reaching the population the Legislature intended it to serve. To ensure that it adequately prepares its target student population to obtain positive employment outcomes after graduation, Calbright should, by November 2021, develop and implement a process for selecting and expanding educational programs that will provide value to that population; that process should include the following: • Collaboration with employers and industry groups to inform the content of the programs. • Consideration of market demand for graduates of such programs. • Determination of whether the programs can help its target student population obtain positive employment outcomes including jobs, earning gains, and upward mobility. • Available resources for program implementation. • An evaluation of student demand for the programs. California State Auditor Report 2020-104 51 May 2021 To ensure that it is fulfilling its mission to help students obtain positive employment outcomes, including jobs in their field of study, earning gains, and upward mobility, Calbright should do the following: • By November 2021, develop and implement a specific plan that describes how it will assist its students in acquiring jobs, earning more income, or being upwardly mobile after graduation; the plan should include a path toward securing job placements for its students. • By the same date, also establish a method to collect and review data on student use of its career services, employment outcomes following graduation, and employer satisfaction with Calbright’s preparation of its students. • By July 2022, Calbright should begin demonstrating that it has been successful at assisting its graduates in obtaining positive employment outcomes, including jobs in their field of study, earning gains, and upward mobility. To ensure that it is providing students with the assistance they need to graduate, Calbright should, by November 2021, establish systems to monitor the effectiveness of its student support efforts. Specifically, it should take the following actions: • Establish a monitoring system to ensure that it provides each student with the supports it has identified in its student support plan. • Conduct an annual survey of enrolled students to assess their satisfaction with its support services and instruction and with their own progress toward their educational goals. • Every six months, evaluate the effectiveness of the student support plan, including reviewing data on its provision of support to its students, student progress, and its annual student survey. Following its review, it should adjust the plan as necessary. • Include in its annual report to the Board of Governors and the public the results of its annual student survey and the steps it has taken to address student feedback. To assist its students in completing its programs, Calbright should immediately establish an efficient process to recognize previous training and experience and allow students to bypass areas of curriculum in which they have demonstrated competence. 52 California State Auditor Report 2020-104 May 2021 We conducted this performance audit in accordance with generally accepted government auditing standards and under the authority vested in the California State Auditor by Government Code sections 8543 et seq. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on the audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, ELAINE M. HOWLE, CPA California State Auditor May 11, 2021 California State Auditor Report 2020-104 53 May 2021 Appendix A RESULTS OF OUR ONLINE SURVEY OF CALBRIGHT STUDENTS The Audit Committee asked us to identify certain information about Calbright’s students, including why they enrolled and whether they are part of Calbright’s target student population. As we note in Chapter 2, Calbright has not collected sufficient data to make those determinations. Given Calbright’s lack of data, we conducted an online survey of Calbright students enrolled as of December 2020. We aggregated and anonymized responses to protect students’ identities. Figure A presents the results of our survey. 54 California State Auditor Report 2020-104 May 2021 Figure A Results of Online Survey of Calbright Students Enrolled as of December 2020 95 of Calbright’s 492 students responded to our survey. OUR ANALYSIS OF STUDENTS’ RESPONSES INDICATES THAT . . . Calbright appears to have reached some groups in its target student population.* 62 44 33 24 7 1 Working Care for children Low Immigrants Veterans Incarcerated adults or family members income† or formerly at home incarcerated Many students considered other programs before choosing Calbright.* The majority of students considered one or more Students chose Calbright because educational programs besides Calbright. other programs were . . . 50 Too expensive 37 Conflicted with work 27 responsibilities 23 23 Conflicted with family 19 responsibilities 16 15 Were not 13 available online Did not offer 8 desired courses Four-year Community For-profit Other None Other 4 university college online college Overall, students were satisfied with their Calbright experience.‡ The majority said Calbright has provided adequate guidance and assistance for The majority were satisfied them to achieve their educational goals. with their progress 12 NEUTRAL 22 6 NEUTRAL NOT ADEQUATE 13 59 NOT SATISFIED SATISFIED 76 ADEQUATE Source: Analysis of results of online survey of Calbright students conducted in January and February 2021. * Some students appeared in more than one category. † Combined annual income under $30,000 before taxes and deductions. ‡ One student did not respond to these survey questions. California State Auditor Report 2020-104 55 May 2021 Appendix B SCOPE AND METHODOLOGY The Audit Committee directed the State Auditor to perform an audit of Calbright to review the current status of its efforts to meet its goal of providing increased economic mobility to working adults who lack easy access to traditional forms of postsecondary education. The table below lists the audit objectives and the methods we used to address them. Audit Objectives and the Methods Used to Address Them AUDIT OBJECTIVE METHOD 1 Review and evaluate the laws, rules, and Reviewed relevant state laws, rules, and regulations related to California regulations significant to the audit objectives. community colleges generally and Calbright specifically. 2 Identify the following information regarding Calbright’s students: a. The total number enrolled and the number Reviewed Calbright’s enrollment data as of October 2020 to determine the number of enrolled in each course offering and program. students enrolled in the college and in each of its three programs. b. To the extent possible, demographic • Reviewed Calbright’s enrollment data to identify demographic information. information, including ethnic diversity, age, • Conducted an online survey of Calbright students to determine their reasons for income, previous educational experience, and enrolling and to identify their demographic information that Calbright did not collect. reason for enrolling in Calbright as opposed to a local community college. c. To the extent possible, whether the enrolled • Analyzed Calbright’s enrollment data and the demographic information we obtained students are those that Calbright was through our online student survey to evaluate the extent to which Calbright has created to serve, including whether the enrolled its target population. Because of the deficiencies in Calbright’s collection of students already had access to affordable student data that we describe in the report, we could not fully determine whether its educational opportunities. students already had access to affordable educational opportunities. d. To the extent possible, whether the students • Reviewed data in Calbright’s case management system to identify the extent to which are achieving their educational goals and students made progress in their educational programs. progressing in courses and programs. • Conducted an online survey of Calbright students to determine whether students were satisfied with their progress. 3 Determine whether Calbright has met key Reviewed Calbright’s progress toward achieving the majority of its milestones by milestones including, but not limited to, the performing the procedures to address the other objectives listed in this table. We milestones specified in the California Online describe below the additional procedures that we performed to assess the remainder of Community College Act. Further, evaluate the milestones. Calbright’s efforts to meet these key milestones and fulfill its mission, including its efforts to do the following: a. Become an accredited college. • Reviewed Calbright’s accreditation plan and application documentation to determine whether it is on track to achieve accreditation by the deadline set in state law. • Interviewed Calbright executives and collected documentation to identify the college’s plan for establishing a process for recognition of prior learning experiences. b. Facilitate internships and job placements • Interviewed Calbright executives and reviewed documentation to assess the and work in partnership with employers and college’s attempts to partner with employers and industry groups for internships and industry groups. job placement. • Reviewed documentation of meetings with employers to assess employer input on Calbright’s curriculum and program development. continued on next page . . . 56 California State Auditor Report 2020-104 May 2021 AUDIT OBJECTIVE METHOD c. Prepare for receiving and educating students, • Evaluated Calbright’s planned and existing student support programs and its process including efforts to recruit qualified faculty for helping students establish and achieve educational goals. and staff. • Evaluated hiring guidance documents and instructors’ resumes to assess faculty qualifications. • Conducted additional review of faculty and staff recruitment as part of Objective 6. 4 Review and assess Calbright’s efforts to create • Reviewed Calbright’s outreach plan and documentation of its outreach activities and implement an inclusive outreach plan to evaluate the effectiveness of its efforts to reach and enroll students from its to recruit students throughout California. target population. Determine whether Calbright’s outreach efforts • Reviewed Calbright’s enrollment data as of October 2020 and conducted an online are specifically targeted to those students that survey of Calbright students to determine the extent that Calbright has successfully Calbright was intended to train and educate enrolled its target population. and whether those outreach efforts are likely to result in it meeting its enrollment goals. 5 Compare Calbright’s programs and courses Compared the course content, delivery methods, and credentials earned through to those of a selection of community colleges to Calbright’s pathways to fully online courses that other community colleges offer statewide. determine whether Calbright’s courses and programs are duplicative. 6 To the extent possible, determine whether • Interviewed Calbright staff and reviewed documentation to identify its policies and Calbright has complied with applicable laws practices for contracting. and regulations, including those related to • For a selection of contracts, reviewed contract files to assess compliance with employment, procurement, and recruitment. applicable laws and regulations, including competitive bidding requirements. • For a selection of staff and faculty, reviewed hiring files to assess Calbright’s compliance with recruitment and employment laws. • Reviewed Calbright’s email records to determine whether and to what extent hiring and contracting decisions may have been influenced by personal interests or bias. 7 Assess Calbright’s efficiency and effectiveness in • Compared Calbright’s spending to date to its projected spending in its seven-year how it uses its resources. To the extent possible, implementation plan. compare the efficiency and effectiveness of • Assessed the feasibility of comparing the efficiency and effectiveness of Calbright’s Calbright’s use of resources with a selection of use of resources to other colleges. We compared salaries for Calbright’s faculty other community colleges or other worthwhile and current and former executive team to those at other community colleges. We initiatives and programs, and identify any areas determined that a similar comparison for Calbright’s other uses of resources was not needing improvement. feasible given that such an assessment would require evaluating the efficiency and effectiveness of the other colleges’ uses of their resources, which would be time- prohibitive. Additionally, Calbright’s size, status as a start-up college, and different nature of programming render the type and amount of resources that it requires too dissimilar to other colleges for an effective comparison. Finally, our review of Calbright’s use of public funding under objectives 6 and 8 was sufficient to conclude that Calbright has not effectively or efficiently used its resources. 8 Evaluate Calbright’s expenditures and internal • Compared the salaries of Calbright’s executives and faculty to the salaries for equivalent controls for reasonableness and appropriateness positions at other California community colleges. We discuss our findings regarding including, but not limited to, those related executive salaries in Chapter 1. We found salaries for faculty to be reasonable. to salaries, office and satellite campus space, • Reviewed a selection of expenditures from fiscal years 2018–19 through 2020–21 electronic devices, online technologies, for reasonableness, including expenditures for office and satellite campus space, consulting contracts, and timekeeping. electronic devices, online technologies, and timekeeping. We found Calbright did not separately track its timekeeping expenses. We identified some deficiencies in Calbright’s documentation of earlier expenditures, and we discussed those deficiencies with Calbright’s management team. We found no significant problems with Calbright’s documentation of its more recent expenditures. • Used additional procedures to evaluate the reasonableness of Calbright’s expenditures as described under objectives 6 and 7. 9 Review and assess any other issues that are We did not identify any other issues of significance. significant to the audit. Source: Audit Committee’s audit request number 2020-104, and information and documentation identified in the table column titled Method. California State Auditor Report 2020-104 57 May 2021 Assessment of Data Reliability The U.S. Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of computer-processed information that we use to support our findings, conclusions, and recommendations. In performing this audit, we relied on electronic data to evaluate the extent to which Calbright has enrolled its target student population, to identify the progress that Calbright’s students have made in its educational programs, and to assess the degree of support that Calbright has provided to support its enrolled students. Specifically, we obtained data from Calbright’s learner management system, case management system, and the California Community College’s application system. To evaluate the data, we reviewed existing information about the data, interviewed Calbright staff who were knowledgeable about the data, and performed electronic testing of the data. As a result, we identified limitations with the data. Specifically, some of the data in the California Community College’s application system are self-reported. For example, students self-report their demographic information, including their educational history, gender, and ethnicity. Although these limitations may affect the precision of the numbers we present, we determined that the data were sufficiently reliable to support our audit findings, conclusions, and recommendations. 58 California State Auditor Report 2020-104 May 2021 Blank page inserted for reproduction purposes only. California State Auditor Report 2020-104 59 May 2021 Calbright College April 21, 2021 Elaine M. Howle California State Auditor * 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 Dear Ms. Howle: Calbright College has reviewed the California State Auditor’s draft audit report titled “Calbright College Must Take Immediate Corrective Action to Accomplish Its Mission to Provide Underserved Californians With Access to Higher Education.” Calbright appreciates the opportunity to respond to the recommendations contained in the report. Calbright is focused on the urgency of this moment for the millions of Californians bearing the brunt of this pandemic, navigating an economic recession, and facing another recovery that leaves them stranded without a path into good jobs in the new economy. It’s clear how these trends have borne out historically. As the economy begins to open up but uncertainty lingers – predatory, for-profits seize the opportunity to target, recruit, and leave behind students in these same communities, calcifying existing inequities. As we work to build a more equitable recovery, we must step up to do more to serve the hardest-hit communities across the state – not less – with a viable, public option for these Californians. Accomplishing this mission is a complex endeavor that requires, among other things, understanding of our target population and the barriers they face to accessing higher education; ability to perform labor market analysis and to determine areas of statewide and regional needs; expertise in higher education and andragogy; knowledge of course design, development, and selection; expertise to develop, build, and maintain technology infrastructure to deliver instruction, supports, and services; and the ability to rapidly build and scale a new public institution within the seven year start-up window specified by the Legislature. Calbright is pleased that the State Auditor reached the conclusion that: “Calbright’s pathways are not duplicative when compared to the other programs we reviewed.” This matches the determination made by the California Community Colleges Chancellor’s Office in 2020 regarding duplication. As noted by the California State Auditor, “the need remains for flexible educational opportunities for California adults who face barriers to attending traditional community colleges, and if successful, the competency-based education model that Calbright * California State Auditor’s comments begin on page 67. 60 California State Auditor Report 2020-104 May 2021 offers could provide those opportunities.” We are committed to continuing our work in this area as we strive to serve students who are not currently accessing a postsecondary education. Calbright appreciates the State Auditor’s recognition of the progress that Calbright has made in many key areas under the current leadership team. As a forward-looking organization, we agree that we need to make continued progress in the areas of improvement that the State Auditor has identified in the report, and we are fully committed to implementing the recommendations, as we describe below. Below we reiterate the audit recommendations and our response to each specific recommendation. Recommendation to Calbright: To provide greater accountability regarding its spending and to ensure that it effectively uses the public funds it receives to accomplish the goals for which it was created, Calbright should do the following: ● By November 2021, incorporate into its implementation plan a spending plan that details how and when it expects to spend the funds the Legislature allocates to it. At a minimum, the spending plan should identify the estimated costs to accomplish the tasks set forth in its implementation plan and a timeline for when it expects to incur those costs. The spending plan should also describe Calbright’s strategy for staying within its budget while completing necessary start-up activities and achieving its milestones. ● On an annual basis, Calbright should review the spending plan and make adjustments as necessary. Annually, Calbright should report on its spending to date and explain how its spending has furthered its progress in achieving its mission. Calbright Response: Calbright agrees with this recommendation. We were pleased that, under Calbright’s new leadership, the State Auditor “found no significant problems with Calbright's documentation of its more recent expenditures.” Calbright is subject to (Educ. Code Section 70902) and by law presents a proposed budget including actual expenditures to the Board of Trustees annually, and we will continue to review the spending plan and make adjustments as necessary. Under 1 new leadership, Calbright has already completed a collaborative research and visioning process that lays out a Strategic Vision for Calbright’s next three years out of the seven-year start-up timeline given by the Legislature. This vision includes ambitious and achievable goals relating to program pathway development and organizational maturity at scale. This Strategic Vision has been approved by the Calbright Board of Trustees. Calbright is using the Strategic Vision in the development of its revised implementation plan, including a spending plan, and is fully committed to implementing this recommendation according to the State Auditor’s timeline. 2 California State Auditor Report 2020-104 61 May 2021 Recommendation to Calbright: To ensure that its hiring process is fair and results in the hiring of well-qualified staff, Calbright should, by November 2021, finalize its development of human resources policies and procedures for recruitment and hiring that comply with state law and regulation. Calbright Response: Calbright agrees with this recommendation. As acknowledged in the audit, “Calbright is now under new leadership that has begun taking positive steps toward correcting the deficiencies we identified, such as through developing stronger hiring practices.” Calbright fully intends to implement this recommendation by November 2021. Recommendation to Calbright: To ensure that it uses state resources responsibly, Calbright should do the following: ● Immediately commence the process of hiring a procurement director with significant experience in public sector procurement. ● By July 2021, complete the development of a fully functioning procurement process that aligns with state law, regulations, and to the extent practicable, the State Contracting Manual. That process should include policies and procedures that ensure that all of Calbright’s contracts provide clear expectations of the work that contractors will perform. It should further include strong contract management processes for ensuring contractors perform that work satisfactorily before receiving payment. ● By August 2021, provide training on relevant laws, policies, and procedures related to procurement to all staff involved in approving or managing contracts. Calbright Response: Calbright agrees with this recommendation. Under new leadership, Calbright has established a more robust contract oversight process that involves a written initial request that tracks scope of work and deliverables sought of independent contractors, which is then reviewed by multiple administrators before the request is approved for processing, and includes notification to and approval by the Calbright Board of Trustees, in accordance with Board policies. Calbright has begun the hiring process for an experienced hire to manage procurements and fully intends to implement these recommendations along the stated timeline. With respect to the recommendation that Calbright align its procurement process “to the extent practicable, the State Contracting Manual” we note that the State Contracting Manual applies, by its own terms, to state agencies. Calbright is a community college district, and, as such, is a local agency, so it is not subject to the State Contracting Manual. The State Auditor acknowledges this by stating that it used the State Contracting Manual as comparative criteria. We understand the “to the extent practicable” part of the recommendation to mean that Calbright should adopt some of the best practices in the State Contracting Manual including: clear and concise descriptions of the work to be performed by the contractor; clear, measurable deliverables; specific due dates for completion of the work; and the maximum amount to be paid to the contractor. These best practices complement the requirements set out in statute that 3 62 California State Auditor Report 2020-104 May 2021 apply to Calbright and all community college districts. Calbright fully intends to implement this recommendation within the required timeline. Recommendation to Calbright: To ensure that the compensation it provides its employees is reasonable, Calbright should establish a pay schedule for all employees by November 2021 that: ● Includes all forms of compensation including salary, benefits, and other forms of compensation. ● Establishes compensation comparable to that of similar positions within the community college system. Calbright Response: Calbright agrees with this recommendation. Under new leadership, Calbright has already made strides in this area, and the State Auditor found that “salaries Calbright pays its new executives are comparable to those earned at other community colleges." Calbright is a labor employer and the intent of the statute included prioritization of faculty labor negotiations regarding salary before the establishment of a formal pay schedule for staff and administration. Calbright fully intends to comply with this recommendation to establish a formal pay schedule, however, the timeline may be impacted as we must do so in partnership and respecting the bargaining process with our employee labor associations. Recommendation to Calbright: To adequately address its foundational purpose for existing, Calbright should immediately develop a robust implementation plan that aligns with best practices, translates its mission into actionable goals and strategies. It should complete that plan and begin implementing it by November 2021. At minimum, Calbright should include in its implementation plan all of the following: ● Its goals, which should include both its goals for completing the setup of the college, and its student outcome goals. It should develop its student outcome goals based, at minimum, on a comparison of the student outcomes for multiple other reasonably comparable educational programs. ● The major steps necessary to achieve its goals. ● The estimated resources and specific deliverables that each step will require. ● The due dates and assigned staff for each deliverable or major step. ● The criteria it will use for measuring its success and monitoring its progress. ● A strategy and timeline for ending its reliance on the Foundation. After completing its implementation plan Calbright should review it at least every six months, and revise and update it as needed to account for major changes relevant to the college's implementation. By July 2022, Calbright should demonstrate that it has made consistent progress in implementing its plan. To improve its accountability for its actions toward fulfilling its mission, Calbright should annually report to the Legislature its progress related to each step in its implementation plan. 4 California State Auditor Report 2020-104 63 May 2021 Calbright response: Calbright agrees with this recommendation. Under new leadership, Calbright has completed a 1 collaborative research and visioning process that lays out a 3-year Strategic Vision for Calbright’s future, including ambitious and achievable goals relating to program pathway development and scaling, and organizational maturity at scale. This Strategic Vision has been approved by the Calbright Board of Trustees and includes goals for completing the setup of the college and its student outcome goals. We are in the process of further developing this vision into a robust implementation plan with more specific detail. As envisioned by the Legislature, Calbright contracted with the Foundation for California Community Colleges during the seven year start-up window. We have taken a number of steps towards setting up key organizational administrative functions and will develop a plan for reducing contractual administrative services during this period. Calbright is fully committed to implementing these recommendations according to the State Auditor’s timeline. Recommendation to Calbright: To effectively reach and enroll the students the Legislature intended it to serve, Calbright should, by November 2021, do the following: ● Develop and implement a specific plan for conducting outreach to individuals within its target student population. The plan should reflect its current outreach strategies and long-term goals, including strategies for reaching each group within its target population. ● Establish methods for measuring whether it has successfully enrolled its target student population. These methods should include collecting and reviewing the information necessary to ensure that it is reaching its target student population, including data on student income level, veteran status, employment status, incarceration history, and reasons for enrolling in Calbright instead of a traditional community college. ● By July 2022, Calbright should demonstrate that its efforts have been effective at reaching the population the Legislature intended it to serve. Calbright response: Calbright agrees with this recommendation. Calbright has already engaged in proactive 2 outreach planning to reach its statutorily mandated target population (Educ. Code Section 75001). It is relevant in context that the period examined (October 2019-2020) included a statewide shelter-in-place order that impacted Calbright’s in-person and traditional outreach strategies. We are considering the disproportionate impacts of the pandemic on our target population to better understand the effects and better connect with these Californians and address their unique needs. More than 90% of Calbright’s initial cohort of students were 2 students over the age of 25 and more than 50% were students of color – exactly the working adults that Calbright was designed to serve. Calbright is planning to do outreach to additional populations who may also benefit from these flexible programs. Calbright is committed to iterating and improving these outreach efforts to reflect the full diversity of the state. The audit recognizes Calbright’s “notable” success in enrolling African American students, a critically important and historically hard-to-reach population for the community college system (23% at 5 64 California State Auditor Report 2020-104 May 2021 Calbright versus 6% in the community college system). Additionally, Calbright can meet the federal student population threshold to be considered an Hispanic-Serving Institution (31% of Calbright’s student body is Hispanic or Latino). Calbright fully intends to implement these recommendations within the stated timeline. Recommendation to Calbright: To ensure that it adequately prepares its target student population to obtain positive employment outcomes after graduation, Calbright should, by November 2021, develop and implement a process for selecting and expanding educational programs that will provide value to that population and that process should include the following: ● Collaboration with employers and industry groups to inform the content of the programs. ● Consideration of market demand for graduates of such programs ● Determination of whether the programs can help its target student population obtain positive employment outcomes including jobs, earning gains, and upward mobility. ● Available resources for program implementation. ● An evaluation of student demand for the programs. Calbright response: Calbright agrees with this recommendation. Each of Calbright’s programs goes through a formal approval process, including Calbright’s senior leadership, curriculum committee, approval by the 3 Calbright Board of Trustees at a public meeting, and approval by the Chancellor’s office. Under the new leadership, each new pathway is researched and validated to ensure it will directly support our target population to achieve labor market outcomes including but not limited to, moving from one job to another with higher compensation, moving from un- and underemployment into a good job, transitioning to a new role and/or industry with career potential, and keeping a current job that now requires new skills. Calbright is fully committed to implementing these recommendations according to the State Auditor’s timeline. Recommendation to Calbright: To ensure that it is fulfilling its mission to help students obtain positive employment outcomes, including jobs in their field of study, earning gains and upward mobility, Calbright should do the following: ● By November 2021, develop and implement a specific plan that describes how it will assist its students in acquiring jobs, earning more income, or being upwardly mobile after graduation; the plan should include a path toward securing job placements for its students. ● By the same date, also establish a method to collect and review data on student use of its career services, employment outcomes following graduation, and employer satisfaction with Calbright's preparation of its students. ● By July 2022, Calbright should begin demonstrating that it has been successful at assisting its graduates in obtaining positive employment outcomes, including jobs in their field of study, earning gains, and upward mobility. 6 California State Auditor Report 2020-104 65 May 2021 Calbright response: Calbright agrees with this recommendation. As part of Calbright’s Board-approved Strategic 1 Vision, we have set clear goals for labor market outcomes dependent on the pace of the economic recovery. Calbright will be collecting data on positive labor market outcomes within six months of students completing a Calbright program including, but not limited to, moving from one job to another with higher compensation, moving from un- and underemployment into a good job, transitioning to a new role and/or industry with career potential, and keeping a current job that now requires new skills. Calbright is fully committed to implementing these recommendations according to the State Auditor’s timeline. While Calbright has every intention of exploring and intending to demonstrate earnings gains and other labor market outcomes for students, we recognize that collecting certain data may be infeasible on this timeline. Given the lag on obtaining the employment outcome data relied upon by the California Community College system from the Employment Development Department, Calbright may not be able to reflect post-audit changes in this data. EDD reports a lag time of one year post-training for this reporting, so July 2022 data would reflect students who began their training at the latest in July 2020 (prior to changes made by new Calbright leadership) and completed their program by July 2021. This lag time is particularly consequential given the global pandemic, economic downturn, and labor market uncertainty. However, Calbright intends to explore additional ways to collect this data on the timeline given by the State Auditor. Recommendation to Calbright: To ensure that it is providing students with the assistance they need to graduate, Calbright should, by November 2021, establish systems to monitor the effectiveness of its student support efforts. Specifically, it should take the following actions: ● Establish a monitoring system to ensure that it provides each student with the supports it has identified in its student support plan. ● Conduct an annual survey of enrolled students to assess their satisfaction with its support services and instruction and with their own progress toward their educational goals. ● Every six months, evaluate the effectiveness of the student support plan, including through reviewing data on its provision of support to its students, student progress, and its annual student survey. Following its review, it should adjust the plan as necessary. ● Include in its annual report to the Board of Governors and the public the results of its annual student survey and the steps it has taken to address student feedback. ● To assist its students in completing its programs, Calbright should immediately establish an efficient process to recognize previous training and experience and allow students to bypass areas of curriculum in which they have demonstrated competence. Calbright response: Calbright agrees with this recommendation and has already begun to implement new student support measures, including building out new tools for monitoring student activity in the Learner Management System, and an Integrated Student Support Model (ISSM) to track and monitor 7 66 California State Auditor Report 2020-104 May 2021 student persistence and completion in a self-paced learning environment. Calbright continues to make progress towards the full implementation of the ISSM including tracking and monitoring data points for both proactive outreach and early interventions based on student progress. It is important to note that the data on student progress reflected in the report shows a period from October 2019 - October 2020 during which our initial student population was deeply impacted by the global pandemic, including job and child care loss, as well as disproportionate exposure and health impacts. While Calbright is committed to continued improvement to our instruction and supports to better 4 serve our student population, it is also important to consider that in the Auditor’s January 2021 survey of Calbright's enrolled students, “the majority reported that they were satisfied with the support that the college's instructors and counselors had provided.” Calbright is fully committed to implementing these recommendations according to the State Auditor’s timeline. Competency-based education by design allows students to apply previous knowledge to accelerate their learning and progress more quickly through the course content. For example, our new Customer Relationship Management (CRM) platform administrator program incorporates in-tool opportunities to demonstrate mastery of skills to be able to progress more quickly through the pathway’s coursework. Calbright is also ahead of schedule to achieve 5 accreditation, as specified in statute, which is necessary to award credit for prior learning and at which point Calbright intends to do so. Calbright is fully committed to implementing this recommendation according to the State Auditor’s timeline. We appreciate the opportunity to respond to the audit. If you have any questions, please contact Jenny Johnson, Vice President of Government Relations, at 916-634-8120. Sincerely, Pamela Haynes Ajita Talwalker Menon President President and CEO Calbright Board of Trustees Calbright College 8 California State Auditor Report 2020-104 67 May 2021 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM CALBRIGHT COLLEGE To provide clarity and perspective, we are commenting on Calbright’s response to our audit. The numbers below correspond to the numbers we have placed in the margin of the college’s response. Calbright references the strategic vision report that the Board of 1 Governors approved in March 2021, which contains its short-term goals for the next three years. Although Calbright’s strategic vision report is a step in the right direction, we found that Calbright did not present adequate support for the goals in that report, as we note on page 47. For example, Calbright did not adequately explain how it will measure student outcomes, and it based its enrollment goals solely on one other college’s past enrollment. Calbright needs to revisit some of its goals and incorporate additional points of comparison in order to demonstrate that these goals are reasonable. Establishing reasonable goals for student outcomes is critical for Calbright to demonstrate that it can fulfill its mission. As we describe beginning on page 36, we found Calbright’s 2 outreach planning to be inadequate. Its current plans do not articulate specific strategies for how it will reach the various groups within its target population, or how it will assess the effectiveness of its outreach efforts. Additionally, Calbright’s assessment of its success in reaching its target student population is selective. As we further note on pages 37 and 38, although Calbright has successfully enrolled individuals from certain groups within its target student population, it has not sufficiently reached others, including adults who do not have a college degree, women, and individuals who identify as Hispanic or Latino. Moreover, as we state on page 36, Calbright has not collected sufficient data to evaluate its success in enrolling other groups that form its target student population, including veterans, immigrants, and low-income individuals. Until Calbright implements our recommendation to develop specific strategies for reaching each group within its target population, it will likely continue to struggle to adequately reach them. Calbright describes actions it has taken only in the last several 3 months for the two educational programs it recently developed. As we note in the report on page 35, Calbright has yet to document these processes. It is critical to Calbright’s ability to achieve its mission that it formalize effective processes for selecting educational programs that can benefit its target population. We look forward to reviewing Calbright’s progress in implementing our recommendation to do so. 68 California State Auditor Report 2020-104 May 2021 4 Calbright has not included the full consideration we gave to the results of our survey. Calbright is correct that, as we say on page 40, the majority of survey respondents stated that they were satisfied with the support they received from Calbright. However, on that same page we describe how less than 20 percent of Calbright’s enrolled students responded to our survey and that students’ actual progress is a more accurate measure of Calbright’s success at supporting its students. We found that the majority of enrolled students—471 of 904 students as shown in Figure 3 on page 32— had either dropped out or stopped progressing in their programs after Calbright’s first year of instruction, and Calbright could not demonstrate that it offered those students adequate support. 5 Calbright incorrectly asserts that until it is accredited it is incapable of helping students move more quickly through its programs by recognizing their previous training and experience. As we note on page 42, regardless of whether Calbright offers formal academic credit, it can still assess students’ prior experiences and allow them to skip coursework related to areas in which those students have already demonstrated competency. Until Calbright does so, its students may continue to face unnecessary barriers to obtaining a postsecondary education.