Recommendations

K–12 Strong Workforce Program State and Regional Administrative Shortcomings Limit the Program’s Effectiveness in Supporting Grant Applicants February 2022 REPORT 2021‑101 CALIFORNIA STATE AUDITOR 621 Capitol Mall, Suite 1200 | Sacramento | CA | 95814 916.445.0255 | TTY 916.445.0033 For complaints of state employee misconduct, contact us through the Whistleblower Hotline: 1.800.952.5665 Don’t want to miss any of our reports? Subscribe to our email list at auditor.ca.gov For questions regarding the contents of this report, please contact our Public Affairs Office at 916.445.0255 This report is also available online at www.auditor.ca.gov | Alternative format reports available upon request | Permission is granted to reproduce reports Michael S. Tilden Acting State Auditor February 10, 2022 2021-101 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As directed by the Joint Legislative Audit Committee, my office conducted an audit of the K–12 component of the Strong Workforce Program, which awards grants to local educational agencies (LEAs) to create, support, or expand career technical education (CTE) programs. We determined that various administrative shortcomings by the Chancellor of the California Community Colleges (Chancellor’s Office) and regional groups of community college districts (regional consortia) have resulted in the program not serving grant applicants as effectively as it could have. The Chancellor’s Office has not disclosed in the request for grant applications (RFA) all of the factors that regional selection committees consider in making grant award decisions. As a result, applicants have not had access to information that could help them make decisions about their applications. In fiscal year 2020–21, three selection committees denied funding for applications based on criteria that the RFA had not disclosed. Furthermore, the Chancellor’s Office has not directed applicants to provide key details that would help selection committees ensure that they fund the applications that best meet the program’s goals. The Chancellor’s Office has also not ensured that LEAs have equal access to staff who support applicants. Currently, the number of LEAs that each staff position serves varies, thereby affecting the level of support they are able to provide to LEAs. Due to hiring delays, approximately $2.6 million that the State appropriated to fund these support positions for fiscal year 2018–19 remains unused. State law requires that this unused funding be added to the amount appropriated to the regional consortia for CTE programs, but the Chancellor’s Office has not yet done so. We also found that the California Department of Education inaccurately reported that LEAs’ demand for a related CTE grant program was three times higher than what was actually requested, which may have misled policymakers and stakeholders. Finally, some selection committees have insufficient safeguards in place to prevent unfair grant decisions. Only two of the eight regional consortia documented their efforts to mitigate selection committee members’ potential conflicts of interest when awarding fiscal year 2020–21 grants. Respectfully submitted, MICHAEL S. TILDEN, CPA Acting California State Auditor 621 Capitol Mall, Suite 1200 | Sacramento, CA 95814 | 916.445.0255 | 916.327.0019 fax | www.auditor.ca.gov iv California State Auditor Report 2021-101 February 2022 Selected Abbreviations Used in This Report CTE career technical education Education California Department of Education LEA local educational agencies RFA request for applications California State Auditor Report 2021-101 v February 2022 Contents Summary 1 Introduction 5 Audit Results The Chancellor’s Office Has Not Directed Applicants to Provide Sufficient Detail to Demonstrate That Their CTE Programs Will Meet Regional Needs 11 The Chancellor’s Office Has Not Given Applicants Sufficient Information on Grant Selection Criteria 13 The Chancellor’s Office Has Not Ensured That LEAs Have Equal Access to Local Support Staff 19 Most Regional Consortia Have Not Ensured That Selection Committees Adopt Strong Safeguards Against Unfair Decisions 22 Recommendations 24 Other Area We Reviewed 27 Appendix Scope and Methodology 29 Responses to the Audit Chancellor of the California Community Colleges 33 California State Auditor’s Comments on the Response From the Chancellor of the California Community Colleges 37 Bay Area Community College Consortium 39 Central/Mother Lode Regional Consortium 41 California State Auditor’s Comment on the Response From the Central/Mother Lode Regional Consortium 43 Inland Empire/Desert Regional Consortium 45 Los Angeles Regional Consortium and Orange County Regional Consortium 47 California State Auditor’s Comment on the Response From the Los Angeles Regional Consortium and Orange County Regional Consortium 49 vi California State Auditor Report 2021-101 February 2022 North/Far North Regional Consortium 51 California State Auditor’s Comments on the Response From the North/Far North Regional Consortium 53 South Central Coast Regional Consortium 55 California State Auditor’s Comments on the Response From the South Central Coast Regional Consortium 57 California Department of Education 59 California State Auditor’s Comment on the Response From the California Department of Education 61 California State Auditor Report 2021-101 1 February 2022 Summary Audit Highlights… Results in Brief Our audit of the K–12 workforce program highlighted the following: California’s vision for the future of workforce development seeks shared success for both employers and employees. Consistent with » The Chancellor’s Office has not directed the vision’s objective to align workforce programs and education applicants to provide sufficient detail to programs, the State has provided $150 million in grants annually demonstrate that their CTE programs will through the Strong Workforce Program (workforce program) to meet regional needs. create, support, or expand career technical education (CTE) at the K–12 level. CTE integrates academic knowledge with technical • As a result, the selection committees and occupational knowledge to prepare students for college and have been unable to consider careers. The K–12 component of the workforce program awards relevant evidence—such as wage grants (workforce grants) to local educational agencies (LEAs) to information for industries related to create, support, or expand CTE programs that are aligned with CTE programs—that would help them workforce development efforts at the community college level. State select applications that best meet the law requires the Chancellor of the California Community Colleges workforce program’s goals. (Chancellor’s Office) to apportion funding for the grants to regional groups of community college districts (regional consortia). The » The Chancellor’s Office has not given regional consortia are responsible for administering a competitive applicants sufficient information about grant program to distribute the grants to LEAs. As the result of a the criteria the regional committees will number of shortcomings in the Chancellor’s Office’s and regional use to select eligible grant recipients. consortia’s administration of the grant program, this program has not served grant applicants as effectively as it could have. • This information would help applicants make decisions when The Chancellor’s Office has not directed workforce grant applicants preparing their applications. to provide key details in their applications demonstrating their ability to meet the program’s goals. Each year, the Chancellor’s » LEAs do not have equal access to the Office issues a request for applications (RFA) that announces the support staff who help them pursue availability of workforce program funding and establishes minimum grants because each community college requirements that applications must meet to be considered for district receives one support position funding (eligibility criteria). However, the Chancellor’s Office’s regardless of how many LEAs that instructions and guidance have not directed applicants to district includes. demonstrate that their CTE programs address all of the factors established in state law. Our review of 30 awarded applications • $2.6 million appropriated to fund from the Bay Area, Los Angeles, and North/Far North regions those staff in fiscal year 2018–19 included 16 applications that sought grants to support CTE remains unused. programs for specific industries. Of those 16, eight did not contain sufficient information on the industry’s demand for skilled workers » Most regional consortia of community and 15 did not include wage data for the industry, both of which are college districts have not ensured that elements that help demonstrate that grants would meet regional selection committees adopt strong needs. Without complete and detailed information, the selection safeguards against unfair grant committees that distribute grant funding (selection committees) award decisions. are unable to consider relevant evidence that would help them ensure that they fund the applications that best meet the workforce program’s goals. 2 California State Auditor Report 2021-101 February 2022 The Chancellor’s Office also has not adequately informed LEAs about the factors that selection committees consider in making grant award decisions. In the RFA for fiscal year 2020–21, the Chancellor’s Office disclosed the standard eligibility criteria but not the criteria that each selection committee would use to allocate limited funds when eligible applications requested more funding in total than was available. Generally, selection committees decide either to exclude some eligible applications by creating additional criteria, which we describe as selection criteria, or to fund all eligible applications at some level of the amounts requested, which we describe as allocation criteria. Because the RFA did not disclose which of these criteria selection committees would use, applicants did not have access to information that could have helped them make informed decisions about their applications. For example, the RFA did not state that selection committees in two regions would address the issue of limited funds by prioritizing the approval of only one application per LEA. Had it disclosed that information, some applicants that submitted multiple applications in those regions might have focused their efforts on a single application instead. We also found one instance in which a selection committee did not apply its selection criteria consistently and, as a result, inappropriately denied an application. Chairs of some of the regional consortia have raised concerns that LEAs do not have equal access to the support staff who help them pursue grants. Access can be unequal because of the manner in which support positions are currently assigned—one per community college district regardless of how many LEAs that district includes. State law gives the Chancellor’s Office, along with the state superintendent of public instruction, discretion over how the support positions are assigned, yet the Chancellor’s Office has not exercised this discretion to better serve LEAs. Further, the Chancellor’s Office was slow to issue guidance for the entities hiring these support positions. Due to hiring delays, $2.6 million of the $12 million that the State appropriated to fund those positions for fiscal year 2018–19 remains unused. State law requires that this unused funding be added to the amount appropriated to the regional consortia to create, support, or expand CTE programs, but the Chancellor’s Office has not yet done so. Finally, some selection committees have insufficient safeguards in place to prevent unfair grant decisions. Because selection committee members may work for the same LEAs that apply for workforce grants, there is an inherent risk that a conflict of interest—that is, a real or seeming incompatibility between their private interests and public duties—will influence their grant decisions. However, when awarding fiscal year 2020–21 grants, only two of the eight regional consortia documented their efforts to ensure that selection committee members did not make California State Auditor Report 2021-101 3 February 2022 decisions about applications for which they had potential conflicts. In addition, all eight selection committees score each application based on standard eligibility criteria, but the committees do not share a common criterion for identifying scores from individual reviewers that vary significantly and warrant additional review of applications. The lack of a common process introduces inconsistency into the application review process. Selected Recommendations Chancellor’s Office To enhance the quality of information the selection committees have available when determining whether applications best meet the workforce program’s goals, beginning in fiscal year 2022–23, the Chancellor’s Office should specify in the RFA that applicants should include detailed information addressing all eligibility criteria, including information about the wage rates and demand for skilled workers in industries aligned with their CTE programs. To ensure that all applicants can make well‑informed decisions when applying for workforce grants, beginning in fiscal year 2022–23, the Chancellor’s Office should do the following: • Request selection committees to determine—before the Chancellor’s Office issues the RFA—how they will address requests for funding that exceed the total amount they are allocated and inform the Chancellor’s Office of their decision and any selection criteria they will use, so that it can include this information in the RFA. • Include in the RFA transparent and complete information about whether each selection committee has decided to fund all eligible applications and, if not, what selection criteria it will use. To provide LEAs equal access to support staff who assist them with pursuing workforce grants, the Chancellor’s Office should establish and implement a process by June 2022 for a regional consortium to propose modifications to the areas that its support staff members are assigned to serve. 4 California State Auditor Report 2021-101 February 2022 Regional Consortia To ensure consistency in scoring applications, the regional consortia should do the following: • Maintain internal documentation demonstrating that they reviewed selection committee members’ potential conflicts of interest and that members did not review applications for which they had conflicts. • Collaborate to establish a standard for addressing score variations that selection committees statewide will use when evaluating whether applications meet eligibility criteria. Agency Comments The Chancellor’s Office indicated that it will take steps to implement some of our recommendations, although it disagreed with our recommendation to improve the transparency and completeness of information provided to potential applicants in the RFA. Additionally, the Chancellor’s Office did not respond to two other recommendations. The regional consortia and the California Department of Education, to which we made a recommendation that appears later in the report, generally agreed with our recommendations. The San Diego/ Imperial Regional Consortium did not provide a response. California State Auditor Report 2021-101 5 February 2022 Introduction Background The Legislature established the Strong Workforce Program (workforce program) to expand the availability of high‑quality, industry‑valued California’s Workforce Development career technical education (CTE) and workforce Policy Objectives development curricula and credentials. The • Foster attainment of demand-driven skills to provide workforce program must comply with the employers with a skilled workforce. California Strategic Workforce Development Plan, which establishes the State’s vision for the future of • Enable upward economic mobility for all Californians workforce development. That vision is to develop through access to workforce and education programs. a workforce that enables economic growth and • Align workforce and education programs to economize shared prosperity for employers and employees by resources and achieve impact. focusing on meaningful engagement with industry Source: California’s Unified Strategic Workforce Development and placement of Californians in quality jobs that Plan, prepared by the California Workforce Development Board. provide economic security. One of the objectives for achieving that vision is to align workforce development programs and education programs, as the text box shows. Consistent with that objective, the workforce program consists of two components. The community college component involves administrative groupings of community college districts that coordinate CTE efforts in eight regions statewide (regional consortia). It requires each regional consortium to develop a regional plan that analyzes labor market needs and establishes goals and priorities to meet those needs. The K–12 component provides competitive grant funding to local educational agencies (LEAs).1 To be eligible to apply for the grants, an LEA must partner with a community college or district to develop clearly defined pathways from K–12 CTE programs to careers or postsecondary education. Each LEA also must align its K–12 CTE efforts with its respective regional plan and commit to providing matching funds—generally, $2 from an LEA for every grant dollar awarded, with the exception of regional occupational centers or programs, which must commit $1 of matching funds for every grant dollar awarded. The text box on the following page presents elements of the workforce program’s two components. This report focuses on the K–12 component. “Components of the Workforce Program” text box The K–12 component provides $150 million annually in grant funding to LEAs to create, support, or expand CTE programs that are aligned with workforce development efforts at the community college level (workforce grants). In particular, workforce grants 1 For purposes of this report, we use the term LEA to refer to eligible applicants for workforce grants, which consist of one or more of any of the following: a school district, county office of education, charter school, or a regional occupational center or program meeting certain conditions. 6 California State Auditor Report 2021-101 February 2022 support the development of CTE curricula that Components of the Workforce Program enable students to follow coordinated pathways from education to employment within career K–12 component: fields for which there is a demonstrated demand • Grant competition to create, support, or expand K–12 for skilled workers and an opportunity to earn a CTE programs. living wage. Accordingly, workforce grants can • Started in fiscal year 2018–19. focus on CTE programs that provide students with • $150 million appropriated for fiscal year 2020-21. work‑based learning opportunities that pertain to specific industries, such as health science, • Requires grant applicants to align K–12 CTE programs with information technology, and agriculture. CTE regional plans that the regional consortia must submit. programs supported by the workforce grants Community college component: may also include opportunities for students to • Allocation of funds to improve and implement complete community college coursework while community college CTE programs. still enrolled in high school (dual enrollment). • Started in fiscal year 2016–17. Figure 1 depicts an example of an industry‑specific • $248 million appropriated for fiscal year 2020–21. CTE dual‑enrollment program for which an LEA [Figure 1] • Requires each regional consortium to submit a regional received a workforce grant in fiscal year 2019–20. plan, updated each year, that analyzes labor market needs, including wage data, and prioritizes projects and programs that close relevant labor market and Program Roles and Responsibilities employment gaps. Source: State law. Several entities are involved in administering and implementing the workforce program’s K–12 component. Figure 2 on page 9 identifies these entities and their respective roles. In fiscal year 2018–19, the Chancellor of the California Community Colleges (Chancellor’s Office) began annually apportioning $150 million in state funding to the regional consortia for workforce grants. The Chancellor’s Office apportions workforce grant funding to each regional consortium according to a statutory formula based on each region’s unemployment rate, the region’s total average daily attendance for pupils in grades seven through 12, and the region’s proportion of the State’s total projected job openings. The Chancellor’s Office also annually issues a request for applications (RFA) that announces the availability of funding through the workforce program and establishes minimum requirements that an application must meet to be considered for [Figure 2] [Figure 3] funding (eligibility criteria). Each regional consortium is required to administer a competitive grant program to distribute the funding it receives under the K–12 component of the workforce program to LEAs. To do so, each regional consortium establishes a workforce grant selection committee (selection committee) made up of individuals with expertise in K–12 CTE and workforce development, such as current or former K–12 CTE teachers and administrators, community college faculty or administrators, and other K–12 education stakeholders. These individuals may include employees of the LEAs that apply for workforce grants. The regional consortia are California State Auditor Report 2021-101 7 February 2022 Figure 1 A Workforce Grant Provided Funding for a Dual-Enrollment Pharmacy Technician Pathway The CTE program allows students to complete high school graduation requirements and enroll in community college pharmacy technician courses. Students participate in a series of community college courses offered on the LEA’s campus. AHS 51 PHT 101 PHT 102 PHT 103 Health Pharmacy Pharmacy Pharmacy Careers Careers A Careers B Careers C Exploration Students graduate from high school and earn a pharmacy technician certification that can be used to pursue further education or career options. UNIVERSITY Source: Fiscal year 2019–20 workforce grant application and award data. 8 California State Auditor Report 2021-101 February 2022 responsible for training selection committees and for ensuring that selection committees use strong safeguards to prevent conflicts of interest from influencing their decisions. Figure 3 on page 10 shows the eight regions statewide that the regional consortia represent. Each selection committee has exclusive authority under state law to determine the recipients of workforce grants in its region and the specific amount of funding for each grant. In addition to the eligibility criteria disclosed in the RFA, selection committees can use other criteria to decide how to award limited funds to all eligible applications (allocation criteria) or to decide which eligible applications to fund and which to exclude (selection criteria). LEAs have generally requested more grant funding for CTE than the State has provided through both the workforce program and another grant program, the CTE Incentive Grant Program, which we describe below. The State provides $12 million annually to fund two categories of positions (support positions) referenced in Figure 2—technical assistance providers and workforce pathway coordinators—that support both the workforce program and the CTE Incentive Grant Program. The State funds one technical assistance provider for each regional consortium and one workforce pathway coordinator within the geographical boundaries of each community college district, unless otherwise determined by the state superintendent of public instruction and the Chancellor’s Office.2 Technical assistance providers serve as consultants to the selection committees by supporting their application review processes and training selection committee members. Among other responsibilities, workforce pathway coordinators help LEAs implement CTE programs and integrate available local, regional, state, and private resources to ensure that students achieve successful work outcomes. CTE Incentive Grants LEAs may also apply for funding to support their CTE initiatives through the CTE Incentive Grant Program, which the California Department of Education (Education) has administered since fiscal year 2015–16. Similar to workforce grants, CTE incentive grants are intended to encourage, maintain, and strengthen the delivery of high‑quality CTE programs. The technical assistance providers and workforce pathway coordinators that the State funds through the workforce program are also responsible for supporting LEAs that apply for CTE incentive grants. From fiscal years 2018–19 2 The State’s online community college district does not encompass a specific geographical boundary and is not assigned a workforce pathway coordinator. California State Auditor Report 2021-101 9 February 2022 Figure 2 Several Entities Are Involved in the Workforce Grant Process CHANCELLOR’S OFFICE Apportions state funds to regional consortia based on each region’s: • Unemployment rate • Total average daily attendance for pupils in grades 7 to 12 • Proportion of projected job openings REGIONAL CONSORTIA Each regional consortium establishes a grant selection committee and facilitates its training and operations. 72 Workforce LEAs Pathway Coordinators Apply for grants to support, create, or expand CTE. Assist LEAs with implementing The grant recipients may be: CTE programs funded with workforce • School districts • County offices of education and CTE incentive grants, and • Charter schools • Regional occupational centers or certain programs collaborate with community colleges to develop CTE pathways. 8 Technical Assistance SELECTION COMMITTEES Providers Determine who is eligible for an award and the amount. Support LEAs and serve as consultants Members include representatives of: to the selection committees by training • K–12 education selection committee members and • Community colleges • Industries prioritized by the consortium supporting their application review processes. CTE PROGRAMS Integrate core academic knowledge with technical and occupational knowledge to provide students in grades 7 to 12 with pathways to careers and postsecondary education. Source: State law and Chancellor’s Office guidance. 10 California State Auditor Report 2021-101 February 2022 through 2020–21, the State funded both CTE incentive grants and workforce grants at $150 million annually for each program. In fiscal year 2020–21, applicants requested approximately $311 million in CTE incentive grants and approximately $222 million in workforce grants. Starting in fiscal year 2021–22, annual funding for CTE incentive grants increased to $300 million. Funding for workforce grants remains at $150 million per year. Figure 3 Eight Regional Consortia Administer Workforce Grants NORTH / FAR NORTH BAY AREA CENTRAL / MOTHER LODE SOUTH CENTRAL COAST INLAND EMPIRE / DESERT LOS ANGELES ORANGE COUNTY SAN DIEGO / IMPERIAL Source: State law, workforce program regional plans, and the Chancellor’s Office website. California State Auditor Report 2021-101 11 February 2022 Audit Results The Chancellor’s Office Has Not Directed Applicants to Provide Sufficient Detail to Demonstrate That Their CTE Programs Will Meet Regional Needs The Chancellor’s Office has not directed applicants for workforce grants to provide key details Eligibility Criteria for demonstrating their CTE programs’ ability to meet Fiscal Year 2020–21 Workforce Grants some of the workforce program’s goals. State law requires selection committees to give the greatest To be considered eligible for funding, applications had to weight to applications that best meet regional obtain an average score of 75 points or more based on the following factors: economic needs, benefit underserved students (such as English learners and foster youth), focus on POINTS students from populations with high dropout rates, SCORING FACTOR (OF 100 MAXIMUM) and are located in areas of high unemployment. Address a problem or need We refer to these factors collectively as the that, among other things, is 9 factors of greatest weight. The fiscal year 2020–21 informed by the regional plan. RFA included these factors of greatest weight Provide clear, concrete within the eligibility criteria, shown in the text objectives to address the 8 box, that all selection committees were to use to problem or need. determine which applications would be considered Describe the activities for funding. and strategies that will be 60 implemented to achieve the objectives. The absence of detailed information in applications on the factors of greatest weight prevents selection Prepare a budget that provides committees from identifying those applicants that descriptions and identifies 15 matching funds. can best support the State’s priorities for CTE. For example, our review found several instances Serve areas of substantial in which the information applicants provided unemployment, rural school 8 districts, or underserved did not sufficiently address regional needs. We student populations. reviewed applications submitted to the selection committees in the Bay Area, Los Angeles, and Source: Fiscal year 2020–21 workforce program RFA. North/Far North regions. Although our review of 30 awarded applications found that the selection committees generally made decisions appropriately, many applications did not contain information at a sufficient level of detail for the regional needs factor. Table 1 shows that for 16 applications we reviewed that sought grants for CTE programs pertaining to specific industries, eight did not contain quantifiable information about the demand for skilled workers in the industry and 15 did not include wage data for the industry, both of which would have helped demonstrate that the CTE programs seeking funding would meet regional needs. 12 California State Auditor Report 2021-101 February 2022 Table 1 Many of the 16 Awarded Applications That We Reviewed Pertaining to Specific Industries Did Not Address Characteristics Related to Regional Needs CHANCELLOR’S OFFICE’S NUMBER OF GUIDANCE CONTAINED APPLICATIONS MISSING AN EXAMPLE OF THIS CHARACTERISTIC THE CHARACTERISTIC PERCENTAGE CHARACTERISTIC Address demand for skilled workers as a factor ü 0 0% demonstrating regional economic need Quantify demand for skilled ü workers in the industry the 8 50 grant would support Address wages as a factor demonstrating regional 10 63 X economic need Quantify wages in the industry 15 94 X the grant would support Source: Application and award data for fiscal years 2019–20 and 2020–21, and Chancellor’s Office guidance. Although LEAs are ultimately responsible for the quality of the applications they submit, it is reasonable to expect that the Chancellor’s Office’s guidance would address the nature and quantity of information to include in their applications. For example, the RFA could instruct applicants to include information from the regional plans on an industry’s economic needs and priorities, such as the number of jobs that are expected to be available in the near future and the wages that workers filling those jobs could expect to earn. For instance, North/Far North’s regional plan for 2019 through 2022 prioritizes both the retail, North/Far North Regional Plan Information hospitality, and tourism industry and the information and communications technology industry. As the text Regional living wage: box demonstrates, the regional plan’s wage data • $22,000 to $27,000 annually (depending on the county) indicate that potential earnings could vary Industry median earnings— significantly, which is information that could help retail, hospitality, and tourism: selection committees evaluate how to award grant • $27,000 annually funds to achieve the greatest positive impact. Industry median earnings— However, instead of directing applicants to provide information and communications technology: data on demand for skilled workers and wages to • $83,000 annually demonstrate how grants will meet the needs Source: North/Far North Regional Consortium’s 2019 through identified in their regional plan, the RFA simply stated 2022 regional plan. that the information provided should include “local/ regional workforce need informed by your region’s regional plan.” Moreover, the examples that the Chancellor’s Office provided on its website to indicate the level of detail applicants should provide in their workforce grant applications did not include detailed wage information, such as hourly pay. California State Auditor Report 2021-101 13 February 2022 In contrast to the elements described above, the RFA specifically instructed applicants to provide information regarding underserved student groups, and the examples from the Chancellor’s Office included descriptions of the specific number of students from an underserved population that the example CTE program would enroll. The inclusion of this information in the RFA and the examples appears to have had a significant impact on the information included in applications, as 27 of the 30 awarded applications we reviewed included sufficient detail regarding underserved students to be served by the CTE program. Furthermore, consistent with the Chancellor’s Office’s examples, most of these 27 applications specified the number of underserved students the applicant expected to complete the proposed CTE program. Such detail allows selection committees to evaluate whether applications align with the workforce program’s intent. The lack of complete and sufficient information in many Incomplete and insufficient applications prevented the selection committees from considering information prevented selection relevant evidence that would have helped them ensure that they committees from ensuring that they awarded grants to those applicants that would best meet the awarded grants that would best workforce program’s goals. The assistant vice chancellor of the meet the workforce program’s goals. workforce and economic development division of the Chancellor’s Office (assistant vice chancellor) explained that the Chancellor’s Office’s approach to providing additional guidance to applicants has been to ensure that the program support staff—technical assistance providers and workforce pathway coordinators—provide support and information to applicants and direct them to work with their regional consortium to present more accurate information in their applications. However, as our review shows, this approach has been insufficient. The Chancellor’s Office Has Not Given Applicants Sufficient Information on Grant Selection Criteria The Chancellor’s Office has not adequately informed LEAs about the factors that selection committees use to award workforce grants. As we describe in the Introduction, the Chancellor’s Office began apportioning workforce grant funds to the regional consortia in fiscal year 2018–19. From the first year of the workforce program, the Chancellor’s Office has assumed responsibility for annually issuing an RFA on behalf of the regional consortia. The RFA establishes certain components of the application process that are uniform for applicants and selection committees statewide. For example, the fiscal year 2020–21 RFA described the entities that were eligible to apply for funds, certain information applicants were required to provide, the minimum number of selection committee reviewers who would score each application, and the appeals process applicants could follow. However, the RFA did not 14 California State Auditor Report 2021-101 February 2022 establish a uniform process for selecting the recipients of workforce grant awards or disclose the factors individual selection committees would use to do so. In fiscal year 2020–21, each selection committee established additional criteria because the total requested funds in each region’s eligible applications exceeded the amount allocated to that region. Thus, the selection committees decided either to exclude some eligible applications by creating additional criteria, which we describe as selection criteria, or to fund all eligible applications at some level of the amounts requested, which we describe as allocation criteria.3 Five of the eight regions used allocation criteria in fiscal year 2020–21. For example, the eligible applications in the Bay Area region contained requests for a combined $50.4 million in funding, but the region was allocated only $30.8 million. The Bay Area selection committee chose to fund all eligible applications, and it offered the eligible applicants from 38 percent to 100 percent of the amounts they requested, based primarily on their eligibility scores and the amount of funding requested. The other three regions used various types of selection criteria to exclude certain eligible applications during fiscal year 2020–21. For example, eligible applications in the South Central Coast region requested a total of $17.6 million in funding, but the region was allocated approximately $10 million. In contrast to the Bay Area’s approach, the South Central Coast’s selection committee addressed this difference by awarding funds to only one application submitted by each LEA, regardless of the number of eligible applications the LEA submitted. The fiscal year 2020–21 RFA did not The fiscal year 2020–21 RFA did not disclose either the selection describe the criteria each region’s criteria or the allocation criteria that selection committees selection committee would use to would use to determine which applications would receive grant allocate these limited funds among awards or the amounts of those awards. The RFA specified the the eligible applications. eligibility criteria, summarized in the text box at the beginning of the Audit Results section, that the selection committees were to use to identify which applications were eligible to be funded. However, it did not describe the criteria each region’s selection committee would use to allocate these limited funds among the eligible applications. Instead, the RFA stated that selection committees could take a variety of factors into consideration in making their funding decisions and were not required to fund the highest‑scoring applications. 3 State law requires selection committees to consider past performance of grantees before awarding additional funds to those reapplying for grants. The fiscal year 2020–21 RFA stated that selection committees would consider this factor, and some selection committees chose not to fund otherwise eligible applications from applicants that performed poorly in their use of a previous workforce grant. California State Auditor Report 2021-101 15 February 2022 In contrast, Education discloses the criteria it plans to use to select CTE incentive grant recipients. Education issues a CTE incentive grant RFA that describes the eligibility criteria and specifies that the number of grant awards will be based on the number of eligible applications and the amount of available funding. When the total amount Education’s Method for requested for fiscal year 2020–21 CTE incentive Allocating CTE Incentive Grants grants exceeded the funds available, according to the director of Education’s career and college Grant applicants are grouped into small, medium, and large categories based on their average daily attendance, and transition division (division director), Education within each category they are allocated funds as follows: awarded a share of available funding to all eligible applications, using the method the text box • Base amount: 70 percent of available funding that describes. Education has used the same or very Education awards to each eligible application based on similar allocation formulas for the base amount in the applicant’s average daily attendance. each year since the beginning of the program. It has • Supplemental amount: 30 percent of available funding also publicly disclosed the base amount and that Education awards to each eligible application that supplemental allocation formulas when submitting meets certain characteristics established in state law, its recommended grant recipients to the State such as serving a pupil subgroup that has a higher Board of Education for approval at public meetings than average dropout rate. A portion of the funds is since fiscal year 2018–19. As a result, applicants assigned to each characteristic and, depending on the have access to details that can help them make characteristic, allocated to applicants based on their average daily attendance or pupil count. informed decisions about applying for CTE incentive grants. Source: State law and the State Board of Education’s January 2021 meeting agenda and minutes. Similarly, guidance about the information federal agencies should disclose regarding certain grant funding processes—which represents a potential best practice for the Chancellor’s Office to follow—also focuses on ensuring that applicants can make informed decisions. The guidance in federal regulations for federal agencies that award grants to nonfederal entities indicates that when an agency announces that funding is available, it should include both the criteria it will use to evaluate applications, which we refer to as eligibility criteria, and the selection criteria. Specifically, a federal agency’s announcement generally must list program policy or other factors that may be used to select applications for awards, such as geographical dispersion or diversity. In addition, federal regulations require the disclosure of award information, such as the expected amounts of individual awards or the average amount of funding per award experienced in previous years. By requiring federal agencies to disclose this information, the federal regulations are intended to maximize the fairness of the process by making it transparent so that applicants can make informed decisions when preparing their applications and deciding which grant opportunities to pursue. 16 California State Auditor Report 2021-101 February 2022 Conversely, a lack of transparency about the grant process can increase the risk of unfair outcomes. In fiscal year 2020–21, the Inland Empire/ Desert selection committee used selection criteria to eliminate some eligible applications. The committee decided to award a grant to every eligible application from LEAs that submitted only one application, but for LEAs that submitted multiple applications, to award a grant only to the application with the highest eligibility criteria score. With the remaining funding, the committee chose to award a grant to a second application from some LEAs. According to the chair of the Inland Empire/Desert regional consortium, these applications were selected based on their eligibility scores and requested amounts. However, we found the selection committee did not apply its selection criteria consistently. Specifically, we determined that it inappropriately denied an application from one LEA, the Elite Academic Academy— Lucerne charter school, which submitted a single eligible application. Based on the selection committee’s criterion of fully funding every eligible application for LEAs that submitted only one application, the committee should have funded this application. However, the Chancellor’s Office’s grant application system displayed miscalculated scores, which indicated the application was ineligible. The consortium chair stated that she was aware of the grant application system’s miscalculations, which the Chancellor’s Office has addressed for subsequent funding cycles, and that she communicated the issue to the selection committee co‑chairs to discuss before the committee made grant decisions. She also stated that, because the issue was under the co‑chairs’ purview, she allowed them to address it. The selection committee co‑chair we interviewed could not recall why the selection committee chose not to consider the application for funding. Ultimately, the application was not funded. Had the selection criteria been disclosed in a transparent manner, such as through the RFA, the charter school may have questioned why the selection committee did not follow its criteria and might have pursued an appeal. The lack of transparency about The lack of transparency about the workforce grant process also the workforce grant process limits limits potential applicants’ ability to make informed decisions about potential applicants’ ability to their applications. As Figure 4 shows, the selection committees in make informed decisions about the Inland Empire/Desert, Los Angeles, and South Central Coast their applications. regions all used selection criteria, and each denied grants to several applications that met the published eligibility criteria. Had the selection committees disclosed their selection criteria for workforce grants in advance, potential applicants would have benefited in several ways. First, understanding the selection criteria would have helped LEAs assess which of the State’s CTE grant programs to pursue, the type and quantity of information to include in their applications, and whether to invest the time to create more than one workforce grant application. Second, because LEA recipients of both workforce grants and CTE incentive grants typically must pledge $2 of matching funds for every dollar of grant funding, such detail would have allowed applicants to California State Auditor Report 2021-101 17 February 2022 Figure 4 Regional Selection Committees Used Various Criteria to Allocate Limited Funding or Exclude Eligible Applications in Fiscal Year 2020–21, None of Which Were Disclosed to Applicants in the RFA The RFA disclosed standard eligibility criteria. None of the regions were allocated enough money to fully fund all eligible applications. Five committees awarded funds to Three committees excluded some eligible all eligible applications and used applications through the use of additional allocation criteria selection criteria to award each eligible application a that were not disclosed in the RFA:* portion of the funds requested. INLAND EMPIRE / DESERT LOS ANGELES SOUTH CENTRAL COAST Selected each LEA’s Selected the highest-ranking Selected the highest-scoring highest-scoring eligible eligible applications, eligible application application for generally based on from each LEA, regardless of full funding and the LEA average daily attendance the number of eligible next three highest-scoring and eligibility score. applications submitted. eligible applications for some funding. Selection committee Selection committee Selection committee denied denied denied 3 of 26 9 of 46 6 of 14 eligible applications eligible applications eligible applications Source: Application award data, fiscal year 2020–21 workforce program RFA, selection committee meeting minutes for fiscal year 2020–21, and interviews with regional consortia personnel. * In addition to excluding some eligible applications, based on selection criteria, these three committees awarded certain other applications less than the requested amount of funding. 18 California State Auditor Report 2021-101 February 2022 better decide how to allocate scarce financial resources between workforce grant and CTE incentive grant applications. Providing the selection criteria in each RFA is also important because selection committees can modify their selection criteria for each year’s funding, and thus applicants cannot [Figure 4] assume that the selection criteria from a prior year are still relevant. Including additional detail about each committee’s selection criteria in the RFA would likely alter some LEAs’ strategies for applying for these funds. For instance, in fiscal year 2020–21, five LEAs in the South Central Coast region each submitted two or three applications, which aligned with the RFA’s statement that LEAs were limited to submitting no more than three applications. However, in accordance with South Central Coast’s fiscal year 2020–21 selection criteria, they each received funding for only one application. Had the selection criteria been transparent from the beginning, the applicants might have pursued funding for only their highest‑priority application rather than submitting multiple applications for the selection committee to consider. Similarly, had applicants in the Los Angeles region known about the selection committee’s decision not to award funds to all applications, they might have made different decisions regarding their applications or chosen instead to focus their efforts on applying for funding through the CTE incentive grant program, which awarded some level of funding to all eligible applicants. One reason the RFA did not include selection criteria is that selection committees generally did not convene to determine selection criteria until after the RFA was published and the deadline to submit applications had passed, as Figure 5 shows. This sequence of events prevented the Chancellor’s Office from including in the RFA information that would have maximized the fairness of the competitive grant process for applicants. However, selection committees are not precluded from convening to determine their selection criteria before the Chancellor’s Office issues the RFA. Some regional consortia chairs also stated that the consortia have had limited opportunity to clarify the content in the RFA. According to the chair of the Inland Empire/Desert consortium, that region’s selection committee did meet to discuss its priorities before the RFA was released and, during regional engagement meetings for fiscal year 2020–21, the consortium shared with applicants that the selection committee would try to fund grants broadly across all geographic areas of the region. She stated that it would have been best to share this information in the RFA to ensure that all applicants had the same information, but she said that the consortium did not have an opportunity to provide regional information for inclusion in the RFA. The assistant vice chancellor The assistant vice chancellor agreed agreed that publishing selection criteria would increase the fairness that publishing selection criteria and transparency of the competitive grant process. She stated that the would increase the fairness and Chancellor’s Office could include regional addenda in the RFA to provide transparency of the competitive applicants with more information regarding regional selection criteria grant process. prior to the deadline to submit applications. California State Auditor Report 2021-101 19 February 2022 Figure 5 After LEAs Submitted Applications, Selection Committees Adopted Additional Criteria to Determine Awards SUBMITTED The Chancellor’s Office LEAs submitted workforce Members of each selection committee The total amount of funding released the workforce grant grant applications in the scored their region’s applications requested by eligible applications RFA, which provided Chancellor’s Office’s grant application using the eligibility criteria disclosed exceeded the amount available, eligibility criteria to applicants. system for consideration by the in the RFA to determine which applications so the selection committees adopted respective regional selection committee. were eligible to receive funds. allocation criteria or selection criteria to determine awards. Source: RFAs for the workforce grant program from fiscal years 2018–19 through 2021–22, regional consortia selection committees’ meeting minutes, application scoring data, and award data. The Chancellor’s Office Has Not Ensured That LEAs Have Equal Access to Local Support Staff LEAs do not have equal access to the workforce pathway coordinators that help them build partnerships with community colleges and pursue workforce grant funding. Unless otherwise determined by the Chancellor’s Office and the superintendent of public instruction, state law provides one workforce pathway coordinator for each community college district. The workforce pathway coordinators’ role is to improve the performance of K–12 and community college CTE programs. They do so by providing support services to the LEAs within their district, such as helping LEAs implement CTE programs and collaborating with community colleges on behalf of LEAs so that LEAs can develop CTE pathways from the K–12 system to community college. Because workforce grants require partnerships between LEAs and community colleges, workforce pathway coordinators serve an important role in providing support for applicants. However, the number of LEAs within community college districts can vary, as Figure 6 depicts, thereby affecting workforce pathway coordinators’ ability to provide sufficient support to those LEAs. [Figure 6] The chairs of some of the regional consortia informed us that they have expressed concerns about unequal access to workforce pathway coordinators for LEAs in different community college districts during informal conversations that included the Chancellor’s Office. According to the chairs, the Chancellor’s Office indicated that state law requires workforce pathway coordinators to be assigned one per community college district. However, the general counsel for the Chancellor’s Office said that state law gives the Chancellor’s Office and the state superintendent of public instruction the authority to alter the assignment of workforce pathway coordinators within or across the boundaries of community college districts. Education’s 20 California State Auditor Report 2021-101 February 2022 Figure 6 Some LEAs Do Not Have Equal Access to Local Support Staff An LEA within Palomar Community College District may have a harder time obtaining program support than an LEA within MiraCosta Community College District even though both LEAs are in the San Diego/Imperial region. The workforce pathway coordinator for The workforce pathway coordinator for Palomar Community College District MiraCosta Community College District is responsible for assisting: is responsible for assisting: 38 10 LEAs LEAs Source: State law and San Diego/Imperial Regional Consortium data on workforce pathway coordinator service areas. division director indicated that Education is willing to collaborate with the Chancellor’s Office to exercise that authority. The assistant vice chancellor stated that workforce pathway coordinators have already been collaborating to address inequities in their regions. Nonetheless, all of the regions indicated that the current method of assigning workforce pathway coordinators can affect LEAs’ access to support and that more flexibility—such as restructuring service areas—would be beneficial. Furthermore, the Chancellor’s Office did not issue hiring guidance to entities in time for them to hire some of the support staff positions that would have benefited applicants during the fiscal year 2018–19 and 2019–20 rounds of the workforce and CTE incentive grant programs. Beginning in fiscal year 2018–19, the State has appropriated $12 million annually for technical assistance providers and workforce pathway coordinators. State law requires the Chancellor’s Office, with the state superintendent of public instruction, to administer competitive California State Auditor Report 2021-101 21 February 2022 processes for selecting the technical assistance providers and workforce pathway coordinators. The Chancellor’s Office provided guidance to the regional consortia chairs directing the regional consortia to select local entities, such as LEAs and community college districts, that would hire the support positions. However, as Figure 7 shows, the Chancellor’s Office did not release guidance on the hiring of workforce pathway coordinators until after the application deadlines for the first two rounds of workforce grant funding. Figure 7 Two Rounds of Grant Applications Were Due Before the Chancellor’s Office Issued All Guidance for Hiring Local Support Positions MARCH 15, 2019 DECEMBER 18, 2019 OCTOBER 16, 2020 Applications Due Applications Due Applications Due ROUND 1 ROUND 2 ROUND 3 (Fiscal year 2018–19) (Fiscal Year 2019–20) (Fiscal Year 2020–21) MARCH 25, 2019 DECEMBER 20, 2019 The Chancellor’s Office issued guidance The Chancellor’s Office issued guidance for hiring technical assistance providers. for hiring workforce pathway coordinators. APRIL 20, 2020 The Chancellor’s Office proposed that all workforce pathway coordinators would be hired by this date. Source: State law, fiscal years 2018–19 through 2020–21 workforce program RFAs, and Chancellor’s Office memoranda. The assistant vice chancellor indicated that as a result of changes in leadership at the Chancellor’s Office, staff did not promptly obtain approval from its board of governors to release funds for the support positions. In addition, chairs of several of the regional consortia explained that some of the community college districts and LEAs that hired support staff members were slow to do so because of several factors, including the time needed to create new positions and, subsequently, complications caused by the COVID‑19 pandemic. All but one of these positions had been initially filled by January 2022.4 Because of the delay in hiring individuals to fill technical assistance provider and workforce pathway coordinator positions, a significant portion of funds appropriated for fiscal year 2018–19 remains unused. The Chancellor’s Office granted one community college district $12 million in both fiscal years 2018–19 and 2019–20 for a number of 4 According to the Los Angeles region’s technical assistance provider, the position not yet filled is for the Compton Community College District, and the LEA that was initially responsible for filling the position failed to do so. She also stated that a new LEA was selected to fill the position. The newly selected LEA’s CTE coordinator stated that the position would be filled in early 2022. 22 California State Auditor Report 2021-101 February 2022 purposes, including administering subgrants to other community Although the regional consortia all college districts and to LEAs to hire support positions statewide. indicated that they had processes The subgrants for fiscal year 2018–19 funding ended in December to identify and avoid conflicts of 2020. However, several community college districts and LEAs that interest, only two maintained received amounts from the $12 million in fiscal year 2018–19 funds internal documentation proving to hire support positions either withdrew from the subgrants or did that they followed their processes. not use all of the funds. According to the assistant vice chancellor, this was due to hiring delays. These unused funds total $2.6 million. The Chancellor's Office has not yet Although state law requires these unused funds to be added to the added $2.6 million in unused funds amount appropriated to the regional consortia to create, support, to the amount appropriated to or expand CTE, the Chancellor’s Office has not yet done so. After create, support, or expand CTE. we discussed this issue with the Chancellor’s Office, the assistant vice chancellor sent an email to the workforce program dean indicating that they needed to create a process to account for and distribute the unspent funds. The workforce program dean expects the Chancellor’s Office to release a formal memo in early 2022 documenting that process. The assistant vice chancellor stated that they are still reconciling the amount of the unspent funds and determining each regional consortium’s share. She said that, beginning in January 2022, selected community college districts will be responsible for managing the contracts and funds for support positions within their regions, which will allow them to take the lead in calculating and reallocating unspent funds, subject to the Chancellor’s Office’s monitoring and oversight. Most Regional Consortia Have Not Ensured That Selection Committees Adopt Strong Safeguards Against Unfair Decisions Most selection committees have insufficient safeguards in place to avoid unfair decisions. Because selection committee members may work for the same LEAs that apply for workforce grants, there is an inherent risk that conflicts of interest will influence grant decisions.5 In November 2019, the Chancellor’s Office issued guidance requiring selection committee members to disclose any potential conflicts of interest and to avoid being involved in any grant decision in which they have a perceived conflict of interest. However, six of the eight regional consortia did not formally document their efforts to ensure that selection committee members follow this guidance. Without such evidence, the regional consortia increase the risk that their workforce grant processes are, or will be perceived as, unfair. 5 As used in this report, the term conflict of interest means a real or seeming incompatibility between one’s private interests and one’s public or fiduciary duties. California State Auditor Report 2021-101 23 February 2022 purposes, including administering subgrants to other community Although the regional consortia all indicated that their selection Although the regional consortia all college districts and to LEAs to hire support positions statewide. committees had processes to identify and avoid conflicts of indicated that they had processes The subgrants for fiscal year 2018–19 funding ended in December interest when awarding fiscal year 2020–21 grants, only two to identify and avoid conflicts of 2020. However, several community college districts and LEAs that maintained internal documentation proving that they followed interest, only two maintained received amounts from the $12 million in fiscal year 2018–19 funds their processes. Specifically, the Bay Area and Inland Empire/ internal documentation proving to hire support positions either withdrew from the subgrants or did Desert consortia maintained written records identifying the various that they followed their processes. not use all of the funds. According to the assistant vice chancellor, entities that would benefit from each application if awarded. this was due to hiring delays. These unused funds total $2.6 million. They also documented the selection committee members who reported having an interest in those entities and demonstrated The Chancellor's Office has not yet Although state law requires these unused funds to be added to the that individuals with such an interest were not assigned to review added $2.6 million in unused funds amount appropriated to the regional consortia to create, support, those applications. In contrast, the chairs of the other regional to the amount appropriated to or expand CTE, the Chancellor’s Office has not yet done so. After consortia could not provide documentation that they had fulfilled create, support, or expand CTE. we discussed this issue with the Chancellor’s Office, the assistant their oversight duties to prevent conflicts of interest and ensure the vice chancellor sent an email to the workforce program dean integrity of the scoring and selection process. Some of the chairs indicating that they needed to create a process to account for and of the six regional consortia with insufficient safeguards in fiscal distribute the unspent funds. The workforce program dean expects year 2020–21 indicated that they have already started improving the Chancellor’s Office to release a formal memo in early 2022 their conflict‑of‑interest processes, while the others agreed that documenting that process. The assistant vice chancellor stated improvements could be made. For example, the Orange County and that they are still reconciling the amount of the unspent funds San Diego/Imperial regional consortia maintained documentation and determining each regional consortium’s share. She said that, showing that they checked for and avoided conflicts of interest for beginning in January 2022, selected community college districts will all entities involved when assigning selection committee members be responsible for managing the contracts and funds for support to review fiscal year 2021–22 applications. positions within their regions, which will allow them to take the lead in calculating and reallocating unspent funds, subject to the In addition, although selection committees use standard eligibility Chancellor’s Office’s monitoring and oversight. criteria to score applications, they do not have a common criterion for identifying scores from individual reviewers that vary significantly (outlier scores) and that could result in unfair Most Regional Consortia Have Not Ensured That Selection decisions. For example, a very low score from one reviewer Committees Adopt Strong Safeguards Against Unfair Decisions could skew the average score of an application below 75 points— the minimum score the RFA established for applications to be Most selection committees have insufficient safeguards in place considered for funding—even if the application’s scores from other to avoid unfair decisions. Because selection committee members reviewers were above 75. Applicants can appeal grant denials, may work for the same LEAs that apply for workforce grants, and there were 16 appeals of grant decisions statewide during the there is an inherent risk that conflicts of interest will influence first three years of funding. We reviewed documentation related grant decisions.5 In November 2019, the Chancellor’s Office issued to 10 of those appeals and found that seven of them questioned guidance requiring selection committee members to disclose any the variations in scoring among different reviewers for the same potential conflicts of interest and to avoid being involved in any application. However, only some selection committees require grant decision in which they have a perceived conflict of interest. additional review when scores for an application vary significantly However, six of the eight regional consortia did not formally among reviewers, and those committees have different thresholds document their efforts to ensure that selection committee members for what constitutes a significant variation. For example, the follow this guidance. Without such evidence, the regional consortia Bay Area selection committee assigned another selection increase the risk that their workforce grant processes are, or will be committee member to conduct a review of an application when the perceived as, unfair. high and low scores from the initial reviewers varied by more than 15 points. The South Central Coast selection committee did the same for differences greater than 30 points, and the Orange County selection committee allowed the committee chair to decide on a case‑by‑case basis the score variations that warranted additional 5 As used in this report, the term conflict of interest means a real or seeming incompatibility between one’s private interests and one’s public or fiduciary duties. review. The lack of a common threshold for identifying outlier scores introduces inconsistency into the application review process. 24 California State Auditor Report 2021-101 February 2022 In response to our concern, the chairs of the regional consortia agreed that the consortia could collaborate to develop a standard approach for determining outlier scores and the degree of variance that would warrant additional review of an application. Recommendations Chancellor’s Office To enhance the quality of information the selection committees have available when determining whether applications best meet the workforce program’s goals, beginning with the fiscal year 2022–23 grant application period, the Chancellor’s Office should do the following: • Specify in the RFA that applicants should include detailed information addressing all eligibility criteria, including information about the wage rates and demand for skilled workers in industries aligned with their CTE programs. • Provide examples that address all of the eligibility criteria. To ensure that all applicants can make informed decisions when applying for workforce grants, beginning with the fiscal year 2022–23 grant application period, the Chancellor’s Office should do the following: • Request selection committees to determine—before the Chancellor’s Office issues the RFA—how they will address requests for funding that exceed the total amount they are allocated and to inform the Chancellor’s Office of their decision and any selection criteria they will use so that it can include this information in the RFA. • Include in the RFA transparent and complete information about whether each selection committee has decided to fund all eligible applications and, if not, what selection criteria it will use. To provide LEAs equal access to the support staff who assist them with pursuing workforce grants, the Chancellor’s Office should establish and implement a process by June 2022 for a regional consortium to propose modifications to the areas that its workforce pathway coordinators are assigned to serve. The Chancellor’s Office should then obtain the state superintendent of public instruction’s agreement to restructure service areas when it determines that doing so will improve the equality of LEAs’ access to workforce pathway coordinators. California State Auditor Report 2021-101 25 February 2022 To ensure that unspent funds appropriated for support positions in fiscal year 2018–19 are used for the purposes of creating, supporting, or expanding CTE programs, the Chancellor’s Office should, before it issues the fiscal year 2022–23 RFA, add any unspent funds to the amount appropriated to the regional consortia for these purposes. Inland Empire/Desert Regional Consortium To allow the Elite Academic Academy—Lucerne an opportunity to have its inappropriately denied fiscal year 2020–21 application reconsidered, the Inland Empire/Desert regional consortium should encourage the charter school to reapply for a workforce grant and should apply its selection criteria consistently and correctly when making its award decision. Central/Mother Lode, Los Angeles, North/Far North, and South Central Coast Regional Consortia To ensure that their selection committees follow safeguards designed to avoid unfair grant decisions, beginning with the fiscal year 2022–23 grant application period, the Central/Mother Lode, Los Angeles, North/Far North, and South Central Coast regional consortia should each maintain internal documentation demonstrating its review for selection committee members’ potential conflicts of interest. This documentation should include a comparison of the entities that applied and the conflicts reported by each selection committee member. These consortia should also keep records showing that selection committee members did not review applications for which they had conflicts of interest. Regional Consortia To ensure consistency in scoring applications, the regional consortia should collaborate to establish a standard approach for addressing score variations that selection committees statewide will use, beginning with the fiscal year 2022–23 grant application period, when evaluating whether applications meet eligibility criteria. 26 California State Auditor Report 2021-101 February 2022 Blank page inserted for reproduction purposes only. California State Auditor Report 2021-101 27 February 2022 Other Area We Reviewed To address the objectives established for this audit, we reviewed information about the amount of funding applicants for workforce grants and CTE incentive grants requested. Portions of this review resulted in a recommendation that we have not previously presented in the report. During the course of our review, we determined that Education reported to the State Board of Education and the California Workforce Pathways Joint Advisory Committee (Advisory Committee) that applicants requested $900 million in CTE incentive grant funding for fiscal year 2019–20. However, according to Education’s data, applicants actually requested only $311 million. According to the division director, the amount that Education reported was incorrect because of a clerical error. Specifically, one applicant requested approximately $600,000, but Education staff performing data entry erroneously keyed the request as $600 million. This error was not identified before Education reported the $900 million figure. Because the individuals who prepared and presented the information no longer work for Education, the division director was unsure why the error was not identified and corrected. To avoid similar errors in the future, the division director stated that he has implemented multiple levels of review for data entry of CTE incentive grant information as of the 2021–22 funding year. However, multiple sources on Education’s website—including the webpage for the State Board of Education containing its March 2020 agenda and an agenda and presentation available on the Advisory Committee’s webpage—still listed the incorrect $900 million figure as of December 2021. As a result, policymakers and interested members of the public could be misled about the demand for CTE incentive grant funding. Recommendation To ensure that policymakers and interested members of the public have accurate information at their disposal, Education should work with any parties that have used this erroneous figure, including but not limited to the State Board of Education and the Advisory Committee, to publish corrections on their webpages that disclose accurate information. 28 California State Auditor Report 2021-101 February 2022 We conducted this performance audit in accordance with generally accepted government auditing standards and under the authority vested in the California State Auditor by Government Code section 8543 et seq. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, MICHAEL S. TILDEN, CPA Acting California State Auditor February 10, 2022 California State Auditor Report 2021-101 29 February 2022 Appendix Scope and Methodology The Joint Legislative Audit Committee (Audit Committee) directed the California State Auditor to conduct an audit of the K–12 component of the workforce program to determine whether the program’s processes for providing CTE grant funds to LEAs are consistent and appropriate. The table below lists the objectives that the Audit Committee approved and the methods we used to address them. Audit Objectives and the Methods Used to Address Them AUDIT OBJECTIVE METHOD 1 Review and evaluate the laws, rules, and Reviewed and evaluated federal and state laws; state and regional plans; and relevant regulations significant to the audit objectives. policies, procedures, and guidance of the Chancellor’s Office and the regional consortia. 2 Determine how many workforce pathway • Reviewed the Chancellor’s Office’s support position hiring guidance, support position coordinator and technical assistance provider subgrants, and regional hiring information to determine how many support positions positions have been filled since fiscal have been filled as of fiscal year 2020–21. year 2018–19. To the extent possible, determine • Interviewed staff of the Chancellor’s Office, Education, and the regional consortia to why unfilled positions have not been filled. determine the cause of delays in filling positions. 3 Determine how any funds not used to fill • Obtained contracts, expenditure data, and invoices to determine how fiscal year 2018–19 workforce pathway coordinator and technical and 2019–20 funds were spent and how much remains unspent. In addition to the assistance provider positions were used in fiscal $2.6 million in fiscal year 2018–19 funds that has not yet been spent, we determined years 2018–19 and 2019–20, including whether that, in total, approximately $4.1 million was spent for purposes other than filling support the funds were provided to consortia and used positions that were consistent with the program. Approximately $950,000 was spent on fees to support CTE programs. for the community college district that administered subgrants for the support positions during fiscal years 2018–19 and 2019–20. The remainder was used in fiscal year 2018–19, with approximately $460,000 spent on one‑time allocations to the regional consortia and approximately $2.7 million paid to vendors to provide assistance implementing the program. • Interviewed staff of the Chancellor’s Office and the fiscal agent and evaluated whether plans for spending the unused funds are consistent with requirements in state law. 4 Assess the processes that each consortium’s • Reviewed policies, procedures, records of selection committee meetings, and grant selection committee has developed for application and award records to identify the workforce grant award process at each of workforce grant applications, awards, and the eight regional consortia. appeals. Determine whether these processes • Interviewed staff of the regional consortia to understand their respective application are appropriate and consistent across all of the review, award selection, and appeals processes. selection committees. • Compared the regional grant review, award, and appeal processes to determine whether the selection committees were consistent. • Assessed the regional processes against criteria established in law for the workforce program and against recommended practices for administering competitive grant programs to determine whether the processes were appropriate. We determined the appeals process generally aligned with best practices. 5 For each selection committee, evaluate • Evaluated the sufficiency of training and instruction provided to selection committee the following: members by reviewing the standardized statewide training materials and each regional a. The training and instructions the individuals consortium’s training materials. We did not identify any significant differences or responsible for scoring applications deficiencies that could affect the consistency of application evaluations among regions. receive to determine whether they • Reviewed Chancellor’s Office guidance; obtained documentation from each regional are sufficient to ensure consistency in consortium; and interviewed regional consortia chairs, technical assistance providers, application evaluations. and selection committee members to identify the processes and procedures that are in b. The processes and procedures in place to place to prevent conflicts of interest from influencing decisions about public funds. avoid conflicts of interest when awarding • Assessed the strength of regional processes and procedures by comparing them to workforce funding. each other and to the Chancellor’s Office guidance and federal recommended practices for awarding competitive grants. continued on next page… 30 California State Auditor Report 2021-101 February 2022 AUDIT OBJECTIVE METHOD 6 For a selection of workforce funds awarded by • Judgmentally selected three selection committees—the Bay Area, Los Angeles, and three workforce program selection committees, North/Far North—taking into account factors including the process review performed determine whether funds were awarded fairly in Objective 4 and geographic location. and appropriately to LEAs. If funds were not • For the three selection committees, judgmentally selected a total of 30 awarded awarded fairly and appropriately, determine workforce grant applications from fiscal years 2019–20 and 2020–21 based on factors to the extent possible the reasons behind the including the amount of funding requested, the type of CTE programs proposed awarding decision. for funding, the type of LEAs applying for funding, and whether applications included characteristics prioritized in state law such as serving an area with a high unemployment rate or a rural region. • Compared each application in our selection against criteria established in federal and state laws and the RFA to determine whether the funds were awarded for CTE programs that fulfill program requirements, such as meeting regional economic needs. • We attempted to determine whether conflicts of interest might have influenced award decisions for these applications. Because state data on application reviewers’ names contained inaccuracies and one regional consortium whose committee we selected for testing did not maintain documentation showing committee members’ affiliations with grant applicants, we reviewed selection committees’ processes for preventing conflicts of interest. 7 Assess how the separation of CTE incentive • Reviewed the impact of splitting funding for CTE incentive grants and workforce grants grant and workforce program funding affects by comparing, from fiscal years 2018–19 through 2020–21, the number of applications the equity of awarding CTE funds to LEAs. received and awarded and the amount of requested funds awarded. • Reviewed the overlap in recipients of grants for both programs. • Evaluated whether the information disclosed in the workforce program RFA affects the equitable distribution of funds, and whether the CTE incentive grant RFA provides best practices for the workforce program to emulate. As described under Objective 4, we compared regional workforce grant review, award, and appeal processes to criteria and recommended practices to determine whether they were consistent and appropriate. • Identified changes that could improve how funds are awarded and obtained perspective on potential changes from relevant personnel at the Chancellor’s Office, Education, and the regional consortia. 8 Review and assess any other issues that are • For the three selection committees reviewed in Objective 6, judgmentally selected a significant to the audit. total of 15 workforce grant applications that were denied based on factors such as fiscal year, amount requested, type of LEA, and whether the LEA was located in an area of high unemployment or in a rural region. • Determined whether the application denials aligned with criteria established by state law, the RFA, and the selection committees. Source: Audit workpapers. Assessment of Data Reliability The U.S. Government Accountability Office, whose standards we are statutorily obligated to follow, requires us to assess the sufficiency and appropriateness of the computer‑processed information we use to support our findings, conclusions, and recommendations. In performing this audit, we relied on the following data and systems: Applications To select and review applications, we obtained access to the Chancellor’s Office’s electronic application review system used by LEAs to submit their workforce program applications and by California State Auditor Report 2021-101 31 February 2022 regional consortia for the application review and grant awarding processes. We also used reviewers’ individual scores recorded in this system to recalculate average application scores for some regions, and we attempted to use reviewers’ names recorded in the system and applications’ average scores, as described further below. We interviewed staff of the Chancellor’s Office regarding the data and reviewed key data fields to ensure that they contained logical data. The application review system is a paperless system, and because applicants submit applications and reviewers enter eligibility scores directly in the system, it was not feasible to perform accuracy testing of these data. To obtain assurance of their completeness, we compared summary totals to information the Chancellor’s Office has presented publicly. We determined that the Chancellor’s Office data were sufficiently reliable for the purpose of selecting applications for further review. We determined that the data were of undetermined reliability for the purpose of reviewing individual reviewers’ scores; however, there is sufficient evidence in total to support our findings, conclusions, and recommendations. While conducting the audit, we found that data we were provided for the purpose of testing for conflicts of interest in application decisions contained inaccurate information on reviewers’ names. As a result, we determined that the data were not sufficiently reliable for our purpose. Instead of testing for individual conflicts of interest, we assessed selection committees’ processes for avoiding conflicts of interest. In addition, we were informed of an issue with the application review system that affected the accuracy of the average scores recorded in the system for fiscal year 2020–21 applications. As a result, we concluded that the average score data contained within the application review system for these applications were not sufficiently reliable. For this reason, we reviewed data extracts that five of the regional consortia had obtained from the Chancellor’s Office’s system at the time they were assessing applications and we used these extracts in place of the average scores from the Chancellor’s Office’s system. We obtained these data extracts because they included scores that could be used to determine each application’s actual average score. Our assessment of those data is described in the next section. Regional Consortia Documentation for Fiscal Year 2020–21 Applications We obtained electronic data for fiscal year 2020–21 applications from five regional consortia that used data extracts from the Chancellor’s Office to assess applications: the Bay Area, Central/Mother Lode, Los Angeles, North/Far North, and South Central Coast regional consortia. To verify the completeness of the data, we compared the totals in the regional documentation 32 California State Auditor Report 2021-101 February 2022 against the list of applications in the Chancellor’s Office’s system in which all applications are submitted and found the data to be complete. We performed accuracy testing by tracing individual scores recorded in the Chancellor’s Office’s application system to data in the regional documentation for a haphazard selection of the five regions’ applications and did not identify any issues. Consequently, we found that the fiscal year 2020–21 application data for the Bay Area, Central/Mother Lode, Los Angeles, North/Far North, and South Central Coast accurately reflected the individual reviewers’ scores recorded in the Chancellor’s Office’s application system. As described above, we found those Chancellor’s Office data to be of undetermined reliability; however, there is sufficient evidence in total to support our findings, conclusions, and recommendations. Financial Data We obtained expenditure data from the Rancho Santiago Community College District (Rancho Santiago)—the entity responsible for distributing fiscal year 2018–19 and 2019–20 funds for the 80 support positions statewide—to determine whether all of the funds allocated for support positions were spent and, if not, how much money remained. To evaluate these data, we interviewed staff members knowledgeable about the expenditure data, verified key figures and the number of records in the data, and reviewed key data fields to ensure that they contained logical data. To determine whether the expenditure data were complete, we compared the allocated amounts listed on the expenditure data to the amounts appropriated in the state budget. To verify the accuracy of the expenditure data, we compared them to a selection of supporting documents for 10 haphazardly chosen workforce pathway coordinator positions and eight technical assistance provider positions. We identified two errors during this testing, and Rancho Santiago subsequently provided us with corrected data. We did not perform additional accuracy testing, and for this reason we determined that the data were of undetermined reliability. Although this determination may affect the precision of the numbers we present, the errors we identified were not material to our findings, conclusions, and recommendations. California State Auditor Report 2021-101 33 February 2022 * 1 * California State Auditor’s comments begin on page 37. 34 California State Auditor Report 2021-101 February 2022 2 California State Auditor Report 2021-101 35 February 2022 3 4 5 36 California State Auditor Report 2021-101 February 2022 Blank page inserted for reproduction purposes only. California State Auditor Report 2021-101 37 February 2022 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM THE CHANCELLOR OF THE CALIFORNIA COMMUNITY COLLEGES To provide clarity and perspective, we are commenting on the response to our audit from the Chancellor’s Office. The numbers below correspond to the numbers we have placed in the margin of the response. 1 The Chancellor’s Office’s response—that it will make explicit that applicants that do not address all of the eligibility criteria in their proposal may not be considered—does not align with our recommendation. We recommended that the Chancellor’s Office specify in the RFA that applicants should include detailed information addressing all eligibility criteria. As we describe beginning on page 11, such information would provide selection committees with relevant evidence that would help them ensure they award grants that best meet the workforce program’s goals. Further, although the selection committees are responsible for evaluating applications, the Chancellor’s Office can provide more consistent guidance regarding how to address all of the criteria, which benefits both applicants and selection committees. 2 The Chancellor’s Office appears to misunderstand our recommendation. We did not recommend the Chancellor’s Office to provide examples of eligibility criteria in the RFA. As we describe on page 12, the Chancellor’s Office has provided applicants with supplemental information to the RFA on its website. However, its examples of detail to include in applications were incomplete. Specifically, it did not provide examples demonstrating how to address certain required elements, such as industry‑specific wage information to address regional needs for certain CTE programs. We also disagree with the Chancellor’s Office’s statement suggesting that the regions develop these examples. As we describe on page 11, the Chancellor’s Office establishes the eligibility criteria that all selection committees use to evaluate applications and issues a single RFA that applicants in every region use to apply for funds. To ensure consistency in applicants’ responses to that standardized RFA, we believe that the Chancellor’s Office should provide standard examples of how to address all eligibility criteria. 3 It is unclear why the Chancellor’s Office disagrees with this recommendation, given that it agreed with the related previous recommendation. The Chancellor’s Office’s response to the previous recommendation indicates that it will work with the regional consortia to obtain information regarding selection criteria and include it in the RFA, so we would expect that it would also be willing to include complete information about whether selection committees decide to fund all eligible applications. As we describe on page 16, disclosing information regarding selection criteria in the RFA would help applicants make informed decisions about their applications. 38 California State Auditor Report 2021-101 February 2022 4 The Chancellor’s Office’s comment on maximizing funding misses the point of our recommendation. This recommendation does not pertain to the nature of the grants selection committees prioritize. Rather, it addresses the information applicants have at their disposal, which is intended to maximize the transparency and fairness of the grant process for applicants and enable them to make informed decisions about applying for grant funds. 5 The Chancellor’s Office did not address whether it intends to implement our recommendations to provide LEAs equal access to support staff and to provide unspent funds appropriated for support positions in fiscal year 2018–19 to the regional consortia. We look forward to receiving and evaluating the Chancellor’s Office’s 60‑day response describing its efforts to implement all of our recommendations. California State Auditor Report 2021-101 39 February 2022 January 24, 2022 Berkeley City College Michael S. Tilden, Acting California State Auditor 621 Capitol Mall, Suite 1200 Cabrillo College Sacramento, CA 95814 Cañada College Subject: Bay Area Community College Consortium Response to Chabot College Audit 2021-101, K-12 Strong Workforce Program City College of San Francisco Dear Mr. Tilden: College of Alameda Thank you for the opportunity to review and respond to the draft College of Marin audit report on the K-12 Strong Workforce Program. We heartily support any efforts to improve the outcomes of this annual College of San Mateo investment in strengthening the career pathway focused connection between K-12 and postsecondary education and we Contra Costa College appreciate the time and effort dedicated by the California State De Anza College Auditor’s Office and its staff in conducting your audit. Diablo Valley College As the K-12 SWP legislation recognizes, post-secondary education is virtually a requirement for obtaining employment that Evergreen Valley College pays family supporting wages. The state makes enormous Foothill College investments of general fund and local taxes in K-12 (~$60B) and postsecondary (~$21B) education. Substantial work has been Gavilan College done in connecting academic pathways across these systems, but far less has been done to align career oriented pathways Hartnell College from K-12 to postsecondary and into employment that pays family supporting wages. The investment of $162M ($150M for Laney College grants and $12M for technical assistance providers and pathway Las Positas College coordinators) provides the means to work towards better alignment of the far larger investments in these systems. Los Medanos College Focusing this investment on the connection between K-12 and Merritt College California Community Colleges is particularly important given Mission College more students enroll in community colleges in the year following completion of high school than enroll in UC, CSU and all public Monterey Peninsula College and private two- and four-year colleges combined (55% Community Colleges, 45% UC, CSU, all other). Aligning K-12 Napa Valley College and Community Colleges will benefit the largest number of Ohlone College students. Better alignment will improve students' preparation for college, improve retention and completion, and ultimately San Jose City College improve rates of job placement and transfer to career-oriented four-year college programs. Perhaps more important is the Santa Rosa Junior College opportunity to attract to a K14 career pathway the 36% of high school completers who are not showing up in postsecondary Skyline College education*. Solano College * Source: CDE, DataQuest - 2017-18 College-Going Rate for California High West Valley College School Students by Postsecondary Institution Type https://dq.cde.ca.gov/dataquest/DQCensus/CGRLevels.aspx?agglevel=State &cds=00&year=2017-18 40 California State Auditor Report 2021-101 February 2022 K-12 SWP is a relatively new program. The data systems to measure how effective it is are still being put in place, but anecdotally we are seeing significant progress in the development of programs and the alignment of these programs with community college programs. As with any new program of this magnitude there will be opportunities for improvement. We fully support the one recommendation the auditors have shared with us in the draft report. Recommendation: To ensure consistency in scoring applications, the regional consortia should collaborate to establish a standard for addressing score variations that selection committees statewide will use, starting with the fiscal year 2022-23 grant application period, when evaluating whether applications meet eligibility criteria. We look forward to working with our colleagues from the other regions to implement this recommendation. We appreciate the time and resources the State Auditor’s office dedicated to reviewing this program and your team's professionalism and diligence in their efforts to understand and evaluate the program. Sincerely, Rock Pfotenhauer Kit O'Doherty Co-chair Co-chair Don Daves-Rougeaux Sharon Turner K-14 Technical Assistance Provider K-14 Technical Assistance Provider California State Auditor Report 2021-101 41 February 2022 January 24, 2022 * Michael Tilden State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 Dear Mr. Tilden: On behalf of Central/Mother Lode Regional Consortium (CRC), thank you for your letter of January 18, 2022 and the opportunity to review and respond to your report, "K-12 Strong Workforce Program: State and Regional Administrative Shortcomings Limit the Program’s Effectiveness in Supporting Grant Applicants." We also want to thank your team members as they were very professional, flexible and kept us advised of the status of the audit and responded to our inquiries as it progressed. · redacted I have reviewed the recommendations included in the agency draft report. 1 Central/Mother Lode Regional Consortium agree with the recommendations made by the CA State Auditor Team. CRC has already implemented changes reflecting the recommendation given, specifically page 41, for the 2022-23 award year. Thank you again for the opportunity to review and comment on this report. If you have any questions, please contact me at 559-494-3705. Janice Offenbach Interim Regional Chair for CRC (exited as of 1/1/2022) 1171 Fulton Street, 4th Floor Fresno, CA 93721 * California State Auditor’s comment appears on page 43. 42 California State Auditor Report 2021-101 February 2022 Blank page inserted for reproduction purposes only. California State Auditor Report 2021-101 43 February 2022 Comment CALIFORNIA STATE AUDITOR’S COMMENT ON THE RESPONSE FROM CENTRAL/MOTHER LODE To provide clarity and perspective, we are commenting on the response to our audit from Central/Mother Lode. The number below corresponds to the number we have placed in the margin of the response. During the publication process for the audit report, some page 1 numbers shifted. The recommendation Central/Mother Lode cites in its response is our recommendation regarding safeguards to avoid unfair grant decisions, which appears on page 25 of our report. 44 California State Auditor Report 2021-101 February 2022 Blank page inserted for reproduction purposes only. California State Auditor Report 2021-101 45 February 2022 Inland Empire/Desert Regional Consortium c/o Riverside Community College District 3801 Market Street Riverside, CA 92501 (951) 222-8026 January 24, 2022 Michael S. Tilden Acting California State Auditor 621 Capital Mall, Suite 1200 Sacramento, California 95184 Dear State Auditor Tilden: The Inland Empire/Desert Regional Consortium (IEDRC) welcomes the opportunity to respond to the draft audit report, K-12 Strong Workforce Program, State and Regional Administrative Shortcomings Limit the Program’s Effectiveness in Supporting Grant Applicants, Report 2021-101. We appreciate the time and effort dedicated by the California State Auditor’s Office and its staff in conducting this important audit. The IEDRC fully recognizes the critical importance of ensuring that K12 LEAs have equitable access to K12 Strong Workforce funds. The IEDRC concurs with the intent of the recommendations noted in the draft audit report. Detailed responses to two recommendations follow: 1. Recommendation: Inland Empire/Desert Regional Consortium: To allow the Elite Academic Academy – Lucerne an opportunity to have its inappropriately denied fiscal year 2020-21 application reconsidered, the Inland Empire/Desert regional consortium should encourage the charter school to reapply for a workforce grant and should apply its selection criteria consistently and correctly when making its award decision. IEDRC Response: The IEDRC agrees with the recommendation and will encourage Elite Academic Academy – Lucerne to apply for K12 SWP funding in the 2022-23 grant application cycle. Further the K12 Pathway Coordinator assigned to Elite Academic Academy’s service area will ensure that Elite has access to regional resources necessary to submit an application. The IEDRC will also provide training and guidance to the region’s K12 Selection Committee to ensure its selection criteria are consistently and correctly applied when making all award decisions. 2. Recommendation: Regional Consortia: To ensure consistency in scoring applications, the regional consortia should collaborate to establish a standard for addressing score variations that selection committees statewide will use, starting with the fiscal year 2022-23 grant application period, when evaluating whether applications meet eligibility criteria. IEDRC Response: With input from the IEDRC’s K12 Selection Committee, the IEDRC Chair and K12 Technical Assistance Provider will collaborate with other regional consortia to establish a standard for addressing score variations used to evaluate whether applications meet eligibility criteria, and these criteria will be applied beginning with the 2022-23 grant application review cycle. 46 California State Auditor Report 2021-101 February 2022 Letter to the California State Auditor January 24, 2022 Page Two Thank you and your staff for your professionalism and cooperation during this audit. We note that improvements are needed and will work diligently to assess and implement your recommendations. Sincerely, Julie Pehkonen Chair, Inland Empire/Desert Regional Consortium California State Auditor Report 2021-101 47 February 2022 January 24, 2022 Mr. Michael S. Tilden * Acting State Auditor California State Auditor 621 Capitol Mall, Ste. 1200 Sacramento, CA 95814 Dear State Auditor Tilden: Thank you for the opportunity to review and respond to the audit regarding the Los Angeles and Orange County Regional Consortia K12 Strong Workforce Program State and Regional Administrative Shortcomings Limit the Program’s Effectiveness in Supporting Grant Applicants. As such, the K12 Strong Workforce Program key personnel agree with the following recommendations for the Los Angeles Regional Consortium and Orange County Regional Consortium. Recommendations 1. Los Angeles Regional Consortium: a) “To Ensure that their selection committees follow safeguards designed to avoid unfair grant decisions, beginning with the fiscal year 2022-2023 grant application and annually thereafter, the (redacted) Los Angeles, (redacted) regional consortia should each maintain internal documentation demonstrating its review of selection committee members’ potential conflicts of interest, including a comparison of the entities that applied and the conflicts reported by each selection committee member. These consortia should also keep records showing that selection committee members did not review applications when they had conflicts.” Response: Conflicts of Interests are critical practices to ensure objective review, scoring, and recommendations for funding for applications that meet the cut scores. To ensure this recommendation is implemented in the Los Angeles Regional Consortium, the Pasadena 1 Area Community College District key personnel should be contacted and provided this audit recommendation as Rancho Santiago Community College District key personnel no longer has purview over the Los Angeles Regional Consortium. As of January 1, 2022, the host district with administrative responsibility is Pasadena Area Community College District. b) “To ensure consistency in scoring applications, the regional consortia should collaborate to establish a standard for addressing score variations that selection committees statewide will use, starting with the fiscal year 2022-23 grant application period, when evaluating whether applications meet eligibility criteria.” Response: Developing consistency in scoring applications statewide is beneficial to streamlining and important to creating continuity and efficiencies in scoring practices from * California State Auditor’s comment appears on page 49. 48 California State Auditor Report 2021-101 February 2022 one regional consortium to another. To ensure this recommendation is implemented in the Los Angeles Regional Consortium, the Pasadena Area Community College District key personnel should be contacted and provided this audit recommendation as Rancho Santiago Community College District key personnel no longer has purview over the Los Angeles Regional Consortium. As of January 1, 2022, the host district with administrative responsibility is Pasadena Area Community College District. 2. Orange County Regional Consortium: a) “To ensure consistency in scoring applications, the regional consortia should collaborate to establish a standard for addressing score variations that selection committees statewide will use, starting with the fiscal year 2022-23 grant application period, when evaluating whether applications meet eligibility criteria.” Response: Developing consistency in scoring applications statewide is beneficial to streamlining and important to creating continuity and efficiencies in scoring practices from one regional consortium to another. The key personnel in the Orange County Regional Consortium looks forward to collaborating with regional consortia colleagues across the state to begin discussing and solving for this recommendation. The Orange County Regional Consortium key personnel further understands the significance of regional administrative oversight for the K12 Strong Workforce Program deliverables and outcomes and, upon publishing of this audit, will begin discussions and look forward to statewide collaborative opportunities to solve for consistency in scoring applications. Furthermore, it should be noted that the CEOs in the Los Angeles Regional Consortium were advised at the Los Angeles Orange County Regional Consortium was participating in a K12 Strong Workforce Program 1 audit with the California State Auditor office, during the regular standing meetings. The Los Angeles Regional Consortium is separate from the Orange County Regional Consortium and is now hosted at Pasadena Area Community College District. The Orange County Regional Consortium administrator, hosted at Rancho Santiago Community College District, no longer has purview over the Los Angeles Regional Consortium. As such, we commit to connecting you to the key personnel at Pasadena Area Community College District who can implement the recommendations from this audit. Thank you, again, for the opportunity to review the audit draft and respond to the recommendations. Best, Adriene L. Davis, Ed.D. Assistant Vice Chancellor, Economic and Workforce Development I Educational Services Regional Chair, Orange County Regional Consortium Former Executive Director, Los Angeles Orange County Regional Consortium Hosted at Rancho Santiago Community College District 2 | Los Angeles and Orange County Regional Consortia K12 SWP Audit Response 2323 N. Broadway, Suite 328, Santa Ana California 92706 • 714-480-7457 • laocrc@rsccd.edu California State Auditor Report 2021-101 49 February 2022 Comment CALIFORNIA STATE AUDITOR’S COMMENT ON THE RESPONSE FROM LOS ANGELES AND ORANGE COUNTY To provide clarity and perspective, we are commenting on the response to our audit from Los Angeles and Orange County. The number below corresponds to the number we have placed in the margin of the response. We provided the draft audit reports for the Los Angeles consortium 1 and the Orange County consortium to the regional chair who administered the workforce program for both entities at the time we were conducting the audit. We were aware that responsibilities would shift subsequently, and we look forward to receiving the Orange County consortium’s 60‑day response from its regional chair and the Los Angeles consortium’s 60‑day response from its new leadership as both take steps to implement the audit recommendations. 50 California State Auditor Report 2021-101 February 2022 Blank page inserted for reproduction purposes only. California State Auditor Report 2021-101 51 February 2022 January 24, 2022 TO: Michael S. Tilden, Acting California State Auditor * CC: Bonnie Roy, Team Leader FROM: Blaine Smith, Chair North Far North Regional Consortium (NFNRC) SUBJECT: 2021-101 – Confidential Draft Audit Report for Review (K-12 Strong Workforce Program) Dear Mr. Tilden: Thank you for providing the opportunity to respond to the draft audit. Enclosed for your review is a Word and PDF formatted document with the North Far North Regional Consortium written response to the draft report currently titled; K-12 Strong Workforce Program – State and Regional Administrative Shortcomings limit the Programs Effectiveness in Supporting Grant Applicants. A response to each recommendation directed to the North Far North Regional Consortium and additional comments can be found within the contents of the draft report. 1. To ensure that their selection committees follow safeguards designed to avoid unfair grant decision, beginning with the fiscal year 2022-23 grant application period and annually thereafter, the North/Far North regional consortia should each maintain internal documentation demonstrating its review for selection committee members’ potential conflicts of interest; including a comparison of the entities that applied and the conflicts reported by each selection committee member. These consortia should also keep records showing that selection committee members did not review applications where conflicts existed. The NFNRC agrees that starting in FY 2022-23 that they will maintain internal documentation demonstrating its review and process for selection committee members’ potential conflicts of interest. The region does have strong safeguards to avoid unfair grant decisions and will work to further document them. 2. To ensure consistency in scoring applications, the regional consortia should collaborate to establish a standard for addressing score variations that selection committees statewide will use, starting with the fiscal year 2022-23 grant application period, when evaluating whether applications meet eligibility criteria. The NFNRC does have a standard for addressing score variation and a process that is undertaken to protect the review processes. The region hopes to work with the other state region’s in adopting this process Statewide or an agreeable alternative for FY 2022-23. * California State Auditor’s comments appear on page 53. 52 California State Auditor Report 2021-101 February 2022 The NFNRC would also like to provide the following feedback on the audit title and statements made within the report. NFNRC believe some of the statements could be taken out of context, the law extends to provide flexibility by region. It should be noted that within different regions this audit highlights some areas of improvement and some 1 regions have more audit findings than others. Using the following title and statements does not define all regions. Most importantly, we think the title and excerpts do not accurately reflect the implementation in the NFNRC. I have selected two examples below: • Title of report – “State and Regional Administrative Shortcomings Limit the Program’s Effectiveness in Supporting Grant Applicants” NFN understands the two specific findings listed in the report regarding NFNRC, but do not agree that the findings limited how effective the administration of the program is and that it has had substantial “shortcomings” in administering the program in the NFNRC. • “Regional Consortia have not ensured that selection committees adopt strong safeguards against unfair decisions” 2 NFN understands the finding that we did not document our safeguards effectively enough for audit purposes regarding documentation of conflicts of interest. However, the NFN has adopted safeguards in training, reviewing applications, re-reviewing applications, selection committee deliberations, conversations, and final funding decision of the selection committee that a fair and equal award process takes place. Each NFN selection committee has taken the conflict of interest and awarding of funding controls seriously, to make sure there has been a fair review and award process. 3 Many times, throughout the draft report the term “some selection committees” OR “some of the regional consortia” is used. With using such vague descriptors, it could put un-do stress and questions directed toward the NFNRC because the reader of the document will be unsure if what is being said includes the NFNRC or doesn’t include the NFNRC in the explanation of the draft audit summary. We appreciate the time the State Auditor’s office has taken to identify improvements to strengthen our use of State CTE funds, maximize the benefit of those funds to students, and ensure compliance with applicable program requirements. Sincerely, Blaine Smith Blaine Smith- Chair – NFNRC California State Auditor Report 2021-101 53 February 2022 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM NORTH/FAR NORTH To provide clarity and perspective, we are commenting on the response to our audit from North/Far North. The numbers below correspond to the numbers we have placed in the margin of the response. We stand by the title of our report and the statements within it. 1 Our findings and conclusions are based on the evidence we obtained from all of the entities involved in the administration of the workforce program. North/Far North’s statement that it did not document its safeguards 2 effectively enough for audit purposes mischaracterizes our finding. We assessed whether its safeguards were sufficient to protect the public interest and the fairness of the grant process for applicants. As we describe on page 22, because selection committee members may work for the same LEAs that apply for workforce grants, there is an inherent risk that conflicts of interest will influence grant decisions. Consequently, strong safeguards are essential, and we are pleased that North/Far North states that it will implement our recommendation intended to prevent unfair grant decisions. Although certain portions of the report, such as the Results in Brief, 3 include statements that summarize information about multiple consortia, the Audit Results clearly identify the regional consortia to which our findings and recommendations pertain. 54 California State Auditor Report 2021-101 February 2022 Blank page inserted for reproduction purposes only. California State Auditor Report 2021-101 55 February 2022 TO: Michael S. Tilden, CPA * Acting California State Auditor 621 Capital Mall, Suite 1200 Sacramento, California 95814 FROM: Luann R. Swanberg, Director/Chair South Central Coast Regional Consortium Hosted at Santa Barbara Community College District DATE: January 24, 2022 SUBJECT: K-12 Strong Workforce Program: State and Regional Administrative Shortcomings Limit the Program’s Effectiveness in Supporting Grant Applicants Dear Mr. Tilden, Thank you for the opportunity to review and respond to the draft audit report on the K12 Strong Workforce Program. We appreciate the time, effort and professionalism of the State Auditor’s staff in conducting this audit and their responsiveness to questions and willingness to provide support during the process. The South Central Coast Regional Consortium (SCCRC) concurs with the two recommendations noted in the draft report. 1. RECOMMENDATION: Maintain internal documentation demonstrating its review for selection committee members’ potential conflict of interest, including a comparison of the entities that applied and the conflicts reported by each selection committee member. The consortia should also keep reports showing that selection committee members did not review applications when they had conflicts. SCCRC RESPONSE: The South Central Coast Regional Consortium agrees to refine how we document conflicts of 1 interest. Currently, the SCCRC maintains the following documents pertaining to conflict of interest: 1) Individual Regional Review Team and Regional Selection Committee member signed Conflict of Interest forms. 2) A detailed spreadsheet showing each Selection Committee member’s applications to be reviewed – including showing those they cannot read due to a potential conflict (area is blacked out for each Selection Committee member). Going forward, the South Central Coast Regional Consortium will institute an additional document: 3) Comparison of the entities that applied and the conflicts reported by each committee member. Each K12 SWP Round’s archived folder will include these 3 documents. 2. RECOMMENDATION: To ensure consistency in scoring applications, the regional consortia should collaborate to establish a standard to address score variations that selection committees statewide will use, starting in the fiscal year 2022-23 grant application period, when evaluating whether applications meet the eligibility criteria. * California State Auditor’s comments appear on page 57. 56 California State Auditor Report 2021-101 February 2022 2 3 4 5 4 California State Auditor Report 2021-101 57 February 2022 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM SOUTH CENTRAL COAST To provide clarity and perspective, we are commenting on the response to our audit from South Central Coast. The numbers below correspond to the numbers we have placed in the margin of the response. As we describe on page 23, the South Central Coast indicated 1 that its selection committee had processes to identify and avoid conflicts of interest, but it could not provide documentation that the committee followed those processes when awarding fiscal year 2020–21 grants. Our report text accurately characterizes the lack of transparency 2 about the grant award process. Regardless of the reasons for not including selection criteria in the RFA, which we address on page 18, that information was not disclosed to applicants before the selection committee deliberated. Nothing precludes selection committees from determining in 3 advance what their approach to awarding grants will be if eligible applications request more funding than is available. As we describe on page 18 another consortium’s selection committee did meet to discuss its funding priorities before the RFA was released. Although we do not dispute South Central Coast’s statement 4 that the total requests for grant funding cannot be known in advance, the consortium also states that requests for funding in its region have historically exceeded available funding. Thus, the South Central Coast’s selection committee should anticipate the need for and prioritize making decisions about the process it will use to address requests for funds that exceed the amount of funds available. To clarify, South Central Coast does not appear to dispute 5 the accuracy of the number of applications denied, nor do we take issue with the selection committee’s method of funding applications in fiscal year 2020–21. Our point on page 18 is that the RFA did not disclose information about selection criteria that potential applicants would have benefited from knowing. 58 California State Auditor Report 2021-101 February 2022 Blank page inserted for reproduction purposes only. California State Auditor Report 2021-101 59 February 2022 January 24, 2022 Michael S. Tilden, CPA * Acting California State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 Subject: K-12 Strong Workforce Program – Report 2021-101 dated February 2022 Dear Mr. Tilden: The California Department of Education (Education) appreciates the opportunity to provide comments and address the recommendation outlined in the California State Auditor’s (CSA) Audit Report titled, K-12 Strong Workforce Program. Recommendation 1 To ensure that policymakers and interested members of the public have accurate information at their disposal, Education should work with any parties that have used this erroneous figure, including but not limited to the State Board of Education and the Advisory Committee, to publish corrections on their webpages that disclose accurate information. Education’s Comments Concur. Education instituted multiple review levels of the information, data, and spreadsheet formula/results to ensure that the figures provided in future reports on CTE Incentive Grant funding are accurate. Additionally, Education will post a statement with the corrected figure shown on the California Workforce Pathways Joint Advisory Committee’s web page, which is located on Education’s web site, and inform the State Board of Education of the correct figure to update to its web page. It is important to note that the erroneous figure had no effect on the allocation results or 1 decision-making for the program. If you have any questions regarding Education’s comments, please contact Alice Lee, Director, Audits and Investigations Division, by phone at 916-323-1547 or by email at AlLee@cde.ca.gov. Sincerely, Mary Nicely Chief Deputy Superintendent of Public Instruction MN:kl * California State Auditor’s comment appears on page 61. 60 California State Auditor Report 2021-101 February 2022 Blank page inserted for reproduction purposes only. California State Auditor Report 2021-101 61 February 2022 Comment CALIFORNIA STATE AUDITOR’S COMMENT ON THE RESPONSE FROM EDUCATION To provide clarity and perspective, we are commenting on the response to our audit from Education. The number below corresponds to the number we have placed in the margin of the response. Education’s assertion that reporting the incorrect information had 1 no effect on program decisions is questionable. As we describe on page 10, annual funding for CTE incentive grants increased to $300 million in fiscal year 2021–22. Thus, Education’s reporting of the incorrect information may have led policymakers to believe the demand for funding was significantly greater than it actually was.