California Community Colleges Chancellor’s Office Report

California Community Colleges Chancellor’s Office Data Management Practices Review June 27, 2019 Michael H. Fine Chief Executive Officer Fiscal crisis & ManageMent assistance teaM June 27, 2019 Christian Osmeña, Vice Chancellor California Community Colleges Chancellor’s Office 1102 Q Street Sacramento, CA 95811 Vice Chancellor Osmeña: In February 2019, the California Community Colleges Chancellor’s Office and the Fiscal Crisis and Management Assistance Team (FCMAT) entered into an agreement to conduct a management assistance study on data management practices across the community college system and provide recommendations and suggested protocols to improve data quality and data management practices for data used in the Student Centered Funding Formula, enacted in 2018. Specifically, the agreement stated that FCMAT would perform the following: 1. Analyze data used in the Student Centered Funding Formula through 2017-18 and summarize trends in the data, noting any anomalies. 2. Conduct a review of data management responsibilities and practices related to the collection, review, reporting and transformation of data elements used in the Student Centered Funding Formula at 10-15 community colleges selected by the team as the study sample. The methodology for identifying the sample will include consideration of the college size, region, and whether a college is in a single-college district or a multicollege district. The review at each college/district in the selected sample will include both examination of documents and interviews with staff regarding data policies, procedures, responsibilities, and flow for the data elements used in the Student Centered Funding Formula. 3. Provide recommendations on best practices for data management designed to improve data quality, data collection practices, data flow and/or reporting at individual community colleges and between colleges, districts and the Chancellor’s Office. 4. In consultation with the Chancellor’s Office, provide sample protocols and audit procedures to improve data management practices designed to address issues identified in the review. This final report contains the study team’s findings and recommendations in the above areas of review. FCMAT appreciates the opportunity to serve the California Community Colleges Chancellor’s Office, and extends thanks to all the staff for their assistance during fieldwork. Sincerely, Michael H. Fine Chief Executive Officer Fiscal crisis & ManageMent assistance teaM i TABLE OF CONTENTS Table of Contents Foreword ............................................................................iii Introduction ........................................................................1 Background .....................................................................................................................1 Study and Report Guidelines ......................................................................................2 Sample Selection............................................................................................................2 Process for the Study ....................................................................................................4 Study Team ......................................................................................................................4 Executive Summary ...........................................................5 Findings and Recommendations .....................................9 Base Allocation ..............................................................................................................9 Supplemental Allocation ............................................................................................13 Student Success Allocation .......................................................................................17 Documentation and Sustainability .........................................................................27 CCCCO Communications .........................................................................................29 Appendices ....................................................................... 31 California Community Colleges ChanCellor’s offiCe ii Fiscal crisis & ManageMent assistance teaM iii ABOUT FCMAT About FCMAT FCMAT’s primary mission is to assist California’s local K-14 educational agencies to identify, prevent, and resolve financial, human resources and data management challenges. FCMAT provides fiscal and data management assistance, professional development training, product development and other related school business and data services. FCMAT’s fiscal and manage- ment assistance services are used not just to help avert fiscal crisis, but to promote sound financial practices, support the training and development of chief business officials and help to create efficient organizational operations. FCMAT’s data management services are used to help local educational agencies (LEAs) meet state reporting responsibilities, improve data quality, and inform instructional program decisions. FCMAT may be requested to provide fiscal crisis or management assistance by a school district, charter school, community college, county office of education, the state Superintendent of Public Instruction, or the Legislature. When a request or assignment is received, FCMAT assembles a study team that works closely with the LEA to define the scope of work, conduct on-site fieldwork and provide a written report with findings and recommendations to help resolve issues, overcome challenges and plan for the future. FCMAT has continued to make adjustments in the types of support provided based on the changing dynamics of K-14 LEAs and the implementation of major educational reforms. Studies by Fiscal Year 90 80 70 60 50 40 30 20 10 0 95/96 96/97 97/98 98/99 99/00 00/01 01/02 02/03 03/04 04/05 05/06 06/07 07/08 08/09 09/10 10/11 11/12 12/13 13/14 14/15 15/16 16/17 17/18 FCMAT also develops and provides numerous publications, software tools, workshops and professional development opportunities to help LEAs operate more effectively and fulfill their fiscal oversight and data management responsibilities. The California School Information Services (CSIS) division of FCMAT assists the California Department of Education with the implementation of the California Longitudinal Pupil Achievement Data System (CALPADS). CSIS also hosts and maintains the Ed-Data website (www.ed-data.org) and provides technical expertise to the Ed-Data partnership: the California Department of Education, EdSource and FCMAT. FCMAT was created by Assembly Bill (AB) 1200 in 1992 to assist LEAs to meet and sustain their financial obligations. AB 107 in 1997 charged FCMAT with responsibility for CSIS and its state- wide data management work. AB 1115 in 1999 codified CSIS’ mission. California Community Colleges ChanCellor’s offiCe seidutS fo rebmuN iv ABOUT FCMAT AB 1200 is also a statewide plan for county offices of education and school districts to work together locally to improve fiscal procedures and accountability standards. AB 2756 (2004) provides specific responsibilities to FCMAT with regard to districts that have received emergency state loans. In January 2006, Senate Bill 430 (charter schools) and AB 1366 (community colleges) became law and expanded FCMAT’s services to those types of LEAs. On September 17, 2018 AB 1840 was signed into law. This legislation changed the how fiscally insolvent districts are administered once an emergency appropriation has been made, shifting the former state-centric system to be more consistent with the principles of local control, and providing new responsibilities to FCMAT associated with the process. Since 1992, FCMAT has been engaged to perform more than 1,000 reviews for LEAs, including school districts, county offices of education, charter schools and community colleges. The Kern County Superintendent of Schools is the administrative agent for FCMAT. The team is led by Michael H. Fine, Chief Executive Officer, with funding derived through appropriations in the state budget and a modest fee schedule for charges to requesting agencies. Fiscal crisis & ManageMent assistance teaM 1 INTRODUCTION Introduction Background In February 2019, the California Community Colleges Chancellor’s Office (CCCCO) requested that FCMAT conduct a management assistance study on data management practices across the California Community Colleges system related to the Student Centered Funding Formula (SCFF) and provide recommendations and suggested protocols to improve data quality and data management practices for data used in the SCFF. The California Community Colleges system serves more than 2.1 million students in 115 colleges and 72 districts. The system is guided by the Vision for Success, adopted by the Board of Governors in 2017. The Vision calls for eliminating achievement gaps, increasing by 35% the number of students transferring to a University of California or California State University campus and increasing by at least 20% annually the number of students who earn a degree, credential, or certificate preparing them for in-demand jobs. The SCFF, written into California state law on June 27, 2018, significantly changes the way California community college districts are funded. The SCFF focuses on narrowing the access and achievement gap for disadvantaged students and improving community college student outcomes as a whole, as outlined in the state’s initiative, Vision for Success. The SCFF divides the state’s community college budget into three allocations. The base alloca- tion, which in 2018-19 comprises 70% of the state’s community college budget, targets access. It is distributed based on full-time equivalent student (FTES) enrollments. The supplemental allocation, which comprises 20%, targets equity and is distributed based on head counts of Pell Grant recipients, AB 540 students, and California College Promise Grant recipients. The student success allocation, which comprises 10% of the budget, targets successful outcomes, distributed based on a variety of weighted metrics that represent various types of student success. Counts included in the student success allocation are: • Associate degrees for transfer (ADTs) • Associate degrees (excluding ADTs) • Baccalaureate degrees granted • Credit certificates (16 units or more) granted • Completion of transfer-level mathematics and English courses within the first academic year of enrollment • Successful transfer to a four-year university • Completion of nine or more career technical education (CTE) units • Attainment of regional living wage Under the new funding formula, noncredit enrollment and some other types of enrollment are funded at current rates. Although student outcome and financial aid data had been reported to the CCCCO prior to the SCFF, the Chancellor’s Office recognized that the SCFF dramatically changed how some data are used. Some data became very high stakes with the new funding formula when previously that California Community Colleges ChanCellor’s offiCe 2 INTRODUCTION data was reported but not used for funding or any other high-stakes decisions. The Chancellor’s Office requested FCMAT conduct this management assistance study to collect a baseline of current data management practices and to make recommendations that could be used to improve data practices statewide. Study and Report Guidelines FCMAT reviewed the data management responsibilities and practices related to the collection, review, reporting and use of data elements used in the SCFF in 11 community colleges within 10 community college districts. The review at each college/district included both examination of documents and interviews with staff regarding data policies, procedures, responsibilities, and flow for the data elements used in the SCFF. FCMAT’s review of data management practices was not an audit; this study was not intended to validate specific counts. The purpose was to examine the processes in place to gather, report and use the data that feeds into the SCFF and to determine if improvements in those processes could be implemented to improve the quality of the data. The SCFF primarily relies on data submitted by the districts through the Chancellor’s Office Management Information System to calculate the base, supplemental and student success allocations; this system is commonly called MIS by districts and so that acronym is used in this report. Additional data is sourced from the Chancellor’s Office Curriculum Inventory (COCI), Employment Development Department (EDD), National Center for Education Statistics (NCES), and CTE outcomes surveys and matched to data reported by the districts in the MIS submissions. Interviews conducted with district and college staff by the FCMAT team focused on these data components and how they were collected, maintained, reported and verified by the districts. FCMAT’s reports focus on systems and processes that may need improvement. Those that may be functioning well are generally not commented on in FCMAT’s reports. In writing its reports, FCMAT uses the Associated Press Stylebook, a comprehensive guide to usage and accepted style that emphasizes conciseness and clarity. In addition, this guide emphasizes plain language, discourages the use of jargon and capitalizes relatively few terms. Sample Selection The CCCCO requested that FCMAT select a sample of 10-15 colleges to be visited during the study. The Chancellor’s Office and FCMAT agreed the sample would include: • Multi-college districts and single college districts • Large, medium and small districts/colleges • Geographic representation from southern, central and northern California FCMAT and CCCCO further agreed that the sample would primarily include only one college from each district but that two colleges from at least one district would be included in the sample. FCMAT selected the sample based upon these parameters established with the Chancellor’s Office. After the sample was selected, FCMAT examined changes in data used in the funding formula reported between 2016-17 and 2017-18 to ensure the sample included some districts with signif- Fiscal crisis & ManageMent assistance teaM 3 INTRODUCTION icant changes and others with fairly consistent data so that the sample was representative of the state. FCMAT also verified the sample included a variety of the student information systems in use by community college districts. Student information systems in the sample included: • Colleague (Ellucian) • Banner (Ellucian) • Peoplesoft (Oracle) • Custom (a locally developed system) Districts and colleges initially selected for the study were: District College Contra Costa CCD Contra Costa College, Diablo Valley College Grossmont-Cuyamaca CCD Grossmont College MiraCosta CCD MiraCosta College Monterey Peninsula CCD Monterey Peninsula College Rio Hondo CCD Rio Hondo College San Bernardino CCD San Bernardino Valley College San Mateo County CCD Skyline College Santa Monica CCD Santa Monica College Shasta-Tehama-Trinity Joint CCD Shasta College West Hills CCD West Hills College-Coalinga Yuba CCD Yuba College FCMAT shared this list with the CCCCO and obtained approval to move forward with this sample. The CCCCO then sent a communication to the districts selected and asked for the name and information for the district contact to coordinate with FCMAT on the study visit. Monterey Peninsula CCD did not respond to multiple attempts to coordinate their participation in the study. Cuesta College in the San Luis Obispo CCD was selected to replace Monterey Peninsula CCD so that the central coast was represented in the sample. San Mateo CCD declined to participate. Since the Contra Costa CCD provided representation from the San Francisco Bay Area, FCMAT and the CCCCO agreed that another district would not be selected to replace San Mateo CCD. As a result, the final list of districts and colleges in the sample was: District College Contra Costa CCD Contra Costa College, Diablo Valley College Grossmont-Cuyamaca CCD Grossmont College MiraCosta CCD MiraCosta College Rio Hondo CCD Rio Hondo College San Bernardino CCD San Bernardino Valley College San Luis Obispo CCD Cuesta College Santa Monica CCD Santa Monica College Shasta-Tehama-Trinity Joint CCD Shasta College West Hills CCD West Hills College-Coalinga Yuba CCD Yuba College California Community Colleges ChanCellor’s offiCe 4 INTRODUCTION Process for the Study In February, March and April of 2019, the FCMAT team visited each district in the sample and typically conducted interviews with individuals from the district/college Admissions & Records, Student Services, Finance/Financial Aid, Institutional Research, and Information Systems/ Information Technology/Management Information Systems departments, although not every visit included individuals from all of these areas due to scheduling conflicts. During the interviews, the FCMAT team and representatives from the selected districts and colleges discussed data management responsibilities, practices, and procedures and the flow of data elements reported to the Chancellor’s Office and used in the SCFF. FCMAT also gathered available documentation/examples of current practices and reports used in monitoring data and/or validating counts submitted to the Chancellor’s Office. This report is the result of those activities and is divided into the following sections: • About FCMAT • Introduction • Executive Summary • Base Allocation • Supplemental Allocation • Student Success Allocation • Documentation and Sustainability • CCCCO Communications • Appendices Although practices at the various colleges and districts differed with regard to the review, reporting and use of data elements, all colleges have two common starting points with the collec- tion of data. Students applying to California’s community colleges use the online CCCApply application and students applying for federal financial aid use the online Free Application for Federal Student Aid (FAFSA) application. Data from each of these systems are subsequently loaded into the various local student information systems (SIS). Several districts also reported using supplemental and/or interim information systems, both official and unofficial, for prelimi- nary reviews, cleansing, and data quality checks prior to loading into the SIS. Data from the local student information systems are used to fulfill mandatory reporting to the MIS and Apportionment Attendance Report (CCFS-320) systems. It is the data from these systems that are used in calculating the SCFF. Study Team The study team was composed of the following members: Michelle Giacomini Lisa Hayes FCMAT Deputy Executive Officer FCMAT/CSIS Implementation Specialist Nancy Sullivan Laura Haywood FCMAT Consultant FCMAT Technical Writer Fiscal crisis & ManageMent assistance teaM 5 EXECUTIVE SUMMARY Executive Summary The California Community Colleges Chancellor’s Office commissioned this management assistance study to gather information about and recommendations for improvement in data management practices across the California Community Colleges system. The Student Centered Funding Formula (SCFF), which began in 2018-19, dramatically changed how funding is deter- mined for community colleges and resulted in some data becoming very high stakes with the new funding formula. Although that data had been previously reported, it had less frequently been used for funding or other high-stakes decisions prior to the funding change. While this report is intended to inform continuous improvement efforts, it was not an audit – this study was not to validate specific counts but rather to examine the processes in place to gather, report and use the data that feeds into the SCFF and to determine if improvements in those processes could be implemented to improve the quality of the data. The California Community Colleges system serves more than 2.1 million students in 115 colleges and 73 districts. The Community Colleges system is guided by the Vision for Success, adopted by the Board of Governors in 2017. The Vision calls for eliminating achievement gaps, increasing by 35% the number of students transferring to a University of California or California State University campus and increasing by at least 20% annually the number of students who earn a degree, credential, or certificate preparing them for in-demand jobs. In all districts visited, district and college staff were working to implement changes to improve student outcomes. Although district and college staff understood the importance of submitting accurate data to the Chancellor’s Office, many indicated that data management activities were often a lower priority than activities focused on services to promote student success. Districts and colleges reported that both the focus on improved student outcomes and the SCFF were resulting in improved data quality, especially for the data elements used to monitor student success and/or calculate the SCFF. The SCFF base allocation is calculated using attendance data reported to the Chancellor’s Office through the CCFS-320 process three times a year. Districts have well-established practices around the collection, reporting and review of attendance data. The supplemental allocation of the funding formula is calculated based on the number of Pell Grants, California Promise Grants, and AB 540 fee waiver recipients. The funding formula is based on disbursements, not eligibility. Data used to calculate the supplemental allocation comes from data submitted to the Chancellor’s Office Management Information System (MIS) in two different annual submissions. Data used for the student success allocation comes from student and course data submitted to MIS at the end of the fall, winter/spring and summer terms. Although some districts have formal structures and defined roles for data governance as well as documentation of local procedures for MIS submissions, many do not. Data governance practices are the practices an organization uses to manage data and information. Best practices for data governance include formalizing local data governance practices in a set of policies and procedures that encompasses the full life cycle of data, including collection; use, including reporting for local, state and federal purposes, and disposal. This includes defining roles and responsibilities for data management, including decision-making authority, as well as policies, procedures, and stan- dards regarding data collection, quality, security and privacy protection, access, and monitoring. In districts without formal data governance structures and/or documentation, efforts rely upon hard-working staff whose individual commitment has resulted in improved MIS submissions over time. Many districts in the sample had staff with years of experience with the MIS submis- sions; those staff indicated their expertise with the submissions had grown over time. They California Community Colleges ChanCellor’s offiCe 6 EXECUTIVE SUMMARY frequently reported making changes in their processes to address errors and make submissions more efficient. They also reported changes in process that resulted in improved services to students, such as better communication processes to help students understand their status with respect to financial aid and/or their progress on their education plan. As some of these staff near retirement, districts are at risk of losing this expertise if they do not document current local data management processes as well as tips for preventing or addressing errors. Many interviewed indicated that, due to resource constraints, local documentation either had not been developed or was out of date. They also noted that there was little, if any, cross training. Districts should develop and maintain local documentation that describes roles, responsibilities, and tasks for each step of the MIS process and how leadership is kept informed about MIS submissions. Documentation should include details about how reports will be reviewed for accuracy and what happens if the counts are not accurate so there is a common understanding across the organiza- tion about the level of review for data accuracy, especially for data elements that impact funding. Ideally, documentation would also include common errors and how to prevent or address them. The appendices in this report include resources to assist districts with their local practices and documentation. Districts identified a need for more training and resources on the MIS submission process, especially for new staff. They indicated all staff would benefit from training that addressed how data from MIS submissions are used in student success metrics and SCFF calculations. The current CCCCO MIS trainings focus on changes to be implemented in the next submission. Although districts/colleges recognize the Chancellor’s Office has resource constraints, they felt more training from the CCCCO would help benefit the entire system and help ensure consis- tent, accurate data. They also felt resources such as mapping guides that document how MIS data elements are used in Data Mart reports would be extremely beneficial. While Data Mart (https:// datamart.cccco.edu/DataMart.aspx) is CCCCO’s online system that provides information and many reports about students/headcounts, student services, outcomes, courses/calendar faculty and staff for the community college system as a whole, more understanding is needing to use the tool more effectively. Districts also indicated that increased dialogue with the CCCCO about upcoming changes would help both them and the CCCCO as they all prepare for implementa- tion. District and college leaders are appropriately making improved student outcomes a priority and are directing resources to improved services for students. Good data management plays an essen- tial role in monitoring student success and can be quite helpful in focusing resources on services that have the most impact. Developing and sustaining good data management practices requires time and resources, as well as an organizational commitment to developing and sustaining policies and practices for data governance. District and college leaders should recognize that quality data is an organizational asset and ensure resources are allocated for data governance and management, including formal identification of roles and responsibilities, written procedures that are maintained over time, and adequate training for those with primary and backup data management responsibilities. District and college leaders are encouraged to engage in an ongoing dialogue with those involved with MIS reporting so that they understand the time, resources and cross-department coordination needed to implement improvements in local data management practices. Given the pace of change in recent years, as district and college leaders set priorities for informa- tion systems/information technology (IS/IT) projects, they should examine whether current IS/ IT resources are being consumed with implementing new requirements or if there are resources available to assist with improvement of existing systems, tools, and resources for data manage- Fiscal crisis & ManageMent assistance teaM 7 EXECUTIVE SUMMARY ment staff. For example, staff currently are hampered in their ability to monitor data quality by lack of access to well-documented local reports that mimic how data will be reported on Data Mart and/or provide easy access to trend data. District and college staff need access to tools to support their efforts to improve processes and data quality. Districts and colleges reported that the increased focus on helping students graduate and transfer has increased the Associate Degree for Transfer (ADT) verification workload. Many reported staffing has not kept pace with the growth in transfers and they are challenged to meet the timelines required by the California State University system for ADT verifications. They also reported that communications from CSU to transferring students has unintentionally resulted in inefficient use of district/college resources. Dialogue between CSU, CCCCO and districts may help eliminate or reduce the problem. District and college staff interviewed indicated that the pace of change in recent years made it difficult to implement continuous improvement processes. They implement one change and are required to move quickly to another change without time to reflect on how to improve their implementation of the first change. Staff also indicated that sometimes communications between the CCCCO and districts/colleges and between the district and colleges can be challenging. Sometimes all involved do not receive the message about an upcoming change and/or there may be different understandings about how a change is to be implemented. The CCCCO and districts are encouraged to explore methods for improving communications to ensure affected staff receive clear and consistent messages about upcoming changes with sufficient time to plan and implement them. California Community Colleges ChanCellor’s offiCe 8 Fiscal crisis & ManageMent assistance teaM 9 BASE ALLOCATION Findings and Recommendations Base Allocation The base allocation is primarily based on units of full-time equivalent students (FTES). One FTES represents a full-time student taking 15 units in 35 weeks of instruction, or 525 hours of attendance. Attendance can be collected by census date accounting, based on a predetermined date during the term for courses scheduled coterminously with regular meeting patterns, or a weekly census for courses that are not coterminous with the primary term. Positive attendance, accounting for actual hours of attendance, is used for courses with short term credit courses, irregularly scheduled credit courses, and open entry/open exit courses. Alternate attendance accounting procedures are used for courses with atypical characteristics or requirements, or an unknown type of meeting pattern; for example, intercollegiate athletics, field trips, labs and hours to be arranged. Attendance is reported to the Chancellor’s Office through the CCFS-320 process three times a year following the end of each term: P1 (first period) July 1 through December 31 (data is annualized); P2 (second period) July 1 through April 15 (data is annualized); Final (annual) July 1 thru June 30. Revisions may be made following the final report if adjustments are needed. Data Collection Prior to the implementation of the SCFF, funding was based primarily on FTES apportion- ment, and as a result, districts have well-established practices around the collection, reporting and reviews of these data. Minor issues were reported with collecting data, such as occasional problems with instructors not returning attendance within the required timelines, difficulty with capturing hours for programs that may meet on weekends due to software limitations, and addi- tional efforts required in accurately collecting attendance for dual enrollment students. Attendance Monitoring and Report Preparation Most of the districts in this study reported some systematic process of monitoring and review of attendance data and many stated that multiple departments collaborated in these processes. Attendance data are regularly audited and heavily scrutinized. Several cited weekly and even daily reviews and analyses of FTES by census type, as well as departmental and divisional trends. Some had developed processes including maintaining duplicate attendance systems for monitoring during the term, custom Structured Query Language (SQL) reports, workbooks, and reverse engineering of the 320 report. Staff verify that instructors substantiate rosters and sign for atten- dance, and run error reports looking for specific errors, such as problems with non-residency and positive attendance. 320 Reporting Process Attendance data are reported at specified times during the year, aligned with the school terms, to the Chancellor’s Office through the Apportionment Attendance Report (CCFS-320) system. The reporting timelines are defined by the California Code of Regulations (CCR), Title 5, section 58003.4. Data are hand-entered for the CCFS-320 reporting, and the process was described as “a little clunky” and cumbersome. Although several interviewed staff reported having incomplete data and issues due to short turnaround times especially for the P2 report, the P2 numbers are California Community Colleges ChanCellor’s offiCe 10 BASE ALLOCATION preliminary and are corrected at the final report. Occasionally recalculations are submitted, but this is not typical. The CCCCO should consider developing a process by which CCFS-320 data could be uploaded rather than being hand entered by districts so key-entry errors are avoided and data more accurate and need less corrections. Several specific issues were raised regarding the CCFS-320 reporting. One district reported errors and underreporting for special admits and incarcerated students in prior years but has rectified its collection and monitoring for current reporting. Positive attendance was carefully examined, and if there were any questions or concerns about the supporting documentation, it was not reported for that course. The ability to report summer attendance for courses that cross fiscal years was noted as beneficial to the districts for maximizing revenue. Staff stated that those courses were carefully tracked to identify when they were reported. Several districts noted that the lack of availability of statewide training for the CCFS-320 reporting was of concern, as was documentation to provide context around the use of the data, especially for new staff. They also reported being unable to find Frequently Asked Questions or a repository of resources on the Chancellor’s Office website. Some voiced frustration that they were unable to reach Chancellor’s Office staff by telephone and were unable to leave messages as voice mailboxes frequently were full. Either staff needs to ensure their mailboxes do not get full, or another method should be implemented so that field staff can leave messages and get responses, such as an auto- mated ticket tracking system. To work around these issues, personnel work with other districts and colleges to get questions answered whenever possible rather than working with CCCCO staff. 320 Data Quality The CCFS-320 reports were reported to be carefully reviewed following each submission. As previously noted, many districts engaged staff in multiple departments (admissions and records, finance, schedulers) in the post submission analysis with internally developed multiyear trend reports. Review procedures were not consistently well-documented, or in some cases not docu- mented at all. Rather, staff were trained in how to review and reconcile the data with the SIS, or other systems in which attendance data are maintained. It is critical to develop and follow desk procedures that document steps taken to complete attendance accounting tasks. These procedures should have sufficient detail so that a new staff member could read them and understand how to complete each necessary task. A schedule should be established and followed for annual review and update as necessary of the procedures for attendance accounting. To be effective, staff must be provided time to complete the review and update, and to be held accountable for it. All attendance data are subject to annual independent audits; as a result, districts reported veri- fying contact hours to ensure they are reported accurately and not reporting course attendance if there were any concerns with data or less than desirable supporting documentation, to avoid potential audit issues. Reports are also reviewed by finance offices for verification and alignment with audit standards as they are more familiar with content of the data. One issue that was reported to be of concern is the reporting of dual and concurrent enrollments because the ability to capture those data were “not as clean.” For example, there may be challenges in obtaining attendance data for courses held on high school campuses because the attendance data may not be recorded the same way or in the same system as courses held on college campuses. Fiscal crisis & ManageMent assistance teaM 11 BASE ALLOCATION Recommendations The CCCCO should: 1. Establish a method that allows people to 1) leave messages for staff if no one is available to take a call when it comes in and 2) receive a response. 2. Consider developing a process by which CCFS-320 data could be uploaded rather than being hand entered by districts. The districts should: 1. Develop desk procedures that document data flow and steps taken to complete attendance accounting tasks relative to MIS reporting. 2. Establish a schedule of the procedures for attendance accounting for annual review and update as necessary. 3. Ensure that staff are provided time to complete an annual review and update, if necessary, of attendance accounting procedures. California Community Colleges ChanCellor’s offiCe 12 Fiscal crisis & ManageMent assistance teaM 13 SUPPLEMENTAL ALLOCATION Supplemental Allocation The supplemental allocation of the funding formula is calculated based on the number of Pell Grants, California Promise Grants, and AB 540 fee waiver recipients. The funding formula is based on disbursements, not eligibility. Students apply for both Pell and California Promise grants through the Free Application for Federal Student Aid (FAFSA) system. Students applying for AB 540 fee waivers complete the California Nonresident Tuition Exemption Request form, also referred to as the AB 540 Affidavit. Pell Grant Pell Grants are federal grants for students with financial need. Pell Grants are limited to 12 semesters, or roughly six years. To remain eligible for federal aid, students must maintain satisfactory academic progress (e.g., cumulative GPA, workload, cumulative pace (attempted vs. completed), quantitative pace). Each college has a satisfactory academic progress policy for financial aid purposes. Financial aid staff expressed concerns that students receiving Pell Grants could use all of the funding before transferring, potentially making it difficult or impossible for the students to afford to finish their bachelor’s degrees. Staff indicated they wanted to ensure students have additional years of Pell eligibility after they transfer to a four-year institution. One college reported specific efforts to get students through in three years or less whenever possible. Staff expressed concern that they do not have time or resources to counsel individual students regarding financial literacy. California Promise Grant The California College Promise Grant waives the enrollment fee for eligible resident students for two years. Students can apply through the FAFSA application or by submitting the California Dream Act application. Students may qualify for the Promise Grant by various criteria. Some reported needing clarification on the various qualifications for Promise Grant financial aid. As with the Pell Grant, academic and progress standards must be maintained for students to retain eligibility. Some students encounter problems with the application and never complete the application process. But overall, colleges reported an increase in the number of Promise Grants awarded. AB 540 Non-resident students who have attended high school in California and received a high school diploma or its equivalent are exempt from paying non-resident tuition due to AB 540. Students apply by completing the California Nonresident Tuition Exemption Request form (AB 540 Affidavit). Free Application for Federal Student Aid Most students start the financial aid process with the FAFSA application, which is maintained and managed by the U.S. Department of Education. The financial aid staff take over the process once the data from the FAFSA web portal are loaded into the student information systems or other interim system for review and to make any necessary follow-up and/or corrections and determine all of the potential aid for which a student may be eligible. If any discrepancies are uncovered in the applications, the U.S. Department of Education requires that they be resolved California Community Colleges ChanCellor’s offiCe 14 SUPPLEMENTAL ALLOCATION prior to awarding any funds, requiring staff intervention. Some data are maintained in supple- mental or “shadow” systems; however, districts ensure the financial aid information pulled for MIS reporting is included in their system of records. Some students are unaware of all of the information required to complete the application, including not knowing their own Social Security number, and may get stalled by the process and discouraged. Some are discouraged by the lengthy application. Homeless students have difficulty due to the lack of a physical address. For students who are not initially eligible for financial aid, staff will review the FAFSA applica- tion. Students can also appeal financial aid decisions. An appeals process typically goes through a committee that can determine if the student actually is eligible. Several colleges commented that not all students who may be eligible for financial aid were applying because students are unaware of the many financial aid options and opportunities. This was especially true of the colleges with more students from families that are not typically consid- ered low income; these students may assume they are not eligible when they actually may qualify for aid based on true cost of college (books, transportation, cost of living). Financial aid staff were working to get more students to be aware of financial aid benefits by stressing that applying for financial aid should be part of the college application. Specific programs have been implemented for high school students, identifying those who may qualify for aid with follow-up during their first year of college. Financial Aid Supports Districts have responded to help students through the financial aid applications by retooling processes, developing supports, and adopting automated solutions to facilitate the completion of applications. Some examples are single-page flyers such as Steps to Financial Aid, and a one-stop model that merged the admissions and financial aid processes. Staff use tablets to work with students waiting in line for staff assistance to confirm they have all needed documents. One college reported that it has adopted a high tech-high touch model to support students. Several colleges have recently implemented commercial solutions to facilitate application moni- toring and follow-up processes. Campus Logic has been adopted at multiple colleges allowing the tracking of the entire financial aid process, verification of Institutional Student Information Record (ISIR) data, electronic verification and secure submission of financial aid documents. Staff can reach out to students on the financial aid website, link their accounts and more easily resolve simple problems, like a missing college code. The financial aid worksheet can be auto- filled with information in the system about the student and school to streamline the process. Then, the student just needs to review it. Campus Logic has allowed colleges to work more proactively with students. Some indicated they have implemented auto-packaging for federal financial aid that provides students with all of the aid they may be eligible to receive. Previously this was a manual process, and the automation has resulted in an increase in the number of awards. A number of colleges reported that they are adopting self-service so that students are able to see awards and make simple corrections. Others have subscribed to FATV, a chatbot that can respond to students’ questions about financial aid. ECMC/iGrad provides students with entrance counseling, so students understand their financial obligations for promissory notes rather than borrowing more than they need and not understanding they must pay it back. Fiscal crisis & ManageMent assistance teaM 15 SUPPLEMENTAL ALLOCATION MIS Reporting Process Financial aid data are submitted to the Chancellor’s Office MIS system annually in the fall for the previous academic year. As with the 320 reporting, practices vary by district as to audits and data reviews prior to submission. With the implementation of the funding formula, more attention is being paid to data quality. Processes have been established for internal reviews and reconciliation and analyzing variance reports from the Common Origination and Disbursement (the COD is a federal system for processing, storing, and reconciling Pell Grants, and other federal financial aid data), and resolving common errors like name changes and incorrect birthdates. Several sites have designated data stewards who have specific responsibilities for ensuring data are complete and correct, but not all have a formalized process. Prior to submissions, some districts reported sending email notifications to district and/or college staff responsible for MIS financial aid data to remind them to review data and reconcile any discrepancies. These districts reported that some staff were more consistent and diligent than others. Some districts reported that existing practices, expectations, and the source and flow are documented, but not all documentation is consistent, complete and up-to-date. Several colleges are building out shared knowledge bases using group tools like Confluence and Google Docs. Data Quality Financial data are maintained and reported in multiple systems, which can make reconciling all student financial aid challenging. Colleges reported that they audited the data prior to the MIS submission to try to get data clean and ensure students who were awarded grants were accurately reported. Most offices reported systematic internal reviews with the other systems (ISIR, Cal-ISIR files). There are four main financial aid disbursements to students during the academic year. Data are reconciled after each disbursement and at the end of the year. However, not all data can be reconciled because the statewide data may include financial aid provided while students attended other colleges. After the MIS processing is complete, many reported reviewing the data and comparing numbers with the Data Mart and stated that numbers were “very close” to internal records. Trend analysis from prior years is reviewed and verified with other departments, like finance, to ensure reason- ability. Several reported concerns about the MIS reporting timelines. Since the reconciliation processes may not be complete by the reporting deadline, there may be insufficient time to report accurate financial aid data to the Chancellor’s Office. One college stated that Pell Grants are underre- ported due to simple errors (name changes, birthdates) that cannot be resolved by the reporting deadline. After the data are submitted to the Chancellor’s Office, reports can be retrieved for review. Staff commented that since no definitions were available for the referential files and the text file is not user friendly, they found it difficult and sometimes impossible to reconcile local data to the reports. Other comments reported were that some data elements are difficult to understand; districts and colleges reported it would be easier for them to understand these data elements if the CCCCO provided more explanation of what, why and how data being reported to the Chancellor’s Office are used. Colleges reported that they are building out local processes in specific areas for ease of reconciling data from their SIS with the MIS referential files and Data on Demand counts. A consortium California Community Colleges ChanCellor’s offiCe 16 SUPPLEMENTAL ALLOCATION of colleges is working with their SIS vendor to develop an automated processing model for Cal Grants that will reduce the time required to report this data by thousands of hours annually. Recommendations The CCCCO should: 1. Provide detailed documentation regarding the financial aid counts produced on Data Mart, including elements, transformations, parameters, and time frames. The districts should: 1. Develop desk procedures that document the data flow and steps taken to complete reporting of financial aid data for MIS reporting. 2. Identify data stewards responsible for pre- and post-MIS submission reviews. Fiscal crisis & ManageMent assistance teaM 17 STUDENT SUCCESS ALLOCATION Student Success Allocation Data used for the student success allocation comes from MIS submissions of student and course data by term at the end of the fall, winter/spring and summer terms. The submissions include these files: • Student Basic File • Student Enrollment File • Course File • Section/Sessions/Assign File • Student Success File • Employee Demographic • Student Disability File • Student EOPS File • Student CalWORKs File • Student VTEA File • College Calendar File (submitted only for the end of spring/winter term) Annually the CCCCO publishes a calendar of submission windows and downtimes for the MIS submissions. CCCCO uses the downtime windows to update the system to prepare for future submissions (e.g., to complete the transition to a new year). Local Processes for Completing Term-End MIS Submissions Typically, a term-end MIS involves the following steps: 1. Creation of the files: Generally, the person responsible for sending the term-end MIS data files to the Chancellor’s Office was in the Information Systems/Information Technology Department (the actual name of the department varied). Files were most frequently generated by the SIS used, but some districts use their own queries to create the files. All districts reported using the Data Element Dictionary, which is available from the CCCCO at http://extranet.cccco.edu/Divisions/TechResearchInfoSys/MIS/DED. aspx, in this process and indicated the document was helpful. The Data Element Dictionary provides a description of the System Office MIS and the technical specifications for the data to be collected and reported to the state. Submission of course data includes the course code. Districts work with local curriculum committees and their Academic Senate on course definitions and then submit course information into the Chancellor’s Office Curriculum Inventory (COCI) system for course approval and assignment of a course code. The process of assigning a course code takes time and must be complete for the course to be included in the course catalog prior to beginning of a session. As a result, work to add any new courses or update course codes if there have been changes is done well before MIS submissions. California Community Colleges ChanCellor’s offiCe 18 STUDENT SUCCESS ALLOCATION 2. Pre-submission check of the files: In some districts, the person responsible for uploading the files to the Chancellor’s Office runs local routines to verify the files before they are submitted. These routines check to be sure the files are in the correct format and contain only allowable characters. Typically, in districts completing this step, the person responsible for the files corrects errors if the solution is obvious (e.g., a field missing leading zeros) and the staff person responsible for the data is contacted if research is needed to resolve the error. 3. Upload files to the Chancellor’s Office: The responsible person uploads the files into the MIS site. At this point, the files go through syntactical, referen- tial and analysis routines. The files either pass the validation checks and are uploaded, or they are rejected due to errors. 4. Correct errors: In most districts visited, the error correction process was a joint effort between the person generating the files and the person responsible for the data, also known as the data owner. In some districts, the person generating the files would correct errors with obvious solutions, especially if the pre-submission check of the files was not completed, and the data owner was responsible for correcting errors that needed to be researched (e.g., missing or invalid birthdate). In other districts, the data owner was always responsible for resolving errors. Key entry errors were reported as the most frequent cause of errors but in some instances, errors were created when the MIS files were generated (e.g., the system right justifying a field when it should be left justified) or incorrect or inconsistent understandings among those responsible for the data. 5. Resubmit the files: Once the errors were corrected, the files were regenerated and uploaded to the Chancellor’s Office. If one or more files error out at this point, the process for correcting and resubmitting is repeated until all files are accepted. Because the validation process includes cross-file validations, each submission must include all required files. 6. Review data reports: Once processed and uploaded, the data are available in reports. The IS/IT staff member typically notified the data owners that the reports were available on the Chancellor’s site and requested that the counts in the reports be verified. Most of the data owners reported they reviewed the reports, but also indicated that the depth of the review varies depending upon how much time is available. In some, only the data that will impact funding was reviewed in the reports to ensure accuracy. Others do a more in-depth probe. If errors in counts are identified before the data submission window closes, the district can resubmit the files and correct the data. If an error is identified after the window closes, the district may resubmit. However, this process can be cumbersome and time consuming, especially if it involves a prior fiscal year, because course information may have changed and the Chancellor’s Office does not maintain historical course information. Data owners reported that whether or not they review reports for accuracy and the depth of their review depends upon the amount of available time and competing priorities. Data owners are typically those involved in providing services to students Fiscal crisis & ManageMent assistance teaM 19 STUDENT SUCCESS ALLOCATION (e.g., Admissions and Records staff) and these staff members may be too busy with other responsibilities to do a thorough review. In some districts, the depth of the review is understood among the district team working on the submission, but in others it may not be, and leadership may not have visibility into the depth of the review. Some districts reported that the window in August when the CCCCO takes the MIS down for system maintenance creates challenges. No submissions can be completed during this downtime. In 2018-19 this maintenance window was August 9-23. All term-end files are due within one month after the end of each term. For summer terms ending in July, the August downtime may impact a district’s ability to complete the submission within the required window. Annual program award files are submitted at the end of August/beginning of September. The August downtime may also impact this submission. Local Data Governance Practices Data governance practices are the practices an organization uses to manage data and information. Best practices for data governance include formalizing local data governance practices in a set of policies and procedures that encompass the full life cycle of data, including collection; use, including reporting for local, state and federal purposes, and disposal. This includes defining roles and responsibilities for data management, including decision-making authority, as well as policies, procedures, and standards regarding data collection, quality, security and privacy protec- tion, access, and monitoring. See Appendix C for information regarding using a Responsible, Accountable, Consulted, and Informed (RACI) matrix to document roles and responsibilities; a RACI matrix can be very useful in documenting the steps in as well as the roles and responsibil- ities of those involved in the data management process, such as MIS submissions. See Appendix D for a sample checklist on data governance. Local data governance practices begin with establishing roles and responsibilities for data management. Generally, staff interviewed had common understandings of roles and respon- sibilities for MIS data in terms of the staff responsible for collecting and reporting the data. Most staff displayed commitment to reporting accurate data and working collegially with others to complete the MIS submissions. A few districts had formal committee structures and regular meetings (e.g., monthly) to ensure consistent practices to collect, maintain, report and use accurate data. Some had completed an inventory of their data systems and had formal documentation of the shared responsibilities, including responsibilities of the data owners and the IS/IT owners in terms of providing technical support for the systems in use. However, most districts and colleges had informal structures to address issues and improvement efforts. In some districts, responsibility for the MIS submissions rested primarily with one person; most of these districts indicated that they had done little, if any, documentation and cross-training and would have difficulty completing the MIS submissions with the same degree of accuracy if that person retired or took another position. Documenting and articulating local data governance roles and responsibilities is a best practice. Districts with a common ownership for data reporting and error correction took individual and collective responsibility for completing the steps necessary to submit accurate MIS data. Often issues were addressed one-to-one by the IS/IT person and the data owner, but if needed, other departments participated in the discussion. For example, several districts indicated that if a trend analysis revealed that data seemed to vary from what would have been expected, a meeting between Institutional Research, the data owner and IS/IT would be called to discuss the data and identify any issues with data quality or the processes for data collection and reporting. California Community Colleges ChanCellor’s offiCe 20 STUDENT SUCCESS ALLOCATION These districts reported modifying local processes when local practices need to change to address MIS submission errors and/or data quality issues. The process for identifying and implementing changes in local practices is informal and generally initiated by data owners. A small number of districts held post submission reviews and developed collective agreements about and priorities for improvements prior to the next submission. In some districts, individuals recorded notes about changes they planned to implement to improve their practices in the next submission. Completing a post submission review and documenting changes needed to improve the next data submission is a best practice; the review typically includes what went well during the submission to capture practices that need to be sustained and what needs to be changed to improve data quality and/or the efficient completion of the next submission. See Appendix E for more details about post submission reviews. Districts that do not currently do a post submission review should adopt this practice. In contrast with the staff in districts with a shared sense of responsibility, a small number of districts had staff who reported being focused on completing individual assignments without much regard for working as a team to address cross-department issues. Time constraints and lack of leadership-level support for districtwide focus on data practices were the most frequent expla- nations for this approach. Districts indicated that executive leadership was focused on providing services to students and that there was often little, if any, discussion with executive leadership about the resources needed for data management and the depth of staff review of reports once data were submitted to CCCCO. Districts with a collaborative, shared responsibility for data had better data management prac- tices. This finding held whether a district had multiple colleges or a single college. Districts with shared responsibility for accurate data worked across organizational lines to investigate data errors and determine the changes in practices needed to prevent errors in the future. When changes took months to implement (e.g., changes that would impact the course catalog), they were able to work together to backward map the time needed to implement the changes by considering the time required for each step toward the final goal. Districts without shared responsibility tended not to do this backward mapping; each department did their part when convenient without the overall coordination to ensure the timeline would result in the task being completed on time. Some districts indicated that because local decision makers may not be involved with the details about data collection and reporting, they may not understand the time required for activities such as designing and implementing process changes and for implementing and adequately testing software systems used for data collection and reporting. These districts stressed the importance of decision makers having ongoing dialogue with those involved with MIS reporting so that the decision makers can be advocates for the time, resources and cross-department coor- dination needed to implement changes. Districts also indicated that executive-level leaders may not thoroughly understand whether or not there are sufficient resources to review reports after submission and the depth of the review if one is completed. Local Efforts to Improve Student Outcomes Districts and colleges are working to successfully implement changes resulting from an increased focus on student success and equity. Staff indicate that student success metrics and the SCFF are resulting in positive changes, including: • Increased or improved support services for students are leading to an increase in the number of students graduating and transferring. For example, districts have increased communications to students about transfers and some districts have opened student Fiscal crisis & ManageMent assistance teaM 21 STUDENT SUCCESS ALLOCATION transfer centers that provide additional support. Some offer priority class registration to transfer students. With these increased and improved services, districts have seen an increase in student transfers. Some districts have implemented financial aid systems that send automated reminders to students who are in the process of applying for financial aid to let students know the status of their application and the steps they need to take to complete the applications (e.g., bring in required documentation); districts report such systems are helping to increase the number of students receiving financial aid. • Improved visibility into student progress is also contributing to student success. For example, some districts have implemented online self-service degree audit programs students can use to examine their progress toward completing their degree. Others have implemented degree audit programs for counselors to use when meeting with students. This increased visibility helps keep students on track and helps them see other degrees they are on track to complete. Some districts/colleges use degree audit and “auto award” programs to identify multiple degrees a student might be eligible to receive. Districts in the study did not automatically provide degrees to students but some did use auto award programs to notify staff of students eligible for multiple degrees. Once staff verified the student’s eligibility, students were notified and could receive the additional degree(s) if they wanted to do so. Some districts auto award certificates, especially if the certificates are needed for job placement (e.g., nursing). • Some districts are examining how to identify students who either are eligible or could be eligible for additional degrees or certificates if they took a small number of additional classes. This effort is driven in part by a desire to help students transition to careers and in part to obtain additional funding under the SCFF. Discussions are underway that the SCFF might be statutorily changed to only reimburse districts for the highest degree or certificate awarded, not all degrees and/or certificates. Since this change occurred during the review, FCMAT was not able to determine if all districts understood the change and if they had time to discuss how the change in that policy might impact current practices. • Increased attention on data quality is leading to examination and improvements in some practices. For example, a number of the districts/colleges visited indicated they are in the process of reviewing course codes as they realized that the career technical education designation for courses was not necessarily correct. Some courses were coded as occupational when they were not, and some were not coded as occupational when they were. The process of updating course codes takes time as it involves action by the Curriculum Committee and the Academic Senate at the district before the district can work with the state on the changes. The process of working with the state also takes time. Districts with large changes in data used for the SCFF between 2016-17 and 2017-18 were able to explain how increased and improved services for students resulted in the improved outcomes. For example, districts and colleges reported that the increased focus on helping students graduate and transfer has resulted in improvements in these metrics. Districts that had updated course codes to correct CTE designations indicated that sometimes the changes resulted in their CTE counts decreasing; districts worked to establish correct course designations even if it meant decreases in that metric because they wanted the data to be accurate. Many reported staffing has not kept pace with the verification required with the growth in ADTs, and they are challenged to meet the timelines required by the CSU system for ADT verifications. Districts also expressed that communications and coordination challenges with CSU sometimes California Community Colleges ChanCellor’s offiCe 22 STUDENT SUCCESS ALLOCATION resulted in staff time being spent on responding to questions from students who were confused or concerned about communications from CSU regarding their transfer status when the district had already informed CSU about that individual student’s status. Districts and colleges also reported challenges in the timely review of transcripts from other community colleges and other institutions of higher education. Districts and colleges ideally would complete this review shortly after a student arrives so that the student was aware of course requirements that had already been met. However, due to workload, most districts/colleges report that this work is completed just prior to graduation. Districts reported challenges with respect to dual/concurrent enrollment students and the completion of transfer-level mathematics and English courses in the first year of enrollment. As the number of high school students taking community college courses beginning in ninth and tenth grade increases, the number of students taking transfer-level mathematics and English courses while they are still in high school also increases. These students do not count in the calculation of the completion of transfer-level mathematics and English courses in the first year of enrollment metric. As a result, many districts reported that the completion of transfer-level mathematics and English courses in the first year of enrollment metric in the SCFF does not adequately reflect their work to get students ready to transfer. Districts also reported challenges with CCCApply. Although they acknowledged efforts underway to improve CCCApply, they indicated that ongoing work is being done to prevent applications from being submitted solely to gain access to free software through the acquisition of a “.edu” email account. Districts said that as soon as CCCApply improved to address these phony applications, those attempting to get the “.edu” email accounts became more sophisticated in how they completed the applications so that their applications would make it through spam filters. As a result, districts expressed their hope that efforts to address phony applications will be ongoing. Districts also indicated that some of the questions in CCCApply result in residency problems for some students. For example, when ninth and tenth grade students who apply are asked where they attended high school in the last two years, they cannot provide a California school because they did not attend high school for the past two years; their response results in the system assuming they are from out of state. District and college staff interviewed indicated that the pace of change in recent years has made it difficult to implement continuous improvement processes. Districts and colleges implement one change and are then required to move quickly to another change without time to reflect on how to improve their implementation of the first change. Districts also reported that the pace of change has resulted in SIS vendors focusing the vast majority of their attention on mandated changes. As a result, some of the changes districts have requested have been waiting in the vendors’ queues for years. Districts also noted that at times, it appears the pace of change is also a challenge for the CCCCO. They reported that on multiple occasions the Chancellor’s Office has released infor- mation about an upcoming change and then received a large volume of communications from districts/colleges indicating why the planned change was problematic. Sometimes when this occurred, the CCCCO issued clarifications or policy adjustments within a few days or weeks of the initial communication, which may increase the workload and confusion among districts. To decrease the risk of this occurring again, districts expressed an interest in increasing the dialogue between the CCCCO and districts about upcoming changes so that implementation challenges and unintended consequences can be discussed and resolved before changes are finalized. Fiscal crisis & ManageMent assistance teaM 23 STUDENT SUCCESS ALLOCATION Training/Knowledge Transfer Districts attend annual webinars by the Chancellor’s Office on changes in the MIS process as well as regional and statewide professional learning opportunities sponsored by professional organi- zations and SIS users’ groups. Districts reported that although the sessions on the MIS changes by the Chancellor’s Office were helpful, they would benefit from additional training from the CCCCO. Districts reported that having training that focuses on the entire MIS submission and the connections to student success, as well as cyber security, would be helpful for all staff involved in the MIS submission process, and especially for new staff. Many districts reported that those working to ensure the data submitted to the Chancellor’s Office were complete and correct needed to understand why data were collected and how individual data elements were used. They indicated that when information about how student success metrics are calculated and monitored is not shared with those doing MIS reporting, those staff don’t necessarily have the context they need to understand how to correct errors or identify changes to improve practices. Districts also requested more training on the interrelationships between data elements and data files. Districts noted that the Chancellor’s Office offers training for new financial aid officers for two years at the Chancellor’s Office, as well as annual refresher training, but noted that this type of training is no longer available for new staff responsible for MIS submissions. With the funding now tied to the accuracy of the MIS submission, districts indicated they would benefit from CCCCO-sponsored training for those new to MIS submissions and requested that their staff be strongly encouraged to attend for the first two years on the job, with optional attendance after that. The Chancellor’s Office staff indicated resources were not currently available to provide this type of training. District and college representatives interviewed indicated that they learned from examples of successful practices from other districts and colleges. They expressed appreciation for conferences and webinars focused on sharing information about practices that were helping to improve student outcomes. User groups, professional organizations and the CCCCO all encourage this type of sharing. Some districts expressed confusion about how specific data elements from the local student infor- mation system are reflected in Data Mart reports. These districts indicated they would like training or more documentation on how data flow from the MIS files into the Data Mart reports. In terms of local knowledge transfer, many districts expressed concern about upcoming retire- ments. Many districts have staff who have worked on the MIS submissions for years and have developed much expertise that potentially will be lost when they retire or if they accept another position. Several districts indicated they were working to develop local documentation of the steps taken to complete each submission but also indicated other priorities and constant change are hampering their efforts to build complete and current local documentation. Districts also expressed concern if responsibility for the MIS submissions primarily rested with one person and no back-up had been designated or trained. Resources & Tools Districts/colleges reported that the Data Element Dictionary was a valuable resource in the MIS submission process. Some districts have adopted a tool or tools to assist in documentation of local practices, including SharePoint and Confluence. These districts are in the process of building documentation and developing the processes for maintaining documentation once built. Some districts are working California Community Colleges ChanCellor’s offiCe 24 STUDENT SUCCESS ALLOCATION to document the processes used to collect, report and use data. Others are doing this and also documenting common errors, the meaning of error messages from MIS, and how to avoid these errors. Some districts have developed local reports designed to provide staff with a preliminary view of data in the same or similar format used in the Data Mart reports. Data owners can use these reports to monitor data quality on an ongoing basis or at least just prior to a submission to try to identify and resolve problems prior to the submissions. Data owners indicated these reports were helpful, but sometimes struggled to understand exactly what data were included and the time frame covered by the reports. Often, there was little or no documentation to help users understand the specific data elements in the report, the effective dates of the data, and how any aggregates displayed were calculated. The number of reports available varied by district, but often districts indicated that there may be multiple reports that contain very similar information, which resulted in confusion about which report should be used. For example, one district reported having more than 1,000 reports. Districts reported that their constraints on the development and implementation of tools, reports and other types of supports to assist with data management were primarily a lack of resources and the pace of change. Tools and resources often need to be implemented by the district’s IS/IT department, and often staff were committed to implementing new legislative requirements and did not have the time to focus on other issues. Recommendations The CCCCO should: 1. Identify resources for and provide training for new staff responsible for the MIS submissions similar to the training for new financial aid officers. 2. Provide training and/or resources for districts that explain how the data elements collected through MIS are used in the SCFF. 3. Develop a mapping guide to show how the data elements submitted in the MIS files are used in the Data Mart reports. 4. Provide documentation specifically focused on what MIS error messages mean and how to avoid them. 5. Communicate frequently with districts on changes to the funding formula to ensure all involved understand the changes as they occur and have the oppor- tunity to develop an understanding of what this means in the local context, including what impacts the changes will have on local practices. 6. Continue opportunities for districts and colleges to share practices that are improving student outcomes and, to the extent possible, provide districts and colleges the time and resources for a continuous improvement process. Districts and colleges need the time to implement and refine the many efforts underway to focus on student success and equity. Fiscal crisis & ManageMent assistance teaM 25 STUDENT SUCCESS ALLOCATION The districts should: 1. Establish the expectation for shared ownership of data management respon- sibilities at the executive level and communicate this expectation to all departments. 2. Encourage and support work across departments to develop and sustain prac- tices that support the collection, reporting and use of accurate data. 3. Develop a matrix of responsibilities for MIS tasks, including the designation of staff members who are trained to complete MIS responsibilities in the event the person with primary responsibility is unexpectedly out when the submission must be completed. 4. Provide those involved with MIS reporting in districts/colleges with infor- mation about how the data are used in the SCFF and in monitoring student success. 5. Establish ongoing communication between local decision makers and those gathering and reporting data such that the decision makers understand the time, resources, and coordination needed to support the collection, use and reporting of accurate data. 6. Once developed by CCCCO, use the mapping guide to build staff under- standing of what data goes into each report. 7. Build a common understanding among those who do the MIS data submis- sion and those who are accountable for the accuracy of the data about how reports will be reviewed for accuracy (e.g., which data elements will be verified and the depth of the verification) and what happens if the counts are not accurate. 8. Identify resources to acquire or sustain tools that help those responsible for data accuracy to monitor data quality and develop local documentation of what data elements are included in the reports, how aggregates are calculated, the time frame for the report, and local processes for data management. Support data owners in building documentation and using reports on an ongoing basis or at least just prior to each submission. 9. Conduct a post submission review and document changes to improve the next submission. California Community Colleges ChanCellor’s offiCe 26 Fiscal crisis & ManageMent assistance teaM 27 DOCUMENTATION AND SUSTAINABILITY Documentation and Sustainability As mentioned in other sections of the report, few districts reported having current documenta- tion of data management processes such that a new staff member could use it to complete data management responsibilities. Lack of time/resources was the most common reason given for not having this documentation. Constant change in requirements also was reported as a cause for out-of-date documentation. Documentation and current processes need to be reviewed whenever a change comes from Sacramento or a policy clarification is received to determine if procedures and the documentation need to be modified to implement the change or policy clarification. Districts reported that it takes individuals time to develop expertise in MIS reporting and data management responsibilities. Many districts and colleges expressed concern about lack of documentation and indicated that if staff members take other jobs or retire, the new staff member may not be able to complete MIS data submission responsibilities and/or collect or report data accurately. Documentation of data management processes would facilitate knowledge transfer from one staff member to another and help ensure current practices can be sustained. Documentation also helps experienced staff remember how to complete tasks that are typically performed only at one specific time during the year. Some districts are working on building/revising documentation to make future submissions easier (e.g., recording what error messages mean, the cause(s), and how to avoid errors in the future). Districts reported that individuals are working hard to have good data, but all of the recent changes and the focus on student success sometimes means that efforts are focused on student services rather than on documentation of data management processes, report verification or process improvements. Districts must ensure that local data management documentation is created and updated. That documentation should include a detailed description of each data management task and responsible parties, including who is responsible for each task, who is accountable for the task, who is consulted, and who is informed about work. Best practice is to include common errors, what they mean and how to avoid them. A schedule should be adopted for periodic review and updating of data management documentation (e.g., annually or every other year) and the responsible parties should be provided with the time and tools necessary to keep documentation current. Recommendations The CCCCO should: 1. Advocate for districts and colleges to allocate resources to help them develop and maintain adequate local documentation of data management practices. 2. Create a forum where examples of local documentation can be shared with other districts as examples of best practices for documentation. The districts should: 1. Create or update local data management documentation. 2. Adopt a local schedule for periodically reviewing and updating data manage- ment documentation and provide the responsible parties with the time and tools necessary to keep documentation current. California Community Colleges ChanCellor’s offiCe 28 Fiscal crisis & ManageMent assistance teaM 29 CCCCO COMMUNICATIONS CCCCO Communications The CCCCO utilizes various alias lists and listservs to communicate with districts and colleges. Alias lists are used to distribute information to individuals holding specific positions in the California Community Colleges system. A communication is sent to an alias list when there is a targeted audience for the message. For example, a message might be designed for and sent to chief business officers or chief student services officers. The CCC Technology Center also hosts listservs for the California Community Colleges regional and system-wide organizations. Districts and colleges appreciate the information they receive via the various lists and listservs. However, sometimes information sent to one list is applicable to others in a district/college and it may take time for the message to reach all whose job responsibilities are impacted by a commu- nication. This can be especially true about communications regarding changes in MIS reporting since it impacts many departments across districts and colleges. Sometimes all involved do not receive the message about an upcoming change and/or there may be different understandings about how a change is to be implemented because the information was stated a bit differently on different lists. The CCCCO could consider establishing a new state listserv that is used to communicate information about upcoming MIS changes. This listserv would be in addition to the existing listservs and would be open to any community college staff who wish to subscribe. When districts/colleges have questions about MIS reporting, they are directed to send questions to the email box for MIS (cccmisedit@cccco.edu). There is no public knowledge base where users can look up previously asked questions and the response. Users indicated they generally received a response after they submitted a question, but noted that the response could take a long time, especially if it required collaboration between multiple divisions within the CCCCO. The CCCCO does not use a help desk software program to track emails received by date or topic, making it difficult to collect metrics and monitor responses to inform improvement efforts. A help desk tracking system should be considered to collect information on volume of tickets, days to resolution, the frequency of questions on a topic, etc. If a tracking system is adopted, the Chancellor’s Office should use the frequency of questions on the same topic to improve trainings, as well as the average response time, to help determine if additional coordination, focus or resources are needed to address questions received. The MIS section of the Chancellor’s website (http://extranet.cccco.edu/Divisions/ TechResearchInfoSys/MIS.aspx) contains information and resources about MIS submissions, including previous web-based trainings and the Data Element Dictionary. However, some resources are difficult to find and may not be readily accessible to users who are not familiar with the site. The Data Element Dictionary is not searchable. The CCCCO indicated a project is underway to revise all data dictionaries to make them more consistent and easier to use. Districts and colleges indicated that changes are not always communicated with sufficient lead time to make changes needed in the SIS and in local practices. Districts indicated they used to receive notice approximately one year prior to the change and frequently had the opportunity to implement changes one term before the data would be officially reported so they could identify and resolve issues related to the data collection before the data were used. They reported that now the lead time is much shorter, often just a few months. This means there is little time to imple- ment the change and adjust the implementation plans if unexpected results or consequences are encountered. Districts also indicated that in the past there was more two-way dialogue with the CCCCO before changes finalized. Districts indicated this dialogue was useful as it helped them understand California Community Colleges ChanCellor’s offiCe 30 CCCCO COMMUNICATIONS and prepare for upcoming changes and it also provided the CCCCO with feedback on imple- mentation challenges and possible unintended consequences. Districts expressed willingness to engage in more of these types of discussions with the CCCCO and felt they would benefit both districts and the Chancellor’s Office. The Chancellor’s Office indicated that this type of dialogue could be useful, but that time and resource constraints make it challenging. Recommendations The CCCCO should: 1. Consider how communication channels can be improved so that individuals in various district/college roles receive the same message about upcoming MIS reporting changes. 2. Communicate MIS reporting changes and the rationale for changes in multiple ways so individuals who missed the first announcement have other opportunities to learn of the change. 3. Gather feedback from districts and colleges about potential changes, imple- mentation challenges, unintended consequences and the amount of lead time necessary to prepare for MIS reporting changes. Use the feedback to refine proposed changes and the planned implementation schedule when possible. 4. Gather input from those involved in the MIS submission process, including new staff, and revise the MIS submission webpages so it is easier to find resources related to the submission. 5. Consider utilizing a help desk tracking system to collect information on volume of tickets, days to resolution, frequency of questions on a topic, etc. to help determine if additional coordination, focus or resources are needed to address questions received. Fiscal crisis & ManageMent assistance teaM 3311 APPENDICDERSAFT Appendices California Community Colleges ChanCellor’s offiCe 3322 DRAAFPTPENDICES Appendix A Study Agreement Fiscal crisis & ManageMent assistance teaM 3333 APPENDICDERSAFT California Community Colleges ChanCellor’s offiCe 3344 DRAAFPTPENDICES Fiscal crisis & ManageMent assistance teaM 3355 APPENDICDERSAFT California Community Colleges ChanCellor’s offiCe 3366 DRAAFPTPENDICES Fiscal crisis & ManageMent assistance teaM 3377 APPENDICDERSAFT California Community Colleges ChanCellor’s offiCe 3388 DRAAFPTPENDICES Appendix B Data Analysis Section 1: Analysis of Student Success Metrics 2016-17 to 2017-18 Data Provided by CCCCO, Dated December 20, 2018 Table 1.1: Associate Degrees (AD) Awarded and Associate Degrees for Transfer (ADT) 2016-17 2017-18 AD % Change 2016-17 2017-18 ADT Changes District AD Total AD Total Since 16-17 ADT Total ADT Total Since 16-17 Allan Hancock 1,076 1,071 -0.5% 259 298 15.1% Antelope 1,303 1,224 -6.1% 430 566 31.6% Barstow 274 291 6.2% 79 74 -6.3% Butte 1,152 1,143 -0.8% 303 367 21.1% Cabrillo 1,035 1,188 14.8% 227 272 19.8% Cerritos 1,090 1,186 8.8% 645 758 17.5% Chabot-Las Positas 1,351 1,254 -7.2% 432 689 59.5% Chaffey 1,775 2,141 20.6% 726 987 36.0% Citrus 2,170 2,291 5.6% 803 1,027 27.9% Coast 4,211 4,207 -0.1% 1,573 1,690 7.4% Compton 493 636 29.0% 0 0 Contra Costa 2,308 2,891 25.3% 1,280 1,697 32.6% Copper Mountain 145 163 12.4% 59 63 6.8% Desert 428 498 16.4% 476 579 21.6% El Camino 2,432 2,203 -9.4% 349 1,096 214.0% Feather River 166 170 2.4% 26 57 119.2% Foothill 2,085 1,983 -4.9% 1,070 1,256 17.4% Gavilan 365 360 -1.4% 157 238 51.6% Glendale 382 402 5.2% 467 545 16.7% Grossmont 2,125 2,453 15.4% 920 1,184 28.7% Hartnell 550 755 37.3% 402 559 39.1% Imperial 952 949 -0.3% 368 412 12.0% Kern 1,329 1,436 8.1% 781 1,112 42.4% Lake Tahoe 102 99 -2.9% 43 64 48.8% Lassen 206 175 -15.0% 47 39 -17.0% Long Beach 749 806 7.6% 782 936 19.7% Los Angeles 8,927 9,409 5.4% 2,540 3,446 35.7% Los Rios 4,597 4,574 -0.5% 1,601 1,894 18.3% Marin 203 211 3.9% 99 134 35.4% Mendocino 233 258 10.7% 93 94 1.1% Merced 632 612 -3.2% 441 572 29.7% MiraCosta 928 1,302 40.3% 223 350 57.0% Monterey 356 397 11.5% 254 331 30.3% Mt. San Antonio 1,685 1,778 5.5% 524 833 59.0% Fiscal crisis & ManageMent assistance teaM 3399 APPENDICDERSAFT 2016-17 2017-18 AD % Change 2016-17 2017-18 ADT Changes District AD Total AD Total Since 16-17 ADT Total ADT Total Since 16-17 Mt. San Jacinto 1,668 1,927 15.5% 337 496 47.2% Napa 521 433 -16.9% 218 253 16.1% North Orange 1,892 2,300 21.6% 1,279 1,590 24.3% Ohlone 720 785 9.0% 260 291 11.9% Palo Verde 133 164 23.3% 14 8 -42.9% Palomar 1,606 1,620 0.9% 312 438 40.4% Pasadena 3,090 4,028 30.4% 1,040 1,393 33.9% Peralta 1,278 1,345 5.2% 505 568 12.5% Rancho Santiago 2,989 3,302 10.5% 1,062 1,237 16.5% Redwoods 424 443 4.5% 62 77 24.2% Rio Hondo 956 1,202 25.7% 552 622 12.7% Riverside 3,477 5,045 45.1% 593 920 55.1% San Bernardino 1,536 1,365 -11.1% 527 611 15.9% San Diego 2,010 2,049 1.9% 1,304 1,473 13.0% San Francisco 1,117 1,180 5.6% 261 294 12.6% San Joaquin Delta 2,631 2,571 -2.3% 203 231 13.8% San Jose 744 789 6.0% 454 594 30.8% San Luis Obispo 674 633 -6.1% 418 428 2.4% San Mateo 1,375 1,336 -2.8% 925 955 3.2% Santa Barbara 1,984 1,953 -1.6% 0 500 Santa Clarita 1,395 1,542 10.5% 729 1,062 45.7% Santa Monica 2,064 3,048 47.7% 572 775 35.5% Sequoias 949 949 0.0% 282 463 64.2% Shasta Tehama 616 689 11.9% 176 260 47.7% Sierra 2,004 2,028 1.2% 724 830 14.6% Siskiyous 223 203 -9.0% 14 17 21.4% Solano 1,152 1,263 9.6% 188 218 16.0% Sonoma 1,522 1,774 16.6% 629 695 10.5% South Orange County 2,009 2,398 19.4% 1,082 1,315 21.5% Southwestern 1,043 1,045 0.2% 653 750 14.9% State Center 1,304 1,609 23.4% 1,409 1,854 31.6% Ventura 3,518 4,106 16.7% 1,871 2,161 15.5% Victor Valley 968 994 2.7% 110 177 60.9% West Hills 762 798 4.7% 120 214 78.3% West Kern 360 355 -1.4% 105 105 0.0% West Valley 616 699 13.5% 532 666 25.2% Yosemite 1,340 1,474 10.0% 470 607 29.1% Yuba 1,061 902 -15.0% 188 265 41.0% Statewide Total 101,546 110,862 9.2% 38,659 49,632 28.4% California Community Colleges ChanCellor’s offiCe 4400 DRAAFPTPENDICES Table 1.2: Credit Certificates Awarded and Students Taking Nine or More Units of CTE 2016-17 2017-18 CTE Certificates % 2016-17 2017-18 Credit Credit Changes District Change Since CTE CTE Certificates Certificates Since 16- 16-17 Total Total Awarded Awarded 17 Allan Hancock 791 750 -5.2% 1,933 1,915 -0.93% Antelope 895 1,049 17.2% 2,138 2,033 -4.91% Barstow 17 80 370.6% 421 374 -11.16% Butte 517 490 -5.2% 2,260 2,192 -3.01% Cabrillo 176 169 -4.0% 1,174 1,135 -3.32% Cerritos 1,504 2,227 48.1% 3,846 3,738 -2.81% Chabot-Las Positas 378 577 52.6% 2,582 2,718 5.27% Chaffey 1,210 1,654 36.7% 2,442 2,382 -2.46% Citrus 1,462 2,031 38.9% 1,919 1,728 -9.95% Coast 4,358 4,322 -0.8% 5,849 5,565 -4.86% Compton 178 104 -41.6% 559 555 -0.72% Contra Costa 1,703 2,126 24.8% 4,526 4,406 -2.65% Copper Mountain 66 76 15.2% 227 232 2.20% Desert 200 300 50.0% 1,628 1,688 3.69% El Camino 557 719 29.1% 2,626 2,690 2.44% Feather River 0 10 #DIV/0! 338 590 74.56% Foothill 682 1,084 58.9% 5,908 5,263 -10.92% Gavilan 562 657 16.9% 813 729 -10.33% Glendale 241 207 -14.1% 2,327 2,231 -4.13% Grossmont 1,497 1,649 10.2% 2,407 2,385 -0.91% Hartnell 673 829 23.2% 869 839 -3.45% Imperial 582 694 19.2% 1,249 1,378 10.33% Kern 536 1,660 209.7% 4,734 4,789 1.16% Lake Tahoe 33 25 -24.2% 287 245 -14.63% Lassen 136 105 -22.8% 289 295 2.08% Long Beach 213 229 7.5% 3,020 2,747 -9.04% Los Angeles 7,983 8,180 2.5% 20,035 17,253 -13.89% Los Rios 2,099 2,532 20.6% 9,058 8,430 -6.93% Marin 59 86 45.8% 464 434 -6.47% Mendocino 63 55 -12.7% 529 517 -2.27% Merced 480 605 26.0% 1,358 1,443 6.26% MiraCosta 1,004 1,463 45.7% 1,891 1,863 -1.48% Monterey 113 254 124.8% 850 880 3.53% Mt. San Antonio 582 542 -6.9% 5,150 4,956 -3.77% Mt. San Jacinto 137 141 2.9% 1,370 1,409 2.85% Napa 368 440 19.6% 863 845 -2.09% North Orange 825 900 9.1% 4,922 4,770 -3.09% Ohlone 37 58 56.8% 1,007 905 -10.13% Palo Verde 51 36 -29.4% 581 738 27.02% Palomar 1,635 1,641 0.4% 3,377 3,432 1.63% Fiscal crisis & ManageMent assistance teaM 4411 APPENDICDERSAFT 2016-17 2017-18 CTE Certificates % 2016-17 2017-18 Credit Credit Changes District Change Since CTE CTE Certificates Certificates Since 16- 16-17 Total Total Awarded Awarded 17 Pasadena 536 548 2.2% 3,251 3,092 -4.89% Peralta 1,188 1,220 2.7% 2,960 2,758 -6.82% Rancho Santiago 2,614 2,679 2.5% 3,761 3,903 3.78% Redwoods 177 122 -31.1% 746 737 -1.21% Rio Hondo 166 1,561 840.4% 2,025 1,972 -2.62% Riverside 871 1,034 18.7% 4,311 4,286 -0.58% San Bernardino 580 565 -2.6% 2,274 2,591 13.94% San Diego 1,141 1,199 5.1% 6,180 6,048 -2.14% San Francisco 730 809 10.8% 3,538 4,363 23.32% San Joaquin Delta 1,101 755 -31.4% 3,453 3,264 -5.47% San Jose 573 735 28.3% 1,870 1,728 -7.59% San Luis Obispo 336 349 3.9% 1,403 1,427 1.71% San Mateo 1,242 1,220 -1.8% 2,817 2,776 -1.46% Santa Barbara 1,581 1,258 -20.4% 2,971 2,682 -9.73% Santa Clarita 1,524 1,420 -6.8% 2,752 2,933 6.58% Santa Monica 1,456 2,388 64.0% 3,822 3,758 -1.67% Sequoias 195 355 82.1% 1,820 1,825 0.27% Shasta Tehama 417 555 33.1% 1,548 1,527 -1.36% Sierra 185 188 1.6% 2,659 2,547 -4.21% Siskiyous 57 114 100.0% 435 317 -27.13% Solano 146 145 -0.7% 1,201 1,155 -3.83% Sonoma 573 614 7.2% 2,868 2,888 0.70% South Orange County 3,538 3,747 5.9% 4,491 4,360 -2.92% Southwestern 651 570 -12.4% 2,159 2,091 -3.15% State Center 675 1,510 123.7% 5,569 5,659 1.62% Ventura 2,144 3,724 73.7% 3,828 3,942 2.98% Victor Valley 206 195 -5.3% 1,915 1,777 -7.21% West Hills 277 346 24.9% 983 939 -4.48% West Kern 60 31 -48.3% 303 313 3.30% West Valley 283 457 61.5% 1,911 1,867 -2.30% Yosemite 364 693 90.4% 2,995 2,865 -4.34% Yuba 61 102 67.2% 1,463 1,365 -6.70% Statewide Total 60,251 71,964 19.4% 192,108 186,472 -2.93% California Community Colleges ChanCellor’s offiCe 4422 DRAAFPTPENDICES Table 1.3: Students Completing Transfer-Level English and Mathematics During Their First Year of Enrollment 2016-17 2017-18 English & Math Transfer English & Math Transfer English & Math Transfer Level Courses Complete District Level Courses Complete Level Courses Complete First Year % Change Since First Year First Year 16-17 Allan Hancock 215 184 -14.4% Antelope 206 187 -9.2% Barstow 26 58 123.1% Butte 261 314 20.3% Cabrillo 148 146 -1.4% Cerritos 156 203 30.1% Chabot-Las Positas 469 482 2.8% Chaffey 255 270 5.9% Citrus 224 328 46.4% Coast 886 1,139 28.6% Compton 33 27 -18.2% Contra Costa 805 889 10.4% Copper Mountain 30 36 20.0% Desert 79 112 41.8% El Camino 487 583 19.7% Feather River 51 47 -7.8% Foothill 1,037 1,172 13.0% Gavilan 79 92 16.5% Glendale 260 257 -1.2% Grossmont 546 592 8.4% Hartnell 196 183 -6.6% Imperial 115 107 -7.0% Kern 283 353 24.7% Lake Tahoe 14 23 64.3% Lassen 25 31 24.0% Long Beach 307 339 10.4% Los Angeles 643 985 53.2% Los Rios 596 650 9.1% Marin 59 56 -5.1% Mendocino 30 30 0.0% Merced 178 210 18.0% MiraCosta 329 341 3.6% Monterey 90 99 10.0% Mt. San Antonio 368 451 22.6% Mt. San Jacinto 348 387 11.2% Napa 172 138 -19.8% North Orange 681 759 11.5% Ohlone 219 223 1.8% Palo Verde 3 2 -33.3% Fiscal crisis & ManageMent assistance teaM 4433 APPENDICDERSAFT 2016-17 2017-18 English & Math Transfer English & Math Transfer English & Math Transfer Level Courses Complete District Level Courses Complete Level Courses Complete First Year % Change Since First Year First Year 16-17 Palomar 225 372 65.3% Pasadena 684 867 26.8% Peralta 273 304 11.4% Rancho Santiago 629 717 14.0% Redwoods 56 57 1.8% Rio Hondo 100 189 89.0% Riverside 376 826 119.7% San Bernardino 169 249 47.3% San Diego 738 904 22.5% San Francisco 214 339 58.4% San Joaquin Delta 182 254 39.6% San Jose 210 308 46.7% San Luis Obispo 166 174 4.8% San Mateo 490 635 29.6% Santa Barbara 368 401 9.0% Santa Clarita 401 464 15.7% Santa Monica 619 566 -8.6% Sequoias 173 224 29.5% Shasta Tehama 122 107 -12.3% Sierra 659 686 4.1% Siskiyou 59 95 61.0% Solano 220 202 -8.2% Sonoma 216 210 -2.8% South Orange County 738 928 25.7% Southwestern 204 263 28.9% State Center 563 790 40.3% Ventura 844 1,006 19.2% Victor Valley 109 131 20.2% West Hills 133 115 -13.5% West Kern 41 53 29.3% West Valley 257 322 25.3% Yosemite 64 83 29.7% Yuba 84 94 11.9% Statewide Total 21,295 25,420 19.4% California Community Colleges ChanCellor’s offiCe 4444 DRAAFPTPENDICES Section 2: Analysis of Student Success Data 2014-15 to 2015-16 Data Obtained from Data Mart June 2019 Table 2.1: Students Completing Associate Degrees 2013-14 to 2015-16 Change Between 2013-14 Associate Degrees Granted & 2015-16 District 2013-14 2014-15 2015-16 Allan Hancock 1,100 1,097 1,006 -9% Antelope 1,322 1,368 1,429 8% Barstow 302 268 272 -10% Butte 1,392 1,241 1,278 -8% Cabrillo 1,334 1,171 1,137 -15% Cerritos 1,333 1,009 1,086 -19% Chabot-Las Positas 1,213 2,024 1,229 1% Chaffey 1,772 1,735 1,827 3% Citrus 1,580 1,883 2,251 42% Coast 3,702 3,989 4,251 15% Compton 262 316 386 47% Contra Costa 2,334 2,251 2,341 0% Copper Mountain 151 155 181 20% Desert 473 482 426 -10% El Camino 1,875 1,800 2,162 15% Feather River 219 162 181 -17% Foothill 2,072 2,205 2,157 4% Gavilan 384 382 361 -6% Glendale 417 401 378 -9% Grossmont 2,042 1,982 1,999 -2% Hartnell 619 591 630 2% Imperial 843 881 826 -2% Kern 1,566 1,420 1,266 -19% Lake Tahoe 133 131 152 14% Lassen 178 210 213 20% Long Beach 702 647 637 -9% Los Angeles 6,746 6,756 7,557 12% Los Rios 5,059 4,648 4,526 -11% Marin 277 238 244 -12% Mendocino 265 243 232 -12% Merced 630 566 632 0% MiraCosta 1,185 1,099 1,288 9% Monterey 412 391 294 -29% Mt. San Antonio 1,704 1,757 1,792 5% Fiscal crisis & ManageMent assistance teaM 4455 APPENDICDERSAFT Change Between 2013-14 Associate Degrees Granted & 2015-16 District 2013-14 2014-15 2015-16 Mt. San Jacinto 1,538 1,541 1,624 6% Napa 571 505 573 0% North Orange 1,964 1,881 1,961 0% Ohlone 667 786 741 11% Palo Verde 104 147 145 39% Palomar 1,887 1,727 1,734 -8% Pasadena 2,135 2,592 3,182 49% Peralta 1,171 1,092 1,235 5% Rancho Santiago 2,781 2,888 3,016 8% Redwoods 389 445 632 62% Rio Hondo 907 865 824 -9% Riverside 2,749 2,810 3,061 11% San Bernardino 1,317 1,296 1,314 0% San Diego 1,711 1,992 1,899 11% San Francisco 1,909 1,507 1,297 -32% San Joaquin Delta 2,472 2,210 2,238 -9% San Jose 1,026 950 878 -14% San Luis Obispo 691 648 616 -11% San Mateo 1,516 1,445 1,291 -15% Santa Barbara 1,763 1,860 2,019 15% Santa Clarita 1,174 1,068 1,109 -6% Santa Monica 1,324 1,935 2,863 116% Sequoias 1,026 1,023 1,045 2% Shasta Tehama 656 585 587 -11% Sierra 2,115 2,046 1,988 -6% Siskiyous 162 165 178 10% Solano 1,379 1,317 1,260 -9% Sonoma 1,557 1,492 1,583 2% South Orange County 1,603 1,590 1,644 3% Southwestern 891 896 1,064 19% State Center 1,607 1,496 1,313 -18% Ventura 2,406 2,403 2,830 18% Victor Valley 947 857 938 -1% West Hills 702 697 742 6% West Kern 326 312 302 -7% West Valley 1,100 939 888 -19% Yosemite 1,447 1,437 1,363 -6% Yuba 825 921 955 16% Statewide Total 95,734 95,081 99,559 4% California Community Colleges ChanCellor’s offiCe 4466 DRAAFPTPENDICES Table 2.2: Students Completing Associate Degrees for Transfer 2013-14 to 2015-16 Change Between 2013-14 Associate Degrees for Transfer & 2015-16 District 2013-14 2014-15 2015-16 Allan Hancock 51 140 194 280% Antelope 27 107 319 1081% Barstow 6 10 20 233% Butte 63 179 263 317% Cabrillo 26 167 224 762% Cerritos 137 347 546 299% Chabot-Las Positas 133 185 264 98% Chaffey 210 402 556 165% Citrus 399 500 703 76% Coast 544 984 1,253 130% Compton 0 0 0 Contra Costa 556 798 1,143 106% Copper Mountain 6 27 63 950% Desert 148 304 353 139% El Camino 123 177 301 145% Feather River 5 12 26 420% Foothill 173 397 768 344% Gavilan 63 58 149 137% Glendale 37 212 395 968% Grossmont 362 590 729 101% Hartnell 128 193 351 174% Imperial 115 249 324 182% Kern 35 296 555 1486% Lake Tahoe 27 38 53 96% Lassen 14 28 22 57% Long Beach 330 463 576 75% Los Angeles 291 645 1,531 426% Los Rios 516 861 1,327 157% Marin 33 62 96 191% Mendocino 47 74 78 66% Merced 190 252 390 105% MiraCosta 134 152 243 81% Monterey 68 175 184 171% Mt. San Antonio 235 330 427 82% Mt. San Jacinto 68 179 266 291% Napa 29 75 149 414% North Orange 638 886 1,117 75% Fiscal crisis & ManageMent assistance teaM 4477 APPENDICDERSAFT Change Between 2013-14 Associate Degrees for Transfer & 2015-16 Ohlone 47 153 206 338% Palo Verde 0 1 0 Palomar 94 165 211 124% Pasadena 435 557 800 84% Peralta 72 202 337 368% Rancho Santiago 389 605 989 154% Redwoods 0 0 0 Rio Hondo 85 298 470 453% Riverside 106 183 343 224% San Bernardino 174 305 379 118% San Diego 482 1,056 1,008 109% San Francisco 122 199 264 116% San Joaquin Delta 53 86 174 228% San Jose 83 177 342 312% San Luis Obispo 145 321 442 205% San Mateo 274 485 669 144% Santa Barbara 0 0 437 Santa Clarita 180 291 509 183% Santa Monica 110 287 499 354% Sequoias 63 99 161 156% Shasta Tehama 61 141 135 121% Sierra 333 498 619 86% Siskiyous 2 9 11 450% Solano 17 39 157 824% Sonoma 335 430 614 83% South Orange County 362 699 964 166% Southwestern 171 407 611 257% State Center 305 569 1,143 275% Ventura 683 1,093 1,514 122% Victor Valley 29 43 94 224% West Hills 16 26 119 644% West Kern 11 28 62 464% West Valley 290 423 578 99% Yosemite 73 270 447 512% Yuba 10 61 110 1000% Statewide Total 11,579 20,760 31,376 171% California Community Colleges ChanCellor’s offiCe 4488 DRAAFPTPENDICES Table 2.3 Percentage of Students Completing Transfer Level English in the First Year of Enrollment 2013-14 to 2015-16 Cohort Year 2013-14 Cohort Year 2014-15 Cohort Year 2015-16 % Completing % Completing % Completing Change from Transfer Level English Transfer Level English Transfer Level English 2013-14 to - 1st Year - 1st Year - 1st Year 2015-16 Allan Hancock CCD 31.5% 35.0% 42.5% 11.0% Antelope CCD 38.0% 37.4% 40.6% 2.6% Barstow CCD 0.3% 0.3% 22.1% 21.8% Butte CCD 51.8% 52.3% 53.9% 2.1% Cabrillo CCD 50.2% 48.1% 51.3% 1.1% Cerritos CCD 26.0% 25.3% 33.0% 7.0% Chabot-Las Positas CCD 40.7% 43.7% 50.1% 9.4% Chaffey CCD 22.7% 23.8% 36.4% 13.7% Citrus CCD 27.4% 28.7% 37.3% 9.9% Coast CCD 45.7% 49.6% 53.6% 7.9% Compton CCD 27.8% 25.1% 26.4% -1.4% Contra Costa CCD 31.3% 33.8% 33.0% 1.7% Copper Mountain 27.6% 30.5% 34.6% 7.0% Desert CCD 35.5% 33.3% 39.5% 4.0% El Camino CCD 45.9% 46.6% 45.9% 0.0% Feather River CCD 44.4% 49.5% 53.5% 9.1% Foothill CCD 47.8% 51.2% 54.0% 6.2% Gavilan CCD 37.7% 38.2% 41.9% 4.2% Glendale CCD 44.7% 50.4% 53.1% 8.4% Grossmont CCD 24.2% 33.7% 36.3% 12.1% Hartnell CCD 29.5% 32.9% 34.9% 5.4% Imperial CCD 16.4% 23.4% 23.3% 6.9% Kern CCD 22.0% 24.6% 27.2% 5.2% Lake Tahoe CCD 62.5% 55.8% 54.3% -8.2% Lassen CCD 44.9% 50.5% 59.2% 14.3% Long Beach CCD 31.2% 35.0% 34.0% 2.8% Los Angeles CCD 22.5% 25.6% 28.9% 6.4% Los Rios CCD 34.3% 36.2% 38.5% 4.2% Marin CCD 35.2% 37.4% 43.4% 8.2% Mendocino CCD 33.2% 38.0% 33.2% 0.0% Merced CCD 22.6% 25.4% 30.8% 8.2% MiraCosta CCD 56.6% 57.4% 61.8% 5.2% Monterey CCD 35.3% 35.9% 44.0% 8.7% Mt. San Antonio CCD 20.1% 27.7% 31.2% 11.1% Mt. San Jacinto CCD 21.5% 25.1% 33.9% 12.4% Napa CCD 19.9% 25.4% 35.5% 15.6% North Orange CCD 36.1% 39.4% 42.2% 6.1% Ohlone CCD 41.6% 44.5% 45.6% 4.0% Fiscal crisis & ManageMent assistance teaM 4499 APPENDICDERSAFT Cohort Year 2013-14 Cohort Year 2014-15 Cohort Year 2015-16 % Completing % Completing % Completing Change from Transfer Level English Transfer Level English Transfer Level English 2013-14 to - 1st Year - 1st Year - 1st Year 2015-16 Palo Verde CCD 15.7% 25.9% 14.9% -0.8% Palomar CCD 38.3% 38.8% 39.2% 0.9% Pasadena CCD 36.4% 43.3% 46.4% 10.0% Peralta CCD 42.4% 43.1% 43.5% 1.1% Rancho Santiago CCD 46.5% 48.2% 52.2% 5.7% Redwoods CCD 29.9% 31.5% 40.3% 10.4% Rio Hondo CCD 36.9% 34.9% 37.4% 0.5% Riverside CCD 23.1% 25.9% 30.3% 7.2% San Bernardino CCD 21.1% 21.5% 24.8% 3.7% San Diego CCD 27.5% 30.7% 40.1% 12.6% San Francisco CCD 22.1% 22.5% 24.5% 2.4% San Joaquin Delta CCD 43.3% 40.5% 39.6% -3.7% San Jose CCD 28.5% 32.5% 32.3% 3.8% San Luis Obispo CCD 60.2% 56.8% 57.6% -2.6% San Mateo CCD 41.3% 45.8% 51.3% 10.0% Santa Barbara CCD 62.3% 53.1% 56.7% -5.6% Santa Clarita CCD 37.9% 41.6% 53.7% 15.8% Santa Monica CCD 43.4% 45.3% 49.1% 5.7% Sequoias CCD 34.4% 39.2% 36.1% 1.7% Shasta Tehama CCD 49.6% 47.7% 50.8% 1.2% Sierra CCD 47.8% 54.8% 58.1% 10.3% Siskiyous CCD 34.0% 49.7% 50.2% 16.2% Solano CCD 43.8% 41.2% 46.6% 2.8% Sonoma CCD 43.7% 47.0% 52.3% 8.6% South Orange County CCD 43.9% 41.8% 44.8% 0.9% Southwestern CCD 66.7% 68.0% 70.7% 4.0% State Center CCD 25.4% 28.2% 30.7% 5.3% Ventura CCD 51.6% 53.3% 56.5% 4.9% Victor Valley CCD 27.9% 30.5% 31.7% 3.8% West Hills CCD 35.7% 38.9% 42.6% 6.9% West Kern CCD 31.4% 42.9% 41.0% 9.6% West Valley CCD 53.8% 56.7% 57.2% 3.4% Yosemite CCD 39.9% 35.1% 34.2% -5.7% Yuba CCD 33.3% 32.2% 34.7% 1.4% 35.7% 37.8% 41.2% 5.5% California Community Colleges ChanCellor’s offiCe 5500 DRAAFPTPENDICES Table 2.4 Percentage of Students Completing Transfer Level Mathematics in the First Year of Enrollment 2013-14 to 2015-16 Cohort Year Cohort Year Cohort Year 2013-14 2014-15 2015-16 % Completing % Completing % Completing Change from Transfer Level Transfer Level Transfer Level 2013-14 to Math - 1st Math - 1st Math - 1st 2015-16 Year Year Year Allan Hancock CCD 15.3% 17.1% 20.2% 4.9% Antelope CCD 10.4% 10.2% 13.0% 2.6% Barstow CCD 0.0% 0.0% 11.3% 11.3% Butte CCD 16.1% 20.6% 20.7% 4.6% Cabrillo CCD 18.2% 18.6% 20.3% 2.1% Cerritos CCD 6.1% 6.3% 6.6% 0.5% Chabot-Las Positas CCD 18.5% 20.8% 23.2% 4.7% Chaffey CCD 9.0% 10.7% 12.1% 3.1% Citrus CCD 10.8% 10.2% 14.6% 3.8% Coast CCD 24.7% 26.7% 28.7% 4.0% Compton CCD 4.5% 5.3% 6.4% 1.9% Contra Costa CCD 26.4% 28.5% 29.8% 3.4% Copper Mountain 13.3% 13.4% 17.6% 4.3% Desert CCD 6.8% 6.2% 5.3% -1.5% El Camino CCD 12.9% 16.7% 16.9% 4.0% Feather River CCD 19.1% 26.1% 23.0% 3.9% Foothill CCD 39.0% 38.8% 39.3% 0.3% Gavilan CCD 11.0% 9.8% 13.1% 2.1% Glendale CCD 15.0% 19.3% 19.5% 4.5% Grossmont CCD 21.8% 19.7% 24.8% 3.0% Hartnell CCD 13.9% 18.6% 20.3% 6.4% Imperial CCD 10.5% 10.2% 10.9% 0.4% Kern CCD 10.0% 11.1% 10.8% 0.8% Lake Tahoe CCD 13.2% 8.0% 13.0% -0.2% Lassen CCD 4.3% 11.2% 7.5% 3.2% Long Beach CCD 12.6% 13.5% 13.6% 1.0% Los Angeles CCD 8.0% 8.3% 8.8% 0.8% Los Rios CCD 14.3% 15.2% 14.8% 0.5% Marin CCD 17.7% 17.4% 19.7% 2.0% Mendocino CCD 13.5% 15.7% 20.2% 6.7% Merced CCD 12.4% 14.0% 14.1% 1.7% MiraCosta CCD 21.3% 17.8% 21.4% 0.1% Monterey CCD 12.4% 14.4% 18.5% 6.1% Mt. San Antonio CCD 11.6% 12.6% 15.3% 3.7% Mt. San Jacinto CCD 12.1% 18.3% 17.9% 5.8% Napa CCD 22.3% 24.2% 25.4% 3.1% Fiscal crisis & ManageMent assistance teaM 5511 APPENDICDERSAFT North Orange CCD 19.7% 21.2% 22.4% 2.7% Ohlone CCD 24.1% 29.8% 29.8% 5.7% Palo Verde CCD 5.7% 1.8% 5.0% -0.7% Palomar CCD 15.7% 15.7% 15.3% -0.4% Pasadena CCD 18.4% 20.9% 22.6% 4.2% Peralta CCD 21.9% 25.7% 26.6% 4.7% Rancho Santiago CCD 24.8% 23.5% 25.8% 1.0% Redwoods CCD 13.4% 14.2% 14.3% 0.9% Rio Hondo CCD 5.0% 3.2% 4.7% -0.3% Riverside CCD 10.0% 12.2% 11.4% 1.4% San Bernardino CCD 10.9% 12.6% 13.0% 2.1% San Diego CCD 19.9% 22.0% 23.8% 3.9% San Francisco CCD 28.8% 28.4% 30.8% 2.0% San Joaquin Delta CCD 9.7% 10.2% 9.8% 0.1% San Jose CCD 22.2% 23.2% 25.0% 2.8% San Luis Obispo CCD 24.1% 23.1% 25.0% 0.9% San Mateo CCD 25.4% 24.4% 28.7% 3.3% Santa Barbara CCD 25.6% 23.5% 25.9% 0.3% Santa Clarita CCD 14.1% 16.1% 17.6% 3.5% Santa Monica CCD 12.7% 14.9% 15.9% 3.2% Sequoias CCD 13.7% 14.7% 12.5% -1.2% Shasta Tehama CCD 20.0% 18.3% 16.0% -4.0% Sierra CCD 21.0% 24.1% 25.1% 4.1% Siskiyous CCD 9.7% 12.8% 15.1% 5.4% Solano CCD 21.0% 22.3% 25.7% 4.7% Sonoma CCD 18.8% 19.7% 17.7% -1.1% South Orange County CCD 23.9% 28.5% 31.5% 7.6% Southwestern CCD 5.3% 5.4% 5.5% 0.2% State Center CCD 12.9% 14.4% 17.1% 4.2% Ventura CCD 22.8% 21.1% 24.0% 1.2% Victor Valley CCD 4.7% 5.8% 8.5% 3.8% West Hills CCD 10.8% 14.0% 12.9% 2.1% West Kern CCD 8.3% 9.7% 11.2% 2.9% West Valley CCD 26.6% 25.7% 27.0% 0.4% Yosemite CCD 4.2% 4.2% 5.6% 1.4% Yuba CCD 10.1% 9.7% 11.1% 1.0% 16.0% 17.0% 18.4% 2.4% California Community Colleges ChanCellor’s offiCe 5522 DRAAFPTPENDICES Appendix C RACI Matrix A Responsible, Accountable, Consulted, and Informed (RACI) Matrix describes general roles and responsibilities of those involved in a specific process, such as MIS submissions. A RACI Matrix is useful in helping to set expectations for who is responsible for tasks, who is accountable for the work, who is consulted in the process, and who is kept informed about the task. If created correctly, a RACI Matrix helps members of an organization see where they need to be involved, and with which tasks. A RACI Matrix can also help eliminate confusion by knowing who is ultimately accountable for a task completion. It is particularly useful to set expectations with more senior stakeholders who are informed on the project; it will allow them to know what information they will receive and the steps involved. Table 1: Definitions of Responsible, Accountable, Consulted, and Informed (RACI) Role Role Code Definition This role completes the actual work and owns the problem; this role is the “doer.” There Responsible R can be multiple R’s. This role approves the completed work and is held accountable for it. There should only Accountable A be one A. This role has information and/or capability to complete the work. There will be two-way Consulted C communication between those responsible and those consulted. There can be multiple C’s. This role is informed of progress and results. This role receives one-way communication, Informed I usually from the R. This role is not involved in completing the work. There can be multi- ple I’s. Instructions for Definitions of Roles for MIS Submissions In the table below, list each role in the Roles column and enter the job title and a brief descrip- tion of the responsibilities of that position in the Definition column. The description is high level, with more specific details about the step(s) that person completes in the MIS submission in the RACI Matrix. Table 2: Definitions of Roles for MIS Submissions Roles Definition Instructions for Completing the RACI Matrix for MIS Submissions In the table below, list each step in the MIS submission in the left-hand column. In the remaining cells of the header row, insert the roles included in Table 2: Definitions of Roles for MIS Submissions. For example, if one role in Table 2 was Dean of Enrollment Services, one of the columns included in Table 3 would be headed as Dean of Enrollment Services. Insert an R and A for each step and include a C and/or I if applicable. Remember for each row, there must be an A. There may be more than one R, but try to limit the number of Rs so that completing the task does not get too complex. Fiscal crisis & ManageMent assistance teaM 5533 APPENDICDERSAFT Table 3: RACI Matrix for MIS Submission Steps Involved Roles in MIS Submission California Community Colleges ChanCellor’s offiCe 5544 DRAAFPTPENDICES Appendix D Data Governance Checklist The U.S. Department of Education’s Privacy Technical Assistance Center (PTAC) has developed a Data Governance Checklist designed to assist stakeholder organizations with establishing and maintaining a successful data governance program. Data governance can be defined as an orga- nizational approach to data and information management that is formalized as a set of policies and procedures that encompass the full life cycle of data, from acquisition to use to disposal. This includes establishing decision-making authority, policies, procedures, and standards regarding data security and privacy protection, data inventories, content and records management, data quality control, data access, data security and risk management, data sharing and dissemina- tion, as well as ongoing compliance monitoring. Although developed for K-12 education, the Data Governance Checklist contains many best practices applicable to higher education data governance. To download the checklist, visit https://studentprivacy.ed.gov/resources/checklist-da- ta-governance. Readers interested in using the checklist should download it from the link above in case PTAC has made updates since the publication of this report. Fiscal crisis & ManageMent assistance teaM 5555 APPENDICDERSAFT Appendix E Post Submission Analysis The purpose of a post submission analysis is to improve data quality and processes by reviewing what worked well and the challenges encountered during the data submission, and identifying action items to address the challenges and improve data quality and processes. If a district has a team responsible for the MIS submission or a portion of the submission, the analysis can be done via a team meeting so that different perspectives can be gathered and analyzed. Then, the group can identify priorities. If each task is primarily done by one person, a post submission analysis can be done by that person reviewing the prompts below and docu- menting changes to be made in the next submission. Process for Post Submission Analysis 1. Before the meeting send participants the purpose, time and place for the meeting. Ask participants to gather their thoughts and notes on experiences during the submission and bring to the meeting what they thought worked well and the pain point(s) during the submission. 2. Begin the meeting by reviewing the purpose of the meeting and the agenda. 3. Review what went well and document responses. 4. Brainstorm without judgment or discuss problems encountered that need to be addressed – focus on the problems and issues, not the solutions. 5. Brainstorm possible solutions to each pain point. These need to be concrete, doable actions. 6. Review results of brainstorming and come to consensus on specific actions that can be taken to address the challenges identified: • Determine what actions need to be sent to the leadership team for discussion/ approval and what can be implemented immediately • Prioritize and assign actions to individuals • Review and document action items and agreements 7. Close the meeting by debriefing on the meeting, including what worked and what might need to be improved in terms of the debrief process and group interactions for the next debrief. If time permits, review any unresolved items or review the list and agree where items should be referred. 8. Document agreements and share with attendees so that the individuals can be held accountable for assigned follow-up action items. California Community Colleges ChanCellor’s offiCe