Recommendations

December 2017 California Community Colleges The Colleges Reviewed Are Not Adequately Monitoring Services for Technology Accessibility, and Districts and Colleges Should Formalize Procedures for Upgrading Technology Report 2017-102 COMMITMENT INTEGRITY LEADERSHIP CALIFORNIA STATE AUDITOR 621 Capitol Mall, Suite 1200 | Sacramento | CA | 95814 916.445.0255 | TTY 916.445.0033 For complaints of state employee misconduct, contact us through the Whistleblower Hotline: 1.800.952.5665 Don’t want to miss any of our reports? Subscribe to our email list at auditor.ca.gov For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255 This report is also available online at www.auditor.ca.gov | Alternate format reports available upon request | Permission is granted to reproduce reports Elaine M. Howle State Auditor Doug Cordiner Chief Deputy December 5, 2017 2017-102 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the California State Auditor presents this audit report concerning the California Community Colleges’ monitoring of services for technology accessibility and the procedures for upgrading information technology (IT). This report concludes that the three community colleges we reviewed are not adequately monitoring their performance in responding to requests from students with disabilities for course materials in accessible media formats, and one has not established a goal for how long it should take to process these requests. Additionally, the Office of the Chancellor of the California Community Colleges (Chancellor’s Office) has not provided guidance to community colleges on monitoring their compliance with accessibility standards. As a result, the colleges cannot demonstrate that they are meeting students’ requests for accessible materials within a reasonable time frame. When students do not have equal access to instructional materials and their requests for an alternate format are not addressed promptly, they do not have equal educational opportunities. Although the colleges and related districts we reviewed have some processes and tools for replacing and upgrading their IT equipment, none of them have formalized these processes to ensure consistency and continuity in the future. Additionally, the Chancellor’s Office does not provide guidance to all of the colleges and districts related to upgrading or replacing IT equipment. We also noted that college instructional departments could not consistently provide documentation showing the stakeholder input received when deciding on what IT to replace or upgrade. Without formalizing their processes by documenting procedures for instructional department staff to follow when making decisions on technology equipment, the community colleges cannot ensure that these processes are consistently followed and are transparent. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 916.445.0255 916.327.0019 fax www.auditor.ca.gov iv Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Selected Abbreviations Used in This Report ACCIC Accrediting Commission for Community and Junior Colleges, Western Association of Schools and Colleges ADA Americans with Disabilities Act American River American River College Cerritos Cerritos Community College [a single-college district] De Anza De Anza College DSPS Disabled student programs and services Foothill–De Anza Foothill–De Anza Community College District Los Rios Los Rios Community College District CALIFORNIA STATE AUDITOR | Report 2017-102 v December 2017 CONTENTS Summary 1 Introduction 5 Community Colleges Are Not Adequately Monitoring Compliance With Accessibility Standards, and the Chancellor’s Office Should Provide Additional Guidance to Assist Community Colleges in Supporting Students With Disabilities 13 Community College Districts Plan for and Fund IT Needs but Lack Written Procedures to Guide Their Processes 27 Scope and Methodology 47 Responses to the Audit California Community Colleges Chancellor’s Office 51 Cerritos Community College District 53 California State Auditor’s Comment on the Response From Cerritos Community College District 59 Foothill–De Anza Community College District 61 California State Auditor’s Comments on the Response From Foothill–De Anza Community College District 67 Los Rios Community College District 69 California State Auditor’s Comments on the Response From Los Rios Community College District 73 vi Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2017-102 1 December 2017 SUMMARY To educate students in an environment in which technology is continually changing, community colleges need to have long‑term plans, as well as plans for periodically replacing and upgrading information technology (IT) equipment and support structures. Community colleges must also address the needs of students with disabilities to access websites and other technology as well as class materials. Federal and state accessibility laws require colleges to make all class materials available, upon request, in a format that is accessible to students with disabilities, and to ensure that websites meet guidelines for accessibility. California’s community colleges are required to plan for IT needs as part of their accreditation process. The foundation of these efforts is a technology master plan that identifies technology needs at the college. These needs may relate to technology infrastructure—items such as networking devices and devices that enable wireless communication, that allow the colleges to transmit information, and that support the general operations of the campuses. Colleges also need instructional technology, such as laptop computers in math labs, to support student learning. For this audit, we reviewed three community college districts: Foothill–De Anza Community College District (Foothill–De Anza), Los Rios Community College District (Los Rios), and Cerritos Community College (Cerritos), a single‑college district. In addition, we reviewed one individual college within each of the multicollege districts: De Anza College (De Anza) in Foothill–De Anza and American River College (American River) in Los Rios. Specifically, we reviewed the colleges’ compliance with federal and state accessibility standards. We also reviewed the colleges’ and districts’ processes for upgrading and replacing IT equipment, including their planning processes and financing. This report draws the following conclusions: Community colleges are not adequately monitoring compliance Page 13 with accessibility standards, and the Chancellor’s Office should provide additional guidance to assist community colleges in supporting students with disabilities. None of the three community colleges we reviewed are monitoring their performance in responding to requests from students with disabilities for course materials in accessible media formats (alternate media), and one college has not established a goal for how long it should take to process these requests. These colleges also do not have processes to monitor whether they comply with accessibility standards for instructional materials, nor has the Office of the Chancellor of the California Community Colleges (Chancellor’s Office) provided guidance to the colleges in either of these areas because it has focused its guidance in other areas and has limited staffing. As a result, the colleges cannot demonstrate that they are meeting students’ requests for accessible materials within a reasonable time frame. When students do not have 2 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 equal access to instructional materials and their requests for an alternate format are not addressed promptly, they do not have equal educational opportunities. Page 27 Community college districts plan for and fund IT needs but lack written procedures to guide their processes. The three community college districts and colleges we reviewed have some processes and tools for replacing or upgrading their technology equipment. However, they have not formalized these processes to ensure consistency and continuity in the future. The Chancellor’s Office does not provide guidance to all of the community college districts and colleges related to upgrading or replacing IT equipment. Additionally, each of the districts and colleges reviewed has a technology master plan, but Cerritos’ plan is not up to date and does not include detailed steps to implement its master plan. American River also lacks steps to implement its master plan. Further, college instructional departments could not consistently provide documentation showing the stakeholder input received when deciding on what information technology to replace or upgrade. Without formalizing their processes by documenting procedures for instructional department staff to follow when making decisions on IT equipment, the community colleges cannot ensure that these processes are consistently followed and are transparent. The Chancellor’s Office provided some guidance on implementing decision‑making processes. However, this guidance does not address documentation of input, attendees, or agreements reached at college governance or department meetings, including those to consider technology equipment requests. Lastly, the colleges offer technology training and assistance to faculty, staff, and students and various opportunities for these stakeholders to provide input on technology training needs. CALIFORNIA STATE AUDITOR | Report 2017-102 3 December 2017 Summary of Recommendations Chancellor’s Office • To ensure that all community colleges are complying with timeliness requirements, by June 2018, the Chancellor’s Office should establish guidance for the colleges on monitoring their effectiveness in responding to students’ requests for instructional materials in alternate media formats in a timely manner. At a minimum, this guidance should provide direction to community colleges on establishing a time‑frame goal for completing student requests and on establishing procedures to periodically monitor their performance in meeting those goals. • To ensure that students with disabilities have equal access to instructional materials, by June 2018, the Chancellor’s Office should develop guidance for the community colleges on periodically monitoring the accessibility of instructional materials. • To assist all community college districts and colleges in ensuring that they have consistent, transparent, and continuous implementation of their processes for upgrading and replacing IT equipment, by September 2018, the Chancellor’s Office should issue guidance to the districts and community colleges on establishing written procedures for those processes. • To assist all community colleges in increasing transparency of their decision‑making processes, by September 2018, the Chancellor’s Office should issue guidance to the community colleges on establishing procedures to document the attendees, input received, and agreements reached during department meetings, including those to consider IT equipment requests. Community Colleges and Districts • To ensure that they are fulfilling requests for alternate media services from students with disabilities in a timely manner, by June 2018, American River, Cerritos, and De Anza should each establish procedures for monitoring their timeliness in responding to such requests so that they can periodically review their performance in completing the requests. Additionally, to evaluate its performance, De Anza should establish a time‑frame goal for completing alternate media requests. 4 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 • To ensure that students with disabilities have equal access to instructional materials, by June 2018, American River, Cerritos, and De Anza should each develop procedures to monitor and periodically review the accessibility of instructional materials. • To ensure the consistent, transparent, and continuous implementation of processes for technology equipment upgrades and replacements, by June 2018, Cerritos, Los Rios, and Foothill–De Anza districts, and American River should each establish written procedures for these processes. • To ensure that it fully implements its technology master plan, by June 2018, American River should establish an implementation plan with detailed steps for achieving the goals in its master plan that it has not yet accomplished. Also, by June 2018, Cerritos should update its technology master plan, including detailed steps to accomplish its master plan goals. • To increase the transparency of their annual review processes, by June 2018, American River, Cerritos, and De Anza should establish procedures to document a summary of the input each receives and agreements reached during meetings to consider instructional IT requests. Agency Comments The Chancellors Office, Cerritos, and Foothill–De Anza agreed with our recommendations. Los Rios agreed with several of our recommendations and indicated that it plans to undertake steps to implement them. However, it disagreed with our recommendation to include a requirement in its next collective bargaining negotiations for instructors to periodically attend accessibility trainings. CALIFORNIA STATE AUDITOR | Report 2017-102 5 December 2017 INTRODUCTION Background The Office of the Chancellor of the California Community Colleges (Chancellor’s Office) reports that the California Community Colleges is the largest system of higher education in the nation, with more than 2.3 million students attending its 113 colleges in the 2016–17 academic year. The Board of Governors for the California Community Colleges sets policy and provides guidance to its colleges, which are organized into 72 community college districts. The community college districts each have a locally elected board of trustees charged with the operations of the local colleges. The colleges operate under a system of shared governance. Intended to ensure that faculty, staff, and students have the right to participate effectively in district and college governance, shared governance consists of procedures established by districts’ governing boards to give these stakeholders the opportunity to express their opinions at the campus level and to ensure that their opinions are given every reasonable consideration. For example, the districts and colleges we reviewed use shared governance committees, such as budget or technology committees made up of faculty, staff, and students. This audit focuses on the community college districts’ processes for replacing and upgrading information technology (IT) equipment. To educate students in an environment in which technology is continually changing, it is important to develop both long‑term plans and ongoing plans for replacing and upgrading IT equipment. The State’s 72 community college districts vary in size and composition, with 49 of the districts consisting of a single college and the remaining 23 districts containing two or more colleges. The multicollege districts average three colleges but have as many as nine. For this audit, we selected three districts that reflect this diversity in size as well as geography. Specifically, we reviewed two multicollege districts—Foothill–De Anza Community College District (Foothill–De Anza) in the Bay Area and Los Rios Community College District (Los Rios) in Northern California— and one single‑college district from Southern California, Cerritos Community College (Cerritos). Los Rios includes four colleges, and Foothill–De Anza has two colleges. College Accreditation Process State regulations require each community college to be an accredited institution. The Accrediting Commission for Community and Junior Colleges, Western Association of Schools and Colleges (ACCJC), is the accrediting agency for the community colleges in California. According to ACCJC’s Manual for 6 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Institutional Self‑Evaluation, accreditation is a system for regulating the quality of higher education by evaluating educational quality and institutional effectiveness on a regular basis. In its manual, ACCJC states that its accreditation process provides assurance to the public that the accredited member colleges meet accreditation standards of quality, that the education earned at the colleges is of value to the student who earned it, and that employers, trade‑ or profession‑related licensing agencies, and other colleges and universities can accept students’ credentials as legitimate. Further, accreditation is one of the requirements for community colleges to be eligible to receive federal aid, including grants for students. ACCJC directs the accreditation process for the California Community Colleges, a process that includes several components of evaluation and review. According to ACCJC’s manual, accredited institutions participate in a comprehensive review for reaffirmation of accreditation once every seven years. As part of the review, institutions complete a self‑evaluation and undergo an external evaluation by a team of peer evaluators. These teams conduct a review following completion of institutional self‑evaluations to determine the extent to which an institution meets ACCJC’s standards. After the review, the teams of peer evaluators make recommendations for compliance and improvement that will help the college better meet ACCJC’s standards. The teams also commend excellent practices when appropriate and provide both the college and ACCJC with a report of their findings. ACCJC has standards that cover a wide variety of college activities, including several related to IT. As shown in Figure 1, ACCJC has four general categories of accreditation standards, and within the resources category are standards that directly relate to IT.1 Our review of districts focused on some of the standards associated with resources and leadership and governance. These included standards involving policies and procedures that guide training in the use of technology, as well as ones involving planning for upgrades and replacements to technology. 1 Because the colleges we reviewed were most recently accredited by ACCJC under the June 2012 standards, we include those standards in Figure 1. ACCJC published updated standards in 2014, and the revised standards became the basis for comprehensive institutional evaluations for reaffirmation of accreditation beginning in spring 2016. CALIFORNIA STATE AUDITOR | Report 2017-102 7 December 2017 Figure 1 Relevant ACCJC Accreditation Standards in Effect for the Most Recent Accreditation Reviews of Selected Colleges I Institutional Mission and Effectiveness II Student Learning Program and Services AREAS OF ACCREDITATION STANDARDS III Resources The institution assures that any technology support it provides is designed to meet the needs of learning, I V Leadership and teaching, collegewide communications, research, and Governance operational systems. • The institution provides quality training in the effective application of its IT to students and The institution establishes and implements a written college personnel. policy providing for faculty, staff, administrator, and • The institution systematically plans, acquires, student participation in decision-making processes. maintains, and upgrades or replaces technology The policy specifies the manner in which individuals infrastructure and equipment to meet bring forward ideas from their constituencies and institutional needs. work together on appropriate policy, planning, and special-purpose bodies. • Faculty and administrators have a substantive and clearly defined role in institutional governance and exercise a substantial voice in institutional policies, planning, and budget that relate to their areas of responsibility and expertise. Students and staff also have established mechanisms or organizations for providing input into institutional decisions. Source: Selected 2012 ACCJC Accreditation Standards that apply to this audit. 8 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Types of Campus IT and State Funding for Community College Districts As part of their mission to educate students, community colleges acquire IT equipment. We determined that this IT equipment falls into two categories: technology equipment supporting the infrastructure of colleges and districts and instructional technology equipment supporting student learning. Infrastructure technology equipment includes items used throughout the college, such as computers for faculty and staff and networking equipment such as printers, switches, and Wi‑Fi devices. These items enable the colleges to receive and transmit information and support the general operations of the campus. In contrast, instructional technology equipment is installed for classroom purposes or for use by specific instructional departments, such as laptop computers for students in math labs and specialized printers. The community college system in California is supported by an allocation of funding through the State’s annual budget process. State law requires the Board of Governors to prepare and adopt an annual statewide budget proposal, which includes funding for state operations of the Chancellor’s Office and local assistance funding for the community college districts. For fiscal year 2016–17, the Board of Governors’ budget request identified several priorities, such as increased funding for general operating and IT expenses and to hire more full‑time faculty. The Legislature appropriated more than $4.6 billion in local assistance funds for the community college districts, consisting of general apportionments of $3.1 billion for the districts and $1.5 billion for categorical programs to accomplish specific program objectives. According to the Chancellor’s Office’s deputy chancellor, the majority of these categorical funds are allocated to districts for local program implementation, but roughly $100 million is directed to statewide initiatives that the Chancellor’s Office manages in partnership with lead districts. This amount did not include a specified allocation for IT from the district apportionments, but it did include funding for IT projects directed by the Chancellor’s Office. For example, the Legislature appropriated nearly $28 million for telecommunications and technology infrastructure. We discuss an example of these projects in greater detail in the Audit Results. Accessibility Law To address the needs of individuals with disabilities, the federal government enacted the Rehabilitation Act of 1973 (Rehabilitation Act). The Rehabilitation Act states that no otherwise qualified individual with a disability shall, solely by reason of his or her disability, be excluded from participating in, be denied the benefits of, or be subjected to discrimination under any program or activity receiving federal assistance. As recipients of federal assistance, California’s community colleges are subject to the provisions of the Rehabilitation Act. In 1990 the federal government reinforced its CALIFORNIA STATE AUDITOR | Report 2017-102 9 December 2017 commitment to individuals with disabilities by enacting the Americans with Disabilities Act (ADA), which provides individuals with disabilities civil rights protection and places emphasis on providing them with equal opportunity. Specific provisions of both the Rehabilitation Act and the ADA apply to programs and activities provided by public entities, including California’s community colleges. In 1998 Congress enacted Section 508 of the Rehabilitation Act (Section 508), which requires federal agencies to make electronic and IT accessible to individuals with disabilities. The law applies to federal agencies when they develop, procure, maintain, or use electronic and IT. Under Section 508, agencies must provide individuals with disabilities access to and use of information and data that are comparable to the access to and use of the information and data available to others. In 2002 the California Legislature amended state law to make the requirements of Section 508 applicable to public entities in California. Because California’s community colleges are public entities, they must comply with the provisions of the Rehabilitation Act, the ADA, and Section 508. Additionally, the Architectural and Transportation Barriers Compliance Board, an independent federal agency that develops guidelines and standards for accessibility, published in January 2017 an update to the requirements for information and Examples of Services That Can Be Provided communication technology covered by Section 508. In by DSPS Offices part, the updated requirements are intended to clarify the types of internal or nonpublic electronic content that • Access to adaptive educational equipment, materials, agencies must make accessible, including electronic and supplies. educational materials. By January 2018, agencies will be • Assessment for learning disabilities. required by federal law to apply accessibility standards to all information and communication • Facilitation of test taking, including adapting tests for and technologies that they develop, purchase, maintain, or use, proctoring test taking by disabled students. such as video, audio, web services, and software • Interpreter or captioning services for hearing-impaired or programs (instructional materials). deaf students. • Job placement and development services related to Disabled student programs and services (DSPS) offices transitioning from school to employment. at California community colleges provide support services and educational accommodations to students • Mobility assistance. with disabilities so that they can equally participate in • Note-taking and manual manipulation for and benefit from the college educational experience. As classroom activities. defined in state guidelines published by the Chancellor’s • Registration assistance. Office, the terms alternate media and accessible formats refer to methods of making information accessible to • Specialized counseling. persons with disabilities. For example, DSPS offices can • Specialized instruction. provide audio versions of textbooks to students with • Specialized tutoring. visual impairments. Additionally, colleges can provide specialized instruction or counseling as part of their • Transcription services , including Braille and print materials. DSPS programs. Examples of DSPS services are Source: Education Code, section 67311. included in the text box. According to Chancellor’s Office data, more than 120,000 students with disabilities attended California community colleges during the 10 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 2015–16 academic year. These students have disabilities such as visual impairments, mobility impairments, hearing impairments, and psychological disabilities. Website Accessibility Levels of Noncompliance With Website Accessibility Guidelines Federal regulations implementing Section 508 incorporate standards that ensure access to Level A: Instances that a web content developer must online services for persons with disabilities, satisfy; otherwise, one or more groups will find it impossible based on the World Wide Web Consortium’s to access the information. Web Content Accessibility Guidelines (web Level AA: Instances that a web content developer should accessibility guidelines). The consortium is an satisfy; otherwise, one or more groups will find it difficult to international community that develops open access the information. standards to ensure long‑term growth of the web. The guidelines present three levels that Level AAA: Instances that a web content developer may address; otherwise, one or more groups will find it describe, in descending order from most severe somewhat difficult to access information. to least severe, the effects of not complying with the standards, as shown in the text box. Source: Web accessibility guidelines. Colleges and universities nationally, including Harvard University, Massachusetts Institute of Technology, Princeton University, and the University of California, Berkeley, have been subject to lawsuits and compliance reviews resulting from difficulties experienced by students with disabilities in accessing their respective web and IT resources. Further, in September 2015, a California community college settled a lawsuit with a student alleging discrimination because of a lack of accessibility in the college’s online instructional platform and related course materials. The district settled this case for $40,000 plus the plaintiff’s legal fees and agreed to take certain corrective actions. Chancellor’s Office Accessibility Activities State law requires the Board of Governors to develop and implement a system for evaluating state‑funded programs and services for disabled students on each campus at least every five years. At a minimum, these systems are to provide for the gathering of outcome data, staff and student perceptions of program effectiveness, and data on the implementation of the program and the physical accessibility requirements of the Rehabilitation Act. Additionally, state law requires that every two years the Board of Governors submits a report to the Governor, the Legislature’s education policy committees, and the California Postsecondary Education Commission describing its efforts to serve students with disabilities. We discuss the most recent report later in this report. CALIFORNIA STATE AUDITOR | Report 2017-102 11 December 2017 State law also assigns statewide responsibility for reviewing and approving all new educational programs in the community colleges to the Board of Governors. The Board of Governors has delegated these responsibilities to the academic affairs division of the Chancellor’s Office. To meet these responsibilities, the Chancellor’s Office developed the California Community Colleges Curriculum Committee (Curriculum Committee), which coordinates efforts between local and statewide curriculum processes and provides guidance and recommendations on curriculum design to districts. Instructional programs that the Board of Governors approves include traditional face‑to‑face instruction as well as distance education, in which the instructor and students interact at a distance through the assistance of communication technology. State regulation specifies that the same quality standards that apply to traditionally delivered courses also apply to distance education, and that each course designed for delivery via distance education must be separately approved by the Curriculum Committee. To help colleges meet accessibility requirements when developing distance education courses, the Chancellor’s Office has established a task force consisting of campus experts in distance education, web accessibility, curriculum, instructional technology, and DSPS program management to produce accessibility guidelines for distance education. 12 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2017-102 13 December 2017 Community Colleges Are Not Adequately Monitoring Compliance With Accessibility Standards, and the Chancellor’s Office Should Provide Additional Guidance to Assist Community Colleges in Supporting Students With Disabilities Key Points • American River College (American River) in Los Rios did not monitor the time it takes to address requests for providing alternate media services to students with disabilities, and consequently, it did not always provide timely responses. Additionally, De Anza (De Anza) in Foothill–De Anza and Cerritos did not record and track sufficient information to be able to review how long they took to respond to these requests. When students do not have access to the same instructional materials and their requests for an alternate format are not addressed promptly, they do not have equal educational opportunities. • The Chancellor’s Office has not provided guidance on monitoring the accessibility of instructional materials, and the community colleges we reviewed are not actively monitoring compliance with accessibility standards for these materials. Without a process to monitor the accessibility of instructional materials, colleges cannot demonstrate that they are complying with accessibility standards and meeting the needs of students with disabilities. • American River and De Anza do not have adequate processes to ensure that their respective websites are accessible to students with disabilities. If the websites are not accessible, students with disabilities do not have equal access to and equal opportunity in the use of information on the web. • The community colleges we reviewed offer training on accessibility of instructional materials, but do not require attendance by all instructors. As a result, the colleges cannot ensure that faculty members are aware of their responsibility to comply with accessibility standards for instructional materials they may choose to use. American River, Cerritos, and De Anza Colleges Are Not Monitoring Their Timeliness in Addressing Requests From Students With Disabilities State regulations related to accessibility require community colleges to provide alternate media, auxiliary aids, and services in a timely manner to ensure equal opportunity for students with disabilities. As discussed in the Introduction, the terms alternate media and accessible formats refer to methods of making information accessible to persons with disabilities. 14 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Also described in the Introduction, the Chancellor’s Office has published guidelines for community colleges to use when producing instructional materials in alternate media in response to requests from students with disabilities. One basic principle of these guidelines is that colleges should establish procedures for responding in a timely manner to such requests. However, neither the law nor state guidelines set specific time frames for what is timely.2 Further, the guidelines do not address the need to monitor the timeliness of responding to requests for alternate media. If students’ requests for alternate media are not addressed promptly, the students will not have equal educational opportunities. The guidelines do not address the need to monitor the timeliness of responding to requests for alternate media. Although American River and Cerritos both indicated that they have a goal of completing alternate media requests within two weeks, or 10 business days, neither college actively monitors how quickly it addresses such requests or tracks how often it exceeds its two‑week goal. Without monitoring the timeliness of completing alternate media requests, the colleges are unable to demonstrate that they are providing timely access to instructional materials for students with disabilities. Although American River tracks the dates of the alternate media requests and dates of their completion, it does not calculate the number of days that it takes to complete the requests or identify the number of instances in which it exceeds its two‑week goal. However, our review of American River’s data for December 2015 through January 2017 shows that American River exceeded its time‑frame goal in 25 of 482 alternate media requests, or 5 percent of the requests received during that time period. These delays ranged from one day to 69 days, averaging 19 days beyond the college’s two‑week goal. American River’s supervisor of DSPS stated that he relies on the program staff to update him on the status of completing requests and that he was not aware of the delays. Further, he agreed that the college should be doing more to track alternate media requests, and he is developing a report that will show the timeliness of requests as well as procedures for reviewing requests that are approaching the two‑week time frame. The supervisor informed us 2 The Chancellor’s Office’s guidelines state that timeliness is a relative term that depends on the context. For a student who requests a textbook in an accessible format, responding in a timely manner would involve providing the book in an alternate format by the time other students in the class are called upon to use the book. If the entire text cannot be supplied in an alternate format by that time, it may be necessary to deliver it in installments that keep pace with the class. CALIFORNIA STATE AUDITOR | Report 2017-102 15 December 2017 in September 2017 that he planned to have these procedures and a reporting tool implemented by October 2017. According to its coordinator of DSPS, American River does not track complaints about accessibility services. She stated that if students, faculty, or staff go to the DSPS with a complaint, the supervisor will discuss it with the person and attempt to provide a resolution, but this process is only verbal and is not tracked or documented in writing. According to its supervisor of DSPS, American River receives only a couple of complaints per semester. However, because American River does not have a process for documenting complaints about its accessibility services, we could not verify the supervisor’s claim. Cerritos does not consistently record certain data that would allow it to measure its timeliness in fulfilling alternate media requests by its two‑week goal. Cerritos’ DSPS division tracks each alternate media request using a spreadsheet, and the dean stated that the tracking spreadsheet could be used to research the specific timeliness concerns. However, Cerritos does not consistently record the dates that alternate media requests are received; therefore, its tracking spreadsheet does not always contain the information necessary to accurately calculate the time it takes for Cerritos to complete students’ requests. The dean stated that the DSPS division handles the few complaints it receives informally and therefore does not have a formal process for tracking complaints. Because Cerritos does not have a process for recording and tracking complaints involving the timeliness of alternate media requests, we could not verify Cerritos’ timeliness in addressing alternate media requests or its claim that there were few complaints. The third college we reviewed, De Anza, has not established a specific goal for completing alternate media requests and does not formally track its timeliness in completing requests. Instead, the dean of DSPS stated that she directs staff to complete requests as soon as possible, and that she meets with staff on a weekly basis and is aware of any issues students are having with accommodations. De Anza tracks the date and type of alternate media requests, but it does not track the completion date of requests in its tracking system. The vice president of student services stated that she did not know that the college needed to establish and track its timelines for completing alternate media requests. However, without De Anza tracks the date and type of alternate media requests, but it does not track the completion date of requests in its tracking system. 16 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 establishing a goal for completing requests and a process to measure its performance, De Anza cannot demonstrate that it completes student requests for alternate media in a timely manner. The vice president of student services stated that De Anza has had only one complaint about its delivery of alternate media services since 2010. Nevertheless, because De Anza does not have a process for tracking complaints specific to the timeliness of alternate media requests, we could not verify this claim. The dean of DSPS said that De Anza has not established a timeliness goal because alternate media requests have historically been processed as quickly as possible. However, when we discussed the lack of any way to measure the timeliness of completing these requests with the college, De Anza’s dean of DSPS and the vice president of student services agreed that De Anza will develop a time‑frame goal and begin tracking the number of days it takes to complete alternate media requests in a weekly report. When the community colleges do not record and monitor their timeliness in addressing requests for instructional materials in alternate media from students with disabilities or have processes to record and track complaints, they cannot demonstrate that they are complying with timeliness requirements. The Chancellor’s Office Should Provide Additional Guidance to Help Community Colleges Monitor Compliance With Accessibility Standards for Instructional Materials As discussed in the Introduction, the Chancellor’s Office is responsible for reviewing and approving all new educational programs in community colleges. To assist community college administrators, faculty, and staff in developing programs and courses and submitting them for review, the Chancellor’s Office developed the Program and Course Approval Handbook (handbook). However, the handbook does not include any requirements for colleges to ensure accessibility of instructional materials. The dean of educational programs and professional development for the Chancellor’s Office stated that the purpose of the handbook is to focus colleges on the process and procedures of submitting courses and programs for state approval and not a guide to developing effective curriculum. However, as discussed in the Introduction, federal law will require colleges, as of January 2018, to comply with accessibility standards for instructional materials. The dean stated that the Chancellor’s Office will include a statement in the next edition of its handbook addressing the requirement to make all instructional materials accessible. Additionally, the Chancellor’s Office developed guidelines for community colleges to follow when producing instructional and other printed materials in alternate media for persons with disabilities. Although these guidelines are not legally CALIFORNIA STATE AUDITOR | Report 2017-102 17 December 2017 binding on districts, they indicate that the Chancellor’s Office will apply them when determining whether a district has met its obligations under state law and regulations related to accessibility of printed materials. Districts that do not follow the guidelines bear the burden of demonstrating that they have met their legal obligation to provide access to printed materials. The Chancellor’s Office guidelines establish basic principles that community colleges should follow to ensure that instructional materials and other information resources are accessible to and usable by persons with disabilities. The guidelines also encourage community colleges to review all existing curriculum, materials, and resources as quickly as possible and to make necessary modifications to ensure access for students with disabilities. At a minimum, the Chancellor’s Office expects the colleges to review and revise the instructional resources or materials used in each course when the course undergoes curriculum review every six years as part of the accreditation process. Further, the guidelines specify that all college administrators, faculty, and staff involved in the development and use of such materials or resources share the responsibility for ensuring that instructional materials are accessible to students with disabilities. The guidelines do not, however, dictate the method for verifying that instructors are ensuring accessible instructional materials. The dean of educational programs and professional development stated that the Chancellor’s Office has generally focused on developing guidance for areas that need the greatest attention, such as ensuring accessibility of materials used in distance education courses. He also noted that the Chancellor’s Office has not developed more guidance because of limited staffing. However, he agreed that it could probably provide more guidance on monitoring instructional materials used in the classroom. He also agreed that community colleges could benefit from specific guidance from the Chancellor’s Office regarding monitoring the adherence to accessibility standards. The guidelines do not dictate the method for verifying that instructors are ensuring accessible instructional materials. As we discussed in the Introduction, state law requires the Board of Governors to develop and implement a system for evaluating state‑funded programs and services for disabled students, which includes gathering of outcome data, staff and student perceptions 18 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 of program effectiveness, and data on the implementation of the program and physical accessibility requirements of the Rehabilitation Act. Additionally, state law requires that the Board of Governors submit a report to the Governor, the Legislature’s education policy committees, and the California Postsecondary Education Commission every two years describing its efforts to serve students with disabilities. The Board of Governors delegated the responsibility for the report to the Chancellor’s Office. According to the report submitted in April 2016, because of budgetary constraints, the Chancellor’s Office suspended the reporting of its annual coordinated student services programmatic site review process, beginning with the 2009–10 academic year. The report noted that the information gathered during the site visits made up a significant portion of the content of past reports. As a result, the Chancellor’s Office reported that it is unable to report on three of the four elements required by statute: staff and student perceptions of program effectiveness, data on the implementation of the program, and physical accessibility requirements of the Rehabilitation Act. Instead, the report focused solely on the analysis of the outcome data reported to the Chancellor’s Office by the community colleges, such as enrollment, retention, transition, and graduation data for students receiving services through DSPS. However, regardless of budgetary constraints, state law continues to require the Chancellor’s Office to implement and report on a system for evaluating state‑funded programs and services for disabled students, which includes staff and student perceptions of program effectiveness, data on the implementation of the program, and physical accessibility requirements of the Rehabilitation Act. In our discussions with administrators at American River, Cerritos, and De Anza colleges, each described challenges with ensuring that the instructional materials comply with accessibility standards. For example, American River’s dean of planning, research, and technology stated that instructors sometimes use materials or free software without informing the college administrators, preventing the accessibility compliance officer from knowing whether the software or materials meet the appropriate accessibility standards. He further stated that it is the responsibility of individual faculty members to ensure that course materials meet the students’ accessibility needs. However, he confirmed that the college has not established a process to routinely verify that the materials instructors use meet accessibility standards. De Anza’s associate vice president of instruction, academic services, and learning resources stated that De Anza does not have the human resources to review all courses for accessibility of instructional materials. However, without a process to periodically review the accessibility of instructional materials, the college cannot demonstrate that it complies with accessibility standards. CALIFORNIA STATE AUDITOR | Report 2017-102 19 December 2017 Further, when describing challenges faced in meeting accessibility requirements, Cerritos’ dean of DSPS stated that some instructional textbooks contain additional materials that are not accessible, such as links to websites that are not accessible or videos without captioning. She also noted that instructors sometimes adopt instructional software that is not accessible. When we asked if Cerritos has a process to review textbooks and add‑ons for accessibility, the dean stated that Cerritos’ curriculum review process is based on the good faith of the instructor. Cerritos’ captioning and accessibility guidelines state that video captions should be displayed whether or not there is an identified student with a disability, and faculty should make sure audio, visual, and written materials are accessible before distributing them to the class. However, Cerritos has not established a process to periodically verify whether the instructional materials comply with accessibility standards. Cerritos’ vice president of academic affairs stated that Cerritos does not review the accessibility of instructional materials because it does not have the staffing resources necessary to perform such reviews. Nevertheless, to comply with federal law, colleges will need to comply with accessibility standards for instructional materials by January 2018. Without a process to monitor the accessibility of instructional materials, colleges cannot demonstrate that they are complying with accessibility standards and meeting the needs of students with disabilities. Oversight of Web Accessibility Compliance Is Necessary to Ensure That Websites Are Accessible to Students With Disabilities To ensure equal access to online services for persons with disabilities, California has adopted standards to address the needs of users who may have disabilities, such as visual, hearing, and mobility impairments. As discussed in the Introduction, California requires that state governmental entities’ websites comply with Section 508 of the Rehabilitation Act, and with the web accessibility guidelines that it incorporates. According to the web accessibility guidelines, website accessibility means that people with disabilities can perceive, understand, navigate, and interact with the website, and that they have equal access to the information on the site. For example, images on a website need to have alternate text so that a person who is blind can listen to a description of the image by using a screen reader, and videos must have captions for people who are deaf or hard of hearing. The Chancellor’s Office funds a grant with one of its college districts, the Butte‑Glenn Community College District (Butte), to operate the California Community Colleges Technology Center (Technology Center). Through the Technology Center, Butte offers training through webinars and workshops to colleges on how to make a website accessible and provides them with tools 20 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 for checking their websites’ compliance with the web accessibility guidelines. Although the Chancellor’s Office directs community colleges to develop their own policies and processes for website accessibility and mentions using website evaluation tools, it does not provide the community colleges with guidance on the need to regularly monitor their websites for accessibility. According to state law, the Board of Governors is to provide leadership and direction in the continuing development of the community colleges. However, the Board of Governors has delegated to the Chancellor’s Office the responsibility for communicating information about educational programs to the districts, including the development of guidelines and best practices for technology accessibility. The Chancellor’s Office does not provide the community colleges with guidance on the need to regularly monitor their websites for accessibility. We examined the processes and, when available, procedures and monthly reports on website accessibility of the colleges we reviewed to determine whether they ensure compliance with web accessibility guidelines. Two of the colleges, American River and De Anza, do not have an adequate process to ensure that their websites are accessible to users with disabilities. American River’s website has multiple known accessibility problems that are ongoing, and it has not dedicated the necessary resources to fully correct those problems. Specifically, American River uses a third‑party vendor to track its website accessibility and receives monthly reports that identify accessibility errors. The monthly accessibility reports for December 2016 through June 2017 show that American River’s website contained an average of 26 Level A accessibility issues, the most severe type of problem. As we described in the text box on page 10, Level A denotes instances that make it impossible for one or more groups of individuals with disabilities to access the information. For example, in April 2017, American River’s website had images without alternate text, meaning that a visually impaired person using a screen reader would not know what the image contains. In another example, its website had multimedia content that lacked captions, preventing a person who is deaf from knowing that the media contains audio descriptions. The Los Rios district’s technology master plan specifies that each of the four colleges, including American River, is responsible for creating and maintaining its own website. When we asked why American River has not addressed the known CALIFORNIA STATE AUDITOR | Report 2017-102 21 December 2017 accessibility issues with its website, its dean of planning, research, and technology stated that it would take a technician three to four months to fix all the issues with the website. Instead, he said that American River is working with Los Rios to develop a new, more robust website that complies with accessibility guidelines, and it has contracted with a consultant to create the new site. According to the contract, the new website will be completed by January 2018. To address any immediate website accessibility issues, American River has an email link on each page of its website for users to notify the web administrator if someone encounters an accessibility issue. According to the IT supervisor, he had not received any emails regarding website accessibility issues. However, American River does not have a process for tracking any accessibility complaints submitted by website users or for documenting their resolution. Without a process for tracking and reviewing the resolution of accessibility complaints submitted through its website, American River cannot demonstrate that it is prepared to promptly address and monitor complaints related to website accessibility. De Anza has not established policies or procedures to monitor its website for accessibility. Instead, the associate vice president of communications and external relations stated that De Anza relies on its senior web coordinator and web support technician to monitor its compliance because they are well versed in the mechanics of ensuring accessibility. According to the senior web coordinator, he scans De Anza’s website monthly for accessibility errors. Further, the associate vice president stated that De Anza’s current content management system has accessibility checks built into it that allow instructors to run a scan to identify accessibility errors. However, because instructors are not required to run the accessibility scan, the content management system does not prevent users from publishing inaccessible materials on the website. De Anza relies on its senior web coordinator and web support technician to monitor its compliance. Additionally, De Anza’s process is informal, and the college does not document its compliance with web accessibility guidelines. According to the senior web coordinator, the web team generally fixes accessibility errors within one business day of identifying them. However, the tools that De Anza uses do not have a tracking mechanism to demonstrate how many accessibility errors it has identified or how long it took to fix those errors. Instead, De Anza’s 22 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 accessibility tools show only the current status of website accessibility, and the college does not maintain records of past errors or reports. The associate vice president of communications and external relations stated that De Anza plans to implement a new version of its content management system in January 2018 that will run the accessibility checks automatically rather than relying on instructors to run them manually. She indicated that the automated checks will enforce compliance with web accessibility guidelines. Additionally, users can submit a website accessibility complaint through an email on the college’s website. However, De Anza did not have a process for tracking accessibility complaints submitted by website users or for documenting their resolution. After we discussed our concern with the senior web coordinator, he developed a process for tracking accessibility complaints and the college’s resolution. In contrast, Cerritos, the third community college we reviewed, has established a process for reviewing all changes made to its website so that it ensures compliance with accessibility standards. When saving any changes to the website, faculty and staff receive a reminder from Cerritos’ content management system to review an accessibility report for the content, and if a page contains any accessibility errors or warnings, the system will prevent the page from being published to the website. Although Cerritos does not maintain records of past accessibility tests, its process prevents inaccessible materials from being published on its website. Additionally, in our review of the accessibility of the homepages of the three colleges we reviewed, Cerritos was the only college that had no accessibility problems, whereas American River and De Anza both had problems that would prevent users with one or more types of disabilities from accessing some of the information. To address website accessibility problems, Cerritos has a form on its website for users to report accessibility complaints. However, it does not have a process for tracking accessibility complaints submitted by website users or for documenting its resolution of the complaints. As discussed previously, without a process for tracking and reviewing the resolution of accessibility complaints submitted through its website, Cerritos cannot demonstrate that it is prepared to promptly address and monitor complaints related to website accessibility. Community Colleges Do Not Require All Instructors to Attend Accessibility Training Courses Although the colleges we reviewed offer training for faculty and staff in meeting federal and state accessibility requirements for instructional materials, they do not require all faculty and staff to attend those trainings. Neither federal nor state law CALIFORNIA STATE AUDITOR | Report 2017-102 23 December 2017 requires community colleges to offer training on procurement or development of accessible educational materials for instructors. However, American River, Cerritos, and De Anza all offer such training as part of their programs to ensure that they comply with accessibility requirements. For example, De Anza offers instruction on how to make Microsoft Word documents, such as course syllabi, accessible to students with disabilities through screen‑reading software. Similarly, Cerritos offers instructors training on how to ensure that their PDF documents are accessible. Additionally, the Chancellor’s Office offers colleges training, workshops, and technical assistance on meeting accessibility requirements. According to the dean of educational programs and professional development for the Chancellor’s Office, there is no law that requires instructors to attend accessibility training, and the Chancellor’s Office does not have the authority to direct how the community colleges should handle accessibility training, given current limitations placed on districts by the collective bargaining agreements with instructors. However, he agreed that the Chancellor’s Office could do more to provide colleges with guidance and best practices related to accessibility. Although they offer training in implementing accessible materials as a resource to instructors, American River, Cerritos, and De Anza have not required all instructors to take this training. Cerritos’ dean of DSPS noted that the college’s collective bargaining agreements with instructors do not include this type of mandated training. Similarly, De Anza’s associate vice president of instruction, academic services, and learning resources noted that its accessibility training resources are optional because the collective bargaining agreements limit the college’s ability to require training. American River’s dean of planning, research, and technology also noted that such a requirement would be a collective bargaining issue. We determined that the collective bargaining agreements all three colleges have with their faculty unions likely prevent them from unilaterally imposing a new training requirement on instructors. However, these agreements could be updated during the next collective bargaining agreement process. Without requiring faculty to attend accessibility training, colleges cannot ensure that faculty are aware of their responsibility to comply with accessibility requirements for the instructional materials they use. 24 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Recommendations Chancellor’s Office To ensure that all community colleges are complying with timeliness requirements, by June 2018, the Chancellor’s Office should establish guidance for the colleges on tracking and monitoring their effectiveness in responding to students’ requests for instructional materials in alternate media in a timely manner. At a minimum, this guidance should provide direction to all community colleges on establishing a time‑frame goal for completing students’ requests. The guidance should also provide direction to colleges for establishing procedures to track and periodically monitor their performance in promptly responding to requests, identify reasons for delays in responding to requests, and take action as needed to improve their timeliness in completing future requests. To ensure that community colleges promptly address any complaints they receive related to alternate media requests and web accessibility, by June 2018, the Chancellor’s Office should provide guidance to the community colleges on developing procedures to track and periodically review complaints received related to accessibility services, and to address any accessibility complaints in a timely fashion. To comply with statutory reporting requirements on its efforts to serve students with disabilities, by June 2018, the Chancellor’s Office should establish and report on a system for evaluating state‑funded programs and services for disabled students, including its gathering of outcome data, staff and student perceptions of program effectiveness, and data on the implementation of the program. To ensure that students with disabilities have equal access to instructional materials, by June 2018, the Chancellor’s Office should develop guidance for the community colleges on periodically monitoring the accessibility of instructional materials and on providing training to all instructors in making their materials accessible to students with disabilities. To ensure that community colleges’ websites comply with accessibility guidelines, by September 2018, the Chancellor’s Office should provide guidance to colleges on establishing policies and procedures to monitor the accessibility of their websites. Additionally, by September 2018, the Chancellor’s Office should provide guidance on best practices for colleges to use in preventing their websites from containing inaccessible information. CALIFORNIA STATE AUDITOR | Report 2017-102 25 December 2017 Community Colleges To ensure that they are fulfilling requests for alternate media services from students with disabilities in a timely manner, by June 2018, American River, Cerritos, and De Anza should each establish procedures for monitoring their timeliness in responding to such requests so that they can periodically review their performance in completing the requests. Specifically, Cerritos and De Anza should record and track sufficient information to be able to review how long they take to complete requests. Additionally, American River, Cerritos, and De Anza should each calculate the number of days they take to complete requests, and periodically evaluate their performance against their time‑frame goals. Further, to evaluate its performance, De Anza should establish a time‑frame goal for completing alternate media requests. To ensure that they promptly address any complaints they receive related to web accessibility and alternate media requests, by June 2018, American River and Cerritos should each establish procedures for tracking and reviewing complaints received related to accessibility and addressing complaints in a timely fashion. Additionally, De Anza should follow its new procedures for tracking and reviewing complaints related to accessibility. To ensure that students with disabilities have equal access to instructional materials, by June 2018, American River, Cerritos, and De Anza should each develop procedures to monitor and periodically review the accessibility of instructional materials. For example, each college could develop an accessibility checklist for instructors to complete when developing or selecting instructional materials, from which the college could periodically review a sample of course content to ensure that instructors completed the checklist and that the instructional materials comply with accessibility standards. To ensure that their websites comply with accessibility standards, by June 2018, American River and De Anza should each develop procedures to monitor website accessibility and incorporate steps to prevent instructors from publishing inaccessible content on the colleges’ respective websites. These procedures should include a tracking mechanism to demonstrate how many accessibility errors each college identifies and how long it takes to fix those errors. To ensure that all instructors are aware of the accessibility standards for instructional materials, American River, De Anza, and Cerritos should each include in their next collective bargaining negotiations a requirement for instructors to periodically attend accessibility trainings. 26 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2017-102 27 December 2017 Community College Districts Plan for and Fund IT Needs but Lack Written Procedures to Guide Their Processes Key Points • Although the three community college districts we reviewed have some processes and tools they use for replacing or upgrading their infrastructure technology equipment, they have not formalized their processes. Providing clearly documented procedures would help the college districts to ensure consistent implementation and continuity in the future. • Each of the three districts we reviewed has a technology master plan to accomplish its institutional technology goals. However, one district’s technology master plan is not up to date and does not include action plans to ensure that it accomplishes its technology goals • Community colleges use an annual review process to consider instructional equipment requests; however, the colleges we reviewed could increase transparency by consistently documenting faculty and staff input. • American River, Cerritos, and De Anza solicit feedback and input from faculty, staff, and students on their technology training needs. • The community college districts we reviewed fund their IT programs at varying levels using different sources. The Chancellor’s Office provides high‑level guidance on budgeting and allows each district to determine the appropriate funding sources to fulfill its technology needs. The Districts We Reviewed Replace and Upgrade Infrastructure Technology Equipment but Lack Formal Processes for Doing So Although all three districts and community colleges we reviewed periodically replace or upgrade infrastructure technology equipment, the entities responsible for these decisions have not established formal procedures that their IT departments should follow when doing so. As we described in the Introduction, ACCJC’s standards include a requirement that community colleges plan for upgrading and replacing technology. However, the standard does not specify how to carry out this planning process, leaving it to the discretion of the community colleges. Specifically, the standard requires the community colleges to systematically plan, acquire, maintain, and upgrade or replace technology infrastructure and equipment to meet institutional needs. To indicate that they have addressed the technology standards, each of the community colleges we reviewed referred to components of its technology master plan in the self‑evaluation it prepared as part of its most recent accreditation process and also 28 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 included various summaries of actions it takes to replace or upgrade its technology equipment. In our review of the most recent reports by the peer review team evaluating whether colleges met ACCJC standards for the three colleges we reviewed—American River in 2015, Cerritos in 2014, and De Anza in 2011—the evaluation reports did not have any recommendations related to meeting the technology standards we reviewed, which we described in Figure 1 on page 7. However, in its 2015 evaluation report of American River, the peer review evaluation team (evaluation team) made a recommendation to Los Rios, the district in which American River is located, that it develop a comprehensive technology plan for the district. Los Rios completed the district plan, and its board of trustees approved it in February 2017. The evaluation reports did not have any recommendations related to meeting the technology standards we reviewed. For network infrastructure equipment, such as servers, switches, and fiber lines (network equipment), all three districts and American River stated that they rely on their respective IT staff to determine the type of equipment to purchase, and on industry standards or manufacturer warranties to determine how often to replace technology equipment.3 Los Rios’ director of technical services acknowledged that the district and its colleges have had difficulty keeping up with replacing and upgrading network infrastructure. He noted that some of the reason is because of a lack of funding to support physical plant needs, equipment replacements, and personnel to perform the replacements. However, he stated that the district has shifted to new practices in which staff now rely on industry standards and manufacturer warranties to determine how often to replace equipment, as well as designating annual funding to support replacement and upgrade needs. The IT departments at Los Rios, American River, and Foothill–De Anza use tracking spreadsheets for network equipment that describe the location of the equipment on their campuses and when the item was last replaced or upgraded to inform decisions on when to replace or upgrade this equipment in alignment with industry standards and manufacturer warranties. Although Cerritos’ tracking spreadsheets for its network 3 The Cerritos and Foothill–De Anza districts are solely responsible for infrastructure replacement for their respective colleges. However, Los Rios and its four colleges—including American River, the college we reviewed—share in the planning and replacement responsibilities for infrastructure technology. CALIFORNIA STATE AUDITOR | Report 2017-102 29 December 2017 equipment is missing information related to the age of some of its equipment, the IT director indicated that IT staff rely on the equipment’s model number listed within the spreadsheet, which provides information on the equipment’s age. In addition to the tracking spreadsheets, the IT departments at the three districts and American River use network monitoring software to continuously monitor infrastructure equipment such as servers, switches, and uninterruptable power supplies. This software provides live feedback on the functioning of equipment to address outages or devices that need to be replaced. Although the three districts we reviewed and American River have some processes and tools they use for identifying equipment to replace or upgrade, they have not formalized these practices to ensure that they are performed consistently and will continue in the future. The three districts we reviewed and American River have not formalized practices to identify equipment to replace or upgrade to ensure that they are performed consistently and will continue in the future. Additionally, one of the districts we reviewed uses project management software to assist it in prioritizing projects and equipment needs that support those projects. Specifically, in 2015 Foothill–De Anza’s IT department created a project management tool that it uses to receive and track the status of technology project requests at its two colleges. Supervisors and other management at the two colleges submit project requests through the online tool, which requires them to identify the funding source for the project and to estimate the cost of the project and the time needed to complete it. Requesters must also specify the names of individuals responsible for approving their projects—which includes the dean or director of the requesting department and the vice president of finance. Next, the vice chancellor of technology reviews and approves the request and assigns it to the IT department staff to determine the project size. For medium and large projects, the IT department prioritizes projects and assigns them to staff within the department. The project management tool provides various reports that are available to anyone with access to the system, allowing them to see the status of all requested projects, thus providing greater transparency in the decision‑making process for technology projects. Projects also can be easily prioritized based on factors such as scope, cost, and status. 30 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Two of the districts have established computer hardware and software standards through their shared governance technology committees (technology committees). Cerritos and Foothill–De Anza each publish, on their respective district websites, the computer hardware and software standards for their college or colleges, which include specifications such as the technology equipment’s make, model, and associated cost. The IT staff use these technology equipment standards when making decisions to replace or upgrade computers or collegewide software, such as Microsoft Office. One of the responsibilities of the technology committees at Cerritos and Foothill–De Anza is to review IT equipment standards. The two districts’ technology committees fulfilled this responsibility by meeting periodically to review and update the specifications for the computer hardware and software each district will use in its replacement schedule. In contrast, Los Rios has not established technology equipment standards for its colleges because the district has assigned responsibility for upgrading or replacing computers to the individual colleges. As part of its 2017 district technology plan, Los Rios’ IT office stated that it will establish and publish equipment standards and replacement cycles for all district‑managed equipment with input from college IT departments and vice presidents of administration. Los Rios’ 2017 district technology plan identifies standard software, such as Microsoft Office, PeopleSoft, and the Canvas learning management system, which the district purchases for the colleges. As part of its 2017 district technology plan, Los Rios’ IT office stated that it will establish and publish equipment standards and replacement cycles for all district‑managed equipment with input from college IT departments and vice presidents of administration. American River’s IT supervisor stated that the IT department determines the computer standards for the college based on the institutional knowledge of the IT staff. Faculty and staff are presented with computer specifications at the time they are notified their computers are eligible for upgrade or replacement. Although end users are informed of the computer and software standards before receiving the equipment, the computer standards are not published on the department’s website and are not developed by the technology committee, which would provide additional opportunities for CALIFORNIA STATE AUDITOR | Report 2017-102 31 December 2017 stakeholder involvement. This is in contrast to the other districts we reviewed, which have their technology committees review and approve hardware and software standards for the district and colleges. The districts and American River have not established formal procedures to guide the implementation process when replacing or upgrading infrastructure technology equipment. Instead, their respective IT staff rely on their institutional knowledge and planning documents. We noted that Cerritos has formalized its process for replacement of computers, including the scheduling of faculty, staff, and lab computer upgrades and a description of the installation process, by publishing it on its website and in its technology master plan. However, the IT director stated that Cerritos has not established procedures to guide the implementation process for the other infrastructure equipment that it supports. Foothill–De Anza’s director of network and client services provided an internal document that includes a high‑level summary of its computer replacement process and indicated that IT staff follow this process when replacing or upgrading computers, printers, and multimedia equipment. However, this summary process does not include procedure steps for IT staff to follow to successfully deploy equipment during the installation process. Cerritos has not established procedures to guide the implementation process for the other infrastructure equipment that it supports. The districts and American River identified various reasons for not developing written procedures for the processes they use to replace or upgrade infrastructure equipment. The IT supervisor at American River stated that there was no requirement that it document procedures. However, he acknowledged that, were IT staff to leave their positions, the successors would find it difficult to compile all the information necessary to perform their jobs. He agreed that formal procedures would benefit the IT department to smooth any future transitions. Cerritos’ IT director said he relies on past practices but also recognizes the benefit of having documented procedures. The supervisors at Foothill–De Anza and Los Rios stated that IT staff rely on institutional knowledge and planning documents, such as inventories, developed in the course of an infrastructure upgrade project rather than documented procedures. However, we believe written procedures are important to ensure consistency and continuity in the replacement and upgrade practices. 32 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 When we discussed with the Chancellor’s Office whether it provides guidance to the community colleges on documenting replacement and upgrade processes, the director of the telecommunications and technology unit provided a document that the Chancellor’s Office gives to community colleges that apply to participate in or lead one of its statewide initiative IT projects, such as Canvas. The document specifies that system configuration and operating procedures are to be documented in a sufficient manner to allow a community college to install, configure, operate, and maintain the system and its functions. The director stated that, other than the document provided, the Chancellor’s Office does not provide guidance to community colleges related to upgrading or replacing technology equipment because of the decentralized nature of the system and the local control authority of the community college districts. However, providing guidance to all districts and community colleges on the importance of establishing written procedures for equipment replacement or upgrades would assist the districts and community colleges in ensuring that their technology replacement practices are consistent and continue in the future. Both the Office of Systems Integration and the California Department of Technology identify best practices for implementing technology equipment upgrades or replacements. The two departments’ implementation plan templates include sections describing necessary operational preparations, which include documenting the steps necessary to ensure the successful installation of new equipment. These sections include items such as identifying affected users who should receive notifications of when the upgrade or replacement will occur, associated training for new equipment or software, when the system will be offline for the new installation, and designated contacts for assistance in case of problems. Documenting the upgrade or replacement processes they use helps IT departments manage the expectations of those affected by the changes and also delineates responsibilities to help minimize the impact on end users. Formalizing these processes also demonstrates to the college community that IT staff performing the upgrade or replacement have a plan and are prepared to handle the disruption and any problems that arise. Two Districts We Reviewed Have Established Implementation Steps to Accomplish Their Technology Master Plans, but the Third District’s Plan Is Not Up to Date The three districts we reviewed—Los Rios, Cerritos, and Foothill–De Anza—have each developed a technology master plan that identifies their technology goals and planned CALIFORNIA STATE AUDITOR | Report 2017-102 33 December 2017 replacements and upgrades of infrastructure technology equipment at the colleges. These plans identify what technology projects are important, provide a framework for implementing those projects, and articulate how the technology plan can support the strategic plan of the district and its colleges. As described earlier, ACCJC standards require community colleges to systematically plan for, acquire, maintain, and update or replace technology infrastructure and equipment in order to meet institutional needs. However, these standards do not specify how to carry out this planning, and instead leave it to the discretion of the colleges. Although a technology master plan is not listed as a requirement within ACCJC standards, ACCJC evaluations of the community colleges refer to technology master plans as evidence that the evaluation teams review to assess whether the community colleges meet the standard that relates to technology planning. The three districts we reviewed each plan for infrastructure technology differently. Los Rios and its affiliated colleges share in the planning for infrastructure technology. The district plans for network infrastructure and the software required to support district and college operations, while each of its colleges is responsible for college‑specific infrastructure such as computers, servers, and wireless access points. In contrast, Foothill–De Anza has a more centralized approach—the district is responsible for all infrastructure technology planning for the colleges; the colleges do not have IT departments and instead rely on the district to upgrade or replace infrastructure technology equipment. Cerritos is a single‑college district, and the IT department is one and the same for the district and college. The three districts we reviewed each plan for infrastructure technology differently. Foothill–De Anza has a complete and up‑to‑date technology master plan, including detailed interim objectives to accomplish its three‑year goals. Specifically, its most recent master plan, approved in June 2017, describes the district’s three‑year goals and includes one‑year objectives to achieve those goals. For example, to help achieve its goal of districtwide infrastructure to support greater speed, reliability, and coverage, the district will improve throughput by upgrading Internet circuits from 1 gigabit per second (Gbps) to 10 Gbps during fiscal year 2017–18. The master plan also lists its two colleges’ goals and implementation steps from their technology plans and identifies where the district can provide support over the next three fiscal years. For instance, as part of De Anza’s teaching, learning, and student 34 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 engagement goal, the district identified in its master plan that it will provide direct assistance to the college in the organized provision of tablets and similar devices in the classrooms and programs. Although the Los Rios district has established detailed plans for implementing its technology master plan, American River—one of the colleges within the Los Rios district—lacks detailed steps to implement its master plan. Los Rios’ 2017 technology master plan includes technology‑related strategies for achieving the goals of the district’s strategic plan, as well as implementation steps and indicators of success. For example, in order to support the district’s strategic plan goal of establishing effective pathways that optimize student access and success, its technology master plan specifies that Los Rios’ IT office, with support from IT staff at the colleges, will complete site‑specific inventories of infrastructure equipment and develop a detailed annual replacement cycle to ensure that the average age of network switches is less than four years and that all production network equipment is supported by uninterruptable power supplies. American River has developed a technology master plan, but unlike the master plan of its district counterpart, American River’s 2014 through 2019 master plan lacks clear steps for implementing its high‑level objectives. According to American River’s dean of planning, research, and technology, the college’s master plan does not include implementation steps or a timeline to accomplish the technology goals because it is intended to be more information‑based. However, he stated that future master plans will include implementation steps that align with the goals identified in the plan. Cerritos does not have a current technology master plan—its most recent plan covered 2014 through 2015. The board of trustees approved a new six‑year educational master plan in May 2017. This plan articulates the college’s most important educational priorities and goals and establishes strategies to achieve those goals. The vice president of business services indicated that Cerritos will now begin creating a new technology master plan that aligns with the goals of the educational master plan. He stated that Cerritos intends to have an approved technology master plan by spring 2018. He also said there is no current documentation available to show any planning at this time, but that the college expects to contract with a vendor to assist it in developing the technology master plan. Although Cerritos’ existing technology master plan did not include action plans or measurable outcomes for listed technology goals, the vice president of business services stated that the new plan will incorporate implementation steps for the goals. He stated that the new technology master plan will work in conjunction with the college’s educational master plan, which includes steps to implement its goals. However, until Cerritos updates its technology master plan, the district is unable to ensure that technology‑related projects or the technology planning process aligns with its strategic plan. CALIFORNIA STATE AUDITOR | Report 2017-102 35 December 2017 Consistently Documenting Faculty and Staff Input in the Annual Review Process Will Increase Transparency Each of the three colleges we reviewed uses an annual review process to consider allocating resources for instructional technology equipment requests, such as technology equipment used in a classroom setting to teach students. Although each college told us it provides opportunities for faculty and staff to provide input when it is considering resource requests, none could demonstrate that they consistently receive and document such input. ACCJC standards require colleges to implement written policies that enable faculty, staff, administrators, and students to participate in decision‑making processes. The standards further state that faculty and administrators are to have a substantive and clearly defined role in institutional governance, and that students and staff are to have established mechanisms or organizations for providing input into institutional decisions. These standards apply to decisions about instructional technology equipment, since the resource requests are approved during a shared governance process at the colleges. When we discussed with the Chancellor’s Office the guidance it has provided to community colleges on shared governance, the dean of educational programs and professional development indicated that the Chancellor’s Office issued a legal advisory in 1997, which is the most current guidance. Specifically, this advisory addresses questions related to the role of faculty, staff, and students in shared governance. Because the application of shared governance has become part of the community colleges’ decision‑making processes, the dean stated that the Chancellor’s Office has not provided additional guidance since the legal advisory. However, the guidance in the advisory does not address documentation of input, attendees, or agreements reached at college governance or department meetings, including those to consider technology equipment requests. The guidance in the advisory does not address documentation of input, attendees, or agreements reached at college governance or department meetings. In the most recent ACCJC evaluations for American River and De Anza, conducted in October 2015 and October 2011, respectively, the evaluation teams’ reports indicate that the colleges 36 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 met the standards for decision‑making roles and processes. In Cerritos’ most recent evaluation in October 2014, the evaluation team’s report recommended that the members of the governing board demonstrate compliance with its policies related to the appropriate roles of the board of trustees and president in order to satisfy the leadership and governance standard. A follow‑up report issued to ACCJC in April 2015 based on a follow‑up visit by the evaluation team indicated that the college had met the leadership and governance standard. To ensure transparency in their budgeting processes, and to be sure that all equipment requests are appropriately vetted and approved, each of the three colleges we reviewed requires its respective departments to make resource requests for instructional technology equipment through an annual review process. Specifically, each of the colleges directs its departments to submit instructional equipment resource requests through the colleges’ planning software. The planning software prompts departments to evaluate the resource requests by addressing various elements such as the impact to student learning or how the resource aligns with the goals of the department. This information helps the colleges ensure that equipment requests support the departments’ requirements and match the needs of students enrolled in the departments’ programs. The instructional equipment requests submitted through the colleges’ annual review processes go through a series of reviews, ending with approval by the vice president or by the college president in conjunction with an appointed council. Figure 2 shows the annual review processes at American River, Cerritos, and De Anza. The figure also shows where the colleges identified faculty and staff as having opportunities to provide input during these processes. Despite indicating that they obtain faculty and staff input into their annual processes, none of the colleges could provide documentation to demonstrate that they do so consistently at the department level. The colleges indicated that their departments are encouraged to include faculty and staff when developing their annual plans, and they rely on department chairs to ensure faculty and staff participation. However, the colleges have not established a formal process for documenting the input, such as requiring instructional departments to record the individuals participating and to document meeting notes that describe the input received when developing their plans. CALIFORNIA STATE AUDITOR | Report 2017-102 37 December 2017 Figure 2 General Annual Review Process for American River, Cerritos, and De Anza American River Meets in open session to review budget proposals and allocations • Meets with from the annual review process. department chairs • i t M n h s a e t k i r r e u d c r e e ti p q o a u n r e a t s m l t t s e e f c n o h t r n c h ol a o ir g s y . to • • t r M F e o o q e r d e u w i t s e a s c s r u w t d s s s i s t t h a o t p h i t n p e h s r e r o t e r d v u q e e c u a d t e n o s . r t s s . • t t r v F e o h i o c q e r d e u w r i p s e e a c q s r r u e t d u s s s s e i s t d s a o t e p s t n p . h t r e . oved b c w M o a i a t n s h k e v e d e t s h r o s e a a n d d t i e e o c a n i n s s s io . n Budget • • a i w R A n l e p l i s o t v t p h r c i u r e a e o c w t x v t io e s i e o c n s b n u u d f a t o d i l i s v r e g t e q r e i u s b t t i u p a t ff m io . e n n t Committee of funds. i Department Instructors Chair Dean Pre V s i i c d e ent President i Cerritos • Holds division • Meets to review the technology meetings with their requests in open session. department chairs • Meets with • After review, forwards its list of to discuss academic deans to recommendations for consideration and prioritize review requests. by the president and executive council. • Make requests for • Meets with instructors the requests. • Forwards approved instructional technology to discuss the requests. Forwards approved requests to the • Make a final decision to their department chairs. • Forwards approved requests to the Planning and on which requests requests to the dean. vice president. Budget Committee. are approved. Planning and Department Instructors Chair Dean Vice Budget Executive Council i President Committee i and President De Anza • Prepares summaries of the • Meets to review the technology departments’ proposals. requests in open session. • Make requests for • Meets with instructors. Forwards them to the • After review, it forwards instructional technology Forwards the proposals Instructional Planning recommendations to the • Make a final decision to the department chairs. to the dean. and Budget Committee. college council and president. on which requests are approved. Instructors Dep C a h rt a m ir ent Dean I n P s l t a ru n c n t i i n o g n a l College Council and President i and Budget i i Committee Instructors may also discuss their requests directly with their dean. Sources: California State Auditor’s analysis of annual budget process documentation, meeting minutes, and interviews with staff at American River, Cerritos, and De Anza. i = Denotes that this step in the process provides an opportunity for faculty and staff to have input. 38 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 We reviewed agendas and minutes for meetings held during the process for the 2016–17 academic year from two departments at each college we reviewed and found that they showed inconsistent approaches to documenting faculty and staff input received. Specifically, American River’s music department prioritized its resource requests, including technology equipment requests, during its department meeting and generated an equipment list for the department chair to request. However, the department’s meeting minutes do not list who attended or what input they provided. Alternatively, while the biology department’s meeting agenda and notes do not list participants, the department generated an equipment wish list discussed during this meeting that identifies the faculty member who requested the item and in which course the equipment will be used. Similarly, although Cerritos’ machine tool technology department documented attendees and the specific equipment requests made, including technology discussed in its meeting minutes, its nursing department did not consistently maintain meeting minutes that included meeting participants and equipment requests. Further, the meeting minutes for De Anza’s automotive department identified discussion of technology equipment requests, but the minutes do not consistently list the participants or the equipment reviewed. The minutes for De Anza’s design and manufacturing technologies program meeting recorded both the meeting participants and the equipment requests made during discussions of technology equipment that occurred during the annual review process. Although the colleges did not consistently document faculty input that occurred at department meetings, we identified some instances within the annual review process that are transparent regarding input. For example, at each college, technology equipment requests are available to the campus community through the college’s planning software or its website. This gives faculty and staff who were unable to attend department meetings the opportunity to review equipment requests from their departments and other departments on campus. We observed that initial equipment requests submitted through the planning software require justification of the request’s merit, which includes how these materials would be used by faculty within the classroom. We also noted that the three colleges involve governance committees during the annual review process, and that the committees either review or make recommendations to approve the requests for instructional technology equipment. The membership of these committees includes representatives from the administration, faculty, staff, and students, and thus encompasses the stakeholder interests of the colleges. Additionally, the colleges include meeting times and locations on the committees’ CALIFORNIA STATE AUDITOR | Report 2017-102 39 December 2017 websites, which provides information for faculty, staff, and students who wish to attend. Each of these processes provides the colleges with opportunities for valuable input from their stakeholders. However, because the community colleges have not established procedures for instructional department staff to follow to consistently document the input received, the colleges cannot always demonstrate to stakeholders that their processes are transparent. Cerritos’ dean of institutional effectiveness, research, and planning acknowledged that the planning software could be updated to include departments uploading their meeting notes to show when the planning discussions took place and who was involved. De Anza previously discussed making modifications to its planning software to include the names of participants at the department level; however, rather than changing the software, the vice president of instruction stated that the college will be asking that all requests moving forward include the names of everyone involved. American River’s dean of planning, research, and technology indicated that the college is in the process of restructuring its annual review process to improve integration and alignment between planning and resource allocation processes, and will be discussing ways to achieve consistency regarding input across the departments. The Community Colleges We Reviewed Offer Various Opportunities for Faculty, Staff, and Students to Provide Input on Technology Training Needs American River, Cerritos, and De Anza colleges each have a training department dedicated to providing ongoing technology training for faculty and staff, and offer resource centers where students can obtain technology assistance. The colleges obtain feedback from faculty, staff, and students on training opportunities or technology services by conducting surveys or through discussions with training department or resource center staff. Faculty and staff have additional opportunities to provide input on technology training needs through the colleges’ shared governance committees. Staff from the training departments are members of the shared governance committees, allowing for information to be shared and incorporated at the training departments. ACCJC standards include standards related to technology resources, one of which is specific to training and requires the community colleges to provide quality training in the effective use of their IT to students and college personnel. This standard does not specifically require that community colleges obtain input from faculty, staff, and students regarding technology training. Nonetheless, the colleges we reviewed do have processes for 40 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 obtaining such input. In our review of the three colleges’ most recent ACCJC evaluation reports—American River in 2015, Cerritos in 2014, and De Anza in 2011—ACCJC did not have recommendations to these colleges for the standard related to technology training for students and college personnel. In their self‑evaluations, the colleges identified their technology training departments for faculty and staff, as well as their student tutoring centers, as examples to satisfy ACCJC requirements. The format of technology training is dependent on whether the training is collegewide or specific to a given department. For the three districts we reviewed, the colleges provide technology training for faculty, staff, and students. The college training departments offer faculty and staff workshops and drop‑in labs or refer them to alternative resources for technical assistance or training. The technology workshops offer training for faculty and staff in the use of tools used collegewide, such as Canvas, the colleges’ learning management system used for delivery of educational courses; Microsoft Office; and instructional applications for the iPad. The training departments at the three colleges provided lists that showed between 35 and 83 technology‑based training workshops offered each year for fiscal years 2014–15 through 2016–17. The three colleges also have student resource centers that provide academic assistance to students, including technology assistance. The centers are staffed with personnel who provide technology assistance and drop‑in labs related to technology instruction. In contrast to the collegewide training, department‑specific training—on hardware such as routing machines or design software like AutoCAD that is used for instructional purposes in the classroom—is individually based. The demand for this training is driven by individual instructors and students, and such training is generally provided on a case‑by‑case basis. The three colleges we reviewed each conduct surveys of faculty and staff to obtain feedback related to the workshops they offer in their training departments. The types of information requested in the surveys generally relate to the usefulness of the content and materials presented and how the content applies to the participants’ work. The surveys include comment sections for personalized responses, including ideas for other workshops or sessions to be offered. Based on the questions asked in the surveys, we believe that the information obtained would assist training department staff in identifying technology training needs. In addition to workshop surveys, two of the three colleges use additional surveys to obtain input from faculty and staff on future technology training. Specifically, American River provides faculty and staff with an end‑of‑semester survey, and De Anza CALIFORNIA STATE AUDITOR | Report 2017-102 41 December 2017 provides faculty and staff with a midsemester survey. De Anza also has an online evaluation form for technology workshops on its training department’s website. American River’s dean of distance education, Cerritos’ Center for Teaching Excellence coordinator, and De Anza’s associate vice president of instruction all indicated that training department personnel at their respective colleges discuss the survey results internally and consider the survey results when selecting future workshop topics. The three colleges we reviewed also offer opportunities for faculty and staff to identify collegewide technology training needs through the shared governance committees. Based on the questions asked in the surveys, we believe that the information obtained would assist training department staff in identifying technology training needs. Each of the colleges we reviewed has a student resource center that provides ongoing tutoring opportunities or computer labs where students can request technology support. The resource centers obtain feedback on the services they provide through student surveys. American River’s Learning Resource Center provides students with an open computer lab and tutoring opportunities and conducts surveys at the end of the semester to obtain information about how students are using these services. The Learning Resource Center coordinator stated that the survey results are reviewed and discussed by the program coordinators and staff to improve the services provided. De Anza offers tutoring in math, science, and technology at its Student Success Center. This center conducts surveys after tutoring sessions and has a form available on its website for students to provide suggestions. A co‑director for the Student Success Center stated that survey results from tutoring and responses received from the online form are discussed by the staff and used to influence workshop topic selection. Currently, Cerritos’ Student Success Center offers student tutoring and workshops focused on writing and math skills, in addition to offering technology assistance in the computer labs. The dean of academic success at Cerritos stated that the college will begin to offer technology‑based workshops at its Student Success Center following the implementation of its student survey in fall 2017. 42 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 District Budgets Do Not Identify the Components of Technology Equipment Replacement and Upgrade Projects Although district budgets include funding for IT staff, maintenance, and training for replacing and upgrading technology equipment, they do not separately identify those costs. Rather, the amounts are included as part of broader budget categories. Similarly, the districts do not use accounting codes to specifically track all expenditures related to replacing and upgrading technology equipment. Instead, their accounting codes allow them to track the costs of their IT staff and all technology equipment purchases, a practice that is consistent with the Chancellor’s Office’s Budget and Accounting Manual. We found that districts do consider these amounts, but perform this consideration as part of their annual budget planning processes. The districts we reviewed fund costs for IT infrastructure and instructional technology equipment through the processes described earlier in this report, in which staff identify costs and funding as they request technology equipment for instructional purposes or develop proposals for specific projects to replace or upgrade technology equipment. However, districts use their existing IT staff resources to implement routine upgrades or replacements, such as faculty and staff computers, which are upgraded or replaced on an ongoing basis—meaning that some portion of staff and faculty computers are upgraded each year. Therefore, districts do not separately budget for those staff hours. For example, when we asked why Cerritos has not included an analysis of staffing needs to implement technology equipment upgrades and replacements in its budgets, Cerritos’ IT director said he has adequate staff available to implement the upgrades or replacements. He added that when existing staffing resources are not adequate to implement an upgrade or replacement, he works with staff to provide flexible schedules or compensatory time off in return for additional hours of work. Further, he stated that he can also help on the implementation of projects because of his technical expertise. Los Rios noted that vendors sometimes provide training for certain IT upgrades, and that these amounts would be included both in the project’s contract and in the district’s budget for the upgrade. The total budget for IT programs varied by district, and the amounts budgeted by two districts have changed over the last three years in response to changes in the districts’ IT needs. Table 1 presents the total IT budget for each district reviewed, as well as the amounts for staffing, equipment, and repairs and maintenance. Two of the three districts significantly increased their spending on IT activities from fiscal years 2014–15 through 2016–17. Specifically, Los Rios stated that it recently increased its budget to upgrade CALIFORNIA STATE AUDITOR | Report 2017-102 43 December 2017 IT infrastructure, in particular its Wi‑Fi systems, using one‑time funding it received from the State. Cerritos told us that its recent budget increase for IT was a result of bond funds available from its bond construction program. Specifically, Cerritos issued bonds in 2004 and 2012 to construct several new buildings on campus, which increased its IT infrastructure needs as the buildings were completed. In contrast, budgets for Foothill–De Anza remained relatively constant. Foothill–De Anza’s vice chancellor of technology noted that equipment purchases and staffing levels have remained consistent over this period. Table 1 Budgets for IT by Community College District Reviewed Fiscal Years 2014–15 Through 2016–17 (In Thousands) COMMUNITY FISCAL REPAIR AND OTHER COLLEGE DISTRICT YEAR IT BUDGET EQUIPMENT MAINTENANCE STAFF IT* Cerritos 2014–15 $5,777 $1,707 $596 $3,104 $370 2015–16 7,845 3,745 596 3,094 410 2016–17 8,810 4,448 589 3,324 449 Foothill–De Anza 2014–15 $15,204 $4,338 $1,492 $8,898 $476 2015–16 14,291 3,521 1,601 8,280 889 2016–17 14,703 2,974 1,289 9,417 1,023 Los Rios 2014–15 $15,912 $4,065 $2,381 $7,593 $1,873 2015–16 23,732 10,030 2,354 8,118 3,230 2016–17 27,243 9,626 4,461 8,331 4,825 Sources: California State Auditor’s analysis of budget data provided by Cerritos, Foothill–De Anza, and Los Rios. * Other IT includes budgets for items such as rents, personal service contracts, and software licenses. Districts Use a Variety of Sources to Fund IT Programs The districts we reviewed fund their IT programs using a variety of sources. Although all three districts use their general fund revenue to support their IT expenditures, they have also needed to rely on other resources to fully fund their IT needs. For example, Foothill–De Anza funds its technology equipment costs in part with the proceeds from bond sales that voters approved in 2006. These bonds were funded through local property tax revenues and provided about 25 percent of the district’s total IT budget for the three fiscal years ending in 2016–17. The revenue from the bonds was intended to be used for several purposes, including to improve safety and disabled access; to repair, upgrade, and expand classroom space; and to upgrade technology. According to its vice chancellor of business services, Foothill–De Anza established a 15‑year plan to use the revenue from the bond sales, which he expects will be spent 44 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 by the 2021–22 academic year. Similarly, Cerritos has used proceeds from the sale of bonds approved by voters in 2004 and 2012 to fund 30 percent of its budget for IT expenditures for fiscal years 2014–15 through 2016–17. Cerritos has also used revenue from a senior housing development located on its property to fund IT needs for students, but this revenue stream represents just 3 percent of its IT funding. The third district—Los Rios—has used lottery revenue and cost savings from prior years to help fund its IT expenditures and, according to its vice chancellor for administration, used one‑time funding it received from the State in fiscal year 2015–16 to update its IT infrastructure. These funding sources represented an average of roughly 23 percent of Los Rios’ total IT budget for fiscal years 2014–15 through 2016–17. The Chancellor’s Office provides high‑level guidance to districts on how to budget and account for their colleges’ expenditures, but each district determines the funding sources to fulfill its technology needs. Additionally, the Chancellor’s Office establishes statewide IT infrastructure projects and obtains the funding for those projects. State law specifies that the Board of Governors is responsible for establishing, maintaining, revising, and updating the uniform budgeting and accounting structures and procedures for the community colleges. As part of this responsibility, the Chancellor’s Office developed and distributed its Budget and Accounting Manual. State law requires the community colleges to follow this manual, which guides the use of their accounting systems, including the uniform fund structure used to record the financial affairs of community college districts. The Chancellor’s Office does not offer any specific guidance to community college districts on how to fund their IT programs. When we asked the vice chancellor for college finance and facilities why it has not published any guidance in this area, he stated that the Chancellor’s Office focuses on providing general guidance in the area of finance, such as the Budget and Accounting Manual, and wants to provide districts with the flexibility to finance their local priorities. The Chancellor’s Office has a significant role in the annual statewide budget process and also works to implement systemwide technology projects with funding received through the State’s annual budget process. An example of the statewide infrastructure projects is the online education initiative, which the Chancellor’s Office reports was initially funded with a base amount in fiscal year 2013–14 and has received funds annually to increase student access and success in online courses. In April 2017, the Chancellor’s Office reported to a legislative budget subcommittee for education finance that this initiative includes several projects: a common course management system for colleges, resources to help faculty design high‑quality courses, online learner readiness modules, tutoring and counseling platforms, exam‑producing solutions, and CALIFORNIA STATE AUDITOR | Report 2017-102 45 December 2017 an online course exchange. According to the vice chancellor for finance and facilities, implementation of IT initiatives statewide may reduce those costs for districts. For example, instead of each district paying for and implementing its own online course management system, the districts can use the statewide version that the Chancellor’s Office implemented. Recommendations Chancellor’s Office To assist all community college districts and colleges in ensuring that they have consistent, transparent, and continuous implementation of their processes for upgrading and replacing IT equipment, by September 2018, the Chancellor’s Office should issue guidance to the districts and community colleges on establishing written procedures for those processes. To assist all community colleges in increasing transparency of their shared governance decision‑making processes, by September 2018, the Chancellor’s Office should issue guidance to the community colleges on establishing procedures to document the attendees, input received, and agreements reached during department meetings, including those to consider technology equipment requests. Districts and Community Colleges To ensure the consistent, transparent, and continuous implementation of processes for technology equipment upgrades and replacements, by June 2018, Cerritos, Los Rios, and Foothill–De Anza districts, and American River, should each establish written procedures for those processes. To ensure that it fully implements its technology master plan, by June 2018, American River should establish an implementation plan with detailed steps for achieving the goals in its technology master plan that it has not yet accomplished. Further, it should develop an implementation plan in conjunction with the development of its future technology master plan. To ensure that its technology master plan supports the strategic goals of the district, Cerritos should update its master plan by June 2018, and should ensure that the plan includes detailed steps to accomplish its goals. 46 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 To increase the transparency of their annual review processes, by June 2018, American River, Cerritos, and De Anza should each establish procedures requiring their departments to document attendees, input received, and agreements reached during meetings to consider instructional technology equipment requests. CALIFORNIA STATE AUDITOR | Report 2017-102 47 December 2017 SCOPE AND METHODOLOGY The Joint Legislative Audit Committee (Audit Committee) directed the California State Auditor to select three community college districts and review their technology equipment implementation and upgrade plans. Table 2 lists the objectives that the Audit Committee approved and summarizes the methods we used to address those objectives. Table 2 Audit Objectives and the Methods Used to Address Them AUDIT OBJECTIVE METHOD 1 Review and evaluate the laws, rules, Reviewed relevant federal and state laws and regulations. and regulations significant to the audit objectives. 2 For a selection of three community • Selected three districts based on location, student population, and number of colleges in the district. college districts, perform the following: Selected districts and colleges reviewed were: – Los Rios and American River – Foothill–De Anza and De Anza – Cerritos (single-college district) • Used information from the Chancellor’s Office to identify the number of students enrolled in the districts as part of our selection of districts to review. We used these data primarily as background information; as such, no data reliability assessment was necessary. a. Identify and evaluate current policies, • Interviewed district and college staff to understand their processes for replacing and upgrading procedures, and practices for replacing technology equipment and reviewed district policies to identify any related to technology equipment. and upgrading technology equipment. • Reviewed technology master plans and educational master plans. • Reviewed equipment replacement processes, including replacement schedules when available, and annual equipment review processes. b. Identify the stakeholders involved • Reviewed district policies to identify any related to input on decision making, including decisions on in the decision-making process at developing and implementing technology policies and equipment. the district and campus/educational • Interviewed key management staff at the districts and colleges to understand stakeholder center levels for developing and involvement in decision making related to replacing and upgrading technology equipment. implementing technology policies • Reviewed meeting agendas and minutes to identify the stakeholder involvement in decision-making and replacing and/or upgrading processes at the colleges. technology equipment. c. Determine whether there are current • Reviewed district policies to identify any related to input into decision making, including input requirements that allow input from regarding the need for new and ongoing training about technology. faculty, staff, and students when • Interviewed managers at the districts and colleges to understand stakeholder roles in this process. determining the need for new • Reviewed training offered related to technologies and educational technology instruction. and/or ongoing training about technologies and educational • Reviewed surveys that colleges conducted to obtain input on training needs related to technology. technology instruction. d. Identify and evaluate the practices • Where available, reviewed colleges’ responses to student requests for alternate formats of course districts/worksites use to ensure materials to determine if the districts complied with requirements to respond in a timely manner. compliance with federal and state • Reviewed college policies and procedures for assisting students with disabilities and for monitoring accessibility requirements related accessibility of instructional materials. to technology services and training • Reviewed college websites to determine whether they are accessible to individuals with for faculty and staff. Determine disabilities. Also, reviewed whether colleges have procedures for monitoring compliance with what challenges, if any, districts, website accessibility. campuses, and educational centers face in adhering to these • Interviewed college managers and reviewed documents related to college training for instructors on accessibility requirements. how to make their course materials accessible to students with disabilities. continued on next page . . . 48 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 AUDIT OBJECTIVE METHOD e. Determine the extent to which • Interviewed managers and reviewed budget documentation where available to understand each district budgets over the last district’s budget processes. three years include amounts for • Obtained budget reports to identify amounts for staff, maintenance, and training related to replacing staff, maintenance, and training and upgrading technology equipment. related to replacing and upgrading technology equipment. 3 Review and assess any other issues that We did not identify any additional issues that are significant to the audit. are significant to the audit. Sources: California State Auditor’s analysis of the Audit Committee’s audit request number 2017-102 as well as information and documentation identified in the column titled Method. CALIFORNIA STATE AUDITOR | Report 2017-102 49 December 2017 Assessment of Data Reliability In performing this audit, we relied on various electronic data files that we obtained from the entities listed in Table 3. The U.S. Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of computer‑processed information that we use to support findings, conclusions, or recommendations. Table 3 describes the analyses we conducted using the data from the information systems we used, our methods for testing them, and the results of our assessments. Although we recognize that these limitations may affect the precision of the numbers we present, there is sufficient evidence in total to support our audit findings, conclusions, and recommendations. Table 3 Methods Used to Assess Data Reliability INFORMATION SYSTEM PURPOSE METHOD AND RESULT CONCLUSION Cerritos and Los Rios: To identify IT budget amounts • We performed data-set verification procedures and Undetermined for the districts. logic testing of key data elements and did not identify reliability for the PeopleSoft Financials system any issues. purposes of this audit. for fiscal years 2014–15 through 2016–17 • To gain further confidence in the data we obtained, Although this we traced and materially agreed the totals to determination may published documentation, such as adopted budgets. affect the precision Foothill–De Anza: • We did not perform accuracy and completeness of the numbers we Ellucian Banner Financial testing of these data because of the number and present, there is software for fiscal variety of data systems associated with this audit, sufficient evidence years 2014–15 making such testing cost-prohibitive. in total to support our findings, through 2016–17 conclusions, and recommendations. American River: To review the length of time • We performed data-set verification procedures and Undetermined the college took to respond logic testing of key data elements and identified that reliability for the DSPS Alternate Media to requests from students over 12 percent of the records contained logical errors purposes of this audit. Requests Tracking with disabilities for alternative in the data. For example, we identified 47 instances in Although this Spreadsheet formats of course materials. which the date the student provided the proof of determination may (December 2015 through purchase came before the date requested. However, affect the precision January 2017) these errors do not significantly affect the precision and completeness and completeness of our analysis. of the time it takes • We did not perform accuracy and completeness the college to testing on these data because after the initial respond to requests, request for alternate media the system is paperless. there is sufficient Additionally, the data do not contain a unique evidence in total to identifier to connect the request data to the initial support our findings, request. We present the results of our analysis to conclusions, and provide context to the magnitude of the effect of recommendations. American River’s lack of procedures for monitoring the timeliness of completing alternate media requests. Sources: California State Auditor’s analysis of various documents, interviews, and data obtained from the community college districts we reviewed. 50 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 We conducted this audit under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives specified in the Scope and Methodology section of the report. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor Date: December 5, 2017 Staff: Tammy Lozano, CPA, CGFM, Audit Principal Richard D. Power, MBA, MPP Jim Adams, MPP Michael Henson Britani M. Keszler, MPA Legal Counsel: J. Christopher Dawson, Sr. Staff Counsel For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. CALIFORNIA STATE AUDITOR | Report 2017-102 51 December 2017 STATE OF CALIFORNIA ELOY ORTIZ OAKLEY, CHANCELLOR CALIFORNIA COMMUNITY COLLEGES CHANCELLOR'S OFFICE 1102 Q STREET, SUITE 4400 SACRAMENTO, CA 95811-6549 (916) 322-4005 http://www.cccco.edu November 9, 2017 Ms. Elaine Howle, State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 Dear Ms. Howle: The California Conummity Colleges Chancellor's Office (CCCCO) appreciates the opportuuity to review and comment on your report titled, "California Community Colleges: Districts and Colleges are not Adequately Monitoring Services for Technology Accessibility and Should Formalize Procedures for Upgrading Technology''. In general, your report recommends the CCCCO provide community colleges guidance to: • Strengthen the respond time to student's requests for instruction materials. • Expand the access of instructional materials to students with disabilities. • Improve the processes for upgrading and replacing information technology equipment. • Increase transparency of participatory governance decision-making processes, including those to consider technology equipment requests. The CCCCO is working to implement the seven recommendations by the established timeframes. We thank the State Auditor staff for its work and we embrace the opportunity to improve our leadership role in these important policy areas. If you have any questions, please contact Frances Parmelee at (916) 445-0540. a�---- Sincerely, Erik E. Skinner Deputy Chancellor cc: On the following page 52 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 CALIFORNIA STATE AUDITOR | Report 2017-102 53 December 2017 Cerritos College November 9, 2017 SENT VIA EMAIL (RickP@auditor.ca.gov) Elaine M. Howle, CPA* State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 Dear Ms.Howle: Enclosed you will find our responses to the recommendations identified from your report. We have provided our responses in PDF and Word format. If you have any further questions, please contact me at (562) 860-2451 ext. 2242. Sincerely, Felipe R Lopez, MBA Vice President of Business Services/ Assistant Superintendent Cerritos Community College District cc: Dr. Jose Fierro, President/Superintendent, Cerritos College Enclosures * California State Auditor’s comment appears on page 59. Cerritos Community College District 11110 Alondra Blvd., Norwalk, CA 90650 • 562.860.2451 • Fax 562.467.5005 • ww.cerritos.edu 54 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Recommendations Community Colleges To ensure that they are fulfilling requests for alternate media services from students with disabilities in a timely manner, by June 2018, Cerritos should each establish procedures for monitoring its timeliness in responding to such requests so that it can periodically review its performance in completingthe requests. Specifically, Cerritos should record and track sufficient information to be able to review how long they take to complete requests. Response to the recommendation 1 To the existing official DSPS Excel spreadsheet, titled “Alternate Media Tracking Spreadsheet” currently documenting the date student request was received and the date alternate media conversion was completed and material ready for student pick-up, a column will be added delineating: type of alternate format of the final production format (Braille, MP3, enlarged print, o PDF or other type needed for accessibility); length of class (9-week/other short-term, semester-length, summer intersession); and, o the number, mean, median, and mode of days elapsed between receipt of student o request and completion of student request, including for tiered requests wherein a student may, in effect, have two or more request dates for a class, time will be calculated as specified (number, mean, median, mode of elapsed days) . A Performance Review Team will meet at least each primary term and intersession to review the performance data. Based on the data, the team will identify improvements as needed. The Team will be comprised of the Senior Accessibility Compliance Specialist, a DSPS staff member handling alternate media production, a DSPS faculty member, and the Team convener and chair will be the Dean of Disabled Student Programs and Services. Recommendation To ensure that they promptly address any complaints they receive related to web accessibility and alternate media requests, by June 2018, Cerritos should each establish procedures for tracking and reviewing complaints received related to accessibility and addressing complaints in a timely fashion. Response to the recommendation 1 The college’s existing student conduct, grievance, concern, or Title IX report submission and case management SaaS, Maxient, will be expanded to include a category or categories for web accessibility-and alternate media request-related complaints. This will include (a) customized form(s), confidential electronic routing to appropriate personnel (Dean of DSPS for student concerns not against DSPS) or the Director, Diversity, Compliance, and Title IX Coordinator who will, as applicable, review, track, and manage or refer the matter to the responsible personnel (e.g., for web accessibility, to the Director, College Relations, Public Affairs, and Government Relations). Complaints related to personnel or confidential employee information will be routed to a separate databasemaintained by Human Resources. The rerouting of these complaints will be noted in the Maxient file. Maxient provides a robust complaint tracking CALIFORNIA STATE AUDITOR | Report 2017-102 55 December 2017 solution that will be employed and it supports effective, confidential review for ensuring timely addressing of complaints. Recommendation To ensure that students with disabilities have equal access to instructional materials, by June 2018, Cerritos should each develop procedures to monitor and periodically review the accessibility of instructional materials. For example, the college could develop an accessibility checklist for instructors to complete when developing or selecting instructional materials, from which the college could periodically review a sample of course content to ensure instructors completed the checklist and that the instructional materials comply with accessibility standards. Response to the recommendation Cerritos Collegewill work with faculty senate in order to develop an accessibility checklist for instructors to complete when developing or selecting instructional materials. This checklist will include: 1) Visual materials are accessible? Provide alternative text for images. Alternative text (or alt text) ensures that images are still accessible for people who are blind because their screen reader will read the alt text aloud for any images. 2) Audio materials accessible? Provide a text transcript for audio files. Text transcripts make audio information accessible to people who are deaf or hard of hearing. Closed captioning will allow learners to read the audio portion of videos. 3) Course software must allow for keyboard input? Allow all functionality via a keyboard. Providing the option for complete keyboard control gives learners who cannot use a mouse the opportunity to use assistive technologies that mimic the keyboard, such as speech input. 4) Does my authoring tool support accessibility? Choose an authoring tool that supports accessibility. This will make it easier for you, as a faculty, to make your course accessible to all your learners. 5) What feedback have I received? Incorporate user testing into your development process. Getting frequent feedback as you’re creating your course will allow you to fix any areas that aren’t accessible. Periodic random monitoring will be facilitated by the Universal Access Committee to ensure instructional materials comply with accessibility standards. Recommendation To ensure that all instructors are aware of the accessibility standards for instructional materials, Cerritos should each include in their next collective bargaining negotiations a requirement for instructors to periodically attend accessibility trainings. 56 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Response to the recommendation Cerritos Collegewill work collaboratively with the official representatives of the College’s constituent groups to provide new and current employees training on universal access. This includes, but is not limited to awareness of Board Policy and Administrative Procedure 3411, embedded training during the onboarding process and periodically offer universal access training for flex credit. Furthermore, Cerritos College will work with the Universal Access Task Force to make available tools, tips, tutorials, and guidelines to all employees to ensure that accessibility is considered at the time of adoption of instructional materials and purchase of information technology products. Recommendations Districts and Community Colleges To ensure the consistent, transparent, and continuous implementation of processes for technology equipment upgrades and replacements, by June 2018, Cerritos should each establish written procedures for these processes. Response to the recommendation Cerritos College will update and revise its current technology equipment replacement plan to include written procedures and expand the plan to includeequipment within the data center and classroom technology equipment. Recommendation To ensure that its technology master plan supports the strategic goals of the district,by June 2018, Cerritos should continue its efforts to update its master plan and should ensure that the plan includes detailed steps to accomplish its goals. Response to the recommendation Cerritos College has recently completed and approved its Educational Master Plan (2017-2023). The College is currently working on a request for qualifications (RFQ) in order to update both its Facilities Master Plan and Technology Master with the hopes of integrating both of these plans. Recommendation To increase transparency in their annual review processes, by June 2018, Cerritos should each establish procedures requiring their departments to document attendees, input received, and the agreements reaching during meetings to consider instructional technology equipment requests. Response to the recommendation In order to increase transparency in the annual planning/review process, Cerritos College will establish a process that will necessitate meeting minutes at both the department and division level where agreements were reached regarding instructional technology equipment. CALIFORNIA STATE AUDITOR | Report 2017-102 57 December 2017 The annual planning process is well defined starting with department chairs filling out annual Unit Plans. These plans are completed in Program Review Plus, locally developed software. Unit plans are developed by first reviewing program review evidence/data in order to identify Strengths, Weaknesses, Opportunities, or Threats (SWOT). After the department completes the SWOT analysis, the unit sets goals. Activities are then determined by the department to accomplish the goals, which may require resources for personnel, software, equipment, etc. Department meeting minutes will be provided to the division office documenting these discussions. The next several steps in the resource allocation process reinforce the importance of dialogue in decision-making processes at Cerritos College. Unit plans are submitted to the responsible administrator for the Division (Dean/ Director). This begins the development of the Division Plan. The division manager will review the Unit Plans submitted and build the Division plan from the contents submitted from the Units; and add Division needs identified through the program review process for non- instructional/administrative offices. Deans will be able to reference this discussion by documenting division level dialogue during their monthly division meeting. Division meeting minutes, along with department meeting minutes, will be available for review by the Vice President(s) of the area. 58 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2017-102 59 December 2017 Comment CALIFORNIA STATE AUDITOR’S COMMENT ON THE RESPONSE FROM CERRITOS COMMUNITY COLLEGE DISTRICT To provide clarity and perspective, we are commenting on the response to our audit from Cerritos. The number below corresponds to the number we have placed in the margin of Cerritos’ response. Although Cerritos indicates some actions it plans to take to 1 address our recommendation, its response does not specify whether it intends to establish the procedures we recommended. We look forward to reviewing the documentation it provides with its future responses that report on its progress in implementing this recommendation. 60 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2017-102 61 December 2017 12345 El Monte Road Los Altos Hills, CA 94022 November 9,2017 Elaine M. Howle, State Auditor* California State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 RE: California State Auditor’s Report No. 2017-102 Dear Ms. Howle, Attached is the response fromFoothill-De Anza Community College District (De Anza College), to the California State Auditor’s Report No. 2017-102 draft report. We sincerely appreciate the work of the CSA audit team members in their development of the audit findings and recommendations. Foothill-De Anza Community College District would like to thank the California State Auditor for the opportunity to respond to the draft report. Sincerely, Kevin McElroy Vice Chancellor, Business Services Foothill-De Anza Community College District 12345 El Monte Road Los Altos Hills, CA 94022 650-949-6201 Attachment Cc: Judy C. Miner, Chancellor, Foothill-De Anza Community College District Brian Murphy, President, De Anza College Joe Moreau, Vice Chancellor, Technology Lorrie Ranck, Associate Vice President, Instruction Marisa Spatafore, Associate Vice President, Communication and External Relations Susan Cheu, Vice President, Finance and College Operations Rob Mieso, Associate Vice President, Student Services Stacey Shears, Division Dean, Disabled Student Programs and Services (DSPS) * California State Auditor’s comments appear on page 67. 62 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Recommendations Community Colleges To ensure that they are fulfilling requests for alternative media services from students with disabilities in a timely manner, by June 2018, Redacted text. De Anza should each establish procedures for monitoring its timelines in responding to such requests so that it can periodically 1 review its performance in completing the requests. Specifically, REDACTED De Anza should record and track sufficient information to be able to review how long they take to complete 1 requests. Additionally, REDACTED REDACTED De Anza should each calculate the number of days that it takes to complete requests, and periodically evaluate its performance against its time frame goal. Further, to evaluate its performance, De Anza should establish a time frame goal for completing alternate media requests. Foothill-De Anza Community College District Response: Foothill-De Anza is in agreement with this recommendation and will take steps necessary to implement the recommendation by June 2018. To ensure that they promptly address any complaints they receive related to web accessibility and alternate media requests, by June 2018, REDACTED REDACTED REDACTED 1 REDACTED REDACTED REDACTED REDACTED REDACTED REDACTED REDACTED REDACTED REDACTED. Additionally, De Anza should follow its new procedures for tracking and reviewing complaints related to accessibility. Foothill-De Anza Community College District Response: Foothill-De Anza is in agreement with this recommendation and will take steps necessary to implement the recommendation by June 2018. CALIFORNIA STATE AUDITOR | Report 2017-102 63 December 2017 To ensure that students with disabilities have equal access to instructional materials, by June 2018, REDACTED De Anza should each develop procedures to monitor and periodically review 1 the accessibility of instructional materials. For example, the college could develop an accessibility checklist for instructors to complete when developing or settling instructional materials, from which the college could periodically review a sample of course content to ensure instructors completed the checklist and that the instructional materials comply with accessibility standards. Foothill-De Anza Community College District Response: De Anza College has an accessibility checklist in place for use in peer review of online courses. In response to this recommendation, we will revise this checklist for broader instructional use and work in collaboration with Academic Senate on a process to periodically review a sample of courses in various modalities for accessibility of instructional materials. To ensure that their websites comply with accessibility standards, by June 2018, REDACTED 1 De Anza should each develop procedures to monitor website accessibility and incorporate steps to prevent instructors from publishing inaccessible content on their respective websites. These procedures should include a tracking mechanism to demonstrate how many accessibility errors each college identifies and how long it takes to fix those errors. Foothill-De Anza Community College District Response: The new website, to be launched in early 2018 with the upgraded CMS, will obviate most of the website accessibilityconcerns. The collegewill require the approximately 300 decentralized users to ensure compliance before pages and edits are published. Additional training opportunities will also assist in preventing instructors from posting inaccessible material, as will encouraging instructors to fully utilize the Canvas LMS, as is being done by the Online Education Center. The new website search tool, Funnelback, can search for errors in binary documents such as PDFs. All other auditor concerns have already been addressed, as follows: 64 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 • The following policy statement regarding accessibility has been added to the Office of Communications website at http://www.deanza.edu/communications/web-accessibility: De Anza College serves a diverse community that values varied experiences and perspectives and strives to fully include everyone. De Anza College strives to ensure that people with disabilities have access to the same services and content available to people without disabilities, including services and content made available through the college’s website. The Office of Communications provides Accessibility Guidelines for website content providers collegewide. The college also ensures accessibility is a featured topic in trainings on the content management system (CMS). Accessibility experts are members of the college Technology Committee, for which accessibility is a standing agenda item. You may also read the Office of Communication's procedures for managing accessibility-related website improvements. If you have a concern or question regarding accessibility of De Anza College website content, please email the Web Team at webteam@deanza.edu. 2 • The above-mentioned Office of Communications' procedures for ensuring website accessibility are: 2 For decentralized CMS users -Ensuring, through the CMS settings, that no webpage can be published with accessibility errors (beginning withpublication of the redesigned website in early 2018). In addition, ensuring that accessibility is part of trainings on the CMS. In fulfilling standard Web Team work 2 -Prior to webpage publication, performing a check, using Tenon or a similar tool, and making any remaining changes. Monthly -Performing a monthly, automated accessibility scan on the website, recording results into a spreadsheet, evaluating the concerns, and providing and recording corrective action. Month-to- month results analyses are also performed. -Accessibility-related emails, phone calls or other contacts are recorded on a spreadsheet for action by the Web Team. Other -In working with vendors, ensuring that products meet accessibility standards. A spreadsheet to be maintained by the senior web coordinator now tracks errors noted by the multiple accessibility tools he uses, way(s) addressed, and speed of response. CALIFORNIA STATE AUDITOR | Report 2017-102 65 December 2017 To ensure that all instructors are aware of the accessibility standards for instructional materials, REDACTED De Anza, REDACTED should each include in their next collective bargaining 1 negotiations a requirement for instructors to periodically attend accessibility trainings Foothill-De Anza Community College District Response: Per the recommendation, the college will discuss required accessibility training for faculty during contractual negotiations. Recommendations District and Community Colleges To ensure the consistent, transparent, and continuous implementation of processes for technology equipment and replacements, by June 2018, REDACTED Foothill-De Anza 1 REDACTED REDACTED should each establish written procedures for these processes. Foothill-De Anza Community College District Response: Foothill-De Anza is in agreement with this recommendation and will take steps necessary to implement the recommendation by June 2018. To increase transparency in their annual review processes, by June 2018, REDACTED REDACTED De Anza should each establish procedures requiring their departments to document 1 attendees, input received, and the agreements reached during meetings to consider instructional technology requests. Foothill-De Anza Community College District Response: Foothill-De Anza is in agreement with this recommendation and will take steps necessary to implement the recommendation by June 2018. 66 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2017-102 67 December 2017 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM FOOTHILL–DE ANZA COMMUNITY COLLEGE DISTRICT To provide clarity and perspective, we are commenting on the response to our audit from Foothill–De Anza. The numbers below correspond to the numbers we have placed in the margin of Foothill–De Anza’s response. We provided Foothill–De Anza with a redacted draft report that 1 contained only those portions relevant to Foothill–De Anza. Foothill–De Anza’s response included the word “REDACTED” where text relating to the other entities we audited was redacted in its draft. We appreciate Foothill–De Anza’s outlining its planned procedures 2 for ensuring website accessibility for its decentralized content management system users and its standard web team work. We look forward to reviewing formal documentation that these procedures have been established and implemented in its future responses that report on its progress in implementing this recommendation. 68 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2017-102 69 December 2017 * * California State Auditor’s comments begin on page 73. 70 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Los Rios Community College District Agency Draft Responses Page 2 of 4 1 RESPONSES TO RECOMMENDATIONS ON PAGE 20 Recommendation No. 1: To ensure that they are fulfilling requests for alternative media services from students with disabilities in a timely manner, by June 2018, American River College should establish procedures for monitoring its timeliness in responding to such requests so that it can periodically review its performance in completing the requests. Additionally, American River College should calculate the number of days it takes to complete request, and periodically evaluate its performance against its time frame goal. Response No. 1: As noted in the report, American River College completed requests for alternate media within its stated goal of two weeks 95% of the time. The data required to make the recommended calculation is readily available and already collected. As a result, American 2 River College believes it has procedures for monitoring its timeliness and will engage in a periodic review of its performance against its goal and document process. Recommendation No. 2: To ensure that they promptly address any complaints they receive related to web accessibility and alternate media requests, by June 2018, American River College should establish procedures for tracking and reviewing complaints received related to accessibility and addressing complaints in a timely fashion. Response No. 2: The Los Rios Community College District identified the need to focus on, and began work on, assuring the accessibility of electronic information technology across the District and its four colleges prior to the Audit. American River College previously added a link on each page of its website that asks if the user has any accessibility issues. While American 3 River College has not received a single complaint about the accessibility of its website through that link, it remains ready, willing, and able to timely respond to any complaint to ensure access to the website materials. American River College will write down this procedure to satisfy this recommendation. Recommendation No. 3: To ensure that students with disabilities have equal access to instructional materials, by June 2018, American River College should develop procedures to monitor and periodically review the accessibility of instructional materials. For example, the College could develop an accessibility checklist for instructors to complete when developing or selecting instructional materials, from which the College could periodically review a sample of course content to ensure instructors completed the checklist and that the instructional materials comply with accessibility standards. Response No. 3: The District will undertake the creation of business practices to determine how American River College will monitor and periodically review the accessibility of instructional materials. Recommendation No. 4: To ensure that their websites comply with accessibility standards, by June 2018, American River College should develop procedures to monitor website accessibility This document is to be kept confidential in accordance with Government Code, sections 8545(b) and 8545.1 CALIFORNIA STATE AUDITOR | Report 2017-102 71 December 2017 Los Rios Community College District Agency Draft Responses Page 3 of 4 and incorporate steps to prevent instructors from publishing inaccessible content on their respective websites. These procedures should include a tracking mechanism to demonstrate how many accessibility errors each college identifies and how long it takes to fix those errors. Response No. 4: American River College currently has software that tracks the accessibility of its website and identifies issues that need correcting and is also in the process of redesigning the website with the goal of addressing these issues. The College will document its practices as suggested by the recommendation. Recommendation No. 5: To ensure all instructors are aware of the accessibility standards for instructional materials, American River College should include in its next collective bargaining negotiations a requirement for instructors to periodically attend accessibility trainings. Response No. 5: American River College objects to this recommendation and is not inclined to 4 follow it. First, the method by which colleges engage in instruction is a matter almost exclusively in the purview of institutions of higher education, like American River College, not the Auditor’s office. There are numerous ways of getting this information to the faculty short of requiring forced attendance at mandatory accessibility trainings. For instance, information campaigns can provide this information to all faculty. Furthermore, collective bargaining is an area that is in the purview of the College and the District and the District should not be placed at a disadvantage in the collective bargaining process based on an unfunded mandate of the Auditor’s office that it must include a particular item in its collective bargaining negotiations. RESPONSES TO RECOMMENDATIONS ON PAGE 40 1 Recommendation No. 6: To ensure the consistent, transparent, and continuous implementation of processes for technology equipment upgrades and replacements, by June 2018, Los Rios district, and American River, should each establish written procedures for these processes. Response No. 6: We agree that having guidelines for standard equipment specifications that correlate with certain activities is helpful as well as documenting the communication protocol used by the IT department. Recommendation No. 7: To ensure that it fully implements its technology master plan, by June 2018, American River should establish an implementation plan with detailed steps for achieving the goals in its technology master plan that it has not yet accomplish. Further, it should develop an implementation plan in conjunction with the development of its future technology master plan. Response No. 7: American River College agrees that establishing detailed steps for implementing the goals within both its current and future Technology Master Plans would assist the college in achieving these goals. By June 2019, The College will develop an This document is to be kept confidential in accordance with Government Code, sections 8545(b) and 8545.1 72 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 Los Rios Community College District Agency Draft Responses Page 4 of 4 implementation plan/schedule for its current Technology Master Plan as well as incorporate said process into future technology master planning. Recommendation No. 8: To increase transparency in their annual review processes, by June 2018, American River, should each establish procedures requiring their departments to document attendees, input received, and the agreements reached during meetings to consider instructional technology equipment requests. Response No. 8: The process faculty use in documenting conversations occurring in department meetings is generally left to each department to determine. Technology needs are just one aspect of what is discussed and reviewed in the development of program and unit plans, let alone the many other items that are the subject of discussion in department meetings so culling just one aspect of those discussions may be viewed as intrusive. In regard to equipment needs, some departments may choose not to attribute a specific request to an individual because the recommendation forwarded is the department’s based upon the consensus of those who 5 participated and/or reviewed. However, the college will work to develop a process whereby the Chair and the Dean are able to verify that all faculty were provided the opportunity for input. This document is to be kept confidential in accordance with Government Code, sections 8545(b) and 8545.1 CALIFORNIA STATE AUDITOR | Report 2017-102 73 December 2017 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM LOS RIOS COMMUNITY COLLEGE DISTRICT To provide clarity and perspective, we are commenting on the response to our audit from Los Rios. The numbers below correspond to the numbers we have placed in the margin of Los Rios’ response. We provided Los Rios with a redacted draft report that contained 1 only those portions relevant to Los Rios. Therefore, the page numbers Los Rios cites in its response do not correspond to the page numbers in our final report. Los Rios states that American River believes it has procedures for 2 monitoring its timeliness and will engage in a periodic review of its performance. On page 15, we acknowledged American River’s plans to have procedures and a reporting tool implemented by October 2017, which was subsequent to the end of our field work. Therefore, we look forward to reviewing the procedures and reporting tool when Los Rios submits its 60‑day response in February 2018. Unfortunately, we cannot verify American River’s claim that it has 3 not received a single complaint about the accessibility of its website. As we state on page 21, American River does not have a process for tracking accessibility complaints submitted by website users or for documenting their resolution. Without a process for tracking and reviewing the resolution of accessibility complaints submitted through its website, American River cannot demonstrate that it is prepared to promptly address and monitor complaints related to website accessibility. As we state on page 23, although American River offers training in 4 implementing accessible materials as a resource to instructors, it has not required all instructors to take this training. American River’s dean of planning, research, and technology noted that such a requirement would be a collective bargaining issue. We recognize that these negotiations are within the purview of the college and the district. However, as we also state on page 23, without requiring faculty to attend accessibility training, colleges cannot ensure that faculty are aware of their responsibility to comply with accessibility requirements for the instructional materials they use. Moreover, our recommendation does not address the method by which American River engages in instruction, only the accessibility of the materials used, which is a legally mandated requirement. 74 Report 2017-102 | CALIFORNIA STATE AUDITOR December 2017 5 Although Los Rios states that American River will work to develop a process whereby the chair and the dean are able to verify that all faculty were provided the opportunity for input, its response is unclear how it will address the lack of consistent transparency in its annual review process. As we state on page 39, because the community colleges—including American River—have not established procedures for instructional department staff to follow to consistently document the input received, the colleges cannot always demonstrate to stakeholders that their processes are transparent. We look forward to its future responses to better understand American River’s new process for the chair and dean to verify that all faculty were provided the opportunity for input and how this increases the transparency of the input received.