Recommendations

June 2015 California State Government Websites Departments Must Improve Website Accessibility So That Persons With Disabilities Have Comparable Access to State Services Online Report 2014-131 COMMITMENT INTEGRITY LEADERSHIP The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the California State Auditor’s Office at the following address: California State Auditor 621 Capitol Mall, Suite 1200 Sacramento, California 95814 916.445.0255 or TTY 916.445.0033 OR This report is also available on our website at www.auditor.ca.gov in PDF and HTML formats. Alternate format reports available upon request. The California State Auditor is pleased to announce the availability of an online subscription service. For information on how to subscribe, visit our website at www.auditor.ca.gov. Permission is granted to reproduce reports. For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. For complaints of state employee misconduct, contact the California State Auditor’s Whistleblower Hotline: 1.800.952.5665. Elaine M. Howle State Auditor Doug Cordiner Chief Deputy June 2, 2015 2014-131 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the California State Auditor presents this audit report concerning the State’s compliance with federal and state web accessibility standards. Since January 2003, state law has required state governmental entities to comply with federal requirements that mandate that persons with disabilities have access to and use of information and data that is comparable to the access and use by those without disabilities. In addition to adopting the federal standards in 2003, California added new standards for accessible websites in June 2006. In 2013, the Public Policy Institute of California reported that 47 percent of Californians said they use the Internet to access government resources. In addition, data on the California state government home page showed that in January 2015 there were millions of unique visits to California government websites. Despite the growing use of government services online and the State’s accessibility requirements, the departments’ websites we reviewed during this audit are not fully accessible to persons with disabilities. Although the violations of web accessibility standards ranged in severity, some of them are so severe that elements of the departments’ websites were completely inaccessible to users with disabilities while other violations may prevent persons with disabilities from completing tasks necessary to access certain online services. For example, we found that, at the time of our review, persons with disabilities who navigate the Internet using a keyboard would not be able to start an online application for health insurance using Covered California’s website. Most of the departments we reviewed did not regularly test updates to their websites to ensure that the websites were accessible after the updates. Of the four departments we reviewed, only Franchise Tax Board conducted regular accessibility testing on updates to its website. In some cases when departments did not thoroughly test updates to their websites for accessibility, we found critical accessibility errors on updated portions of the departments’ websites. Because of the violations of accessibility standards on the websites of the departments we reviewed and the lack of regular accessibility testing at most of those departments, we believe that California would benefit from requiring web accessibility training for staff involved in the procurement or development of websites and from naming the California Department of Technology (CalTech) as the department responsible for overseeing governmental entities’ efforts to test their websites for accessibility. Additionally, we believe CalTech should provide direction to state departments that specifies the method by which departments should conduct web accessibility testing. Increased and standardized web accessibility testing in combination with required training would increase the likelihood that state websites would be accessible to persons with disabilities who attempt to access critical services and information through those websites. Finally, we recommend that the Legislature amend state law to require all state websites to comply with updated standards that could help make California government websites more accessible. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 916.445.0255 916.327.0019 fax www.auditor.ca.gov Blank page inserted for reproduction purposes only. California State Auditor Report 2014-131 v June 2015 Contents Summary 1 Introduction 7 Chapter 1 Department Websites Do Not Fully Comply With State Accessibility Standards 19 Recommendations 34 Chapter 2 Departments Have Not Consistently Tested for Accessibility, and Statewide Web Accessibility Training Should Be Required 37 Recommendations 55 Responses to the Audit California Community Colleges 59 California State Auditor’s Comment on the Response From California Community Colleges 61 California Government Operations Agency, California Department of Human Resources 63 California Government Operations Agency, California Department of Technology 67 California State Auditor’s Comments on the Response From the California Department of Technology 71 Covered California 73 California State Auditor’s Comments on the Response From Covered California 77 California Government Operations Agency, State of California Franchise Tax Board 79 California State Auditor’s Comments on the Response From the State of California Franchise Tax Board 83 vi California State Auditor Report 2014-131 June 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-131 1 June 2015 Summary Results in Brief Audit Highlights . . . A significant number of Californians use the Internet to obtain Our review of the accessibility of key online information about and avail themselves of government services. services offered by four departments— According to a 2013 report by the Public Policy Institute of California Community Colleges (Community California, 47 percent of Californians report they use the Internet Colleges), California Department of Human to access government services, an increase of 4 percentage Resources (CalHR), Covered California, and points from 2008. Also, data featured on the California state State of California Franchise Tax Board government home page showed that in January 2015 there were (Franchise Tax Board)—highlighted the millions of unique visits to California government websites such following issues: as the California Department of Motor Vehicles and Employment » Despite the growing use of government Development Department sites. services online and the State’s accessibility requirements, the California To ensure access to online government services for persons with websites we reviewed are not fully disabilities, California has adopted standards that address the accessible to persons with disabilities. needs of users who may have one or more of a range of disabilities, including those with visual impairments, hearing impairments, » Some of the accessibility violations are so and impairments to mobility. Since January 2003 state law has severe that, under certain circumstances, required California websites to meet requirements stemming they may prevent persons with disabilities from Section 508 of the federal Rehabilitation Act of 1973, as from accessing online services. amended (Section 508). Subsequently, in July 2006, California added the World Wide Web Consortium’s (W3C) Web Content • A CalHR online job exam did not Accessibility Guidelines (WCAG) version 1.0 as additional state appropriately label the exam questions— web accessibility standards for departments that report to the persons using screen readers would governor and the state chief information officer. have had to leave the exam question response area and use other methods to Despite the growing use of government services online and the determine what information to enter. State’s accessibility requirements, the California government • At Franchise Tax Board, users who websites we reviewed during this audit are not fully accessible. cannot use a computer mouse We reviewed one key online service offered by each of four cannot register an online account for departments—California Community Colleges (Community submitting tax returns. Colleges), California Department of Human Resources (CalHR), Covered California, and State of California Franchise Tax Board • At Community Colleges, screen reader (Franchise Tax Board)—to determine whether their online services users were not alerted that they would and the associated portions of their websites complied with the not be able to register for an account State’s accessibility requirements.1 We found violations of applicable if they took too long to complete accessibility standards on each department’s website.2 Among the individual registration pages. four departments we reviewed, Covered California’s website has • At Covered California, users who the largest number of violations of web accessibility standards. cannot use a computer mouse could Although the violations we found ranged in severity, many of not start an application for health them are critical, meaning that they are severe enough to make insurance because the button on the elements of the websites inaccessible to persons with disabilities. Covered California website that begins the application process could not be 1 Throughout this report, we refer to each of these state government entities as departments. accessed using only the keyboard. 2 For the purposes of this report, we use the term violation to mean noncompliance with an accessibility standard a department’s website was expected to conform to because of state law continued on next page . . . or a grant or contract agreement. 2 California State Auditor Report 2014-131 June 2015 » At the start of our audit, most For example, one such critical violation was on a CalHR online job departments did not provide information exam. Users who employ screen reader software—software that on their websites to the public about reads web pages aloud to those who cannot read text—should be how to complain about website able to have exam questions read to them as they navigate through accessibility problems. the exam. However, because CalHR did not appropriately label the exam questions, persons using screen readers would have had to » Three of the four departments we leave the exam question response area and use other methods, such reviewed could improve the type and as browsing the surrounding text on the web page line by line with frequency of accessibility testing that the keyboard’s arrow keys, to try to determine what information they perform on updates to their they should enter. online services. » Best practice guidance suggests that Some of the critical accessibility violations we identified are so departments provide specific training on severe that, under certain circumstances, they may prevent persons web accessibility to staff involved in the with disabilities from completing the core tasks necessary to access development of websites and web‑based online services. These violations occurred at three departments and services; however, there is no statewide would affect users with a variety of disabilities, including motor requirement for such training. disabilities and vision loss. A violation on Franchise Tax Board’s website prevents users who cannot use a computer mouse from » Although updated standards are registering an online account—a required step in submitting tax available that could help California make returns online through the department’s CalFile service. In addition, its websites more accessible, the State has a violation on Community Colleges’ online application for college not adopted them. prevented screen reader users who were applying to college online from registering for an account if they took too long to complete individual registration pages. This occurred because the notification that alerted users that their time to complete the page was almost expired was not programmed correctly, meaning that the screen reader software did not identify the notification when it appeared. Because these users were not made aware of this notification box, they were unable to request additional time to complete the page and would be logged off without warning at the expiration of the allotted time. The user would then need to start the account creation process over again from the beginning. Community Colleges was made aware of this issue through a complaint from a user in February 2015 and has now resolved the problem. Finally, at Covered California, users who cannot use a computer mouse could not start an application for health insurance because the button on the Covered California website that begins the application process could not be accessed using only the keyboard. Further, updated standards are available that could help California make its websites more accessible. In 2008, shortly after California adopted the first version of the WCAG standards, WCAG 1.0, the W3C issued WCAG 2.0. When it did so, the W3C stated that the WCAG 2.0 standards apply more broadly to different types of web technologies and allow for more effective testing of websites’ accessibility. However, California has not adopted these updated standards. In light of these improvements, it is important that the Legislature amend state law to require departments to meet the WCAG 2.0 standards. Further, it is important for the California State Auditor Report 2014-131 3 June 2015 California Department of Technology (CalTech) to monitor commonly accepted accessibility standards going forward to help ensure that California’s standards do not again become outdated in the future. Most of the departments we reviewed could improve the type and frequency of accessibility testing that they perform on updates to their online services. Three of the four departments we reviewed— Community Colleges, Covered California, and CalHR—conducted some accessibility testing on their websites before initially releasing the sites to the public. However, best practice guidance suggests that departments test updates to their websites as well, using both automated and manual techniques. Only Franchise Tax Board performed update tests on its website updates in this manner. At the start of our audit, most of the departments we reviewed did not provide important information on their websites to the public about how to complain about website accessibility problems. State policy requires departments to provide users who may want to complain about website accessibility with multiple forms of contact information, such as an email address, phone number, and mailing address. However, at the start of our audit Franchise Tax Board had failed to publish an email address on its website. Since then, it has added the required information. Although Covered California is not required to comply with policies issued by the state chief information officer, who is the director of CalTech, and it is unclear whether Community Colleges must comply with those policies, we assessed the complaint contact information on both departments’ websites against the requirements that other departments must follow. At the time of our review, neither department provided on its website all of the complaint contact information that state policy requires other departments to provide. When departments do not provide multiple forms of contact information, the risk increases that users will be unable to complain about web accessibility problems they may encounter so that departments can fix those issues. Similar to the reported level of complaints received by federal departments, the departments we reviewed appear to receive very few complaints about the accessibility of their websites. In 2010 and 2011 the U.S. Department of Justice surveyed 89 federal agencies about their compliance with Section 508. In their responses to the survey, those agencies reported that, from June 2001 through 2010, they had received a total of 140 administrative complaints about their web accessibility, or only about 15 complaints per year among all of the agencies. Similarly, we identified no web accessibility complaints received at Covered California since June 2013, no complaints at Franchise Tax Board since 2010, and only five complaints Community Colleges received since November 2012. Most of the complaints we reviewed at Community Colleges related to problems that screen reader 4 California State Auditor Report 2014-131 June 2015 users had with the online application to college. However, it is possible that the low level of complaints we observed is partly related to the lack of complaint contact information provided by some departments. CalHR did not keep adequate records of its accessibility testing or the web accessibility complaints it received. The department’s web manager and the supervisor over development for the Careers in California Government website (jobs site) acknowledged that CalHR does not currently have a level of information about accessibility testing that would allow it to know what portions of the site had been tested, what issues were found, and whether and when those issues were resolved. Additionally, the department lacks a tracking method for the web accessibility complaints it received. CalHR’s web manager stated that only a few accessibility complaints have been made and that he generally tries to assist users in navigating the site if they are experiencing problems. However, there is no procedure in place to ensure that if he discovers a problem with the website while assisting a user, the error will be documented and resolved. Although best practice guidance suggests that departments provide specific training on web accessibility to staff involved in the procurement or development of websites and web-based services, there is no statewide requirement for web accessibility training. At the four departments we reviewed, staff had inconsistent levels of training. We believe that statewide training on web accessibility would benefit all California departments. As the lead agency in California for matters related to information technology, CalTech could provide this training in consultation with other departments, such as the California Department of Rehabilitation. Further, because the departments we reviewed were not consistent in their approach to web accessibility testing, we believe it is important for the Legislature to direct all state government entities to report to CalTech about their web accessibility testing approach. CalTech would then assess each entity’s approach, determine whether it is adequate, and publish the results of that assessment online. Recommendations Legislature The Legislature should amend state law to do the following: • Require that all state websites comply with WCAG 2.0 standards in addition to the Section 508 standards. • Require CalTech to apprise the Legislature of any changes to those standards that California should adopt. California State Auditor Report 2014-131 5 June 2015 • Name CalTech as the lead agency responsible for providing training to state government entities on web accessibility issues. The Legislature should direct all state government entities to report biennially to CalTech regarding the frequency and method of their web accessibility testing. Further, the Legislature should direct CalTech to assess the sufficiency of each government entity’s testing approach and publicize the results of its review online. Departments Each of the four departments should correct the accessibility violations we found during our review. Covered California, CalHR, and Community Colleges should develop and follow written accessibility test approaches that include both manual and automated testing of updates to their websites. Community Colleges and Covered California should update their websites to include all methods of communication described in state policy for someone wishing to report a problem about website accessibility. CalHR should develop tracking tools that will allow it to document its accessibility testing efforts. At a minimum, these tools should track what portions of its jobs site were tested, what errors were found, and whether and when those errors were addressed. CalHR should develop procedures for addressing complaints about the accessibility of its website and methods for tracking the complaints it receives and their resolution. Agency Comments The four departments at which we performed detailed audit work— Community Colleges, CalHR, Covered California, and Franchise Tax Board—all generally agreed with our recommendations to address the problems we found. However, Franchise Tax Board disagreed with the portrayal of our audit results. Further, CalTech outlined actions it plans to take in response to the recommendations we directed to it. 6 California State Auditor Report 2014-131 June 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-131 7 June 2015 Introduction Background Government websites are a popular means by which members of the public obtain information and avail themselves of government services. In a 2010 report on the use of online government services and information, the Pew Research Center stated that in 2009 nearly half of American Internet users searched online to identify services provided by a government agency. Specific to California, the Public Policy Institute of California reported in 2013 that 47 percent of Californians said they use the Internet to access government resources, an increase of 4 percent from 2008. Finally, data featured on the California state government home page showed that in January 2015 there were millions of unique visits to California government websites, such as those for the California Department of Motor Vehicles and the Employment Development Department. However, not all persons interact with and experience the web in the same way. To ensure that persons with disabilities can access web content in a manner comparable to those without disabilities, the federal government and a nongovernmental organization have developed web accessibility standards that address the needs of users who may have one or more of a range of disabilities. For example, the standards encompass the needs of users with visual impairments, hearing impairments, and physical disabilities. These impairments affect many Californians. The results of the 2013 American Community Survey, an ongoing survey conducted by the U.S. Census Bureau, indicate that approximately 4 million Californians live with a disability. Further, the American Foundation for the Blind reported that in 2013 about 789,000 Californians suffered from vision loss. The National Institutes of Health estimates that 17 percent of American adults report some form of hearing loss, which translates to almost 5 million Californians. California’s Requirements for Accessible Websites Since January 2003 state law has required all state governmental entities to comply with Section 508 of the federal Rehabilitation Act of 1973, as amended (Section 508), and its implementing regulations. Section 508 regulations mandate that electronic and information technology (EIT) that federal agencies develop, procure, maintain, or use must be accessible to persons with disabilities and requires that persons with disabilities have access to and use of data that is comparable to the access and use by those without disabilities. Federal regulations establish the specific technical requirements (Section 508 standards) that EIT must meet in order to ensure that it is accessible to persons with disabilities. 8 California State Auditor Report 2014-131 June 2015 Section 508 standards are similar to, but distinct from, the physical accessibility standards established by the federal Americans with Disabilities Act of 1990, as amended (ADA). Along with other, more general, requirements for overall accessibility, ADA regulations require all public agencies to communicate as effectively with people with disabilities as they do with others; Section 508 standards focus specifically on the accessibility of particular categories of electronic products, such as software, websites, telecommunications and multimedia products, and some physical products, such as stand-alone terminals. This audit is focused on the compliance of web-based products developed or procured by four state departments with Section 508 and related state web accessibility standards, which we refer to jointly as state web accessibility standards. In addition to adopting the Section 508 standards in 2003, California further added new standards for accessible websites in June 2006. To address a goal of making government services more accessible to citizens, the former state chief information officer convened the California State Portal Steering Committee (steering committee) in 2005 to guide the development of a new state web portal.3 As part of that effort, the steering committee created the Information Organization, Usability, Currency, and Accessibility Working Group (IOUCA) in 2006 to develop policy and best-practice recommendations to ensure that state websites are accessible, usable, and understandable. In its recommendations, the IOUCA advised that California should go beyond the Section 508 standards and also adopt accessibility standards developed by the World Wide Web Consortium (W3C), an international nongovernmental organization. These standards are known as the Web Content Accessibility Guidelines (WCAG). In July 2006 the steering committee adopted the IOUCA recommendations and made the first version of the WCAG standards—WCAG 1.0—a requirement for California departments and agencies reporting to the governor and the state chief information officer. Figure 1 shows the timing of the release of various web accessibility standards as well as California’s adoption of some of those standards. The Section 508 standards were issued by the Architectural and Transportation Barriers Compliance Board (Access Board), a federal agency that, according to its website, promotes equality for people with disabilities through leadership in accessible design and the development of accessibility guidelines and standards for the built environment as well as for information technology. When it issued the Section 508 standards, the Access Board indicated that 3 A web portal is a website that is or proposes to be a major starting site for users. In this case, the web portal being developed was the website located at www.ca.gov. California State Auditor Report 2014-131 9 June 2015 some of the standards could be met by complying with specific WCAG 1.0 standards. However, the Access Board determined that some WCAG 1.0 standards would be difficult to enforce or were not clearly defined and it, therefore, did not adopt equivalent federal standards. Nevertheless, in recommending that California adopt the WCAG 1.0 standards, the IOUCA determined that the WCAG 1.0 guidelines were important to provide a greater level of usability for the State’s citizens. When the W3C later published the WCAG 2.0 standards, it stated that it designed the updated guidelines to build on the WCAG 1.0 standards, apply to future technology, and be testable with a combination of automated testing and human evaluation. Additionally, the WCAG 2.0 standards encompass Section 508 standards that WCAG 1.0 did not previously address. Table 1 on the following page provides examples of some accessibility standards to demonstrate the relationship between the Section 508 standards and WCAG 1.0 and 2.0 standards. Figure 1 Timeline of the Adoption of Website Accessibility Standards by the Federal Government and California The World Wide Web Consortium (W3C) publishes the first version of the Web Content Accessibility Guidelines (WCAG 1.0). Section 508 standards become enforceable. California adopts the The W3C publishes WCAG 2.0, Section 508 standards. replacing WCAG 1.0. 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 The California Department of Technology issues a policy letter that reminds departments about the The Federal Rehabilitation Act is amended, and 2006 recommendation, reinforcing that California Section 508 of the act mandates that standards websites must meet WCAG 1.0 standards.† establishing technical and functional criteria for comparable access to information and data be created (Section 508 standards). A California working group recommends that state websites meet the WCAG 1.0 standards in addition to the Section 508 standards.* The recommendation is adopted by the State Portal Steering Committee and becomes policy for state departments and agencies reporting to the governor and the state chief information officer. Sources: 1998 amendment to the Federal Rehabilitation Act of 1973; California Statutes of 2002, Chapter 1102; California Office of the State Chief Information Officer, Information Technology Policy Letter 10-10; Information Organization, Usability, Currency, and Accessibility Working Group, Recommendation on Accessibility Standards for California State Web Pages; W3C guidelines and website. * The Information Organization, Usability, Currency, and Accessibility Working Group. † At the time the policy letter was written, the department was known as the Office of the State Chief Information Officer. Later it was reestablished as the California Technology Agency, and in 2013 it became the California Department of Technology. California’s approach to implementing web accessibility standards generally mirrors the federal approach. At the federal level, individual federal departments and agencies are responsible for ensuring that their websites meet accessibility requirements. State law also requires individual state government entities to comply 10 California State Auditor Report 2014-131 June 2015 with applicable accessibility standards and does not identify a department specifically responsible for statewide oversight or enforcement of the accessibility standards. A separate state law assigns general responsibility for information technology oversight and enforcement of most state departments to the California Department of Technology (CalTech). However, this statute does not expressly state that CalTech is responsible for ensuring that all state-owned EIT is accessible to persons with disabilities. Nevertheless, despite not having explicit responsibility for ensuring Section 508 compliance at state departments, CalTech published a policy letter in July 2010 that reinforced the IOUCA recommendations for accessible websites and has devoted a chapter of the Statewide Information Management Manual (SIMM) to information technology accessibility, highlighting the legal requirements for state governmental entities as well as resources that departments can access to help ensure compliance.4 Table 1 Comparison of Selected Accessibility Standards and Guidelines WEB CONTENT ACCESSIBILITY GUIDELINES (WCAG) 1.0 SECTION 508 STANDARDS WCAG 2.0 (NONGOVERNMENTAL (FEDERAL STANDARDS ISSUED (NONGOVERNMENTAL STANDARD STANDARDS ISSUED IN 1999) IN 2000, EFFECTIVE 2001) STANDARDS ISSUED IN 2008) Websites use the clearest and simplest language appropriate  5* † Users may skip certain repetitive material 5   Web pages have titles that describe their topic or purpose 5 5  Sources: California State Auditor’s analysis of the federal Architectural and Transportation Barriers Compliance Board’s (Access Board) Preamble to the Section 508 Standards; World Wide Web Consortium’s Web Accessibility Initiative and WCAG 1.0 and 2.0; and information presented at the 2009 Annual International Technology and Persons with Disabilities Conference.  = Yes 5 = No * The Access Board determined that although this WCAG 1.0 standard is a worthwhile guideline, it is difficult to enforce because requiring the simplest language can be very subjective. † WCAG 2.0 specifically guides web content developers to define words or phrases used in an unusual or restricted way, including jargon, and establishes the lower secondary education level as the maximum reading level required for users. Federal regulations and best-practice guidance documents go beyond technical accessibility standards for websites to provide direction regarding the procurement of accessible EIT and how departments should address complaints that EIT is inaccessible. Federal regulations require departments that procure EIT products to conduct market research and, as part of that research, to evaluate the availability of products that meet federal accessibility standards. 4 At the time the policy letter was written, the department was known as the Office of the Chief Information Officer. Later it was reestablished as the California Technology Agency and in 2013 it became CalTech. California State Auditor Report 2014-131 11 June 2015 Federal and state guidance also recommends that departments test EIT products for compliance during development and as part of ongoing maintenance. Further, the SIMM states that departments should have processes in place for collecting and addressing complaints about inaccessible EIT. Finally, at both the federal and state levels there is guidance concerning responsibilities departments should meet to ensure the accessibility of their EIT. How Persons With Disabilities Use the Web People with disabilities access and navigate the web in different ways, depending on their individual needs and preferences. Some common approaches for interacting with the web include assistive technologies and adaptive strategies. Assistive technologies are software or hardware that people with disabilities use to improve interaction with the web. These include screen readers that read aloud web pages for people who cannot read text, screen magnifiers for people with some types of low vision, and voice recognition software and selection switches for people who cannot use a keyboard or mouse. Adaptive strategies are techniques that people with disabilities use to improve interaction with the web, such as increasing text size, reducing mouse speed, or turning on captions. Figure 2 on the following page displays techniques persons with disabilities may use to interact with websites and also shows elements of a website that, if properly coded, will assist persons with certain disabilities in navigating and using that website.5 According to the W3C, several different components of web development and web interaction must work together in order for web content to be accessible to persons with disabilities. These components include the content of a website, such as the text, the images, and the code that defines the structure of the site; the web browsers and media players that users employ; assistive technologies; and the knowledge and experience of the user. In addition, the W3C highlights the importance of the website developers and the tools they use to author content and validate that the content they create is accessible.6 5 Most of the content of this paragraph was derived from a copyrighted draft report issued by the W3C’s Web Accessibility Initiative in 2012, found at www.w3.org/WAI/intro/people-use-web/Overview.html. 6 Most of the content of this paragraph was derived from a copyrighted document issued by the W3C’s Web Accessibility Initiative in 2005, found at www.w3.org/WAI/intro/components.php. 12 California State Auditor Report 2014-131 June 2015 Figure 2 How Persons With Disabilities Generally Use the Web Sources: California State Auditor’s analysis of reports issued by the World Wide Web Consortium, Web Accessibility Initiative; Web Content Accessibility Guidelines 1.0; Section 1194 of Title 36 of the Code of Federal Regulations; the Web Style Guide, 3rd Edition; and an interview with our web accessibility consultant (consultant). Note: Our consultant did not perform an accessibility analysis of the State of California website shown above; it is being used for demonstration purposes only. Departments and Web Products Selected for This Audit Our review included four state departments and web-based products that they procured or developed. Table 2 shows the departments and the associated web products that we selected for review. To define a web product at each department, we identified a service that each California State Auditor Report 2014-131 13 June 2015 department offers online and reviewed department web pages that support those services. Further, at each department we reviewed the compliance of additional web pages on which accessibility-related information was located when they were present at the time of our review. Specifically, we reviewed each department’s page describing how to submit a complaint related to website accessibility. As a result, our review did not include a comprehensive examination of any department’s entire web presence. Table 2 Profiles of Web-Based Services We Reviewed KEY COMPONENT OF THE WAS THE KEY COMPONENT PROCURED YEAR DEPARTMENT WEB-BASED SERVICE WEB-BASED SERVICE* OR DEVELOPED IN-HOUSE? IMPLEMENTED California Community Colleges Apply to college OpenCCCApply Procured from vendor† 2012 (Community Colleges) California Department of Human Establish eligibility Careers in California Developed in-house 2012 Resources (CalHR) for state employment Government Portal Covered California Apply for California Healthcare Eligibility, Procured from vendor† 2013 health insurance Enrollment, and Retention System State of California Franchise Tax File state tax return CalFile Developed in-house 2003 Board (Franchise Tax Board) Source: California State Auditor’s analysis of websites operated by Community Colleges, Covered California, CalHR, and Franchise Tax Board; procurement information obtained from Community Colleges and Covered California; and interviews with staff at all four departments. * In addition to reviewing the compliance of the key component associated with the web-based service each department offers, we reviewed the compliance of additional web pages at each department. These included other pages that were in support of the web-based service and the department’s accessibility information page. † According to officials at these departments, the key component was procured from a vendor but was not a commercial off-the-shelf product. Instead, vendors built custom products for these departments. Some departments developed the product we selected for review using their internal staff, while others, specifically Covered California and California Community Colleges (Community Colleges), procured their web-based products from outside vendors. Community Colleges’ Chancellor’s Office (Chancellor’s Office) has an agreement with one of its college districts, the Butte-Glenn Community College District (Butte), wherein Butte receives grant funding from the Chancellor’s Office to operate the California Community Colleges Technology Center (technology center). Through the technology center, Butte procured the OpenCCCApply website from a third-party vendor. According to the assistant project director for Covered California’s California Healthcare Eligibility, Enrollment, and Retention System project, Covered California procured the product from an outside vendor that still maintains the system for Covered California. Conversely, according to a supervisor of the Internet and taxpayer folder section, the State of California Franchise Tax Board developed its online tax filing system—CalFile—internally using staff developers. Finally, the supervisor in the web development and 14 California State Auditor Report 2014-131 June 2015 support unit at the California Department of Human Resources (CalHR) stated that staff who built the Careers in California Government web portal and related applications were CalHR staff or former employees at the California State Personnel Board. The site and the staff responsible for it were transferred to CalHR in the governor’s reorganization in July 2012. Scope and Methodology The Joint Legislative Audit Committee (audit committee) directed the California State Auditor to review the State’s compliance with applicable laws, regulations, policies, and best practices for making EIT accessible to persons with disabilities. Table 3 lists the objectives that the audit committee approved and the methods used to address those objectives. As already described, our fieldwork concerned four state departments and web products developed or procured by those departments. Table 3 Audit Objectives and the Methods Used to Address Them AUDIT OBJECTIVE METHOD 1 Review and evaluate the laws, rules, Reviewed relevant laws, regulations, and other background materials applicable to web accessibility. and regulations significant to the audit objectives. 2 Determine the roles and responsibilities • Reviewed state laws and policies to identify existing roles and responsibilities for statewide within state government for ensuring Section 508 compliance. compliance with electronic and • Interviewed staff at the California Department of Technology (CalTech) to determine its role in information technology (EIT) accessibility statewide compliance. requirements by determining the • At each of the four departments we reviewed under Objective 5, identified key individuals roles and responsibilities that a responsible for ensuring Section 508 compliance at the department and interviewed them regarding selection of departments have for their responsibilities. ensuring compliance with Section 508 standards and related laws, regulations, policies, and best practices when EIT is developed or procured. 3 Review and assess the effectiveness • Identified best practices at the federal and state levels for providing accessible web-based EIT and of any guidance, policies, or protocols compared those best practices to state policy and guidance. We identified no best practices at the the State uses to ensure, monitor, local level. and enforce compliance with EIT • Determined that state guidance was generally consistent with the best practices that we identified. accessibility requirements. Determine • Determined whether the accessibility standards included in state policy were the most up-to-date whether such guidance, policies, standards issued by the World Wide Web Consortium (W3C). or protocols are consistent with best practices, if any, at the federal, state, or • Interviewed staff at each department we reviewed under Objective 5 to identify the sources of local level for providing accessible EIT. web accessibility guidance they use. 4 Determine whether the State provides • Interviewed staff at each department we reviewed under Objective 5 and reviewed training personnel involved in procuring or documents to determine whether department staff received training related to Section 508. developing EIT adequate training and • Interviewed staff at CalTech and the California Department of Rehabilitation to determine whether whether the State has a process for they provide web accessibility training and, if so, identified the type of accessibility training. investigating complaints related to • Determined that the State has no centralized entity responsible for addressing web accessibility noncompliance with Section 508. complaints and reviewed the complaint process for each department under Objective 5(f). California State Auditor Report 2014-131 15 June 2015 AUDIT OBJECTIVE METHOD 5 Identify EIT products (web pages and Identified key services offered online by state departments and considered the web traffic on related web-based services) administered by a websites and the size of the department offering each service to choose four services for review. selection of state agencies and select We selected applications for community college, applications for health care insurance, application a number of EIT products to review. and exam processes for state employment, and filing state income tax returns as our key web-based To the extent possible, the products services. The corresponding departments were California Community Colleges (Community Colleges), selected should include both EIT Covered California, the California Department of Human Resources (CalHR), and State of California developed, hosted, and/or administered Franchise Tax Board (Franchise Tax Board), respectively. For each online service, we identified the web by third-party contractors as well as product that was most closely associated with the service and reviewed that product as described EIT developed and/or hosted by state under subobjectives (a) through (f). agencies; EIT developed or procured within the last two years as well as those from prior years; and EIT offered by both large and small state agencies. For each of the EIT products selected, determine whether: a. The product complies with • Retained a web accessibility consultant (consultant), Deque Systems, Inc., to determine whether Section 508 standards and any each web product selected under Objective 5 complied with key criteria and to assess the severity other relevant state guidance, such of errors against those criteria. as the State Administrative Manual • Covered California is not subject to the policies issued by the state chief information officer and as or the Statewide Information such would not be required to follow Web Content Accessibility Guidelines (WCAG) 1.0 as were Management Manual, or related other departments we reviewed. We assessed Covered California’s California Healthcare Eligibility, laws, regulations, policies, and Enrollment, and Retention System (CalHEERS) against Section 508 standards and WCAG 2.0. This is best practices. because Covered California’s contract with the vendor that developed and maintains CalHEERS required that the vendor provide a product that complied with accessibility standards issued by the W3C’s Web Accessibility Initiative. At the time Covered California entered into this agreement, the most up-to-date version of those standards was WCAG 2.0. • Asked the consultant to review key pages in support of each web product, such as the accessibility page at all departments that featured such a page. • At every department except Covered California, our consultant conducted its entire review on the live version of each department’s website that is available to the public. At Covered California, this was not possible because the live version of the application for health insurance includes a step that uses personal information to verify the applicant’s identity. As a result, the consultant conducted the review primarily in a test version of the application that, according to Covered California, contains the most up-to-date version of the department’s website. • Our consultant conducted its review using the following web browsers and assistive technologies: the most recent versions of the Mozilla Firefox web browser and the NonVisual Desktop Access screen reader available at the time of testing, the consultant’s proprietary testing software, and supplemental testing with the Job Access With Speech screen reader versions 15 and 16 and the VoiceOver feature of the Apple Safari web browser. • Interviewed staff at each department to obtain their perspective on the results of our consultant’s review. b. The product’s accessibility • Reviewed the accessibility options discussed on each department’s website. options are clearly identified • Interviewed staff at departments that did not list accessibility options to determine why there and readily available for persons were no options listed on those websites. with disabilities. • Conducted research to determine which web browsers are the most popular to determine which web browser–specific information state departments should refer users who are seeking additional information about accessibility options. c. The administering entity complied • Interviewed department staff and reviewed relevant documentation to determine whether the with appropriate procurement web products we selected for review were procured. Determined that only Covered California and laws and processes applicable Community Colleges procured their web products. to Section 508 standards if it • Reviewed solicitation and contract documents for procured web products to determine whether contracted with a vendor to the departments complied with procurement laws and best practices applicable to Section 508 develop or procure the EIT product. standards, including whether the documents incorporated specific Section 508 requirements for In addition, determine whether the web product. the contracts incorporate specific • Determined that neither Covered California nor Community Colleges included specific Section 508 Section 508 requirements. requirements in their final contracts. Further, neither department conducted market research related to accessibility. However, we concluded that the effect of these omissions is likely minimal because each department included language in its final contract that requires the vendor to provide a product that complies with Section 508 standards. continued on next page . . . 16 California State Auditor Report 2014-131 June 2015 AUDIT OBJECTIVE METHOD d. The administering entity • Interviewed staff at each department and reviewed documentation to determine how frequently appropriately tested the EIT the departments tested the products selected under Objective 5 for Section 508 compliance product while the product was during the products’ development and maintenance. Also determined what methods each being developed, as it was department used to test its product. implemented, and throughout • Compared each department’s testing approach to best-practice guidance for testing web products’ the life of the product to ensure compliance with web accessibility standards and requirements. compliance with Section 508 standards and requirements. e. The administering entity properly • At Community Colleges, Covered California, and Franchise Tax Board, we used data from oversees and/or enforces the databases described in the “Methods Used to Assess Data Reliability” table to select compliance with the accessibility accessibility-related defects for review. We selected those items for review using keywords related requirements and ensures to web accessibility because none of these departments tracked accessibility-related defects corrective action is taken quickly separately from the other defects in their databases. We reviewed between 20 and 30 issues at when noncompliance is noted. each of these departments, depending upon the results of our keyword search. • Determined when the department discovered the defect, whether the defect had been fixed, and if so, how long the department took to fix those defects. • Interviewed staff at these three departments about defects that were not fixed and those that were not fixed before key releases of their web product. • At CalHR, the quality of record keeping regarding the department’s testing efforts was not complete enough to allow us to reliably review the results of the department’s testing and remediation of issues found during testing. As a result, we did not review a selection of defects found during testing at this department. f. The administering entities of the • Interviewed department staff and reviewed any written policies to determine the department’s EIT products selected have a processes for collecting accessibility complaints from users of its website and web product, process in place to receive and investigating, and addressing those complaints. investigate complaints of alleged • At Community Colleges and Covered California, we used data from the databases described in the barriers to access, and evaluate the “Methods Used to Assess Data Reliability” table to search complaint records at each department timeliness in investigating such using keywords related to web accessibility. We searched email records available at Franchise Tax complaints and the effectiveness Board using the same keywords. We identified complaints, if any, related to web accessibility. of the process. • At CalHR, there was no reliable record of complaints received by the department because the department did not consistently track these complaints. • At Covered California and Franchise Tax Board, our search of complaint records yielded no results for web accessibility complaints. At CalHR we identified isolated complaints in the web manager’s email records. • At Community Colleges, we reviewed the complaints we identified and determined whether the department addressed the complaints and how quickly it did so. 6 Review and assess any other issues Reviewed the September 2012 report issued by the U.S. Department of Justice regarding Section 508 that are significant to the audit. compliance to learn about best practices for Section 508 compliance. Sources: California State Auditor’s analysis of Joint Legislative Audit Committee audit request 2014-131, and information and documentation identified in the table column titled Method. Assessment of Data Reliability In performing this audit, we relied on various electronic data files that we obtained from the entities listed in Table 4. The U.S. Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of computer-processed information that we use to support our findings, conclusions, or recommendations. Table 4 describes the analyses we conducted using data from these systems, our methodology for testing them, and the limitations we identified in the data. Although California State Auditor Report 2014-131 17 June 2015 we recognize that these limitations may impact the precision of the numbers we present, in total there is sufficient evidence to support our audit findings, conclusions, and recommendations. Table 4 Methods Used to Assess Data Reliability INFORMATION SYSTEM PURPOSE METHOD AND RESULT CONCLUSION Covered California We did not perform accuracy Undetermined reliability and completeness testing for the purposes of this Application Lifecycle • To identify a selection of web accessibility of these data because audit. Although this Management system defects Covered California identified while the systems are paperless determination may testing CalHEERS during the development Data related to defects in the systems and hard-copy affect the precision of and maintenance of the product through California Healthcare Eligibility, source documentation the numbers we present, December 16, 2014. Enrollment, and Retention was not available for there is sufficient System (CalHEERS) and the • To determine whether Covered California review. Alternatively, evidence in total to remediation of those defects resolved those defects and, if so, the average following U.S. Government support our audit number of days it took to do so. Accountability Office findings, conclusions, • To identify a selection of accessibility guidelines, we could have and recommendations. complaints made by CalHEERS users from reviewed the adequacy of June 2013 through December 15, 2014. selected system controls that include general and Remedy IT Service To identify a selection of accessibility complaints application controls. Management Suite system made by CalHEERS users from June 2013 through However, we did not conduct December 15, 2014. Data related to tracking and these reviews because this resolving reported incidents, audit involved five such problems, and change requests paperless systems across for CalHEERS three departments and to do so for each would have been California Community Colleges cost-prohibitive. (Community Colleges) • To identify a selection of web accessibility JIRA system defects Community Colleges identified while At the time of our review, testing its online application service during none of these systems Data related to tracking the development and maintenance of this service specifically tracked development process for up to March 6, 2015. accessibility defects or the OpenCCC applications complaints. We therefore • To determine whether Community Colleges searched for keywords resolved those defects and, if so, the average associated with accessibility number of days it took to do so. to identify items for our ZenDesk system • To identify a selection of accessibility analysis. As a result, it is complaints made by online application users possible that defects or Data related to user requests since the release of the OpenCCCApply website complaints exist in those for assistance in using the in November 2012 through February 5, 2015. systems that we did OpenCCC applications • To determine whether Community Colleges not identify. addressed those complaints. State of California Franchise Tax Board (Franchise Tax Board) ClearQuest system • To identify a selection of web accessibility defects Franchise Tax Board identified from the Data related to enhancing, time it began testing the CalFile application for changing, and maintaining the accessibility in 2012 through December 2013. CalFile application • To determine whether Franchise Tax Board resolved those defects and, if so, the average number of days it took to do so. Source: California State Auditor’s analysis of various documents, interviews, and data from the entities listed above. Note: For each data system, the time frame of the data we analyzed was determined by the availability of data in that system related to accessibility, up to the date we completed our review. 18 California State Auditor Report 2014-131 June 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-131 19 June 2015 Chapter 1 DEPARTMENT WEBSITES DO NOT FULLY COMPLY WITH STATE ACCESSIBILITY STANDARDS Key Points » The four department websites we reviewed are not fully accessible under applicable web accessibility standards. Many of the accessibility violations we identified are of critical severity, meaning that elements of these websites were completely inaccessible to users with disabilities.7 » Because of some critical accessibility violations that occur under certain conditions, users with disabilities are not able to complete core online tasks that three departments offer. For example, a violation on the State of California Franchise Tax Board’s (Franchise Tax Board) website prevents users who cannot use a mouse from creating an online account—a required step in submitting tax returns online. » California’s web accessibility standards are out of date and do not reflect current best practices for ensuring accessibility for persons with disabilities. The most up-to-date standards are designed to be consistent with future technologies. Thus, we believe that it is important for the Legislature to amend state law to require compliance with more up-to-date standards issued by the World Wide Web Consortium (W3C). Department Websites Are Not Fully Accessible Under State Mandated Standards We evaluated the websites of four departments to determine whether the sites comply with web accessibility standards designed to ensure that persons with disabilities can access and navigate those sites. As discussed in the Introduction, California requires that state governmental entities’ websites comply with Section 508 of the federal Rehabilitation Act of 1973, as amended (Section 508) and that agencies reporting to the governor and state chief information officer must comply with the Web Content Accessibility Guidelines (WCAG) version 1.0.8 To evaluate the accessibility of 7 For the purposes of this report, we use the term violation to mean noncompliance with an accessibility standard a department’s website was expected to conform to because of state law or a grant or contract agreement. 8 The WCAG 1.0 standards are individual testable checkpoints that organizations can use to review their websites for accessibility. 20 California State Auditor Report 2014-131 June 2015 each department’s website, we identified a specific service each department offers through its website and reviewed the key pages of the department’s website that support that service against the relevant criteria for each department.9 In performing this review, our web accessibility consultant (consultant) identified violations of accessibility standards that occurred on individual web pages, as well as common accessibility violations—violations that occurred in similar or identical ways across multiple pages. One example of a common violation is an accessibility issue located in a website’s header or footer, which would be present on many web pages throughout the site. To avoid double-counting common violations, we counted each of them as a single violation affecting a single page, although each of these violations represents issues occurring across multiple pages. Table 5 shows the total number of pages we reviewed at each department as well as the number of those pages on which our consultant found individual and common violations. All four department websites we reviewed have violations of accessibility standards that make it difficult for users with disabilities to successfully navigate the sites, but the violations vary in how severely they affect users. Our consultant assigned a severity level to each violation based on What a Violation’s Severity Means for its effect on a user’s ability to interact with the Individuals With Disabilities specific element of the web page that contained the violations. The text box defines the different Critical: Makes underlying content completely inaccessible to users. levels of severity. A violation of critical severity is one that makes the underlying content completely Serious: Results in serious barriers for users, making some inaccessible to persons with disabilities. One content inaccessible. example of a critical violation affecting persons Moderate: Results in some barriers to users, but does not with disabilities who navigate the Internet by prevent them from accessing fundamental content. keyboard alone would be if a website did not provide visual cues to those users indicating Source: Our web accessibility consultant’s definitions of the severity levels assigned to each accessibility violation that where on a web page they were navigating. Such a testing identified on departments’ websites. violation would make it prohibitively difficult for these users to navigate a website. By comparison, a moderate violation, such as a violation in which a decorative image is improperly programmed for screen readers, creates barriers for individuals with disabilities but does not prevent 9 Covered California is not subject to the policies issued by the state chief information officer, who is the director of the California Department of Technology (CalTech), and as such would not be required to follow WCAG 1.0. We assessed Covered California against Section 508 standards and a more up-to-date version of the WCAG standards that we discuss in the Introduction—WCAG 2.0. This is because Covered California’s contract with the vendor that developed and maintains the California Healthcare Eligibility, Enrollment, and Retention System required that the vendor provide a product that complied with accessibility standards issued by the W3C. At the time Covered California entered into this agreement, the most up-to-date version of those standards was WCAG 2.0. Additionally, it is unclear whether California Community Colleges (Community Colleges) is required to follow policies issued by the state chief information officer. However, the Community Colleges’ Chancellor’s Office requires that web-based materials developed by its technology center comply with WCAG 1.0 standards. Accordingly, we assessed Community Colleges’ online application against those standards. California State Auditor Report 2014-131 21 June 2015 them from accessing the web page’s content. In this particular example, the violation could cause a screen reader user to be aware of that image but not know that the image was purely decorative and did not contain useful information. As a result, the violation could confuse the user, but would not make the page’s content completely inaccessible. Table 6 on the following page shows the number and proportion of violations at each severity level found at each department. The departments varied in the overall severity of their accessibility violations, but many of the accessibility violations we identified are of critical severity, meaning that elements of these websites were completely inaccessible to users with disabilities. Table 5 Total Web Pages Reviewed at Four Departments and Pages With Violations of California Accessibility Standards CALIFORNIA STATE OF CALIFORNIA DEPARTMENT CALIFORNIA COMMUNITY OF HUMAN COVERED FRANCHISE COLLEGES RESOURCES CALIFORNIA TAX BOARD Pages reviewed 58 78 57 70 Pages with distinct violations* 27 38 55 40 Pages with common violations† 3 7 7 7 Source: California State Auditor’s review of the results of accessibility testing conducted by our web accessibility consultant (consultant). Note: For the purposes of this table, we use the term violation to mean noncompliance with an accessibility standard a department’s website was expected to conform to because of state law or a grant or contract agreement. In this table, we did not include violations of the Web Content Accessibility Guidelines (WCAG) 1.0 that our consultant advised are so out of date that department efforts to comply with them might be inefficient. However, the table does include violations of WCAG 1.0 that were not outdated. * Distinct violations of accessibility standards our consultant identified that occur on individual web pages. † Common violations of accessibility standards are violations that are present in similar or identical ways across multiple pages throughout the site. In this table, we report each common violation as occurring on a single web page, even though the violation affects each page on the website where it is present. As a result, the number of pages with violations at each department is understated. Additionally, because distinct and common violations can both occur on the same web page, the number of pages with distinct and common violations in the table cannot be summed to calculate the total number of web pages with one or more violations at a department. One critical violation on the California Department of Human Resources’ (CalHR) Careers in California Government website (jobs site) concerns the areas on the exam form for state employment as a staff services analyst in which users must enter responses to exam questions. These areas were not always properly labeled so that they were connected to the corresponding exam questions. When these areas are properly labeled, a visually impaired user’s screen reader will read the question the user needs to answer as he or she tabs into the response area. Instead, on the exam we reviewed, persons using screen readers would need to leave the unlabeled response 22 California State Auditor Report 2014-131 June 2015 Table 6 Violations of California Accessibility Standards by Department and Severity CALIFORNIA CALIFORNIA DEPARTMENT STATE OF CALIFORNIA COMMUNITY COLLEGES OF HUMAN RESOURCES COVERED CALIFORNIA FRANCHISE TAX BOARD DISTINCT COMMON DISTINCT COMMON DISTINCT COMMON DISTINCT COMMON SEVERITY VIOLATIONS* VIOLATIONS† VIOLATIONS* VIOLATIONS† VIOLATIONS* VIOLATIONS† VIOLATIONS* VIOLATIONS† Critical Makes underlying content 26 2 26 7 287 18 17 7 completely inaccessible to users Serious Results in serious barriers 64 2 384 16 644 15 20 4 for users, making some content inaccessible Moderate Results in some barriers to users, 6 6 29 0 1,425 1 54 1 but does not prevent them from accessing fundamental content Source: California State Auditor’s review of the results of accessibility testing conducted by our web accessibility consultant (consultant). Note: For the purposes of this table, we use the term violation to mean noncompliance with an accessibility standard a department’s website was expected to conform to because of state law or a grant or contract agreement. In this table, we did not include violations of the Web Content Accessibility Guidelines (WCAG) 1.0 that our consultant advised are so out of date that department efforts to comply with them might be inefficient. However, the table does include violations of WCAG 1.0 that were not outdated. * Distinct violations of accessibility standards our consultant identified that occur on individual web pages. † Common violations of accessibility standards are violations that are present in similar or identical ways across multiple pages throughout the site. We report each common violation as occurring only once in this table, even though each represents multiple individual violations occurring throughout the respective website. area and use other methods, such as browsing the surrounding text on the web page line by line with the keyboard’s arrow keys, to try to determine what information they should enter. Before entering the information, the user would then need to navigate back into the response area using the Tab key. However, because the area was not properly labeled, the user could not be certain the response area was the one he or she left earlier. A similar critical violation on the same exam affects buttons that those taking the exam must use to answer questions. Both of these violations are common violations and occur throughout the exam we reviewed. Figure 3 illustrates how often these violations occur on a sample page of the exam. As a result of these violations, users taking the exam with the assistance of a screen reader could be forced to spend more time and effort to complete it than if the exam complied with state accessibility standards. Further, because it was unclear what the questions require, the risk is higher that users may inadvertently respond to exam questions with incorrect information. Incorrect answers to exam questions could cause applicants who would otherwise have attained eligibility for employment to miss an employment opportunity. The development supervisor for CalHR’s jobs site indicated that the exam we reviewed was an older exam and that he was not aware of it ever having been tested for accessibility. California State Auditor Report 2014-131 23 June 2015 Figure 3 California Department of Human Resources, Staff Services Analyst Exam Accessibility Violations 1. If a user pushes the Tab key to navigate from the top of the page, the screen reader skips from the Logout link to this button and reads: “less than two years, radio button, unchecked, one of four” without indicating that the level of college completed is in accounting. As a result, the user must take additional steps to obtain this information by leaving the response button and navigating the page line by line using the arrow keys. 3. When in this box, the screen reader user hears: “edit, has auto complete, blank”— again without indicating the area of study. The remaining boxes on the page also do not indicate the area of study to the user. Users would, therefore, have to go through the same steps as described at 1 . above to determine what information is requested. 2. The same violation takes place in all 12 sets of buttons on this examination page. As a result, a screen reader user would have to spend more time and effort completing the page and would have difficulty ensuring that the answers he or she provides are accurate. Source: California Department of Human Resources’ online staff services analyst exam. 24 California State Auditor Report 2014-131 June 2015 Staff at CalHR were aware of similar issues affecting another job exam over a year ago but did not fully address all of the barriers in that exam or conduct a review of other exams at that time. In February 2014 the CalHR web manager reviewed an employment exam for an office technician position after the department received a complaint about the exam. After his review, the web manager noted that questions on the exam were missing labels, making it difficult for screen reader users to know how to respond. However, when we asked about these issues in March 2015, the development supervisor for CalHR’s jobs site indicated that only some of the violations on that exam had been addressed. The supervisor stated that he assumed that the issues had been fixed since the developer for that job exam had been notified of the issues. The supervisor also stated that after discovering the issues with this exam, CalHR did not review other exams. If CalHR had done so, it may have known CalHR was unaware that an exam that similar issues existed on the exam we reviewed as part of this on its website had accessibility audit. Instead, CalHR was unaware that this exam had accessibility violations until we found them. violations until we found them. Community Colleges has already addressed a critical violation on its OpenCCCApply website (online application) that blocked some users from knowing whether they had completed all required components of the online college application. The online application form contains tabs for each page of the application, and these tabs include check mark symbols that indicate the user has successfully completed that page. In addition, the online application allows users to pass through pages of the application without fully completing them. However, at the time of our review the check mark symbols were missing text alternatives that screen readers need in order to interpret them for a visually impaired user. Therefore, this efficient way of knowing whether an application is complete was not available to screen reader users. As a result, if a screen reader user had intentionally or unintentionally skipped over a portion of the application, he or she would not be alerted that the application was incomplete until the last application page. Further, these users would have greater difficulty than others in determining which specific pages were missing required information, as they might have had to review the entire application to identify where exactly information was missing. The database that Community Colleges uses to track issues with its online application shows that a user complained about this violation in February 2015, shortly after our review of the online application the previous month. When we asked Community Colleges about the violation, the technology center’s chief technology officer stated that the feature with the check mark symbols had been part of the online application since it was first released in November 2012 but was not tested for accessibility. To its credit, after receiving the complaint, California State Auditor Report 2014-131 25 June 2015 Community Colleges addressed the violation in an update to the online application in February 2015. However, had the department appropriately tested the application for accessibility, it could have corrected the problem earlier and prevented it from potentially affecting college applicants. Our consultant also identified significant numbers of serious accessibility violations at all four departments. Although these Although the significant numbers violations do not make content completely inaccessible to persons of serious violations our consultant with disabilities, they present serious barriers to using the site in identified at all four departments a way that is comparable to the experience of persons without do not make content completely disabilities. For example, our consultant found a serious violation inaccessible to persons with on Covered California’s website that prevents users from skipping disabilities, they present serious over repetitive content each time they visit a new page on the site. barriers to using the site. Section 508 standards require that websites provide users with a method for skipping repetitive navigation links. Such links may appear at the top of each individual web page on a website. This standard helps ensure that websites are accessible to persons with disabilities who navigate the Internet with the keyboard alone, such as persons with motor disabilities. Covered California’s site provides a link that, if placed correctly on the site, would allow users to skip to the main content of each page. However, due to where this link is currently placed, keyboard-only users must navigate through all the repetitive links in the website’s header before they can access it, unnecessarily adding to the time it takes those users to access the main content of the web page. Because the link appears after all the repetitive content on the page, it provides no practical benefit to users and does not satisfy the Section 508 requirement. After analyzing the results of our consultant’s review, we provided each department with supporting details about each violation our consultant identified, including those we report on in this chapter. Subsequently, Covered California, CalHR, and Franchise Tax Board claimed that they began addressing some of the violations we identified. However, as described in the Introduction, our review did not include a comprehensive examination of any of the reviewed departments’ entire web presence. Instead, we focused on the pages in support of a service each department offers online. It is possible that the types of violations we describe in this section, as well as violations that were not present in the reviewed websites, exist in other areas of each department’s online presence that we did not review. Departments could guard against the risk that similar accessibility violations exist elsewhere on their websites by reviewing the violations we found, noting whether and where they are likely to occur throughout the rest of their online presence, and following up to correct the violations where they do occur. 26 California State Auditor Report 2014-131 June 2015 Some Accessibility Violations We Found Present Particular Burdens to Users For three of the department websites we reviewed, certain critical accessibility violations could, under some circumstances, prevent users from completing core online tasks. As discussed in the previous section, our consultant defined critical accessibility violations as violations that make the underlying content We found that particular completely inaccessible to persons with disabilities. Moreover, we critical violations at three of the found that particular critical violations at Community Colleges, departments could, under certain Covered California, and Franchise Tax Board could, under certain conditions, prevent users from conditions, prevent users from accessing the content they need accessing the content they need to to complete the core online tasks we tested at those departments. complete the core online tasks we As with the other accessibility violations we discuss in this report, tested at those departments. these violations affect users with specific types of disabilities, and because our consultant used a specific model and version of web browser and screen reader when performing the accessibility testing, users of different models or versions may not experience these violations in the same way and may be able to navigate past them.10 However, our consultant used the most recent version of the Mozilla Firefox (Firefox) web browser that was available at the time to test the accessibility of the departments’ websites. Firefox is a free browser software used by more than 500 million people worldwide. Therefore, the violations our consultant found are likely to affect a significant number of disabled users. In order to complete Community Colleges’ online application, users must first register for an account. To register, users are required to complete a series of pages by entering personal information and establishing a password and security questions. If a user takes more than eight and a half minutes to complete any of these pages, the application opens a time-out notification indicating that the application will log the user off in another 60 seconds unless he or she selects a button inside the notification box indicating that he or she wants to continue completing the account registration. This occurs even if the user is actively navigating around a page and entering information. If time expires, the application logs the user out of the registration, and the information the user has entered up to that point is lost. Users must restart the registration process from the beginning if they are timed out in this manner. This time-out notification process was inaccessible for some visually impaired users at the time of our review. Due to the way the notification box was programmed, screen reader software did not identify the box when it appeared, and therefore users were not 10 See the Scope and Methodology table, Objective 5(a) on page 15, for details on the software our consultant used. California State Auditor Report 2014-131 27 June 2015 alerted to its presence. As a result, screen reader users would have been unaware that time was about to expire, and they would be logged off if they did not complete the application page in the following 60 seconds. At that point, these users would have to begin the registration process again and would experience the same problem again if they did not more quickly complete the registration pages. Figure 4 on the following page illustrates the issue screen reader users would have with the time-out notification. Because registration is a required step in the online application process, this violation could have prevented some screen reader users from being able to submit an application to college. Community Colleges has now addressed this accessibility violation, but the violation illustrates shortcomings in its approach to accessibility testing. The Community Colleges technology center’s chief technology officer indicated that the time-out feature was added to the online application in December 2014 but that it was not tested for accessibility before being made available to the public. As a result, he became aware of this issue only after receiving a complaint from a screen reader user in February 2015. The chief technology officer also stated that all such updates to the website should include accessibility testing, but acknowledged that the technology center did not have a formal plan for completing accessibility testing on updates to its online application. We verified that, since discussing the issue with Community Colleges, the violation has been addressed and the notification box is functioning properly for screen readers. At Covered California, our review identified a critical violation that At Covered California, our review prevented keyboard-only users from applying for health insurance identified a critical violation that on the Covered California website. Figure 5 on page 29 illustrates prevented keyboard-only users this violation. Before starting an application for health insurance, from applying for health insurance users must identify what type of applicant they are: individual or on the Covered California website. family, employer, or employee. The Covered California website provides a button for each of these three options. However, users were not able to navigate to those buttons using the keyboard alone. As a result, persons with motor disabilities who are unable to use a computer mouse could not begin the application process. At the time of our review in February 2015, Covered California had recently made changes to this page of its website. However, Covered California did not test these changes for how they might affect the accessibility of its website. As a result, it was not aware that the new version of the page was inaccessible to users with certain disabilities and blocked those users from applying for health insurance. After we brought this violation to its attention, Covered California fixed the violation through an update to its website in May 2015. 28 California State Auditor Report 2014-131 June 2015 Figure 4 California Community Colleges, Account Creation Accessibility Violation When a user has been on any single web page in the account creation process for eight and a half minutes, a notification box opens to inform the user that time will expire in 60 seconds and allow him or her to request more time to complete the page. This occurs even if the user is actively entering information on the page. 60 Before California Community Colleges addressed this problem, the box was programmed in a way that did not alert a screen reader user that the box had appeared. As a result, screen reader users would have been unaware that time was about to expire and they would be logged off if they did not complete the page within the next 60 seconds. Those users would then have to start the process over. Source: California Community Colleges’ OpenCCCApply online account creation page. California State Auditor Report 2014-131 29 June 2015 Figure 5 Covered California, Application Accessibility Violation 1. Before beginning an application for health insurance, users must indicate what type of applicant they are by selecting one of these three buttons. 2. However, before Covered California addressed this problem, users could not navigate to the buttons by using only the keyboard. Users attempting to navigate in that manner jumped from the page header (above) directly to this link. As a result, persons with motor disabilities who cannot use a computer mouse were blocked from beginning an application. Source: Covered California’s online account creation page. 30 California State Auditor Report 2014-131 June 2015 Our review also identified a Our review also identified a violation in the registration process violation in the registration process for Franchise Tax Board’s CalFile application that could prevent for Franchise Tax Board’s CalFile some users from being able to complete and file their California application that could prevent tax returns online. When registering for an account, users must some users from being able to choose a security image that will be associated with their account complete and file their California going forward. Users with motor disabilities who navigate tax returns online. the Internet by using only a keyboard are unable to select this required image. Specifically, when one of these users navigates to an image and presses the Enter key to select it, the website submits the incomplete registration page rather than selecting the image. The website then notifies the user that he or she must select an image in order to complete the registration process. Figure 6 illustrates this error. If users are unable to register with the Franchise Tax Board, they cannot use CalFile to file their tax returns online. Franchise Tax Board’s accessibility lead analyst stated that the account registration software is distinct from the CalFile software and has not been tested for accessibility since December 2012. Results from the 2012 testing indicate Franchise Tax Board knew about some keyboard navigation issues at that time but did not identify this particular violation. The accessibility lead analyst stated that she could not be sure why her test report did not identify this issue but suggested that changes to the software or web browser updates since 2012 could have caused it. She also stated that keyboard users do not encounter this violation when using web browsers other than the browser our consultant used to test the website. Nonetheless, as we described in the previous section, our consultant used a widely used web browser when conducting its testing. As a result, violations affecting this browser could affect many persons with disabilities. Another critical violation our consultant found on Franchise Tax Board’s CalFile application could prevent some persons with disabilities from submitting their completed tax returns as the website instructs. Before submitting completed tax returns online, Franchise Tax Board directs users to review a PDF file that summarizes the information they have entered throughout the return process and certify that it is accurate. However, the PDF file that Franchise Tax Board presents to users is not fully accessible, preventing visually impaired screen reader users from being able to review the tax form. As a result, those users cannot verify that they have entered all their tax information correctly before submitting their tax returns. If users submitted their tax returns without reviewing the accuracy of the information, they would violate the Franchise Tax Board’s instructions and risk submitting an inaccurate return. California State Auditor Report 2014-131 31 June 2015 Figure 6 State of California Franchise Tax Board, Account Creation Process Accessibility Violation 1. To finish registering an account, users must select from among the 2. images below. Keyboard-only users are able to browse through the images, but hitting the Enter key triggers the “Continue” button instead of selecting the desired image. Users able to navigate the page with a computer mouse can select an image by clicking on it with the mouse. 3. When a user unsuccessfully attempts to select an image, the page reloads with an error message at the top (shown below) that notifies the user that the registration is incomplete. Source: State of California Franchise Tax Board’s online account creation page 32 California State Auditor Report 2014-131 June 2015 Despite knowing about an Despite knowing about this accessibility issue for more than accessibility issue with CalFile for two years, Franchise Tax Board has not implemented a solution more than two years, Franchise that would provide comparable access to users with disabilities. Tax Board has not implemented Accessibility testing documents at Franchise Tax Board indicate that a solution that would provide staff have known about this violation since at least December 2012 comparable access to users and are aware of how it affects users with disabilities. According with disabilities. to the manager who oversees CalFile, this issue was not given a higher priority because Franchise Tax Board believed that a disabled user could seek assistance from a third party to review the tax form, and it assumed that users would not submit their tax returns online unless they had found a way to review its content. Although users may be able to seek outside assistance in filing their taxes, placing this requirement on disabled users does not provide them comparable access to CalFile, which is a stated purpose of Section 508. The manager indicated that Franchise Tax Board has considered modifying this step in the online filing process so that users would not need to rely on the PDF copy of the tax form, but it has not yet done so. The manager also stated that Franchise Tax Board had hoped to find a way to adapt the PDF so it would be accessible, but it has not been able to do that. An email provided by Franchise Tax Board’s accessibility lead analyst indicates that in November 2014 staff at Franchise Tax Board attempted to make the PDF accessible but were unsuccessful in that effort. According to the manager, Franchise Tax Board plans to address this accessibility violation when it releases its next version of CalFile in January 2016. California’s Accessibility Standards Have Not Been Modernized California’s website accessibility standards are outdated and do not reflect current best practices for ensuring comparable access for persons with disabilities. As shown in Figure 1 on page 9, the W3C updated its WCAG standards in 2008 to version 2.0. The Information Organization, Usability, Currency, and Accessibility Working Group, which recommended California adopt WCAG 1.0 in 2006, recognized that technology changes rapidly and in its recommendations document emphasized the importance of evaluating and updating the State’s accessibility standards as needed. However, California has not revised its 2006 policy that requires websites to comply with WCAG 1.0 standards for departments and agencies reporting to the governor and the state chief information officer. According to the W3C, the WCAG 2.0 standards have certain key advantages over the 1.0 version. Specifically, it has stated that the WCAG 2.0 standards apply more broadly than the 1.0 standards to different types of web technologies and are also designed to be consistent with future technologies. The W3C has also stated that the 2.0 standards are more testable, allowing website providers to better ensure that their sites are accessible. In contrast, some California State Auditor Report 2014-131 33 June 2015 of the WCAG 1.0 standards are unclear or difficult to enforce, according to the federal Architectural and Transportation Barriers Compliance Board (Access Board), which issued the Section 508 standards. For example, one standard requires websites to use the clearest and simplest language appropriate for the content of the site. The Access Board did not include this as a Section 508 standard; instead determining that this standard was difficult to enforce because a requirement to use the simplest language can be very subjective. The WCAG 2.0 standards specify testable criteria that organizations can use to determine whether their websites conform to the standards. For example, one such criterion is that text on a web page can be doubled in size without using assistive technology and without loss of the page’s content or functionality. There are three conformance levels in the WCAG 2.0 standards: A (the lowest level of conformance), AA, and AAA (the highest level of conformance). A website conforming to the AAA level of the WCAG 2.0 standards must meet more stringent accessibility criteria than a website that conforms to either the A or AA level. The W3C does not recommend that organizations adopt the AAA conformance level as policy for their entire website because it is not currently possible to satisfy the related criteria for some types of web content. The Access Board has considered adopting the WCAG 2.0 The federal Access Board has standards as a replacement for the Section 508 standards. In considered adopting the WCAG 2.0 2010, the Access Board issued an Advance Notice of Proposed standards as a replacement for the Rulemaking that proposed updating the existing Section 508 Section 508 standards. standards for web accessibility with slightly modified versions of the WCAG 2.0 standards. In response to public concern that the slightly modified standards were potentially confusing, the Access Board later reissued the draft proposed rule and incorporated the WCAG 2.0 standards in their entirety up to the AA conformance level. More recently, in February 2015, the Access Board issued a Notice of Proposed Rulemaking that incorporated the WCAG 2.0 standards, at the AA level of conformance, as a replacement for the existing Section 508 standards. As of April 2015, the Access Board was scheduled to close public comment on this proposed rule on May 28, 2015. To further compare the WCAG 1.0 and 2.0 standards, we reviewed the departments’ websites against the WCAG 2.0 standards at the AA conformance level. We did not identify any state law or policy that would require the departments we reviewed or any other state department to meet the WCAG 2.0 standards. Nonetheless, we found that the vast majority of violations of the Section 508 and WCAG 1.0 standards also represent missed opportunities for accessibility under WCAG 2.0. For example, earlier in this 34 California State Auditor Report 2014-131 June 2015 chapter we described a violation on a CalHR employment examination where, due to improper labeling, screen reader users would not have exam questions read aloud to them. This was a violation of a WCAG 1.0 standard that CalHR is currently required to meet. However, the labeling problem also does not meet a WCAG 2.0 standard. Our consultant advised us that some of the WCAG 1.0 standards violated do not perfectly align with the corresponding 2.0 standards. As a result, it is possible that departments, in complying with current state standards, could apply outdated techniques for providing accessible websites that would leave them short of the WCAG 2.0 standard. In other words, despite the overlap we observed between many WCAG 1.0 and 2.0 standards, there is still benefit in clearly defining which of these two standards state departments should be required to meet. We believe it is important that We believe it is important that California update its accessibility California update its accessibility requirements to bring them in line with the most up-to-date requirements to bring them in accessibility standards. Currently, that would mean requiring state line with the most up-to-date departments to comply with the WCAG 2.0 standards. According accessibility standards. to the W3C, most websites that conform to the WCAG 1.0 standards should not require significant changes to comply with WCAG 2.0. Therefore, the additional cost of any change to the state accessibility standards is likely to be minimal for departments that already comply with the current state requirements. Because changes in technology are ongoing, it is also important that the State make every effort to prevent its accessibility standards from again becoming outdated in the future. CalTech, as the State’s lead department for matters related to information technology, could help the State keep its accessibility policy up to date by monitoring WCAG standards and alerting policymakers to any further revisions the W3C makes to those standards. After we discussed this matter with her, CalTech’s chief deputy director of policy stated that she sees a role for CalTech in monitoring emerging web accessibility standards and that CalTech could notify the Legislature in the future if commonly accepted standards changed significantly enough that California should adjust what standards it requires. Recommendations Legislature To maximize the accessibility of California’s websites, the Legislature should amend state law to require that all state websites conform to WCAG 2.0 standards at compliance level AA in addition to the Section 508 standards. California State Auditor Report 2014-131 35 June 2015 To help ensure that California’s accessibility standards remain current, the Legislature should amend state law to require CalTech to monitor commonly accepted accessibility standards and apprise the Legislature of any changes to those standards that California should adopt. Departments To ensure that they address barriers to the accessibility of their websites for persons with disabilities, each of the four departments should, no later than December 1, 2015, correct the accessibility violations we found during our review. No later than December 1, 2015, each department should develop a plan to determine whether the accessibility violations we identified exist on other portions of their online presence that we did not include in the scope of our review. At Community Colleges this should include any web presence managed by its technology center. Once this plan is executed, the departments should correct violations wherever they find them and do so no later than June 1, 2016. 36 California State Auditor Report 2014-131 June 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-131 37 June 2015 Chapter 2 DEPARTMENTS HAVE NOT CONSISTENTLY TESTED FOR ACCESSIBILITY, AND STATEWIDE WEB ACCESSIBILITY TRAINING SHOULD BE REQUIRED Key Points » The four departments we reviewed conducted some accessibility testing, but most could improve the method and frequency of their tests. For example, Covered California has not regularly tested updates to its website, and the update testing that the California Department of Human Resources (CalHR) performed did not include automated testing techniques. » State policy requires most departments to provide specific contact information so that website users can complain about accessibility issues. Although not specifically required to provide this information, Covered California and California Community Colleges (Community Colleges) do not provide all of this important information on their websites. This omission limits the number of ways for users who want to file an accessibility complaint to do so. » CalHR lacks adequate processes for tracking the results of its accessibility testing and the complaints it receives about the accessibility of the Careers in California Government website (jobs site) and, as a result, cannot effectively monitor accessibility testing or the management of complaints. » Shortcomings in department web accessibility testing demonstrate a need for a statewide formalized testing approach. In addition, we believe California would benefit from greater oversight of accessibility testing and the California Department of Technology (CalTech) is the logical entity to take on this responsibility. Each Department Conducted Some Accessibility Testing, but Most Do Not Sufficiently or Consistently Test Updates to Their Websites Guidance at both the state and federal levels indicates departments should test web products for accessibility, whether they are developed in-house or procured from outside vendors. The U.S. Department of Justice, which at the federal level is tasked with issuing recommendations regarding federal government compliance with Section 508 of the federal Rehabilitation Act of 1973, as amended (Section 508) standards, recommends that agencies develop procedures to regularly test their web products 38 California State Auditor Report 2014-131 June 2015 for accessibility, using both manual and automated methods of testing. Manual testing involves actual users attempting to use a website, sometimes with the help of assistive technologies such as screen readers. Automated testing can include the use of freely available tools online that scan web pages for accessibility problems. Also at the federal level, the U.S. General Services Administration advises departments to incorporate testing against the Section 508 standards into verification and validation of web products before they are deployed, including completing testing with assistive technologies.11 The Statewide Information Management Manual (SIMM) says that accessibility testing should occur for web products and should include individuals who represent various types of disabilities. The departments we reviewed The departments we reviewed all conducted some accessibility all conducted some accessibility testing on their web-based services, as shown in Table 7. In testing on their web-based services. general, testing occurred before the key release of the supporting web product to the public. Covered California’s vendor completed accessibility testing of the California Healthcare Eligibility, Enrollment, and Retention System (CalHEERS) website before issuing the first major release of the site to consumers in October 2013. This testing included both automated and manual review of the CalHEERS site. Similarly, Community Colleges’ vendor performed both manual and automated accessibility testing before the debut of its OpenCCCApply website (online application) in November 2012. According to the executive director of the Community Colleges Technology Center (technology center), Community Colleges also had a staff developer review some of the vendor’s work to ensure that the online application aligned with accessibility standards. CalHR’s supervisor over web development explained that when the current version of the jobs site, which was released in 2012, was being developed, developers conducted accessibility testing on portions of the site by using an online automated tool and that the California Department of Rehabilitation also reviewed some online job exams at the time using screen reader software.12 However, there is no documentation of this testing or its results. In contrast, the State of California Franchise Tax Board (Franchise Tax Board) did not conduct accessibility testing on its CalFile product until many years after it was developed. According to Franchise Tax Board’s accessibility lead analyst, 11 Verification and validation is a process used to ensure that a system conforms to requirements and satisfies user needs. 12 According to the supervisor over web development at CalHR, the current version of the jobs site was released in January 2012, before CalHR existed. However, the same team that built the jobs site while employed at the California State Personnel Board continues to work on the site as employees of CalHR, although some staff have left CalHR since the development phase. California State Auditor Report 2014-131 39 June 2015 Franchise Tax Board first conducted accessibility testing on CalFile in 2012. However, according to a systems software supervisor, Franchise Tax Board originally developed CalFile in 2003. We reviewed summary accessibility test reports and individual issues found during testing that show that since 2012, the start of the period we reviewed, Franchise Tax Board has regularly tested CalFile for accessibility and has used both automated and manual methods since 2013. Table 7 Accessibility Testing Approaches by Four Departments INITIAL TESTING UPDATE TESTING METHOD METHOD DEPARTMENT AUTOMATED MANUAL AUTOMATED MANUAL FREQUENCY Not Covered California   5 † regular California Community Colleges Not   5  (Community Colleges) regular California Department of Human Not * * 5  Resources (CalHR) regular State of California Franchise Tax 5 5   Regular Board (Franchise Tax Board) Sources: California State Auditor’s interviews with staff and review of available testing documentation at Community Colleges, CalHR, Covered California, and Franchise Tax Board.  = Yes 5 = No * Staff stated that testing was performed, but it was not documented. † This testing was limited to evaluating the Voter Registration page with a screen reader. Although three of the four departments conducted some accessibility testing before the initial release of their web service, Franchise Tax Board is the only department that conducted regular update testing. Other departments we reviewed have not regularly tested subsequent updates to their websites. According to testing reports written by Covered California’s vendor, the vendor performed no dedicated accessibility testing after CalHEERS’ first major release to consumers in October 2013.13 The CalHEERS schedule shows that since that time, Covered California has issued updates to the CalHEERS website. Covered California’s chief of operational readiness confirmed that these updates included changes to content the public interacts with on the site but stated that most of the changes added functionality for Covered California staff. However, 13 Covered California did have a separate contractor perform manual testing in October 2014 with a screen reader on its Voter Registration page as part of an update to its website. However, this was an isolated review of a single page. 40 California State Auditor Report 2014-131 June 2015 as we discussed in Chapter 1, Covered California recently updated a page on its website on which users begin an application for health insurance. That new page had a critical violation that blocked keyboard-only users from being able to begin an application. The assistant project manager for the CalHEERS project confirmed that Covered California was unaware of this accessibility violation until we brought it to its attention. More thorough testing of updates to the CalHEERS website would help Covered California guard against releasing inaccessible content. Table 7 on the previous page summarizes the initial and update testing performed by the four departments we reviewed. Community Colleges’ technology Community Colleges’ technology center does not have a formal center does not have a formal plan to ensure that updates to its online application are tested for plan to ensure that updates to its accessibility, and it acknowledged that the testing that does occur online application are tested for is not comprehensive. The technology center’s agreement with its accessibility, and it acknowledged vendor states that the vendor is responsible for testing the online that the testing that does occur is application, but it does not specifically mention accessibility testing. not comprehensive. The database that the technology center uses to track testing of its online application shows that its vendor conducted some manual testing of the website after the release of the website to the public. However, Community Colleges acknowledged that its vendor does not always test updates to its online application for accessibility. For example, the technology center’s chief technology officer acknowledged that the online application’s security time-out feature was not tested for accessibility before being released in December 2014 and that because of this, the technology center’s vendor did not know about an error on its site wherein screen reader users would not know they were about to be timed out from the site. The technology center was unaware of the issue until a visually impaired user brought the issue to its attention. Similarly, CalHR informally tests updates to its jobs site, and the testing does not follow best practice recommendations to include automated testing as well as manual. According to the supervisor over web development for its jobs site, he will occasionally ask for accessibility tests on new content. He explained that he usually decides to request testing when there have been changes to standard templates that CalHR uses for its online exams. The web manager confirmed that he conducts testing on new exams posted on the jobs site when it is requested, as there is no formal approach for regular testing. Further, the web manager stated that his testing did not include automated methods and involved using only screen reader software to manually test the exams. According to the W3C, automated accessibility evaluation tools can reduce the time and effort required to carry out testing, because they can determine websites’ compliance with standards that can be evaluated automatically, as well as assist reviewers in performing testing on some standards that need to be evaluated manually. California State Auditor Report 2014-131 41 June 2015 Three departments we reviewed track the results of their accessibility testing. Covered California and Community Colleges both use databases to track testing results from the time an issue is identified through resolution. Through 2013, according to its accessibility lead analyst, Franchise Tax Board used a database to track testing results. According to the accessibility lead analyst, in 2014 the department tracked unresolved accessibility issues through an annual accessibility report because it was transitioning to a new tracking database, which the department began using in 2015. Further, the accessibility lead analyst indicated that as Franchise Tax Board transitions to its new tracking database, the accessibility testers have exchanged emails with the CalFile development team about problems they identify with CalFile, and any issues not resolved by the end of each calendar year will be reflected in the annual accessibility report. Despite using a database to track accessibility defects with its online application, Community Colleges has not provided adequate direction to its vendor about how to indicate that a defect has been resolved. Throughout the audit, we relied in part on Community Colleges’ database in order to determine whether it had addressed accessibility problems that it was made aware of through testing or because of a complaint from a user. However, in several cases, we found that Community Colleges’ vendor had closed the record of a defect without indicating that the defect had been resolved. Because of this, it was difficult for us and for the technology center’s chief technology officer to determine when or whether the accessibility problems underlying the database records had actually been resolved. According to the chief technology officer, the database was sometimes inadequate for determining whether a defect was fixed. He further indicated there would be value in changing how defects are tracked in the technology center’s database so that it is clear when they are resolved. According to its web manager, CalHR does not track the results of its accessibility testing in a systematic manner that would allow it to easily determine whether the issues identified in testing were addressed. As previously described, CalHR staff reported that they perform accessibility testing when requested to do so after changes are made to existing website templates. According to CalHR’s web According to CalHR’s web manager, manager, who performs the testing, the only records of testing who performs the testing, the only results are his email records. The web manager acknowledged that records of testing results are his the emails he retains do not necessarily reflect all accessibility testing email records. However, the level emails he sends or receives. We reviewed the web manager’s email of information in the emails we records and found incomplete information about testing efforts. reviewed would not allow CalHR The level of information in the emails we reviewed would not allow to know consistently whether CalHR to know consistently whether issues were resolved and, if so, issues were resolved and, if so, how quickly. The web manager and the supervisor over development how quickly. for the jobs site acknowledged that CalHR does not currently have 42 California State Auditor Report 2014-131 June 2015 a level of information about accessibility testing that would allow it to know what portions of the site had been tested, what issues were found, and whether those issues were resolved. According to the supervisor over development for the jobs site, since the audit began, CalHR has begun drafting new procedures for accessibility testing and tracking the results of that testing, but he did not have an estimate for when those procedures would be finalized. Departments Generally Address Problems Found During Testing, but Not Always Before Releasing Their Websites to the Public Three of the departments we reviewed did not always resolve issues found during accessibility testing before releasing their web service to the public or before issuing significant updates to the service. As discussed in Objective 5(e) of the Scope and Methodology table on page 16, we did not review a selection of testing results at CalHR due to the poor quality of the department’s record keeping related to accessibility testing. We explained in the previous section that the lack of comprehensive tracking at CalHR prevents it from effectively knowing what it has tested and whether it has addressed all issues it finds during testing. In the case of both Covered California and Community Colleges, we assessed whether issues were identified and resolved before the debut of the departments’ products to the public. For Franchise Tax Board, because the department did not begin testing CalFile for accessibility until after it was initially developed, we assessed whether defects that were identified after the product was released were resolved and how quickly. Covered California did not resolve Covered California did not resolve before the first major release of its before the first major release of its CalHEERS site in October 2013 all of the defects that we reviewed that CalHEERS site in October 2013 all it had identified at that time. As shown in Table 8, of the 22 defects of the defects that it had identified Covered California identified by October 2013 that we reviewed, at that time. 20 were not resolved before that date, despite Covered California knowing that the issues existed. In fact, Covered California took an average of 245 days from its release to resolve these defects. One of these unresolved defects was that alternative text, which screen reader users rely on to describe to them what images are found on a web page, was missing from most of the images on the site’s pages. Without these text alternatives, visually impaired users cannot understand what images are found on the page that they are navigating. According to similar types of violations that our web accessibility consultant (consultant) found at all four departments that we reviewed, an accessibility issue with alternative text would, at a minimum, be considered a serious violation that makes some content inaccessible.14 14 For the purposes of this report, we use the term violation to mean a violation of an accessibility standard a department’s website was expected to conform to because of state law or a grant or contract agreement. California State Auditor Report 2014-131 43 June 2015 According to the CalHEERS assistant project director, the accessibility defects that were not resolved before the October 2013 release had to be prioritized along with all other CalHEERS defects or they were duplicates of other already existing defects. Further, she explained that the release date could not be pushed back due to the legally mandated date for the State’s first open enrollment to begin. She also stated that some of these defects were later rejected, meaning they were ultimately determined not to be true defects. However, CalHEERS staff did not know that these defect records did not represent actual accessibility problems or that they were duplicates of existing defects until they were reviewed and subsequently rejected. As of December 2014, all 20 of these defects were marked as resolved in the department’s tracking database. In addition, Covered California identified eight other accessibility defects that we reviewed after the first major release of CalHEERS and resolved these within an average of about 50 days of discovering the defects. Table 8 Defects Found During Accessibility Testing at Three Departments CALIFORNIA STATE OF CALIFORNIA COMMUNITY COLLEGES COVERED FRANCHISE TAX BOARD (COMMUNITY COLLEGES) CALIFORNIA (FRANCHISE TAX BOARD) Defects we reviewed* 23 30 20 Defects department resolved 23 30 19 Resolved before release 12 2 NA† Resolved after release of product‡ Identified before release 6 20 NA† Average days from release 219§ 245 NA to resolution Identified after release 5 8 19 Average days to resolution 12 49 182 Sources: California State Auditor’s analysis of data from Community Colleges Technology Center’s JIRA database; Covered California’s Application Lifecycle Management database; Franchise Tax Board’s ClearQuest database; email records of Franchise Tax Board staff; and interviews with staff at Covered California, Community Colleges, and Franchise Tax Board. NA = Not Applicable. * Although we planned to review 30 defects at each department, the keyword search to identify defects at Community Colleges and Franchise Tax Board resulted in fewer than 30 accessibility-related defects. † Franchise Tax Board did not conduct accessibility testing on CalFile until 2012, so no defects were identified prior to its 2003 release. ‡ In this table, release refers to the date on which the product supporting the online service we reviewed became available for public use at each department. § This average includes two items that took 827 days and 398 days, respectively, to resolve which was notably longer than the other four items included in this average. 44 California State Auditor Report 2014-131 June 2015 At Community Colleges, the Similarly, at Community Colleges, the technology center and technology center and its vendor its vendor were aware of some issues prior to the release of the were aware of some issues prior to online application but did not resolve all of those issues prior the release of the online application to the release of the application. We reviewed 23 defects at this but did not resolve all of those department, including 18 defects identified before Community issues prior to the release of Colleges released the online application. Community Colleges the application. did not resolve six of the 18 defects despite the technology center and vendor knowing about them before the application’s release in November 2012. Four of these issues were resolved on average within about a month of the application’s release. However, the remaining two were not resolved for more than 300 days after the application’s release. One of these issues, in which error messages were not being read by screen readers, took 827 days to resolve. If visually impaired screen reader users do not have error messages read to them, the users will be unaware of the specific problem that prohibited them from continuing their application. The technology center’s chief technology officer was not in his current position in November 2012 when the online application was first released. However, he stated that the vendor who built the site told him that this problem was addressed shortly after the online application’s initial release. According to the chief technology officer, the vendor states that code change logs indicate the problem was addressed in December 2012. Nevertheless, the defect record in the technology center’s tracking database contains notes that show that the problem persisted after December 2012. Further, we were not able to locate any evidence in Community Colleges’ tracking database that demonstrates that it addressed this error message defect any earlier than February 2015. In the other case, the technology center’s vendor closed the defect with a “Won’t Fix” status prior to the release of the application. However, the notes for this defect show that the technology center’s vendor knew that this defect meant a security measure that the online application used was not accessible to screen reader users because it relied on an image that did not have alternative text. The security feature that the online application featured was designed to help tell human users apart from malicious software. According to the technology center’s chief technology officer, the development team originally had to make a judgment call between security and accessibility and chose to focus on the security of the online application in this case. The technology center eventually addressed the accessibility of this security feature 398 days, or more than one year, after the site was released. According to the chief technology officer, this was possible because the security feature’s manufacturer released an accessible version of the feature, which the technology center adopted. California State Auditor Report 2014-131 45 June 2015 As discussed in the previous section, Franchise Tax Board did not begin accessibility testing on CalFile until 2012, after the product’s initial 2003 release. According to the manager overseeing CalFile, Franchise Tax Board releases an update to CalFile in January of each year at the start of each tax filing season. After that release, the department does not issue another update to CalFile until the following year, unless there are issues that would cause incorrect filings to occur. Of the 20 defects we reviewed, Franchise Tax Board addressed 15 before the next annual update of CalFile. However, four additional defects we reviewed were defects Franchise Tax Board was aware of prior to releasing the next update, but it did not release a fix for those defects until the following year’s update. Three of these defects were related to keyboard navigation errors in the CalFile application, and the fourth was related to screen reader use. For two of these defects, the accessibility lead analyst stated that due to constraints on time and resources to address problems, they were not addressed until the following year. For the third defect, Franchise Tax Board staff were unable to explain why the defect had not been addressed. According to a supervisor in Franchise Tax Board’s Internet and taxpayer folder section, the fourth defect was corrected before the next CalFile update, but the item was not closed in its database until after the update. However, he could not provide any documentation to that effect. Finally, out of 20 total defects we reviewed, we found one unresolved defect related to screen reader users being unable to review information they had entered in the CalFile application. We discussed this defect in greater detail on page 30 in the Chapter 1 section on burdensome violations of web accessibility standards. Some Departments Must Improve the Complaint Contact Information Presented on Their Websites Before the start of this audit, most departments we reviewed did not provide important information to the public regarding how to complain about accessibility problems with websites. At the federal level, Section 508 states that individuals with disabilities can file complaints Key Elements Required on State Websites regarding failure to provide accessible electronic • “Contact Us” or “Accessibility” links information technology, such as websites. State • A statement affirming the department’s commitment policies issued by the state chief information officer to accessibility establish a variety of elements that most departments must include on their website, as • Contact information, including telephone number, mailing shown in the text box. Some of the elements are address, email address, and teletypewriter (TTY) number designed to help members of the public make • Instructions for reporting accessibility problems complaints about website accessibility issues. In Sources: Information Organization, Usability, Currency, and addition, state guidance provides that complaints Accessibility Working Group recommendations from July 2006; about website accessibility should be directed to California Department of Technology, Information Technology Policy Letter 10-10. a person or group with the responsibility and 46 California State Auditor Report 2014-131 June 2015 technical knowledge necessary to effectively respond to such complaints. Although Covered California is not required to comply with policies issued by the state chief information officer and it is unclear whether Community Colleges must follow those same policies, we assessed the complaint contact information on both departments’ websites against the guidance and best practices that are the basis of the requirements that other departments must follow. At the time of our review, Covered California did not provide all of the complaint contact information on its website that state policy requires other departments to provide and Community Colleges’ online application featured only a contact page for general help that lacked any reference to web accessibility At the time of our review, only or complete contact information. Also, at the time of our review, CalHR’s website met all state Franchise Tax Board lacked complete contact information on the criteria for providing information portion of its website that discussed accessibility. Only CalHR’s about how to complain about web website met all state criteria for providing information about how accessibility problems. to complain about web accessibility problems. As of November 2014 Covered California had updated the instructions on its website for submitting a complaint so that they are now more specific to website accessibility. This update was part of a website redesign initiated before this audit began. The previous version of the website had no accessibility page; instead, it had only a contact page that directed all requests for assistance to a service center. The site now features an accessibility page that includes a statement of commitment to accessibility. However, it provides only an email address as contact information for submitting a complaint and does not feature the remaining contact information that other state websites are required to provide. According to its marketing director, Covered California made an organizational decision to post only the email address because high turnover made it difficult to identify one designated person as the contact for accessibility issues. However, departments may create barriers to reporting web accessibility issues when they do not provide more than one type of contact information. To make itself more reachable by persons with disabilities, Covered California could also include a telephone number, teletypewriter number, and mailing address on its accessibility page as other departments are required to do. Until the start of this audit, Franchise Tax Board’s website did not provide specific instructions for users wishing to file a complaint about website accessibility. Franchise Tax Board’s accessibility page outlined its Americans with Disabilities Act (ADA) grievance process, which it operates from its Equal Employment Opportunity (EEO) office and which is specific to ADA-related issues. The page did not mention web accessibility or Section 508. Additionally, the accessibility page did not include an email address to which complaints could be directed. A different area of Franchise Tax Board’s website provided a general complaint form for users who California State Auditor Report 2014-131 47 June 2015 experience problems with the website but did not indicate whether it was the correct method for submitting an accessibility-related complaint. A lack of clear information about how to report problems with website accessibility could result in individuals who experience problems not knowing where to direct a complaint. Franchise Tax Board’s EEO officer was not able to identify a reason that the department’s accessibility page did not include information specific to Section 508 web accessibility, and she stated that she had been unaware of the requirement regarding full contact information for web accessibility complaints until we informed her. After we asked Franchise Tax Board about the content of its accessibility page, it modified its site to provide more clarity on how to report web accessibility problems. The website’s accessibility page now instructs users to contact Franchise Tax Board’s ADA coordinator if they have any difficulties accessing the site materials. Franchise Tax Board also updated the page to include the ADA coordinator’s email address. This change brings Franchise Tax Board’s accessibility page into full compliance with state requirements. The Community Colleges online application lacks most of the The Community Colleges information that state policies require other departments to include online application lacks related to accessibility complaints. The site features a telephone most of the information that number and email address for technical support but none of the state policies require other other forms of contact information or a statement of commitment departments to include related to accessibility. According to Community Colleges’ technology to accessibility complaints. center executive director who oversees this site, he was unaware that there were state requirements for an accessibility page. As we indicate in the next section, some users followed the technical support contact information on the online application to complain about its accessibility. However, by not providing all forms of contact information that state guidance indicates should be included on a website, Community Colleges is limiting the number of ways in which persons with disabilities can complain if they encounter problems with the online application. The Departments We Reviewed Appear to Receive Few Complaints About Web Accessibility None of the departments we reviewed tracked web accessibility complaints separately before the start of our audit. When we began this audit, staff at both Covered California and Community Colleges indicated that both entities logged complaints about their web-based services into a database. However, neither department’s database included a separate flag that would allow the department to identify all complaints related to website accessibility. Both departments have since changed their approach to tracking accessibility complaints. After the start of our audit, the 48 California State Auditor Report 2014-131 June 2015 Community Colleges technology center added a flag to its database specifically to track complaints from persons with disabilities. Also, resulting from a website redesign initiated before this audit began, Covered California now directs all persons with web accessibility questions, suggestions, or complaints to submit them using a single email address. Finally, as discussed in the previous section, before the start of this audit, Franchise Tax Board did not provide clear instruction regarding where users should direct web accessibility complaints. As a result, users with accessibility problems may have reasonably used either Franchise Tax Board’s ADA grievance process or its general website complaint form, or they may not have complained at all. However, Franchise Tax Board now instructs website users to direct all questions about web accessibility to its ADA coordinator. By doing so, Franchise Tax Board has helped ensure that web accessibility questions and complaints will not be mixed among other website issues, which should assist it in accurately tracking any complaints about web accessibility. Although Covered California now Although Covered California now directs all accessibility complaints directs all accessibility complaints to one email address, there is no formal process in place for handling to one email address, there is no these emails. According to Covered California’s chief of information formal process in place for handling technology services, most of the issues users have emailed about are these emails. related to user accounts and choosing a health plan, which are issues better handled by Covered California’s service centers. However, when we asked about Covered California’s process for ensuring that accessibility-related emails were addressed, the chief of information technology services confirmed that no written procedure exists for handling accessibility complaints. She stated that someone reviews the emails daily. She also stated that the department planned to develop procedures for processing the emails and explained what steps Covered California would take if it received a web accessibility complaint before finalizing these procedures. However, until it develops such procedures, Covered California is at a greater risk that a user’s accessibility complaint will go unnoticed or will not be adequately addressed. CalHR provides clear direction on its website for how users can report problems with the accessibility of its site, but it does not track complaints that it receives. The contact information on CalHR’s accessibility web page connects complainants with its web manager for web accessibility issues. However, the web manager stated that there is no process for tracking user issues. The web manager stated that he receives few calls and emails regarding web accessibility and that he assists users who call to complain about the website. However, there is no procedure in place to ensure that if he discovers a problem with the website while assisting a user, the error is documented and resolved. For example, the web manager recalled helping a visually impaired complainant through a problem with web page navigation and in so doing California State Auditor Report 2014-131 49 June 2015 realized that the “skip to content” option on CalHR’s jobs site was disabled, which contributed to the complainant’s issue with the site. However, when we asked whether this issue on the jobs site had been addressed, the web manager indicated that it had not. He acknowledged that CalHR could benefit from better tracking of the receipt and resolution of complaints. We found no record of accessibility complaints at two of the We found no record of accessibility departments that kept records of complaints related to their complaints at two of the websites. At both Covered California and Franchise Tax Board, we departments that kept records of searched available complaint records for keywords related to web complaints related to their websites. accessibility, such as “screen reader,” “508,” “blind,” and “deaf.” At Covered California, we searched complaints received at its service centers from June 2013 through December 2014 and the record of emails it received at its new accessibility complaint email address since November 2014. At Franchise Tax Board, we reviewed the only complaint its EEO officer stated that it received through its ADA grievance process since 2010, along with emails submitted through its general website complaint form, which at the time of our review was a method website users could use to submit complaints. Our search of those records was limited to the emails that Franchise Tax Board retains. According to the staff member in charge of responding to general website complaints, Franchise Tax Board retains a record of those complaints for only 60 days after the issue has been resolved. As a result, our search was limited to the records that existed when we performed our search in March 2015. At both of these departments, we found no complaints related to web accessibility. At Community Colleges, we identified several complaints related to web accessibility. We searched the records of complaints received about Community Colleges’ online application since it was first made public in November 2012 using keywords related to web accessibility. Through this process, we identified five web accessibility complaints. Most of these complaints concerned the interaction of the online application with screen reader software. One such complaint was already discussed in Chapter 1 and related to a problem with the time-out feature of the online application not being accessible to those persons using a screen reader. The same complainant also identified several other problems he experienced while using his screen reader to complete the application, such as the screen reader not identifying a check mark symbol that shows if a section has been completed, which was also identified by our consultant and discussed in Chapter 1. This complaint was forwarded to Community Colleges’ development team so that it and other issues the user identified could be addressed. Records from Community Colleges’ tracking database indicate that the check mark issue was resolved in late February 2015, which was within one month of receiving the complaint about this issue. 50 California State Auditor Report 2014-131 June 2015 The few complaints we observed across the departments we reviewed appear consistent with the few complaints reported at the federal level. In 2010 and 2011, the U.S. Department of Justice surveyed 89 federal agencies about their implementation of Section 508. In their responses to the survey, those agencies self-reported that, from June 2001 through 2010, they had received a total of 140 administrative complaints about their web accessibility, or only about 15 complaints per year among all of the agencies. However, because our search for complaints at the departments we reviewed relied on keywords, it is possible that complaints exist that we did not identify. It is also possible that the few complaints we observed are related to the lack of clear complaint contact information provided by some departments, as we discussed in the previous section. Levels of Training at Each Department Varied, and Statewide Training on Web Accessibility Is Not Required We identified no requirement for statewide training specific to the development and maintenance of accessible websites. As discussed in the Introduction, the state law that requires departments to maintain their websites in accordance with the Section 508 standards does not establish a department or agency responsible for statewide enforcement or oversight of these standards. Additionally, no state law we reviewed as part of this audit requires web accessibility training for state employees. As part of its general activity as California’s lead agency for matters related to information technology, CalTech maintains a website dedicated to conveying information and resources to state webmasters about a variety of topics, including web accessibility.15 CalTech also facilitates a webmaster users group that meets periodically to discuss topics related to website management. According to the agendas for this users group going back to 2009, discussion at the meetings has included accessibility on several occasions, including as recently as February 2015, when the California Department of Rehabilitation (Rehabilitation) offered a refresher session on basic web accessibility requirements. According to the web services manager at CalTech, attendance at these sessions is optional. At the departments we reviewed At the departments we reviewed during this audit, the level of during this audit, the level of training training offered by the departments or obtained by staff was offered by the departments or inconsistent. CalHR’s web content manager stated that he leads obtained by staff was inconsistent. training for department staff on how to ensure that documents uploaded to its site are accessible. Staff from Franchise Tax Board attended a training session provided by an accessibility consulting 15 This website is located at webtools.ca.gov. California State Auditor Report 2014-131 51 June 2015 firm in 2013 about the accessibility of PDF documents. In contrast, staff at Covered California and the Community Colleges technology center reported that they did not receive training on web accessibility from their respective department or attend any such third-party training. Although both of these departments procured the websites associated with the web-based services we reviewed Even at departments that procure as part of this audit, neither department’s staff were trained on best their websites, we believe staff practices for procuring an accessible website. Even at departments should receive some level of that procure their websites, we believe staff should receive some accessibility training to familiarize level of accessibility training to familiarize them with best practices them with best practices for for procuring an accessible website. procuring an accessible website. Best practice guidance regarding accessibility training advises that department personnel should attend such training. At the state level, CalTech’s SIMM recommends that agencies provide specific training to content creators and refers to federal and state resources for training in the areas of procurement, accessibility testing, and specific Section 508 standards. The U.S. Department of Justice, in a September 2012 report on Section 508 compliance among federal departments, recommended that agencies provide or facilitate the provision of more training to department Section 508 coordinators, agency personnel involved in the procurement process, and developers—including software, website, video, and multimedia developers. One state department has offered training on the Section 508 standards and continues to offer training on accessible documents. Shortly after California adopted the recommendations of the Information Organization, Usability, Currency, and Accessibility Working Group (IOUCA) in July 2006, Rehabilitation offered several training sessions to state web professionals on Section 508 web accessibility requirements. It does not appear that attendance at these trainings was required, and Rehabilitation’s chief information officer stated that the classes were free to attend. Training materials from the classes show that the training covered the Section 508 standards and how to achieve or maintain compliance. The chief information officer also indicated that Rehabilitation continues to offer training to all departments at a small cost about how to ensure that documents posted to department websites are accessible to persons with disabilities. He stated that in 2013 and 2014, the department provided 62 such training classes for 697 attendees. Given the level and frequency of training that we observed at each department we reviewed and the type of accessibility standards violations that we discussed in Chapter 1, we believe that California would benefit from a requirement that all relevant state personnel receive training on web accessibility on a regular basis. In doing so, California could adopt a model similar to the federal model for 52 California State Auditor Report 2014-131 June 2015 technical assistance with Section 508 standards. At the federal level, the U.S. General Services Administration is responsible for providing federal agencies with technical assistance regarding Section 508 requirements. If the Legislature required training for state employees involved in developing department websites and web-based services, it could assign CalTech the responsibility for coordinating and offering the training sessions. CalTech is in a unique position to coordinate and offer training, as it currently publishes guidance on how to incorporate Section 508 standards into the development cycle of an information technology project. Training could reinforce the importance of addressing accessibility requirements during key stages in the development of websites and web-based services and would help ensure that state employees receive training on current best practices in web accessibility. A web consulting supervisor at CalTech stated that the training that CalTech provides on web accessibility is limited to hosting Rehabilitation subject-matter experts, which CalTech does at some quarterly webmaster group meetings and on an ad hoc basis. Rehabilitation’s chief information officer indicated that his department would welcome the opportunity, with sufficient resources, to provide regular web accessibility training to state personnel. Although CalTech and Rehabilitation could partner to offer such training in the future, CalTech is California’s lead agency for information technology and a logical department to take the lead in coordinating and offering web accessibility training in consultation with other departments. Although the website it maintains Further, although the website it maintains for state web for state web professionals includes professionals includes information on web accessibility, it has been information on web accessibility, almost five years since CalTech issued its policy letter concerning it has been almost five years since accessible websites in July 2010. The issues we observed at the CalTech issued its policy letter four departments we reviewed indicate that all state departments concerning accessible websites would benefit from a reminder from CalTech about the State’s in July 2010. web accessibility requirements. Those issues are likely to exist at a number of other state departments, and thus it would be beneficial to remind all state departments of their responsibilities in this area. The web services manager at CalTech agreed that there is value in reminding departments about their obligations. Some Departments Could Enhance Their Websites’ Accessibility by Listing Additional Usability Features Popular web browsers offer standard features that enhance users’ ability to access and navigate websites. Generally, these features allow users to take actions such as magnifying a website’s content; adjusting the font, size, and color of a website’s text; and more quickly navigating within and among web pages using the keyboard. These specific features are in some cases related to, but distinct California State Auditor Report 2014-131 53 June 2015 from, steps departments must take to ensure that their websites comply with California’s web accessibility standards; rather, they are built into web browser software and are not specific to individual websites. Users with disabilities may not always be aware of the availability of these features or how to use them. Despite the value these features can have for users, only one of the departments we visited, Covered California, had taken steps to describe these features on its website before our review. Covered California’s website also provides links directing users to additional information that is specific to individual web browsers, such as Apple Safari, Google Chrome, Microsoft Internet Explorer, and Mozilla Firefox (Firefox), which are four very popular web browsers. Covered California posted these features after our audit began but as part of a planned update to its website that the department completed in November 2014. At the time we began our audit in November 2014, none of the At the time we began our audit other three departments’ websites included information on in November 2014, none of the standard accessibility features or links to more specific browser websites for the three departments features. At that time, Franchise Tax Board and CalHR had other than Covered California accessibility pages on their websites stating that the departments included information on standard intend to comply with accessibility requirements, but they did accessibility features or links to not list any additional accessibility features like those previously more specific browser features. described. Community Colleges’ online application included a technical support page, but that page did not include any information specific to website accessibility. At the time, CalHR’s web manager and the executive director at the Community Colleges technology center both stated that their respective sites did not list browser usability features because those features are not unique to the sites. According to Franchise Tax Board’s accessibility lead analyst, these features are not specific to the Franchise Tax Board’s website, and the main focus has been on the site’s web content and online applications, given the limited resources available. Adding information on these browser features to state websites is a simple and low-cost way to increase the accessibility of those sites. As a result of our conversations with the departments about this issue, CalHR and Franchise Tax Board have already updated their websites to include standard keyboard navigation features as well as links to browser-specific information on additional features. When we asked Franchise Tax Board about these features, the accessibility lead analyst stated that adding this information was something she could quickly implement. Community Colleges has not made these additions to its online application despite having already developed a replacement for its current help page that includes all of this information. According to the chief technology officer for Community Colleges’ technology center, the technology center prefers to package changes to the website into larger releases rather 54 California State Auditor Report 2014-131 June 2015 than making small and frequent changes. He stated that the new version of the website’s help page was scheduled to be released in late May 2015. Because providing this information could benefit users of all state websites, we believe CalTech could enhance the overall accessibility of California’s online services by posting information on these browser features on the website it operates as a resource for state web professionals and adopting a policy that requires all state departments to feature this information on their websites. CalTech’s chief deputy director of policy stated that CalTech could accomplish this easily and efficiently. Oversight of Web Accessibility Testing Is Necessary to Help Ensure That California State Websites Are Accessible to Persons With Disabilities California needs better guidance and oversight to ensure that departments adequately test their websites for accessibility and thereby maintain websites that are accessible to users with disabilities. As discussed earlier in this chapter, three of the four departments we reviewed did not always regularly test their websites to ensure that updates they made to those sites were accessible, and they also used inconsistent methods for testing. Moreover, in some cases we found critical accessibility errors on updated portions of departments’ websites that the departments did not thoroughly test before they released those updates to the Shortcomings in departments’ public. These shortcomings in departments’ approaches to approaches to accessibility testing accessibility testing demonstrate a need for a statewide formalized demonstrate a need for a statewide testing approach. Although CalTech has issued general guidance to formalized testing approach. departments that outlines a broad approach to accessibility testing, it can go further in its direction to departments. Specifically, it could issue a policy that sets a minimum standard for the testing approach used for all state websites. Adopting a policy to require a standard testing approach would align California’s approach to web accessibility with a recommendation made by the IOUCA and with the World Wide Web Consortium’s (W3C) suggested testing approach. When it issued its recommendations for improving web accessibility in California in June 2006, the IOUCA recommended that a centralized body explore the use of a single testing tool that could be used across departments. The IOUCA stated that such a tool would likely boost compliance as well as provide a centralized method of reviewing accessibility and holding departments accountable. Further, the W3C recommends that organizations establish an accessibility support baseline, or minimum combination of operating systems, browsers, and assistive California State Auditor Report 2014-131 55 June 2015 technologies that is expected to work with their websites, when evaluating the accessibility of the websites. An effective accessibility support baseline would include various forms of assistive technology to ensure that testing would consider the needs of users with a range of disabilities. Given that three of the four departments we reviewed did not regularly test their websites for accessibility, we believe California We believe California would benefit would benefit from greater oversight of the accessibility testing from greater oversight of the conducted by all state governmental entities, including those not accessibility testing conducted by currently under the direction of CalTech. As the lead technology all state governmental entities, agency for the State, CalTech is the logical state entity to take on including those not currently under this responsibility. To facilitate CalTech’s ability to conduct such the direction of CalTech. oversight, under this approach, state governmental entities should be required to report to CalTech about their accessibility testing methods and the frequency of their testing. These reports to CalTech should include certifications of the testing signed by the department’s highest-ranking technology officer and supporting documentation to demonstrate that the testing occurred and what the department did or plans to do about any problems it identified during testing. Once it had received these certifications, CalTech could then make a determination about the sufficiency of each department’s testing approach as well as the department’s approach to fixing identified issues and publish the results of its review online. Any such determination should be made against the criteria found in the statewide accessibility testing policy discussed earlier. If CalTech publicized its assessment of each department’s accessibility testing approach, departments would be motivated to follow that policy. According to CalTech’s deputy director of policy, CalTech does not currently have adequate resources to provide web accessibility training to relevant staff throughout the State, develop a standard testing approach that all government entities should use when testing for web accessibility, and assess the sufficiency of accessibility testing approaches. However, we believe that increased and standardized web accessibility testing across all state departments in combination with required training on web accessibility would increase the likelihood that state websites would be accessible to persons with disabilities who attempt to access critical information and services through state websites. Recommendations Legislature To ensure that state governmental entities have a clearly identified resource for web accessibility training, the Legislature should amend state law to name CalTech as the lead agency responsible 56 California State Auditor Report 2014-131 June 2015 for providing training to state governmental entities on web accessibility issues, in consultation with Rehabilitation and other state departments as it determines necessary. To ensure that governmental entity personnel have the information and tools necessary to develop and maintain accessible websites, the Legislature should require governmental entities to provide or obtain web accessibility training at least once every three years for staff involved in the procurement or development of websites or web-based services. To help ensure that all state governmental entities appropriately test their websites for accessibility, the Legislature should direct all state governmental entities to report every other year to CalTech regarding the frequency and method of their web accessibility testing and their efforts to resolve accessibility issues they identify. Such reporting should include signed certifications from the highest-ranking technology officer at the governmental entity and documentation that supports the claimed testing as well as the entity’s effort to fix identified issues. Further, the Legislature should direct CalTech to assess the sufficiency of each governmental entity’s testing and remediation approach and publicize the results of its review online. Departments To ensure that updates to their websites are tested for accessibility, by July 31, 2015, Covered California, Community Colleges, and CalHR should develop and follow written test approaches that describe how and when changes to their websites will be reviewed. These plans should describe how the departments will include both automated and manual forms of accessibility testing. To ensure that it can accurately track whether accessibility issues found during testing have been resolved, Community Colleges should direct its vendor to more clearly and consistently document when the fix for an accessibility defect has been implemented in the live version of the online application. To ensure that it can adequately track the results of its accessibility testing, by July 31, 2015, CalHR should develop tracking tools that will allow it to document its testing efforts. At a minimum, these tools should track what portions of its jobs site were tested, what errors were found, and whether and when those errors were addressed. To ensure that individuals have a wider variety of contact information available to them for reporting problems with website accessibility, by July 31, 2015, Community Colleges and California State Auditor Report 2014-131 57 June 2015 Covered California should update their accessibility pages to include all methods of communication that state requirements mandate for other departments. To ensure that it appropriately addresses any complaints it receives related to web accessibility, Covered California should develop procedures to regularly review the complaints it receives at its accessibility email address and address any web accessibility complaints in a timely fashion. So that complaints from the public do not go unaddressed, by July 31, 2015, CalHR should develop procedures for addressing complaints about the accessibility of its website and methods for tracking the complaints it receives and their resolution. To ensure that all state departments are reminded about web accessibility requirements and best practices, by July 31, 2015, CalTech should issue an official reminder that directs state departments to key policy documents and the SIMM for additional information about how to meet their obligation to provide accessible websites. To enhance the overall accessibility of its website, by July 31, 2015, Community Colleges should list general web browser usability features on its websites after verifying that its sites are compatible with those features. Additionally, Community Colleges should add links to its website directing users to browser-specific usability information for these four popular web browsers: Apple Safari, Google Chrome, Microsoft Internet Explorer, and Firefox. To help state websites achieve a higher degree of overall accessibility, by July 31, 2015, CalTech should post standard browser usability features and links to further information on its resources website, and direct all state departments to include this information on their websites. To standardize California’s approach to web accessibility testing, CalTech should issue a policy that specifies the method by which state departments should conduct web accessibility testing. This policy should include information about a minimum combination of operating systems, browsers, and assistive technologies that should be used during testing. 58 California State Auditor Report 2014-131 June 2015 We conducted this audit under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives specified in the scope section of the report. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor Date: June 2, 2015 Staff: John Billington, Audit Principal Bob Harris, MPP Michaela Kretzner, MPP Mark Reinardy, MPP Julie Nguyen Web Accessibility Consultant: Deque Systems, Inc. Legal Counsel: Stephanie Ramirez-Ridgeway, Sr. Legal Counsel For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. California State Auditor Report 2014-131 59 June 2015 STATE OF CALIFORNIA BRICE W. HARRIS, CHANCELLOR CALIFORNIA COMMUNITY COLLEGES CHANCELLOR’S OFFICE 1102QSTREET SACRAMENTO,CA 95811 (916)322-4005 http://www.cccco.edu May 7, 2015 Elaine M. Howle* California State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 Dear Ms. Howle: Thank you for the opportunity to respond to the draft audit report. The California Community Colleges Chancellor’s Office welcomes the oversight of your office and is committed to ensuring that our web sites fully comply with accessibility requirements so that we can continue to offer students and the public high levels of service. The California Community Colleges Chancellor’s Office agrees with all findings, and we have taken or will take the following actions: • We will follow a written test approach that describes how and when changes to our web sites will be reviewed. We are in the process of contracting with WebAIM, a well-respected third-party accessibility expert organization based at Utah State University to perform full testing before each release. We have developed reusable test cases to validate that all issues found in the audit have been addressed. • We have made improvements to our processes and systems to more clearly andconsistentlytrack accessibility defects found in testing and user reported issues from initial report through repair and release. • We have developed an Accessibility Page that was reviewed by the State Auditor’sOffice with 1 contact information for persons with disabilities to report problems and browser-specific usability information. This page willbe released in May 2015. • In addition we have hired an accessibility consultant and have included a full-time Accessibility Manager position into our web development organization to ensure that we are following best practices as well as state and federal requirements and guidelines. Thank you once again for your office’s findings and recommendations. Please do not hesitate to call me if you have any questions. Sincerely, Erik Skinner Deputy Chancellor * California State Auditor’s comment appears on page 61. 60 California State Auditor Report 2014-131 June 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-131 61 June 2015 Comment CALIFORNIA STATE AUDITOR’S COMMENT ON THE RESPONSE FROM CALIFORNIA COMMUNITY COLLEGES To provide clarity and perspective, we are commenting on California Community Colleges’ (Community Colleges) response to our audit. The number below corresponds to the number we have placed in the margin of Community Colleges’ response. 1 In April 2015 Community Colleges shared with us a draft version of an accessibility page that, at the time, it planned to release publicly in May 2015. However, after we reviewed the draft page, we informed Community Colleges that it did not include one form of contact information that best practices indicate should be included on state websites so that persons with disabilities can report web accessibility issues they encounter. Specifically, the draft page did not include a teletypewriter number as a form of contact information. We expect that Community Colleges will inform us in its 60-day response to this audit of its plan to include this contact information on its new accessibility page. 62 California State Auditor Report 2014-131 June 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-131 63 June 2015 64 California State Auditor Report 2014-131 June 2015 California State Auditor Report 2014-131 65 June 2015 66 California State Auditor Report 2014-131 June 2015 California State Auditor Report 2014-131 67 June 2015 * * California State Auditor’s comments appear on page 71. 68 California State Auditor Report 2014-131 June 2015 1 California State Auditor Report 2014-131 69 June 2015 2 70 California State Auditor Report 2014-131 June 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-131 71 June 2015 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM THE CALIFORNIA DEPARTMENT OF TECHNOLOGY To provide clarity and perspective, we are commenting on the California Department of Technology’s (CalTech) response to our audit. The numbers below correspond to the numbers we have placed in the margin of CalTech’s response. 1 Throughout its response, CalTech makes statements that indicate that it believes the California Department of Rehabilitation (Rehabilitation) is the lead department in the State for matters related to web accessibility. As we state on pages 9 and 10, the state law that requires government entities to maintain their websites in accordance with the Section 508 standards does not establish a department specifically responsible for statewide enforcement or oversight of these standards. However, state law does assign CalTech the general responsibility for information technology oversight and enforcement. Therefore, in our recommendations related to greater oversight of accessibility testing, increased training regarding web accessibility, posting specific usability features to all state department websites, and developing a standardized approach to web accessibility testing, we identified CalTech as the logical department that should be responsible for such activities. As we indicate in our recommendation for increased web accessibility training on pages 55 and 56, CalTech may need to partner with other departments such as Rehabilitation. 2 As we state in our report on page 54, the World Wide Web Consortium, which is the organization that developed the Web Content Accessibility Guidelines California first adopted in 2006, recommends that organizations establish an accessibility support baseline, or minimum combination of operating systems, browsers, and assistive technologies that is expected to work with a website. Therefore, on page 57 we recommend that CalTech issue a policy that specifies the method by which state departments should conduct web accessibility testing, including information about the minimum combination of operating systems, browsers, and assistive technologies to be used in such testing. 72 California State Auditor Report 2014-131 June 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-131 73 June 2015 * 1 2 * California State Auditor’s comments appear on page 77. 74 California State Auditor Report 2014-131 June 2015 3 4 California State Auditor Report 2014-131 75 June 2015 76 California State Auditor Report 2014-131 June 2015 Blank page inserted for reproduction purposes only. California State Auditor Report 2014-131 77 June 2015 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM COVERED CALIFORNIA To provide clarity and perspective, we are commenting on Covered California’s response to our audit. The numbers below correspond to the numbers we have placed in the margin of Covered California’s response. 1 Covered California’s statement that Section 508 issues identified by testers of early releases of the California Healthcare Eligibility, Enrollment, and Retention System (CalHEERS) were fixed prior to use by the public is incorrect. As we state on page 42 of our report, not all of the accessibility defects that Covered California identified before its first major release of CalHEERS were fixed before that release went public. 2 It is unclear to us how Covered California could know that the issues we found during our review were only recently introduced. As we discuss on page 39 of our report, Covered California did not conduct dedicated web accessibility testing after the first major release of CalHEERS to the public in October 2013. Since that time, Covered California has released several updates to CalHEERS without testing the updates for accessibility. Therefore, it is entirely possible that some of the web accessibility problems we identified had been introduced to the public as early as 2013. 3 As of the date of its response, we had not received a request from Covered California for additional information related to our review. On May 9, 2015, three days after the date of its response to our draft report, we did receive a request for additional information from Covered California. We then worked with Covered California to provide additional information about the results of our review as appropriate. 4 Although we are pleased to see that Covered California plans to identify web accessibility training opportunities and make them available to its staff, we make no recommendation to Covered California regarding increased training on web accessibility. On pages 55 and 56 in our report, we recommend that the Legislature require governmental entities provide or obtain web accessibility training for their staff, and we also recommend that the California Department of Technology be assigned the responsibility for providing this training. Our report includes information on page 51 about the absence of staff training at Covered California to support our conclusion that increased training statewide is merited. 78 California State Auditor Report 2014-131 June 2015 Blank page inserted for reproduction purposes only. CalCifaolrinfoiarn Sitaa Steta Ateu dAiutodrit Roer pRoerpto 2r0t 1240-1143-1131 7979 JunJue n2e0 125015 * * California State Auditor’s comments begin on page 83. 80 California State Auditor Report 2014-131 June 2015 chair Betty T. Yee m ember Jerome E. Horton member Michael Cohen 05.06.15 To: Elaine M. Howle, State Auditor California State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 From: Selvi Stanislaus Subject: Response to California State Auditor’s Draft Report: Section 508 Compliance Audit Memorandum Thank you for the opportunity to respond to the California State Auditor’s (state auditor) Report No. 2014-131, which addresses recommendations to the Franchise Tax Board (FTB) resulting from its audit of compliance with electronic and information technology accessibility requirements (specifically regarding CalFile, our web-based tax return filing service). FTB’s Response: FTB agrees with the state auditor’s recommendations and looks forward to addressing them. We are very committed to making our website as accessible as possible for California taxpayers and welcome recommendations that further that end. 1 We do feel, however, that the state auditor’s report does not provide an accurate picture of the overall level of accessibility that we have achieved and the effort that has accompanied that over recent years. FTB has a strong commitment to providing services and information to help taxpayers file accurate and timely tax returns and pay the proper amount owed. We provide several services to our taxpayers to make this possible, one of them being CalFile, our web-based tax return filing service. The report should have provided greater context in relation to the utility of our CalFile service and the related application of the browser and screen reading tool that was tested for purposes of the audit. To provide context regarding our CalFile service in relation to overall tax return filing, for tax 2 year 2014, 14.9 million California income tax returns have been filed through early May, with 13.3 million electronically filed. Of the 13.3 million electronically filed returns, 240,000 3 returns were filed using CalFile. This represents less than two percent of both electronically filed tax returns and total tax returns. Executive Office MS A390 tel 916.845.4543 PO Box 115 fax 916.845.3191 Rancho Cordova CA 95741-0115 ftb.ca.gov California State Auditor Report 2014-131 81 June 2015 05.06.15 Section 508 Compliance Audit/Response to CSA Draft Report Page 2 FTB customers use four main browsers to access our website: Google Chrome (38%), 2 Microsoft Internet Explorer (27%), Safari (20%) and Firefox (14%). As indicated in the state auditor’s report, the Firefox web browser is used by more than 500 million people worldwide, however, it is used by only 14% of our customers and its use is trending downwards, which is parallel to global trends. The state auditor conducted its review using NonVisual Desktop Access (NVDA), which is a free screen reader most commonly used by approximately 19%1 of individuals who use screen readers to access web content. FTB tests for website accessibility using Job Access With Speech (JAWS), most commonly used by 50% of individuals using screen readers (Ibid). We test JAWS against most commonly used browsers by screen reader users, including Firefox. Those testing results show CalFile achieving an accessibility rate of 94%. 4 The state auditor’s testing combination, only, of the Firefox web browser with NVDA, does 5 not, in our opinion, provide an accurate context of FTB’s overall CalFile (or website in general) accessibility levels. In fact, FTB’s statistical data estimates that the combination of NVDA and 2 Firefox is used by less than two dozen CalFile customers, who using another browser or screen reader (approximately two-thirds of screen reader users use multiple readers (Ibid)) would not experience the critical NVDA/Firefox errors noted in the state auditor’s report. 6 Therefore, although the audit report portrays FTB’s accessibility picture somewhat negatively, the actual numbers are not nearly as egregious as portrayed. 7 FTB puts in a tremendous amount of effort into our ongoing commitment to accessibility. Each year, on a year-round basis, we devote in excess of 3,000 hours to accessibility testing and improvements. We have dedicated numerous resources, program structure and oversight towards improving accessibility and ensuring individuals with impairment challenges can easily file their returns and pay their tax obligations. Accessibility is a core component within our Software Development Life-Cycle. In attempts to make all FTB web services accessible to persons with disabilities, FTB also strives to maximize the effectiveness of its accessibility resources by prioritizing its efforts and making the most frequently used services the most accessible. We are proactive in our accessibility testing and are continuously logging, prioritizing, and fixing any identified errors. Recommendation #1: Ensure keyboard-only users that navigate the Internet using the Firefox 8 browser are able to select a security image when registering to use the CalFile application in order to file their return online. FTB’s Response #1: We are actively working on evaluating a solution to enable keyboard-only users the ability to register for CalFile when utilizing the Firefox browser. Part of our evaluation is determining whether the inability to select a security image in Firefox is a browser issue for which we can’t overcome. This issue occurs in specific browsers and not in others. FTB customers use four main browsers to access our website: Google Chrome (38%), 2 Microsoft Internet Explorer (27%), Safari (20%) and Firefox (14%). FTB supports these website browsers that keyboard-only users can utilize. Also, our website directs users that experience issues while using the latest version of any browser to try one of the other browsers we support. 1 WebAIM Screen Reader User Survey #5, January 2014. 82 California State Auditor Report 2014-131 June 2015 05.06.15 Section 508 Compliance Audit/Response to CSA Draft Report Page 3 8 Recommendation #2: Ensure the PDF file that summarizes return information that has been entered into CalFile is fully accessible to visually impaired users so the information can be verified prior to submission of their online return. FTB’s Response #2: We are actively pursuing a PDF solution and plan to have a solution in place by the end of the year. Recommendation #3: Develop a plan to determine whether the accessibility issues identified in the audit exist on other portions of FTB’s online presence that were not included in the scope of the review. FTB’s Response #3: We will continue to assess accessibility gaps across our online services and update our prioritized plan by December 1, 2015, with the goal of fixing those gaps by June 1, 2016. In closing, I’d like to reiterate that FTB takes accessibility very seriously and we are constantly striving to adhere to accessibility standards and guidelines and to improve our products and services to meet the needs of our customers. Sincerely, Executive Officer cc: Marybel Batjer California State Auditor Report 2014-131 83 June 2015 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM THE STATE OF CALIFORNIA FRANCHISE TAX BOARD To provide clarity and perspective, we are commenting on the State of California Franchise Tax Board’s (Franchise Tax Board) response to our audit. The numbers below correspond to the numbers we have placed in the margin of Franchise Tax Board’s response. 1 We clearly identify in the Scope and Methodology section for Objective 5 on page 15 of our report that we reviewed a key online service at each department; we did not claim to assess the overall level of accessibility at Franchise Tax Board. Our report accurately conveys, in Table 5 on page 21, the number of web pages we reviewed at Franchise Tax Board and the number of those pages where we identified violations of web accessibility standards that Franchise Tax Board is required to follow. Further, in Table 6 on page 22, we provide an accurate count of the number and severity level of accessibility violations we found. However, as we state on page 25 of our report, it is possible that the types of violations that we describe, as well as other violations, exist in other areas of Franchise Tax Board’s online presence that we did not review. 2 We have no independent verification of the data that Franchise Tax Board presents in its response. 3 In providing context, Franchise Tax Board appears to downplay the significance of its CalFile service because, according to the data it presents, the number of taxpayers who use CalFile is relatively small. Nevertheless, regardless of how many individuals use this online service, state law and policy require this service to be accessible by persons with disabilities. Our report shows that at the time of our review, this was not always the case. 4 Franchise Tax board makes a misleading claim about the compliance with web accessibility standards of its CalFile application. Franchise Tax Board claims the CalFile application has an accessibility rate of 94 percent. This refers to the findings of accessibility reports that are completed by its internal accessibility testers that pertain to compliance with federal accessibility standards. However, these test reports do not include information about CalFile’s compliance with Web Content Accessibility Guidelines 1.0, which are standards adopted by California that Franchise Tax Board is required to follow in addition to the federal standards. Additionally, these accessibility reports show that Franchise Tax Board believes the PDF-related violation we describe on page 30 of our report is the difference between 94 percent or 100 percent compliance. However, as we 84 California State Auditor Report 2014-131 June 2015 state on that same page, the PDF file that Franchise Tax Board presents to users before they submit their tax returns is inaccessible to visually impaired users with screen readers. Therefore, despite its claim of an accessibility rate of 94 percent, visually impaired screen reader users cannot complete the process of filing tax returns using Franchise Tax Board’s product without risking submitting an inaccurate return. 5 Franchise Tax Board incorrectly states that our web accessibility consultant’s (consultant) use of Mozilla Firefox and NonVisual Desktop Access (NVDA) to assess Franchise Tax Board’s compliance with web accessibility standards provides inaccurate context about the accessibility of CalFile. According to our consultant, testing with the Job Access With Speech (JAWS) screen reader as Franchise Tax Board indicates it does would lead to an inaccurate assessment of the accessibility of CalFile. This is because JAWS sometimes takes inaccessible website code and guesses what the accessible code should have been. In addition, testing with JAWS as the primary assistive technology can lead to critical accessibility errors being missed in testing. Conversely, NVDA, the primary screen reader our consultant used, does not guess at what the website code should have contained. NVDA, more comprehensively than JAWS, identifies instances where a website is not properly designed and therefore does provide an accurate assessment of Franchise Tax Board’s compliance with California’s web accessibility standards. 6 Franchise Tax Board is incorrect when it states that if users chose different web browsers or screen readers they would not experience the critical errors we note in our report. We identified several critical violations that users would experience even if they switched browsers. For example, our consultant identified a critical violation on Franchise Tax Board’s website wherein the closed captioning button on a video player was not available and confirmed that the violation occurred in multiple browsers. Further, on page 30 of our report, we describe a problem with the PDF that Franchise Tax Board requires CalFile users to review prior to submitting their tax returns. This issue prevents visually impaired screen reader users from reviewing the PDF prior to submitting their returns. As noted on page 32, Franchise Tax Board has known about this issue for more than two years and considered it an accessibility violation that required remediation despite not using the screen reader our web accessibility consultant used to conduct its accessibility testing. 7 It is unclear what “actual numbers” Franchise Tax Board is referring to. We are very clear in our report that it contains information regarding the number of accessibility violations that occur on the web pages we reviewed. Franchise Tax Board suggests throughout its response that the violations we identified are either California State Auditor Report 2014-131 85 June 2015 not valid due to the testing tools our consultant employed or not likely to affect a large number of users. Franchise Tax Board’s effort to minimize the importance of our findings because they are only likely to affect a small number of users is puzzling given that the web accessibility standards it violated are designed to ensure website access for a minority population of users. As described in comment three, Franchise Tax Board is required to provide an accessible online product regardless of the popularity of that product. Moreover, the type of violations we found may occur on other web pages that we did not review as part of this audit. 8 Franchise Tax Board narrowed its response to our recommendations to focus only on examples of violations we describe in the report text. In doing so, it has rewritten the recommendation we made on page 35 of our report. To be clear, we recommended that Franchise Tax Board resolve all of the accessibility violations that our consultant identified, not just the violations we describe in detail in our report text. Further, we have provided Franchise Tax Board the information necessary to address all of those violations.