Consultant's Accessibility Report

Five Cedars Alternate Format Ballot Usability, Accessibility and Privacy Test Report CDL-010-UAPTR-01 Prepared for: Vendor Name Five Cedars Vendor System Alternate Format Ballot (AFB) v4.3 Prepared by: 4720 Independence St. Wheat Ridge, CO 80033 303-422-1566 www.SLICompliance.com Accredited by the Election Assistance Commission (EAC) for Selected Voting System Test Methods or Services Test Report v1.1 Page 1 of 21 Report Number CFC-17018-STR-01 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report Copyright 2017 by SLI ComplianceSM, a Division of Gaming Laboratories International, LLC Revision History Date Release Author Revision Summary 8.28.17 1.0 M. Santos Initial Release 8.28.17 1.1 M. Santos Updates for CASOS comments Disclaimer The information reported herein must not be used by the client to claim product certification, approval, or endorsement by NVLAP, NIST, or any agency of the Federal Government. Trademarks • SLI is a registered trademark of SLI Compliance. • All products and company names are used for identification purposes only and may be trademarks of their respective owners. Page 2 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report TABLE OF CONTENTS OVERVIEW ................................................................................................................................................... 4 PHASE I - DOCUMENTATION REVIEW...................................................................................................... 4 PHASE II - USABILITY & ACCESSIBILITY TESTING ................................................................................ 5 USABILITY .................................................................................................................................................... 5 ACCESSIBILITY ........................................................................................................................................... 14 PHASE III - PRIVACY TESTING ................................................................................................................ 19 SUMMARY .................................................................................................................................................. 21 Page 3 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report Overview Testing was divided into three phases. Phase I includes review of all pertinent documents as an inspection of the manufacturer’s documentation of usability and accessibility testing performed during system development. Phase II includes all Usability and Accessibility testing. Phase III includes Privacy testing. This testing will verify California Election Code specific requirements, • The RAVBM shall not have the capability, including an optional capability, to use a remote server to mark a voter’s selections transmitted to the server from the voter’s computer via the Internet. [EC19295(a)] • The RAVBM shall not have the capability, including an optional capability, to store any voter identifiable selections on any remote server. [EC19295(b)] • The RAVBM shall not have the capability, including the optional capability, to tabulate votes. [EC19295(c)] The Accessibility, Usability and Privacy testing reviewed the Five Cedars Alternate Format Ballot system against relevant California Election Code requirements and other relevant standards. The California Voting Systems Standards (CVSS) were written in such a way to be applicable to a wide variety of voting technology. Therefore, the relevant portions of the CVSS were reviewed as they relate to the Remote Accessible Vote by Mail (RAVBM) for purposes of this report. The use of “voting system” shall apply to the RAVBM system. Phase I - Documentation Review In this phase, documentation was reviewed to verify and validate the following: • Review of the manufacturer’s documentation of usability and accessibility testing performed during system development. Review of the Technical Data Package (TDP) validated that the requirement was satisfactorily covered. Page 4 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report Phase II - Usability & Accessibility Testing In this phase, functional tests were exercised in order to verify and validate the requirements noted in the following sections. The tools listed below were utilized during testing in Phase II. • Wave Web Accessibility evaluation tool o is a tool to evaluate accessibility of Web sites, to WCAG 2.0 and section 508. o The manufacturer provided url’s were run through MAUVE to help determine accessibility • Evaluera website o Is a tool to evaluate web applications regarding conformance to the (WCAG) 2.0. o The manufacturer provided url’s were run through 508 Checker to help determine compliance. • A11y.css o Is a tool to evaluate web applications regarding conformance to ISO 9241-171: Ergonomics of human-system interaction - Guidance on software accessibility. o The manufacturer provided url’s were run with A11y to help determine compliance. • ChromeVox o Is a screen reader that is an extension of Google Chrome, to assist visually impaired users. Usability The following CVSS aspects of the voting system were evaluated: • The voting system shall allow the voter, at the voter’s choice, to submit an undervoted ballot without correction. (3.2.2.2) • Review of the requirement validated that the requirement was satisfactorily covered. Page 5 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report • The voting system shall provide the voter the opportunity to correct the ballot for an undervote before the ballot is cast and counted. (3.2.2.3) • Review of the requirement validated that the requirement was satisfactorily covered. • VEBD shall prevent voters from selecting more than the allowable number of choices for each contest. (3.2.2.4) • Review of the requirement showed that while the ballot informs the voter that they have overvoted a contest and that their votes won’t count, the voter is allowed to overvote the contest if they choose to do so. • VEBD shall provide feedback to the voter, before final casting of the ballot that identifies specific contests for which the voter has selected fewer than the allowable number of choices (i.e., undervotes). This feature shall not be disabled. (3.2.2.6) • Review of the requirement validated that the requirement was satisfactorily covered. • VEBD shall provide the voter the opportunity to correct the ballot before it is cast and counted. This correction process shall not require external assistance. The corrections to be supported include modifying an undervote and changing a vote from one candidate to another. (3.2.2.7) • Review of the requirement validated that the requirement was satisfactorily covered. • VEBD shall allow the voter to change a vote within a contest before advancing to the next contest. (3.2.2.8) • Review of the requirement validated that the requirement was satisfactorily covered. • VEBD shall provide navigation controls that allow the voter to advance to the next contest or go back to the previous contest before completing a vote on the contest(s) currently being presented (whether visually or aurally). (3.2.2.9) • Review of the requirement validated that the requirement was satisfactorily covered. Page 6 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report The features specified in this CVSS section are intended to minimize cognitive difficulties for voters. a. Completeness of instructions - The voting system shall provide instructions for all operations inherent to the voting system or that are generated by default. Instructions that are part of a ballot definition are not subject to this requirement. (3.2.5.a) • Review of the requirement validated that the requirement was satisfactorily covered. Instructions needed for a RVABMS implementation are sufficient. b. Availability of assistance from the system - The voting system shall provide a means for the voter to get help directly from the system at any time during the voting session. (3.2.5.b) • Review of the requirement validated that the requirement was satisfactorily covered. Assistance needed for a RVABMS implementation are sufficient. c. Plain language – Operational instructions and voting system help material shall conform to norms and best practices for plain language. (3.2.5.c) i. Clarity of warnings - Warnings and alerts issued by the voting system shall clearly state: o The nature of the problem; (3.2.5.c.i.1) o Whether the voter has performed or attempted an invalid operation or whether the voting equipment itself has malfunctioned in some way; and (3.2.5.c.i.2) o The set of responses available to the voter. (3.2.5.c.i.3) • Review of the requirement validated that the requirement was satisfactorily covered. Language and clarity of warnings needed for a RVABMS implementation are sufficient. ii. Context before action - When an instruction is based on a condition, the condition shall be stated first, and then the action to be performed. (3.2.5.c.ii) • Review of the requirement validated that the requirement was satisfactorily covered. iii. Start each instruction on a new line - The system shall start the visual presentation of each new instruction on a new line. (3.2.5.c.iii) Page 7 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report • Review of the requirement validated that the requirement was satisfactorily covered. iv. Use of positive - The system shall issue instructions on the correct way to perform actions, rather than telling voters what not to do. (3.2.5.c.iv) • Review of the requirement validated that the requirement was satisfactorily covered. v. Use of imperative voice - The system's instructions shall address the voter directly rather than use passive voice constructions. (3.2.5.c.v) • Review of the requirement validated that the requirement was satisfactorily covered. vi. Gender-based pronouns - The system shall avoid the use of gender-based pronouns. (3.2.5.c.vi) • Review of the requirement validated that the requirement was satisfactorily covered. d. No bias among choices - Consistent with the California Elections Code, the voting system shall support a process that does not introduce bias for or against any of the contest choices to be presented to the voter. In both visual and aural formats, the choices shall be presented in an equivalent manner. (3.2.5.d) • Review of the requirement validated that the requirement was satisfactorily covered. e. Ballot design - The voting system shall provide the capability to design a ballot with a high level of clarity and comprehensibility. (3.2.5.e) i. Contests split among pages or columns - The voting system shall visually present a single contest on a single page or column except where the number of choices in a contest makes it impossible. (3.2.5.e.i) (cid:1) Review of the requirement validated that the requirement was satisfactorily covered. ii. Indicate maximum number of candidates - The voting system shall require that the ballot clearly indicate the maximum number of candidates for which one can vote within a single contest. (3.2.5.e.ii) Page 8 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report (cid:1) Review of the requirement validated that the requirement was satisfactorily covered. iii. Consistent representation of candidate selection - The relationship between the name of a candidate and the mechanism used to vote for that candidate shall be consistent throughout the ballot. (3.2.5.e.iii) (cid:1) Review of the requirement validated that the requirement was satisfactorily covered. f. Conventional Use of Color - The use of color by the voting system shall agree with common conventions: (a) green, blue or white is used for general information or as a normal status indicator; (b) amber or yellow is used to indicate warnings or a marginal status; (c) red is used to indicate error conditions or a problem requiring immediate attention. (3.2.5.f) (cid:1) Review of the requirement validated that the requirement was satisfactorily covered. g. Icons and language - When an icon is used to convey information, indicate an action, or prompt a response, it shall be accompanied by a corresponding linguistic label. (3.2.5.g) (cid:1) Review of the requirement validated that the requirement was satisfactorily covered. The requirements of this CVSS section are designed to minimize perceptual difficulties for the voter. e. Accommodation for color blindness - The default color coding shall support correct perception by voters with color blindness. (3.2.6.e) (cid:1) Review of the requirement validated that the requirement was satisfactorily covered. f. No reliance solely on color - Color coding shall not be used as the sole means of conveying information, indicating an action, prompting a response, or distinguishing a visual element. (3.2.6.f) Page 9 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report (cid:1) Review of the requirement validated that the requirement was satisfactorily covered. The requirements of this CVSS section are designed to minimize interaction difficulties for the voter. a. No page scrolling - Voting systems shall not require page scrolling by the voter. (3.2.7.a) (cid:1) Review of the requirement validated that the requirement was satisfactorily covered. Scrolling is available, though tab key and arrow key navigation is also available. b. Unambiguous feedback for voter's selection - The voting system shall provide unambiguous feedback regarding the voter’s selection, such as displaying a checkmark beside the selected option or conspicuously changing its appearance. (3.2.7.b) (cid:1) Review of the requirement validated that the requirement was satisfactorily covered. Checkmark is applied, as well as coloring of entire row a light yellow background. c. Accidental Activation - The location and sensitivity of the input mechanisms shall be designed to minimize accidental activation. i. Size and separation of touch areas (3.2.7.c.i) (cid:1) Review of the requirement validated that the requirement was satisfactorily covered. Voter will be using their own equipment, so can enlarge screen as need, if utilizing a touch screen. Navigation/selection is done per individual environment. ii. No repeating keys - No key or control on a voting system shall have a repetitive effect as a result of being held in its active position. (3.2.7.c.ii) (cid:1) Review of the requirement showed that navigation keys (tab key, tab- shift key combination, up and down arrows) did have a repetitive effect. (cid:1) AFB fails this requirement. These CVSS requirements address how long the system and voter wait for each other to interact. Page 10 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report a. Maximum initial system response time - The initial system response time of a VEBD shall be no greater than 0.5 seconds. o Review of the requirement validated that the requirement was satisfactorily covered. b. Maximum completed system response time for vote confirmation - When the voter performs an action to record a single vote, the completed system response time of the VEBD shall be no greater than one second in the case of a visual response, and no greater than five seconds in the case of an audio response. (3.2.8.b) o Review of the requirement validated that the requirement was satisfactorily covered. c. Maximum completed system response time for all operations - The completed system response time of a VEBD for visual operations shall be no greater than 10 seconds. (3.2.8.c) o Review of the requirement validated that the requirement was satisfactorily covered. d. System response indicator - If a VEBD has not completed its visual response within one second, it shall present to the voter, within 0.5 seconds of the voter's action, some indication that it is preparing its response. (3.2.8.d) o Review of the requirement validated that the requirement was satisfactorily covered. e. Voter inactivity time - The VEBD shall detect and warn about lengthy voter inactivity during a voting session. Each system shall have a defined and documented voter inactivity time, and that time shall be between two and five minutes. (3.2.8.e) o AFB fails this requirement o Review of the requirement showed that after five minutes, no inactivity warning was issued. As remote access system in an environment of the voters choosing, this may be more beneficial. f. Alert time - Upon expiration of the voter inactivity time, the voting system shall issue an alert and provide a means by which the voter may receive additional time. The alert time shall be between 20 and 45 seconds. If the voter does not respond to the alert within the alert time, the system shall go into an inactive state. (3.2.8.f) Page 11 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report o AFB fails this requirement. Review of the requirement showed that after five minutes, no alert was issued, nor did the system convert to an inactive state. As remote access system in an environment of the voters choosing, this may be more beneficial. Languages a. General Support for Alternative Languages - The voting system shall be capable of presenting the ballot, contest choices, review screens, vote verification records, and voting instructions in any language that Elections Code section 14201 or the Section 203 of the federal Voting Rights Act requires in any California jurisdiction. i. Voter control of language - A VEBD shall allow the voter to select among the available languages throughout the voting session while preserving the current votes. (3.2.9.a.i) o Review of the requirement showed that English, Spanish, Hindi, Khmer, Korean, Tagalog, Thai, Vietnamese and Chinese are supported. As a RAVBMS, the user selects language, which is then downloaded. If the voter then chooses to vote a ballot in a different language, they will need to access a ballot in the new language via a different link. The ballots contain only one language.. ii. Complete information in alternative language - Information presented to the voter in the typical case of English-literate voters (including instructions, warnings, messages, contest choices, and vote verification information) shall also be presented when an alternative language is being used, whether the language is written or spoken. (3.2.9.a.ii) o Review of the requirement validated that the requirement was satisfactorily covered. iii. Auditability of records for English readers - Any records, including paper ballots and paper verification records, shall have sufficient information to support auditing by poll workers and others who can read only English. (3.2.9.a.iii) o Review of the requirement validated that the requirement was satisfactorily covered. Page 12 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report iv. Usability testing by S-ATA for alternative languages - The S-ATA shall conduct summative usability tests for each of the system's supported languages, using subjects who are fluent in those languages but not fluent in English and shall report the test results, using the Common Industry Format. (3.2.9.a.iv) o Review of the requirement validated that the requirement was satisfactorily covered. Note that internal testing was performed on this system. Testing verified that the voting system is designed to achieve high usability for voters in the following ways: Voter Effectiveness • Verify voting devices are designed to allow voters to accurately cast a ballot that will be counted as their selections are intended. • Verify voting devices incorporate best practices for user interface design in order to reduce undervotes. o Review of the requirements validated that these requirements were satisfactorily covered. “Check ballot for mistakes” and “Summary” pages reduce opportunity for un-intended undervotes (and overvotes). Voter Efficiency • Verify voting device interfaces are designed to support a voter’s ability to accomplish completion of their ballot such that they are confident that his/her ballot will be counted as intended. • Verify voting devices are designed to minimize the number of interactions that are required to complete typical actions during a voting session. o Review of the requirements validated that these requirements were satisfactorily covered. Voter Satisfaction • Verify voting systems are designed to support a voting process that is easy and intuitive. • Verify voting systems interfaces make it clear to the voter how to begin the voting process. • Verify voting systems interfaces make it unequivocally clear to the voter whether a ballot has been successfully cast or not. Page 13 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report • Verify voting systems do not overload voters with excessive information at any one time. • Verify voting systems offer voters indicators of the overall progress in the voting session, to make it clear what steps have already been completed, and what steps remain before completion. • Verify voting systems make it clear when the voter has completed each step or task in the voting process. • Verify voting systems allow users to easily find the races they wish to vote on. • Verify voting systems minimize the presentation of extraneous information in the visual or audio interface. Review of the requirements validated that these requirements were satisfactorily covered. Accessibility The CVSS requirements of this section are relevant to a wide variety of disabilities. a. Accessibility throughout the voting session - A VEBD shall be integrated into the manufacturer’s complete voting system so as to support accessibility for disabled voters throughout the voting session. i. Documentation of Accessibility Procedures - The manufacturer shall supply documentation describing: o recommended procedures that fully implement accessibility for voters with disabilities; and (3.3.1.a.i.1) o how a VEBD supports those procedures. (3.3.1.a.i.2) • Review of the requirement validated that the requirement was satisfactorily covered. b. Complete information in alternative formats - When the provision of accessibility involves an alternative format for ballot presentation, then all information presented to non-disabled voters, including instructions, warnings, error and other messages, and contest choices, shall be presented in that alternative format. (3.3.1.b) • Review of the requirement validated that the requirement was satisfactorily covered. c. Secondary means of voter identification - If a voting system provides for voter identification or authentication by using biometric measures that require a voter to possess particular biological characteristics, then the Page 14 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report system shall provide a secondary means that does not depend on those characteristics. (3.3.1.d) • Review of the requirement showed that no biometric measures are required, so this requirement is not applicable. d. Accessibility of paper-based vote verification - If a VEBD generates a paper record (or some other durable, human-readable record) for the purpose of allowing voters to verify their votes, then the system shall provide a means to ensure that the verification record is accessible to all voters with disabilities. i. Audio readback for paper-based vote verification - If a VEBD generates a paper record (or some other durable, human-readable record) for the purpose of allowing voters to verify their votes, then the system shall provide a mechanism that can read that record and generate an audio representation of its contents. (3.3.1.e.i) • If a voter has equipment that utilizes either OCR technology, or a QR reader with audio playback, then they will be able to obtain an audio read back of their printed ballot. Note that the system itself does not provide feedback on the marked, printed ballot, once the voter has printed it out. These CVSS requirements specify the features of the accessible voting system designed to assist voters with low vision. a) Usability testing by S-ATA for voters with low vision - The S-ATA shall conduct summative usability tests on the voting system using individuals with low vision and shall report the test results, using the Common Industry Format. (3.3.2.a) (cid:1) Voter will implement their own hardware, which will determine visual usability. Standard equipment provides sufficient visual properties. b) Adjustable saturation for color displays - An accessible voting station with a color electronic image display shall allow the voter to adjust the color saturation throughout the voting session while preserving the current votes. At least two options shall be available: a high and a low saturation presentation. (3.3.2.b) a. Voter will implement their own hardware, which will determine color saturation. Standard equipment provides sufficient color saturation. c) Distinctive Buttons and Controls - Buttons and controls on accessible voting stations shall be distinguishable by both shape and color. This applies to buttons and controls implemented either "on-screen" or in hardware. This Page 15 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report requirement does not apply to sizeable groups of keys, such as a conventional 4x3 telephone keypad or a full alphabetic keyboard. (3.3.2.c) • Voter will implement their own hardware, and will determine what types of buttons and controls are used. d) Synchronized audio and video - The voting station shall provide synchronized audio output to convey the same information as that which is displayed on the screen. There shall be a means by which the voter can disable either the audio or the video output, resulting in a video-only or audio-only presentation, respectively. The system shall allow the voter to switch among the three modes (synchronized audio/video, video-only, or audio-only) throughout the voting session while preserving the current votes. (3.3.2.d) • AFB does provide for both video and audio presentation to the voter. The voter will implement their own hardware. Synchronized audio/video, video-only, or audio only are all obtainable. If voter environment contains a screen, audio output and a screen reader, synchronized output is available. Voter can turn off audio to have video only. Likewise, voter can turn off video display to have audio only These CVSS requirements specify the features of the accessible voting station designed to assist voters who are blind. a. Usability testing by S-ATA for blind voters - The S-ATA shall conduct summative usability tests on the voting system using individuals who are blind and shall report the test results, using the Common Industry Format. b. Audio-tactile interface - The accessible voting station shall provide an audio-tactile interface (ATI) that supports the full functionality of the visual ballot interface. Full functionality includes at a minimum: o Instructions and feedback on initial activation of the ballot (such as insertion of a smart card), if applicable; (3.3.3.b.1) o Instructions and feedback to the voter on how to operate the accessible voting station, including settings and options (e.g., volume control, repetition); (3.3.3.b.2) o Instructions and feedback for navigation of the ballot; (3.3.3.b.3) o Instructions and feedback for contest choices, including write-in candidates; (3.3.3.b.4) o Instructions and feedback on confirming and changing votes; (3.3.3.b.5) o Instructions and feedback on final submission of ballot. (3.3.3.b.6) Page 16 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report i. Equivalent functionality of ATI - The ATI of the accessible voting station shall provide the same capabilities to vote and cast a ballot as are provided by its visual interface. (3.3.3.b.i) ii. ATI supports repetition - The ATI shall allow the voter to have any information provided by the voting system repeated. (3.3.3.b.ii) iii. ATI supports pause and resume - The ATI shall allow the voter to pause and resume the audio presentation. (3.3.3.b.iii) iv. ATI supports transition to next or previous contest - The ATI shall allow the voter to skip to the next contest or return to previous contests. (3.3.3.b.iv) v. ATI can skip initiative or referendum wording - The ATI shall allow the voter to skip over the reading of an initiative or referendum so as to be able to vote on it immediately. (3.3.3.b.v) Review of the requirements validated that these requirements were not applicable, as voter will implement their own hardware. c. Audio features and characteristics - Voting stations that provide audio presentation of the ballot shall do so in a usable way, as detailed in the following sub-requirements. iv. Intelligible audio - The audio presentation of verbal information by both recorded and synthetic speech shall be readily comprehensible by voters who have normal hearing and are proficient in the language. This includes such characteristics as proper enunciation, normal intonation, appropriate rate of speech, and low background noise. Candidate names shall be pronounced as the candidate intends. This requirement applies to those aspects of the audio content that are inherent to the voting system or that are generated by default. (3.3.3.c.vii) o AFB does provide for intelligible audio presentation to the voter. Voter will implement their own hardware, which will dictate how that audio is heard. d. Ballot activation - If the voting station supports ballot activation for non- blind voters, then it shall also provide features that enable voters who are blind to perform this activation. (3.3.3.d) o Review of the requirement validated that the requirement was satisfactorily covered. e. Ballot submission and vote verification - If the voting station supports ballot submission or vote verification for non-blind voters, then it shall Page 17 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report also provide features that enable voters who are blind to perform these actions. (3.3.3.e) o Review of the requirement validated that the requirement was satisfactorily covered. These CVSS requirements specify the features of the accessible voting station designed to assist voters who lack fine motor control or use of their hands. a) Usability Testing by S-ATA for Voters With Dexterity Disabilities - The S- ATA shall conduct summative usability tests on the voting system using individuals lacking fine motor control and shall report the test results, using the Common Industry Format. (3.3.4.a) o This requirement is a hardware requirement that, for this type of system, is attributed to the voter. As the ballot is accessed, marked and printed by the voter in their own environment, it is the voter that will provide whatever hardware is needed for them to suitably mark and print their ballot b) Support for Non-Manual Input - The accessible voting station shall provide a mechanism to enable non-manual input that is functionally equivalent to tactile input. All the functionality of the accessible voting station (e.g., straight party voting, write-in candidates) that is available through the conventional forms of input, such as tactile, shall also be available through non-manual input mechanisms such as mouth sticks and "sip and puff" switches. (3.3.4.b) (cid:1) This requirement is a hardware requirement that, for this type of system, is attributed to the voter. As the ballot is accessed, marked and printed by the voter in their own environment, it is the voter that will provide whatever hardware is needed for them to suitably mark and print their ballot These CVSS requirements specify the features of the accessible voting station designed to assist voters with hearing disabilities. a. Reference to Audio Requirements - The accessible voting station shall incorporate the features listed under the requirements for voting equipment that provides audio presentation of the ballot. (3.3.6.a) o AFB does provide for intelligible audio presentation to the voter. Voter will implement their own hardware, which will dictate how that audio is heard. Page 18 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report b. Visual Redundancy for Sound Cues - If the voting system provides sound cues as a method to alert the voter, the tone shall be accompanied by a visual cue, unless the station is in audio-only mode. (3.3.6.b) o Review of the requirement validated that the requirement was satisfactorily covered. These CVSS requirements specify the features of the accessible voting station designed to assist voters who lack proficiency in reading English. Use of ATI- For voters who lack proficiency in reading English, the voting equipment shall provide an audio interface for instructions and ballots. (3.3.7) o Review of the requirements validated that this requirement was not applicable, as voter will implement their own hardware. Speech not to be required by equipment - Voting equipment shall not require voter speech for its operation. (3.3.8) o Review of the requirement validated that the requirement was satisfactorily covered. Phase III - Privacy Testing In this phase, functional tests were exercised in order to verify and validate the following requirements: The RAVBM shall not have the capability, including an optional capability, to use a remote server to mark a voter’s selections transmitted to the server from the voter’s computer via the Internet. [EC19295(a)] o Review of the requirement validated that the requirement was satisfactorily covered. The RAVBM shall not have the capability, including an optional capability, to store any voter identifiable selections on any remote server. [EC19295(b)] Page 19 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report o Review of the requirement validated that the requirement was satisfactorily covered. The RAVBM shall not have the capability, including the optional capability, to tabulate votes. [EC19295(c)] o Review of the requirement validated that the requirement was satisfactorily covered. Visual privacy - The ballot, any other visible record containing ballot information, and any input controls shall be visible only to the voter during the voting session and ballot submission. (3.2.4.1.a) o Review of the requirement validated that the requirement was satisfactorily covered, as the voter will utilize their own hardware in the environment of their choosing. Auditory privacy - During the voting session, the audio interface of the voting system shall be audible only to the voter. (3.2.4.1.b) o Review of the requirement validated that the requirement was satisfactorily covered, as the voter will utilize their own hardware in the environment of their choosing. Privacy of warnings - The voting system shall issue all warnings in a way that preserves the privacy of the voter and the confidentiality of the ballot. (3.2.4.1.c) o Review of the requirement validated that the requirement was satisfactorily covered, as the voter will utilize their own hardware in the environment of their choosing. No receipts - The voting system shall not issue a receipt to the voter that would provide proof to another of how the voter voted. (3.2.4.1.d) o Review of the requirements validated that this requirement was not applicable, as voter will implement their own hardware, and print their own marked ballot, as a RAVBMS system. No information shall be kept within an electronic CVR that identifies any alternative language feature(s) used by a voter. (3.2.4.2.a) Page 20 of 21 Five Cedars AFB California Certification Accessibility, Usability & Privacy Test Report o Review of the requirement validated that the requirement was satisfactorily covered, as CVR’s are not created. No information shall be kept within an electronic CVR that identifies any accessibility feature(s) used by a voter. (3.2.4.2.b) o Review of the requirement validated that the requirement was satisfactorily covered, as CVR’s are not created. Summary The AFB application is an HTML 5 SPA (Single Page Application), which means that once the initial server call for the application is processed the entire application runs in the current browser session. Voter Privacy is achieved by removing client side storage of marked selections, the voter is allowed to verify and print a ballot summary card for use in currently setup jurisdiction absentee /mail in voting programs. The system was written to WCAG 2.0 guidelines to implement accessibility features. Usability was generally met, with the exception of repetitive keys. The voter is given the ability to proof and confirm ballot selections within the AFB interactive ballot system as well as the paper ballot summary. Usability was generally met, with the exception of repetitive keys. One discrepancy finding was identified within the Five Cedars AFB RAVBMS. When a key is pressed, if held, the action is repeated multiple times. Notable also is that the AFB does not prevent overvoting. While it notifies the voter that they have overvoted a contest, the voter is allowed to continue, if they so choose. As per the direction given by the California Secretary of State, this security testing report does not include any recommendation as to whether or not the system should be approved. End of Accessibility, Usability and Privacy Test Report Page 21 of 21