Los Angeles County Vsap 3.0 Consultant’s Accessibility Testing Report

Los Angeles’ County Voting Solutions for All People (VSAP) 3.0 Voting System Accessibility, Usability, and Privacy Test Report for California Secretary of State CHI-21008-AUPTR-01 Vendor Name Los Angeles County Vendor System VSAP 3.0 Prepared by: SLI ComplianceSM 4720 Independence St. Wheat Ridge, CO 80033 303-422-1566 www.SLICompliance.com Accredited by the Election Assistance Commission (EAC) for VSTL status. Los Angeles County VSAP 3.0 California Certification Accessibility, Usability, and Privacy Test Report v2.0 Revision History Date Release Author Revisions January 28th, 2022 v1.0 M. Santos Initial Release February 8th, 2022 V2.0 M. Santos Updates for CASOS comments Disclaimer The Certification Test results reported herein must not be used by the client to claim product certification, approval, or endorsement by NVLAP, NIST, or any agency of the Federal Government. Results herein relate only to the items tested. Trademarks • SLI is a registered trademark of SLI Compliance, a Division of Gaming Laboratories International, LLC. • All other products and company names are used for identification purposes only and may be trademarks of their respective owners. The tests referenced in this document were performed in a controlled environment using specific systems and data sets and results are related to the specific items tested. Actual results in other environments may vary. Opinions and Interpretations There are no SLI opinions or interpretations included in this report. Copyright 2022 SLI ComplianceSM , a Division of Gaming Laboratories International, LLC. California Certification AUP Test Report Page 2 of 7 Report Number CA-21008-AUPTR-01 Los Angeles County VSAP 3.0 California Certification Accessibility, Usability, and Privacy Test Report v2.0 TABLE OF CONTENTS INTRODUCTION ................................................................................................................................... 4 REFERENCES....................................................................................................................................... 4 Equipment Tested .......................................................................................................................... 4 Ballot Marking Device .................................................................................................................... 4 CERTIFICATION TEST RESULTS SUMMARY ................................................................................... 4 TESTING SUMMARY .............................................................................................................................. 4 Volunteer One ................................................................................................................................ 5 Volunteer Two ................................................................................................................................ 6 FINAL CONSIDERATIONS .................................................................................................................. 7 California Certification AUP Test Report Page 3 of 7 Report Number CA-21008-AUPTR-01 Los Angeles County VSAP 3.0 California Certification Accessibility, Usability, and Privacy Test Report v2.0 INTRODUCTION SLI Compliance is submitting this test report as a summary of the certification testing efforts for the Los Angeles County’s Voting Solutions for All People 3.0 (VSAP 3.0) voting system. The purpose of this document is to provide an overview of the certification testing and the findings of the testing effort for the VSAP 3.0 voting system. This effort included accessibility, usability, and privacy testing of the VSAP 3.0 voting system’s Ballot Marking Device (BMD) component. References California Voting System Standards (CVSS) Equipment Tested The VSAP 3.0 voting system component involved in this test was the BMD. Ballot Marking Device The BMD is the primary touchpoint for the voter and the hub of the voting system, guiding users with screen prompts and symbols. The BMD features a touchscreen, an audio-tactile interface (controller and headphones), paper handler (scanner and printer), QR code scanner, and dual-switch input which voters use to generate, verify, and cast paper ballots. Completed ballots are transferred to the integrated ballot box, which can be detached for unloading. CERTIFICATION TEST RESULTS SUMMARY Testing Summary The sessions were conducted with Secretary of State personnel acting as election workers and with volunteers from the disability community voting on the VSAP 3.0 BMDs. When the volunteers arrived, they were given a briefing on the testing and the devices. The sessions were a free form, ad hoc test where the volunteers were able to vote a ballot in any manner that they chose. The volunteers provided feedback in real-time as they were voting, as well as completing an Accessibility Test survey for each device. Additionally, all observations were recorded as each volunteer navigated their way through the process. California Certification AUP Test Report Page 4 of 7 Report Number CA-21008-AUPTR-01 Los Angeles County VSAP 3.0 California Certification Accessibility, Usability, and Privacy Test Report v2.0 Volunteer One Table 1 – Volunteer One Survey Agree Agree Disagree Disagree strongly somewhat somewhat strongly The voting method was private. X I feel I can use this system to vote X independently. I am confident that my vote was X recorded accurately. The voting instructions were clear X and complete. The voting method was easy to use. X I could read the display easily. X I could understand the speech X The assistive device(s) were easy to See reach and use. Summary I found the system confusing to use. X The timeframe it took to vote was X what I expected. Volunteer One Summary Volunteer One was surveyed after the testing and responded that they were highly satisfied and would prefer using this system to vote. The tester noted the headset could not adjust to fully fit their head size and recommended headsets that could be adjusted for larger head sizes. No concerns were voiced over privacy issues. California Certification AUP Test Report Page 5 of 7 Report Number CA-21008-AUPTR-01 Los Angeles County VSAP 3.0 California Certification Accessibility, Usability, and Privacy Test Report v2.0 Volunteer Two Table 2 – Volunteer Two Survey Agree Agree Disagree Disagree strongly somewhat somewhat strongly The voting method was private. X I feel I can use this system to vote X independently. I am confident that my vote was X recorded accurately. The voting instructions were clear X and complete. The voting method was easy to use. X I could read the display easily. N/A I could understand the speech See output. Summary The assistive device(s) were easy to X reach and use. I found the system confusing to use. X The timeframe it took to vote was X what I expected. Volunteer Two Summary Volunteer Two was surveyed after the testing and responded that they would prefer using this system to vote. The tester identified that when the speech speed was adjusted, the instructions stopped reading the current field. This required the voter to cycle through the entire page again to re-listen to the current field. The tester also noted that two different voices used between instructions and candidate names was distracting and that when the rate of speech was adjusted only one voice sped up and slowed down while the other stayed at the same rate. No concerns were voiced over privacy issues. California Certification AUP Test Report Page 6 of 7 Report Number CA-21008-AUPTR-01 Los Angeles County VSAP 3.0 California Certification Accessibility, Usability, and Privacy Test Report v2.0 FINAL CONSIDERATIONS The general consensus of the volunteers was that they felt the technologies implemented for accessibility and usability improved the experience for voters that are most in need of them. No concerns were voiced over privacy issues while voting during an election. As directed by the California Secretary of State, this accessibility, usability, and privacy testing report does not include any recommendation as to whether or not the system should be approved. End of Accessibility, Usability and Privacy Test Report California Certification AUP Test Report Page 7 of 7 Report Number CA-21008-AUPTR-01