Hart Verity Voting 3.1 Voting System Accessibility and Usability Testing Report

Hart InterCivic Verity Voting 3.1 Accessibility, Usability, and Privacy Test Report for California Secretary of State CHI-19047-AUPTR-01 Vendor Name Hart InterCivic Vendor System Verity Voting 3.1 Prepared by: SLI ComplianceSM 4720 Independence St. Wheat Ridge, CO 80033 303-422-1566 www.SLICompliance.com Accredited by the Election Assistance Commission (EAC) for VSTL status. California Secretary of State Hart InterCivic Verity Voting 3.1 Accessibility, Usability & Privacy Test Report Copyright 2019 SLI ComplianceSM, a Division of Gaming Laboratories International, LLC Revision History Date Release Author Revisions 11/1/2019 v1.0 M. Santos Initial Release Disclaimer The Certification Test results reported herein must not be used by the client to claim product certification, approval, or endorsement by NVLAP, NIST, or any agency of the Federal Government. Results herein relate only to the items tested. Trademarks • SLI is a registered trademark of SLI Compliance, a Division of Gaming Laboratories International, LLC. • All other products and company names are used for identification purposes only and may be trademarks of their respective owners. The tests referenced in this document were performed in a controlled environment using specific systems and data sets and results are related to the specific items tested. Actual results in other environments may vary. Opinions and Interpretations There are no SLI opinions or interpretations included in this report. California Certification AUP Test Report v1.0 Page 2 of 8 Report Number CHI-19047-AUPTR-01 California Secretary of State Hart InterCivic Verity Voting 3.1 Accessibility, Usability & Privacy Test Report TABLE OF CONTENTS INTRODUCTION ................................................................................................................................... 4 REFERENCES....................................................................................................................................... 4 SYSTEM OVERVIEW ........................................................................................................................... 4 SCOPE OF THE HART INTERCIVIC VERITY VOTING 3.1 VOTING SYSTEM .................................................. 4 CERTIFICATION TEST RESULTS SUMMARY ................................................................................... 5 TESTING SUMMARY .............................................................................................................................. 5 Volunteer One ................................................................................................................................ 6 Volunteer Two ................................................................................................................................ 7 Volunteer Three ............................................................................................................................. 8 FINAL CONSIDERATIONS .................................................................................................................. 8 California Certification AUP Test Report v1.0 Page 3 of 8 Report Number CHI-19047-AUPTR-01 California Secretary of State Hart InterCivic Verity Voting 3.1 Accessibility, Usability & Privacy Test Report Introduction SLI Compliance is submitting this test report as a summary of the certification testing efforts for the Hart InterCivic Verity Voting 3.1 voting system. The purpose of this document is to provide an overview of the certification testing and the findings of the testing effort for the Hart InterCivic Verity Voting 3.1 voting system. This effort included Accessibility, Usability, and Privacy testing of the Hart InterCivic Verity Voting 3.1 voting system’s Verity Touch Writer, Verity Reader, and Verity Scan components. References • California Voting System Standards (CVSS). System Overview Scope of the Hart InterCivic Verity Voting 3.1 Voting System This section provides a description of the scope of Hart InterCivic Verity Voting 3.1 voting system components involved in this test: • Verity Touch Writer firmware/hardware, polling place. • Verity Reader firmware/hardware, polling place. • Verity Scan firmware/hardware, polling place. The Hart Verity Touch Writer ballot marking platform is used for the creation of paper ballots. These ballots are later scanned and tabulated by the Hart Verity Scan optical ballot counter. The Hart Verity Reader system employs a precinct-level optical scanner which is designed to scan paper ballots, interpret voting marks, and display the ballots onto a viewable tablet such that the voter is able to review and verify their choices. The Hart Verity Scan system employs a precinct-level optical scan ballot counter (tabulator) in conjunction with an external ballot box. This tabulator is designed to scan paper ballots, interpret voting marks, and deposit the ballots into the secure ballot box. California Certification AUP Test Report v1.0 Page 4 of 8 Report Number CHI-19047-AUPTR-01 California Secretary of State Hart InterCivic Verity Voting 3.1 Accessibility, Usability & Privacy Test Report Certification Test Results Summary Testing Summary For the accessibility testing, three devices were utilized in this testing phase. These devices were Verity Touch Writer, Verity Reader, and Verity Scan. The sessions were conducted with Hart personnel acting as poll workers and volunteers voting on the Hart devices. When the volunteers arrived, they were given a quick briefing on the testing and the devices. The sessions were a free form, ad hoc test where the volunteers were able to vote a ballot in any manner that they chose. The volunteers voted on the Verity Touch Writer device, read the ballot on the Verity Reader device, and scanned the ballot on the Verity Scan device. The volunteers provided feedback in real-time as they were voting, as well as completing an Accessibility Test survey for each device. Additionally, any observations were recorded as each volunteer navigated their way through the process. California Certification AUP Test Report v1.0 Page 5 of 8 Report Number CHI-19047-AUPTR-01 California Secretary of State Hart InterCivic Verity Voting 3.1 Accessibility, Usability & Privacy Test Report Volunteer One Volunteer One primarily used a combination of the Audio Tactile Interface (ATI) and headphones for the Verity Touch Writer and the Verity Reader devices. Once they completed voting the ballot and verifying their votes using the Verity Reader, they scanned the ballot through the Verity Scan. Afterwards they answered a simple survey. Results are shown in Table 1. Table 1 – Volunteer 1 Survey Volunteer One Summary Volunteer One was surveyed after the testing and responded that they felt the system was very good and they would feel comfortable using it. Volunteer One offered a suggestion to allow voting by voice, speaking to mark their choices. No concerns were voiced over privacy issues. California Certification AUP Test Report v1.0 Page 6 of 8 Report Number CHI-19047-AUPTR-01 California Secretary of State Hart InterCivic Verity Voting 3.1 Accessibility, Usability & Privacy Test Report Volunteer Two Volunteer Two primarily used the jelly switches for the Verity Touch Writer and Verity Reader devices. This voter required assistance to insert and remove the paper ballot on the three devices. Once they completed voting the ballot and verifying their votes using the Verity Reader, they scanned the ballot through Verity Scan. Afterwards, they answered a simple survey. Results are shown in Table 2. Table 2 – Volunteer 2 Survey Volunteer Two Summary Volunteer Two was surveyed after the testing and responded that they would prefer some type of remote accessible vote by mail system. Volunteer Two offered suggestions about positioning of screens to reduce glare and for Verity Reader to produce more noticeable/visible warnings about any under votes that may have occurred. No concerns were voiced over privacy issues. California Certification AUP Test Report v1.0 Page 7 of 8 Report Number CHI-19047-AUPTR-01 California Secretary of State Hart InterCivic Verity Voting 3.1 Accessibility, Usability & Privacy Test Report Volunteer Three Volunteer Three primarily used a combination of the Audio Tactile Interface (ATI) and headphones for the Verity Touch Writer and Verity Reader devices. Once they completed voting the ballot and verifying their votes using the Verity Reader, they scanned the ballot through Verity Scan. Afterwards, they answered a simple survey. Results are shown in Table 3. Table 3 – Volunteer 3 Survey Volunteer Three Summary Volunteer Three was surveyed after the testing and responded that they felt the system was very good and they would feel comfortable using it. Volunteer Three offered suggestions about having additional text size options other than just the “Large” option, as well as having a voice activated option for those who cannot use jelly switches or touch screens. No concerns were voiced over privacy issues. Final Considerations The general consensus of the volunteers was that they felt the technologies implemented for accessibility and usability improved the experience for voters that are most in need of them. As directed by the California Secretary of State, this accessibility, usability, and privacy testing report does not include any recommendation as to whether or not the system should be approved. End of AUP Test Report California Certification AUP Test Report v1.0 Page 8 of 8 Report Number CHI-19047-AUPTR-01