Hart Verity Voting 3.2 Voting System Functional Testing Report

Hart InterCivic Verity Voting 3.2 Voting System Functional Test Report for California Secretary of State CAF-22009-FTR-04 Vendor Name Hart InterCivic Vendor System Verity Voting 3.2 Prepared by: 4720 Independence St. Wheat Ridge, CO 80033 303-422-1566 www.SLICompliance.com Accredited by the U.S. Election Assistance Commission (EAC) for Selected Voting System Test Methods or Services California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 Revision History Date Release Author Revision Summary June 7th, 2023 1.0 A. Nestico Initial Release July 14th, 2023 2.0 A. Nestico Added RCTab Clarify Verity Scan is a Precinct Scanner Removed parentheses explaining acronyms after first usage Changed EC to Election Code Changed AB1416 to Ballot DISCLOSE Act Added RCTab to list of System Description July 21st, 2023 3.0 M. Santos Removed RCTab July 27th, 2023 4.0 M. Santos Fixed typo Disclaimer The information reported herein must not be used by the client to claim product certification, approval, or endorsement by NVLAP, NIST, or any agency of the Federal Government. Trademarks • SLI Compliance is a registered trademark of Gaming Laboratories International, LLC. • All products and company names are used for identification purposes only and may be trademarks of their respective owners. Copyright 2023 by SLI ComplianceSM, a division of Gaming Laboratories International, LLC. California Certification Functional Test Report Page 2 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 TABLE OF CONTENTS INTRODUCTION ................................................................................................................................. 4 TESTING RESPONSIBILITIES ................................................................................................................ 4 SCOPE OF THE HART VERITY VOTING 3.2 VOTING SYSTEM ..................................................... 4 SYSTEM DESCRIPTION ........................................................................................................................ 4 BLOCK DIAGRAM ................................................................................................................................ 7 FUNCTIONAL TESTING .................................................................................................................... 8 PHASE ONE – PHYSICAL CONFIGURATION AUDIT PHASE ...................................................................... 8 PHASE TWO – INSTALLATION PHASE ................................................................................................... 9 PHASE THREE – FUNCTIONAL CONFIGURATION AUDIT PHASE ............................................................ 11 PHASE FOUR – FUNCTIONAL TEST PHASE ......................................................................................... 11 EVALUATION OF TESTING ............................................................................................................ 23 California Certification Functional Test Report Page 3 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 INTRODUCTION This Functional Test Report details the testing performed during functional testing of the Hart InterCivic Verity Voting 3.2 (Hart Verity Voting 3.2) voting system against the California Voting System Standards (CVSS). Testing Responsibilities All testing was conducted under the guidance of personnel verified by the California Secretary of State (CASOS) to be qualified to perform the testing. Scope of the Hart Verity Voting 3.2 Voting System This section provides a description of the scope of Hart Verity Voting 3.2 voting system components. System Description The Hart Verity Voting 3.2 voting system is a paper-based, digital scan voting system that consists of the following major components: • Verity Data application • Verity Build application • Verity Count application • Verity Central application • Verity Election Management • Verity Desktop • Verity User Manager • Verity Scan firmware/hardware • Verity Touch Writer with Access firmware/hardware • Verity Reader firmware/hardware • Verity Key Verity Data Data is used by election officials to enter election data for contests, candidates, proposition text, translations, and audio. Data also provides the user with controls for proofing of data, layout, and performs validation prior to locking the data to ensure its readiness for use in Verity Build, the election definition software. California Certification Functional Test Report Page 4 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 Verity Build An election definition and device settings component. Build is a required component of the Verity Voting system, used by officials to complete pre-voting tasks for creating and generating an election definition and ballots. Build provides a ballot layout proofing process. The process establishes relationships between election data, jurisdiction, and polling place data, for the shared election definition. Build will create the portable media, vDrives, to provide a method of transferring the shared election definition to Verity Voting devices and other Verity components. vDrive uses an “air-gap,” or non-networked transfer method, to provide more secure exchange of election data. Verity Central A central ballot scanning and adjudication component used by officials for paper ballot scanning, contest resolution, and conversion of voter selection marks to electronic Cast Vote Records (CVRs) capabilities. Once the CVRs are written to vDrive(s) they can be transferred into Verity Count for vote tabulation and reporting of election results. Verity Central records cast vote records only; it does not tabulate. Verity Count Used by officials to complete post-voting functionality to tabulate election results and generate reports. Verity Count receives the CVRs from portable media devices (vDrives) used to record CVRs from Hart voting devices or Verity Central workstations. Verity Count can be used by officials to resolve Verity Scan or Verity Central write-in votes for paper ballots that were manually marked. Verity Count can also be used to collect and store all election logs from every Verity component/device used in the election, allowing for complete election audit log reviews. Verity Election Management The Verity Election Management application is available only on Verity server workstations. This software enables authorized users to add, import, export, archive, restore, and manage elections. Once an election is added or imported in the Election Management application, the election can be opened and handled per the features available within the Verity software installed on that workstation. User Management This software enables authorized users to create and manage user accounts within the Verity system. California Certification Functional Test Report Page 5 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 Verity Desktop Allows authorized users to manage a very limited set of operating system functions. Verity Desktop is workstation management software used for: • Setting the system date and time. • Exporting Verity application file hashes to removable USB media. • Accessing the operating system for a limited time (less than 24-hours per access code). User access to the operating system’s functionality is restricted to software updates and database management. • Importing printer configuration files. Verity Touch Writer Verity Touch Writer is a touch-screen Ballot Marking Device (BMD) that prints voter-marked ballots to a COTS printer. Voters use the electronic touch display interface to privately and independently make their selections on the ballot. Voters can also make selections with Verity Access, an Audio-Tactile interface (ATI) component with three tactile buttons, one audio port (for headphones), and one port for external two-switch devices. When voters finish making their selections, they print the marked ballot. Verity Reader Verity Reader is an optional paper ballot review device suitable for use by all voters, including non-disabled voters and voters with disabilities. Voters can insert their marked paper ballot to visually verify how their ballot will be counted when the ballot is cast in the Verity system, and/or hear audio read-back of their ballot choices. (For voters with disabilities, Verity Reader offers the same accessibility features as the Verity Touch Writer ballot marking device.) Verity Scan Verity Scan is Verity’s precinct tabulator scanning solution for paper ballots. Verity Scan is paired with a purpose-built ballot box to ensure accurate, secure, and private ballot scanning and vote casting. When opening the polls, authorized users activate the Verity Scan device to prepare it to receive marked paper ballots. Verity Scan indicates when it is appropriate to insert ballots, and when ballots have been successfully cast. Verity Scan records Cast Vote Records (CVRs) and audit log data in redundant, secure storage locations, including the Verity vDrive. The Verity vDrive storage is portable flash memory and allows the CVRs to be transferred to the Verity Count tabulation and reporting system. California Certification Functional Test Report Page 6 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 Verity Access Verity Access is an interface module that is connected to Verity Touch Writer and Verity Reader. The module has three tactile buttons, one audio port, and one port for external tactile buttons or sip-n-puff devices. Jacks for headphones and adaptive devices are located on the top edge of the device, and the device has grip surfaces on either side. Verity vDrive Verity vDrive is a required Verity Voting component, used as a portable media device generated by Verity Build. Verity vDrive allows election definitions to be moved from Verity Build to Verity Scan, Verity Touch Writer, Verity Reader, and Verity Print. Verity vDrive supports the transfer of Cast Vote Records (CVRs) in Verity Scan and Verity Central. Verity Key Verity Key is electronic media that is created by Verity Build for a specific election. Key is a required Verity component. Key is the electronic media that provides user authentication and configures election security throughout the Verity Voting system. Block Diagram The system overview of the submitted voting system is depicted in Figure 1. Figure 1. Hart Verity Voting 3.2 Voting System Overview California Certification Functional Test Report Page 7 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 Functional Testing Prior to all testing, the Trusted Build of the software and firmware was created. Functional testing was divided into four phases. • In phase one, the Physical Configuration Audit compared components submitted to the actual documentation. • In phase two, the Installation Phase included the steps necessary to install the system. • In phase three, the Functional Configuration Audit verified the system’s hardware and software perform all the functions listed in the documentation. • In phase four, Functional Testing exercised the system using operations necessary to conduct elections following the California Use Procedures for the system, documented the test results, and prepared benchmark data that can be used for system validation by the CASOS. During installation and functional testing, it was necessary to make minor edits to the California Use Procedures to provide clarity for end users. During examination and review performance, the system shall be configured as it would be for normal field use. This includes connecting all supporting equipment and peripherals. Phase One – Physical Configuration Audit Phase The Physical Configuration Audit (PCA) compared the voting system components submitted for certification to the manufacturer's technical documentation. This was an audit of all hardware and software in the system to compare the Technical Data Package (TDP) to the actual system. For the PCA, Hart provided: • Identification of all items that are to be a part of the software release. • Specification of compiler (or choice of compilers) to be used to generate executable programs. • Identification of all hardware that interfaces with the software. • Configuration baseline data for all hardware that is unique to the system. • Copies of all software documentation intended for distribution to users, including program listings, specifications, operations manual, voter manual, and maintenance manual. • User acceptance test procedures and acceptance criteria. • Identification of any changes between the physical configuration of the system submitted for the PCA and that submitted for the Functional California Certification Functional Test Report Page 8 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 Configuration Audit (FCA), with a certification that any differences do not degrade the functional characteristics. • Complete descriptions of its procedures and related conventions used to support this audit by: o Establishing a configuration baseline of the software and hardware to be tested. o Confirming that the system documentation matches the corresponding system components. Phase Two – Installation Phase System Installation, Configuration, and Validation During Installation testing, the following was verified: • All boxes, system components, etc. have been labeled correctly and accurately. • The voting system has been labeled correctly. (CVSS 8.2.a). • That a Configuration Log has been established. (CVSS 8.2.b). • All hardware that was used in the testing, including servers, workstations, monitors, printers, voting devices and peripherals, was documented. Build Software, Servers, and Workstations The following activities occurred: • All computers were wiped with DBAN. • All software and firmware components to be compiled in the trusted build were included and validated with HASHes from the manufacturer for COTS software/firmware components. • All hardware, compilers, and components needed to compile the trusted build were included and available. • The hardware provided was verified to meet or exceed the minimum requirements in the installation procedure manuals. • The required COTS software was verified to be available. • The component list of software and firmware exactly matched what was prescribed to be installed. • All software and firmware components were built per the California Use Procedures. • Hart Installation Procedures for the server and clients were successfully followed. California Certification Functional Test Report Page 9 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 • The sequence of steps in the installation procedures manual were followed. After the installation, COTS applications, proprietary applications, hardening, and configuration, images of each component of the system were taken. • Post installation, HASHes were taken of every piece of software and firmware. (Note: These HASHes will be provided to California Counties along with the Trusted Build so that they may validate the system at any time, such as a post-election reinstall to meet California airgap requirements). • System security policies, data sources, and registry were verified to be properly documented. • System configuration and setup were audited against specifications in the manuals to verify that scripts used in installation and configuration achieved the specifications. • The Hart System Verification Procedures were verified to be applicable for each machine. • All components were HASH’d and verified to be correct. • The voting system was verified to deploy COTS protection against viruses, worms, Trojan horses, and logic bombs. (CVSS 7.4.2). Install Firmware on Hardware Devices • Hardware devices were examined and determined to have the correct version of firmware installed. • Instructions for firmware upgrades in use procedures and other system documentation were verified to be correct. • After firmware was installed, hardware devices were verified to be operational. • The Hart System Verification Procedures for each hardware device were verified to be correct to install the firmware on that device. • Verified that no compilers, assemblers, or source code were resident on the system. • Election specific firmware was verified to not be installed on the same component that the operating system is installed on. (CVSS 7.4.1.b.iv). • Verified all software setup validation requirements of CVSS 7.4.6. Post Installation • Screenshots of all computer-based machines listed on the “Installed Programs” screen were taken. California Certification Functional Test Report Page 10 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 • Screenshots of all computer-based machines listed on the “drivers” page from Windows were taken. • Photos of all proprietary machines for the version numbers were taken. • Images of all equipment were taken. • All system software and firmware were taken. • A master copy of the “Trusted Build” directory structure, files, and “County Release” for distribution to counties was created. • “Trusted Build” software and Golden (County Release) images were created. Phase Three – Functional Configuration Audit Phase The Functional Configuration Audit was conducted by SLI to verify that the system performs all the functions described in the system documentation. In accordance with CVSS 9.11.2, The Hart Verity Voting 3.2 voting system does: a. Completely describe its procedures and related conventions used to support this audit for all system components b. Provide the following information to support this audit: i. Copies of all procedures used for module or unit testing, integration testing, and system testing. ii. Copies of all test cases generated for each module and integration test, and sample ballot formats or other test cases used for system tests. iii. Records of all tests performed by the procedures listed above, including error corrections and retests. The criteria of this phase were adequately met. Phase Four – Functional Test Phase During the Functional Test Phase, the system was examined to determine that every functional piece of the system is accurate and complete. During the Functional Test Phase, an issue log was maintained of any errors and omissions found in the documentation or anomalies encountered that were not identified during the PCA. Throughout the Functional Test Phase, Verity Print (BOD) was used to generate ballots that have been fatigued or need replacement, as well as new ballots used for casting votes. California Certification Functional Test Report Page 11 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 The system was maintained in an air-gapped fashion: The architecture shall allow transfer of the election definition and tally database from the permanent server(s) to the sacrificial server (CVSS 7.4.1.a.i). The voting system architecture allows each installation to use its own Ethernet network, infrastructure devices like switches, and central-scan vote-recording units, including optical scan units, permitting the two installations to be segregated and air-gapped to ensure that there are no cross connections. An air gap is established by keeping two installations/networks physically separate and seeing that no devices attached to the sacrificial installation/network is connected (directly or indirectly) to the first network, ensuring that data cannot flow from one installation/network to the other. Functional Testing Preparation Functional aspects of this phase included: • Kickoff meeting. Reviewed test plan and determine resource usage. • Inspected the test decks prepared by Hart Intercivic and audited them against the Ballot Marking and Expected Results files. • Using Verity Data/Build, built an election from scratch and verified all options in the Election Management System (EMS). • Completed extensive review of Verity Data/Build and Verity Count to verify end user documentation. • Evaluated methods to import elections into Verity. • Proofed the test deck for the Presidential Primary, Presidential General, Gubernatorial Primary, Gubernatorial General, Special Recall, Ranked Choice Voting, and start to finish Special Local election. All listed items completed successfully. Functional Testing Summary The tests conducted on the Hart Verity Voting 3.2 voting system included: • Presidential Primary Election • Presidential General Election • Gubernatorial Primary Election • Gubernatorial General Election • Special Recall Election • Special Local California Certification Functional Test Report Page 12 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 Test Presidential Primary Election A primary election was conducted utilizing: • One Verity Touch Writer polling place device • Two Verity Scan polling place devices • One Verity Data/Build workstation • One Verity Count workstation • One Verity Central workstation The Presidential Primary was conducted in English and Spanish. The election included two precincts, two political parties (Republican and Democrat) and one non-partisan. It included four contests and fifteen choices. Paper ballots included at least one bi-lingual ballot style consisting of English and Spanish. Ballot layouts met the requirements for Elections Code 13111, 13109, 13105 and the Ballot DISCLOSE act. This election was printed on the 8.5”x14” ballot size, both front and back to a single card. The following steps were completed with results as noted: • Prepared system for test election. • Configured Verity Central workstation and Verity Count workstation for receiving election data. • Prepared all precinct components (Verity Touch Writer and Verity Scan) for election. • Evaluated system for air-gap requirements. • Opened Polls in accordance with California Use Procedures. • One Verity Scan device setup as Election Day voting while the other Verity Scan device was setup as Early Voting. • Printed and verified zero reports for all devices. • Voted two ballots from each party on a Verity Touch Writer. • Scanned twenty ballots from each party on each Verity Scan. • The rest of the ballots were scanned on the Verity Central workstation. • Verified that the voter can review, confirm, and change their selections on the Verity Touch Writer and Verity Scan. • Printed final reports. During voting, the following was tested: • Fed ballots in all directions/sides on all devices. California Certification Functional Test Report Page 13 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 • Closed polls in accordance with California Use Procedures. • Printed results from all scanners. • Removed results media to transfer results back to Verity Count. • Shut down devices. • Consolidated and reported: o Uploaded results to Verity Count from all units. o Canvass reconciliation. Using adjudication component, adjudicated ballots containing write- ins. o Generated final reports and verified: Generated all reports available on the system. Canvass – Statement of Votes (SOV). Verified accuracy of report. Cumulative Results. Precinct Results Cast Vote Record Export and Report Saved all reports to a flash drive as artifacts of testing. • Verified system logging for all events. • Verified results. All listed items of election completed successfully. No issues were encountered. Test Presidential General Election A Presidential General election was conducted utilizing: • One Verity Touch Writer polling place device • Two Verity Scan polling place devices • One Verity Data/Build Standalone workstation • One Verity Central Standalone workstation • One Verity Count Standalone workstation Presidential General was conducted in English, Korean, Chinese, and Vietnamese. The election included 10 precincts, 13 contests, and 5 propositions, with one or two-write-ins depending on the contest. Paper ballots included one bi-lingual ballot style. Hart Verity Voting 3.2 does not support tri-lingual ballots. Ballot layouts met the requirements for Election Code 13111, 13109, 13105 and the Ballot DISCLOSE Act. California Certification Functional Test Report Page 14 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 This election was printed on the 8.5”x11” ballot size and included multiple cards. Startup and maintenance procedures for the polling place machines were evaluated to make sure the procedures were correct. Electronic devices and interfaces were programmed to support the entire election in all four languages. The following steps were completed with results as noted: • Prepared system for test election. • Configured Verity Central Standalone workstation and Verity Count Standalone workstation for receiving election data. • Prepared all precinct components (Verity Touch Writer, and Verity Scan) for election. • Evaluated system for air-gap requirements. • Opened Polls in accordance with California Use Procedures. • Printed and verified zero reports for all devices. • Voted two ballots from each party on a Verity Touch Writer. • Scanned two ballots from each party on a Verity Scan. • Verified that the voter can review, confirm, and change their selections on the Verity Touch Writer and Verity Scan. • Printed final reports. During voting, the following was tested: • Fed ballots in all directions/sides on all devices. • Closed polls in accordance with California Use Procedures. • Printed results from all scanners. • Removed results media to transfer results back to Verity Count. • Shut down devices. • Consolidated and reported: o Uploaded results to Verity Count from all units. o Canvass reconciliation. Processed provisional ballots. Using adjudication component, adjudicated ballots containing write- ins. o Generated final reports and verified: Generated all reports available on the system. Canvass – SOV. Verified accuracy of report. California Certification Functional Test Report Page 15 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 Cumulative Results. Precinct Results. Cast Vote Record Export and Report. Saved reports and backup of election to a flash drive as artifacts of testing. • Verified system logging for all events. Verified results. All listed items of election completed successfully. During the central scanning phase, all ballots were separated by their page number, such that the first pages of all the ballots were together, while all the second pages were also together. The Verity Central Standalone workstation was able to scan the ballots and tabulate the correct results. No issues from Hart Verity Voting 3.2 voting system devices encountered. Test Gubernatorial Primary Election A Gubernatorial Primary election was conducted utilizing: • One Verity Touch Writer polling place device • Two Verity Scan polling place devices • One Verity Data/Build Server/Client workstation • One Verity Central Server/Client workstation • One Verity Count Server/Client workstation Gubernatorial Primary was conducted in English, and Spanish. The election included ten precincts. The election included two precincts with five contests and 27 choices. Paper ballots included one bi-lingual ballot style. Ballot layouts met the requirements for Election Code 13111, 13109, 13105 and the Ballot DISCLOSE Act. Two hundred and fifty-five test deck ballots were printed on demand using Verity Print and its IntoPrint SP1360 printer. This election was printed on the 8.5”x14” ballot size using multiple cards. Electronic devices and interfaces were programmed to support the entire election in both languages. The following steps were completed with results as noted: • Prepared system for test election. California Certification Functional Test Report Page 16 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 • Configured Verity Central Server/Client workstation and Verity Count Server/Client workstation for receiving data. • Prepared all precinct components (Verity Touch Writer, and Verity Scan) for election. • Evaluated system for air-gap requirements. • Opened Polls in accordance with California Use Procedures. • Verity Scan setup as Election Day and the other Verity Scan setup as Early Voting. • Printed and verified zero reports for all devices and workstations. • Removed ten ballots for each party from the test deck and recreated them by voting on a Verity Touch Writer. • Simulated the experience of a fleeing voter and voter who took longer than normal to vote. o A longer than typical voting experience voter was able to complete their voting session. o A fleeing voter ballot was spoiled and voting session started for another voter. • Scanned twenty ballots on each Verity Scan device (40 total) with the remaining ballots scanned in the Verity Central workstation. • Printed final reports. During voting, tested the following: • Fed ballots in all directions/sides on all devices. • Closed polls in accordance with California Use Procedures. • Printed results from all scanners. • Removed results media to transfer results back to Verity Count Server/Client workstation. • Shut down devices. • Consolidated and reported: o Uploaded results to Verity Count from all units. o Canvass reconciliation. Using adjudication component, adjudicated ballots containing write- ins. o Generated final reports and verified: Generated all reports available on the system. Canvass – SOV. Verified accuracy of report. California Certification Functional Test Report Page 17 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 Cumulative Results. Precinct Results. Cast Vote Record Export and Report. Audit reports (Including tabulation devices). • Verified system logging for all events. • Verified results. All listed items of election completed successfully. Test Gubernatorial General Election A Gubernatorial General election was conducted utilizing: • One Verity Touch Writer polling place device • Two Verity Scan polling place devices • One Verity Data/Build Server/Client workstation • One Verity Central Server/Client workstation • One Verity Count Server/Client workstation Gubernatorial General was conducted in English, and Spanish. The election included ten precincts with five contests and 27 choices. Paper ballots included one bi-lingual ballot style. Ballot layouts met the requirements for Election Code 13111, EC 13109, EC 13105 and Ballot DISCLOSE Act. Two hundred and fifty-five test deck ballots were printed on demand using Verity Print with its IntoPrint SP1360 printer. This election was printed on the 8.5”x17” ballot size using multiple cards. Electronic devices and interfaces were programmed to support the entire election in both languages. The following steps were completed with results as noted: • Prepared system for test election. • Configured Verity Central Server/Client workstation and Verity Count Server/Client workstation for receiving data. • Prepared all precinct components (Verity Touch Writer, and Verity Scan) for election. • Evaluated system for air-gap requirements. • Opened Polls in accordance with California Use Procedures. California Certification Functional Test Report Page 18 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 • Verity Scan setup as Election Day and the other Verity Scan setup as Early Voting. • Printed and verified zero reports for all devices and workstations. • Scanned twenty ballots on each Verity Scan device (40 total) with the remaining ballots scanned in the Verity Central workstation. • Printed final reports. During voting, the following was tested: • Fed ballots in all directions/sides on all devices. • Simulate a polling place power loss by unplugging the Scan machine and scanning ten ballots over thirty minutes of battery power. • Closed polls in accordance with California Use Procedures. • Printed results from all scanners. • Removed results media to transfer results back to Verity Count Server/Client workstation. • Shut down devices. • Consolidated and reported: o Uploaded results to Verity Count from all units. o Canvass reconciliation. Using adjudication component, adjudicated ballots containing write- ins. o Generated final reports and verified: Generated all reports available on the system. Canvass – SOV. Verified accuracy of report. Cumulative Results. Precinct Results. Cast Vote Record Export and Report. • Verified system logging for all events. • Verified results. All listed items of election completed successfully. Test Special Recall Election A Special Recall election was conducted utilizing: • One Verity Touch Writer polling place device • Two Verity Scan polling place devices California Certification Functional Test Report Page 19 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 • One Verity Reader polling place device • One Verity Central Standalone workstation • One Verity Data/Build Standalone workstation • One Verity Count Standalone workstation The Special Recall was conducted in English, Khmer, Japanese, and Hindi. The election consisted of one precinct and one contest. Ballots were pre-printed with English and Khmer, Hart Verity Voting 3.2 voting system did not support tri-lingual ballots. The contest contained 48 candidates with one write-in gubernatorial contest. Standalone configuration was used to conduct the election. Ballot layouts met the requirements for Election Code 13105. The election was printed on 8.5”x22” ballots. The following steps were completed with results as noted: • Prepared all components for election. • Configured Verity Central Standalone workstation and Verity Count Standalone workstation for receiving election data. • Prepared all precinct components (Verity Touch Writer, and Verity Scan) for election. • Evaluated system for air-gap requirements. • Open Polls in accordance with California Use Procedures. • One Verity Scan device has been setup as Election Day voting while the other Verity Scan device has been setup as Early Voting. • Printed and verified zero reports for all devices. • Two ballots of each language were pulled from the pre-printed stack, spoiled and then recreated on the Verity Touch Writer. • Recreated ballots were verified on the Verity Reader and then inserted into the Verity Scan. Printed the final report. During voting, the following was tested: • Fed ballots in all directions/sides on all devices. • Accessibility options were verified on the Verity TouchWriter. • Verify language support for alternative languages: Khmer, Japanese, and Hindi using the: o Verity Touch Writer display o Verity Touch Writer audio California Certification Functional Test Report Page 20 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 o Verity Reader display o Verity Reader audio ballot o Verity Scan • Closed polls, in accordance with California Use Procedures. • Printed results from all scanners. • Removed results media to transfer results back to Verity Count Standalone workstation. • Shut down devices. • Consolidated and reported: o Uploaded results to Verity Count from all units o Canvass reconciliation: Using adjudication component, adjudicated ballots containing write- ins. o Generated final reports & verified: Canvass – SOV. Verify accuracy of report. Cumulative Results. Precincts Results. Cast Vote Record Export and Report Saved reports and backup of election to a flash drive as artifacts of testing. • Verified system logging for all events. • Verified results. All listed items of election completed successfully. No issues were encountered. Special Local Election A Special Local election was created using: • Verity Data/Build Standalone workstation • Verity Count Standalone workstation • Verity Central Standalone workstation The Special Local was conducted in English. The election included two precincts. It included one contest with six choices and one proposition. Ballot layouts met the requirements for Election Code 13111, 13109, 13105 and the Ballot DISCLOSE Act. California Certification Functional Test Report Page 21 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 This election was printed on the 8.5”x11” ballot size. The following steps were completed with results as noted: • Prepared system for test election. • Configured Verity Central workstation and Verity Count workstation for receiving election data. • Evaluated system for air-gap requirements. • Printed and verified zero reports for all devices. • Ballots were hand marked. • The ballots were scanned on the Verity Central workstation. • Printed final reports. During voting, the following was tested: • Consolidated and reported: o Uploaded results to Verity Count from all units. o Canvass reconciliation. o Generated final reports and verified: Generated all reports available on the system. Canvass – SOV. Verified accuracy of report. Cumulative Results. Cast Vote Record Export and Report. Saved all reports to a flash drive as artifacts of testing. • Verified system logging for all events. • Verified results. All listed items of election completed successfully. No issues were encountered. Final Data Capture and Analysis • Took hashes and images from all computers. • Validated the system on each server and desktop, as well as each polling place device using the procedure provided by Hart Intercivic. • Generated “Trusted Build” software and Golden (County Release) images to be distributed by Secretary of State to County. • Verified with vendor that the “Trusted Build” software was complete and ready for distribution. Hart Intercivic attested to the correctness of the files. California Certification Functional Test Report Page 22 of 23 Document No. CAF-22009-FTR-04 California Secretary of State Hart InterCivic Verity Voting 3.2 California Certification Functional Test Report v4.0 • Verified that all drivers including monitor video drivers are included in the County Release. Evaluation of Testing The above tests were conducted using the executables created in the Trusted Build, in association with the appropriate hardware versions as declared during the current certification project for the Hart InterCivic Verity Voting 3.2 voting system, for the State of California. No functional issues were encountered during testing. As directed by the California Secretary of State, this report does not include any recommendation as to whether or not the system should be approved. End of Functional Test Report California Certification Functional Test Report Page 23 of 23 Document No. CAF-22009-FTR-04