Consultant's Functional Test Report

Dominion Democracy Suite 5.19 Voting System Functional Test Report for California Secretary of State CAF-23001-FTP-01 Vendor Name Dominion Voting Systems Vendor System Democracy Suite 5.19 Prepared by: SLI ComplianceSM 4720 Independence St. Wheat Ridge, CO 80033 303-422-1566 www.SLICompliance.com Accredited by the Election Assistance Commission (EAC) for Selected Voting System Test Methods or Services. Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report Copyright © 2025 SLI ComplianceSM, a Division of Gaming Laboratories International, LLC Revision History Date Release Author Revisions August 12, 2024 v1.0 E. Bickley Initial Release Disclaimer The Certification Test results reported herein must not be used by the client to claim product certification, approval, or endorsement by NVLAP, NIST, or any agency of the Federal Government. Results herein relate only to the items tested. Trademarks • SLI is a registered trademark of SLI Compliance, a Division of Gaming Laboratories International, LLC. • All other products and company names are used for identification purposes only and may be trademarks of their respective owners. The tests referenced in this document were performed in a controlled environment using specific systems and data sets and results are related to the specific items tested. Actual results in other environments may vary. Opinions and Interpretations There are no SLI opinions or interpretations included in this report. California Certification Functional Test Report v1.0 Page 2 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report TABLE OF CONTENTS INTRODUCTION ................................................................................................................................... 4 REFERENCES....................................................................................................................................... 4 SYSTEM OVERVIEW ........................................................................................................................... 4 SCOPE OF THE DOMINION DEMOCRACY SUITE 5.19 VOTING SYSTEM ..................................................... 4 CERTIFICATION TEST RESULTS SUMMARY ................................................................................... 5 PHYSICAL CONFIGURATION AUDIT PHASE ............................................................................................. 5 INSTALLATION PHASE ........................................................................................................................... 6 System Installation, Configuration and Validation ......................................................................... 6 Build Software, Servers, and Workstations ................................................................................... 7 Install Firmware on Hardware Devices .......................................................................................... 7 Post Installation Phase .................................................................................................................. 8 FUNCTIONAL CONFIGURATION AUDIT PHASE ........................................................................................ 8 FUNCTIONAL TEST PHASE .................................................................................................................... 8 Functional Testing Summary ....................................................................................................... 10 Test Top-Two Gubernatorial Primary .......................................................................................... 10 Test Gubernatorial General Election ........................................................................................... 11 Test Presidential Primary Election ............................................................................................... 12 Test Presidential General Election .............................................................................................. 14 Rank Choice Voting (RCV) Election ............................................................................................ 15 Test Special Recall Election ........................................................................................................ 16 Test Special Local Election .......................................................................................................... 18 Retesting for ICC Code Fix .......................................................................................................... 19 Final Data Capture and Analysis ................................................................................................. 20 Post Test Analysis ....................................................................................................................... 21 EVALUATION OF TESTING ................................................................................................................... 21 California Certification Functional Test Report v1.0 Page 3 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report Introduction SLI Compliance is submitting this test report as a summary of the certification testing efforts for the Dominion Democracy Suite 5.19 voting system (Dominion DS 5.19). This test report provides results for the functional testing of the Dominion DS 5.19. Reference California Voting System Standards (CVSS) System Overview Scope of the Dominion Democracy Suite 5.19 Voting System This section provides a description of the scope of the Dominion DS 5.19 components: • Election Management System (EMS) – Election Event Designer (EED) application • EMS – Results Tally Reporting (RTR) application • EMS – Adjudication Services (Adjudication) application • ImageCast Central (ICC) application • ImageCast Evolution (ICE) firmware/hardware • ImageCast X (ICX) firmware/hardware • ImageCast Precinct 2 (ICP2) firmware/hardware The Dominion DS 5.19 EMS represents a set of N-Tier software applications (EMS, RTR, Adjudication) for pre-voting and post-voting election project activities that are applicable to jurisdictions of various sizes and geo-political complexities. The Dominion DS 5.19 ICC system consists of a central, high-speed, optical scan ballot counter (tabulator) called the ICC Ballot Counter and is used for processing absentee ballots (such as vote by mail). This ballot counter unit is based on commercial off-the-shelf (COTS) hardware coupled with custom-made ballot processing application software. It is used for high-speed, accurate, and reliable centralized scanning and counting of paper ballots. The Dominion DS 5.19 ICE system employs a precinct-level optical scan ballot counter (tabulator) in conjunction with an external ballot box. This tabulator is designed to mark and/or scan paper ballots, interpret voting marks, communicate these interpretations back to the voter (either visually through the integrated LCD California Certification Functional Test Report v1.0 Page 4 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report display or audibly via integrated headphones) and, upon the voter’s acceptance, deposit the ballots into the secure ballot box. The Dominion DS 5.19 ICX ballot marking platform is used to create paper ballots. These ballots are later scanned and tabulated by the ICC optical ballot counter and/or scanned, verified, and cast by the ICE. The Dominion DS 5.19 ICP2 ballot counter device is a precinct optical scan ballot counter designed to provide three major functionalities: ballot scanning and tabulation, accessible voting, and ballot review and second chance voting. Certification Test Results Summary Prior to all testing, the trusted build of the software and firmware was created. This process created a contractor compiled “Trusted Build” from components provided by the vendor and witnessed by the California Secretary of State (CASOS). Functional testing was divided into five phases. 1. The Physical Configuration Audit (PCA) compared components submitted to the actual documentation. 2. The Installation Phase included the steps necessary to install the system, develop any additional test elections from scratch, and provided Dominion with the data needed to print test ballots for any additional elections. 3. The Functional Configuration Audit (FCA) verified that the systems hardware and software perform all the functions listed in the documentation. Note that numerous documentation issues were found which resulted in a fifth phase for documentation verification. 4. The Functional Test Phase exercised the system through the remaining operations necessary to conduct elections following the California Use Procedures for the system, documented the test results, and prepared benchmark data that can be used for system validation by the CASOS. 5. A fifth phase was not required to perform any regression testing, though numerous documentation issues were found which resulted in a fifth phase for documentation verification. During installation and functional testing, it was necessary to edit the California Use Procedures to provide clarity for end users. A list was maintained of all changes and updated California Use Procedures were generated. Physical Configuration Audit Phase The PCA compared the voting system components submitted for certification to the manufacturer's technical documentation. This is an audit of all hardware and software in the system to compare the Technical Data Package (TDP) to the actual system. For the PCA, Dominion: California Certification Functional Test Report v1.0 Page 5 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report • Identified all items that were to be a part of the software release. • Specified the compiler (or choice of compilers) to be used to generate executable programs. • Identified all hardware that interfaces with the software. • Configured baseline data for all hardware that is unique to the system. • Made copies of all software documentation intended for distribution to users, including program listings, specifications, operations manual, voter manual, and maintenance manual. • Complied with user acceptance test procedures and acceptance criteria. • Identified any changes between the physical configuration of the system submitted for the PCA and that submitted for the FCA, with a certification that any differences did not degrade the functional characteristics. • Completed descriptions of its procedures and related conventions used to support this audit by: o Establishing a configuration baseline of the software and hardware to be tested. o Confirming whether the system documentation matches the corresponding system components. Note that the PCA was conducted in conjunction with testing activities; as such, all anomalies and omissions in the documentation identified by SLI during the PCA were not corrected prior to starting any testing. A re-review of the documentation was conducted in order to verify the documentation was accurate and free of anomalies or omissions. Installation Phase System Installation, Configuration and Validation During Installation Phase testing, • An issue log was created of any errors and omissions found in the documentation or anomalies encountered that were not identified in the PCA. • Verified that all boxes, system components, etc. had been labeled correctly and accurately. • Verified that the voting system had been labeled correctly. • Verified Configuration Log. The Installation Phase completed successfully. California Certification Functional Test Report v1.0 Page 6 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report Build Software, Servers, and Workstations • All hardware used in the tests, including servers, workstations, monitors, printers, voting devices, and peripherals was documented and photographed. • Verified that the servers and workstations have been wiped. • Verified that the hardware provided meets or exceeds the minimum requirements in the installation procedure manuals. • Verified that the required COTS software licenses are available. • Verified that the component list of software and firmware exactly matches what should be installed. • Followed the Dominion Installation Procedures for the server and clients. • Followed the sequence of steps in the installation procedures manual. After the installation, COTS applications, proprietary applications, hardening, and configuration, an image of the system was taken. • Installed/setup Mobile Ballot Printing. • Installed imprinters on all scanners that will provide this option. • Verified the functionality of ICE. • When the installation was completed, in addition to the images, HASH values were taken for each component software. • Verified that system security policies, services, data sources, and registry were documented. • Audited the system configuration and setup against specifications in the manuals to verify that scripts used in installation and configuration achieved the specifications. • Followed the Dominion System Verification Procedures applicable for each machine and logged any errors and omissions or anomalies in the issue log. • Verified the voting system has deployed COTS protection against viruses, worms, Trojan horses, and logic bombs. All listed items completed successfully. Install Firmware on Hardware Devices • Examined the hardware devices and determined which version of firmware is currently installed. • Took steps necessary to allow installation of the trusted build of firmware on each hardware device. • Took steps as necessary to validate instructions for firmware upgrades in use procedures and other system documentation. • After firmware was installed, verified that hardware devices were operational. California Certification Functional Test Report v1.0 Page 7 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report • Followed the Dominion System Verification Procedures for each hardware device to validate the firmware installed on that device. • Verified no compilers, assemblers, or source code were resident on the system. • Verified that election specific firmware was not installed on the same component that the operating system was installed on. • Verified all validation requirements. • Logged any errors and omissions or anomalies in the issue log. • Provided a copy of the issue log to Dominion personnel for their resolution. All listed items completed successfully. Post Installation Phase • Took Acronis images of all equipment. • All images sent to SLI for restoration on equipment used for Security testing. • Created a Master copy of the “Trusted Build” directory structure, files, and “Golden Images” for distribution to counties. Post Installation Phase completed successfully. Functional Configuration Audit Phase The Functional Configuration Audit is conducted to verify that the system performs all the functions described in the system documentation. The manufacturer shall: • Completely describe its procedures and related conventions used to support this audit for all system components. • Provide the following information to support this audit: o Copies of all procedures used for module or unit testing, integration testing, and system testing. o Copies of all test cases generated for each module and integration test, and sample ballot formats or other test cases used for system tests. o Records of all tests performed by the procedures listed above, including error corrections and retests. Dominion provided test cases, results records, procedures used for module or unit testing, integration testing, and system testing. All required information was verified to satisfactorily meet the requirements of this section. Functional Test Phase Functional Test Phase testing examined the system to determine that every functional piece of the system is accurate and complete. During Functional Phase California Certification Functional Test Report v1.0 Page 8 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report testing, an issue log was maintained of any errors and omissions found in the documentation or anomalies encountered that were not identified during the PCA. Throughout the Functional Testing Phase, EED attached printers were used to generate ballots that have been fatigued or needed replacement. The system was maintained in an air-gapped fashion: The architecture allowed transfer of the election definition and tally database from the permanent server(s) to the sacrificial server using USB devices. The voting system architecture does allow each installation to use its own Ethernet network, port server, and central-office vote-recording units and optical scan units, permitting the two installations to be segregated and air-gapped to ensure that there are no cross connections. An air gap was established by keeping two installations/networks physically separate and seeing that no device attached to the sacrificial installation/network was connected (directly or indirectly) to the first network, ensuring that data cannot flow from one installation/network to the other. • Inspected the test decks prepared by Dominion and audited them against the Ballot Marking and Expected Results Excel files. • Using EED, built an election from scratch and verified all options in EED. This election contained a multiple candidate contest for a “choose two of three” contest. • Printed ballots. • Scanned ballots. • Tabulated and generated reports. • Extensive review of EED and RTR to verify end user documentation. • Evaluated methods to import elections into EED. • Evaluated Access Controls. • Verified all CVSS requirements. • Proofed the test deck for the Top-Two Gubernatorial Primary, Gubernatorial General, Presidential Primary, Presidential General, Special Recall, Ranked Choice Voting, and Special Local elections. • Scanned and tabulated the test deck for the elections on ICC. • Compared the tabulation results of this scan to expected results. Investigated any anomalies encountered in the results and resolved them by modifying either the deck or the expected results. • Conducted logic and accuracy (L&A) testing in accordance with California Use Procedures. • Logic and accuracy testing was inherent to the testing performed and verified across all of the elections. California Certification Functional Test Report v1.0 Page 9 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report o Reviewed ICE, ICP2, ICX Classic and ICX Prime, ICC, and EMS for L&A procedures per Democracy Suite 5.19 documentation. o Printed and verified L&A results from scanners. o Uploaded and consolidated results to EMS. o Printed and verified L&A results. o Verified that all components are ready to go after L&A or needed to be reprovisioned/reimaged prior to the actual election. All listed items completed successfully. Functional Testing Summary The tests run on the Democracy Suite 5.19 voting system included: o Top-Two Gubernatorial Primary o Gubernatorial General o Presidential Primary o Presidential General o Special Recall o Ranked Choice Voting o Special Local Election Top-Two Gubernatorial Primary Test A Top-Two gubernatorial primary election was run utilizing: • Two ICX polling place devices • One ICE polling place device • One ICP2 polling place device • Two ICC environments (One HiPro and one Cannon) • One EMS (EED, Adjudication, RTR) The Top-Two gubernatorial primary election was conducted in English and the election was imprinted by the HiPro scanner on the last run. The election included 19 precincts and five parties. It included five contests and ten choices. Ballots were sized (8.5” x 17”) for the system, and single sided. The following steps were completed with results as noted: • Prepared system for test election. • Finalized the EMS and set up for reporting. • Prepared all precinct components (ICE, ICP2, ICX Classic, and ICX Prime) for election. • Evaluated system for air-gap requirements. California Certification Functional Test Report v1.0 Page 10 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report • Opened polls in accordance with California Use Procedures. • Printed and verified zero reports for all devices. • Verified that the voter can review, confirm, and change their selections on the ICX Classic, ICX Prime, and ICE. • Marked one ballot from each party on an ICX Classic. • Marked one ballot from each party on an ICX Prime. • Marked ballots from pattern of precincts on both an ICX Classic and Prime. • Verified ballots from pattern on scanners. • Created eight marginal marks and scanned on all scanners to verify all scanners scanned consistently and adjudication was consistent across all scanners. • Simulated early voting and suspended voting and voted more ballots. • System tested label capability. • Scanned half the ballot on an ICE. • Scanned half the ballots on an ICP2. • Scanned all ballots from each party on an ICC Cannon. • Scanned all ballots from each party on an ICC HiPro with Imprinter. • Printed summary reports. • Loaded all results from each scanner. • Adjudicated hand marked ballots with marginal marks. • Used preprinted ballots for the ICX’s above so the results did not need to be added to the expected results. • Used RTR to adjudicate ballots. o Generated final reports and verified. o Audited reports (including tabulation devices). • Verified system logging for all events and saved system logs to archive. All listed items of election completed successfully. No issues were encountered. Gubernatorial General Election Test A gubernatorial general election was run utilizing: • Two ICX polling place devices • One ICE polling place device • One ICP2 polling place device • Two ICC environments (One HiPro and One Cannon) California Certification Functional Test Report v1.0 Page 11 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report • One EMS (EED, Adjudication, RTR) The gubernatorial general election was conducted in English. The election included 3000 precincts or the maximum for the system. It included five contests and ten choices. Ballots were of medium size (8.5” x 14”) for the system, and double sided. The following steps were completed with results as noted: • Prepared system for test election. • Finalized EMS and set up for reporting. • Prepared all precinct components (ICE, ICP2, ICX Classic, and ICX Prime) for election. • Opened polls in accordance with California Use Procedures. • Printed and verified zero reports for all devices. • Scanned two ballots from each party on an ICE. • Scanned two ballots from each party on an ICP2. • Marked one ballot from each party on an ICX Classic. • Marked one ballot from each party on an ICX Prime. • Verified that the voter can review, confirm, and change their selections on the ICX Classic, ICX Prime, ICE, and ICP2. • Used preprinted ballots for the ICX’s above so the results did not need to be added to the expected results. • Printed Summary reports. • Uploaded results to EMS from all units. • Used RTR to adjudicate ballots. o Generated final reports and verified. o Audited reports (including tabulation devices). • Verified system logging for all events and saved system logs to archive. All listed items of election completed successfully. No issues were encountered. Test Presidential Primary Election A presidential primary election was run utilizing: • Two ICX polling place devices • One ICE polling place device • One ICP2 polling place device • Two ICC environments (One HiPro and One Cannon) • One EMS (EED, Adjudication, RTR) California Certification Functional Test Report v1.0 Page 12 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report The presidential primary election was conducted in English and Spanish. The election included ten precincts and eight party splits per precinct. It included 20 contests and 65 choices. Electronic devices and interfaces were programmed to support the entire election in English and Spanish. Ballots were of medium size (8.5” x 14”) for the system, and double sided. The following steps were completed with results as noted: • Prepared system for test election. • Finalized EMS and set up for reporting. • Prepared all precinct components (ICE, ICP2, ICX Classic, and ICX Prime) for election. • Opened polls in accordance with California Use Procedures. • Printed and verified zero reports for all devices. • Printed ten ballots for every language available in this election. Added these to the expected results. • Hand marked five ballots with write-ins. Two from each language and added to expected results. • Marked at least ten ballots from each party on both an ICX Classic and Prime. • Marked one ballot from each party on an ICX Classic. • Marked one ballot from each party on an ICX Prime. • Scanned ballots from each party on an ICE. • Scanned ballots from each party on an ICP2. • Verified that the voter can review, confirm, and change their selections on the ICX Classic, ICX Prime, ICE, and ICP2. • Fed ballots in all directions/sides on all devices. • Used preprinted ballots for the ICX’s above so the results did not need to be added to the expected results. • Printed summary reports. • Uploaded results to EMS from all units. • Used RTR to adjudicate ballots. o Generated final reports and verified. o Audited reports (including tabulation devices). • Verified system logging for all events and saved system logs to archive. All listed items of election completed successfully. No issues were encountered. California Certification Functional Test Report v1.0 Page 13 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report Presidential General Election Test A presidential general election was run utilizing: • Two ICX polling place devices • One ICE polling place device • One ICP2 polling place device • Two ICC environments (One HiPro and One Cannon) • One EMS (EED, Adjudication, RTR) The presidential general was conducted in English, Korean, Chinese and Vietnamese. The election included ten precincts and 12 contests. Paper ballots included one bi-lingual ballot style consisting of English, Chinese and Vietnamese. Ballots were of small size (8.5” x 11”) and were multi-card for the system, and double sided. The following steps were completed with results as noted: • The candidates were rotated per EC 13111. • Order of office on the ballot were per EC 13109. • Prepared system for test election. • There were 2 measures on the ballot style and a second ballot card was used. (This includes using the front and back of the first ballot card if two- sided ballots are supported.) • Finalized EMS and set up for reporting. • Prepared all precinct components (ICE, ICP2, ICX Classic, and ICX Prime) for election. • Opened Polls in accordance with California Use Procedures. • Printed and verified zero reports for all devices. • Print ten ballots for each supported language in the election. Hand Marked and voted ballots on the ICX Classic and Prime per language. • Verified that the voter can review, confirm and change their selections on the ICX Classic, ICX Prime, ICE and ICP2. • Used preprinted ballots for the ICX’s above so the results did not need to be added to the expected results. • Scanned all ballots on an ICE. • Scanned all ballots on an ICP2. • Printed Summary reports. • Uploaded results to EMS from all units. • Used RTR to adjudicate ballots. o Generated final reports & verified. California Certification Functional Test Report v1.0 Page 14 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report o Audit reports (Including tabulation devices). • Verified system logging for all events. Saved system logs to archive. All listed items of election completed successfully. No issues were encountered. Rank Choice Voting (RCV) Election A Ranked Choice Vote election was run utilizing: • 2 ICX polling place devices • 1 ICE polling place device • One ICP2 polling place device • Two ICC environments (One HiPro and One Cannon) • One EMS (EED, Adjudication, RTR) The ranked choice voting election was conducted in English, Ilocano, Tagalog, and Thai. A fictitious single and multiple seat RCV Election with one ballot style with two contests containing six to twelve candidates with four write-ins for the multi seat and one write-in for the single seat was utilized. Ballots were of small size (8.5” x 11”) and double sided. The following steps were completed with results as noted: • Prepared all components for election. • Initialized and loaded election definition on one ICE, ICP2, ICX Classic, ICX Prime, and all ICCs. • Opened polls in accordance with California Use Procedures. • Printed and verified zero reports for all devices. • Scanned all ballots on an ICC scanner. • Marked ballots on an ICX Classic. • Marked ballots on an ICX Prime. • Removed ballots from the pre-marked test deck and remade these on the ICX Classics and ICX Primes. • Printed zero reports from EMS. • Cast all ballots on each scanner. • During voting, tested the following: o Fed ballots in all directions/sides on all devices. ➢ Language supported included alternative languages: Ilocano, Tagalog, and Thai. o ICE display. o ICE audio. California Certification Functional Test Report v1.0 Page 15 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report o ICX Classic display. o ICX Classic audio. o ICX Classic printing. o ICX Prime display. o ICX Prime audio. o ICX Prime printing. • Closed polls, in accordance with California Use Procedures. • Printed results from all scanners. • Removed results media as necessary to transfer results back to EMS. • Shut down devices. • Ranked results and determined winner. • Verified and documented all one seat and multi seat RCV on the system. • Uploaded results to EMS from all units. • Used RTR to adjudicate ballots. o Generated final reports and verified. o Audited reports (including tabulation devices). • Verified system logging for all events and saved system logs to archive. All listed items of election completed successfully. No issues were encountered. Special Recall Election Test A special recall election was run utilizing: • Two ICX polling place devices • One ICE polling place device • One ICP2 polling place device • Two ICC environments (One HiPro and One Cannon) • One EMS (EED, Adjudication, RTR) The Special Recall election was conducted in English, Khmer, Japanese, and Hindi. The election consisted of one precinct and one contest. The contest included 135 choices with one write-in for a gubernatorial contest. The election was printed on the longest ballot size possible for the system. Ballots size was (8.5” x 21”) and double sided. The following steps were completed with results as noted: • Prepared all components for election. California Certification Functional Test Report v1.0 Page 16 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report • Installed election definitions on devices, printed zero reports, and opened polls. • Scanned test decks through each tabulating device. • During voting, tested the following: o Fed ballots in all directions/sides on all devices. o Printed five ballots for each supported language in the election. o Voted five on ICE, five on the ICX and hand marked five. o Added/replaced these ballots to the expected results. • Language support for alternative languages Khmer, Japanese, and Hindi were tested on the following: o ICE display. o ICE audio ballot. o ICX Classic display. o ICX Classic audio ballot. o ICX Classic printing. o ICX Prime display. o ICX Prime audio ballot. o ICX Prime printing. o ICP2 display. • Scanned the marked ballot on each scanner device. • Closed each machine and printed results. • Tested the ability to select candidates across multiple screens on the ICEs, ICX Classics and ICX Primes. • Closed polls in accordance with California Use Procedures. • Printed results from all scanners. • Transferred results back to EMS. • Shut down devices. • Uploaded results to EMS from all units. • Used RTR to adjudicate ballots. o Generated final reports and verified. o Audited reports (including tabulation devices). • Verified system logging for all events and saved system logs to archive. All listed items of election completed successfully. No issues were encountered. California Certification Functional Test Report v1.0 Page 17 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report Special Local Election Test A special local election was run utilizing: • Two ICX polling place devices • One ICE polling place device • One ICP2 polling place device • Two ICC environments (One HiPro and One Cannon) • One EMS (EED, Adjudication, RTR) The special local election was built from scratch by SLI in English. A fictitious single and multiple seat RCV Election with one ballot style with five contests containing two to ten candidates was utilized. The following steps were completed with results as noted: • Prepared all components for election in EED. • Printed ballots. • Marked ballots from pattern of precincts on both an ICX Classic and Prime and hand marked. • Initialized and loaded election definition on one ICE, ICP2, ICX Classic, ICX Prime, and two ICCs for Cannon and HiPro scanners. • Evaluated system for air-gap requirements by creating and restoring election demonstrating the air gap. • Opened polls in accordance with California Use Procedures. • Printed and verified zero reports for all devices. • Cast all ballots on each scanner. • During voting, tested the following. o Fed ballots in all directions/sides on all devices. • Closed polls in accordance with California Use Procedures. • Printed results from all scanners. • Final scan on ICC Cannon scanner with Imprinter on to mark ballots on final run. • Removed results media as necessary to transfer results back to EMS. • Shut down devices. • Exercised adjudication workstation. • Uploaded results to EMS from all units. • Used RTR to adjudicate ballots. o Generated final reports and verified. o Audited reports (including tabulation devices). California Certification Functional Test Report v1.0 Page 18 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report • Verified system logging for all events and saved system logs to archive. All listed items of election completed successfully. The HiPro scanner motherboard failed and was replaced and then regression tested. A software irregularity was detected in the Canon scanner, which caused misalignment of text imprinted by the scanner. After scanning, the device imprints details about the scanning event onto the ballot and adds this information digitally to the ballot image. This was resolved by the manufacturer, and appropriate regression testing was performed to validate the resolution. Retesting for ICC Code Fix Testing was conducted to verify that the implemented code fix for the ICC Imprinter issue encountered during the Special Local Election was resolved. Two elections were run in order to perform regression testing for accuracy and retesting Imprinter functionality on the Cannon scanner. Presidential General Election Test A presidential general election was run utilizing: • Eight ICC environments (Four HiPro and four Cannon) • One EMS (EED, RTR) The presidential general election was conducted in English, Korean, Chinese, and Vietnamese. The election included ten precincts and 12 contests. Paper ballots included one bi-lingual ballot style consisting of English, Chinese, and Vietnamese. Ballots were of small size (8.5” x 11”), were multi-card for the system, and double sided. The following steps were completed with results as noted: • Prepared system for test election. • There were two measures on the ballot style and a second ballot card was used. (This includes using the front and back of the first ballot card if two- sided ballots are supported.). • Opened polls in accordance with California Use Procedures. • Scanned 180 ballots on each ICC. • Printed and verified zero reports for all devices. • Scanned all ballots on an ICC workstation. • Printed summary reports. • Uploaded results to EMS from all units. • Used RTR. California Certification Functional Test Report v1.0 Page 19 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report o Generated final reports and verified. o Audited reports (only for ICC workstations). All listed items of election completed successfully. No issues were encountered. Gubernatorial General Election Test A gubernatorial general election was run utilizing: • Eight ICC environments (Four HiPro and four Cannon) • One EMS (EED, RTR) The gubernatorial general election was conducted in English. The election included 3000 precincts or the maximum for the system. It included five contests and 10 choices. Ballots were of medium size (8.5” x 14”) for the system and double sided. The following steps were completed with results as noted: • Prepared system for test election. • Opened polls in accordance with California Use Procedures. • Scanned 110 ballots on each ICC. • Printed summary reports. • Uploaded results to EMS from all units. • Used RTR. o Generated final reports and verified. o Audited reports (including tabulation devices). All listed items of election completed successfully. No issues were encountered. Final Data Capture and Analysis • Took hashes and images from all computers. • Validated the system on each server, desktop, and one each of the ICE, ICP2, ICC, ICX Classic, and ICX Prime using the procedure provided by Dominion. • Generated “Trusted Build” software and Golden (County Release) images to be distributed by CASOS to Vendor/County. • Verified with vendor that the trusted build software was complete and ready for distribution. Dominion attested to the correctness of the files. • Verified that all drivers, including monitor video drivers, are included in the County Release. California Certification Functional Test Report v1.0 Page 20 of 21 Report Number CAF-23001-FTP-01 Dominion Democracy Suite 5.19 Voting System California Certification Functional Test Report Post Test Analysis Verified against test results that the CVSS requirements and California Elections Code and Federal requirements were met, as applicable. Evaluation of Testing The above tests were conducted using the executables created in the Trusted Build, in association with the appropriate hardware versions as declared during the current certification project for the Dominion DS 5.19 voting system, for the State of California. Summary of testing includes: • Functional test issues that were encountered were addressed and resolved by the vendor and appropriately retested and systems regression tested as required for confirmation of resolution. • Documentation updates were noted which resulted in an additional round of review and verification. Documentation is deemed sufficient at this time. • Two ICE devices were tested, one with and one without a smart card reader. In the 5.19 version of Dominion Democracy Suite, the smart card reader is not supported and is not approved for use by CASOS. As a result, the smart card reader device installed on the ICE was not tested. End of Functional Test Report California Certification Functional Test Report v1.0 Page 21 of 21 Report Number CAF-23001-FTP-01