Consultant's Source Code Test Report

Election Systems & Software EVS 6.3.0.2 Voting System Source Code Review Test Report for California Secretary of State CAF-ESS22002-SCRTR-01 Vendor Name Election Systems & Software (ES&S) Vendor System EVS 6.3.0.2 Prepared by: SLI ComplianceSM 4720 Independence St. Wheat Ridge, CO 80033 303-422-1566 www.SLICompliance.com Accredited by the Election Assistance Commission (EAC) for VSTL status. Election Systems and Software EVS 6.3.0.2 California Certification Software Test Report v1.0 Copyright © 2022 by SLI ComplianceSM, a Division of Gaming Laboratories International, LLC Revision History Date Version Author Revision Summary November 30, 2022 1.0 M. Santos Initial Release Disclaimer The information reported herein must not be used by the client to claim product certification, approval, or endorsement by NVLAP, NIST, or any agency of the Federal Government. Trademarks • SLI is a registered trademark of SLI Compliance. • All products and company names are used for identification purposes only and may be trademarks of their respective owners. Opinions and Interpretations There are no SLI opinions or interpretations included in this report. This document shall not be reproduced outside of SLI Compliance without written consent. California Certification Software Test Report Page 2 of 10 Document No. CAF-ESS22002-CSTR-01 v1.0 Election Systems and Software EVS 6.3.0.2 California Certification Software Test Report v1.0 TABLE OF CONTENTS INTRODUCTION ........................................................................................................................................... 4 REFERENCES .............................................................................................................................................. 4 SYSTEM OVERVIEW ................................................................................................................................... 4 VOTING SYSTEM SCOPE ............................................................................................................................... 4 REVIEW SPECIFICATIONS ......................................................................................................................... 5 SOURCE CODE REVIEW ............................................................................................................................... 5 REVIEW RESULTS ...................................................................................................................................... 7 DISCREPANCIES ........................................................................................................................................... 7 VULNERABILITIES ......................................................................................................................................... 8 FINAL REPORT .......................................................................................................................................... 10 California Certification Software Test Report Page 3 of 10 Document No. CAF-ESS22002-CSTR-01 v1.0 Election Systems and Software EVS 6.3.0.2 California Certification Software Test Report v1.0 Introduction This report outlines the test approach SLI Compliance (SLI) followed when performing Software Testing on the Election Systems & Software EVS 6.3.0.2 (ES&S EVS 6.3.0.2) voting system against the California Voting System Standards (CVSS). References California Voting System Standards (CVSS) System Overview Voting System Scope This section provides a description of the ES&S EVS 6.3.0.2 voting system components: • Precinct Scanners • Central Scanners • CoRE • Electionware Next • Electionware (EMS) • Scanner_C8051 • ImageUtil • libCoNG • PowerManagement The ES&S EVS 6.3.0.2 Election Management System (Electionware, PaperBallot, ImageUtil, libCoNG) represents a set of N-Tier software applications [Election Management System (EMS), Results Tally and Reporting (RTR), and Adjudication] for pre-voting and post-voting election project activities that are applicable to jurisdictions of various sizes and geo-political complexities. The ES&S EVS 6.3.0.2 Central Scanners (DS450, DS850, DS950, Scanner C8051) are central, high-speed, optical scan ballot counters (tabulators) and are used for processing absentee ballots (such as vote by mail). These ballot counter units are based on commercial-off-the-shelf (COTS) hardware coupled with custom-made ballot processing application software. They are used for high-speed, accurate, and reliable centralized scanning and counting of paper ballots. The ES&S EVS 6.3.0.2 Precinct Scanner (DS200, DS300, PowerManagement) system employs a precinct-level optical scan ballot counter (tabulator) in conjunction with an California Certification Software Test Report Page 4 of 10 Document No. CAF-ESS22002-CSTR-01 v1.0 Election Systems and Software EVS 6.3.0.2 California Certification Software Test Report v1.0 external ballot box. This tabulator is designed to scan paper ballots, interpret voting marks, and deposit the ballots into the secure ballot box. Review Specifications The following are the specifications for source code testing conducted on the ES&S 6.3.0.2 voting system. Source Code Review The ES&S EVS 6.3.0.2 voting system includes proprietary software and firmware. The ES&S EVS 6.3.0.2 voting system code base was tested to the applicable CVSS requirements. Review of the code included: • Adherence to the applicable standards in sections 5 and 7 of the CVSS. • Adherence to other applicable coding format conventions and standards including best practices for the coding language used. o ESSSYS_1’0_P_CodingStandards.pdf1 • Analysis of the program logic and branching structure. • Evaluate whether the system is designed in a way that allows meaningful analysis, including: o Whether the architecture and code are amenable to an external review. o Whether code analysis tools can be usefully applied. o Whether the code complexity is at a level that obfuscates its logic. Security considerations reviewed against the code base included: • Search for exposures to commonly exploited vulnerabilities. • Evaluate the use and correct implementation of cryptography and key management. • Analysis of error and exception handling. • Evaluate the likelihood of security failures being detected: o Evaluate whether audit mechanisms are reliable and tamper resistant. o Evaluate whether data that might be subject to tampering is properly validated and authenticated. 1 This is the standard selected and provided by ES&S that is used in development as allowed by the CVSS Requirements California Certification Software Test Report Page 5 of 10 Document No. CAF-ESS22002-CSTR-01 v1.0 Election Systems and Software EVS 6.3.0.2 California Certification Software Test Report v1.0 • Evaluate the risk that a user can escalate his or her capabilities beyond those authorized. • Evaluate the design and implementation to ensure that sound, generally accepted engineering practices are followed, checking to verify that code is defensively written against: o Bad data. o Errors in other modules. o Changes in environment. o User errors. o Other adverse conditions. • Evaluate for embedded, exploitable code (such as “Easter eggs”) that can be triggered to affect the system. • Evaluate the code for dynamic memory access features which would permit the replacement of certificated executable code or control data or insertion of exploitable code or data. • Evaluate the code for use of runtime scripts, instructions, or other control data that can affect the operation of security relevant functions or the integrity of the data. Components and coding languages involved in ES&S EVS 6.3.0.2 applications are shown in Table 1. Table 1 – ES&S EVS 6.3.0.2 Components Lines of Component Language/s Standard Code Central Scanners C/C++ 520,182 ESSSYS_1’0_P_CodingStandards.pdf Precinct Scanners C/C++, Java 507,180 ESSSYS_1’0_P_CodingStandards.pdf C/C++, CoRE 488,604 ESSSYS_1’0_P_CodingStandards.pdf TypeScript libCoNG C/C++ 301,844 ESSSYS_1’0_P_CodingStandards.pdf PowerMangament C/C++ 16,781 ESSSYS_1’0_P_CodingStandards.pdf ImageUtil C/C++ 3,612 ESSSYS_1’0_P_CodingStandards.pdf Scanner C8051 C/C++ 19,398 ESSSYS_1’0_P_CodingStandards.pdf C#, ElectionwareNext 9,122 ESSSYS_1’0_P_CodingStandards.pdf TypeScript PaperBallot C/C++ 531,426 ESSSYS_1’0_P_CodingStandards.pdf Electionware Java, SQL 1,044,954 ESSSYS_1’0_P_CodingStandards.pdf Source Code Review Tools utilized by SLI included: California Certification Software Test Report Page 6 of 10 Document No. CAF-ESS22002-CSTR-01 v1.0 Election Systems and Software EVS 6.3.0.2 California Certification Software Test Report v1.0 • Module Finder: an SLI proprietary application used to parse module names from C/C++, Java and VB code and populate the identified module names into the review documents. • Understand: a commercial application used to review code to stated requirements. Review Results Discrepancies Discrepancies are reported such that the California Secretary of State is provided with a basis for evaluating the extent to which the source code meets applicable standards. Electionware source code review No source code requirements were found to be an issue within the Electionware source code base reviewed; as a result, no discrepancies were written against the code base. PaperBallot source code review No source code requirements were found to be an issue within the Electionware PaperBallot source code base reviewed; as a result, no discrepancies were written against the code base. Central Scanners source code review No source code requirements were found to be an issue within the Central Scanners source code base reviewed; as a result, no discrepancies were written against the code base. Precinct Scanners source code review No source code requirements were found to be at issue within the Precinct Scanners source code base reviewed; as a result, no discrepancies were written against the code base. ImageUtil source code review No source code requirements were found to be at issue within the ImageUtil source code base reviewed; as a result, no discrepancies were written against the code base. libCoNG source code review No source code requirements were found to be at issue within the libCoNG source code base reviewed, as a result; no discrepancies were written against the code base. California Certification Software Test Report Page 7 of 10 Document No. CAF-ESS22002-CSTR-01 v1.0 Election Systems and Software EVS 6.3.0.2 California Certification Software Test Report v1.0 PowerManagement source code review No source code requirements were found to be at issue within the PowerManagement source code base reviewed; as a result, no discrepancies were written against the code base. CoRE source code review No source code requirements were found to be at issue within the CoRE source code base reviewed; as a result, no discrepancies were written against the code base. ElectionwareNext source code review No source code requirements were found to be at issue within the ElectionwareNext source code base reviewed; as a result, no discrepancies were written against the code base. Scanner C8051 source code review No source code requirements were found to be at issue within the Scanner C8051 source code base reviewed; as a result, no discrepancies were written against the code base. Vulnerabilities For any vulnerabilities discovered, SLI was tasked with identifying the particular standards applicable to each vulnerability. To the extent possible, reported vulnerabilities include an indication of whether the exploitation of the vulnerability would require access by: • Voter: Usually has low knowledge of the voting machine design and configuration. Some may have more advanced knowledge. May carry out attacks designed by others. They have access to the machine(s) for less than an hour. • Poll worker: Usually has low knowledge of the voting machine design and configuration. Some may have more advanced knowledge. May carry out attacks designed by others. They have access to the machine(s) for up to one week, but all physical security has been put into place before the machines are received. • Elections official insider: Has a wide range of knowledge of the voting machine design and configuration. May have unrestricted access to the machine for long periods of time. Their designated activities include: o Set up and pre-election procedures. o Election operation. o Post-election processing of results. o Archiving and storage operations. California Certification Software Test Report Page 8 of 10 Document No. CAF-ESS22002-CSTR-01 v1.0 Election Systems and Software EVS 6.3.0.2 California Certification Software Test Report v1.0 • Vendor insider: Has great knowledge of the voting machine design and configuration. They have unlimited access to the machine before it is delivered to the purchaser and, thereafter, may have unrestricted access when performing warranty and maintenance service, and when providing election administration services. SLI will not verify or demonstrate exploitability of the vulnerability, but the report of the vulnerability will identify factors involved in the exploitation. Any vulnerability theories developed by the source code review team members shall, to the extent possible, be referred to Secretary of State staff. Electionware source code vulnerability review No vulnerabilities were found were found within the Electionware source code base reviewed. PaperBallot source code vulnerability review No vulnerabilities were found within the PaperBallot source code base reviewed. Precinct Scanners source code vulnerability review No vulnerabilities were found within the Precinct Scanners source code base reviewed. Central Scanners source code vulnerability review No vulnerabilities were found within the Central Scanners source code base reviewed. ImageUtil source code vulnerability review No vulnerabilities were found within the ImageUtil source code base reviewed. libCoNG source code vulnerability review No vulnerabilities were found within the libCoNG source code base reviewed. PowerManagement source code vulnerability review No vulnerabilities were found within the PowerManagement source code base reviewed. CoRE source code vulnerability review No vulnerabilities were found within the CoRE source code base reviewed. ElectionwareNext source code vulnerability review No vulnerabilities were found within the ElectionwareNext source code base reviewed. California Certification Software Test Report Page 9 of 10 Document No. CAF-ESS22002-CSTR-01 v1.0 Election Systems and Software EVS 6.3.0.2 California Certification Software Test Report v1.0 Scanner C8051 source code vulnerability review No vulnerabilities were found within the Scanner C8051 source code base reviewed. Final Report No discrepancy findings were located within the ES&S EVS 6.3.0.2 code base. No potential vulnerabilities were identified within the ES&S EVS 6.3.0.2 code base. As per the direction given by the California Secretary of State, this software testing report does not include any recommendation as to whether or not the system should be approved. End of Software Test Report California Certification Software Test Report Page 10 of 10 Document No. CAF-ESS22002-CSTR-01 v1.0