Consultant's Source Code Test Report

Dominion Democracy Suite ImageCast Remote 5.2 (RAVBM system) Source Code Test Report for California DOM-18001-SCRTR-01 Prepared for: Vendor Name Dominion Voting Democracy Suite ICR 5.2, Vendor System RAVBM system Prepared by: 4720 Independence St. Wheat Ridge, CO 80033 303-422-1566 www.SLICompliance.com Accredited by the Election Assistance Commission (EAC) for Selected Voting System Test Methods or Services Dominion Democracy Suite ICR 5.2 RAVBM system Software Test Report Copyright 2018 by SLI ComplianceSM, a Division of Gaming Laboratories International, LLC Revision History Date Release Author Revision Summary 3.19.2018 v1.0 M. Santos Initial Release 4.4.2018 v2.0 M. Santos Updates for SOS comments Disclaimer The information reported herein must not be used by the client to claim product certification, approval, or endorsement by NVLAP, NIST, or any agency of the Federal Government. Trademarks • SLI is a registered trademark of SLI Compliance. • All products and company names are used for identification purposes only and may be trademarks of their respective owners. California Certification Software Test Report Page 2 of 8 v2.0 Dominion Democracy Suite ICR 5.2 RAVBM system Software Test Report TABLE OF CONTENTS INTRODUCTION ........................................................................................................................................... 4 REVIEW SPECIFICATIONS ......................................................................................................................... 4 SOURCE CODE REVIEW ................................................................................................................................ 4 REVIEW RESULTS....................................................................................................................................... 6 DISCREPANCIES ........................................................................................................................................... 6 VULNERABILITIES ......................................................................................................................................... 6 FINAL REPORT ............................................................................................................................................ 7 California Certification Software Test Report Page 3 of 8 v2.0 Dominion Democracy Suite ICR 5.2 RAVBM system Software Test Report INTRODUCTION The California Voting Systems Standards (CVSS) were written in such a way to be applicable to a wide variety of voting technology. Therefore, the relevant portions of the CVSS are reviewed as the relate to the Remote Accessible Vote By Mail (RAVBM) system for the purposes of this report. The use of “voting system” shall apply to the RAVBM system. This report outlines the testing SLI Compliance (SLI) followed when performing Software Testing on the Dominion Democracy Suite ImageCast Remote 5.2 (RAVBM) system (DS ICR 5.2 RAVBM system) against the California Voting System Standards (CVSS). The DS ICR 5.2 RAVBM system enables the voter to mark their ballot using a secure web-based interface, and generate and download a PDF representation of choice selections. Voters then print that ballot, and then return it to their clerk. REVIEW SPECIFICATIONS The following are the specifications for source code testing conducted on the DS ICR 5.2 RAVBM system. Source Code Review The DS ICR 5.2 RAVBM system includes proprietary software. The DS ICR 5.2 RAVBM system code base was tested to the applicable CVSS requirements. Review of the code included: • Adherence to the applicable standards in sections 5 and 7 of the CVSS • Adherence to other applicable coding format conventions and standards including best practices for the coding language used • Analysis of the program logic and branching structure • Evaluate whether the system is designed in a way that allows meaningful analysis, including: o Whether the architecture and code is amenable to an external review o Whether code analysis tools can be usefully applied o Whether the code complexity is at a level that obfuscates its logic Security considerations reviewed against the code base included: • Search for exposures to commonly exploited vulnerabilities • Evaluate the use and correct implementation of cryptography and key management California Certification Software Test Report Page 4 of 8 v2.0 Dominion Democracy Suite ICR 5.2 RAVBM system Software Test Report • Analysis of error and exception handling • Evaluate the likelihood of security failures being detected o Evaluate whether audit mechanisms are reliable and tamper resistant o Evaluate whether data that might be subject to tampering is properly validated and authenticated • Evaluate the risk that a user can escalate his or her capabilities beyond those authorized • Evaluate the design and implementation to ensure that sound, generally accepted engineering practices are followed, checking to verify that code is defensively written against: o Bad data o Errors in other modules o Changes in environment o User errors o Other adverse conditions • Evaluate for embedded, exploitable code (such as “Easter eggs”) that can be triggered to affect the system • Evaluate the code for dynamic memory access features which would permit the replacement of certificated executable code or control data or insertion of exploitable code or data. • Evaluate the code for use of runtime scripts, instructions, or other control data that can affect the operation of security relevant functions or the integrity of the data. Coding languages involved in the DS ICR 5.2 RAVBM system are shown in Table 1. Table 1 – Dominion DS ICR 5.2 RAVBM System Components Lines of Component Language/s Standard Code RAVBM system JavaScript 19175 CVSS, dvs_javaCodingStandards.pdf RAVBM system C# 184143 CVSS, StyleCop Coding Standards RAVBM system ASPX/ASCX 46310 CVSS RAVBM system Comment Lines 68546 CVSS, Relevant standard from above Source Code Review Tools utilized by SLI included: California Certification Software Test Report Page 5 of 8 v2.0 Dominion Democracy Suite ICR 5.2 RAVBM system Software Test Report • Module Finder: an SLI proprietary application used to parse module names from C/C++ and VB code and populate the identified module names into the review documents • StyleCop: a commercial application used to review code to stated requirements • Understand: a commercial application used to review code to stated requirements REVIEW RESULTS Discrepancies Discrepancies are reported such that the California Secretary of State is provided with a basis for evaluating the extent to which the source code meets applicable standards. RAVBM System Source Code Review There were 180 source code requirements found to be at issue within the RAVBM system source code base reviewed; as a result, 180 discrepancies were written against the code base. • Eighty (80) instances were noted where lines of source code exceeded more than 120 characters in length. • Seventy-one (71) instances were noted where numbers were not set to constant. • Twenty-three (23) instances were noted where variable declarations were without comment. • Ten (10) instances were noted where no default case existed. • Two (2) instances were noted where variable names not differing by more than one character were utilized. • One (1) instance was noted where inconsistent indentation was implemented. Vulnerabilities For any vulnerabilities discovered, SLI was tasked with identifying the particular standards applicable to each vulnerability. To the extent possible, reported vulnerabilities include an indication of whether the exploitation of the vulnerability would require access by: California Certification Software Test Report Page 6 of 8 v2.0 Dominion Democracy Suite ICR 5.2 RAVBM system Software Test Report • Voter: Usually has low knowledge of the voting machine design and configuration. Some may have more advanced knowledge. May carry out attacks designed by others. They have access to the machine(s) for less than an hour. • Poll worker: Usually has low knowledge of the voting machine design and configuration. Some may have more advanced knowledge. May carry out attacks designed by others. They have access to the machine(s) for up to one week, but all physical security has been put into place before the machines are received. • Elections official insider: Wide range of knowledge of the voting machine design and configuration. May have unrestricted access to the machine for long periods of time. Their designated activities include: o Set up and pre-election procedures; o Election operation; o Post-election processing of results; and o Archiving and storage operations. • Vendor insider: With great knowledge of the voting machine design and configuration. They have unlimited access to the machine before it is delivered to the purchaser and, thereafter, may have unrestricted access when performing warranty and maintenance service, and when providing election administration services. SLI will not verify or demonstrate exploitability of the vulnerability but the report of the vulnerability will identify factors involved in the exploitation. Any vulnerability theories developed by the source code review team members shall, to the extent possible, be referred to the Secretary of State staff. RAVBM System Source Code Vulnerability Review No vulnerabilities were found within the RAVBM system source code base reviewed, as a result, no findings were written against the code base. Final Report A total of 180 discrepancy findings were located within the DS ICR 5.2 RAVBM system code base. No vulnerabilities were identified within the DS ICR 5.2 RAVBM system code base. As per the direction given by the California Secretary of State, this software testing report does not include any recommendation as to whether or not the system should be approved. California Certification Software Test Report Page 7 of 8 v2.0 Dominion Democracy Suite ICR 5.2 RAVBM system Software Test Report End of Software Test Report California Certification Software Test Report Page 8 of 8 v2.0