Secretary of State Staff Test Report

A P LEX ADILLA | SECRETARY OF STATE | STATE OF CALIFORNIA OFFICE OF VOTING SYSTEMS TECHNOLOGY ASSESSMENT 1500 11th Street | Sacramento, CA 95814 | Tel 916.695.1680 | Fax 916.653.4620 | www.sos.ca.gov Dominion ImageCast Remote Version 5.10 Remote Accessible Vote by Mail System Staff Report Prepared by: Secretary of State’s Office of Voting Systems Technology Assessment November 25, 2019 Table of Contents I. Introduction .....................................................................................................1 1. Scope ...............................................................................................1 2. Summary of the Application ...........................................................1 3. Contracting and Outsourcing ..........................................................1 II. Summary of the System .................................................................................2 III. Testing Information and Results ....................................................................2 1. Background .....................................................................................2 2. Functional Testing Summary ..........................................................2 3. Software Review Testing Summary ...............................................3 4. Security and Telecommunications Testing Summary ....................4 5. Usability, Accessibility & Privacy Testing Summary ...................5 IV. Compliance with State and Federal Laws and Regulations............................6 1. Elections Code Review ....................................................................6 2. Section 508 and WCAG Review .....................................................7 V. Conclusion .......................................................................................................7 SECRETARY OF STATE’S STAFF REPORT – Dominion ImageCast Remote 5.10 2 | Page I. INTRODUCTION 1. Scope This staff report presents the test results for all phases of the certification test of Dominion Voting Systems ImageCast Remote 5.10 Remote Accessible Vote by Mail (RAVBM) system. The purpose of the testing is to test the compliance of the remote accessible vote by mail system with California and federal laws. Testing also uncovers other findings, which do not constitute non-compliance, and those findings are reported to the remote accessible vote by mail system vendor to address the issues procedurally. The procedures for mitigating any additional findings are made to the documentation, specifically the ImageCast Remote 5.10 Use Procedures. ImageCast Remote 5.10 was evaluated against the applicable portions of the California Voting Systems Standards (CVSS). The CVSS were written in such a way to be applicable to a wide variety of voting technology, therefore the relevant portions of the CVSS were reviewed as it relates to the ImageCast Remote 5.10 RAVBM system for purposes of this report. The use of “voting system” shall apply to RAVBM. 2. Summary of the Application Dominion Voting Systems Inc. submitted an application for the ImageCast Remote 5.10 remote accessible vote by mail system on October 24, 2018. ImageCast Remote 5.10 is a vendor hosted application, solely for the purposes of ballot marking pursuant to Elections Code sections 303.3 and 19283. Dominion was required to submit the following: 1) the technical documentation package (TDP); 2) two test environments; 3) source code; and 4) the Dominion ImageCast Remote 5.10 Use Procedures. 3. Contracting and Consulting Upon receipt of a complete application, the Secretary of State released a Request for Quote (RFQ) for assistance with the Security and Telecommunications, Usability, Accessibility and Privacy, and a Software Test (Source Code Review). Through the formal California contracting process and pursuant to California Elections Code section 19285, the Secretary of State awarded a contract to SLI Compliance, a division of Gaming Laboratories International, LLC, to serve in the capacity as the state- approved testing agency or expert technician to examine the remote accessible vote by mail system. II. SUMMARY OF THE SYSTEM ImageCast Remote 5.10 remote accessible vote by mail system is a web-based interface. The voter can optionally use a screen reader to navigate the ballot. After marking their ballots and reviewing their selections, the voter must print their selections. The printed paper cast record is returned to the local elections official, where it will be remade into a ballot for tabulation. The printed paper cast record and the ballot are kept together for auditing purposes. III. TESTING INFORMATION AND RESULTS 1. Background The ImageCast Remote 5.10 remote accessible vote by mail system application is a fully hosted service provided by Dominion. The application platform used in the ImageCast Remote 5.10 platform is based on Windows Server 2012 R2 and the development environment is Visual Studio 2015. Users can access the application platform using industry standard web browsers. Functional Accessibility testing of this system was conducted by SOS Staff. The testing took place in June, July and October of 2019. Approximately 29 volunteers throughout the state of California participated. Testers were given the option to take a pre-test survey, of which 14 of the testers responded. The survey results can be found in Attachment A. Additionally, testers provided general written feedback to the SOS about their experience. The Software, Security and Telecommunications, and Usability, Accessibility, and Privacy testing was performed by SLI in a mirroring test environment. 2. Functional Testing Summary System Configuration: The ImageCast Remote 5.10 remote accessible vote by mail system is supported by a fully redundant network and application architecture protecting against a single point of failure. For the functional test, the SOS had two test elections configured based on California elections as described below: 1. State of California General Election 2. Local Jurisdiction – Fictional Rank Choice Election Each election performed as expected, however the instructions for the Rank Choice Election were consistently noted to be confusing. SECRETARY OF STATE’S STAFF REPORT – Dominion ImageCast Remote 5.10 2 | Page Undervotes The system warned of undervotes, and allowed the tester to proceed to the end of the ballot to print the paper cast record. Overvotes The system warned of overvotes, and allowed the tester to proceed to the end of the ballot and print the paper cast record. Write-Ins The system allowed the tester to make write in selections with a fifty (50) character limit. Printed Paper Cast Record Each paper cast record printed displays the Election Name, Election Type (General/Primary) and the date of the Election within the header. The appropriate Ballot Style is listed, in addition to one or more QR code, and the voter’s precinct is also listed. Each contest along with the voter’s selections in a human readable format, are printed on the paper cast record. The QR code contains the voter’s selections, however it cannot be used for tabulation or read using QR code readers from application stores. The QR code is encrypted, and can only be decrypted by an election official, using a decoder built into the adjudication system. The election official can verify the selections correspond to those listed on the paper cast record, and either hand duplicate or utilize the adjudication system to duplicate the paper cast record onto a traditional ballot for tabulation. The paper cast record is kept with the duplicated ballot for auditing purposes. 3. Software Review Testing Summary SLI performed Software Testing (Source Code Review) of the ImageCast Remote 5.10 RAVBM system. SLI utilized the following tools: Module Finder: an SLI proprietary application used to parse module names from C/C++ and VB code and populate the identified module names into the review documents. StyleCop: a commercial application used to review code to stated requirements. Understand: a commercial application used to review code to stated requirements. SLI found no discrepancy findings or vulnerabilities, within the ImageCast Remote 5.10 RAVBM system. See the testing consultant’s Source Code Test Report located on the Secretary of State website for more information. SECRETARY OF STATE’S STAFF REPORT – Dominion ImageCast Remote 5.10 3 | Page 4. Security and Telecommunications Testing SLI performed Security and Telecommunications Testing on the ImageCast 5.10 Remote System over three Phases. The first phase was a comprehensive documentation review for compliance with the applicable portions of the CVSS. The second phase consisted of Functional Security Testing, followed by a third phase of Telecommunications and Data Transmission Testing. The issues identified during Security and Telecommunications testing are listed below, along with mitigations by the vendor. Table 4A: Security and Telecommunications Issue Identified Vendor Mitigation CVSS Section 7.4.5 – The documentation Dominion provided high-level documentation doesn’t describe a process used to verify the about the process and procedure to verify the software distributed is the software software version. provided by the NSRL or designated repository. The documentation doesn’t provide a procedure or functionality to verify that the software is the certified software by comparison. CVSS Section 7.4.6 – The documentation Dominion provided high-level documentation doesn’t reference any type of digital about the process and procedure to verify the verification on software prior to installation. software version. No documentation on the method provided by external interface or equipment used to verify software on the system. No documentation about a mechanism for detecting unauthorized software. CVSS Section 7.2.1 – General Access This is the behavior in Firefox (since version 38), Control – On the administrative application Google Chrome (since 34), and Internet Explorer it was found that the password field has (since version 11). The vendor has no control over autocomplete enabled. this functionality. Server-Side JavaScript Code Injection The value posted in the ddlValidStatus parameter is not dynamically evaluated on the server side. The System compares the submitted value with all available pre-defined values on the server side. If such situation occurs, the default value will still be displayed in the drop-down. Cross-Site Scripting (Reflected) This issue can manifest itself only on the DevLogViewer.aspx (Developer Log) page on the administrative site where the system displays errors, amongst other information, which is usually of little meaning to election administration users. In this case system logs Error Message: A potentially dangerous Request.QueryString value SECRETARY OF STATE’S STAFF REPORT – Dominion ImageCast Remote 5.10 4 | Page Table 4A: Security and Telecommunications was detected from the client, but also logs the entire error message which is rendered on the client as html. Thus, this cannot affect regular election operations. See the testing consultant’s Security and Telecommunications Report located on the Secretary of State website for more information. 5. Usability, Accessibility and Privacy Testing Summary SLI performed Security and Telecommunications Testing on the ImageCast 5.10 Remote System over three Phases. The first phase was a comprehensive documentation review for compliance with the applicable portions of the CVSS. The second phase consisted of Usability and Accessibility Testing, followed by a third phase of Privacy Testing. The issues identified during Usability, Accessibility and Privacy testing are listed below, along with mitigations by the vendor. Table 5A Usability, Accessibility and Privacy Testing Issue Identified Vendor Mitigation CVSS Section 3.2.6 - Perceptual Issues No voter will be accessing the administrative portion of - One anomaly was found with the ImageCast Remote 5.10. Administrative portion of the application. The white or grey text on red background does not meet the WCAG contrast ratio. CVSS Section 3.2.7c.ii- No Repeating The vendor couldn’t reproduce the issue on supported Keys - Review of the requirement browsers inside the voter portion of the ImageCast showed that navigation keys (up and Remote 5.10 application. down arrows, tab) did have a repetitive effect. CVSS Section 3.2.8.e - Review of the All sessions timeouts are handled by the server, ImageCast Remote 5.10 showed that because of the nature of the ImageCast Remote 5.10 after five minutes, no application (being disconnected from the server), the inactivity warning was issued. ImageCast Remote 5.10 user interface doesn't invalidate the session. By the same token, voters can enter the application unlimited times, rules and regulations in California for the ImageCast Remote 5.10 application have safeguards in place to prevent multiple ballots by the same voter from being accepted in the election office. CVSS Section 3.2.8.f - Review of the All sessions timeouts are handled by the server, ImageCast Remote 5.10 showed that because of the nature of the ImageCast Remote 5.10 SECRETARY OF STATE’S STAFF REPORT – Dominion ImageCast Remote 5.10 5 | Page Table 5A Usability, Accessibility and Privacy Testing after five minutes, no application (being disconnected from the server), the alert was issued, nor did the system ImageCast Remote 5.10 user interface doesn't convert to an inactive state. invalidate the session. By the same token, voters can enter the application unlimited times, rules and regulations in California for the ImageCast Remote 5.10 application have safeguards in place to prevent multiple ballots by the same voter from being accepted in the election office. See the testing consultant’s Usability, Accessibility, and Privacy Report located on the Secretary of State website for more information. IV. COMPLIANCE WITH STATE AND FEDERAL LAWS AND REGULATIONS 1. Elections Code Review 1)§19283 (b) Remote accessible vote by mail system standards adopted by the Secretary of State pursuant to subdivision (a) shall include, but not be limited to, all of the following requirements: (1) The machine or device and its software shall be suitable for the purpose for which it is intended. (2) The remote accessible vote by mail system shall preserve the secrecy of the ballot. (3) The remote accessible vote by mail system shall be safe from fraud or manipulation. (4) The remote accessible vote by mail system shall be accessible to voters with disabilities and to voters who require assistance in a language other than English if the language is one in which a ballot or ballot materials are required to be made available to voters. The system meets these requirements. The system demonstrated English, Spanish and Chinese languages, however with external translation services; the system can support other languages as required. 2) §19295 A remote accessible vote by mail system or part of a remote accessible vote by mail system shall do not any of the following: (a)Have the capability, including an optional capability, to use a remote SECRETARY OF STATE’S STAFF REPORT – Dominion ImageCast Remote 5.10 6 | Page server to mark a voter’s selections transmitted to the server from the voter’s computer via the Internet. (b) Have the capability, including an optional capability, to store any voter identifiable selections on any remote server. (c) Have the capability, including the optional capability, to tabulate votes. The system meets these requirements. 2. Section 508 and WCAG Compliance Review ImageCast Remote 5.10 was written to WCAG 2.0 guidelines to implement accessibility features. The system also complies with the applicable portions of Section 508. V. CONCLUSION The ImageCast Remote 5.10 remote accessible vote by mail system, in the configuration tested and documented by the Dominion Voting Systems Inc.’s ImageCast Remote 5.10 Use Procedures, meets all applicable California and federal laws. The ImageCast Remote 5.10 remote accessible vote by mail system is compliant with all California and federal laws. SECRETARY OF STATE’S STAFF REPORT – Dominion ImageCast Remote 5.10 7 | Page Dominion ImageCast Remote 5.10 Remote Accessible Vote by Mail Pre-Test Survey Wave I Q1 Name Answered: 14 Skipped: 0 # RESPONSES DATE Q2 Age Answered: 14 Skipped: 0 # RESPONSES DATE 1 30 6/23/2019 10:25 PM 2 30 6/22/2019 12:04 AM 3 49 6/21/2019 10:16 PM 4 66 6/21/2019 9:34 PM 5 73 6/21/2019 6:23 PM 6 30 6/21/2019 2:57 AM 7 71 6/20/2019 6:25 AM 8 52 6/19/2019 3:59 AM 9 60 6/19/2019 1:28 AM 10 53 6/19/2019 12:58 AM 11 38 6/18/2019 10:13 PM 12 66 6/18/2019 6:35 PM 13 48 6/18/2019 6:12 PM 14 57 6/18/2019 3:34 PM Q3 Phone Answered: 14 Skipped: 0 # RESPONSES DATE 1 / 9 Dominion ImageCast Remote 5.10 Remote Accessible Vote by Mail Pre-Test Survey Wave I Q4 Email Answered: 14 Skipped: 0 # RESPONSES DATE Q5 Please describe the disabilities or difficulties you have that may interfere with you being able to mark a ballot privately and independently. Answered: 12 Skipped: 2 # RESPONSES DATE 1 visual impairment 6/23/2019 10:27 PM 2 I am blind. 6/22/2019 12:07 AM 3 Unable to read regular print due to total blindness 6/21/2019 10:35 PM 4 Totally blind 6/21/2019 9:35 PM 5 Iegally blind 6/21/2019 6:26 PM 6 Low vision, blindness 6/21/2019 3:00 AM 7 I am blind. 6/20/2019 6:26 AM 8 Disabling AIDS which can limit my mobility and abilities to vote. 6/19/2019 4:00 AM 9 I am totally blind, therefor I cannot read any of the different fforms of ballots 6/19/2019 1:31 AM 10 I am totally blind 6/18/2019 10:14 PM 2 / 9 Dominion ImageCast Remote 5.10 Remote Accessible Vote by Mail Pre-Test Survey Wave I 11 visual impairment 6/18/2019 6:37 PM 12 Mobility and dexterity impairments 6/18/2019 6:15 PM Q6 Which language would you prefer voting in ? Answered: 12 Skipped: 2 English Spanish Chinese Japanese Tagalog Thai Vietnamese Hindi Khmer Korean 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ANSWER CHOICES RESPONSES English 100.00% 12 Spanish 0.00% 0 Chinese 0.00% 0 Japanese 0.00% 0 Tagalog 0.00% 0 Thai 0.00% 0 Vietnamese 0.00% 0 Hindi 0.00% 0 Khmer 0.00% 0 Korean 0.00% 0 TOTAL 12 # OTHER (PLEASE SPECIFY) DATE There are no responses. Q7 Have you ever voted before? Answered: 13 Skipped: 1 3 / 9 Dominion ImageCast Remote 5.10 Remote Accessible Vote by Mail Pre-Test Survey Wave I Yes No 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ANSWER CHOICES RESPONSES Yes 100.00% 13 No 0.00% 0 TOTAL 13 Q8 If "No", what has prevented you from voting in the past? Answered: 2 Skipped: 12 # RESPONSES DATE 1 Exactly what happened during last presidential election (my first and only vote): My helper marked 6/21/2019 10:35 PM the person I said should not be chosen because that was his candidate. I knew because the ballot was returned to me for no signature. 2 Sometimes I do not vote because I am unable to vote on my own. 6/19/2019 1:01 AM Q9 What method of voting do you use most often? Answered: 13 Skipped: 1 Polls By Mail Early Voting 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ANSWER CHOICES RESPONSES Polls 38.46% 5 By Mail 30.77% 4 Early Voting 30.77% 4 TOTAL 13 # OTHER (PLEASE SPECIFY) DATE 1 Online electronic 6/21/2019 3:00 AM 2 talking voting machine 6/19/2019 1:31 AM 4 / 9 Dominion ImageCast Remote 5.10 Remote Accessible Vote by Mail Pre-Test Survey Wave I Q10 Have you ever used a voting system with any special accessibility accommodations? Answered: 13 Skipped: 1 Yes No 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ANSWER CHOICES RESPONSES Yes 61.54% 8 No 38.46% 5 TOTAL 13 Q11 If "Yes", which type of voting system? Answered: 10 Skipped: 4 Electronic Ballot Marking Device 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ANSWER CHOICES RESPONSES Electronic 60.00% 6 Ballot Marking Device 40.00% 4 TOTAL 10 # OTHER (PLEASE SPECIFY) DATE There are no responses. Q12 What operating system are you using? Answered: 11 Skipped: 3 5 / 9 Dominion ImageCast Remote 5.10 Remote Accessible Vote by Mail Pre-Test Survey Wave I Windows 10 Windows 8.1 Windows 7 macOS 10.12 Android Linux 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ANSWER CHOICES RESPONSES Windows 10 54.55% 6 Windows 8.1 9.09% 1 Windows 7 18.18% 2 macOS 10.12 9.09% 1 Android 0.00% 0 Linux 0.00% 0 TOTAL 11 # OTHER (PLEASE SPECIFY) DATE 1 IOS12 6/22/2019 12:15 AM 2 Ios 12.3 6/21/2019 6:41 PM 3 I am not sure but my system is brand new 6/19/2019 1:05 AM Q13 What internet browser are you using? Answered: 11 Skipped: 3 Microsoft Edge 14 Internet Explorer 11 Safari 10.1 Google Chrome 59.0 Mozilla Firefox 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 6 / 9 Dominion ImageCast Remote 5.10 Remote Accessible Vote by Mail Pre-Test Survey Wave I ANSWER CHOICES RESPONSES Microsoft Edge 14 9.09% 1 Internet Explorer 11 36.36% 4 Safari 10.1 18.18% 2 Google Chrome 59.0 18.18% 2 Mozilla Firefox 18.18% 2 TOTAL 11 # OTHER (PLEASE SPECIFY) DATE There are no responses. Q14 List the auxiliary devices and/or software you are using for electronic accessibility. Answered: 10 Skipped: 4 JAWS NVDA Apple Voiceover Windows Narrator 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ANSWER CHOICES RESPONSES JAWS 40.00% 4 NVDA 20.00% 2 Apple Voiceover 30.00% 3 Windows Narrator 0.00% 0 TOTAL 10 # OTHER (PLEASE SPECIFY) DATE 1 Windows accessibility settings 6/18/2019 6:21 PM Q15 What is the make and model of the hardware you are using (PC, Laptop, Tablet, etc.)? Answered: 11 Skipped: 3 # RESPONSES DATE 1 iPhone7 6/22/2019 12:15 AM 2 Dell laptop 6/21/2019 10:40 PM 7 / 9 Dominion ImageCast Remote 5.10 Remote Accessible Vote by Mail Pre-Test Survey Wave I 3 Dell Laptop 6/21/2019 9:37 PM 4 Iphone 6 6/21/2019 6:41 PM 5 Dell Computer 6/20/2019 6:28 AM 6 ASUS all in one 6/19/2019 4:02 AM 7 Desktop PC Processor: Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz 2.79 GHz Installed memory 6/19/2019 1:43 AM (RAM): 8.00 GB System type: 64-bit Operating System Pen and Touch: No Pen or Touch Input is available for this Display 8 PC 6/19/2019 1:05 AM 9 TC 6/18/2019 10:17 PM 10 Dell 6/18/2019 6:40 PM 11 Dell OptiPlex 3060 6/18/2019 6:21 PM Q16 What is the printer make and model you are using? Answered: 10 Skipped: 4 # RESPONSES DATE 1 HP ENVY 5530 6/22/2019 12:15 AM 2 HP Laser Jet 1100 6/21/2019 10:40 PM 3 HP 6/21/2019 9:37 PM 4 H 8750 6/21/2019 6:41 PM 5 Brother Printer 6/20/2019 6:28 AM 6 hp 6/19/2019 4:02 AM 7 HP LaserJet P2055d UPD PCL 6 on network 6/19/2019 1:43 AM 8 Epson Workforce WF2750 6/19/2019 1:05 AM 9 Kyocera 6/18/2019 10:17 PM 10 HP 3010 series 6/18/2019 6:21 PM Q17 What antivirus software are you using? Answered: 11 Skipped: 3 McAfee Norton Windows Defender Kaspersky 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ANSWER CHOICES RESPONSES McAfee 9.09% 1 Norton 9.09% 1 8 / 9 Dominion ImageCast Remote 5.10 Remote Accessible Vote by Mail Pre-Test Survey Wave I 54.55% 6 Windows Defender Kaspersky 18.18% 2 TOTAL 11 # OTHER (PLEASE SPECIFY) DATE 1 None 6/22/2019 12:15 AM 2 AVG free and windos defender 6/19/2019 1:43 AM 3 not sure 6/19/2019 1:05 AM 4 MalWarebytes 6/18/2019 10:17 PM 5 Nod32 6/18/2019 6:40 PM Q18 Is your antivirus software up to date? Answered: 11 Skipped: 3 Yes No 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ANSWER CHOICES RESPONSES Yes 90.91% 10 No 9.09% 1 TOTAL 11 9 / 9