County of Los Angeles Vsap 2.1 Staff Report

A P LEX ADILLA |SECRETARY OF STATE | STATE OF CALIFORNIA OFFICE OF VOTING SYSTEMS TECHNOLOGY ASSESSMENT 1500 11th Street | Sacramento, CA 95814 | Tel 916.695.1680 | Fax 916.653.4620 | www.sos.ca.gov County of Los Angeles’ Voting Solutions for All People (VSAP) Tally 2.1 Voting System Staff Report Prepared by: Secretary of State’s Office of Voting Systems Technology Assessment August 14, 2020 Table of Contents I. Introduction ......................................................................................... 1 1. Scope ................................................................................... 1 2. Summary of the Application ................................................. 1 3. Contracting and Outsourcing ............................................... 2 II. Summary of the System .................................................................... 2 III. Testing Information and Results ........................................................ 3 1. Background .......................................................................... 3 2. Functional Testing Summary ............................................... 3 3. Software (Source Code) Testing Summary .......................... 4 4. Security and Telecommunications Testing Summary .......... 7 5. Volume Testing Summary .................................................... 10 6. Accessibility, Usability and Privacy ....................................... 12 7. Hardware Testing ................................................................. 18 IV. Compliance with State and Federal Laws and Regulations ............... 19 V. Conclusion .......................................................................................... 26 I. INTRODUCTION 1. Scope This report presents the test results for all phases of the certification test of the County of Los Angeles’ Voting Solutions for All People (VSAP) 2.1 voting system. The purpose of the testing is to test the compliance of the voting system with California and federal laws, including the California Voting System Standards (CVSS). Testing also uncovers other findings, which do not constitute non- compliance, and those findings are reported to the County of Los Angeles to address the issues procedurally. The procedures for mitigating any additional findings are made to the documentation, specifically the County of Los Angeles’ VSAP 2.1 Use Procedures. 2. Summary of the Application The County of Los Angeles submitted an application for the VSAP 2.1 voting system on February 27, 2020. The system is comprised of the following major components: a. Tally Version 2.2.2.31 b. Ballot Marking Device (BMD) Version 1.6 c. FormatOS Version 1.6.1 d. BMD BASI Version 1.6 e. BMD BESI Version 1.6 f. BMD Manager (BMG) Version 1.5 g. VSAP Ballot Layout (VBL) Version 1.1.3 h. Enterprise Signing Authority (ESA) (commercial off-the-shelf- equipment [COTS], Version 1.0 i. IBML - ImageTrac 6400 (COTS) In addition to each of the aforementioned components, which includes the executable code and the source code, the County of Los Angeles was required to submit the following: (1) the technical documentation package (TDP); (2) all the hardware and software components, including all peripheral devices needed for all phases of testing; (3) and the VSAP 2.1 Use Procedures. 1 | P age 3. Contracting and Outsourcing Upon receipt of a complete application, the Secretary of State released a Request for Quote (RFQ) for assistance with testing of the VSAP 2.1 voting system. Through the formal California contracting process and pursuant to California Elections Code section 19285, the Secretary of State awarded a contract to SLI Compliance, a division of Gaming Laboratories International, LLC, to serve in the capacity as the state-approved testing agency or expert technician to examine the voting system. Hardware testing of the ballot marking devices was subcontracted by SLI to National Technical Systems (NTS) Laboratories. II. SUMMARY OF THE SYSTEM The VSAP 2.1 voting system consists of the following components: • Tally Version 2.2.2.31 —Hardware and software that captures and processes ballot images ensuring that votes on paper ballots are digitally represented and counted, storing the images as Cast Vote Records (CVRs). • Ballot Marking Device (BMD) Version 1.6—The central component of the voting system and the main interface for the voter. It includes a touchscreen, an audio-tactile interface, a paper handler, a QR code scanner, a dual-switch input, and an integrated ballot box. The BMD is used by voters to generate, verify, and cast paper ballots. • FormatOS Version 1.6.1 – Application used to wipe new BMD devices. • BMD BASI Version 1.6 – Application software for the BMD. • BMD BESI Version 1.6 – Application for election software for BMD. • BMD Manager (BMG) Version 1.5 — Ballot marking device manager application for managing BMDs including software, ballot configurations, and post-election data. • VSAP Ballot Layout (VBL) Version 1.1.3—Defines ballot print formats for BMD, Vote by Mail (VBM), Remote Accessible Vote by Mail (RAVBM) and Uniformed Overseas Citizens Absentee Voting Act (UOCAVA) ballots. VBL also generates data files and packages to configure the BMD, BMG, ISB, and Tally. • Enterprise Signing Authority (ESA) Version 1.0—A cryptographic sub- system (hardware and software) that ensures components of the VSAP 2 | P age conform to security standards and that the data passed to components is secure and authenticated. • IBML - ImageTrac 6400 – High speed scanner used in conjunction with Tally tabulation software. III. TESTING INFORMATION AND RESULTS 1. Background The Secretary of State staff in conjunction with SLI, oversaw all phases of testing of the system, including Functional, Software Testing (Source Code Review), Security and Telecommunications (Red Team Penetration Testing), Volume, and Accessibility, Usability and Privacy Testing, and Hardware Testing. 2. Functional Testing Summary System Configuration: The system is self-contained on an air gapped network, per the CVSS requirements. SLI performed the task of creating the Trusted Build of the VSAP 2.1 voting system. The artifacts produced, will be kept, and distributed by the Secretary of State. This version is solely for the use of Los Angeles County. Functional Testing: The Functional Testing consisted of following the Use Procedures to import the following four (4) test elections into the environment: ▪ Presidential Primary (2020 Election) – This election tested the limitations of ballot styles that can be used within the system, in addition to language support (audio and visual). ▪ General Election (Los Angeles County 2016) ▪ Recall Election (2003 Election) – This election tested the capacity to list 135 candidates. ▪ Special Election – A special election with two congressional districts and one municipality. Temporary workers hand marked each of the ballots, including some marginal marks to test out stacking functionality. Additionally, ballots were cast using the ballot marking devices (BMD). Each election was tabulated using the IBML high speed scanner. A manual tally of the results was conducted, and the reporting results confirmed accurate. 3 | P age A detailed report of the Functional Testing conducted on the system can be found on our website. 3. Software Testing (Source Code) Review Summary The review was conducted by SLI. SLI evaluated the security and integrity of the voting system by identifying any security vulnerabilities that could be exploited to: • Alter vote recording, • Alter vote results, • Alter critical data (such as audit logs), or • Conduct a “denial of service” attack on the voting system. SLI’s review of the VSAP 2.1 source code against the applicable standards of the CVSS in sections 5 and 7 determined the following discrepancies were found in the source code Table 3A: VSAP 2.1 Source Code Findings: Table 3A: VSAP 2.1 Source Code Findings 1 catch()” issue, five instances (CVSS 5.2.5.a) Known Language Vulnerability, one instance (CVSS 5.2.8.b.v) 2 Unused function issue, one instance (CVSS 5.2.7.e) 3 compArray issue, one instance (CVSS 5.2.5.a) 4 Each of the findings have no impact on the functionality of the system, and therefore are deemed non-issues. Additionally, SLI was tasked to evaluate the Source Code for previous findings in the VSAP 2.0 source code, classified as “low.” The corresponding numbers of the findings begin on page 82 of the VSAP 2.0 Software Report and are identified as #15 - 25. The results are listed below in the following table 3B: VSAP 2.0 Source Code Findings (Review): 4 | P age 3B: VSAP 2.0 Source Code Findings (Review): # Finding Staff Analysis Severity 15 Third-party code Resolved - All systems leverage third Low provides an attack party tools in one form or another. Los vector and must be Angeles County continues to actively monitored for changes monitor all components for potential and reviewed when threats. If the need arises to address they occur. those threats, the County must apply to the Secretary of State’s office to make any modifications to the system, pursuant to California Elections Code Section 19216. 16 SQL database Resolved - This issue is classified as Low initialization seed data resolved, as SLI’s assessment is entirely optional, determined there was no instance of the and INSERT IGNORE setting. can lead to unforeseen consequences. 17 Database creation Resolved - This issue is classified as Low sets resolved, as SLI’s assessment only_full_group_by to determined there was no instance of the null, creating the setting. possibility of inconsistent data on select. 18 Static code analysis of Resolved - This finding is informational, Low Go source code. as the tool used to assess the code scans for preferences in the code. Functionally, this finding has no impact on the system. 19 Static code analysis of Resolved - This finding is informational, Low JavaScript source as the tool used to assess the code scans code. for preferences in the code. Functionally, this finding has no impact on the system. 20 Public vulnerability Resolved - The system is air gapped, Low search. See sections thus reducing the attack surface. Los 4.2, Published Angeles County continues to actively Vulnerabilities, and monitor all components for potential 5.1, Public threats. If the need arises to address those threats, the County must apply to 5 | P age 3B: VSAP 2.0 Source Code Findings (Review): # Finding Staff Analysis Severity Vulnerability Search, the Secretary of State’s office to make for complete results. any modifications to the system, pursuant to California Elections Code Section 19216. 21 Description: The CA Resolved - The County has procedures Low certificate and key are in place, including detailed documentation stored in tmp and set in the VBL Build Guide to remedy this to 777 file issue. The issue is considered resolved. permissions. Programmatic copy of the CA cert and key to the cluster machines makes sense as its going to be necessary for later steps in the process, but in this case the file is set to 777 permissions, which means that all users have all permissions on these files. 22 Point of origin is not Resolved - SLI’s evaluation of the Low taken into template file did not reveal usage of the consideration with standard syntax of “%” to signify the any authentication entries. host wild card for users being inserted into the database. This issue is considered resolved. 24 Python 2 reaches end Resolved - This issue is classified as Low of life at the end of resolved as the system’s use of python this year and will not code is now Python 3. be supported after 2019-12-31. Any future security vulnerabilities found in Python 2 will not be fixed. 6 | P age 3B: VSAP 2.0 Source Code Findings (Review): # Finding Staff Analysis Severity 25 Calico container Unresolved pending Action from Low securityContext set to Calico - This item is actively being privileged = true. monitored by the County and will be securityContext: true addressed in a future version pending is set for the container action from Calico, a third-party tool used Calico, which controls in the system. network functions. A Source Code Review report, including the findings and vendor responses and/or mitigations can be found on our website. 4. Security and Telecommunications Testing (Red Team) Summary Security and Telecommunications (Red Team Penetration) testing of the VSAP 2.1 system was conducted in July of 2020, by SLI. The Security and Telecommunications Testing resulted in four findings requiring a response and/or mitigation. Each is described in Table 4A: Security Findings: Table 4A: Security Findings Finding County Mitigation/Response Staff Analysis Access Control Authorization (BMD Full disk encryption will remedy Partially & BMG) this issue and the county has Addressed: submitted a plan to implement The County of full disk encryption. Los Angeles’ plan for Full Disk Encryption is currently being evaluated by the Secretary of State’s office. Access Control Tally and VBL Full disk encryption is Partially planned for the VSAP Tally 3.0 Addressed: release. We believe this will go The County of a long way towards mitigating Los Angeles’ 7 | P age Table 4A: Security Findings Finding County Mitigation/Response Staff Analysis these concerns. During the plan for Full process of addressing full disk Disk encryption and FIPS, we will Encryption is also be evaluating more robust currently being options for secret storage evaluated by across the system. Importantly, the Secretary most of these secrets are of State’s generated during the install office. process (and the rest are loaded Additionally, in from the ESA), so secrets are the county has environment specific. This until June 30, means that in order to extract of 2021 to production secrets, an attacker meet the FIPS would need to gain access to requirement. the production environment. BMG Telecommunications Access In response to the Use Partially Control Conditions on the VSAP 2.0 Addressed: Certificate, a plan for Full Disk The County of Encryption implementation was Los Angeles submitted by Los Angeles plan for Full RR/CC on July 24th. Disk Encryption is currently being evaluated by the Secretary of State’s office. Nessus Scan – 2 High Risk and 14 Disallowing USB access to the The results are Medium, and 5 Low system components, alongside mitigated by the available physical security physical measures, logging, and use of security paper ballots by all voters measures as provide measures of security, noted. technical, tactical, and strategic. Additionally, if The County diligently maintains the county the air gapped configuration of addresses the the system, too. findings by making updates, those updated must be verified, tested, and approved by the SOS. 8 | P age Additionally, SLI was tasked to evaluate the VSAP 2.1 for previous findings in the VSAP 2.0 Security Report. The findings begin on page 17 of the VSAP 2.0 Security and Telecommunications Report. The results are listed below in the following table 4B: VSAP 2.0 Security Findings (Review): Table 4B: VSAP 2.0 Security Findings (Review) Finding County Mitigation/Response Staff Analysis FIPS 140-2 Compliant N/A In Process – Cryptographic Module Utilization The County of Los Angeles has until June 30, of 2021 to meet the FIPS requirement. Full Disk Encryption The county has submitted a plan Partially to the Secretary of State to Addressed: implement Full Disk Encryption. The County of Los Angeles plan for Full Disk Encryption is currently being evaluated by the Secretary of State’s office Dependency on Root Access Reducing programmatic root Partially access will require significant Addressed: architecture changes to BMG. The results are For now, as the Report shows, mitigated by we successfully removed USB physical access to the system to better security prevent an intruder from being measures as able to access and thus noted. leverage root access. Additionally, if the county addresses the findings by making updates, those updated must be verified, tested, and approved by the SOS. Shared/Static Secrets In response to the Use Partially Conditions on the VSAP 2.0 Addressed: 9 | P age Table 4B: VSAP 2.0 Security Findings (Review) Finding County Mitigation/Response Staff Analysis Certificate, a plan for Full Disk The County of Encryption implementation was Los Angeles’ submitted by Los Angeles plan for Full RR/CC on July 24th. Disk Encryption is currently being evaluated by the Secretary of State’s office. A detailed report of the Security and Telecommunications Testing (Red Team Penetration) can be found on or website. 5. Volume Testing Summary The Volume Test simulates conditions in which the ballot marking devices would be used on Election Day. Fifty (50) BMD units were tested during the volume test, with nine (9) temporary workers marking and casting one hundred (100) ballots per device over 2.5 days. During the VSAP 2.0 Volume Test, the ballot marking devices experienced a jamming rate of 0.0096%. Some of the devices were later determined to have faulty gears within the printing units, resulting in those devices having the gears replaced. The Volume Test for VSAP 2.1 consisted of an even split of twenty-five (25) ballot marking devices impacted by the replacement gears in the printing unit and twenty -five (25) devices that did not require the gears to be replaced. The BMD devices were each loaded with the March 2020 Presidential Primary Election, including 13 languages, with complete audio files for the testers to choose from. The testing resulted in the following findings: 1. Timid Feeds – Out of the 5,000 ballots fed into the ballot marking device units during testing 149 experienced timid feeds or Ballot Page Metadata (BPM) errors warranting the ballot to be refed into the device. The 149 timid feeds occurred in 23 of the 50 units. The BMD device by design gently pulls the ballot into the device, in a timid, push/pull manner. Testers would sometimes attempt to force feed the ballot or would pull the ballot back after the device began to gently pull the ballot into the device. When the tester did one of those two things, the BMD would appear to pull the ballot in, but immediately eject it. Clear instructions to voters and poll workers regarding the timid feed will mitigate the chances of this occurring. 2. QR Code Errors – 10 | P age a. Five BMDs encountered a “QR Code not Read” error upon initial insertion into the ballot box. In all instances, the ballots were tried on another device, and successfully read. Further examination of the ballots determined that the QR codes printed on the ballots, used to activate the voting session were slightly skewed when printed, thus causing random anomalies in some machines, but not others. b. One BMD unit, #40,would intermittently reject seemingly valid QR codes. When inserting the ballot, the feeder would start to take the ballot in, and then reject it. The BMD was restarted, but the device continued to reject the ballots. Preventative maintenance was also performed on the unit, but the unit continued to reject ballots. The device was removed from the test and taken for additional troubleshooting. The following diagnosis was given by the repair technician: “The unit was taken to the Repair Center for further diagnosis. There it was found that scans from the unit had a grey vertical band present. The grey vertical band extends through the Ballot Page Metadata (BPM) code located in the upper left of the ballot paper and to the right of the BPM, and from the top (leading) edge of the ballot to the bottom edge of the ballot. The BMD software depends on being able to discern the leading edge of the ballot. Initiation of the scan of the BPM code, and detection of an attempt to insert the ballot skewed relative to the printer intake slot are two functions dependent on accurate leading edge discernment of the leading edge, causing the unit to reject the ballot. The grey band is on the edge of visibility to the unit’s software, meaning that if the band were darker in color it would result in a hard failure, thus rejection of every insertion attempt, and not the somewhat random behavior observed in the test, where only some fraction of attempted insertions lead to rejection.” 3. Blank Screen & Paper Jam – One device encountered a blank screen and paper jam after the tester inserted the ballot. The ballot was removed, following protocol, but the screen remained blank. After a hard reboot of the system, the tester was able to proceed without further incident. Los Angeles County was required as a condition of certification of the VSAP 2.0 voting system to submit to our office a plan to improve the ballot marking device paper handling and remediate the jamming and misfeed rate of the ballot marking device. The former rate was 0.0096%, which was not acceptable as prescribed in the California Voting Systems Standards (CVSS), which has an allowable rate of no more than 0.002%. One device out of the 51 overall devices tested experienced a jam, thus bringing the paper handling rate to 0.0002%, which is within the allowable rate as prescribed by the CVSS. A detailed Volume Test report, including error logs can be found on our website. 11 | P age 6. Accessibility, Usability and Privacy Testing Summary The Accessibility, Usability and Privacy testing took place over two days in July 2020. Eight (8) volunteer testers participated in the testing. Due to the COVID 19 pandemic, volunteers tested the device either at their homes, outside or at a local community center parking lot. The volunteers were from the Los Angeles County accessibility community. The BMD devices used for this test were programmed with the March 2020 Presidential Primary Election. Each volunteer tester was asked to complete two (2) voting sessions, using the BMD. Upon completion of the session, all volunteer testers were asked to participate in a post-test survey regarding their experience. Voters consistently reported that they liked the BMD. Most test voters felt that they could independently vote, without assistance, and that their votes were recorded accurately. However, a few voters did note that there was some confusion with the system instructions. One test voter pointed out that the selection button on the tactile remote was the only button that did not have Braille signage near it and suggested that would be nice to have. Another voter felt the “More” button should be brighter to draw your attention to it. Multiple test voters felt the tactile buttons were too sensitive. Los Angeles County provided the following responses in Table 6: Accessibility Survey Results: Table 6: Accessibility Survey Results Tester Issue Category Description Response 4 Contest Audio The tester felt that This will be taken the different into consideration. voices used in the The audio files audio could be are treated as confusing. system and election configuration files therefore can be improved and updated at any time without affecting certification versions. 4 Contest Audio The tester pointed The candidate’s out that the audios are played candidate names every time that 12 | P age Table 6: Accessibility Survey Results Tester Issue Category Description Response were not re-read the user navigates (audio) when through them, changing back moving between and forth between contests. contests. 4 Contest Audio The tester pointed This scenario out that the text of occurs when the the contest user is in the (Yes/No) had to voting experience be touched on the setting: Audio with screen for the Touchscreen, audio to start. where the keypad shouldn't be used. There are two sets of Audios in the BMD for each Voting Experience: Audio with Touchscreen and Audio with Keypad. These are automatically set when the user selects their language. If the user selects their language with the touchscreen, the Audio with Touchscreen experience will be activated. On the other hand, if the user selects the language with the keypad, the experience of Audio with Keypad will be activated. 4 Contest Audio Tester felt that the The layout was text of the designed to avoid proposition should pagination on be read first and Yes/No contests 13 | P age Table 6: Accessibility Survey Results Tester Issue Category Description Response then the contest with large text options (Yes/No) proposition. The after. audio will be played in the same order as the text appears in the page layout. 6 Contest Audio The tester felt the The candidate system was name is played confusing when the voter because the navigates on it tester had to using the arrow select each buttons. To select contest option to the candidate, the have it read to voter must click them. the round button. The system was designed to give the voter the control of the candidate navigation, therefore on each candidate they can decide either to continue to the next candidate (down arrow), go back to previous candidate (up arrow) or select the candidate (round button). 1 Instructions The tester noted When the user that when there goes back to a was more than contest already one candidate, navigated, the the instructions audio starts to did not repeat read the number when you of contests, the switched between name of the contests. For contests, the example, if they number of moved forward candidates that and then went are allowed to be selected, and the 14 | P age Table 6: Accessibility Survey Results Tester Issue Category Description Response back to make a instructions to change. start reading the candidates. These instructions are the same instructions that are reproduced the first time that the user navigates into the contest. 1 Instructions The tester When the Audio inquired about with Keypad adjusting the voting experience speed of the is activated, audio. before showing the first contest to the voter, the system shows instructions on how to use the keypad, explaining each button, including the playback rate buttons. The voter is not able to start voting without going through these instructions. 6 Instructions The tester felt the When the Audio voting contest with Keypad options were voting experience clear, but the is activated, System Help before showing instructions were the first contest to not. Once the the voter, the tester knew how system shows to adjust the rate instructions on settings, the how to use the tester thought it keypad, was easier to explaining each understand. button, including the playback rate buttons. The voter is not able to start voting without 15 | P age Table 6: Accessibility Survey Results Tester Issue Category Description Response going through these instructions. 1 Keypad The tester noted The specification Sensitivity that the tactile in CVSS address switch seemed that the force too sensitive and required to wanted to know if activate controls there was a way and keys shall be to adjust the no greater than 5 sensitivity. lbs (22.2N). The system is designed to comply with the specification. 2 Keypad The tester felt that The specification Sensitivity the down arrow in CVSS address key on the that the force assistive device required to was too sensitive. activate controls The tester felt it and keys shall be repeated too no greater than 5 much. lbs (22.2N). The system is designed to comply with the specification. 2 Keypad The tester was This button Sensitivity concerned that distinguishes itself there was no by its unique braille for the round shape. The selection button. buttons that have braille are rectangular with a relief. 5 More Button The tester felt the This will be taken “More” button on into consideration the screen should for future iteration. be brighter. 6 More Button The tester pointed The More button out that the select screen overlay button on the appears only on keypad could not the Audio with be used to exit Touchscreen out of the “More” experience, so it is expected that it 16 | P age Table 6: Accessibility Survey Results Tester Issue Category Description Response button screen cannot be overlay. handled with the keypad. 2 Time Frame The tester felt that * 2 out of 8 testers the process to (25%) felt that the complete the time-frame was ballot took longer longer than than expected. expected. * 1 out of 8 testers (12.5%) felt that the time-frame was faster than expected. * 5 out of 8 testers (62.5%) felt that the time-frame was as expected. The majority of the voters agreed that the time- frame was as expected. 6 Time Frame The tester felt the * 2 out of 8 testers process took at (25%) felt that the little longer than time-frame was expected. longer than expected. * 1 out of 8 testers (12.5%) felt that the time-frame was faster than expected. * 5 out of 8 testers (62.5%) felt that the time-frame was as expected. The majority of the voters agreed that the time- frame was as expected. 2 Time Frame The tester felt the There are several latency response latency responses was annoying. in the system. VSAP Tally 2.1 is designed to 17 | P age Table 6: Accessibility Survey Results Tester Issue Category Description Response comply with CVSS requirements. 6 Contest Audio The tester felt the The candidate system was name is played confusing when the voter because the navigates on it tester had to using the arrow select each buttons. To select contest option to the candidate, the have it read to voter has to click them. the round button. The system was designed to give the voter the control of the candidate navigation, therefore on each candidate they can decide either to continue to the next candidate (down arrow), go back to the previous candidate (up arrow) or select the candidate (round button). A detailed Accessibility, Usability and Privacy Test Report, including the survey results, can be found on our website. 7. Hardware Testing Summary NTS conducted Environmental and Dynamics Testing of the ballot marking devices. The devices were tested during the VSAP 2.0 testing efforts, however because of the replacement gears in some units, a sampling of units was tested 18 | P age against the applicable Hardware Section of the CVSS. The ballot marking devices passed each phase of the hardware testing. A detailed Hardware Testing report can be found on our website. IV. COMPLIANCE WITH STATE AND FEDERAL LAWS AND REGULATIONS 1. Elections Code Requirements Six (6) sections of the California Elections Code, Sections 19101, 19203, 19204, 19204.5, 19205, and 19270, describe in detail the requirements any voting system must meet in order to be approved for use in California elections. These sections are described in detail and analyzed for compliance below. a) §19101 (b) (1): The machine or device and its software shall be suitable for the purpose for which it is intended. − The system meets this requirement. All phases of the testing, specially, Functional demonstrated this to be true. b) §19101 (b) (2): The system shall preserve the secrecy of the ballot. − The system meets this requirement. Vote by Mail and BMD ballots can both be secretly cast. c) §19101 (b) (3): The system shall be safe from fraud or manipulation. − The system meets this requirement. d) §19101 (b) (4): The system shall be accessible to voters with disabilities pursuant to section 19242 and applicable federal laws. − The system meets this requirement. Accessibility conducted with voters during this and the previous tests efforts have demonstrated voters with accessibility needs have little or no trouble using the ballot marking devices. e) §19101 (b) (5): The system shall be accessible to voters who require assistance in a language other than English if the language is one in which a ballot or ballot materials are required to be made available to voters pursuant to Section 14201 and applicable federal laws. − VSAP 2.1 supports all 14201 languages. The system is capable of adding additional languages, to produce ballots or ballot materials, and accessible audio files pursuant to Section 14201, utilizing system functionality and outside translation. 19 | P age f) §19203: The system shall use ballot paper that is of sufficient quality that it maintains its integrity and readability throughout the retention period specified in sections 1700 through 17306. − The system meets this requirement. g) §19204: The system shall not include procedures that allow a voter to produce, and leave the polling place with, a copy or facsimile of the ballot cast by that voter at that polling place. − The system meets this requirement as the ballot marking devices retain the ballots, in the integrated ballot box, upon casting. h) §19204.5: The Secretary of State shall not certify or conditionally approve a voting system that cannot facilitate the conduct of a ballot level comparison risk-limiting audit. − The system meets this requirement. The systems components in addition to processes and procedures, prepare ballots during the tabulation process for a risk-limiting audit. i) §19205 (a): No part of the voting system shall be connected to the internet at any time. − The system meets this requirement. j) §19205 (b): No part of the voting system shall electronically receive or transmit election data through an exterior communication network, including the public telephone system, if the communication originates from or terminates at a polling place, satellite location, or counting center. − The system meets this requirement. k) §19205 (c): No part of the voting system shall receive or transmit wireless communications or wireless data transfers. − The system meets this requirement. l) §19270 (a): The Secretary of State shall not certify or conditionally approve a direct recording electronic voting system unless the system includes an accessible voter verified paper audit trail. − The system meets this requirement. 2. Elections Code Review 1) §305.5(b): A paper cast vote record is a ballot only if the paper cast vote record is generated on a voting device or machine that complies with ballot layout requirements and is tabulated by a separate device from the device that created the paper cast vote record. − The system meets this requirement. 2) §13109.7(a): Notwithstanding Section 13109, for a period of three years commencing with the date that the county elections official for the County of Los Angeles declares that the voting system modernization project 20 | P age underway in 2018 is complete and ready for operation, the county elections official for the County of Los Angeles shall conduct elections using the alternate ballot order described in Section 13109.8. (b) The county elections official shall prepare a report regarding the effect of using the alternate ballot order for elections conducted during the time period described in subdivision (a). The report shall include, but not be limited to, the following information: (1) Statistics and information on the cost of transitioning to the use of the alternate ballot order. (2) The overall turnout of voters in the jurisdiction for each election conducted using the alternate ballot order. (3) For different contests listed on the ballot, including, but not limited to, local offices and local ballot measures, state offices and state ballot measures, and federal offices, the following information: (A) The turnout of voters for each contest. (B) The number of overvotes and undervotes for each contest. (C) The dropoff rates for each contest. (4) Legislative recommendations. (c) The report described in subdivision (b) shall, whenever possible, compare an election conducted pursuant to this section and using the alternate ballot order described in Section 13109.8 to similar elections conducted using the ballot order described in Section 13109 in the same jurisdiction or in a comparable jurisdiction. (d) Three years after the declaration date described in subdivision (a), the county elections official shall submit the report described in subdivision (b) to the Secretary of State and to the Legislature in accordance with Section 9795 of the Government Code. The county elections official shall also post a publicly accessible copy of the report on the Internet Web site of the county elections official. (e) Notwithstanding any other law, the county elections official may adjust ballot instructions to the extent necessary to comply with this section. (f) Immediately after making the declaration described in subdivision (a), the county elections official shall post the declaration on his or her Internet Web site and send the declaration to the Secretary of State, the Secretary of the Senate, the Chief Clerk of the Assembly, and the Legislative Counsel. (g) This section shall remain in effect only until the first January 1 that occurs at least four years after the declaration date described in subdivision (a), and as of that date is repealed. - The system meets this requirement. 3) §15360: During the official canvass of every election in which a voting system is used, the official conducting the election shall conduct a public manual tally of the ballots tabulated by those devices cast in one percent of the precincts chosen at random by the elections official. If one percent 21 | P age of the precincts should be less than one whole precinct, the tally shall be conducted in one precinct chosen at random by the elections official. In addition to the one percent count, the elections official shall, for each race not included in the initial group of precincts, count one additional precinct. The manual tally shall apply only to the race not previously counted. − The system fully supports this requirement. 4) §19300: A voting machine shall, except at a direct primary election or any election at which a candidate for voter-nominated office is to appear on the ballot, permit the voter to vote for all the candidates of one party or in part for the candidates of one party and in part for the candidates of one or more other parties. − The system meets this requirement. 5) §19301: A voting machine shall provide in the general election for grouping under the name of the office to be voted on, all the candidates for the office with the designation of the parties, if any, by which they were respectively nominated. The designation may be by usual or reasonable abbreviation of party names. − The system meets this requirement. 6) §19302: The labels on voting machines and the way in which candidates’ names are grouped shall conform as nearly as possible to the form of ballot provided for in elections where voting machines are not used. − The system meets this requirement. 7) §19303: If the voting machine is so constructed that a voter can cast a vote in part for presidential electors of one party and in part for those of one or more other parties or those not nominated by any party, it may also be provided with: (a) one device for each party for voting for all the presidential electors of that party by one operation, (b) a ballot label therefore containing only the words “presidential electors” preceded by the name of the party and followed by the names of its candidates for the offices of President and Vice President, and (c) a registering device therefore which shall register the vote cast for the electors when thus voted collectively. If a voting machine is so constructed that a voter can cast a vote in part for delegates to a national party convention of one party and in part for those of one or more other parties or those not nominated by any party, it may be provided with one device for each party for voting by one operation for each group of candidates to national conventions that may be voted for as a group according to the law governing presidential primaries. No straight party voting device shall be used except for delegates to a national convention or for presidential electors. 22 | P age − The system meets this requirement. 8) §19304: A write-in ballot shall be cast in its appropriate place on the machine, or it shall be void and not counted. − The system supports this requirement. 9) §19320: Before preparing a voting machine for any general election, the elections official shall mail written notice to the chairperson of the county central committee of at least two of the principal political parties, stating the time and place where machines will be prepared. At the specified time, one representative of each of the political parties shall be afforded an opportunity to see that the machines are in proper condition for use in the election. The party representatives shall be sworn to perform faithfully their duties but shall not interfere with the officials or assume any of their duties. When a machine has been so examined by the representatives, it shall be sealed with a numbered metal seal. The representatives shall certify to the number of the machines, whether all of the counters are set at zero (000), and the number registered on the protective counter and on the seal. − The system supports this requirement. 10) §19321: The elections official shall affix ballot labels to the machines to correspond with the sample ballot for the election. He or she shall employ competent persons to assist him or her in affixing the labels and in putting the machines in order. Each machine shall be tested to ascertain whether it is operating properly. − The system supports this requirement. 11) §19322: When a voting machine has been properly prepared for an election, it shall be locked against voting and sealed. After that initial preparation, a member of the precinct board or some duly authorized person, other than the one preparing the machines, shall inspect each machine and submit a written report. The report shall note the following: (1) Whether all of the registering counters are set at zero (000), (2) whether the machine is arranged in all respects in good order for the election, (3) whether the machine is locked, (4) the number on the protective counter, (5) the number on the seal. The keys shall be delivered to the election board together with a copy of the written report, made on the proper blanks, stating that the machine is in every way properly prepared for the election. − The system supports this requirement. 12) §19340: Any member of a precinct board who has not previously attended a training class in the use of the voting machines and the duties of a board member shall be required to do so, unless appointed to fill an emergency vacancy. − The system does not adversely impact this requirement. 23 | P age 13) §19341: The precinct board shall consist of one inspector and two judges who shall be appointed and compensated pursuant to the general election laws. One additional inspector or judge shall be appointed for each additional voting machine used in the polling place. − The system does not adversely impact this requirement. 14) §19360: Before unsealing the envelope containing the keys and opening the doors concealing the counters the precinct board shall determine that the number on the seal on the machine and the number registered on the protective counter correspond to the numbers on the envelope. Each member of the precinct board shall then carefully examine the counters to see that each registers zero (000). If the machine is provided with embossing, printing, or photography devices that record the readings of the counters the board shall, instead of opening the counter compartment, cause a “before election proof sheet” to be produced and determined by it that all counters register zero (000). If any discrepancy is found in the numbers registered on the counters or the “before election proof sheet” the precinct board shall make, sign, and post a written statement attesting to this fact. In filling out the statement of return of votes cast, the precinct board shall subtract any number shown on the counter from the number shown on the counter at the close of the polls. − The system supports this requirement. 15) §19361: The keys to the voting machines shall be delivered to the precinct board no later than twelve hours before the opening of the polls. They shall be in an envelope upon which is written the designation and location of the election precinct, the number of the voting machine, the number on the seal, and the number registered on the protective counter. The precinct board member receiving the key shall sign a receipt. The envelope shall not be opened until at least two members of the precinct board are present to determine that the envelope has not been opened. At the close of the polls the keys shall be placed in the envelope supplied by the official and the number of the machine, the number written on the envelope. − The system supports this requirement. 16) §19362: The exterior of the voting machine and every part of the polling place shall be in plain view of the election precinct board and the poll watchers. Each machine shall be at least four feet from the poll clerk’s table. − The system supports this requirement. 3. Review of Federal Statutes or Regulations. 24 | P age a) The Voting Rights Act (VRA) of 1965, as amended (42 U.S.C. 1973), requires all elections in certain covered jurisdictions to provide registration and voting materials and oral assistance in the language of a qualified language minority group in addition to English. Currently in California, there are ten VRA languages (English, Spanish, Chinese, Hindi, Japanese, Khmer, Korean, Tagalog, Thai, and Vietnamese) as prescribed under the law. − The system meets this requirement. The system’s paper ballots can be easily printed in these languages, as well as any others. Further, BMD can be programmed to display the ballot in any of these languages on the touch screen interface and to provide audio instruction in any of these languages. b) The National Voter Registration Act of 1993 (42 U.S.C. 1973gg and 11 CFR 8) allows for the casting of provisional ballots through Fail-Safe Voting procedures. − The system meets this requirement. Provisional ballots can easily be cast with this system. The BMD only marks ballots (or verifies the marking of a ballot), it has no impact on provisional voting. c) The Voting Accessibility for the Elderly and Handicapped Act of 1984 (42 U.S.C. 1973ee through 1973ee-6) requires each political subdivision conducting elections within each state to assure that all polling places for federal elections are accessible to elderly and handicapped voters, except in the case of an emergency as determined by the state’s chief election officer or unless the state’s chief election officer: (1) determines, by surveying all potential polling places, that no such place in the area is accessible or can be made temporarily accessible, and (2) assures that any handicapped voter assigned to an inaccessible polling place will, upon advance request under established state procedures, either be assigned to an accessible polling place or be provided an alternative means of casting a ballot on election day. − This system supports this requirement. d) The Retention of Voting Documentation (42 U.S.C. 1974 through 1974e) statute applies in all jurisdictions and to all elections in which a federal candidate is on a ballot. It requires elections officials to preserve for 22 months all records and papers which came into their possession relating to an application, registration, payment of a poll tax, or other act requisite to voting. Note: The US Department of Justice considers this law to cover all voter registration records, all poll lists and similar documents reflecting the identity of voters casting ballots at the polls, all applications for absentee ballots, all envelopes in which absentee ballots are returned for 25 | P age tabulation, all documents containing oaths of voters, all documents relating to challenges to voters or absentee ballots, all tally sheets and canvass reports, all records reflecting the appointment of persons entitled to act as poll officials or poll watchers, and all computer programs used to tabulate votes electronically. In addition, it is the Department of Justice’s view that the phrase “other act requisite to voting” requires the retention of the ballots themselves, at least in those jurisdictions where a voter’s electoral preference is manifested by marking a piece of paper or by punching holes in a computer card. − The system meets this requirement. All votes in this system are recorded on paper ballots that can be easily retained. 4. Help America Vote Act (HAVA) Requirements The Help America Vote Act (HAVA) §301(a) mandates several requirements for voting systems, including: 1) The ability to verify the vote choices on the ballot before that ballot is cast and counted, 2) Notification to the voter of over-votes on a ballot, 3) Auditability with a permanent paper record of votes cast, 4) Accessibility for individuals with disabilities, including nonvisual accessibility for the blind and visually impaired, in a manner that provides the same opportunity for access and participation (including privacy and independence) − This system supports these requirements in the following manner: a) The paper ballots themselves lend themselves to visual inspection and verification. b) The BMD provides its users with a ballot review screen prior to printing the ballot. Further, any voted ballot can be inserted into the unit for review and verification. c) The BMD prevents over-voting a contest. d) Because all ballots in this system are paper based, there is a fully auditable and permanent record of the election. e) Deployment of the BMD in a precinct provides accessibility for persons with disabilities at the polling place. V. CONCLUSION The VSAP 2.1 voting system satisfactorily passed the Functional and Accessibility phases of testing. The Volume Test phase verified the fixes applied to the BMD have brought the devices well within an acceptable rate of paper handling as dictated by the CVSS. Finally, the county has addressed the findings 26 | P age required as a condition of certification of the VSAP 2.0 voting system, specifically those tied to the Source Code and Security and Telecommunications Review. 27 | P age