LHC
California Election Infrastructure: Making a Good System Better
Read the report at Little Hoover Commission ↗
California Election Infrastructure:
Making a Good System Better
Report #259 | April 2021
Milton Marks Commission on California State
Government Organization and Economy
www.lhc.ca.gov
LITTLE HOOVER COMMISSION Dedicated to Promoting Economy
and Efficiency in California State
Pedro Nava
Government
Chair
Sean Varner The Little Hoover Commission, formally known as the Milton
Marks “Little Hoover” Commission on California State Government
Vice Chair
Organization and Economy, is an independent state oversight agency.
David Beier
Dion Aroner By statute, the Commission is a bipartisan board composed of
five public members appointed by the governor, four public
Cynthia Buiza
members appointed by the Legislature, two senators and two
Bill Emmerson†
assemblymembers.
Assemblymember Chad Mayes
In creating the Commission in 1962, the Legislature declared its
Senator Jim Nielsen
purpose:
Assemblymember Bill Quirk
Senator Richard Roth ...to secure assistance for the Governor and itself in
promoting economy, efficiency and improved services in the
Cathy Schwamberger
transaction of the public business in the various departments,
Janna Sidley*
agencies and instrumentalities of the executive branch of
†Served as subcommittee chair the state government, and in making the operation of all
state departments, agencies and instrumentalities, and
*Served on study subcommittee
all expenditures of public funds, more directly responsive
COMMISSION STAFF to the wishes of the people as expressed by their elected
representatives...
Ethan Rarick
Executive Director
The Commission fulfills this charge by listening to the public,
Tamar Foster consulting with the experts and conferring with the wise. In the
course of its investigations, the Commission typically empanels
Deputy Executive Director
advisory committees, conducts public hearings and visits government
Krystal Beckham
operations in action.
Ashley Hurley
Its conclusions are submitted to the Governor and the Legislature
Rachel Mattioli
for their consideration. Recommendations often take the form of
Sherry McAlister
legislation, which the Commission supports through the legislative
Tristan Stein process.
Contacting the Commission
All correspondence should be addressed to the Commission Office:
Little Hoover Commission
925 L Street, Suite 805, Sacramento, CA 95814
(916) 445-2125 | LittleHoover@lhc.ca.gov
This report is available from the Commission’s website at www.lhc.ca.gov.
Table of Contents
EXECUTIVE SUMMARY ......................................................................3
INTRODUCTION .................................................................................4
THE MOST SECURE ELECTION IN HISTORY ....................................4
AN OPEN SOURCE ELECTIONS SYSTEM .........................................4
Incentivizing Security through Open Source Intellectual Property ..........4
Benefits of an Open Source System ..............................................................5
Recommendation ............................................................................................6
RISK-LIMITING AUDITS .....................................................................7
The 1 Percent Manual Tally ............................................................................7
An Alternative: Risk-Limiting Audits.............................................................7
What is a Risk-Limiting Audit? .............................................................7
How Risk-Limiting Audits Address the 1 Percent Manual Tally’s
Shortcomings .........................................................................................8
Voluntary Risk-Limiting Audits for Upcoming Elections ............................8
Challenges in Implementing Risk-Limiting Audits ....................................10
Supporting Best Practices ............................................................................10
Recommendation ..........................................................................................11
OTHER REFORMS .............................................................................11
TOP-TIER TRAINING .........................................................................11
Training Starts at the Top ............................................................................11
Training Must Not Stop at the Top ..............................................................11
Creating Incentives for People to Become Election Workers .........11
Recommendations ........................................................................................12
COMPLIANCE AUDITS ......................................................................13
Recommendations ........................................................................................14
APPENDIX A: PUBLIC HEARING WITNESSES ................................15
NOTES ................................................................................................16
CALIFORNIA ELECTION INFRASTRUCTURE: MAKING A GOOD SYSTEM BETTER | 1
Letter from the Chair
April 19, 2021
The Honorable Gavin Newsom
Governor of California
The Honorable Toni Atkins The Honorable Scott Wilk
President pro Tempore of the Senate Senate Minority Leader
and members of the Senate
The Honorable Anthony Rendon The Honorable Marie Waldron
Speaker of the Assembly Assembly Minority Leader
and members of the Assembly
DEAR GOVERNOR AND MEMBERS OF THE LEGISLATURE:
California’s elections are free, fair, and secure but the state must not be complacent. The Commission found
that California currently relies on a for-profit model of developing election infrastructure and its limitations leave
equipment designers and manufacturers without the financial incentive to create security upgrades for existing
models. Furthermore, the Commission found that some counties are left to rely on outdated post-election audits
when there is a better, more statistically rigorous auditing alternative available.
In order to address these challenges and further improve the security of California’s election infrastructure, the
Commission recommends that the state invest in and adopt an open source elections system, implement risk-
limiting audits, improve training of election officials, and adopt the use of compliance audits. California must
build on its good work to make our voting even more secure.
The Commission respectfully submits this work and stands prepared to help you address this challenge.
Sincerely,
Pedro Nava, Chair
Little Hoover Commission
2 | LITTLE HOOVER COMMISSION
Executive Summary
Letter from the Chair
California’s elections are free, fair, and secure. As There is an alternative: a “risk-limiting audit.” Such
technology and knowledge evolve, however, the state an audit reviews randomly selected ballots until the
can take steps to improve its election infrastructure. risk limit – a pre-determined chance that a wrong
California should adopt an open source elections outcome will not be discovered – is reached. These
system, require the use of “risk-limiting audits” audits have a high chance of correcting a previous
as soon as funding allows, improve the training count that is wrong.
of election officials, and delineate standards for
However, risk-limiting audits might increase costs for
compliance audits.
counties.
The Commission held a hearing on this topic in 2018
The Commission recommends that the state require
and in 2019 released a letter to the Governor and
risk-limiting audits as soon as needed funding is
legislative leadership to consider important questions
available.
related to election security, such as the need for
funding to improve equipment.
Training
This report builds on the Commission’s past work
Most chief election officials have many duties,
and adds specific policy recommendations. The
and thus have vastly different levels of expertise
Commission outlines potential improvements in four
in elections administration. Additionally, elections
broad areas of election security:
rely on thousands of relatively inexperienced poll
Open Source Elections System workers. Often it is difficult for county election
officials to find enough poll workers.
The state currently relies on for-profit producers of
The Commission recommends that the state create
election equipment. An open source system would
a training program for county elections officers and
be more transparent, save money, increase versatility
for lower-level elections workers. The state should
for counties, and align with a state goal to use open
also create incentives for people to become poll
source software across government.
workers. The Secretary of State should provide staff
Such a system must be accompanied by policies to to jurisdictions that lack enough trained poll workers.
ensure its proper use, and its adoption must not be
Compliance Audits
rushed.
The Commission recommends that the state invest in California’s elections code outlines security
a publicly owned, open source elections system. procedures, but lacks any system to ensure
compliance.
Risk-Limiting Audits
The Commission recommends that the state create a
The state’s current requirement for a manual tally framework for mandatory compliance audits.
of 1 percent of precincts as a way to check results is
outdated. It does not require all types of ballots to be
audited; it does not require scrutiny of close races;
and it does not ensure that an incorrect result will be
corrected.
CALIFORNIA ELECTION INFRASTRUCTURE: MAKING A GOOD SYSTEM BETTER | 3
Introduction
Consequently, equipment designers and
The Most Secure Election
manufacturers do not have large research and
in History
development budgets, nor the financial incentive to
create security upgrades for existing models. The
California’s elections are free, fair, and secure.
service and support side of the industry is where
The nonprofit news organization CalMatters
they make their money.5 As a result, voters vote on
surveyed county election officials and found that
machines that meet the security standards of the
the November 2020 election had “few glitches [and]
year they were certified.
little drama.”1 Californians turned out in rates not
seen since Eisenhower was elected.2 This was true Incentivizing Security through
despite the fact that the election occurred during a
Open Source Intellectual
worldwide pandemic, which at the time of publication
Property
had infected 3.5 million and killed more than 55,000
Californians.3 Every voter was provided a mail ballot,
Experts recommend investing in and utilizing open
and unspent funds originally intended to upgrade
source intellectual property, though the state would
election system infrastructure were rerouted to keep
need to create a governance framework.
voters safe while voting.
There are different models of open source systems,
Despite this striking success, California can improve
and the model election experts often recommend
its elections. As technology and statistical knowledge
may be familiar to many in the IT industry as a
change, new systems can make our voting even
shared source system. This means that the source
more secure. This report focuses principally on two
code is freely available for anyone to inspect, but only
changes: the use of an open source elections system
authorized individuals may change the code. The
and the implementation of risk-limiting audits.
“shared source” nomenclature is closely associated
with a type of licensing by a software company,
An Open Source Elections
however, so some, including the Commission, use
System
the broader “open source” terminology to prevent
confusion. Any open source system adopted by
The limitations of a for-profit model of developing California, then, would be available to security
election infrastructure in California are researchers, “white hat” hackers who try to exploit
straightforward. The customer base is tiny: There a system so they can report security concerns,
are only 58 potential customers who historically students, election officials, and anyone else
have only purchased new voting equipment every interested, while only the Secretary of State’s office or
couple of decades. The cost to sell in California is its designee would be able to modify the code.
high: California does not charge applicants a fee
Currently, open source applications typically involve
for certification, but equipment manufacturers are
the software that manages election systems or
expected to cover the costs of the process, which is
individual components within it.6 The Department
extensive. Manufacturers who update their systems
of Defense, however, currently is working on open
must re-complete the testing and certification
source election hardware, which would protect itself
process. The Secretary of State’s office’s overview of
against security threats such as users who try to
what that entails can be found on page 5.4
tamper with the equipment.7
4 | LITTLE HOOVER COMMISSION
Review and Testing for Certification Process Overview For
Voting Systems and Remote Accessible Vote By Mail Systems
1 2 3 4
Application Pre-Testing Testing Report Issuance & Post-
Phase Activities Phase Phase Test Activities Phase
◊ The Application, ◊ Project Test Schedule ◊ Functional Testing ◊ Final Reports
including technical
◊ Statement of Work ◊ Usability/Accessibility/ ◊ Public Hearing &
documentation, is
Privacy Testing Comment
◊ Open Escrow Account
submitted.
◊ Trusted Build ◊ Hardware Testing ◊ Certification
◊ The Application
(Equipment & ◊ Software Testing ◊ Awarded or Denied
is reviewed for
Software) to SOS
completeness. ◊ Telecommunications
◊ Request for Proposal ◊ Testing
Bid
◊ Security Testing
◊ Request for Proposal
◊ Quality Assurance and
Review
Configuration
◊ Contract Awarded &
◊ Management
Signed Non-Disclosure
Agreements between
SOS, Applicant, and
Test Laboratory Staff
◊ Test Plan
California Secretary of State. January 13, 2017. “Review and Testing for Certification Process Overview For Voting Systems and Remote Accessible Vote
By Mail Systems.” https://votingsystems.cdn.sos.ca.gov/cert-and-approval/review-testing-overview.pdf.
Benefits of an Open Source ◊ Transparency promotes security. To the
layperson, allowing anyone to look at source code
System
seems like a gift to hackers instead of a security
measure. However, the opposite approach, called
There are several benefits to the state investing in
security through obscurity, does not work. In that
and utilizing open source election systems:
case, the technology owners, their testers, and
◊ It’s more transparent. If we want elections to malicious actors are the only ones with eyes on
be transparent, said UC Davis computer security the source code, and the latter are not likely to
professor and researcher Matt Bishop, then alert anyone to security weaknesses. Since no
people should be able to see how their votes are technology is unhackable, no matter how talented
counted.8 and conscientious its designers, more review of
CALIFORNIA ELECTION INFRASTRUCTURE: MAKING A GOOD SYSTEM BETTER | 5
the source code means more opportunities to be more efficient than doing it themselves.12 There
correct security flaws before they impact elections. would be costs associated with updating and
maintaining a publicly-owned open source system,
Switzerland provides a case in point. Swiss
but we believe those costs would be less than the
Post, which oversees the country’s election
current cost of updating a proprietary system. In
infrastructure, worked with an outside contractor
an open source system, California would remove
to develop a new elections system, employed
some of those expenses from the counties while
professional auditors to review the code, then
promoting better security and ultimately saving
launched resilience testing with pre-approved
money.
testers from the public. Colloquially referred to as
◊ It is versatile. An open source system would
bug bounties, resilience testing provides financial
allow counties to structure the acquisition and
incentives to “white hat” hackers to find and
support of their election infrastructure in a way
report flaws in the code back to the organization,
that benefits them. Counties with significant IT
often with terms limiting how public these testers
resources, like Los Angeles and San Francisco,
can make their findings. A whistleblower leaked
would be able to continue creating and managing
the source code for Switzerland’s system to the
their own voting system. Counties with fewer
general public. Security researchers were appalled
resources would have the flexibility to work
by what they found.
with vendors on hardware, service, and support
packages. The goal is not to put equipment
Within the “poorly constructed and convoluted
vendors out of business; rather, it is to improve
maze that made it difficult to follow what
security by providing an alternative to the part
was going on in the system and effectively
of the industry where the invisible hand leads to
evaluate whether security measures deployed
insecurity.
in the system were done properly,” researchers
discovered a serious flaw: A malicious actor with ◊ It aligns with state goals. California has
access to Swiss Post’s IT system (i.e. an insider) committed to implementing open source software
could replace all the ballots in the system without across state government. The Department of
detection and change the reported election Technology asks state agencies to use open source
outcome.9 The takeaway is that at the end of a software when possible and to make their custom
strenuous testing process, such a critical flaw was code available as open source when practical to
missed by developers and auditors and was only decrease duplicative costs, reduce vendor lock-
detected through a public, albeit illegal, review of in, improve security, and facilitate information
the source code.10 sharing.13
◊ It is cheaper. California counties have twice
Recommendation
updated their election equipment since the 2000
general election at a cost of about $600 million.
1. The State of California should invest in a publicly-
Witnesses testified that designing, testing, and
owned, open source elections system designed to
training election officials on an open source
be accessible, secure, reliable, and auditable. Open
publicly-owned election technology platform
source should mean that security researchers and
would cost about $40 to $50 million.11 Counties
other interested parties should be able to test the
would then be responsible for the cost of the
code and report vulnerabilities to the state. This
hardware and set-up, and could contract with
system should be designed to provide maximum
vendors for maintenance and support if it would
flexibility for the different needs of different counties.
6 | LITTLE HOOVER COMMISSION
◊ As part of the development of open source recount, but they must pay for it.16
election infrastructure, California should create
◊ It does not guarantee election officials will
a policy framework governing how the system
catch and correct errors. If auditors find a
should be used to maximize the benefits of
discrepancy between how voters voted and how
adopting this system.
their votes were recorded or reported, there is no
◊ Election officials should resist pressure to use mechanism in place to fix the outcome.17 Changing
open source systems in elections until the election results in California requires a recount,
code has been released and inspected and the which someone must be willing to pay for. In
equipment is functioning as intended. effect, the requirement for a 1 percent manual
tally is useful only as a warning siren that there
may be a problem with the original count. But
Risk-Limiting Audits
if no one is willing to pay for a recount, nothing
happens.
The 1 Percent Manual Tally
The exclusion of so many ballots from the process
California counties currently conduct post-election
and lack of follow-up requirements led Dr. Philip
audits called the 1 percent manual tally. In these
Stark, UC Berkeley associate dean for mathematical
audits, election officials randomly select 1 percent of
and physical sciences and expert on election
participating precincts and review the ballots cast in
auditing, to testify that California’s manual tally,
those precincts to ensure the paper record matches
“wastes resources, and does not accomplish anything
reported results.”14 Though innovative when first
in particular.”18
implemented in 1965, there are some shortfalls to
the manual tally: An Alternative: Risk-Limiting
Audits
◊ It does not require all types of ballots to be
audited. Election officials are not required to
Statistical methodology has advanced since 1965
include provisional ballots in their tally, nor are
and there are other auditing options available with
they required to include vote-by-mail ballots that
fewer shortcomings than the manual tally. Risk-
are properly postmarked by but have not arrived
limiting audits in particular are a statistically rigorous
before Election Day. The number of excluded
alternative.
ballots is significant: More than 970,000 of the 12.7
million ballots cast in the November 2018 election WHAT IS A RISK-LIMITING AUDIT?
were provisional.15 Having such large categories of A risk-limiting audit is a review of randomly-selected
ballots that are not required to be audited leaves ballots until the risk limit – the chance that a wrong
big holes in California’s election security net. outcome will not be discovered and corrected in the
audit19 – is reached. For example, wrote Dr. Stark
◊ It does not require scrutiny of close races.
and his colleague, political scientist Mark Lindeman,
Election officials stop auditing when they’ve
a risk limit of 10 percent means that if the reported
completed 1 percent of participating precincts. If
outcome is wrong, there is at most a 10 percent
auditing shows that a very tight race is even tighter
chance that the audit will not correct the outcome
than previously thought or there were errors in
and at least a 90 percent chance that it will.20 In other
a close race, auditors still stop at 1 percent. If a
words, risk-limiting audits provide another layer of
candidate or member of the public has concerns
protection against a wrong outcome. They do not
about accuracy in a tight race, they can request a
CALIFORNIA ELECTION INFRASTRUCTURE: MAKING A GOOD SYSTEM BETTER | 7
guarantee the reported outcome is right, stated Dr. certification of election results. If it looks like there
Stark and Dr. Lindeman, but with a low risk limit they might be a problem, anyone can ask and pay for a
have a large chance of correcting the outcome if recount. In a risk-limiting audit, the recount is built
wrong.21 into the auditing system before the final certification
of election results.
The chart on page 9 outlines the process of
conducting a risk-limiting audit. First, the election Voluntary Risk-Limiting Audits
jurisdiction must settle on the audit’s risk limit. A risk
for Upcoming Elections
limit of 5 percent often is chosen, meaning there
is no more than a 5 percent chance that wrongly- In 2018, the Legislature passed and the Governor
reported results would not be corrected. Then the signed AB 2125 (Quirk), which allowed counties to
ballots to be audited must randomly be selected. voluntarily conduct a risk-limiting audit in lieu of the
All counties in California must create a manifest manual tally for elections in 2020.22 The legislation
detailing the physical location of every ballot cast in also required the Secretary of State to design a
the election, so one way of choosing ballots would regulatory framework for risk-limiting audits that
be to randomly select (using methods proven to included all ballots, including provisional and vote-
produce random results) positions on the manifest by-mail ballots. In August 2020, the option to conduct
– say 27, 441, and 10,297, then pull the ballots risk-limiting audits was extended through the end of
corresponding to those positions. If Candidate A 2022 through AB 2400 (Quirk), which also provided
was reported to have won by a landslide and the counties added flexibility in the contests they audit.23
randomly-selected ballots reflected that voters
indeed overwhelmingly voted for that candidate, This legislation was a strong step forward, but
then not too many ballots would have to be audited California must now go further and require counties
to be sure there is only a 5 percent chance that to conduct risk-limiting audits. There is abundant
a wrong outcome went uncorrected. However, if evidence that risk-limiting audits work. Risk-limiting
Candidate A was reported to have won by a landslide audit pilot programs have been conducted in
and randomly-selected ballots showed many people California since 2010, when the federal Election
voted for Candidate B, auditors would have to keep Assistance Commission awarded grants to the
examining ballots until they reached the risk limit. state for that purpose.24 Pilot audits have been
completed in a diverse array of counties, including
HOW RISK-LIMITING AUDITS ADDRESS THE 1
Inyo, Alameda, Humboldt, and San Luis Obispo. The
PERCENT MANUAL TALLY’S SHORTCOMINGS
Commission is confident that additional voluntary
Risk-limiting audits solve some of the shortcomings
risk-limiting audits conducted under AB 2125 and
of the current auditing system. If a race is very close
AB 2400 will continue to show the advantages of this
or the audit catches errors, election officials will have
system.
to examine more ballots to be sure of an accurate
outcome. If necessary, this could result in a manual No one doubts that risk-limiting audits are sounder
examination of 100 percent of ballots. This solves statistically than the 1 percent manual tally. It is time
the problem of remedying an incorrect outcome to ensure that all California counties – and thus all
in the initial count, provided Legislators give a full California voters – employ the best election auditing
hand count the same legal significance as a recount. system that is available.
Under the manual tally, officials stop auditing after 1
percent of precincts and report the outcome in their
8 | LITTLE HOOVER COMMISSION
Risk-Limiting Audits
Conduct elections with voter-verified and
machine-scanned paper ballots.
Identify contest(s) to be audited
according to state law and rule.
Store, organize, and
catalog paper ballots
for later retrieval.
Select a scientific random sample
of ballots for hand auditing.
Pull sampled paper ballots, examine,
and record information.
Does the information from the sampled paper ballots give
enough evidence to support the reported outcome(s)?
YES. Voters can have confidence NO. More ballots are needed to provide
in reported outcome! evidence, up to and including a full hand
count of all validly cast ballots.
Source: Verified Voting. “A Verified Voting Flowchart for Conducting Risk-Limiting Audits.” July 2018. https://verifiedvoting.org/publication/a-verified-
voting-flowchart-for-conducting-risk-limiting-audits/.
CALIFORNIA ELECTION INFRASTRUCTURE: MAKING A GOOD SYSTEM BETTER | 9
be audited under a risking-limiting audit system than
Challenges in Implementing
the manual tally.
Risk-Limiting Audits
Pilot projects have borne this out. For a 2018 special
Election officials’ concerns about risk-limiting audits
election, Inyo County election officials conducted
revolve around resources and funding. A large hand
both a risk-limiting audit and a manual tally. With a
count would present a significant logistical challenge
10 percent risk limit, the risk-limiting audit required
in heavily-populated counties like Los Angeles: There
reviewing 19 ballots while the manual tally required
could be millions of ballots cast and in a risk-limiting
auditing 74 ballots.26 In a November 2018 Orange
audit, election officials must find specific ballots,
County pilot project for three local contests, officials
whereas ballots can be audited by batch in the 1
anticipated a sample size of 16,000 ballots for
percent manual tally.25 Some officials support the
the combined three contests would be needed to
idea of machine-assisted audits: Kenneth Bennett,
meet their confidence level, versus the more than
who oversaw the Voting Solutions for All People
57,000 ballots that would need to be reviewed for
program in Los Angeles County, testified that, “a
the manual tally. For one of the three contests, the
100% manual recount is not only a time-consuming
confidence level was met after auditing only 540
logistical challenge requiring hundreds of staff
ballots. Auditors unfortunately had to stop reviewing
resources, it is questionable that such a hand count
ballots beyond 1,000 each for the other two contests,
of 2 to 3 million ballots would be demonstrably more
because the staff were needed for the manual tally.
accurate than the machine-tallied results.”
More than 60 employees spent three weeks auditing
the 57,178 ballots required under the 1 percent
Proving with better accuracy manual tally. Dr. Stark, who worked with Orange
County on this project, testified that unless there
that the reported results of an
were errors or a too-close-to-call race, the number of
election are correct is worth
ballots required by a risk-limiting audit even in highly-
the cost for the legitimacy it populated counties would be relatively few.27
adds to elections, particularly Supporting Best Practices
in an era of misinformation.
It is inevitable that there will be errors or very close
races, and in those instances having to audit large
Incorporating another piece of technology in the numbers of ballots is a feature, not a flaw. California
auditing process would add another security risk, must provide the support necessary to prove the
however, and negate the purpose for the risk-limiting accuracy of results or remedying mistakes. Past
audit: to see if machines tallied and reported votes history has taught county election officials that the
correctly. The potential logistical enormity, however, costs of implementing a new system will be their
is not lost on the Commission. problem. The state must rectify that. Often, a risk-
limiting audit will require fewer resources than
Others testified that it was unlikely a 100 percent the manual tally, though it would be a new cost to
hand count would be required unless there was the state due to funded mandate requirements.
an exceptionally close race or serious errors in Proving with better accuracy that the reported
the voting process. On the contrary, if things run results of an election are correct is worth the cost
smoothly, it is likely that fewer ballots would need to for the legitimacy it adds to elections, particularly
10 | LITTLE HOOVER COMMISSION
in an era of misinformation. The Commission is that California undertake a needs assessment
cognizant of current fiscal conditions, but California to understand the gaps between state security
should require risk-limiting audits statewide and standards and how the standards are implemented
be prepared to fund that mandate as soon as the by local election officials. Then a training course or
budget allows. The system should build in additional curriculum could be developed to fill those gaps.28
support and flexibility in case an extremely close The state’s involvement is critical, as county budgets
election requires a manual review of a significant can only go so far. Commission witness Kammi Foote,
number of ballots. for example, heads Inyo County’s election office, and
also serves as the county’s recorder and registrar.
Recommendation
Her combined annual budget for all three divisions at
the time of the hearing was approximately $500,000,
2. Risk-limiting audits should be conducted to verify
with about $60,000 allocated for elections.29 There
all election results. This could be done by the state
was not much left over for extensive security training.
directly or through a mandate to counties. The
change should be implemented as soon as the Training Must Not Stop at the
state can provide the needed funding either to the
Top
Secretary of State’s office or to mandated counties.
Aside from county election officials and their staff,
◊ Should a risk-limiting audit result in a 100 percent
elections rely on tens of thousands of relatively
manual inspection of the ballots, the Legislature
inexperienced poll workers.
should award that hand count the same legal
significance as an official recount.
California election workers must meet a basic set of
standards, but it is up to counties to package training
Other Reforms
options that help their election workers reach those
standards. In many counties, this is a one-time class
The state can also improve training and adopt the
plus optional hands-on time with the county’s voting
use of compliance audits.
equipment. “Being a poll worker is complicated,”
Top-Tier Training Stanford emeritus professor and founder of Verified
Voting David Dill told staff. He observed that many
From the county election chief to the volunteer poll poll workers are not taught the reasons behind the
worker, an accurate election outcome depends on procedures they must follow and consequently do
everyone being fluent in their job responsibilities, not understand how a procedure protects against
prepared for security threats, and knowledgeable threats. It’s easier to take shortcuts, intentionally
about election policies and procedures. or accidentally, when election workers do not
understand the purpose of what they’re doing. And
Training Starts at the Top California has no systemic way of monitoring what is
happening on the ground.30
Most election administrators in California have many
duties. Counties often lump election administration CREATING INCENTIVES FOR PEOPLE TO
BECOME ELECTION WORKERS
in with the registrar’s or a similar position’s duties.
The obvious solution seemingly is more training for
Commission witnesses stressed the need to election workers, but the answer is not that simple.
provide election officers with comprehensive Putting aside cost, officials already have a hard time
security training. Dr. Bishop recommended to staff
CALIFORNIA ELECTION INFRASTRUCTURE: MAKING A GOOD SYSTEM BETTER | 11
finding enough election workers. Poll workers are
Recommendations
paid little, must work on a weekday, and in many
cases must take time off from jobs or school. Two- 3. The State of California should conduct a needs
thirds of local election officers nationwide report assessment to understand where to prioritize
finding it difficult to staff their polls.31 security training.
Adding further requirements without creating 4. The State of California should create a training
corresponding incentives would only make it harder curriculum individually tailored to the needs of each
to recruit election workers. The state must create county election officer and key staff members to
incentives to encourage people to sign up to be an ensure they are operating at a high security standard.
election worker with additional enhanced training.
California should consider: ◊ Ideally, this curriculum should be informed by the
needs assessment the Commission recommended
◊ Granting counties funds to supplement base pay above.
in areas that have historically struggled to find
5. The State of California should create a
election workers.
comprehensive election worker training program
◊ Working with high schools and colleges to create
to ensure its election workforce meets election
election administration courses for credit in
security best practices. Concurrently, the state should
election years and requiring the students to serve
create incentives for Californians to become election
as poll workers as part of the curriculum, which
workers despite the increased requirements. These
should include enhanced training. Poll workers
include but are not limited to:
should continue to be paid to avoid limiting
the opportunity to students who can afford an
◊ Using its resources to train election workers on
unpaid internship. Election officials should be
topics not specific to any county, leaving only
able to waive the minimum GPA requirement at
county-specific training for the counties. Hands-on
their discretion if a consultation with the student
training with the equipment voters will use should
indicates that the student’s participation would be
be mandatory for all election workers.
mutually beneficial.
◊ Supplementing counties’ base pay with bonuses
◊ Allowing the state’s 233,000 employees to for people who train to be election workers in
complete enhanced training without having to use
historically difficult-to-fill locations.
their vacation time, just as California already does
◊ Working with high schools and post-secondary
for employees who serve as election workers.
institutions to create applied election
Additionally, department heads should actively
administration courses so students could receive
encourage interested employees to become
course credit for training as an election worker.
election workers.
They should still receive their stipends so that
students from all economic backgrounds can
Finally, if a county does not have enough trained
afford to participate.
election workers to administer a secure and
accessible election, then the Secretary of State’s ◊ Allowing and encouraging the state workforce
office must supply the rest. The county should to commit to becoming election workers with
be responsible for all expenses incurred by the enhanced training requirements without having to
Secretary of State’s office in this situation. use their annual leave or vacation time.
12 | LITTLE HOOVER COMMISSION
6. The Secretary of State’s office should provide ◊ Physical Chain of Custody. Are the custody logs
well-trained staff to any voting jurisdiction that is detailing the minutia of interactions with voting
unable to supply enough trained election workers to equipment and ballots complete? Do, say, cross
run an accessible election that meets best practices references between an action recorded in the
for security. The jurisdiction should compensate the log and video surveillance show that record was
Secretary of State for the workers’ pay, including accurate?
overtime and travel expenses if they need to use
◊ Due Diligence. What do the voting machine
this option outside of a Governor-declared state of
event logs show? How were complaints by voters
emergency.
addressed? What problems did poll workers
report and how did election officers respond?
Compliance Audits
Many election officers already perform many of the
Even with the best training and the latest equipment,
components that would be included in a compliance
human beings will make mistakes. California’s
audit as part of their regular canvass procedures.
Election Code outlines security procedures, but the
Witnesses recommended the state create a high
state lacks any system to ensure compliance. Many
standard, formalize the auditing process, require
witnesses recommended that the state institute a
standardized reporting, and require the auditing to
compliance audit to provide an additional layer of
be conducted in a publicly verifiable manner. Multiple
security.
witnesses pointed out that few places have robust
compliance audits; California could set the gold
The core feature of evidence-based elections is a
standard for others to follow.
voter-verified paper trail, which in California is a
ballot. A compliance audit is the process of assessing
The Secretary of State administers an election
the paper trail’s trustworthiness from the moment
observation program that sends trained staff into the
that ballot is created to when it is properly destroyed
field to monitor and inspect in-person voting location
after the election.32 Additionally, election officials
operations. These officials report any problems they
must show that every validly cast vote was cast
discover, and investigate reports made by voters and
exactly once and their records are complete and
others outside of the Secretary of State’s office. With
intact from election to audit.33
necessary funding from the Legislature, the Secretary
of State should expand this program by randomly
The Commission asked Dr. Stark what a compliance
auditing additional voting jurisdictions during and
audit should look like. His high-level sketch is
after the election and during the off-season to make
included in his written testimony to the Commission,
sure that ballots and voting equipment are properly
but in short, the following factors must be included in
secured. These audits should be approached in the
the audit in order for it to be effective:
spirit of cooperation and a shared understanding
◊ Ballot Accounting. Do the numbers add up? For that all parties involved are trying to ensure a secure,
example, if a polling place issued 200 ballots of a accessible election. The audit’s action items should
given kind, were they all fed through the optical not just detail steps the county will take to better
scanner? secure elections, but also how the Secretary of State’s
office will support them. The Secretary of State
◊ Eligibility. Were all of the eligible ballots and no
should regularly report the results of county and
others included in the final count? What about the
state audits to the Legislature.
provisional ballots?
CALIFORNIA ELECTION INFRASTRUCTURE: MAKING A GOOD SYSTEM BETTER | 13
Finally, the Secretary of State should work with
the Administration and Legislature to create a
series of actions that will be implemented should a
compliance audit indicate that election process may
have been compromised.
As with risk-limiting audits, enhanced training
requirements and compliance audits will be
mandates that must be funded by the state.
Though the Commission typically does not make
recommendations that would enact new state
mandates, ensuring the integrity of the democratic
process in California for generations to come is
worthwhile stewardship of taxpayer dollars.
Recommendations
7. Led by the Secretary of State, the State of
California should delineate standards and create the
framework for mandatory compliance audits.
◊ The state should aim to set the gold standard for
compliance audits.
◊ As part of this process, the Secretary of State
should work with the Administration and
Legislature to codify next steps if the audit
uncovers serious errors.
◊ Compliance audits must be publicly verifiable.
◊ Audit results should be available in writing, shared
with the Secretary of State’s office, and be easily
accessible to the public.
8. The Secretary of State should expand its election
observation program by randomly auditing additional
voting jurisdictions during, after, and between
elections to ensure they are following laws and
regulations. These audits should be conducted in the
spirit of collaboration and if errors are uncovered,
the Secretary of State should commit to working with
the jurisdiction on improving its compliance with
security measures. The Secretary of State should
regularly report its findings and next steps to the
Legislature. The Legislature should provide sufficient
funding to support these activities.
14 | LITTLE HOOVER COMMISSION
Appendix A: Public Hearing Witnesses
The lists below reflect the titles and positions of witnesses at the time of the hearings.
Public Hearing on Voting Equipment Security
July 26, 2018
Sacramento, California
Kenneth Bennett, Program Manager, Voting Solutions E. John Sebes, Chief Technology Officer, Open Source
for All People (VSAP), Los Angeles County Registrar- Election Technology (OSET) Institute
Recorder/County Clerk
Pamela Smith, Senior Advisor, Verified Voting
Kammi Foote, Clerk-Recorder, County of Inyo
Philip Stark, Associate Dean, Mathematical and
Susan Lapsley, Deputy Secretary of State, HAVA Physical Sciences and Professor, Department of
Director and Counsel, California Secretary of State Statistics, University of California, Berkeley
Noel Runyan, Owner, Personal Data Systems
CALIFORNIA ELECTION INFRASTRUCTURE: MAKING A GOOD SYSTEM BETTER | 15
Notes
1. Lewis Griswold. December 3, 2020. “How a-dollar10-million-open-source-secure-voting-
California reached historic voter turnout system. Accessed August 28, 2020.
despite pandemic, distrust.” CalMatters. https://
8. Matt Bishop, Professor, Computer Science,
calmatters.org/politics/votebeat/2020/11/
University of California, Davis. April 26, 2018.
california-historic-voter-turnout-despite-
Phone call with Commission staff.
pandemic-distrust/. Accessed March 11, 2021.
9. The system in question was an electronic voting
2. California Secretary of State. “Historical Voter
system, which is not a best practice in election
Registration and Participation in Statewide
infrastructure security.
General Elections 1910-2020.” https://www.sos.
ca.gov/elections/statistics. Accessed March 11,
10. Kim Zetter. March 12, 2019. “Researchers Find
2021.
Critical Backdoor in Swiss Online Voting System.”
Motherboard: Tech by Vice. https://www.vice.
3. California Department of Public Health. January
com/en_us/article/zmakk3/researchers-find-
13, 2020. Covid by the Numbers: As of March
critical-backdoor-in-swiss-online-voting-system.
10, 2021. https://www.cdph.ca.gov/Programs/
Accessed August 28, 2020.
CID/DCDC/Pages/Immunization/ncov2019.aspx.
Accessed March 11, 2021.
11. E. John Sebes, Chief Technology Officer, Open
Source Election Technology (OSET) Institute. July
4. California Secretary of State. January 13, 2017.
26, 2018. Sacramento, California. Testimony to
“Review and Testing for Certification Process
the Commission. Also, Philip Stark, Associate
Overview For Voting Systems and Remote
Dean, Mathematical and Physical Sciences and
Accessible Vote By Mail Systems.” https://
Professor, Department of Statistics, University
votingsystems.cdn.sos.ca.gov/cert-and-approval/
of California, Berkeley. July 2, 2018. Sacramento,
review-testing-overview.pdf. Accessed August 27,
California. Testimony to the Commission.
2020.
12. Kammi Foote, Clerk-Recorder, County of Inyo.
5. E. John Sebes, Chief Technology Officer, Open
July 26, 2018. Sacramento, CA. Testimony to the
Source Election Technology (OSET) Institute. July
Commission.
26, 2018. Sacramento, California. Testimony to
the Commission.
13. California Department of Technology. May 2018.
Technology Letter 18-02. https://codecagov-
6. U.S. Election Assistance Commission. “HAVA
playbook.readthedocs.io/en/latest/policy/.
Funds: State Chart View.” https://www.eac.gov/
Accessed August 28, 2020.
payments-and-grants/hava-funds-state-chart-
view/. Accessed June 15, 2018. Also, SB 840.
14. Joseph Lorenzo Hall. February 11, 2009.
Mitchell. Budget Act of 2018-19. http://leginfo.
“Improving the Security, Transparency and
legislature.ca.gov/faces/billTextClient.xhtml?bill_
Efficiency of California’s 1% Manual Tally
id=201720180SB840.
Procedures.” USENIX/ACCURATE Electronic Voting
Technology Workshop 2009. https://www.usenix.
7. Kim Zetter. March 14, 2019. “DARPA Is Building a
org/legacy/event/evt08/tech/full_papers/hall/
$10 Million, Open Source, Secure Voting System.
hall_html/. Accessed September 14, 2018.
Motherboard: Tech by Vice. https://www.vice.
com/en_us/article/yw84q7/darpa-is-building-
15. California Secretary of State. March 27, 2019.
16 | LITTLE HOOVER COMMISSION
“2018 November General Election – Provisional Voting. https://www.stat.berkeley.edu/~stark/
Ballot Statistics as of 3/27/19.” https://elections. Preprints/gentle12.pdf. Accessed September 19,
cdn.sos.ca.gov/statewide-elections/2018-general/ 2018.
provisional-statistics.pdf. Accessed August
21. Mark Lindeman and Philip B. Stark. 2012. “A
17, 2020. Also, California Secretary of State.
Gentle Introduction to Risk-Limiting Audits.” IEEE
“Historical Voter Registration and Participation in
Security and Privacy, Special Issue on Electronic
Statewide General Elections 1910-2018.” https://
Voting. https://www.stat.berkeley.edu/~stark/
elections.cdn.sos.ca.gov/sov/2018-general/
Preprints/gentle12.pdf. Accessed September 19,
sov/04-historical-voter-reg-participation.pdf.
2018.
Accessed August 17, 2020.
22. AB 2125 (Quirk, 2018). Chapter 913, Statutes of
16. California Secretary of State. Statewide Recounts
2018.
– Frequently Asked Questions. https://www.sos.
ca.gov/elections/statewide-recounts/statewide-
23. AB 2400 (Quirk, 2020). Chapter 33, Statutes of
recounts-faq/. Accessed December 20, 2018.
2020.
17. Philip Stark, Associate Dean, Mathematical and
24. California Secretary of State. March 1, 2011. “AB
Physical Sciences and Professor, Department of
2023 (Saldaña), Chapter 122, Statutes of 2010
Statistics, University of California, Berkeley. July
Post-Election Risk-Limiting Audit Pilot Program
2, 2018. Written testimony to the Commission.
March 1, 2012, Report to the Legislature.” https://
https://lhc.ca.gov/sites/lhc.ca.gov/files/
admin.cdn.sos.ca.gov/reports/2012/post-election-
Reports/247/WrittenTestimony/StarkJuly2018.
audit-report-20120301.pdf. Accessed August 18,
pdf. Page 3.
2020.
18. Philip Stark, Associate Dean, Mathematical and
25. Kenneth Bennett, Program Manager, Voting
Physical Sciences and Professor, Department of
Solutions for All People (VSAP), Los Angeles
Statistics, University of California, Berkeley. July
County Registrar-Recorder/County Clerk. July 26,
2, 2018. Written testimony to the Commission.
2018. Written testimony to the Commission.
https://lhc.ca.gov/sites/lhc.ca.gov/files/
Reports/247/WrittenTestimony/StarkJuly2018.
26. Kammi Foote, Clerk-Recorder & Registrar of
pdf. Page 3.
Voters, County of Inyo. April 20, 2018. “Report
on the Inyo County Risk Limiting Audit Pilot.”
19. Dwight Shellman, County Support Manager,
Can be viewed as an attachment to Ms. Foote’s
Elections Division, Colorado Department of State.
testimony at lhc.ca.gov.
June 15, 2015. “Risk-Limiting Audits in Colorado.”
Presentation to the National Conference of State
27. Philip Stark, Associate Dean, Mathematical and
Legislatures conference: The Future of Elections.
Physical Sciences and Professor, Department
http://www.ncsl.org/Portals/1/Documents/
of Statistics, University of California, Berkeley.
Elections/RLA_20170615.pdf. Accessed
July 26, 2018. Sacramento, CA. Testimony to the
September 19, 2018.
Commission.
20. Mark Lindeman and Philip B. Stark. 2012. “A
28. Matt Bishop, Professor, Computer Science,
Gentle Introduction to Risk-Limiting Audits.” IEEE
University of California, Davis. April 26, 2018.
Security and Privacy, Special Issue on Electronic
Phone call with Commission staff.
CALIFORNIA ELECTION INFRASTRUCTURE: MAKING A GOOD SYSTEM BETTER | 17
29. Kammi Foote, Clerk-Recorder, County of Inyo. July
26, 2018. Written testimony to the Commission.
30. David Dill, Emeritus Professor, Stanford
University and Founder, Verified Voting. April 12,
2018. Phone call with Commission staff.
31. Matt Vasilogambros. October 22, 2018.
“Few People Want to be Poll Workers, and
That’s a Problem.” Stateline. https://www.
pewtrusts.org/en/research-and-analysis/blogs/
stateline/2018/10/22/few-people-want-to-be-
poll-workers-and-thats-a-problem. Accessed
September 4, 2020.
32. Philip Stark, Associate Dean, Mathematical and
Physical Sciences and Professor, Department of
Statistics, University of California, Berkeley. July
26, 2018. Written testimony to the Commission.
Also, California Elections Code. Division 17.
Chapter 4. Sections 17301 and 17302.
33. Philip Stark, Associate Dean, Mathematical and
Physical Sciences and Professor, Department of
Statistics, University of California, Berkeley. July
26, 2018. Written testimony to the Commission.
18 | LITTLE HOOVER COMMISSION
Little Hoover Commission Members
CHAIRMAN PEDRO NAVA | Santa Barbara ASM. CHAD MAYES | Yucca Valley
Appointed to the Commission by Speaker of the Assembly Appointed to the Commission by Speaker of the Assembly
John Pérez in April 2013 and reappointed by Speaker of the Toni Atkins in September 2015. Elected in November
Assembly Anthony Rendon in 2017. Government relations 2014 to represent the 42nd Assembly District. Represents
advisor. Former State Assemblymember from 2004 to Beaumont, Hemet, La Quinta, Palm Desert, Palm Springs,
2010, civil litigator, deputy district attorney and member San Jacinto, Twentynine Palms, Yucaipa, Yucca Valley, and
of the state Coastal Commission. Elected chair of the surrounding areas.
Commission in March 2014.
SEN. JIM NIELSEN | Gerber
VICE CHAIRMAN SEAN VARNER | Riverside Appointed to the Commission by the Senate Rules
Appointed to the Commission by Governor Edmund G. Committee in March 2019. Elected in January 2013 to
Brown Jr. in April 2016 and reappointed in January 2018. represent the 4th Senate District. Represents Chico,
Managing partner at Varner & Brandt LLP where he Oroville, Paradise, Red Bluff, Yuba City, and surrounding
practices as a transactional attorney focusing on mergers areas.
and acquisitions, finance, real estate, and general counsel
work. Elected vice chair of the Commission in March 2017. ASM. BILL QUIRK | Hayward
Appointed to the Commission by Speaker of the Assembly
DION ARONER | Berkeley Anthony Rendon in 2017. Elected in November 2012 to
Appointed to the Commission by the Senate Rules represent the 20th Assembly District. Represents Hayward,
Committee in April 2019. Partner for Aroner, Jewel, and Union City, Castro Valley, San Lorenzo, Ashland, Cherryland,
Ellis. Former State Assemblymember from 1996 to 2002, Fairview, Sunol, and North Fremont.
chief of staff for Assemblymember Tom Bates, social
worker for Alameda County, and the first female president SEN. RICHARD ROTH | Riverside
of Service Employees International Union 535. Appointed to the Commission by the Senate Rules
Committee in February 2013. Elected in November 2012
DAVID BEIER | San Francisco to represent the 31st Senate District. Represents Corona,
Appointed to the Commission by Governor Edmund G. Coronita, Eastvale, El Cerrito, Highgrove, Home Gardens,
Brown Jr. in June 2014 and reappointed in January 2018. Jurupa Valley, March Air Reserve Base, Mead Valley,
Managing director of Bay City Capital. Former senior officer Moreno Valley, Norco, Perris, and Riverside.
of Genentech and Amgen, and counsel to the U.S. House of
Representatives Committee on the Judiciary. CATHY SCHWAMBERGER | Calistoga
Appointed to the Commission by the Senate Rules
CYNTHIA BUIZA | Los Angeles Committee in April 2018 and reappointed in January
Appointed to the Commission by Speaker of the Assembly 2019. Associate general counsel for State Farm Mutual
Anthony Rendon in October 2018. Executive director of the Automobile Insurance Company. Former board member
California Immigrant Policy Center. Former policy director of the Civil Justice Association of California and the Capital
for the American Civil Liberties Union, San Diego, and Political Action Committee.
policy and advocacy director at the Coalition for Humane
Immigrant Rights of Los Angeles. JANNA SIDLEY | Los Angeles
Appointed to the Commission by Governor Edmund
BILL EMMERSON | Redlands G. Brown Jr. in April 2016 and reappointed in February
Appointed to the Commission by Governor Edmund G. 2020. General counsel at the Port of Los Angeles since
Brown Jr. in December 2018. Former senior vice president 2013. Former deputy city attorney at the Los Angeles City
of state relations and advocacy at the California Hospital Attorney’s Office from 2003 to 2013.
Association, State Senator from 2010 to 2013, State
Assemblymember from 2004 to 2010, and orthodontist. Full biographies are available on the Commission’s
website at www.lhc.ca.gov.
CALIFORNIA ELECTION INFRASTRUCTURE: MAKING A GOOD SYSTEM BETTER | 19
“DEMOCRACY ITSELF IS A PROCESS OF CHANGE, AND
SATISFACTION AND COMPLACENCY ARE ENEMIES OF
GOOD GOVERNMENT.”
By Governor Edmund G. “Pat” Brown,
addressing the inaugural meeting of the Little Hoover Commission,
April 24,1962, Sacramento, California
Milton Marks Commission on California State
Government Organization and Economy
www.lhc.ca.gov