LAO
The 2017-18 Budget: California State Payroll System
Read the report at Legislative Analyst's Office ↗
The 2017-18 Budget:
California State Payroll System
MAC TAYLOR • LEGISLATIVE ANALYST • FEBRUARY 28, 2017
Summary
The State Controller’s Office (SCO) is responsible for issuing pay to the state workforce,
including employees of the state, California State University (CSU), and judicial council. The
SCO has recently renewed its effort to replace the state’s payroll system. In this year’s budget, the
Governor proposes $3 million to begin an analysis of proposed alternatives for replacing the system.
At this time, we agree it makes sense for the state to assess the potential problems, both in
terms of functionality and stability, to the state’s payroll system and explore potential solutions to
these problems. Nonetheless, the appropriate selection of a project alternative will be critical to the
success of the future project. Substantial delays and cost overruns are particularly acute risks for a
project of this complexity. As such, we recommend the Legislature require the SCO to present its full
findings from the alternatives analysis before it is granted additional funding. This would provide
the Legislature with a clear opportunity to provide meaningful input and weigh in with its own
priorities for the payroll system before a solution is procured.
Background problems during its pilot stage, the SCO terminated
its contract with the project’s primary vendor, SAP
Recent Unsuccessful Attempt to Replace
Public Services, Inc. (SAP). Since then, the SCO
Legacy Payroll Systems. The SCO is responsible
has entered into a settlement agreement with SAP
for issuing pay to the state workforce, including
for its lawsuit related to the TFC project, resolved
employees of the state, CSU, and judicial council.
the various payroll errors produced by the system
In 2004, the SCO proposed the Twenty-First
while it was in use during the pilot, and closed out
Century (TFC) project, the information technology
the project. For a full history of the TFC project
(IT) replacement for its existing human resources
through the settlement agreement in June 2016,
management and payroll systems. In February
please see the Appendix.
2013, after the project experienced various
2017-18 BUDGET
Still May Be Necessary to Update the State’s Stage 2 alternatives analysis, Stage 3 procurement
Payroll System. In the 2016-17 budget, the SCO analysis, and Stage 4 bid analysis and finalization of
received funding for eight positions to restart the project details. Each stage: (1) requires sponsoring
replacement of the state’s payroll system (now departments to conduct specific planning-related
renamed the California State Payroll System, or analyses and submit an associated planning
CSPS). When the TFC project was first proposed document to CDT, and (2) provides CDT with
in 2004, the SCO justified the project by noting a discrete decision point in its approval process.
there were functionality and stability issues that Departments cannot begin their projects without
warranted pursuit of an updated payroll system. receiving approval from CDT for each of the four
For example, the SCO believed there would be stages. (Please refer to our February 2017 report,
ongoing challenges with maintenance as the The 2017-18 Budget: The New IT Project Approval
legacy systems are technically challenging and and Funding Process, to learn more about the PAL
labor-intensive to update. They also noted that process.)
a new system would reduce the likelihood of
Governor’s Proposal
significant payroll disruptions in future years.
However, the department has now suggested that
Budget Request
the primary reason to replace the state’s payroll
system is to improve functionality, not to address Governor’s Budget Proposes $3 Million to
potential future stability issues. For example, Begin Stage 2 of the PAL. The Governor’s 2017-18
functionality improvements in a new system would Budget includes nearly $3 million to begin Stage 2
allow the SCO to respond more quickly to payroll of the PAL process and continue internal efforts
changes, issue reports to other state agencies and to replace the state’s payroll system. (The SCO
stakeholders, and allow employees online access to anticipates it will complete Stage 1 of PAL in
payroll and tax information. April 2017 using currently authorized resources.)
New State IT Project Approval Process This proposal includes $1.1 million in one-year
Aims to Improve Success of State IT Projects. limited-term funding to support 11 positions
Historically, when departments proposed IT (in addition to the 8 positions authorized in the
projects, the California Department of Technology 2016-17 budget package). These positions would
(CDT) required them to prepare Feasibility Study provide information on existing payroll policies
Reports (FSRs). Various shortcomings with the and procedures, provide technical expertise on the
FSR approval process meant that projects often legacy payroll systems, and respond to the PAL
experienced challenges once they were underway. documentation requirements.
These challenges were frequently associated with Proposal Includes Funding for Consulting
significant cost increases and schedule extensions Services. The proposal also includes $1.8 million
for IT projects. The CDT has begun implementing in 2017-18 for consulting and professional services.
a new IT project approval process—known as the These services include an interagency agreement
Project Approval Lifecycle (PAL)—with the goal of with the Department of Human Resources (CalHR)
helping to bolster project planning and reduce the to ensure the new system appropriately applies
likelihood of project challenges or failure. legal and regulatory requirements and supports
PAL Divides Approval Process Into Four statewide human resources and labor relations
Stages. The stages are: Stage 1 business analysis, policies and practices. The SCO also includes
2 Legislative Analyst’s Office www.lao.ca.gov
2017-18 BUDGET
external consulting services to: (1) help the state • Decentralized Payroll Systems.
rethink its payroll process so they are more efficient Payroll practices can vary significantly
(known as business process re-engineering), across state departments, and some
(2) improve its project management, and departments—such as the Department
(3) support a procurement support vendor. of Forestry and Fire Protection (CalFire)
Tentative Proposed Timeline for Completion and the Department of Corrections and
of PAL. With these (and other previously Rehabilitation (CDCR)—have particularly
authorized) resources, SCO anticipates it will complex payroll needs that complicate the
complete Stage 1 of the PAL process in April 2017. state’s ability to integrate these systems.
The SCO plans to complete Stage 2 by August 2018, (Employees in these departments, for
Stage 3 by May 2019, and Stage 4 by December example, work unusual hours or receive
2019 (with the resources requested in this proposal variable compensation based on a variety of
and additional requests in future years). The SCO conditions.) This alternative would assess
anticipates project work would begin in 2020-21. updating payroll systems in a decentralized
fashion that integrates less complex payroll
Initial Alternatives Under Consideration
departments together and considers
SCO Has Identified an Initial List of alternative approaches for modernizing the
Alternatives to Consider. The SCO has identified payroll systems of complex departments.
the initial list of alternatives it will evaluate during
• Using Commercial Off-the-Shelf (COTS)
Stage 2 of the PAL process. Below, we summarize
Software. This alternative would assess
these alternatives:
modifying existing software—known as
• Modular Approach Toward Centralized
COTS software—as necessary to address
Payroll System. Largely as a means of
the state’s payroll needs. The SCO would
reducing IT project risk, departments
contract with a vendor to make the
across the state are beginning to consider
necessary software modifications. The
developing and deploying their IT projects
SCO anticipates evaluating the viability of
in modules rather than all at once, which
several COTS applications, including SAP
has been the traditional approach. The
software and the Financial Information
modular approach works by breaking
System for California’s (FI$Cal’s) People
projects down into multiple discrete
Soft application. The TFC project also relied
units of functionality, prioritizing them,
on SAP COTS software and contracted
and then deploying them over relatively
with SAP to modify the software for use
short periods of time. In the case of the
by the state. FI$Cal is a state IT project
CSPS, modules would likely focus on
that, when fully built, would integrate the
discrete aspects of the state’s payrolling
state’s financial management systems in
processes, such as benefits administration
the areas of budgeting, accounting, cash
or deductions. Collectively, the modules
management, and procurement.
would create a centralized payroll system
for the state. • Internally Custom-Built System. Rather
than rely on existing software and vendors,
this alternative would assess building
www.lao.ca.gov Legislative Analyst’s Office 3
2017-18 BUDGET
a custom system primarily using SCO The Legislature should note that these
employees. Under this approach the state alternatives are not mutually exclusive. For example,
would not have to conform its payroll the SCO could evaluate using a COTS application
process to accommodate the particular to modernize the payroll systems for less complex
functions of a COTS software. According departments. Meanwhile, to address the needs of
to SCO, this would position it to effectively departments with complex payroll processes, the
maintain and operate the system without SCO could evaluate using a modular approach.
the reliance of a vendor. This could
LAO Comments
allow the SCO to relatively easily modify
the system as payroll practices change, State IT Projects Are Expensive and
such as when a new memorandum of Carry a Great Deal of Risk. The state has a
understanding (MOU) is ratified between history of significant challenges in successfully
the state and bargaining units. implementing IT projects. In some cases, projects
have experienced significant cost overruns and
• Implementing a New Front-End System
multiyear delays. In other cases, the IT projects
on Top of Existing Technology. The
have failed altogether, resulting in either project
existing payroll systems rely on mainframe
suspension or termination and receiving significant
technology. This alternative would assess
legislative and media attention. While the PAL
building a new “front-end” system (the
should increase the quality of up-front planning
interface a user sees) using modern
and result in more accurate cost and schedule
technology while continuing to rely on
estimates than the prior FSR process, it does not
the current mainframe technology in
ultimately guarantee success.
the “back-end” (the code and software
Replacing Payroll System Will Be
that support the system). Despite SCO’s
Challenging . . . The CSPS would be a huge
concerns with the stability of the existing
undertaking. The state’s human resource
systems when the TFC project was first
management and payroll systems are very
proposed in 2004, SCO now believes the
complex and updating those systems (even if it is
existing mainframe technology is stable
through a decentralized or modular approach)
and reliable well into the future. However,
carries significant risk. As we have noted, some
SCO has identified some challenges with
departments, like CalFire and CDCR, have
hiring employees with the technical
particularly complex payroll needs that complicate
expertise to maintain and operate
the state’s ability to integrate these systems. The
mainframe technology.
state’s payroll needs are also continually evolving
as new MOUs institute different compensation
• Initiating “Software as a Service.” This
schemes. As such, a new payroll system may be met
alternative would assess establishing a
with significant, and unforeseen, challenges. While
subscription for an online application to
we do think there is merit to the state pursuing
process the state’s payroll. This option is
an updated payroll system, we would caution the
similar to the COTS alternative. However,
Legislature that the risks common to IT projects
a third-party would own and operate the
are particularly acute in the case of CSPS. As a
software, rather than state.
4 Legislative Analyst’s Office www.lao.ca.gov
2017-18 BUDGET
result, substantial delays and cost overruns are even No Concerns With the Governor’s Current
more likely than usual. Proposal. At this point, we have no concerns
. . . But Rigorous Planning May Result in with the Governor’s proposal to provide one-year
a More Favorable Outcome. While unforeseen limited-term funding to the SCO for the PAL
challenges surely will arise, we are cautiously alternatives analysis. We agree it makes sense for
optimistic about the SCO’s current effort to replace the state to assess the potential risks, both in terms
the state’s payroll system. In particular, the new of functionality and stability, to the state’s payroll
PAL process, while still relatively untested, has system and explore potential solutions to these
the potential to reduce the likelihood of project problems.
challenges or failure through a more rigorous Recommend SCO Report to the Legislature
planning process. Through the PAL, the SCO is With Analysis and Preferred Alternative. The
currently collaborating with CDT and, through PAL process presents the Legislature with an
this year’s proposal, would formally institute its early opportunity to weigh in on its own priorities
ongoing collaboration with CalHR. The SCO’s for IT projects. However, the SCO’s anticipated
initial list of alternatives suggests it is exploring timeline for completing the PAL does not neatly
a wide range of possible project types. These align with the budget cycle. In particular, the SCO
developments are encouraging. anticipates it will complete Stage 2 and move on to
State Should Give Alternative Selection Stage 3 (which initiates the procurement process
Careful Consideration. The alternative selected based on the solution selected in Stage 2) in August
at the end of Stage 2 by SCO and CDT will have 2018, just after the adoption of the 2018-19 state
significant consequences for the future of the state’s budget. If so, the Legislature may not have an
payroll processes. Once an alternative is selected, opportunity to evaluate SCO’s alternatives analysis
it could be years or decades before the state would and express its preferences in the selection of a
implement a different system. (We would note project alternative. We recommend the Legislature
the SCO proposed the TFC project in 2004, over require SCO to present its full findings from the
12 years ago.) The appropriate selection of a project Stage 2 alternatives analysis, alongside its preferred
alternative will be critical to the success of the alternative, before it receives funding for Stage 3.
future project. This would provide the Legislature with a clear
opportunity to provide meaningful input and
weigh in with its own priorities for the CSPS before
a solution is procured.
www.lao.ca.gov Legislative Analyst’s Office 5
2017-18 BUDGET
6 Legislative Analyst’s Office www.lao.ca.gov
2017-18 BUDGET
Appendix delays, SCO issued a breach-of-contract notice to
BearingPoint. The vendor and SCO then reached
History of the TFC Project a plan to address project failures and integration
continued. These delays extended the schedule by
Modernization of State’s Human Resources
two years and raised estimated total costs to about
and Payroll Management Systems. In 2004,
$180 million.
the State Controller’s Office (SCO) proposed
Vendor Contract Terminated. After several
the Twenty-First Century (TFC) Project, the
months, BearingPoint once again fell behind
information technology (IT) effort to replace the
schedule, unable to complete project activities
existing statewide human resources management
and provide deliverables on time. With the
and payroll systems used to pay approximately
project’s schedule and development in jeopardy,
260,000 state employees. The new system, also
the Department of General Services (DGS) issued
called MyCalPAYS, was intended to allow the
a default notice to the vendor on December 3,
state to improve management processes such as
2008. The notice stated that the vendor failed
payroll, benefits administration, and timekeeping
to: (1) properly manage the project, (2) complete
and include self-service access for employees and
designs in a timely manner, and (3) make progress
managers, among other capabilities. The SCO noted
toward development. On January 6, 2009, SCO
that the existing systems, commonly referred to
formally terminated the contract, and primary
as “legacy systems,” were developed more than 30
work on the TFC project stopped.
years ago and are inflexible, fragmented, and costly
Strategy to Move Project Forward Developed.
to maintain. In 2005, the Legislature approved the
Following the termination of the primary vendor
project with an estimated total cost of $130 million
contract, SCO developed a new strategy. In
with full implementation scheduled for July 2009.
particular, the project scope was narrowed by
Two-Phase Procurement. In conjunction
excluding California State University (CSU) from the
with state IT oversight officials, SCO decided to
project. The CSU has different payroll requirements
pursue a two-phase, or “unbundled,” procurement
from those applicable to state civil service employees.
approach. This meant the state sought two vendors
The legacy system would continue to process payroll
and undertook two procurements. The first vendor
for CSU until a revised system for CSU employees
would supply the software package, and the second
was completed as a separate project. The SCO also
vendor (the primary vendor) would integrate the
decided to select a new system integrator using a
software to the state’s business requirements. In
two-stage procurement approach, discussed below.
April 2005, SAP Public Services, Inc. (SAP) was
The new strategy was documented in Special Project
selected to supply the software package. The system
Report (SPR) 3.
integrator contract was awarded to BearingPoint in
New Contract Procured. In March 2009, DGS
June 2006.
released a request for proposal for a new system
Early Issues Delayed Project Development.
integrator. The procurement was conducted as
During 2006 and 2007, SCO asserted that
a two-stage procurement approach. Stage I was
multiple problems had emerged with the work of
the selection of contractors to evaluate the work
BearingPoint. The vendor asserted that issues with
completed to date and its possible reuse, and to
the software package and with SCO caused delays.
better understand the requirements of the project.
In October 2007, following multiple schedule
Accenture, LLP, and SAP were selected in Stage I.
www.lao.ca.gov Legislative Analyst’s Office 7
2017-18 BUDGET
Both companies submitted Stage II proposals, calculated, and medical benefits were denied for
which detailed the approach, cost, and schedule some employees and their dependents. In one
for completing the project. In February 2010, case, employees that took vacation time during the
SAP was awarded the contract, at which point payroll cycle received compensation in addition to
the project costs and schedule were revised. The their base salary. Attempts to correct these errors
project schedule was extended to October 2012, created further problems in the following payroll
and estimated total costs rose to $283 million. cycle. In early August, project staff determined
Implementation was to occur in five phases— that the severity of these issues warranted the
known as pilots and waves—where Pilots 1 and 2 delay of Pilot 2, an expansion of the new system to
would bring a small number of employees into the 15,000 employees across numerous departments.
new system in order to test it prior to Waves 3, 4, Pilot 2 was initially delayed from September 2012
and 5, which would fully implement the system in to March 2013. As a result of continued challenges,
three large and roughly proportional stages. (SPR 4 discussed below, Pilot 2 was not implemented.
documented these changes.) Cure Notice Issued to SAP. On October 25,
Subsequent challenges occurred when the 2012, SCO issued a cure notice to the system
project began converting data from the legacy integrator, SAP, expressing serious concern
system to the new system. Project management regarding SAP’s ability to successfully implement
issued a cure notice to the primary vendor, the new system. According to SCO, SAP’s lack of
which then subcontracted with a data migration expertise and strategic planning lead to inadequate
vendor, BackOffice Associates, to improve the data scheduling, staffing, knowledge transfer, deliverable
conversion process. SPR 5 accounted for these management, and quality assurance. The SCO
delays, increased the estimated total project costs also identified concerns regarding design, testing,
to $373 million, and extended the final wave of the organizational change management, and training
project, Wave 5, by one year—from October 2012 to weaknesses. In total, the cure notice cited 13
September 2013. grievances and prompted SAP to correct these
Pilot 1 Test. On June 11, 2012, the first major problems by November 30, 2012, so that the project
test of the state’s new payroll system took place. The could move forward.
test, known as Pilot 1, produced payroll, benefits, SAP Responds to Notice. The SAP submitted a
timekeeping, and position management activities response to the cure notice on November 30, 2012.
for about 1,500 SCO employees. This pilot program In its response, SAP did not assume responsibility
tested the new system’s functionality with a small for the grievances outlined by SCO and took no
number of employees in an effort to identify and action to resolve the issues.
correct potential problems before expanding the Contract Terminated in Early 2013. On
number of employees covered by the new system. February 8, 2013, SCO terminated its vendor
Problems Encountered During Pilot 1. contract with SAP, citing inaction regarding issues
Although SCO expected minor discrepancies listed in the cure notice and a lack of confidence in
during Pilot 1, significant errors surfaced during the vendor to implement the project successfully.
the first payroll cycle and persisted through each At the same time, the Technology Agency—now
of the subsequent seven monthly payroll cycles. the California Department of Technology (CDT)—
In particular, incorrect paycheck deductions were suspended further work on the project until a new
made, payroll and pension wages were erroneously plan could be established.
8 Legislative Analyst’s Office www.lao.ca.gov
2017-18 BUDGET
State Payroll Reverted to Legacy Systems. legacy systems in order to identify inaccuracies and
Beginning in March 2013, SCO returned the ensure that no pay or benefits discrepancies were
payroll processing for its Pilot 1 employees to left unresolved. As a result of this precaution, SCO
the existing legacy systems. According to project indicated the return to the legacy payroll systems
staff, SCO began running parallel payrolls on the for these employees should not pose a problem.
Figure A1
Twenty-First Century Project Timeline
• May 2004—The Department of Finance approves the TFC project FSR and the project begins.
• April 2005—The TFC project procures the SAP Public Services, Inc. (SAP) software solution for a new system
and begins a second procurement for an integration vendor to design, develop, and deploy the solution.
• June 2006—The TFC project contracts with BearingPoint, the winning system integration vendor.
• January 2009—After experiencing multiple serious problems, the state issues a notice of default to
BearingPoint and terminates the contract.
• February 2010—After completing a second vendor procurement, the TFC project contracts with SAP to
complete the new system. With the start of the SAP system integration contract, the TFC project updates its
costs and schedule with SPR 4.
• Spring 2011—Initial data conversion tests between the state’s existing payroll system and the new system are
problematic. The TFC project staff identify additional implementation issues.
• August 2011—The TFC project issues a cure notice to the system integrator, SAP, requiring SAP to improve
data conversion, among other requests. The SAP subcontracts with BackOffice Associates in order to remedy
the cure notice, and the project continues.
• November 2011—Project staff and SAP review data conversion and revise timeline, delaying the first pilot test
by nine months. SPR 5 includes new cost and schedule estimates.
• June 2012—Pilot 1 goes live, processing payroll for about 1,500 SCO employees.
• August 2012—The TFC project staff report significant errors during the go-live payroll, including overpayments,
incorrect deductions, and leave balance discrepancies. Staff tentatively delay Pilot 2 from September 2012 to
March 2013.
• October 25, 2012—The SCO issues a second cure notice to the system integrator, SAP, requiring SAP to
increase personnel on the project, reschedule project milestones, and stabilize the software so that Pilot 2 and
Waves 3, 4, and 5 may go forward, among other requests.
• November 30, 2012—SAP issues its response to SCO’s cure notice, denying responsibility for the Pilot 1
payroll inconsistencies and timeline delay.
• February 8, 2013—The SCO terminates its vendor contract with SAP and returns Pilot 1 employee payroll to
the existing legacy system.
• February 8, 2013—The California Technology Agency (now the California Department of Technology)
suspends the TFC project, citing the vendor’s failure to finish the project and unwillingness to remedy the
issues SCO presented in the cure notice.
• November 2013—The SCO files a lawsuit against SAP, seeking to recover payments made to the vendor, and
SAP files a counter-claim against the SCO.
• June 6, 2016—The SCO reaches a settlement agreement with SAP. Under the terms of the agreement, SAP
pays the SCO $59 million and drops its own claims.
• December 2016—The SCO closes out the TFC project and renames the renewed effort the California State
Payroll System.
TFC project = Twenty-First Century project (also known as MyCalPAYS system); FSR = Feasibility Study Report; SPR = Special Project Report;
and SCO = State Controller’s Office.
www.lao.ca.gov Legislative Analyst’s Office 9
2017-18 BUDGET
Lawsuits Filed. In November 2013, the System/Software Assessment Released. In
SCO filed a lawsuit against SAP for breach of late 2015, Grant Thornton, a contractor hired
contract, seeking to recover payments made to on behalf of CDT, released the results of an
SAP for system integration costs prior to project assessment of the TFC system and software. The
termination. (The primary vendor payments made assessment detailed which system requirements
to SAP totaled $50 million of the $90 million were satisfied, determined which of the delivered
contract.) SAP later filed counter-claims against products could be leveraged to complete the
the SCO for damages and declaratory relief in the project, and estimated the cost for a vendor to
amount of $23 million. complete the system. CDT had planned to complete
SCO Corrects System Errors Produced During two additional assessments of the TFC project—a
Pilot 1. In 2014, SCO took on a “Do the Math” project management assessment and an alternatives
reconciliation effort to identify and correct the assessment. However, these were later rolled into
errors produced during Pilot 1. Of the 1,542 SCO the Project Approval Lifecycle (PAL), the state’s
employees that participated in Pilot 1, the SCO new IT project approval process.
corrected errors for 267 employees who were Post-Implementation Evaluation Report
underpaid and 541 employees who were overpaid. (PIER) Completed. In November 2016, the SCO
In addition, the SCO completed reconciliations submitted a PIER to the CDT. The PIER includes a
for the 100 interfaces that connect with other detailed history of the TFC project, compares the
payroll system payees, such as healthcare providers, project objectives to the actual project outcomes,
retirement savings systems, and tax agencies. documents the failures and successes experienced
Lawsuits Settled. The terms of a June 6, 2016, on the project, and describes the corrective actions
settlement agreement resulted in SAP paying the the SCO will take to improve the chances of future
SCO $59 million and abandoning its own claims success. The submittal of the PIER represents the
against the SCO. The $59 million settlement was official end of the TFC project. For SCO, it also
distributed to the state’s General Fund and special represents the start of a renewed effort to update
funds, using the same proportions established the state’s human resources management and
to support the TFC project. Figure A1 shows the payroll systems. In December 2016, the SCO named
timeline of major events from the start of the the renewed effort the California State Payroll
project in 2004 through the close of the project at System.
the end of 2016.
10 Legislative Analyst’s Office www.lao.ca.gov
2017-18 BUDGET
www.lao.ca.gov Legislative Analyst’s Office 11
2017-18 BUDGET
LAO Publications
This brief was prepared by Ann Hollingshead and Lourdes Morales, and reviewed by Jason Sisney. The Legislative
Analyst’s Office (LAO) is a nonpartisan office that provides fiscal and policy information and advice to the Legislature.
To request publications call (916) 445-4656. This brief and others, as well as an e-mail subscription service,
are available on the LAO’s website at www.lao.ca.gov. The LAO is located at 925 L Street, Suite 1000,
Sacramento, CA 95814.
12 Legislative Analyst’s Office www.lao.ca.gov