The 2017-18 Budget: The New it Project Approval and Funding Process

The 2017-18 Budget: The New IT Project Approval and Funding Process MAC TAYLOR • L E G I S L A T I V E A N A L Y S T • FEBRUARY 2017 2017-18 BUDGET 2 Legislative Analyst’s Office www.lao.ca.gov 2017-18 BUDGET EXECUTIVE SUMMARY State Has Initiated a New IT Project Approval Process. The state has experienced considerable challenges successfully implementing technology projects. There are various factors that can contribute to project challenges, one such factor is poor project planning. In recent years, the California Department of Technology (CDT) has begun implementing a new information technology (IT) project approval process—known as the Project Approval Lifecycle (PAL)—with the goal of helping to bolster project planning and reduce the likelihood of project challenges or failure. Prior Project Approval Process Created Challenges. Historically, when departments proposed IT projects, CDT required them to prepare Feasibility Study Reports (FSRs). The FSR identified the problem, evaluated alternatives, and identified a technical solution. Various shortcomings with the FSR approval process meant that projects often experienced challenges once they were underway. These challenges were often associated with significant cost increases and schedule extensions. New Project Approval Process—PAL. In response to the issues with the FSR approval process, in 2016 CDT fully implemented a new IT project approval process—PAL. It divides CDT’s approval process into four stages—business analysis, alternatives analysis, procurement analysis, and bid analysis and finalization of project details. Each stage (1) requires sponsoring departments to conduct specific planning-related analyses and submit an associated planning document to CDT and (2) provides CDT with a discrete decision point in its approval process. Collectively, the planning documents from the four stages create a comprehensive plan for implementing the proposed IT project. Departments cannot begin their projects without receiving approval from CDT for each of the four stages. PAL Addresses Shortcomings of Prior Approval Process . . . The new project approval process allows departments to refine their plans and analysis collaboratively with CDT to arrive at more accurate cost estimates and sound project plans at the time of project approval. With a more accurate cost and schedule baseline, sponsoring departments are anticipated to experience fewer challenges once the project is underway. . . . And Introduces Potential Trade-Offs. While the potential benefits of PAL appear clear, the new project approval process comes with trade-offs and implications. Since the PAL process requires more detailed analysis upfront and includes new activities (mainly procurement) that previously occurred after approval, it is likely to take longer upfront and some departments may request a budget augmentation to support the effort. It is uncertain how long it will take departments to move through the entire PAL process. If the project approval and budget process do not align, the Legislature could be asked to approve funding for project design, development, and implementation without the benefit of a complete project plan. This could compromise the Legislature’s effective budgetary oversight of the project. Legislative Decision Points Under New PAL Process. The Legislature has two key decision points under the new project approval process: (1) whether to fund planning activities associated with the PAL process for proposed IT projects and (2) whether to fund project design, development, and implementation for projects ultimately approved by CDT. Additionally, like in the FSR process, www.lao.ca.gov Legislative Analyst’s Office 3 2017-18 BUDGET the Legislature will retain its oversight role of approved projects, which may include decisions regarding future changes to the project. Issues for Legislative Consideration. Although the PAL process has the potential to improve the quality of IT project implementation in theory, we raise a number of issues for the Legislature to consider as it exercises oversight of this new process: • In Some Cases, Funding for Planning May Have Merit . . . The merits of providing funding for project planning proposals should be determined on a case-by-case basis. We note a few issues the Legislature may want to consider when determining whether to support a request for planning funds or require a department to absorb the cost of planning a proposed IT project. • . . . And Gives Legislature an Opportunity to Weigh in Early. When sponsoring departments request funding for PAL-related planning activities, it presents the Legislature with an early opportunity to weigh in on its own priorities. If the Legislature has certain priorities it would like reflected in the project, it could build in requirements that ensure that the department considers those priorities as part of its budget approval. • Legislature May Need to Build in Additional Oversight Methods. When the PAL process does not neatly align with the budget cycle, the Legislature may need to build in additional oversight methods. • Actual Benefits of New Project Approval Process Should Be Evaluated. Several years often elapse between project approval and full system implementation. We recommend that CDT report at budget hearings on the quantitative and qualitative measures it will use to evaluate the effectiveness of PAL and project success. 4 Legislative Analyst’s Office www.lao.ca.gov 2017-18 BUDGET INTRODUCTION Technology Can Improve State Operations state IT project failures. These failures have resulted and Delivery of Services. Technology has the in either project suspension or termination and potential to improve how Californians interact have received considerable legislative and media with government—making this interaction more attention. In other cases, projects have been efficient, reliable, and convenient. Strides in ultimately completed, but only after significant cost private sector technology allow the public to access overruns and multiyear delays. tools that improve their daily lives—from online State Has Initiated a New IT Project Approval portals that facilitate communicating with doctors Process. With so much at stake, the Legislature to mobile applications that schedule restaurant has looked to CDT to determine what changes are reservations. The public has come to expect a necessary to ensure that IT projects are successfully similar level of service from government. The state completed. While there are various factors that can is currently undertaking numerous information contribute to project challenges, one such factor technology (IT) projects that are intended to is poor project planning. (Please refer to our May increase the quality of services provided to 2015 report, The 2015-16 Budget: Centralizing State the public and improve the efficiency of state IT Project Management, to lean about CDT’s efforts programs. (When an IT project is completed and to address another major contributor to project fully operational, it is then referred to as an IT challenges—poor project management.) In recent system.) Specifically, the state currently has about years, CDT has begun implementing a new IT 30 IT projects in various phases of development project approval process—known as the Project that are approved by and under the oversight of Approval Lifecycle—with the goal of helping to the California Department of Technology (CDT), bolster project planning and reduce the likelihood the state’s central IT organization. The total cost, of project challenges or failure. should the state complete all of these IT projects In this report, we discuss CDT’s as currently envisioned, is estimated to be about responsibilities, describe the challenges with the $3.2 billion. prior project approval process; describe the new State Has Had Challenges Implementing IT project approval process; discuss the potential Projects. The state has experienced considerable benefits, trade-offs, and implications of the new challenges successfully implementing technology process; identify the key legislative decision points projects. While there have been some project within the new approval process; and identify successes, there have also been various high-profile issues for legislative consideration. BACKGROUND What Are the Responsibilities of CDT? Below, we describe CDT’s role in the approval and oversight of state IT projects. CDT, a department within the California Approves and Oversees IT Projects. One of Government Operations Agency, is the state’s CDT’s responsibilities is to review and approve central IT organization and has broad authority IT project proposals developed by a “sponsoring over all aspects of technology in state government. www.lao.ca.gov Legislative Analyst’s Office 5 2017-18 BUDGET department”—that is, the state department role. In some cases, projects change considerably undertaking the IT project. (We discuss below and several SPRs are required over the life of the the concept of a “reportable” IT project, meaning project. projects that are subject to CDT’s approval and Suspends, Terminates, and Reinstates IT oversight authority.) CDT evaluates department Projects. As part of its project approval and proposals to ensure that proposed projects: (1) are oversight responsibilities, CDT has the authority based on well-defined programmatic needs, to suspend, terminate, or reinstate an IT project (2) consider feasible alternatives to address the based on its performance. CDT also has the identified needs, (3) identify a sound technical authority to hold departments accountable for poor solution, (4) implement project management best performance, including by restricting future project practices, and (5) comply with state policies and approvals pending demonstration of successful procedures, among other CDT considerations. correction of the identified performance failure. Once CDT approves a department’s project What Projects Are Subject to CDT’s proposal, its role changes to one of providing Approval and Oversight Authority? project oversight. Specifically, CDT provides an independent review and analysis of the project Only “Reportable” IT Projects. The most to determine if it is on track to be completed on significant state IT projects are considered to be schedule and within budget, and whether it will reportable and therefore subject to the approval provide the benefits identified by the sponsoring and oversight of CDT. In contrast, non-reportable department. As part of this review, CDT routinely projects are completely within the authority of reports to sponsoring departments on issues sponsoring departments to manage. Projects that of concern that it has identified, shares lessons meet one or more of the following characteristics learned from other projects, and recommends risk are reportable: mitigation and issue resolution strategies. • Estimated project cost exceeds the Reviews Revised Project Plans, as Necessary. sponsoring department’s cost threshold Over time, a project may change in scope or deviate assigned by CDT. The assigned cost from the schedule and/or cost that was established thresholds generally range from $200,000 during the approval process. Any significant to $2 million, and usually reflect changes to the project plan are documented CDT’s assessment of the department’s and justified in Special Project Reports (SPRs). performance in previous projects. The revised project plans are developed by the sponsoring department and submitted to CDT • Projects with costs under the department’s for its review and approval. Approval of an assigned cost threshold but for which the SPR constitutes a new agreement between the costs are not absorbable by the sponsoring sponsoring department and CDT. This process department and therefore require an resets the scope, schedule, and cost from which the appropriation by the Legislature. project’s progress and performance are assessed. • Projects that are specifically mandated by Once CDT approves the SPR and the Legislature the Legislature. approves the associated funding, the department can move forward with the project based on the revised plan and CDT continues its oversight 6 Legislative Analyst’s Office www.lao.ca.gov 2017-18 BUDGET For What Period Does CDT Oversee Projects? learned, and provides a final summary of actual versus expected costs—to CDT. Once the PIER CDT continues to oversee projects until is submitted, the project becomes an IT system the sponsoring department has submitted a maintained and operated by the sponsoring Post-Implementation Evaluation Report (PIER)—a department and is no longer subject to CDT’s report that details whether and how the project oversight. objectives were accomplished, documents lessons PRIOR PROJECT APPROVAL PROCESS CREATED CHALLENGES Historically, when departments proposed IT planning activities and developed FSRs largely projects, CDT required them to prepare Feasibility independently of CDT and then submitted the Study Reports (FSRs). The FSR identified the completed analysis to CDT. Because departments problem, evaluated alternatives, and identified a largely planned projects without guidance from technical solution. The FSR was developed by the CDT, it was possible for a department to make an sponsoring department and submitted to CDT for early erroneous assumption that jeopardized the review and approval. Various shortcomings with quality of the entire planning effort. Even if CDT the FSR approval process meant that projects often ultimately approved a sponsoring department’s experienced challenges once they were underway. FSR, it was difficult for CDT to identify potential These challenges were often associated with planning deficiencies because it generally had not significant cost increases and schedule extensions. been involved in the planning process. Deficiencies Below, we detail the primary challenges associated in the planning generally manifested during the with the FSR process. design, development, and implementation (the Cursory Business Analysis Lead to Unmet system deployment) of the project, when issues are Needs. While the FSR required some analysis significantly more challenging to resolve. regarding the programmatic needs that motivated Approval Process Did Not Incorporate the proposed IT project, this “business analysis” Knowledge From Project Vendors. Once a was generally done rather superficially. As a department completed an FSR, it would submit result, some departments ultimately discovered a it to CDT for approval. Once CDT approved mismatch between the functionality offered in its the project, the FSR and budget-related requests newly implemented IT system and the needs of the would be submitted to the Legislature for approval system users—including department staff and/or and funding. Under this former project approval members of the public. Departments then faced a process, procurement occurred after CDT approved difficult decision—they either had to make do with a department’s FSR and the Legislature approved a new system that did not fully meet their needs initial project funding. (Procurement is the and expectations or invest in potentially costly and process through which departments work with time-consuming system enhancements. potential project vendors to plan for the purchase Limited Collaboration Between Sponsor of technology for IT projects and select vendors Department and CDT Resulted in Poor Planning. to install the technology.) Generally, because Under the FSR process, departments undertook departments learn more about the actual cost of www.lao.ca.gov Legislative Analyst’s Office 7 2017-18 BUDGET achieving their technology objectives through planning and/or management.) Refer to the nearby interactions with potential project vendors box for an example of a large IT project that has through the procurement process, the timing of evolved considerably since its FSR was approved in procurement typically resulted in unanticipated 2005. cost increases, and schedule extensions beyond Limited Ability to Track Overall Project those established in the FSR. Departments would Success. SPRs reset the baselines from which CDT have to submit a revised project plan—an SPR— tracked a project’s progress and performance. soon after a vendor was selected to update the This meant that, at its conclusion, a project’s project baseline, scope, schedule, and cost. performance was largely evaluated relative to The Legislature Would Approve Projects the last approved SPR, which generally aligned Based on Incomplete Planning. Historically, the closely with the actual project scope, schedule, Legislature authorized funding for an IT project and cost, but potentially deviated significantly based on the baselines established in the FSR. from the approved FSR. Additionally, “project However, given the shortcomings with the FSR success” was not a formally defined concept. approval process, it was common for departments Project performance was tracked based on easily to discover the project would be costlier and take quantifiable measures, such as the number of longer to implement than previously anticipated. months behind schedule or percent over budget Projects frequently required several SPRs to revise the project was compared to its most recent the project scope, schedule, and/or cost once it was SPR. However, those measures excluded more underway. The SPRs were sometimes problematic difficult to assess but more important measures because the Legislature had approved an IT project of success, such as how well the IT system worked based on expectations that were laid out in the for its intended users. The absence of a clear and FSR, but the final project may have ended up with commonly accepted definition for project success, significant scope, schedule, and/or cost differences. and the inability to track project performance (We note that SPRs can reflect reasonable changes because it was constantly evaluated based on reset that can improve the quality and ultimate success baselines, limited CDT’s and the Legislature’s of a project. SPRs do not always reflect poor project ability to gauge overall project success. WHAT IS THE NEW PROJECT APPROVAL PROCESS? In response to the issues with the FSR approval document to CDT and (2) provides CDT with a process described above, in 2016 CDT fully discrete decision point in its approval process. implemented a new IT project approval process— Upon review of the planning document associated Project Approval Lifecycle (PAL). It divides CDT’s with each stage, CDT can offer sponsoring approval process into four stages—business departments one of three decisions: analysis, alternatives analysis, procurement • Approve. Approval of the planning analysis, and bid analysis and finalization of document associated with the stage means project details. Each stage (1) requires sponsoring CDT finds it to be of adequate quality and departments to conduct specific planning-related the sponsoring department is authorized to analyses and submit an associated planning proceed to the next PAL stage. 8 Legislative Analyst’s Office www.lao.ca.gov 2017-18 BUDGET Evolution of the FI$Cal Project The Financial Information System for California (FI$Cal) is an information technology (IT) project currently underway by a partnership of control agencies including the Department of Finance, the State Controller’s Office, the State Treasurer’s Office, and the Department of General Services. FI$Cal replaces the state’s aging and decentralized IT financial systems with a new system that will integrate state government processes in the areas of budgeting, accounting, cash management, and procurement. Since the project began in 2005, it has changed many times in scope, schedule, and cost from what was initially anticipated in the Feasibility Study Report (FSR). FI$Cal is considered to be the state’s largest IT project to date. The project is currently operating under its sixth Special Project Report (SPR), which was approved by the California Department of Technology in February 2016. The figure below provides a description of the evolution in the scope, schedule, and cost of the project over the last 12 years, and illustrates how dramatically a project can change after the FSR. Because FI$Cal is an extremely ambitious and complex IT project, the changes in scope, schedule, and cost from SPR to SPR are of a magnitude not observed in other IT projects. However, the general fluctuations over time are common to many IT projects approved through the FSR process. Evolution of the FI$Cal Project’s Cost, Schedule, and Scope (In Millions) Estimated Final Total Estimated Implementation Project Plan Project Cost Date Summary of Project Scope and Status FSR $138 July 2011 Budgeting system for one state department. Estimated cost and July 2005 timeline largely based on state analysis. SPR 1 $1,334 June 2015 Expanded project scope to include additional functionality for all December 2006 state departments. Estimated cost and timeline largely based on state analysis. SPR 2 $1,620 June 2017 Scope unchanged. Estimated cost and timeline based on December 2007 additional analysis by the state. SPR 3 Unspecified Unspecified The procurement process began. Estimated cost and timeline for November 2009 project left unspecified until after the software and vendor were selected through the procurement process. SPR 4 $617 July 2016 Estimated cost and timeline revised based on selected March 2012 software, state and vendor analysis, and a less risky “phased” implementation approach. SPR 5 $673 July 2017 Scope remained largely unchanged. Estimated cost and January 2014 timeline based on state and vendor analysis and a revised implementation schedule. SPR 6 $910 July 2019 Scope remained largely unchanged. Estimated cost and February 2016 timeline based on state and vendor analysis and a revised implementation schedule. FI$Cal = Financial Information System for California; FSR = Feasibility Study Report; and SPR = Special Project Report. www.lao.ca.gov Legislative Analyst’s Office 9 2017-18 BUDGET • Reject. Rejection of the planning document alternatives for accomplishing the project objectives associated with the stage means CDT identified in Stage 1. Based on this analysis, finds it to be of inadequate quality and the departments identify the recommended alternative sponsoring department is not authorized to and develop a procurement strategy. Departments proceed. often rely on market research to gather the information necessary to successfully complete this • Rethink and Resubmit. This means the stage. This is the first time in the PAL process that planning document associated with the CDT requires sponsoring departments to provide stage is of inadequate quality but the project a financial analysis for the project, including a has merit. CDT recommends the sponsoring comparison of the cost of not implementing a new department rethink, revise, and resubmit IT system—that is, maintaining existing technology the analysis for future consideration. or manual processes—to the various alternatives. Sponsoring departments must secure approval Stage 3—Procurement Analysis. The third from CDT for each of the four stages before the stage requires departments to identify the department can begin the IT project. In the detailed requirements for the project based on the following section, we describe the specific analysis recommended alternative selected in Stage 2 and associated with each stage. develop a solicitation—a request for information Stage 1—Business Analysis. In the first stage of from vendors. The solicitation documents the the PAL process, a department that is considering project requirements, terms, and conditions. an IT project must first layout the issue that could Stage 4—Bid Analysis and Finalization potentially be solved by an IT project. This business of Project Details. The fourth stage requires case is centered on (1) the programmatic problems departments to release the solicitation developed in that substantially and adversely affect the operation Stage 3. Prospective vendors use the solicitation to and delivery of a service, (2) the programmatic develop their bid for an IT project. The department opportunities that may substantially improve evaluates the bids that respond to the solicitation operation and delivery, (3) the expected revenue and selects a vendor. The Stage 4 planning generation or cost savings, or (4) compliance with document also outlines the final project details, legislative mandate. In this first stage, sponsoring including the project scope, schedule, cost, and departments also document the project objectives resource needs. These project details serve as a and assess their readiness to take on an IT project. baseline for monitoring the project’s progress and (All departments are required to submit Stage 1 performance moving forward. planning documents regardless of whether the Completed PAL Process. Each stage in the proposed projects are anticipated to be reportable PAL process builds off the analysis from the or non-reportable. However, non-reportable prior stage. Collectively, the planning documents projects with an approved Stage 1 planning from the four stages create a comprehensive document are not required to proceed with the plan for implementing the proposed IT project. subsequent PAL stages. Instead, departments Departments cannot begin their projects without undertaking non-reportable projects are internally receiving approval from CDT for each of the four responsible for the success of their own projects.) stages. Because this process is still relatively new, Stage 2—Alternatives Analysis. The second it is uncertain how long it will take for sponsoring stage requires departments to evaluate various departments to complete the PAL process. In some 10 Legislative Analyst’s Office www.lao.ca.gov 2017-18 BUDGET cases, a department may be able to move through sponsoring department’s capacity for planning the four stages in a single fiscal year. In others IT projects, (2) the complexity of the project, and cases, planning may extend across several fiscal (3) CDT’s workload. Refer to Figure 1 for a visual years. The duration will likely be affected by (1) the depiction of the new project approval process. Figure 1 New Project Approval Process: Project Approval Lifecycle (PAL) Stage 1 Stage 2 Stage 3 Stage 4 Business Alternatives Procurement Bid Analysis Analysis Analysis Analysis and Finalization of Project Details Identify Programmatic Assess Existing Develop Solution Solicitation Release Problem/Opportunity Programming Processes Requirements Select Vendor Establish Business Market Research Develop Solicitation Case/Need Contract Management Develop Mid-Level Identify Objectives Solution Requirements Baseline Project CDT Decision Assess Departmental Identify Solution Readiness Alternatives Recommend Solution CDT Decision Financial Analysis = Reject CDT Decision = Rethink and Resubmit CDT Decision = Approve AAwwaarrdd CCoonnttrraacctt aanndd SSttaarrtt PPrroojjeecctt CDT = California Department of Technology. HOW DOES THE NEW PROJECT APPROVAL PROCESS ADDRESS SHORTCOMINGS OF PRIOR PROCESS? CDT designed the PAL process to improve the collaborate with CDT at various points during the planning quality and increase the likelihood of planning process. Should CDT identify deficiencies success for IT projects undertaken by the state. The during the planning process, it can communicate new process attempts to accomplish these goals by its concerns and recommended solutions early addressing some of the shortcomings of the FSR to sponsor departments. This structured and approval process. guided approach towards planning and approving Requires Collaboration Between Sponsor IT projects may lead to a better planned project Department and CDT During Project Planning and bolster the likelihood of successful project Stage. The PAL stages require departments to implementation. www.lao.ca.gov Legislative Analyst’s Office 11 2017-18 BUDGET Emphasizes Business Analysis. The new baseline estimates because of the detailed analysis project approval process emphasizes due diligence required by PAL, collaboration with CDT, and in defining the business case before an IT solution of the movement of procurement into the project is selected. As discussed previously, projects approval process. Ultimately, it allows departments approved via the FSR process often resulted in a to refine their plans and analysis collaboratively mismatch between the programmatic needs and with CDT to arrive at more accurate estimates and the functions available in the IT system. The new sound project plans at the time of project approval. project approval process intends to address this With more accurate cost and schedule baselines issue by requiring a detailed business analysis in there should be far fewer SPRs. Stage 1. Departments are required to consider the New Process May Save Time and Money in programmatic needs that motivated the project Long Term. Because of the hoped-for higher quality proposal and the project objectives that most upfront planning in the PAL process, departments serve the interest of the affected programs and its may save time and money over the life of the IT constituents. project. Under the FSR process, departments Approval Process Incorporates Knowledge often had to course correct when a major issue From Working With Vendors. Unlike the FSR was discovered or the project fell behind schedule. process, PAL incorporates procurement activities Project staff were often redirected to problem into the project approval process. When the project solve the issue and then tasked with updating the is fully approved under the new process, the project plan and securing approval from CDT administration and the Legislature should have through the SPR process. Diverting resources a better understanding of, and confidence in, the away from project execution activities and towards project schedule and cost because it reflects the issue resolution and the SPR process can be costly vendor selection. In this way, the new process not and time consuming. A sounder project plan that only could provide more reliable information, but mitigates the need for course corrections during also moves projects further along in the planning project development could help departments and development process relative to the FSR realize the benefits of their IT systems more quickly process. and reduce project costs. Figure 2 provides a Increases Likelihood That Final Project comparison of the new and former project approval Aligns With Expectations. The new process should processes. provide for more accurate cost and schedule WHAT ARE THE POTENTIAL TRADE-OFFS AND IMPLICATIONS OF THE NEW APPROVAL PROCESS? While the potential benefits of PAL appear process will likely take longer for departments to clear, the new project approval process comes with complete than the former FSR approval process. trade-offs and implications of which the Legislature This is a natural consequence of making the should be aware. project approval process more robust with the New Project Approval Process Will Likely expanded requirements for detailed upfront Take Longer Upfront. The new project approval analysis. Departments may need to call upon a 12 Legislative Analyst’s Office www.lao.ca.gov 2017-18 BUDGET Figure 2 Comparison of New and Former Project Approval Processes New Project Approval Process Project Approval Lifecycle (PAL) Stage 1 Stage 2 Stage 3 Stage 4 Project Design, Business Alternatives Procurement Bid Analysis Project Development, and Analysis Analysis Analysis and Finalization Starts Implementation of Project Details Former Project Approval Process Project Feasibility Project Solicitation Solicitation Project Design, Concept Study Starts Development Release and Development, and Report Contract Award Implementation (FSR) Roughly Akin to Stage 3 and Stage 4 Roughly Akin to Stage 1 and Stage 2 significant number of programmatic and technical budget includes several proposals that request experts to successfully complete the detailed funds to support various stages of the new project analysis CDT requires for project approval. In some approval process. (Please refer to the box on page 14 cases, departments may leverage the expertise of for examples of some of the budget requests to fund contractors for specific planning-related activities, PAL-related activities.) such as identifying system requirements. This is PAL Process May Not Align With Budget especially true for departments that have limited Cycle. It is uncertain how long it will take experience implementing IT projects. departments to move through the entire PAL Some Departments May Seek Budget process. The duration depends on several factors, Augmentation for Planning Effort. Under the FSR including the complexity of the proposed project process, departments largely absorbed the cost of and the sponsoring department’s experience planning an IT project. However, since the PAL planning IT projects. It is possible that the process requires more detailed analysis upfront stages of the PAL process will not align with the and includes new activities (mainly procurement) state’s traditional budget process. For example, a that previously occurred after approval, some department may receive approval from CDT for departments may request a budget augmentation Stage 4 midway through a state fiscal year. The to support the effort. These budget requests may sponsoring department may not want to delay seek staff resources and/or contract services. CDT the project start for several months until a budget requires departments to have an approved Stage 1 request could be secured for the following fiscal planning document before a department submits year. If a department anticipates that development a budget request for any of the remaining three activities could occur in the same fiscal year stages. Essentially, CDT requires departments to that Stage 4 is anticipated to be complete, the absorb the cost of Stage 1. The Governor’s 2017-18 department could request funding to support www.lao.ca.gov Legislative Analyst’s Office 13 2017-18 BUDGET development activities before the Stage 4 analysis considered a success? Alternatively, if a project is approved by CDT. In this scenario, however, the is completed on time and on budget, but the IT Legislature could be asked to approve funding for system does not effectively meet the needs of users, project design, development, and implementation is the project considered a success? Evaluating without the benefit of a complete project plan. project success likely involves a combination of This could compromise the Legislature’s effective quantitative measures and, harder-to-evaluate, budgetary oversight of the project. qualitative measures. Measuring Project Success Remains Unclear. Unclear How PAL Will Align With Other The PAL process should increase the quality of IT Process Changes. At the same time the state upfront planning and result in more accurate is implementing the PAL process for IT project cost and schedule baseline estimates than the approval, it is also making other IT process related FSR process. However, PAL does not ultimately changes. Most notably, departments are beginning guarantee project success. This is because CDT to consider developing and deploying their IT has still not formally defined project success. projects in increments rather than all at once as one Evaluating success can be complicated. For big project. The Legislature will need to make sure example, if a project runs over budget and takes it understands how any new project development longer to implement than anticipated, but the IT and deployment methods work with the PAL system works effectively for its users, is the project process. Governor’s 2017-18 Budget: Requests to Fund Project Approval Lifecycle (PAL)-Related Activities The Governor’s 2017-18 budget includes several proposals that request funds to support proposed information technology projects undergoing planning through the new project approval process. Below are a few of the budget requests to fund PAL-related activities: • Department of Health Care Services (DHCS). DHCS requests $6.6 million ($727,000 General Fund) to support the planning effort of the proposed Medi-Cal Eligibility Data System Modernization Project primarily during Stage 3 and Stage 4 activities. • Employment Development Department (EDD). EDD requests $4 million in special funds to support the planning effort of the proposed Benefit System Modernization Project during Stage 2 activities. • State Controller’s Office (SCO). SCO requests $3 million ($1.7 million General Fund) to support the planning effort for the California State Payroll System during Stage 2 activities. This request is notable because SCO previously attempted, unsuccessfully, to modernize the state’s payroll systems. SCO terminated the project (21st Century Project) in 2013 following various problems during the pilot stage of the project. SCO is now proposing to undertake a project with similar objectives under the new, more comprehensive, planning and approval process. 14 Legislative Analyst’s Office www.lao.ca.gov 2017-18 BUDGET WHAT ARE THE LEGISLATIVE DECISION POINTS UNDER THE NEW PROJECT APPROVAL PROCESS? The Legislature has two key decision points than one budget request over the planning period Graphic Sign Off under the new project approval process: (1) whether for specific PAL activities. The Legislature will Secretary to fund planning activities associated with the PAL consider the merits of these requests through the Analyst process for proposed IT projects and (2) whether traditional budget process. Some departments may MPA to fund project design, development, and continue to absorb the cost of planning IT projects. implementation for projects ultimately approved In these cases, the Legislature will not learn of the Deputy ARTWORK #170051 by CDT. Additionally, like in the FSR process, the project until Stage 4 is complete. Legislature will retain its oversight role of approved DepaTretmmepnltas tWe_ilLlA ROeqRueepsto Frtu_nladringge .faoirt Project projects, which typically includes decisions Design, Development, and Implementation. regarding future changes to the project. Figure 3 Once CDT approves the project (after Stage 4), depicts the legislative Figure 3 decision points under the new project approval Legislative Decision Points process and through the Project Approval Process life of the project. We discuss the legislative Stage 4 Project Stage 1 Stage 2 Stage 3 Bid Analysis decision points below. Business Alternatives Procurement and Finalization Approval by CDT Analysis Analysis Analysis of Project Details Some Departments May Request Funding Potential Legislative Decision: Some departments may request funding for some or all for Project Planning. of the Stage 2 through Stage 4 planning activities. As described earlier, in some cases, sponsoring departments may request funding to support Legislative Decision: Departments will request funding to start approved project. the planning activities associated with the new project approval Legislative Oversight process. Departments may request funding Project Closure Project Modification PIER from the Legislature for Starts SPR, if activities associated with Necessary all but the first stage of Legislative Decision: the PAL process. The Departments may request funding for project changes and ongoing costs. Legislature can project approval process request periodic updates regarding project status. may span across multiple state fiscal years, and the CDT = California Department of Technology; SPR = Special Project Report; and PIER = Post-Implementation Evaluation Report. Legislature may see more www.lao.ca.gov Legislative Analyst’s Office 15 2017-18 BUDGET the sponsor department will request funding to Legislative Role and Decision Points begin designing, developing, and implementing Following Project Approval. After a project the IT project. This will be an opportunity for has been approved and funded, the Legislature the Legislature to review the complete project has a key oversight role regarding the state’s IT plan and determine if the project objectives and project portfolio. It can require periodic updates approach have merit given the cost, schedule, regarding the status of IT projects of key interest and the Legislature’s own priorities. Typically, or projects that have experienced prior challenges. the Legislature approves funding for IT projects The Legislature also has the opportunity to review on a year-by-year basis. This provides assurances budget proposals associated with revised project that the sponsoring department will return in plans—SPRs. Through this budget review process, a subsequent fiscal year with a status update the Legislature has the opportunity to assess if the and a request for additional funding. Should project changes are reasonable. Near the conclusion project challenges arise, the department would of an IT project, the Legislature often considers be accountable to the Legislature regarding how requests for ongoing funding to support the the challenge developed and steps to address maintenance and operation of the IT system. and mitigate future challenges before additional funding was approved. ISSUES FOR LEGISLATIVE CONSIDERATION Although the PAL process has the potential to development and implementation. improve the quality of IT project implementation The merits of providing funding for these in theory, we raise a number of issues for the project planning proposals should be determined Legislature to consider as it exercises oversight of on a case-by-case basis. When determining this new process. whether to support a request for planning funds or In Some Cases, Funding for Planning May require a department to absorb the cost of planning Have Merit . . . In some cases, departments may a proposed IT project, the Legislature may want to request significant funding to support the PAL consider such things as: process. While we discussed the potential benefits • The sponsoring department’s performance/ of providing resources in the planning effort—a experience planning and implementing sound project plan may make successful project IT projects in the past, which may inform implementation more likely—it is possible that how much support a department may need the proposed project ultimately does not move fulfilling the PAL planning requirements. forward into development and implementation and the funding does not actually lead to a project. A • The complexity of the proposed project— project may not move forward for several reasons, more complex projects typically require including because the department decided not more complex planning. to pursue the project, CDT did not approve all • Whether the project is a priority for the four stages of the PAL process, or the Legislature Legislature. rejected the budget augmentation for project 16 Legislative Analyst’s Office www.lao.ca.gov 2017-18 BUDGET • Whether the project is necessary to comply without a complete project plan versus delaying with federal laws or regulations, especially the implementation of a proposed IT project. If if noncompliance may jeopardize federal delaying implementation would have significant funding. negative consequences, such as preventing a department from meeting a statutorily established • Whether the Legislature is comfortable deadline, the Legislature might consider other approving a budget request that ultimately options that build in its own approval and oversight may not result in a new IT system. role. For example, the Legislature could consider . . . And Gives Legislature an Opportunity to using provisional budget bill language to withhold Weigh in Early. When sponsoring departments the portion of the authorized budget for project request funding for PAL-related planning development until the department meets certain activities, it presents the Legislature with an early conditions, such as submitting an approved Stage 4 opportunity to weigh in on its own priorities for planning document. the project. Under the FSR process, the Legislature Actual Benefits of New Project Approval was not informed of a potential project until the Process Should Be Evaluated. Several years often planning process was complete. A request for elapse between project approval and full system planning funding for the PAL process provides the implementation. This lag will make it difficult Legislature with an earlier opportunity to learn for CDT and the Legislature to determine early about projects that departments are considering. on if the proposed potential benefits of the PAL If the Legislature has certain priorities it would process are realized. Additionally, to fully assess like reflected in the project, it could build in if IT projects are implemented more successfully requirements as part of its budget approval that under the PAL process than under the prior FSR ensure the department considers those priorities. process, the evaluation of several projects that For example, the Legislature could require were approved and fully implemented since PAL departments to consider specific alternatives went into effect in 2016 would be needed. An during Stage 2, include certain functionality in early indicator that PAL may not fully realize its the solicitation developed during Stage 3, or in anticipated benefits could be if SPRs continue to be other ways ensure the system planning reflects its common among IT projects approved through the priorities. PAL process. However, for now, the Legislature may Legislature May Need to Build in Additional want to direct CDT to report at budget hearings Oversight Methods. When the PAL process on: (1) whether there are additional changes to the does not neatly align with the budget cycle, project approval process that may further improve the Legislature may need to build in additional the likelihood for project success and (2) the oversight methods. The planning documents quantitative and qualitative measures it will use developed during the four PAL stages offer the to evaluate the effectiveness of PAL and project Legislature critical information necessary to success. The latter could ensure that CDT, the evaluate the merits of the proposed IT project. Legislature, and IT system stakeholders can speak The Legislature would need to consider the to the performance of IT projects using a common trade-offs between authorizing project funding definition of success. www.lao.ca.gov Legislative Analyst’s Office 17 2017-18 BUDGET 18 Legislative Analyst’s Office www.lao.ca.gov 2017-18 BUDGET www.lao.ca.gov Legislative Analyst’s Office 19 2017-18 BUDGET LAO Publications This report was prepared by Lourdes Morales and reviewed by Ginni Bella Navarre. The Legislative Analyst’s Office (LAO) is a nonpartisan office that provides fiscal and policy information and advice to the Legislature. To request publications call (916) 445-4656. This report and others, as well as an e-mail subscription service, are available on the LAO’s website at www.lao.ca.gov. The LAO is located at 925 L Street, Suite 1000, Sacramento, CA 95814. 20 Legislative Analyst’s Office www.lao.ca.gov