Evaluation of the Statewide Project Management Office

Evaluation of the Statewide Project Management Office MAC TAYLOR LEGISLATIVE ANALYST DECEMBER 7, 2017 analysis full gutter AN LAO REPORT LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT Executive Summary Project Management Office (PMO). As part of the 2014-15 Budget Act, the Legislature approved a Governor’s proposal to establish a centralized project management office within the California Department of Technology (CDT) to improve the day-to-day management of information technology (IT) projects and, eventually, overall IT project outcomes across the state. The PMO would address critical state deficiencies in project management by consolidating a team of experienced project management professionals within CDT. These project managers would provide their services to state IT projects as needed. With support from the Legislature, CDT envisioned that the PMO would eventually manage some of the state’s largest, most complex, and high-costing projects. Given that the PMO has been directly involved with managing state IT projects for two years, we recently evaluated the performance of the PMO to assess whether the office is meeting—or on track to meet—its original objectives. Analysis of the PMO to Date. Thus far, six state IT projects have sought direct project management services from the PMO. For the first six projects, the PMO selected projects likely to progress smoothly, preferring smaller, less complex projects. This allowed the PMO to test the potential of the centralized project management approach and build the reputation and capacity of the office gradually, with the intention of scaling up to larger, more complex projects once the office was more established. Based on our interviews with the six departments utilizing the direct PMO services, we found that the PMO was a valuable asset to projects in need of project management services. Significant Recent Developments. Although the PMO has been operating largely as intended with a high level of satisfaction from departments, during our evaluation we learned of two recent developments that significantly impact the operation of the PMO going forward. First, a reorganization of CDT creates a potential conflict of interest as it places CDT oversight and project management responsibilities under the same chief deputy director. Additionally, CDT implemented a policy change that significantly reduces the number and complexity of IT projects eligible for PMO services. LAO Findings and Recommendations. The Legislature approved the establishment of the PMO with the expectation that the office would meet a critical need in the state for project management expertise and help the state avoid the high-profile IT failures of the past. We found that the PMO was showing success towards meeting the original legislative intent by first establishing the internal processes and capacity to ultimately scale up to larger, more complex projects. However, under the recent policy change, the PMO will now primarily serve smaller, low-complexity projects, which is not in line with the Legislature’s intent when it established the office. Given recent developments, we have concerns about the path forward for the PMO and its ability to succeed and meet the Legislature’s original expectations. In light of our positive findings of the PMO’s operation under the original eligibility policy, we make the following recommendations: • Reassert PMO’s Mission. We recommend the Legislature reassert its original objectives for the PMO in statute. Codifying the objectives of the office would provide CDT with legislative guidance for the operation of the office and ensure that the goals and functions of the office remain aligned with legislative intent. www.lao.ca.gov 1 analysis full gutter AN LAO REPORT • Establish Firewall. We recommend the Legislature require in statute that CDT maintain an organizational “firewall” between its project management and oversight responsibilities. Requiring a firewall would help address and mitigate potential conflicts between the two responsibilities. • Express Intent Regarding PMO’s Independence. We recommend that the Legislature express intent in statute that it expects the PMO to operate independently and serve in the best interests of the projects. This would ease sponsoring departments’ concerns of inappropriate and unnecessary information sharing between the PMO and CDT oversight and cultivate an open pathway of communication between the PMO and sponsoring departments. Additionally, we note that even with the continuation of the PMO as originally envisioned, there is still room for departments with the capacity to manage their own projects to do so. 2 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT INTRODUCTION As part of the 2014-15 Budget Act, the Legislature Given that the PMO has been directly involved approved a Governor’s proposal to establish a with managing state IT projects for two years, we centralized project management office within the recently evaluated the performance of the PMO to California Department of Technology (CDT) to improve assess whether the office is meeting—or on track to the day-to-day management of Information technology meet—its original objectives. In this report, we provide (IT) projects and, eventually, overall IT project outcomes background regarding the objectives and operations across the state. The Project Management Office (PMO) of the PMO, detail our findings in evaluating the PMO, would address critical state deficiencies in project introduce two significant recent developments and management by consolidating a team of experienced their impacts on the PMO, and make associated project management professionals within CDT. These recommendations on how the Legislature should project managers would provide their services to proceed to better align the office with the original state IT projects as needed. With support from the legislative intent. Legislature, CDT envisioned that the PMO would eventually manage some of the state’s largest, most complex, and high-costing projects. BACKGROUND What Is IT Project Management? IT project and Funding Process, to learn about CDT’s efforts management is the application of skills, tools, and to address another major contributor to project techniques to an IT project so that the project is challenges—poor project planning—by developing and completed on time, within budget, and fulfilling its implementing a new project approval process.) objectives. Project managers are the people trained Historical State Project Management with the skill set to keep projects on track and address Structure. In the past, the state has primarily relied issues when problems arise. on the individual departments to manage their own State Challenges Implementing IT Projects. IT projects—using state staff, contractors, or a Historically, the state has experienced considerable combination of the two. Under such a decentralized challenges implementing IT projects. Despite some approach, project managers are expected to develop successes, various high-profile state IT project failures, and implement project management processes and resulting in either project suspension or termination, practices, and create a team to support the project have received extensive legislative and media attention. implementation process. One of the primary challenges In other cases, projects were ultimately completed, but of utilizing a decentralized project management only after significant cost overruns and multiyear delays. approach can be the lack of experienced project Given recent failures, the Legislature and administration management staff at the departmental level. Because have expressed strong interest in reforming existing smaller departments generally undertake IT projects processes to enhance the likelihood of IT project infrequently, these departments typically acquire staff success. An internal review by CDT—the state’s central resources for project management on an as-needed, IT organization—revealed that the challenges commonly case-by-case basis. These project managers also may experienced by troubled projects largely connect back have a limited understanding of project management to deficiencies in the project management process, processes and practices, as they may be unable to which often fails to implement best practices to ensure draw from lessons learned in previous projects. There that projects remain on track and are completed are some exceptions to this lack of experienced successfully. (Please refer to our February 2017 report, project management staff at the departmental level. The 2017-18 Budget: The New IT Project Approval For instance, some larger departments, such as the www.lao.ca.gov 3 analysis full gutter AN LAO REPORT Franchise Tax Board, often simultaneously manage management, whereby individual departments assume multiple IT projects of high complexity and have full responsibility of managing their own projects. CDT demonstrated strong project management capacity. envisioned that the PMO would eventually manage Nevertheless, inadequate project management some of the state’s largest, most complex, high-cost expertise remains a challenge for most other projects. departments. Primary PMO Services. To achieve its objectives, The Statewide Project Management Office Is the PMO has two primary functions: (1) to provide Created. To address state deficiencies in project direct project management services to departments not management, in 2014-15 the Governor proposed capable of managing projects independently (referred to establishing a centralized PMO within CDT. The PMO as direct services for purposes of this report) and (2) to would create a team of skilled project management develop project management tools to indirectly support professionals, who would provide project management departments in need of some assistance (referred services to departments that are not equipped to to as indirect services for purposes of this report). handle this responsibility independently. Specifically, Departments may engage the PMO for direct services in 2014-15 the Governor requested two positions through three pathways: (1) the department undertaking and $208,000 (General Fund) to establish the initial the IT project (known as the sponsoring department) elements of the PMO. Subsequently, the Governor’s can request PMO services, (2) CDT can require PMO 2015-16 budget requested 11 additional positions engagement as a condition of project approval, and (13 positions total) and $1.5 million (special fund) (3) CDT can intervene to provide PMO services for to continue implementation activities—including at-risk projects under development. establishing standards, tools, and guidelines, and to Regardless of the pathway for direct services, PMO begin providing project management services to three offers three distinct service models based on the projects. The Legislature adopted the Governor’s sponsoring department’s existing capacity to manage proposals and in 2015-16 the PMO began managing IT projects. These service models, in ascending order state IT projects. of degree of PMO engagement, include: (1) short-term Objectives of the PMO. Specific objectives consulting services, whereby the PMO provides envisioned for the PMO in the Governor’s budget assistance on specific issues (such as testing) to proposal included: (1) successfully managing a wide support the sponsoring department’s existing project range of projects, including large, high-complexity manager; (2) providing the department a designated projects, for departments lacking capacity and/or CDT project manager, who would be absorbed into expertise; (2) recruiting and retaining a team of qualified, and work alongside the sponsoring department’s trained project management professionals to allocate project team to manage the project; and (3) providing to projects as needed; and (3) establishing and sharing a project manager and a full project management team best practices and project management standards to assume day-to-day management of the project in that better equip departments to manage their own lieu of the sponsoring department’s staff resources. projects successfully. The potential benefits of creating (Our analysis focuses on the extended support CDT a centralized project management office include: provides in the latter two service models. Consequently, making information sharing among project managers hereafter consulting services are excluded when we more feasible, allowing IT project-related training to refer to direct services.) Project managers from the be applied in a more uniform fashion, and allowing PMO remain employees of CDT but are loaned to the staff resources utilized for project management to sponsoring departments. Which service model is become experienced professionals who are capable selected, and the terms of the direct services to be of managing complex projects—all of which may provided by the PMO, are set in a memorandum of better position projects to succeed. While the PMO understanding between CDT and the sponsoring would attempt to address project management department. Following completion of their project work, deficiencies on projects engaged with the office, IT the project manager returns to the PMO to be allocated projects not engaged with the PMO would continue to another project in need of services. to operate under the decentralized model of project 4 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT The PMO also provides indirect project management under two different branches of the department, each services. The office develops and shares general project with its own chief deputy director. The idea behind management tools—including frameworks, templates, the firewall is that it would separate CDT’s oversight and trainings—that are available to all departments. and project management responsibilities, allowing it to These resources allow departments to build internal continue performing its oversight functions even with project management expertise and positions them to the PMO responsibilities. As the two functions are still more effectively manage projects independently. part of the department, this design does not eliminate— Organizational Firewall Between CDT’s Oversight but rather mitigates—possible conflicts of interest. and Project Management Roles Is Created. CDT Current PMO Project Portfolio. Although originally has oversight responsibilities over IT projects as part approved by the Legislature to provide direct project of its control agency functions—chiefly to evaluate managements services to three projects, CDT the management of IT projects and make associated determined that the PMO worked well enough to recommendations. Establishing the PMO created expand direct services to three additional projects the possibility that CDT could be evaluating the in 2016. Four of the six projects range in estimated effectiveness of its own project management, thereby project cost from $4 million to $7 million. The smallest creating a potential conflict of interest between CDT’s project is estimated to cost $165,000, while the largest current oversight responsibilities and its proposed is estimated to cost $69.1 million. Figure 2 (see next project management responsibilities. As shown in page) identifies key characteristics of the six projects Figure 1, the Governor’s proposal included a strategy currently receiving direct PMO services. for preventing potential conflicts by creating a “firewall” These six projects share common characteristics, between CDT’s oversight and project management particularly related to the circumstances of PMO responsibilities. The proposed, and ultimately engagement. Thus far, all projects have independently implemented, solution placed these two functions requested direct services (with the exception of one Figure 1 California Department of Technology’s Organizational Firewall Between Project Management and Oversight Rolesa Director of Technology Other Offices Policy Operations (Legal Services, Chief Deputy Director Chief Deputy Director External Affairs, IT Services) Statewide Technology Administration Office of Technology Enterprise Solutions Procurement Division Services Office of Information State Geographic Broadband/Digital Project Oversight Security Information Literacy Office of Professional Customer Delivery Development Project Management Office a The organization chart has been simplified to highlight only the aspects relevant for this report. IT = information technology. www.lao.ca.gov 5 analysis full gutter AN LAO REPORT Figure 2 Current Project Portfolio for Statewide Project Management Office (In Millions) Estimated Estimated Completion Project Sponsor Department Project Project Description Datea Cost Department of Conservation Well Statewide Tracking and Well management system to meet state June 2020 $69.1 Reporting (WellSTAR) and federal requirements Department of Fair California Civil Rights System Case management system for October 2017 6.5 Employment and Housing (CCRS) discrimination complaints Commission on Teacher Streamline and Strengthen the Data collection system for CTC June 2017b 6.5 Credentialing (CTC) Accreditation Process (SSAP) accreditation process California Workforce Cross-System Analytics and Data repository and analytics platform December 2018 4.2 Development Board Assessment for Learning and for workforce development activities Skills Attainment (CAAL-Skills) Department of Insurance Fraud Data Analytics System Analytics tools to identify fraud May 2018 4.1 (FDAS) Office of Traffic Safety Grant Electronic Management Automating paper-based grants January 2018 0.2 System (GEMS) application process a Estimated completion date as of summer 2017. b Actual project completion date. project that was encouraged by CDT to use PMO approach and build the reputation and capacity of services). CDT has not required direct PMO services the office gradually, with the intention of scaling up to as a strict condition of project approval or intervened larger, more complex projects once the office was more to provide services due to project underperformance, established. although CDT originally envisioned both of these PMO Funding Structure. Most of CDT is funded scenarios for engaging the PMO. Additionally, the office through the Technology Services Revolving Fund has yet to exercise the highest level of engagement, (TSRF), which collects fees paid by departments for whereby the PMO provides a full project management services provided by CDT, including those charged team to assume day-to-day management of the for PMO services. The PMO service rate charged to project in lieu of the sponsoring department’s staff. a sponsoring department is largely set based on the All six projects currently receive a designated PMO salary and benefits of the designated project manager, project manager, who is expected to integrate into while other PMO responsibilities are excluded from the the department’s project team for the duration of the service rates, including indirect PMO services (such as project. For the first six projects, the PMO selected the development of frameworks, methodologies, and projects likely to progress smoothly, preferring smaller, other tools) and various administrative responsibilities. less complex projects. This allowed the PMO to test The 2017-18 budget provides the PMO with the potential of the centralized project management 14 positions and $2.5 million (special fund). 6 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT ANALYSIS OF THE PMO TO DATE We recently evaluated the performance of the PMO In contrast, other departments requesting PMO to date to assess whether the office is meeting—or services claimed to have the expertise to manage on track to meet—its original objectives and to better projects independently but temporarily lacked available understand the strengths and limitations of the PMO, staff. These larger departments had established as implemented. Specifically, our evaluation focuses on IT project management operations, including their the direct, rather than consulting and indirect, services own departmental PMO, and were simultaneously offered by the PMO. For this work, we interviewed managing several projects of varying complexity. These key staff members of the six sponsoring departments departments turned to CDT’s PMO to provide project receiving direct PMO services, as well as executive management services on a new project when internal staff at CDT. Given that the PMO has only been directly IT staff were already fully dedicated to ongoing projects. involved with managing state IT projects for two years, Without adequate project management, the new project we acknowledge that the PMO is in the early stages may have experienced delays until resources were of establishing its capacity and processes as an office, secured, or the department would need to reassign especially as the PMO is intended to scale up to resources from other projects, potentially jeopardizing eventually manage larger, more complex projects. As a other project work, or contract for services. Under such result, this analysis has limitations that we acknowledge circumstances, departments found the PMO helpful more fully later. Below, we discuss our findings from the in addressing internal capacity deficiencies. These evaluation of the PMO as the office is operating to date. departments aim to have sufficient internal capacity in the future to manage their projects independently, but Departments Had Several would leverage the PMO should capacity constraints Motivations for Engaging PMO arise again. (We discuss later why this may change moving forward.) Departments Sought PMO Services Due to Lack Departments Identified Several Benefits of of Internal Expertise and/or Capacity. According PMO Over Other Alternatives. Departments largely to the departments we interviewed, the needs that prioritized successful project implementation when motivated department requests for PMO services considering their options for project management, and depended on the expertise and capacity of the viewed PMO services as one option among several— department’s internal IT operation and the complexity including using state resources or contracting—to of the project. We found that departments with less achieve project success. However, sponsoring established IT operations requested PMO services to departments provided several reasons for preferring address gaps in both expertise and capacity. Because PMO services: they take on projects so infrequently, most small departments, and some larger departments, have • Perceived Advantage Navigating CDT. Since limited opportunity and need to develop the internal PMO project managers are CDT employees, operations necessary to effectively complete complex departments believed that using direct services IT projects. For instance, a smaller department may from the PMO facilitates smoother navigation be capable of managing small scale IT projects, such of CDT processes, including initial project as hardware replacements or application updates, approval, procurement, approval of revised but may lack expertise beyond these routine projects. project plans, and project oversight. The project Hence, on the rare occasion when the department managers would be more familiar with CDT’s pursues a more significant IT project, they require processes and help departments manage these additional project management services to successfully processes successfully. If challenges were to complete the project. Many of the departments that arise, departments believed the PMO project engaged the PMO because of expertise and capacity managers could leverage their direct access to issues anticipate returning to the PMO for direct project CDT colleagues, as appropriate, to remedy issues management services in the future for similar reasons. more expeditiously. Those CDT employees may www.lao.ca.gov 7 analysis full gutter AN LAO REPORT be more understanding when interacting with a success or failure of state IT projects. Ultimately, CDT colleague (the project manager assigned to projects managed by the PMO are still the the project) than with a non-CDT project manager. responsibility of the sponsoring departments. • Administrative Ease. According to departments, Nevertheless, departments may perceive it requesting PMO services is less administratively valuable to engage CDT, particularly when they burdensome than the other available options. For can shift responsibility for project challenges to example, contracting for project management CDT. services requires completing the state procurement process, which may be lengthy PMO Was Positioned to Hire and result in project delays. Hiring for a new High Quality Project Managers temporary or permanent position may require The project managers hired by the PMO came from formal budget approval. Alternatively, reassigning a range of backgrounds, including within CDT, from internal resources is only a viable solution if other departments, and from the private sector. In adequate expertise already exists internally and recruiting, the PMO looked for candidates who were the reassignment does not result in delays on certified as project managers, had experience on other work. projects of different sizes, and maintained knowledge • Cost-Effectiveness. For the projects currently in a variety of project management-related disciplines, using direct PMO services, PMO rates generally including organizational change and various software have ranged from $15,000 to $18,000 per development models. Below, we outline the tools in month, depending on the complexity of the place that have positioned the PMO to recruit high project. Departments considered these rates quality project managers and discuss sponsoring more cost-effective than contracting for project departments’ experiences with the project managers to management services, indicating that PMO date. service rates were either competitive or lower than Several Factors Positioned PMO to Recruit contractors’ rates. As we discuss below, because Qualified Project Managers. When the PMO was these rates do not fully reflect CDT’s cost for initially proposed, we noted that recruiting and retaining operating the PMO, other fees paid to CDT qualified project managers would be critical to the subsidize the operation of the office. success of the PMO. We stated at the time that while • Cultivate In-State Expertise. Some the state would likely continue to have problems departments’ interest in the PMO was partially competing with the private sector for qualified IT staff, driven by an interest in building and retaining CDT may be better positioned than departments to project management expertise for the benefit of recruit and retain staff resources through the PMO the state. They viewed engaging PMO services because of the career pathway the office could offer. as a means to contribute to the state’s long-term We found that several factors enabled the PMO project management capacity, as the expertise to attract, and ultimately hire, experienced project and experience that the PMO project manager managers: developed through the project would remain in-state. These departments noted that, in • Career Advancement. According to the PMO, contrast, outside contractors would leave the many project manager candidates viewed the state after completing their contractual project PMO as an opportunity for career advancement management responsibilities and take with them by working on high-visibility, complex, and any added expertise. challenging projects which were not available at • Sharing Responsibility With CDT. One their previous position. For candidates from the department noted that its interest in PMO services private sector, the PMO offered an opportunity was partially motivated by an interest to share to manage projects that could affect the entire responsibility for any project with CDT. As the state of California, a scale and impact level state’s central IT organization, this department incomparable to most projects seen in the private believed CDT should be accountable for the sector. Depending on the department, candidates 8 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT within state government may have reached their project manager a strong asset, as they helped the top of their professional development, with mitigate project delays and were able to act as a limited opportunities to work on more complex IT neutral mediator when issues arose. For instance, when projects. For these candidates, the PMO provided projects involved collaboration across entities—such the possibility of more challenging and engaging as data sharing across departments or convening project work that would better leverage their skill teams within the department—and disagreements set and expertise. Although the PMO intended to arose over project changes or next steps, project start with smaller projects, the appeal of working managers attempted to resolve the conflict as a neutral in the future on state projects that were large in party. One department noted that when an unforeseen scale and intellectually challenging positioned the issue arose, the project manager leveraged his/her PMO to attract strong candidates. knowledge of the state procurement process to • Pay Differential for Complex Projects. Through contract with the necessary technical specialists in a an arrangement between CDT and the California short time frame. Thus far, there has been no turnover Department of Human Resources, CDT secured of project managers on any of the six projects receiving a 7.5 percent to 15 percent pay increase, or PMO services. differential, for project managers working on We note that the project manager selection process the most complex projects. While none of the may have played a role in the high department projects currently receiving PMO services qualifies satisfaction with PMO services. Because these were for the pay differential, the possibility of the pay the first projects the PMO undertook, after initially differential in the future allowed the PMO to be a screening candidates, the PMO worked collaboratively competitive employer. (We note that one project with most sponsoring departments to hire candidates manager initially qualified for the pay differential. considered a good match for the project needs and However, a reassessment of the project’s department culture. Departments had the opportunity complexity resulted in an adjustment down and a to review candidate resumes, interview candidates, and removal of the pay differential.) select their preferred candidate—who would then be • Job Security. For project managers already hired by the PMO. Through this process, departments in state government, the PMO was better were able to provide the PMO substantial input on positioned to provide job security than other project manager selection, which was likely critical in departments. Typically, IT projects are short-term high department satisfaction with their PMO resource. endeavors, whereby project managers would Progress of PMO Projects find themselves in need of employment after a project is completed. However, under the Projects Largely Progressing as Expected. PMO model, project managers are hired into As of June 2017, of the six projects receiving PMO permanent positions and are assured to receive services, three projects are under development, and a new project assignment after the previous one the other three projects are in the planning phase. Two concludes, allowing for continuous work and of the three projects under development are generally employment without having to apply for jobs in progressing as expected based on the approved other departments. project plans. The third project noted project delays largely due to external vendor challenges. Despite Departments Highly Satisfied With Project the delays, the department noted that they were Managers. All of the sponsoring departments that we pleased with the project manager’s proactive approach interviewed were highly satisfied with the qualifications to mitigating any further delays. One department and performance of their project manager. Project acknowledged their project manager as the main managers integrated with department project staff and reason for the project staying on schedule. There is not managed daily operations of the project. According to as much information available to fully assess the three departments, project managers were able to develop projects still in the planning phase. However, projects a working understanding of programmatic needs and engaged with the PMO during the planning phase may departmental culture. Several departments considered find their project manager provides critical support that www.lao.ca.gov 9 analysis full gutter AN LAO REPORT positions it for success. For example, one department services without growing the office too large and leaving attributed being farther along in the planning phase PMO staff underutilized during times of lesser demand than otherwise expected due to the work of the project for their services. manager. Alignment With Legislative Intent Perceived Conflict Between PMO Operated Largely as Legislature Intended. CDT’s Oversight and When the PMO was first proposed, CDT asserted that Project Management Roles Remained statutory authority for a PMO already existed, citing statute that broadly laid out the Director of Technology’s Some Departments Concerned About Adequacy authority to perform project management activities, of CDT Firewall. Despite CDT’s organizational firewall, but the statute was never modified to explicitly lay some sponsoring departments expressed some out the role and objectives for a statewide PMO. concern that a conflict of interest still existed. For Despite the fact that the PMO was not explicitly example, there was concern that the PMO project established in statute, we generally found the PMO manager would expose the department’s internal to be operating in line with legislative intent, based deliberative process to other CDT staff. Departments on the budget proposal approved by the Legislature. had concerns that such exposure would result in The PMO had already expanded direct services from stricter or more involved oversight than was warranted. three projects to six projects, and was positioned to These departments asked their CDT project manager potentially expand services to larger, more complex to refrain from sharing any project information, beyond projects. The office was meeting, if not surpassing, what was necessary, with other units within CDT, sponsoring departments’ expectations, both in including oversight. In effect, these departments regards to project manager qualifications and project attempted to strengthen the firewall between the progress under the CDT project manager. With the project managers and the rest of CDT to maintain exception of one project delayed due to external vendor control over the information given to CDT. challenges, the remaining five projects are progressing Additionally, when CDT suggests that a project seek as expected. Although the PMO encountered some project management services, sponsoring departments initial challenges—including limited PMO capacity and may perceive some pressure to use PMO services departments’ internal concerns about CDT oversight— over other options, such as contracting from a third we found the PMO to be largely meeting or working to party. Given the authority of CDT over project approval meet its original objectives. and oversight, departments may have concerns that not accepting PMO services, even when not required, PMO May Encounter Future Limitations would invite a different level of CDT oversight. Although our analysis thus far indicates a high level PMO Capacity Limitations of satisfaction with the PMO to date, there are some significant limitations to these findings, which we Unable to Fully Meet Demand for Services. The highlight below. PMO was unable to fulfill requests for direct services High Department Satisfaction May Be Difficult to from at least seven departments, largely because Maintain. Because a department’s ability to interview PMO staff was already engaged on other IT projects. and select its project manager was a critical factor in Although it is unclear whether these departments would high department satisfaction, department satisfaction have pursued direct PMO services should project with PMO services may differ moving forward. In managers have been available, it is an indicator of the future, because the PMO will be fully staffed interest in direct project management services through (barring any intermittent vacancies), new projects the PMO. We also note there is a tension between requesting PMO services will have limited options serving all interested departments and developing based on the project managers available at the time excess capacity. The PMO needs to continuously of request. Additionally, as previously mentioned, the balance the trade-offs between permanently expanding PMO intentionally started with smaller projects that the office to address requests for project management requested PMO services and were likely to progress 10 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT smoothly. Hence, the projects currently receiving PMO the costs associated with the salary and benefits for services are not fully representative of the projects project managers. However, to keep rates competitive the PMO will likely engage in the future. If the office with contractor rates and to increase initial take-up for continues to operate as today, we would anticipate services, CDT was incentivized to keep rates below that the PMO will engage a broader range of projects what was necessary for full cost recovery of the PMO. under varied circumstances, including engagement For instance, the service rates exclude key aspects as a requirement of project approval and on projects of the PMO’s responsibilities, including indirect PMO already underway considered at-risk. While we found services (such as the frameworks and methodologies that departments that invited PMO involvement through made available to all departments) and various request were satisfied with CDT’s service, it is unclear administrative responsibilities. As a result, CDT relies whether department satisfaction would remain high on surplus balances in the TSRF, which, as previously when CDT requires a department to use direct project discussed, collects fees paid by departments for CDT management services from the PMO. services, to fully cover the cost of operating the PMO. Future PMO Service Rates May Be Less In the future, CDT may need to increase service rates Competitive. Although the PMO was expected to fully to factor in the cost of indirect PMO services and office recover the cost of its operation through the collection administration to work towards full cost recovery, as of fees for PMO services paid by departments, CDT was originally envisioned. If direct service rates were indicated to us that the PMO has not been able to fully to increase, the PMO might be a less attractive project recover costs since the office was established. The management option for departments moving forward, PMO service rates are largely calculated to recover especially if the PMO were no longer able to compete with contractor rates. SIGNIFICANT RECENT DEVELOPMENTS Although the PMO has been operating largely policy and the other focused on operations. The as intended with a high level of satisfaction from reorganization collapsed the department’s functions departments, during our evaluation we learned of and responsibilities under one chief deputy director. two recent developments that significantly impact the Figure 3 (see next page) illustrates the new operation of the PMO: (1) a reorganization of CDT that organizational structure of CDT. eliminates the firewall between CDT’s oversight and Potential Conflict of Interest Between CDT’s project management roles and (2) a policy change that Oversight and Project Management Roles narrows the number of IT projects eligible for PMO Resurfaces. To address the potential conflict of services. Below, we describe the recent developments interest between the project management and and discuss the relevant potential implications they may oversight functions of CDT, the two responsibilities have on the PMO going forward. were previously placed under separate branches. However, the reorganization under new CDT REORGANIZATION OF CDT leadership consolidated the project management and oversight functions under the same office—the CDT Reorganized. In spring 2016, a new Office of Statewide Project Delivery—with both director assumed leadership of CDT and, shortly functions reporting to the same chief deputy director. thereafter, proposed an organizational realignment This effectively eliminated the firewall, making it so of the department. The reorganization sought to that the two responsibilities are in potential conflict. achieve organizational efficiencies across a variety Consequently, the current departmental structure may of CDT’s functions, not specific to the PMO. be thought to compromise CDT’s ability to maintain Formerly, the functions of CDT were divided between independent and objective oversight for projects two chief deputy directors, one that focused on managed by the PMO. www.lao.ca.gov 11 analysis full gutter AN LAO REPORT Figure 3 California Department of Technology’s Revised Organization Director of Technology Chief Deputy Director Other Offices Office of Statewide (Digital Innovation, Project Delivery Enterprise Architecture, Technology Services, Customer Engagement, Information Security, Administrative Services, Project Project Statewide Technology Professional Development, Approvals and Oversight Management Office Procurement Governmental Affairs) a The organization chart has been simplified to highlight only the aspects relevant for this report. CDT NARROWED ELIGIBILITY FOR approval or oversight. (Please refer to the nearby box for a detailed definition of reportable IT projects.) In DIRECT PMO SERVICES summer 2016, CDT instituted a new policy that makes Change in Project Eligibility for Direct PMO most reportable projects under the oversight of CDT— Services. State IT projects are either “reportable” or that is, generally the larger, more complex projects— “non-reportable.” Reportable projects are generally ineligible for PMO direct services. Accordingly, more complex and costly, and as such, they require combined with other aspects of the policy discussed approval and oversight from CDT. Non-reportable below, the new eligibility policy significantly limits the projects, on the other hand, can be independently types of projects directly served by the PMO. In fact, managed by sponsoring departments without CDT under the narrowed eligibility policy, a majority of the Reportable State Information Technology Projects Reportable projects generally face a higher level of scrutiny, as they are approved by and under the oversight of the California Department of Technology (CDT), whereas non-reportable projects are completely within the authority of sponsoring departments to manage. Projects that meet one or more of the following characteristics are considered reportable: • Estimated project cost exceeds departmental delegated cost threshold authority assigned by CDT. The delegated cost thresholds generally range from $200,000 to $2 million. • Projects for which the costs are not absorbable by the sponsoring department and therefore require an appropriation. • Projects that are specifically mandated by the Legislature. • Projects that meet previously imposed conditions for reportability set by CDT. Non-reportable projects either do not meet these criteria or are sponsored by state entities outside of CDT’s jurisdiction, such as Covered California. 12 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT previously eligible projects could no longer use direct consulting and indirect services—that may position project management services. This group of projects departments to successfully manage their own projects collectively represent about $3 billion, or the vast independently. CDT noted that departments primarily majority of the state’s IT project costs. (We note that, view its role as a control agency, and that departments despite the narrowed eligibility criteria for direct PMO with adequate resources would prefer to keep project services, all departments and projects remain eligible management internal, limiting their interactions with a for PMO consulting services, which provides short-term control agency and building their own long-term project project assistance.) management capacity. According to CDT, the narrowed Specifically, CDT’s new eligibility policy narrows eligibility policy addresses these concerns by allowing direct PMO services to (1) all non-reportable projects, centralized project management services, through (2) reportable projects only during the project approval the PMO, for smaller projects that are likely to lack process (known as the Project Approval Lifecycle internal capacity or expertise but otherwise relying on [PAL]), and (3) reportable projects that have completed a decentralized approach for project management for the PAL process but are determined by CDT to be most large, complex projects. “low-complexity,” and therefore receive very limited CDT’s decision to strictly limit direct PMO services oversight from CDT. Figure 4 shows whether or not in cases where projects are under the oversight of CDT the six projects currently engaged with the PMO would has the practical effect of reducing the potential for still qualify under the new eligibility requirements. Even a conflict of interest. CDT reported that departments among these smaller projects, only three of them would repeatedly raised concerns about CDT’s ability to meet the narrowed eligibility requirement for direct maintain independent oversight while providing PMO services. The other three projects, despite finding project management services on a project. As PMO services valuable, would be ineligible under the discussed previously, we heard similar concerns from narrowed eligibility policy. (We note that the current departments. projects receiving direct PMO services will continue to In the following sections, we discuss the practical do so despite the policy change.) implications of the narrowed eligibility policy, the office’s Finally, because PMO staff were already fully ability to continue meeting its original objectives, and dedicated to projects when the policy change went into the relevant legislative considerations under the new effect, the PMO has not yet engaged any new projects policy. under the narrowed eligibility policy. (The first project managed by the PMO was completed in summer 2017. However, the project Figure 4 manager has not been assigned to Eligibility of Six Currently Engaged Projects for a new project for direct services; PMO Services if Under New Eligibility Policya instead, the project manager is being Reportable IT Projects used for another internal effort.) Non-Reportable Low Higher Administration’s Reasons IT Projects Complexity Complexity for Narrowing Eligibility. In our discussions with CDT, the Eligible GEMS SSAP CAAL-Skills administration expressed an interest in continuing to build project Ineligible WellSTAR management capacity within CCRS FDAS individual departments as opposed a to maintaining a centralized PMO as Although the six projects managed by the PMO were engaged under the original eligibility policy, this figure shows that three projects would be ineligible for PMO services beyond the planning original envisioned. The administration stage because they are reportable and higher complexity. thereby expressed its preference for PMO = Project Management Office; IT = information technology; GEMS = Grant Electronic Management System; CAAL-Skills = Cross-System Analytics and Assessment for Learning a decentralized approach to project and Skills Attainment; SSAP = Streamline and Strengthen the Accreditation Process; management, whereby CDT provides WellSTAR = Well Statewide Tracking and Reporting; CCRS = California Civil Rights System; and FDAS = Fraud Data Analytics System. departments support—such as www.lao.ca.gov 13 analysis full gutter AN LAO REPORT Practical Implications of of services to include project implementation after Narrowed Eligibility Policy anticipating that the change would be disruptive. This suggests that mainly departments with projects likely to Reportable, Higher-Complexity Projects be non-reportable would engage PMO services during Unlikely to Engage PMO Services if Solely for the planning phase, because only these projects have Project Planning. As discussed above, reportable, the certainty of retaining the same project manager higher-complexity projects can only use direct through project completion under the narrowed PMO services during the planning phase. Generally, eligibility policy. departments have a sense of whether their projects Pathways for Engaging PMO Limited. As discussed are likely to be reportable prior to initiating project previously, the new eligibility policy significantly limits the planning. Whether a project is actually reportable projects eligible for PMO services. The new policy also and its complexity level is determined by CDT limits the pathways CDT can utilize to support projects through the planning process. (Refer to the nearby in need of project management services. CDT originally box for a detailed definition of “complexity” for state envisioned three pathways for engaging PMO services: IT projects.) Although reportable, higher-complexity (1) by department request; (2) as a CDT requirement projects are eligible for project management services for project approval; and (3) through CDT intervention during the planning phase, the challenges associated on at-risk projects, likely at the recommendation of with finding new project management services once CDT oversight. We note that the latter two pathways planning is complete may discourage these projects could be used by CDT as tools to intervene and ensure from seeking PMO services solely for the planning projects are managed appropriately. (Thus far, only the phase. Even though the PMO may help reportable, first pathway has been used.) The projects that would higher-complexity projects more efficiently navigate be eligible for direct PMO services under the new policy the project approval process, losing a project have little or no approval and oversight requirements manager after receiving project approval also means from CDT. In effect, this means CDT would seldom be losing the valuable insight and knowledge gained in a position to require PMO services as a condition of during planning—such as a better understanding of project approval or intervene based on issues identified programmatic needs or anticipated risks and mitigation through its oversight. Without these tools, CDT is unable strategies. One department currently receiving PMO to intervene with direct PMO services on the largest, services originally intended to use PMO services solely most complex projects that are potentially troubled or for project planning, but then extended the duration proven to be troubled. As a result, we anticipate most, if Complexity of State Information Technology (IT) Projects The California Department of Technology (CDT) determines the level of complexity for each state IT project after assessing both the programmatic and technical components of the project. The programmatic complexity takes into consideration factors such as the stability of the program needs, the clarity of project objectives, the political sensitivities, and the financial risk to the state. The technical complexity incorporates factors such as whether the software or hardware has proven effective through its use, the level of system integration, and the security requirements associated with the project. The complexity level enables CDT to determine the appropriate level of oversight to provide on the project. For instance, a project of the lowest complexity—which may rely on established technology solutions, have a clear business objectives and requirements, pose little financial risk to the state, and be routine in nature—would receive very limited oversight from CDT. In contrast, a higher-complexity project may be larger in scale, have higher visibility, face multiple contentious issues, and employ new technological solutions that must be integrated with multiple existing systems. These projects require much closer oversight from CDT. Generally, a CDT employee is embedded full time on a project and routinely reports on the project’s status. 14 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT not all, future projects to engage PMO services through projects in need of project management services would a request to CDT by the sponsoring department. This have to find support elsewhere. These projects would pathway depends on departments accurately assessing likely revert to the same project management options the adequacy of their internal IT project management available prior to the establishment of the PMO—such capacity, which is something departments have as contracting or reassigning staff internally—which struggled with historically. were inadequate in some cases and originally prompted the need for a statewide PMO. Issues With the Potential Challenges Recruiting and Retaining Narrowed Eligibility Policy Project Managers. According to the departments receiving PMO services, the current PMO project Unclear Whether Smaller Projects Stand to managers are highly qualified, experienced, and Benefit Most From PMO Services. Given that knowledgeable. However, it seems likely that the non-reportable projects tend to be less costly, smaller narrowed eligibility policy will create significant in scope, and lack the need for direct CDT oversight, recruitment and retention challenges. The narrowed we can infer that these projects are considered to eligibility policy eliminates two key factors that attracted be less risky and therefore less likely to need project project managers to the PMO: (1) the possibility of management services. Of the six projects the PMO working on complex, high-visibility, and professionally currently engages, three projects would remain challenging projects that could advance their careers eligible for PMO services under the new eligibility and (2) a favorable pay differential for working on policy, suggesting some need for services among projects that are more complex. The loss of these small projects. However, the need for PMO services incentives may make it difficult to retain the current on non-reportable or low-complexity projects may be project managers. Further, the less complex projects limited, suggesting that dedicating state resources and still eligible for direct PMO services may require funding to provide direct PMO services to these types less experienced and skilled project managers than of projects is likely not the best use of these resources. the current PMO team, given that the qualification Further, the potential cost savings to the state from requirements for working on smaller, less complex more effective project management on these smaller, projects will likely be lower. As a result, the state stands low-costing projects is limited. The state stands to to lose valuable project management expertise and realize more benefits by focusing on complex reportable capabilities formerly centralized at the PMO, especially projects that risk significant cost increases without as current project managers consider opportunities at proper project management. These are the projects other departments or outside of state government. which the state has historically struggled with and which the PMO was originally established to support. Misalignment With Legislative Intent. Given the new eligibility policy, the PMO is no longer operating Departments Have Lost a Project Management as originally envisioned. The PMO will continue to Option. In our analysis, we found an interest in provide consulting and indirect services (such as receiving direct project management services from the project management frameworks) for all projects PMO among departments that are no longer eligible. while bolstering the successful management and For instance, three projects currently receiving PMO implementation of smaller projects through direct services are under CDT oversight and would not be PMO services. However, the original motivation eligible for services under the new policy. Additionally, behind the PMO focused on serving a wide range of several of the unfulfilled requests noted previously projects, including those most challenging to the state would not be eligible for direct PMO services under the historically. Now, the state’s highest-visibility, largest, new policy. Finally, the PMO already declined a request most complex projects may continue to face the same for services from a project that is no longer eligible project management challenges that first motivated under the narrowed criteria. This indicates that some the creation of the PMO. By only supporting smaller departments prefer direct PMO services over other projects that lack expertise or capacity, the initial project management options—such as consultants— problems that prompted the creation of the PMO may but have lost the PMO as an option for direct project go unaddressed. management under this new policy. Newly ineligible www.lao.ca.gov 15 analysis full gutter AN LAO REPORT LAO FINDINGS AND RECOMMENDATIONS Prior to the recent developments, we found the the office and the pathways through which projects PMO to be operating as originally envisioned by the engage the PMO. A potential objective of the PMO is to Legislature, with the office providing critical services to manage IT projects for departments that lack capacity departments and in a position to potentially expand to and/or expertise, irrespective of a project’s oversight larger, more complex projects. Departments receiving status. In other words, all IT projects would be eligible direct PMO services were largely satisfied and noted to receive direct PMO services, including the largest the significant contributions of their CDT project and most complex projects. Codifying the objectives of manager in managing their projects efficiently and the office would provide CDT with legislative guidance effectively. On balance, we found that the PMO was for the operation of the office and ensure that the a valuable asset to departments in need of project goals and functions of the office remain aligned with management services. However, under the recent legislative intent. policy change, the PMO will now primarily serve smaller, Establish Strong Firewall Between low-complexity projects, which is not in line with the CDT’s Oversight and Project Management Legislature’s intent when it established the office. In light Responsibilities. We recommend the Legislature of our positive findings of the PMO’s operation under require in statute that CDT maintain an organizational the original eligibility policy, we make several findings firewall between its project management and oversight and recommendations that assert the original intent of responsibilities—similar to the firewall that existed prior the PMO and position the office to help state IT projects to the most recent reorganization. Requiring a firewall succeed. would help address and mitigate potential conflicts There Is a Role for Both Centralized and between the two responsibilities. Decentralized Approaches to Project Management. Clarify Intent That PMO Act Independently. As we have pointed out, CDT sees a continued role for Concerns raised by CDT and departments signal the decentralized approach to project management, that the firewall between CDT’s oversight and project and has used this to partially explain the recent change management responsibilities, even prior to the recent in the eligibility criteria for direct PMO services. We reorganization, could be further strengthened to allow agree that for some departments this approach makes the PMO to function in a way that relieves departments’ sense. However, we find that there can be a role for concerns about a potential conflict of interest between both a centralized and decentralized approach to CDT’s project management and oversight functions. project management, and that there is no need to To address this concern, we recommend that the only have one or the other model available for state Legislature express intent in statute that it expects departments. the PMO to operate independently and serve in Reassert Original Objectives of PMO. The the best interests of the projects. This would ease recent changes to the PMO have caused the office to sponsoring departments’ concerns of inappropriate diverge from its original intent to provide direct project and unnecessary information sharing between the management services to the most complex IT projects PMO and CDT oversight and cultivate an open and to engage projects that may be or already are pathway of communication between the PMO and challenged due to project management deficiencies. sponsoring departments. This recommendation also We recommend the Legislature reassert its original would allow the department to continue to provide objectives for the PMO in statute. The Legislature could project managers with the necessary information to establish the types of projects under the purview of successfully manage a project. 16 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT CONCLUSION The Legislature approved the establishment of the meet the Legislature’s original expectations. Our PMO with expectations that the office would meet recommendations enable the Legislature to reassert a critical need in the state for project management the original intent of the PMO to centralize project expertise and help the state avoid the high-profile management services and improve the state’s success IT failures of the past. We found that the PMO was in implementing IT projects by equipping departments showing success towards meeting the original with the necessary expertise and capacity. Additionally, legislative intent by first establishing the internal we note that even with the continuation of the PMO processes and capacity to ultimately scale up to as originally envisioned, there is still room to continue larger, more complex projects. However, given recent the decentralized approach to project management for developments, we have concerns about the path departments with the capacity to do so. forward for the PMO and its ability to succeed and www.lao.ca.gov 17 analysis full gutter AN LAO REPORT LAO PUBLICATIONS This report was prepared by Lourdes Morales and Amy Li, and reviewed by Ginni Bella Navarre. The Legislative Analyst’s Office (LAO) is a nonpartisan office that provides fiscal and policy information and advice to the Legislature. To request publications call (916) 445-4656. This report and others, as well as an e-mail subscription service, are available on the LAO’s website at www.lao.ca.gov. The LAO is located at 925 L Street, Suite 1000, Sacramento, CA 95814. 18 LEGISLATIVE ANALYST’S OFFICE