Recommendations

California High‑Speed Rail Authority Its Flawed Decision Making and Poor Contract Management Have Contributed to Billions in Cost Overruns and Delays in the System’s Construction November 2018 REPORT 2018-108 CALIFORNIA STATE AUDITOR 621 Capitol Mall, Suite 1200 | Sacramento | CA | 95814 916.445.0255 | TTY 916.445.0033 For complaints of state employee misconduct, contact us through the Whistleblower Hotline: 1.800.952.5665 Don’t want to miss any of our reports? Subscribe to our email list at auditor.ca.gov For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255 This report is also available online at www.auditor.ca.gov | Alternate format reports available upon request | Permission is granted to reproduce reports Elaine M. Howle State Auditor November 15, 2018 2018‑108 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the California State Auditor presents this audit report regarding the California High-Speed Rail Authority’s (Authority) contracting and cost control practices. This report concludes that the Authority’s flawed decision making regarding the start of high-speed rail system construction in the Central Valley and its ongoing poor contract management for a wide range of high-value contracts have contributed to billions of dollars in cost overruns for completing the system. The Authority began construction in the Central Valley in October 2013 despite being aware of the risks associated with beginning construction early—the fact that the Authority had not acquired sufficient land for building, had not determined how it would relocate utility systems, and had not obtained agreements with external stakeholders. These unmitigated risks have contributed to $600 million in costs overruns thus far for the three active Central Valley construction projects, with another $1.6 billion in additional costs needed to complete the projects. The Authority has cited the terms of a 2010 federal grant—which originally required construction to be complete by 2017—as the primary factor in its decision to begin construction when it did. However, we determined that even with a grant deadline extension until December 2022, the Authority could miss the new deadline unless Central Valley construction progresses twice as fast as it has to date. Missing the deadline could expose the State to the risk of having to pay back as much as $3.5 billion in federal funds. The Authority has partially offset Central Valley cost overruns, as well as those projected elsewhere in the system, by planning to share existing rail infrastructure where possible. However, the Authority acknowledges that it has identified every feasible option to do so and therefore cannot continue to use this approach to offset costs. Moreover, despite its challenging financial situation, we determined that the Authority has failed to implement sound contract management practices. As a result, it cannot demonstrate that the large amounts it has spent on its contracts have been necessary or appropriate. Respectfully submitted, ELAINE M. HOWLE, CPA California State Auditor 621 Capitol Mall, Suite 1200 | Sacramento, CA 95814 | 916.445.0255 | 916.327.0019 fax | www.auditor.ca.gov iv California State Auditor Report 2018-108 November 2018 Selected Abbreviations Used in This Report CMSU Contract Management Support Unit DBE Disadvantaged Business Enterprises DGS Department of General Services DVBE Disabled Veteran Business Enterprise GAO Government Accountability Office MTC Metropolitan Transportation Commission PG&E Pacific Gas and Electric RDP Rail delivery partner California State Auditor Report 2018-108 v November 2018 Contents Summary 1 Introduction 7 Chapter 1 The Authority’s Decision to Begin Construction Before Completing Proper Planning Led to Cost Overruns and Delays 17 Recommendations 36 Chapter 2 The Authority Has Not Successfully Enforced the Policies It Adopted to Address Ongoing Deficiencies With Its Contract Management 37 Recommendations 58 Chapter 3 The Authority Can Improve the Quality and Transparency of Its Monitoring and Reporting For Key Goals 61 Recommendations 68 Appendix Scope and Methodology 71 Response to the Audit California High‑Speed Rail Authority 73 California State Auditor’s Comments on the Response From the California High‑Speed Rail Authority 85 vi California State Auditor Report 2018-108 November 2018 California State Auditor Report 2018-108 1 November 2018 Summary Audit Highlights . . . Results in Brief Our audit of the California High‑Speed Rail Authority and its contracting and cost Conceived as the nation’s first bullet train, the California high-speed control practices highlighted the following: rail system promises to transform how Californians travel across » Although the Authority has secured and the State. However, the California High-Speed Rail Authority identified funding of over $28 billion that (Authority)—the state agency responsible for planning, building, it expects will be sufficient to complete and operating the system—faces serious challenges. Although the initial segments, that funding will not Legislature created the Authority in 1996, voters did not approve be enough to connect those segments or major funding until 2008, when they authorized $9.95 billion in finish the rest of the system—estimated general obligation bonds, $7.5 billion of which is for the system’s to cost over $77 billion. planning and construction. Two years later, the Authority received $2.6 billion in funding through the American Recovery » It has incrementally modified its plans for and Reinvestment Act (Recovery Act) to begin planning and a fully dedicated high‑speed rail system construction. In 2011 it received an additional $929 million in since 2012 and now intends to share— federal grant funding, bringing total federal support to $3.5 billion. blend—existing transit infrastructure The Authority receives 25 percent of the revenues from the State’s wherever feasible. Although blending is cap-and-trade program, resulting in $1.7 billion as of December 2017. less costly, it subjects high‑speed trains In total, the Authority has secured $12.7 billion in funding and to lower speed limits and may require identified up to $15.6 billion in possible future funding. The Authority sharing time on the tracks with other expects this funding will be sufficient to complete initial segments rail operators. of the system between Madera and north of Bakersfield and between » The fact that it has now exhausted San Francisco and Gilroy, but not to connect those segments or finish all feasible options to use existing the rest of the system between San Francisco and Los Angeles. Its infrastructure raises concerns about its most recent cost estimate for the larger system, which it presented ability to mitigate future cost increases. in its 2018 business plan, is $77.3 billion. » The risk of additional cost increases is high. Costs to date have been significantly Since 2012 the Authority has incrementally modified its plans for greater than originally projected because a fully dedicated high-speed rail system. Instead, it now intends the Authority moved forward before it to share existing transit infrastructure wherever feasible, an completed many critical tasks such as approach known as blending that has helped to offset rising costs purchasing land, planning how to relocate in the system. It currently plans to blend with local rail service on utility systems, or obtaining agreements the San Francisco Peninsula and in Los Angeles, as well as to share with external stakeholders. a freight corridor between San Jose and Gilroy. Although less costly than the dedicated approach, blending also subjects high-speed • This risk contributed to $600 million in trains to lower speed limits and sometimes requires sharing time on changes to construction contracts. the tracks with other rail operators. The extent to which blending • The Authority estimates that finishing will negatively affect rail service will not be known until a private the construction that is currently sector operator, which will ultimately run the system for the underway will require another Authority, makes service decisions, such as how fast and frequently $1.6 billion. to operate the trains. The fact that the Authority has now exhausted all feasible options to use existing infrastructure raises concerns • If the Authority does not complete about its ability to mitigate future cost increases. construction by the federal government’s December 2022 The Authority’s spending to date and future projections suggest deadline, it may need to repay that the risk of such additional cost increases is high. Costs for the $3.5 billion. three current construction projects in the Central Valley have been significantly greater than the Authority originally projected, in large continued on next page . . . 2 California State Auditor Report 2018-108 November 2018 » It needs to improve its contract part because the Authority did not complete many critical planning management to control soaring costs— tasks before moving forward with construction. Although the it currently has 56 contract managers Authority has asserted that the early start was necessary to comply throughout its organization, but these with the requirements for the system’s federal grant funding, it was individuals generally do not serve in aware that beginning construction without completing sufficient contract management roles full time. planning would expose the construction projects to a number of Moreover, it has placed portions of its risks it had not addressed. The risks associated with beginning oversight of large contracts into the construction early—the fact that the Authority had not acquired hands of outside consultants. sufficient land for building, had not determined how it would relocate utility systems, and had not obtained agreements with • In reviewing nine planning, external stakeholders, including impacted local governments and engineering, and consulting other railroad operators—developed into costly problems. These contracts, few contract managers risks have contributed to more than $600 million in changes to could provide evidence of reviewing construction contracts to pay for work for which the Authority had each monthly invoice for accuracy, not sufficiently planned or budgeted. none maintained tracking logs of deliverables, and most were unable Despite being aware of these risks, the Authority did not account to demonstrate how they ensured the for them in its project cost estimates until this year. It now forecasts quantity and quality of the work for that finishing the construction that is currently underway will which the Authority paid. require still another $1.6 billion in contract changes. In addition, » Although it has estimated the it estimates that it will need to push completion dates back as far environmental impacts of its current as March 2022—close to the federal government’s December 2022 construction, it has not comprehensively grant deadline. If the Authority does not complete the construction evaluated its performance against by this deadline, it may need to repay $3.5 billion in federal those estimates. funding, $2.6 billion of which it reports it has already spent. To meet the current schedule, the Authority will need to ensure that construction proceeds twice as fast as it has thus far. Meeting this schedule, which the Authority acknowledges is aggressive, will be possible only if it effectively monitors and mitigates risks—tasks it has performed inconsistently to date. Moreover, looking beyond the Central Valley, the Authority’s precarious funding situation means it cannot repeat past mistakes. In addition, the Authority will need to do more to control the soaring costs of its contracts by improving its contract management. After the Authority conducted two internal audits in 2015 and 2016 that identified significant deficiencies with its contract management practices, it established a contract administration organization in 2016, which included the Contract Management Support Unit (CMSU). This unit then oversaw the development of revised policies and procedures that emphasize the specific processes contract managers must perform and document. The Authority also tasked CMSU to monitor compliance with the policies. However, the potential effectiveness of the policies has been limited by the Authority’s contract management structure. The Authority has 56 contract managers throughout its organization, but these individuals generally do not serve in contract management roles full-time. Moreover, the Authority has in essence placed portions of its oversight of large California State Auditor Report 2018-108 3 November 2018 contracts into the hands of outside consultants, for whom the State’s best interests may not be the highest priority. In addition, CMSU—which is staffed by consultants rather than Authority employees—has performed only weak and inconsistent oversight. Likely as a consequence of these organizational weaknesses, when we reviewed nine planning, engineering, and consulting contracts, with a combined value of $1.3 billion, we noted significant problems with the Authority’s adherence to its requirements related to invoice review, deliverables monitoring, and change management. For example, Authority procedures require a systematic approach to ensure that contractors bill only appropriate and allowable costs. Although Authority contract managers asserted through standardized checklists that they had complied with those procedures by reviewing each monthly invoice for accuracy, few could provide evidence of those reviews. We found similar problems when we reviewed the Authority’s monitoring of deliverables—the services or work products for which it contracted. To document the timeliness and quality of a contract’s deliverables, the Authority requires each contract manager to maintain a tracking log of those deliverables and to provide the contractor with written notices of acceptance. Nonetheless, none of the contract managers for the nine contracts we reviewed maintained tracking logs. Further, only two contract managers could demonstrate any formal documentation for the acceptance of deliverables, and we have concerns regarding the timeliness with which these two contract managers evaluated and accepted deliverables. Moreover, for nearly all the contracts we reviewed, the only documented information about the timeliness and status of deliverables came from the contractors themselves, leaving us unable to determine how the Authority ensured it received the quantity and quality of work for which it paid. Because of the lack of documentation, we were also generally unable to determine how the Authority identified and resolved problems with deliverables. Without the contract management documentation its policies require, the Authority cannot demonstrate that the hundreds of millions of dollars it has spent to date on the contracts we reviewed has been necessary or appropriate. We also found that the Authority often amended its contracts to add time or additional funds and that when doing so, it relied on the contractors’ own estimates and projections of the associated costs and delays. The Authority designed its contract management procedures, as well as the related tracking requirements, to ensure that it identifies the need for contract changes in a timely manner and that it appropriately ensures the justification of those changes before adopting them as amendments. However, we found little documentation demonstrating whether or how the Authority 4 California State Auditor Report 2018-108 November 2018 independently evaluated the validity and size of the amendments to contracts we reviewed. In some instances, we noted that the Authority approved the amendments based wholly on the information the contractors reported to it. Construction of the high-speed rail system is not only a major undertaking in terms of its costs, but it also affects the State’s environment. Although the Authority is aware that it needs to manage the environmental effects of construction, we identified ways it could improve its monitoring and measurement of these impacts. For example, the Authority intends for the system to be a model for future rail infrastructure, but it has not sufficiently identified key objectives in its sustainability policy to ensure that its active construction projects follow sustainable practices. Further, an expert we retained determined that although the Authority appropriately estimated the environmental impacts of its current construction before beginning work, it has not comprehensively evaluated its performance against those estimates. Summary of Key Recommendations Before executing its next construction contract, the Authority should establish formal prerequisites for beginning construction to prevent avoidable cost overruns and project delays. At a minimum, these prerequisites should identify specific benchmarks related to property acquisition, utility agreements and relocations, and agreements with external stakeholders, including impacted local governments and other railroad operators. To enable policymakers and the public to track the Authority’s progress toward meeting the Recovery Act deadline in 2022, the Authority should begin providing quarterly updates to the Legislature detailing the progress of Central Valley construction by January 2019. To improve its contract management, increase accountability, and demonstrate that the significant amounts it pays for contracted services are justified, the Authority should take the following steps by May 2019: • Prioritize contract management efforts by establishing a process for hiring and assigning full-time, experienced contract managers. • Require CMSU to establish a schedule to monitor contract manager compliance, and help ensure the unit’s integrity by staffing it with full-time contract managers who are state employees. California State Auditor Report 2018-108 5 November 2018 • Hold contract managers accountable for performing the duties that the Authority’s policies assign to them. The Authority should require and review documentation of the contract managers’ compliance with these policies and related procedures. To help ensure that it meets its sustainability goals, the Authority should comprehensively compare the environmental impact of its construction to its baseline estimates on a quarterly basis by May 2019. Agency Comments The Authority agreed with our recommendations and identified actions it is taking or planning to take to implement them. 6 California State Auditor Report 2018-108 November 2018 California State Auditor Report 2018-108 7 November 2018 INTRODUCTION Background California considered developing a high-speed rail network for many years; as early as 1981, the State evaluated working with Japanese partners to construct a high-speed rail line in Southern California. However, planning did not begin in earnest until the mid-1990s. In 1996, the California Intercity High-Speed Rail Commission, which had investigated whether a high-speed train system could be possible in California, issued a report concluding that high-speed rail in the State was feasible. Following that report, the Legislature formed the California High-Speed Rail Authority (Authority), which is responsible for planning, building, and operating high-speed, intercity passenger rail service in the State. If completed, the high-speed rail system promises to transform how people travel in California and would be the first bullet train in the nation. A part of the California State Transportation Agency, the Authority is overseen by board of directors (board). The Governor appoints five of the board’s voting members, and the Legislature selects the other four, along with appointing one senator and one assemblymember to serve as nonvoting members. The board supervises the Authority’s employees, who numbered more than 190 as of June 2018. In addition, since 2006 a consulting firm has helped manage the high-speed rail project by acting in a role that the Authority labels rail delivery partner (RDP consultants). As of June 2018, the RDP consultants had 485 staff working on the project. Figure 1 on the following page summarizes key events in the Authority’s history, as well as projected completion dates for portions of the eventual rail system. Funding The Authority has secured a total of $12.7 billion in funding and has identified an additional $15.6 billion in possible future funding for the high-speed rail system. Although the Authority completed some preliminary planning tasks in the years following its creation, it did not have a dedicated revenue stream until November 2008, when voters approved Proposition 1A, which provided funding for the system. Also known as the Safe, Reliable High-Speed Passenger Train Bond Act for the 21st Century, Proposition 1A allowed for the issuance of $9.95 billion in state general obligation bonds, $7.5 billion of which is for the system’s planning and construction. Two years later, the State secured federal funding for the project through the American Recovery and Reinvestment Act (Recovery Act), which provided $2.6 billion through a matching grant. 8 California State Auditor Report 2018-108 November 2018 Figure 1 Timeline of High-Speed Rail Development 11999966 The Authority is formed by Senate Bill 1420, the High-Speed Rail Act. 22000000 The Authority releases its first business plan. 22000088 Proposition 1A passes, authorizing $9.95 billion in general obligation bonds. 22001100 The Authority is awarded $2.6 billion in a federal American Recovery and Reinvestment Act grant. 2011 The Authority’s 2012 business plan introduces blending between San Francisco and San Jose, and it plans to have initial 22001122 construction connect the Central Valley and Los Angeles. 22001133 The Authority executes its first construction contract for work in the Central Valley. 22001155 Central Valley construction officially breaks ground. 22001166 The Authority’s 2016 business plan focuses on finishing the Silicon Valley to Central Valley line (Valley-to-Valley segment) in Northern California first. 22001188 The Authority’s business plan shows construction overruns and other cost increases, which raise the overall system cost estimate to $77.3 billion. 2020 Completion date for Central Valley construction based on current contracts. March 2022: New expected completion date for Central Valley construction based on projected contract changes. 2022 December 2022: Federal Recovery Act deadline for Central Valley construction. 2027 Potential start of high-speed rail operations between Madera and Bakersfield and, separately, between San Francisco and Gilroy. 2029 Planned completion of Valley-to-Valley segment with service between San Francisco and Bakersfield. 2033 Assumed completion of Phase 1 system. Source: State law, state election records, federal grant agreements, and the Authority’s business plans, contracts, and press releases. California State Auditor Report 2018-108 9 November 2018 In 2011, the State received an additional $929 million in federal grant funding, bringing the total federal support to $3.5 billion. The Authority also receives a continuous appropriation of 25 percent of revenues from the State’s Greenhouse Gas Reduction Fund, which is funded by the State’s cap-and-trade program. As of December 2017, this funding stream had provided the Authority $1.7 billion, and the Authority projects it will receive between $4 billion and $4.5 billion in future revenues from the fund through 2030. The Authority also projects receiving an additional $3.9 billion to $11.1 billion if it is able to use federal loan programs or public-private partnerships to borrow against future cap-and-trade revenue. The Authority presented its most recent cost estimate for the system—$77.3 billion—in its 2018 business plan. The Authority stated in the 2018 business plan that it would continue to pursue all possible options for funding the project, including additional federal grants and private sector partnerships. However, these funding sources have not yet materialized. The Authority is also exploring additional funding scenarios with local governments in the cities and counties where it plans to build stations. In recognition of the expected economic impact of the new rail stations, the local governments that partner with the Authority would use future property tax revenues to help support the development of such stations within their jurisdictions. Although recent state legislation has expanded these types of funding options and the Authority has worked with cities to evaluate their feasibility, it is still in the early stages of this planning. Similarly, the Authority has researched the prospect of receiving advertising and station parking revenue in the future. System Planning and Construction According to its 2000 business plan, the California high-speed rail system was originally conceived as a stand-alone, dedicated system spanning over 700 miles and connecting some of the State’s largest cities, including San Diego, Los Angeles, San Francisco, and Sacramento. Early plans for the system also called for stations throughout the Central Valley, including in Bakersfield and Fresno. Since that initial vision, the Authority’s plans have changed dramatically. Most notably, in its 2012 revised business plan, the Authority introduced the concept of blending—the practice of sharing existing infrastructure with other rail operators instead of constructing dedicated infrastructure for high-speed trains— which partially offset the system’s rising cost estimates. Although previous business plans had considered opportunities to integrate the system with other railways, the 2012 plan was the first time the Authority formally introduced blending into the system by deciding to share the corridor between San Francisco and San Jose with an 10 California State Auditor Report 2018-108 November 2018 existing regional carrier, Caltrain. Since then, the Authority has incrementally adopted blended options elsewhere in the system, including between Burbank and Los Angeles and—as it announced in its 2018 business plan—between San Jose and Gilroy. Figure 2 details how planned blending has expanded across portions of the system over time. We further discuss the projected cost savings associated with blending, as well as the service implications, in Chapter 1 of this report. In the 2012 business plan, the Authority also announced that it would use a segmented approach to building the system, in part due to the lack of sufficient funding for the full system. The segmented plan initially called for the Authority to construct a segment between Merced and the Los Angeles basin on which it would operate high-speed service before completing the rest of the system. However, in its 2016 business plan, the Authority changed course and stated that it planned to build first between San Jose and an interim station north of Bakersfield, an approach the Authority has named the Silicon Valley to Central Valley Line (Valley-to-Valley segment). In 2018, the Authority announced that it would expand the planned Valley-to-Valley segment to San Francisco in the north and downtown Bakersfield in the south; previously, the southern terminus had been Shafter, a small farming community north of Bakersfield. However, the Authority’s 2018 business plan acknowledges that although it has the funds necessary to complete work in the Central Valley between Madera and Bakersfield, as well as between San Francisco and Gilroy, the funds will not be sufficient to complete the tunnels necessary to connect those two lines or extend the system south to Los Angeles. Figure 3 on page 12 illustrates the Authority’s current and projected funding as compared to the estimated cost of different segments. Construction of the system is now underway in the Central Valley. Current construction is focused on developing the initial infrastructure, such as bridges and viaducts. Future construction will lay the physical track and will install other needed support services and maintenance facilities. As Figure 4 on page 13 shows, the Authority has divided its current construction into three projects, which this report refers to as Project 1, Project 2/3, and Project 4. The Authority has entered into contracts with different construction firms for the delivery of each project. These construction contracts represent $3.1 billion of the $5.6 billion in contracts that the Authority currently oversees. Chapter 1 of this report includes our review of each construction project, including its cost and status. California State Auditor Report 2018-108 11 November 2018 Figure 2 The Authority’s System Plans for Phase 1 Have Evolved Over Time Sacramento 2000 Sacramento 2012 San San Francisco Francisco MillbraeSan Francisco International Airport (SFO) MillbraeSFO San Jose San Jose Merced Merced Gilroy Madera Gilroy Madera Fresno Fresno Introduced blending between San Francisco and San Jose Kings/Tulare Kings/Tulare Bakersfield Bakersfield Palmdale Burbank Burbank Los Angeles Union Station Los Angeles Union Station San Diego San Diego Sacramento 2016 Sacramento 2018 San San Francisco Francisco MillbraeSFO MillbraeSFO San Jose San Jose Merced Merced Gilroy Madera Gilroy Madera Fresno Fresno Extended blending Kings/Tulare south to Gilroy Kings/Tulare Bakersfield Bakersfield Palmdale Palmdale Burbank Burbank Los Angeles Union Station Los Angeles Union Station Added blending between Burbank and Los Angeles San Diego San Diego Dedicated high-speed rail infrastructure Change to previous business plan Blended (shared) infrastructure Source: The Authority’s published business plans, budgets, and planning documents. Note 1: The Authority has not consistently planned to operate service between Los Angeles and Anaheim. As recently as 2014, the Authority did not plan to operate trains on this segment. The Authority now plans to operate high‑speed trains on this segment by sharing track with Metrolink. However, because these plans do not represent a shift from dedicated to blended infrastructure over time, this segment is not included in the above analysis. Note 2: The Authority has consistently stated that it intends to eventually complete Phase 2, which will extend the system to San Diego (via the Inland Empire) and Sacramento after it finishes Phase 1 between San Francisco and Los Angeles. The maps above show Phase 1 only. 12 California State Auditor Report 2018-108 November 2018 Figure 3 The Authority Has Secured Funding to Finish Construction in the Central Valley but Not the Rest of the System $80 70 60 50 40 30 20 10 0 Funding Segment Costs )snoilliB ni( sralloD PHASE 1 TOTAL COST: $77.3 billion Remaining cost for Phase 1 (San Francisco to Anaheim): $47.8 billion VALLEY-TO-VALLEY TOTAL COST: $29.5 billion Financing of cap-and-trade revenue through 2050: UNSECURED Remaining cost for up to $11.1 billion FUTURE FUNDING: Valley-to-Valley segment up to $15.6 billion* (San Francisco to Bakersfield): $18.9 billion Future cap-and-trade revenue: up to $4.5 billion SECURED FUNDING: $12.7 billion Proposition 1A bonds: Cap-and-trade revenue: $7.5 billion $1.7 billion Central Valley cost: $10.6 billion Federal grants: $3.5 billion Source: The Authority’s 2018 business plan. * The Authority presents its unsecured funding in ranges. This graph uses the high end of its estimates; the Authority’s low‑end estimates of unsecured future funding is $7.9 billion. California State Auditor Report 2018-108 13 November 2018 Figure 4 Current Construction Projects Are Between Madera and Bakersfield Madera FRESNO COUNTY PROJECT 1 Fresno South of Fresno PROJECT 2/3 KINGS COUNTY TULARE COUNTY North of Kings County Line PROJECT 4 Shafter (North of Bakersfield) Source: The Authority’s maps and construction contracts. Past Audits of the Authority’s Contract Management and Oversight The Authority has been aware of shortcomings in its processes for managing its wide range of contracts for at least the past three years. It published internal audits in 2015 and 2016 that identified significant deficiencies in its contract management and oversight, including its failure to implement necessary policies and procedures, its lack of documentation of contract management activities, and its failure to establish sufficient oversight structures to ensure effective 14 California State Auditor Report 2018-108 November 2018 contract management. In February 2015, the Authority’s Audits Office completed the first of these audits to determine whether the Authority was managing contracts in accordance with state and Authority expectations. The Authority conducted a follow-up audit in May 2016 with the same focus. The 2015 audit emphasized that a structured contract management process is necessary to define responsibilities for administering contracts and for monitoring and evaluating contractors’ performance. However, the audit found that the Authority’s contract managers did not always receive proper guidance and often lacked the technical expertise to thoroughly manage contracts. The audit also identified numerous instances in which poor communication between the RDP consultants and the Authority meant that the Authority’s contract managers were uncertain whether invoices that the RDP consultants approved were for appropriate services and whether contractor work products were reasonable based on their costs. Released 15 months later, the 2016 audit confirmed that these contract management issues persisted and were widespread. After reviewing a wider range of contracts and contract managers, the Authority’s auditors concluded that contract managers continued to delegate core contract management tasks to the RDP consultants, that roles and responsibilities for contract managers and the RDP consultants were not clearly defined, and that oversight of contract management was still insufficient. Further, this report highlighted that contract managers did not proactively document Key Recommendations From the their expectations for work products, putting the Authority’s Internal Audits Authority at risk of paying for inappropriate or February 2015: unsatisfactory work. • Develop effective, comprehensive contract management processes unique to the types of contracts that the The Authority has repeatedly acknowledged the Authority manages. need to address its contract management • Implement a formal system of review and oversight of deficiencies. Each internal audit report contained contract managers. recommendations intended to address inadequate • Establish performance standards for contract management. contract management policies and structures. The • Develop individual contract work plans that include how the Authority’s management concurred with these Authority will manage segmented responsibilities through a recommendations, which the text box summarizes. communication plan that identifies roles and responsibilities. In response to the 2015 internal audit, management May 2016: asserted that the Authority had contract • Establish oversight for contract managers, along management policies, procedures, roles, and with clearly documented expectations and regular responsibilities in place. However, it acknowledged communication to ensure contract requirements are met. that contract managers had not consistently • Ensure that persons performing contract management adopted those policies and procedures. After the have sufficient information to determine if invoiced costs 2016 internal audit revealed ongoing widespread are reasonable and deliverables meet requirements. deficiencies, the Authority’s management Source: The Authority’s February 2015 and May 2016 internal recognized that the implementation of existing audits of contract management. contract management policies had not yet yielded the desired results. California State Auditor Report 2018-108 15 November 2018 Current Contract Management Policies and Procedures In response to its 2015 and 2016 internal audits, the Authority established a contract administration organization in 2016, which included the Contract Management Support Unit (CMSU). The Authority tasked CMSU with developing revised and improved policies and procedures, governance structures, and training and other resources applicable to all contract management areas. In April 2017, the Authority approved nine new policies and procedures related to contract management. These policies cover key areas for contract management, including invoicing and payment, performance monitoring and reporting, and contract compliance. Additionally, the procedures outline processes to ensure that contract managers effectively manage contract documents, deliverables, risks, changes, and disputes. When we compared the 2017 policies with requirements in California’s State Contracting Manual, we found that the Authority’s policies were consistent with those requirements in areas related to controlling costs, such as prompt invoice review and comprehensive monitoring of contractor work products, referred to as deliverables. We also found that the 2017 policies improved upon the Authority’s previous policies by specifying required contract management processes and the documentation of those processes. That is, although both the 2014 contract manager handbook and the 2017 policies are broadly consistent with the State Contracting Manual, the 2017 policies emphasize specific process steps that contract managers must perform, require the use of tracking logs and other documents, establish a standardized file system for maintaining contract management documentation, and create a process for transferring files to new contract managers to preserve the consistency of contract oversight. Table 1 on the following page lists the documentation that the policies require contract managers to maintain to demonstrate how they are CMSU’s Oversight Responsibilities monitoring each contract. In these ways, the policies • Review and report on contract managers’ compliance. recognize both the deficiencies uncovered by the • Communicate noncompliance to appropriate supervisors. Authority’s internal audits and the need to document • Follow up on concerns until completely resolved. processes to facilitate and demonstrate compliance. Source: The Authority’s contract management policies and procedures. Although the Authority as a whole is responsible for writing contracts in a manner that safeguards the State’s interests, its contract management policies identify contract managers as the personnel responsible for overseeing those contracts once executed. Further, the Authority’s 2017 policies and procedures assign CMSU oversight responsibility to help ensure contract managers’ compliance with policies, as the text box describes. The Authority also requires CMSU to collect feedback to help develop lessons learned and identify areas for continued improvement. In Chapter 2, we evaluate the Authority’s implementation of its new contract management policies and procedures. 16 California State Auditor Report 2018-108 November 2018 Table 1 The Authority’s Procedures Require Contract Managers to Keep Clear Documentation Related to Ensuring Contract Value and Controlling Costs CONTRACT MANAGEMENT REQUIRED DOCUMENTATION TO ENSURE CONTRACT VALUE, CONTRACT MANAGERS MUST... AREA Invoice tracking log Use this log to track the contract’s invoices, expenditures, budget, and forecast. This process is essential to managing the contract and ensuring the contractor does not incur costs over the contract’s limit. Invoices Invoice approval checklist Complete an approval checklist to verify that each invoice submitted for payment is true, correct, and in accordance with law for all contracts with value greater than $5 million. Dispute tracking log Use this log to track and document invoice dispute information and the status of the resolution process. Deliverables tracking log Use this log to document the contract deliverables’ status and the determination of whether the deliverables are timely and meet the quality terms of the contract. Deliverable acceptance notice Use this notice to attest that a deliverable meets the acceptance criteria and to indicate Deliverables acceptance of the final deliverable. Recovery plan Request this plan from the contractor if a contract deliverable has fallen behind schedule, does not meet contract requirements or documented acceptance criteria, or may require repeated work. Risk register Use this register to identify risks, as well as strategies to accomplish contract objectives in the face of those risks. Change tracking log Assess any potential amendments to a contract for merit, and use this log to document Performance and the outcome of this assessment, the potential amendment’s impact on the project’s Amendments schedule and proposed cost, and a description of the issue or need for change. Contract compliance assessment Conduct assessments of contract requirements, including—at a minimum— insurance, small business utilization, deliverables, invoicing, schedule, change orders, and subcontracts. Source: The Authority’s contract management policies and procedures. California State Auditor Report 2018-108 17 November 2018 Chapter 1 THE AUTHORITY’S DECISION TO BEGIN CONSTRUCTION BEFORE COMPLETING PROPER PLANNING LED TO COST OVERRUNS AND DELAYS Chapter Summary After years of planning for a fully dedicated high-speed rail system, mounting costs led the Authority to decide instead to use existing infrastructure wherever possible—a cost control technique known as blending. Blending requires lower train speeds and imposes other service limitations, but the Authority will not know the full effect of these limitations until service planning and operations begin. Although blending has resulted in significant projected savings, those savings have only partially offset cost overruns. Further, potential time savings from reduced construction needs will be at least partially offset by the years that the Authority spent studying the dedicated options rather than pursuing blended options. The Authority has now exhausted every major opportunity available to share infrastructure with existing rail systems; thus, sharing infrastructure no longer represents a source of future cost savings. The Authority’s decision to begin construction despite not having sufficiently accounted for known risks contributed to its significant cost overruns. The Authority told us it decided to proceed with construction because it was concerned about deadlines for using $2.6 billion in federal grant funds. However, the risks in question—not having acquired the land to build on, a lack of agreements with existing utility systems, and uncertainty about the requirements that external stakeholders might impose— manifested in changes to its construction contracts that have thus far increased the three current construction projects’ costs by more than $600 million. Further, the Authority estimates that it will need an additional $1.6 billion in contract changes to finish these three projects, pushing its total cost overruns above $2 billion. The contract changes have also resulted in significant time delays, and consequently the Authority has had to continually extend the projects’ expected completion dates, pushing them back from 2018 to March 2022. Even with the extended schedules, construction will need to proceed much faster than it has to date for the Authority to meet the federal government’s construction completion deadline of December 2022. If the Authority misses this deadline, the federal government could require it to repay the grant funds it received; therefore, it is vital that the Authority do all it can to ensure its time and cost projections are accurate so that it can detect and address any further risks. Moreover, as it moves forward with the 18 California State Auditor Report 2018-108 November 2018 construction of the rest of the system, the Authority must take steps to ensure that it does not repeat the types of decisions that led to its significant cost and time overruns to date. After Years of Planning a Dedicated High-Speed Rail System, the Authority Has Now Pursued Every Option to Reduce Costs by Using Existing Infrastructure By incrementally modifying its plans for the high-speed rail system, the Authority reduced planned costs for some segments. However, these cost savings have also resulted in decreased service capabilities. When the Authority elects to blend a segment of the system by sharing existing rail corridor owned by another railroad, it significantly reduces planned costs by limiting the preparation needed to lay track. However, the Federal Railroad Administration sets a speed limit of 125 miles per hour for high-speed trains sharing a corridor or track with other rail traffic and of 110 miles per hour limit if the tracks intersect roads, which can be avoided by elevating tracks over or tunneling under roads. These blended segment speeds are significantly lower than those for dedicated high-speed segments of the system, where regulations allow speeds up to 220 miles per hour. Further, sharing track means that high-speed rail trains must split time on the tracks with other operators, limiting how frequently high-speed trains can operate on a segment. For example, on the San Francisco Peninsula, sharing tracks with Caltrain means that the Authority can only operate four high-speed trains per hour, instead of 12 per hour, as it originally planned. The Authority similarly plans to share track between Burbank and Los Angeles with Metrolink, Amtrak, and Union Pacific Railroad (Union Pacific). Figure 5 shows how these limitations will affect eventual service options for the three segments where the Authority has implemented blending: San Francisco to San Jose, San Jose to Gilroy, and Burbank to Los Angeles. Although blending a segment of Although blending will impose limitations on eventual high-speed the system by sharing existing rail rail operations, the extent to which the limitations will negatively corridor owned by another railroad affect actual rail service is not yet clear. According to the Authority’s will impose limitations on eventual deputy chief of rail operations, one reason why the limitations are high‑speed rail operations, the not yet known is that service decisions, such as how fast and how extent to which the limitations will frequently to operate the trains, have not yet been determined by negatively affect actual rail service the private sector operator that the Authority will select to run the is not yet clear. system. Until the operator decides how many trains are needed, the Authority will not know the effect of sharing track. Similarly, although reducing speed limits imposes a restriction with which the train operator must contend, other service considerations also will influence how fast the operator will run the trains. For example, California State Auditor Report 2018-108 19 November 2018 Figure 5 The Authority Has Adopted Blending in Three Segments SAN FRANCISCO to SAN JOSE Original Plan Blended Infrastructure High-Speed Rail Caltrain Shared tracks with Caltrain Shared Corridor Shared Tracks 125 110 HSR trains per hour HSR trains per hour MPH MPH speed limit speed limit Sacramento San Francisco MillbraeSFO SAN JOSE to GILROY San Jose Merced Original Plan Blended Infrastructure Gilroy Madera Fresno Shared tracks with Caltrain Kings/Tulare Dedicated Viaduct Shared Tracks Bakersfield 220 110 HSR trains per hour Undetermined† MPH MPH Palmdale speed limit* speed limit Burbank Los Angeles Union Station BURBANK to LOS ANGELES Original Plan Blended Infrastructure San Diego Shared tracks with Metrolink OR Dedicated Viaduct or Tunnel Shared Tracks 160 125 HSR trains per hour HSR trains per hour MPH MPH speed limit* speed limit Source: Review of the Authority’s business plans, capital cost basis of estimate reports, preliminary and supplemental alternative analysis reports, preliminary engineering for project design reports, service planning studies, and additional cost estimates provided by Authority staff. * Maximum speed limit shown; speed limited to 140 miles per hour in some segments. † The Authority stated that it plans to run 12 trains per hour on this segment but did not provide us with any studies or agreements showing how it will accomplish this number. 20 California State Auditor Report 2018-108 November 2018 the distance necessary to safely accelerate or decelerate high-speed trains means that the trains may not be able to operate at 220 miles per hour in parts of the system even if speed limits allow it because of the need to navigate curves and stop in stations. We attempted to identify how the new speed limits have affected the Authority’s projections of the blended segments’ travel times, but the Authority was largely unable to provide any supporting documentation for the travel times it projected for these segments before 2016. Because the Authority had already implemented much of the system blending by then, it was generally unable to demonstrate how much time blending added to its travel time estimates. One exception was the segment between San Jose and Gilroy, for which the Authority did not adopt blending until 2018. For this forty-mile segment, the projected travel time increased from fourteen to eighteen minutes when the Authority switched from a dedicated line to shared track, decreasing the maximum speed for this segment. Blending has allowed the Authority Blending has allowed the Authority to expedite the system’s planned to expedite the system’s planned time for construction by eliminating the time needed to design time for construction, but the and build tunnels, viaducts, and other dedicated infrastructure, effect of those time savings may but the effect of those time savings may be offset by the Authority’s be offset by the Authority’s past past decisions to continue to study dedicated options. Rather than decisions to continue to study adopting a blended model for as much of the system as possible early dedicated options. on, the Authority has incrementally accepted blended alternatives over the past six years. As of 2012, the Authority planned to construct two new, dedicated tracks, including tunnels and viaducts, between San Jose and San Francisco. Rising cost estimates for this section contributed to the $98 billion system cost the Authority reported in its draft 2012 business plan. To address the rising costs and local governments’ concerns about the potential impacts to environmental and community resources on the Peninsula, the Authority proposed a blended model in its revised plan. Shortly thereafter, the State Legislature mandated for this segment that the Authority could not use state funds to expand beyond Caltrain’s existing tracks in the corridor. In its revised 2012 business plan, the Authority reported the segment’s estimated costs had decreased from $13.6 billion to $5.6 billion, or 59 percent, after it adopted the blended approach. Table 2 provides the Authority’s estimates for the decreased costs of the blended segments, which have partially offset increases in its systemwide cost estimates. Despite its adoption of the blended model in its 2012 revised business plan for one segment, the Authority continued to study dedicated options for at least two more years and did not begin studying a blended option in Los Angeles until 2015, limiting the time savings it might have realized had it acted more quickly. In 2012, the Authority’s original plans for Burbank to Los Angeles California State Auditor Report 2018-108 21 November 2018 called for either a tunnel under central Los Angeles or an aerial viaduct—similar to a bridge—running through it. In its May 2014 analysis document supporting the 2014 business plan, the Authority was still planning for dedicated options including a tunnel, ground-level track, and viaducts. The Authority did not introduce the blended, shared-track model for this segment in its planning document and business plan until 2016. Table 2 Blending Significantly Reduced Planned Costs for Affected Segments (Dollars in Billions) EFFECTS OF BLENDING COST CHANGE SEGMENT BEFORE AFTER AMOUNT PERCENTAGE San Francisco to San Jose $13.6 $5.6 ‑$8.0 ‑59% San Jose to Gilroy 4.4 2.8 ‑1.6 ‑36 Burbank to Los Angeles 2.9 1.6 ‑1.3 ‑45 Source: The Authority’s published business plans, budgets, and internal planning documents. Note: This table only reflects cost estimates before and after blending was implemented on each segment in order to demonstrate the effect of blending on costs. Other factors, such as a reduction in the planned number of bridges in a segment, have lowered cost estimates after the implementation of blending. The Authority’s Southern California regional director confirmed that the Authority did not seriously begin studying a blended option for Los Angeles until 2015, when it procured a new planning contractor, and that it waited this long to ensure the blended model would not have unexpected consequences. Additionally, the regional director stated that because very little funding was available during this time period, the Authority could not conduct the study of the blended option. However, the 2012 revised business plan states that the Authority’s position is that the system’s benefits will be delivered faster through the blended approach. We therefore question why it waited three years to begin studying the blended option in Los Angeles to determine whether it was viable. Had the Authority acted earlier, it could have captured more of the time savings blending represents. For example, the Authority’s 2012 decision to use blending on the San Francisco Peninsula has led to construction already beginning in that location. By comparison, the Authority has yet to finalize its planned route between Burbank and Los Angeles. 22 California State Auditor Report 2018-108 November 2018 Although the process took several years, the Authority states it has now adopted blending in every segment of the system where sharing infrastructure is possible. In its 2018 business plan, it indicated for the first time that it intends to blend the segment between San Jose and Gilroy by operating within existing freight corridors and possibly sharing track with other carriers. Because the Authority is already pursuing blending on the San Francisco Peninsula and in Los Angeles, its chief of rail operations asserted that no further blending options are available. Additionally, he stated that travel time requirements mean that the Authority cannot implement additional blended segments even if opportunities become available. State law requires that the system be designed to achieve a nonstop travel time from San Francisco to Los Angeles Union Station of two hours and 40 minutes; according to the Authority’s model, the travel time incorporating the current level of blending is expected to be two hours, 36 minutes, and 56 seconds. Our review similarly noted that the blending of additional segments is unlikely because of characteristics of the remaining segments. For example, the only existing rail line traversing the Tehachapi Mountains in Southern California is a winding freight line built in the 1870s. In the north, where the Authority plans to connect the Central Valley to the Bay Area via the Pacheco Pass, no current rail system exists. In both regions, the Authority plans to pursue complicated tunneling projects that include tunnels over 20 miles long and more than 2,000 feet underground. Blending has allowed the Authority to partially offset significant Our analysis shows that the cost overruns for the system as a whole. Our analysis shows that Authority’s cost estimates would have the Authority’s cost estimates would have increased by 111 percent increased by 111 percent since the since the publication of its 2009 business plan had the Authority publication of its 2009 business plan not implemented blending; instead, overall costs have increased by had the Authority not implemented 81 percent. However, the fact that the Authority has now exhausted blending; instead, overall costs have all blending options limits its ability to mitigate the effects of future increased by 81 percent. cost overruns through additional blending. The Authority Has Approved Hundreds of Millions of Dollars’ Worth of Change Orders to Date, Most of Which Were for Changes It Initiated Changes and additions that the Authority has made to its three active construction contracts in the Central Valley have driven costs significantly higher than it originally projected. The Authority uses the change order process to account for unexpected developments, project delays that generate new contractor costs, and other changes to its construction contracts. The construction contracts allocate a specific dollar amount for each component of a project’s design and construction. For any additional work that is not contained in the contract, the Authority must authorize a California State Auditor Report 2018-108 23 November 2018 change order, which assigns a cost for the new work and increases the overall contract value. The Authority may direct a contractor to do additional work through a change order, or the contractor may request a change order for work it identifies as necessary. Change orders can also extend a project’s timeline either to allow additional work or to account for delays. To date, the Authority has approved more than $600 million worth of change orders related to the three construction projects in the Central Valley. The Authority relies on contracted construction oversight firms (oversight firms)—which are responsible for overseeing the construction contracts on behalf of the Authority—to evaluate potential change orders’ merits and provide independent estimates of how much they should cost. However, we found that the Authority did not always follow the oversight firms’ advice. We reviewed 11 of these change orders with a total value of The Authority approved change $38 million and found that the Authority obtained the required orders for dollar amounts that levels of management approval before executing each.1 However, were more than its oversight firms in four instances, the Authority approved change orders for dollar recommended or for work that the amounts that were more than its oversight firms recommended or oversight firms initially determined for work that the oversight firms initially determined was already was already covered under covered under the contracts. the contracts. Specifically, in two of these four change orders, the Authority executed changes for amounts that were greater than the oversight firms recommended. For example, the construction contractor for Project 1 requested more than $21 million for unanticipated bridge construction. The oversight firm disagreed with the contractor, estimating a cost of only $7.4 million. The Authority ultimately authorized a change order for $18.6 million—more than twice the amount the oversight firm recommended. When we discussed with the Authority’s director of design and construction why the Authority authorized more than the oversight firm recommended, he stated that the Authority’s initial position was that the construction contractor would cover the cost of some of the new work because it should already have been aware of the need for that work. However, he was unable to provide any documentation showing how the Authority determined the higher number was appropriate. In the other change order involving a higher amount than the oversight firm recommended, the Authority authorized an $868,000 change when the oversight firm had recommended only $854,000. The Authority’s documentation did not explain why the higher amount was appropriate. 1 We also reviewed two change orders that the Authority’s legal unit settled through a different process. Including these two change orders, the total value of our selection is $139 million. As of June 2018, the Authority had executed more than $600 million in change orders. 24 California State Auditor Report 2018-108 November 2018 In two other instances, the Authority approved change orders involving work the oversight firms initially determined was already required by the existing contract terms. However, the contract language was undermined by the assumptions that the Authority had made during the bidding process. For example, the contractor for Project 2/3 requested additional compensation for increased costs associated with disconnecting existing utility lines. In response, the oversight firm correctly identified that the contract assigned responsibility for utility disconnection tasks to the contractor, and therefore the contractor bore responsibility for these costs. However, the oversight firm also noted that Pacific Gas and Electric (PG&E), the utility owner, had begun to charge a fee for disconnections that the Authority had not specified in The Authority’s failure to adequately the information it provided to the construction contractor during coordinate with a key external the bidding process. As a result, the Authority and the contractor stakeholder before beginning negotiated the issue, and the Authority agreed to a change order the bidding process undermined of $2.7 million. In this case, the Authority’s failure to adequately its subsequent ability to use the coordinate with a key external stakeholder before beginning oversight firm to enforce contract the bidding process undermined its subsequent ability to use the terms and limit costs. oversight firm to enforce contract terms and limit costs. In the other case we identified, the oversight firm initially concluded that the construction contractor should bear the cost to redesign a bridge that did not meet Union Pacific’s standards. The Authority later approved a change order against this advice because it had not previously executed an agreement with Union Pacific, thereby limiting the contractor’s ability to coordinate with the railroad. We discuss this change order in greater detail later in this chapter. We found that the majority of all executed change orders came at the request of the Authority rather than the request of the contractors. Some of these change orders were the result of fundamental changes to the construction projects’ plans. For example, the Authority requested and executed a $153 million change order to extend one of the projects 2.7 miles north to connect to the Madera County Amtrak station. Because the Authority did not include this work in the original contract, it clearly required a change order. However, as we discuss in the following section, the Authority requested many other changes that were not the result of fundamental changes to the planned system, but rather related to its decision to begin construction before completing critical tasks. California State Auditor Report 2018-108 25 November 2018 The Authority Did Not Sufficiently Account for Known Risks in Its Initial Cost Estimates The Authority approved the start of construction in the Central Valley in 2013 despite knowing that moving forward was likely premature from a planning perspective and thus carried significant risks for unknown costs. According to its chief engineer, the primary factor in the Authority’s decision to execute a construction contract in August 2013 was to meet deadlines for project completion and spending of funds under the terms of its 2010 grant agreement with the federal government. The agreement provided $2.6 billion for the project under the Recovery Act. Because the purpose of the Recovery Act was to create and preserve jobs and revitalize state and local economies, the agreement required that the Authority complete the Central Valley construction by 2017.2 In coordination with the federal government, the Authority determined that it needed to begin construction as soon as possible to meet that deadline. Therefore, the Authority executed its first construction contract in August 2013 and authorized the construction contractor to begin work in October 2013. Figure 6 on the following page details the three projects currently underway—all of which are funded in part by federal money—and provides a timeline of Central Valley construction. The Authority did not complete many critical planning tasks before beginning construction, which ultimately resulted in significant delays and led to increased costs. Because the Authority’s planning Because the Authority’s planning was incomplete, it has used change orders to direct its contractors was incomplete, it has used change to perform additional work and to compensate them for delays. orders to direct its contractors Quantifying the total cost of the change orders resulting from to perform additional work and to the Authority’s insufficient planning is difficult, largely because the compensate them for delays. Authority’s change order summaries do not show which changes stemmed from the early start of construction. However, the majority of the change order costs relate to risk areas that the Authority had identified but not effectively quantified when it decided to move forward with construction. Some of these risks, such as not securing the property on which it intended to build, directly led to cost overruns and project delays. In other instances, the Authority did not sufficiently account for the costs arising from issues it knew it would eventually need to address, such as relocating utility infrastructure from project sites and addressing the concerns of external stakeholders. At the time, it indicated that it did not have the information or finalized agreements it needed to plan or budget for the mitigation of these issues. Figure 7 on page 27 summarizes the total impact that executed change orders have had on the three current construction projects in terms of cost and delay. 2 As we discuss later in this chapter, the federal government extended this deadline to December 2022. 26 California State Auditor Report 2018-108 November 2018 Figure 6 The Authority’s Three Current Construction Projects Have Been Phased In but Share a Deadline CONSTRUCTION MAP Madera Station Madera 2010 2011 PROJECT 1 2012 Fresno Station 2013 South of Fresno 2014 2015 Project 4 started Kings/Tulare Regional Station 2016 PROJECT 2/3 2017 2018 Original/current end date 2019 Expected end date 2020 North of Kings County Line 2021 PROJECT 4 2022 Shafter (North of Bakersfield) 4 TCEJORP Project 2/3 started Original end date Current end date Expected end date 3/2 TCEJORP Project 1 started Original end date Current end date Expected end date 1 TCEJORP CONSTRUCTION TIMELINE Signed Recovery Act Recovery Act deadline Source: The Authority’s contracts, business plans, baseline schedules, and maps. California State Auditor Report 2018-108 27 November 2018 Figure 7 The Authority’s Change Orders Have Increased the Cost and Length of Its Construction Contracts ORIGINAL CONSTRUCTION CONTRACTS CHANGE ORDERS CURRENT CONSTRUCTION CONTRACTS FOR ALL THREE PROJECTS (78% AUTHORITY-DIRECTED) FOR ALL THREE PROJECTS Original Cost: Additional Cost: Current Cost: $2.5 Billion + $600 Million = $3.1 Billion Original End Date: Schedule Delay: Current End Date: August 2019 9 Months May 2020 Source: The Authority’s change order records and construction contracts. Land Acquisitions The Authority’s decision to enter construction contracts despite not owning the required land—as well as its subsequent inability to acquire land on schedule—directly resulted in delays to the construction schedules. These delays in turn led to additional costs related to labor, materials, and equipment under contract but not in use. The Authority’s acquisition of the land was delayed in part by a 2011 lawsuit over whether the Authority had met legal requirements to issue bonds, which the Authority stated it needed to do in order to purchase property. Despite knowing that the lawsuit could restrict access to its funds, the Authority still initiated the request for proposals for Project 1 in March 2012 and executed its first construction contract in August 2013. In fact, the Authority signed the contract the same day that the superior court ruled against the Authority—effectively freezing its bond funds. Although the superior court’s decision was eventually overturned, the delay significantly set back the construction contractor’s schedule. Land acquisition delays have cost $64 million for Project 1 and extended its completion deadline by 17 months. The Authority also issued change orders because of land acquisition delays in Project 2/3 and Project 4. In total, these change orders have resulted in more than $115 million in additional costs. Utility Infrastructure Relocations The Authority also proceeded with construction in the Central Valley without completing agreements with utility companies or ensuring it had a full understanding of the magnitude of the utility infrastructure that it would need to relocate or how it would relocate those utilities. As a result, it could not properly budget for these costs. For example, the Authority originally expected to directly pay utility providers, such as PG&E and AT&T Inc., to relocate utilities for Project 1’s planned sites before construction, 28 California State Auditor Report 2018-108 November 2018 and it set aside nearly $69 million for this work. However, the Authority later determined the utilities would not be able to complete the relocations in time to meet construction deadlines, and in June 2015, it reassigned the work to the construction contractor. In February 2017, the Authority estimated that completing all of this work for Project 1 would ultimately cost $216 million. However, in April 2018, it provided the board with another revised estimate, which projected that costs would rise to $396 million. Unlike delays in land acquisition, the Authority’s poor estimates for utility work did not create costs where it might Given that Project 1’s original otherwise have had none. However, given that Project 1’s original construction contract was for construction contract was for $970 million, we find it concerning $970 million, we find it concerning that the Authority failed to anticipate what it now expects will be that the Authority failed to nearly $400 million in additional costs. Had it developed a better anticipate what it now expects understanding of the costs related to relocating utilities before will be nearly $400 million in beginning construction, it might have explored ways of mitigating additional costs. those costs. For the next project—Project 2/3—the Authority preemptively assigned utility relocations to the construction contractor. However, the Authority still did not execute an agreement with PG&E specifying the distribution of relocation work between the contractor and the utility for over a year after signing the construction contract. As a result, it accounted for this delay, along with delays related to right-of-way acquisition and other issues, by approving additional cost and time for the construction contract. The Authority has not had to add time for Project 4, but utility relocations have created additional costs. According to information it provided to us in July 2018, problems with utilities across the three Central Valley projects had already accounted for $215 million in costs not included in the Authority’s original budgets. The majority of these costs—$167 million—have come from Project 1, likely because it is the furthest along. Project 2/3 and Project 4 have experienced $29 million and $19 million in additional costs, respectively. External Stakeholders’ Requirements The Authority also did not ensure it was fully aware of the requirements that other external stakeholders, such as other railroads, would impose on the three projects. These requirements led to still more costs for which the Authority did not originally budget. For example, the Authority asked construction contractors to bid on Project 1 and in fact began construction before it finalized a coordination agreement with Union Pacific that specified the circumstances under which the construction contractor could build within Union Pacific’s right of way. The lack of such an agreement led to the construction contractor incorrectly assuming California State Auditor Report 2018-108 29 November 2018 that it could construct a pillar for a bridge within Union Pacific’s right of way. When Union Pacific declined to permit the pillar, the construction contractor argued that the Authority was at fault because it had not previously identified this issue and alerted the construction contractor. Although the Authority initially disagreed, it later agreed to share the costs with the construction contractor, with the Authority’s share being $414,000. The lack of an executed agreement was not unique to Union Pacific; in a June 2014 letter to the Authority, the construction contractor for Project 1 noted that the Authority had not executed needed agreements with several other external stakeholders—including two other railroad operators and the city of Fresno—which resulted in delays to the project. In total, the Authority has approved change orders worth In total, the Authority has approved $27 million related to requirements from external stakeholders change orders worth $27 million across the three Central Valley projects, and it anticipates that related to requirements from these extra costs will increase significantly during remaining external stakeholders across the construction. For example, freight carriers are currently insisting three Central Valley projects. that the Authority construct intrusion protection barriers to prevent freight trains from derailing onto high-speed rail tracks along certain portions of the Central Valley segment. The Authority has projected that these barriers will cost an additional $315 million. The Authority Was Aware That Its Early Start to Construction Could Lead to Significant Additional Costs Although its 2018 business plan asserted that the early start of construction in the Central Valley resulted in unforeseen or underestimated costs, the Authority had long been aware that there were risks associated with its decisions to begin construction without completing key preconstruction tasks and that these decisions could lead to significant additional costs. For example, a plan that consultants prepared for the Authority in 2011 identified land acquisition, utility relocation, and external stakeholder coordination as risks that would require mitigation. Similarly, the Authority’s chief engineer confirmed that the Authority knew in 2013 that it had not acquired sufficient land, and although it had a plan to secure the needed land, it also knew that its ability to use its bond funding for that purpose was uncertain because of legal challenges. The Authority noted in a March 2013 report to the Legislature that delays in acquiring property could affect costs and deadlines, and it disclosed in the same report that it had not yet entered into the necessary agreements with utility companies, which could lead to additional costs and delays. The Authority was also aware that it would need to work closely with Union Pacific to coordinate construction work on and around its right-of-way, yet the 2013 report noted that it had not yet executed an agreement 30 California State Auditor Report 2018-108 November 2018 with the railroad. It acknowledged that if the Authority could not reach such an agreement, design work in progress or already completed might be affected, leading to potentially significant cost increases or schedule delays. The Authority reported total Nonetheless, the Authority did not account for the potential costs cost estimates when beginning of these risks in its estimates until recently. As a result, it reported construction in 2013 that were total cost estimates when beginning construction in 2013 that were unreasonably low. unreasonably low. In fact, the Authority’s May 2018 business plan was the first to assign costs to known program risks, even though we identified concerns with the Authority’s risk management processes in our 2012 audit report.3 In that report, we found that although the Authority had identified risks that could affect the system’s cost and schedule—such as the lack of a finalized agreement with Union Pacific—it was not promptly and effectively addressing these risks. A 2013 report by the U.S. Government Accountability Office (GAO) expressed similar concerns about the Authority’s risk management, specifically identifying that the Authority had not completed a risk analysis to determine how the risks it had identified, such as right-of-way delays, would affect cost estimates. For example, according to the report, the Authority acknowledged the risk that its acquisition of 100 of 400 properties it needed for construction Project 1 would be delayed. However, the Authority did not include the possible effect of this delay in its reported cost estimates. The GAO consequently determined that the Authority only partially met best practices intended to help ensure the credibility of its cost estimates. Although the Authority has acknowledged that beginning construction when it did resulted in inaccurate cost estimates and contributed to additional costs, it has not yet taken sufficiently detailed steps to ensure a similar situation does not occur on future segments. For example, the Authority’s director of design and construction told us that the Authority planned to address these planning issues at a high level in a new program management plan that it was developing. We reviewed the program management plan, published in October 2018, and confirmed that it discusses the need to ensure land acquisition and utility relocations do not adversely affect construction timelines. However, it does not explain in detail about how the Authority will do so. Similarly, the Authority’s recently released comprehensive schedule delineates that these “early work” tasks should begin before the design and construction phase on future segments, but the schedule does 3 California High-Speed Rail Authority Follow-Up: Although the Authority Addressed Some of Our Prior Concerns, Its Funding Situation Has Become increasingly Risky and the Authority’s Weak Oversight Persists, January 2012, Report 2011‑504. California State Auditor Report 2018-108 31 November 2018 not establish specific benchmarks the Authority must achieve before procuring a construction contractor—though we note that the Authority does not plan to procure another construction contractor until 2020. When discussing the Authority’s future plans, its chief engineer stated that the uncertain nature of the Authority’s funding makes it continually reliant on the terms of available funding sources. The chief engineer stated that if the Authority were to receive another grant with short timeframes like those of the Recovery Act grant agreement, it might have to reevaluate its plans. He further stated that the decision to begin construction when the Authority did was partially driven by a desire to show visible progress as various groups were trying to stop the program, raising concerns that future external pressures may drive the Authority to make similarly poorly planned decisions again. Although we acknowledge that the Authority must secure additional funding to complete the high-speed rail system, we disagree that it should accept similar levels of risk brought on by beginning construction before it adequately performs planning. Looking past the Central Valley, the next planned construction segments, between San Francisco and Gilroy and then over the Pacheco Pass between the South Bay and the Central Valley, will present new challenges beyond what the Authority has faced in the Central Valley, including performing construction in dense urban areas and boring 15 miles of tunnels through the mountains. It is therefore imperative that the Authority formalize the lessons it indicates that it has learned in the Central Valley and that it implement a process to incorporate those lessons into its future planning. To Complete Construction of Its Three Current Projects, the Authority Believes It Will Need $1.6 Billion in Additional Change Orders and Extended Project Timelines Baseline estimates that the Authority and its oversight firms provided to us in July 2018 indicate that the Authority will need $1.6 billion in additional change orders to complete the three current construction projects, for an anticipated total cost of $4.7 billion. These changes include additional costs for ongoing activities we discuss in the previous section, such as utility relocation and land acquisition. The $4.7 billion total also includes costs for new activities, such as construction of intrusion protection barriers and efforts to mitigate problems with soil stability in the Central Valley. According to the RDP consultants, who are responsible for coordinating the Authority’s estimation process, 32 California State Auditor Report 2018-108 November 2018 the new baseline estimates represent a budget—and accompanying timeline—that the Authority believes is achievable and realistic, provided it takes appropriate actions as needed. Because of these projected changes, the Authority will need to extend its schedule significantly. As of June 2018, the Authority had extended the contract completion dates for Project 1 from 2018 to 2019 and for Project 2/3 from 2019 to 2020. However, given the work needed to complete the anticipated $1.6 billion in additional change orders, the Authority’s new baseline schedule indicates that it will need to extend the completion dates once again, with the latest—for Project 2/3—in March 2022. If the work does not progress as quickly as planned or if more changes become necessary, the Authority will likely need to push completion dates further into the future. Additional delays to the three current construction projects pose their own significant risks for the Authority, which must finish the Central Valley construction by December 2022 to avoid violating its federal grant agreements. The Authority received two federal grants for the Central Valley segment, one under the Recovery Act for $2.6 billion and a second for $929 million. Violating the grant agreements could require the Authority to repay this $3.5 billion in federal grant funds, $2.6 billion of which it reports it has now spent. The Recovery Act grant agreement’s deadline has been extended once before—from 2017 to 2022—at the Authority’s request. The Authority has not indicated any plans to request a second extension or to request an extension for its other grant, and it has no guarantee it would receive such an extension if it asked. In a legal opinion, the GAO concluded that the federal government could require the State to repay all $2.6 billion of the Recovery Act funds if it determines the Authority has violated the agreement, and it could recover the funds by offsetting any other payments by the federal government to the State. Consequently, the Authority’s 2018 business plan listed meeting the December 2022 deadline as Earned Value Analysis its first priority. According to the Project Management Institute, earned Meeting the federal deadlines for the Central value analysis is a tool that allows entities to measure the Valley projects will be challenging and will require progress of projects and to forecast their total costs and construction to occur significantly faster than it has dates of completion. in the past. Figure 8 illustrates how the Authority’s Planned value: How far along the project work is supposed change orders have added more work and extended to be at any given point in the project schedule and the construction schedule over time. The Authority cost estimate. uses a project management tool called earned Earned value: Actual progress to date in terms of project value analysis, which is described in the text box. schedule and cost. Figure 8, which is based on this tool, demonstrates the Authority’s planned and actual progress on its Source: Project Management Institute. three projects. Change orders affect the planned schedule (blue line) by increasing the amount of California State Auditor Report 2018-108 33 November 2018 work, which is expressed in terms of cost and time. As a result, the Authority’s actual progress (yellow line) needs to accelerate from its historical average for the Authority to complete the three projects by the federal deadline. As the green line that represents the required rate to meet the December 2022 deadline demonstrates, finishing the projects in time will require the Authority to work twice as fast over the next four years as it has since it began construction in 2013. If the Authority continues to work at its current rate, it will not complete all anticipated work until 2027, as the red line in Figure 8 shows. Further, the federal grant requires the Authority to lay track across the segment, a task for which the Authority has not yet procured a contractor. It plans to lay the track concurrently with the current construction projects beginning in 2020. Figure 8 The Authority Must Double the Rate of Central Valley Construction to Meet the Federal Deadline 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 Year noitelpmoC rof deriuqeR noitcurtsnoC )snoilliB ni sralloD( $5 4 3 2 1 0 2202 rebmeceD enildaeD tnarG laredeF Anticipated construction Anticipated total cost: $4.7 billion change orders REQUIRED RATE Executed construction change orders Original construction plan CURRENT RATE PLAN A N C E T D UAL Jun P e r 2 o 0 g 1 r 8 e — ss t $ o 1 . d 4 a b te il : lion Source: Analysis of the Authority’s estimates at completion, baseline schedules, construction contracts, monthly invoices, and monthly reports. Note: This figure only presents information on Projects 1, 2/3, and 4, which cover project infrastructure between Madera and north of Bakersfield. The Authority’s recently adopted schedule predicts that it can finish on time, but only if it effectively monitors and mitigates risks. When presenting the new schedule to its board, the Authority’s deputy chief operating officer stated his belief that the Authority can achieve the planned schedule, but he conceded that the approach 34 California State Auditor Report 2018-108 November 2018 is “very aggressive.” For the Authority to effectively mitigate future problems and accelerate the rate of construction, it must have accurate, realistic information on all the risks it faces. According to the Authority’s chief engineer—who oversees risk management for the system—the oversight firms play a significant role in the risk management process. However, as we discuss in detail in the next chapter, the Authority’s management of its contracts with the oversight firms has been flawed. Further, when we asked about certain information that the oversight firms had provided through the risk management process, an RDP consultant responsible for managing the schedule stated that the firms’ risk assessments were sometimes potentially misleading. He attributed this issue to the Authority not always closely or consistently monitoring the oversight firms. As the cost overages and delays As the cost overages and delays the Authority has experienced the Authority has experienced in the in the Central Valley to date demonstrate, insufficient risk Central Valley to date demonstrate, identification and management can have serious implications. If insufficient risk identification the Authority allows deficiencies in its risk assessment process and management can have to continue, it may not properly identify and respond to threats to serious implications. the system’s development. This could in turn prevent the Authority from meeting its December 2022 deadline. The High-Speed Rail Project Might Benefit From the Establishment of an Independent Oversight Committee Our review identified several similarities between the high-speed rail project and the California Department of Transportation’s (Caltrans) Toll Bridge Seismic Retrofit Program (retrofit program), another major transportation infrastructure project our office has evaluated several times. Our past audits noted that the retrofit program—which was tasked with retrofitting or replacing state-owned toll and highway bridges—had experienced cost overruns in part because of its management’s failure to perform adequate risk management to quantify potential cost increases. For example, our 2004 report on the retrofit program noted that Caltrans had identified certain risks, but it had not quantified the potential dollar costs until August 2004, when it reported soaring cost estimates to the Legislature. Our current audit identified similar problems with the Authority’s failure to effectively account for preconstruction risks in its cost estimates, as we note earlier in this chapter. In response to these and other concerns with the retrofit program’s costs and schedule, the Legislature required that Caltrans and the Metropolitan Transportation Commission (MTC) create an independent oversight committee to provide program direction, review costs and schedules, and approve significant change orders, California State Auditor Report 2018-108 35 November 2018 which the oversight committee deemed to be those over $1 million. Our most recent audit report on the retrofit program, released in August 2018, concluded that the oversight committee’s actions had resulted in $866 million in cost avoidance and savings, as well as the avoidance of seven years of potential delays. As a result, the retrofit program was completed generally on budget. We recommended in that report that the Legislature implement similar oversight committees for other large transportation projects that the State undertakes. The Authority’s efforts to deliver the eventual rail system might The Authority’s efforts to deliver benefit from similar additional oversight. That said, differences the eventual rail system might between the Authority’s and retrofit program’s governance benefit from similar additional structures make it unclear exactly what role a high-speed rail oversight as the retrofit program, oversight committee would play and which specific public entities but it is unclear exactly what role a should serve on it. Before the implementation of the retrofit project high‑speed rail oversight committee oversight committee, Caltrans managed the program directly. would play and which specific In contrast, the Authority’s board, which the Legislature and the public entities should serve on it. Governor appoint, governs the Authority. In 2008 the Legislature also required the Authority to create a peer review group composed of experienced individuals appointed by the state treasurer, state controller, director of finance, and secretary of transportation to review and analyze the Authority’s planning, engineering, and financing, and to report its findings to the Legislature. If the Legislature appointed a high-speed rail oversight committee, it would need to determine how that committee would work with the board and peer review group, as well as which entities would serve on the committee. Not all of the members that served on the retrofit program’s oversight committee—the chief executives of Caltrans, the MTC, and the California Transportation Commission—would be appropriate for the high-speed rail project. Of these entities, the California Transportation Commission could potentially provide additional guidance based on its statewide responsibility to manage transportation improvements. However, Caltrans is a current contractor on the high-speed rail system, which may limit its ability to provide objective oversight. Nonetheless, the Authority’s history of cost overruns and delays suggests that additional oversight may be warranted, especially considering the impending federal deadline for the Central Valley projects and the funding challenges the Authority faces in completing the system. The Authority has previously set cost estimates and timelines that its board allowed to be revised as challenges arose; an independent oversight committee may be better positioned to push back against changes to help maintain the current schedule and budget in the Central Valley and beyond. 36 California State Auditor Report 2018-108 November 2018 Recommendations To ensure that the change orders it approves are necessary and that their costs are appropriate, the Authority should adhere to the guidance and estimates the oversight firms provide to it. If the Authority chooses to deviate from the oversight firms’ recommendations, it should clearly document why it made those deviations. Before executing its next construction contract, the Authority should establish formal prerequisites for beginning construction to prevent avoidable cost overruns and project delays. At a minimum, these prerequisites should identify specific benchmarks related to land acquisition, utility agreements and relocations, and agreements with external stakeholders, including impacted local governments and other railroad operators. To better position itself to complete the three Central Valley projects by the December 2022 federal grant deadline, the Authority should improve its monitoring and evaluation of the oversight firms’ risk assessment processes and should take steps to ensure that these processes are consistent across the three projects by May 2019. To enable policymakers and the public to track the Authority’s progress toward meeting the federal grant deadline of December 2022, the Authority should, by January 2019, begin providing quarterly updates to the Legislature detailing the progress of the three Central Valley construction projects using an earned value model that compares construction progress to the projected total completion cost and date. The Authority should base these updates on the most current estimates available. To ensure that it is adequately prepared if it is unable to meet the federal grant deadline of December 2022, the Authority should, by May 2019, develop a contingency plan for responding to such a scenario. California State Auditor Report 2018-108 37 November 2018 Chapter 2 THE AUTHORITY HAS NOT SUCCESSFULLY ENFORCED THE POLICIES IT ADOPTED TO ADDRESS ONGOING DEFICIENCIES WITH ITS CONTRACT MANAGEMENT Chapter Summary Although the Authority recently adopted contract management policies to help ensure that it monitors its contracts and controls costs, it lacks an effective organizational structure to implement them. Under its current structure, the Authority’s contract managers experience high rates of turnover and receive little oversight. Further, they generally serve as contract managers in addition to their other professional responsibilities. Although the RDP consultants assist in contract management, they may not always have the State’s best interests as their primary motivation. Likely in part as a result of these weaknesses in the Authority’s contract management structure, we identified significant problems in its management of its contracts. We reviewed a selection of engineering and other service contracts to assess the Authority’s compliance with its policies and procedures most directly relevant to controlling costs and ensuring value: invoice review, deliverables monitoring, and change management. We found that although the contract managers complied to varying degrees with the invoice review procedures, they consistently did not document the receipt or evaluation of contractor deliverables, nor did they independently evaluate the need for contract changes that added cost and time. In fact, the contract managers’ lack of documented, independent review prevented us from reaching conclusions about the Authority’s effectiveness in assessing the quality, timeliness, or cost of the work performed under these contracts. Without such documentation, the Authority cannot demonstrate that the hundreds of millions of dollars it has spent to date on the selected contracts—including for cost overruns in the form of amendments—has been necessary or appropriate. Similarly, our review of the Authority’s construction contracts found that it has implemented a construction invoicing process capable of significantly limiting the risk that it overpays these contractors for the work they perform. However, the Authority has not provided reliable monitoring of the oversight firms that are responsible for managing this invoicing process. Further, it has only recently developed formal monitoring to evaluate the performance of the oversight firms. 38 California State Auditor Report 2018-108 November 2018 The Authority Has Not Established an Effective Contract Management Structure As we discuss in the Introduction, the Authority adopted policies related to contract management and oversight in April 2017. However, these policies will only prove effective if Authority staff follow them, and the Authority has yet to create a contract management structure that adequately ensures such adherence. Instead, its contract managers experience high turnover and receive little oversight. Moreover, weaknesses in the Authority’s contract management structure have contributed to its reliance on contractors for important functions—such as contract oversight— that state employees should perform. The Authority’s Contract Managers Experience High Turnover and Receive Little Oversight The Authority’s contract managers’ official responsibilities do not always specify contract management duties. As of September 2018, the Authority’s 56 contract managers were collectively responsible for 204 contracts with values totaling $5.6 billion. However, according to the contract administration manager, only three of those 56 contract managers serve in contract management roles full-time. Although records from the Contract Management Support Unit (CMSU) show that each of the eight contract managers responsible for the contracts we reviewed completed the Authority’s required contract management training, our review of these individuals’ duty statements found that only three specifically mentioned contract management duties.4 Further, only one of the individuals mentioned in this chapter carries the job title of contract manager. Although we refer to them all as such for the purposes of their responsibilities, their actual titles include administrator, engineer, and executive. The former manager of CMSU acknowledged that the Authority has not established a formal practice for selecting contract managers and assigning them to contracts. Perhaps as a result of that fact, the Authority frequently changes the individuals responsible for managing each contract, resulting in high rates of turnover among its contract managers. For the nine contracts we reviewed, CMSU’s roster of contract managers shows that five contracts had two or three different contract managers in the past year alone. In fact, the contract managers for three of the contracts changed during the period of our review from March through June 2018. High turnover has not only likely contributed to noncompliance with policies, but it also underscores the need for strong documentation practices. 4 One contract manager was responsible for two of the nine contracts we reviewed. California State Auditor Report 2018-108 39 November 2018 For example, one new contract manager told us she received a transfer form from the previous contract manager indicating that all the required contract documentation was available and up to date. However, she did not sign the form because much of the documentation listed on the form was not actually available. The responsibility for ensuring that contract managers perform required tasks lies with their direct supervisors, who themselves are not consistently trained in the Authority’s contract management policies and procedures. Six supervisors managed the eight contract Six supervisors managed the managers in our review, and half of them had not received training on eight contract managers in our the Authority’s contract management policies and procedures at the review, and half of them had not time of our review. If they do not understand the Authority’s specific received training on the Authority’s requirements for contract management, supervisors may not be able contract management policies and to intervene effectively when contract concerns arise or to provide procedures at the time of our review. strong oversight to ensure contract managers adhere to policies and procedures. Additionally, these supervisors also have full-time responsibilities unrelated to contract management. In fact, some of the supervisors are in executive leadership positions, including the chief executive officer and the chief financial officer (CFO), each of whom oversees multiple contract managers in addition to their responsibilities for large segments of the Authority’s operations. The Authority established CMSU within the Contract Administration Branch in part to oversee compliance with contract management policies and procedures, but its oversight has been weak and inconsistent. The extent of its oversight activities to date has been a fall 2017 review of whether contract managers had filed documentation in the locations and structure that the Authority’s policies and procedures required. Although this review consistently found that contract managers had not filed required tracking logs as expected, CMSU did not take any additional steps to determine whether the contract managers were actually using those logs or to verify compliance with any other policy requirements. If CMSU had performed such additional reviews, it could have identified some of the more significant compliance issues we discuss later in this chapter. In response to our concerns, the CFO asserted that the current set of contract management policies and procedures took significant effort to develop, as did developing and conducting the Authority’s contract management training. However, he also acknowledged that the Authority has not yet taken additional steps to ensure compliance. Similarly, the director of the Contract Administration Branch (contracts director) asserted that only a short time has elapsed since the implementation of the Authority’s new contract management policies and procedures and that the branch is relatively new. Given that these policies and procedures have been in effect for over a year and that the contract managers signed forms pledging to comply 40 California State Auditor Report 2018-108 November 2018 with them, we believe ample time has passed for the Authority to have conducted meaningful oversight. The CFO informed us that the Authority intended for CMSU to conduct contract manager compliance assessments and submit the results of these assessments to the contract managers’ supervisors, as the Authority’s policies and procedures specify. However, he confirmed that CMSU has not conducted these assessments and thus has not implemented the process of notifying supervisors of any noncompliance. The CFO further expressed his belief that timing and leadership transitions have contributed to the Authority’s general challenges in ensuring its staff comply with its contract management policies and procedures. He stated that since adopting its contract management policies and procedures in April 2017, the Authority has directed its efforts toward creating two new executive positions—the chief deputy director and the chief operating officer—to oversee new offices that would more appropriately include contract management. In October 2018, the Authority adopted a project management plan that includes placing contract management under the direction of these new executive positions. However, the plan is not sufficiently detailed to address shortcomings in the Authority’s current contract management related to defining contract managers’ formal duties and ensuring enforcement of those duties. We also do not agree that the time spent rearranging the high-level organization of contract management supplants the responsibility to simultaneously strengthen the existing system. If the Authority believes, as we do, that strong and accountable contract management is key to controlling the system’s costs, it must commit to fully implementing and enforcing its contract management policies and procedures. Weaknesses in the Authority’s Contract Management Structure Have Likely Contributed to Its Overreliance on Contractors The Authority’s inadequate enforcement of its contract management policies and procedures may encourage its reliance on contractors to perform important functions, further hindering its ability to control costs. As we noted in our 2012 audit report, the Authority’s organizational structure places large portions of its program planning, construction, and oversight in the hands of the RDP consultants, who may not have the best interests of the State as their primary motivation. Further, as we discuss in the Introduction, the Authority’s internal audits concluded that roles and responsibilities for contract managers and RDP consultants were not clearly defined. To address this issue, the Authority’s contract management policies and procedures clearly assign contract managers the responsibility for tracking and monitoring all aspects of the contracts they manage. Nonetheless, we observed that the contract managers for the California State Auditor Report 2018-108 41 November 2018 regional planning contracts—which are for preliminary engineering Even though RDP consultants may and environmental work in locations where the Authority plans not have the best interests of the to develop the high-speed rail system—still often rely on the State as their primary motivation, RDP consultants to provide the oversight for which the contract the contract managers for the managers are ultimately responsible. In fact, during our review, the regional planning contracts still Authority’s contract managers for the regional planning contracts often rely on the RDP consultants directed our contract management questions to the RDP consultants to provide the oversight for for answers and were generally unable to provide documentation which the contract managers are related to contract management that did not originate from the ultimately responsible. RDP consultants. As a result, the RDP consultants have become the de facto contract management body, working closely with contractors with insufficient Authority oversight. Further, the Authority has also placed the oversight responsibility for contract management with the RDP consultants, which creates a potential conflict of interest. Specifically, although an Authority employee heads CMSU, the RDP consultants fill its seven positions. When the Authority’s contract managers inappropriately rely on the RDP consultants to perform their contract management responsibilities, it may not be reasonable to expect CMSU staff— who are also RDP consultants—to tell state contract managers to stop this practice. Consequentially, CMSU’s current composition raises questions about the Authority’s ability to use the unit as a tool to prevent the Authority’s continued overreliance on the RDP consultants to perform contract management, which we believe should be among CMSU’s priorities. The Authority tasked contractors with duties that state employees could have performed in other instances as well. For example, the Authority’s documentation for its $40 million contract for financial advisory services states that contracting for those services is justified because the tasks are of a highly technical nature and equivalent expertise is unavailable within state civil service. However, in December 2016, the Authority’s former chief executive officer (CEO) sent the CFO an email in which he expressed concern over high spending rates for the contract, particularly in the areas of accounting support and budgets. In his email, the CEO stated that contract spending should focus on nontraditional areas of work, such as financing analyses and commercial and real estate strategies, and that employing state staff to perform basic budgeting and accounting work would be more appropriate and cost-effective. The CFO responded to the CEO’s concerns by stating that a core group of state staff provided services for budgeting and accounting, but that the financial advisory contractors were needed to help perform responsibilities that had no precedent in state service, including implementing information technology systems. The CFO reiterated this position to us during our audit. However, our review of the contract’s work plans and invoice materials determined 42 California State Auditor Report 2018-108 November 2018 that, although some of the contractor’s duties included information Given the fact that the contractor technology tasks, many of the tasks it reported performing were billed the Authority $3.5 million in described as support for general budgeting and accounting activities. fiscal year 2016–17 and $1.1 million Given the fact that the contractor billed the Authority $3.5 million in fiscal year 2017–18 for the in fiscal year 2016–17 and $1.1 million in fiscal year 2017–18 for budgeting and accounting portions the budgeting and accounting portions of the contract, we believe the of the contract, we believe the Authority should have taken steps to ensure and document that its Authority should have taken steps use of contracted resources was necessary and prudent. to ensure and document that its use of contracted resources was The Authority also assigned work related to contract management necessary and prudent. to outside contractors that may have been more appropriately performed by state employees. For example, the Authority tasked its financial advisory contractor to perform analysis and support for its Contract Administration Branch from July 2016 through June 2018. As part of this work, the contractor developed the contract management policies we reviewed during this audit. The CFO explained that these policies and procedures took significant effort because no equivalent state criteria directly apply to the Authority. However, we do not believe that the Authority is so unique in its contract management needs that state resources could not develop similarly adequate tools. Further, the Authority also tasked the contractor with monitoring and reporting on the status of the RDP consultants’ deliverables and with supporting CMSU in its compliance reviews of the Authority’s contract managers. As we discuss above, these compliance reviews have been insufficient. We do not believe that any of these tasks are so highly technical or specialized that state employees could not have performed them. Nonetheless, the financial advisory contractor billed the Authority almost $4 million for these tasks over two years. Finally, just as the Authority staffed CMSU with RDP consultants to oversee contract manager compliance, it also staffed its separate administrative unit for supporting contracts entirely with RDP consultants, who filled all 17 positions as of June 2018. We question why using RDP consultants in place of state employees to perform contract management oversight and support is necessary. The Authority agreed that it should place state employees in these positions in the future. The Authority Has Not Ensured That Its Contract Managers Actively Manage Expenditures and Deliverables in Compliance With Its Requirements We found that the contract managers generally complied with the Authority’s documentation requirements for reviewing invoices but did not comply with the procedures for documenting timely and thorough review of deliverables. Figure 9 illustrates that although contract managers were often able to provide us documentation to demonstrate their compliance with invoice requirements for using California State Auditor Report 2018-108 43 November 2018 a tracking log and completing an invoice approval checklist, they could not consistently demonstrate how they validated invoiced costs to ensure they were appropriate and allowable. The contract managers were also generally unable to demonstrate their review of deliverables or their efforts to monitor contractor performance. This lack of documented, independent review prevented us from reaching conclusions about the quality, timeliness, or cost of work performed under these contracts. Further, when we did identify references to concerns with contractors’ work products, the lack of documentation meant that we were generally unable to determine how contract managers identified or resolved such issues. Without clear documentation that its contract managers ensured deliverables were consistent with requirements before approving payments or that they appropriately monitored contractor performance, the Authority cannot demonstrate that the hundreds of millions of dollars it has spent to date on those contracts—including for cost overruns—has been necessary and appropriate. Figure 9 Contract Managers Failed to Consistently Document Completion of Tasks Necessary to Control Costs and Ensure Value 100 95% 80 60 38% 40 20 14% 4% 0 sreganaM tcartnoC yb detnemucoD sksaT fo egatnecreP Task Required to Tasks Required to Pay Contractors Control Costs and Ensure Value Review Ensure Costs Evaluate Assess Performance Invoices Are Allowable Deliverables and Amendments Source: Analysis of the Authority’s available contract management documentation. 44 California State Auditor Report 2018-108 November 2018 The Contract Managers Could Not Consistently Demonstrate That They Performed Thorough Reviews of Contractors’ Invoices In order to assess the Authority’s compliance with its own policies and procedures, we reviewed nine active contracts overseen by eight contract managers. The contracts we reviewed include planning, engineering, consulting, and construction oversight services and have a total value of $1.3 billion.5 The services on these contracts include a combination of discrete deliverables and day-to-day work involving on-site staff support. Our review focused on the Authority’s compliance with those contract management policies and procedures most closely linked to controlling costs and ensuring value, including invoice and deliverables review. The Authority has established procedures requiring a systematic approach to its contract managers’ receipt and handling of invoices, appropriate invoice documentation and review, and required approvals for timely processing of payments. For example, Authority contract managers must ensure that contractors bill hourly rates appropriately, that all direct costs are eligible and supported with receipts, and that the totals of each invoice are calculated correctly. In addition, the Authority’s procedures require contract managers to complete an approval checklist for each invoice for contracts with values greater than $5 million. These checklists require contract managers to attest to the accuracy of the invoices and the sufficiency of supporting documentation by checking boxes stating that the invoices comply with relevant contract provisions and that all expenses are eligible and supported with receipts. The procedures also require contract managers to keep logs that track invoiced and approved amounts, key dates, and any disputed costs. When we examined a selection of invoice approval checklists for the nine contracts we reviewed, we found that the contract managers completed checklists indicating that they evaluated The contract managers for each invoice for accuracy. However, the contract managers for only three of the nine contracts only three of the nine contracts could provide documentation we reviewed could provide to support the checklists’ assertions that they ensured invoiced documentation to support the rates and expenses were allowable. For example, one contract checklists’ assertions that they manager provided us with a review spreadsheet in which, in ensured invoiced rates and addition to monitoring monthly spending against the contract’s expenses were allowable. value, he tabulated each individual billing rate, the hours reported by task, and all direct expenses. The manager of the Contract Administration Support Unit told us that contract managers can 5 We also reviewed the Authority’s oversight of its three construction contracts, which have a combined current value of $3.1 billion. We discuss the management of these contracts in the following sections of this report. California State Auditor Report 2018-108 45 November 2018 request support from this unit to verify that contractors bill hours against allowable rates, but only one of the contract managers in our review used this resource. We expected to see documentation from contract managers verifying that costs were allowable and accurate because most of the invoices we reviewed were complex; they included many individual pay rates for contractor staff, tens or hundreds of thousands of dollars in direct costs that the contract managers had to verify, and many subcontractor costs. When it relies primarily on the contract managers’ attestations on the invoice approval checklists, the Authority risks that those contract managers may not have performed thorough and comprehensive invoice reviews. As a result, the Authority could pay for unauthorized costs. We An internal audit that the Authority observed one such instance in which an internal audit that the issued in June 2017 identified that it Authority issued in June 2017 identified that it had paid for more had paid for more than $1.2 million than $1.2 million in questioned and disallowed costs to one of its in questioned and disallowed costs contractors—costs that either were not supported by adequate to one of its contractors—costs that documentation or were not in compliance with the contract terms either were not supported or were or applicable rules and regulations. not in compliance with requirements. Similarly, although the contract managers we reviewed consistently used required invoice tracking logs, these logs had limited utility. Specifically, the invoice tracking logs serve as a record of the amounts that the contractors bill and the Authority approves each month. When the contract managers enter approved invoice amounts, the tracking logs automatically calculate the remaining contract balance. Although they help ensure that the Authority does not overspend the contract balance, the tracking logs do not demonstrate detailed invoice review any more than the invoice checklists. More importantly, none of the required invoice documents demonstrate whether or how the contract managers performed the critical task of determining that invoiced costs were appropriate for the amount and quality of the contractors’ work. The Authority Lacks Evidence Supporting the Quality, Timeliness, and Cost of Contract Deliverables Most of the managers for the contracts we reviewed asserted that when they approved invoices, they reviewed narratives summarizing the work performed. However, these narratives are generated by the contractors and do not serve to independently verify that the contractors have, in fact, performed the work to the standards of the contracts. In establishing its 2017 policies, the Authority acknowledged the limitations of this invoice review process by creating a parallel but distinct process through which it requires contract managers to identify, document, track, receive, review, and accept contract deliverables. 46 California State Auditor Report 2018-108 November 2018 The Authority’s policies and procedures clearly Requirements for Contract Managers’ Oversight state that this process—known as deliverables of Deliverables: management—is meant to ensure the Authority is able to construct the high-speed rail system on • Identify all deliverables, along with associated timelines. schedule and within budget. The text box • Define and document tasks, performance expectations, describes the Authority’s procedural requirements and timelines for contracts that do not identify for managing deliverables and identifies the specific deliverables. documentation required to track that process. • Organize the deliverables and all relevant information, The fundamental purpose of these tracking including objectives, due dates, responsible individuals, activities is ensuring that the contractors’ estimated budget, and acceptance criteria, into a deliverables are consistent with quality, timeliness, deliverables tracking log. and cost requirements. • Review deliverables to ensure they meet all requirements before formally accepting them as complete. Despite the specificity of the Authority’s • Document each complete deliverable by issuing an requirements related to deliverables, the contract acceptance notice to the contractor. managers we reviewed did not comply with the • Request a recovery plan, including an updated schedule required actions. None of the contract managers and budget, for late or unsatisfactory deliverables. for the nine contracts we reviewed used the • Track and escalate contractor performance issues if the standardized deliverables tracking log—or contractor continues to provide inadequate deliverables any other document—to independently track and is unable to meet its contractual obligations. the status and review of contract deliverables. Source: The Authority’s deliverables management procedures. Further, contract managers did not document their formal reviews of deliverables; instead, they described various alternative methods for these reviews. Some contract managers claimed to monitor contracted work simply by observing and working closely with contracted staff on a daily basis. Others relied on the RDP consultants to approve contractor-generated progress reports and individual deliverables, after which the contract managers would approve payment. Still others stated they relied on other Authority staff and subject matter experts to evaluate and approve individual deliverables, although those delegated evaluations were also generally not documented. Though we recognize that subject matter experts play an important role in assessing the quality of work, their expertise does not supplant the responsibilities of the contract managers. Finally, instead of using the Authority’s required templates for tracking the receipt and sufficiency of deliverables, contract managers for four of the nine contracts we reviewed provided documents that they use to verify the extent to which deliverables were timely and adequately completed. However, we noted that the contractors prepared these documents. In fact, for nearly all the contracts we reviewed, the only documented source of information regarding the timeliness and status of deliverables came from the contractors themselves. As a result, we were generally unable to determine how the Authority independently ensured it received the deliverables for which it paid and that they were of the quality that it required. California State Auditor Report 2018-108 47 November 2018 The Authority Paid for Deliverables Without Performing Proper Review Our review of nine contracts also found that the contract managers rarely had documentation for their acceptance of deliverables. In fact, the contract managers for seven of the nine contracts did not have any acceptance notices on file. Authority policies and procedures require contract managers to review deliverables for compliance with contract requirements and the Authority’s acceptance criteria. The contract managers must then officially accept the deliverables in writing. This documented communication marks the point in time when the Authority takes care, custody, and control of the deliverable. Despite this requirement, when we asked the contract manager for an environmental contract about the missing acceptance notices, he stated that the contract had only two real deliverables, one of which is the environmental impact report. He stated that he planned to evaluate that report upon its completion and that he would issue the acceptance notice at that time. However, the contract documentation makes clear that the contractor is responsible for many individual work products as part of developing this larger report. When we asked about these work products, the contract manager stated that instead of reviewing deliverables as the contractor submits them, he plans to later issue a memorandum to summarize all deliverables. We received similar responses from other contract managers, who stated that acceptance notices were not applicable because the contracts they managed did not include what they considered to be formal deliverables. However, we disagree with this assertion; for each of these contracts, we identified evidence of discrete work products—or deliverables— that the Authority policies and procedures would require contract managers to evaluate and formally accept. In fact, only two of the nine contracts had acceptance notices Contrary to Authority policies on file that documented the timing of deliverable submission and procedures, only two of the and review, and this documentation was minimal and used nine contracts had acceptance inconsistently. When we requested acceptance notices for one of notices on file that documented the the two contracts—a four-year, $40 million contract for financial timing of deliverable submission and advising services—the contract manager provided nine notices, review, and this documentation was all of which he signed and dated June 11, 2018—after the date of minimal and used inconsistently. our request on June 4. Further, none of the notices contained any comments in the template fields that contract managers can use to record the details of their reviews. When we subsequently followed up to ask about additional deliverables for which the contract manager had not provided acceptance notices, he produced another three notices, each of which he signed on July 9, 2018—again, after our request. In some instances, the contractor had submitted the deliverables in question as early as October 2017. Because the Authority pays the contractor for its work on a monthly basis, 48 California State Auditor Report 2018-108 November 2018 even if the contract manager had performed a detailed review in June and July 2018 and had found issues with the deliverables, the Authority had already long since paid the contractor for the work. When we asked the contract manager about his signing the nine notices on the same day, he asserted that the process was for the contractor to submit the acceptance notices to him at the end of the fiscal year, rather than with the deliverables. However, as we state above, he provided the original set of acceptance notices before the end of the fiscal year, and soon after we requested them. Further, because the acceptance notices are a part of the Authority’s policies and procedures, we would expect the contract manager to initiate the process, not the contractor. When we expressed concern about the contract manager’s approach, he replied that he believed the process was sufficient because he would complete the notices before the contract was closed. However, if the Authority waits until the contract’s end to review deliverables, it will have already substantially paid for these deliverables. When we asked about any other evidence to demonstrate his review of deliverables, the contract manager provided emails intended to demonstrate his review. Our review of those emails found that the contract manager was not included on many of them, some of which were between contractors only. The contract manager told us that going forward, he plans to complete an acceptance notice contemporaneously with receipt of each deliverable. To the extent that the contract manager also performs and documents his comprehensive review of each deliverable at this time, we agree it is appropriate for him to complete the process by signing and issuing acceptance notices. We also identified significant concerns with the Authority’s collection and evaluation of deliverables for its $666 million RDP consulting contract, which includes tasks in 30 subject matter areas. In late 2017, the contract manager tasked a contractor—from a firm The Authority was missing formal other than the RDP consultants—with conducting an assessment acceptance notices for 70 of of the deliverables and corresponding acceptance notices that the the 80 deliverables and did not Authority had on file dating back to the start of the RDP contract in have 145 of the 184 deliverables July 2015. To conduct this assessment, the contractor compared the that the RDP consultants had RDP consultants’ self-reported information about the deliverables reported as having completed it had submitted to the Authority to the deliverables the Authority since the beginning of the contract actually had on file. The review determined that for the work in July 2015. Nonetheless, from plans that were active when the Authority’s contract management July 2015 through December 2017, the policies and procedures went into effect, the Authority was missing Authority paid the RDP consultants formal acceptance notices for 70 of the 80 deliverables that the over $200 million for the tasks that RDP consultants reported as complete. Further, the Authority did included these deliverables. not have 145 of the 184 deliverables that the RDP consultants had reported as having completed since the beginning of the contract. These missing deliverables ranged from engineering documents to software updates to white papers and other strategic documents. California State Auditor Report 2018-108 49 November 2018 Nonetheless, from July 2015 through December 2017, the Authority paid the RDP consultants over $200 million for the tasks that included these deliverables. When we asked about the Authority’s efforts to follow up on the missing and unapproved deliverables, the contract manager told us that because of funding constraints, the non-RDP contractor was no longer working on the deliverables issue. However, he stated that he had requested help from other Authority staff in identifying and recovering the deliverables. Nonetheless, as of July 2018, he had not yet received any new information. Without documentation of formal review and approval, the Authority cannot demonstrate that it received the quantity and quality of work for which it paid the RDP consultants. Moreover, the Authority’s tracking and evaluation of this contract’s deliverables has continued to be an issue. In July 2018, the contract manager provided us a list of deliverables that the RDP consultants had reported submitting, which he originally represented as an updated tracking log. However, the contract manager later stated that because of the aforementioned funding constraints and a lack of support staff, he had not had the opportunity to verify the submission and timing of the deliverables on this list. He asserted that subject matter experts, who are state employees, are involved in developing the monthly status reports that the RDP consultants submit to the Authority, which include the status of deliverables. However, he acknowledged at the time that he had not reached out to subject matter experts to collect the deliverables that the RDP consultants had reported as complete and therefore had not yet completed deliverable reviews in order to issue acceptance notices. Despite having confirmed this lack of acceptance notices on multiple occasions, the contract manager informed us in October 2018 that he did in fact have completed acceptance notices for some of the RDP consultants’ deliverables. He then provided 77 signed acceptance notices dated as far back as January 2018. These acceptance notices do not alleviate the need for detailed Even though the contract manager deliverable tracking and review documentation. The acceptance asserted that the acceptance notices on their own do not allow the contract manager to notices are the formal deliverable determine whether the RDP consultants’ work is generally on review documents, neither the schedule. Along with the acceptance notices, the contract manager acceptance notices that he also provided a log his staff began compiling in August 2018 to track provided nor the log contain detail the status of deliverable acceptance. If used going forward, the log about how the contract manager will help the contract manager more proactively track the status determined that deliverables met of all deliverables. However, even though the contract manager contract requirements. asserted that the acceptance notices are the formal deliverable review documents, neither the acceptance notices that he provided nor the log contain detail about how the contract manager determined that deliverables met contract requirements. 50 California State Auditor Report 2018-108 November 2018 Given the large dollar amount associated with this contract, it is crucial that the Authority improve its practices for tracking the status of deliverables and reviewing them for quality. Because of the Authority’s Failure to Track Deliverables, Determining How It Resolved Quality Issues Is Difficult, If Not Impossible The contract managers’ insufficient tracking of deliverables also means that when the contract documentation alluded to issues with contractors’ work products, we generally could not determine how the Authority identified the issues or confirm that they were resolved appropriately. When contractor deliverables are unsatisfactory, a contract manager must not only identify the unsatisfactory work, but also ensure the Authority does not pay for the hours spent to fix it. For example, in September 2017, a regional planning contractor in Southern California had to revise its design for a train station access road because it adversely affected a historical bridge. The Authority considered the hours spent revising the design as repeated work for which it had already paid. Although the contract documentation indicated that the Authority contract manager formally disputed the invoice that included charges for this work, it provided no further information. When we Neither the contract manager nor asked for details about the dispute, the contract manager stated that the RDP consultants documented the RDP consultants were responsible for identifying the issue and how they ensured that they determining the total amount that the Authority should not pay. accounted—and therefore avoided However, neither the contract manager nor the RDP consultants paying—for the contractor’s documented how they ensured that they accounted—and therefore repeated work. avoided paying—for the contractor’s repeated work. In another example from January 2018, an RDP consultant expressed concern to his RDP supervisor about the quality and timeliness of a contractor’s deliverables for a Northern California environmental planning project. When we asked how the issue was resolved, the RDP consultant stated that he did not dispute the related invoice because the contractor did not claim the deliverables for payment because of their deficiencies. To demonstrate this, the consultant provided an invoice from the contractor that showed hours worked for which the contractor did not bill the Authority. However, because of the lack of detail in the RDP consultant’s tracking documentation and the fact that the contract manager had no additional documentation, we could not substantiate that the hours on the invoice accounted for all of the repeated work the contractor performed. Both of these examples also demonstrate the Authority’s overreliance on the RDP consultants to provide oversight of certain contracts. In the first example regarding the train station access road, the available documentation about the origin of the California State Auditor Report 2018-108 51 November 2018 issue was limited to communications among the RDP consultants, and we identified no evidence that the contract manager took steps to independently evaluate the situation. Similarly, in the second example, the contract manager for the environmental planning contract asserted that because his background is engineering, he relies on the RDP consultants to recommend when the Authority should dispute invoices for environmental work. However, as of April 2018, the contract manager was still not aware of the issues the RDP consultants discovered in January 2018. Instead, the contract manager believed that the contractor had not had to repeat work on the contract. Despite these and other quality concerns, the contract manager for only one of the nine contracts we reviewed requested that a contractor submit a recovery plan—a document Authority policy directs contract managers to request if deliverables have fallen behind, do not meet contract requirements, or may require repeated work. Specifically, the contract manager for the RDP consulting contract requested a recovery plan in December 2017 for a single deliverable for the development of cost management software. This deliverable had delays spanning multiple years. The contract manager communicated his expectations for the recovery plan in a formal letter to the RDP consultants, stating that he expected the plan to include a schedule with detailed implementation activities, a list of all remaining project scope items and necessary resources, and a detailed mitigation strategy should the RDP consultants miss any milestones. The contract manager’s request was consistent with Authority contract management policies and procedures. However, because this example is the Authority’s only use of a recovery plan for any of the nine contracts we reviewed, we are concerned that the Authority may have missed other opportunities to address untimely or unsatisfactory deliverables. In fact, the RDP consultants missed deadlines for other deliverables for this same contract, suggesting problems may exist that the Authority has not actively tried to mitigate. The contract’s current Although the contract’s current work work plan was originally scheduled to be eight months, with plan was originally scheduled with the RDP consultants completing all deliverables by the end of the RDP consultants completing February 2018. However, as of January 2018, the RDP consultants all deliverables by the end of had reported submitting only 10 of 81 deliverables, despite February 2018, as of January 2018, spending nearly $70 million of the work plan’s $90 million budget. the RDP consultants had reported According to the contract manager, the Authority extended this submitting only 10 of 81 deliverables, work plan twice, for a total of seven months, because the RDP despite spending nearly $70 million consultants had not yet completed the deliverables. When it of the work plan’s $90 million budget. extended the work plan, the Authority also assigned additional deliverables and added funds to the work plan, now valued at $157 million. 52 California State Auditor Report 2018-108 November 2018 Although the contract manager asserted that the Authority expects the RDP consultants to complete outstanding deliverables with no additional resources, the Authority added dollar amounts with the extensions that do not appear proportional to the additional deliverables it assigned. For example, the most recent extension added nearly $30 million but just eight new deliverables. As of October 2018, the RDP consultants reported to us that they had submitted 101 of the 111 deliverables due by the work plan’s revised September 30, 2018 deadline. However, as we explain in the previous section, the contract manager provided acceptance notices for only 77 deliverables and has only recently begun proactively tracking the timeliness of the RDP consultants’ work. Overall, the work plan’s shifting deadlines and large dollar increases make us question why the Authority did not initiate the formal corrective actions that its policies and procedures indicate it should. Given the types of challenges Given the types of challenges some of the contracts we reviewed some of the contracts we reviewed have presented, we found the lack of formal intervention by have presented, we found the the Authority concerning. Most of the contract managers, and lack of formal intervention by even the RDP consultants who identified the quality issues that the Authority concerning. resulted in repeated work, insisted that there had been no need to establish recovery plans. Moreover, we identified evidence of formal invoice disputes for only two of the nine contracts we reviewed, and the disputed items generally had to do with incorrect billing rates or unsubstantiated costs for invoiced expenses, rather than deliverable quality. Some of the other contract managers explained the lack of formal intervention by asserting that when quality issues arose, they resolved them informally with the contractors. Although Authority policies allow for the informal resolution of issues as an alternative to formal documented disputes, the lack of documented deliverable review and tracking by contract managers, along with the deliverable delays we discussed above, creates the risk that the Authority is not detecting or resolving issues with contractor performance. Further, without the contract management documentation its policies and procedures require, the Authority cannot demonstrate that the hundreds of millions of dollars it has spent to date on these contracts—including for cost overruns—has been necessary and appropriate. Although the Authority Has Amended Many of Its Contracts, Contract Managers Have Not Sufficiently Documented Attempts to Control Costs or the Reasons for Overruns The Authority frequently amends its contracts to add additional time or funds. Of the nine contracts totaling more than $1.3 billion in our review, the Authority used amendments to increase the value of six by a total of $183 million and to extend the contract terms for five of those six. In addition, it amended two other contracts to California State Auditor Report 2018-108 53 November 2018 reduce their value by nearly $40 million when reassigning tasks and funds to other contractors. Only one contract that we reviewed, for financial advisory services, has no amendments. As Table 3 shows, the amendments increasing the contracts’ value represent significant additional costs. These amendments more than doubled the value of two contracts and increased the value of three others by more than 40 percent. Amendments have thus unquestionably contributed to the high-speed rail system’s cost overruns. Table 3 The Authority Has Added Large Amendments to Many of the Contracts We Reviewed TOTAL VALUE NUMBER OF ORIGINAL CURRENT TOTAL TIME ORIGINAL CONTRACTOR/SERVICE OF AMENDMENTS AMENDMENTS CONTRACT VALUE CONTRACT VALUE EXTENSION CONTRACT TERM Caltrans January 2013– $64,200,000 3 $225,900,000 $290,100,000 — State Road 99 construction June 2020 Wong+Harris, Joint Venture May 2013– 35,500,000 3 34,209,000 69,709,000 12 months Construction oversight firm December 2018 T.Y. Lin International February 2014– Bakersfield to Palmdale 26,558,000 3 46,100,000 72,658,000 24 months January 2019 regional consultant* Parsons Transportation Group December 2008– Central Valley Wye 25,640,000 6 55,000,000 80,640,000 63 months June 2014 regional consultant* Nossaman, LLP January 2009– 18,500,000 10 500,000 19,000,000 114 months Legal services† June 2011 HNTB Corporation January 2016– 12,800,000 1 30,064,000 42,864,000 1 month Construction oversight firm* November 2020 KPMG, LLP June 2016– — 0 40,000,000 40,000,000 — Financial advisors June 2020 WSP USA, Inc. July 2015– (33,630,000) 3 700,000,000 666,370,000 — RDP consultant‡ June 2022 Arcadis US, Inc. November 2014– (6,000,000) 1 71,885,000 65,885,000 — Construction oversight firm‡ April 2019 Totals $143,568,000 30 $1,203,658,000 $1,347,226,000 Source: Authority’s contracts and contract amendments. * The board recently approved amendments for these three contracts; although the amendments have not yet been officially executed, they are included in this table. † State law requires that the Authority, as a state agency, obtain written consent of the attorney general before contracting with outside counsel. The contract manager explained that the attorney general typically gives approval for outside counsel in two‑year increments, which has necessitated regular contract amendments for legal services. ‡ Two of these contracts received amendments that decreased their original values. The Authority moved $6 million from the Arcadis US, Inc. contract into an increase for the Wong+Harris, Joint Venture contract. The Authority removed $33.6 million from the WSP USA, Inc. contract as a result of removing certain tasks from the contract’s scope of work. 54 California State Auditor Report 2018-108 November 2018 The Authority designed its contract management policies and procedures, as well as its associated tracking requirements, to ensure that it identifies the need for changes in a timely manner and that it appropriately ensures those changes’ justification before adopting them as amendments. For example, when an involved party identifies a potential need for a change, such as a change in the scope of the contract or in required deliverables, the policies and procedures require the contract manager to include the change in the change tracking log; document who identified the need for it; describe the issue or potential change; and identify its proposed cost, its impact to the contract schedule, and the relevant dates in the process. The policies and procedures also require the contract manager to assess the potential change for merit and discuss this determination with an appropriate supervisor for approval. The Authority may also provide amendment details in a staff report to the board, which during the period of our review had to approve any amendments to existing engineering and architectural contracts with values that exceed $5 million. The Authority presented all of the proposed amendments we reviewed to the board. Despite these requirements, the contract managers for the contracts we reviewed could not consistently provide documentation demonstrating the Authority’s independent evaluations of potential amendments. Since April 2017, when the Authority established its new policies and procedures, it has approved 13 amendments for We reviewed three amendments, eight of the nine contracts in our review. We reviewed three of each for a different contract, and these amendments, each for a different contract. However, found that none of the contract when we tried to identify the details related to the amendments’ managers had maintained change necessity and size, we found that none of the contract managers tracking logs, despite a requirement had maintained change tracking logs. Further, when we asked for to do so. supporting documents with the information we expected to find in the tracking logs, the contract managers frequently provided the documentation that Authority staff had used to present the recommended amendments to the board. When we requested documentation showing how the Authority determined these three amendments had merit as the board materials attested, contract managers for two of the contracts had to request this documentation from the RDP consultants or from the contractors themselves. The Authority documented its analysis of the third amendment we reviewed, but that analysis was incomplete. An example involving a $3 million amendment with a one-year extension demonstrates why relying on contractor-provided evidence is problematic. When we asked about this amendment, the contract manager first provided us with the formal high-level amendment request that the former contract manager signed. According to Authority procedures, a contract manager should sign this form only after determining that the proposed amendment has California State Auditor Report 2018-108 55 November 2018 merit. The contract manager also provided the staff report that the Authority presented to the board when the board considered the amendment. The staff report asserted that a number of unforeseen changes had impacted the scope, budget, and overall schedule of the contract’s work, precipitating the need for the amendment. When we asked for evidence supporting the claims that the board documents made regarding the amendment’s merit, the current contract manager provided additional documentation that the Authority received from an RDP consultant working with the contractor. This documentation showed that the contractor was the source of information regarding the amendment’s necessity and accompanying costs, and it included no evidence that the Authority independently verified the contractor’s claims. In fact, the content in the Authority’s report to the board came directly from the contractor’s amendment request. When it does not independently verify and document the need for contract amendments, the Authority risks authorizing additional funds for unnecessary or unwarranted changes. Contract managers were also unable to demonstrate if and how Contract managers were also they independently determined that the approved amendment unable to demonstrate if and how costs were appropriate and justified. When we reviewed another they independently determined that amendment, we found that the contract manager had originally the approved amendment costs drafted an amendment request for $26.7 million based on the were appropriate and justified. contractor’s estimates for the cost of land acquisitions and utility relocations. The contractor later determined that it required an additional $2.5 million, causing the contract manager to increase this amendment proposal to $29.2 million, which the board ultimately approved. When we questioned how the Authority had evaluated the need to add the $2.5 million or the appropriateness of the $29.2 million total cost, the contract manager stated that the contractor had increased the amount of additional funds it claimed to need after further consideration and analysis. However, the contract manager was unable to provide documentation showing the Authority had independently determined the accuracy of the amendment’s costs either before or after the contractor increased them. For the third amendment we reviewed, the Authority documented a justification for why it required the amendment, but its analysis was incomplete. Specifically, when one of the oversight firms that manage the Authority’s construction contracts was projected to deplete its contract funds nine months ahead of schedule, Authority staff requested that the board approve a third amendment for $28.5 million to retain the oversight firm’s services for an additional year. The amendment documents the Authority presented to the board indicate that the Authority considered seeking a new vendor to replace the oversight firm, but it decided to recommend the amendment instead. The Authority’s documents assert that the 56 California State Auditor Report 2018-108 November 2018 need for the amendment stemmed from its assigning work to the oversight firm that was outside of the original contract because of changes to the construction contract. However, the contract manager could not tell us how much money went to the oversight firm for performing out-of-scope work and acknowledged that he had not documented the oversight firm’s adequate performance, as we discuss in more detail later. The board expressed concerns about substantial cost increases, stating that the Authority generally had some issues in the past with oversight of construction management, and it directed Authority staff to return in 90 days to update it on the oversight firm’s progress in fulfilling the objectives of the contract. However, it also unanimously approved the amendment. When staff reported back, they presented the board with a template they planned to use to evaluate the three oversight firms’ performance in the future. Our analysis of the amendment documents indicates that future cost overruns may be likely for this contract. In the documents, Authority staff calculated that retaining the existing oversight firm was more cost-effective because of its familiarity and experience with the project. In reaching this conclusion, the staff compared the amendment amount to what they indicated was the remaining value of the work still to be completed for the corresponding construction contract. However, when making this comparison, the Authority failed to account for a significant amount of expected future construction costs. The amendment documents, dated March 2018, projected that $545 million of additional construction work would remain as of July 2018. However, a status report from July 2018 shows $676 million remaining, for a total contract value of $1.5 billion. Further, cost projections from December 2017, three months before the amendment discussion, indicated that the construction contract would ultimately cost more than $2 billion due to expected changes. The Authority’s failure to account for this expected work suggests that it may need additional amendments for the oversight firm’s contract in the future. If the Authority continues to account for expected construction cost increases only after those increases take effect, it must develop a process for tracking the actual impacts of those increases on its oversight firms to ensure their spending is reasonable and to reduce the likelihood that the contracts run out of funds prematurely. Although the Authority’s Construction Contracts Include Potentially Effective Methods for Verifying Progress and Costs, Those Methods Require Improved Oversight The Authority has separate management structures to oversee its three active construction projects. As we discuss in Chapter 1, the Authority has assigned an oversight firm to manage—under its California State Auditor Report 2018-108 57 November 2018 direction—each of the three current construction contracts. In addition, the Authority has assigned a contract manager to be its authorized representative for each construction contract and to also manage the pertinent oversight firm’s contract. Because managing its construction contracts is distinctly different than managing its other contracts in terms of how it measures and pays for work, the Authority has developed requirements specific to this process. Specifically, the Authority’s policy requires oversight firms to provide primary, day-to-day oversight of the construction contractors’ progress in meeting contract requirements. Because the current construction contracts include provisions that require the contractors to base their invoices on their progress instead of on the costs they incur, the total amount each construction contractor can earn is the total value of the contract— inclusive of contract increases due to change orders—regardless of the contractor’s actual costs to perform the work. The Authority tasks the oversight firms with evaluating and verifying the construction contractors’ claimed progress as part of reviewing the contractors’ monthly invoices. When an oversight firm has verified a construction contractor’s progress, it communicates that fact to the Authority’s contract manager, who then approves payment based on the agreed-upon value of the work as identified in the invoice materials. The Authority’s construction invoicing approach has the potential The Authority’s construction to significantly limit the risk of it overpaying the construction invoicing approach has the contractors, but the extent to which the Authority can rely on it potential to significantly limit the as a cost control tool depends on how well the Authority oversees risk of it overpaying the construction the work of the oversight firms. When we reviewed the invoicing contractors but depends on how processes for the three construction contracts, we determined that well the Authority oversees the work the invoicing documents contained the required documentation of the oversight firms. to allow the oversight firms to evaluate construction progress. However, as a matter of practice, the Authority’s contract managers stated that they rely on formal review and approval from the oversight firms without reviewing additional information to independently verify the accuracy of the invoices. Determining whether the oversight firms have effectively performed their roles would require the Authority to actively monitor the oversight firms and to document that monitoring. However, the contract managers are only now beginning to institute this type of active monitoring. In early 2018, in response to board concerns, the contract managers for the three oversight firms developed a performance evaluation template for the oversight firms, as we discuss above. Authority staff presented the preliminary performance evaluation template to the board in June 2018, but this presentation did not include actual evaluation data. Instead, the contract managers stated that they expected to conduct formal evaluations using the template on a quarterly basis moving forward. 58 California State Auditor Report 2018-108 November 2018 The Authority has not yet established any formal methodology or procedures for using the performance evaluation template. For example, the template asks contract managers to rank the oversight firms’ performances on a scale from one (poor) to five (excellent) across a range of categories, but it does not make clear how contract managers are to determine the scores. The fact that three contract managers will use the template while monitoring different oversight firms creates the risk that their evaluations will be inconsistent. Although the contract managers have discussed setting specific criteria for determining how they will assign values, they have not yet done so. Further, the contracts director acknowledged that Because the Authority’s oversight as of September 2018, the Authority is still evaluating the initial structure for construction contracts methodology and that it plans to monitor monthly trends shown relies heavily on the oversight through the reports. Because the Authority’s oversight structure firms, it is crucial that the Authority for construction contracts relies heavily on the oversight firms, provide a clear methodology it is crucial that the Authority provide a clear methodology for for consistently evaluating the consistently evaluating the oversight firms’ performance. oversight firms’ performance. The Authority’s construction contract management structure also does not mitigate the risks of cost overruns by the oversight firms. In fact, two of the three oversight firms expended their funds ahead of schedule and needed substantial contract amendments to continue performing their oversight duties. When we asked the pertinent contract managers about the reasons for these cost overruns, they referred to changes that the Authority had made to the construction contracts that the firms oversee. These change orders, which we discuss in Chapter 1, added significant costs and time to the construction contracts and thus—according to the contract managers—caused significant additional work and expense for the oversight firms. However, the Authority did not amend the oversight firms’ scopes of work and funding contemporaneously with these changes. Rather, as we discuss in the previous section, the Authority amended one of the oversight firms’ contracts to add $28.5 million when it was projected to deplete its contract funds nine months ahead of schedule. Recommendations To improve its contract management, increase accountability, and justify the significant amount it pays for contracted services, the Authority should take the following steps by May 2019: • Prioritize contract management efforts and reduce the frequency with which contract management responsibilities shift among Authority staff by establishing a formal process for hiring and assigning full-time, experienced contract managers. These contract managers should have duty statements reflecting their contract oversight responsibilities, and they should report to supervisors who understand those responsibilities and California State Auditor Report 2018-108 59 November 2018 have extensive knowledge about the contracts’ deliverables. In addition, those supervisors’ duty statements should clearly lay out their responsibility for addressing any contract manager noncompliance with the Authority’s contract management policies and procedures, whether reported by CMSU or identified by another means. • Require CMSU to establish a schedule to monitor individual contract manager compliance and report annually the results of this monitoring to Authority executive leadership. To help ensure the integrity of its oversight role, CMSU should be composed of state staff in place of RDP consultants. • Hold contract managers accountable for performing the duties that the Authority’s policies assign to them. Specifically, CMSU and, to the extent necessary, contract managers’ supervisors should require and review evidence from contract managers demonstrating their approval of deliverables, detection and resolution of contractor performance issues, and assessment of contract amendments for merit. The Authority should not accept observations and reports from its contractors or the RDP consultants in place of this evidence. To prevent the inappropriate use of contractors to perform state functions, the Authority should develop procedures by May 2019 for evaluating whether new and existing administrative duties should be assigned to contractors or to state employees. To ensure that contract managers’ invoice reviews are complete and that invoiced costs are allowable under contract terms, the Authority should amend its applicable procedures by May 2019 to require contract managers to document their review of invoiced rates and expenses. To ensure the consistency and effectiveness of its efforts to monitor the performance of the oversight firms with which it contracts, the Authority should develop a formal methodology by May 2019 for using the performance evaluation tool it has implemented. This methodology should include procedures for assessing the sufficiency of the oversight firms’ review and approval of invoices for construction contracts. To ensure that the oversight firms’ spending is reasonable, the Authority should develop a formal process by May 2019 for tracking any out-of-scope work that the oversight firms perform. To reduce the likelihood that its contracts with the oversight firms run out of funds prematurely as a result of this additional work, the Authority should also develop a formal process for amending the oversight firms’ contracts contemporaneously to change orders that significantly extend the timelines or increase the scope of work of the construction contracts that oversight firms oversee. 60 California State Auditor Report 2018-108 November 2018 California State Auditor Report 2018-108 61 November 2018 Chapter 3 THE AUTHORITY CAN IMPROVE THE QUALITY AND TRANSPARENCY OF ITS MONITORING AND REPORTING FOR KEY GOALS Chapter Summary As one of the most expensive transportation projects in the United States, the construction of the high-speed rail system has significant implications for the State’s environment, its small and disadvantaged businesses, and its economy. However, we identified several ways that the Authority can better measure and report on these impacts. The Authority’s sustainability policy has relevant goals to limit the negative environmental impacts of the high-speed rail system, but it does not sufficiently focus on the environmental impacts of construction activities. Additionally, the Authority has not collected complete and accurate data on the environmental impact of its construction activities, and thus far it has not comprehensively measured construction impact trends and set targets for future construction. The Authority reports consistently on its contracting with small, disabled veteran, and disadvantaged businesses, having set goals for the percentage of its total expenditures that will go to those businesses. However, its reporting could be more complete and transparent, as that reporting has omitted $930 million in contracts. In contrast, although the Authority faces some limitations that may affect the precision with which it estimates the overall economic impact of its activities in the Central Valley and elsewhere, it has appropriately disclosed these limitations as part of its reporting. The Authority Can Better Account for the Environmental Impact of the System’s Construction by Strengthening Its Sustainability Policy, Monitoring, and Measurement According to its sustainability policy, the Authority intends its approach to the design, construction, and operation of the high-speed rail system to contribute to a more sustainable California. The policy also states the Authority’s commitment to employing leading edge construction methods to make the project a model for future rail infrastructure. Described by the Authority as “all-encompassing,” the sustainability policy is supposed to guide the system’s energy and natural resource use, impact on local communities, construction practices, and operations. State and federal commitments mandate certain aspects of the Authority’s focus. For example, Proposition 1A—which provided funding for the high-speed rail system—requires the Authority to plan 62 California State Auditor Report 2018-108 November 2018 and construct the system in a manner that minimizes the impact on the natural environment. Further, in a 2010 memorandum of understanding with the federal government, the Authority pledged to be environmentally conscious throughout the design, construction, and operation of the system. Other aspects of the Authority’s sustainability policy, such as encouraging transit development in local communities, are compatible with legislative priorities for sustainable transportation planning. Although the Authority’s sustainability policy includes goals consistent with best practices, it also has shortcomings that limit its effectiveness. According to the expert we retained to assist us in assessing the sustainability policy, it includes valuable objectives that generally align with established best practices. However, our expert found the policy does not sufficiently distinguish between construction of the system—which has a significant impact on the State’s environment—and its eventual operation. Some of the policy’s priorities, such as reducing car and other vehicle travel, clearly focus on the effects of system operations and have no direct construction implications. However, the Authority has not identified construction-related objectives for all of its priorities even though the priorities themselves have relevance during the construction stage. For example, although the policy lists conservation of nonrenewable energy as a priority, the related objectives pertain only to the system’s operations. Our expert noted that best practices concerning this priority, including those from the Global Reporting Initiative—with whose standards the Authority claims compliance—require a project to consider the total amount of energy used during construction.6 Our expert observed that because the policy does not consistently and explicitly address the impacts caused by the construction phase of the system, the Authority’s implementation plan—which details how it will assess compliance with the policy—is not always specific about what the Authority should measure during construction in order to determine success. Further, the implementation plan The Authority’s implementation does not include measurable, process-focused metrics related to plan does not include measurable, construction for many of the objectives. For example, the plan states process‑focused metrics related that the Authority will monitor the degree to which the system’s to construction for many of the eventual operation improves air quality by tracking the number objectives in its sustainability of emergency room visits for asthma sufferers; however, the plan policy; this lack of actionable does not include a metric to measure the degree to which the detail makes ensuring the system’s construction affects current air quality. Our expert concluded that current and future compliance with this lack of actionable detail makes ensuring the system’s current the policy’s goals challenging. and future compliance with the policy’s goals challenging. 6 The Global Reporting Initiative is an international organization that develops sustainability standards that many of the world’s largest corporations use. California State Auditor Report 2018-108 63 November 2018 Because construction is significantly underway in the Central Valley, we asked our sustainability expert to review best practices for monitoring sustainable construction and compare those practices to the Authority’s plans and actions. After reviewing 13 comparable infrastructure projects as well as guidelines published by the American Public Transportation Association, our expert determined that current industry standards call for organizations to estimate the material impacts resulting from construction before beginning projects. Organizations should then establish specific goals and—once construction has begun— measure actual progress against those goals to determine where they have been successful. The Authority has set initial estimates for some construction impacts, but it has not comprehensively measured actual progress against those estimates. Before beginning construction on the system, the Authority estimated the level of greenhouse gases and other pollutants that construction activities would emit. Our expert reviewed the Authority’s calculations and found them to be reasonable and in line with guidance from the California Air Resources Board. Further, the Authority required each of its construction contractors to submit estimates of impacts, such as greenhouse gas emissions, within sixty days of receiving approval to begin work. Relying on contractors to determine these estimates presents a risk that they may overestimate emissions, but our expert concluded that doing so is a standard industry practice. Although it properly completed initial estimates, the Authority Although it properly completed did not ensure the accuracy of subsequently collected sustainability initial estimates, the Authority data. The Authority relies on each of its construction contractors did not ensure the accuracy to self-report information on their sustainability performance— of subsequently collected such as their usage of heavy equipment and water—into a central sustainability data, which could database. The oversight firms for the three current construction limit the Authority’s ability projects then review these submissions and pass them on to to reliably compare actual the Authority’s sustainability unit, which RDP consultants lead performance against estimates. and almost completely staff. Despite these protocols, when we attempted to validate a selection of nine database entries against supporting documentation, we found that the Authority could provide sufficient supporting documentation for only three entries. We therefore were unable to determine the accuracy of the Authority’s data. An inability to ensure accurate data could limit the Authority’s ability to reliably compare actual performance against estimates. Further, our expert found that the Authority has not comprehensively evaluated the sustainability performance of the currently active construction projects. Neglecting to monitor all pertinent aspects of performance continuously throughout construction could result in the individual construction projects falling short of their goals; 64 California State Auditor Report 2018-108 November 2018 if it reviews progress only after it completes a specific project, the Authority will have missed any opportunity to intervene in order to improve sustainability outcomes. The Authority provided documentation showing that it is tracking two environmental impacts, greenhouse gas emissions and other air pollutants, against benchmark estimates. However, that documentation did not include equivalent comparisons related to the environmental impact of waste produced from construction, despite the fact that each construction contractor provided a benchmark estimate for waste. In addition, our expert observed that the documentation the Authority provided also does not allow it to effectively project whether contractors will meet or exceed their estimates because it does not account for actual construction progress to date. Finally, we noted that the current construction contracts do not require contractors to estimate the environmental impact of water usage for the project, and as a result, construction contractors did not provide benchmark estimates for water use. Evaluating the sustainability impact of the system’s construction on an ongoing basis is also critical because it will enable the Authority to set standards for future construction. The Authority has not yet developed a systemwide baseline or identified a universal metric against which to anchor future construction projects, which may differ in scope and type from current projects. For example, the Authority’s plans for completing the system call for significant tunneling, which it has not yet attempted. Although its sustainability report from 2016 stated that the Authority planned to adopt 2015 as its baseline year, the Authority has not moved forward with this plan. Therefore, it is not yet prepared to hold future construction contractors to a baseline it has established using current construction activities. The Authority’s sustainability director told us that her team is in the process of using recently completed systemwide construction plans and current construction data to develop a metric that will allow the Authority to set standards for future projects. She stated that the Authority plans to complete this process before it enters into any additional construction contracts, but that the process is complex and will be challenging. Although the Authority Reports Regularly on Its Utilization of Small, Disabled Veteran, and Disadvantaged Businesses, It Excludes $930 Million in Contracts From Its Reporting State law, regulations, and policy, as well as federal regulations, mandate that the Authority report on its contracting activity with small businesses, disabled veteran owned businesses, and disadvantaged businesses. In particular, executive orders require certain state agencies, including the Authority, to establish California State Auditor Report 2018-108 65 November 2018 processes to meet a small business participation goal of 25 percent and a Disabled Veteran Business The Authority’s Small Business Program Categories Enterprises (DVBE) participation goal of 3 percent. Additionally, as a condition of receiving federal Small Business: State law defines small businesses as financial assistance from the Federal Railroad independently owned and operated businesses that Administration, the Authority must report on its are located in California and whose officers also live in actual use of Disadvantaged Business Enterprises California. Small business must have 100 or fewer employees (DBEs), with a goal of 10 percent utilization. The and average annual gross receipts of $10 million or less. Authority has established its own small and Of these small businesses, a business is a microbusiness if disadvantaged business enterprise program (small it has 25 or fewer employees and average gross receipts of business program) that, as the text box describes, $2.5 million or less. includes small and microbusinesses, as well as Disabled Veteran Business Enterprise (DVBE): State DVBE and DBE firms. The Authority established an regulations define DVBEs as businesses that are at least overall 30 percent participation goal for its small 51 percent owned and controlled by one or more business program participants, inclusive of the disabled veterans who live in California. 10 percent DBE goal and 3 percent DVBE goals. Disadvantaged Business Enterprise (DBE): Federal regulations define DBEs as for‑profit, small businesses To determine the extent to which the Authority that are at least 51 percent owned and controlled by contracts with these types of businesses, we one or more individuals who are both socially and reviewed the small business utilization reports economically disadvantaged. (utilization reports) that the Authority posts Source: California Government Code section 14837(d); on its website and provides to the Federal California Code of Regulations, title 2, section 1896.81; and Railroad Administration. These reports include Federal Code of Regulations title 49, section 26.5. the Authority’s actual utilization rates for 37 of its professional services contracts as well as its three construction contracts, with a total contract value for all 40 contracts of just over $4 billion. When we reviewed 10 of these contracts, we found that the documentation the Authority had collected from contractors sufficiently supported the utilization that the contractors reported. The documentation also supported the utilization percentages that the Authority reported for the six of these contracts that were active at the time of our review. In its most recent report, which it issued in June 2018, the Authority reported a nearly 30 percent actual utilization rate—inclusive of small businesses, DVBE firms, DBE firms, and microbusinesses— for its professional service contracts and a utilization rate of almost 16 percent for its construction contracts. Although both state and federal programs require reporting, neither of their reporting formats fully capture the extent of the Authority’s contracting activities with relevant businesses. For example, in its role administering the State’s small business and DVBE programs, the Department of General Services (DGS) instructs state departments to exclude contracts with federal and state entities, as well as contracts with any counties or cities. Furthermore, DGS requires reporting only on the dollar amounts awarded to small businesses and DVBE firms, rather than actual expenditures; thus, its reporting requirements do not provide the Authority’s actual impact on these businesses. In contrast, the federal Department of 66 California State Auditor Report 2018-108 November 2018 Transportation and the Federal Railroad Administration instruct the Authority to report on commitments, awards, and payments to DBE firms for federally funded contracts. According to the Authority’s contract compliance administrator, its quarterly utilization reports list active contracts that are at least partially paid for with federal funding and that are subject to small business utilization goals. However, the utilization report does not include federally funded contracts for highly specialized services, such as expert witness services, for which the Authority has asserted it cannot find small business contractors. We reviewed the Authority’s utilization report and found that it As of January 2018, two‑thirds of excluded $930 million of the Authority’s contracts. As of January 2018, the contracts that the Authority two-thirds of the contracts that the Authority excluded were with excluded in its utilization report public entities, such as California cities and counties. These contracts were with public entities, which represent $627 million in contract value and $326 million in actual represent $627 million in contract expenditures. The utilization report also excludes $303 million of value and $326 million in contract value for services from private entities. Given the limitations actual expenditures. and exclusions inherent in each reporting format, the Authority does not currently release a report that includes its total small business program utilization out of its total contracting dollars. Because the Authority does not mention that it exempts contracts with public entities on its reports—including its quarterly utilization reports, business plans, and economic impact reports—we asked the Authority about its reasons for not reporting on these contracts. The Authority’s recently appointed chief administrative officer explained that its contracts with public entities and utilities generally do not contain small business provisions primarily because they are not the result of a competitive bidding process. Instead, the Authority enters into contracts with these entities because of specific program needs that only the public entities or utilities can serve. An example is contracting with a city or a county for permitting activities in its jurisdiction. In such instances, the Authority has no bargaining power to compel the public entities to participate in its business utilization program. The chief administrative officer also stated that DGS specifically instructs state entities not to include these types of contracts in their small business and DVBE reporting, as we previously describe. Nonetheless, we believe that the Authority could better account for its contracts with these public entities in its reporting. For example, Caltrans—with whom the Authority has a $290 million contract—participates in the Authority’s small business program. However, the Authority does not include this contract on its utilization report. When we asked the Authority why it does not include the Caltrans contract on its utilization report, a contract compliance administrator stated that it had not received utilization reporting from Caltrans on a consistent enough basis to allow the California State Auditor Report 2018-108 67 November 2018 Authority to confidently include the agency in its reporting. The contract compliance administrator asserted that as it receives more information from Caltrans, the Authority will incorporate Caltrans’ small business utilization rates into its reports, as well as monitor Caltrans’ small business utilization through to the contract’s completion. The Authority provided a different reason for not reporting on all of its contracts with private entities. Although it is required to include all of its contracts with private entities on its quarterly utilization reports, those reports have not included $303 million in such contracts. The chief administrative officer stated that the Authority exempts certain contracts because the scopes of the work are too specific for it to require the contractors to be small, DVBE, or DBE businesses or to subcontract with such businesses. For example, many of the contracts omitted from the report are for expert witness or legal services. The chief administrative officer also explained that partly due to our inquiries, she realized that the Authority has no written policy explaining these exemptions or the process by which it determines whether to include these provisions in its contracts. She stated that she intends to oversee the development of such a policy. Although there are few consequences if the Authority or its contractors do not meet their utilization goals, the emphasis on reporting in both state and federal requirements suggests that transparency and accountability related to contracting practices are key principles of the business utilization programs. However, by limiting the contracts it includes in its public reporting, the Authority is not accurately reflecting the proportion of its total expenditures that go to these businesses. We believe that given We believe that given the the magnitude of the Authority’s contracting, the public would magnitude of the Authority’s be best served by a higher degree of transparency in its reporting. contracting, the public would An important part of increasing that transparency would be for be best served by a higher the Authority to disclose both the total value of its contracts degree of transparency in its as well as the extent to which it has exempted contracts from reporting of its utilization of small business requirements. The Authority’s reasons for making small, disabled veteran, and those exemptions—whether its own prospective policies or state disadvantaged businesses. guidelines—would be an important part of that disclosure. The Authority Followed Industry Standards When Estimating Its Economic Impact and Has Adequately Disclosed the Limitations of Those Estimates The Authority used two widely accepted economic modeling programs to measure the total economic impacts of its spending from its contracting and construction activities. In September 2017, the Authority issued a retrospective report regarding the 68 California State Auditor Report 2018-108 November 2018 total economic impact from its spending activities from fiscal years 2006–07 through 2015–16, which it estimated to be between $3.5 billion and $4.1 billion. In its 2018 business plan, it updated this information to include fiscal year 2016–17, estimating that its economic impact increased an additional $1.6 billion to $1.8 billion during this time. Although we identified some inconsistencies between the data that the Authority used for its economic modeling and the documentation supporting those data, these inconsistencies were relatively minor. To determine the accuracy of the Authority’s data, we randomly selected and reviewed 58 data entries, including expenditure amounts and geographic locations at the zip code level, and we verified the information’s accuracy by comparing it to the original invoices or other expenditure data sources. We identified certain inconsistencies: for example, the Authority attributed some expenditures to incorrect counties, which affected the Authority’s estimations of jobs in a particular county. In other instances, the Authority explained that due to incomplete expenditure information, it used higher-level financial or past geographical data to make assumptions about the size of expenditures, which could affect the precision of the dollar amounts it reported. Nonetheless, given the magnitude of the total amounts that the Authority reported and the fact that the amounts are intended to be estimates, we are not concerned that the Authority’s economic impact reporting is misleading or substantially under- or overstated. Further, the Authority disclosed the assumptions it made when designing its methodology by discussing those assumptions in the public technical memorandum that accompanied its reports. Recommendations To help improve the effectiveness of its sustainability policy, the Authority should revise the policy by May 2019 to more clearly differentiate between the construction and operation phases of the high-speed rail system. Further, it should ensure that each objective in each section of the policy is associated with quantifiable metrics for evaluating implementation. To allow it to evaluate the sustainability of the high-speed rail system’s construction, the Authority should, by May 2019, perform and document a review of its compliance with its existing quality controls related to ensuring the validity and completeness of contractor-reported data. The Authority should also establish a formal process to perform such reviews periodically. California State Auditor Report 2018-108 69 November 2018 To help ensure that it meets its sustainability goals, the Authority should comprehensively compare the three construction projects’ performances to their construction contractors’ original baseline estimates on a quarterly basis. It should perform the first of these comparisons no later than May 2019. To help ensure that its contractors’ proposed environmental impacts are reasonable and to measure the progress of its sustainable construction efforts over time, the Authority should, by November 2019, identify and track standardized measures—such as project miles—that will allow it to compare construction impacts across the high-speed rail system’s different construction projects. To increase the transparency of its reporting, the Authority should, by May 2019, expand its quarterly small business, DVBE, and DBE utilization reporting to account for the total value of all its contracts and to identify the reasons it has exempted specific contracts. We conducted this audit under the authority vested in the California State Auditor by section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives specified in the Scope and Methodology section of the report. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, ELAINE M. HOWLE, CPA California State Auditor November 15, 2018 70 California State Auditor Report 2018-108 November 2018 California State Auditor Report 2018-108 71 November 2018 Appendix Scope and Methodology The Joint Legislative Audit Committee (Audit Committee) directed the California State Auditor to perform an audit related to the Authority’s costs and contracting processes, as well as several other audit objectives. Table A below outlines the Audit Committee’s objectives and our methods for addressing them. Table A Audit Objectives and the Methods Used to Address Them AUDIT OBJECTIVE METHOD 1 Review and evaluate the laws, rules, and Reviewed relevant laws, regulations, policies, and procedures. regulations significant to the audit objectives. 2 Review and assess the Authority’s policies, • Reviewed prior audit findings from our office and the Authority’s internal audits office procedures, and processes for managing related to contract management policies, procedures, and practices. contracts and containing costs for the project, • Reviewed the Authority’s 2017 revised contract management policies and procedures. including its processes for tracking, reviewing, and paying contractor invoices. • Identified areas of the 2017 policies and procedures most closely related to controlling costs and ensuring value. We compared those areas to the Authority’s previous policies and to the State Contracting Manual. • Selected nine active environmental, engineering, or professional services contracts with a total value of $1.3 billion and reviewed the contract managers’ compliance with the Authority’s policies and procedures. We also reviewed the Authority’s management of its three active construction contracts, with a total value of $3.1 billion. • When necessary, reviewed contract materials provided by persons other than contract managers, such as other Authority staff, outside consultants, or the contractors themselves. 3 Evaluate the Authority’s process for reviewing and • Reviewed Authority construction data and documentation to determine the number approving design‑build contract change orders. and value of the change orders it has executed to date and of those it expects to execute before the completion of the Central Valley construction. • Judgmentally selected 16 change orders, including three rejected changes. We reviewed supporting documentation to determine if the Authority followed the processes outlined in its policy for processing change orders. • Analyzed the Authority’s future cost estimates to determine the major risk areas contributing to executed and expected change orders. We interviewed Authority staff and reviewed documentation to assess how the Authority’s decisions contributed to changes. • Analyzed the Authority’s baseline schedules and other planning documents to determine the current and expected effects of change orders on the system’s cost and schedule. 4 To the extent possible, review and evaluate the • Reviewed the Authority’s reporting on the economic impact of its expenditures from Authority’s efforts to determine the economic July 2006 through June 2017. impact the project has had on communities in • Reviewed documentation to determine the extent to which the Authority supported its those areas where construction is underway. reported expenditures. continued on next page . . . 72 California State Auditor Report 2018-108 November 2018 AUDIT OBJECTIVE METHOD 5 Determine the extent to which the • Reviewed state laws and other state and federal criteria related to utilizing small, Authority contracts with small and disadvantaged, and disabled veteran owned businesses. disadvantaged businesses. • Selected 10 contracts included on the Authority’s utilization reports and reviewed supporting documentation. • Identified the number, type, and value of contracts for which the Authority does not report its utilization of small, disadvantaged, and disabled veteran owned businesses. 6 Review the Authority’s sustainability policy and • Reviewed the Authority’s sustainability policy and implementation plan. assess its compliance with the policy. Assess the • Reviewed nine entries from the Authority’s sustainability database to assess the extent Authority’s efforts to evaluate the economic and to which the data are valid and supported by required documentation. environmental outcomes of its policy. • Engaged a sustainability consulting firm to evaluate the Authority’s sustainability policy, implementation plan, reporting, and progress toward goals in comparison to industry best practices and other, similar projects. 7 Determine whether there are opportunities • Reviewed the Authority’s business plans to determine how the system plans have for the Authority to expedite the project and evolved over time to increasingly include blending with existing infrastructure. reduce costs through cooperation with other • Interviewed Authority staff and reviewed Federal Railroad Administration regulations transportation entities, such as other transit or to determine how blending is likely to affect rail service operations. rail lines, or through capturing additional value through construction of project facilities. • Reviewed the Authority’s environmental and engineering planning documents to determine when the Authority made key decisions about blended infrastructure. • Compared the Authority’s system cost estimates over the past six years to assess how blending has led to reduced costs. • Interviewed staff and reviewed documentation to determine whether additional options exist to blend elsewhere in the system. • Reviewed academic and industry materials on opportunities to capture additional value through construction of project facilities and determined that the Authority has worked with cities to evaluate their feasibility, but is still in the early stages of this planning. 8 Review and assess any other issues that are We did not identify any additional issues that are significant to the audit. significant to the audit. Source: Audit Committee’s audit request number 2018‑108, planning documents, and information and documentation identified in the table column titled Method. Assessment of Data Reliability In performing this audit, we relied on electronic data files we obtained from the Authority’s oversight firms related to the dollar amount and nature of changes to the Authority’s construction contracts. The GAO, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of any computer-processed information we use to support our findings, conclusions, or recommendations. To perform this assessment, we evaluated the oversight firms’ data against sources of corroborating documentation from both the Authority and its construction contractors. We determined that the data was sufficiently reliable for the purposes of summarizing change orders in this report. California State Auditor Report 2018-108 73 November 2018 * * California State Auditor’s comments begin on page 85. 74 California State Auditor Report 2018-108 November 2018 October 22, 2018 Brian Annis, Secretary California State Transportation Agency 915 Capitol Mall, Suite 350-B Sacramento, CA 95814 Dear Secretary Annis: The California High-Speed Rail Authority (Authority) appreciates the opportunity to respondto the California State Auditor (CSA) draft audit report issued on October 15, 2018. The CSA conducted this audit at the request of the Joint Legislative Audit Committee. The audit’s scope addressed the efficiency and efficacy of the policies and practicesemployed by the Authority. We concur with and willwork to implement the CSA’srecommendations as an integral part of the Authority’s commitment to excellence and continuous improvement. We are pleased to report that we have begun implementing these and other corrective actionsto remedy the issues identified. The Authority is making progress on delivering the Californiahigh-speed rail system. The Authority’schallenge has beento evolve its organization and improve its program management processes, while concurrently deliveringthis complex mega-program. We are working to meet this challenge through a continuous improvement process by whichwe methodically and regularly identify and apply lessons learned. Through this ongoing process, we are taking systematic steps to expedite the Authority’s transition to a more rigorous program management and delivery organization. This evolution included establishing new governance structures in mid-2017--that we further strengthened in 2018--to more effectively manage the program through highlystructured configuration management and change processes. To advance and expedite this organizational transition, the Board of Directors recruited new leadership in 2017. The Board appointed me as the Authority’s Chief Executive Officer and my tenure began in February 2018. I was joined immediately thereafter by two newlyappointed senior executives Iselected to support me in leading this organization –a new Chief Operating Officer and a Chief Deputy Director. Those individuals brought specific skill sets in construction and program delivery and in administrative management. Together, we and the rest of the senior executive team take the audit’s recommendations seriously. My senior executive team has reviewed the audit and is working with staff to address the CSA’s recommendations. This will include conducting root cause analyses and implementingadditional corrective actions beyond the audit’s recommendations,if necessary. We regard this as an opportunity to further solidify the systemic improvements that the Authority has continued to establishandthat we have strengthened over the last year. Also, theCSA’s auditcoincides with the Authority’s development of three key foundational governance documentsthat are essential to our ongoing processof continuous improvement. 770 L Street, Suite 620,Sacramento, CA 95814 • T: (916) 324-1541 • F: (916) 322-0827 • www.hsr.ca.gov California State Auditor Report 2018-108 75 November 2018 Secretary Brian Annis Page 2 October 22, 2018 First, in May 2018, the Board of Directors adopted the 2018 Business Plan,which lays out the Authority’s implementation strategy for delivering California high-speed rail. Second, in June, the Board adopted our 2018 Program Baseline, which is an essential management document that outlines the scope, schedule and budget for the Authority Program’s early delivery objectives. Third, inOctober 2018 we finalized our Project Management Plan(PMP), which clarifies our integrated organizational framework and presents a governance structure stressing program management and delivery. As detailed below, we are actively addressing the audit’s recommendations through implementation and other remedial steps. As it has done in the past, the Authority will move swiftly to fully implement the recommendations of the State Auditor that bear on the efficacy of program delivery. We hope and trust that our rapid implementation of these recommendations will bolster confidence in the Authority’s commitment to active management and continuous improvement of this most important program. The CSA recommendations and the Authority’s responses (in bold) are as follows: Chapter 1: The Authority’s Decision to Begin Construction Before Completing Proper Planning Led to Cost Overruns and Delays 1. To ensure that the change orders it approves are necessary and that theircosts are appropriate, the Authority should adhere to the guidance and estimates the oversight firms provide to it. If the Authority chooses to deviate from the oversight firms’ recommendations, it should clearly document why it made those deviations. Response: The Authority concurs with this recommendation.In 2017, the Authority initiated a governance process to assess the construction, financial, legaland other programperspectivesfor all changes. The Board of Directors adopted the 2018 Business Plan that included the new governance (see page 63of the Plan)and adopted the programBaseline in June 2018. The Authority then updated and formalized this process in the Program Management Plan (PMP). Thecurrent process includes:aProgram Delivery Committee,which is a management committee that holds the functional groups accountable for program delivery and evaluates all pending and potential change orders;and aBusiness Oversight Committee, whichacts as a change control committee and must approve all change orders before they go to the Executive Committee and,when needed,the Board of Directors(Board)for approval to execute. Each governance committee has an approved charter that outlines itspurpose and decision- making authority. In accordance with the CSArecommendation, the Authority will revise the charters to require documentation when a governancecommittee overrules a Project Construction Management (PCM) firm’srecommendation. This will be accomplished through a Business Case, which is the document forrequestinga proposed change order or achange to the Baseline. The Business Case is required to provide a summary andjustification of the recommended actions/changes and includes signatures from the relevant functional, legal, construction and program teams. For construction change orders, the accompanying Business Case will documentthe PCM recommendationand cost estimate with an explanation of any differences. Planned completion date: June 2019 76 California State Auditor Report 2018-108 November 2018 Secretary Brian Annis Page 3 October 22, 2018 2. Before executing its next constructioncontract, the Authority should establish formal prerequisites for beginning construction to prevent avoidable cost overruns and project delays.At a minimum, these prerequisites should identify specific benchmarks related to landacquisition, utility agreements and relocations, and agreements with externalstakeholders, including impacted local governments and other railroadoperators. Response: The Authority concurs with this recommendation and has placed significantfocus on this issue. The 2018 Business Plan identifies this as a critical lesson learned and it continues to be a point of emphasis (see Chapter 4, Lessons Learned and Managing Risk, page 53of the Plan).Key among the lessons learned wasthat the Authority’s decision to awarddesign-build contracts before acquiring right of way and completing agreements with utilities, local governments and railroads meant there were many unknowns thatcreated risks of delays and highercosts. The same chapter also describes the governance andmanagementprocedures initiated in 2017 to strengthen decisionmaking through a highlystructuredprocess, which our response to Recommendation 1 outlines briefly. 1 More specifically, the Business Oversight Committeeconsidersbenchmarks prior to progressing to procurement or to the next phase of project delivery.For example, the Business Oversight Committeewould consider the progress of pre-construction activities, including right-of-way acquisition, prior to approving procurement activities to select a construction contractor. Further, delivering projects in accordance with the Program Baseline, which was adopted by the Board of Directors in June 2018, also ensuresthat certain predecessor tasks (or prerequisites) are 2 sufficiently advanced prior to beginning construction,asthe Baseline incorporatesthe lessons learned outlined in the 2018 Business Plan.The Authority continues todevelop Baseline project work plans composedof discretely definedtasks.Thesetasks are linked together based on project delivery sequencing, whichestablisheswhattasks must be completed prior to beginning other tasks (i.e., predecessortasks and successortasks). Planned completion dates: Project Work Plans (with benchmarks) -Draft: January 2019 Project Work Plans (with benchmarks) -Final: February 2019 3. To better position itself to complete the three Central Valley projects by the December 2022 federal grant deadline, the Authority should improve its monitoring and evaluation of the oversight firms’ risk assessment processes and should take steps to ensure that these processes are consistent across the three projects by May2019. Response: The Authority concurs with this recommendation.The Authority recognizes the risks associated with complying with the deadline established by the American Recovery and Reinvestment Act of 2009 and is in the process of reorganizing its risk team overseeing the Central Valley projects. It is hiring additional risk management personneland will prioritize both risk management and mitigation in a way that is consistent across the three construction projects. Further, to verify that the risk assessment processesare consistent across the three projects, the Authority will continue to enforce all policies and procedures related to PCM California State Auditor Report 2018-108 77 November 2018 Secretary Brian Annis Page 4 October 22, 2018 oversight, revise the PCM manual to be more explicit on risk management, and will publish a program estimate-to-complete and risk management manual. Planned completion date: April 2019 4. To enable policymakers and the public to track the Authority’s progress toward meeting the federal grant deadline of December2022, the Authority should, by January 2019, begin providing quarterly updates to the Legislature detailing the progress of the three Central Valley construction projects using an earned value model that compares construction progress to the projected total completion cost and date.The Authority should base these updates on the most current cost estimates available. Response: The Authority concurs with this recommendation.The Authority recentlydeveloped a Program Delivery Status Report (PDSR) that itproduces on a monthly basis.The PDSR includes comprehensive status information within the Authority’s three main areas of project delivery: 1) right-of-way procurement, third party agreements,and environmental clearance; 2) engineering/designand construction; and 3) rail infrastructure. This PDSR and associated procedures will be codified within the Program Controls Manual to be published. The PDSR provides detailed information on the progress of the three Central Valley construction projects withinthe Infrastructure Deliverysection, including cost varianceand schedule performance index (SPI). Moving forward, the Authority will use the most current cost information and an earned value model to refine the cost variance and SPI for each of the three Central Valley construction projects.On a monthly basis, the cost variance and SPI information will be used to estimate the projected total completion cost and date for each of the Central Valley construction projects. This information will be included in the PDSR. By January 2019, the Authority will use information from the PDSR–including earned value, cost variance and SPI–to develop and provide quarterly updates to the Legislature.The updates will include detailed information on the progressof the three Central Valley construction projects.This information will be used to actively manage the construction projects to ensure that the 2022federal grant deadline is met. Planned completion dates: PDSR Quarterly Update -Draft: December 2018 PDSR Quarterly Update -Final: January 2019 PDSR Manual –May 2019 5. To ensure that it is adequately prepared if it is unable to meet the federal grant deadline of December2022, the Authority should, by May2019, develop a contingency plan for responding to such a scenario. Response: The Authority concurs withthis recommendation.The Authority intends to meetthe federal grant deadline and,to achieve that, we continuously monitor and assess the program through the ProgramDelivery Committee and the Business Oversight Committee. As part of this monitoring process we routinely update the individual project risk registers in coordination with the Federal Railroad Administration(FRA) on a quarterly basis. The ProgramDelivery 78 California State Auditor Report 2018-108 November 2018 Secretary Brian Annis Page 5 October 22, 2018 Committee and the Business Oversight Committee use the Project Update Report, Business Plan and Baselinein their monitoring activities. We will continue to assess different contingency plans/options through eachof these activitiesand will prepare a contingency plan in accordance with the CSA recommendation, which will be updated on an annual basis. Planned completion date: May2019 Chapter 2: The Authority Has Not Successfully Enforced the Policies It Implemented to Address Ongoing Deficiencies WithIts Contract Management 1. To improve its contract management, increase accountability, and justify the significant amount it pays for contracted services, the Authority should take the following steps by May2019: • Prioritize contract management efforts and reduce the frequency with which contract management responsibilities shift among Authority staff by establishing a formal process for hiring and assigning full-time, experienced contract managers. These contract managers should have duty statements reflecting their contract oversight responsibilities and they should report to supervisors who understand those responsibilities and have extensive knowledge about the contracts’ deliverables. In addition, those supervisors’ duty statements should clearly lay out their responsibility for addressing any contract manager noncompliance with the Authority’s contract management policies and procedures, whether reported by Contract Management Support Unit (CMSU)or identified by another means. Response: The Authority concurs with the recommendation. The Authority will create a formal process for hiring and assigning full-time experienced contract managersto reduce the frequency with which contract management responsibilities shift among Authority staff. This will include emphasizing contract management experience/skills as well as desirable contract manager qualifications.In addition, all new advertised positions that require contract management will specify the skills required for a contract manager within the duty statement. In addition, all existing duty statements will be reviewed and modified to reflect contract management/oversight responsibilities for all contract managers and their supervisors. Contract manager supervisors’ duty statements will also address their responsibility to hold their contract management staff accountable for compliance with the Authority's contract management policies and procedures. While only contract managers are currently required to be trained in contract management, contract manager supervisors will also be required to attend contract management training to ensure that the contract managers they supervise are adhering to the Authority’s policies and procedures. The Authority will also create a separate contract management training specifically for supervisors. Planned completion date:May2019 California State Auditor Report 2018-108 79 November 2018 Secretary Brian Annis Page 6 October 22, 2018 • Require CMSU to establish a schedule to monitor individual contract manager compliance and report annually the results of this monitoring to Authority executive leadership. To help ensure the integrity of its oversight role, CMSU should be composed of state staff in place of RDP consultants. Response: The Authority concurs with the recommendation. A schedule to monitor contract managers’ compliance has been created and assessments of contract manager performance are to begin no later than November 2018, to comply with the Authority’s PROC-FIS-038, Contract Compliance Procedure.The assessments will be electronically tracked for resolution status and an executive report will be prepared no less than annually. In addition, the Authority concurs that CMSU should be composedof state staff. The Authority will prepare a Budget Change Proposal for the 2020/21 fiscal year to request staff 3 augmentation to removecontractedRail Delivery Partner (RDP)consultants and replace withstate employees. Planned completion date:The contract manager assessment schedule has been developed and the Budget Change Proposal concept will be submitted to the Authority Executive Committee for consideration in May2019. • Hold contract managers accountable for performing the duties that the Authority’s policies assign to them.Specifically, CMSU and, to the extent necessary, contract managers’ supervisors should require and review evidence from contract managers demonstrating their approval of deliverables, detection and resolution of contractor performance issues, and assessment of contract amendments for merit.The Authority should not accept observations and reports from its contractors or RDP consultants in place of this evidence. Response: The Authority concurs with the recommendation. The Authority’s contract compliance policy POLI-FIS-038 established requirements for performing assessments and reports on contract manager compliance with the Authority’s policies and procedures. The Authority will begin performing assessments of contract managers no later than November 2018. We will begin with the contracts identified in this audit and continue until all contracts have been fully assessed. These assessments will ensure that contract managers are following the Authority's policies and procedures, which arethe foundation of a well-managed contract. The assessments will also provide documented evidence that contract managers, not RDP consultants,are properly approving deliverables/invoices, resolving contractor disputes or performance issues appropriately, and justifying contract amendments with verifiable documentation in all contracts. The assessmentswill not only provide supervisors/management with documentation demonstrating accountability (conformance/nonconformance), but also contain recommendations for best practices and opportunities for improvement. Once an assessment is issued, it will be tracked to closure/resolution. The process also maintains the tracking mechanism to verify the implementation of the corrective action. This plan provides a framework for review of the contract managers’ performance by contract manager and by contract, providing a quantitative assessment of contract manager performance. 80 California State Auditor Report 2018-108 November 2018 Secretary Brian Annis Page 7 October 22, 2018 4 Planned completion date: The Authority will begin performing assessments of contract managers no later than November 2018 and anticipates that all contract manager assessments will be completed by or before November 2020 and be ongoing thereafter. 2. To prevent the inappropriate use of contractors to perform state functions, the Authority should develop procedures by May 2019 forevaluating whether new or existing administrative duties should be assigned to contractors or to state employees. Response: The Authority concurs with this recommendation.The Authority’s Administration Office will develop procedures for evaluating whether new or existing administrative duties should be assigned to contractors or to state employees. Planned completion date: May 2019 3. To ensure the completeness of contract managers’invoice reviews and that invoiced costs are allowable under contract terms, the Authority should amend its applicable procedures by May 2019 to require contract managers to document their review of invoiced rates and expenses. Response: The Authority concurs with this recommendation.As part of a larger plan to revise, fieldtest, and finalize the contract management procedures,the Authority will revise its invoice review procedures (FIS-PROC-033) to require contract managers to document their review of invoiced rates and expenses. Planned completion date: May2019 4. To ensurethe consistency and effectiveness of its efforts to monitor the performance of the oversight firms with which it contracts,the Authority should develop a formal methodologyby May 2019for using the performanceevaluation tool it has implemented.This methodology should include procedures for assessing the sufficiency of the oversight firms’review and approval forconstruction contracts. Response: The Authority concurs with this recommendation.The Authority has developed and implemented monthly performance-based evaluations for its construction oversight firms that 5 are updated by each contract manager.A formal methodology will be developed for the performance evaluations. Contract Performance Monitoring and Reporting Policy POLI-FIS- 034 and Procedure PROC-FIS-034 will berevised to document this formal methodology. The Authority’s contract manager and quality team will evaluate the PCM’s procedure for compliance with section 3.9 Contractor’s Monthly Payments of the Project and Construction Management Manual (For Design-Build Contracts) on all construction packages;specifically, for the purpose of assessing construction progress to validate invoice submissions and earned value from the design-builder.The Authority will amend its Quality Manual to reflect this process and the frequency of the assessments.Any non-conformance will be formally documented. Planned completion date: May2019 California State Auditor Report 2018-108 81 November 2018 Secretary Brian Annis Page 8 October 22, 2018 5. To ensure oversights firms’spending is reasonable, the Authority should develop a formal process by May2019, for tracking any out-of-scope work the oversight firms perform.To reduce the likelihood that its contracts with these oversight firms run out of funds prematurely as a result of this additional work, the Authority should also develop a formal process for amending these oversight firm’s contracts contemporaneous to change orders that significantly extend timelines or increase the scope of work of the construction contracts that theyoversee. Response: The Authority concurs with this recommendation.The June 2018 Program Baseline, approved by the Boardof Directors, details the scope, schedule and budget for all work on the high-speed rail program, including PCMfirms’ contract budgets and schedule. Any PCM contract 6 amendment requires approval from the Business Oversight Committee.As outlined in the response to Recommendation 1, the Business Oversight Committeerequires a Business Case with subject matter expert signatures beforeconsideringany contract change. Additionally, the Authority will revisethe PCM manual to reflect their responsibility to identify, manage and request amendments for out-of-scope work. The PCM contract managers will separately be responsible for tracking and identifying any potential out-of-scope workand the necessity to amend contracts at each PCM work plan revision and approval. The Program Delivery Committeeis responsible for governance oversight of contracts and will consider the impact on PCMcontracts by programmatic trends (such as schedule delays) or changesapproved by the Business Oversight Committee(such as a change to adesign-build contract).A key component of Program Delivery Committee meetings will focus on upcoming key milestones and decision-points, including when or if contracts need amending. Planned completion date: May2019 Chapter 3: The Authority Can Improve the Quality and Transparency of Its Monitoring and Reporting for Key Goals 1. To help improve the effectiveness of its sustainability policy, the Authority should revise the policy by May2019 to more clearly differentiate between construction and operation phases of the high- speed rail system.Further, it should ensure that each objective in each section of the policy is associated with quantifiable metrics for evaluating implementation. Response: The Authority concurs with this recommendation.The Authority will revise its Sustainability Policy to more clearly differentiate between construction and operations.Since the end of audit fieldwork, the Authority has revised the implementation plan to match each existing policy 7 objective with a quantitative metric, posted here: http://www.hsr.ca.gov/docs/programs/green_practices/sustainability/Sustainability_implementation_p lan_SUMMARY_Oct2018.pdf. Planned completion date: The Policy will be updated by May2019. 2. To allow it to evaluate the sustainability of the high-speed rail system’s construction, the Authority should perform and document a review of its compliance with its existing quality controls related to ensuring the validity and completeness of contractor-reported data by May2019.The Authority should also establish a formal process to perform such reviews periodically. 82 California State Auditor Report 2018-108 November 2018 Secretary Brian Annis Page 9 October 22, 2018 Response: 8 The Authority concurs with this recommendation.The Authority improved its environmental and sustainability data gathering and analysis system in July 2017. As an additional improvement, the Authority will develop a quality assurance process to perform periodic reviews of its compliance with the quality controls related to validity and completeness of contractor-reported data. The updated system has a datafield validation feature, which is a quality control measure that involvesa cross check at the data entry level, next level review and confirmation of data by PCMs, and a final data quality review and acceptance by the Authority.This process is enforced by the system and isongoing and continuous. The recommended periodic reviews will verify that the above-described controls are functioning as expected. Planned completion date: May2019 3. To help ensure it meets its sustainability goals, the Authority should comprehensivelycomparethe three construction project’s performance to their construction contractors’ original baseline estimates on a quarterly basis by May2019. Response: The Authority concurs with this recommendation.Since 2015, the Authority has collected and organized data which it uses to analyze construction activities related to air quality and 9 greenhouse gasemissions. The Authority thencompares that to the contractuallyidentified baseline estimates;this comparison is done on a quarterly basis. The Authority compares the contractor’s required performance for waste, absolute targets for recycling of concrete and steel and a percentage target for remaining nonhazardous waste, as recycling records are submitted.Other sustainability performancetracking, related to fuel usage, water usage, and recycling, had previously been analyzed and compared to baseline estimates on an annual basis. The Authority will begin comparison of these performance areason a quarterly basisand will adjust the analysis to include the relationship to construction progress. Planned completion date: May2019 4. To help ensure that itscontractorsproposed environmental impacts are reasonable and to measure the progress of its sustainable construction efforts overtime, the Authority should, by November2019, identify and track standardized measure–such as project miles –that will allow it to compare construction impacts across the high-speed rail system’s different construction projects. Response: The Authority concurs with this recommendation.Currently, the Authority has a model that assessesprogramenvironmental impacts using standardized metrics, including tons of CO2e, kgCO2e/kg, kgCO2e/gallon, kgCO2e/kWh, normalized by miles and construction typology. The Authority is in the process of updating thismodel to incorporate refined project lengths and infrastructure typologies adopted in the Baselineand to incorporate relevant, validated data (e.g.,tons of CO2eand environmental product declarations for actual materials installed) tracked on each construction package.This updated model will allow comparison across construction projectsand support establishing targets for future construction contracts. Planned completion date: November2019 California State Auditor Report 2018-108 83 November 2018 Secretary Brian Annis Page 10 October 22, 2018 5. To increase the transparency of its reporting, the Authority should, by May 2019,expand its quarterly small business, DBVE, and DBEutilization reporting to account for the total value of all its contracts and to identify the reasons it has exempted specific contracts. Response: The Authority concurs with this recommendation.In an effort toincrease transparency, the Authority will post the total value of all current contracts within the Small Business Program section of the Authority’s website.The Authority will also post the total value of any contracts that require small business, DVBE and/or disadvantaged business, which will appropriately reflect the Authority’s utilization percentages per state and federal guidelines. While the Authority reports utilization on a quarterly basis for federally funded contracts, the stateprocess,administeredby the Department of General Services, only requires an annual report from all state agencies/departments.However, the Authority will create an internal process to closely align the state timelinetothe federal quarterly process.In addition, a policy will be created that will clearly specify what contracts are exempt from small business, DVBE and/or disadvantaged business according to state and/or federal regulations, policies, and guidelines. Planned completion date:May2019 Again, weappreciate the opportunity to provide a response to our plans to implement the California State Auditor’s recommendations.If you have any questions, please contact Paula Rivera, Chief Auditor, at paula.rivera@hsr.ca.govor (916) 403-2679 Sincerely, for Brian P.Kelly Chief Executive Officer 84 California State Auditor Report 2018-108 November 2018 California State Auditor Report 2018-108 85 November 2018 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM THE CALIFORNIA HIGH‑SPEED RAIL AUTHORITY To provide clarity and perspective, we are commenting on the Authority’s response to the audit. The numbers below correspond to the numbers we have placed in the margin of its response. Although the Authority states that the Business Oversight 1 Committee (BOC) considers benchmarks prior to progressing to procurement or the next phase of project delivery, this committee, which the Authority established in September 2017, has not yet overseen the procurement of a construction contractor. As Figure 6 on page 26 shows, the Authority executed the most recent of its three construction contracts—for Project 4—in 2016. Therefore, any oversight the committee provides regarding key preconstruction activities is still prospective. As we discuss on pages 30 and 31 of the report, the Authority’s 2 recently released comprehensive schedule delineates that early work tasks, such as land acquisition, should begin before the design and construction phase. However, the schedule does not establish specific benchmarks the Authority must achieve before procuring a construction contractor. Therefore, to avoid cost overruns and delays from moving to the construction phase too soon, as our recommendation on page 36 specifies, the Authority should establish formal prerequisites for beginning construction and these prerequisites should identify specific benchmarks related to land acquisition, utility agreements and relocations, and agreements with external stakeholders. As we state on page 39, CMSU’s oversight of contract management 3 policies and procedures has been weak and inconsistent. We also explain on page 41 that placing oversight responsibility with RDP consultants creates a potential conflict of interest. However, under the Authority’s proposed approach, it would not have funding for state employees to staff CMSU until at least July 2020. Given the issues we found and the amount of public funds at stake, we believe the Authority should move faster to secure professional state staff to perform CMSU’s crucial oversight duties. We state on page 38 that as of September 2018, the Authority’s 4 56 contract managers were collectively responsible for 204 contracts. Although we recognize the amount of work necessary to hold each contract manager accountable for each of their assigned contracts, we do not believe the State can afford for the Authority to take two years to complete its assessments of contract managers, as its 86 California State Auditor Report 2018-108 November 2018 response indicates. Additionally, we encourage the Authority to focus its initial efforts not only on the contracts identified in this audit but also its other largest and highest risk contracts. 5 The Authority’s response does not specify whether its methodology for evaluating its PCMs—which we refer to in our report as construction oversight firms (oversight firms)—will include procedures for assessing the sufficiency of the oversight firms’ reviews and approvals of invoices for construction contracts. As we state in our recommendation on page 59, such procedures are important to ensure consistency and the effectiveness of the Authority’s efforts to monitor the performance of the oversight firms with which it contracts. Therefore, the Authority should include these procedures in its methodology. 6 The Authority states that any oversight firm contract amendment requires approval by the BOC. However, we identified concerns with an oversight firm contract amendment that the Authority approved in March 2018. Specifically, as we describe on pages 55 and 56, the Authority’s documents assert that the amendment stemmed from it assigning work to the oversight firm that was outside the original contract. However, the contract manager was unable to tell us the amount of funding that went to the oversight firm for this out-of-scope work and acknowledged that he had not documented the oversight firm’s adequate performance. Therefore, until the Authority implements our recommendation on page 59 to track any out-of-scope work that oversight firms perform, the Authority, including its BOC, will not know whether the oversight firms’ spending rates are reasonable or if amendments are appropriate. 7 We appreciate that the Authority is taking steps to match existing sustainability policy objectives with quantitative metrics in its implementation plan. However, our recommendation on page 68 is that the Authority first revise its sustainability policy to more clearly differentiate between construction and operation phases of the system, a process we would expect to generate new policy goals and objectives. At that point, the Authority should reevaluate its implementation plan to ensure that the plan contains appropriate metrics for those new policy objectives. 8 Although the Authority asserts that it improved its environmental and sustainability data gathering and analysis system in July 2017, the Authority’s sustainability director confirmed during our audit that the Authority was still in the process of collecting and reviewing sustainability data for calendar year 2017 and that the transition to this system was still ongoing. Additionally, the Authority’s response describes quality control measures for the new California State Auditor Report 2018-108 87 November 2018 data system, but these measures are not fundamentally different from those that were in place under the previous system. We describe our concerns with the accuracy of the data on page 63. 9 The Authority’s assertion that it has completed these comparisons since 2015 is inconsistent with the evidence it provided during our audit. During the audit, the Authority’s sustainability director confirmed for us that the Authority was beginning the process of comparing actual sustainability impacts to contractor estimates, but that the Authority did not expect to have preliminary findings until the end of 2018. The Authority also provided documentation showing that some comparisons were underway. However, as we state on page 64 of our report, our expert observed that this documentation does not allow the Authority to effectively project whether contractors will meet or exceed their estimates because it does not account for actual construction progress to date. On page 64 we also state that documentation did not include equivalent comparisons related to the environmental impacts from waste or water use. Therefore, we stand by our recommendation on page 69 that the Authority should comprehensively compare the three construction projects’ performances to their original baseline estimates on a quarterly basis, and it should begin doing so no later than May 2019.