Recommendations

March 2016 California Department of Transportation Its Maintenance Division’s Allocations and Spending for Field Maintenance Do Not Match Key Indicators of Need Report 2015-120 COMMITMENT INTEGRITY LEADERSHIP The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the California State Auditor’s Office at the following address: California State Auditor 621 Capitol Mall, Suite 1200 Sacramento, California 95814 916.445.0255 or TTY 916.445.0033 OR This report is also available on our website at www.auditor.ca.gov. The California State Auditor is pleased to announce the availability of an online subscription service. For information on how to subscribe, visit our website at www.auditor.ca.gov. Alternate format reports available upon request. Permission is granted to reproduce reports. For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. For complaints of state employee misconduct, contact the California State Auditor’s Whistleblower Hotline: 1.800.952.5665. Elaine M. Howle State Auditor Doug Cordiner Chief Deputy March 17, 2016 2015-120 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the California State Auditor presents this audit report concerning the methods used by the California Department of Transportation (Caltrans) to make spending decisions related to the Maintenance Program (program) and to assess the timeliness and effectiveness of the program’s funding. This report concludes that the Caltrans division of maintenance’s (maintenance division) allocations and spending for field maintenance do not match key indicators of maintenance need. Specifically, the maintenance division developed a budget model (model) for allocating field maintenance funding based on key indicators of maintenance need such as traffic volume and climate. However, we found that the maintenance division abandoned this approach, and instead has based funding allocations to the 12 Caltrans districts on a simple average of historical spending rather than using level of maintenance performance (service scores) or other information about maintenance need, despite reporting to the Legislature that it was using a more sophisticated method. Additionally, the maintenance division’s current process for evaluating service scores does not provide the same in-depth information as the model would have provided. The maintenance division also does not use the information regarding service scores to strategically plan its work or to inform its funding allocations. Further, the maintenance division cannot demonstrate that it promptly performs certain field maintenance work. Specifically, data indicate that more than 30,000 service requests received by the three districts we audited in fiscal years 2010–11 through 2014–15 remained unresolved for more than 90 days. Not surprisingly, we found that the maintenance division’s actual spending for field maintenance in the three districts we reviewed was not consistent with key indicators of need—climate and traffic volume. To better align allocations with field maintenance needs we recommend that the Legislature include language in the Budget Act that requires the maintenance division to develop and implement a budget model that takes into account key indicators of field maintenance need, such as traffic volume, climate, service scores, and any other factors it deems necessary. We made additional recommendations to the maintenance division to ensure that it performs field maintenance work consistently on highways with similar needs. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 916.445.0255 916.327.0019 fax www.auditor.ca.gov Blank page inserted for reproduction purposes only. California State Auditor Report 2015-120 v March 2016 Contents Summary 1 Introduction 7 Audit Results After Beginning to Develop a Logical Approach for Addressing Field Maintenance Needs, the Maintenance Division Abandoned It 23 The Maintenance Division Cannot Demonstrate That It Promptly Performs Field Maintenance Work 39 Weak Controls Over Field Maintenance Work Orders Create Opportunities for Fraud, Waste, and Abuse 42 The Maintenance Division Appropriately Manages Highway Maintenance Projects, but Some Backlogged Maintenance Work Is Increasing 43 Recommendations 45 Appendix Spending by Zone and Demographics in Districts We Reviewed 49 Responses to the Audit California State Transportation Agency 59 California Department of Transportation 61 California State Auditor’s Comments on the Response From the California Department of Transportation 67 vi California State Auditor Report 2015-120 March 2016 Blank page inserted for reproduction purposes only. California State Auditor Report 2015-120 1 March 2016 Summary Audit Highlights . . . Results in Brief Our audit concerning the methods used by the California Department of The California Department of Transportation (Caltrans) is Transportation (Caltrans) to make spending responsible for constructing, improving, and maintaining decisions related to its maintenance California’s highway system (state highway system). This report program revealed the following: focuses on Caltrans’ maintenance activities. Maintenance is » Caltrans’ division of maintenance defined by state law as the preservation and upkeep of roadway (maintenance division) paid $250,000 for structures in the safe and usable condition to which they have development of a budget model (model) been improved or constructed. Caltrans’ division of maintenance for allocating field maintenance funding (maintenance division) administers the maintenance program, based on key indicators of maintenance which focuses on preventative work to correct small problems need, however it abandoned it. before they grow to require more costly repairs. The maintenance program consists of two types of work: highway maintenance and » Although the model was not field maintenance. Highway maintenance includes more significant implemented, Caltrans reported to the work that contractors perform to repair pavement, bridges, and Legislature that it is using the model to drainage culverts. Field maintenance, on the other hand, is generally allocate funding to its districts. performed by maintenance division staff and includes activities » The maintenance division’s process for such as repairing minor pavement damage, clearing vegetation, and evaluating maintenance performance picking up litter. Although we reviewed the maintenance division’s scores (service scores) is lacking: processes for both highway maintenance and field maintenance, the concerns we identified relate primarily to field maintenance. • It does not provide the same in-depth We were asked to review district 7 (Los Angeles) and two other information as the model would districts; we selected district 4 (Oakland) and district 6 (Fresno). have provided. Although it developed a logical approach for addressing field • It does not weight districts’ service maintenance needs, the maintenance division abandoned the scores to account for differences in approach. Specifically, the maintenance division never implemented traffic volume. a budget model (model) that it paid $250,000 to develop in 2009. • It only sets performance goals Use of that model would have allowed the maintenance division for improving three of the many to identify the resources needed to maintain highways with categories of maintenance activities similar conditions at a similar level of maintenance performance it evaluates. (known as a service score). The model categorized sections of the state highway system into zones based on climate and on traffic » The maintenance division does not levels, and it took into consideration the service scores for several use information from districts’ service maintenance activities and the inventory of road elements, such as scores to strategically plan its work or to bridges and tunnels. The maintenance division asserted to us that allocate funding. the model produced funding allocations that were unreasonable, such as the need to reduce more than 100 staff positions at district 7 » Spending for field maintenance was not (Los Angeles); but instead of trying to determine why the model consistent with key indicators of need— produced such allocations, the maintenance division decided to traffic volume and climate. abandon it. » The maintenance division cannot demonstrate that it promptly performs We found that the maintenance division did not implement field maintenance work. the zone‑level evaluation of service scores but instead samples conditions broadly across each district, which does not provide » Caltrans’ weak cost controls over the same level of information. The maintenance division also does field maintenance work orders create not establish goals for all categories of maintenance activities opportunities for fraud, waste, and abuse. 2 California State Auditor Report 2015-120 March 2016 it evaluates and does not use the information it gathers about maintenance needs to meet its service goals or to plan its work. For example, some districts’ service scores, which indicate the level of maintenance need, have remained generally low over the last five fiscal years. A low service score means the district has a high maintenance need. While the maintenance division averages the districts’ service scores to calculate a statewide score, it does not weight the districts’ scores according to traffic volumes, which misrepresents the status of the majority of the state’s most heavily used roads. When we adjusted the service scores to account for traffic volumes, the statewide scores fell several points. The maintenance division also could not demonstrate that it uses the information on low service scores to strategically plan its work to address maintenance needs and improve the scores. Instead of holding districts accountable for improving their scores, the maintenance division establishes spending goals for some field maintenance activities. While we found that the districts we reviewed sometimes exceeded their spending targets, the corresponding service scores generally remained stagnant or even worsened. In addition, the three districts we reviewed could provide only short‑term plans for activities they intended to complete, and these plans did not adequately establish priorities or facilitate the monitoring of progress toward improving the districts’ scores. For example, some of the plans consisted of emails from supervisors that listed brief descriptions of selected activities to complete over the ensuing week. The maintenance division did have robust requirements for planning work, but it decided to do away with these requirements because it believed the requirements demanded too much time. Without adequate plans for completing field maintenance work, district staff have little accountability for how well they meet maintenance needs. In addition, although the maintenance division never implemented its model, the division has been reporting to the Legislature that it is using this sophisticated model to allocate field maintenance funding to its districts that takes into account key maintenance need indicators, such as traffic volume and climate. However, the maintenance division informed us that instead of using the model, it has actually been distributing funding based on a simple historical average of each district’s spending. In fact, we found the districts’ allocations remained largely unchanged from fiscal years 2010–11 through 2014–15. As a result, the Legislature and other decision makers may have believed that headquarters was using a more robust approach to allocate funding to the districts than it actually was, causing those decision makers to be less likely to question the allocations. California State Auditor Report 2015-120 3 March 2016 We found that simply using historical averages to allocate field maintenance funding to the districts does not align with key indicators of field maintenance needs. For example, the two districts that handle the biggest percentage of the state’s traffic volume (43 percent) only receive 27 percent of the funding. Additionally, although some districts had low service scores, headquarters did not adjust its overall allocations to those districts to better align with their needs. As a result, districts may not be able to fully address such needs, which could delay maintenance work and result in more costly repairs in the future. These effects may also be exacerbated because headquarters is potentially underfunding the districts with relatively greater traffic volume. Because Caltrans did not implement the budget model and instead based its allocations of field maintenance funding on historical averages, it is not surprising to find that the three Caltrans districts we reviewed spent significantly different amounts for field maintenance on highways with similar indicators of maintenance need. For example, in comparing highway sections based on climate and traffic levels, we found the maintenance division spent significantly less per mile for field maintenance on some sections of highway in each district than it did on other similar sections of highways in those same districts. These results further indicate that Caltrans needs to assess whether districts are using funds in a manner commensurate with need. The maintenance division also cannot demonstrate that it promptly performs certain field maintenance work. Specifically, the maintenance division does not establish time frames for completing maintenance service requests (service requests) it receives from the public, so we calculated how long each service request had been outstanding. Data from Caltrans’ online service request system indicate that more than 30,000 service requests that were received through the online system by the three districts from fiscal years 2010–11 through 2014–15 remained unresolved after more than 90 days. Caltrans was unable to tell whether those service requests still remained incomplete or whether it had acted on the request and simply failed to update the system. In addition, districts receive service requests via methods other than Caltrans’ website, but the districts have poor processes for tracking whether they complete them. Failure to perform or delay of needed maintenance work can create unsafe conditions. Moreover, we found that the maintenance division has weak cost controls over field maintenance work orders, which creates opportunities for fraud, waste, and abuse. Specifically, our review of work orders found limited evidence to support whether the costs and resources used were reasonable and appropriate. Despite requirements to review and approve work orders, district 4 California State Auditor Report 2015-120 March 2016 supervisors do not document their review and approval that the resources used to complete field maintenance work orders were necessary and appropriate and that costs were reasonable. Additionally, audits of field maintenance work orders that Caltrans’ internal auditors conducted in 2013 and 2015 found instances in which physical inventory counts of the chemicals used to control vegetation did not match recorded amounts and staff time sheets were not promptly reviewed and approved and had errors. These weak controls make resources and equipment susceptible to theft or misuse. Finally, although the maintenance division is generally managing highway maintenance projects appropriately, the backlog of pavement and drainage culverts that need maintenance work is increasing. The maintenance division allocates funding for highway maintenance proportionally to the districts based on the condition of pavement and bridges. In contrast, Caltrans asserts that it generally distributes funds for culverts equitably among the districts because the budget is relatively small compared to the identified needs of the districts. Our review of pavement and bridge projects found that the maintenance division generally ensured that contractors completed the projects on time and within budget. However, since 2011 the number of highway lane miles that need maintenance has increased from 11,053 to 15,272, and the number of culverts that need maintenance has increased from an estimated 13,185 to 27,346. Although the maintenance division’s goal given its current funding level is to repair only 2,100 lane miles of pavement and 140 culverts each year, Caltrans has reported that even if it were to receive additional funding it would prioritize more significant rehabilitation work ahead of this maintenance backlog. Without further action, the backlog of pavement and drainage culverts that need maintenance work will likely grow. Recommendations To better align the maintenance division’s allocations with districts’ maintenance needs, the Legislature should include language in the Budget Act that requires the maintenance division to develop and implement a budget model for field maintenance by June 30, 2017, that takes into account key indicators of maintenance need, such as traffic volume, climate, service scores, and any other factors the maintenance division deems necessary to ensure that the model adequately considers field maintenance need. Once the model is developed, Caltrans should use it to inform appropriate allocations to the districts. California State Auditor Report 2015-120 5 March 2016 Caltrans should revise the language in its future five‑year maintenance plans to accurately describe the method it uses to allocate field maintenance funding to its districts. To ensure that it performs field maintenance work consistently on highways with similar needs, the maintenance division should do the following: • Assess whether districts are using funds in a manner commensurate with indicators of need included in its new budget model. • Implement the zone‑level evaluation of service scores contemplated in the earlier budget model that it abandoned. • Establish zone‑specific service score goals for all of the field maintenance activities it deems critical to ensuring a safe and usable state highway system and require districts to meet those goals for all the zones within their borders. • Implement requirements for strategically planning field maintenance work that it believes are feasible and ensure that supervisors plan and schedule field maintenance work in accordance with those requirements to monitor progress toward improving service scores. Caltrans should require its staff to verify and update the status of all outstanding service requests. Additionally, Caltrans should require supervisors to monitor completion of service requests by reviewing the data from the service request system monthly to identify service requests not completed after a period of time that Caltrans deems appropriate, such as 30 days. For all service requests outstanding after this period, Caltrans should require its supervisors to determine the status of the service requests and ensure the work is appropriately prioritized. To prevent and detect fraud, waste, and abuse and to ensure costs are appropriate, the maintenance division should strengthen its controls over reviewing and approving work order costs by requiring its supervisors and superintendents to document their review and approval of work orders. Agency Comments The California State Transportation Agency and Caltrans agreed with our findings and indicated they will implement our recommendations. 6 California State Auditor Report 2015-120 March 2016 Blank page inserted for reproduction purposes only. California State Auditor Report 2015-120 7 March 2016 Introduction Background The California Department of Transportation (Caltrans) is responsible for constructing, improving, and maintaining California’s highway system (state highway system). The state highway system is composed of more than 50,000 lane miles, more than 13,100 bridges, and an estimated 205,000 drainage culverts.1 It includes interstate highways, U. S. highways, and state highways. These highways are also referred to as routes. For example, the state highway system includes State Highway 50, which runs from Sacramento to the Nevada state line, and Interstate 5, which runs north and south through California. The state highway system does not include county highways and local roads. Caltrans mainly cares for the existing state highway system through two programs: the state highway operation and protection program (SHOPP) and the maintenance program. Caltrans rehabilitates and reconstructs the state highway system through the SHOPP. Projects in this program include capital improvements for safety and the rehabilitation of state highways and bridges. These projects do not add capacity to the state highway system; adding capacity is the responsibility of Caltrans’ state transportation improvement program (STIP). Multiyear plans for both SHOPP and STIP projects are adopted by the California Transportation Commission (commission). According to its 2015 annual report, the commission approved allocations of approximately $1.6 billion for SHOPP projects and $531.3 million for STIP projects for fiscal year 2014–15. Our review focused on the maintenance program, which Caltrans’ division of maintenance (maintenance division) administers. Unlike the SHOPP, which handles more significant and costly rehabilitation projects, the maintenance program focuses on preventative work and corrects small problems before they worsen and require more costly repairs. State law defines maintenance as the preservation and upkeep of roadway structures in the safe and usable condition to which they have been improved or constructed. The maintenance program includes pavement, bridges, roadside and drainage, traffic guidance, and electrical maintenance. Maintenance also includes the special or emergency maintenance or repair necessitated by accidents, weather conditions, slides, settlements, or other unusual or unexpected damage of a roadway, structure, or facility. Maintenance does not include construction of new assets 1 A lane mile is a unit of measure for pavement measuring one mile long and one lane wide. A mile stretch of a two‑lane road equals two lane miles, and a segment of road one mile long and four lanes wide is four lane miles. 8 California State Auditor Report 2015-120 March 2016 or rehabilitation or reconstruction of roadways. However, according to Caltrans, adequate maintenance can significantly reduce future SHOPP costs for roadway rehabilitation. The maintenance program consists of two types of maintenance work: highway maintenance and field maintenance. Highway maintenance includes more significant work to repair pavement, bridges, and drainage culverts, among other things. For example, highway maintenance work includes different types of surface treatments to extend the service Examples of Field Maintenance Activities life of a segment of pavement. These treatments keep the roadway safe and in usable condition, • Sealing cracks and patching potholes on pavement but they do not include structural capacity • Clearing vegetation and drainage improvement or reconstruction. Caltrans generally hires contractors to perform this work. • Picking up litter and debris • Removing graffiti Field maintenance, on the other hand, is generally performed by maintenance division staff and • Maintaining and painting bridges includes activities such as repairing minor • Replacing highway lighting and traffic signals pavement damage, clearing vegetation, picking • Preserving roadway striping, signs, and guardrails up litter, removing graffiti, and other activities as listed in the text box. Districts generally identify • Removing snow, patrolling for storms, controlling field maintenance work in two ways: maintenance floods and slides personnel travel all highways to observe Source: California Department of Transportation's Maintenance conditions and identify maintenance needs, or Manual, volume II. the public submits service requests notifying the maintenance division of needed maintenance. Funding for the Maintenance Program In the state budget each year, the Legislature appropriates funding for Caltrans’ programs, including the maintenance program. California’s budget process generally uses incremental budgeting, which employs a department’s current level of funding as a base amount. State law requires Caltrans to prepare a five‑year maintenance plan (maintenance plan), which it must update every two years, as the basis for its budget request. The plan addresses the maintenance needs of the state highway system but includes only maintenance activities that could result in increased SHOPP costs if not performed. The maintenance plan attempts to balance resources between SHOPP and maintenance activities to achieve identified milestones and goals at the lowest possible long‑term cost. State law also requires Caltrans to develop a budget model to achieve this balance of resources.2 Additionally, if the maintenance 2 Caltrans includes the statutorily required budget model in its maintenance plan in the section titled “Analysis of Alternative Levels of Maintenance Investment,” not to be confused with the section of the maintenance plan titled “Maintenance Program Budget Model,” which is discussed in more detail in the Audit Results. California State Auditor Report 2015-120 9 March 2016 plan recommends increases in maintenance spending, the maintenance plan is supposed to identify projected future SHOPP costs that would be avoided by implementing that increased maintenance spending. As Table 1 shows, the maintenance program received approximately $1.4 to $1.5 billion in funding in each fiscal year from 2010–11 through 2014–15. This amount represented approximately 14 percent of Caltrans’ total annual funding in fiscal year 2014–15. The maintenance program receives nearly all of its funding from the state highway account, the fund in which the State accumulates most of the revenues from gasoline taxes. Other funding sources for the maintenance program include federal funds that the maintenance division uses for pavement projects and bridge inspections, which make up approximately 8 percent of its total funding, and reimbursements for work that the maintenance program performs for local agencies.3 Table 1 California Department of Transportation Maintenance Program Appropriations, Expenditures, and Unspent Funds Fiscal Years 2010–11 Through 2014–15 (in Millions) FISCAL YEAR 2010–11 2011–12 2012–13 2013–14* 2014–15* State Highway Account Appropriation $1,259 $1,406 $1,321 $1,366 $1,413 Expenditures 1,259 1,395 1,311 1,327 988 Unspent 0 11 10 39 425 Federal Trust Fund Appropriation 103 105 117 118 119 Expenditures 92 102 111 97 23 Unspent 11 3 6 21 96 Total appropriations $1,362 $1,511 $1,438 $1,484 $1,532 Total expenditures 1,351 1,497 1,422 1,424 1,011 Total unspent $11 $14 $16 $60 $521 Percentage of 99% 99% 99% 96% 66% appropriations expended Sources: State Controller’s Appropriation Control Ledger and Budgetary/Legal Reporting System for fiscal years 2010–11 through 2014–15. * The California Department of Transportation still has additional fiscal years to spend against its appropriations for fiscal years 2013–14 and 2014–15. 3 For example, Caltrans personnel may perform routine maintenance of traffic control systems or facilities on county‑owned roads or city‑owned streets through cooperative agreements. 10 California State Auditor Report 2015-120 March 2016 Caltrans has one year to spend or commit all or part of an appropriation for future expenditures and then two additional years to pay off such expenditures from its appropriation of funds. After this three‑year period, any unspent funds revert to the originating fund for future reappropriation. The maintenance division spent most of its maintenance program funding that was appropriated in fiscal years 2010–11 through 2012–13, but it still has time to spend amounts appropriated in the most recent two fiscal years. We noted that approximately $144 million appropriated in the two years before our audit period had reverted to the state highway account during our audit period. The reverted funds generally resulted from the effects of the economic downturn: cost‑savings the maintenance division achieved on projects as prices decreased, an influx of funding from the American Recovery and Reinvestment Act of 2009 (recovery act), and funds it did not spend when it complied with the governor’s 2009 executive order to halt purchases of new vehicles. In each of the first three fiscal years of our audit period (2010–11 through 2012–13), the maintenance division spent 99 percent of its total appropriation. During the last five fiscal years, the maintenance division has spent about one‑fourth of its appropriations on contracts, primarily for highway maintenance. The maintenance division has discretion in distributing the major part of its maintenance program appropriation. Figure 1 shows how headquarters allocated the roughly $1.5 billion the maintenance program received in fiscal year 2014–15. Included in the total is an appropriation of approximately $338 million for distributed administration and equipment programs (administration program).4 The Budget Act of 2014 specified amounts that must be spent for certain highway maintenance program items, including major highway maintenance pavement contracts and storm water discharge: $231.7 million and $50.6 million, respectively. The remaining $917 million is discretionary. From these discretionary funds, the maintenance division allocated an additional $72 million for highway maintenance and $682.7 million for field maintenance to Caltrans’ 12 districts, and $161.9 million for headquarters in fiscal year 2014–15. The $161.9 million the maintenance division allocated was for overhead and other costs to the following four headquarters divisions: maintenance, engineering services, procurement and contracts (specifically, warehouse), and audits and investigations. The allocation was primarily for employee costs, external and 4 Administration program costs are the indirect costs of a program, typically a share of the costs of the administrative units serving the entire department (for example, legal, personnel, and accounting). Distributed administration costs represent the distribution of the indirect costs to the program activities of a department. California State Auditor Report 2015-120 11 March 2016 interdepartmental contracts, and other general expenses. Specifically, in fiscal year 2014–15, the maintenance division retained $128.4 million and allocated $17.5 million to the division of engineering services, $15.6 million to the warehouse, and $317,000 to audits and investigations. In fiscal years 2010–11 through 2014–15, the allocation for these other headquarters’ functions ranged between 14 and 20 percent of program funding, excluding distributed administration program costs. Figure 1 California Department of Transportation Maintenance Program Funding Allocation for Fiscal Year 2014–15 (in Millions) Headquarters—$162 (11%) Distributed administration— $338 (22%)* Discretionary—$917 (60%) District highway maintenance— District field $354 (23%)† maintenance— $683 (44%) Source: California Department of Transportation’s financial system. * Administration program costs are the indirect costs of a program, typically a share of the costs of the administrative units serving the entire department (for example, legal, personnel, and accounting). Distributed administration costs represent the distribution of the indirect costs to the various program activities of the department. † Highway maintenance includes $231.7 million for major maintenance pavement contracts and $50.6 million for storm water discharge, appropriated separately in the Budget Act of 2014. Caltrans received $2.5 billion in federal funds in 2009 through the recovery act, including $56.2 million that the California Department of Finance approved for the maintenance program. According to reports from the California Division of the Federal Highway Administration as of September 30, 2015—the last day recovery act funds were available—Caltrans has spent 99.7 percent of recovery act funds, including the funds approved for the maintenance program. Caltrans asserted that the unspent $7.9 million represented savings from projects for other programs that were completed under budget. 12 California State Auditor Report 2015-120 March 2016 The Maintenance Program’s Organizational Structure Examples of the Maintenance Division’s Offices at Headquarters The maintenance division has staff at Caltrans’ headquarters located in Sacramento and at • Pavement Management and Performance Caltrans’ 12 districts. The districts and their counties • Structure (bridge) Maintenance and Investigations are shown in Figure 2. The maintenance division is divided into several offices at headquarters, • Roadway Maintenance including those shown in the text box. Headquarters • Maintenance Equipment and Training is responsible for establishing policies, providing technical assistance to the districts, and reviewing • Budgets and Planning districts’ compliance with standards and policies. At • Maintenance Management Systems and Studies headquarters, the chief of the maintenance division • Administration Management (division chief) has overall responsibility for the statewide maintenance program. The division chief • Emergency Management is responsible for establishing goals, developing Source: California Department of Transportation headquarters justification for and documenting resource needs, division of maintenance organization chart. and determining resource allocations among the districts. The maintenance division structures its management according to geographic areas within each of Caltrans’ 12 districts. Specifically, a deputy district director of maintenance (district maintenance deputy) directs maintenance efforts at each of the 12 districts. District maintenance deputies oversee district activities and communications, engineering, and region manager operations; they are also responsible for allocating resources within their respective districts, updating district plans to achieve expected goals, and reviewing and approving region work plans. Each district is divided into regions, and region managers are responsible for field operations and activities within each region. Regions are further divided into areas. Area superintendents are responsible for activities within their assigned areas, and they oversee the supervisors of maintenance staff crews. Each supervisor is also responsible for specific segments of the state highway system and specific field maintenance activities within a superintendent’s area, such as landscaping. Figure 3 on page 14 illustrates the management structure using district 7 (Los Angeles) as an example. The maintenance division has provided a manual of guidance and a computer system to help its staff manage maintenance work. The maintenance manual presents general practices and procedures intended to provide for a uniform approach to maintaining the state highway system. The maintenance division’s maintenance staff use its integrated maintenance management computer system (maintenance management system) to plan, perform, record, and manage maintenance work. The maintenance management system allows supervisors and managers to inventory assets, track work performed and the associated costs, manage materials and equipment, and provide decision‑making tools to managers California State Auditor Report 2015-120 13 March 2016 Figure 2 California Department of Transportation Districts DEL NORTE SISKIYOU MODOC EUREKA 2 REDDING LASSEN HUMBOLDT TRINITY SHASTA 1 TEHAMA PLUMAS BUTTE GLENN MARYSVILLE SIERRA MENDOCINO LAKE COLUSA SUTTER YUBA 3 NEVADA EL P L D A O C R E A R DO YOLO HEADQUARTERS SONOMA NAPA ALPINE MAR 4 IN OAKL S C A O O L N N A T D N R O A SAC S R S A T A M O E N N C TO K A T M O A N D C O AL R AV 1 ERAS 0 TUOLUMNE MONO SAN F S R A A N N M C A IS T C E O O AL C A O C S M S A L E T A N D A R T A A A JOAQ ST U AN IN ISLAU M S ERCED MARIPO M S A A DERA BISHOP 9 SANTA CRUZ FRESNO SAN 6 FRESNO BENITO INYO TULARE MONTEREY 5 KINGS SAN LUIS OBISPO KERN SAN BERNARDINO SAN LUIS OBISPO 8 SANTA BARBARA 7 VENTURA LOS ANGELES SAN BERNARDINO LOS ANGELES IRVINE RIVERSIDE ORANGE 12 SAN DIEGO SAN DIEGO 11 IMPERIAL Source: California Department of Transportation. 14 California State Auditor Report 2015-120 March 2016 and supervisors. For example, a maintenance supervisor must fill out a work order in the maintenance management system for field maintenance work. The work order records the expenditures of labor, production units, vehicles, and materials, and the location where the crew performed the work. Figure 3 Example of California Department of Transportation Division of Maintenance Region Boundaries and Management Structure Within District 7 (Los Angeles) North Region REGION MANAGER SUPERINTENDENTS (Areas) SUPERVISORS (Territories) West Region CREWS East Region DISTRICT MANAGEMENT • A district deputy director of maintenance oversees activities and operations at each district. • Region managers are responsible for field operations and activities in each region. South Region • Regions are divided into areas. Area superintendents oversee supervisors assigned to their area. • Supervisors manage crews of maintenance staff. Sources: California Department of Transportation’s maintenance manual and district 7 (Los Angeles) maintenance region boundary map. California State Auditor Report 2015-120 15 March 2016 System for Evaluating Maintenance Performance The maintenance division has a program for evaluating the maintenance level of service or performance that helps determine how well it maintains the state highway system under the maintenance program. The maintenance division annually conducts maintenance performance evaluations (service evaluations) for several categories of maintenance activities, representing a snapshot of the roadway conditions. These evaluations are separate from the general reviews maintenance personnel perform to identify needed maintenance work mentioned previously. Some examples of the categories are shown in Table 2 on the following page. The maintenance division calculates maintenance performance scores (service scores) for these categories. However, the maintenance division has set service score goals only for picking up litter and debris, maintaining lane striping, and repairing guardrails. According to the maintenance performance reports, these goals apply only to the state overall, not to the individual districts. To perform the service evaluations, the maintenance division divides the state highway Level of Service Rating System for Litter system into one‑mile segments. It annually Pass (100): no deficiency conducts service evaluations on a random sample of 20 percent of the one‑mile segments within Need 1 (50): one small area each district. Evaluators visually observe highway Need 2 (0): more than one area attributes to determine whether conditions are deficient and to determine the overall needs of Source: California Department of Transportation's Fiscal Year 2014–15 Maintenance Level of Service Statewide Report each segment. Specifically, evaluators inspect a Executive Summary. one‑mile segment using a rating system such as the one shown in the text box and provide a score for that segment. Evaluators total and average the points for all the evaluated segments to calculate that district’s service score. Low service scores indicate that the district has a high maintenance need. The maintenance division then averages all of the districts’ service scores to calculate an overall statewide score for each category, with the exception of storm water. Rather than using the established service score goals to measure district performance, the maintenance division has established spending goals for five maintenance activities, shown in Table 2, that it requires each district to meet. We discuss the maintenance division’s approaches for evaluating maintenance performance further in the Audit Results. 16 California State Auditor Report 2015-120 March 2016 Table 2 Examples of Service Score Categories and Goals SERVICE SCORE SERVICE SCORE GOAL SPENDING GOAL SERVICE SCORE CATEGORY IS CALCULATED HAS BEEN SET HAS BEEN SET Flexible Travelway Cracks  5  Potholes  5 5 Paved shoulders  5 5 Rigid Travelway Joint separation  5 5 Slab failure  5 5 Ramps  5 5 Drainage Surface drains  5 5 Ditches  5 5 Slopes  5 5 Roadside Vegetation  5 5 Fences  5 5 Litter and debris    Graffiti  5 5 Traffic Guidance Lane striping    Raised markers  5 5 Signs  5 5 Guardrails   * Landscaping Weed control  5 5 Mulch  5 5 Irrigation system  5 5 Storm Water Storm water NA NA  Sources: California Department of Transportation’s (Caltrans) Fiscal Year 2014–15 Maintenance Level of Service Statewide Report Executive Summary and maintenance division expenditures dashboards.  = Yes 5 = No NA: While the maintenance division has set a spending goal for storm water, storm water represents funding Caltrans receives that is not for any particular maintenance activity so the maintenance division does not calculate a service score or set a service score goal for storm water. * Caltrans set a spending goal for safety barriers, which include guardrails. California State Auditor Report 2015-120 17 March 2016 Scope and Methodology The Joint Legislative Audit Committee (audit committee) directed the California State Auditor to review the methods Caltrans used to make spending decisions related to the maintenance program. Table 3 lists the objectives that the audit committee approved and methods we used to address those objectives. Table 3 Audit Objectives and the Methods Used to Address Them AUDIT OBJECTIVE METHOD 1 Review and evaluate the laws, Reviewed relevant laws, rules, regulations, and other background materials related to the rules, and regulations significant to maintenance program. the audit objectives. 2 Identify the actual, estimated, and For our audit period of July 1, 2010, through June 30, 2015, we did the following: proposed statewide expenditures • Reviewed budget documents and accounting records from the State Controller’s Office (State for the program. Additionally, Controller) Appropriation Control Ledger, which shows cash expenditures for each fiscal year by fund, identify any trends in expenditures agency, and program. and reasons for the trends. • Reviewed budgeted and actual expenditure amounts and compared year‑over‑year increases and decreases to identify trends. Interviewed relevant staff to determine reasons for the trends. 3 Identify the sources of funding • Reviewed annual budget documents to identify funding sources. for the program and assess the • Reviewed Caltrans’ five‑year maintenance plans and other relevant documentation of Caltrans’ method used by the California process for determining the total amount of funding for the statewide program. Department of Transportation • Reviewed budget documents and State Controller's accounting records to determine whether (Caltrans) to determine the total Caltrans was using all the funds it received. Identified funds that reverted to the state highway amount of funding allocated to account during our audit period and gathered evidence of the reasons for the reversions. the statewide program each year. Additionally, determine whether • Interviewed staff and reviewed relevant documentation of Caltrans’ efforts to identify and secure Caltrans has taken advantage of all additional funding from the federal government. opportunities to maximize funding for the program and whether any additional sources of funding exist. 4 Review and assess Caltrans’ • Interviewed staff about the process for determining allocations for positions and operating expenses. method of allocating program • Interviewed staff and reviewed documentation regarding the budget model the division of funding throughout the State maintenance (maintenance division) developed to allocate field maintenance funding to the and determine whether the 12 districts but later abandoned. distribution of funding is fair • Reviewed memorandums from fiscal years 2010–11 through 2014–15 that the maintenance and reasonable. division used to notify the 12 districts of their annual allocations. Compared the amounts in the memorandums with reports from Caltrans’ accounting system showing final allocations to identify any adjustments Caltrans may have made during the year. • Compared each district’s allocation with its traffic volume and maintenance performance scores to determine whether the distribution was commensurate with these indicators of maintenance needs and therefore fair and reasonable. • Reviewed the maintenance division’s processes for allocating funding for pavement, bridge, and culvert projects to determine whether they were needs‑based, fair, and reasonable. continued on next page . . . 18 California State Auditor Report 2015-120 March 2016 AUDIT OBJECTIVE METHOD 5 To the extent possible, identify • Determined that all 12 districts received funding in each of the five fiscal years we reviewed. We the current socioeconomic determined the amounts received by each district, as described in objective 4. However, to analyze demographics of each district more precisely where the districts spent their field maintenance funding, we focused on the three receiving funding. Additionally, districts we selected under objective 6. determine the amounts received • Retained a geographic information systems consultant (consultant), ENPLAN, to identify socioeconomic by those districts in each of the demographics and determine where funds were spent in the three districts. We selected median income years reviewed. and race/ethnicity as the socioeconomic demographics we would review. The consultant used data from the U.S. Census Bureau’s American Community Survey Five‑Year Estimate (2009–2013). The consultant used census tract‑level data to create the demographic map layer. • Compiled data related to the maintenance division's spending associated with field maintenance. Using data obtained from Caltrans' Integrated Maintenance Management System (IMMS), we filtered the IMMS data to arrive at the universe we used to complete our analysis. Specifically, the original IMMS data table contained a total of 17,348,866 records with work order cost details. We removed 7,465 records that contained illogical data. In addition, we identified only those work order cost details that were completed during the period July 2010 through June 2015 and were associated with work orders for specific roadway activity expenses—labor, vehicle, and materials—further reducing our universe to 11,296,096 records, which was the starting point of our analysis. • The consultant plotted spending data on maps to identify where spending occurred within each of the three selected districts. • Using Caltrans’ climate and traffic zone data, the consultant identified areas of low spending (less than 50 percent of the average spending per mile in each zone) and high spending (more than 150 percent of the average spending per mile in each zone). • Compared areas of low and high spending with socioeconomic demographic information for the three districts. We found no clear correlation between the spending and demographics. We include the information in the Appendix to provide additional context for the areas of high and low spending. 6 For three districts, including Selected two additional districts—district 4 (Oakland) and district 6 (Fresno)—based on geographic District 7 (Los Angeles and Ventura location, traffic volumes, population, socioeconomic demographics, and maintenance performance levels. counties), perform the following to determine whether program funding is used in an effective and timely manner: a. Evaluate the methodology Confirmed with district deputy directors of maintenance at the three selected districts that they do used to estimate and propose not estimate and propose district maintenance expenditures. Rather, the maintenance division at maintenance expenditures headquarters determines the districts’ allocations. pertaining to the district. b. Review and assess the method • Interviewed relevant staff to determine what processes they had in place to identify and prioritize used to identify and prioritize maintenance projects. maintenance projects. • Reviewed the maintenance division’s process for identifying and prioritizing field maintenance work and pavement, bridge, and culvert projects. • For both pavement and bridge projects at each of the three selected districts, compared projects identified in multiyear plans with the report in which the maintenance division prioritizes the projects it commits to complete each fiscal year. • Reviewed the process for addressing maintenance service requests submitted to each of the selected districts during the last five fiscal years. c. Determine the extent to • Interviewed relevant staff to determine whether they set goals for the districts. which the program is properly • Reviewed maintenance performance goals and analyzed them to determine whether districts were managed and meeting its goals meeting the goals. Interviewed staff to determine whether maintenance performance was used to and objectives. manage maintenance activities. • Obtained reports regarding pavement, bridge, and culvert maintenance that is backlogged. Interviewed relevant staff to understand the reasons for the backlogs. • Reviewed Caltrans’ five‑year maintenance plans, which contain goals for repairing pavement, bridges, and drainage culverts. We compared outstanding bridge maintenance recommendations with completed bridge projects to ensure the maintenance division was completing the bridge work it reported. We found that the number of pavement lane miles requiring maintenance is increasing and that the maintenance division is not making significant progress in repairing drainage culverts, as we discuss further in the Audit Results. California State Auditor Report 2015-120 19 March 2016 AUDIT OBJECTIVE METHOD d. For a selection of maintenance • At each of the three districts, judgmentally selected five highway maintenance projects projects, determine whether (four pavement projects and one bridge project). For each project performed the following: the costs of the projects are – Compared bid amounts to final cost to determine whether costs were reasonable. reasonable, the completion is – Compared number of working days proposed in the bid to actual working days to determine timely, and the reasons for any whether projects were completed in a timely manner. backlogs that exist. • Selected four field maintenance work orders at district 4 and attempted to obtain supporting documentation for the labor, materials, and equipment costs but found that supporting documentation does not exist. Reviewed compensating controls that the maintenance division has put in place to ensure costs are reasonable. Reviewed Caltrans’ internal audits related to field maintenance labor, materials, and equipment costs to identify issues. • Obtained data from Caltrans’ maintenance service request system and calculated the number of days service requests remained open and unresolved. • Reviewed documentation related to goals for backlogged projects as described in objective 6c. e. For a selection of program • Selected the five largest expenditures for highway maintenance projects and five largest expenditures, determine expenditures for field maintenance activities at each of the three selected districts. Reviewed the whether they were reasonable expenditures to determine whether Caltrans complied with applicable provisions in its maintenance and allowable. manual and contract manager’s handbook for approving expenditures and to determine whether the expenditures were reasonable and allowable. • Our review found that the expenditures were generally reasonable and allowable. Specifically, payments were supported by invoices and Caltrans staff had reviewed and approved the invoices for payment. f. Determine the extent to which • Obtained data from the maintenance division regarding the amount it spent for contracts in fiscal the district hires, monitors, and years 2010–11 through 2014–15. evaluates contractors. • For the 15 highway maintenance projects reviewed in objective 6d, reviewed relevant documentation to determine whether districts monitored and evaluated contractors. Our review found Caltrans generally had monitored and evaluated contractors appropriately. g. Identify total available and filled • Obtained data from Caltrans’ personnel system. Caltrans’ data for the first two fiscal years of our audit positions, employees, and period (2010–11 and 2011–12) did not identify maintenance program positions. Analyzed the data vacancies in the program for fiscal years 2012–13 through 2014–15 to identify authorized, filled, and vacant positions and to and the impact vacancies have calculate a vacancy rate for the three selected districts. We also reviewed blanket hiring authority and on the program. calculated how the vacancy rate increased when considering these data. • Interviewed relevant staff to obtain their perspective on the vacancies. 7 Determine if Caltrans is on track • Obtained reports related to recovery act funding from Caltrans. to spend the remaining American • Obtained reports from the California division of the Federal Highway Administration that show Recovery and Reinvestment Act the total recovery act amounts awarded to Caltrans and the total amounts spent as of the last (recovery act) funding before it day the funding was available—September 30, 2015. Calculated the amount that reverted to the reverts to the federal government federal government. and the extent to which these funds can and are being used for the program. 8 Review and assess any other issues Reviewed projects included in the 2000 through 2012 State Highway Operation and Protection that are significant to the audit. Program (SHOPP) plans to determine whether projects had been performed on highways with low and high spending for field maintenance in the districts we reviewed. Sources: California State Auditor’s analysis of the Joint Legislative Audit Committee’s audit request 2015‑120 and information and documentation identified in the table column titled Method. Assessment of Data Reliability In performing this audit, we obtained electronic data files extracted from the information systems listed in Table 4 on the following page. The U.S. Government Accountability Office, whose standards we are statutorily required to follow, requires us to assess the sufficiency and appropriateness of the computer‑processed 20 California State Auditor Report 2015-120 March 2016 information that we use to support our findings, conclusions, and recommendations. Table 4 describes the analyses we conducted using the data from these information systems, our methods for testing them, and the results of our assessments. Although these determinations may affect the precision of the numbers we present, there is sufficient evidence in total to support our audit findings, conclusions, and recommendations. Table 4 Methods Used to Assess Data Reliability INFORMATION SYSTEM PURPOSE METHODS AND RESULTS CONCLUSION California • To quantify the costs and • We performed data‑set verification procedures and Not sufficiently Department of labor hours for roadway field found no issues. Further, we performed electronic reliable for these audit Transportation maintenance work orders testing of key data elements. We identified a number purposes. Although (Caltrans) completed by location. of records with illogical data. In some instances, this determination may • To make a selection of field we removed these records from our analysis as affect the precision Integrated maintenance work orders discussed in Table 3. of the numbers we Maintenance completed in Caltrans districts 4, • We reviewed existing information to determine what present, there is Management 6, and 7. is already known about the data and found that prior sufficient evidence System (IMMS) audit results indicate there are pervasive weaknesses in total to support in Caltrans’ general controls. our audit findings, Maintenance work conclusions, and data from recommendations. July 1, 2010, through June 30, 2015 Position Tracking • To determine the average yearly • We performed data‑set verification procedures and Automated System authorized and vacant full‑time electronic testing of key data elements and did not (PTAS) equivalent maintenance program identify any significant issues. positions for Caltrans districts 4, 6, • We reviewed existing information to determine Position data from and 7 for the period from July 2012 what is already known about the data, and found July 1, 2011, through through June 2015. that prior audit results indicate there are pervasive June 30, 2015 • To determine the average vacancy weaknesses in Caltrans’ general controls. rates for authorized and blanket maintenance program positions in Caltrans districts 4, 6, and 7 in fiscal year 2014–15. Environmental To identify the latitude and longitude • We performed data‑set verification procedures and Systems Research coordinates for postmile points on electronic testing of key data elements and did not Institute (ESRI) California state highways. identify any significant issues. Arc Map system • We reviewed existing information to determine what is already known about the data and found that prior Postmile latitude audit results indicate there are pervasive weaknesses and longitude in Caltrans’ general controls. coordinates as of • To gain some assurance of the completeness of ESRI September 22, 2015 Arc Map data, we traced 29 haphazardly selected latitude and longitude coordinates on California state highways to the ESRI Arc Map data and found no errors. California State Auditor Report 2015-120 21 March 2016 INFORMATION SYSTEM PURPOSE METHODS AND RESULTS CONCLUSION Maintenance Service To determine the status of • We performed data‑set verification procedures and Not sufficiently Request System maintenance service requests electronic testing of key data elements and did not reliable for these audit submitted through Caltrans’ online identify any significant issues. purposes. Although Maintenance service system for Caltrans districts 4, 6, and 7. • We reviewed existing information to determine what this determination may requests submitted is already known about the data and found that prior affect the precision from July 1, 2010, audit results indicate there are pervasive weaknesses of the numbers we through in Caltrans’ general controls. present, there is June 30, 2015, for sufficient evidence Caltrans districts 4, in total to support 6, and 7 our audit findings, conclusions, and A Caltrans‑generated To determine the climate and traffic • We performed data‑set verification procedures and recommendations. data file containing zones of the state highway system in electronic testing of key data elements and did not climate and traffic Caltrans districts 4, 6, and 7. identify any significant issues. volume data for • We reviewed existing information to determine what state highways as is already known about the data and found that prior of 2010. audit results indicate there are pervasive weaknesses in Caltrans’ general controls. Consultants to To make a selection of highway • We performed data‑set verification procedures and Complete for these Government and maintenance program expenditures, electronic testing of key data elements and did not audit purposes. Industry Advantage bridge projects, and pavement projects identify any significant issues. system (Advantage) in Caltrans districts 4, 6, and 7. • These purposes did not require a data reliability assessment. Instead, we gained assurance that the Financial data from population was complete. To test the completeness July 1, 2010, through of Caltrans’ Advantage system, we haphazardly June 30, 2015 selected 29 maintenance program expenditures from the 2014 and 2015 source documents located at Caltrans’ headquarters and five maintenance program expenditures from Caltrans’ archived source documents. We then traced the maintenance program expenditures back to the Advantage data and found the data to be complete. Advantage • To determine allocations • We performed data‑set verification procedures and Not sufficiently of maintenance program electronic testing of key data elements and did not reliable for these audit funding to Caltrans' 12 districts identify any significant issues. purposes. Although and headquarters. • We reviewed existing information to determine what this determination may • To determine the amounts the is already known about the data and found that prior affect the precision maintenance division spent audit results indicate there are pervasive weaknesses of the numbers we on contracts. in Caltrans’ general controls. present, there is sufficient evidence in total to support our audit findings, conclusions, and recommendations. U.S. Census Bureau, To determine the median income We did not assess the reliability of these data because, Undetermined 2013 American and race/ethnicity that composes according to standards of the U.S. Government reliability for this Community Survey the majority of each census tract in Accountability Office, it is not necessary to conduct audit purpose. Five‑Year Estimates Caltrans districts 4, 6, and 7. procedures to verify information that is used for widely (2009–2013) accepted purposes and is obtained from sources generally recognized as appropriate, such as the Socioeconomic U.S. Census. information related to median income and race/ethnicity by census tract State Controller’s To determine the maintenance We assessed the reliability of these expenditure records Sufficiently reliable Office Appropriation division’s expenditures for each by reviewing testing of the appropriation control ledger for the purposes Control Ledger fiscal year from July 2010 through system’s major control features performed as part of the of presenting data June 2015. State’s financial and federal compliance audits. on expenditures. Sources: California State Auditor’s analysis of various documents, interviews, data obtained from Caltrans, and data from State Controller’s Office Appropriation Control Ledger. 22 California State Auditor Report 2015-120 March 2016 Blank page inserted for reproduction purposes only. California State Auditor Report 2015-120 23 March 2016 Audit Results After Beginning to Develop a Logical Approach for Addressing Field Maintenance Needs, the Maintenance Division Abandoned It Through its prior budget model (model) and other planning documents, the California Department of Transportation (Caltrans) had laid out a logical approach for assessing and addressing field maintenance needs.5 This included categorizing all state highways into specific zones based on climate and traffic volume. It also included calculating maintenance performance scores (service scores) for maintenance activities in each zone and setting goals for improving them. In concept this information should logically inform where maintenance needs are greatest and therefore how funding should be allocated and prioritized. However, the maintenance division abandoned this approach. In practice the maintenance division identifies field maintenance work based primarily on physical observations made by maintenance personnel who drive along sections of district roads and the division’s reactions to public complaints. Further, instead of using the more localized zone approach for evaluating service scores, it uses a much higher‑level evaluation by calculating an overall service score for certain maintenance activity categories each district is responsible for and averaging the district scores to derive statewide scores. In addition, the maintenance division only sets performance goals for improving three of the many categories of maintenance activities it reports on annually. It also has based annual funding allocations to the 12 districts on a simple average of historical spending rather than using service scores or zone‑level information, despite reporting to the Legislature that it was using a more sophisticated method. Thus, it is not surprising that we found actual spending has not improved service scores and has been inconsistent among roadways in similar zones. The Maintenance Division Does Not Use Key Indicators and Performance Information to Strategically Plan Field Maintenance Activities The Federal Highway Administration, other state departments of transportation, and additional entities have established that traffic volume, climate, and environmental factors are key factors that influence highway maintenance needs. Further, Caltrans’ 5 This budget model is not the same as the statutorily required budget model described in the Introduction. The maintenance division intended to use the field maintenance budget model to allocate field maintenance funding among the 12 districts. In contrast, the statutorily required budget model is intended to balance resources between the State Highway Operation and Protection Program and the Maintenance Program. Caltrans includes the statutorily required budget model in the section of its five‑year maintenance plan titled “Analysis of Alternative Levels of Maintenance Investment.” 24 California State Auditor Report 2015-120 March 2016 maintenance manual states that the need for maintenance activities, such as routine litter removal and road sweeping frequencies, is very sensitive to traffic volumes. In 2009 the maintenance division developed a budget model for its field maintenance program that accounted for some of these key factors. Specifically, that model would have allowed the maintenance division to identify the resources needed to maintain highways with similar climate and traffic volume at a given maintenance performance level. However, despite paying a consultant roughly $250,000 to develop the model, the maintenance division never implemented it. According to the deputy director for maintenance and operations (deputy director)— who was the chair of the steering committee responsible for overseeing the consultant’s efforts in developing and implementing the model—the model suggested unreasonable allocations to the districts, such as reducing district 7’s (Los Angeles) staff by roughly 100 positions, so the maintenance division abandoned it. In concept, the model would have shifted the maintenance division’s assessment of maintenance Highway Zone Climate and Traffic Categories need from a district‑level analysis to a more Climate Region granular and informative route‑level analysis. After dividing all state highways, or routes, • Inland Valley into segments by county, the model grouped • Desert these segments into one of five climate regions and six traffic volume categories, as shown • Low Mountain in the text box. Generally, roads with higher • High Mountain traffic volume have greater field maintenance • North Coast needs. For example, the model categorized the segment of Interstate 80 in Sacramento County • South Coast as having inland valley climate and the highest Traffic Volume (average daily traffic) traffic volume, level 5 (Inland‑5). The model • Level 1: 0–2,500 referred to each combination of climate region and traffic volume category, such as Inland‑5, • Level 2: 2,501–10,000 as zones. The model established service scores • Level 3: 10,001–25,000 at the zone level and, using historical trends of expenditures and labor hours, estimated the • Level 4: 25,001–100,000 resources needed to maintain those scores. Finally, • Level 5: 100,001 + using this zone‑level information and data about Source: California Department of Transportation’s budget the inventory of assets subject to maintenance model User Manual and Technical Guide. activities, the model would have established baseline funding needs and allocations to maintain similar highways within the state to a specified service score.6 6 Inventory refers to individual elements of statewide infrastructure that include lane miles of roads, bridges, tunnels, etc. California State Auditor Report 2015-120 25 March 2016 However, the maintenance division did not implement the zone‑level evaluation of service scores it had envisioned in its abandoned budget model. While that approach would have allowed it to monitor each district’s maintenance performance by zone, its current methodology of sampling roads for evaluation does not provide this level of information. Specifically, the current methodology gathers information only for each district as a whole and does not accurately identify differences in maintenance needs that exist within each district. As mentioned in the Introduction, the maintenance division calculates service scores for many maintenance activities but only sets service score goals for three of them: litter and debris, guardrails, and striping. In addition, these goals apply only to overall statewide performance, not to individual districts. In Figure 4 on the following page we present the statewide service goals and service scores for the categories of maintenance activities for which the maintenance division has established service score goals. We also include the service scores for each of the three districts we reviewed: district 4 (Oakland), district 6 (Fresno), and district 7 (Los Angeles). The maintenance division calculates the statewide service scores by averaging the respective scores of each district. From fiscal year 2010–11 through 2014–15, the state average service score was below the goal for striping and guardrails, but was above the goal for litter and debris. However, the service scores for the three districts we reviewed were generally below the state average. Moreover, although service scores for these districts have been volatile, scores for districts 4 and 7 have generally been well under the state average, as Figure 4 shows. However, the maintenance division does not weight districts’ service scores to account for differences in traffic, terrain, or climate, even though the maintenance performance statewide report for fiscal year 2014–15 stated that readers should consider these characteristics when comparing scores for each district. For example, districts 4 (Oakland) and 7 (Los Angeles), which handle almost 43 percent of the State’s traffic volume, have the same impact on the overall state service score as do districts 1 (Eureka) and 9 (Bishop), which together handle about 1 percent of the State’s traffic. Evaluating maintenance performance without Evaluating maintenance including traffic volume allows the several small districts with performance without including minimal traffic to outweigh the few large districts with the majority traffic volume allows the several of the State’s traffic, skewing the statewide score higher. When small districts with minimal traffic we weighted the score for fiscal year 2014–15 to reflect the traffic to outweigh the few large districts volume in each district, we found that although the statewide score with the majority of the State’s for guardrails did not change, the scores for striping and litter and traffic, skewing the statewide debris worsened by between five and nine points. score higher. 26 California State Auditor Report 2015-120 March 2016 Figure 4 Service Scores for Categories With Service Goals for Districts 4 (Oakland), 6 (Fresno), and 7 (Los Angeles) Fiscal Years 2010–11 Through 2014–15 100 90 80 70 60 50 40 100 90 80 70 60 50 40 100 90 80 70 60 50 40 erocS erocS erocS Litter and Debris Level of Service State Average Goal District 6 (Fresno) District 4 (Oakland) District 7 (Los Angeles) 2010–11 2011–12 2012–13 2013–14 2014–15 Striping Level of Service Goal District 6 (Fresno) State Average District 7 (Los Angeles) District 4 (Oakland) 2010–11 2011–12 2012–13 2013–14 2014–15 Guardrail Level of Service Goal District 6 (Fresno) District 4 (Oakland) State Average District 7 (Los Angeles) 2010–11 2011–12 2012–13 2013–14 2014–15 Years Sources: California Department of Transportation’s Maintenance Level of Service Statewide Reports Executive Summaries for fiscal years 2010–11 through 2014–15. California State Auditor Report 2015-120 27 March 2016 Instead of holding individual districts accountable for meeting service score goals for field maintenance activities, the maintenance division established spending goals for specific activities that the districts monitor as indicators of performance. When we inquired as to how these spending goals were derived, the division was unable to support how the spending goals were originally established—it only asserted that they are adjusted based on past expenditures and service scores. According to the assistant maintenance division chief for the office of administration and budgets (assistant division chief), these spending goals provide direction to the districts on improving service scores. Although there is value in monitoring spending, this approach alone does not account for the actual activities that need to be completed or the outcomes from that spending that would result in improved service scores. In fact, our review found instances where the districts’ service scores decreased even though the districts sometimes exceeded their spending targets. For example, despite spending more than its target for litter and debris since fiscal year 2012–13, district 7’s (Los Angeles) service scores for this category declined. Despite the fact that the service scores for the districts we reviewed have generally not met the statewide goals, the maintenance division could not demonstrate that it is using this information to strategically plan its work to address maintenance needs and improve service scores. While the maintenance division’s service score evaluation guide states that managers can use service scores in conjunction with their own knowledge and other evaluation data to develop plans and set priorities, we found that districts do not use the service scores to identify and prioritize their field maintenance work. Deputy district directors of maintenance Because districts are not (district maintenance deputies) we spoke with stated that they do using information regarding field not use service scores to inform how they plan or prioritize the maintenance needs to plan and field maintenance work they perform. Because districts are not prioritize their field maintenance using information regarding field maintenance needs to plan and work, the work appears to be prioritize, their field maintenance work appears to be performed on performed on an impromptu basis an impromptu basis and is not consistent with need. and is not consistent with need. In addition, the three districts we reviewed generally create only short‑term plans for activities they intend to complete during the following one or two weeks; such plans do not adequately establish priorities or facilitate the monitoring of progress toward improving service scores. For example, district 6 (Fresno) provided us with emails that supervisors sent to superintendents that briefly described activities they intended to complete over the next week. In the 2005 version of its maintenance manual, the maintenance division included requirements for districts to formally plan and monitor field maintenance work using the maintenance management system and requirements for region managers and 28 California State Auditor Report 2015-120 March 2016 district maintenance deputies to review annual workload plans. The manual also required districts to take into account service scores at the route level when developing plans. These plans would have enabled management to establish and document priorities and develop a baseline by which to measure progress. For example, the maintenance manual specifies that districts are to establish routine litter removal and sweeping frequencies for each highway segment based on the rate at which litter, debris, and sediment accumulate. However, according to the division chief, the maintenance division subsequently removed the field planning requirements from the maintenance manual because districts determined that using the planning and scheduling module in the maintenance management system was very time‑consuming. Despite the significant benefits of formally planning field maintenance work, the maintenance division did Without adequate plans for not develop alternate planning methods that it believed were more completing field maintenance feasible. Without adequate plans for completing field maintenance work, district staff are not being work, district staff are not being held accountable for how well they held accountable for how well they address field maintenance needs, as evidenced by the low service address field maintenance needs. scores and the inconsistent spending discussed in a later section. The division chief also asserted that maintenance crews cannot always adhere to work plans because they are regularly redirected from completing planned work to address emergency work, such as clearing accident scenes and addressing other urgent highway issues. However, we do not agree with that rationale for not planning work. If crews cannot strictly follow their plan because of the responsive nature of maintenance work, these deviations from the plan will shed light on what work activity is not getting accomplished and what changes to resource allocations should be made. While we agree that the maintenance division must respond to emergencies and may need to interrupt or reschedule routine work, those interruptions make it especially important to plan and prioritize the maintenance division’s other work to ensure that it is adequately addressing field maintenance needs. The Maintenance Division Does Not Allocate Field Maintenance Funding to Its Districts Based on Key Indicators of Need The maintenance division does not use key indicators of field maintenance need, such as traffic volume and service scores, to allocate funding to the districts. Thus, some districts may not be receiving adequate funding to meet their field maintenance needs, which may delay the work and result in more costly future repairs. We found that the proportion of maintenance funds the maintenance division allocated to each district remained roughly the same for fiscal years 2010–11 through 2014–15, as Table 5 shows. California State Auditor Report 2015-120 29 March 2016 5 elbaT sretrauqdaeH dna stcirtsiD eht ot gnidnuF margorP ecnanetniaM fo snoitacollA ecnanetniaM fo noisiviD noitatropsnarT fo tnemtrapeD ainrofilaC 51–4102 hguorhT 11–0102 sraeY lacsiF 51–4102 RAEY LACSIF 41–3102 RAEY LACSIF 31–2102 RAEY LACSIF 21–1102 RAEY LACSIF 11–0102 RAEY LACSIF %4 960,662,24$ %4 680,393,54$ %4 937,116,74$ %3 529,768,23$ %3 266,920,34$ )akeruE( 1 tcirtsiD 6 974,960,07 6 911,809,46 5 746,718,16 5 341,776,16 5 128,373,66 )gniddeR( 2 tcirtsiD 01 582,698,611 9 178,572,901 8 266,169,29 9 644,863,001 9 016,878,511 )ellivsyraM( 3 tcirtsiD 31 518,504,061 31 776,294,151 51 629,165,271 31 859,011,051 21 774,158,451 *)dnalkaO( 4 tcirtsiD 5 104,159,75 4 414,352,74 3 477,125,63 5 247,803,35 4 408,905,25 5 tcirtsiD )opsibO siuL naS( 8 465,964,39 7 078,964,18 7 979,017,18 7 028,585,18 6 232,302,08 *)onserF( 6 tcirtsiD 41 728,094,761 31 776,959,151 31 483,775,541 51 094,344,371 31 143,854,461 *)selegnA soL( 7 tcirtsiD 9 196,699,601 7 928,494,58 8 323,578,39 01 642,607,801 9 899,899,701 )onidranreB naS( 8 tcirtsiD 3 061,851,33 3 228,352,13 3 733,275,03 2 941,484,72 2 650,043,03 )pohsiB( 9 tcirtsiD 6 843,509,96 5 549,115,16 5 783,335,75 6 448,963,46 5 575,571,46 )notkcotS( 01 tcirtsiD 6 243,307,57 6 609,567,56 5 713,817,75 6 997,145,66 6 938,770,77 )ogeiD naS( 11 tcirtsiD 4 598,196,24 4 848,911,54 4 641,128,34 4 478,335,74 4 507,031,74 )enivrI( 21 tcirtsiD 41 421,078,161 81 567,407,012 81 359,513,802 51 465,892,271 02 088,671,442 †sretrauqdaeH ‡%001 000,578,891,1$ ‡%001 928,306,151,1$ ‡%001 475,995,031,1$ ‡%001 000,792,041,1$ ‡%001 000,502,842,1$ slatoT .metsys laicnanfi s’noitatropsnarT fo tnemtrapeD ainrofilaC :ecruoS .tidua ruo fo trap sa weiver deliated rof )selegnA soL( 7 tcirtsid dna ,)onserF( 6 tcirtsid ,)dnalkaO( 4 tcirtsid detceles eW * fo noisivid ,secivres gnireenigne fo noisivid ,ecnanetniam fo noisivid :snoisivid ruof gniwollof eht ot yliramirp saw sretrauqdaeh ot gnidnuf ecnanetniam fo noitacolla s’noisivid ecnanetniam ehT † .snoitagitsevni dna stidua fo noisivid dna ,)esuoheraw ,yllacfiiceps( stcartnoc dna tnemerucorp .gnidnuor ot eud sraey lacsfi lla rof tnecrep 001 yltcaxe ot dda ton od segatnecreP ‡ 30 California State Auditor Report 2015-120 March 2016 At most, the percentage of allocations to each of the districts changed by zero to two percentage points year to year. The allocations for headquarters had slightly wider swings; its allocations ranged from a high of 20 percent in fiscal year 2010–11 to a low of 14 percent in fiscal year 2014–15. Although these allocations generally aligned with the number of lane miles in each district, they were not commensurate with other indicators of maintenance need. One key indicator of each district’s maintenance need is traffic volume, which was a key element of the abandoned budget model and, as previously mentioned, a key indicator used by several other governmental entities. Figure 5 shows that although some districts have a significantly higher proportion of traffic volume, the maintenance division does not allocate a commensurate proportion of funding to those districts. For example, as previously mentioned, together districts 4 (Oakland) and 7 (Los Angeles) handle 43 percent of the State’s traffic volume, but these two districts together received only 27 percent of maintenance program’s funding in fiscal year 2014–15. On the other hand, multiple districts with significantly lower traffic volume received disproportionately large allocations. For instance, districts 1 (Eureka), 2 (Redding), and 9 (Bishop) together received 13 percent of total program funding in fiscal year 2014–15, despite only handling 3 percent of the state’s traffic volume. Although traffic volume is not the only indicator of maintenance need, the large differences between districts’ proportions of traffic volume and allocations suggest that the maintenance division should revise its allocation methodology. The maintenance division does not The maintenance division also does not allocate funding based allocate funding based on districts’ on districts’ service scores. Consequently, some districts may not service scores. Consequently, some have the resources to improve those scores. As we describe in the districts may not have the resources Introduction, the maintenance division annually determines service to improve those scores. scores to measure districts’ field maintenance performance. The maintenance division’s performance evaluation guide states that one important objective of evaluating performance is to assess statewide maintenance needs. However, although service scores at some districts were generally low over the last five fiscal years, the maintenance division did not revise those districts’ overall allocations to address those low service scores. For example, from fiscal year 2010–11 through 2014–15, service scores for litter and debris, striping, and guardrails at districts 4 (Oakland) and 7 (Los Angeles) have, for the most part, been significantly below both the state goal and the state average. Because the maintenance division does not allocate its funding commensurate with key indicators of maintenance needs, some districts may not be able to fully address these needs, which can delay maintenance work and result in more costly repairs in the California State Auditor Report 2015-120 31 March 2016 future. This effect is exacerbated because headquarters is potentially underfunding those districts where a significant portion of the State’s highway traffic is occurring and that have low service scores. Figure 5 Percentages of Maintenance Program Funding Districts Received in Fiscal Year 2014–15 Compared to Districts’ Percentages of 2013 Daily Vehicle Miles of Travel on State Highways District 7 (Los Angeles)* District 4 (Oakland)* District 8 (San Bernardino) District 11 (San Diego) District 12 (Irvine) District 3 (Marysville) District 6 (Fresno)* District’s percentage of total daily vehicle District 10 (Stockton) miles of travel in 2013 on state highways District 5 (San Luis Obispo) District’s allocation as a percentage of total maintenance program funding in fiscal year 2014–15† District 2 (Redding) District 1 (Eureka) District 9 (Bishop)‡ 0 5 10 15 20 25% Percentage Sources: California State Auditor’s analysis of allocation data for fiscal year 2014–15 from California Department of Transportation’s (Caltrans) financial system and daily vehicle miles of travel on state highways data from Caltrans’ 2013 California Public Road Data report. * We selected district 7 (Los Angeles), district 4 (Oakland), and district 6 (Fresno) for detailed review as part of our audit. † Caltrans headquarters received 14 percent of maintenance program funding in fiscal year 2014–15. ‡ District 9 (Bishop) handles 0.4 percent of the State’s daily vehicle miles of travel on state highways. 32 California State Auditor Report 2015-120 March 2016 The Maintenance Division Has Mischaracterized the Sophistication of the Method It Uses to Allocate Field Maintenance Funding to the Districts Although it did not implement the model that it developed for its field maintenance activities, the maintenance division has been reporting to the Legislature and other decision makers that it was using the model’s sophisticated methodology for allocating field maintenance funds. As we previously Excerpt from the California Department noted, the maintenance division has continued to of Transportation’s 2015 allocate funding in roughly the same proportions Five‑Year Maintenance Plan from fiscal year 2010–11 to 2014–15. For example, The Maintenance Program Budget Model was developed district 7 (Los Angeles) has received between 13 and to enhance budget management capabilities on an annual 15 percent of the total allocation in each of the last basis. The performance‑based model uses a combination five fiscal years. Although the maintenance division’s of historical expenditures, Level of Service performance process for allocating funding to the districts is not measures, and inventory data to project future resource well documented, the maintenance division asserted needs with performance‑level expectations for the that it has allocated field maintenance funding entire State Highway System. While the budget model to the districts using the number of positions in does include all resources, the model is used primarily each district and averaging the districts’ historical to determine a performance‑based budget for field expenditures. The maintenance division annually maintenance activities. informs districts of their allocations and instructs Historically, the budget model grouped Caltrans’ 12 districts districts not to exceed them. We verified with the into sets of comparable units using geographic, population, three districts we reviewed that they do not estimate and traffic volume characteristics. Current versions of the and propose allocations to the maintenance division. model have shifted the focus from district‑level analysis to The deputy director and the division chief stated route‑level analysis by grouping all routes based on traffic that they do not know when the maintenance volumes and geographic locations. Through route‑level division originally started using this allocation analysis, the budget model provides detailed comparative methodology or how it originally determined analysis for determining relationships between performance and resource needs for each highway inventory unit which the number of staff positions and the funding creates standardized allocation and efficiency rates for allocations for field activities for each district. each route. However, Caltrans consistently reported that it was The budget model is used: using the model “to determine a performance‑based • To measure the direct relation between funding and budget for field maintenance activities” in the 2011, Level of Service, and the effects of changes to either. 2013, and 2015 updates to its five‑year maintenance • To measure an efficiency curve at the route level for plans (maintenance plan). As shown in the text box, statewide comparisons. in these plans Caltrans also provided additional details and reported that “through route‑level • To measure a standardized allocation process for analysis, the budget model provided detailed inventory items at a route level for life‑cycle cost and asset comparative analysis for determining relationships management practices. between performance and resource needs for each • To assist decision makers in determining the best course of highway inventory unit which created standardized action relative to budgetary and performance issues. allocation and efficiency rates for each route.” • To predict funding needs based on project delivery actions However, as previously mentioned, according to the and decisions. deputy director, the maintenance division has never used the model to calculate route‑level allocations to Source: California Department of Transportation’s 2015 Five‑Year Maintenance Plan (maintenance plan). the districts for the entire field maintenance budget. Note: The 2011 and 2013 maintenance plans contained the Consequently, the Legislature and other decision same descriptions of the budget model, except for minor makers may have believed that the maintenance word changes. division was using a more sophisticated approach to allocate funding to the districts than it actually Excerpt from the California Department of Transportation’s 2015 Five‑Year Maintenance Plan “The Maintenance Program Budget Model was developed to enhance budget management capabilities on an annual basis. The performance‑based model uses a combination of historical expenditures, Level of Service performance measures, and inventory data to project future resource needs with performance‑level expectations for the entire State Highway System. While the budget model does include all resources, the model is used primarily to determine a performance‑based budget for field maintenance activities. Historically, the budget model grouped Caltrans’ 12 districts into sets of comparable units, using geographic, population, and traffic characteristics. Current versions of the model have shifted the focus from district‑level analysis to route‑level analysis by grouping all routes based on traffic volumes and geographic locations. Through route‑level analysis, the budget model provides detailed comparative analysis for determining relationships between performance and resource needs for each highway inventory unit which creates standardized allocation and efficiency rates for each route. The budget model is used: • To measure the direct relation between funding and Level of Service, and the effects of changes to either. • To measure an efficiency curve at the route level for statewide comparisons. • To measure a standardized allocation process for inventory items at a route level for life‑cycle cost and asset management practices. California State Auditor Report 2015-120 33 March 2016 • To assist decision makers in determining the best course of action relative to budgetary and performance issues. • To predict funding needs based on project delivery actions and decisions.” Source: California Department of Transportation’s 2015 Five‑Year Maintenance Plan (maintenance plan). has, and thus they may have been less willing to challenge or question Note: The 2011 and 2013 maintenance plans contained the the allocations. The division chief stated that his staff will revise the same descriptions of the budget model, except for minor language in the subsequent maintenance plans to more accurately word changes. depict how the maintenance division currently allocates funding for field activities to the districts. Although the division chief agreed that allocating funding for field activities using traffic volume and service scores would result in allocations commensurate with districts’ needs, he stated it would not improve large urban districts’ service scores because these districts would not be able to spend their increased allocations. According to the deputy director and the division chief, large urban districts—such as districts 4 (Oakland) and 7 (Los Angeles)—cannot fill many of their field staff positions because of staff turnover resulting from the high cost of living, terms in union agreements that affect how the program advertises field positions, and the department’s lengthy hiring process. However, if indicators of maintenance need show that funding allocations should be adjusted, we believe the maintenance division needs to work on resolving any challenges that it encounters. In addition, our analysis showed that the vacancy rates in the three districts that we reviewed remained consistently low. Specifically, the average vacancy rate for authorized positions was 3 percent or less from fiscal year 2012–13 through 2014–15. In fiscal year 2014–15, districts 4 (Oakland), 6 (Fresno), and 7 (Los Angeles) on average had 912, 325, and 876 authorized positions, respectively. Additionally, Caltrans asserted that it allocates blanket positions to the districts, and when we considered that information, average vacancy rates at the reviewed districts did not increase significantly and ranged from 4 percent to 10 percent in fiscal year 2014–15.7 According to the assistant division chief, Caltrans is reviewing its existing hiring processes and policies to expedite the hiring process for field maintenance staff. The Maintenance Division Does Not Spend Its Field Maintenance Funds Consistent With Some Indicators of Need Given the shortcomings with the maintenance division’s approach to planning and allocating funds we discussed in previous sections, we were not surprised to find that actual spending for field maintenance work was not consistent with key indicators of need. Our review of Caltrans districts 4 (Oakland), 6 (Fresno), 7 In its March 2012 Budget Letter 12‑03, the Department of Finance stated that departments can use blanket hiring authority to fill positions beyond approved position authority on a temporary basis for operational needs. Specifically, it stated that departments can use blanket authority for temporary issues, such as hiring retired annuitants or seasonal staff, or payment of leave balances. According to this budget letter, departments also may hire permanent employees within the blanket authority if no vacant positions exist; however, permanent employees must be moved from the blanket authority once authorized positions become vacant. 34 California State Auditor Report 2015-120 March 2016 and 7 (Los Angeles) found that each district spent different amounts for field maintenance on highways with similar maintenance needs based on key indicators Caltrans identified—traffic volume and climate. For example, each of the three districts spent different average amounts per mile on certain highways that had similar field maintenance needs. In addition, we found that each district spent significantly less on some highways in its district than it spent on The maintenance division’s others with similar maintenance needs. While other indicators inconsistent spending for field of maintenance need may also affect spending, the inconsistent maintenance suggests that it should spending we identified suggests that the maintenance division assess whether it is effectively should assess whether it is effectively addressing maintenance needs. addressing maintenance needs. By potentially underspending, it increases the personal safety and environmental risks posed by unmet maintenance needs, such as excess litter or graffiti, guardrails in need of repair, unfilled potholes, and untended landscaping. Further, unperformed maintenance activities can result in more significant future repair costs. When we used the abandoned model’s zone methodology to compare average spending per mile during our audit period, we found districts sometimes spent significantly different amounts for field maintenance on highways with similar maintenance needs. For example, each of the three districts we reviewed has highways in an inland valley climate with traffic volume levels 1 through 4. Table 6 shows that district 7 spent roughly twice as much per mile on highways classified as Inland‑3 as did districts 4 and 6: $154,000 compared to $71,000 and $62,000, respectively. In addition, district 7 spent more than $1 million per mile to maintain its four‑mile‑long section of Inland‑2 highway, while it spent only $334,000 per mile to maintain its Inland‑5 highways. According to the district maintenance deputy for district 7 (Los Angeles), the high spending along this section of Inland‑2 highway was driven by work including landscaping, highway sweeping, and removal of debris and graffiti, which required costly lane closures. Table A in the Appendix shows the average spending per mile for each zone in the three Caltrans districts we reviewed. Because highways with similar characteristics should give an indication of similar maintenance and resource needs regardless of which district they are in, these spending variances among the districts may mean that the districts are not spending sufficient resources to meet maintenance needs, not receiving sufficient resources, or they are not spending their resources efficiently. For the three reviewed districts, we plotted the maintenance division’s field maintenance expenditure data from fiscal years 2010–11 through 2014–15 on highway zone maps to see how each district’s field maintenance spending corresponded to maintenance need. Specifically, we first calculated the average dollars that each district spent per mile in each of the zones. Then we compared sections of highways that are in similar zones and identified those sections where the district spent below 50 percent of the average. We considered California State Auditor Report 2015-120 35 March 2016 this to be low spending. We also identified high spending for each zone as more than 150 percent of the average spent per mile in the respective zone. As part of our analysis, we reviewed SHOPP projects planned since 2000 that rehabilitated the pavement and might have reduced the amount of spending for field maintenance needed on the highways we identified. We found that SHOPP projects had been performed on some roads with low spending as well as on some roads with high spending; thus, it appears that the inconsistent spending is not entirely explained by those projects. Table 6 Average Spending per Mile in Inland Valley Climate Zones Fiscal Years 2010–11 Through 2014–15 CLIMATE / TRAFFIC ZONE AVERAGE DOLLARS SPENT PER MILE PER ZONE CLIMATE TRAFFIC VOLUME* DISTRICT 4 (OAKLAND) DISTRICT 6 (FRESNO) DISTRICT 7 (LOS ANGELES) Inland Valley 1 (0–2,500) $22,000 $47,000 $79,000 Inland Valley 2 (2,501–10,000) 41,000 32,000 1,007,000 Inland Valley 3 (10,001–25,000) 71,000 62,000 154,000 Inland Valley 4 (25,001–100,000) 90,000 115,000 107,000 Inland Valley 5 (100,001+) 159,000 NA 334,000 Sources: California Department of Transportation’s (Caltrans) integrated maintenance management system and Caltrans’ climate region and traffic volume data. Note: Average dollars spent per mile are weighted for mileage and rounded to the nearest thousand. NA = District 6 (Fresno) does not have any highways in the inland valley climate with a traffic volume 5. * Traffic volume indicates average daily traffic. Figure 6 on the following page identifies in blue the sections of highways on which district 4 spent less for field maintenance than it spent on other highways in the same district with similar maintenance needs. For example, the blue text box in Figure 6 shows a section of an approximately 41‑mile stretch of Route 82 in district 4 that is in the North Coast‑4. On average, district 4 spent $104,000 per mile to maintain North Coast‑4 roads. However, it spent only an average of $28,000 per mile for field maintenance on this particular section of Route 82, despite this section having maintenance needs similar to the district’s other North Coast‑4 roads. This is a difference of 73 percent— or almost $76,000 less per mile in spending—compared to other similar roads in district 4. Similarly, Figure 6 shows in red the sections of highways in district 4 with high spending. In particular, the red text box in Figure 6 shows district 4 spent $66,000 more per mile than the average of $56,000 for its Low Mountain‑3 highways, or $122,000 per mile on an 11‑mile section of Route 116 in Sonoma County, which is more than twice the average. Both district 6 and district 7 also spent field maintenance funds inconsistently on highways with similar maintenance needs. Figures 7 and 8 on pages 37 and 38 show sections of highways in districts 6 and 7, respectively, on which these districts spent more than 150 percent of the average or more than 50 percent below the average. 36 California State Auditor Report 2015-120 March 2016 Figure 6 Field Maintenance Per‑Mile Spending, District 4 (Oakland) Highways Distric t 1 Distric t 3 Route 116 Climate: Low Mountain Traffic Volume: 3* Length: 11 miles Spending: $66,000 more per mile SONOMA than zone average of $56,000 NAPA Santa Rosa Fairfield SOLANO MARIN CONTRA COSTA Oakland SAN FRANCISCO Route 82 Climate: North Coast Distric t 10 Traffic Volume: 4† Length: 41 miles ALAMEDA Spending: $76,000 less per mile than zone average of $104,000 San Jose SAN MATEO SANTA CLARA LEGEND District headquarters Highways with low spending (<50 percent of zone average) Highways with high spending (>150 percent of zone average) Other Caltrans maintained highways Distric t 5 District 4 boundary Miles Counties 0 17 Sources: California State Auditor geographic information system consultant’s analysis of cost data from California Department of Transportation’s (Caltrans) integrated maintenance management system for fiscal years 2010–11 through 2014–15 and Caltrans’ climate region and traffic volume data. * Traffic volume 3 indicates average daily traffic of 10,001–25,000. † Traffic volume 4 indicates average daily traffic of 25,001–100,000. California State Auditor Report 2015-120 37 March 2016 Figure 7 Field Maintenance Per‑Mile Spending, District 6 (Fresno) Highways Nevada District District 9 10 Route 198 MADERA Climate: Inland Valley Traffic Volume: 3* Length: 72 miles Madera Spending: $67,000 more per mile FRESNO than zone average of $62,000 Fresno Visalia TULARE Coalinga KINGS District 5 Route 5 McFarland Climate: Inland Valley Traffic Volume: 4† Length: 103 miles Spending: $78,000 less per mile KERN than zone average of $115,000 LEGEND District headquarters Highways with low spending (<50 percent of zone average) Highways with high spending District (>150 percent of zone average) 7 Other Caltrans maintained highways District 6 boundary Miles Counties 0 26 Sources: California State Auditor geographic information system consultant’s analysis of cost data from California Department of Transportation’s (Caltrans) integrated maintenance management system for fiscal years 2010–11 through 2014–15 and Caltrans’ climate region and traffic volume data. * Traffic volume 3 indicates average daily traffic of 10,001 – 25,000. † Traffic volume 4 indicates average daily traffic of 25,001 – 100,000. 38 California State Auditor Report 2015-120 March 2016 8 erugiF syawhgiH )selegnA soL( 7 tcirtsiD ,gnidnepS eliM‑reP ecnanetniaM dleiF 6 tcirtsiD retsacnaL ARUTNEV tcirtsiD 5 tcirtsiD 8 SELEGNA SOL 101 etuoR tsaoC htuoS :etamilC †5 :emuloV cffiarT selim 23 :htgneL dranxO elim rep ssel 000,182$ :gnidnepS 000,954$ fo egareva enoz naht anomoP selegnA soL DNEGEL sretrauqdaeh tcirtsiD notpmoC 1 etuoR gnidneps wol htiw syawhgiH )egareva enoz fo tnecrep 05<( tcirtsiD 21 tsaoC htuoS :etamilC gnidneps hgih htiw syawhgiH *3 :emuloV cffiarT )egareva enoz fo tnecrep 051>( selim 51 :htgneL elim rep erom 000,64$ :gnidnepS syawhgih deniatniam snartlaC rehtO seliM 000,87$ fo egareva enoz naht yradnuob 7 tcirtsiD 21 0 seitnuoC metsys tnemeganam ecnanetniam detargetni )snartlaC( s’noitatropsnarT fo tnemtrapeD ainrofilaC morf atad tsoc fo sisylana s’tnatlusnoc metsys noitamrofni cihpargoeg rotiduA etatS ainrofilaC :secruoS atad emulov cffiart dna noiger etamilc ’snartlaC dna 51–4102 hguorht 11–0102 sraey lacsfi rof .000,52–100,01 fo cffiart yliad egareva setacidni 3 emulov cffiarT * .+100,001 fo cffiart yliad egareva setacidni 5 emulov cffiarT † California State Auditor Report 2015-120 39 March 2016 We also compared the three districts’ spending with their median income and their race and ethnicity demographics and found there was no clear correlation. In the Appendix beginning on page 49, we describe our analysis and present the socioeconomic demographic information as additional context for the districts’ spending. Specifically, in the Appendix, Figures A–C on pages 52 to 54 show the median income and Figures D–F on pages 55 to 57 show the demographics for the predominant racial and ethnic makeup for the three districts we reviewed. As part of our review, we brought the field maintenance spending discrepancies to the attention of the districts, and we asked for their perspective on why field maintenance spending varied among highways with similar needs in their respective district. The district maintenance deputies indicated that several factors could be affecting the spending, such as previous SHOPP projects that reduced the need for field maintenance, the age of the road, whether the road is in a rural or urban area, the number and cost of lane closures required to perform the field maintenance work, the number of service requests received for a particular road, and the amount of landscaping on a particular road. However, as we described previously, the maintenance division does not consider these factors in its planning or allocating for field maintenance work. Although we are not suggesting that Caltrans spent money on projects that were not worthwhile or where there was no indicator of need, we are concerned that there are many highways in each district on which average spending per mile over a five‑year period was inconsistent with the maintenance needs indicated by the nature of the climate and traffic volumes in those zones. The Maintenance Division Cannot Demonstrate That It Promptly Performs Field Maintenance Work The three districts we reviewed do not effectively manage the service requests they receive from the public to ensure that they appropriately prioritize and address them in a timely manner. Specifically, data indicate that more than 30,000 service More than 30,000 service requests requests received by the three districts in fiscal years 2010–11 received by the three districts in through 2014–15 remained unresolved after more than 90 days. fiscal years 2010–11 through 2014–15 Although these districts asserted that many of those service remained unresolved after more requests have been completed, neither we nor the districts’ than 90 days. management could easily verify this assertion. Unperformed maintenance work can create safety risks and can also result in increased future repair costs. The public can identify needed maintenance work by using an electronic form on Caltrans’ website that allows the public to notify the maintenance program of a specific maintenance issue—for 40 California State Auditor Report 2015-120 March 2016 example, graffiti, litter, or a pothole—and its location. The maintenance division tracks these service requests in its statewide maintenance service request system (service request system). After receiving a service request from the public, a district assesses the request to decide how it should respond. Specifically, the district determines whether the reported issue occurred on the state highway system and if the district needs to address it. After conducting this assessment, the district responds to the individual who submitted the service request, stating whether the district commits to perform the work or stating the reasons why it is not going to perform the work. Although the maintenance division has not established a time frame for completing service requests, we analyzed the service requests that the maintenance division had committed to addressing by grouping them in 30‑day increments to determine how long they had been outstanding. Data from the service request system suggests that two of the three districts we reviewed frequently let service requests remain outstanding for more than 90 days, as shown in Table 7. For example, from fiscal year 2010–11 through 2014–15, more than 16,000 service requests submitted to district 7 remained outstanding for more than 90 days. Similarly, during the same time period, the system data showed that more than 15,000 service requests submitted to district 4 had not been addressed after more than 90 days. Table 7 Status of Maintenance Service Requests Received by California Department of Transportation Districts 4 (Oakland), 6 (Fresno), and 7 (Los Angeles) Fiscal Years 2010–11 Through 2014–15 DISTRICT 4 (OAKLAND) DISTRICT 6 (FRESNO) DISTRICT 7 (LOS ANGELES) NUMBER PERCENT OF NUMBER PERCENT OF NUMBER PERCENT OF OF SERVICE DISTRICT’S SERVICE OF SERVICE DISTRICT’S SERVICE OF SERVICE DISTRICT’S SERVICE REQUESTS* REQUESTS REQUESTS* REQUESTS REQUESTS* REQUESTS Completed service requests: Completed in 30 or fewer days 14,893 41% 1,103 77% 1,400 7% Completed in 31 to 60 days 3,116 9 57 4 121 1 Completed in 61 to 90 days 823 2 51 4 62 0.3 Completed in more than 90 days 1,295 4 148 10 184 1 Outstanding service requests: Outstanding for 30 or fewer days as of 413 1 2 0.1 327 2 June 30, 2015 Outstanding for 31 to 60 days as of 377 1 2 0.1 227 1 June 30, 2015 Outstanding for 61 to 90 days as of 376 1 5 0.3 388 2 June 30, 2015 Outstanding for more than 90 days as of 15,288 42 61 4.3 16,783 86 June 30, 2015 Totals 36,581 100%† 1,429 100%† 19,492 100%† Source: California State Auditor’s analysis of data from California Department of Transportation’s maintenance service request system. * In this analysis we included only those service requests for which the districts committed to perform the work. † Percentages do not add to exactly 100 percent for all fiscal years due to rounding. California State Auditor Report 2015-120 41 March 2016 The districts’ current processes for tracking service requests make it difficult and time‑consuming to determine whether maintenance staff have completed service requests and if data in the service request system are accurate. Districts use the service request system to track service requests from the public while district maintenance staff use the maintenance management system to record that they completed the work associated with these service requests. However, the two systems do not automatically exchange information between one another, and the districts have to manually update the service request system to show which service requests were completed. Otherwise, the data on the status of service requests could be inaccurate. District maintenance deputies could not verify whether the large number of service requests that were outstanding for more than 90 days had been completed, and they indicated that doing so would require a significant effort. Nevertheless, we believe the maintenance division should have a process for following up on those service requests that it does not promptly address. Furthermore, the districts we reviewed do not capture in the service request system those service requests they receive via methods other than Caltrans’ website, making it even more difficult to monitor the progress of addressing those additional service requests. The district maintenance deputies at the three districts we reviewed confirmed that their districts do not have documented policies that describe how they should handle service requests received by phone, mail, or email. Although some districts have tools to record such service requests, none of the districts we reviewed record such requests in one central repository. For example, some regions in district 4 (Oakland) use paper logs to record phone calls. In another example, according to the district As a result of the districts’ poor maintenance deputy for district 6 (Fresno), the district does not processes for recording service have any process to record and monitor service requests it receives requests they receive via methods by phone, mail, or email. As a result of the districts’ poor processes other than Caltrans’ website, for recording service requests they receive via methods other than neither we nor the three districts’ Caltrans’ website, neither we nor the three districts’ staff could staff could ensure that such service ensure that such service requests were addressed in a timely way or requests were addressed timely or addressed at all. addressed at all. Similarly, the maintenance program does not have processes to ensure that maintenance staff complete work orders in a timely manner. As we described in the Introduction, the maintenance manual requires maintenance staff to create work orders in the maintenance management system for all field maintenance work they perform. The system has fields to record the dates when a work order is initiated and when it is completed, and these dates can be used to determine how long it took to complete the work. However, according to the office chief of management systems and studies within the maintenance division, supervisors do not consistently 42 California State Auditor Report 2015-120 March 2016 create work orders as soon as they identify field maintenance needs; rather, they often create work orders after their crews start the work. Thus, the true length of time it takes from the date the work is identified to the date it is completed is not captured, and the maintenance division cannot accurately monitor whether it is addressing field maintenance needs in a timely manner. Weak Controls Over Field Maintenance Work Orders Create Opportunities for Fraud, Waste, and Abuse Caltrans’ weak controls over field maintenance do not adequately ensure that work order costs are reasonable and allowable and that the resources used were necessary and appropriate. Our review of field maintenance work orders found that the integrated maintenance management system is paperless and no supporting documentation is maintained for work order costs, such as labor, equipment, and materials used to complete field maintenance work. We reviewed the compensating internal controls the maintenance division has developed to ensure that costs are reasonable and allowable. The maintenance manual describes controls including superintendent reviews of work orders, which are intended to monitor work order costs. Internal controls are essential to ensure that the maintenance division achieves its objectives and mitigates the risk of fraud, waste, and abuse. Maintenance supervisors record on each work order all costs incurred to complete the work. For example, a supervisor records in the work order how many hours each staff member works and the costs of all materials and vehicles Although the maintenance manual used to complete maintenance activities. Although the maintenance requires superintendents to review manual requires superintendents to review work orders for material work orders for material usage and usage and accurate time reporting, there is no evidence that they accurate time reporting, there is no do so. In particular, according to the office chief of management evidence that they do so. systems and studies, the current maintenance management system does not allow reviewers to document that they have reviewed a work order. Without such evidence, the maintenance division cannot ensure that the labor, equipment, and materials used are reasonable and appropriate. We also reviewed Caltrans’ internal division of audits and investigations’ (internal audits) reviews or audits of these costs and found that the internal auditors had identified weaknesses. For example, a 2015 Caltrans internal review of district 8 (San Bernardino) found weaknesses in controls over materials such as the chemicals used for controlling vegetation. Specifically, internal audits’ review noted variances in 20 of 40 inventory counts between the count and the inventory records for those items. The review indicated that when physical inventory levels are not reconciled to balances recorded in the maintenance management system, theft or misuse of the chemicals can go undetected. Internal California State Auditor Report 2015-120 43 March 2016 audits made recommendations to address these issues but has not yet followed up to determine whether the recommendations have been implemented. Additionally, an internal audit released in 2013 found that some maintenance staff entered erroneous time sheets into the system, including time sheets that exceeded available leave balances and time sheets showing total numbers of hours worked that did not match the employees’ work schedules. The audit also found that staff failed to promptly correct errors and that superintendents did not review and approve time sheets until the end of the month, which, according to the internal auditors, could result in errors and misuse of leave. Caltrans’ internal auditors made recommendations for these findings that the maintenance division has reported addressing. The Maintenance Division Appropriately Manages Highway Maintenance Projects, but Some Backlogged Maintenance Work Is Increasing In contrast to the field maintenance activities, the maintenance division currently uses a needs‑based approach to allocate funding for the majority of highway maintenance activities and to identify highway maintenance projects. The maintenance division allocates The maintenance division currently funding for highway maintenance proportionally to the districts uses a needs-based approach based on the condition of the pavement and bridges in each district. to allocate funding for the For example, it allocates funding for bridge projects based on each majority of highway maintenance district’s portion of outstanding bridge maintenance work that activities and to identify highway regular bridge inspections identify. Similarly, it determines districts’ maintenance projects. allocations for pavement projects using data about the amount and condition of the pavement in each district. In contrast, according to the office chief for storm water and environmental compliance, the maintenance division generally allocates highway maintenance funds for drainage and culverts equally among the districts because the budget is relatively small compared to the identified needs of the districts. Currently, the maintenance division generally identifies projects to repair pavement by conducting field reviews and considering data from the pavement condition survey, which uses customized vehicles fitted with sensing equipment traveling on the roadway at or near highway speeds to collect pavement condition data. Based on these pavement surveys, each lane mile is rated as good, fair, or poor. Further, the maintenance division stated that it is in the process of implementing a new technology called PaveM that it indicates will optimize the project selection process by targeting future repairs that provide the best value for the least amount of money. According to the 2015 State of the Pavement report, PaveM will make suggestions based on pavement condition, pavement type, climate, and project history. However, the division has not set a final implementation date for this new technology. 44 California State Auditor Report 2015-120 March 2016 Our review of highway maintenance projects for pavement and bridges found that generally the maintenance division ensured that contractors completed projects within budget and on time. While the maintenance division lacks strong controls for field maintenance, controls for highway maintenance projects to contain costs generally appear to be adequate. For highway maintenance projects that contractors perform, Caltrans’ resident engineers review contract payment requests before authorizing contractor payments, which entails comparing completed work with the contract. We found that contractors generally completed the 15 projects we reviewed at an actual cost that was reasonably close to their bid amounts and within the number of days estimated in the bid. However, the maintenance division’s goals to repair pavement statewide fail to address its growing backlog of needed pavement maintenance. Specifically, pavement that is in fair condition with minor surface distress, such as minor cracking and potholes, but that still requires corrective maintenance has increased. According to Caltrans’ State of the Pavement reports, the backlog of lane miles in need of corrective maintenance has increased from 11,053 in 2011 to 15,272 in 2015 due to the increase in construction costs and a greater percentage of higher‑cost maintenance strategies. In Caltrans’ 2015 maintenance plan, the maintenance division established a goal to repair 2,100 lane miles a year based on the funding it received, although Caltrans reported that it did not meet this goal. The maintenance division is not The maintenance division is also not sufficiently addressing sufficiently addressing backlogged backlogged inspections and repairs of drainage culverts even inspections and repairs of drainage though culvert damage or failure can seriously damage roadways, culverts, even though culvert create the need for extensive repairs, and threaten the mobility damage or failure can seriously and safety of the traveling public. In the 2015 maintenance plans, damage roadways, cause extensive Caltrans set goals to inspect 12,000 culverts and repair 140 culverts repairs, and threaten the mobility annually. Caltrans reported that it inspected 13,168 culverts in and safety of travelers. fiscal year 2013–14, which exceeded its goal. Although Caltrans has generally met its culvert repair goals during our audit period, the goal to repair 140 culverts annually does not sufficiently address the significant increase in backlogged maintenance needs. The number of culverts Caltrans reported as needing repair has consistently grown since fiscal year 2010–11, from an estimated amount of 13,185 to 27,346 as of June 2015. The backlog is increasing in part because Caltrans’ culvert inspection program has identified and assessed the condition of 107,000 of the estimated 205,000 culverts in the State. As this process continues, Caltrans is identifying more culverts in need of repair. Additionally, Caltrans reports that a large percentage of culvert inspections and repairs it identified through previous maintenance plans have included “easier” access and California State Auditor Report 2015-120 45 March 2016 repairs, and it anticipates that the remaining culvert inspections and repairs will be more difficult to address and will require additional time and planning to complete. According to the division chief, Caltrans has taken an approach of developing SHOPP projects that focus on repairing pavement and culverts that are in the worst condition first. In fact, Caltrans reported in its 2015 maintenance plan that if it were to receive additional revenue, it would prioritize SHOPP pavement rehabilitation ahead of preventative maintenance. Caltrans also reported that it would recommend funding both maintenance and SHOPP to manage and maintain backlogged culverts. Given the need to balance resources between SHOPP and maintenance, we believe Caltrans should continue to monitor and report on its backlogged maintenance work to ensure that the backlogged work does not deteriorate to such a degree that it results in increased future SHOPP costs. In contrast to its lack of progress on pavement and culverts, in its 2015 Five‑Year Maintenance Plan Caltrans reports that it is reducing its backlogged highway maintenance projects for bridges. During our audit period Caltrans set a goal to reduce the number of bridges with backlogged maintenance needs to 8 to 10 percent of the total inventory of bridges. The bridge inventory reportedly varied between 12,900 and 13,100 during our audit period. Caltrans defines bridge maintenance needs as backlogged if two years pass without completion of the maintenance after a bridge inspector recommends maintenance work. During our audit period, Caltrans reported that bridges with backlogged maintenance needs decreased from 2,575 in the beginning of fiscal year 2010–11 to 1,771 in the beginning of fiscal year 2013–14. Caltrans reports it is consistently reducing the backlog and estimates that the backlog will continue to decrease at the current level of funding. In 2015 Caltrans set a goal to reduce the number of backlogged bridges to 1,090 over the next five years. Recommendations To better align the maintenance division’s allocations with districts’ maintenance needs, the Legislature should include language in the Budget Act that requires the maintenance division to develop and implement a budget model for field maintenance by June 30, 2017, that takes into account key indicators of maintenance need, such as traffic volume, climate, service scores, and any other factors the maintenance division deems necessary to ensure that the model adequately considers field maintenance need. Once the model is developed, Caltrans should use it to inform appropriate allocations to the districts. 46 California State Auditor Report 2015-120 March 2016 Caltrans should revise the language in its future five‑year maintenance plans to accurately describe the method it uses to allocate field maintenance funding to its districts. To ensure that it performs field maintenance work consistently on highways with similar needs, the maintenance division should do the following: • Assess whether districts are using funds in a manner commensurate with indicators of need included in its new budget model. • Implement the zone‑level evaluation of service scores contemplated in the earlier budget model that it abandoned. • Establish zone‑specific service score goals for all of the field maintenance activities it deems critical to ensuring a safe and usable state highway system and require districts to meet those goals for all the zones within their borders. • Implement the requirements for strategically planning field maintenance work that it previously included in its maintenance manual or develop similar requirements that it believes are feasible and ensure that supervisors plan and schedule field maintenance work based on service scores. Caltrans should require superintendents and regional managers to approve those plans. Caltrans should also require supervisors and superintendents to monitor progress toward improving service scores. Caltrans should require its staff to verify and update the status of all outstanding service requests. Additionally, Caltrans should require supervisors to monitor completion of service requests by reviewing the data from the service request system monthly to identify service requests not completed after a period of time that Caltrans deems appropriate, such as 30 days. For all service requests outstanding after this period, Caltrans should require its supervisors to determine the status of the service request by reviewing the related work order that records what work Caltrans completed and ensure the work is appropriately prioritized. Also, Caltrans should require its staff to record all service requests it receives via methods other than Caltrans’ website, such as by phone, mail, or email, in its service request system to ensure it captures all service requests in one central repository. To detect and prevent fraud, waste, and abuse and to ensure costs are appropriate, the maintenance division should strengthen its controls over reviewing and approving work order costs by requiring its supervisors and superintendents to document California State Auditor Report 2015-120 47 March 2016 their review and approval of work orders in the maintenance management system. For example, supervisors or superintendents could include a note in the comment field of the work order indicating their review and approval. The maintenance division could also establish a reasonable dollar threshold for those work orders that would require documented review and approval. To ensure that field maintenance work orders are completed in a timely manner, the maintenance division should require supervisors to initiate work orders in the integrated maintenance management system at the time that they identify field maintenance work that needs to be performed and record the date that work was started and the date the work was completed. Superintendents should periodically review work orders to ensure that identified work is completed in a timely manner. We conducted this audit under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives specified in the Scope and Methodology section of the report. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor Date: March 17, 2016 Staff: Nicholas Kolitsos, CPA, Audit Principal Jordan Wright, CFE Martin Lee, CPA Caitlin Palmer Natalja Zvereva IT Audits: Michelle J. Baur, CISA, Audit Principal Lindsay H. Harris, MBA, CISA Ryan P. Coe, MBA, CISA Shauna M. Pellman, MPPA, CIA Legal Counsel: Heather Kendrick, Sr. Staff Counsel Geographic Information System Consultant: ENPLAN For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. 48 California State Auditor Report 2015-120 March 2016 Blank page inserted for reproduction purposes only. California State Auditor Report 2015-120 49 March 2016 Appendix SPENDING BY ZONE AND DEMOGRAPHICS IN DISTRICTS WE REVIEWED The Joint Legislative Audit Committee requested that the California State Auditor, to the extent possible, identify the current socioeconomic demographics of each district receiving funding and determine the amounts received by those districts in each of the fiscal years from 2010–11 through 2014–15. Our review found that all 12 of Caltrans’ districts received funding in each of the five years we reviewed, as we show in Table 5 on page 29 in the Audit Results. However, to analyze more precisely where the districts spent their field maintenance funding, we focused on three selected districts: district 4 (Oakland), district 6 (Fresno), and district 7 (Los Angeles). Table A on the following page shows the amounts that these districts spent by zone—that includes information on the respective district’s climate and average daily traffic volume—for field maintenance during those fiscal years. Additionally, we retained a geographic information systems consultant (consultant) to plot on maps each of the three districts’ spending for field maintenance activities. Because we had this detailed spending information for the three districts, we attempted to focus our review of socioeconomic demographics in the three districts as well. Specifically, the consultant identified median income, race, and ethnicity using tract‑level data from the U.S. Census Bureau and displayed that information on maps of each of the three districts. The consultant then overlaid the spending information on the socioeconomic demographic information to allow for comparison. However, when compared with the areas of high and low spending, which we described more fully in the Audit Results, we found no clear correlation between the levels of spending and income levels or race and ethnicity. In particular, we found there were areas of both high and low spending in each of the three districts on highway sections that pass through areas where the income level and race and ethnicity of the majority of the population varies. Figures A, B, and C beginning on page 52 show the median income by census tract for districts 4, 6, and 7, respectively, as well as the sections of highway with high and low spending. Figures D, E, and F beginning on page 55 show the race and ethnicity that make up the majority of the population by census tract and also show sections of highway with high and low spending for districts 4, 6, and 7, respectively. 50 California State Auditor Report 2015-120 March 2016 Table A Average Field Maintenance Spending for All Zones, Fiscal Years 2010–11 Through 2014–15 CLIMATE/TRAFFIC VOLUME ZONE DISTRICT 4 (OAKLAND) DISTRICT 6 (FRESNO) DISTRICT 7 (LOS ANGELES) AVERAGE AVERAGE AVERAGE TOTAL DOLLARS DOLLARS PER LOW HIGH MILES TOTAL DOLLARS DOLLARS PER LOW HIGH MILES TOTAL DOLLARS DOLLARS PER LOW HIGH MILES CLIMATE TRAFFIC VOLUME SPENT PER ZONE MILE PER ZONE* SPENDING† SPENDING‡ PER ZONE SPENT PER ZONE MILE PER ZONE* SPENDING† SPENDING‡ PER ZONE SPENT PER ZONE MILE PER ZONE* SPENDING† SPENDING‡ PER ZONE Inland Valley 1 (0–2,500) $375,710.45 $22,287.45 $11,143.73 $33,431.18 16.86 $3,666,054.71 $47,208.53 $23,604.26 $70,812.79 77.66 $259,193.54 $79,239.62 $39,619.81 $118,859.43 3.27 Inland Valley 2 (2,501–10,000) 944,121.07 40,713.21 20,356.61 61,069.82 23.19 25,433,601.30 31,791.50 15,895.75 47,687.25 800.01 4,069,583.12 1,006,514.37 503,257.19 1,509,771.56 4.04 Inland Valley 3 (10,001–25,000) 4,845,630.08 71,484.97 35,742.48 107,227.45 67.79 20,074,759.03 62,147.17 31,073.59 93,220.76 323.02 3,740,766.02 153,715.30 76,857.65 230,572.96 24.34 Inland Valley 4 (25,001–100,000) 6,941,588.39 90,478.41 45,239.20 135,717.61 76.72 45,667,935.76 114,704.49 57,352.25 172,056.74 398.14 11,748,779.09 106,949.49 53,474.75 160,424.24 109.85 Inland Valley 5 (100,001 +) 19,891,237.73 159,026.46 79,513.23 238,539.69 125.08 – – – – – 83,071,283.38 334,360.57 167,180.29 501,540.86 248.45 Low Mountain 1 (0–2,500) – – – – – – – – – – – – – – – Low Mountain 2 (2,501–10,000) 3,531,767.55 52,872.49 26,436.25 79,308.74 66.80 1,957,977.08 58,868.65 29,434.32 88,302.97 33.26 – – – – – Low Mountain 3 (10,001–25,000) 3,638,269.71 55,509.59 27,754.80 83,264.39 65.54 1,004,738.56 42,658.46 21,329.23 63,987.70 23.55 – – – – – Low Mountain 4 (25,001–100,000) 11,375,363.86 72,260.75 36,130.37 108,391.12 157.42 – – – – – – – – – – Low Mountain 5 (100,001 +) 7,799,224.64 269,418.77 134,709.39 404,128.16 28.95 – – – – – – – – – – High Mountain 1 (0–2,500) – – – – – 1,629,771.50 43,966.07 21,983.04 65,949.11 37.07 1,598,298.95 68,962.35 34,481.18 103,443.53 23.18 High Mountain 2 (2,501–10,000) – – – – – 3,755,295.38 29,519.39 14,759.70 44,279.09 127.21 1,631,350.60 38,086.00 19,043.00 57,128.99 42.83 High Mountain 3 (10,001–25,000) – – – – – 2,252,342.02 61,383.75 30,691.88 92,075.63 36.69 5,599,175.72 82,278.42 41,139.21 123,417.63 68.05 High Mountain 4 (25,001–100,000) – – – – – 3,348,310.01 326,445.69 163,222.85 489,668.54 10.26 1,672,756.96 80,457.81 40,228.91 120,686.72 20.79 High Mountain 5 (100,001 +) – – – – – – – – – – 3,153,884.31 113,928.26 56,964.13 170,892.39 27.68 North Coast 1 (0–2,500) 1,320,531.03 69,575.17 34,787.59 104,362.76 18.98 – – – – – – – – – – North Coast 2 (2,501–10,000) 5,753,410.81 35,380.75 17,690.37 53,071.12 162.61 – – – – – – – – – – North Coast 3 (10,001–25,000) 4,348,776.18 34,858.65 17,429.33 52,287.98 124.75 – – – – – – – – – – North Coast 4 (25,001–100,000) 21,129,074.56 103,678.39 51,839.20 155,517.59 203.79 – – – – – – – – – – North Coast 5 (100,001 +) 60,365,108.91 206,707.77 103,353.88 310,061.65 292.03 – – – – – – – – – – Desert 1 (0–2,500) – – – – – – – – – – – – – – – Desert 2 (2,501–10,000) – – – – – 3,805,550.52 72,276.70 36,138.35 108,415.05 52.65 120,105.24 26,004.91 13,002.45 39,007.36 4.62 Desert 3 (10,001–25,000) – – – – – 4,045,807.77 39,869.42 19,934.71 59,804.13 101.48 1,626,026.53 30,268.86 15,134.43 45,403.29 53.72 Desert 4 (25,001–100,000) – – – – – – – – – – 1,012,904.02 51,328.44 25,664.22 76,992.66 19.73 Desert 5 (100,001 +) – – – – – – – – – – – – – – – South Coast 1 (0–2,500) – – – – – – – – – – 303,194.93 34,605.00 17,302.50 51,907.50 8.76 South Coast 2 (2,501–10,000) – – – – – – – – – – 3,125,485.66 64,012.32 32,006.16 96,018.48 48.83 South Coast 3 (10,001–25,000) – – – – – – – – – – 4,503,036.79 78,138.87 39,069.44 117,208.31 57.63 South Coast 4 (25,001–100,000) – – – – – – – – – – 13,898,230.08 104,498.70 52,249.35 156,748.05 133.00 South Coast 5 (100,001 +) – – – – – – – – – – 107,138,000.23 458,915.41 229,457.71 688,373.12 233.46 Total miles with low or high spending: 304.29 Total miles with low or high spending: 646.60 Total miles with low or high spending: 217.06 Percentage of miles in district with low or high spending: 21% Percentage of miles in district with low or high spending: 32% Percentage of miles in district with low or high spending: 19% Sources: California State Auditor geographic information system consultant’s analysis of cost data from California Department of Transportation’s (Caltrans) integrated maintenance management system for fiscal years 2010–11 through 2014–15 and Caltrans’ climate region and traffic volume data. * Calculations of average dollars per mile per zone are weighted for number of miles in each section of the highway. † Low spending is field maintenance spending more than 50 percent below the average per‑mile spending for the zone. ‡ High spending is field maintenance spending more than 150 percent of the average per‑mile spending for the zone. – District does not have highways in this zone. California State Auditor Report 2015-120 51 March 2016 Table A Average Field Maintenance Spending for All Zones, Fiscal Years 2010–11 Through 2014–15 CLIMATE/TRAFFIC VOLUME ZONE DISTRICT 4 (OAKLAND) DISTRICT 6 (FRESNO) DISTRICT 7 (LOS ANGELES) AVERAGE AVERAGE AVERAGE TOTAL DOLLARS DOLLARS PER LOW HIGH MILES TOTAL DOLLARS DOLLARS PER LOW HIGH MILES TOTAL DOLLARS DOLLARS PER LOW HIGH MILES CLIMATE TRAFFIC VOLUME SPENT PER ZONE MILE PER ZONE* SPENDING† SPENDING‡ PER ZONE SPENT PER ZONE MILE PER ZONE* SPENDING† SPENDING‡ PER ZONE SPENT PER ZONE MILE PER ZONE* SPENDING† SPENDING‡ PER ZONE Inland Valley 1 (0–2,500) $375,710.45 $22,287.45 $11,143.73 $33,431.18 16.86 $3,666,054.71 $47,208.53 $23,604.26 $70,812.79 77.66 $259,193.54 $79,239.62 $39,619.81 $118,859.43 3.27 Inland Valley 2 (2,501–10,000) 944,121.07 40,713.21 20,356.61 61,069.82 23.19 25,433,601.30 31,791.50 15,895.75 47,687.25 800.01 4,069,583.12 1,006,514.37 503,257.19 1,509,771.56 4.04 Inland Valley 3 (10,001–25,000) 4,845,630.08 71,484.97 35,742.48 107,227.45 67.79 20,074,759.03 62,147.17 31,073.59 93,220.76 323.02 3,740,766.02 153,715.30 76,857.65 230,572.96 24.34 Inland Valley 4 (25,001–100,000) 6,941,588.39 90,478.41 45,239.20 135,717.61 76.72 45,667,935.76 114,704.49 57,352.25 172,056.74 398.14 11,748,779.09 106,949.49 53,474.75 160,424.24 109.85 Inland Valley 5 (100,001 +) 19,891,237.73 159,026.46 79,513.23 238,539.69 125.08 – – – – – 83,071,283.38 334,360.57 167,180.29 501,540.86 248.45 Low Mountain 1 (0–2,500) – – – – – – – – – – – – – – – Low Mountain 2 (2,501–10,000) 3,531,767.55 52,872.49 26,436.25 79,308.74 66.80 1,957,977.08 58,868.65 29,434.32 88,302.97 33.26 – – – – – Low Mountain 3 (10,001–25,000) 3,638,269.71 55,509.59 27,754.80 83,264.39 65.54 1,004,738.56 42,658.46 21,329.23 63,987.70 23.55 – – – – – Low Mountain 4 (25,001–100,000) 11,375,363.86 72,260.75 36,130.37 108,391.12 157.42 – – – – – – – – – – Low Mountain 5 (100,001 +) 7,799,224.64 269,418.77 134,709.39 404,128.16 28.95 – – – – – – – – – – High Mountain 1 (0–2,500) – – – – – 1,629,771.50 43,966.07 21,983.04 65,949.11 37.07 1,598,298.95 68,962.35 34,481.18 103,443.53 23.18 High Mountain 2 (2,501–10,000) – – – – – 3,755,295.38 29,519.39 14,759.70 44,279.09 127.21 1,631,350.60 38,086.00 19,043.00 57,128.99 42.83 High Mountain 3 (10,001–25,000) – – – – – 2,252,342.02 61,383.75 30,691.88 92,075.63 36.69 5,599,175.72 82,278.42 41,139.21 123,417.63 68.05 High Mountain 4 (25,001–100,000) – – – – – 3,348,310.01 326,445.69 163,222.85 489,668.54 10.26 1,672,756.96 80,457.81 40,228.91 120,686.72 20.79 High Mountain 5 (100,001 +) – – – – – – – – – – 3,153,884.31 113,928.26 56,964.13 170,892.39 27.68 North Coast 1 (0–2,500) 1,320,531.03 69,575.17 34,787.59 104,362.76 18.98 – – – – – – – – – – North Coast 2 (2,501–10,000) 5,753,410.81 35,380.75 17,690.37 53,071.12 162.61 – – – – – – – – – – North Coast 3 (10,001–25,000) 4,348,776.18 34,858.65 17,429.33 52,287.98 124.75 – – – – – – – – – – North Coast 4 (25,001–100,000) 21,129,074.56 103,678.39 51,839.20 155,517.59 203.79 – – – – – – – – – – North Coast 5 (100,001 +) 60,365,108.91 206,707.77 103,353.88 310,061.65 292.03 – – – – – – – – – – Desert 1 (0–2,500) – – – – – – – – – – – – – – – Desert 2 (2,501–10,000) – – – – – 3,805,550.52 72,276.70 36,138.35 108,415.05 52.65 120,105.24 26,004.91 13,002.45 39,007.36 4.62 Desert 3 (10,001–25,000) – – – – – 4,045,807.77 39,869.42 19,934.71 59,804.13 101.48 1,626,026.53 30,268.86 15,134.43 45,403.29 53.72 Desert 4 (25,001–100,000) – – – – – – – – – – 1,012,904.02 51,328.44 25,664.22 76,992.66 19.73 Desert 5 (100,001 +) – – – – – – – – – – – – – – – South Coast 1 (0–2,500) – – – – – – – – – – 303,194.93 34,605.00 17,302.50 51,907.50 8.76 South Coast 2 (2,501–10,000) – – – – – – – – – – 3,125,485.66 64,012.32 32,006.16 96,018.48 48.83 South Coast 3 (10,001–25,000) – – – – – – – – – – 4,503,036.79 78,138.87 39,069.44 117,208.31 57.63 South Coast 4 (25,001–100,000) – – – – – – – – – – 13,898,230.08 104,498.70 52,249.35 156,748.05 133.00 South Coast 5 (100,001 +) – – – – – – – – – – 107,138,000.23 458,915.41 229,457.71 688,373.12 233.46 Total miles with low or high spending: 304.29 Total miles with low or high spending: 646.60 Total miles with low or high spending: 217.06 Percentage of miles in district with low or high spending: 21% Percentage of miles in district with low or high spending: 32% Percentage of miles in district with low or high spending: 19% Sources: California State Auditor geographic information system consultant’s analysis of cost data from California Department of Transportation’s (Caltrans) integrated maintenance management system for fiscal years 2010–11 through 2014–15 and Caltrans’ climate region and traffic volume data. * Calculations of average dollars per mile per zone are weighted for number of miles in each section of the highway. † Low spending is field maintenance spending more than 50 percent below the average per‑mile spending for the zone. ‡ High spending is field maintenance spending more than 150 percent of the average per‑mile spending for the zone. – District does not have highways in this zone. 52 California State Auditor Report 2015-120 March 2016 Figure A Median Income and Field Maintenance Per‑Mile Spending, District 4 (Oakland) Distric t 1 Distric t 3 SONOMA NAPA Santa Rosa Fairfield SOLANO MARIN CONTRA COSTA Oakland SAN FRANCISCO Distric t 10 ALAMEDA LEGEND District headquarters Highways with low spending San Jose (<50 percent of zone average) Highways with high spending SAN MATEO SANTA CLARA (>150 percent of zone average) Other Caltrans maintained highways District 4 boundary Counties MEDIAN INCOME $12,018 - $59,492 $59,493 - $80,187 $80,188 - $106,567 Distric t 5 $106,568 - $250,000 No data Miles 0 17 Sources: California State Auditor geographic information system consultant’s analysis of cost data from California Department of Transportation’s (Caltrans) integrated maintenance management system for fiscal years 2010–11 through 2014–15 and Caltrans’ climate region and traffic volume data. Median income categories classify the district’s census tracts into four groups of an equal number of tracts based on their distribution of median income using data from the U.S. Census Bureau’s 2009–2013 American Community Survey Five‑Year Estimate. California State Auditor Report 2015-120 53 March 2016 F igure B Median Income and Field Maintenance Per‑Mile Spending, District 6 (Fresno) Nevada District 10 MADERA Madera District FRESNO 9 Fresno Visalia TULARE Coalinga KINGS District 5 McFarland LEGEND District headquarters Highways with low spending KERN (<50 percent of zone average) Highways with high spending (>150 percent of zone average) Other Caltrans maintained highways District 6 boundary Counties MEDIAN INCOME $11,070 - $31,885 District 7 $31,886 - $43,646 $43,64L7 E- G$5E9N,0D92 $59,093 - $126,713 Miles No data 0 26 Sources: California State Auditor geographic information system consultant’s analysis of cost data from California Department of Transportation’s (Caltrans) integrated maintenance management system for fiscal years 2010–11 through 2014–15 and Caltrans’ climate region and traffic volume data. Median income categories classify the district’s census tracts into four groups of an equal number of tracts based on their distribution of median income using data from the U.S. Census Bureau’s 2009–2013 American Community Survey Five‑Year Estimate. 54 California State Auditor Report 2015-120 March 2016 C erugiF )selegnA soL( 7 tcirtsiD ,gnidnepS eliM‑reP ecnanetniaM dleiF dna emocnI naideM 6 tcirtsiD tcirtsiD retsacnaL 5 tcirtsiD 8 ARUTNEV SELEGNA SOL DNEGEL sretrauqdaeh tcirtsiD gnidneps wol htiw syawhgiH dranxO )egareva enoz fo tnecrep 05<( gnidneps hgih htiw syawhgiH anomoP )egareva enoz fo tnecrep 051>( syawhgih deniatniam snartlaC rehtO selegnA soL yradnuob 7 tcirtsiD seitnuoC notpmoC EMOCNI NAIDEM tcirtsiD 21 776,93$ - 092,01$ 754,55$ - 876,93$ 579,67$ - 854,55$ seliM 21 0 410,722$ - 679,67$ atad oN tnemeganam ecnanetniam detargetni )snartlaC( s’noitatropsnarT fo tnemtrapeD ainrofilaC morf atad tsoc fo sisylana s’tnatlusnoc metsys noitamrofni cihpargoeg rotiduA etatS ainrofilaC :secruoS na fo spuorg ruof otni stcart susnec s’tcirtsid eht yfissalc seirogetac emocni naidem tcirtsiD .atad emulov cffiart dna noiger etamilc ’snartlaC dna 51–4102 hguorht 11–0102 sraey lacsfi rof metsys .etamitsE raeY‑eviF yevruS ytinummoC naciremA 3102–9002 s’uaeruB susneC .S.U eht morf atad gnisu emocni naidem fo noitubirtsid rieht no desab stcart fo rebmun lauqe California State Auditor Report 2015-120 55 March 2016 Figure D Race and Ethnicity and Field Maintenance Per‑Mile Spending, District 4 (Oakland) Distric t 1 Distric t 3 SONOMA NAPA Santa Rosa Fairfield SOLANO MARIN CONTRA COSTA Oakland SAN FRANCISCO Distric t 10 ALAMEDA LEGEND District headquarters Highways with low spending San Jose (<50 percent of zone average) Highways with high spending SAN MATEO SANTA CLARA (>150 percent of zone average) Other Caltrans maintained highways District 4 boundary Counties RACE and ETHNICITY Asian Black or African American Hispanic or Latino (of any Race) Distric t 5 White Miles No data 0 17 Sources: California State Auditor geographic information system consultant’s analysis of cost data from California Department of Transportation’s (Caltrans) integrated maintenance management system for fiscal years 2010–11 through 2014–15 and Caltrans’ climate region and traffic volume data. The consultant also calculated the race and ethnic group that made up the highest percentage of each census tract’s population using data and terms from the U.S. Census Bureau’s 2009–2013 American Community Survey Five‑Year Estimate. 56 California State Auditor Report 2015-120 March 2016 Figure E Race and Ethnicity and Field Maintenance Per‑Mile Spending, District 6 (Fresno) Nevada District 10 MADERA Madera District FRESNO 9 Fresno Visalia TULARE Coalinga KINGS District 5 McFarland KERN LEGEND District headquarters Highways with low spending (<50 percent of zone average) Highways with high spending (>150 percent of zone average) Other Caltrans maintained highways District 6 boundary District 7 Counties RACE and ETHNICITY Hispanic or Latino (of any Race) Miles 0 26 White Sources: California State Auditor geographic information system consultant’s analysis of cost data from California Department of Transportation’s (Caltrans) integrated maintenance management system for fiscal years 2010–11 through 2014–15 and Caltrans’ climate region and traffic volume data. The consultant also calculated the race and ethnic group that made up the highest percentage of each census tract’s population using data and terms from the U.S. Census Bureau’s 2009–2013 American Community Survey Five‑Year Estimate. California State Auditor Report 2015-120 57 March 2016 F erugiF )selegnA soL( 7 tcirtsiD ,gnidnepS eliM‑reP ecnanetniaM dleiF dna yticinhtE dna ecaR 6 tcirtsiD tcirtsiD retsacnaL 5 ARUTNEV tcirtsiD 8 SELEGNA SOL DNEGEL sretrauqdaeh tcirtsiD gnidneps wol htiw syawhgiH )egareva enoz fo tnecrep 05<( dranxO gnidneps hgih htiw syawhgiH )egareva enoz fo tnecrep 051>( anomoP syawhgih deniatniam snartlaC rehtO selegnA soL yradnuob 7 tcirtsiD seitnuoC notpmoC YTICINHTE dna ECAR tcirtsiD 21 naisA naciremA nacirfA ro kcalB )ecaR yna fo( onitaL ro cinapsiH seliM etihW 21 0 atad oN ecnanetniam detargetni )snartlaC( s’noitatropsnarT fo tnemtrapeD ainrofilaC morf atad tsoc fo sisylana s’tnatlusnoc metsys noitamrofni cihpargoeg rotiduA etatS ainrofilaC :secruoS pu edam taht puorg cinhte dna ecar eht detaluclac osla tnatlusnoc ehT .atad emulov cffiart dna noiger etamilc ’snartlaC dna 51–4102 hguorht 11–0102 sraey lacsfi rof metsys tnemeganam .etamitsE raeY‑eviF yevruS ytinummoC naciremA 3102–9002 s’uaeruB susneC .S.U eht morf smret dna atad gnisu noitalupop s’tcart susnec hcae fo egatnecrep tsehgih eht 58 California State Auditor Report 2015-120 March 2016 Blank page inserted for reproduction purposes only. California State Auditor Report 2015-120 59 March 2016 * * California State Auditor’s comments appear on page 67. 60 California State Auditor Report 2015-120 March 2016 California State Auditor Report 2015-120 61 March 2016 1 62 California State Auditor Report 2015-120 March 2016 2 California State Auditor Report 2015-120 63 March 2016 2 64 California State Auditor Report 2015-120 March 2016 3 California State Auditor Report 2015-120 65 March 2016 4 66 California State Auditor Report 2015-120 March 2016 California State Auditor Report 2015-120 67 March 2016 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM THE CALIFORNIA DEPARTMENT OF TRANSPORTATION To provide clarity and perspective, we are commenting on the California Department of Transportation’s (Caltrans) response to our audit. The numbers below correspond to the numbers we have placed in the margin of Caltrans’ response. To clarify, as we state in the Appendix on page 49, we analyzed more 1 precisely where the districts spent their field maintenance funding by focusing on three selected districts: district 4 (Oakland), district 6 (Fresno), district 7 (Los Angeles). Because we had this detailed spending information for the three districts, we attempted to focus our review of socioeconomic demographics in the three districts as well. As we describe on page 24, the purpose of the budget model was to 2 establish baseline field maintenance funding needs and allocations needed to maintain similar highways within the state at a specified maintenance performance level. Notwithstanding the challenges with implementing the budget model that Caltrans described in its response, we are pleased that it agrees with our recommendation to develop and implement a budget model that takes into account key indicators of maintenance need and use the model to inform appropriate allocations to the 12 districts. As we state on page 40, the maintenance division has not established 3 a time frame for completing service requests. The 10 day alert that Caltrans mentions in its response refers to the requirement to respond to the individual who submitted the service request within 10 days, stating whether the district commits to perform the work or stating the reasons why it is not going to perform the work. In contrast, on page 46 we recommend that Caltrans require supervisors to monitor actual completion of service requests to identify those not completed within a certain period of time that Caltrans deems appropriate, such as 30 day and determine their status. We are concerned that in its response Caltrans’ proposed revision 4 to its current guidance will not adequately address our concern. In particular, on page 47 we recommend that supervisors initiate work orders in the integrated maintenance management system once they identify needed field maintenance work rather than when the work begins as Caltrans mentions in its response. As we state on pages 41 and 42, unless the maintenance division initiates work orders when work is identified, the true length of time it takes from the date the work is identified to the date it is completed is not captured, and the maintenance division cannot accurately monitor whether it is addressing field maintenance needs in a timely manner.