Summary

Department of General Services Strengthening the Division of the State Architect’s Workload Management and Performance Measurements Could Help It Avoid Delays in Processing Future Increases in Workload May 2012 Report 2011-116.2 Independent NONPARTISAN Accountability TRANSPARENT The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 916.445.0255 or TTY 916.445.0033 OR This report is also available on the World Wide Web http://www.bsa.ca.gov The California State Auditor is pleased to announce the availability of an on-line subscription service. For information on how to subscribe, please contact the Information Technology Unit at 916.445.0255, ext. 456, or visit our Web site at www.bsa.ca.gov. Alternate format reports available upon request. Permission is granted to reproduce reports. For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. CALIFORNIA STATE AUDITOR Elaine M. Howle State Auditor Doug Cordiner B u r e a u o f S t a t e A u d i t s Chief Deputy 555 Capitol Mall, Suite 300 Sacramento, CA 95814 916.445.0255 916.327.0019 fax www.bsa.ca.gov May 31, 2012 2011-116.2 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the California State Auditor (state auditor) presents this audit report concerning the Department of General Services, Division of the State Architect’s (division) processes related to reviewing and approving plans for school construction. This report concludes that the division has been able to keep the amount of time plans wait for review—bin time—under six weeks primarily due to a decrease in workload. Should the division’s workload return to previous levels, it will likely struggle to maintain this goal. Recently, the division was prevented from contracting for plan review and instead drew on staff from its construction oversight activities. Further, although the division has a goal for keeping bin time below six weeks, it does not have goals for how long it should take to review plans. Without a plan review time goal, the division has less assurance that it is reviewing plans efficiently and school districts’ design professionals have little certainty about how long they should expect to wait for the division to return plans to them for correction. Also, the division’s monthly performance reports on the length of each phase of the plan approval process do not report clear or accurate information. For example, the division includes in its reports some projects for which no plan review activity has occurred because the applications were incomplete, counting the length of time it took to complete each phase as zero days. Finally, the division cannot provide assurance that it has received and approved all plan changes before the start of related construction. After the division approves plans, districts must submit changes to the division for review and approval before undertaking related construction. However, the division does not have a process to ensure that it has received and approved all relevant plan changes. If the division does not approve plan changes before construction, construction may not comply with building standards and risks being unsafe. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor California State Auditor Report 2011-116.2 vii May 2012 Contents Summary 1 Introduction 5 Audit Results Increases in Future Workload and a Lack of Plan Review Goals May Jeopardize the Division’s Ability to Promptly Process Plans 17 The Division’s Performance Reports Include Statistics That Do Not Reflect Current Activities and Are Distorted 25 The Division Cannot Ensure That It Has Approved All Changes to Plans Before the Start of Related Construction 27 The Division Could Improve Its Documentation of Final Reviews and Staff Attendance at Trainings 29 Recommendations 30 Appendix Status of the Division’s Efforts to Address Recommendations Made in Recent Reviews 33 Response to the Audit State and Consumer Services Agency, Department of General Services 39 California State Auditor’s Comment on the Response From the State and Consumer Services Agency, Department of General Services 45 viii California State Auditor Report 2011-116.2 May 2012 Blank page inserted for reproduction purposes only. California State Auditor Report 2011-116.2 1 May 2012 Summary Results in Brief Audit Highlights . . . As mandated by a state law known as the Field Act (act), the Our audit of the Department of General Division of the State Architect (division), part of the Department Services (department), Division of State of General Services (department), supervises the design and Architect’s (division) processes related to construction of K–12 schools and community colleges. The act the review of plans for school construction requires the department—which delegates its responsibilities to highlighted the following: the division—to review plans for school construction projects and to ensure that they comply with requirements in Title 24 » Although the division has kept the of the California Code of Regulations (building standards). When average time between the division’s the division is satisfied that a set of plans meets legal requirements, receipt of a complete application and the it issues an approval letter, which the school district must obtain start of its plan review (bin time) under before beginning construction. six weeks since early 2010, its workload is significantly lower than the level of work The division and the school districts’ licensed architects and it faced in late 2008 and early 2009. structural engineers (design professionals) share responsibility for the time it will take the division to approve construction » The division does not have goals for how plans. Once the division receives a complete application for long the plan review phase should last a project, the plan approval timeline moves into three sequential and thus has less assurance that it is phases: bin time and plan review time, for which the division reviewing plans efficiently. is primarily responsible, and client time, which is the time it takes the design professionals to make corrections based on » To help it manage any future increases in the division’s comments. The division has a goal of managing workload without negative consequences its workload to keep bin time—the time between the division’s to its other activities, the division will receipt of a complete application and the start of its plan review— need to use the retainer contracts it has below six weeks. Although the division has kept average bin times available, an option it was prohibited under six weeks since early 2010, it has done so with a workload from using in the recent past. that is significantly lower than the level of work it faced in late 2008 and early 2009. From July 2008 through June 2009, the division » The division’s performance reports that received an average of 858 new projects each quarter; however, relate to the length of each phase of the from October 2010 through September 2011, the division received plan approval process do not report clear an average of 638 projects each quarter. If the division’s workload or accurate information to stakeholders were to return to previous levels, it would likely again have difficulty and division management. meeting its bin time goal and it would risk an increase in the total time it takes to approve plans. » The division cannot assure that it has received and approved all plan changes Although the division has a goal for keeping bin time below before the start of related construction six weeks, it does not have goals for how long the plan review phase because it lacks processes to ensure that should last. After bin time, projects enter plan review, during which it has received all relevant plan changes the division ensures that plans meet the minimum requirements and recorded and managed all changes to of law and of building standards. The division notes any issues it plans that it must review. has with the plans and returns them to the design professionals for correction. Without a plan review time goal, the division has less assurance that it is reviewing plans efficiently. Further, without such a goal, design professionals have little certainty about how long they should expect to wait for the division to return the plans. 2 California State Auditor Report 2011-116.2 May 2012 To help it manage any future increases in workload without negative consequences to its other activities, the division will need to use the retainer contracts it has available. State law allows the division to issue contracts whenever the division deems it necessary to expedite plan review. The division must maintain a list of qualified plan review firms that have signed retainer contracts for plan review work. In the recent past, however, the division was prohibited from using contractors to address high bin times. Instead of contracting, the division shifted staff away from its construction oversight responsibilities. We noted that the division’s regional managers believed that the shift affected construction oversight, although they could not quantify the impact. Because the division is likely to need contractors in the future, we believe the division risks lengthening plan approval times if it does not develop a formal policy that defines when it will expedite plan review by using its contracting authority. Furthermore, the division’s performance reports that relate to the length of each phase of the plan approval process do not report clear or accurate information to stakeholders and division management. Instead of reporting information that reflects a snapshot of its current performance, the division reports only on projects that have fully concluded the plan approval process. Because the plan approval process can be long, this means that the division’s monthly reports reflect some activities that occurred significantly earlier than the report date, in one case over a year earlier. Further, the division includes data in its reports that cause reported averages for the different phases of the plan approval process to be understated or overstated. For example, the division has included some projects for which no plan review activity occurred because the applications were incomplete, counting the length of time it took to complete each phase as “zero days.” Finally, the division cannot provide assurance that it has received and approved all plan changes before the start of related construction. After the division approves plans, districts must submit plan changes to the division for review and approval before undertaking related construction. However, the division does not have a process to ensure that it has received all relevant plan changes. It also does not have a process for recording and managing all changes to plans that it must review. If the division does not approve plan changes before construction, construction may not comply with building standards and risks being unsafe. According to the former acting state architect, the division is making changes to improve its management of plan changes, which it intends to complete by the end of 2012. California State Auditor Report 2011-116.2 3 May 2012 Recommendations To better gauge the timeliness of its plan review and better communicate with design professionals, the division should develop goals for the time spent on the plan review phase, and measure and report its success at meeting these goals. In order to avoid delays in plan review, the division should develop a policy that defines when it will expedite plan review using its statutory authority to contract for additional plan review resources. To more accurately report on its plan review activities to stakeholders and provide relevant information to management, the division should: • Provide current information on its performance, by phase, at the time of the reporting period. • Exclude values from its calculations related to projects that cause the average time for a particular phase to be understated or overstated. To appropriately oversee changes to approved plans, the division should develop policies and procedures to ensure that it: • Receives all relevant plan changes. • Reviews and approves all relevant plan changes before the start of related construction. • Documents its approval of all relevant plan changes. Agency Comments The department agreed with our recommendations and outlined steps to implement them. 4 California State Auditor Report 2011-116.2 May 2012 Blank page inserted for reproduction purposes only. California State Auditor Report 2011-116.2 5 May 2012 Introduction Background On March 10, 1933, a magnitude 6.3 earthquake hit Long Beach, California. According to the Department of General Services (department), that earthquake destroyed 70 schools and inflicted major structural damage on another 120 schools. The earthquake struck when the buildings were unoccupied; otherwise, according to the department, hundreds of children might have died. On April 10, 1933—only one month after the earthquake—the Field Act (act) became law to protect the safety of pupils, teachers, and the public. The act requires that the department supervise the design and construction of any school building or the reconstruction or alteration of any school building to ensure that plans and specifications comply with the act and the building standards published in Title 24 of the California Code of Regulations (building standards). Although the act directs the department to approve plans and supervise construction of school buildings for the protection of life and property, the department assigned this responsibility to the Division of the State Architect (division). The division consists of a headquarters in Sacramento and four regional offices located in Oakland, Sacramento, Los Angeles, and San Diego. The headquarters includes the office of the state architect, a branch dedicated to codes and standards, and sections or units dedicated to human resources, inspector certification, fiscal services, contracts, training, performance metrics, and information technology. The state architect is appointed by the governor and heads the division; the current state architect has held the position since December 2011. In each regional office, a principal structural engineer serves as a regional manager, planning, organizing, and directing the plan review and field oversight activities for that office. The regional offices serve the counties assigned to them, as shown in Figure 1 on the following page. For fiscal year 2011–12, the division had a total of 345 authorized positions and a budget of $53.8 million. The division receives its revenue from fees it charges for its services.1 This report focuses on the division’s process for reviewing and approving school district plans, although the division is also responsible for overseeing and certifying construction. We discussed construction oversight in our December 2011 report titled Department of General Services: The Division of the State Architect Lacks Enforcement Authority and Has Weak Oversight Procedures, Increasing the Risk That School Construction Projects May Be Unsafe, Report 2011-116.1. 1 Two of those fees in particular come from activities related to the act. One fee supports structural safety and fire and life safety plan reviews and construction oversight, while the other supports access compliance reviews. 6 California State Auditor Report 2011-116.2 May 2012 Figure 1 Territories of the Division of the State Architect’s Regional Offices DEL Region I - Oakland NORTE SISKIYOU MODOC Region II - Sacramento Region III - Los Angeles Region IV - San Diego SHASTA LASSEN HUMBOLDT TRINITY TEHAMA PLUMAS GLENN BUTTE SIERRA MENDOCINO LAKE COLUSA S U YUBA NEVADA PLACER TTE R YOLO EL DORADO SONO M M A A RIN NAPA C S O O N LA T N R O A SACR J A O ME S A N A T Q O N UIN AMAD CA O L R AVERAS TUO A L L U P M IN N E E MONO COSTA SAN FRANCISCO SAN MATEO ALA S M A E N D T A A STANISLAUS MARIPOSA CLARA MERCED MADERA SANTA CRUZ SAN FRESNO BENITO INYO TULARE MONTEREY KINGS KERN SAN LUIS OBISPO SAN BERNARDINO SANTA BARBARA VENTURA LOSANGELES RIVERSIDE ORANGE SANTA CATALINA ISLAND IMPERIAL SAN DIEGO SAN CLEMENTE ISLAND Source: The Division of the State Architect’s Web site. California State Auditor Report 2011-116.2 7 May 2012 In order to apply for state funds for school construction projects, school districts must first receive approval from the division for all construction projects that are subject to the act.2 Figure 2 on the following page illustrates the division’s role in the school construction process. Between July 2008 and September 2011, the division received applications for 9,339 projects, Plan Review Disciplines with a total estimated cost of $22.2 billion. In each region, under the direction of the regional manager, Structural Safety—Ensures construction safety regarding supervising structural engineers and supervising resistance to the forces of gravity, wind, and earthquake. architects oversee technical staff who review plans Fire and Life Safety—Ensures construction safety regarding for school projects. The division has 142 authorized fire-resistive building materials, fire alarms, and fire suppression plan review positions—over 40 percent of its equipment; allows occupants to safely exit a facility; and allows workforce—to process and approve school districts’ firefighting equipment to access a facility. plans for construction. The division divides plan Access Compliance—Ensures construction compliance with review staff into three disciplines, described in laws and regulations requiring that buildings be accessible by the text box. While there are 83 positions in the persons with disabilities. structural safety discipline, there are only 32 in the fire and life safety discipline and only 27 in the Sources: Title 24 of the California Code of Regulations and the Division of the State Architect’s Web site. access compliance discipline. The Division’s Process for Approving Project Plans When the division first receives an application for a project, an engineer or architect performs a preliminary review to ensure that the application is complete. Generally, a complete application includes a number of documents such as an application form, a project submittal checklist, and three sets of plans and specifications. If the division deems the application incomplete, the division sends a letter requesting the missing items to the school district and the licensed architect or structural engineer (design professional) responsible for the project. The division reviews plans for both school and nonschool projects. Between July 2008 and September 2011, the division received at least 7,869 applications for school projects, representing 84 percent of all projects received, with an estimated value of $17 billion.3 Once the division deems a project application complete, its process for approving plans encompasses three sequential parts: bin time, plan review time, and client time, as shown in Figure 3 on page 9. The first phase of the process, bin time, is the time between the division’s receipt 2 The act, building standards, and the division’s Interpretation of Regulations use several terms, including school district, school board, and governing board of a school district, to refer to local entities with responsibilities under the act. In our report, we use the term school district(s) or district(s) when referring to those entities. 3 The division is also responsible for reviewing plans for some construction not covered by the act. For example, the division reviews plans for some state-owned buildings. This report focuses on school construction projects, but in our analysis we included all projects undergoing plan review because they all go through the division’s review process. 8 California State Auditor Report 2011-116.2 May 2012 of the complete application and the start of its plan review. During bin time, projects are on hold until assigned to a reviewer. The division generally ensures that it moves projects from the bin into plan review in the order in which the projects were received. However, it gives priority to projects with funding deadlines and those that are “shovel ready”—projects that are fully funded, will not use state bond funds, and will begin construction within 90 days of the division’s approval of their plans. Figure 2 A General Overview of the Division of the State Architect’s Involvement in the Public School Construction Process School districts Districts work with the School Facilities Planning identify construction Division of the California Department of Education or facility needs. to identify an appropriate school site. The School Facilities Planning The division closes projects, Division approves plans certifying those projects that for aspects related to comply with the Field Act. educational program delivery and classroom size, among others. Districts construct projects under Districts may apply to the The Division of the State Architect the oversight of the division. Office of Public School (division) ensures that construction plans meet building standards for seismic safety, Construction for state funding. fire and life safety, and accessibility. Sources: School Facility Program Handbook and Public School Construction Process Web site. Notes: Seven other state agencies can become involved under certain conditions, which can increase the complexity and duration of the process of building or remodeling a school. The text in the green boxes highlights the division’s involvement in the public school construction process. California State Auditor Report 2011-116.2 9 May 2012 Figure 3 The Division of the State Architect’s Process for Reviewing and Approving a Project, by Phase PROJECT APPROVAL PHASE Application Bin Time The division requests any Plan Review missing items. Client Time and Approval A school district submits an application and plans An intake architect reviews the to the Division of the State Architect (division). application for completeness. The project waits for available reviewers. When the division determines the plans comply with state requirements, within five days, the division issues a letter approving the project, which allows the school The division generally reviews projects district to contract for construction in the order received. and apply for state funding. REVIEW Division staff review plans for structural Structural safety, fire and life safety, and accessibility The design professional and the division meet to Fire & Life Safety and mark comments on the plans. review the corrected plans. Accessibility The division performs a quality control check of the plan review, called a The design professional final review. addresses the comments. The division returns the plans to the design professional. Sources: Title 24 of the California Code of Regulations, the division’s plan review guidelines and policies, reports on bin time and plan review workload, and the division’s Web site. During the second phase, plan review time, the division performs concurrent reviews in the three disciplines described in the text box on page 7. According to the former acting state architect, the structural safety review is the critical review path because this part of the review generally takes the longest to complete. Consequently, as noted on page 7, the division dedicates significantly more staff to structural safety than to either fire and life safety or access compliance reviews. Plan reviewers generally work on one project at a time, although regional managers 10 California State Auditor Report 2011-116.2 May 2012 stated that they will sometimes assign multiple similar projects to the same plan reviewer to gain efficiency. Reviewers mark corrections and note errors or omissions onto the plans. As discussed in more detail below, the division’s plan review supervisors or other technical experts conduct quality control checks of the comments on the plans to ensure that the review has been done appropriately before returning the plans for needed corrections to the design professionals who prepared them for the districts. The division has provided guidance for reviewers in each discipline on how to review plans and comment on potential design problems. The division has produced guidelines for its structural safety plan reviews, and makes several documents from different sources—including the State Fire Marshal—available on its Web site to describe fire and life safety aspects of construction. Further, the division has a reference manual for access compliance. The final phase, client time, occurs between the end of plan review and plan approval. After plan review is complete, the division returns the plans, with the division’s comments, to the design professionals. When the design professionals have addressed these comments, they make an appointment for what is called a “back check,” during which the design professionals and the division’s plan reviewers meet to compare the corrected plans with the division’s comments on the original plans. When the division is satisfied that the corrected plans meet legal requirements, it places its stamp on the plans and issues a letter to the school district, signed by the regional manager, approving the plans and specifications.4 The division has a shorter process for two types of applications. Specifically, the division has an “over-the-counter process” to handle reviews of schools and state-funded structures that are primarily for single-story, relocatable buildings and other simple projects that use preapproved designs; these reviews typically take two hours or less to complete. Between July 2008 and September 2011, the division received 2,928 over-the-counter projects—31 percent of all projects it received—which had a total estimated cost of $1.2 billion. The division took an average of 13 days to approve this type of project once the client provided a complete application. Second, because the division is responsible for ensuring that public schools and state-funded buildings are accessible to persons with disabilities, the division has a separate “access-only” process for projects that require only access compliance review. This requirement is outside of the purview of the act and can include both school and nonschool projects. Between July 2008 and September 2011, the division received 1,230 access-only projects. 4 According to division performance reports, the process of stamping and approving the plans generally takes one or two days. Thus, we have not separated it from the rest of client time in our analysis. California State Auditor Report 2011-116.2 11 May 2012 These represented 13 percent of all projects received, for a total estimated cost of $5 billion. The division took an average of 63 days to approve this type of project once the client provided a complete application. Construction plans may change after the division has approved them. According to the building standards, several different documents can be submitted to the division for review and approval of plan changes. For example, the division may learn of changes to plans that happen before the award of a construction contract through revisions and addenda. Once a school district awards a construction contract, it communicates changes to plans to the division via “change orders” or “field change documents.” The division uses these documents to review and approve changes made to plans that the division has already approved.5 We discuss change orders and field change documents in the Audit Results. Projects do not always complete the approval process. The division will cancel a project when cancellation is requested by the district. Also, in some instances the division may void a project. For example, if the design professional does not submit corrected plans for a back check within six months from the date the division returned the plans, division policy is to void the project. However, the school district may request a six-month extension. Second, if the back check is not complete within two months, the division’s policy is to void the project. According to the division’s database, school districts cancelled 282 projects between July 2008 and June 2011, and the division voided 1,034 projects during that same time. The Division’s Processes for Ensuring Consistent Review Practices Throughout the State The division uses a number of mechanisms to ensure that its four regional offices implement plan review policies and procedures consistently. One example involves the use of final reviews. The division expects its plan reviewers to exercise independent judgment in verifying the work of the design professionals. To ensure that plans are reviewed consistently, supervising engineers or other qualified experts perform a final review after the division’s plan reviewers complete their work. According to the division’s Structural Final Review Guidelines, final review is a critical step intended to ensure a thorough quality plan check, with uniform application of codes and standards and consistency among staff. Additionally, the division has statewide teams that help ensure that reviewers apply the code consistently and that they address concerns raised by clients and staff by providing a forum for the discussion and resolution 5 In some cases, the division may also defer approval of a portion of the plans when, according to building standards, a portion of the construction cannot yet be adequately detailed because of variations in product design or manufacturer. For example, the division states that it is not efficient to design elevator guide rails until after an elevator supplier has been chosen. 12 California State Auditor Report 2011-116.2 May 2012 of issues as they arise. The division’s statewide teams include teams that focus on each of the three review disciplines. Each team includes member s representing each regional office and headquarters, and one member of each team is designated the team lead. Decisions from the statewide teams can be specific to a single project or have a broader impact. The division disseminates the decisions of statewide teams to the different parties within the division so that all staff are aware of agreed-upon approaches to technical and administrative questions. Team members serve as their discipline’s resource person for regional office staff. A team’s decision on an issue becomes the official policy of the division. Finally, the division has a process in place to handle disputes over the application of building standards. According to the former acting state architect, when disputes between the division and a design professional arise over the application of building standards, the division attempts to address them informally through discussions between the design professional and the plan reviewer, the review supervisor, and the regional manager. Should the division not be able to resolve such disputes informally, the design professional may make use of the division’s Code Appeal Process (appeal process). A design professional wishing to invoke the appeal process writes out a complaint and submits it to the division. The division routes the complaint first to the plan reviewer, then to the discipline lead, then the supervisor, then the regional manager, and finally the statewide team. Before elevating the appeal to the next level, the design professional and the relevant division staff discuss the issue. If at any point the design professional and division reach agreement, the process ends. Otherwise, the statewide team will make a decision. According to division records, between July 2008 and September 2011, the division issued 32 decisions through the appeal process. The division communicates appeal process decisions to its staff through e-mail and by making the decisions available through the division’s appeal process database. Similar to statewide team decisions, some appeal process decisions are specific to an individual project, while others have a broader application. Other Recent Audits and Evaluations The Department of Finance, Office of State Audits and Evaluations (OSAE), conducted two recent reviews of the division. One review, released in April 2010, focused on the division’s revolving fund and its fees.6 The other, released in March 2011, focused on the division’s plan review and construction oversight processes. OSAE made a 6 Fees collected from applicants submitting plans for the construction or alteration of school buildings are deposited to the Public School Planning, Design, and Construction Review Revolving Fund. Money from this fund is used to cover the expenses involved in the review and approval of plans and construction oversight. California State Auditor Report 2011-116.2 13 May 2012 total of 41 recommendations in the two reports, and we reviewed the division’s implementation of 35 of those recommendations. The Appendix lists the recommendations we reviewed and the status of their implementation. Scope and Methodology The Joint Legislative Audit Committee (audit committee) directed the California State Auditor (state auditor) to review the division’s implementation of the act, which we describe in the Introduction. The audit committee’s request divided the audit into two phases: Phase one focused on the division’s construction oversight and project closeout functions. The state auditor released a report on phase one in December 2011. This report covers phase two of the audit request, focusing on the division’s plan review function. Table 1 outlines the audit committee’s objectives for phase two and our methodology for addressing each objective. Table 1 Methods of Addressing Audit Objectives AUDIT OBJECTIVE METHODOLOGY 1 Review and evaluate the laws, rules, and We reviewed relevant laws, rules, and regulations. regulations significant to the audit objectives. 2 Review and evaluate the management, control, We reviewed laws and regulations that govern plan review, reviewed division policies and and operational structure of the Division of the guidelines related to plan review, and interviewed division staff responsible for plan review. State Architect’s (division) construction plan We determined that the management, control, and operational structure, as discussed in the review process. Determine whether the existing Introduction, is adequate for plan review and appears to provide for efficient service delivery structure provides for efficient service delivery. except as further discussed in the Audit Results. 3 Describe the policies, procedures, and practices We reviewed laws and regulations that govern plan review, reviewed division policies and currently used in the plan review process. guidelines related to plan review, and interviewed division staff responsible for plan review. We describe the policies, procedures, and practices currently in use in the plan review process in the Introduction. In determining the average length of time a project spends in the bin and in plan review, we followed the division’s convention of focusing on structural review for nonaccess-only projects and on the access compliance review for access-only projects. Although fire and life safety and access reviews may start earlier or end later, the division considers the structural review to be the critical path. 4 Determine what processes the division uses We interviewed regional managers and headquarters staff about processes they use to ensure to ensure its policies and practices for plan consistency and reviewed documents that describe and result from those processes. We reviewed review functions are consistently applied in the division’s processes for communicating policy changes to its staff and checked for the all its offices. Assess whether such processes completion of the final reviews of plans, which help ensure consistent plan review. We determined are effective. that the division’s processes and practices, as described in the Introduction, provide reasonable assurance that the division consistently applies its policies and practices for plan review. 5 Assess whether, and to what extent, the division We analyzed information from the division’s database and performance reports and assessed has backlogs in its project plan review functions. whether the division’s bin times exceeded six weeks, the division’s stated bin time goal. We If the division has a backlog, review and evaluate reviewed division records and interviewed division staff related to its efforts to manage the division’s actions to reduce the backlog. plan review workload. continued on next page . . . 14 California State Auditor Report 2011-116.2 May 2012 AUDIT OBJECTIVE METHODOLOGY 6 Review studies performed in the past year by the We interviewed division staff and reviewed documents related to the division’s progress in division and the Department of Finance’s Office addressing OSAE’s recommendations. We limited our review to those recommendations of State Audits and Evaluations (OSAE) on the relevant to plan review and the division’s fee structure and present the status of the division’s division’s operations and determine the extent implementation of those recommendations in the Appendix. Additionally, we determined that to which the division made any changes as a neither the division nor the Department of General Services (department) have conducted studies result of these reviews to improve its operations, related to plan review since January 2010. including recommendations related to the division’s fee structure. 7 If the division has not done so, determine We interviewed division staff and reviewed policy documents to identify changes the division how changes in key policies and practices made to its plan review policies and practices since January 2010. We determined that the since January 2010 at the division have division has not completed its own analysis of the effect of these changes. We determined that measurably impacted operations (e.g., more a division management practice focusing on bin time was a change to a key practice; however, efficient management of caseload, timely the effect of the change was obscured by a bin time reduction effort the division undertook in communication with school districts and other early 2010 that involved shifting staff from other responsibilities to plan review, and a general stakeholders, and ensuring effective operations drop in workload. throughout the State). 8 Review and evaluate existing division We reviewed the division’s performance metrics reports and interviewed division staff. We performance measures for plan review processes identified other local and state plan review entities to determine what industry standards exist and, to the extent possible, determine whether and determined that the Facilities Development Division of the Office of Statewide Health those performance measures align with industry Planning and Development (OSHPD) was most similar to the division. We interviewed the deputy standards or best practices. director of OSHPD’s Facility Development Division and compared OSHPD’s practices and goals to those of the division. 9 Determine if the division has any plans to We interviewed division and department staff and determined that neither had plans for future modify key policies and practices and what its changes to policy or practices related to plan review as of March 2012. justification and projected outcomes are for these modifications. 10 Review and assess any other issues that are To provide context for the division’s role in the State’s school construction process, we reviewed general significant to the division’s plan review function. information on the department’s Web site about school construction and related state funding. In phase one of this audit, we identified elements We reviewed the division’s contracting and contract amendment processes by testing a selection related to contracting in four objectives which, of contracts and amendments against requirements in state law and the division’s internal policies. because the division contracts primarily for We noted no exceptions in this testing. We interviewed the division’s contracting staff and regional plan review services, we chose to address in this managers regarding the division’s methods for evaluating the competency of contractors, training phase. The specific objectives from phase one requirements for contractors, and the division’s evaluation of contractors’ work. addressed in this phase relate to the division’s processes for evaluating, training, and monitoring its contractors. We also chose to review plan changes in We reviewed laws and regulations that govern changes made to plans after division approval phase two, rather than phase one, because and reviewed division policies related to such changes. We also interviewed division staff about of their relationship to approved plans; for the process for approving changes. Using data from phase one of our audit, we assessed the example, the division stated it sometimes uses division’s process for approving changes that are made to division-approved plans. contractors to review changes to approved plans as well as to perform plan reviews. To support its work, the division uses a database called Tracker (database), which it developed in 1997 to manage projects. This database tracks project applications, key dates—such as plan approval and construction start and end dates, and the types of project closure. The database also generates invoices and calculates the fees owed to the division for certain aspects of its work. The database links to scanned copies of documents, when available. To address several of the audit committee’s objectives, we relied on data the division provided. We adhere to the standards of California State Auditor Report 2011-116.2 15 May 2012 the U.S. Government Accountability Office, which require us to assess the sufficiency and appropriateness of computer-processed information. Table 2 shows the results of this analysis. Table 2 Methods of Assessing Data Reliability INFORMATION SYSTEM PURPOSE METHOD AND RESULT CONCLUSION Department of For the purpose of identifying the • We performed data-set verification procedures and electronic Undetermined General Services: following for the period between testing of key data elements. We identified no issues when reliability for Division of the July 1, 2008, and June 30, 2011: performing data-set verification procedures, but we found errors the purposes State Architect in our electronic testing, some of which we were able to correct. of this audit • The percentage of projects (division) closed that had a plan change • We performed completeness testing by selecting 29 projects document submitted after from regional files and verifying that these projects existed in the Tracker database construction ended. database; we found no errors in this testing. Data as of • The number of plan change • We also tested the accuracy of the database by testing key data June 30, 2011 documents that were submitted elements for a random sample of 29 projects and tracing the after construction ended, and the selected elements to the project files. In this sample, we found percentage of these documents one error, so we continued testing until we had tested a total of that were change orders. 47 randomly selected projects and found no additional errors. However, because the division did not have a consistent method • The number of projects voided for identifying the date construction ended, we were unable to or canceled. test the accuracy of this field. • Our review of existing information identified two data limitations: The division’s database does not track information on any projects submitted to the division before November 1997. Further, the database does not identify which projects are reopened regardless of whether the project was initially recorded in the database. Because some applicants are required to pay a fee when they reopen projects, we were able to identify a portion of the reopened projects using the fee information. Although we were not able to identify all of the reopened projects, we included those we did identify in our analysis. Department of For the purpose of calculating the • We performed data-set verification procedures and electronic Undetermined General Services: following for the period between July testing of key data elements. The results of electronic testing reliability for Division of the 1, 2008, and September 30, 2011: identified minor errors in a few key data elements. the purposes State Architect of this audit • The number and total estimated • We performed completeness testing by tracing a haphazardly dollar value of projects the division selected sample of projects from regional files to the database and Tracker database received by region and quarter. testing the sequential numbering of the projects in the database, and found no errors in this testing. Data as of • The average time between September 30, 2011 application receipt date and plan • We performed accuracy testing by testing key data elements for a approval date (total plan approval random sample of 29 projects with plan review activity and tracing time) for approved projects. the selected elements to the project files. In addition, we selected a supplemental sample of 10 randomly selected projects with no • The number of projects and plan review activity. We found no errors in the fields we were able average amount of total plan to test. However, we were not able to verify the accuracy of the approval time for over-the-counter project receipt and plan review dates because the documentation and access-only projects. needed to verify this information is not consistently documented • The average bin time, plan review or retained by the division. time, and client time for projects completing these phases of the plan review process. • The number and total estimated dollar value of over-the-counter and access-only projects. 16 California State Auditor Report 2011-116.2 May 2012 Blank page inserted for reproduction purposes only. California State Auditor Report 2011-116.2 17 May 2012 Audit Results Increases in Future Workload and a Lack of Plan Review Goals May Jeopardize the Division’s Ability to Promptly Process Plans The Division of the State Architect (division) is responsible for two phases of the plan approval process, as described in the Introduction, but it has a goal for the length of only one of these phases. The division has established a goal to keep bin time— which once averaged over seven weeks and exceeded 12 weeks for some projects—to under six weeks. Largely due to a reduced workload, it has been successful at meeting this goal. If workload were to increase, however, the division would likely have difficulty maintaining bin times below six weeks using only its existing plan review staff. Further, the division has no goals for how long its staff should take to finish the plan review phase. Consequently, the division is less able to gauge whether it is reviewing plans and returning them to the school districts’ design professionals as quickly as possible. Together, bin time and plan review time comprise the portion of the total plan approval process that is under the division’s control. How long the division takes to complete its part of the process is significant because school districts must obtain plan approval before receiving state funding for projects or before beginning construction. When the division determines it needs to speed up plan review, it has the authority to obtain additional plan review assistance by contracting with plan review firms. However, in the recent past the division has been prevented from using this resource. If Workload Increases to Past Levels, the Division Risks Lengthening the Plan Approval Process Although the division maintained average bin times below its goal of six weeks between January 2010 and September 2011, it could have trouble continuing to meet this goal if its workload were to return to past levels or to fluctuate significantly, thus risking delays in plan approval. The division’s success in meeting its goal has coincided with a downturn in workload. In the past, during periods of heavier workload, the division had difficulty maintaining average bin times of six weeks. Therefore, without the use of additional resources, an increase in the number of applications for plan review would likely cause bin times to exceed the division’s current goal. As a significant part of the plan approval process, the division’s bin time affects overall plan approval time. Plan approval is also part of a larger process that school districts must navigate before constructing a building project; for example, without division approval, school districts cannot apply for state funding for projects or contract for construction. 18 California State Auditor Report 2011-116.2 May 2012 The average amount of time it took the division to approve construction plans trended down between July 2008 and September 2011. As discussed in the Introduction, the plan approval process consists of three parts: bin time, plan review time, and client time. The division is primarily responsible for how long the bin and plan review phases of the plan approval process last, while the design professionals who work for the school districts are primarily responsible for the length of time spent in the client phase. Figure 4 shows the average duration of each phase in the plan approval process during the third quarter of 2011. Plan approval time took 28.6 weeks on average between July 2008 and June 2009; however, between October 2010 and September 2011 approval time took 26.1 weeks on average. Figure 4 Average Time in Weeks Spent in Each Phase of the Plan Approval Process Third Quarter of 2011 18 16 14 12 10 8 6 4 2 0 Bin Plan Client Time Review Time Phase* skeeW Source: California State Auditor’s analysis of data obtained from the Division of the State Architect’s Tracker database. * We calculated bin time, plan review time, and client time for projects that completed each phase during the third quarter of 2011. Because some projects may not have finished all three phases or may have completed multiple phases during this quarter, the sum of the averages shown above does not represent the average total plan approval time for the quarter. California State Auditor Report 2011-116.2 19 May 2012 To avoid delays in plan approval, the division established a goal to begin the structural safety review of projects within six weeks of receiving complete applications.7 According to the division’s former acting state architect, since early 2010 the division has actively managed its workload to keep structural safety bin time for projects under six weeks. He stated that the division developed this goal with the Department of General Services (department) and the In early 2010 the division had State and Consumer Services Agency (agency) in early 2010. At some projects that had been in the that time, according to the manager of the division’s performance bin between 11 and 13 weeks. In metrics unit, the division had some projects that had been in the order to meet its six-week goal, bin for between 11 and 13 weeks. He explained that currently, the division shifted staff from to assist in managing to the bin time goal, regional managers other responsibilities to plan receive weekly updates from headquarters about projects in their review, temporarily increasing the region that are nearing the six-week mark. The regional managers resources focused on plan review. stated that they use these updates to assign projects for plan review as the submitted projects approach six weeks in the bin. As shown in Figure 5 on the following page, the division had average bin times at or near seven weeks in three quarters of 2009. Figure 5 also shows a dramatic decrease in bin times in the first and second quarters of 2010. During this time, in order to meet its six-week goal, the division shifted staff from other responsibilities to plan review, temporarily increasing the resources focused on plan review. We discuss this effort further on page 24. Since the summer of 2010, when the former acting state architect says the division stopped shifting staff, average quarterly bin times have remained below six weeks through the third quarter of 2011. The division’s success in maintaining its average bin time below six weeks after its 2010 effort concluded appears to be largely related to decreased workload. The division’s workload—the number of new project applications it received—decreased substantially during the period we reviewed. As shown in Figure 6 on page 21, from July 2008 through June 2009, the division received an average of 858 projects each quarter. In contrast, from October 2010 through September 2011, the division received an average of 638 projects each quarter, approximately 25 percent less. Further, the dollar value of projects the division received dropped even more over the same period, indicating that the division has been receiving smaller projects on average. The total estimated cost of projects received from July 2008 through June 2009 was nearly $8.5 billion, but the estimated cost of projects received from October 2010 through September 2011 was nearly $5.3 billion, approximately 38 percent less. The former acting state architect stated that although it is difficult to 7 According to the manager of the division’s performance metrics unit, the division measures bin time from the date it receives an application unless it determines the application to be incomplete. In that case, it measures bin time from the date it receives the additional documentation that makes the application complete. 20 California State Auditor Report 2011-116.2 May 2012 pinpoint reasons for the decline in workload, the economic recession, the slowdown in the housing market, demographic changes, and a lack of state bond funding seem to be contributing factors. Figure 5 Average Bin Time for Structural Plans in Weeks July 2008 Through September 2011 8 7 6 5 4 3 2 1 0 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 2008– 2008– 2009– 2009– 2009– 2009– 2010– 2010– 2010– 2010– 2011– 2011– 2011– Quarter skeeW Weeks Trendline Source: California State Auditor’s analysis of data obtained from the Division of the State Architect’s Tracker database. Aside from adopting a bin time goal and more closely monitoring projects in the bin, since January 2010 the division has not made changes to its policies that would significantly affect plan approval time. The changes that were made by the division either affected only certain portions of its plan review or had no relationship to the amount of time the division takes to approve plans. According to the division’s former acting state architect, these policy changes included requiring clients to submit a checklist of required items along with their project applications, requiring plans for automatic fire sprinkler systems to be submitted as part of the initial project application, suspending a requirement that clients electronically submit plans for projects with estimated costs up to $400,000, and increasing the division’s access compliance-related fee in January 2010. According to the manager of the division’s performance metrics unit, the division has not measured the effect any of these changes have had on the division’s plan review operations. California State Auditor Report 2011-116.2 21 May 2012 Figure 6 Applications for Plan Review Received by the Division of the State Architect by Quarter July 2008 Through September 2011 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 2008– 2008– 2009– 2009– 2009– 2009– 2010– 2010– 2010– 2010– 2011– 2011– 2011– Quarter dettimbuS snoitacilppA Number of applications 1,000 900 Average of 858 applications 800 700 600 500 Average of 638 applications 400 300 200 100 0 Source: California State Auditor’s analysis of data obtained from the Division of the State Architect’s Tracker database. If the Division Set Goals for Plan Review Time, It Could Better Assess Its Efficiency and Better Communicate With Design Professionals Although the division has set a goal for bin time, it has not set a goal for plan review time. As discussed in the Introduction, plan review time is the amount of time the division spends reviewing plans, commenting on design problems, and returning comments to design professionals. Just as with bin time, the time the division spends on this phase has an impact on the total approval time and thus affects a school district’s ability to obtain funding and begin construction. As Figure 4 on page 18 shows, plan review lasted approximately four and a half weeks on average in the quarter ending September 2011. The division does not have goals or standards for how long projects should take to move through the plan review phase. According to the manager of the division’s performance metrics unit, regional managers and supervisors budget time for plan review based on the project’s size and complexity. In other words, estimates vary 22 California State Auditor Report 2011-116.2 May 2012 depending on project characteristics. He further stated, however, that the division does not currently have standards for completing plan reviews within a set amount of time for a given project category, and it cannot globally determine its success in meeting an expected length of plan review, owing to other factors, such as the quality or complexity of plans. Without such goals, however, the division has less assurance that it is conducting plan reviews efficiently. For example, once a week the division currently identifies all plans that have remained in the bin for longer than six weeks. The division cannot perform a similar assessment for projects in the plan review phase without goals for plan review. Establishing a set of plan review goals based on general project characteristics would better allow the division to determine if it is completing this phase of the plan approval process in a timely manner and to take action as necessary. The division’s lack of plan review The division’s lack of plan review goals also leaves design goals leaves design professionals professionals with little certainty about how long they should with little certainty about how long expect to wait for the division to return plans. On its Web site, they should expect to wait for the the division provides a tool to help design professionals estimate division to return plans. when they will receive plans back for correction. However, the tool is imprecise and relies on the user to determine the expected timeline. For example, a design professional using the division’s tool must gauge the complexity of the project and determine whether the buildings included on the plans are large or small. According to the tool, such determinations should be used to adjust the division’s timeline. In the case of complexity, the division states that review time can increase by “up to 50 percent,” while several small buildings can add “up to 20 percent.” The tool directs design professionals to add or subtract these amounts of time from a baseline number of days that is set by the dollar value and type of project, whether new construction or modernization. In contrast, the Office of Statewide Health Planning and Development (OSHPD) does set goals for the time it will take to review plans. Similar to the division, OSHPD’s Facilities Development Division is responsible for reviewing plans for new construction of, alterations to, additions to, or renovations of health facilities in the State. Its goals for the amount of time it needs to review plans are specific to project categories that are based on the project’s size and complexity. For example, OSHPD’s goal for small projects that do not contain major structural work is to complete plan review within 60 days.8 According to the deputy director of its Facilities Development Division, OSHPD based these goals on historical information on how long OSHPD took to review 8 OSHPD’s plan review period starts when it receives a submittal and ends when it returns the submittal for corrections. This period is thus equivalent to a combination of the division’s bin time and plan review time. California State Auditor Report 2011-116.2 23 May 2012 plans. OSHPD uses its goals to clearly communicate on its Web site how many days it expects to take to complete its review. For example, new hospital building projects have a review turnaround goal of 100 days. The state architect stated that it would be difficult for the division to develop plan review goals similar to OSHPD’s because of the wide variety in the types of projects that the division reviews. He also cited the quality of plans as another issue. The manager of the division’s performance metrics unit also stated that the quality of plans is one reason why the division cannot determine if plan review is meeting expectations for time to completion. He noted that plans requiring a larger number of comments take longer to review than those that are more in agreement with Title 24 of the California Code of Regulations (building standards). However, OSHPD also reviews diverse projects, setting different goals for different projects, ranging from small, nonstructural projects to new hospital buildings. OSHPD also may face situations where plans require a large number of comments. In fact, in a report to the Legislature, OSHPD stated that poorly designed submittals require higher review efforts and may cause turnaround times to exceed goals. Additionally, the division already captures certain project details The division already captures certain that could help it create goals for plan review time. The division project details that could help it records a project’s estimated cost, which, according to the former create goals for plan review time. acting state architect, is generally associated with the size and type of a project and the quality of construction materials used. Also, the division categorizes projects into different classes, generally according to the size of the project and the type of construction materials used, such as a class for wood-frame projects and a class for single-story relocatable buildings. The Division Has Authority to Ensure the Timely Return of Plans to Design Professionals if Its Workload Increases When the division determines that it needs to speed up plan review, state law allows it to take specific actions that include temporarily employing additional staff, having existing staff work overtime, and contracting for plan review services. The manager of the division’s performance metrics unit said that the division’s preferred method for addressing surges in workload is the use of contractors. However, the division has not always been able to use this tool to address high workload. As required by state law, the division maintains a list of qualified plan review firms that have signed retainer contracts for plan review work. When the division identifies the need for additional 24 California State Auditor Report 2011-116.2 May 2012 plan review resources, it amends one of these contracts and assigns work to a plan review firm. The division chooses firms for plan review on a rotating basis, contacting firms in the order in which they appear on the division’s randomly ordered contractor list. Before amending a contract, the division’s policy is to negotiate a maximum amendment cost and a timeline for completing plan review. If the division and the contractor cannot reach agreement on the amendment cost and timeline, state law requires the division to contact the next contractor on the list. However, during recent periods of increased workload and high bin times, the division was unable to contract for plan review work. During recent periods of increased During most of 2009, the division had average quarterly bin times at workload and high bin times, the or near seven weeks, and the manager of the division’s performance division was kept from contracting metrics unit said that in January 2010 the division had some projects for review work. that had been in the bin for up to 13 weeks. A July 2009 e-mail from the state architect to the division’s regional managers stated that the State and Consumer Services Agency had directed the division to stop using plan review contractors and staff overtime. Additionally, according to the manager of the division’s performance metrics unit, the department informally discouraged contracting for plan review. He noted that an executive order prohibited contracting to make up for the loss of staff time related to a furlough program that started in early 2009, making it difficult to justify contracting for plan review services. Instead of contracting to address long bin times, in January 2010 the division required regional managers and construction oversight staff to shift up to 50 percent of their time to plan review tasks. In our December 2011 report on the division, we noted that regional managers believed that the shift affected construction oversight, though they could not quantify the impact. Based on its past performance, future increases in the division’s workload will likely challenge the division’s ability to meet its bin time goal using only its current level of plan review staff. In fact, the division recently used its authority to contract in order to manage long bin times in its Oakland region, which, according to the Oakland regional manager, were the result of a surge in applications at the end of 2011. The division’s weekly bin time report from early February 2012 showed that some projects in the Oakland region had been in the bin for up to eight weeks. As previously discussed, the division’s bin time and plan review time have an impact on the total time it takes for the division to approve plans, and plan approval is a key step in the school construction process. The department’s chief deputy stated that he was not familiar with the specific details of past directions to the division but that the division is currently able to contract for services when it determines that plan review assistance is critical to its mission and it has sufficient budgeted spending authority to cover the cost California State Auditor Report 2011-116.2 25 May 2012 of contracting. However, because the division is likely to require plan review assistance in the future and the division’s authority to contract has been curtailed in the recent past, we believe the division risks lengthening plan approval times if it does not develop a formal policy that defines when it will expedite plan review by using its contracting authority. The Division’s Performance Reports Include Statistics That Do Not Reflect Current Activities and Are Distorted On its Web site, the division posts monthly reports on the length of time projects spend in the bin and in plan review, but many statistics in the report do not reflect current activities. Reported statistics include the minimum, maximum, and average number of days for bin time, plan review time, and client time. We expected to find that the division’s reports would reflect performance during the report period—for example, that the bin times reported for September 2011 would reflect the minimum, maximum, and average bin times of The division’s reports reflect activities projects that exited the bin in that month. However, the division that occurred a significant amount reports information only on projects that have concluded all plan of time before the reporting month review activities—after they have been approved or voided by the because it reports information only division, or cancelled by the school district. on projects that have concluded all plan review activities—after they As a result, the division’s reports reflect activities that occurred have been approved or voided by a significant amount of time before the reporting month. For the division, or cancelled by the example, one project the division approved in September 2011 school district. left the bin and finished plan review in May 2010. Because of the division’s methodology, its monthly reports did not reflect activity related to this project until the plan was approved in 2011, meaning that the bin and plan review times reported for this project occurred more than a year before the report that included this project. The manager of the division’s performance metrics unit stated that the division chose to report project information only after all plan review activity had concluded because it wanted to maintain consistency across all of the phases of the approval timeline. Consequently, the information the division presents on each phase is not a snapshot of how long projects have remained in a phase at the time of the report, making it difficult for stakeholders to judge the division’s performance at a specific time. In addition, the division uses data that can distort the average values presented in its reports. Specifically, the reports contain information for projects that the division’s database shows never entered the bin for review. These projects had applications that the division determined were incomplete and that it eventually voided. For example, we found a project that had never entered the bin and that the division had voided in August 2011. Despite this fact, the division included the project in its report and counted the project 26 California State Auditor Report 2011-116.2 May 2012 as having taken zero days to complete the plan review and client time phases of the plan approval process. Including zero values for projects that never experienced activity during a particular phase of the plan approval process artificially reduces the average time reported for these phases. The division has also distorted its reported values by including information on projects that exceed the time limits defined in the division’s policy. Division policy states that it will void projects that a design professional has not scheduled for back check within six months of the end of the plan review phase, unless the client requests an extension for an additional six months. Beyond this 12-month maximum, the division will void projects and clients must resubmit them as new projects, with new applications and filing fees. According to the division’s policy, it voids projects because excessive elapsed time causes ineffective use of staffing resources and because comments on plans may become outdated as new codes, standards, and regulations become effective. Additionally, the same policy provides that the division will void projects that do not finish back check within two months. However, the division’s monthly reports included some projects in which well over a year went by before back check began or the project was voided. For example, division records show that for a number of projects, more than 1,000 days elapsed between the end of plan review and back check. According to the former acting state architect, design professionals for large projects, such as a high school with an estimated cost of over $80 million, may require more time to address the division’s comments and schedule a back check appointment. He stated that voiding projects in these cases would result in a loss of time and resources for both the design professionals and the division, with no material gain in the safety of the project’s design. Nevertheless, including these projects in its performance reporting—instead of removing them as outliers— causes the division to report longer average client times than may be typical. We found that the division’s reports When we adjusted for the above-mentioned issues, we found that overstate the average length of the division’s reports overstate the average length of time spent by time spent by design professionals design professionals on the client phase. For projects completing on the client phase. plan review activity in the third quarter of 2011, an adjusted methodology that removed zero-day counts for projects on which no activity occurred and that removed client time days for projects that exceeded time limits showed average client times that were between 17 and 26 days shorter than the client times reported by the division. According to the performance metrics analyst at the division responsible for creating the monthly report, the division reviews the data used in its reports to identify and exclude any outliers. However, we found numerous examples in the division’s reports of projects that were incorrectly included even though California State Auditor Report 2011-116.2 27 May 2012 they had zero values and no review activity or they had spent significantly more time in the client time phase than the division’s policy allows. The division’s review for outliers did not exclude these projects. Including such projects distorts the average days reported for all phases of the plan approval process and appears to particularly distort reported averages for client time. The Division Cannot Ensure That It Has Approved All Changes to Plans Before the Start of Related Construction As discussed in the Introduction, school districts may make changes to their plans after the division has approved them. Regulations require that the school districts’ design professionals submit plan changes to the division for review and approval before Several holes in the plan change undertaking related construction. However, several holes in the process create a situation where plan change process create a situation where the division cannot the division cannot demonstrate demonstrate that it has approved all plan changes before the start that it has approved all plan of related construction, risking construction that does not meet changes before the start of related building standards and that may be unsafe. In fact, the division’s construction, risking construction Project Certification Guide states that there have been many that does not meet building instances where the field change process was not followed and standards and that may be unsafe. change orders did not receive division approval, yet construction was completed. During project construction, design professionals communicate plan changes to the division through either “change orders” or “field change documents” (field changes). Change orders are documents school districts and design professionals use to communicate changes to the contractor and may affect construction plans, estimated project costs, or the project’s schedule. According to the division, the process of developing change orders may be time-consuming because school boards must review and approve them. In order to expedite construction, regulations allow school districts to submit plan changes to the division through field changes, ahead of change orders dealing with the same issue. However, school districts can also use field changes in situations that do not require change orders, and conversely, change orders may include plan changes that were never covered by field changes. Because both change orders and field changes may affect approved plans, we expected the division to have a process to ensure that it has reviewed all of these plan changes before construction and to maintain evidence of its review. However, this is not the case. The division’s process for handling change orders is significantly more robust than its process for handling field changes. The division retains the change orders it receives, which are numbered sequentially, and logs their receipt and approval dates into its database. As part of its process for closing a project, the division 28 California State Auditor Report 2011-116.2 May 2012 also requires the construction contractor to report the total number of change orders for the project. The division uses this information to ensure that it has received all change orders. In contrast, the division does not receive all field changes—only those involving changes that are covered by building standards; the division does The division does not require not receive field changes involving paint color, for example. In contractors to report the number addition, it does not have a statewide process for recording or of field changes associated with managing field changes. Further, the division does not require a project, nor does it have any contractors to report the number of field changes associated with other process to ensure that it has a project, nor does it have any other process to ensure that it has reviewed all relevant field changes. reviewed all relevant field changes. Thus, the division cannot be certain it has received and approved all relevant plan changes. Further, the division does not require cross-references between change orders and field changes that overlap. Division policy states that change orders do not need to include approved field changes, although they may. The lack of consistent cross-referencing between change orders and field changes makes it more difficult for the division to determine whether it has approved plan changes on late-arriving change orders through field changes before construction occurred. This is a significant concern, given that division data show that out of nearly 8,800 projects that closed between July 2008 and June 2011, more than 3,000—34 percent of all closed projects—had at least one plan change document submitted after construction ended. Change orders made up nearly 95 percent of the total of 18,248 late-arriving plan changes. In addition, the lack of cross-referencing can lead to inefficient use of time when staff, unaware that the division already approved a field change, review the same change a second time when the district submits it as a change order. According to the former acting state architect, when the division receives change documents after construction is complete, the division ensures that the plan change documents complied with building standards and then gains assurance that the construction was performed appropriately. This may require additional onsite verification from the division’s field engineers or possibly reconstruction, at additional cost to districts. According to the former acting state architect, the division has made changes to its regulations to improve its management of plan changes. Specifically, regulations published in January 2012 no longer refer to “change orders” or “field changes.” Instead, they require design professionals to submit all “construction change documents” related to structural safety, fire and life safety, or access compliance portions of the project for division review and approval. This change to regulations has the potential to streamline the division’s work by focusing its review on plan changes that are subject to the building standards. However, because the division has not yet revised its related policies, the issues we identified remain unresolved. Until the division develops and implements related policies to ensure, for example, that it receives all relevant changes California State Auditor Report 2011-116.2 29 May 2012 and that it records all relevant approvals into its database, problems will remain. According to the former acting state architect, the division will begin implementing some related policies and procedures in May and that it intends to complete implementation by the end of 2012. The Division Could Improve Its Documentation of Final Reviews and Staff Attendance at Trainings The division needs to improve documentation of some of its activities. Specifically, the division does not record the final The division does not record the review performed in the fire and life safety and access compliance final review performed in the fire disciplines. Further, the division does not maintain records and life safety and access demonstrating that all its plan review staff have participated in compliance disciplines. training related to recent changes to building standards. The division cannot demonstrate that it performed final plan reviews in all three disciplines. The division has a written policy that requires a supervisor or other in-house expert to perform a final structural safety review once staff complete their initial review of plans. According to the former acting state architect, a final review of the plans is performed in each of the three disciplines before the division returns the plans to the design professional for correction. In addition, after a contracted firm completes its initial plan review, the division performs a final review of the work, in part to evaluate contractor performance. According to division policy, the extent of the final review varies with the complexity of the project. For example, an abbreviated final review may be adequate for a simple modernization project, while a more in-depth final review may be necessary for a larger, more complicated project. We expected to find that the division maintained a record of these reviews; however, the division does not do so consistently. The division does not maintain copies of the plans with its review comments once it has accepted the design professional’s corrections, but the division does record the date of the final structural safety review and the name of the reviewer in its database. Nevertheless, three of 34 projects for which contracted firms performed plan review in fiscal year 2010–11 did not have a record of final structural review by division staff. Further, the division does not capture the date and reviewer name for the final review of fire and life safety or access compliance work. Without a record of the second review in each of the three disciplines, the division cannot demonstrate that it has completed final reviews that provide an additional level of quality control and consistency and that allow the division to evaluate the performance of contracted firms. 30 California State Auditor Report 2011-116.2 May 2012 The division also cannot demonstrate that all of its plan review staff are current on changes to building standards. State law requires that updated building standards be published every three years and that any changes in the interim be made available through a supplement to the building standards. In order to provide an overview of revisions and to highlight amendments to building standards, the division provides training on these updates when they become effective. However, the division did not confirm that all of its plan review staff participated in the recent training on the triennial update, held in November 2010. Specifically, the division did not maintain attendance rosters, and other corroborating records are incomplete. In this case, the division relied on staff surveys as evidence of participation, but not all staff returned the survey. Without verification that all plan review staff participated in the training, the division cannot ensure that they are all familiar with the latest building standards. Recommendations To better gauge the timeliness of its plan review and better communicate with design professionals, the division should develop goals for the time spent on the plan review phase, in the style of those used by OSHPD, and measure and report its success at meeting these goals. In order to avoid delays in plan review, the division should develop a policy that defines when it will expedite plan review using its statutory authority to contract for additional plan review resources. To more accurately report on its plan review activities to stakeholders and provide relevant information to management, the division should: • Provide current information on its performance, by phase, at the time of the reporting period. • Exclude zero values from its calculations related to projects that did not have activity in a particular phase. • Exclude projects from client phase calculations that were not returned to the division for back check within the division’s deadlines. To appropriately oversee changes to approved plans, the division should develop policies and procedures to ensure that it: • Receives all relevant plan changes. California State Auditor Report 2011-116.2 31 May 2012 • Reviews and approves all relevant plan changes before the start of related construction. • Documents its approval of all relevant plan changes. To ensure that the division performs a final review in all disciplines, the division should require and provide a means for recording final plan review of fire and life safety and access compliance-related work in the database. To ensure that staff are current on building standards, the division should document its staff’s participation in building standards update trainings by maintaining attendance rosters. We conducted this audit under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives specified in the scope section of the report. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor Date: May 31, 2012 Staff: Jim Sandberg-Larsen, CPA, CPFO, Audit Principal John Lewis, MPA Bob Harris, MPP Jordan Wright, MPA, CFE IT Audit Support: Michelle J. Baur, CISA, Audit Principal Ben Ward, CISA, ACDA Richard W. Fry, MPA Legal Counsel: Donna L. Neville, Associate Chief Counsel Stephanie Ramirez-Ridgeway, JD For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. 32 California State Auditor Report 2011-116.2 May 2012 Blank page inserted for reproduction purposes only. California State Auditor Report 2011-116.2 33 May 2012 Appendix STATUS OF THE DIVISION’S EFFORTS TO ADDRESS RECOMMENDATIONS MADE IN RECENT REVIEWS In April 2010 and March 2011, the Department of Finance’s Office of State Audits and Evaluations (OSAE) issued reports that described the results of two reviews it performed of the Division of the State Architect (division). The first report included findings and recommendations related to the division’s Public School Planning, Design, and Construction Review Revolving Fund.9 The second report included findings and recommendations about the division’s plan review and construction oversight processes. As part of our audit, we reviewed OSAE recommendations related to the division’s fee structure and plan review function and the division’s implementation of those recommendations. The results of our review, including our determination regarding the implementation status of the recommendations, appear in Table A. Table A Analysis of the Division of the State Architect’s Progress in Addressing Selected Recommendations From the Department of Finance OFFICE OF STATE AUDITS AND EVALUATIONS SUBJECT STATUS OF NUMBER RECOMMENDATION REPORT* MATTER ANALYSIS IMPLEMENTATION 1 Develop fee structures based on the April 2010, Fees The Division of the State Architect (division) has Partially cost of providing services. page 6 developed an updated fee structure for access implemented services. However, according to the manager over the division’s fiscal unit, the division has not yet developed a new fee structure related to structural services. The division reported that depending on the availability of resources, the performance metrics unit is to conduct a study of this issue. 2 Ensure the methodology for each fee April 2010, The division has developed an updated fee Partially structure is adequately supported page 6 structure for access services. However, according implemented with detailed documentation, to the manager over the performance metrics analysis, and formal legal opinions. unit, the division has not yet developed a new fee Furthermore, formal legal structure related to structural services. The division opinions should be obtained for said it will ensure that the Office of Legal Services of policy decisions that significantly the Department of General Services (department) affect the division’s costs and/or fees is consulted on future policy decisions. prior to implementation. continued on next page . . . 9 Fees collected from applicants submitting plans for the construction or alteration of school buildings are deposited in the Public School Planning, Design, and Construction Review Revolving Fund. Money from this fund is used to cover the expenses involved with the review and approval of plans and construction oversight. 34 California State Auditor Report 2011-116.2 May 2012 OFFICE OF STATE AUDITS AND EVALUATIONS SUBJECT STATUS OF NUMBER RECOMMENDATION REPORT* MATTER ANALYSIS IMPLEMENTATION 3 Consider revising the statute April 2010, Fees The division provided proposed legislation to the Partially and regulations so that new fee page 6 department that would remove its fee structure implemented structures can be implemented from statute. The manager over the division’s with annual adjustments to fiscal unit is not aware of any further action on reflect changes in workload, the this item. construction market, the division’s costs, and other relevant factors. 4 Revise regulations and procedures April 2010, According to the division, it plans to develop Pending to ensure that refund policies are page 9 proposed regulatory changes as part of the equitable to both the division annual regulatory update cycle. and clients. 5 Revise regulations to establish a April 2010, According to the division, it plans to develop Pending minimum dollar threshold rather page 9 proposed regulatory changes as part of the than a percentage threshold for annual regulatory update cycle. assessing further fees. 6 Develop a cost-reporting system April 2010, The division has implemented revised Partially that accurately tracks all costs and page 10 accounting cost centers, position control implemented revenue by specific program at the components, and planned financial adjustments. level of detail needed to develop The division says that additional levels of appropriate fee structures. detailed tracking will be analyzed by the performance metrics unit. 7 Consider tracking revenue and April 2010, The division reported that the performance Pending expenditures for each school page 10 metrics unit will analyze this. The manager over construction project to identify the that unit said that the division could currently average project cost and significant identify the income and direct plan review costs cost overruns. related to particular projects, but that it does not have the ability to determine which exact functions (structural safety, fire and life safety, or access compliance) are responsible for costs. 8 Review the policy for charging April 2010, The division reported that its fiscal unit is Pending hours to general assignment codes page 10 currently assessing and reviewing the details of and determine if those hours its accounting database and project accounting can be charged to specific school and leave system as part of ongoing activities construction projects. related to developing a cost-reporting system. 9 Review administrative personnel April 2010, The division has implemented revised Fully duties, develop an equitable page 11 accounting cost centers, position control implemented methodology to allocate components, and planned financial adjustments. administrative personnel costs to the Public School Planning, Design and Construction Review Revolving Fund and the Disability Access Account Fund, and document the methodology. 10 Develop, document, and implement April 2010, The division has implemented revised Fully an equitable cost allocation for page 11 accounting cost centers, position control implemented division indirect costs. components, and planned financial adjustments. 11 The division and the department April 2010, The division has implemented revised Fully should review and revise the page 11 accounting cost centers, position control implemented department’s overhead allocation components, and planned financial adjustments. to ensure indirect costs charged to Fund 0328, the Public School Planning, Design, and Construction Review Revolving Fund, are equitable. California State Auditor Report 2011-116.2 35 May 2012 OFFICE OF STATE AUDITS AND EVALUATIONS SUBJECT STATUS OF NUMBER RECOMMENDATION REPORT* MATTER ANALYSIS IMPLEMENTATION 12 The division should track staff time March 2011, Fees The division reported that it has explored the Not by project and activity. page 12 feasibility of tracking hours on a project basis implemented and found it to be inefficient and impractical because of the labor-intensive effort involved in tracking staff time spent on numerous projects. 13 Headquarters should more timely March 2011, Statewide Each regional office has a member on each of Fully disseminate the statewide team’s page 9 consistency the division’s statewide teams. According to the implemented interpretation of regulations, former acting state architect, it is these members’ building codes, and decisions responsibility to inform their direct supervisor on policy to facilitate timely and and the rest of the staff in their region of any consistent implementation. statewide team decisions. 14 Enhance staff accessibility to all March 2011, The division’s public Web site and/or its intranet Fully regional offices’ Code Interpretations, page 9 feature all of the recommended items. implemented Reporting, and Tracking (Form 60) decisions, including the basis for decisions. 15 When a new interpretation of March 2011, According to the division, interpretation Fully regulations is issued, update the page 9 of regulations are posted within two days of implemented Web site immediately. approval and a notification e-mail is sent to stakeholders and staff. 16 Ensure intake requirements are March 2011, Plan review guidance is available on the Fully clearly communicated and available page 9 division’s Web site. Additionally, as of implemented to design firms and other parties March 2010, the division requires districts to involved in project development. complete a plan review checklist as part of their project submittals. 17 Provide periodic training to plan March 2011, The division reported that it conducted Partially reviewers. To promote consistency page 10 training for plan review staff ahead of the last implemented among staff, the division should update of the building code. As discussed in develop a periodic training curriculum the Audit Results section, the division did not for seasoned plan reviewers that adequately document staff attendance at this covers new building code and policy November 2010 training. The division also changes, and serves as a reminder of said that it intends to continue training efforts plan review nuances. as significant building code or procedural changes occur. 18 The statewide team for project March 2011, The division reported that it has tasked the Pending services should develop standard page 10 project services statewide team with reviewing intake guidelines. Training should existing standard intake guidelines and be provided to staff performing this recommending training to improve consistency. process to ensure consistency in the application of the intake guidelines. 19 Ensure all estimates of value of April 2010, Operations The division updated its procedures to verify Partially services (EVS) are approved prior to page 11 that EVS are approved prior to contract implemented contract negotiations, EVS templates negotiations as part of its review of contract are standardized, and methodology amendment requests. The division is developing supporting contract rates is a standardized EVS. documented. EVS are developed by the division to estimate the cost of contracting plan review work with a third party. continued on next page . . . 36 California State Auditor Report 2011-116.2 May 2012 OFFICE OF STATE AUDITS AND EVALUATIONS SUBJECT STATUS OF NUMBER RECOMMENDATION REPORT* MATTER ANALYSIS IMPLEMENTATION 20 Establish criteria that would trigger a March 2011, Operations The division reported that it participates in Pending mandatory preapplication meeting page 10 preliminary meetings with districts on request, for certain projects. When possible, encourages meetings on large or complex these preapplication meetings projects, and understands that clients are satisfied should involve the same intake, with available processes. It says that it will plan review, and field engineer staff review the feasibility of making preapplication that will eventually be assigned to meetings mandatory, including consideration the project so that agreed upon of cost implications, staffing requirements, and decisions can be carried forward. regulatory changes necessary for implementation. 21 Inform school districts and design March 2011, The division reported that it participates in Pending firms that preapplication meetings page 10 preliminary meetings with districts on request, are required for projects meeting encourages meetings on large or complex certain criteria, and be available projects, and understands that clients are satisfied on request for others. Post with available processes. It says that it will the established criteria on the review the feasibility of making preapplication division’s Web site. meetings mandatory, including consideration of cost implications, staffing requirements, and regulatory changes necessary for implementation. 22 Periodically survey school districts March 2011, The division reported that it will determine the Pending for upcoming projects and identify page 11 feasibility of developing a reporting process those that require or could benefit for large school districts to identify from a preapplication meeting. upcoming projects that would benefit from preapplication meetings. 23 The division should review the March 2011, The division reported that the California Not state budget to identify page 11 Community Colleges Chancellor’s Office has a implemented upcoming community college dedicated liaison within its facilities planning projects and identify those that unit who works with the division to coordinate require or could benefit from a planned community college facilities projects preapplication meeting. with the division’s workload. The division also noted that it participates in preliminary meetings on request. 24 The electronic plan review statewide March 2011, The division reported that an analysis of its Not team should conduct a cost/benefit page 11 electronic plan review process indicates that the implemented analysis that compares the cost process is not efficient. However, the division did of the current manual processes with not provide this analysis to the California State the costs of electronic plan review, Auditor (state auditor). including the costs to upgrade electronic plan review equipment. 25 Create a Web portal for electronic March 2011, The division reported that by the end of Pending submission of division documents page 11 fiscal year 2011–12, it will begin piloting and including applications, other division studying the feasibility of a shared electronic forms, and post-approval documents. document exchange. If a Web portal cannot be created, the division should consider using the tracking component in Tracker, other technology such as a bar code system to track post-approval documents, or a dedicated e-mail address to submit certain documents electronically. 26 Identify one person as the single March 2011, The division stated that maintaining a single Not point of contact to coordinate page 13 point of contact may be impractical because of implemented and monitor project progress. the complexity and differences in the plan review This individual would coordinate and construction oversight processes. It noted the various disciplines involved in the that it provides two primary contacts, one for plan review and serve as a liaison. plan review and one for construction oversight. California State Auditor Report 2011-116.2 37 May 2012 OFFICE OF STATE AUDITS AND EVALUATIONS SUBJECT STATUS OF NUMBER RECOMMENDATION REPORT* MATTER ANALYSIS IMPLEMENTATION 27 Assign staff to coordinate the receipt March 2011, Operations The division reported that it has identified the Fully of California Geological Survey page 13 senior structural engineer responsible for implemented (survey) reports. This will minimize structural plan review—the same person delays in the final approval of plans identified as the single point of contact for a due to the survey report not being project—as the staff person responsible for received timely. ensuring that survey materials are obtained and retained. 28 Expand cross-discipline training March 2011, The division reported that it implemented Partially where applicable and allowed page 13 cross-training for its fire and life safety and implemented by unions. accessibility plan reviewers. The division said it is aware of constraints on further cross-training and is taking them into account in evaluating additional cross-training opportunities. The percentage of fire and life safety and accessibility plan review staff that have been cross-trained is relatively small, at 15 percent. 29 Require a back check for plan March 2011, The division’s plan review guidelines allow for Not revisions, deferred approvals, and page 13 a back check of plan changes. According to implemented addendums that are a certain size the former acting state architect, requiring a or complexity. back check is left to the discretion of the division staff or the request of the school district. 30 Evaluate whether the change March 2011, The division reported that it revised regulations Pending order triage process in San Diego is page 13 related to change orders in January 2011. It said successful. Determine if this would that development of procedures to implement help streamline the change order these regulations to streamline review and process in other regions. If triaging approval of construction change documents change orders is efficient, formalize should be completed in 2012. the process for all regions. 31 The performance metrics unit March 2011, The division reported that as part of its Pending should establish performance page 13 evaluation of revised regulations related metrics related to time frames to change orders, its performance metrics for post-approval documents. unit can establish guidelines for monitoring Once performance metrics are post-approval documents. established, monitor construction oversight staff’s performance. 32 Research and resolve all Tracker April 2010, Tracker The division says that it has determined that the Partially variances and reconciling page 8 Tracker database does not have a mechanism for implemented items timely. capturing refund information. It estimates that this problem accounts for about 70 percent of noted variances. According to the performance metrics unit manager, however, the division has not yet implemented changes needed to reconcile system information. 33 Correct Tracker system errors to April 2010, The division says that its performance metrics Unknown ensure the system is operating page 8 unit detailed proposed solutions to Tracker properly and recording financial issues in a September 2010 memo. However, information accurately. the division did not provide this analysis to the state auditor. continued on next page . . . 38 California State Auditor Report 2011-116.2 May 2012 OFFICE OF STATE AUDITS AND EVALUATIONS SUBJECT STATUS OF NUMBER RECOMMENDATION REPORT* MATTER ANALYSIS IMPLEMENTATION 34 The division should conduct periodic April 2010, Tracker The division reported that this recommendation Pending tests of the Tracker system to verify page 8 will be addressed through ongoing activities of data integrity. the performance metrics unit. According to the performance metrics unit manager, the division does not currently perform any testing of the financial data in the Tracker database. 35 Continue to strengthen system March 2011, According to the division, it has identified Partially controls for Tracker to improve page 11 numerous issues and solutions for Tracker implemented data reliability. database problems and is addressing them as resources allow. Sources: State auditor’s analysis of the Office of State Audits and Evaluations’ (OSAE) recommendations, the division’s recommendations update as of December 2011, interviews with division staff, and division policies and procedures. * The April 2010 OSAE report is titled Department of General Services Division of the State Architect Public School Planning, Design, and Construction Review Revolving Fund. The March 2011 OSAE report is titled Department of General Services Division of the State Architect Plan Review and Construction Oversight. California State Auditor Report 2011-116.2 39 May 2012 (Agency comments provided as text only.) May 15, 2012 State and Consumer Services Agency 915 Capitol Mall, Suite 200 Sacramento, CA 95814 Elaine Howle* California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Re: Bureau of State Audit’s Report No. 2011-116.2 Pursuant to the Bureau of State Audit’s (BSA) Report No. 2011-116.2, enclosed are the Department of General Services’ comments pertaining to the results of the audit. The State and Consumer Services Agency would like to thank the BSA for its comprehensive review. The results provide us with the opportunity to better serve our clients and protect the public. Sincerely, (Signed by: Anna M. Caballero) Anna M. Caballero Secretary State and Consumer Services Agency Enc. * California State Auditor’s comment appears on page 45. 40 California State Auditor Report 2011-116.2 May 2012 MEMORANDUM Date: May 15, 2012 To: Anna M. Caballero, Secretary State and Consumer Services Agency 915 Capitol Mall, Suite 200 Sacramento, CA 95814 From: Fred Klass, Director Department of General Services Subject: RESPONSE TO BUREAU OF STATE AUDITS’ REPORT NO. 2011-116.2 Thank you for the opportunity to respond to the Bureau of State Audits’ (BSA) Report No. 2011-116.2 which addresses recommendations to the Department of General Services’ (DGS) Division of the State Architect (DSA). The audit focused on the DSA’s plan review functions. The following response addresses each of the recommendations. OVERVIEW OF THE REPORT The DGS appreciates the BSA’s in-depth and professional audit of the DSA’s plan review functions. In summary, the BSA identified a number of areas for improvement with the DSA’s current processes for managing workload and measuring performance. As noted below, these issues will be promptly addressed. In addition, the BSA determined1 that the DSA has established: (1) a management, control, and operational structure that is adequate for plan review and appears to provide for efficient service delivery, except in a few areas; (2) processes which provide reasonable assurance that it consistently applies its policies and practices for plan review; and, (3) contracting processes that ensure compliance with State law and its internal policies. The DGS is pleased that the BSA found that the DSA has established adequate and effective systems of operational control over these important functions and activities. With the recent appointments of a State Architect and a Deputy State Architect, a new executive management team is overseeing the DSA’s operations. Both the State Architect and Deputy State Architect have extensive experience in overseeing public and private building projects and are actively scrutinizing operations to ensure the efficient and effective use of DSA resources. As part of these efforts, a high priority has been placed on fully addressing issues raised during both the current BSA audit of plan review functions and its prior audit of the DSA’s construction oversight and project close-out functions. 1 See Table 1, Methods of Addressing Audit Objectives, contained in the report’s Introduction chapter. California State Auditor Report 2011-116.2 41 May 2012 Anna M. Caballero -2- May 15, 2012 Based on the results of its fieldwork, the BSA developed the following recommendations to further improve the DSA’s plan review functions. In general, the BSA’s recommendations have merit and will be promptly addressed. RECOMMENDATIONS RECOMMENDATION # 1: To better gauge the timeliness of its plan review and better communicate with design professionals, the division should develop goals for the time spent on the plan review phase, in the style of those used by the OSHPD, and measure and report its success at meeting these goals. DGS RESPONSE # 1: The DSA is in the process of developing systems, which will include a performance measurement and reporting component, to identify estimated dates for plans to be returned with division comments to design professionals. As part of this effort, OSHPD will be consulted on its methodology used for calculating the estimated time spent on the plan review phase of its plan approval process. RECOMMENDATION # 2: In order to avoid delays in plan review, the division should develop a policy that defines when it will expedite plan review using its statutory authority to contract for additional plan review resources. DGS RESPONSE # 2: Within sixty days, the DSA will develop additional policies which specifically address the use of its contracting authority to expedite plan review. These policies will include guidelines which provide for the consideration of various issues such as budget constraints, workload demands and available staff resources in determining if contractors will be used to perform the plan review function. RECOMMENDATION # 3: To more accurately report on its plan review activities to stakeholders and provide relevant information to management, the division should: • Provide current information on its performance by phase at the time of the reporting period. • Exclude zero values from its calculations related to projects that did not have activity in a particular phase. • Exclude projects from client phase calculations that were not returned to the division for back check within the division’s deadlines. 42 California State Auditor Report 2011-116.2 May 2012 Anna M. Caballero -3- May 15, 2012 DGS RESPONSE # 3: The DSA is committed to providing relevant plan review information that ensures transparency to stakeholders and management. Toward this end, the division will enhance and/or revise its plan approval performance measurement and reporting process to address the three areas for improvement identified by the BSA. RECOMMENDATION # 4: To appropriately oversee changes to approved plans, the division should develop policies and procedures to ensure that it: • Receives all relevant changes. • Reviews and approves all relevant plan changes before the start of related construction. • Documents its approval of all relevant plan changes. DGS RESPONSE # 4: As noted in the report, the DSA recently made changes to its regulations that result in a more simplified and streamlined process for approving construction change documents. The regulations no longer refer to change orders or field changes whose tracking led to some of the concerns developed by the BSA. Instead, they require design professionals to submit all construction change documents related to the structural safety, fire and life safety, or access compliance portions of the project for division review and approval. The DSA is committed to ensuring that the new regulations are effectively implemented. Upon completion of the implementation phase, which is currently planned by the end of 2012, the DSA will have implemented additional processes that provide further assurance that all relevant plan changes are received, reviewed, approved and documented by the division. Recently, the DSA initiated the first phase of implementing the new requirements which involved the creation of a process for K-12 school districts and community college districts to self-report the final costs of their construction projects. RECOMMENDATION # 5: To ensure that the division performs a final review in all disciplines, the division should require and provide a means for recording final plan review of fire and life safety and access compliance-related work in the database. DGS RESPONSE # 5: Although the DSA’s supervisors or other technical experts currently conduct quality-control checks (referred to as final review) in all three disciplines prior to returning plans for needed corrections to design professionals, a record of this activity is only maintained in the database for the structural safety discipline. As recommended by the BSA, the DSA will revise its processes to ensure that the final review activity for the fire and life safety and access compliance disciplines are also recorded in the project database. California State Auditor Report 2011-116.2 43 May 2012 Anna M. Caballero -4- May 15, 2012 RECOMMENDATION # 6: To ensure that staff are current on building standards, the division should document its staff’s participation in building standards update trainings by maintaining attendance rosters. DGS RESPONSE # 6: As recognized in the audit report, the DSA proactively provides training to staff, school employees, private 1 architects and engineers, construction engineers, project inspectors and others on changes to building standards as they become effective. To assist in ensuring that its staff attends the required training, the DSA recently issued requirements that attendance rosters in the form of sign-in sheets be maintained for each class and that attendee information be entered into the DGS’ integrated human resources and fiscal management system. CONCLUSION The DGS is firmly committed to effectively and efficiently overseeing the plan review functions performed by the DSA. As part of its continuing efforts to improve that process, the DGS will take appropriate actions to address the issues presented in the report. If you need further information or assistance on this issue, please contact me at (916) 376-5012. (Signed by: Fred Klass) Fred Klass Director 44 California State Auditor Report 2011-116.2 May 2012 Blank page inserted for reproduction purposes only. California State Auditor Report 2011-116.2 45 May 2012 Comment CALIFORNIA STATE AUDITOR’S COMMENT ON THE RESPONSE FROM THE STATE AND CONSUMER SERVICES AGENCY, DEPARTMENT OF GENERAL SERVICES To provide clarity and perspective, we are commenting on the response to our audit report from the Department of General Services (department). The number below corresponds to the number we placed in the margin of the department’s response. The department characterizes our conclusions about the Division 1 of the State Architect’s training efforts too broadly. In particular, our report does not address training related to school employees, construction engineers, or project inspectors. 46 California State Auditor Report 2011-116.2 May 2012 cc: Members of the Legislature Office of the Lieutenant Governor Little Hoover Commission Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press