Summary

Blackout Preparedness: The Office of Emergency Services and the California National Guard Each Have Weaknesses in Their Blackout Preparations September 2001 2001-111.1 rotiduA etatS ainrofilaC S T I D U A E T A T S F O U A E R U B The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TDD (916) 445-0255 x 216 OR This report may also be available on the World Wide Web http://www.bsa.ca.gov/bsa/ The California State Auditor is pleased to announce the availability of an online subscription service. For information on how to subscribe, please visit our Web site at www.bsa.ca.gov/bsa. If you need additional information, please contact David Madrigal at (916) 445-0255, ext. 201. Alternate format reports available upon request. Permission is granted to reproduce reports. C S A ALIFORNIA TATE UDITOR ELAINE M. HOWLE STEVEN M. HENDRICKSON STATE AUDITOR CHIEF DEPUTY STATE AUDITOR September 11, 2001 2001-111.1 Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the Bureau of State Audits presents its report titled Blackout Preparedness: The Office of Emergency Services and the California National Guard Each Have Weaknesses in Their Blackout Preparations concern- ing the Office of Emergency Services’ and the California National Guard’s preparedness for a power outage. SUMMARY T he Governor’s Office of Emergency Services’ (OES) respon- sibility is to help the State mitigate, plan, and prepare for, respond to, and recover from the effects of emergencies. With regard to blackouts, the OES must ensure that it can function should it lose power. The OES has an emergency generator to power its headquarters, which includes its State Operations Center and it told us that it regularly tests and maintains the generator so that it will run when necessary. In addition, the OES has worked with the emergency response community to share information about the energy crisis and assist them in planning for blackouts. For example, the OES has held meetings and workshops, developed a notification process to warn appropriate parties when blackouts are coming, and developed a guide for local governments to use in plan- ning for power outages. BUREAU OF STATE AUDITS 1 555 Capitol Mall, Suite 300, Sacramento, California 95814 Telephone: (916) 445-0255 Fax: (916) 327-0019 www.bsa.ca.gov/bsa However, the OES may have other issues that weaken its preparedness for blackouts. In March 2001 the OES distributed an Energy Shortage Response Matrix (response matrix) to its staff. The response matrix provides background and insight into potential public safety impacts, state actions to date, and the OES’ policy relating to energy response. The OES asserts that it has taken steps to address some of the activities found in that document, but we are uncertain if or how it has resolved a few key concerns it raised. For example, the OES’ response matrix indicates that it needs to refine its Business Continuity Plan (continuity plan). Although the OES asserts that its staff reviewed this plan in June 2001, it did not provide any evidence of changes it made or changes that may be pending. The OES also does not have a method in place to periodically evaluate its process for notifying the public, public safety agencies, and the media about anticipated blackouts nor has it designated staff or trained them to assist the local governments in using its planning guide. Consequently, the OES cannot ensure the effectiveness of its notification process or that local governments are receiving the planning assistance they seek. When disasters or emergencies occur in the State, such as floods, earthquakes, or blackouts, the California National Guard (CNG) does not automatically mobilize to assist. Rather, the CNG must wait for the OES to direct it to where it is needed and how to assist in the disaster or emergency. Similar to the OES, the CNG’s primary responsibility is to insulate itself from the effects of a blackout so it can function should OES mobilize it. In June 2001 the CNG issued its Power Outage Plan (outage plan). The plan appears to cover most issues critical to the CNG such as methods of communication, temporary electrical supplies, and weapons security. However, because the CNG does not ensure that it is ready to implement all of the outage plan, this raises doubts about the CNG’s preparedness. For example, the CNG does not ensure that staff check and charge their cell phone batteries or periodically test and maintain their tactical generators. Most importantly, the CNG’s outage plan is silent on the steps to be taken to ensure that its back-up generator, which supplies power to parts of its headquarters building will remain operable. Thus, the CNG has limited assurance that its back-up generator will be readily available to provide the necessary power to operate its Joint Operations Center so that the CNG can respond fully to any OES request. 2 BACKGROUND In 1996 the California Legislature passed Assembly Bill 1890, deregulating the State’s electrical industry. Among other things, the legislation created a nonprofit, public-benefit corporation, the California Independent System Operator (ISO), to manage the power distribution grid that supplies electricity to almost all of the State. The ISO continuously monitors its operating reserves to keep the electric power system stable and functioning. However, at times there has not been enough energy available to meet the State’s needs. Therefore, in order to keep the entire system from shutting down, the ISO ordered rotating customer outages in January 2001, and at other times since, leaving millions of people within the State without power for periods of time. The OES coordinates the overall response of state agencies to major disasters in support of local government. The OES is responsible for assuring the State’s readiness to respond to and recover from natural, man-made, and war-caused emergencies; this may include electrical blackouts. The OES’ response to disasters in California is based on a statewide system of mutual aid, which obligates the State to provide available resources to assist local governments in their emergency preparedness, response, and recovery efforts. In the event of a power loss, the local governments such as cities, counties, and special districts must immediately put their emergency response plans into operation. If the situation escalates beyond their ability to control it, they can submit a formal request for assistance to the OES. As such, the OES may call upon state agencies to help provide support. Due to their expertise, the CNG is one of the agencies the OES most often asks to assist with emergency situations. The CNG is comprised of the Army National Guard and the Air National Guard and has roughly 23,000 members occupying 127 armories and 9 air bases and stations throughout the State. The CNG’s primary responsibility is to mobilize its units for combat and peacekeeping missions at the direction of the President of the United States. As a response agency, the CNG typically does not deploy its units until after it receives a request for assistance from the OES. 3 SCOPE AND METHODOLOGY The Joint Legislative Audit Committee asked the Bureau of State Audits to determine whether the CNG has a plan to deal with blackouts resulting from the State’s energy shortage. Our review also includes an evaluation of the OES’ plan since it is primarily responsible for assuring the State’s readiness to respond to and recover from man-made emergencies such as electrical blackouts. To assess the OES’ preparedness to address blackouts and its efforts to assist local governments to prepare for blackouts, we interviewed OES staff. We also reviewed its response matrix and its Electric Power Disruption Toolkit for Local Government (toolkit), which contains guidance on preparedness, response, recovery, and mitigation actions relevant to electric disruptions. Finally, we reviewed its agendas and minutes from certain committees that discussed blackout issues. To assess CNG’s ability to remain viable so that it is available to assist others during a blackout, we interviewed CNG staff. We also reviewed its plans and procedures for mitigating the effects of a blackout. We focused primarily on its headquarters and Army National Guard facilities. Its headquarters facility houses the Joint Operations Center, which activates and coordinates CNG support throughout the State. Additionally, the Army National Guard responds most frequently to the OES’ requests for assistance. THE OFFICE OF EMERGENCY SERVICES HAS AN ALTERNATIVE POWER SOURCE DURING A BLACKOUT BUT OTHER CONCERNS ABOUT ITS PREPAREDNESS EXIST In the event of a blackout, the OES has a generator at its headquarters as an alternative power source. According to the OES, it runs and inspects the generator on a regular basis, which is a reasonable precautionary step to ensure that this critical facility will have power. However, the OES may have other weaknesses that can affect its blackout preparedness. In March 2001 the OES distributed to its staff a response matrix, which provides background and insight into potential public safety impacts, state actions to date, and its policy relating to energy responses. The OES asserts that it has taken steps to 4 address some of the activities found in the matrix, but we are uncertain if or how it has resolved a few key concerns that have been raised in its response matrix. The OES’ Headquarters Generator Can Help to Minimize the Effects of a Blackout When a blackout occurs, the OES will need to ensure that it insulates itself from the effects or can quickly recover in order to coordinate the State’s overall response to the emergency. The OES headquarters houses its State Operations Center, which is one of the key locations it uses to receive and process local governments’ requests for assistance. According to the OES, it: (cid:127) Tests the generator weekly. (cid:127) Periodically inspects the fuel levels. (cid:127) Maintains an annual service contract which conforms to the manufacturer’s specifications. (cid:127) Has certain staff on 24-hour shifts that can start the generator if it fails to start automatically. If the OES is taking these steps as it asserts, it is helping to ensure that the generator will run and power its State Operations Center in the event of a blackout. Some Concerns Raised by the OES In Its Energy Shortage Response Matrix About Its Blackout Preparedness May Still Exist In March 2001 the OES distributed to its staff a response matrix, which provides background and insight into potential public safety impacts, state actions to date, and its policy relating to energy responses. For certain potential public safety impacts, the OES identified additional steps to be taken to minimize disruption to its operations. Specifically, the OES found that an evaluation of its plans for transferring responsibilities for critical functions to unaffected units and relocating staff to an alternative work site was necessary to refine its continuity plan. It also recognized the need to evaluate its continuity plan and emergency proce- dures to ensure back-up systems are operational and whether it could handle a natural disaster during an energy crisis. 5 These issues appear to represent valid concerns the OES had about its own preparedness and could prove to be weaknesses in the OES’ ability to internally function during a blackout; there- fore, we asked the OES to demonstrate how it had addressed them. The deputy director of Emergency Operations, Planning, and Training Division asserts that in 1999 the OES updated its Relocation and Transfer Plan. However, his statement is incon- sistent since the OES’ March 2001 response matrix continues to highlight concerns that there are facilities without back-up generators and there are those with a back-up generator that may not be large enough to house additional staff and activities that may be necessary. Further, regarding the OES’ continuity plan and emergency procedures, the deputy director asserts that staff reviewed this plan in June 2001 and that the OES continually reviews and updates its emergency procedures as it identifies the need. Despite his claims that the continuity plan was just reviewed and emergency procedures are examined continuously, the deputy director did not provide any evidence such as changes the OES made or changes that may be pending. As a result, the steps the OES has taken to deal with the concerns it raised in its response matrix are unclear. THE OES HAS TAKEN STEPS TO INFORM THE EMERGENCY RESPONSE COMMUNITY AND OTHERS ABOUT BLACKOUTS BUT SOME EFFORTS COULD BE STRONGER In addition to preparing itself for blackouts, the OES has worked with the emergency response community to share information about the energy crisis and assist them in planning for blackouts. The OES shares information with local governments, state agencies, and the private sector regarding the impacts of blackouts and what these groups could do to mitigate their effects. By meeting and sharing information in this way, the OES is raising awareness of the effects of blackouts and helping these groups to better prepare for them. The OES has also implemented a notification process that provides for a series of alerts prior to a potential blackout. However, the OES lacks a way to evaluate its effectiveness and therefore, may overlook necessary changes or improvements. Finally, the OES developed a guide for local governments in planning for power outages. Although this guide addresses many critical planning issues, the OES may not be able to assist local governments because it has not designated staff to respond to inquiries nor has it trained its staff on how to use the planning document. 6 Through Committee Meetings and Workshops the OES is Sharing Information About Blackouts With the Emergency Response Community The OES organizes and hosts regular meetings of two committees where various emergency response agencies can disseminate information and discuss disasters in California so that they can better anticipate and mitigate the effects of disasters. Specifically, there are the Statewide Emergency Planning Committee (SWEPC) and the Mutual Aid Regional Advisory Committee (MARAC). Staff from nearly 47 different state agencies, departments, offices, and commissions involved in handling disasters in California are on the SWEPC. MARAC is not a single body, rather there are six such committees throughout the State, with members from local government, state agencies, and other organizations such as the American Red Cross. Both SWEPC and MARAC meet quarterly and from January through April 2001 some of these meetings have included discussions on energy issues. For example, at a January 2001 MARAC meeting the topic was the energy crisis and the importance of energy conservation. In March 2001 the ISO and the Department of General Services (DGS) gave a presentation on the blackout notification process to SWEPC. In addition, in May 2001 the OES held three workshops throughout the State. Attendees were representatives in fields such as law enforcement, fire service, and emergency medical services. The intent of the workshops was to better inform emergency managers and elected officials about potential out- ages and their impact on local public safety by providing an overview of the energy situation, power outage procedures and impacts, and suggestions on conservation measures. Through its committee meetings and workshops the OES continues to inform local governments about energy issues. The OES Has a Blackout Notification Process, but Has Not Documented Its Assessment of the Process On June 5, 2001, the governor issued Executive Order D-38-01 requiring the OES to develop an implementation plan, by June 15, 2001, for notifying public safety agencies, the media, and the public of potential electrical blackouts. Although the OES’ implementation plan appears reasonable, if it were necessary to 7 use the process, the OES does not have a way to evaluate its effectiveness. As a result, the OES may not identify and make necessary changes or improvements so that the notification process is as effective and efficient as possible. The notification process is a series of messages the ISO initiates at 48-, 24-, and 1-hour intervals before a blackout. The ISO uses forecasts of demand considering the weather throughout the State and supply conditions within its control area to generate notices that convey the potential for blackouts. The ISO sends the notices it initiates to the OES and utilities using multiple means of communication including an Emergency Digital Infor- mation Service, the telephone, e-mail, and paging services. In addition, the ISO sends notices to the media who share this information with the public. This process varies slightly for the ISO’s 1-hour notice. Specifically, when the OES receives the ISO’s 1-hour notice, through its Response Information Management System, the California Law Enforcement Telecommunications System, the California Warning and Alerting System, e-mail, and paging services, the OES notifies the public safety agencies. In addition, after receiving the 1-hour notice, the utilities must tell the media, general public, and public safety agencies in their potentially affected service areas. The notice includes the time and duration of the potential blackout, common geographical boundaries, grid or block numbers, and maps or similar identi- fying information readily understood by the public and affected customers. On July 3, 2001, the ISO issued a 1-hour notice; however it subsequently canceled the notice. Although this was an opportunity to evaluate the process to see if all parties received notifications promptly, the deputy director of Emergency Operations, Planning, and Training Division, told us that the OES did not specifically assess this event to identify any necessary changes or improvements to the process. However, the deputy director asserts that both before and after the July 2001 event, the OES has had an ongoing series of scheduled communications with those involved in the notification process to refine and improve the operational details of implementing it. Even though the deputy director did provide dates for conference calls that took place and the agencies who were part of the stakeholders group, he did not provide sufficient information such as minutes to substantiate these communications or their outcome. As a result, we could not concur that its scheduled communications are the OES’ attempt to assess the notification process and improve it or determine if they have an entirely different purpose. 8 Because the OES could not demonstrate that it has a method in place to evaluate the notification process’ effectiveness, the OES may not be able to identify and make necessary changes or improvements to ensure an effective and efficient process exists. Although the OES Has Prepared Blackout Planning Guidance for Local Governments, It May Not Be Able to Provide Assistance to Them Recently, to identify possible actions that local governments such as cities and counties can take to protect public health and safety during blackouts, the OES released a toolkit planning guide. By providing this type of guidance to local governments, the OES helps them to recognize how to prepare for, respond to, and recover from blackouts. Although the toolkit appears to convey to local governments many ideas critical to dealing with a power outage, the OES may not be fully prepared to assist them in implementing the toolkit’s guidance. Specifically, the OES expects that the local governments will contact them if they need assistance in planning for a blackout. However, until the OES designates and trains staff to respond to local governments’ call for assistance, it may not provide the appropriate and necessary information they need to implement the toolkit’s guidance. In July 2001 the OES distributed the toolkit to local governments, including cities, counties, and special districts. The toolkit identifies generally the types of blackouts a community may face, customers potentially affected by the blackouts, and facilities and populations with critical electrical needs. The toolkit also includes OES’ general planning assumptions for local governments to consider such as determining which emergency service providers need electricity to conduct their public safety duties and creating plans flexible enough to enable them to respond to short- and long-term power outages. In addition, the toolkit provides some criteria the local governments could use to identify critical facilities that are vital to the community’s well-being and their vulnerable customers such as people on electronically powered medical support equipment, the elderly who live alone, and people with mental and physical disabilities. So that local governments use the toolkit correctly, we would expect the OES to have staff who are familiar with it and are able to assist them. According to the deputy director of Emergency Operations, Planning, and Training Division, if local governments have questions about the toolkit and need technical assistance 9 they can call the OES. However, as the OES did not include specific contact information in the toolkit itself, as part of the distribution package, or on its Web site and it did not assign staff to accept inquiries, when local governments contact the OES for technical assistance, there is a potential that they may get passed on to multiple staff and not receive the assistance they need at all. The deputy director also told us that it is not necessary to train staff on the toolkit specifically because the OES’ staff possess emergency planning and response experience. Since it did not designate and train staff to accept the inquiries, local govern- ments could get assistance from staff with less expertise and may get erroneous information. THE CALIFORNIA NATIONAL GUARD’S POWER OUTAGE PLAN COVERS MOST CRITICAL AREAS, BUT THE GUARD DOES NOT ENSURE THAT IT STAYS PREPARED FOR BLACKOUTS In April 2001 the DGS required all state agencies and departments to immediately update their emergency preparedness plans to address blackouts resulting from California’s unprecedented period of electrical shortages. In June 2001 the CNG distributed its outage plan to all staff. The plan outlines how the CNG intends to maintain its ability to conduct daily operations and to respond to emergency missions. The plan appears to ad- equately cover most issues critical to the CNG when a blackout occurs such as methods of communication, temporary electrical supplies, and weapons security. For example, in the event of a blackout, the outage plan instructs CNG’s armory personnel to test its vault alarm system, and if necessary commanders must order staff to secure the vault. Additionally, the CNG uses a contractor to perform ongoing electronic monitoring and periodic physical inspections of the alarm system, which in- cludes a primary power failure and back-up endurance test. However, the CNG does not ensure that it is ready to implement other aspects of the outage plan and that raises doubts about the CNG’s preparedness for power outages. For example, the plan specifies that staff are to use cell phones if commercial telephone systems are down, but the CNG does not ensure it periodically checks and charges the cell phone batteries. In addition, the outage plan calls for the use of tactical generators if necessary, but the CNG also does not make sure that it periodically tests 10 and maintains the generators housed at the armories or other locations. Most importantly, the plan is silent on the steps to be taken to ensure that the CNG’s back-up generator, which sup- plies electricity to certain parts of its headquarters building will remain operable. We found that, although the CNG had a con- tract for semiannual maintenance, the contract expired on June 30, 2001. Without ongoing monitoring and maintenance of its headquarters’ generator, the CNG has limited assurance that its back-up generator will be readily available to provide the necessary power to operate its Joint Operations Center so that the CNG can respond fully to any OES request. Although Its Communication Systems Are Redundant, the CNG’s Lack of Maintenance Weakens These Systems The CNG’s outage plan specifies that the armories are to rely on commercial telephone systems as the primary means of commu- nication. If commercial services are unavailable, the plan directs staff to use two alternative communication methods: high frequency radios (HF radios) and cellular phones. Although the CNG’s outage plan appears reasonable in that it provides for redundant methods of communication, because the CNG does not ensure that its HF radios and cell phones are intact and operational, it cannot be certain that these alternatives will be available when necessary. HF radios To allow for long-distance communications, the CNG has issued HF radios to 19 strategic locations. However, despite weekly tests, the CNG has done a poor job making sure that all radio operators participate in the tests and that the HF radios are operational. As a result, the CNG cannot be sure that this system of communication will be available when it is needed. On a weekly basis, according to its State Area Command Emergency Net Standard Operating Procedures, the CNG is to test the HF radios by calling each location individually. However, not all operators participate in the test and respond to the signal. For example, from January 4, 2001, through August 23, 2001, the CNG’s highest rate of participation was 56 percent, or 11 of 19 radios. For one week in August 2001 the CNG’s HF radio test participation rate dropped to 11 percent, or 2 of 19 11 radios. According to the Military Support to Civilian Authorities (MSCA) communications officer, some radios are broken or the operator is unavailable to participate in the test for reasons such as their attendance in training activities. In late 2000 the CNG prepared a schedule to assess and service the HF radios. Between December 2000 and February 2001 the CNG visited 13 locations and by mid-April 2001 at least 10 of these locations had radios that were operational. Although the CNG made a great deal of progress up to April 2001, its adherence to the schedule appears to have stalled. For example, the CNG identified 6 locations it needed to inspect and repair during the second quarter of 2001. However, none of this work was done. Although the MSCA communications officer told us that the CNG intends to assess additional HF radios, his schedule is not final. In addition, the CNG has no plans in place to visit all locations with HF radios on an ongoing basis to conduct periodic maintenance checks. Specifically, the CNG told us that it lacks sufficient funding to continue to conduct these checks. Without such plans in place, the CNG risks not being able to quickly communicate with its units when commercial phone systems and cell phones are unavailable. Cell Phones In December 1999 the CNG distributed one cell phone to each armory. The outage plan instructs the Army National Guard to “test the cellular phone issued for the armory. Make sure the battery is charged.” As the first alternative means of communi- cation and considering that the CNG’s HF radio network is in poor condition, we would expect the CNG to take steps to ensure that its cell phone communications are reliable. However, although the plan instructs the armories to charge the cell phone battery, the CNG does not periodically follow up with the armories to ensure that this is done. According to the director of Plans, Operations and Security, the CNG relies on its commanders to make sure the cell phone battery is charged and does not conduct battery checks. More- over, the director stated that because of the rate plan the CNG has, it incurs cell phone monthly service and per minute charges only when the phones are in use. As such, to turn the phones on and make calls to test the batteries would incur unnecessary phone expenses. 12 Based on its outage plan, the CNG’s cell phones appear to be a critical link in its communication system. In fact, the cell phones may be more critical than the CNG expects considering that its HF radio network, the other alternative specified in its outage plan, may not be reliable. Although the CNG’s decision to arrange for a rate plan where it incurs charges only when the cell phones are in use seems prudent, this should not limit the CNG’s ability to protect the integrity of their communications. Also, the CNG could test cell phone batteries in ways that do not require placing a phone call and incurring needless phone charges, such as simply turning the phone on and off. Moreover, by not periodically testing each phone, the CNG seems to be missing out on some other important benefits. For example, these tests would help make certain that the: (cid:127) Users can physically locate the phones. (cid:127) Appropriate personnel have access to the phone. (cid:127) Phones do not have other mechanical problems. Until the CNG sets in place a process to periodically check that each phone is operating, its communications system may be weakened. The CNG Does Not Monitor Its Tactical Generators’ Operability The CNG’s outage plan specifies that tactical generators may be used in CNG facilities when power is essential for safety, security, and mission requirements. The CNG normally uses tactical generators when staff are in the field and need a power supply for their equipment. Although these generators cannot be connected to the buildings’ electrical system to supplant tradi- tional power sources, they can be used to operate portable light fixtures and radios thereby contributing to the normal operation of a CNG facility during a blackout. However, the CNG does not ensure its facilities periodically test its tactical generators. Prior to the release of its outage plan, on March 20, 2001, the CNG issued Warning Order 01-6 requiring its commanders to check and test their tactical generators to make sure the equipment was operational. However, this specific requirement is absent from the CNG’s formal outage plan. According to the director of Plans, Operations and Security, staff report monthly on the operational status of some generators, but at any one time, only 13 about 85 percent of these generators are operational. Moreover, he told us that the generators are not critical equipment for most of the missions the CNG conducts, unlike vehicles and aircraft. Although the generators may not be critical equipment on an ongoing basis, during a blackout, it seems reasonable that their importance would increase. Therefore, we would expect the CNG to test them periodically and closely monitor their operational status. If it does not, the CNG has limited its assurance that it can use these generators in the event of a blackout. The CNG Does Not Include In Its Plan or Adequately Monitor its Headquarters’ Back-Up Generator The DGS expects state agency and department emergency plans to address how they will ensure that any back-up generator sources are tested and readily available. Although the CNG’s plan addresses tactical generators, it does not address the back-up generator in its headquarters building. Should power to the headquarters building be interrupted for any reason, the back-up generator will automatically turn on and supply electricity so that the Joint Operations Center will have access to its telephone and computer systems, and limited lighting. According to the director of Plans, Operations and Security, once a week an automatic timer trips and the back-up generator will start up and run for several minutes to ensure the generator is working properly. Because the back-up generator is critical to the Joint Operations Center’s operations during a blackout, we would expect the CNG to include this generator in its plans and much like the OES asserted it does, to have policies and procedures in place for tracking the weekly generator test and as part of that test, inspecting the generator for sufficient fuel, leaks, or other malfunctions. However, according to the MSCA communications officer responsible for the headquarters’ generator, no such policies or procedures exist; he simply listens for the generator to start up each week. Until June 30, 2001, the CNG had a maintenance agreement on the generator. Twice a year the CNG’s contractor was to perform a 20-point inspection and once a year change the generator’s oil and oil filter. The last time the contractor inspected the generator under this agreement was October 2000. As of August 2001, the CNG had not renewed its maintenance agreement. The director of Plans, Operations and Security told us that he expects to renew the maintenance contract but will not initiate that process until after necessary environmental compliance modifications are completed at the end of September. The director believes it will 14 take 30 to 60 days to arrange for a new contract. Therefore, the CNG will not have a new maintenance agreement in place until November 2001, possibly December 2001—more than 1 year after the last maintenance inspection. Because the CNG does not fully inspect the generator as part of its periodic tests and it currently has no maintenance contract in place, until such time as it lets a maintenance contract, the CNG has limited its assurance that the generator will be readily available to supply necessary power during a blackout. RECOMMENDATIONS To strengthen its blackout preparedness, the OES should take the following actions: (cid:127) At a minimum, review and document its efforts to ensure that its relocation and transfer plan, business continuity plan, and emergency procedures address sufficiently the State’s energy situation. (cid:127) Establish a method to periodically evaluate its notification process, which includes documenting the results of its evalua- tions and following up with participants to ensure that all necessary changes are made. (cid:127) Assign specific staff to be responsible for responding to local governments’ inquiries about its toolkit. It should also train these staff on how to use the toolkit and advise local govern- ments on their planning efforts. To strengthen its readiness for blackouts, the CNG should take the following actions: (cid:127) Develop a plan that sets forth inspection dates for each location with an HF radio, the person responsible for the inspection, and a date certain for the completion of all repairs; and continue with these maintenance checks on an ongoing basis. (cid:127) Establish a process to periodically check that each cell phone is operating and the batteries are fully charged. (cid:127) Develop policies and procedures for testing and maintaining its tactical generators and include these policies and procedures in its outage plan. In addition, continue to monitor the operational status of these generators. 15 (cid:127) Update its outage plan to address its headquarters’ back-up generator that it needs to operate its Joint Operations Center, periodically inspect it for leaks, check its fuel levels and the other critical elements, and execute a maintenance contract to ensure that more extensive inspections occur on an ongoing basis. We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, ELAINE M. HOWLE State Auditor Date: September 11, 2001 Staff: Joanne Quarles, CPA, Audit Principal Sharon L. Smagala, CPA Stephanie Chan, CPA 16 Agency’s comments provided as text only. Department of the Army and the Air Force Office of the Adjutant General California National Guard 9800 Goethe Road Sacramento, California 95826-9101 September 7, 2001 Ms. Elaine M. Howle California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: Thank you for forwarding the draft copy of Blackout Preparedness, Audit No. 2001-111.1. I would like to provide the following responses to the respective four recommendations made by the auditors in the draft audit report. It is my understanding that you will include my responses as a part of the published audit report. Auditor’s Recommendation No. 1: Develop a plan that sets forth inspection dates for each location with a HF radio, the person responsible for the inspection, and a date certain for the completion of all repairs; and continue with these maintenance checks on an ongoing basis. Military Department’s Response to Recommendation No. 1: The Plans and Operations Communications Officer will monitor the weekly requirement for remote Army National Guard facilities with HF radios to make radio contact with the Guard Head- quarters in Sacramento by developing a checklist which ensures that all 19 remote locations make contact, their radios are functioning, inoperable radios are reported for repair purposes, and a follow-up system to ensure that inoperable radios are repaired on a timely basis with available state funded resources. Additional State General Fund dollars will be pursued to maintain the HF radios, purchase new radios to replace outdated and non-repairable radios, and continue periodic inspection visits to the 19 HF armories. The 19 remote HF radios were procured and are maintained with State funds. In addition, federal military HF radio systems can be transported to any of the state resourced 19 remote HF locations if and when they are inoperable during state emergency conditions. 17 -2- Auditor’s Recommendation No. 2: Establish a process to periodically check that each cell phone is operating and the batteries are fully charged. Military Department’s Response to Recommendation No. 2: The Military Department’s Power Blackout Plan dated 31 May 2001, will be revised in order to require Armory Commanders to turn on their armory cell telephones on a monthly basis and ensure the batteries are charged without incurring message charges. In addition, the Military Department’s three (3) statewide Emergency Response Planners will spot check the operability of armory cell telephones and provide the results to the Director of Plans, Operations and Security in quarterly written reports. Auditor’s Recommendation No. 3: Develop policies and procedures for testing and maintaining its tactical generators and include these policies and procedures in its power outage plan. In addition, continue to monitor the operational status of these generators. Military Department’s Response to Recommendation No. 3: Directions to Army Guard Commanders contained in Warning Order 01-6, which required them to check and test their tactical generators to make sure the equipment is operational, will be added to the Military Department’s revised Power Blackout Plan. In addition, the Military Department’s three (3) statewide Emergency Response Planners will spot check the operability of tactical generators located at armories and provide a written quarterly report to the Director, Plans, Operations and Security. Auditor’s Recommendation No. 4: Update its power outage plan to address its headquarters’ back-up generator that it needs to operate its Joint Operations Center, periodically inspect it for leaks, check its fuel levels and the other critical elements; and execute a maintenance contract to ensure that more extensive inspec- tions occur on an ongoing basis. Military Department’s Response to Recommendation No. 4: The ongoing maintenance contract is being renewed for the backup emergency generator at Guard Headquarters. Instructions will be included in the Department’s revised Power Blackout Plan that will require a weekly checklist be implemented by the Plans and Operations Communica- tions Officer which will ensure the backup generator is exercised weekly, and that fuel, oil, and water levels are checked on a weekly basis. 18 -3- Thank you for considering the Military Department’s responses. If further dialogue is neces- sary on the contents in the draft audit report, Colonel Charles Clifton remains the Department’s point of contact. He can be reached at (916) 854-3699. Sincerely, (Signed by: Paul D. Monroe, Jr.) Paul D. Monroe, Jr. Major General The Adjutant General 19 20 Agency’s comments provided as text only. Governor’s Office of Emergency Services To: Elaine M. Howle, State Auditor* Bureau of State Audits FROM: Dallas Jones, Director (Signed by: Dallas Jones, Director) DATE: September 7, 2001 SUBJECT: RESPONSE TO DRAFT AUDIT REPORT The purpose of this memorandum is to respond to the Bureau of State Audits’ (BSA) draft report entitled “Blackout Preparedness: The Office of Emergency Services and the California National Guard Each Have Weaknesses in Their Blackout Preparations,” which was forwarded to our office by letter dated September 5, 2001. OES has reviewed the findings and recommendations outlined in BSA’s draft report, and its overall response is that a better understanding of OES’ primary mission and overall emergency management activities would alleviate many of the concerns presented. While the audit report focuses on the area of blackout preparedness, it is important to understand that OES, under the Emergency Services Act and other authorities, is required to oversee the state’s preparedness for all hazards, whether natural or technological. What are characterized in the 1 report as “weaknesses” in blackout-specific preparedness activities, are areas which are already addressed by pre-existing, all-hazard emergency management practices. In fact, in such a large and diverse state that is vulnerable to so many types of disasters, standardized practices and a multi-hazard approach are necessary to ensure that all California citizens are prepared for whatever hazard may arise, blackout or otherwise. Specific responses to issues raised by the BSA report are as follows: “Some Concerns Raised by the OES In Its Energy Shortage Response Matrix About Its Blackout Preparedness May Still Exist” The report asserts that in March 2001, OES identified additional steps to be taken to minimize disruptions to its own operations, and that these may still be unresolved. As discussed with BSA previously, the OES headquarters facility has been deemed a “critical facility” and is thus exempt from rotating outages that may occur as a result of the state’s energy crisis, which is the focus of this audit. However, due to its all-hazards approach OES recognizes that other situations, such as floods or earthquakes, can result in loss of power or otherwise require the relocation of critical operations functions. We feel that this potential situation is addressed by 2 the existing headquarters backup generation capability, the Business Continuity Plan (which was revised in preparation for the Year 2000 rollover), as well as the Relocation and Transfer * California State Auditor’s comments begin on page 23. 21 Plan. OES is also in the process of permanently relocating to a new facility within the next several months. The issues of backup generation, relocation and business continuity will be carried over as pertinent to the new facility upon completion of this move. “The OES Has Taken Steps to Inform the Emergency Response Community and Others About Blackouts But Some Efforts Could Be Stronger” The report asserts that OES lacks a way to evaluate its effectiveness in providing blackout notifications and may thus overlook necessary changes or improvements. We feel there is no 3 need to specifically evaluate effectiveness of blackout notifications because OES utilizes these same notification processes and tools for all other types of disasters and emergencies on a daily basis. Therefore we are constantly in the process of exercising, refining and improving notification systems and processes. Further, as added value, OES has developed a system to receive feedback from its customers pertaining specifically to blackout planning, including notifications, through its scheduled stakeholders conference calls. The BSA report further asserts that OES may not be able to assist local governments because it lacks designated, trained staff to respond to inquiries and to use the blackout planning document. Because of its all-hazards approach, OES has numerous staff specifically assigned to local government and other customers in the state whose sole purpose is to provide technical expertise and assistance in all areas and phases of emergency management. As discussed with BSA, OES conducts regular meetings with its customers through its MARAC, SWEPC, and other forums to continuously receive feedback and input on emergency management operations as a whole from both local government and other state agencies. It is important to note that while there are a few issues that are unique to blackouts, the planning documents and guidance developed for this particular hazard are merely extensions of existing all-hazard plans (State Emergency Plan, SEMS, etc.). Therefore there is no need for staff 4 “designated to respond to inquiries” or “trained on how to use the planning document” because OES has these capabilities within its existing structure. BSA Recommendations In response to the BSA’s recommendations: (cid:127) OES has and will continue to review and update its business continuity plan and emergency procedures to address the state’s energy situation and other hazards. (cid:127) OES does feel that it adequately evaluates its notification process because it is in the 3 business of providing statewide notifications, 24 hours a day, for all types of disasters and emergencies. OES agrees that as part of this continuous evaluation, it is important to notify participants that necessary changes have been made when a shortfall is identified. (cid:127) OES believes that it currently has staff identified and trained within its existing organization 4 that are capable of addressing local government inquiries about the Electric Power Disruption Toolkit for Local Government, and advise local governments on their planning efforts. In addition, OES has other, regularly utilized forums to meet these needs as mentioned above. (cid:127) Finally, OES believes that the title of the BSA report: “Blackout Preparedness: The Office of Emergency Services and the California National Guard Each Have Weaknesses in Their 5 Blackout Preparations,” in reference to OES, is overstated and a misrepresentation of the findings by BSA. Notwithstanding our responses above, we do not concur that the findings of the BSA substantiate weaknesses in our system. OES strongly recommends that BSA change this title to be consistent with the report findings and more accurately reflect OES’ overall efforts. 22 COMMENTS California State Audtor’s Comments on the Reponse From the Office of Emergency Services T o provide clarity and perspective, we are commenting on the Governor’s Office of Emergency Services’ (OES) response to our audit report. The numbers below corre- spond to the numbers we placed in the margins of OES’ response. 1 We disagree with the OES’ statement that the weaknesses we describe in the OES’ blackout preparation are already addressed by its pre-existing, all-hazard emergency management practices. As we indicate on page 5, the OES prepared an Energy Shortage Response Matrix (response matrix) and for certain potential public safety impacts, the OES identified additional steps it should take to minimize disruptions to its operations. For example, it recognized the need to evaluate whether it could handle a natural disaster during an energy crisis. Because the OES identified these concerns itself, it seems clear that they were not already addressed by pre-existing practices as the OES is now claiming. 2 We disagree with the OES’ belief that its Business Continuity Plan (continuity plan) and Relocation and Transfer Plan adequately address a potential blackout situation. As we state on pages 5 and 6, the OES identified concerns with its continuity plan and Relocation and Transfer Plan. Moreover, since the OES did not provide us with any evidence, we question whether it has taken the necessary steps to resolve the concerns raised in its response matrix about its own preparedness. However, if the OES is follow- ing the procedures as it asserts, we acknowledge on page 5 that its headquarters generator can help to minimize the effects of a blackout. 3 We disagree with the OES’ statement that there is no need to specifically evaluate the effectiveness of its blackout notification process. In fact, in a meeting held on August 14, 2001, the deputy director of Emergency Operations, Planning, and Training Division agreed that a formal, periodic assessment of how the notification process is working would be beneficial to identify process improvements. Moreover, we also disagree with the OES’ 23 statement that it uses these same notification processes and tools for all other types of disasters and emergencies on a daily basis. The deputy director told us that the blackout notification process has improved upon its prior notification procedures. For example, the process allowed for expanded use of its Emergency Digital Information Service and the incorporation of its Response Information Management System. Therefore, we would expect OES to ensure that these new enhancements are effective. Finally, as we describe on pages 8 and 9, although the deputy director asserted that the OES has had ongoing communications with those involved in the notification process to refine and improve it, he did not provide sufficient information to substantiate these communications or their outcome. Thus, we could not concur that these communications are the OES’ attempts to assess the process. 4 We disagree with the OES’ belief that there is no need for it to designate and train staff to respond to local government inquir- ies. As we state on page 10, because OES did not designate and train staff to accept these inquiries, there is a potential that when the local governments contact the OES for assistance, they may get passed on to multiple staff and not receive the help they need at all. Moreover, because as the OES states there are issues that are unique to blackouts, despite their technical expertise in overall emergency management operations, staff may not be able to assist the local government in using OES’ Electric Power Disrup- tion Toolkit for Local Government (toolkit). 5 We disagree with the OES and believe that the title of our report is supported by relevant, sufficient evidence, and is consistent with our conclusions about the OES’ blackout preparation efforts. Further, we believe that we are presenting a complete and accurate picture of the OES’ blackout readiness based on the information the OES provided to us. For example, as we already discussed in the comments above, we are uncertain if or how the OES has resolved the issues it identified in its response matrix. Further, the OES could not demonstrate that it has a method in place to evaluate the notification process’ effectiveness. Finally, the OES has not designated or trained staff to respond to inquiries and may not be ready to assist local governments in using the toolkit. Consequently, these findings have led us to conclude that weak- nesses exist in the OES’ blackout preparations. 24 cc: Members of the Legislature Office of the Lieutenant Governor Milton Marks Commission on California State Government Organization and Economy Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press 25