Summary

Technology, Trade and Commerce Agency: Its Strategic Planning Is Fragmented and Incomplete, and Its International Division Needs to Better Coordinate With Other Entities, but Its Economic Development Division Customers Generally Are Satisfied December 2001 2001-115 rotiduA etatS ainrofilaC S T I D U A E T A T S F O U A E R U B The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TDD (916) 445-0255 x 216 OR This report may also be available on the World Wide Web http://www.bsa.ca.gov/bsa/ The California State Auditor is pleased to announce the availability of an online subscription service. For information on how to subscribe, please visit our Web site at www.bsa.ca.gov/bsa. If you need additional information, please contact David Madrigal at (916) 445-0255, ext. 201. Alternate format reports available upon request. Permission is granted to reproduce reports. C S A ALIFORNIA TATE UDITOR ELAINE M. HOWLE STEVEN M. HENDRICKSON STATE AUDITOR CHIEF DEPUTY STATE AUDITOR December 13, 2001 2001-115 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the Bureau of State Audits presents its audit report concerning strategic planning, coordination, and customer service efforts of the Technology, Trade and Commerce Agency (agency). This report concludes that the agency does not have an agency-wide strategic plan despite starting two agency-wide strategic planning processes in recent years. In addition, plans at the program level often lack elements of strategic planning such as goals for all significant aspects of program missions, targets for significant goals or targets that challenge performance, and a com- parison of results with targets in external reports. This report also finds that external coordination of export services provided by the agency’s International Trade and Investment Division is lim- ited, but recent activities indicate a renewed focus on this issue. Finally, programs in the agency’s Economic Development Division generally satisfy their customers but could benefit from formal pro- cesses to target and measure customer satisfaction. Respectfully submitted, ELAINE M. HOWLE State Auditor BUREAU OF STATE AUDITS 555 Capitol Mall, Suite 300, Sacramento, California 95814 Telephone: (916) 445-0255 Fax: (916) 327-0019 www.bsa.ca.gov/bsa CONTENTS Summary 1 Introduction 5 Chapter 1 The Agency’s Planning Efforts Continue to Lack Many Elements of a Strategic Approach 11 Recommendations 29 Chapter 2 The International Trade and Investment Division’s Coordination Efforts Are Uneven 31 Recommendations 41 Chapter 3 The Economic Development Division Generally Provides Good Customer Service, but It Could Benefit From Formal Processes to Measure Customer Satisfaction 43 Recommendations 49 Appendix Description of Technology, Trade and Commerce Agency Programs Reviewed in This Report 51 Response to the Audit Technology, Trade and Commerce Agency 55 California State Auditor’s Comments on the Response From the Technology, Trade and Commerce Agency 71 SUMMARY RESULTS IN BRIEF T he Technology, Trade and Commerce Agency (agency) is responsible for promoting economic development in California. The agency administers programs to support economic development, promote international trade and for- Audit Highlights . . . eign investment, and support technology use and development Our review of the Technology, in the State. These diverse programs range from assisting small Trade and Commerce Agency business development centers to promoting California exports (agency) found that: abroad. Although the agency reports success in many of its The agency has no agency- endeavors, it does not use agency-wide strategic planning to wide strategic plan, and enhance its effectiveness. After making some progress in estab- many program plans continue lishing an agency-wide strategic plan in 1997, the agency has to lack elements of strategic discontinued an agency-wide planning process and backed away planning including: from a results-oriented approach. Program-specific plans now (cid:1) Goals for all significant guide its actions, but all the plans we reviewed are weakened by aspects of program a lack of strategic planning elements. Many plans do not include missions. all goals significant to their mission; some do not set quantified (cid:1) Targets for significant targets for their goals; and some have targets so low they do not goals or targets that challenge performance. For example, the foreign office in Mexico, challenge performance. whose primary mission is to promote the State’s products overseas, (cid:1) A comparison of results to does not include outcome goals for the office as a whole but targets in external reports. instead an outcome goal—increasing exports—for two staff Further, external coordination members. Moreover, this plan lacks quantified targets for any of its of export services is limited for stated goals. In addition, the Infrastructure State Revolving Fund the agency’s International Program, a program that finances infrastructure throughout the Trade and Investment State, has a plan with no outcome goals. Further, the programs’ Division, but recent activities indicate a renewed focus on internal and external reports rarely compare targets that do exist this issue. with actual results. Such planning deficiencies reduce accountabil- ity within the agency and to stakeholders such as the Legislature. Finally, programs in the agency’s Economic Development Division The agency contends that the recent energy crisis, a dynamic generally satisfy their environment, and staff vacancies have hampered its strategic customers but lack formal planning. It says the governor’s administration, the budget pro- processes to measure customer satisfaction. cess, and legislative initiatives, as well as individual program plans, now direct the agency’s activities. Nevertheless, by de-emphasizing strategic planning, the agency is forgoing the benefits of taking a broad, outcome-based approach to focus the efforts of programs on overall goals and to evaluate the programs’ success in meeting goals. The agency, by relying instead on individual program plans that omit essential elements of strategic planning, lacks a 1 basis to integrate its diverse programs and allocate limited resources to fulfill its mission most effectively. Also, without adequate strategic planning, the agency lacks an effective way to demonstrate that it is wisely using the more than $200 million spent on its programs each year. On another front, the agency’s International Trade and Investment Division (International Division) has done an uneven job of coordinating with other entities working in the international arena. Without effective coordination, the agency cannot ensure that it has fully leveraged the State’s resources and addressed possible gaps and redundancies in the delivery of services. The International Division appears to have adequately coordinated its foreign investment services with other entities, but its coordi- nation for its export-related services has been limited. The International Division does not hold regular, broad-based coordi- nation meetings with other entities working in the international arena. It also has experienced problems coordinating with the California Department of Food and Agriculture and the California Energy Commission. The agency explains that, during the transition between state administrations, it focused on rebuilding its operations rather than on coordination. Despite indications that the International Division is trying to better coordinate with other entities, it has much to do. To improve coordination, the agency needs to meet with other entities, identify avenues of cooperation, and reach mutual agreement on roles and responsi- bilities. In addition, the delivery system for export-related services needs further study to determine the best way to leverage the State’s limited resources. In another area, programs in the agency’s Economic Develop- ment Division are generally responsive to customers’ needs despite a lack of processes to measure customer satisfaction. These programs’ relatively small circles of customers generally are satisfied with the services they receive, as shown by their responses to a customer survey that we conducted. Nevertheless, the Small Business Loan Guarantee Program needs to work with its customers to resolve concerns they expressed on our survey. Additionally, all programs could benefit from some formal processes to measure customer satisfaction, such as setting targets for customer satisfaction levels and using surveys to obtain feedback. 2 RECOMMENDATIONS To ensure that the diverse programs and activities administered by the agency demonstrate their worth and perform at their optimal level of efficiency and effectiveness, the agency needs to take the following actions: (cid:127) Develop a long-term, agency-wide strategic planning process that includes all goals that are significant to its mission, outcome-related goals, quantified targets for goals, and report- ing that compares actual results with targets. (cid:127) Report to the Legislature biennially on its progress in imple- menting a strategic approach to its planning. To ensure that the International Division adequately coordinates with other entities in the international arena, the agency should hold regular meetings with other entities to discuss goals and operations, analyze opportunities to reduce service gaps and redundancies within the service delivery system, and establish agreements that spell out its roles and interactions with other entities. In addition, the Legislature should consider commissioning an independent statewide study of the existing delivery system for export services to determine the best division of efforts among international trade entities. To ensure that the Economic Development Division is meeting the needs and expectations of customers, the agency should set customer service goals and targets and periodically survey customers to determine their level of satisfaction and areas that may need improvement. AGENCY COMMENTS Overall, the agency states that it will use the input from our report to enhance its planning process and its provision of services to the international trade and economic development communities. With respect to strategic planning, although the agency contends that the planning process it has adopted is the best means of ensuring that its programs are delivered in an effective and efficient manner, it indicates that it is planning to implement many of our specific recommendations. Additionally, the agency commented that it is committed to increasing the 3 level of coordination with federal, state, and local entities; however, it did not specifically address what it plans to do in response to some of our recommendations to improve coordina- tion. Finally, the agency indicates that it is planning to implement our recommendations regarding customer service in its Economic Development Division. (cid:2) 4 INTRODUCTION BACKGROUND T he Trade and Commerce Agency was created in 1992 to focus the State’s efforts on economic development and job creation in an increasingly competitive business environment. In January 2001, with technology playing a greater role in California’s economy, state law changed the agency’s name to the Technology, Trade and Commerce Agency (agency) and established its Division of Science, Technology, and Innovation. The agency’s vision is to serve as the State’s principal catalyst for innovation, investment, and economic opportunity, thus enhanc- ing the quality of life for all Californians. To accomplish its vision, the agency does the following: (cid:127) Promotes economic development, job creation, and business retention. (cid:127) Promotes international trade and foreign investment through its foreign trade, export, and investment functions. (cid:127) Helps businesses use and access technology, and supports the development and commercialization of technology. To accomplish its overall vision, the agency is responsible for providing leadership, oversight, coordination, and direct assis- tance through diverse programs that range from promoting California exports to reusing military bases. The Appendix describes each of the 15 programs we selected for review during our audit. The agency is composed of five operating divisions, shown in Table 1 on the following page, supported by administration, finance, policy, and planning groups. In fiscal year 2000–01, the agency spent about $204 million on its programs, with the State’s General Fund directly providing about 38 percent of the total funding. The California Infrastructure and Economic Development Bank Fund, which obtained its start-up money from the General Fund in 1999, provided an additional 47 percent 5 of the funding. (Borrowers of the California Infrastructure and Economic Development Bank will provide for this fund’s con- tinuing operations through loan repayments and fees.) Various other funds provided the remaining 15 percent of the funding. TABLE 1 Description of Five Operating Divisions and Expenditures for Fiscal Year 2000–01 Number of Division Name General Description Programs Expenditures California Infrastructure Provides loans for local infrastructure programs 2 $98.3 million and Economic and issues bonds, including industrial development Development Bank bonds, for manufacturing companies and bonds for nonprofit organizations. Economic Development Provides funding and technical support to local 15 55.5 million Division organizations that attract and retain businesses. Provides loans and technical support to businesses. Division of Science, Provides grants and other assistance to businesses 5 27.8 million Technology, and Innovation and local communities seeking to create or distribute new technologies. International Trade and Promotes exports and foreign investment through 3 11.1 million of Investment Division trade missions, trade shows, loan guarantees, and which 5.5 million (International Division) technical assistance. was spent on 12 foreign offices Division of Tourism Promotes California as a travel destination through 1 7.8 million advertisements and information. Note: In addition to the amounts spent for each of the five operating divisions, the agency spent $4 million for other purposes, such as for contract administration and economic research. PAST AUDITS HAVE RECOMMENDED IMPROVEMENTS IN PLANNING The Bureau of State Audits has conducted three audits at the agency that addressed planning and performance measurement. In 1995, we reviewed the Enterprise Zone program, which oversees and provides technical assistance to cities and counties that operate special incentive zones. We found that it needed to set performance measures and obtain complete and reliable performance data. In 1996, we conducted an agency-wide audit that found the agency lacked an overall plan and that various programs did not have targets that measured performance or comparisons between actual performance results and targets. In 1999, we audited the Manufacturing Technology Program, which provides funding to regional manufacturing technology centers that assist manufacturing companies. We found that this program did not set performance targets. All these audits recommended that programs set performance targets and 6 compare their results to their targets. In addition, the 1996 audit recommended that the agency implement an integrated approach that would include all the necessary elements of effective strategic planning. SCOPE AND METHODOLOGY The Joint Legislative Audit Committee (audit committee) requested the Bureau of State Audits to conduct a performance audit of the agency, focusing on its progress in implementing a strategic plan, mission, goals, and performance measures. The audit committee also requested that we evaluate the agency’s coordination activities with external entities involved in export promotion and foreign Thirteen Programs Reviewed for Strategic Planning investment. In addition, the audit committee requested that we examine the effect of state Economic Development Division policy guidance provided by the World Trade • Enterprise Zone Program Commission, how the agency gathers and uses (cid:127) Military Base Reuse Program data on state exports and foreign investment to (cid:127) Sacramento Regional Office measure the contribution of its programs, and the (cid:127) Small Business Development responsiveness of the agency’s Economic Devel- Center Program opment Division to its customers. (cid:127) Small Business Loan Guarantee Program California Infrastructure and Economic To gain an understanding of the agency’s Development Bank responsibilities and the environment in which it (cid:127) Infrastructure State Revolving operates, we reviewed the laws and rules relevant Fund Program to the agency in general and to the audit mandate International Division in particular. (cid:127) California Export Finance Office (cid:127) Japan Foreign Office To evaluate the effectiveness of the agency’s plan- ning efforts, we interviewed executive management (cid:127) Mexico Foreign Office and program staff and reviewed strategic plans, (cid:127) Office of Export Development missions, goals, objectives, and performance (cid:127) Office of Foreign Investment targets. Since there is not an agency-wide plan, but Division of Science, Technology, instead an array of plans for various programs and Innovation and offices, we selected and reviewed plans for (cid:127) Manufacturing Technology Program 13 programs or offices after analyzing budget Division of Tourism information, background materials, and the role (cid:127) Tourism Program of offices or programs within the agency. In this report, we refer to all 13 areas reviewed as programs. We interviewed the division deputies and program managers to obtain additional information about the missions, goals, objectives, and performance measures for the programs reviewed. (Objectives are lower-level goals or sub-goals that, if reached, lead to accomplishing higher goals. For the 7 purpose of our review, we evaluated goals and objectives similarly because both express desired accomplishments. We therefore refer to both goals and objectives as goals in this report.) We evaluated the programs’ plans using criteria we developed, which include having a long-term plan, a survey of the operating environment, a mission statement, significant goals, including outcome goals, and targets for goals. We based these criteria on guidelines developed by the federal government and others who have studied strategic planning, as well as guidelines the agency circulated to its programs in May 2000. In considering whether program plans included all significant goals, we concentrated on programmatic purposes such as the creation of jobs, rather than issues of service delivery or internal operations. In cases where a program accomplishes its mission through third parties, we evaluated the program’s contracts or agreements to determine if they included appropriate goals and targets. When the contracts did have goals and targets, we evaluated them in conjunction with those in the plans. To determine the effect of World Trade Commission guidance on state policy, we reviewed the commission’s activities since January 1999. We also interviewed the commission’s executive director and its chairperson. To determine how well the agency’s International Division coordinates with other trade entities, we used criteria developed by the federal General Accounting Office in its report on barriers to interagency coordination. We considered aspects of coordina- tion such as joint planning meetings, formal agreements, and data sharing to evaluate the same five International Division programs selected for our planning review. We interviewed the directors of International Division programs and reviewed documents related to coordination to determine how the division coordinates its planning and other activities. We also identified agencies and organizations with which the International Division coordinates, and then interviewed officials at 23 of these entities to obtain their view of the division’s coordination efforts. To determine the responsiveness of the Economic Development Division to its customers, we used criteria based upon guidelines developed by other states and the federal government. The criteria include identification of customers, collection of customer feedback, and establishment of customer service goals and measures. We evaluated the same five Economic Development Division programs selected for the planning review discussed 8 previously, plus two additional programs—the Main Street Program and the Small Business Assistance and Advocacy Program. We interviewed program staff to determine who their customers are and what the program does to ensure customer satisfaction. We also interviewed 5 to 16 customers for each program, based on a program’s number and type of customers, to determine the level of satisfaction with program services. In the case of the Small Business Assistance and Advocacy Program, we were unable to obtain enough customer responses to determine the level of customer satisfaction. (cid:2) 9 Blank page inserted for reproduction purposes only. 10 CHAPTER 1 The Agency’s Planning Efforts Continue to Lack Many Elements of a Strategic Approach CHAPTER SUMMARY A fter making progress toward an agency-wide strategic plan in 1997, the Technology, Trade and Commerce Agency (agency) has discontinued a fully-integrated planning process and placed less emphasis on a results-oriented approach. As in our 1996 audit findings, the agency’s activities are now guided by program-specific plans. Although some plans we reviewed are better than others, many program plans do not include all goals that are significant to their missions. Further, some plans do not include any outcome goals and thus do not focus on the benefits that programs are trying to achieve. Some plans do not set quantified targets for their goals, and some do not set targets that challenge performance. Moreover, the programs’ internal and external reports on program accomplishments rarely compare targets that do exist with actual results, reducing accountability within the agency and to stakeholders such as the Legislature. The agency states that the recent energy crisis, the dynamic external environment, and staff vacancies have hampered its strategic planning efforts. It says the governor’s administration, the budget process, and legislative initiatives, as well as individual program plans, now direct the agency’s activi- ties. However, by de-emphasizing strategic planning, the agency misses the benefits of a broad, outcome-oriented approach, which is vital to integrating diverse programs, allocating resources to efforts that best advance overall goals, and demonstrating the value of the agency’s activities. Some agency programs suffer from other weaknesses as well. For instance, some programs may misstate their true benefits either by not verifying the estimated data they receive from customers who may have an incentive to bias that information or by not systematically following up on estimated data their clients provide. In addition, vacancies have hurt the agency’s planning and operations in certain areas. Some foreign offices within the International Trade and Investment Division (International Division) had extended vacancies in positions where appointees 11 must be nominated by the agency, reducing planning and activities. Similarly, the World Trade Commission was inactive for more than a year while it lacked a chairperson and has provided limited guidance to the State on international trade policy since starting to meet again in 2000. THE AGENCY DISCONTINUED AGENCY-WIDE STRATEGIC PLANNING AFTER MAKING SOME PROGRESS After hiring two consultants and beginning two agency-wide strategic planning processes, the agency still does not have an agency-wide strategic plan. It has reverted to using individual program plans, which are often incomplete and vary widely because the agency has not set standards. The agency has instead The agency started issued optional guidelines that programs generally do not agency-wide strategic follow. By backing away from comprehensive strategic planning, planning processes twice the agency is forgoing an important process to help it focus since 1996 but still does programs on what is most important for the entire organization, not have an agency-wide determine overall success, and evaluate how much particular strategic plan. actions contribute to overall success. Considering the demands on its funds and staff, the agency needs these benefits of strategic planning, which could provide a basis for reallocating resources. In response to our 1996 audit recommending that the agency develop an integrated approach to planning and to the California Economic Strategy Panel’s recommendation that the agency recast its mission of economic development, the agency hired a consultant who helped it develop an agency-wide strategic plan. By February 1997, the agency had developed a plan that contained an agency-wide vision, mission, and goals. In the next phase, the agency planned to establish performance standards, targets, and outcomes for individual programs. However, not all programs completed this stage, and several that did dropped the outcome goals and targets they had established. Subsequently, the agency did not update the agency-wide plan, which eventually fell out of use. In June 2000, the agency hired another consultant to help it develop a new agency-wide vision statement and guiding prin- ciples, intended as the first step in a two-phase strategic planning effort. After finalizing a vision statement, however, the agency did not adopt guiding principles or begin the second phase of 12 planning. Explaining that the “energy challenge” of 2001 changed the agency’s focus and halted the planning process, the agency’s acting undersecretary said the agency is now unsure when it will resume this process. The acting undersecretary states that the agency continues to use the 1997 plan as guidance, in conjunction with the agency’s new direction and goals. He explains that the governor’s admin- istration, individual program plans, the budget process, and legislative initiatives now direct the agency’s activities. Unfortu- nately, these disparate processes do not provide the coherence of an agency-wide strategic plan. Moreover, despite a renewed emphasis on program plans rather than an agency-wide plan, the agency has not set standards for planning at the program level. In May 2000, the agency issued internal guidelines for evaluating programs. These guidelines are a good basis for a sound strategic planning process. They point out the importance of setting quantitative goals that identify the results that will be used to measure programs and of discussing the need to compare program results with targets to analyze performance. However, the agency did not require the individual programs to follow these guidelines, so program planning has remained inconsistent. STRATEGIC PLANNING IS ESSENTIAL FOR FOCUSING ON IMPORTANT GOALS AND INTEGRATING PROGRAMS To manage its programs effectively, an agency’s executives must ensure consistent, high-caliber planning efforts that yield useful information for decision making. This is particularly true with organizations, such as the agency, that have diverse and dissimilar programs. A unified strategic plan helps such organizations gain focus, integrate programs, and allocate resources to activities that most effectively advance overall goals. Within the past 10 years, the federal government and various states have started requiring their agencies to establish agency strategic planning. For example, the federal government required The federal government all its agencies to establish agency strategic plans covering at and various states require least 6 years by September 1997. Similarly, Texas began requiring agency strategic planning. its agencies to submit 5-year strategic plans in 1993, and in 1996, the State of Washington enacted requirements that its agencies submit 6-year strategic plans. Other states requiring agency strategic plans include Florida, Maine, and Arizona. California 13 also has recognized the importance of strategic planning. In 1994, the State began requiring the Department of Finance to survey agencies’ strategic planning annually and to recommend which agencies should develop or update strategic plans. In 1996, the Department of Finance recommended that all agencies have strategic plans and subsequently issued a directive requiring the same. Although it did not require plans to cover a specific minimum time frame, the Department of Finance said that plans addressing growth for a 5- to 10-year period offer programs a greater control over their destiny. Additionally, strategic planning processes generally include long- and short-term components. For example, federal agencies develop strategic plans that include mission statements and 6-year goals and use associated perfor- mance plans to set 1-year goals and targets that lead to achieving the long-term goals. Strategic planning is a long-term, future-oriented process of assessment, goal setting, and decision making that maps an explicit path between the present and a vision of the future. Rather than focusing exclusively on the efforts expended in activities, or on outputs, a strategic plan should stress the benefits, or outcomes, of the efforts. A sound strategic planning process includes these essential elements: (cid:127) Defining a mission. (cid:127) Analyzing the external environment to identify opportunities and threats that could affect the mission. (cid:127) Formulating goals consistent with the mission, including outcome goals, and establishing priorities among them. (cid:127) Establishing actions necessary to achieve goals. (cid:127) Defining quantified targets for goals, including targets for desired results, or outcomes. (cid:127) Measuring the results of operations. (cid:127) Comparing results to targets to evaluate and report performance. (cid:127) Explaining under-performance and the actions planned to meet goals. (cid:127) Revising the plan in light of performance and changing circumstances. 14 Strategic planning is fundamentally dynamic. It is not a one- time project that remains static after completion, but an iterative process that is refined as performance is measured and evaluated, targets are reset, and new information becomes available. A successful planning process offers many benefits to the agency, its clients, and its stakeholders, including its legislative body. For example, as an agency clarifies its purpose and direction, it is better able to focus on activities that enhance its mission. Strategic planning also improves an agency’s ability to anticipate and accommodate the future by identifying issues, opportunities, and problems. In addition, good planning enhances decision making by focusing attention on results, or outcomes. Finally, strategic planning yields information needed to guide resource allocation and to report success on an agency’s planned accomplishments. STRATEGIC PLANNING FOR INDIVIDUAL PROGRAMS IS INCOMPLETE AND INCONSISTENT Many of the plans for the 13 agency programs we reviewed lacked several elements of sound strategic planning, and all Many plans for the lacked at least one element. All plans generally included mission 13 programs we reviewed statements, goals that are aligned with the mission, and detailed lacked several elements of lists of tasks to accomplish each goal. In addition, all 13 programs sound strategic planning, recently updated their plans. However, as shown in Table 2 on and all lacked at least the following page, many plans did not include all goals that are one element. significant to the program; some did not set targets for all their significant goals; and some contained goals that do not chal- lenge performance. Further, no program compared targets with actual results when reporting to outside parties. These problems are similar to those reported in our 1996 audit, including the lack of targets for significant goals, the failure of targets to challenge performance, and the failure of programs to compare results with targets. Besides the elements described in Table 2, no plans cover five or more years, and many plans lack an analysis of their operating environment to provide better focus for their efforts. Many Plans Did Not Include All Significant Goals Of the 13 program plans we reviewed, 8 did not include goals for all significant aspects of their mission or vision statements or for outcomes included in external reports. In fact, some pro- grams included various outcomes in their vision or mission statements but did not establish related goals. When a program 15 TABLE 2 Programs Often Lack Important Planning Elements Quantified targets Targets that External report set for all challenge performance compares results Plan includes all significant goals for significant with targets for Programs significant goals in the plan goals in the plan significant goals Economic Development Division Military Base Reuse Yes No Yes* No Enterprise Zone† No No No No Small Business Development Center† No Yes No No‡ Small Business Loan Guarantee† No Yes No No Sacramento Regional Office Yes No Yes* No§ International Trade and Investment Division Export Finance Yes Yes Yes No Export Development No Yes Yes No Foreign Investment Yes Yes No No Japan Office No No No No Mexico Office No No No No Other Infrastructure State Revolving Fund No Yes Yes No Manufacturing Technology† Yes Yes Yes No‡ Tourism No No No No *The program had challenging targets for some goals but did not set targets for other significant goals in the plan. † To determine if a planning element was present, we considered both the program plan and the agreements that the program has with third parties to carry out its activities. ‡ The program provided documents showing that it compares results with targets internally. § State law does not require this office to produce an external report. does not set goals for major aspects of its mission, it implicitly reduces the importance of those aspects and ends up directing its attention to areas that may be of lesser importance. For example, the plan for the International Trade and Investment Office in Mexico (Mexico office) contains numerous activities the office intends to undertake but does not include outcome goals for the office as a whole. The plan only has an outcome goal—increasing exports—for two staff members. This contrasts with the office’s mission to promote exports and attract foreign investment and its performance reports, which tout those outcomes as well as job creation. The director of the Mexico office attributes the lack of comprehensive goals to staff vacan- cies that led to a slowdown in operations and a consequent 16 absence of sufficient performance data. However, the Mexico office has been in place for more than 12 years and has reported its successes in annual reports to the Legislature. Thus, the office has had enough time to define its major goals. Similarly, a primary part of the Office of Export Development’s (Export Development) mission is to help companies expand their export sales. It also reports on the number of jobs and the value of export sales it has helped create in its annual perfor- mance report to the Legislature. Its plan does not, however, have related goals. Instead, its only outcome goals relate to achieving a specified number of clients that report an export success. The director of Export Development said the program reports annually on the value of export sales and jobs created because that is the type of information that has been requested, but she said she does not believe these outcomes comprehensively reflect the program’s efforts and successes. She says other factors, such as the number of companies it assists and its ability to obtain federal grants, also determine Export Development’s success. We agree that a program’s efforts and outputs are necessary ingredients to its success and may warrant emphasis in reports to stakehold- ers. Nevertheless, programs need to set goals for significant aspects of their missions and for results that they present as their main outcomes. Of the eight program plans that did not include all significant goals, four focus exclusively on outputs or on the efforts the programs expend in carrying out activities, lacking any goals Four plans lacked any for the expected outcomes of their efforts. Outcome goals goals for expected communicate the results that a program is trying to accomplish, program outcomes, the such as creating employment or increasing exports. These four benefits programs hope programs, because they lack any outcome goals, are at a special to achieve. disadvantage in evaluating and demonstrating how successful they are in achieving their intended purpose. For example, the vision statement for the agency’s California Infrastructure and Economic Development Bank (bank) indicates that it will help the California economy by advancing statewide economic development, revitalization, and job creation efforts. The bank’s revolving fund program, which finances infrastructure projects, also places considerable importance on a project’s potential for job creation when evaluating applications for funding. Despite the obvious priority given to job creation, however, none of the goals for the revolving fund program addresses job creation or any other outcome. The program’s assistant executive director stated that the program’s goal to approve $200 million in loans for infrastructure projects addresses its vision statement, 17 including creating employment. However, since projects of similar cost may vary widely in the number of jobs they create, we believe job creation should be a separate outcome goal with a corresponding target. A job creation goal would highlight the importance of selecting projects that best align with the bank’s vision of advancing statewide job creation efforts. In contrast, the California Export Finance Office (Export Finance), which guarantees loans for California exporters, has separate goals for increasing exports and for creating jobs in its program plan. Some Programs Did Not Set Targets for Significant Goals Although programs we reviewed typically developed at least some significant goals for themselves, 6 of the 13 programs did Six of 13 plans did not not set targets to effectively assess progress in meeting those goals. include targets to effectively When programs do not establish quantified targets for their assess performance. significant goals, they lack a basis to evaluate effectively whether they have performed well, and they do not take advantage of the motivational benefit of strategic planning. For example, one goal of the International Trade and Investment Office in Japan (Japan office) was to promote new Japanese investment in California. Although the Japan office recently decided to measure its success based on the average number of actual investment dollars landed per investment inquiry, it did not set any target for this performance indicator. Consequently, the Japan office cannot readily measure whether it has achieved its goal of promoting new investment in California and cannot use that measure to motivate employees. The acting director of the Japan office said he could not set a target for this measure in fiscal year 2001–02 because the measure is new, unlike those used in previous years, and because Asian economies are undergoing fundamental shifts. The Japan office, however, reported on the number of investment leads it received and the amount of foreign investment it supported in the International Division’s performance report to the Legislature in fiscal year 1999–2000, the latest report prepared at the time of our review. Thus, it should have had data available to calculate the average number of investment dollars per investment in- quiry and thus set a target in fiscal year 2001–02. Moreover, as discussed later, it is possible to set targets using expectations about economic variables. The acting director said the Japan office plans to set targets for subsequent years, using results it is compiling now. 18 Although some programs we reviewed acknowledge the need to set targets for outcome goals, others are reluctant to do so. For instance, the Economic Development Division’s Enterprise Zone Program, which oversees the cities and counties that manage economic development areas, does not believe it is feasible to include targets for outcome goals in its agreements with the local governments. The manager for the Enterprise Zone Program stated that the environment in which the program operates is dynamic, with external economic factors significantly affecting local governments’ ability to influence economic development within their business-incentive areas. In addition, the assistant secretary of the Economic Development Division stated that she does not believe the program has the statutory authority to require local governments to set performance measures for outcomes such as job creation. Although external factors can affect a program’s success, it is important to set targets and track progress for outcome goals because this process reveals whether a program is truly successful. It is important to set Even when other variables affect success, a program can set targets for outcomes even outcome goals and targets based on an analysis of expected when external factors economic variables. It then can analyze how the performance affect success. differed from expectations in the context of how the variables differed from expectations. The feasibility of setting outcome targets is shown by the fact that local governments, currently applying for time extensions for their economic development areas, are now drafting such targets. For instance, a city and county have proposed goals to create an average of 50 jobs per year, to assist in decreasing the unemployment rate by 1 percent, and to increase non-farm related jobs by 3 percent over the next five years within an economic development area they co-manage. This indicates that the economic development areas can establish outcome goals and targets despite external factors that could inhibit their ability to meet them. In addition, a review of state laws governing economic development areas reveals that the law does not restrict the program from setting performance measures. In fact, the law gives the program broad powers to establish and oversee these areas. We noted a similar issue in our 1995 audit report and recommended that the Enterprise Zone Program establish performance measures and determine how to evaluate reported achievements against those performance measures. 19 In contrast, the Manufacturing Technology Program established in its contracts targets for its significant goals. Two of its goals were to provide manufacturing assistance to small manufactur- ers and to create jobs. The program set related targets for its contractors to serve 2,100 small manufacturers and create 1,800 jobs in fiscal year 2000–01. By establishing these targets, the Manufacturing Technology Program can know and report on how well it meets its expected performance levels. Some Plans Included Targets That Do Not Challenge Performance By setting targets for performance far below what programs can do, some plans did not challenge performance. For example, in Some plans set targets 2000 the Small Business Development Center Program set an that were significantly output target for one of the small business development centers lower than actual with which it contracts to provide 47 training sessions. The center performance for the was able to provide 194 sessions during the year, exceeding the previous year. target by more than 300 percent. In another instance, the program set an output target of 2,468 hours for counseling for another small business development center, which then accumulated about 3,710 hours of counseling, exceeding the target by more than 50 percent. We looked at performance for the prior year to see if it would explain the low level of targets and were surprised to find that performance in 1999 also significantly exceeded the targets for 2000. In fact, 1999 outputs exceeded 2000 targets by 53 percent and 29 percent, respectively, for each center. The program director stated that targets are set according to formulas mutually agreed upon by the Small Business Adminis- tration, the Chancellor’s Office of the California Community Colleges, and the small business development centers. These formulas incorporate the size of a small business development center’s budget and a minimum statewide threshold for spending on particular tasks, such as counseling. In addition, he said that in negotiating targets for each center consideration is given to elements such as local needs, past performance, and the capacity of centers to accomplish targets. He also said that if it appears that a center is continually exceeding its goals, the agency, in consultation with its funding partners, would make a determina- tion to amend the targets. The program demonstrated that it used the minimum threshold formula to compute targets. It could not, however, provide any analysis that it considered the other elements it discusses to develop challenging targets. The Small Business Development Center Program loses the motivational 20 effect of targets when it sets them far below what centers can achieve. We noted this concern in our 1996 audit report when we found that some programs, including the Small Business Development Center Program, reported results that significantly exceeded their targets. Plans also fail to challenge performance by omitting targets altogether. As mentioned earlier, many plans we reviewed did not include quantified targets for their significant goals. A plan without targets does not challenge performance because staff do not know the level of success they are expected to reach or how this compares with their past achievements. The Mexico office, for instance, did not set targets for any significant goals in its strategic plan. Instead, it outlined activities, such as trade shows, that it would participate in or attend during the year. Although taking part in trade shows is an important activity of the Mexico office, scheduling events does little to motivate staff or indicate higher levels of performance. Consequently, the plan does not challenge performance. Programs Did Not Report Results Compared With Targets, Lessening Accountability When reporting results to outside parties, none of the 13 reviewed programs gave the targets they had set for their goals. Also, most None of the 13 programs programs did not include comparisons of results and targets in we reviewed compared internal reports. When programs do not publish targets along results to targets in with results, report readers cannot assess whether programs have external reports. met their goals and cannot hold program managers accountable for program performance. When programs do not even compare results with targets internally, they do not have the information they need to see if changes to their operational strategies are warranted. Goals and targets also fail to motivate the staff when managers do not present staff with information showing program performance compared with expectations. For example, during fiscal year 1998–99, Export Finance set targets to approve $24.6 million in loan guarantees to California exporters and to create or maintain 2,006 jobs in California as a direct result of its services. In its annual performance report to the Legislature, Export Finance said it approved nearly $9.1 million in loan guarantees and helped create or maintain 551 jobs—but did not also report its targets. Accordingly, the Legislature had no way of knowing the program significantly missed its targets. When asked about this reporting problem, programs provided 21 various reasons including the fact that the Legislature does not require them to report targets as well as results. Meaningful performance reporting, however, requires that results be provided in a context that allows for their evaluation. Performance targets provide such a context. Also, most programs could not provide evidence that they compare results with targets in internal performance reports. Some programs stated that they informally compare targets with actual results. For example, the Small Business Development Center Program makes a side-by-side comparison of targets with results in its performance reports. This allows program management to determine whether small business development centers are meeting the targets for various goals and strengthens accountabil- ity. However, only 2 of the 13 programs we reviewed could provide us with documents showing a comparison of results with targets. The absence of such documentation raises questions as to whether the programs are performing meaningful analyses of their performance by comparing results with targets. No Programs’ Plans Were Long Term, and Many Lacked Environmental Assessments No reviewed programs developed plans covering five or more years, and some did not consider opportunities and threats from their external environment in establishing their plans. When the planning process omits a long-term view and an assessment of threats and opportunities, such as economic conditions or technological advances that could impact program goals, it diminishes the program’s ability to take a broad view of its functions and to position itself for maximum effectiveness. None of the 13 programs we reviewed had established goals or objectives that focused on their expectations over five or more No plans covered five or years. In fact, only 4 plans exceeded one year. Program managers more years, and only four provided various reasons for lacking long-term goals, including plans covered more than that agency management did not direct them to set long-term one year. goals. In addition, some program managers stated that the dynamic nature of their operating environment precluded long- term goals—yet a dynamic environment is even more reason to plan and understand the environment. In addition, 8 of the 13 programs did not assess external opportu- nities and threats as part of their planning process. Some programs claimed to have analyzed the threats and opportunities but could not provide any evidence to support their assertions. For 22 example, the Division of Tourism’s chief of operations stated that various external conditions, such as the strength of the dollar, make it hard to identify opportunities and threats in advance. She further said the division evaluates its threats and opportunities when it implements each phase of its marketing plan and makes adjustments accordingly. She could not, how- ever, provide any evidence of such evaluations. We agree that changing circumstances warrant a reevaluation of threats and opportunities, but when a program does not document such evaluations, it cannot show that it thoroughly considered the relevant environmental factors and positioned itself to be as effective as possible. CONCERNS EXIST REGARDING THE RELIABILITY OF SOME REPORTED BENEFITS AND OUTCOMES Generally, the agency’s programs do not verify data that may be considered inherently unreliable, such as data from clients who may have an incentive to exaggerate results. When programs base the results in their performance reports on such unverified data, they risk misstating the true benefits of their programs. Some programs rely solely Although we reported a lack of data verification in our on estimates from clients 1996 report, unreliable data continue to be a concern. who may have an incentive to exaggerate The agency reports a variety of benefits and outcomes for its results. various programs each year, typically using two sources to generate the information it reports: its clients and internal data. Some programs rely solely on estimates of clients who may have an incentive to distort data. The Small Business Loan Guarantee Program, for instance, relies on estimates provided by borrowers during the loan application process. These estimates show the jobs that borrowers expect to create or retain through guaranteed loans. These clients may perceive an incentive to overestimate the number of jobs created or retained in hopes of securing a loan guarantee. Currently, the program bases its performance reports on these estimates and does nothing to verify the data. We noted this problem in our 1996 audit and recommended the agency consider verifying some of the inherently less reliable, client-supplied information on a sample basis. In our report, we described how the Small Business Loan Guarantee Program could use Employment Development Department (EDD) data on a sample basis to confirm independently the number of jobs created. According to the manager who oversees the program, the agency received approval for $11,000 in additional funding 23 in fiscal year 1998–99 to obtain EDD data to verify job estimates for several programs, including the Small Business Loan Guaran- tee Program and Export Finance. However, according to the deputy director for the Office of Small Business, the agency later learned that EDD would not be willing to provide the required data because of confidentiality concerns, so the agency did not pursue the matter further. However, a misunderstanding appears to have occurred. During our current audit, we spoke with the The Employment chief of the Labor Market Information Division at EDD, who Development Department stated that EDD would be able to provide the needed information can provide inexpensive to the agency and that confidentiality would not preclude data information useful for sharing. The EDD estimated that information on 500 companies, verifying data. provided twice a year, would cost $3,800 to $4,500 annually, depending on the type of data. Also, if the agency followed our recommendation to verify data on a sample basis, the number of companies verified could be considerably lower, decreasing the costs even further. When a program has access to reliable and relatively inexpensive data that it can use to measure and report its performance better, it should take advantage of the opportunity. Some Data Used by International Division Programs May Be Inaccurate or Incomplete Other concerns about the reliability of data surfaced when we reviewed specific data gathered by the International Division. The Joint Legislative Audit Committee asked us to determine how the agency gathers and uses data on state exports and foreign investment, including how it uses such data to measure the contribution of state programs to enhance exports and foreign investment in the State. Various programs and offices within the agency’s International Division manage these programs. Table 3 illustrates the specific types of data the International Division gathers on state exports and foreign investment, as well as the sources and uses of such data. In addition to the data shown in Table 3, the International Division collects and reports data on its efforts. For example, Export Finance tracks the dollar value of the loans that it guar- antees; Export Development accumulates the number of trade inquiries it services; and foreign offices record the number of trade shows and seminars in which they participate. Previously, in this chapter, we discussed the extent to which International Division programs, as well as others, use data to measure their contributions as part of a strategic planning and performance measurement process. Table 2 on page 16 summarizes the results of our review. 24 TABLE 3 Sources and Uses of Data Generated by the International Division Data Used Data Used To Source of Data as To Set Report Program Program Data Gathered Reported by the Agency Targets Success* Amount of export sales Purchase orders submitted Yes† Yes by clients Export Finance Jobs created Written confirmations from clients Yes† Yes Amount of foreign Client-provided documents and Yes† Yes investment telephone calls to clients Foreign Investment Jobs created Client-provided documents and Yes† Yes telephone calls to clients Amount of export sales Post-event surveys and periodic No Yes telephone calls to clients Export Jobs created Periodic telephone calls to clients No Yes Development Number of companies with Post-event surveys and periodic Yes Yes successful export sales telephone calls to clients Amount of foreign investment Client-provided documents and No Yes telephone calls to clients Foreign Amount of export sales Post-event surveys and periodic No Yes Offices telephone calls to clients Jobs created Post-event surveys and periodic No Yes telephone calls to clients * Each of the programs report their past year’s results in the International Division’s annual report to the Legislature. However, none of the programs that sets targets compare these results to the targets in the annual report or in any other documented analysis. † Each of the programs annually sets targets for the listed data elements. However, although the programs claim that they use the previous year’s data to set targets for the subsequent year, they could not demonstrate how they did so. Certain data for Export Finance shown in Table 3 has the same problem as that identified for the Small Business Loan Guaran- tee Program. Namely, borrowers who receive loan guarantees from Export Finance to complete an export sale may have an incentive to overstate the number of jobs created because the loan guarantee decision depends in part on expectations about these outcomes. In such situations, it is prudent to verify client-reported data on at least a sample basis. Nevertheless, Export Finance relies on written confirmations from the borrowers on the jobs they have created or maintained. It does not take steps to con- firm this data independently. As noted previously, the agency recognized this need when it sought funding to obtain data from EDD to verify jobs created for Export Finance, among 25 others. The risk of unreliability does not extend to Export Finance’s export sales data, which are based on purchase orders, a more reliable source of data. Other client-reported data on state exports and foreign investment may be inaccurate or incomplete. For example, Foreign Investment obtains estimates from its clients on the number of jobs they expect to create and the amount of money they plan to invest in California. According to the deputy director for Foreign Invest- ment, the program works closely with its clients and updates the estimates if it learns that clients have changed their plans. The program does not, however, have a process for systematically rechecking these figures at the completion of projects, when actual figures should be available. Consequently, the program’s reported results may contain inaccurate data. In a different situation, Export Development uses post-event surveys to collect data from its clients on the jobs they create and the amount of their export sales related to program events. In compiling information for its annual performance report, it Data reported by Export contacts these clients again to confirm and update this data. Development in its annual However, the director of Export Development says that, because performance report may there is little ongoing contact, clients are not always forthcoming be incomplete because of with information regarding their successes when contacted limited ongoing contact months after an event. In such cases, reported data may be with clients. incomplete. The director of Export Development says she has taken steps to improve ongoing contact with the clients, plan- ning to have staff contact them quarterly rather than annually. Also, she says she has required all staff to maintain files on these clients to keep track of contact information. Such positive steps, if followed, may help ensure that clients are more forthcoming with their successes and that reported outcomes are complete. Typically, foreign offices work with Export Development and Foreign Investment to increase foreign investment, increase export sales, and to create and maintain jobs. In some cases, their efforts are primary to reported successes; at other times, they play an assisting role. Nevertheless, they work on many of the same activities as Export Development and Foreign Investment and collect the same type of data. Foreign offices thus face similar data reliability issues as those mentioned for Foreign Investment and Export Development. 26 VACANCIES IN THE INTERNATIONAL DIVISION WEAKENED PLANNING AND OPERATIONS AT THE FOREIGN OFFICES AND WORLD TRADE COMMISSION Lengthy vacancies for appointed positions at some of the Inter- national Division units weakened planning and operations. Vacancies at foreign offices, which provide overseas support for the division’s export and foreign investment activities, resulted in a lack of plans and focus during two recent years. For instance, almost half the positions at the Mexico office were vacant for about a year or more, causing the office to function at a minimal level. In addition, the World Trade Commission, which advises the State on trade policy, did not even meet for more than a year because it lacked a chairperson. It has provided limited guidance to the State since starting to meet again in 2000. By state law, all positions at the International Division’s foreign offices, from directors to office assistants, are nominated by the agency’s secretary and appointed by the governor. The agency’s acting undersecretary indicated that these positions must be appointed, and thus outside the civil service system because the application of civil service requirements would be impractical to implement in foreign countries. He explained that these requirements would, for example, restrict the candidate pool for foreign offices to persons qualified under federal immigration laws to work for the State. This would preclude the agency from employing most foreign nationals that apply for jobs in foreign offices. The director of Foreign Office Operations also explained that the agency put off nominating staff for some of the foreign offices until after the appointment of directors at those offices, a process that took as long as 13 months. Consequently, some positions remained vacant for more than a year at certain foreign offices after the administration changed in January 1999. Moreover, our review of appointments made to all foreign offices since January 1999 showed that, on average, the positions were vacant 10.5 months with the agency taking nearly 9 months, on average, to submit nominations. Because of vacancies, the two offices that we reviewed—Mexico and Japan—lacked plans and focus for two recent years. For Vacancies caused foreign example, since the start of the latest administration in offices in Mexico and January 1999, the director position at the Mexico office was Japan to lack plans and vacant for about a year, and 4 of the 10 staff positions were focus in two recent years. vacant from 11 to 18 months. The director of Foreign Operations attributed the Mexico office’s lack of plans during fiscal years 1999–2000 and 2000–01 to this situation. In addition, the 27 director of the Mexico office stated that the vacancies left this office in such disarray and slowed its operations so drastically that it did not have the necessary data to set goals in fiscal year 2001–02. He also pointed out that, before his arrival, the office did not have a policy manual or a filing system to keep track of various activities. The director said he spent the first nine months of his appointment organizing the office, training its staff, establishing direction, and setting up a tracking and filing system. Although we understand the desire to have directors involved in staff nominations, we question whether it is prudent to allow numerous vacancies at foreign offices for extended periods. Foreign offices benefit the State because they can readily interact with potential foreign investors and buyers. When there are insufficient staff to carry out office functions, they may not be as effective in helping California businesses to penetrate foreign markets or in attracting foreign investors to California. Also, a foreign office’s relationship with both California businesses and foreign partners could suffer from a lack of continuity. Thus, it is important that the agency ensure positions are filled promptly, even in the absence of a director, to prevent disruption of essential operations. Similarly, the World Trade Commission (commission) lacked a chairperson so it met infrequently and provided little policy advice to the State during the last three years. In this case, there The World Trade was no appointed chairperson to fill the vacancy left by the Commission has provided departure of the former chairperson. Unlike the foreign offices, little policy advice in the however, the agency does not have the statutory responsibility last three years. to nominate commissioners. The governor, the speaker of the Assembly, and the Senate president pro tempore appoint the commission’s 15 members. A structure of overlapping terms theoretically ensures a quorum of appointees and minimizes the impact of changes in administration. However, the term of the former chairperson expired at the end of 1998, and a new chairperson was not appointed until January 2000. The commission’s executive director, who is also the deputy secretary of the International Division, said that, because there was not a chairperson, there was no one to call a meeting during this time. The commission’s primary purpose is to advise the State on policies such as the North American Free Trade Agreement that could affect California’s ability to trade internationally. However, lacking a chairperson, the commission did not meet between October 1998 and March 2000 and did not provide any policy 28 direction during that time. Subsequently, the commission has provided limited policy direction. The executive director says the commission has helped the agency identify opportunities that may generate jobs and incomes for California and has supported the agency in advocating for U.S. free trade agreements with Chile and Singapore. It is now considering initiating a study on the Free Trade Area of the Americas, the first study it will have initiated since 1998. RECOMMENDATIONS To manage its programs more effectively, the agency should take the following actions: (cid:127) Develop an agency-wide strategic plan covering at least five years to better integrate program efforts and to highlight current state priorities. The agency also should ensure that short-term plans for programs are aligned with the agency- wide strategic plan. (cid:127) Include in its strategic planning process the elements this report identifies as missing in many of the agency’s program plans. These elements include goals and targets for all signifi- cant aspects of its vision and mission and for significant accomplishments noted in external reports, outcome goals that focus efforts on results where they matter most, targets that are challenging in light of prior performance and expected economic assumptions, and outcome goals and related targets in agreements with third parties who deliver program services. (cid:127) Compare targets with results in internal and external reports to evaluate success and use this information to revise goals and targets as appropriate. (cid:127) Periodically scan the environment to identify opportunities and threats that could significantly affect goals and include such assessments in its plans. (cid:127) Give high priority to nominating persons to appointed man- agement positions in the International Division, and nominate persons to appointed staff positions that are necessary for program continuity even if managers are not yet appointed. 29 (cid:127) Ensure the data it reports as the results of its programs are as accurate as possible, including performing follow-up on client estimates as needed. The agency also should verify some of the inherently less reliable, client-supplied information on a sample basis. Additionally, the agency should report to the Legislature bienni- ally on its progress in implementing a strategic approach to its planning, including specific recommendations in this chapter. Finally, the World Trade Commission should consider imple- menting procedures so it can continue to advise the agency even if a chairperson is not appointed. (cid:2) 30 CHAPTER 2 The International Trade and Investment Division’s Coordination Efforts Are Uneven CHAPTER SUMMARY T he International Trade and Investment Division (Interna- tional Division) within the Technology, Trade and Commerce Agency (agency) has done an uneven job of coordinating its services with other entities working in the international arena. Without effective coordination, the agency cannot ensure that it has fully leveraged the State’s resources and addressed gaps and redundancies in the delivery of services. Among other things, the International Division promotes foreign investment in California, finds foreign buyers and matches them with exporters, counsels businesses on exporting, organizes trade missions and shows, and guarantees export-related loans. Although it appears to be coordinating its foreign investment services adequately, the International Division’s efforts are more limited in coordinating its export-related services. For example, its Office of Foreign Investment (Foreign Investment) coordinates with local economic development organizations to find locations in California for foreign investors. However, the International Division’s Office of Export Development (Export Development), rather than coordinating with other entities, generally uses its own resources to match potential foreign buyers with California exporters, sending foreign buyer leads to other entities only if it cannot find an appropriate exporter match. In addition, the International Division does not hold regular, broad-based coordination meetings with other entities working in the international arena. Further, the International Division has experienced problems coordinating with the California Department of Food and Agriculture and the California Energy Commission. The agency explains that during the transition between administrations it focused its attention on rebuilding its operations rather than on coordination. Although we have seen indications that the International Division is trying to coordinate with other entities, it still has much to do. To improve coordination, the agency needs to meet with other entities, identify avenues of cooperation, and reach agreement on roles 31 and responsibilities. In addition, the service delivery structure for export-related services needs further study to determine the best way to leverage the State’s limited resources. MANY OTHER ENTITIES OFFER THE INTERNATIONAL DIVISION OPPORTUNITIES Primary Entities That Coordinate With the International Division TO IMPROVE EFFICIENCY AND EFFECTIVENESS THROUGH COORDINATION Federal Export Assistance Centers offer export To California exporters, the International Division marketing and trade finance support to offers trade shows, trade leads, and loan guarantees, small- and medium-sized businesses through a partnership among five federal entities. among other services. To potential foreign inves- tors, the International Division offers information The Small Business Administration offers services to small business exporters, on location sites and conducting business in including helping them obtain loans by California. Many governmental and nonprofit providing lenders with guarantees of repayment. entities offer similar services that complement and sometimes overlap those of the International The Export-Import Bank of the U.S. offers services to exporters, including credit Division. Because these entities have developed insurance and loan guarantees. their own members and clients, coordination State offers the International Division the chance to cross-promote its services to targeted customers The California Department of Food and Agriculture promotes agricultural efficiently and effectively. These entities range exports through various activities, including from industry-specific agencies at the federal trade shows and missions. level, such as the U.S. Department of Agriculture, The California Energy Commission to trade organizations at the local level, such as promotes international sales of energy technology products. the world trade centers located in major cities. Despite working sporadically with many different Local coordinating entities such as trade associations Centers for International Trade Development, located at community and chambers of commerce, the International colleges, offer training and technical aid to Division deals mainly with entities that provide a small- and medium-sized businesses broad range of international services or that focus considering or engaged in international trade. on major California industries. The text box at left World Trade Centers located in major cities provides brief descriptions of the principal entities offer various services, including trade with which the International Division coordinates. counseling and leads to export-ready companies. The range of services each entity offers allows for Economic Development Organizations are nonprofits or governmental entities that integration and consolidation of services where promote business investment and job entities can agree to specialize and to refer clients creation in their communities. based on this specialization. Currently, for example, the agency’s foreign offices use their overseas presence to collect trade leads, match California exporters with foreign buyers, and organize foreign investment and buying missions to California. Local organizations may be able to integrate these efforts with their own services to exporters. At the same time, Export 32 Development and various local organizations offer businesses counseling on exporting. Thus, opportunities exist to take advantage of each other’s work and to gain efficiencies through coordination and specialization. Recognizing the limited resources of state agencies and the benefits of coordinating related activities, the Legislature and governor enacted state law that gave the agency’s International Division primary responsibility for coordinating state activities The International Division that foster international trade. By state law, the International has primary responsibility Division is responsible for coordinating the various export for coordinating state development, finance, research, policy, and promotion programs activities that foster in state government. State law also requires Export Development international trade. to coordinate its trade promotional activities with the California Department of Food and Agriculture and the California Energy Commission. In 1999, state law instructed the agency to develop, if feasible, a statewide alliance of public-private trade develop- ment organizations that would provide “one-stop” delivery of export services. However, this effort was contingent on state funding, which the Legislature has never approved. COORDINATION EFFORTS FOR FOREIGN INVESTMENT APPEAR ADEQUATE The International Division, drawing on local entities, appears to have adequately coordinated services to promote foreign investment in California. The foreign investment process involves gaining leads on foreign investors and matching them to California locations that meet the foreign investors’ specific needs. By involving local economic development organizations in proposing locations, the International Division’s Foreign Investment is leveraging its resources and giving equal treatment to all areas of the State. For example, over several years Foreign Investment—with help from two economic development organizations and several cities—provided site selection information to a German company that eventually set up a wire rope manufacturing plant in Northern California. This investment was 1 of 19 new foreign investments the International Division reported in fiscal year 1999–2000 (the most recent year for which results were reported at the time of our review). It provided 25 of the 714 jobs initially created by Foreign Investment’s clients in that year. 33 The International Division’s 12 foreign offices generally find potential foreign investors, while Foreign Investment matches those possible investors with suitable California locations. The foreign offices have less opportunity to coordinate with other entities because of their distant location and their unique task of finding foreign investors. Foreign Investment, however, Foreign Investment coordinates extensively with other trade entities at two vital coordinates with local points in the matchmaking process. First, it uses a formal Request economic development for Proposal process to enlist local economic development organizations to meet the organizations in locating sites that meet investor needs. Foreign needs of foreign investors. Investment sends an investor’s specifications to local economic development organizations, asking them to propose specific sites in their communities that meet the investor’s requirements. The economic development organizations are uniquely suited for this task because they know their communities well. Second, after foreign office staff present the proposals to the foreign investor and Foreign Investment works with the investor to make a selection, Foreign Investment brings the associated economic development organization back in to complete the deal. This process benefits all parties—foreign investors get better service; California communities get an equal chance to compete for foreign investment; and Foreign Investment gets to leverage its resources. COORDINATION OF EXPORT SERVICES HAS BEEN LIMITED BUT SHOWS RECENT IMPROVEMENT The International Division’s coordination efforts related to its export services have been limited, with coordination for export financing stronger than that for export development. The International Division works to increase California exports primarily by counseling potential California exporters, bringing exporters and foreign buyers together through trade shows and missions, and guaranteeing export loans. Export Development provides counseling, while Export Development and the foreign offices coordinate on trade shows and missions, and the Interna- tional Division’s California Export Finance Office (Export Finance) guarantees loans. Export Finance does a good job of coordinating with federal agencies and partnering with financial institutions. However, Export Development has had limited success in its efforts to coordinate with other entities. The International Division’s strongest coordination processes for its export services relate to its financing operation. Export Finance coordinates export activities through formal agreements 34 with the federal Small Business Administration and the federal Export-Import Bank. These agreements allow Export Finance, in conjunction with the federal agencies, to guarantee loans that would otherwise exceed its statutory lending threshold of $750,000. Both of these federal agencies say they are satisfied with the level of coordination with Export Finance. Export Finance also has assembled more than 100 financial institutions that participate in its loan guarantee program, helping borrowers who do not meet conventional loan requirements obtain financ- ing through the loan guarantees from Export Finance. These banks provide Export Finance with about 50 percent of its customers. In contrast to Export Finance’s coordination with federal agencies and its partnership with financial institutions, the International Division’s efforts to coordinate export services are more limited. These efforts are informal and consist mainly of promoting the International Division’s export services, rather than coordinating any services. For instance, Export Development says it distributes its yearly trade event calendar to other trade entities with the goal of having them jointly promote Export Development’s events and select events in which to participate. Export Development also says it has persuaded a handful of other entities to agree informally to promote its events through their newsletters and Web sites. In turn, Export Development promotes the work of other entities by financing informational materials such as reference guides of international trade organizations. The International Division Has Had Limited Success in Coordinating Export Services Although promoting its export services through other trade entities is worthwhile and probably helps reduce its marketing costs, the International Division has had limited success in Most of the officials of integrating its activities with those of other entities. In fact, 23 entities we interviewed most of the officials of 23 coordinating entities we interviewed, indicated that little ranging from local centers for international trade development to coordination exists with federal export assistance centers, indicated that little coordination the International Division. exists. The International Division tends to share what it already has decided to do, rather than work with others to decide what should be done and by whom. As the director of one world trade center put it, the International Division usually presents its events as a fait accompli, requesting mailing lists for advertising but rarely requesting input beforehand. The International Division’s deputy secretary contends that the division jointly develops projects with other entities in some cases, but said that it is very difficult to coordinate planning with the myriad of 35 other trade organizations. In addition, he believes that some other trade entities have a financial incentive to independently develop trade promotion activities that may compete with those of the International Division. Given this situation, he believes someone needs to take the lead, and the International Division is the logical choice. By acting on its own rather than coordinating efforts, however, the International Division may be missing out on opportunities to share expertise, deliver services more effi- ciently, and gain synergism from combined forces. The International Division’s unilateral approach is evident in the lack of other coordinating activities. For instance, neither Export Development nor Export Finance brings other entities Neither Export into their overall planning processes. Moreover, the International Development nor Export Division does not hold regular, broad-based coordination meet- Finance brings other ings with other entities. Three of the coordinating entities we entities into their overall interviewed said they attended regularly scheduled coordination planning processes. meetings during the previous state administration. They said the meetings, which reportedly included the agency and federal, state, and local entities, offered an ongoing forum for discussing export-related goals and operations. These entities could not provide records of the meetings; however, we believe meetings such as these could be useful. Additionally, the International Division has experienced problems coordinating with the California Department of Food and Agriculture’s Agricultural Export Program and the California Energy Commission’s Energy Technology Export Program. Both programs organize trade shows and missions as part of their respective efforts to promote agricultural and energy technology exports. The Agricultural Export Program was once strongly involved in the International Division’s export activities, even directing the work of agricultural trade specialists, located in the International Division’s foreign offices. In 1999, the California Department of Food and Agriculture and the agency signed a one-year agreement giving the International Division day-to-day management of the trade specialists, but continuing the California Department of Food and Agriculture’s handling of programmatic and planning responsibilities for them. However, according to the deputy secretary of the International Division, in 2000 the division took over these responsibilities as well. The deputy secretary of the International Division believes the change was necessary to better integrate the work of the trade specialists with that of the foreign offices. He says that at this time the California Department of Food and Agriculture did not 36 have a director of international trade, and that under the agree- ment he does not believe the program’s acting director was in a position to independently create trade policy. The agencies did not renew this agreement when it lapsed in July 2000. The acting director of the Agricultural Export Program says his program did not renew the agreement and ended its involvement with the trade specialists because it believed its role was being reduced and it could not reach a meeting of the minds with the International Division. He said his program nevertheless, coordi- nates with the foreign offices on marketing and trade missions and receives some trade leads from them. As for the Energy Technology Export Program, it worked with the International Division to reach a formal agreement to The International Division coordinate its trade shows and missions in fiscal year 1999–2000. and the California Energy The program’s manager says he backed away from an agreement Commission could not because he believed the International Division’s proposal would reach an agreement on place the Energy Technology Export Program in a secondary coordinating their trade position rather than on equal footing. He says the program now shows and missions. has limited coordination with the International Division. The deputy secretary of the International Division believes the agency’s proposal would have better leveraged the expertise of each entity. Although realigning control of staff or consolidating services may be needed at times, coordination is likely to suffer unless entities mutually agree to such changes. Without coordi- nation, there is a greater likelihood that gaps and overlaps in services will not be identified and addressed appropriately. The International Division Has Reduced the Availability of Trade Leads to Other Entities Weaknesses in coordination also affect the International Division’s system for distributing “trade leads,” requests of foreign buyers for specific products. This system is designed to match trade leads with California exporters. The International Division significantly changed its trade lead system in 2000, without gaining the input of coordinating entities. Before then, the International Division entered trade leads on an Internet site called TradePort, available to trade entities statewide. This site, operated by a group of international trade entities in the San Francisco Bay and Los Angeles areas, offers a range of trade-related information. According to the director of Export Development, TradePort halted operations in 1999 because of lack of funding. TradePort later resumed operations, but the Export Development director does not consider it an option because she believes its viability is uncertain. Because of having to change systems, the International 37 Division established a very different way of handling its leads. It now has Export Development handle trade leads by itself instead of widely distributing them. Except for environmental technology and agricultural products, Export Development says it seeks the help of other entities only if it cannot locate an appropriate California exporter. However, the director of Export Development stated that even when it does seek help, it does not hand off the lead but rather asks for contact information to ensure it is sent to the appropriate foreign office. The director of Export Development explained that a new process was needed to ensure that leads are addressed adequately and Export Development that the International Division gets credit for its efforts. She said shares few non-agricultural that when the leads were put on TradePort, the International or non-environmental Division rarely learned if a lead resulted in an export success for technology trade leads which it could take credit. However, under the new system, the with other entities. International Division has shared few non-agricultural or non- environmental technology trade leads. The director of Export Development recognizes this fact but says that greater sharing is not feasible until the program gets an interactive database capable of disseminating and tracking leads. Considering that other entities also can match trade leads, however, sharing leads more frequently could offer the International Division an effective way to leverage its resources. Recent Activities Indicate a Renewed Focus on Coordination Acknowledging it needs to put more effort into coordination, the International Division has begun some initiatives to coordinate export services. Although they are steps in the right direction, their effectiveness remains to be seen, and further initiatives are needed. According to the director of Export Development, during the first few years of the new administration (beginning in 1999), it had to focus the bulk of its efforts on rebuilding the program’s foundation, including establishing internal processes, creating marketing materials, and building relations with the foreign offices. Internal matters rather than coordination with external entities were the priority. She says Export Development now is taking the initiative on coordinating with other trade entities to better leverage the State’s resources. For example, in May 2001, the agency joined with two Bay Area trade organiza- tions in successfully applying for a $400,000 federal Market Development Cooperator Grant. Under this grant, the agency will receive up to $109,000 and be responsible primarily for organizing trade missions in conjunction with trade shows, as well as partnering meetings in Europe. Bay Area trade organizations will 38 assist in organizing educational trade seminars, targeting start-up companies, and drawing on the resources of a French partner. Export Development also tried to organize a meeting with the world trade centers in June 2001. In addition, Export Develop- ment and Export Finance conducted some road shows around the State in summer 2001 to better explain their programs to coordinating entities. Moreover, Export Development plans to develop and implement an agreement to coordinate activities with the centers for international trade development. Some federal and local entities we interviewed said they have noted an improvement in the International Division’s willingness to coordinate after a March 2001 legislative hearing related to the agency’s planning and coordination efforts. POSSIBLE REDUNDANCY IN THE EXISTING SERVICE DELIVERY STRUCTURE MERITS FURTHER STUDY For this audit, we were asked to review the International Division’s methods for coordinating foreign investment and export services. In examining the International Division’s actions and in interviewing coordinating entities, however, we found an underlying issue that colors most discussions on coordination—who should provide these services to business clients. Further investigation into changing the delivery system is warranted to determine if entities are duplicating services and if gaps in service delivery leave some needs unmet. The current service delivery structure seems to perpetuate redun- dant services. Under the existing service delivery structure, the International Division promotes its services, generates trade leads, matches trade leads with exporters, counsels businesses on Trade entities appear to exporting, organizes trade missions and shows, and guarantees offer overlapping services. loans to exporters. Various other entities provide similar types of services. For example, world trade centers and centers for inter- national trade development offer counseling services to exporters and match exporters to foreign buyers. Other state agencies run trade missions. The federal Small Business Administration and the federal Export-Import Bank provide loan guarantees to exporters. Some entities believe the existing service delivery structure should be changed. The California Association for Local Economic Development, a statewide public/private organization dedicated to advancing the delivery of economic development services, published a paper in 1999 calling on the agency to reassess its activities and hand off “retail” services to local entities. In this 39 context, retail services means services to businesses, the ultimate customers of agency programs. The California Association for Local Economic Development indicated that the number and capacity of local entities has risen to the point that they should take over these duties and specifically recommended that the State enhance and support the capacity of local economic organizations to deliver international marketing and investment opportunities. Similarly, some world trade centers and centers for international trade development speak about the need for local entities rather than the agency to provide services to business customers, contending that local entities are specially situated to meet business needs at the local level because they already work with local businesses. One entity commented that the agency is “seeking to provide all levels of services itself, some of which it is not best positioned to do.” Some entities also say the International Division should limit its activities to “whole- sale” issues, such as statewide policy formulation, or to special situations, such as the operation of foreign offices, which the State is uniquely able to handle. In June 2001, two of the world trade centers proposed to com- bine resources with the agency in an effort they said would Trade entities have complement one another’s activities. These activities included divergent perspectives on providing marketing and consulting support for trade missions who should provide and matching foreign firms with California businesses. This services. proposal would have cost the agency about $200,000, funding that the International Division says its budget could not support. Further, the International Division states that the proposal did not request that it work with the world trade centers to develop joint projects, but rather sought funding that would allow the hiring of trade consultants for each world trade center. Statements on both sides underscore the divergent perspectives that trade entities bring to the issue of coordination. Under the existing service delivery structure, other state and federal agencies offer services similar to what the agency does. As discussed previously, the California Department of Food and Agriculture and the California Energy Commission promote exports by organizing trade shows and missions, which the International Division also does. Thus, it appears that redundant efforts may occur at the state level. In addition, the representatives of the federal export assistance centers we interviewed indicated that the International Division might be duplicating some services that they provide. The manager of one center, for example, said the center provides export counseling to many 40 businesses in its region. Because the export assistance center has a larger staff, the center’s manager believes Export Development could refer its customers to the center and thus more effectively serve them. We recognize that the service delivery issue is complicated. Although some entities may provide similar services, their overall mission, focus, and policies on charging for services may be different. Further complicating matters, entities represent different levels of government and some are not even a part of government. This raises issues of control and of accountability. For example, if world trade centers were to provide counseling services to International Division customers, how would the State ensure that services were performed adequately and who would foot the bill? Despite these complications, this matter warrants further attention, with an eye toward leveraging each party’s efforts. Although it is clear that the International Division can do more to coordinate with other entities within the existing structure, the efficiencies and other benefits it can achieve may be limited without basic changes to the service delivery system. RECOMMENDATIONS The Legislature should consider commissioning an independent statewide study of the existing delivery system for export ser- vices to determine the best division of work and resources among the various entities in the international trade arena. To better leverage its resources, the International Division should increase its efforts to coordinate with other trade entities, including doing the following: (cid:127) Hold regular meetings with other entities to discuss goals and operations. (cid:127) Analyze the service delivery system and discuss reducing service gaps and redundancies with relevant entities. This effort should include, but not be limited to, any study of the issue commissioned by the Legislature. (cid:127) Establish agreements that spell out its roles and interactions with other entities. 41 (cid:127) Discuss the trade lead system with other entities and consider whether sharing its leads with them to a greater extent would be beneficial. (cid:2) 42 CHAPTER 3 The Economic Development Division Generally Provides Good Customer Service, but It Could Benefit From Formal Processes To Measure Customer Satisfaction CHAPTER SUMMARY T he Economic Development Division within the Technol- ogy, Trade and Commerce Agency (agency) primarily assists local organizations to develop their local economies and provides financial and technical assistance to local busi- nesses. Economic Development Division programs are generally responsive to customers, and their relatively small circles of customers generally are satisfied with most programs’ overall services, as shown by our survey of a sample of customers. However, the programs could benefit from some formal processes to measure customer satisfaction, such as setting targets for customer satisfaction levels and using surveys to obtain feedback. Also, the Small Business Loan Guarantee Program needs to resolve concerns with its customers, who ranked the program low in our survey. ECONOMIC DEVELOPMENT DIVISION PROGRAMS TYPICALLY GATHER FEEDBACK INFORMALLY The seven Economic Development Division programs we reviewed for customer service generally provide technical services to a small number of entities and receive feedback from their customers informally, typically through customer telephone calls, e-mails, and meetings. None of the programs currently uses formal feedback tools such as customer satisfaction surveys to gauge the level of customer approval of the programs’ overall services. A few survey customers on specific services, such as training sessions. Division management says it has not used formal feedback tools more widely because it lacks the funds and staff to administer them. Nevertheless, the deputy secretary for the Economic 43 Development Division states that the division recognizes the importance of customer service and is incorporating customer satisfaction surveys, goals, and targets into its work plans for 2002. Customers for most of the seven Economic Development Division programs we reviewed are local organizations that assist their communities in attracting and expanding businesses. These customers include cities and counties, economic development corporations, financial development corporations, and small business development Seven Economic Development Division centers. The economic development corporations Programs Reviewed for Customer Service often receive funding from cities and counties; the financial development corporations and (cid:127) Enterprise Zone Program small business development centers receive (cid:127) Main Street Program funding from the Economic Development (cid:127) Military Base Reuse Program Division through contracts. In each case, the (cid:127) Sacramento Regional Office local organizations provide services to the (cid:127) Small Business Assistance and ultimate customers of Economic Development Advocacy Program Division programs—businesses. (cid:127) Small Business Development Center Program The Small Business Loan Guarantee Program, (cid:127) Small Business Loan Guarantee Program for example, provides funding, oversight and technical services to 11 financial development corporations. In turn, the financial develop- ment corporations guarantee loans for small businesses in their regions. Similarly, the Office of Small Business provides funding, outreach, and technical services to 52 small business development centers, which then counsel, train, and advise small businesses at the local level. The Economic Development Division’s programs sometimes deal directly with businesses. For instance, its Sacramento Regional Office serves businesses interested in investing in the region stretching from Kern County to the Oregon border. This office typically assists these businesses by providing information on available business location sites and on incentive programs offered by the State. The Small Business Assistance and Advocacy Program also directly serves businesses by offering information on topics such as licensing and permitting requirements through publications, an Internet site, and telephone services. More information on reviewed Economic Development Division services and customers is in the Appendix. The Economic Development Division programs we reviewed generally receive feedback from customers through informal mechanisms such as telephone calls and e-mails initiated by 44 customers. Customers will call with a question, request, or complaint, and program staff will respond. Customers also may bring up issues at meetings and conferences. Programs rarely take the first step in obtaining feedback. Training is one of the few areas where programs are proactive in obtaining feedback. For example, the Main Street Program surveyed its customers to determine their satisfaction with a training workshop that it held. Also, one program’s customers were surveyed on their satisfaction level with one product. Specifically, the Small Business Assistance and Advocacy Program received customer satisfaction information in 2000 on one of its products, the Professional and Business License Handbook, which it jointly produces with the Employment Development Department (EDD). The handbook assists businesses with California licensing requirements. The EDD surveyed customers on its various products, including the Professional and Business License Handbook, and forwarded the results to the Small Business Assistance and Advocacy Program. However, none of the reviewed programs currently uses formal feedback mechanisms such as customer surveys to determine the level of satisfaction with their overall service. CUSTOMERS GENERALLY ARE SATISFIED WITH PROGRAM SERVICES Although programs lack formal feedback mechanisms, our survey of a sample of customers yielded average customer service rankings between 3.6 and 4.5 on a 5-point scale, with 1 being poor and 5 excellent, for five of the seven reviewed programs. The Customers for five of Enterprise Zone, Small Business Development Center, Sacramento seven reviewed programs Regional Office, Military Base Reuse, and Main Street programs rated customer service as scored well. The Small Business Loan Guarantee Program cus- above average. tomers—financial development corporations that provide loan guarantees to businesses—ranked service as below average. We did not receive enough responses to determine a level of customer satisfaction for the Small Business Assistance and Advocacy Program. Customers of the five programs that scored well gener- ally believe those programs are responsive to their needs and expectations. For example, surveyed customers for the Sacramento Regional Office said the office provides accurate information and is generally responsive to their inquiries. Likewise, customers of the Main Street Program stated that the program staff are readily available to answer questions and are responsive to requests. In addition, some customers said the program directs them to the right place when they need a referral to another organization, including the name of the contact person. 45 Although the Military Base Reuse Program received an overall ranking of 3.6, some customers expressed dissatisfaction with its services. Two of the eight customers we talked with ranked the program’s customer service at 1, or poor. One customer said the program does not respond to inquiries and does not return calls. The other customer said the program promised to send informa- tion about program services but did not send the information promptly and then sent inadequate information. Consequently, at the time of the survey, this customer still did not know what services the program offers. In the case of the Small Business Assistance and Advocacy Program, too few customers responded to our e-mailed inquiries to determine their satisfaction level. The deputy director for the program told us that staff do not keep track of customers who contact them by telephone for information, but they do keep the e-mail addresses of customers who e-mail the program with questions or requests. We surveyed a sample of these customers but did not receive sufficient responses to determine if customers are satisfied. With a mission of providing information on how to conduct business in California, the Small Business Assistance and Advocacy Program should ensure this information is meet- ing customers’ needs. However, the Small Business Assistance and Advocacy Program currently does nothing to follow up with its customers who receive technical assistance and so cannot identify those customers’ satisfaction levels or service areas that need improvement. THE SMALL BUSINESS LOAN GUARANTEE PROGRAM NEEDS TO WORK OUT DIFFERENCES WITH THE FINANCIAL DEVELOPMENT CORPORATIONS Although many Economic Development Division customers are satisfied with its services, the financial development corporations, which provide loan guarantees for small businesses, are generally The Small Business Loan dissatisfied with services provided by the Small Business Loan Guarantee Program got Guarantee Program. We tried to survey all eight existing financial low ratings for customer development corporations located throughout California, but only service. five responded. (Eight corporations were in existence at the time of our survey; three more were incorporated in November 2001; another is expected in 2002.) The five financial development corporations that responded gave the program an average score of only 2.2 on a 5-point scale. The Economic Development Division maintains its responsibility for overseeing the financial development corporations is sometimes incompatible with 46 pleasing these customers. This may be a valid point, but the low level of customer satisfaction indicates the program needs to discuss concerns with the financial development corporations and needs to establish ongoing methods for gauging customer satisfaction. The financial development corporations’ concerns included inconsistent and slow technical service, lack of continuity during the latest transition in state administrations, lack of a statewide marketing effort for the program, and no efforts to gain their feedback. For instance, the president of one financial development corporation complained that program staff told the corporation to undertake a particular activity to solve a problem, but the agency’s legal department later told the corporation to do the opposite. Such inconsistency can lead to confusion and tension between the agency and the financial development corporations. Some financial development corporations also complained that Financial development the Small Business Loan Guarantee Program has not done enough corporations are to promote increased state funding for the program. State law concerned about slow enacted in 2000 mandated four new financial development technical service, lack of corporations. Some existing financial development corporations statewide marketing were concerned that their individual funding levels would efforts, and the level of decrease in order to provide money for the new organizations. state funding. They believe the program could have done more to increase overall funding. When we discussed these complaints with the assistant secretary for the Economic Development Division, she said the agency’s primary role is to monitor the work of the financial development corporations, so it cannot always please them while also fulfilling its oversight duties. She also stated that many questions the financial development corporations ask require research and consultation that delay response times. Meanwhile, the manager of the Financial Assistance Unit for the Office of Small Business, which oversees the Small Business Loan Guarantee Program, said the program had attempted to obtain more funding. In May 2000, the agency requested the financial development corporations’ funding be increased by 50 percent, or $16 million, to maintain the current level of funding for the eight existing corporations while providing similar funds for the four new corporations. The Legislature approved an increase of half that request, or $8 million. 47 PROGRAMS COULD BENEFIT FROM CUSTOMER SATISFACTION SURVEYS AND TARGETS As said earlier, the Economic Development Division recognizes the importance of customer service. In fact, some program plans include goals and activities related to providing quality service. However, none of the plans for the seven programs we reviewed contains quantified targets for service quality or customer satisfaction. Without methods for measuring service quality, programs do not know how well they are satisfying customers or whether a change in methods is producing the desired effect. For example, the Sacramento Regional Office has planned to collect feedback from businesses to find ways to improve the office’s customer service but did not establish an associated performance measure or evaluation method. Also, the plan for the Small Business Development Center Program includes a goal to provide and maintain consistent, quality customer service to small businesses, but the plan does not contain a related performance target. However, the program will need to develop a strategic plan and performance targets that include customer service as part of its upcoming recertification process with the Association of Small Business Development Centers. According to the assistant state director for the Small Business Development Center Program, the recertification will take place in 2003, and the program expects to update its strategic plan in spring 2002 to meet the requirements. According to the recertification standards, this plan should reflect systematic processes for identifying customer needs and expectations, meeting those needs and expectations, and determining the level of customer satisfaction. To measure customers’ satisfaction and more clearly determine their needs and expectations, programs should survey their Customer surveys offer a customers regularly. By surveying customers and setting quantified way to measure targets for customer satisfaction, the Economic Development satisfaction levels and to Division can better focus its efforts on activities that make a more clearly determine difference to its customers and lead to improved program effec- customer needs. tiveness. Although many customers are satisfied with the services they receive, the survey ratings indicate room for improvement, with some customers noting specific concerns. Customers’ suggestions included improving the timeliness of information, being more proactive in obtaining feedback, communicating to customers what the program intends to do with the feedback, and improving the transition process during changes in adminis- tration. By using formal methods, such as customer satisfaction surveys, the programs would be able to measure their performance and more reliably determine customers’ unmet needs and 48 expectations. The surveys also would allow the programs to show what they are doing well and to improve upon those efforts. The programs also should set goals for customer service, such as targets for turnaround time in answering customers’ questions or expected levels of satisfaction. As discussed earlier, the programs rely on informal feedback mechanisms. In the past, however, at least one program surveyed its customers. The Office of Business Development, which oversees the Enterprise Zone Program, developed a survey and measured customer satisfaction in 1997. In that survey, many customers ranked the Enterprise Zone Program staff highly on their helpfulness, responsiveness, and knowledge. They also found the Enterprise Zone Program’s newsletter, annual report, and legislative updates helpful. However, some customers com- plained that the program’s information on businesses that were considering relocating or expanding in California was not very helpful. The assistant secretary for the Economic Development Division said the Office of Business Development stopped surveying after that year because of resource constraints. However, the deputy secretary for the Economic Development Division states that the division recognizes the importance of customer service and is incorporating customer satisfaction surveys, goals, and targets into its work plans for 2002. RECOMMENDATIONS The Economic Development Division should take these actions to improve customer satisfaction: (cid:127) Develop goals and associated targets for customer satisfaction and service for each of its programs. (cid:127) Periodically survey customers to gauge the quality of customer service and to identify areas that need improvement. (cid:127) Evaluate performance by comparing results with targets, and change services as needed. In addition, the Small Business Loan Guarantee Program should work with the financial development corporations to discuss concerns and determine what actions it should take to resolve them. 49 We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, ELAINE M. HOWLE State Auditor Date: December 13, 2001 Staff: Karen L. McKenna, CPA, Audit Principal Jim Sandberg-Larsen, CPA Roberta A. Kennedy Kris D. Patel 50 APPENDIX Description of Technology, Trade and Commerce Agency Programs Reviewed in This Report ECONOMIC DEVELOPMENT DIVISION T he Enterprise Zone Program within the Office of Busi- ness Development designates enterprise and other tax incentive zones and oversees and provides technical assistance to the cities or counties that operate the 48 existing zones throughout the State. The cities or counties in turn encour- age business and industrial growth by helping businesses take advantage of state and local tax incentives for operating in these economically depressed areas. The Main Street Program within the Office of Business Devel- opment designates downtown and neighborhood commercial districts as “Main Street” communities and provides technical assistance and training to the local community organizations dedicated to revitalizing these districts. The Military Base Reuse Program within the Office of Military Base Retention and Reuse helps communities market their former defense facilities and identifies matching funds for federal grants aimed at shifting national defense resources into alternative economic endeavors. The Regional Offices provide services to businesses, including one-on-one problem solving, expansion assistance, and employ- ment and training for companies interested in locating to California. The regional offices also provide cities, counties, and economic development corporations information on companies that are considering relocating or expanding to the region. We reviewed planning and customer service for the Sacramento Regional Office. The Small Business Assistance and Advocacy Program within the agency’s Office of Small Business assists small businesses and individuals by answering questions on how to conduct business in California and by analyzing general issues impeding the success of a small business. 51 The Small Business Development Center Program within the agency’s Office of Small Business provides funding, oversight, and technical services to the 52 independently operated small business development centers throughout the State. The small business development centers in turn offer business counseling, referrals, and training activities to small businesses. The Small Business Loan Guarantee Program within the agency’s Office of Small Business provides funding, oversight, and contracting services to 11 financial development corporations throughout the State. The financial development corporations in turn provide loan guarantees for small businesses that are unable to obtain conventional loans and funding. CALIFORNIA INFRASTRUCTURE AND ECONOMIC DEVELOPMENT BANK The Infrastructure State Revolving Fund Program provides loans to local governmental entities for infrastructure projects. These entities include cities, counties, redevelopment agencies, and special districts. INTERNATIONAL TRADE AND INVESTMENT DIVISION The California Export Finance Office provides loan guarantees that help qualified companies acquire short-term working capital loans to complete specific export sales. The Office of Export Development promotes exports by organiz- ing participation in trade shows focused on high-value products with strong export potential. It also uses trade leads to match California exporters to potential foreign buyers. The Office of Foreign Investment provides potential investors with information on the location of sites and facilities, the availability of labor, and business permit processes. The office also organizes foreign investment missions to introduce foreign companies to potential California business partners. The International Trade and Investment Offices or Foreign Offices provide overseas coordination and support for the agency’s trade shows, investment promotions, and business missions, as well as direct assistance to California companies seeking business 52 opportunities in foreign countries. The International Division operates 12 foreign offices. We reviewed planning and coordina- tion for the Mexico office and the Japan office. DIVISION OF SCIENCE, TECHNOLOGY, AND INNOVATION The Manufacturing Technology Program provides funding, oversight, and technical services to two regional manufacturing technology centers. The manufacturing technology centers in turn provide business assistance to industries identified by the program as significant to the State’s economic development. DIVISION OF TOURISM The Tourism Program promotes travel to and within the State through national and international advertising, sales missions, and heightened California presence at major trade shows. It also conducts cooperative marketing campaigns, operates a toll-free visitor information service, and produces California visitor guides and other promotional material. 53 Blank page inserted for reproduction purposes only. 54 Agency’s comments provided as text only. CALIFORNIA TECHNOLOGY, TRADE AND COMMERCE AGENCY 801 K Street, Suite 1918 Sacramento, CA 95814-3520 December 5, 2001 Elaine M. Howle* State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Re: Response to Auditor Report on the Technology, Trade and Commerce Agency Dear Ms. Howle: Attached please find the response of the California Technology, Trade and Commerce Agency (TTCA) to the Bureau of State Audit’s December 2001 report on the agency. The TTCA is an exciting and dynamic agency, responsible for delivering a diverse set of programs to boost the California economy. In the two years since Governor Davis took office, California has moved from the 7th to the 5th largest economy in the world. To continue to achieve this level of success, the TTCA must maintain maximum flexibility to adjust to the frequent and hard to predict shifts in the global, state and local economies. For this reason, TTCA developed a strong overarching and clearly articulated vision statement to provide an overall direction, and now relies on work plans at the program level to acknowledge the important differences in the nature of the programs the agency delivers and maximize flexibility to respond to changes in the economy. The TTCA is dedicated to improving all aspects of its operations, and appreciates the positive input 1 from the Bureau of State Audits. The TTCA is gratified by the Bureau’s findings that our customers are highly satisfied with the services we provide. We will use the input from the audit report to enhance and inform our strategic planning process and our provision of services to the interna- tional trade and economic development communities. In addition to responding to the report recommendations, our response provides background information on the TTCA, its strategic planning process, and the International Trade and Invest- ment and Economic Development Divisions. We are hopeful that this information will be of assis- tance to readers of the report who are not familiar with the agency. Sincerely, (Signed by: Lon S. Hatamiya) Lon S. Hatamiya Secretary Attachment *California State Auditor’s comments begin on page 71. 55 CALIFORNIA TECHNOLOGY, TRADE AND COMMERCE AGENCY RESPONSE TO BUREAU OF STATE AUDITS REPORT DECEMBER 5, 2001 INTRODUCTION The California Technology, Trade and Commerce Agency is the State’s lead agency for innovation, investment and economic opportunity. Its 300-plus employees are responsible for delivering 37 diverse programs to boost the California economy, including financing and investment, community development, business start-ups and expansion, site selection, permit processing, export develop- ment, industry-specific assistance, economic information and data, marketing, and domestic and international trade shows and missions. They are located in seven offices throughout the State and twelve offices overseas. Since Governor Davis took office in 1999, California has moved from the 7th to the 5th largest economy in the world. Several unique characteristics of the TTCA must be considered in determining an appropriate strategic planning process for the TTCA. First, the TTCA operates in a dynamic environment and must remain responsive to the policy priorities of the Governor and the Legislature. For example, in January 1999, Governor Gray Davis defined general policy priorities for the (then) Trade and Commerce Agency, which included building a partnership with Mexico and expanding export and investment opportunities with Latin American markets; strengthening and expanding export and investment opportunities with the Asian and European markets; addressing the chronic unemploy- ment, distressed communities, e-commerce-related infrastructure needs and economic diversifica- tion needs of the Central Valley; facilitating the innovation, investment and growth of technology, especially for educational and competitiveness objectives; and investing in the infrastructure needs in California. In January 2001, a new law changed the name of the agency to the Technology, Trade and Commerce Agency and created the new Division of Science, Technology and Innova- tion.1 The TTCA also receives regular input from a variety of advisory groups, composed of different industry leaders, established by statute to advise and assist the TTCA in the development of its programs. These groups include the World Trade Commission, the California Travel and Tourism Commission, the California Film Commission, the California Defense Retention and Conversion Council, the California Rural Development Council, the California Infrastructure and Economic Development Bank Board, the Rural Economic Development Infrastructure Panel, the California Export Finance Board, and the California Small Business Board. Second, in order to ensure that the programs and services of the TTCA are best tailored to the current business climate of California, the TTCA is constantly adapting to changes in the external environment, in particular, frequent shifts in the global, state and local economies. In 2001 alone, for example, the TTCA aggressively shifted focus several times to respond to issues that could not have been foreseen or anticipated even with the most rigorous strategic planning process. In January, the TTCA shifted a major part of its focus to respond to the energy challenge. Recogniz- ing the TTCA’s direct and long-standing relationship with the California business community and 1 Chapter 1056, Statutes of 2000 (SB 1136, Vasconcellos). 56 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 2 trade organizations, Executive Order D-19-01 charged the TTCA with communicating the Governor’s order to reduce maximum outdoor lighting capacity to the public, retail establishments and local officials. In the Spring, the TTCA again shifted some of its focus to technology research and development in response to the downturn in the dot-com industry. Then in October, the TTCA redirected it energy towards an aggressive campaign to stimulate Californians to travel within the State to shore up the travel and tourism industry, which was hit hard by the terrible events of September 11th. Shortly thereafter, in early November, the TTCA played a major role in the Governor’s Economic Summit. The TTCA’s activities in the immediate future will be affected by the budget shortfalls and the nationwide economic recession, two events that could not have been anticipated as recently as the beginning of the year. Third, the TTCA “product” – jobs, investment and a healthy economy – is influenced by numerous and complex external factors that impact the amount of control that the TTCA has over its final product. The one factor that the TTCA can control in a rapidly shifting environment is the method used for delivery of services, which the agency can adjust to achieve the best results given exter- nal constraints. However, even the best processes cannot guarantee maximal results when unexpected and uncontrollable crises or challenges occur. In recognition of these unique factors, and taking advantage of the relatively small size of the TTCA, the Secretary has determined that the most effective way to plan strategically for and manage the complex role of this agency is to provide an overriding long-term vision for the TTCA as a whole, to hold weekly and often daily meetings with senior management to modify and refine the short-term goals, and to charge the program areas with the task of detailed work plans to 2 implement the vision and priorities of the TTCA. The Secretary believes that this hands-on, inter- active strategic planning approach is a better fit for this TTCA than the traditional model, as it provides the strongest vehicle to maximize the effectiveness of the TTCA.2 To implement this interactive and collaborative planning process, in April 2000, the Secretary initiated a visioning process that resulted in an agency-wide vision statement. This inclusive, agency-wide process was conducted over a six-month period, facilitated by an outside consultant. In November 2000, the Secretary held an agency-wide meeting, attended in person by the majority of the TTCA staff and over the intranet by most other employees, at which he announced the new vision statement and the new technology division, and articulated his vision and goals for the TTCA. In the interim, as noted in the report, the TTCA had issued internal guidelines for evaluating 2 Current planning theory recognizes this model, as well as the traditional model favored by the Bureau. See Michael Allison and Jude Kaye, “Strategic Planning for Nonprofit Organizations,” 1997; George Albert Steiner, “Strategic Planning: What Every Manager Must Know,” 1997. See also, David Osborne with Ted Gaebler, “Reinventing Government: How the Entrepreneurial Spirit is Transforming the Public Sector,” 1993, Pages 250-54 (decentralized government). 57 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 3 programs, which the report describes as “a good basis for a sound strategic planning process.” In addition, since assuming office, the Secretary has conducted weekly meetings of senior staff, which are attended by the TTCA’s Deputy Secretaries, Assistant Secretaries and Directors, during which changes or shifts in the economy and environment and the current priorities for the TTCA are discussed. These weekly meetings enable senior management to understand programs throughout the TTCA, and to get a sense of how their program fits into the bigger picture. They also facilitate proactive communication within the TTCA and create opportunities for collaboration. The Secretary also meets with members of senior staff on a daily basis. The Deputy Secretaries are charged with bringing the information back to their divisions and ensuring that the information is incorporated into the program work plans. A more decentralized planning approach allows the TTCA to develop the infrastructure to respond to a rapidly changing environment within our economy. And there is no question that the TTCA has 3 been highly effective in delivering new and old programs using this model. Since the Davis administration took office in 1999, the TTCA has: (cid:127) Implemented the Infrastructure and Economic Development Bank, which has provided more than $1.2 billion of financing to promote the economic growth and revitalization of California communities through the low-cost financing of vital infrastructure and economic development projects throughout the State. These financings are projected to generate substantial public benefits, including an estimated 12,000 new jobs, and an improved environment, increased health and safety, and an enhanced overall quality of life in com- munities throughout the State. (cid:127) Helped international trade grow to record levels, with California firms benefiting from the increased export opportunities. For instance, California’s exports jumped to a record $129.7 billion in 2000, surging 20.8 percent over 1999. An estimated 273,000 direct and indirect jobs were added in 2000 based on the U.S. Department of Commerce job estimate formula of 12,000 jobs created per $1 billion in trade. (cid:127) Through its Small Business Development Center program the Agency created or retained 9,215 jobs, provided technical assistance to enable small businesses to obtain 486 SBA loans (totaling $99,793,298) and 728 non-SBA loans (totaling $81,499,866). (cid:127) Established the new Division of Science, Technology and Innovation. The Division primarily provides grants and partnership opportunities to new and emerging innovators and entre- preneurs. DSTI’s programs are responsible for awarding 246 grants totaling $72,071,000, resulting in a corresponding job impact of approximately 18,900 jobs created and retained. 58 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 4 (cid:127) Created three new Financial Development Corporations to provide loan guarantees to small businesses. The three programs, when added to the State’s existing FDC network, will help leverage nearly $160 million in small business loan guarantees and farm loans. (cid:127) Developed the Film California First program, which provides financial incentives to film companies to keep film production in California. Since inception, it has received more than 1,000 applications for reimbursement of fees totaling more than $5 million. (cid:127) Assisted in the attraction, retention and/or expansion of numerous corporate operations, resulting in nearly 16,544 jobs and $380 million in investment in California. (cid:127) Responded to more than 8,200 direct small business requests through the Small Business Advocacy Program. (cid:127) Guaranteed 1,664 loans worth more than $215 million through the Small Business financ- ing programs. (cid:127) Certified eight new communities as California Main Street communities, designated two new Local Agency Military Base Recovery Area (LAMBRA) zones, and provided five-year extensions to 13 of the State’s 39 Enterprise Zones. The TTCA also implemented a com- prehensive multimedia marketing campaign that encompasses all of the State’s Enterprise Zones. (cid:127) Launched the “Adventures in Wild California” IMAX film to market California as a travel destination throughout the world. Opened four new California Welcome Centers. (cid:127) Opened new overseas trade offices in Argentina, Shanghai and Singapore. Opened a new trade office in the Central Valley. Opened two new international travel trade offices in Australia and Brazil. (cid:127) Helped bring the SR Technics aircraft maintenance and repair facility project to Palmdale, resulting in the creation of more than 600 high-paying jobs in the aerospace industry. In the TTCA’s view, there is no one model of strategic planning that is the best fit for any given organization, and strategic planning is an inherently dynamic process and is therefore never 2 “complete.” The TTCA believes that the planning process it has adopted is the best model for a small, agile agency like the TTCA. The TTCA also believes that, as with any process, there is 59 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 5 always room for improvement, and the TTCA does not object to a number of the Bureau’s recom- mendations for incorporating certain elements of the Bureau’s preferred strategic planning process into the TTCA’s strategic planning process. The TTCA’s response to the specific recommendations in the report follows. CHAPTER 1 As noted in the Introduction, the TTCA has developed and is using a number of elements of the strategic planning model preferred by the Bureau. The TTCA has a vision statement, developed through an extensive six-month process in 2000. The TTCA has a draft of its guiding principles, which will be put into final form. The TTCA has individual work plans for its programs, which are updated on an annual basis. The TTCA has a structure in place to conduct environmental scans and compare targets to revise goals and will update and formalize reporting of these activities where feasible. The Bureau acknowledges in its report that all of the 13 plans they reviewed had been recently updated and included mission statements, had goals that aligned with the mission, and included detailed lists of tasks to accomplish each goal. In the report, the Bureau makes the following specific suggestions regarding ways to improve the TTCA’s planning process. Recommendation #1: “To more effectively manage its programs, the Technology, Trade and Commerce Agency should take the following actions: (cid:127) “ Develop an agency-wide strategic plan covering at least five years in order to better integrate program efforts and to highlight current State priorities. The agency should also ensure that short-term plans for programs are aligned with the agency-wide strategic plan.” Response: In light of the rapidly changing and unpredictable environment in which the 2 TTCA operates, the TTCA does not agree that the strategic planning model supported by the Bureau will maximize the effectiveness of the TTCA. The TTCA has used its informed judgment to determine how best to spend the limited resources available to the agency to implement its multiple, diverse programs. The TTCA is concerned about the impact on the public and the State’s economy of the Bureau’s suggestion that it divert increasingly limited 4 resources away from program delivery to develop a five-year agency-wide strategic plan. Nevertheless, the TTCA agrees to review the 1997 strategic plan, which was not time- specific, and to update the plan and incorporate elements of the Bureau’s model, where appropriate. The TTCA will also finalize and distribute the guiding principles for incorpora- tion into its work plans. (cid:127) “ Include in its strategic planning process the elements this report identifies as missing in many of the agency’s program plans. These elements include goals and targets for all significant aspects of its vision and mission, and for significant accomplishments noted in external reports; outcome goals that focus efforts on results where they matter most; 60 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 6 targets that are challenging in light of prior performance and expected economic assumptions; and outcome goals and related targets in agreements with third parties who deliver program services.” Response: The TTCA agrees to review the format of its work plans with an eye toward incorporating more of the elements of the Bureau’s strategic planning model into its work plans. The TTCA notes that outcome goals will vary from program to program, depending on the purpose of the program. The TTCA also notes that with respect to external reports, the Legislature has identified with particularity in the statutes requiring the reports the issues and statistics it wishes to see addressed in the reports. (cid:127) “ Compare targets and results in internal and external reports to evaluate success and use this information to revise goals and targets as appropriate.” Response: The TTCA agrees to conduct this comparison in a more rigorous fashion. (cid:127) “ Periodically scan the environment to formally identify opportunities and threats that could significantly affect goals and include such assessments in its plans.” Response: The TTCA already regularly conducts these scans, however, it does not cur- rently formally document the results of these scans, but rather incorporates the information into its work plans. The TTCA agrees to revise the format of its work plans to document the results of these scans. (cid:127) “ Give high priority to nominating persons to appointed management positions in the Inter- national Division, and nominate persons to appointed staff positions that are necessary for program continuity even if managers are not yet appointed.” Response: The TTCA fully supports the Governor’s appointments process, and has always given its highest priority to locating and recommending to the Governor qualifiedcandidates for all of its exempt management positions, including the Directors for the International Division’s overseas offices. The TTCA cannot commit to the recommendation to fill 5 lower level positions before managers are appointed, since the TTCA questions whether this is a sound management principle. (cid:127) “ Ensure the data it reports as the results of its programs are as accurate as possible, including performing follow-up on client estimates as needed. The agency should also verify some of the inherently less reliable, client-supplied information on a sample basis.” 6 Response: The TTCA disagrees that its clients are inherently unreliable in reporting antici- pated job creation. The Small Business Loan Guarantee Program requires that job cre- ation information be collected and reported on all borrowers. Financial Development 61 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 7 Corporations have individual job creation goals and they market the Loan Guarantee program to lenders and business with those goals in mind. However, as established by 6 statute, job creation is not one of the factors that determines whether an individual borrower’s application for a loan guarantee is granted or denied by an FDC. Similarly, job creation estimates do not qualify or disqualify applicants for an Export Finance Office loan guarantee. Therefore, there is no incentive for borrowers under either of these programs to distort job creation figures included on their applications. Nevertheless, the TTCA will contact the Employment Development Department to obtain EDD data to verify job esti- mates on a sample basis. Recommendation #2: “[T]he agency should report to the Legislature biennially on its progress in implementing a strategic approach to its planning, including specific recommendations in this chapter.” 7 Response: The TTCA will be subject to the traditional sixty day, six month, and one year follow-up with the Bureau. The TTCA questions whether biennial review is necessary. Recommendation #3: “[T]he World Trade Commission should consider implementing procedures so that it can continue to advise the agency even if a chairperson is not yet appointed.” Response: As was discussed at the exit interview, the World Trade Commission is revising its bylaws to address this issue. CHAPTER 2 The TTCA was pleased that the Bureau acknowledged several programmatic areas of the Interna- tional Trade and Investment Division that were providing quality services to California companies. This is consistent with the feedback that the Division has received from the companies, financial institutions, and partner organizations that it works with on a day-to-day basis. The Division is committed to increasing the level of coordination with federal, state and local entities, and was pleased with the Bureau’s acknowledgement that the Division is making progress in this area. As directed by the Secretary, a primary focus for the Division over the past two years has been to strengthen its internal management structure and to establish clear operational procedures. The successes in these areas have been important to building a foundation upon which to pursue external coordination. Prior to 1999, the Division had not had an internal staff meeting for more than one year. In February 1999, the Division initiated weekly planning meetings for its California- based directors and policy staff. The directors of the core programs also lead weekly or semi- monthly staff coordinating meetings. The Division’s decision to relocate the managers, who had been working in three different cities, to Sacramento, strengthened the working relationship among the Division’s managers. 62 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 8 In 1999, for the first time, the Division initiated weekly conference calls to coordinate its activities with the foreign offices. These meetings built upon the Division’s strategic plan to install three regional managing directors for Asia, the Americas, and Europe/the Middle East/Africa. The managing directors coordinate trade and investment activities for their respective regions. By reshaping its internal management and planning structure, the Division has built a stronger infra- structure and more coordinated programs to assist California companies and to work with partner organizations. The Division is composed of five core operating units. The Office of Foreign Investment, the Office of Export Development, the Office of Export Finance, the 12 Foreign Trade Offices and the Interna- tional Policy Unit. These functional areas offer a broad set of services that are dedicated to in- creasing international trade and investment for the State of California. These programs specialize in assisting international companies to locate their operations in California, assisting California companies to penetrate international markets, providing loan guarantees for working capital for exporters, representing the State’s international business priorities in 12 locations worldwide, organizing delegations of private companies and business missions for the Executive and Legisla- tive branches of California government, and providing program management and trade policy direction to the State of California. While each functional unit is dedicated to promoting international business for the State, each unit achieves its objectives by working with a unique set of companies and partners. Expertise is provided in the form of direct interaction with clients, ongoing promotional and technical work- shops, joint development of marketing campaigns including trade shows and inbound and out- bound trade missions, review and approval of loan applications, and sector analysis of the New Economy trends in California. Services are provided in various formats, including directly by the TTCA staff and coordinated through six regional offices in San Diego, Long Beach, Sacramento, Fresno, Santa Clara and San Mateo; through contracts with third party consultants in five of the international trade offices; through cooperative agreements with private sector trade associations and local, state and federal agencies; and through informal networked relationships. The Division’s “clients”—those entities that receive TTCA services—are also varied, and include international and domestic small-, medium- and large-sized companies, local governments, finan- cial institutions and banks, the Legislature, foreign government officials, international trade promo- tion entities of all sizes, and local economic development organizations. The California Export Finance Office (CEFO) provides loan guarantees of up to $750,000 for California exporters who need access to working capital to finance their export transactions. By statute, CEFO may leverage its $10 million fund up to four times, allowing the program to guaran- tee up to $40 million in loans. The program works closely with more than 100 financial institutions in California, which provide financing to the companies. CEFO’s partners include the US Small Business Administration and the US Export Import Bank. CEFO assists clients from its regional 63 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 9 offices in San Diego, Long Beach, Sacramento, Santa Clara, and its new Central Valley office in Fresno, opened in January 2001. The TTCA agrees with the Bureau’s assessment that CEFO has been successful in its coordination efforts. The Office of Foreign Investment (OFI) assists international companies to understand how to do business in California and advises companies about how to locate their operations in California. OFI uses a wide network of resources to attract foreign investments into California, including state and local agencies, foreign governments and private resources. OFI also serves as a business advocate and liaison between foreign companies and state government. OFI’s partners include economic development organizations, state and local government agencies, private developers, chambers of commerce, port authorities, utility companies and industrial real estate brokers. The program provides companies with information on labor availability, worker training and wage rates that are project-specific. The Division operates OFI offices in Long Beach, Sacramento and San Mateo. The TTCA agrees with the Bureau’s assessment that OFI has been successful in its coordination efforts. The Office of Export Development (OED) assists California’s small- and medium-sized companies to sell products to international markets. The program places emphasis on facilitating California technology exports in the area of hardware and software, biomedical, aerospace, environmental, and agriculture. OED organizes international trade shows and trade missions, provides match- making services for California sellers and global buyers, and provides market-specific trade infor- mation to California suppliers. It works closely with the California Centers for International Trade, chambers of commerce, economic development bodies, federal, state and local government agencies, and private industry associations. OED also operates the Environmental Technology Export Program, which facilitates the export of environmental technologies, and the Overseas Procurement Opportunities Program, which assists California companies to facilitate sales to international governments. The Division established a new OED office in Fresno in January 2001 to serve as its state hub for the Rural Export Assistance Program. OED assists clients from re- gional offices in San Diego, Long Beach, Sacramento and Santa Clara. The TTCA offers its views on the success of OED’s coordination efforts in response to the Bureau’s recommendations, below. Recommendation #1: “The Legislature should consider commissioning an independent study of the existing delivery system for export services to determine the best division of work and re- sources among the various entities in the international trade arena.” Response: The TTCA supports the Bureau’s recommendation, although it cautions that given the complexity involved in the delivery of export services and the varying interests of the parties involved, such a study would have to be conducted by a truly independent entity with a recognized expertise in trade issues to give the results the credibility necessary to provide the foundation upon which to build a statewide plan. 64 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page10 Recommendation #2: “To better leverage its resources, the International Division should increase its efforts to coordinate with other trade entities, including doing the following: (cid:127) Hold regular meetings with other entities to discuss goals and operations. (cid:127) Analyze the service delivery system and discuss reducing service gaps and redundancies with relevant entities. This effort should include, but not be limited to, any study of the issue commissioned by the Legislature. (cid:127) Establish agreements that spell out its roles and interactions with other entities. (cid:127) Discuss the trade lead system with other entities and consider whether sharing its leads with them to a greater extent would be beneficial.” Response: On Coordination Efforts: The TTCA is proud of the central role played by its International Division in coordinating and interacting with California’s diverse trade organizations since 1999. The Division has planned annual events, organized workshop series and one-time activities that increase the awareness of the State’s trade services. While there are more than 1,000 trade promotion organizations in California, the Division has succeeded in identifying many projects and partners that complement and enhance the State’s trade development mission. In March 1999, the Division held its first international trade workshop for the State’s trade organiza- tions at the annual conference of the California Association for Local Economic Development (CalED). The Division built upon this relationship and organized similar seminars at CalED in 2000 and 2001. Further, for the past three years (June 1999-2001) the Division has continued to serve as a primary co-sponsor with the US Department of Commerce for the University of Southern California’s premier Asia Pacific Business Outlook Conference. Each year, the Division organizes trade promotion workshops for California companies and partners. In April 1999 and in December 1999, the California Export Finance Board held public meetings that sought input on the State’s trade finance program. These meetings are widely attended by trade organizations throughout the State. In July 1999, the Division organized and hosted a full-day trade development seminar in Sacra- mento, which brought together 20 representatives from Bay Trade, LA Trade, the Bay Area World Trade Center, and the Centers for International Trade Development from Chula Vista, Fresno and San Mateo, among others. A primary focus of the meeting was AB 180, the private-public partner- ship bill that Governor Davis signed in 1999. In December 1999, the Division began planning its Technology 2000 trade mission to Mexico, which took place in September 2000. The organizations that planned and arranged the mission 65 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 11 included the Centers for International Trade Development in Chula Vista, San Bernardino, and Torrance as well as California’s 15 US Export Assistance Centers. Many other organizations were invited to participate. The International Division built on this success by conceptualizing and organizing the Technology 2001 trade mission in September 2001 in cooperation with six Centers for International Trade Development, the US Foreign Commercial Service and other State trade organizations. In February 2000, the Energy Commission approached the Division to sponsor an inbound trade mission from China. The TTCA agreed to co-sponsor the event and the deputy secretary spoke at the 200-person conference. In March 2000, the California State World Trade Commission held its first public meeting since September 1998, which was open to all trade organizations. Since that time, the Commission has held four public meetings, all open to all trade organizations, at which it has outlined the projects and priorities of the Commission and the Division. In June 2001, the Division invited the State’s nine World Trade Centers to Sacramento to discuss the most effective ways to coordinate and promote their respective activities. In July and August 2001, the International Division organized a series of five outreach workshops in San Diego, Newport Beach, San Bernardino, Los Angeles and Sacramento in which it partnered with the US Department of Commerce, the Centers for International Trade Development and local chambers of Commerce in Los Angeles, San Bernardino and Sacramento. In September 2000 and in May 2001, the Division coordinated with California’s trade community a one-week tour of the State’s international trade directors. The directors participated in one-on-one meetings with companies recruited by the Bay Area World Trade Centers, networking receptions with the Los Angeles Chamber of Commerce, the San Francisco Chamber of Commerce, and attended the Quarterly Board Meeting events for the California Chamber of Commerce. The Division partnered with the US Export Assistance Center in Newport Beach to arrange directors’ meetings with medical technology companies and organized an agricultural trade promotion workshop in Sacramento, which included the participation of CDFA, the International Agricultural Center of Tulare and the Center for International Trade Development of Fresno. 8 The Division is committed to continuing its coordination efforts in the manner described. On Redundancy of Services: The International Division is unique among all trade entities in several respects. The Division is the only entity charged with implementing the Governor’s and Legislature’s international trade policy and it is the entity that facilitates State-to-State trade rela- tions. The Division organizes and implements trade missions for the Governor as well as the Legislature. It provides market research and background materials for trade mission participants, identifies businesses within California to attend State-led trade missions and organizes the busi- ness-to business meetings in the foreign countries. 66 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 12 The Division is the State entity that officially interacts with the Consuls General and the Economic Attaches from around the world that seek to develop stronger relationships with California. The Division also represents the business interests of the State on behalf of the Governor when hosting incoming delegations of foreign dignitaries and government officials. In many countries, trade is conducted through governments. Foreign government trade associations look to the Division as their governmental counterpart. No private sector organization can fulfill these functions. In addition, although the Division may sometimes provide the same services as a private organiza- tion, it does so for free or at cost, rather than for profit. While the Bureau criticizes the Office of 9 Export Development for using its own resources to match potential foreign buyers with California exports, rather than sending foreign buyer leads to other entities, the TTCA continues to believe it is appropriate to provide these services to the foreign buyers and California exporters for free, rather than simply act as a referral for fee based organizations. On the Trade Lead System: The Division will continue to discuss the trade lead system with other entities in the context of the regularly held meetings. However, given the Division’s goal to provide 9 free or cost-based services to its clients, and the private organizations’ goal to earn a profit from providing these services, it is unclear that a mutually satisfactory arrangement is obtainable. CHAPTER 3 The TTCA was pleased that the Bureau’s customer service survey found a high level of customer satisfaction with the services provided by the programs of the Economic Development Division. These results are consistent with the informal feedback that the Division’s programs receive and respond to on a regular basis. The Economic Development Division offers an extensive array of diverse programs in the field of economic development. Overall areas of expertise include community economic development planning; downtown and urban neighborhood revitalization; the development of small business and entrepreneurship; site location-related issues involving large and medium-sized industry; marketing the California business climate, and mitigating economic issues impacting the State’s rural areas. Expertise is provided in the form of direct technical assistance and training, through grants and loans, through education forums, in marketing venues such as trade shows and missions, by staffing or sitting on various councils and boards, and by facilitating partnership networks. Services are provided in various formats, including directly by the TTCA staff and coordinated through four regional offices in San Diego, Los Angeles, Sacramento, and the Bay Area; through contracts with third party organizations, through cooperative agreements with other entities (such as the Employment Training Panel), and through informal networked relationships. 67 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 13 The Division’s “clients”—those entities that receive TTCA services—are also varied, and include local government, community groups, employers of all sizes, and local economic development organizations. The Division’s services are currently offered in the specific programs of Enterprise Zones, LAMBRAs, Main Street, the Rural Development Council, corporate site selection through Major Corporate Projects and the regional offices, permit assistance, Military Base Retention and Reuse, Small Business Advocacy, Small Business Development Centers, and small business loans and loan guarantees. In addition, specialty programs have been added in the last two years, such as the reuse of lumber mills, the support of biomass energy generators, and job creation to address the transition from welfare to work. The success of the Division’s programs is directly attributable to the long established practice of annual planning that all programs and offices must complete. Plans and contracts are all designed to be performance-based in their results as much as possible. A primary focus for the Division over the past two years has been on improving the quality of services to small business; increasing the level and depth of services to communities with military installations; and enhancing the net- work of state, local and federal leadership on behalf of rural issues. This emphasis is reflected in all current and pending planning documents and the Division would not be on target without this planning emphasis. Recommendation #1: “The Economic Development Division should take the following actions to improve customer satisfaction: (cid:127) Develop goals and associated targets for customer satisfaction and service for each of its programs. (cid:127) Periodically survey customers to gauge the quality of customer service and to identify areas for further improvement. (cid:127) Evaluate performance by comparing results to targets, and change services as needed.” Response: The TTCA agrees with the importance of gauging customer satisfaction and commits to incorporating such measurements into work plans, as resources allow. The TTCA currently reviews work plans and results through various mechanisms, including routine review and numer- ous and frequent reporting opportunities. The Division will continue to work to improve goal setting and performance monitoring as much as possible within given resources. 68 California Technology, Trade and Commerce Agency Response to Bureau of State Audits Report December 5, 2001 Page 14 Recommendation #2: “[T]he Small Business Loan Guarantee Program should work with the financial development corporations to discuss concerns and determine what actions it should take to resolve them.” Response: The TTCA has demonstrated tremendous program success and has always strived to develop and maintain the best possible relations with all partners, contractors, and clients. The Division is committed to developing and sustaining a positive and productive working relationship with the Financial Development Corporations. However, the TTCA notes that given the current fiscal constraints, it will be unable to address the corporations’ primary concern, namely, increasing their funding base. The TTCA does concur with auditor findings that steps can be taken to develop a more comprehensive statewide marketing effort for the loan guarantee program and commits to do so within allowable resources. CONCLUSION While the TTCA respects that there are varying views and approaches with respect to organiza- tional development, management and strategic planning, the TTCA has adopted a model that we believe is the best means of ensuring that the TTCA’s diverse programs are delivered in the most effective and efficient manner. The TTCA accepts that there is always room for improvement in any process, and appreciates the Bureau’s constructive suggestions for ways to improve the TTCA’s planning. The TTCA is pleased that the six months of intensive scrutiny by the Bureau 1 resulted in findings that customer satisfaction with the TTCA’s services is high, and helpful recom- mendations for improving the TTCA’s planning process and delivery of export development ser- vices in the future. 69 Blank page inserted for reproduction purposes only. 70 COMMENTS California State Auditor’s Comments on the Response From the Technology, Trade and Commerce Agency T o provide clarity and perspective, we are commenting on the Technology, Trade and Commerce Agency’s (agency) response to our audit report. The numbers correspond to the numbers we placed in the agency’s response. 1 To provide perspective on the agency’s statement that we found that its customers are highly satisfied, we need to remind the reader that we concluded that customers of its Economic Devel- opment Division are generally satisfied. We did not review customer satisfaction for the agency’s other divisions. 2 The agency contends that, because it operates in a dynamic environment, its current approach to planning is a better fit for it than what it calls “the traditional model favored by the Bureau.” On page 60, it further states that it does not agree that the strategic planning model “supported by the Bureau” will maximize the agency’s effectiveness. It describes its current planning process, which includes providing a “vision” for the agency and holding meetings with management staff to discuss goals and priorities. Although we recognize that such efforts may be valuable, they do not take the place of the fully- integrated, results-oriented strategic planning process we outline in Chapter 1. Further, although the agency, in various parts of its response, characterizes our findings and recommendations as the process “favored,” “preferred,” or “supported” by us, it is important to recognize that the process we describe in Chapter 1 focuses on basic elements that are commonly recognized as essential to strategic planning. Finally, the fact that the agency operates in a dynamic environment should not preclude the agency from implementing our recommendations. As we state in Chapter 1, strategic planning is a dynamic process that is refined as performance is measured and evaluated, and new information becomes available. This process by its very nature accommodates changes in the environment. 71 3 Although the agency lists many accomplishments, it is difficult to determine the agency’s effectiveness without the context of intended goals and associated targets. These are basic elements of strategic planning, as described in Chapter 1. 4 The agency is concerned with the cost of developing a long- term, agency-wide strategic plan. Such a plan should, however, yield significant dividends by helping the agency to determine which activities are most effective in accomplishing its mission and how it can best allocate its resources. 5 The agency contends that it has always given its highest priority to nominating candidates for its appointed management posi- tions, and it questions whether it is a sound practice to fill staff positions before managers are appointed. However, as noted in Chapter 1, it took the agency up to 13 months to get managers appointed to some foreign offices, and the agency chose to leave staff positions unfilled. Further, at one foreign office we reviewed, we noted that operations slowed drastically because of vacancies. If the lack of staff causes essential services to be compromised, it may be necessary to nominate staff at foreign offices before managers are appointed. 6 The agency has missed our point. We did not say that the agency’s clients are inherently unreliable but rather that clients for some programs, such as the Small Business Loan Guarantee Program, may have an incentive to exaggerate the data they provide to the agency, and thus the data may be considered inherently unreliable. In addition, although state law lists certain requirements that must be met to obtain a loan guaran- tee, it does not preclude the consideration of other factors. Financial development corporation staff told us that they use job creation as one of the factors in determining who gets loan guarantees under the program. Further, the director of Export Finance told us that the loan guarantee decision for that program depends in part on expectations about job creation. During these loan application processes, borrowers are asked to estimate the number of jobs they expect to create or retain, a request that borrowers may perceive as important to securing a loan guarantee. Thus, we remain concerned that borrowers may exaggerate the job creation figures they report to the agency. As such, we believe the agency should verify some of the client supplied information on a sample basis. 72 7 The agency questions whether reporting to the Legislature biennially on its progress in implementing a strategic approach is necessary. As noted in Chapter 1, our findings related to planning are similar to those in our 1996 audit report. Given the agency’s spotty track record in implementing our previous recommendations, we believe that the agency needs to biennially report to the Legislature on its progress. 8 The agency states that the International Trade and Investment Division (International Division) is committed to continuing its coordination efforts, and on page 62 it says that it is committed to increasing the level of coordination with other entities. Unfortunately, it does not specify what it plans to do in response to our recommendations on holding regular meetings to discuss goals and operations, analyzing the service delivery system, or establishing agreements with other entities. We look forward to learning more about the steps the agency is taking in its 60-day, 6-month, and 1-year responses to the audit. 9 The agency points out that the International Division may sometimes provide the same services as a private organization, but it draws a distinction, saying that it offers services for free or at cost rather than for profit. Additionally, it states that although it will discuss the trade lead system with other entities, it questions whether a mutually satisfactory agreement is obtainable in light of differing financial goals. In Chapter 2, however, we note instances where it provides services similar to those of federal and state agencies, as well as private organizations. In addition, interviews with governmental and private entities indicate that they sometimes offer services for free, sometimes at cost, and sometimes for profit. As noted in Chapter 2, we recognize entities may have different missions, focuses, and policies on charging for services. We believe, however, that it is worthwhile to analyze the service delivery system and to discuss reducing service redundancies with other entities. Further, we continue to believe that it is important to discuss the trade lead system with other entities and consider whether sharing leads to a greater extent would be beneficial. 73 cc: Members of the Legislature Office of the Lieutenant Governor Milton Marks Commission on California State Government Organization and Economy Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press 74