Summary

California Trade and Commerce Agency: It Has Not Demonstrated Strong Leadership for the Manufacturing Technology Program, Collected Data Necessary to Measure Program Effectiveness, or Ensured Compliance With Program Requirements December 1999 99025 The first copy of each California State Auditor report is free. Additional copies are $3 each. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TDD (916) 445-0255 x 216 OR This report may also be available on the World Wide Web http://www.bsa.ca.gov/bsa/ Permission is granted to reproduce reports. C S A ALIFORNIA TATE UDITOR KURT R. SJOBERG MARIANNE P. EVASHENK STATE AUDITOR CHIEF DEPUTY STATE AUDITOR December 21, 1999 99025 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As required by the Budget Act of 1999, the Bureau of State Audits presents its audit report concerning the effectiveness of the Trade and Commerce Agency’s Manufacturing Technology Program. This report concludes that the Trade and Commerce Agency’s Office of Strategic Technology has not taken a strong leadership role in managing the Manufacturing Technology Program. Specifically, the Office of Strategic Technology has failed to establish statewide goals and performance measures for the program. Consequently, we were unable to assess the effectiveness of the program. It has also not exercised sufficient oversight to ensure that the Manufacturing Technology Program centers comply with state funding requirements. In addition, this report concludes that the centers generally appear to have spent public funds appropriately and stayed within current statutory limitations pertaining to lobbying. Finally, this report concludes that although the centers’ administrative costs are fairly low, a closer look at how center staff spent their time indicates that true administrative costs may be higher. Respectfully submitted, KURT R. SJOBERG State Auditor BUREAU OF STATE AUDITS 555 Capitol Mall, Suite 300, Sacramento, California 95814 Telephone: (916) 445-0255 Fax: (916) 327-0019 CONTENTS Summary 1 Introduction 5 Audit Results 11 Recommendations 24 Responses to the Audit California Trade and Commerce Agency 27 California Manufacturing Technology Center 29 Corporation for Manufacturing Excellence 31 San Diego Manufacturing Extension Center 33 SUMMARY RESULTS IN BRIEF T he Manufacturing Technology Program (program), a program administered by the Trade and Commerce Agency’s Office of Strategic Technology (OST), provides Audit Highlights . . . funding to public agencies and nonprofit organizations (centers) that assist small- and medium-sized manufacturing companies The Trade and Commerce within California. Before the program was established, the Agency has not taken an National Institute of Standards and Technology (NIST) funded active leadership role over the Manufacturing centers through a similar federal program. Since 1996, the Technology Program. state and federal programs have jointly funded three centers Specifically, it has not: in California. (cid:1) Developed a comprehensive statewide strategy. OST has not taken a strong leadership role in overseeing this (cid:1) program. It has failed to develop statewide goals for the pro- Collected consistent and gram, set reporting standards for the centers, closely monitor meaningful data necessary to measure the centers’ progress toward meeting the goals in their grant program effectiveness. agreements, and ensure that the centers are meeting funding (cid:1) requirements. Although the centers are fulfilling their mission of Ensured that centers meet serving small- and medium-sized manufacturers, insufficient all program requirements. data exists to evaluate the effectiveness of the program. Further, OST’s lack of comprehensive oversight could adversely affect future funding for the centers and ultimately service delivery to the manufacturing companies the program is intended to serve. Ensuring that California’s small- and medium-sized manufactur- ers stay competitive is important because these companies have a strong impact on the economy: In 1996 alone, manufacturing contributed $134 billion to the gross state product and employed one in every eight workers in the State. Additionally, according to the National Research Council, small- and medium-sized manufacturers represent more than 98 percent of all manufacturing firms in the United States; however, these companies often lack the resources necessary to improve their manufacturing performance. Even though ensuring that these manufacturers remain competi- tive is important, it is difficult to determine how effective the program is at this task. OST has not yet established statewide goals and performance measures for the program or gathered the information needed to evaluate program effectiveness. Although OST sets goals for the individual centers and requires them to C A L I F O R N I A S T A T E A U D I T O R 1 submit quarterly progress reports, it has not given the centers a reporting format. As a result, the centers do not collect consistent data or report on consistent performance measures. Additionally, data in the reports is not documented or is inaccu- rate. We had similar concerns with the information in the 1999 Report to the Legislature. Thus, it is also of little use in assessing the program. Studies claiming to show the success of the program should be viewed with caution. For example, using an economic analysis performed by NIST, OST claimed that the program was responsible for increasing the value of goods and services produced in California by $64.1 million; increasing personal income by $43.6 million; raising an additional $10 million in federal, state, and local taxes; and adding 873 jobs. We question the reliability of these numbers, however, because of our concerns over the methods used to collect the data. Other studies purporting to show program success do not specifically address California’s centers, so they do not provide unequivocal evidence of the effectiveness of California’s program. Besides failing to set statewide goals and gather sufficient data to assess the program, OST has not ensured that the centers comply with state funding requirements. For example, one center overstated its share of program costs for two grant periods by more than $216,000 because it incorrectly included funds from other federal programs. OST also did not ask centers to correct reporting and performance deficiencies. Indeed, despite poor performance, one center received an increase in funding of $200,000. As required by the Budget Act of 1999, we reviewed additional aspects of the centers’ operations: whether they appropriately spent state and federal funding for travel, lobbying activities, and the amount of their administrative costs. Our review found that, in general, the centers spent public funds appropriately, although two centers did not in a few instances. One center used $1,300 in federal funds to purchase alcohol and paid more than $120,000 to a consultant over a 32-month period after its written agreement with the consultant had expired. Another paid more than $5,100 in unauthorized bonuses to external consultants. Regarding lobbying activities in which two centers were involved, we determined no current statutory limitations were violated and the centers spent no federal funds on these 2 C A L I F O R N I A S T A T E A U D I T O R activities. Lastly, although the centers’ administrative costs appear fairly low, a closer look at how center staff spent their time indicates that true administrative costs may be higher. RECOMMENDATIONS To ensure that the program meets California’s needs, OST should do the following: (cid:1) Develop a statewide strategy that identifies its role, the program goals and objectives, and specific performance measures and milestones that it can use to assess the progress of the program in meeting California’s needs. (cid:1) Develop a standard reporting format and require that the centers’ quarterly and legislative reports address all perfor- mance measures. (cid:1) Use the quarterly reports to monitor center performance and require centers to promptly address deficiencies. (cid:1) More closely monitor the centers’ compliance with program funding requirements and consider reducing grant amounts when centers do not comply. (cid:1) Evaluate the propriety of the time centers spend on adminis- trative tasks. Finally, the centers should reimburse the federal government for disallowed costs. AGENCY COMMENTS The California Trade and Commerce Agency agrees with our recommendations and has begun taking corrective actions. The three centers agreed with our conclusions. (cid:1) C A L I F O R N I A S T A T E A U D I T O R 3 Blank page inserted for reproduction purposes only. 4 C A L I F O R N I A S T A T E A U D I T O R INTRODUCTION CALIFORNIA’S TRADE AND COMMERCE AGENCY T he California Trade and Commerce Agency (agency) works to promote economic development, create jobs, and retain businesses in the State. Its Office of Strategic Technology (OST) contributes to the mission of creating and retaining jobs by assisting California companies improve their technology and encouraging the development of new, commercially viable products and services. One of the programs OST administers is California’s Manufacturing Technology Program (program). THE MANUFACTURING TECHNOLOGY PROGRAM The Manufacturing Technology Program provides matching grants and technical assistance to nonprofit organizations and public agencies in California that assist manufacturing enterprises. The program was established in 1993 but was not funded until 1996. Since then, OST has awarded approximately 97 percent of its program funds, which totaled $6.7 million in fiscal year 1997-98, to organizations that also receive funding from the federal Manufacturing Extension Partnership program (federal partnership program). Although the specific require- ments of the two programs differ, both programs strive to improve manufacturing. THE FEDERAL TECHNOLOGY PROGRAM The National Institute of Standards and Technology (NIST) administers the federal partnership program, which was established in 1988. The primary mission of the federal partnership program is to provide technical assistance, such as appropriate technologies and training, to small- and medium-sized manufacturers, enabling them to become more competitive. The federal partnership program distributes funding for this assistance to more than 70 centers located throughout the United States. C A L I F O R N I A S T A T E A U D I T O R 5 The centers, which are often nonprofit organizations, offer a variety of services to assist small- and medium-sized manufactur- ers. They frequently partner with other business-assistance providers, such as Small Business Development Centers, community colleges, and federal laboratories, to help companies solve individual manufacturing problems, obtain training for their employees, create marketing plans, and upgrade their equipment and computers with the latest technology. The centers focus on assisting small- and medium-sized manu- facturers, defined as businesses with fewer than 500 employees, because research by the National Research Council and others has indicated that these companies often lack the resources necessary to improve their manufacturing performance. PROGRAM FUNDING REQUIREMENTS Both the state program and the federal partnership program require the centers to obtain a share of their funding from other sources, commonly referred to as matching. For example, OST requires the centers to obtain one-third of their funding from sources other than the agency and the federal government. To meet this obligation, the centers can obtain funds from other state or private entities and charge fees for their services. Their share can also include the value of in-kind contributions, such as donated services, equipment, and space. CALIFORNIA’S CENTERS The State and federal programs jointly fund three centers in California: the California Manufacturing Technology Center in Hawthorne, the Corporation for Manufacturing Excellence in Fremont, and the San Diego Manufacturing Extension Center. Table 1 shows the current grants the State and federal programs awarded to these centers. The centers help companies solve specific business and manufacturing problems such as product design, plant modernization, production control, product quality, workforce development, materials requirement plan- ning, and business planning. Although each center operates independently of the others, some collaboration does occur. 6 C A L I F O R N I A S T A T E A U D I T O R TABLE 1 Awards for Grant Periods Ending in 1999 National Institute California Trade and of Standards and Center Commerce Agency Technology Manufacturing Manufacturing Extension Partnership Technology Program Program California Manufacturing Technology Center (CMTC)* $4,700,000 $10,989,444 Corporation for Manufacturing Excellence (Manex) 1,140,000 1,666,666 San Diego Manufacturing Extension Center (SanMEC) 700,000 1,425,000 *Amount from the federal partnership program includes three grants. The California Manufacturing Technology Center Established in 1992 and incorporated as a nonprofit organiza- tion in 1994, the California Manufacturing Technology Center (CMTC) currently has more than 130 employees. Headquartered in Hawthorne, CMTC serves a target population of 53,000 small- and medium-sized manufacturers through seven regional outreach centers: Los Angeles, San Fernando/ San Gabriel, Inland Empire, Fresno, Sacramento, Ventura, and Orange County. CMTC assists a complete range of smaller manufacturers in its targeted industries of aerospace, medical technology, and electronics. It also targets the apparel, food processing, furniture, and automobile aftermarket industries that are of particular importance to the local regions it serves. The largest and most established center in California, CMTC, reported it served 3,028 clients in 1998. The Corporation for Manufacturing Excellence The Corporation for Manufacturing Excellence (Manex), incorporated as a nonprofit organization in November 1995, currently has 26 employees. Located in Fremont, Manex serves a population of approximately 19,700 manufacturers in 10 Bay Area counties. Manex has targeted five key industry clusters as its primary market: apparel, biotechnology, electron- ics, food, and metal and machine shops. Manex provides C A L I F O R N I A S T A T E A U D I T O R 7 consulting and other services for business development, manu- facturing and technology operations, workforce development, and access to other public resources and organizations. Accord- ing to Manex, it served 322 clients in 1998. The San Diego Manufacturing Extension Center The San Diego Manufacturing Extension Center (SanMEC), incorporated as a nonprofit organization in 1996, has 15 employees. Located in San Diego, SanMEC serves 4,600 manufacturers in the San Diego area. SanMEC has targeted six industry sectors as its primary market: electronics, telecommunications, instruments and medical devices, recreational equipment, industrial machinery, and fabricated metal and related plastic parts. The center assists manufacturers with projects and workforce development. It also offers targeted seminars and group activities related to supply-chain development and common manufacturing issues. According to SanMEC, it served 271 clients in 1998. Partners of the Centers The centers partner with many other organizations within their regions to provide more services to small- and medium-sized manufacturers. For example, the three Regional Technology Alliances in the State help entrepreneurs and new companies find investors and other financing options. Some of these companies may later be served by the centers. Other partners include the Centers for Applied Competitive Technologies, part of the community college system, which provide training semi- nars for the centers’ clients. SCOPE AND METHODOLOGY The Budget Act of 1999 required that we conduct a program audit of the Trade and Commerce Agency’s Manufacturing Technology Program (program). Specifically, we were asked to assess the program’s effectiveness; the appropriateness of the centers’ expenditures for lobbying, travel, and hotel costs; and the ratio of administrative costs to service delivery. To understand state and federal requirements for the program, we reviewed applicable state and federal laws, regulations, and grant agreements. We interviewed staff at both the State and federal levels to better understand reporting requirements placed 8 C A L I F O R N I A S T A T E A U D I T O R on the centers as well as their roles in managing the programs. We visited each center and interviewed management about its role in the programs. We also contacted partners of the centers, such as the Regional Technology Alliances and community colleges, and reviewed formal agreements between these organi- zations to determine the extent of their relationships with the centers. To assess program effectiveness, we reviewed OST’s role in providing oversight and leadership to the centers, including its efforts to set goals and measure progress for itself, the centers, and the program as a whole. During our visits to the centers, we examined how they set client fees, their compliance with reporting requirements, their progress toward meeting grant agreement goals, and the accuracy of quarterly reports and the 1999 Report to the Legislature. Further, we evaluated the extent of each center’s collaboration with partners, with other Califor- nia centers, and with the State. We also analyzed data collected at the centers to determine whether the centers were adhering to their mission of serving small- and medium-sized manufac- turers. To determine the appropriateness of center expenditures, we reviewed a sample of expenditures for travel, hotels, lobbying, and other purposes at each center for state grant years ending in 1997, 1998, and 1999, and we assessed compliance with relevant state and federal laws and regulations. We also reviewed the lobbying activities of the centers and their compliance with current restrictions. To assess the ratio of administrative costs to service delivery, we reviewed the centers’ reported administrative costs and compared them to the total operating expenditures for the centers. Additionally, we compiled data from the centers, which summarized how staff spend their time. (cid:1) C A L I F O R N I A S T A T E A U D I T O R 9 Blank page inserted for reproduction purposes only. 10 C A L I F O R N I A S T A T E A U D I T O R AUDIT RESULTS SUMMARY T he Trade and Commerce Agency’s (agency) Office of Strategic Technology (OST) has not taken an active leader- ship role in managing the Manufacturing Technology Program (program). Specifically, it has not set statewide goals for the program. In addition, although it collects some data from the centers, it has failed to standardize the manner in which the centers report the data, so this information is inconsistent and cannot be used to assess program effectiveness. An economic analysis performed by the National Institute of Standards and Technology (NIST) indicates that the program has had a positive impact. However, because of flawed data collection practices, NIST’s analysis must be viewed with caution. Finally, OST has not adequately ensured that the centers comply with state and federal program funding requirements. OST’S ROLE IS EVOLVING Since 1996, OST and NIST have jointly funded three centers in California. The purpose of these centers is to improve the com- petitiveness of small- and medium-sized manufacturers by providing consulting and other services that these manufactur- ers might otherwise be unable to obtain. Because of the impact small- and medium-sized manufacturers have on the economy, ensuring that they remain competitive is important to the State. In 1996, manufacturing contributed $134 billion to the gross state product and employed one in every eight workers in the State. In 1993, the National Research Council reported that small- and medium-sized manufacturers represented more than 98 percent of U.S. manufacturing compa- nies. OST Has Demonstrated Minimal Leadership Despite the importance of the program to the State, OST’s management of the program can be described as minimal. Until recently, its primary role has been to review grant applications and award program funds. Its oversight of the program has been limited to having staff sit on the centers’ boards, if asked to do C A L I F O R N I A S T A T E A U D I T O R 11 so, and reviewing the invoices and reports received from the centers. Although its grant agreements establish goals and reporting requirements for the individual centers it funds, OST has not established statewide goals or performance measures for the program. As a result, it has gathered little information with which to evaluate the effectiveness of the State’s program, as we will discuss later in this report. Ideally, before awarding any money to the centers, OST would Although it has awarded have assessed the State’s needs and developed a statewide funds, OST has not strategy for satisfying those needs. The strategy would articulate established statewide the program’s goals, identify specific program objectives to goals or performance achieve these goals, and establish a common set of performance measures for the measures for the centers. Such a strategy would permit OST to Manufacturing clarify its role and direction and thus establish a stronger pres- Technology Program. ence in the program. It would also help OST direct program funds where they best address the State’s needs as well as enable OST to anticipate and accommodate future issues, opportunities, and problems. Finally, performance measures would allow OST to monitor progress towards its goals and objectives and con- tinually evaluate the success or failure of the program. Instead of taking an active role in setting the direction for the program, OST has relied on NIST to do so. This approach makes sense if the State’s goals and the goals of NIST are the same. After all, by the time the State began funding the program, NIST had already established goals and objectives, a clear reporting protocol, and a stringent center review process. However, OST must evaluate California’s needs and articulate the program’s goals before it can be sure they indeed are the same. Recent Legislation Expands OST’s Role Recent legislation requires OST to take a more active leadership role in the program by mandating that it develop a Small Manu- facturing Competitiveness Strategy and specific program goals by September 1, 1999. As of October, however, OST had released only a draft of Phase I of this strategy. Phase I, the first of three phases, outlines the process for developing the competitiveness strategy and policy changes within the agency, the general goals of small manufacturing companies, and planned efforts for the program. Unfortunately, OST still has not identified its own management goals, set specific program objectives to achieve these goals, or developed a common set of performance measures for the centers. Currently, OST estimates that it will complete its competitiveness strategy by April 30, 2000. 12 C A L I F O R N I A S T A T E A U D I T O R INSUFFICIENT DATA EXISTS TO MEASURE PROGRAM PERFORMANCE The OST has not collected enough consistent, meaningful data to measure program performance. Even though other studies indicate the program is successful, we could not rely on these studies to evaluate the effectiveness of the State’s program. Nonetheless, we found that the centers serve clients with fewer than 500 employees. Quarterly Reports Are Not Sufficient to Analyze Program Effectiveness Due to OST’s poor data-reporting guidelines and oversight, there are substantial problems with the data that it collects to measure the program’s performance. As a result, the quarterly reports and the 1999 Report to the Legislature are not useful tools to analyze the program’s performance. Each grant award contains a specific set of goals for the center to meet with the funding provided, called the “scope of work.” The centers are to use quarterly reports to update OST periodically regarding progress on their scope of work. According to the grant agreements, each center is to “submit quarterly project and financial status reports,” which shall include “information on the progress of this Grant in terms of the . . . scope of work.” We found substantial problems with the centers’ quarterly The quarterly reports reports. They are generally inconsistent, unsupported, and and the 1999 Report to inaccurate. They are also incomplete because OST has not the Legislature include required the centers to report on progress made toward all of the inconsistent, inaccurate, goals laid out in their scope of work. Therefore, it is quite diffi- and unsupported data. cult to periodically assess the centers’ progress toward meeting their goals by the end of the grant year. In addition to allowing the centers to submit incomplete reports, OST has not designed a standard reporting format or mandated that each center report on the same goals. This lack of consistency has resulted in the centers’ reporting on different performance measures. Consequently, the centers’ reports are not comparable, making cross-center or collective analysis difficult, a shortcoming that greatly inhibits the analysis of overall program performance. C A L I F O R N I A S T A T E A U D I T O R 13 We further found that the centers could not support much of the data in their reports. Therefore, we were not able to assess the accuracy of this data. For example, one of the three centers, SanMEC, could not provide documentation for the amounts it reported in years preceding the current calendar year. In addi- tion, SanMEC could not support many of the figures for this calendar year. Specifically, it could not provide documentation supporting its reported project revenue, average client size (by Because the three centers’ number of employees or by revenue), or type of service provided reports are inconsistent, to clients. This large amount of unsubstantiated data calls into cross-center analysis is question the usefulness of these reports. difficult, inhibiting an evaluation of overall Our review also showed that the figures in the quarterly reports program performance. for all three centers are not completely accurate. There seems to be no systematic intent to misrepresent their performance; instead, the inaccuracies generally seem to be simple mistakes. For example, CMTC reported that it had booked 85 new projects in November 1998, but its detailed records showed 113 new projects. A project is any type of technical service or assessment provided to a client that requires more than eight hours to complete. The error in reporting occurred because CMTC included new projects for only two of its three grants. Based on the many inaccuracies, our review shows that the centers need to place more importance on the accuracy of their reports to OST. We have similar concerns about the data in OST’s report for the Legislature highlighting the performance of the centers. Like the data in the centers’ quarterly reports, a significant amount of the data in the 1999 Report to the Legislature was inaccurate. For example, the report stated that SanMEC had completed 70 projects. However, the detailed documentation shows that SanMEC completed only 41 projects while its part- ners completed 28 projects. Similarly, the data was either not documented or the documentation did not support the amounts reported. Therefore, the true usefulness of this report is also suspect. Studies Purporting Program Success Should Be Viewed With Caution Despite OST’s failure to compile consistent and meaningful data, studies conducted and data collected by the NIST, United States General Accounting Office (GAO), and others indicate that the program is successful. NIST collects data on the federal partnership program’s economic impact on sales, job creation, 14 C A L I F O R N I A S T A T E A U D I T O R and inventory levels. However, because we have concerns regarding the methods NIST uses to collect the economic impact data, and because other studies do not specifically relate to California, we feel that the data cannot be used to draw defini- tive conclusions regarding the success of the State’s program. In its March 1999 Report to the Legislature, OST claimed that the program had a positive effect on California’s economy. Based on the results of an economic analysis performed by NIST, the report stated that in 1998, the program increased the value of goods and services produced in California by $64.1 million; increased personal income by $43.6 million; increased federal, state, and local taxes by $10 million; and created 873 jobs. However, we have several concerns about NIST’s data collection methods that lead us to question the reliability of this economic analysis. NIST conducts an annual telephone survey to determine the economic impact of projects funded by the federal partnership program one to two years after project completion. The survey is designed to question all companies receiving project assistance from centers regarding monetary impacts, the number of jobs created and retained, and the overall satisfaction with services received. One of our concerns is that centers are allowed to “coach” their clients for the survey. As the time for the survey approaches, the Although a national centers contact their clients and help prepare them for the study indicates that the survey questions. This may affect the independence of the program benefits the survey results. The centers indicated to us that preparing clients economy, due to concerns for the survey is necessary to get fair results. They stated that in about data collection the past, clients have been inappropriately excluded from later methods, we question the questions because they responded to the first question incor- reliability of the analysis. rectly. The person conducting the telephone survey does not assist the respondents by answering questions or eliciting addi- tional information to confirm responses. Even though we did not determine whether the centers inappropriately coached their clients, we are concerned that responses may have been skewed. The centers also have the authority to eliminate clients from the survey process by labeling them as “Do Not Survey.” According to its regional manager, NIST allows them to do this to ensure that the centers report the names of all clients served, even those who do not want to be surveyed. The regional manager stated that clients may not want to participate in the survey for C A L I F O R N I A S T A T E A U D I T O R 15 confidentiality reasons. For example, manufacturers obtaining assistance with inspections to identify potential problems with E. coli bacteria would not want it known that they had sought such assistance for fear of ruining their business reputation. Despite the intended use of the option not to survey some clients, the regional manager stated that centers across the country have begun to use the option excessively and for purposes other than confidentiality. NIST does monitor the number of clients marked “Do Not Survey” and has identified a desirable level as no more than 2 percent to 4 percent of clients for large centers; the national average is currently 19 percent. The three centers in California are well under the national average, but one is above the desirable range. Specifically, for calendar year 1998, CMTC requested that 5 percent of its clients not be surveyed, Manex did so for 4 percent of its clients, and SanMEC did so for 2 percent of its clients. We are concerned that the ability of centers to eliminate clients from the survey in this way may skew the data obtained from the survey. Although such omissions are likely to understate the economic impact data, their effect on other survey data, such as client satisfaction, is unknown. Another concern is that neither NIST nor OST has assessed the reliability of data from the centers’ clients. The data collected There is no process regarding economic impact is taken directly from the clients as to verify economic fact, without any verification of its accuracy. Understandably, impact data collected this data would be difficult to confirm because there are many from clients. factors that may affect improvements in a business. However, without any process for checking the data, and the possibility that the data may be skewed because centers’ coach their clients or eliminate them from the survey, the validity of the data is questionable. We found two other studies purporting to show program success, although neither specifically addresses California’s centers. The GAO surveyed a sample of manufacturers receiving services in 1993 from centers funded by the federal partnership program to obtain their views regarding the impact of the centers’ services on business performance. Of those surveyed, 73 percent reported that the centers had positively affected business performance. The GAO did not verify either positive or negative impacts reported by manufacturers. Finally, a study appearing in the Winter 1999 issue of the Journal of Policy Analysis and Management concluded that manufacturing businesses receiving services between 1987 and 1992 from 16 C A L I F O R N I A S T A T E A U D I T O R centers funded by the federal partnership program enjoyed between 3.4 percent and 16 percent more growth in labor productivity than those that did not. Centers Are Accomplishing Their Mission to Serve Small- and Medium-Sized Manufacturers Even though the program lacks reliable reports and we have concerns regarding the data OST uses to determine the program’s effect, we found that the centers are fulfilling their mission to serve clients with fewer than 500 employees. Because we could not use OST data to effectively measure the overall performance of this program, we performed a limited analysis of program performance using data collected directly from the centers. According to our analysis, the centers predominantly serve small- and medium-sized manufacturers. Table 2 shows that the centers are primarily serving clients with fewer than 100 employees. Our analysis also shows that the centers have begun to serve more clients with over 500 employees; however, these clients continue to make up only a small percentage of the total number. TABLE 2 Percentages of Projects Completed by California’s Centers Client Size (Number of Employees) Center Year 0-100 101-250 251-500 >500 Corporation for Manufacturing Excellence (Manex) 1997 81% 13% 6% 0% 1998 79 19 2 0 1999 87 8 3 2 California Manufacturing Technology Center (CMTC) 1997 77 18 5 0 1998 73 20 6 1 1999 67 24 7 2 San Diego Manufacturing Extension Center (SanMEC) 1997 90 10 0 0 1998 83 2 13 2 1999 78 1 12 9 Average Percentage 1997 78 17 5 0 1998 74 19 6 1 1999 71 21 6 2 C A L I F O R N I A S T A T E A U D I T O R 17 C A L I F O R N I A S T A T E A U D I T O R A L I F O R N I A S T A T E A U D I T O R OST’S LACK OF OVERSIGHT MAY HAVE A NEGATIVE IMPACT ON CENTERS Not only have OST’s efforts to collect consistent and meaningful data been ineffective, its oversight did not reveal the centers’ failure to comply with funding requirements. Noncompliance with funding requirements exposes the centers to a risk of reduced funding. Further, OST did not formally comment on— or impose consequences for—the centers’ deficiencies in report- ing and performance. When centers submit incomplete or inaccurate reports, OST lacks important information to monitor the program and detect deficiencies. Deficiencies in center performance that are not addressed ultimately affect services to the manufacturing companies that the program is intended to serve. Two Centers Failed to Comply With Some Program Funding Requirements Although state and federal regulations require the centers to meet certain funding requirements, two of the centers, Manex and SanMEC, did not always do so. These centers also did not comply with some requirements for documenting or recording services donated toward their share of program costs. Under the State’s program, the centers’ grant agreements specify that they must obtain 33 percent of their funding from sources other than the agency and the federal government. The centers can include in-kind contributions, or the value of services donated by other entities, in their share of the program funds. Federal regulations allow the donation of professional services but specify that the rates must be documented and consistent One center incorrectly with reasonable market rates. Federal regulations also limit included federal funds in-kind contributions to 50 percent of the centers’ share. in its share; another exceeded the limit on As Table 3 indicates, Manex fell short of obtaining one-third in-kind contributions. of its funding from other sources for all three grant periods reviewed. For grant periods July 1997 through August 1998 (1998) and September 1998 through August 1999 (1999), it incorrectly included federal funds in its share of the program’s funding, inflating its overall contribution by $86,422 and $130,066, respectively. Manex claims it was not aware that federal funds from agencies other than NIST could not be used. Although OST received reports on the matching funds, when we brought the matter to its attention, the office seemed surprised that Manex had included federal sources. 18 C A L I F O R N I A S T A T E A U D I T O R C A L I F O R N I A S T A T E A U D I T O R C A L I F O R N I A S T A T E A U D I T O R TABLE 3 Two Centers Failed to Comply With All Program Funding Requirements Grant Amount Under 33 Percent Did Not Comply With Center Period Match Requirement In-Kind Requirements Corporation for Manufacturing Excellence (Manex) 1997 9% ✖ 1998 2 ✖ 1999 1 ✖ California Manufacturing Technology Center (CMTC) 1997 ✖ 1998 ✖ 1999† ✖ San Diego Manufacturing Extension Center (SanMEC) 1997 26 ✖ 1998 7 ✖ 1999 N/A* N/A* †We did not analyze one of three grants received because the grant period has not yet ended. * This grant is not included because the grant period has not yet ended. Additionally, many of Manex’s board members used a standard rate of $1,000 per day when they contributed their services for attending board meetings. Because Manex did not document the basis for valuing the board members’ donated services, the accuracy of these in-kind contributions is unclear. Altogether, board members’ contribution of services totaled $92,297, $26,950, and $42,715 for grant periods 1997, 1998, and 1999, respectively. These contributions represent 7 to 22 percent of total in-kind amounts for the three grant periods. A second center, SanMEC, did not meet its requirement to obtain one-third of its funding from other sources during both its first grant period, calendar year 1997 (1997), and its second grant period, February 1998 through December 1998 (1998). Our review of SanMEC’s program funding revealed that SanMEC did not comply with the federal regulations that limit in-kind contributions to 50 percent of the center’s share of program funding. We recalculated its share for both grant periods, using the allowable 50 percent, and determined that SanMEC obtained only 7 percent of its funding from other sources during the first grant period and 26 percent during the second grant period. Both amounts fall below the required 33 percent funding share. C A L I F O R N I A S T A T E A U D I T O R 19 The third center, CMTC, records its in-kind contributions in the same fiscal year in which it receives the report summarizing contribution amounts. For example, El Camino Community College provided more than $340,000 of in-kind services between June 1997 and July 1998; however, because CMTC received the final report from the college in October 1998, it included those services in its share of funding for the grant period September 1998 through September 1999 (1999). According to the Statement of Financial Accounting Standards No. 116, nonprofit organizations should account for contribu- tions of services in the period the services are received. Because CMTC consistently records its in-kind contributions when they are reported rather than when they are received, this noncom- pliance has thus far not affected its ability to meet its matching requirements. Noncompliance with funding requirements puts the centers at risk of losing some funding. Both NIST and OST have the authority to reduce a center’s funding when it does not provide its share of the program funds. Also, when centers do not document the basis for valuing donated services, their unsub- stantiated in-kind contributions may be disallowed, further reducing funding. OST Does Not Provide Formal Comments on or Impose Consequences for Poor Reporting and Performance Although OST receives quarterly reports on center performance, it does not provide formal comments on incomplete reports. We also found no evidence that OST requires the centers to explain poor performance or their planned corrective actions. Further, two centers indicated to us that OST provides minimal feedback or oversight. The lack of prompt formal response Despite continuing poor indicates that OST does not hold the centers accountable for performance, OST reporting and performance deficiencies, so they may continue to awarded one center a perform poorly. $200,000 increase in funding. For one center reviewed, SanMEC, when we compared its reported performance to milestones set forth in its scope of work, we found that it met only half of its service delivery and revenue goals for 1998. Failure to meet performance measures, such as service delivery goals, indicates that the center is assist- ing fewer clients than projected. Despite this continuing poor performance, OST increased SanMEC’s funding by $200,000 for the 1998-99 grant period. Our review of SanMEC’s performance so far in 1999 indicates that it will again fall short of its revenue 20 C A L I F O R N I A S T A T E A U D I T O R goal. Although SanMEC provided planned corrective actions for past performance problems in its annual proposal to OST for new funding, OST missed an important opportunity to address performance problems earlier: It could have required SanMEC to take corrective action immediately following the poor perfor- mance identified in its quarterly reports. In a second example, CMTC did not report periodically as required on its progress toward completing many of the tasks outlined in the grant agreement, including one of its three performance measures. Instead, CMTC gave a more complete report on its progress after the end of the grant year. There is no evidence to indicate that OST took exception to CMTC’s lack of complete reporting during the year, even though OST did not have information it needed to monitor progress effectively. If OST monitored performance shortfalls more closely, it could analyze problems and require corrective action more promptly, thus helping the centers improve their services to the manufacturers. CENTERS GENERALLY SPENT PROGRAM FUNDS APPROPRIATELY; HOWEVER, TRUE ADMINISTRATIVE COSTS MAY BE HIGH Although the centers generally spent public funds appropriately, we found instances when two centers did not. On the other hand, the two centers engaged in lobbying activities did not use public funds for these activities or violate current statutory limitations. Additionally, despite fairly low administrative costs, the high levels of staff time spent in administrative functions raises questions about the true costs of administration. Centers Spent Some Public Funds Inappropriately Our review of expenditures indicates that the centers generally spent public funds in compliance with state and federal regula- tions. However, we noted some instances when they did not. Specifically, Manex inappropriately used $1,170 in public funds to buy alcohol for its kickoff reception and $130 in public funds to buy wine for another event. Federal regulations prohibit the use of public funds for this purpose. This center also paid an outside consultant more than $120,000 for services between June 1996 and January 1999, after its written agreement with the consultant had expired. C A L I F O R N I A S T A T E A U D I T O R 21 In addition, SanMEC inappropriately used more than $5,100 in public funds to pay unauthorized bonuses to three consult- One center used over ants. Before receiving approval from NIST or OST, SanMEC $1,000 in public funds entered into written contracts with the consultants agreeing to pay for alcohol and to pay them a bonus for obtaining clients. The bonus increased another used public according to the level of revenue from the clients. NIST funds to pay $5,100 in subsequently informed SanMEC that these bonuses were not unauthorized bonuses. allowable. According to the regional manager at NIST, it was inappropriate for SanMEC to pay these bonuses from any funds approved in its budget for the program, including funds generated from client fees. The regional manager stated that a program that pays consultants a bonus for essentially no additional work is a significant deviation from the program that NIST approved. During the time of this audit, SanMEC discontinued its practice of paying bonuses. Lobbying Efforts Do Not Violate Statutory Limits Even though some centers used public funds inappropriately, these uses did not include paying for lobbying activities. Two of the three centers, Manex and CMTC, engaged in lobbying activities for the three-year period we reviewed; however, they did not violate statutory limits. Although federal law prohibits the centers from using public funds to pay for lobbying activi- ties, they are permitted to use other funding as long as lobbying does not become a substantial portion of their activities. According to the California Franchise Tax Board, which monitors expenditures of nonprofit organizations, it becomes concerned when lobbying expenditures exceed 5 percent of the operating costs. Neither of the centers that engaged in lobbying used federal funds to pay for these activities. Instead, the centers used fees generated from client services. Manex contracted with a registered lobbyist and a governmental consulting firm to advocate legislation on its behalf and maximize Manex’s state funding. The center paid the two consultants $38,663, less than 1 percent of its operating costs during calendar year 1998. In addition, CMTC had a vice president of government relations with responsibilities that included building relationships with federal and state officials to develop strong support for govern- ment funding. CMTC spent less than 1 percent of its operating costs on this individual’s salary. 22 C A L I F O R N I A S T A T E A U D I T O R Administrative Hours Are Higher Than Costs Indicate Although the percentage of administrative costs with respect to total expenditures appears reasonable, a closer look at the amount of time center staff spend on administrative activities raises questions about the true costs of administrative time. Only two of the three centers, CMTC and Manex, currently track administrative costs, which typically include the salaries, ben- efits, and operating expenditures for staff not directly linked to delivering client services. For example, both CMTC and Manex include salaries and benefits for their executive staff and accounting and finance staff in administrative costs. In contrast, they classify salaries and benefits for staff that work directly with the manufacturing companies as client service costs. The third center, SanMEC, does not track administrative costs separately. For the fiscal year ending June 30, 1999, administrative costs made up 18.6 percent and 26.5 percent, respectively, of CMTC’s and Manex’s total expenditures. However, comparing expendi- Costs for lobbying tures in the administrative category to overall expenditures may activities comprised less be misleading since they do not fully reflect all administrative than 1 percent of the costs. Salaries of center staff that work directly with clients centers’ total expenses— on projects are not included in administrative costs even though well within Franchise Tax these employees appropriately spend some time on administra- Board limits. tive functions. Therefore, these costs must be included in the total administrative costs to arrive at the true costs of adminis- tration. Using data provided by the centers to determine how all staff spent their time for the fiscal year ending June 30, 1999, we found the ratio of time charged to administrative categories with respect to total hours is significantly higher than the administra- tive costs suggest. As Table 4 illustrates, the centers spend about half of their time on administrative tasks, which may indicate that the centers are less efficient than their current administrative cost percentages reflect. C A L I F O R N I A S T A T E A U D I T O R 23 C A L I F O R N I A S T A T E A U D I T O R C A L I F O R N I A S T A T E A U D I T O R TABLE 4 Allocation of Center’s Staff Hours for Fiscal Year Ending June 30, 1999 Center Percentage of Hours Staff Duties Corporation for Manufacturing Excellence (Manex) 57% Administrative 29 Marketing 14 Project-related California Manufacturing Technology Center (CMTC) 52 Administrative 14 Marketing 34 Project-related San Diego Manufacturing Extension Center (SanMEC) 48 Administrative 26 Marketing 26 Project-related C A L I F O R N I A S T A T E A U D I T O R C A L I F O R N I A S T A T E A U D I T O R RECOMMENDATIONS To ensure that the program meets California’s needs, OST should do the following: (cid:1) Develop a statewide strategy that identifies its role, the program’s goals and objectives, and specific performance measures and milestones that it can use to assess the progress of the program in meeting California’s needs. It should also ensure that its strategy establishes a common set of performance measures to evaluate the centers’ progress towards achieving the program’s goals. (cid:1) Develop a standard reporting format and require that the centers’ quarterly and legislative reports address all perfor- mance measures. To minimize the centers’ reporting burden, OST should consider using the data the centers report to NIST when that data also meets its needs. (cid:1) Use the quarterly reports to monitor center performance, communicate its concerns to centers in writing, require centers to promptly address deficiencies, and consider reducing funding when centers do not address problems. 24 C A L I F O R N I A S T A T E A U D I T O R (cid:1) More closely monitor the centers’ compliance with program funding requirements and consider reducing grant amounts when centers do not comply. (cid:1) Evaluate the propriety of the time centers spend on adminis- trative tasks. Finally, the centers should reimburse the federal government for disallowed costs. We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, KURT R. SJOBERG State Auditor Date: December 21, 1999 Staff: Sylvia L. Hensley, CPA, Audit Principal Tammy Lozano, CPA Kimberly Bootman Miles Burnett, Ph.D. Nathan Checketts Leah Northrop Nicette Short C A L I F O R N I A S T A T E A U D I T O R 25 Blank page inserted for reproduction purposes only. 26 C A L I F O R N I A S T A T E A U D I T O R Agency’s comments provided as text only. California Trade and Commerce Agency 801 K Street, Suite 1918 Sacramento, California 95814 December 10, 1999 Kurt R. Sjoberg, State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Mr. Sjoberg: Thank you for the opportunity to respond to the audit results of the Manufacturing Technology Program. I am pleased to report that we already have taken steps that are consistent with the recommendations provided in the report. For example, the Trade and Commerce Agency has adopted the following policy changes: (cid:1) The Office of Strategic Technology (OST), currently in the Division of Economic Development, will become the Division of Strategic Technology with a Deputy Secretary reporting directly to the Secretary of the Trade and Commerce Agency. (cid:1) The new Division of Strategic Technology will be moved to Sacramento from its current location in Pasadena to provide statewide leadership, presence and accountability for all initiatives in 1) the information technology and e- commerce industry cluster; 2) the air and space industry cluster; and 3) the general manufacturing industry cluster. The Manufacturing Technology Program is a component in the general manufacturing industry cluster initiative. This organizational alignment provides the Division of Strategic Technology the appropriate authority to conduct its policy setting, administrative and fiscal matters with federal and state agencies, the legislature and industry. (cid:1) The Trade and Commerce Agency is working with representatives from the National Institute of Standards and Technology, which, as you are aware, provides grant funds directly to the Manufacturing Technology Centers; the three centers in the Bay Area, Los Angeles and San Diego; the Center for Applied Competitive Technology Centers in the community college system and the Chancellor’s Office; small manufacturing associations; and owners of manufacturing firms to develop a statewide strategy. The strategy will C A L I F O R N I A S T A T E A U D I T O R 27 Bureau of State Audits December 10, 1999 Page two provide goals and objectives, performance measures and outcomes and strategic initiatives to meet the diverse and rapidly changing demands of manufacturers in industry sectors. · The Trade and Commerce Agency and the National Institute of Standards and Technology are mutually working towards an improved, coordinated program and addressing any deficiencies and performance of Manufacturing Technology Centers. The results of the audit support our efforts to make organizational, investment and accounting changes. The Trade and Commerce Agency will immediately work with representatives from the Manufacturing Technology Centers and the National Institute of Standards and Technology to incorporate specific recommendations from the audit and resolve any reimbursement requirements due to disallowed costs. I appreciate the efforts of the Bureau of State Audits. Sincerely, (Signed by: Lon Hatamiya) Lon Hatamiya Secretary 28 C A L I F O R N I A S T A T E A U D I T O R Agency’s comments provided as text only. California Manufacturing Technology Center 13430 Hawthorne Boulevard Hawthorne, CA 90250 December 9, 1999 Bureau of State Audits Attn: Kurt Sjoberg State Auditor 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Mr. Sjoberg: Thank you for providing CMTC the opportunity to respond to the draft report on the Manu- facturing Technology Program. We are very proud of the program and the results that it has produced for the State as reflected in the March 1, 1999 Report to the Legislature by the California Trade and Commerce Agency. This report contained the survey results, conducted by the U.S. Department of Census for the three California centers (including CMTC), which showed among other results, that there had been a positive change in employment of 1,365 jobs and an increase in client company sales of $56 million. The report also indicated a three-year State impact of a $196 million increase for Gross State Product as well as a $15 million increase in State and local tax revenues. Our comments on the draft report are as follows: 1. Consistent Performance Measures (page 2) We strongly agree with the statement that consistent performance measures are needed for the MTP program. 2. Administrative Costs (page 3) We agree with the conclusion that “center administrative costs are relatively low” not withstanding the apparent large percentage of time shown in our time cards which includes program activity such as client report writing, infrastructure and partner alliance develop- ment and presentation preparation. Consequently, the 52% reported through our time cards include program content and we need to do a better job of segregating activities such as market development, product development, and partnering activity. We recognize this and are taking corrective action. A more accurate indicator of our low administration cost is the comparative figures between all MEP centers (as compiled by NIST) of total expenses per FTE (full time equivalent) Field agent/technical staff, which show that CMTC is below both the mean and median averages for all of these centers. C A L I F O R N I A S T A T E A U D I T O R 29 Mr. Kurt Sjoberg December 9, 1999 Page Two 3. Quarterly Reports (page 15, 16, 17, 26, table 4) While we have never previously been requested to do so, we agree that reporting could be improved. We work very hard and are very diligent in providing accurate and timely re- ports. In fact, all our systems have been set-up to address the reporting requirements of our funding partners. We are continuously improving our systems to capture consistent and accurate information. Data is recorded on an on-going basis, compiled monthly, and reviewed by senior management prior to submittal. 4. Impact collection ,“Do not survey” (page 19, 20) We agree with the conclusion that our center is “well under the national average” when it comes to using the “Do not survey” option on impact collection process. While CMTC has a 5% “Do not survey” percentage compared to a desirable 2- 4% level and a 19% national average, CMTC also has a comparatively large repeat business (over 60%). “Repeat” client contracts may be marked “do not survey” to avoid the client being surveyed multiple times in a year. We recognize that by not including them in the survey multiple times, the “jobs created and retained” indicator is under reported. We also recognize that the “Cus- tomer Satisfaction” indicator might be over or under reported. Again, thank you for the opportunity to respond. Should you have any questions, please don’t hesitate to contact me at (310) 355-3067. Sincerely, (Signed by: John J. Van Buren, C.F.O. for) David Braunstein President and Chief Executive Officer 30 C A L I F O R N I A S T A T E A U D I T O R Agency’s comments provided as text only. Corporation for Manufacturing Excellence 48001 Fremont Blvd. Fremont, CA 94538 December 8, 1999 Mr. Kurt R. Sjoberg, State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Subject: Response to Audit Report Dear Mr. Sjoberg Thank you for the opportunity to respond to the draft audit report entitled "California Trade and Commerce Agency: It Has Not Demonstrated Strong Leadership for the Manufacturing Technology Program, Collected Data Necessary to Measure Program Effectiveness, or Ensured Compliance with Program Requirements." Manex does not dispute the criticisms contained in the report regarding inconsistent, unsupported and inaccurate data. These inaccuracies were "simple mistakes" and the preponderance of errors occurred early in our program when the infrastructure was not solid. We have remedied this by having a single-point-of-control responsible for the collec- tion and dissemination of data. Though we did have proper documentation bearing signatures from our board members' donated services, we did not have appropriate documentation for valuing the basis of these in-kind donations. We have been addressing the in-kind issue by reducing it from $676,172 in 1997 to $193,952 in 1999. The in-kind issue will become a moot point in FY 2004 when in-kind contributions will not be an acceptable form of cost share by the federal program and will thus be eliminated in total. We will engage in discussions and resolutions with the federal government regarding any disallowed costs. Manex is committed to serving small and medium-sized manufacturers and to the continuous improvement of its own performance. Sincerely yours, (Signed by: Kulbir “Bill” Singh) Kulbir "Bill" Singh Interim President C A L I F O R N I A S T A T E A U D I T O R 31 Blank page inserted for reproduction purposes only. 32 C A L I F O R N I A S T A T E A U D I T O R Agency’s comments provided as text only. San Diego Manufacturing Extension Center 9663 Tierra Grande Street, Suite 204 San Diego, CA 92126 December 8, 1999 Kurt R. Sjoberg State Auditor 555 Capital Mall, Suite 300 Sacramento, CA 95814 Dear Mr. Sjoberg, We have reviewed our portion of Report No. 99025 and have only a few comments. In general, we would like to complement the audit team for their professionalism. They asked good, pertinent questions and were open to our observations. We agree with their overall perspective. There is a need for the Office of Strategic Technol- ogy and the Manufacturing Technology Program to agree on a common set of goals, objective and performance criteria and a means by which to measure performance. If possible, this must be balanced with similar objectives of NIST/MEP. With respect to the inappropriate use of funding to pay third parties in excess of their billing rate, we wish to point out that the potential for this to become a problem was pointed out to the former director of the center prior to implementation. Several members of the manage- ment team strongly suggested (in writing) that approval for this be obtained from NIST/ MEP and OST before the plan was implemented. Once the inappropriateness of these expenditures was brought to the attention of new management it was immediately halted. Again, we thank your staff for their diligence and look forward to supporting positive efforts to move the manufacturing technology program forward. Sincerely, (Signed by David L. Dinerman) David L. Dinerman Acting President C A L I F O R N I A S T A T E A U D I T O R 33 Blank page inserted for reproduction purposes only. 34 C A L I F O R N I A S T A T E A U D I T O R cc: Members of the Legislature Office of the Lieutenant Governor Attorney General State Controller Legislative Analyst Assembly Office of Research Senate Office of Research Assembly Majority/Minority Consultants Senate Majority/Minority Consultants Capitol Press Corps C A L I F O R N I A S T A T E A U D I T O R 35