Summary

California State University: Its Common Management System Has Higher Than Reported Costs, Less Than Optimal Functionality, and Questionable Procurement and Conflict-of-Interest Practices March 2003 2002-110 rotiduA etatS ainrofilaC S T I D U A E T A T S F O U A E R U B The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TDD (916) 445-0255 x 216 OR This report is also available on the World Wide Web http://www.bsa.ca.gov/bsa/ The California State Auditor is pleased to announce the availability of an online subscription service. For information on how to subscribe, please contact David Madrigal at (916) 445-0255, ext. 201, or visit our Web site at www.bsa.ca.gov/bsa Alternate format reports available upon request. Permission is granted to reproduce reports. � � � ��������� ���� ������ ������������� ������������������� ������������ ����������������������� March 11, 2003 2002-110 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the Bureau of State Audits presents its audit report concerning the California State University’s (university) Common Management System (CMS). This report concludes that the university did not establish a business case for CMS by preparing a feasibility study report or other analysis demonstrating that CMS is a worthwhile expenditure of university resources. Further, its previous CMS cost projections have understated the full costs of CMS, most recently projected at $662 million, including $269 million for maintenance and operations, over the now nine-year development and implementation period. Moreover, it has yet to establish a mechanism to adequately monitor overall systemwide CMS costs, nor establish a systemwide funding plan for the project. Additionally, we noted problems that cast doubt on whether CMS will achieve all the objectives the university intended, nor offer what could have been achieved from a systemwide project. Although the university followed recommended procurement practices to acquire data center services, its procurements for software and consultants raise questions about the fairness and competitiveness of its practices. For example, its solicitation document for the CMS software procurement did not provide for a method to objectively select one winning vendor, and the university could not demonstrate how it resolved evaluation team concerns. Finally, the university did not do enough to prevent or detect apparent conflicts of interest on CMS-related procurements. Respectfully submitted, ELAINE M. HOWLE State Auditor ������������������� �������������������������������������������������� ���������������������������������������� ������������������ CONTENTS Summary 1 Introduction 7 Chapter 1 The University Did Not Develop a Business Case for the Common Management System and Its Supporting Data Center 17 Recommendations 37 Chapter 2 The University’s Project Costs Exceed Initial Estimates, and Cost Monitoring Procedures Are Inadequate 39 Recommendations 67 Chapter 3 The Common Management System May Not Achieve All Its Business Objectives Nor All the Possible Benefits of a Systemwide Software Project 69 Recommendations 92 Chapter 4 The Processes the University Used to Select the Software Vendor and Consultants on the Project Did Not Clearly Demonstrate Best-Value Procurements 95 Recommendations 110 Chapter 5 Data Center Services Have Improved, but Data Warehousing Needs Remain 113 Recommendations 120 Chapter 6 The University’s Oversight Over Potential Conflicts of Interest Needs Improvement 121 Recommendations 134 Appendix A Chancellor’s Office Positions and Workload Will Increase, but It Is Too Early to Determine the Impact of Project Implementation on Campus Positions and Workload 137 Appendix B Projected Total Investment and Maintenance and Operations Costs of the Common Management System Project by the Chancellor’s Office and Campuses 143 Appendix C Projected Total Costs of the Common Management System Project by Type of Costs 145 Appendix D Consultant Costs, Actual and Projected, for the Common Management System Project 149 Appendix E Common Management System Functionality at Each Campus 155 Response to the Audit California State University 163 California State Auditor’s Comments on the Response From the California State University 175 California State Auditor Report 2002-110 11 SUMMARY RESULTS IN BRIEF With 23 campuses and an annual budget of more than $5.2 billion, the California State University Audit Highlights . . . (university) provides a broad education along with undergraduate and graduate instruction for professional Our review of the California and occupational goals to more than 400,000 students each State University’s (university) year. Supporting this instruction is a structure for handling Common Management System (CMS) revealed numerous administrative details, including the following: the following: student services such as admissions, registration, and grades; human resources activities of processing and paying faculty, þ The university did not establish a business classified staff, and student workers; and financial services case for CMS to define related to purchasing, billing, inventory, and accounting for its intended benefits funds. To enhance administrative productivity and quality, the and associated costs university is developing and implementing an administrative and ensure that the expenditure of university software project, the Common Management System (CMS), resources is worthwhile. which is replacing all university finance, human resources, þ The university’s previous and student administration systems with the PeopleSoft suite cost projections understated of administrative software. The university plans to support this the full costs of CMS software with one outsourced data center to process all CMS over its now nine-year data. One expected benefit of CMS is enabling students to go project period; these online to look up admissions status and grades. Recent estimates for costs—including an estimated $269 million the CMS project’s total costs are about $662 million—$393 million for maintenance and for one-time costs and $269 million for maintenance and operations—are now operations—for the nine-year development and implementation expected to total period from fiscal year 1998–99 through 2006–07. $662 million. þ Problems exist that The university originally anticipated significant cost savings cast doubt on whether from its CMS and data center efforts, even believing these CMS will achieve all the objectives intended, nor savings could fund other information technology initiatives. offer what could have It now says it did not have cost savings as a primary goal, but been achieved from a expected CMS would avoid or minimize costs for improved and systemwide project. expanded administrative software services over the long term. continued on the next page Regardless of its reasons for pursuing CMS and its supporting data center, the university has not established a business case for the CMS project by preparing a feasibility study report or a similar analysis that clearly defined its intended benefits and associated costs when compared with the pre-CMS environ- ment. In addition, when the university decided to direct a single administrative system at all campuses, it did not evaluate the cost variance in alternatives that allowed for implementation California State Auditor Report 2002-110 11 plans ranging from running one shared copy of the software to running several copies focusing on differing campus needs. Without compelling evidence of its need for new administrative þ Although the university software, or a cost-benefit analysis for the project, the university followed recommended cannot ensure that the CMS project is a worthwhile expenditure procurement practices of resources. to acquire data center services, its procurements for software and When asked why it never conducted a formal return-on- consultants on the investment analysis on the CMS project, the university project raise questions explained that the magnitude of potential savings estimated about the fairness and competitiveness of the by its consultants, IBM and Pacific Partners Consulting Group university’s practices. (Pacific Partners), led it to believe that such a formal analysis was unnecessary. However, the university cannot rely on its þ The university did not consultants’ reports as justification for not developing the do enough to prevent or detect apparent conflicts business case for CMS because these studies were not intended of interest on CMS-related for such a purpose—as evidenced by their scope. In fact, both procurements. studies recommended that the university conduct subsequent cost-benefit analyses before proceeding with the project. Further, although the university may have intended to conduct its own cost-benefit analysis for the CMS project, it never completed one. Also, the university’s stated reasons for CMS, given in its “Why CMS?” document, are insufficient to justify a significant investment without analysis demonstrating that the problems it described were severe or systemic. Further, the university cannot support that most of its campuses were planning to replace prior administrative systems in July 1999 when the decision was made for all campuses to implement CMS. The university’s chancellor cites various reasons for pursuing CMS, such as drawing all cam- puses into a more common set of business practices. However, these reasons are inadequate to forgo documenting the problems being addressed by CMS and the associated costs to fix them. The strategic benefits the chancellor is hoping to achieve come at a sig- nificant cost annually; the most recent university estimates indicate it will cost more than $65 million each year to maintain the fully implemented CMS software. Recent data indicate that the university’s 1998 cost projections of $332 million to $400 million and 1999 projection of $440 million for its CMS project understated the project’s costs. Because it did not collect actual project cost information from campuses, the university, at our request, expedited a comprehensive cost survey of actual CMS expenditures and projections. This survey revealed that the total project cost for the types of expenses it initially estimated—what the university considers to be “new” costs—now total $482 million. 22 California State Auditor Report 2002-110 California State Auditor Report 2002-110 33 The $482 million includes maintenance and operations of the data center during the implementation period, but it excludes certain campus project costs the university did not estimate. The university maintains these are not “new” costs. They include $63 million in implementation costs charged to other campus budgets and $117 million related to campus maintenance and operations costs for CMS during the nine-year project period. However, the accuracy of these figures is uncertain given that 73 percent of the projected $662 million in implementation and maintenance and operations costs through fiscal year 2006–07 is estimated. Additionally, the university has not established a mechanism to monitor overall systemwide project costs adequately, contribut- ing to a lack of complete project cost information for university management and for the Legislature. The university told us it planned to gather cost data from campuses in fall 2002. After we expressed our concern about the incomplete information, the university surveyed campuses in June 2002 for this cost informa- tion. Also, the university lacks a systemwide funding plan for the CMS project and, therefore, lacks a full picture of how the project may affect future funding priorities. Further, we noted problems that cast doubt on whether CMS will achieve all the objectives the university intended, nor offer what could have been achieved from such a systemwide project. Some problems stem from the university’s weak efforts early in the planning process. For instance, one business objective was to minimize costs and time to implement and maintain the software. One of the ways the university intended to do this was by limiting modifications to the vendor software to only those needed to meet its business needs. However, the university had no basis to anticipate the modifications it needed to make because, before it purchased the software, it did not sufficiently evaluate its specific business processes to understand which business processes the potential vendors’ software products could accommodate and which software products would require modification to meet its business needs. Additionally, it must often continue to reapply these modifications when the vendor software is updated, thus increasing the costs to maintain the CMS software. Further, the university plans to continue to use existing processes for systemwide reports because it did not design CMS to replace these processes. Finally, its piecemeal approach of identifying, procuring, and implementing its own CMS solution did not share risk with vendors and consultants. 22 California State Auditor Report 2002-110 California State Auditor Report 2002-110 33 Thus, it assumed substantially all the considerable financial and business risk involved in ensuring that the software meets its business needs and is implemented successfully. Additionally, the CMS software procurement process raises questions about whether the university used a fair and objective competitive process. Originally, the university planned to identify one or more vendors that campuses could select, and its solicitation document did not provide a method to select only one vendor. When it decided late in the process to recommend one software vendor, the university did not use a quantitative scoring process to select objectively between the two finalists and could not demonstrate that it had resolved questions raised by the procurement evaluation teams. After procuring the software, the university and the campuses hired CMS consultants through sole-source contracts that appear to be contrary to its own policy on when such agreements are appropriate. The university also has not required solicitations for offers from various consulting firms under its master agreements. Without such additional offers, the university cannot demonstrate that it procured best-value services. Unlike these procurements, the university used recommended procurement practices to select the best-value vendor when procuring the outsourced data processing services needed to run CMS. Also, it shared risk with the outsourced data center vendor by establishing contract terms aimed at holding the vendor accountable for meeting preestablished service levels. When it experienced inadequate service from the data center in the early months of the contract, the university used procedures in the contract to help raise services to agreed levels, and recent months show improvements in the levels. However, although it has worked to address its CMS data processing needs, the university only now is starting to address campus CMS data warehousing needs. Data warehousing can provide for optimum data storage and reporting, such as enabling the production of reports that contain historical analysis of university operations. Earlier in the project planning, the university removed data warehousing services from the CMS project scope. It is now revisiting and starting to address campus interest for those services. Finally, the university did not do enough to ensure that indi- viduals participating in the project’s procurement decisions were free from apparent conflicts of interest, casting a shadow on the project. The university did not designate certain university 44 California State Auditor Report 2002-110 California State Auditor Report 2002-110 55 positions, such as some CMS project directors, as responsible for filing annual forms to disclose economic interests. Also, the university did not provide appropriate guidance to employees to identify potential conflicts. Finally, the university lacks a policy that spells out for employees what constitutes “incom- patible activities” and does not require designated employees to receive regular ethics training. Conflicts of interest or incom- patible activities could compromise the university’s reputation for honest and fair business practices and undermine public confidence in the university’s procurement decisions. In fact, we found an employee who appeared to have had a conflict of interest while participating in a CMS procurement, and one employee who possibly may have used nonpublic information to benefit personally. RECOMMENDATIONS The university should adopt policies and procedures that require a feasibility study before the acquisition and implementation of significant future information technology projects. The university should ensure that it monitors systemwide project costs adequately, including establishing a mechanism to collect data on campus costs periodically. Also, it should estab- lish a systemwide funding plan for CMS that includes campuses. The university should take steps to ensure that it meets its business objectives for the CMS project, including taking action to minimize the costs and time associated with implementing the software. Further, it should determine how it could improve the design of CMS to report systemwide information. The university also should use recommended practices, such as ensuring that it shares project risk with vendors and using a quantitative evaluation method to select best-value vendors. Further, when procuring information technology systems or software in the future, the university should evaluate its specific business processes against vendor products before procurement, then select vendors that best accommodate the university’s specific needs. Finally, it needs to strengthen its procedures for prevent- ing and detecting potential conflicts of interest for individuals participating in procurement decisions. For example, it should 44 California State Auditor Report 2002-110 California State Auditor Report 2002-110 55 conduct periodic ethics training for designated employees. Additionally, it should establish an incompatible activities policy that it formally communicates to employees. AGENCY COMMENTS The university states that it agrees in nearly all cases with our recommendations but does not agree fully with all the findings of the audit. The university believes the audit recommendations will be beneficial for the continued development and improve- ment of the CMS effort, and states it has already implemented or begun to implement some of the recommendations and will be acting on the others. n 66 California State Auditor Report 2002-110 California State Auditor Report 2002-110 77 INTRODUCTION BACKGROUND The California State University’s (university) mission is to provide a broad education and undergraduate and gradu- ate instruction for professional and occupational goals. The university’s annual budget is more than $5.2 billion. It offers more than 1,800 bachelor’s and master’s degree programs in 240 subject areas and serves more than 400,000 students annually. Since 1961, the university has awarded about 2 million bachelor’s, master’s, and joint doctoral degrees. The State vests responsibility for the university in a board of 25 trustees, 19 of whom are appointed by the governor; in turn, the board of trustees (board) appoints the chancellor and the campus presidents. The university’s 23 campuses, located throughout the State, vary in student population and annual budget. The San Diego campus reported student enrollment of 34,171 students in the fall of 2001, while the Maritime Academy reported 606 students during this same period. The system’s oldest campus was founded in 1857 and became the first public institution of higher education in California, while the 23rd campus was added to the system in 2002. THE UNIVERSITY’S INTEGRATED TECHNOLOGY STRATEGY PLAN In January 1993, the university’s former chancellor challenged the board and university employees to “do business differently” in using technology to manage the overall university and to instruct students. In response to this challenge, the university developed an Integrated Technology Strategy (ITS) plan that it presented to the board in March 1996. The driving force behind the ITS plan was its belief that student demand for public instruction would be satisfied elsewhere unless the university adopted an information age approach to conducting business. The university’s underlying vision for the ITS plan is that learn- ing can be enhanced by information technologies that address individual learning styles and the unique needs of diverse student groups. As stated in its ITS plan, the university believes these “technologies can increase student access to faculty and information providers and resources by making them available independent of time and place and by reducing or removing the 66 California State Auditor Report 2002-110 California State Auditor Report 2002-110 77 geographic, economic, and social barriers to learning.” Through successfully implementing the ITS plan, the university hopes to achieve an integrated electronic environment that enables all university students, faculty, and staff to communicate with one another and to interact with information resources from any- place, to anyplace, at any time. Consistent with the university-commissioned work of IBM in 1996, the university’s ITS plan and its associated technology initiatives are aimed at achieving one of four strategic outcomes: excellence in learning and teaching, quality of student experi- ence, administrative productivity and quality, and personal productivity. Table 1 depicts these 11 technology initiatives and their associated strategic outcomes. TABLE 1 Strategic Goals and Initiatives of the University’s ITS Plan Initiative Strategic Goals Number Technology Initiative 1 Distributed Learning and Teaching 2 Centers for Instructional Technology Development Excellence in Learning and Teaching 3 Multimedia Repositories 4 Library Resources 5 Student Friendly Services Quality of Student Experience 6 One Card 7* Collaborative Management Systems–subsequently evolved into Common Management System (CMS) Administrative Productivity and Quality 8* Streamlining Information Technology Delivery—subsequently evolved into the CMS data center effort 9 Procurement Process Improvement 10 Baseline User Hardware, Software Access, Training, and Support Personal Productivity 11 Access Infrastructure Source: The university’s Integrated Technology Strategy (ITS) plan. * Focus of the audit report. In keeping with the university’s primary vision of improving education through information technology, six of the 11 initiatives under ITS (1-6 in Table 1) aim to improve the university’s instruction and student services. The strategic outcomes associated with the remaining five initiatives are split roughly between improving administrative efficiency and 88 California State Auditor Report 2002-110 California State Auditor Report 2002-110 99 providing the prerequisite technology needed for all initiatives. Because the scope of this audit is limited to a review of the Common Management System (CMS) and its supporting data center, this report discusses initiatives 7 and 8. Additionally, in the next section, we discuss the university’s efforts to obtain the prerequisite technology for CMS through initiatives 10 and 11. THE UNIVERSITY IS WORKING TO ENSURE THAT ALL CAMPUSES HAVE THE PREREQUISITE TECHNOLOGY FOR CMS AND ITS OTHER TECHNOLOGY INITIATIVES The university reports that it has been working since 1996 toward ensuring that all campuses have the prerequisite technology in place to realize the full benefits of CMS and the remaining ITS technology initiatives. As of November 2002, the university’s work in this area (initiatives 10 and 11 in Table 1) was continuing. The university refers collectively to these two initiatives, aimed at increasing personal productivity, as the Technology Infrastructure Initiative (TII). According to the university’s ITS plan, “none of the ITS initiatives can be successfully implemented, nor their goals achieved, without access to robust inter-campus and intra-campus telecommunications infrastructures.” In its November 2002 Measures of Success report to the Legislature, the university states that “a major premise of the ITS [plan] is that success of [its] initiatives depends upon the provision of a minimum baseline technical infrastructure of hardware, software and network communications as well as training and support for those who use them. The TII provides the underlying minimum baseline infrastructure for the ITS [plan].” As of November 2002, none of the 23 campuses had attained the baseline level of technology required for the ITS initiatives. In its November 2002 Measures of Success report to the Legislature, the university reported that no campus met the baseline level of physical infrastructure, access to computer workstations, and training components, but that three campuses met the baseline related to the high-speed network access and access to technical support. However, the university anticipates that the first campus will achieve the full prerequisite level of technology sometime in fiscal year 2002–03, expecting the TII project to conclude in fiscal year 2005–06, a year before CMS will be installed on all campuses. 88 California State Auditor Report 2002-110 California State Auditor Report 2002-110 99 Although it was not discussed in its annual report to the Leg- islature, the university projects that the TII project’s total cost will exceed $271 million through fi scal year 2005–06. The TII project’s senior director states that $26.8 million already has been spent on TII through June 2002. He further indicates that campuses have funded 68 percent of the project’s costs through the use of their legislatively approved capital outlay programs, while the chancellor’s offi ce has provided the remaining funding through annual allocations from the university’s annual support budget. The annual support budget is funded primarily from the State’s General Fund. THE CMS PROJECT UNDER THE UNIVERSITY’S INTEGRATED TECHNOLOGY STRATEGY PLAN As stated earlier, one strategic outcome of the university’s ITS plan is enhanced administrative productivity and quality, which the university hopes to achieve through technology initiatives that redesign administrative support functions to be more effec- tive and cost-effi cient. CMS is one such initiative (number 7 in Table 1 on page 8). The CMS project entails replacing all university fi nancial reporting, human resources, and student The CMS “Target Administrative Environment” administration systems with the PeopleSoft suite of administrative software. When the “target By the year 2006, the chancellor’s offi ce and administrative environment” is achieved, the all 23 campuses expect to do the following: university expects to have more unified and effi- • Perform administrative functions in concert cient administrative systems (see textbox). with a common set of administrative “best practices” approaches. The university expects CMS to yield many signifi - • Support administrative functions (initially cant benefi ts, such as allowing students to look including human resources, fi nancial, and student services) with a shared, common up admission status and grades online, and better suite of applications software. enabling faculty to advise students through online • Operate the administrative software suite information. It also expects CMS to enable each at a shared service center. campus to provide all levels of users access to data when they want and in a format that is meaningful to them without having to wait for programming Source: The university’s “CMS Project Charter.” resources to become available to write a program for a special report. The university issued a request for qualifi cations (RFQ) to procure software under the CMS initiative in April 1997 and confi rmed the selection of the PeopleSoft suite of administrative software in July 1998. After procuring a software license from 1100 California State Auditor Report 2002-110 California State Auditor Report 2002-110 1111 PeopleSoft, the university began modifying this software to meet its specific business needs. PeopleSoft was responsible for initially delivering the software, but not for its successful implementation. Eleven of the university’s 23 campuses, collectively known as the “first wave,” began implementing CMS in early 2001. According to its December 2002 project timeline, the university expects all campuses to have implemented CMS fully by October 2006. A June 2002 cost survey indicates that total CMS costs, including an estimated $393 million in investment costs and an estimated $269 million in costs for maintenance and operations, will exceed $662 million by the end of fiscal year 2006–07. The university is implementing CMS through a phased approach. Because campuses have decided to begin imple- menting CMS at different times, some are further along in this process than others. For example, some campuses are well underway, while one campus will not begin implementation until April 2005. Figure 1 on the following page demonstrates the varying degrees of CMS implementation at all 23 campuses, depicting each campus with its current and projected remaining CMS investment costs as of June 2002. These costs represent campus one-time costs associated with CMS such as implementation, integration, in-kind, and upgrade costs. They are described in more detail in Chapter 2. Some campuses are delaying implementing CMS for strategic or financial reasons. For example, the Sacramento campus decided to wait to implement CMS so it could learn from the experiences of other campuses. The San Diego campus will not begin to implement CMS until fiscal year 2004–05 to gain maximum value from a new Oracle system purchased before July 1999, when the university decided to implement CMS at all campuses. Even though the university began to procure software for CMS before this time, it had left campus participation in this initiative voluntary. From the initial stages of the initiative until July 1999, the term CMS referred to a “collaborative management system,” reflecting that campus participation was voluntary and the university did not expect all campuses to use the same administrative systems. The July 1999 decision to implement CMS at all campuses fundamentally changed the scope of the project, and CMS now refers to a “common management system,” indicating the policy of common business practices and their supporting software at all campuses. 1100 California State Auditor Report 2002-110 California State Auditor Report 2002-110 1111 FIGURE 1 Campus CMS Implementation Status as of June 2002 Measured by Investment Costs ��� �� �� �� �� �� � � �������� 1122 California State Auditor Report 2002-110 California State Auditor Report 2002-110 1133 ���������� ��������������������������������� ���������������������� � � �� � � � � � �� � � � � � � � ��� � � � � � � � � � � � � �� � � �� ����� �� �� � � � � �� ��� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��� � � � � � � � � � � � � � � � � � � � � � � � � � � �� � � � � � � � � � � � � � � � � � � ��� � � � � � � � � � � � � � �� � � � � � � �� � � � � ��� � � � � � � � � � � � ���� � �� � Source: The university’s June 2002 cost survey. THE CMS DATA CENTER UNDER THE UNIVERSITY’S INTEGRATED TECHNOLOGY STRATEGY PLAN The university’s efforts to establish an outsourced CMS data center under contract to a private company is a major component of the project. The CMS data center, which evolved from the original “streamlining information technology delivery” initiative (number 8 in Table 1 on page 8), falls under the ITS strategic goal of enhancing administrative productivity and quality. Specifically, the university hopes that its outsourced CMS data center will reduce the administrative and hardware costs associated with CMS by maintaining the CMS software and data at a single location. In April 2000, the university solicited proposals from IBM and Unisys regarding an outsourced data center to host the CMS software and data. The university initially awarded the data center contract to IBM, and it allowed that vendor to begin preparing CMS processing capability at its data center in Rochester, New York, while final negotiations took place. Never able to finalize an agreement with IBM, the university ended negotiations in December 2000, citing significant differences over technical requirements and IBM’s proposed price. According to the university’s chief of staff for information technology services, the university temporarily expanded its own data center after terminating negotiations with IBM in order to continue CMS development. After successfully negotiating a contract and receiving data center services from Unisys, one of the two original bidders on the contract, the university no longer used this contingency data center for CMS software and data. In March 2001 the university entered into a contract with Unisys for a data center, which is located in Salt Lake City, Utah. Under the terms of the contract, the university will pay Unisys $60 million through February 2006. In return, Unisys will provide data center services to support the university’s CMS software and data. SCOPE AND METHODOLOGY The Joint Legislative Audit Committee (audit committee) requested that the Bureau of State Audits review the CMS proj- ect. Specifically, the audit committee asked us to identify the initial cost estimates and current projected costs for CMS. Our review of CMS costs included specific areas the audit committee asked us to review, such as system integration costs, consultant costs, data center costs, and the university’s funding sources for these related expenditures. Also, the audit committee asked us to identify the university’s needs, benefits, and return on investment from CMS and its supporting data center. The audit committee also asked us to review a number of specific areas, including the university’s management and oversight for CMS and its supporting data center; the university’s process to select the software, hardware, and consultants contributing to the CMS project; and how implementation has affected growth in employee positions and workload. 1122 California State Auditor Report 2002-110 California State Auditor Report 2002-110 1133 As part of our review of CMS project costs, we identified the university’s initial and subsequent total cost estimates and inter- viewed university personnel to understand their methodology and assumptions for preparing these figures. Early in our audit, we learned that campus costs were only estimated, with actual campus cost data not kept centrally. Our inquiries prompted the university to prepare a cost survey to capture current and pro- jected campus expenditures for the CMS project. We provided input during the preparation of the survey to ensure that the university collected sufficient information on specific areas we were asked to review, such as system integration and consultant costs. After reviewing the expenditures reported in this survey, we issued a follow-up survey to collect funding information for the expenditures previously reported. To assure that the survey responses were reasonably accurate, we visited the chancellor’s office and two campuses—Long Beach and Sonoma—to identify and verify selected project costs in more detail. To understand the university’s business need for CMS and its supporting data center, we interviewed university personnel to identify and obtain all studies and analyses supporting the existence of such need. We confirmed our understanding of the studies with university personnel and, when feasible, obtained clarification from the university’s consultants who originally performed these analyses. We surveyed each campus president to determine whether their support of CMS was based on campus-specific business needs, whether the campus completed the university’s optional campus financial planning tool, and whether the campus had analyses supporting a conclusion that a data center was the most cost-effective approach to implementing CMS. Our review of the university’s business need for CMS also involved comparing the university’s justification of the project to how other state departments are required to justify similar information technology (IT) projects. In addition, we asked the university for all benefit and return- on-investment calculations it had conducted before proceeding with the CMS project. To review the university’s management and oversight for the CMS project, we reviewed a report from the Legislative Analyst’s Office that identified IT recommended practices. In addition to the identified practices, we considered those included in requirements for other state agencies, the Project Management Body of Knowledge (PMBOK), and the Control Objectives for Information and related Technology (COBIT). Based on interviews with university staff and a review of various 1144 California State Auditor Report 2002-110 California State Auditor Report 2002-110 1155 project documents, we then reviewed how well the university managed its CMS and supporting data center in accordance with the identified recommenced practices. We also evaluated whether the CMS project was on track to achieve the university’s key business objectives. Finally, we reviewed the university’s management of its CMS data center, including its handling of problems. To evaluate the university’s process to select the software, hardware, and consultants contributing to the CMS project, we reviewed the laws and university policies relevant to the university’s IT procurement process. Based on our review of the laws, we identified the university’s procurement responsibilities and the extent to which it is statutorily exempt from state law and oversight. Our review of university IT procurement policies also involved comparing these policies to industry recommended practices and state policies. To determine if the university followed its own procurement policies and industry recommended practices, we selected software, hardware, and consultant agreements related to the CMS initiative at the chancellor’s office and three campuses—Fresno, Long Beach, and Sonoma. The review of these agreements included various factors such as determining the procurement method used and evaluating it against the university’s policies and practices. To identify potential conflicts of interest, we also reviewed statement of economic interests forms of individuals playing key roles in CMS and its supporting data center. Based on a review of the CMS project organization chart, and lists of participants contained in procurement documents, we developed a list of individuals directly involved in the university’s decision to solicit and award contracts to PeopleSoft, Unisys, and other CMS-related contracts. After developing this list, we attempted to obtain each individual’s statement of economic interests forms covering the period of solicitation and award. When our audit uncovered potential conflicts of interest, we obtained details of the facts surrounding the issue and an understanding of what steps the university took to mitigate the potential conflict. If our audit revealed individuals who were not included in the university’s conflict-of-interest code, we evaluated their job descriptions for responsibilities that would require inclusion in the code. Finally, we were asked to determine how project implementation has affected growth in positions and workload at campuses and the chancellor’s office. To conduct our analysis, we collected data on full-time-equivalent (FTE) positions from 1144 California State Auditor Report 2002-110 California State Auditor Report 2002-110 1155 the chancellor’s office and the 11 “first wave” campuses that were furthest along in the project’s implementation. The FTE data covered each October from 1997 through 2002 and was separated between IT and non-IT designated employees. In order to ascertain the degree to which any significant FTE growth was attributable to the CMS project, we asked the chancellor’s office and the three campuses we visited to explain select FTE increases. The results of our analysis on the CMS project’s impact on employee positions appear in Appendix A, along with our discussion of the university’s efforts to monitor changes in employee workload attributable to the CMS project. n 1166 California State Auditor Report 2002-110 California State Auditor Report 2002-110 1177 CHAPTER 1 The University Did Not Develop a Business Case for the Common Management System and Its Supporting Data Center CHAPTER SUMMARY When it began planning in 1996 for what are now its Common Management System (CMS) and its supporting data center, the California State University (university) anticipated these efforts would yield significant savings that could fund other information technology (IT) initiatives. However, the university’s expectations have changed. Today the university maintains that implementing a systemwide suite of administrative software did not have effecting cost savings as a primary goal; rather it was expected that this implementation would result in cost avoidance or minimized cost for improved and expanded administrative software over the long term. Regardless of the university’s reasons for pursuing CMS and its supporting data center when planning began in 1996, or its reasons today, the university cannot demonstrate that it documented the extent and severity of problems existing in its prior administrative software or that today’s CMS project represents the most cost-effective solution to these problems. Without compelling evidence of a need or requirement for new administrative software, as well as a cost-benefit analysis for the project, the university cannot ensure that the CMS project is a worthwhile expenditure of more than $662 million—an estimated $393 million for one-time costs and an estimated $269 million for maintenance and operations—through fiscal year 2006–07. If the university implemented a systemwide CMS for the reasons it stated in its “Why CMS?” document—such as “severe problems” with its existing administrative software—then it should have documented the severity and pervasiveness of these problems, as well as conducted a cost-benefit analysis demonstrating that CMS was the most effective solution. On the other hand, if the university implemented a systemwide CMS for other reasons—such as a desire for increased functionality and “anytime/anywhere” access—then it should have stated these objectives explicitly and prepared a cost-benefit analysis that 1166 California State Auditor Report 2002-110 California State Auditor Report 2002-110 1177 compared them against detailed projections of the necessary cost. In addition, when the university decided to implement a single administrative system at all campuses, it should have evaluated the cost variance in alternatives that allowed for implementation plans ranging from running one shared copy of the software to running several copies that focused on campus size and needs. The university said it believed that a formal return-on-investment analysis was unnecessary because of the magnitude of potential savings estimated in studies by its consultants, IBM and Pacific Partners Consulting Group (Pacific Partners). However, the scopes of these studies show they were not intended to establish the business case for CMS and its supporting data center, and both consultants recommended that the university conduct subsequent cost-benefit analyses. Thus, the university cannot rely on either report as a rationale for not developing a business case for CMS. Moreover, the university is implementing CMS and its data center in a manner contrary to the assumptions behind both consultants’ savings estimates. For example, the university did not consolidate all campus data center operations for administrative and academic purposes and shut down the existing campus data centers, as assumed in IBM’s and Pacific Partners’ cost savings estimates. Although the university may have intended to conduct its own cost-benefit analysis for the CMS project, and even developed an assessment tool for this purpose, it ultimately did not use this tool. The university’s stated reasons for CMS are insufficient to justify its significant investment without analysis documenting that the problems with its administrative systems described in its “Why CMS?” document were severe or systemic. Further, the university cannot support that most of its campuses were planning to replace prior administrative systems in July 1999, when the university decided to implement CMS at all campuses. Although the university’s chancellor has various reasons for pursuing CMS, such as drawing all campuses into a more common set of business practices, these reasons do not remove the need to document the problems being addressed by CMS, the associated costs to fix them, and the expected cost- efficiencies from common business practices. In addition to the initial implementation costs, the benefits of CMS, as envisioned by the chancellor, come at a significant annual cost. According to the June 2002 cost survey, the university expects to spend $65 million annually to maintain and operate CMS once it is fully implemented in fiscal year 2006–07. 1188 California State Auditor Report 2002-110 California State Auditor Report 2002-110 1199 ORIGINALLY ANTICIPATING SIGNIFICANT SAVINGS FROM THE CMS AND DATA CENTER CONSOLIDATION INITIATIVES, THE UNIVERSITY’S EXPECTATIONS FOR BOTH EFFORTS CHANGED Based on a 1996 presentation to its board of trustees (board), the university apparently expected to realize significant sav- In September 1996, ings through CMS, data center consolidation, or both. In a the university’s CIO September 1996 meeting, the university’s executive director of informed the board of the integrated technology strategy initiative, currently the chief trustees that a feasibility information officer (CIO), described CMS to the board’s commit- study would be done; tee on technology utilization as an initiative “with the aim of however, this study was achieving significant cost savings that can be redirected to other never performed. information technology investments.” At this meeting, the university’s current CIO informed the board’s committee that “a feasibility study will be done this fall [fall 1996] to identify the costs and timing involved in moving to shared manage- ment information software for [university] campuses.” However, as described in the following section, this feasibility study was never performed. The executive vice chancellor and chief financial officer (executive vice chancellor) described the evolution of CMS in an October 2002 letter to us as follows: “The projection of significant cost savings was one factor supporting the identification of an [enterprise resource planning system] and data center consolidation as high- priority initiatives in the Integrated Technology Strategy. As the CMS collaborative developed, the CMS task force identified the variety of goals and objectives that formed the basis for the CMS Charter. The CMS Charter does not speak to savings. Rather it points out the objective of achieving efficiencies and minimizing costs. Therefore there was no need to do a stand-alone analysis of potential cost savings.” The executive vice chancellor also clarified the goal of CMS by stating that, “the implementation of a systemwide suite of administrative software did not have effecting savings as a primary goal; rather it was expected that this implementation would result in cost avoidance or minimized costs for improved and expanded administrative software services over the long term.” Whether the objective is described as cost savings, achieving efficiencies or minimizing costs, it is important to document the business case to establish the problem being resolved as well as the expected benefits and the associated costs to achieve those benefits. 1188 California State Auditor Report 2002-110 California State Auditor Report 2002-110 1199 THE UNIVERSITY DID NOT ESTABLISH A BUSINESS CASE FOR CMS The university did not establish a business case for CMS by preparing a feasibility study report or a similar analysis that evaluated the need for and the costs and benefits of this new An economic analysis of system. Had the university adopted IT procurement policies CMS would have forced and procedures that require a feasibility study before procuring university executives to IT goods and services, such as CMS, it might have performed an evaluate in detail the economic analysis of the life-cycle costs and benefits of CMS, expected benefits against comparing these costs with those under its prior administrative costs before deciding systems. Assuming that cost savings was never the university’s to invest significant primary goal for CMS, this analysis still would have had resources in the project. value by forcing executives to evaluate in detail the new efficiencies and functionality of CMS against its projected costs before deciding to invest significant resources. Without a feasibility study, the university lacks persuasive answers to the Legislature’s questions about the use of state resources for CMS and its supporting data center. Chapter 3 of the California Public Contract Code requires state agencies to follow the State Administrative Manual (SAM) when acquiring IT goods and services. To ensure compliance with Chapter 3’s intent, the SAM procedures include a need and cost- benefit analysis. According to SAM, a feasibility study “must establish the business case for the investment of state resources in [an IT] project by setting out the reasons for undertaking the project and analyzing its cost and benefits.” Further, the SAM indicates that “the scope of the feasibility study must be commensurate with the nature, complexity, risk, and expected cost of the proposed use of information technology.” However, under Public Contract Code, Section 12100.5, which is within Chapter 3, the university is exempt from certain state oversight and approval of its IT procurements. The university believes the Public Contract Code further exempts it from following the SAM regarding feasibility study reports, although the statute requires the university to adopt policies and procedures that further the legislative policy expressed in Chapter 3. The university’s Policy Manual for Contracting and Procurement, which the university believes furthers the legislative policy of Chapter 3, does not require feasibility studies before IT procurements. Regardless of the applicability of SAM feasibility study pro- cedures to its own practices, the university would have been in a stronger position to answer legislative and public questions concerning the need for CMS if it had performed a need and cost-benefit analysis consistent with SAM. In a cost-benefit 2200 California State Auditor Report 2002-110 California State Auditor Report 2002-110 2211 analysis that mirrored the SAM requirements (see textbox), the university would have described the business problem or opportunity that CMS and its supporting data center were addressing. Also, the university would have had to maintain suffi cient documentation supporting Feasibility Study Requirements Under the feasibility study to resolve any future questions the State Administrative Manual about the project’s intent, justification, nature, and scope. Using the SAM model, the university • A description of the business problem or would have had to clearly defi ne why it was opportunity the project is intended to address. pursuing each initiative: whether cost savings, the replacement of inadequate administrative systems, • The project objectives (specifi cally which signifi cant results must be achieved for or other reasons were the “opportunities” being the alternatives to be an effective response addressed by CMS. to the problem or opportunity being addressed). In addition to quantifying IT savings, a feasibility • A complete description of the IT conditions study would have provided the university with and capabilities that must exist to satisfy each defi ned objective. an opportunity to quantify the increased business process effi ciencies expected from CMS. Rather • A thorough description of the hardware, software, and personnel to be involved in than quantify business process savings, the the project. university elects to measure qualitative gains that • A discussion and economic analysis of are not measured easily or compared with costs, each of the alternatives considered in such as improved services, while ignoring the costs the feasibility study, and the reasons for to achieve and support the services. Conceding rejecting the alternatives that were not selected. the university has not established a quantitative measure of CMS business process effi ciencies, such • An economic analysis of the life cycle costs and benefi ts of the project and the as the cost avoidance generated through manual costs and benefi ts of the current method processes no longer needed because of CMS, the of operation during the life cycle of the project. executive vice chancellor indicates such measures cannot be done accurately because values must • The source of funding for the project. be assigned to students “not waiting in line” or • A detailed project schedule showing key to employees “not needing to make or receive milestones during the project’s life. a phone call” as they did before CMS. However, other indications of effi ciency can be measured quantitatively—such as the increased or decreased number of students served by each registrar staff because of CMS. Additional measures might include comparing the number of personnel transactions a human resources analyst can execute in a day before and after CMS. Though focused on qualitative measures, the university asserts that it has established a method to measure the expected benefi ts from more effi cient systems. According to the executive vice chancellor, the university has established a voluntary quality improvement program that uses performance measures and customer satisfaction surveys as assessment tools. Although it is voluntary and not set up specifi cally for 2200 California State Auditor Report 2002-110 California State Auditor Report 2002-110 2211 measuring business processes as a result of CMS, the program asks participating campuses to contribute information regarding their key processes while students, faculty, and staff A survey of the university assess the services provided. The executive vice chancellor staff found that prior expects customer satisfaction scores to improve with the broad administrative systems implementation of CMS. However, according to the university’s received higher ratings November 2002 Measures of Success report, an annual report than the CMS software to the Legislature describing progress on CMS and other IT in each of the three projects, a survey of its staff found prior administrative systems application areas. received higher satisfaction ratings than PeopleSoft (the CMS software) in each of the three application areas (financial, human resources, and student administration) and across the three performance indicators (response time, ease of use, and quality of information). The survey, which was conducted in the summer of 2002, reflected the responses of 392 individuals who used the CMS financial application, 276 who used the CMS human resources application, and 148 who used the CMS student administration application. Nevertheless, we recognize it is possible that campuses will grow increasingly satisfied with CMS as they become more familiar with it. Although lacking a framework to measure the cost justification of CMS, the university believes the project is cost-justified, suggesting as much to the Joint Legislative Audit Committee (audit committee). In the university’s statement to the audit committee, it cited a projected difference of $50 million in annual costs between its prior administrative systems and estimates of annual CMS costs once fully implemented. In an October 2002 letter to us explaining the university’s January 2002 statement to the audit committee, the university’s executive vice chancellor wrote the following: “The January 2002 statement to the [audit committee] was not an assertion that the [university] would save $50 million with the CMS project. This statement was a response to the question of why a formal [return on investment] was not warranted in 1996. The [university] stated that the projected differences between the estimated annual CMS expenditures of $50 million, once the project was fully implemented, versus the $100 million annual cost reported in the 1996 IBM study led [the university] to conclude that a formal [return on investment] study was not warranted or necessary. The $100 million figure is a rounded total of the costs of the total Administrative Portfolio ($34 million) and Central Computing ($65 million) reported in the IBM study. The $50 million is a cost figure loosely derived from the projected range of potential savings in the IBM study for re-engineering IT delivery ($20 million to $35 million) and Collaborative Administrative systems ($8 million to 2222 California State Auditor Report 2002-110 California State Auditor Report 2002-110 2233 $20 million). This rough savings estimate was further corroborated by the Pacific Partners study that projected a 50 [percent] savings in a fully standardized IT environment. In 1996, staff considered the order of magnitude of the potential savings, rather than a specific savings number. The example of $50 million was used in the initial response to the [audit committee] regarding reasons for no [return on investment] because it gave a clear and representative picture of the thinking in 1996. It has also been stated previously, that [a return on investment] was not necessary based on the fact that the [university] campuses intended to replace and improve administrative systems. The option not [to] replace or improve these systems did not exist making a study of return on investment irrelevant. The projection of significant cost savings was one factor supporting the identification of an [enterprise resource planning system] and data center consolidation as high- priority initiatives in the Integrated Technology Strategy. As the CMS collaborative developed, the CMS task force identified the variety of goals and objectives that formed the basis for the CMS Charter. The CMS Charter does not speak to savings. Rather it points out the objective of achieving efficiencies and minimizing costs. Therefore there was no need to do a stand-alone analysis of potential cost savings.” It is unclear whether the $100 million figure in annual costs cited by the executive vice chancellor reasonably represents the university’s true cost to maintain the systems that were replaced by CMS. For example, of the $34 million in cost for the “administrative portfolio,” only $25 million relates to the CMS functions of finance, human resources, and student administration. The $9 million difference represents the administrative services of facilities services and development. Additionally, the $65 million in “central computing” is questionable because it attributes all campus central computing It is unclear whether the costs to supporting the administrative portfolio rather than $100 million cited by the allocating a proportionate share. However, the survey tool executive vice chancellor used by IBM to collect this cost information does not indicate reasonably represents the whether “central computing” costs includes only administrative true cost to maintain the support costs rather than both administrative and academic systems replaced by CMS. support costs. IBM’s analysis of the university’s IT expenditures indicates that only 21 percent of campus academic and administrative IT costs related to finance, human resources, and student administration systems. The remaining 79 percent relates to campus non-CMS administrative functions, as well as the academic portfolio of systems used by schools and colleges, libraries, instructional media centers, and others. 2222 California State Auditor Report 2002-110 California State Auditor Report 2002-110 2233 The president of the San Diego campus described the supportive role of its campus’s data center to both academic and adminis- trative systems when he said: “Each campus data center has a myriad of other systems that must be supported whether or not the administrative systems are run off-campus. Among these are learning management and other academic systems, campus web servers, 911 systems, cashiering systems, student health systems, calendaring systems, one-card systems, and fax and printer servers. Servicing these systems requires database administrators, operating systems analysts, and operations specialists whose positions and services cannot be eliminated. Running the administrative applications locally, as [the San Diego campus] does currently with Oracle, certainly adds some marginal costs to the local data center.” An alternative analysis of the data in the IBM study would indicate that these marginal annual costs for systems replaced An alternative analysis by CMS would be $43 million rather than $100 million. The indicates that the $43 million is composed of the $25 million related to the university would have CMS functions of finance ($8 million), human resources spent closer to $43 million ($1 million), and student administration ($16 million), as to maintain the systems well as a proportionate share of the total central computing replaced by CMS. and telecommunications costs ($18 million). Although this $43 million is for fiscal year 1994–95, it is remarkably close to the university’s fiscal year 1999–2000 estimate of $42 million in annual pre-CMS costs, presented in the last section of the chapter. According to the executive vice chancellor, as a result of IBM’s and Pacific Partners’ savings calculations, as well as the university’s belief that the option not to replace and improve administrative systems did not exist, the university believed that a return-on-investment analysis was irrelevant. However, as discussed in the next two sections, neither the IBM study nor the Pacific Partner studies make the business case for CMS and its supporting data center because neither was designed to do so and because the university did not implement these initiatives as envisioned in these studies. 2244 California State Auditor Report 2002-110 California State Auditor Report 2002-110 2255 THE UNIVERSITY CANNOT RELY ON THE IBM STUDY AS A BUSINESS CASE FOR CMS The university has said that an IBM study it commissioned was the key document justifying its need to replace legacy administrative systems. However, the study that the university refers to was not designed to evaluate the existing administrative systems and further did not conclude the university should replace them. Rather, IBM compared the university’s existing technology in 1995 with its goal for “anywhere/anytime” electronic access for all university students and employees. Further, IBM’s cost savings estimates assume an entirely different implementation strategy than the university ultimately adopted, so the university cannot use the study to justify the CMS project. In April 1996, IBM presented a report that recommended various technology initiatives, including CMS software and a data center consolidation. IBM describes the CMS initiative as a multi-campus collaboration in the operation and management of administrative software, resulting in savings of $8 million to $20 million that could be redirected to other initiatives. Similarly, IBM said data center consolidation could save $20 million to $35 million. Overall, IBM presented the university with cost savings of $28 million to $55 million annually, money that could help fund other technology initiatives the report recommended. IBM seemed focused on recommending initiatives that could pay for themselves or for other initiatives. In its report, IBM indicates, “due to the signifi cant cost and lack of viable funding sources, the CMS initiative must be closely tied to the The Goals and Objectives of benefi ts from data center consolidation.” In other the IBM Study words, IBM was proposing that savings derived from a consolidated data center be used to fund the The IBM study was charged with identifying CMS project, whose savings in turn would help key existing or new technology initiatives that would do the following: fund the remaining IT initiatives. • Directly or indirectly support the learning and teaching mission. IBM’s purpose in the 1996 study was not to identify and assess the systemwide problems • Achieve economies of scale through collaborative efforts among campuses; of existing administrative systems. Rather, IBM leveraging systemwide resources; and compared the university’s existing technology partnerships with external entities. capabilities to a desired goal for “anywhere/ anytime electronic access to information resources Source: Integrated Technology Strategies Initiative: for all students, faculty, and staff.” IBM concluded Building the Strategic Information Technology (IT) Plan that a “signifi cant performance gap” separated by IBM—April 1996. the university’s current state of technology from its desired state. However, the performance gap 2244 California State Auditor Report 2002-110 California State Auditor Report 2002-110 2255 appears to relate primarily to the university’s ability to deploy technology in teaching and serving students, not in using technology for administrative purposes. According to IBM’s principal consultant on the project, “we built several potential models that varied from one shared copy of the software (requiring only one large programming staff to support it) up to several programming staffs to run several copies that focused on campus size and needs.” He further stated that, “the underlying philosophy behind these initiatives [CMS and data center consolidation] was to commoditize these systems [financial reporting, human resources, and student administration systems] and use the savings to improve learning and teaching on the campuses.” IBM’s report to the university describes CMS as an initiative that “will promote multi-campus cooperation and collaboration in the operation and management of software portfolios, and thereby achieve significant cost efficiencies/savings that can be re-directed to other initiatives.” To understand more fully the amount of potential savings expected from the various models, we asked the university for working papers supporting IBM’s report, which the principal consultant said were left with the university. However, the university could not locate them. The university did not implement the ideas on which IBM’s cost savings were premised, so it is not reasonable to use IBM’s report as the basis for the current CMS project’s business case. IBM recognized that As discussed in Chapter 3 of our report, the university did not the university needed implement one shared copy of the software or several copies to conduct additional that focused on campus size and needs. Also, it did not shut cost-benefit analyses down campus data centers and consolidate all campus comput- to develop a sound ing at consolidated data center(s), as envisioned by IBM. When business case for informed the university’s CIO could not provide us the detailed analysis decisions regarding CMS. and assumptions supporting the IBM study, he gave the best description of the relevance of the IBM study to the CMS cost savings: “Since the current CMS project was not directly based on the potential benefits indicated in the IBM study, compari- sons to the IBM projections would be nothing more than an interesting look back at historical ideas.” Nevertheless, by the time CMS was conceptualized in 1996, IBM recognized that a cost-benefit study should be conducted. It realized that an additional cost-benefit analysis was needed to help campuses develop a sound business case for informed decisions regarding CMS. Based on CMS task force meeting minutes, it appears that IBM even prepared a CMS feasibility workplan to assist campuses in determining whether participation in CMS, which was voluntary at that 2266 California State Auditor Report 2002-110 California State Auditor Report 2002-110 2277 time, was a sound business decision. IBM’s proposed cost- benefit analysis recommended identifying annual costs of existing administrative systems over a three-year period, using a comprehensive focus on hardware, software, installation services, modifications, training, and systems maintenance costs. To explain why the university never used IBM’s proposed feasibility workplan for CMS, the executive vice chancellor contends that “staff working on the project believed that university and campus staff could best handle the work required to further develop the CMS project.” According to the chancellor, he and the CIO and executive vice Although the university chancellor reasoned as follows: states it engaged in significant internal study, “[We] determined that additional study was not necessary to support the campus and system belief that a common it did not prepare a management system would be beneficial to the university cost-benefit analysis that and the corresponding desire of the campuses to develop would make the business that system. At the same time, however, with the full support case for CMS and its and involvement of the campuses, we engaged in significant supporting data center. internal study and development of the collaborative management and data center consolidation concepts.” However significant the chancellor may believe this internal study was, it did not prepare a need or cost-benefit analysis that would make the business case for CMS and its supporting data center. THE UNIVERSITY CANNOT RELY ON THE PACIFIC PARTNERS STUDY AS A BUSINESS CASE FOR A CMS DATA CENTER The university also has cited a 1996 study by Pacific Partners as support for pursuing a CMS data center. However, the Pacific Partners study was not charged with concluding whether a data center that was outsourced, consolidated, or both was the most cost-effective approach to implementing CMS. The scope of this study was limited to identifying the experiences of other organizations that consolidated data centers and applying them to a potential consolidation within the university. As a result, the university’s reliance on the Pacific Partners study is misplaced because the university has adopted a form of data center consolidation contrary to the assumptions that support the consultant’s savings calculations. 2266 California State Auditor Report 2002-110 California State Auditor Report 2002-110 2277 The Pacifi c Partners study tried to determine estimates for potential savings from data center The Purpose and Scope of the Pacifi c Partners Study consolidation across the entire university and estimated the university could reasonably expect The focus of this study was to survey the to save $6 million to $40 million annually, fi eld of known data center consolidations to depending on the form of consolidation selected. examine the following: However, the consultant concluded, “a point • The issues. estimate of what [the university] might reasonably • The benefi ts. expect to save is $22 million in on-going savings based on the benchmark data obtained from • The risks. other institutions and companies and assuming • The best practices found through the a full consolidation of [university] operations experiences of industry, government, and higher education entities. centers and systems support into two centers.” The university did not adopt the Pacific Another purpose was to provide benchmarks, decision criteria, and scenarios for evaluating Partners’ model of consolidating all campus data a possible data center consolidation at the centers into two data centers. It elected instead to university. consolidate only the administrative data center Source: Data Center Consolidation Feasibility Study by operations that are needed to support CMS and Pacifi c Partners—November 1996. outsource the operation of the consolidation, thus leaving the academic and remaining administrative data center functions under the purview of the existing campus data centers. Although it reported that the university might reasonably expect $22 million in annual savings through campus data center consolidation, Pacifi c Partners also recognized that the university needed to evaluate data center costs in more detail. Pacifi c Partners’ savings estimates were based on comparisons to various industry benchmarks and relied on university cost data from the prior IBM report. Knowing that its own study did not try to compute the university’s specifi c hardware, software, telecommunications, personnel, and other needs to consoli- date data centers, Pacifi c Partners told the university that one of the most important actions going forward was “a feasibility/ implementation study that looked at the full life-cycle costs.” Pacifi c Partners also informed the university that “estimating the savings that might exist throughout the [university] system through data center consolidation is diffi cult because detailed information for data center expenditures is not available. To do a proper estimate, each facility should be fully examined and fully costed on a life-cycle basis.” As stated earlier, the university did not perform such a cost analysis, and the Pacifi c Partners study did not consider whether a consolidated or outsourced data center was the most cost-effective way to implement CMS. As a result, there is no basis for the university to rely on this study as support for its efforts to establish the CMS data center. 2288 California State Auditor Report 2002-110 California State Auditor Report 2002-110 2299 The executive vice chancellor informed us that, “the participat- ing campuses in the CMS effort collectively determined that data center consolidation would be the most effective means to provide the necessary hardware and staffing to serve the CMS data processing requirements.” However, when we asked campus presidents whether their campuses had any studies, reports, or similar analyses supporting this determination, they responded that the campus was not involved in this decision or that the campus did participate in this determination, but could not refer us to any studies other than those by IBM and Pacific Partners. THE UNIVERSITY DID NOT REQUIRE CAMPUSES TO USE THE FINANCIAL ANALYSIS TOOL IT CREATED The university created its own financial analysis tool that might have helped make the business case for CMS and its supporting data center; however, it did not require the campuses to use this tool, which would have given the university some analysis of cost savings. The chancellor made the following statement about the financial analysis tool: “The Collaborative Management Systems Campus Financial Planning Tool . . . was developed to help campuses in their own assessment and decision-making regarding participation in a collaborative system. The CMS Task Force did not direct the use of this document. Participation decisions regarding the use of this tool were campus based and the cost of campus implementation would be borne by campuses individually.” According to the executive vice chancellor, “The tool was provided to all campuses through distribution of the Agenda The information from its for the September 23 (1997) Task Force Meeting.” This financial own financial analysis analysis tool seems to be a worthwhile document because it tool would have yielded provided a structure for each campus to compare the projected important comparative cost of maintaining its current systems for the next five years cost data before the with the projected cost of installing and maintaining a new university decided to system over the next five years. Such a comparison would pursue CMS; however, have allowed the chancellor’s office to assess the likelihood of no campus could provide achieving cost savings while also providing campus presidents a copy of the completed with important comparative cost data. Such information likely financial analysis tool. would have proved useful when it was decided in July 1999 to implement CMS systemwide as opposed to allowing campuses to determine their own participation in the CMS project. 2288 California State Auditor Report 2002-110 California State Auditor Report 2002-110 2299 However, when we asked each campus if it completed the financial planning tool, and if so to provide us a copy, no campus could give us a copy of its completed form. A couple of campuses indicated that the financial planning tool might have been prepared, but no longer could be located. Others indicated that they prepared various cost analyses, or relied on cost estimates from the chancellor’s office for small, medium, and large campuses; however, these campuses did not compare existing system costs with new system costs. Still another said it became moot once a decision was reached that the PeopleSoft- based CMS would be installed at all campuses. THE UNIVERSITY’S STATED REASONS FOR CMS AND THE DATA CENTER ARE NOT SUFFICIENT TO JUSTIFY THE SIGNIFICANT INVESTMENT WITHOUT MAKING A BUSINESS CASE Rather than establishing a business case, the university has given various reasons for pursuing a systemwide implementation of Although the university’s CMS, including those stated in the university’s “Why CMS?” various reasons for a document. In this document, the university asserted that it systemwide CMS have was headed for “severe” problems with its prior administrative some merit, they software, so it had to do something. Also, the executive vice do not justify spending chancellor indicated that campuses intended to replace and $662 million over improve administrative systems anyway, thus making a return- a nine-year period on-investment study irrelevant. In addition, the chancellor without establishing says the IBM and Pacific Partners studies validated what the a business case. campus presidents already knew: “Improved administrative systems were necessary in the [university] to provide the level and type of services required.” Although each reason has some merit, individually and collectively they do not justify spending $662 million over the nine-year project period (an estimated $393 million for one-time costs and an estimated $269 million in maintenance and operations costs) without establishing the business case. The University Cannot Support That It Had Severe Problems With Its Administrative Systems as Described in Its “Why CMS?” Document In October 1999, more than a year after it had acquired the software from PeopleSoft, the university explained its reasons for pursuing the project in its “Why CMS?” document. Although identifying general software problems that could reasonably justify analyzing alternatives for software replacement, “Why CMS?” documented neither the severity of the problems nor 3300 California State Auditor Report 2002-110 California State Auditor Report 2002-110 3311 their pervasiveness across the campuses. In this document, the university lists software problem Why the University Is Replacing Prior Administrative Systems With CMS areas (see textbox) such as eroding vendor support and increased maintenance costs for systems • Prior administrative systems have reached that reportedly had reached the limits of their their technical limitations. technical capabilities. • Inability to maintain prior administrative systems and eroding vendor support. To understand the university’s basis for identify- • Interfacing with the state controller’s ing these severe and systemwide problems, we planned new payroll system. (Note: asked the CIO to explain how these problems were ultimately, this planned system did not materialize.) discovered. In response, he indicated that “the systemwide business need for the CMS project was • University demand for timely and accurate management information marginally met. a direct refl ection of individual campus needs for improvement and change and that their need was • Growing cost of maintaining the prior expressed repeatedly by numerous campuses in administrative software. various committees and task forces throughout the Source: The university’s “Why CMS?” document— early nineties and before.” However, the CIO was October 1999. unable to identify specifi c meetings or to provide any documentation to suggest that numerous campuses expressed a desire to replace administra- tive systems at that time. The university’s CIO also was unable to provide any analyses documenting the extent and severity of the problems cited in the “Why CMS?” document. The University Cannot Support That Most of Its Campuses Were Planning to Replace Administrative Systems in July 1999 The executive vice chancellor informed us that a return-on- investment calculation “was not necessary based on the fact that [university] campuses intended to replace and improve administrative systems. The option not [to] replace or improve those systems did not exist, making a study of return on investment irrelevant.” However, the extent to which campuses were intending to replace their software administrative systems is not as clear-cut as the executive vice chancellor suggests. When the university began its software procurement process in April 1997, the request for qualifi cations (RFQ) it sent to potential bidders indicated that six campuses planned to replace fi nancial reporting systems, six campuses planned to replace human resource systems, and two campuses planned to replace student administration systems. The RFQ indicates specifi c campuses planning to replace certain administrative systems and names the planned replacements, but it does not mention any other campuses planning to replace systems that had not identifi ed potential replacements. 3300 California State Auditor Report 2002-110 California State Auditor Report 2002-110 3311 We surveyed all campus presidents to determine if they supported the common system because of their specific campus needs in 1999 or for the university’s common good. Many campuses indicated The specific campus responses varied. Many indicated they that they had campus- had campus-specific wants, needs, or requirements for new specific wants, needs, administrative software, but only a few told us they tried to or requirements for new address campus-specific concerns by soliciting information or administrative software; proposals from vendors. The response from the Los Angeles however, only a few told campus was consistent in certain respects with various other us they tried to address campuses. In the area of student administration systems, these before CMS. Los Angeles indicated that it was informed that its system no longer would be supported because another vendor bought the vendor that developed its software. Los Angeles was one of nine campuses using this software based on the April 1997 RFQ. Los Angeles did not have a human resource system and relied on the state controller’s payroll system for human resource information. It was one of seven campuses in this situation based on the April 1997 RFQ. On the financial side, the campus was running a heavily modified version of a university-developed financial reporting system. According to the Los Angeles campus, the design of this system was based on requirements to meet the chancellor’s office’s specific needs and systemwide reporting requirements, not local campus needs. All the campuses were running the same university financial reporting system based on the April 1997 RFQ. The critical nature of these shortcomings is unclear because the university indicated to its prospective vendors in 1997 that it was planning to replace so few systems. The Los Angeles campus differs from most other campuses in three important respects. Unlike the other campuses, the April 1997 RFQ indicated that Los Angeles was the only campus planning to replace all three of its administrative systems at that time. Second, only the Los Angeles campus sent us a feasibility study that presented a business case for software replacement, even though we asked all campus presidents to submit all reports, studies, or similar analyses that led them to conclude that the business needs at their respective campuses were severe enough to warrant replacing existing financial, human resources, and student administration systems. Third, according to the campus president, by the time the university decided all campuses would implement CMS, the campus already had implemented most of a PeopleSoft suite of administrative software, and it was in the process of implementing the student administration application to solve its problems. 3322 California State Auditor Report 2002-110 California State Auditor Report 2002-110 3333 Similarly, by the time of the decision, the Fresno, San Marcos, and San Diego campuses already had acquired administrative systems to address their needs. As a result, the need for these campuses to replace their administrative systems is unclear. Further, 19 campuses indicated they had campus-specific wants, needs, or requirements for new administrative software, but only three campus presidents told us they had tried to address campus-specific concerns by soliciting information or proposals from software vendors. This lack of specific action by all but a few campuses does not rule out the possibility of making a business case for the other campuses or for a systemwide implementation. Although undoubtedly, the administrative systems at some campuses would have been replaced, the 1997 RFQ status of planned administrative system replacement, along with the absence of campus feasibility studies and the lack of campuses completing the voluntary September 1997 Collaborative System Financial Planning Tool previously discussed, collectively call into question how seriously most campuses were planning to replace administrative systems by July 1999 when the university decided to install a PeopleSoft- based system at all campuses. The Chancellor’s Reasons For Supporting the Need For CMS Are Not Sufficient to Eliminate the Need to Develop a Business Case The chancellor believes that one of the most critical factors in CMS’s success and value to the university is its ability to draw campuses into more common business practices. He also Without cost-benefit or asserts that CMS will benefit the university through increased similar feasibility studies, economies of scale that will allow it to direct more energy there is insufficient toward its primary mission—education. Common business analysis to document the practices and increased focus on education are undoubtedly system will accomplish worthwhile objectives, but these reasons by themselves do the maximum possible not override the need to establish a business case for CMS by benefit at the least explicitly stating these objectives and performing a cost-benefit possible cost. study or similar feasibility study report. Without these studies, there is insufficient analysis to document that the system is being installed for the least possible cost to accomplish the maximum possible benefit. We asked the chancellor to provide us with the specifics behind his belief that “improved administrative systems were neces- sary in the [university] to provide the level and type of services 3322 California State Auditor Report 2002-110 California State Auditor Report 2002-110 3333 required.” We asked the chancellor this question to understand his basis for concluding that new systems were required as opposed to wanted. The chancellor replied as follows: “‘Required’ and ‘wanted’ are synonymous in this context. When faculty, staff, and students want certain functionality or services to be able to teach, support, and learn, then that functionality and service is ‘required.’ One could certainly argue that faculty, staff, or student wants fall in varying levels of requirements. However, today’s wants often become tomorrow’s requirements. For example, Internet access to personnel records was a want eight years ago; today it is a requirement for a modern competitive university.” We believe that at some point in the process of making CMS a systemwide project as opposed to a voluntary collaborative, the university should have halted the process and reevaluated the At some point in the approach for implementing the system. This important decision, process of making CMS which may have been the correct decision, greatly expanded the a systemwide project, university’s potential to achieve its business objective to “mini- the university should mize cost to implement and maintain application software.” have reevaluated By reevaluating, the university would have had the opportunity the approach for to analyze, identify, and manage the least cost implementation implementing the system. systemwide to achieve the maximum benefit systemwide. The chancellor disagreed that at some point in the process of making CMS a systemwide project, as opposed to a voluntary collaborative, the university should have halted the process and reevaluated the rationale for the project. By taking this approach to evaluating the CMS project, he believes we would be missing one of the most critical factors in its success and in the value it has and will bring to the university. The chancellor expresses this value as follows: “CMS has helped draw together all campuses in the [university] not only into a common management system, but also into more common management and business practices. We are sharing our expertise and knowledge to collaboratively develop our business practices in all three areas [student administration, financial reporting, and human resources]. The economies of scale in cost and in personnel time that we achieve will allow us to focus our energies on our primary mission, education.” In contrast, we discuss in Chapter 3 how the university is not achieving the maximum functionality from a systemwide imple- mentation, and in Chapter 4 we discuss how its procurement and bidding process was not designed to identify the best-value vendor for a systemwide implementation. 3344 California State Auditor Report 2002-110 California State Auditor Report 2002-110 3355 All the university’s reasons have some merit. We also appreciate that whether a particular functionality is a requirement or a The university currently want is subjective and can change over time. However, none recognizes the need to of these reasons is a basis for not making the business case show a return on its for CMS by documenting the need for new software and investment in CMS in terms performing a cost-benefit study or similar feasibility study of better and more efficient report. The university currently recognizes the need to show a administrative practices. return on its investment. As stated in its April 2002 update to the board regarding its information technology strategy, “These projects must begin to show a return on investment in terms of improved environments for learning and teaching and better and more efficient administrative processes.” However, absent such an analysis, a judgment cannot be made whether the increased functionality obtained by the new system is worth the additional recurring costs to maintain and operate it. As discussed in the following section, it appears that these recurring costs will exceed similar costs in the pre-CMS environment. THE UNIVERSITY IS LIKELY TO SPEND MORE, RATHER THAN LESS, TO MAINTAIN AND OPERATE ITS NEW SOFTWARE Based on a rough comparison of fiscal year 1999–2000 maintenance and operations costs under the pre-CMS environment and the projected costs for the fully implemented CMS in fiscal year 2006–07, it appears the university may spend about $23 million more annually to maintain and operate this new software compared with its pre-CMS environment. We recognize that the CMS software provides campuses with some functionality that did not previously exist. Further, some figures presented in Table 2 on the following page are disputed by the university or represent rough projections of future costs. Finally, the cost comparison does not account for the effects of inflation or university growth. Nevertheless, the comparison raises doubt about the university’s ability to achieve one of its stated business objectives for CMS, namely minimized cost to maintain application software. 3344 California State Auditor Report 2002-110 California State Auditor Report 2002-110 3355 TABLE 2 Comparison of Annual Maintenance and Operations Costs Between Prior Administrative Systems and CMS Actual Projected Pre-CMS Annual CMS Annual Maintenance and Maintenance and Operations Costs Operations Costs (fiscal year (fiscal year Increase 1999–2000) 2006–07) (Decrease) Chancellor’s office $ 1,380,000* $29,346,376 $27,966,376 Combined campuses 40,340,730† 35,760,740 (4,579,990) Total annual maintenance and operations costs $41,720,730 $65,107,116‡ $23,386,386 Source: The university’s November 2000 Measures of Success report; the university’s June 2002 cost survey. * The director of business management services estimated that the chancellor’s office spent this amount supporting administrative systems that were subsequently replaced by CMS. † This amount only reflects maintenance and operations costs reported by the 22 campuses in existence at the time. Channel Islands’ annual maintenance and operations costs are reflected in fiscal year 2006–07, amounting to $73,158. Chancellor’s office “campus” costs, representing the cost of maintaining and operating human resources and finance applications at the chancellor’s office, are also included in fiscal year 2006–07, amounting to $779,300. ‡ This amount includes the projected ongoing costs to maintain and operate the version of CMS that would exist in fiscal year 2006–07. However, it does not reflect the costs for upgrades or additional development efforts that the CMS software may require in the future. As Table 2 shows, the university reported to the Legislature in November 2000 that campuses spent approximately $40 million annually to maintain and operate its pre-CMS systems during fiscal year 1999–2000. The university further estimates that the chancellor’s office spent more than $1 million in additional costs during this period to maintain and operate its pre-CMS environment, amounting to almost $42 million in pre-CMS systemwide costs. Although the university subsequently has questioned the validity and usefulness of the $40 million it reported to the Legislature in campus pre-CMS costs, that figure is the only information campuses have reported for that time period. This $40 million figure, taken with the chancellor’s office costs of more than $1 million, approximates the comparable pre-CMS costs for fiscal year 1994–95 mentioned earlier in our discussion of the IBM study. 3366 California State Auditor Report 2002-110 California State Auditor Report 2002-110 3377 The university projects that its annual CMS maintenance and operations costs will exceed $65 million in fiscal year 2006–07. During June 2002, our office surveyed campuses to quantify their to-date and projected expenditures on the CMS project. According to the June 2002 cost survey conducted at the bureau’s request, campuses project that CMS maintenance and operations costs will be nearly $36 million annually once fully implemented, almost $5 million less than the pre-CMS environment. However, the chancellor’s office will incur $29 million in annual costs to support the new software, which is $28 million more than its costs under the pre-CMS environment. This $29 million is composed of $14 million for support personnel; $8 million in annual payments to Unisys in support of the CMS data center; and $7 million annually for other CMS costs such as consultants, hardware, and software licenses. RECOMMENDATIONS To ensure that the university’s future IT projects are appropriate expenditures of state resources, the university should adopt policies and procedures that require a feasibility study before the acquisition and implementation of significant IT projects. Such a feasibility study should include at least a clearly defined statement of the business problems or opportunities being addressed by the project, as well as an economic analysis of the project’s life-cycle costs and benefits compared with the current method of operation. To measure the benefits achieved through common management and business practices, the university should establish quantitative measures of increased business process efficiencies through CMS, in addition to any qualitative factors being assessed. Such measures should identify the increased productivity of staff, reduced operational costs, or both. n 3366 California State Auditor Report 2002-110 California State Auditor Report 2002-110 3377 Blank page inserted for reproduction purposes only. 3388 California State Auditor Report 2002-110 California State Auditor Report 2002-110 3399 CHAPTER 2 The University’s Project Costs Exceed Initial Estimates, and Cost Monitoring Procedures Are Inadequate CHAPTER SUMMARY Recent data indicate that the California State University’s (university) earlier estimates understated costs for its Common Management System (CMS) project, both in the 1998 projections of $332 million to $400 million and 1999 projections of $440 million. At the bureau’s request, the university conducted a more comprehensive review of actual CMS expenditures and projections in June 2002, including individual campuses’ projections of their costs for the project. This June 2002 cost survey revealed that total project costs for the types of expenses the university initially estimated—what it considers to be “new” costs—now total $482 million, which includes the cost for operations and maintenance of the data center during the implementation period. The $482 million excludes costs the university did not estimate related to the project, including $63 million in implementation costs charged to other campus budgets and $117 million in maintenance and operations costs at the campuses during the nine-year development and implementation period. Adding these costs to the university’s latest cost estimate brings the CMS total project costs to $662 million for implementation, maintenance, and operations through fiscal year 2006–07. However, the accuracy of these figures is uncertain because 73 percent of the projected $662 million is estimated costs. The bulk of these projected expenditures are for consulting and university personnel costs, which add up to more than $463 million, or 70 percent of the total project cost. Another major cost is the university’s expenditures on the related data center, which, including telecommunications and other costs, is expected to exceed $78 million over the project’s nine-year term. About $3.7 million of this is associated with the university’s failed contract negotiations with IBM and the need to establish a contingency data center in fiscal year 2000–01. 3388 California State Auditor Report 2002-110 California State Auditor Report 2002-110 3399 Contributing to a lack of complete cost information for univer- sity management and the public, the university has monitored overall systemwide project costs inadequately since beginning the CMS project in 1998. In a striking lapse, the university lacked actual and projected campus cost information for the project when we began our review, resulting in incomplete public reports of estimated project costs. The university told us it planned to gather cost data from campuses in fall 2002. It subsequently surveyed campuses for this information in June 2002 after we expressed our concern about the incomplete information. Similarly, the university does not have a system- wide funding plan for the CMS project. RECENT COST PROJECTIONS FOR THE CMS PROJECT EXCEED INITIAL COST ESTIMATES, WHICH DID NOT INCLUDE ALL RELATED COSTS Updated university cost projections indicate that the amount it expects to spend over the life of the CMS project exceeds the original estimates. Further, additional costs—such as ongoing maintenance and campus employees working on the project while being paid from other campus budgets—should be Nine months after it considered when computing the project’s total cost over the purchased the PeopleSoft nine-year project period. In a January 2002 statement to the software, the university Joint Legislative Audit Committee, the university reported that prepared its first its first “rough” estimate of costs for the CMS project in 1998 comprehensive CMS cost totaled $350 million to $400 million over a projected seven-year projection in June 1999. development and implementation period. The university’s chief information officer (CIO) told us this estimate was developed very cursorily to give an overall picture of the project’s potential cost, not for decision-making purposes. In fact, we saw no documentation from 1998 related to an estimate of $350 million to $400 million for a seven-year period. However, the university does have documents showing a “preliminary estimate” in December 1998 that projected total costs of $332 million for an eight-year period. Although this estimate indicated that the university was at least starting to consider the cost of implementing CMS at all campuses, the CIO contends that the systemwide project was not defined sufficiently for the university to develop a comprehensive cost projection until mid-1999. In June 1999, nine months after it purchased the PeopleSoft software, the university prepared what it refers to as its “first comprehensive cost projection” for the CMS project, in which 4400 California State Auditor Report 2002-110 California State Auditor Report 2002-110 4411 the university’s estimate grew to $440 million over a nine- year development and implementation period. This projection documented detailed costs by major types of expenses such as personnel and consultants, estimating centralized development costs of $210 million and campus costs of $230 million. The university based the $230 million campus cost estimate on its average campus cost estimate of $10 million multiplied by 23 campuses (the 22 campuses in existence at the time and the chancellor’s office’s campus costs). However, recent cost projections from the June 2002 cost survey of the now 23 campuses and the chancellor’s office conclude that the university should expect to spend about $662 million on the CMS project, including an estimated $393 million for one- time costs and an estimated $269 million for maintenance and operations costs, by the end of the projected nine-year development and implementation period ending June 2007. Early in the audit, our requests for project cost information The chancellor’s office prompted the chancellor’s office to survey the 23 campuses never tried to gather in June 2002. Until then the chancellor’s office had never project cost data from tried to gather cost data on the project from the campuses, the campuses until although it told us it planned to do so in fall 2002. In addition June 2002. to providing a total projected cost of $662 million for the CMS project, the university’s 2002 cost survey provided specific information on project costs, such as actual and estimated costs by type of expense such as personnel, training, travel, consultants, hardware, and software. Figure 2 on the following page shows this breakdown of expenses. This survey also provided information by the primary areas of the project such as implementation and maintenance and operations, which we discuss later in the chapter. 4400 California State Auditor Report 2002-110 California State Auditor Report 2002-110 4411 FIGURE 2 Projected Total Costs for the CMS Project Including Maintenance and Operations Costs Fiscal Years 1998–99 Through 2006–07 ��������� ����� ��� �� ����������� ����������� ��������� ��� ����������� ��������� ��� ������������ ����������� ��� ������������ ������ �������� �� �� ����������� ����������� Source: The university’s June 2002 cost survey. * Software costs include PeopleSoft software costs. † Hardware costs include Unysis data center costs. In the June 1999 estimate, the university considered only what it termed “new costs.” The university’s 2002 cost survey highlighted an increase of about $42 million over the June 1999 estimate of $440 million for these kinds of costs. Table 3 shows the increase from the June 1999 cost estimate to those projec- tions recently collected in the June 2002 cost survey, by type of cost, for central costs incurred by the chancellor’s office, and for campuses. 4422 California State Auditor Report 2002-110 California State Auditor Report 2002-110 4433 TABLE 3 A Comparison of the 1999 and 2002 CMS Project Cost Estimates for the Nine-Year Project Period June 1999 June 2002 Estimate Growth Cost Item Cost Estimate Cost Estimate or (Shrinkage) Central Costs: Personnel $ 26,735,486 $ 73,714,004 $46,978,518 Training and Travel 5,144,228 5,144,228 Consultants 70,840,335 31,264,032 (39,576,303) Hardware 400,000 1,313,555 913,555 Data Center* 75,000,000 75,176,426 176,426 Software 4,126,357 4,126,357 PeopleSoft Software 34,876,961 39,876,812 4,999,851 Other 1,757,000 5,800,515 4,043,515 Central Costs Total 209,609,782 236,415,929 26,806,147 Campus Costs:† Personnel 110,400,000 74,374,321 (36,025,679) Training and Travel 23,575,000 21,869,074 (1,705,926) Consultants 85,100,000 127,679,650 42,579,650 Hardware and Software 10,925,000 15,214,506 4,289,506 Other 6,236,258 6,236,258 Campus Costs Subtotal 230,000,000 245,373,809 15,373,809 Subtotals $439,609,782 481,789,738 $42,179,956 Other Campus Costs:‡ In-Kind 63,378,034 Maintenance and Operations 116,714,146 Total Project Costs $661,881,918 Source: The university’s June 1999 implementation approach analysis document and June 2002 cost survey. * The June 2002 cost estimate for the data center costs include other nonpersonnel costs of $728,683, outsourced data center costs of $70,223,133, and data center related telecommunications costs of $4,224,610. † For its June 1999 estimate, the university estimated implementation costs of $10 million for one campus and extrapolated to $230 million for 22 campuses plus the chancellor’s office “campus” costs. This estimate excluded campus in-kind costs as well as maintenance and operations costs. Further, although upgrade and integration costs are additional or new costs resulting from the CMS project, the university also excluded these from its 1999 estimate. ‡ The campus in-kind and maintenance and operations costs, which are shown separately on this table, are included in the various types of costs, such as personnel, shown in Figure 2. Thus, the individual cost items in Figure 2 and Table 3 are not comparable. As Table 3 shows, the largest increase in the estimated central project costs, almost $47 million, occurred in personnel costs in the chancellor’s office, as staff grew to accommodate the workload involved with modifying the software and providing 4422 California State Auditor Report 2002-110 California State Auditor Report 2002-110 4433 support for campuses. Some of this increased personnel cost, according to the university, reflects an effort to move from a consultant-based staff to a university personnel-based staff ahead of schedule. This is reflected in a reduction of the projected consultant cost at the chancellor’s office from nearly $71 million in the June 1999 estimate to about $31 million in the June 2002 cost survey. In exactly the opposite pattern, the campuses have seen a drop in the projected university personnel costs while seeing a similar increase in the projected consultant costs. Further, the university reportedly did not consider integration and upgrade costs (discussed later) in its 1999 estimate, although these are additional costs that result from the implementation of CMS. Finally, the university did not consider inflationary or growth factors in its 1999 estimate, which may have contributed to the increase. A More Complete Picture of What the University Will Spend Includes Other Campus Costs The June 2002 cost survey revealed two campuses’ costs—their maintenance and operations costs and in-kind costs—that the university did not include in its June 1999 estimate because its The university’s June focus was only on new costs. The university expects to spend 1999 CMS estimate more than $63 million for in-kind costs, which represent indi- did not include about rect costs, or costs that are paid from other campus budgets. For $180 million in campus example, in-kind costs occur when a manager from the finance costs because its focus department spends a month or more working a majority of the was only on new costs. time on “fit-gap” sessions that are necessary for the business process reengineering required by CMS implementation, but the finance department, not the CMS budget, pays for the manager’s time. In its June 1999 estimate, the university did not consider in-kind costs because they did not represent a new cost to the university. However, paying CMS bills out of a different budget does not take that expense out of the CMS project’s total cost. Further, the June 2002 cost survey reported that the university expects to spend $117 million in campuses’ maintenance and operations costs through fiscal year 2006–07. The university did not consider such costs in its June 1999 estimate, contending that campuses’ CMS maintenance and operations costs are not new costs because this type of cost already existed with the exist- ing legacy systems being replaced by the CMS project. However, at the chancellor’s office, the maintenance and operations costs were considered in the June 1999 estimate because they repre- sented a cost that did not exist previously. To compute the total cost the university will spend during the project period, we 4444 California State Auditor Report 2002-110 California State Auditor Report 2002-110 4455 have included the $117 million in maintenance and operations costs for the campuses to provide the most comprehensive and consistent accounting of the university’s costs in terms of personnel, consultants, and other expenses related to the CMS project, which we detail in the remainder of this chapter and the related appendices. The University’s Projected Total Costs Are Subject to Uncertainty, and Campus Cost Projections Vary Widely The projected total costs of CMS are uncertain for several reasons, the main ones being that 73 percent of the $662 million total costs are based on estimates for future years and some of these costs, such as in-kind costs, are especially hard to estimate. Seventy-three percent Most costs are based on estimates because the cost figures for of the $662 million in the project cover nine years and there are only four years of projected total costs for actual data included, which accounts for about $176 million CMS over the nine-year out of the $662 million the university expects to spend on the project period represents CMS project. The fifth through ninth years are projected costs estimates for future years. prepared by each campus and the chancellor’s office to estimate what they think their CMS project costs will be during those years. Also, some types of costs are more uncertain than others. For example, the $63 million in in-kind costs discussed in the previous section are particularly rough estimates because the university does not track the hours each employee spends on a given task. In an effort to estimate those costs, campuses were instructed to report only the costs for staff spending more than 50 percent of their time on CMS-related tasks in a given month. Further, the accuracy of the cost estimates is likely influenced by how far along campuses are in the implementation process. Nearly two-thirds of the reported total costs come from the campuses. Figure 3 on the following page provides an annual snapshot of how many campuses have and will have implemented the three CMS applications each year through fiscal year 2006–07. It does not take into consideration the different versions of each CMS software application a campus will be implementing, but instead counts a campus’s initial implementation of a CMS application. As of the end of fiscal year 2001–02, 11 campuses had begun using the human resources application, and 10 had begun using the finance application. The first campuses did not start using the student administration application until fiscal year 2001–02. 4444 California State Auditor Report 2002-110 California State Auditor Report 2002-110 4455 FIGURE 3 Number of Campuses “Live”* on CMS by Application �� �� �� �� � � ��������� ������� ������� ������� ������� ������� ������� ������� ����������� Also, project costs vary greatly between campuses. For example, the San Diego campus is at the high end with projected costs at almost $37.6 million, and the new Channel Islands campus is at the low end with about $862,000 in projected costs. Although it is reasonable to assume costs differ significantly among campuses for various reasons, such as differences in size, other more unique factors can cause costs to differ. For example, the Channel Islands campus opened just recently, and this may contribute to its lower estimate of costs. Further, it is important to note that some campuses are further along in the implementation process and presumably would have a better understanding of what total actual costs might be. For instance, in September 2002 the Fresno, Sonoma, and California Maritime Academy campuses became the first campuses to have 4466 California State Auditor Report 2002-110 California State Auditor Report 2002-110 4477 ���������������� �������������� ������� ��������������������� ��� �� �� ��� �� �� �� �� �� �� �� � � � � � � Source: CMS deployment timeline as of December 2002. * “Live” means a campus has completed testing on at least some functionality of its initial version of the application, and is now using it in the day-to-day operation of the campus. † Includes all 23 campuses and chancellor’s office. ‡ Includes all campuses except Channel Islands. implemented all three applications. Although they still are planning to implement some upgrades, one might expect the costs they submitted as part of the June 2002 cost survey to be the most complete. A campus’s progress in the implementation process can have another effect on reported costs. Campuses implementing earlier in the nine-year period would report higher maintenance and operations costs than if they had implemented later. For example, Fresno, Sonoma, and California Maritime Academy’s reported costs would reflect not only the already incurred implementation costs, but also projections for nearly a full five years of maintenance and operations costs for all three applications until the systemwide project is implemented fully. On the other end of the spectrum, the San Diego campus does not plan to start the implementation process on its first two CMS applications until April 2005 and does not expect to complete implementation of any of its three applications until July 2006. Thus, San Diego’s cost projections generally would include less than one year of maintenance and operations costs, so one might expect its costs to be lower than those campuses who were further ahead in the process. In fact, the cost projection reported by the San Diego campus was one of the two highest systemwide. Campus staff indicated that the high cost projections were based partly on the campus’s experience with implementing similar systems. Table B.1 in Appendix B shows how much of each campus’s total projected costs are for maintenance and operations. Another factor driving the cost of campus implementation is the differing needs of the campuses. The CMS software is modular by design, allowing for different levels of functionality. This allows each campus to implement only the functionality elements, such as modules, that they believe best meet their needs. However, the flexibility of this approach has other We found many consequences, such as limits on systemwide reporting, which are inconsistencies in the cost discussed in Chapter 3. information submitted by campuses—not surprising Finally, we found many inconsistencies in the June 2002 cost because this was the survey submissions, not surprising because this was the first first time the chancellor’s time the chancellor’s office gathered project cost data from the office gathered such campuses. Some campuses continued to report implementation information. costs for a given application in years after the implementation was complete, while other campuses failed to show costs for the third-party software being used to integrate other campus systems. Sixteen campuses failed to show all costs for their 4466 California State Auditor Report 2002-110 California State Auditor Report 2002-110 4477 ongoing maintenance and operations of CMS. One campus listed costs not related to the CMS project. After contacting each campus to discuss the inconsistencies we noted in its survey submission, each campus was given the opportunity to make corrections and resubmit its June 2002 cost survey. Eighteen campuses responded with some changes, but considering that 73 percent of the total project costs are projections, more inaccuracies could exist and more refinement of these estimates is likely to be warranted. Our review also found that reported project costs are understated because, as we discuss in Chapter 5, the university has not included the related CMS data storage and retrieval (data warehousing) costs as a central cost of the project. Further, we noted that only a few campuses included CMS data warehousing costs in the June 2002 cost survey. OTHER COSTS CONTRIBUTE TO IMPLEMENTATION AND TOTAL PROJECT COSTS Of the $662 million, the university projects a total one-time investment of $393 million for the CMS project. Included in this investment total are costs for implementation as well as other costs such as those for upgrading to the most recent version of Maintenance and the software. Ongoing costs of maintaining and operating CMS operations costs add will add more than $269 million to the total costs during the $269 million to the total nine years of systemwide development and implementation. costs during the project’s nine-year term. The university’s June 2002 cost survey provided information on the primary project areas. Typically, these areas are implemen- tation and maintenance and operations. However, in part, to disclose the more detailed information required for this audit report, the university further isolated certain costs: integration, in-kind, and upgrade costs. Table 4 displays the total projected costs of the CMS project by these five areas and breaks out the total investment versus the continuing maintenance and operations costs. 4488 California State Auditor Report 2002-110 California State Auditor Report 2002-110 4499 TABLE 4 Summary of Projected Total Costs by Project Area Fiscal Years 1998–99 Through 2006–07 Cost Percent of Cost One-Time Investment Costs Implementation $291,349,354 44.0% Integration 12,868,156 1.9 In-Kind* 63,433,034 9.6 Upgrade 24,957,016 3.8 Investment Total 392,607,560 59.3 Ongoing Costs Maintenance and Operations 269,274,358 40.7 Total† $661,881,918 100.0% Source: The university’s June 2002 cost survey. * The In-Kind costs, which include $55,000 of central costs, have been broken out as follows: Implementation $50,889,187, Integration $4,873,451, and Upgrade $7,670,396. † Included in the total costs are $83,855,717 in investment costs and $152,560,212 in maintenance and operations costs incurred by the chancellor’s office for its “central” efforts on the CMS project. The chancellor’s office does not differentiate between investment and maintenance and operations costs when it accounts for its costs. Thus, it allocated its costs between these two areas based on when the campuses begin using each application. The $662 million total represents both the one-time and the recurring costs that the CMS project has incurred, or is projected to incur, during the nine-year systemwide project development and implementation effort. Campuses reported costs in the following areas: • Implementation—Costs related to the initial design, devel- opment, and implementation of CMS. [Traditionally the definition of these costs covers the entire investment, or the one-time expenditures, in the project cost. However, in part because of the information required for this report, the university separately reported the integration, in-kind, and upgrade costs.] • Integration—Costs related to the development and imple- mentation of interfaces between the CMS and other existing campus systems such as legacy systems, telephone registra- tion, or other service packages. 4488 California State Auditor Report 2002-110 California State Auditor Report 2002-110 4499 • In-Kind—Costs related to the development and implementa- tion, integration, and upgrade of CMS, but paid from other campus budgets. The university reported costs for staff working on CMS project-related tasks more than 50 percent of the time in a given month, and who are not reported in The $662 million total the CMS campus budget or replaced with staff during the represents the cost to project assignment. achieve and maintain the implemented • Upgrade—Costs related to software upgrades and additional and planned CMS development and implementation of functionality elements. functionality identified This category covers upgrades from the 7.x versions to the by the campuses during 8.x versions of the CMS software and the implementation the nine-year systemwide of additional CMS functionality elements purchased under development and the PeopleSoft agreement but not provided to the campuses implementation effort. during their initial implementation. [Traditionally, upgrades are considered a recurring cost because software companies continually release new versions of the software that must be modified for systems such as CMS and implemented every few years. However, to compute the cost to achieve the imple- mented and planned functionality that campuses expect to achieve by the end of fiscal year 2006–07, as displayed in Appendix E, we have reported the cost of campus upgrades to achieve this target functionality as an investment cost.] • Maintenance and Operations—Costs related to ongoing main- tenance and operations once the CMS application is “live” at the campus. A campus generally goes “live” after it completes testing and begins using certain functionality of an applica- tion. Maintenance and operations is the recurring cost of the daily operation and support of CMS and includes all the costs of labor for hardware and software maintenance, and other costs related to the daily operation and maintenance of CMS after it is implemented. University campuses differ in size, needs, organizational structure, and what kind of legacy system was in place before the CMS project. Therefore, not all campuses incur the same kinds of costs. The San Diego campus does not anticipate any upgrade costs during the CMS project because it is waiting until late in the project so it can implement CMS when the latest version of the software is available. Thus, it expects to avoid the cost of having to upgrade from earlier versions. The Fresno campus used an earlier version of PeopleSoft as its legacy system, so it believes it has no need to integrate any existing campus systems because that was done before CMS. 5500 California State Auditor Report 2002-110 California State Auditor Report 2002-110 5511 Systemwide, the university expects to spend almost $18 million to integrate existing systems such as telephone registration, directory services, and other existing campus systems that feed information into or retrieve information from CMS. However, CMS functionality will replace some existing systems, such as Web registration, making their integration unnecessary. The nearly $18 million in expected costs for integrations includes almost $13 million in direct costs and nearly $5 million in in-kind costs. The integration in-kind costs are mainly a result of using university personnel not assigned to a CMS position to work on the integrations. THE UNIVERSITY EXPECTS TO SPEND MORE THAN $296 MILLION IN PERSONNEL COSTS AND $167 MILLION IN CONSULTANT COSTS FOR THE CMS PROJECT THROUGH FISCAL YEAR 2006–07 The legislative request for this audit asked us to report on personnel costs for the CMS project as well as to identify the number of and costs for consultants working on each campus and at the chancellor’s office. More than two-thirds of the university’s total projected expenditures on the CMS project, about $463 million through fiscal year 2006–07, will support university personnel and consultant costs for the project. Figure 2, shown previously, shows that of the $662 million in overall projected costs for the CMS project, 45 percent is for university personnel and 25 percent for consultants. Table C.1 in Appendix C shows the actual and projected costs by type of cost and by fiscal year, and Table C.2 in Appendix C displays the total costs by type of cost for the project’s central costs and by each campus. The university’s CIO says consultants play a valuable role in The university, which successfully implementing enterprise resource planning systems expects to spend such as CMS. In fact, the CIO believes using consultants is almost $167 million the only way the university could rapidly find enough people on consultants for the with the skills and experience to ensure the successful early project, believes using implementation of the CMS project. The CIO states that, consultants is the only without consultants, the university would have had to take way it could rapidly an excessive amount of time to train or hire employees. find enough skilled Furthermore, the CIO believes the knowledge transfer that and experienced people happens by teaming experienced consultants with university to ensure successful employees is invaluable to support rapid development of staff early implementation. and institutional knowledge and skills. The CIO stated the university intends to use consultants extensively during the 5500 California State Auditor Report 2002-110 California State Auditor Report 2002-110 5511 implementation period and transition to university personnel after staff have developed the necessary skills on CMS, an intention supported by the annual CMS cost projections. As Figure 4 shows, the project begins with higher consultant costs that eventually dwindle as the personnel costs increase. The university has spent only $43 million, or 14 percent, through fiscal year 2001–02, yet projects to spend more than $296 million on personnel costs through fiscal year 2006–07. In contrast, the university already has spent nearly $56 million, or 34 percent, through fiscal year 2001–02, but expects to spend almost $167 million on consultants. FIGURE 4 A Comparison of Personnel and Consultant Costs Over the Life of the Project ��������������� �������������� ��� �� ���� ���� ���� �� ���� �� ���� ���� �� ���� ���� ���� ���� ���� ���� �� ��� ��� �� ���� ��� ��� ��� � ������� ��������� ������� ������� ������� ������� ������� ������� ������� ����������� 5522 California State Auditor Report 2002-110 California State Auditor Report 2002-110 5533 ���������� ������ ��������� Source: The university’s June 2002 cost survey. Consulting represents about 25 percent of the total projected CMS costs. Table 5 shows that, systemwide, the university plans to spend almost $167 million on consultants during the nine-year development and implementation period. At the chancellor’s office, which expects to spend more than $31 million on consultants for its central efforts, consultants have worked on various tasks including developing and coding the modifications to the PeopleSoft software deemed necessary to meet the university’s needs. Of the nearly $22 million already spent, Cedar (formerly known as the Hunter Group), Io Consulting, and PeopleSoft have received more than $4 million each as of June 2002. PeopleSoft has been paid the most for consultants as of June 2002—more than $7 million. Table D.1 in Appendix D provides a complete list of the consulting companies working at the chancellor’s office and their respective costs, actual and projected, as reported by the university. Also shown, the chancellor’s office has not yet determined which consultants will be used for more than $7 million of the $9 million in projected costs for July 2002 through June 2007. TABLE 5 Consultant Costs for Developing the CMS Software Centrally and Implementing It at Campuses Actual Projected Projected July 1998 Through July 2002 Through Total June 2002 June 2007 Costs Central $21,815,624 $ 9,448,408 $ 31,264,032 Campuses 34,044,036 101,377,450 135,421,486 Total Consultant Costs $55,859,660 $110,825,858 $166,685,518 Source: The university’s June 2002 cost survey. On the campuses, $34 million already has been spent on consultants as of June 2002, but an additional $101 million in spending is projected through 2006–07. These consultants have worked primarily as implementation partners on campuses. They also have worked on specific projects such as integrating existing campus systems into CMS, reengineering the university’s business processes to fit CMS better, and in a few instances maintaining CMS at smaller campuses such as the California Maritime Academy. Of the $34 million already spent, Io Consulting, Cedar, and KPMG Consulting have received more than $6 million each, with Io Consulting receiving the most, nearly $10 million. Also, campuses expect to pay Cedar an additional $23 million for the period July 2002 through June 2007 to assist with student administration implementations, partly because of this firm’s experience as an implementation partner with the three student administration pilot campuses. Campuses have not determined which consultants will be used for nearly $62 million of expected costs. Table D.2 in Appendix D provides a summary list of consultants working at the campuses, and Table D.3 in Appendix D provides a complete list, by campus, of consulting companies working on each campus. 5522 California State Auditor Report 2002-110 California State Auditor Report 2002-110 5533 THE UNIVERSITY EXPECTS TO SPEND MORE THAN $78 MILLION THROUGH FISCAL YEAR 2006–07 ON DATA CENTER COSTS Our audit scope also included determining the amount of expenses incurred and projected for both the IBM Rochester data center and the current Unisys Corporation (Unisys) data center. In support of its CMS project, the university expects to spend more than $78 million through fiscal year 2006–07 on these costs, including $3.7 million expended because of the failed negotiations with IBM and the need to establish a contingency data center, $4 million in telecommunications costs, and more than $70 million through fiscal year 2006–07 to a data center provider, currently Unisys. As discussed in Chapter 5, the university released a request for proposal (RFP) in April 2000 for an outsourced data center in support of the CMS initiative. The data center RFP went to two vendors, IBM and Unisys, the only two firms that successfully completed the university’s prequalification process. After evalu- The university originally ating both proposals, the university selected IBM for its data selected IBM for its center contract based primarily on IBM’s significantly lower data center contract; price. In September 2000, the university entered into an interim however, neither side agreement with IBM to establish data center operations in was able to negotiate Rochester, New York, while negotiations on a final contract took a final contract. place. In this interim agreement, the university agreed to pay IBM up to $2.9 million through December 2000 for establishing data center operations for CMS. However, the university and IBM were never able to negotiate a final contract. Correspondence between the university and IBM state that there were “irreconcilable differences in pricing and scope for the project.” The university maintained that it spent considerable time defining data center requirements that IBM should have understood, and IBM stated that its work provided for discovery of more precise operational requirements desired by the university for the data center. According to university correspondence, IBM requested that the price on its original bid be increased to accommodate what IBM perceived as required increases to cost and scope, an increase in price the university noted would have resulted in another vendor winning the con- tract. Finally, the university acknowledged that IBM did agree to honor its original bid price, but only with what the university believed to be a significantly reduced scope that required the university to absorb considerable additional costs. As a result of the disagreement, the university terminated negotiations with IBM in December 2000 but was still liable under the previously executed 5544 California State Auditor Report 2002-110 California State Auditor Report 2002-110 5555 interim agreement for IBM’s costs to establish data center operations for CMS. University meeting minutes show that IBM was close to providing data center services for CMS in Rochester, New York when the university terminated negotiations. Before the university terminated its negotiations, it received a bill from IBM for $2.9 million in connection with its costs to establish data center operations for CMS. In September 2002, more than 11⁄2 years after the university formally terminated negotiations with IBM, the parties entered into a settlement agreement that required the university to pay IBM $320,000. Both parties agreed to discharge any future claim under the prior interim agreement. This agreement effectively concluded the data center dispute with IBM. In addition to the settlement pay- ment, the university reported to us that it incurred costs totaling $351,000 for salaries, consultant costs, and equipment during the failed contract negotiation period. In total, the university reports that it incurred about $671,000 directly associated with IBM’s data center operations in Rochester, New York. Meanwhile, the university had established a contingency plan for data center services if its negotiations with IBM were unsuccessful. After terminating negotiations with IBM As a result of its in December 2000, the university executed this plan and unsuccessful negotiations amplified the capabilities of an existing data center to provide a with IBM, the university hardware environment robust enough to handle the scheduled established and operated development and testing of CMS. The university believed it was a contingency data center important to establish this contingency data center to allow the costing $3 million. first campuses to develop and implement CMS on schedule. The university used this contingency data center from January 2001, after IBM negotiations terminated, until July 2001, when the Unisys data center became operational. During this period, the university reports spending about $3 million to establish and operate the contingency data center, with most funds supporting the center’s hardware, software, and staffing needs. Finally, in March 2001, the university entered into a contract with Unisys for the data center services specified under its RFP. The cost of the outsourced data center is projected to be more than $70 million through fiscal year 2006–07, $60 million of which is under the current five-year contract with Unisys. The Unisys data center, located in Salt Lake City, Utah, became operational in July 2001. When considering related telecommunication costs of $4 million along with the $3.7 million spent because of the failed negotiations with IBM and the need to establish the contingency data center, 5544 California State Auditor Report 2002-110 California State Auditor Report 2002-110 5555 the university expects to spend more than $78 million in connection with its CMS data center. Although Table 3, as previously shown, reflects only $75 million in data center costs, the university says that $3 million of the $3.7 million in contingency data center and IBM costs are accounted for separately as hardware, software, and personnel costs. THE UNIVERSITY HAS NOT MONITORED SYSTEMWIDE COSTS ADEQUATELY The university lacks adequate procedures to monitor and control the systemwide costs of the CMS project, so the chancellor’s office cannot accurately report on those expected costs on an ongoing basis. Full and accurate reporting will continue to be impossible until the university sets up an ongoing reporting process to capture campus costs. Further, even for the central costs that it considers to be its responsibil- ity, the chancellor’s office does not use project status reports that periodically track and compare the variances between actual and projected costs and the initial and revised project budgets. The lack of campus cost reports and sufficient project reports hinder management and other interested parties from accurately judg- ing the university’s progress in meeting budget expectations. The University Lacks a Process to Gather and Monitor Campus Costs The university cannot accurately report on the project’s expected systemwide costs because it has not established an ongoing The chancellor’s office process to capture and monitor the costs campuses actually has not tracked campus are incurring or projecting to incur. Although it tracks central CMS costs because it project costs, the chancellor’s office does not track campus costs believes these costs are a because it believes they are a campus responsibility. When we campus responsibility. initially asked about the CMS project costs, the chancellor’s office provided an April 2002 CMS expenditure worksheet that estimated $407 million for the CMS project over an eight- year period, of which $216 million represented campus costs. However, the chancellor’s office had based this campus cost amount on its June 1999 estimate—an estimate prepared nearly three years earlier. This 1999 estimate of $230 million was very rough, merely estimating costs of $10 million for 22 campuses plus the chancellor’s office “campus” costs. The April 2002 cost estimate of $216 million differed only in excluding $14 million deferred because a few campuses’ implementation delays extended beyond the eight-year period on the worksheet. 5566 California State Auditor Report 2002-110 California State Auditor Report 2002-110 5577 The university explained it never collected the campus cost information centrally because it believes that each campus is responsible for its own implementation costs. As a result, the university was not aware of its total systemwide costs for the CMS project. In response to our concerns during the audit about the incomplete cost information, the university expedited its collection efforts and asked campuses to report their actual and projected CMS costs. This resulted in the June 2002 cost survey, which is the primary source for the campus cost information we present. The CIO told us the university already was planning to collect cost information from campuses in fall 2002, when campuses would collectively be better able to develop a more accurate assessment of their current and future CMS costs and would be better able to project and report actual and future costs based on the body of experience gained from the system. Although we recognize that estimates probably will improve as the project progresses, we believe that a regular reporting process should have been established in the beginning of the project before campuses incurred significant costs. Campuses reported in the June 2002 cost survey that they already have spent more than $85 million on the project as of June 2002. Although most of the costs have occurred in the last two fiscal years, campuses reported costs as early as fiscal year 1998–99. The university has said it intends to continue the CMS campus cost survey process in future years and to include a high-level summary of those results in annual reports to the Legislature. However, the university has not been reporting a clear picture of the project’s financial status in its annual Measures of Suc- The university has not cess report to the Legislature. This report communicates the been reporting a clear university’s status in implementing initiatives in its Integrated picture of the project’s Technology Strategy plan, including CMS, and one measure that costs to the Legislature. the university reports on is the tracking of costs of CMS project components against the published budget. In its November 2002 report, the university reported the project budget for fiscal years 2000–01 and 2001–02 as being $30 million and $31 million, respectively, and the actual costs as “at budget.” However, the university did not report campus costs, which totaled $29 million and $47 million in those respective fiscal years. 5566 California State Auditor Report 2002-110 California State Auditor Report 2002-110 5577 The University Lacks Sufficient Project Status Reports to Measure Performance Finally, the chancellor’s office does not use project status reports that periodically track variances between the actual and projected CMS costs on the one hand and the initial and revised CMS project budgets on the other. Prudent project management calls for establishing approved initial budgets and tracking actual costs, enabling managers to report and monitor project progress through periodic status reports that analyze variances between the planned budget and the actual costs. These variances measure project performance and assist management in controlling the project schedule and costs by predicting shortcomings and reducing the risk of going over budget. The chancellor’s office does not use status reports that track variances, even for the central costs that it considers its responsibility. According to the chief of staff for information The chancellor’s office technology services, there is no approved initial CMS project does not use status budget because the budget is approved annually. Each year’s reports that track cost budget for the central CMS effort is developed and funded based variances nor track CMS on project needs and available funding. The chief of staff for costs by development, information technology services further states that as project implementation, and needs have become clearer, funding has been established and ongoing maintenance. shared among the participants. The project funding is adjusted when new costs are identified. The participants’ share of that funding, and the results of the various changes, are recorded in the CMS budget forecast documents. The university’s CIO asserts that the chancellor’s office uses these CMS budget forecast documents to continually track actual project costs against revised budgets. However, these CMS budget forecast documents do not provide an approved initial project budget that the chancellor’s office can use to compare against actual and projected CMS project costs, creating the variance analysis discussed earlier. Instead, the CMS budget forecast documents simply report the funding the chancellor’s office has allocated or expects to allocate to the CMS central effort throughout the fiscal years and the annual or projected CMS central costs. Also, because the chancellor’s office does not track CMS central project costs by development, implementation, and ongoing maintenance, it has been unable to tell us accurately the amount spent for developing modifications to the software or for ongo- ing central CMS maintenance. Earlier, in a footnote to Table 4 on page 49, we discussed the university’s allocation of CMS 5588 California State Auditor Report 2002-110 California State Auditor Report 2002-110 5599 central costs to maintenance based on when campuses begin using CMS, and in Chapter 3 we discuss how the university could not tell us how much it spent to develop the CMS modifications. ALTHOUGH THE UNIVERSITY DOES NOT HAVE A COMPREHENSIVE FUNDING PLAN FOR THE CMS PROJECT, MOST EXPENDITURES ARE SUPPORTED BY THE UNIVERSITY GENERAL FUND The CMS Project Charter of 1999 lists a comprehensive, systemwide cost and funding plan as a critical factor for the project’s success. However, our audit revealed that the university’s funding plan addressed only expected CMS expenditures at the chancellor’s office, not any campuses’ funding needs. When it does not finalize funding for all CMS costs up front, the university lacks a clear understanding of how the project affects its ability to meet other priorities such as academic needs. Chancellor’s office documents indicate that most central CMS expenditures will be supported by the university general fund (general fund), which receives most of its money from the State’s General Fund. Our survey of campus funding for CMS similarly shows that campuses have relied and will rely heavily on their general fund revenues to support CMS expenditures. However, a few campuses identified alternative funding sources, including the financing of CMS costs, as potential funding solutions. The University Does Not Have a Systemwide Funding Plan for the CMS Project Although in 1999 it judged the development of a comprehen- sive, systemwide cost and funding plan a critical success factor during implementation, the university has failed to develop Although the CMS a funding plan that addresses all the CMS project’s expected project charter states costs. The university’s funding plan addresses the funding needs that a comprehensive, of only the chancellor’s office during implementation, but it systemwide funding excludes the similar implementation costs of each campus. plan is a critical factor to the success of CMS, In September 1999, the university developed a CMS Project the university failed Charter (charter) that described, among other things, critical to develop a funding factors for the project’s success. One factor listed by the plan that addresses all university was the development of a comprehensive, CMS costs. systemwide cost and funding plan that included CMS and the funding needs of other, nontechnology high priorities. According to the charter, this funding plan was to guide the 5588 California State Auditor Report 2002-110 California State Auditor Report 2002-110 5599 university’s decision making on CMS. The charter further stated that resources that were committed to the CMS project should not be diverted to other competing demands—noting, however, that care should be taken to safeguard the university’s academic mission. Although the university’s executive vice chancellor and chief financial officer (executive vice chancellor) asserts that the university does have a systemwide funding plan as envisioned by the CMS charter, we disagree that the plan is systemwide. In his November 2002 letter to the bureau, the executive vice chancellor asserts “a systemwide funding plan was developed and has been continually refined as demonstrated in the CMS forecast documents.” However, the funding information in the forecast documents focuses solely on the chancellor’s office’s expenditures during CMS implementation. The executive vice chancellor states “campus implementations were understood to be campus-specific efforts from the beginning.” He clarified this point by saying that “campus-specific expenditures are not generally collected by the chancellor’s office but are left to the campuses to fund and track.” In general, the executive vice chancellor indicated that the chancellor’s office’s overall Although campuses management philosophy has been to provide campuses with account for 63 percent significant managerial autonomy, thus explaining why campus of project costs, the funding needs on the CMS project were not addressed in the “systemwide” CMS project’s funding plan. funding plan does not consider campus funding In fact, it appears that the chancellor’s office has taken a needs for CMS. hands-off approach to managing campus implementation of the systemwide CMS project. The chancellor’s office expects campuses to determine on their own the costs and funding necessary to implement CMS. Campus CMS implementation accounts for 63 percent of the project’s overall anticipated cost. In his November 2002 letter to the bureau, the executive vice chancellor states that “campus presidents are held accountable for the successful operations of the campuses” and that “the outcome expectation is to implement all components of the system successfully.” He clarifies this point by explaining that each campus is responsible for determining the specific level of funding needed to cover campus CMS implementation costs. This point of view explains why the chancellor’s office never deemed it necessary to collect campus-level funding information to ensure that adequate funds had been secured to cover expected campus costs. 6600 California State Auditor Report 2002-110 California State Auditor Report 2002-110 6611 Considering that the chancellor’s office directed all campuses to implement CMS, thus making the scope of CMS systemwide, funding for the entire project should have been finalized at the chancellor’s office. It appears that the university recently began to see the importance of having campus cost and funding information when it began collecting this data in response to our audit. The university’s executive vice chancellor writes in his November 2002 letter that “given the evolution to the current systemwide implementation of CMS, the [university] is beginning to collect campus expenditure and funding data” and “the current audit has helped expedite the beginnings of that process.” However, the evolution of the CMS project to a systemwide implementation is not a recent phenomenon—the university’s chancellor directed staff to move forward with systemwide implementation in July 1999. Finally, in collecting systemwide CMS cost and funding informa- tion, the university also should consider the required funding In light of the State’s for other high priorities, as envisioned in the CMS project current budget deficit, charter. Under the charter, the systemwide funding plan was the university likely will to address the funding needs of other significant university find it more difficult priorities, a prudent step that would allow the university to to fund future CMS prioritize projects that may compete for resources. However, the expenditures without university’s current funding plan does not specifically address identifying funding the funding requirements for these other priorities. Although in priorities or sources. recent years the university has benefited from increased support from the State’s General Fund under a partnership agreement, the State’s current budget deficit makes continually increasing state support unlikely. In fact, the governor’s proposed budget for fiscal year 2003–04 reduces the university’s base budget by $326 million, as well as an additional $201 million in antici- pated funding increases related to its partnership agreement with the State. The governor’s proposed budget assumes that future anticipated student fee increases and increased general fund support for enrollment growth will help offset these reductions, but these offsets cover only $409 million of the $527 million removed from the university’s budget. Further, $86 million of the $409 million is to be set aside for financial aid and thus is not available for other purposes, such as CMS. As a result, the university likely will find it more difficult to fund future CMS expenditures when it has not established funding priorities or funding sources for all expected CMS costs. 6600 California State Auditor Report 2002-110 California State Auditor Report 2002-110 6611 The Chancellor’s Office Is Funding Its Share of CMS Costs Through Its General Fund Responding to the June 2002 cost survey, the chancellor’s office provided actual and projected cost information for its central costs for the CMS project as a total cost of $236 million through fiscal year 2006–07. To understand the funding sources supporting these actual and projected expenditures, we issued a subsequent survey in November 2002. Responding to this survey, the chancellor’s office said it will fund the entire $236 million from its general fund—general support fund (university sub-account 001). As Figure 5 shows, budget forecast documents from the chan- cellor’s office indicate that approximately 31 percent of its total central CMS costs will be supported through an “off the top” assessment of all campuses—funding that otherwise would have been passed from the chancellor’s office to the 23 university campuses during the annual budget allocation. These annual “off the top” assessments have consistently been $10 million to all campuses collectively (except for the $5 million assessment in fiscal year 1999–2000). The chancellor’s office expects to fund 29 percent of its CMS costs through its partnership fund- ing from the State, 28 percent through its existing CMS budget, 2 percent through specific charges to campuses, and 10 percent from “other sources.” These other sources represent chancellor’s office allocations of additional funding based on unallocated systemwide funds at the end of the fiscal year. FIGURE 5 Projected Chancellor’s Office Central CMS Funding Sources Fiscal Years 1998–99 Through 2006–07 ����� ��� ������������������ ��������� ��� ����������� ����������������� ��� ��������� �������� ���������������� �� ������� ��� Source: The university’s CMS budget forecast document (December 2002). 6622 California State Auditor Report 2002-110 California State Auditor Report 2002-110 6633 Campus CMS Expenditures Have Been Supported Primarily by the University General Fund, Although Some Campuses Have Found Other Funding Sources The chancellor’s office requires campuses to demonstrate, at a minimum, executive commitment to funding before beginning implementation and, ideally, to develop a comprehensive multiyear funding plan. Given the university’s lack of campus funding information for CMS, the bureau surveyed all 23 cam- puses to ascertain how they funded their CMS expenditures through fiscal year 2001–02, and how each intends to fund its projected CMS expenditures through fiscal year 2006–07. The survey’s results indicate that all campuses rely on their general fund—general support fund for their CMS expenditures and expect to rely on that source in the future. General fund alloca- tions come from the chancellor’s office and represent the largest source of support for each campus. In connection with our audit, all 23 campuses completed a June 2002 cost survey that detailed actual and projected expen- ditures on CMS. To understand the funding sources supporting these reported expenditures, the bureau issued a subsequent survey in October 2002 that required each campus to indi- cate the funding sources for these CMS costs. Tables 6 and 7 on the following pages show the campus-by-campus results of the bureau’s funding survey. Table 6 gives the actual funding through June 30, 2002. As Table 6 on the following page shows, campuses have relied heavily on their general fund accounts to fund CMS costs Sixteen of the through June 30, 2002. In fact, 16 of the 22 campuses that 22 campuses that incurred CMS-related costs during this period reported that incurred CMS costs they used only general fund assets to support CMS costs. through June 2002 Overall, approximately 85 percent of campus CMS expenditures reported that they through June 30, 2002, have been supported through campus used only general allocations from their general fund—general support funding. fund assets to support The predominant use of campus general fund assets to support CMS costs. campus CMS efforts is not surprising because the general funds are the campuses’ largest source of money. Also, because such expenditures are not restricted to any specific program, campuses can spend general fund revenue for a variety of purposes, including such items as salaries, instructional materials, and administration costs. 6622 California State Auditor Report 2002-110 California State Auditor Report 2002-110 6633 TABLE 6 University Campus Funding Sources for CMS Expenditures (July 1998 Through June 2002) General Fund– General Information General Fund– Resources Fund– Various Other Total Total General and Capital Auxiliary University Revenue Funding Expenditures Campus Support Technology Outlay Funds Trust Funds Funds Reported Reported Bakersfield $ 386,455 $ 386,455 $ 386,455 Channel Islands 256,580 256,580 256,580 Chico 3,443,824 $ 1,067 3,444,891 3,444,891 Dominguez Hills 85,600 85,600 85,600 Fresno* 6,938,580 $469,635 $ 652,011 8,060,226 8,060,226 Fullerton 933,440 933,440 933,440 Hayward 5,059,187 5,059,187 5,059,187 Humboldt 813,242 813,242 813,242 Long Beach 13,204,585 13,204,585 13,204,585 Los Angeles 1,673,200 1,673,200 1,673,200 Maritime Academy 2,007,733 2,007,733 2,007,733 Monterey Bay* 409,576 $1,412,276 1,821,852 1,821,852 Northridge 5,242,297 5,239,266 10,481,563 10,481,563 Pomona 3,781,988 3,781,988 3,781,988 Sacramento 2,270,200 2,270,200 2,270,200 San Bernardino 5,032,514 $406,530 5,439,044 5,439,044 San Diego† San Francisco* 12,300 237,100 249,400 249,400 San Jose 2,485,235 2,335,586 95,068 4,915,889 4,915,889 San Luis Obispo 8,874,561 8,874,561 8,874,561 San Marcos* 1,140,200 1,140,200 1,140,200 Sonoma 6,496,734 815,093 523,459 483,054 8,318,340 8,318,340 Stanislaus 187,547 187,547 187,547 Totals‡ $70,735,578 $643,630 $469,635 $2,227,369 $8,750,322 $579,189 $83,405,723 $83,405,723 Percent 84.81% 0.77% 0.56% 2.67% 10.49% 0.70% Source: The Bureau of State Audits’ October 2002 “CMS Funding Survey”; the university’s June 2002 cost survey. * Fresno, Monterey Bay, San Francisco, and San Marcos fund CMS costs through a pooled account that is supported by the applicable funds listed above for each campus. These figures may not represent actual funding sources for CMS at these campuses because the figures are based on the contribution percentages of these funding sources to the pooled account. † No funding information is provided for the San Diego campus because it will not incur CMS implementation costs until after June 2002. ‡ The table does not include chancellor’s office “campus” costs. These costs represent the cost of implementing the human resources and finance applications at the chancellor’s office. For the period above, chancellor’s office “campus” costs totaled $1,647,635. This amount was funded through “general fund–general support.” 6644 California State Auditor Report 2002-110 California State Auditor Report 2002-110 6655 Although Table 6 shows the campuses predominantly use general fund revenue to support CMS costs, our survey revealed that some campuses have used other funding sources in conjunction with the general fund. Six campuses reported that they used auxiliary funds, university trust funds, other revenue funds including parking and dormitory revenue, or a combination of all three to support their CMS costs. As Table 6 demonstrates, four campuses collectively reported that they spent about $8.8 million of university trust fund assets to support their CMS costs, equaling more than 10 percent of all campus expenditures through June 30, 2002. The Northridge and San Jose campuses used trust funds most extensively, drawing $5.2 million and $2.3 million, respectively. The university trust funds are established under Education Code, Section 89722, and contain money obtained from various sources including but not limited to gifts, donations, and collected fees for parking, health facilities and services, and extension programs. According to Northridge, most of the trust fund assets it spent for CMS came through collected fees for accounting services, while the remainder came from its extended learning fund. San Jose indicated that the trust fund money it spent on CMS was obtained primarily through the contributions of auxiliary organizations, while extended learning revenue, health services fees, and parking revenue were the other trust fund assets used to support CMS costs. The bureau’s funding survey also revealed that two campuses— Monterey Bay and Sonoma—have assessed their auxiliary Two campuses have organizations a combined $2.2 million through June 30, 2002, assessed their auxiliary to partly absorb campus CMS costs. Campus auxiliary organizations for organizations provide student housing, parking, athletics, food CMS costs. services, bookstores, and other self-supporting noninstructional services. The services provided by auxiliary organizations are funded through specific user charges and are not subsidized by the State. Both campuses have indicated they chose to assess these auxiliary organizations for CMS costs because the auxiliaries plan to use CMS in their own operations. The survey also revealed that the Chico, San Jose, and Sonoma campuses have used funding sources other than those mentioned above. Specifically, these campuses have collectively obtained $579,189 in support of their CMS costs by using funding sources such as dormitory, parking, lottery, and continuing education revenue. 6644 California State Auditor Report 2002-110 California State Auditor Report 2002-110 6655 TABLE 7 Projected University Campus Funding Sources for CMS Expenditures (July 2002 Through June 2007) General Fund– General Information Fund– Resources Various Other Total General and Auxiliary University Revenue Total Funding Expenditures Campus Support Technology Funds Trust Funds Funds Reported Reported Bakersfield $ 8,077,346 $ 8,077,346 $ 8,077,346 Channel Islands 605,087 605,087 605,087 Chico 14,444,501 $ 556,748 15,001,249 15,001,249 Dominguez Hills 13,744,200 13,744,200 13,744,200 Fresno 10,165,750 $ 156,150 10,321,900 10,321,900 Fullerton* 18,390,193 Hayward† 13,436,439 13,436,439 13,436,439 Humboldt 10,829,622 10,829,622 10,829,622 Long Beach 23,903,543 23,903,543 23,903,543 Los Angeles 11,747,300 11,747,300 11,747,300 Maritime Academy 2,630,452 2,630,452 2,630,452 Monterey Bay‡ 13,561,785 Northridge 7,689,766 47,500 7,737,266 7,737,266 Pomona 16,455,451 16,455,451 16,455,451 Sacramento 17,002,300 17,002,300 17,002,300 San Bernardino 12,645,127 $ 1,824,813 14,469,940 14,469,940 San Diego 37,596,358 37,596,358 37,596,358 San Francisco† 7,203,200 11,839,000 19,042,200 33,032,600 San Jose 9,410,103 804,440 333,770 10,548,313 10,548,313 San Luis Obispo 24,777,513 24,777,513 24,777,513 San Marcos 9,682,100 9,682,100 9,682,100 Sonoma 7,332,595 $1,120,395 2,824,284 1,189,800 12,467,074 12,467,074 Stanislaus 9,329,000 9,329,000 9,329,000 Totals§ $268,707,753 $13,663,813 $1,120,395 $3,832,374 $2,080,318 $289,404,653 $335,347,031 Percent 92.85% 4.72% 0.39% 1.32% 0.72% Source: The Bureau of State Audits’ October 2002 “CMS Funding Survey”; the university’s June 2002 cost survey. * The Fullerton campus indicated it had yet to determine how its projected CMS costs would be funded; however, the campus estimates that at least 75 percent will come from the general fund, while other revenue sources such as parking, housing, and the university trust fund also may be used. † The Hayward campus indicated that it may have to finance one-time CMS costs as necessary. The San Francisco campus indicated that it will finance the difference between its projected CMS expenditures and the funding reported above. ‡ The Monterey Bay campus declined to project how its future CMS expenditures would be funded; however, the campus estimates that the majority of costs will continue to be funded through the general fund. The campus also plans to obtain funding from auxiliary organizations. § The table does not include chancellor’s office “campus” costs. These costs represent the cost of implementing the human resources and finance applications at the chancellor’s office. For the period above, chancellor’s office “campus” costs totaled $5,065,600. This amount was funded through “general fund–general support.” 6666 California State Auditor Report 2002-110 California State Auditor Report 2002-110 6677 Table 7 summarizes how the university’s 23 campuses expect to fund their remaining projected CMS expenditures between fiscal years 2002–03 and 2006–07. The survey’s results show that cam- puses intend to rely even more heavily on general fund assets to support CMS future costs. Specifically, campuses expect to fund approximately 93 percent of their projected CMS costs through their general fund—general support allocations. Based on a comparison between Tables 6 and 7, it appears that some campuses expect to use university trust fund assets to a lesser extent in the future, electing to increase general fund Only seven of 23 campuses support for CMS instead. However, the bureau’s funding survey were able to provide revealed that campuses may be unsure how they will fund future a funding plan that CMS costs. Of the 23 campuses, only seven were able to provide identified funding the bureau with a funding plan for CMS costs that identified sources for projected costs. funding sources for projected costs. Additionally, it appears that some campuses now believe that available campus revenue will not be sufficient to cover all CMS costs, and are considering the financing of these expected expenditures. Both the San Francisco and Hayward campuses’ funding survey responses indicated a potential need to finance a portion of their projected CMS expenditures, stating that existing campus resources are insuf- ficient to fund CMS implementation costs. As a result, as shown on Table 7, the San Francisco campus reported less projected funding than expenditures. RECOMMENDATIONS To ensure that it adequately monitors and controls project costs, the university should take the following actions: • Establish a mechanism to collect and compile systemwide project cost information, including campus costs. The university should determine what level of quarterly cost information it needs from campuses to monitor the project. Further, the university should collect comprehensive cost information annually, including in-kind, upgrades, and integration costs. • Compare the collected project costs against the approved systemwide project budget and publish this information in a quarterly status report. 6666 California State Auditor Report 2002-110 California State Auditor Report 2002-110 6677 Additionally, the university should ensure that it includes all costs of the CMS project in its annual reports to the Legislature. Finally, the university should establish a systemwide funding plan for the CMS project that includes campuses. Further, before it begins any major information technology project in the future, it should ensure that it has a funding plan in place that covers the entire scope of the project. n 6688 California State Auditor Report 2002-110 California State Auditor Report 2002-110 6699 CHAPTER 3 The Common Management System May Not Achieve All Its Business Objectives Nor All the Possible Benefits of a Systemwide Software Project CHAPTER SUMMARY Although the Common Management System (CMS) may achieve some of the benefits the California State University (university) has outlined, we noted problems that cast doubt on whether CMS will achieve all the university’s business objectives related to the project’s mission, as stated in the CMS Project Charter (charter). These business objectives include minimizing costs and time to implement and maintain the software, and providing ready access to current, accurate, and complete administrative information. Several problems we noted stem from the university’s weak efforts early in the planning process and its limited expectations with regard to systemwide reporting. For instance, the university’s stated intent was to minimize cost by limiting the modifications to the vendor software to those needed to meet its business needs. However, the university had no basis to anticipate the extent of modifications it has needed because, before it purchased the software, it did not sufficiently evaluate its specific business processes to understand which business processes the potential vendors’ software products could accommodate and which software products would require modification to meet its business needs. Additionally, it often must continue to reapply modifications when the vendor software is updated, increasing maintenance costs. Further, the university plans to continue to use existing processes for systemwide reports because it did not design CMS to replace these processes. Thus, when CMS is implemented systemwide in fiscal year 2006–07, the way campuses report data to the chancellor will not be substantially different than it is now, except CMS will be the source of the data. Campus data still will be transmitted quarterly or on certain census dates or at semester’s end, when the chancellor’s office will use other systems to manipulate and summarize the data into a format for systemwide reports. 6688 California State Auditor Report 2002-110 California State Auditor Report 2002-110 6699 Although the university did use recommended practices for designing, developing, and testing CMS software modifications, it might have averted certain product quality and information security problems if it had employed other recommended practices relating to quality assurance. Finally, the university’s piecemeal approach of identifying, procuring, and implementing its own CMS solution did not ensure that it shared project risk with vendors. It assumed substantially all the responsibility for the project’s success and the considerable financial and business risk involved in ensuring that the software meets its business needs and is implemented successfully. THE CMS PROJECT MAY NOT FULLY ACHIEVE THE UNIVERSITY’S STATED BUSINESS OBJECTIVES The university expects to accomplish certain business objectives with its CMS project, but problems noted during our review According to the indicate that CMS may neither fully achieve those objectives chancellor, the university’s nor offer what could have been achieved from such a system- decision to implement wide project. In October 2002, the university chancellor told us a systemwide CMS the university’s decision to implement a systemwide CMS was was based on reasons based on reasons expressed in the mission statement, strategic expressed in the CMS outcomes, and business objectives in the charter. The charter project charter. defines the mission of the university’s administrative functions as providing efficient, effective, high-quality service to students, faculty, and staff. The charter also lists several strategic out- comes, including high-quality customer service and improved efficiencies; and identifies the general business objectives the university expects to accomplish through its CMS project. The university expects that upon completion or within a certain period after implementation of the CMS system, it will have accomplished, among others, these objectives: • Minimize costs to implement and maintain application software. • Minimize time to implement application software. • Establish standards to share information for common reporting purposes. • Provide ready access to current, accurate, and complete administrative information and the means to use it in an effective manner. 7700 California State Auditor Report 2002-110 California State Auditor Report 2002-110 7711 However, our review of the CMS project found problems that cast doubt on whether the university will fully achieve its stated business objectives or reap benefits possible from a systemwide implementation of a project such as CMS. Though focusing on the university’s business objectives, the following analysis has broader implications because these objectives also relate to the mission statement and strategic outcomes contained in the charter. Initial Versions of the Vendor’s Software Required About 200 Modifications to Meet the University’s Business Needs The university’s stated business objectives for the CMS project include minimizing costs and time to implement the software. The university’s stated An important element affecting this objective is the cost business objectives and time involved with modifying the software to meet the for the CMS project university’s business needs. The university has made more than include minimizing costs 100 modifications to the initial version of the CMS human and time to implement resources software, more than 50 to the finance software, the software. and 40 to the student administration software. It made these modifications to change functionality or to generate reports or forms. Often, modifications then must be reapplied each time PeopleSoft releases a new version of the CMS software, adding potentially significant maintenance costs to reapply, test, and implement the modifications. Although we recognize that not all modifications take the same amount of time and effort, we are unable to quantify which modifications were most costly because the university did not track modification costs. The CMS charter explicitly states that the design “bias” for the project was to be in favor of minimizing software modifications. Therefore, to minimize the cost and time to develop, implement, and maintain the CMS software, the university planned to use the unmodified, or “vanilla,” version of the PeopleSoft software unless compelling justification could be made for modifying it. In an effort to contain costs, the university has established an approval process for modification requests, and not all campus requests for modifications are approved. The university considers the need for a modification based on whether it affects a campus’s ability to meet business needs including legal, regulatory, or university requirements. Also, the university considers modifications that would result in a significant reduction in manual effort or improvement in administrative productivity. As it develops the software applications, the chancellor’s office defines, with input from selected campus representatives, the system requirements to meet those business 7700 California State Auditor Report 2002-110 California State Auditor Report 2002-110 7711 needs. The chancellor’s office approves modifications to the vendor’s software to produce the CMS version distributed to campuses for implementation. Although it initially planned to make as few modifications as possible to the PeopleSoft software, the university ultimately found that it needed to make about 200 modifications to the One modification was initial versions of the software applications to meet business developed to transmit requirements and other campus needs. Between April 1999 timesheet information and September 2000, the university developed more than entered into CMS each 100 modifications to the human resources version 7.6 software payroll period to the application. One modification was developed to transmit State Controller’s Office timesheet information entered into CMS each payroll period payroll system. about hourly employees and salaried employees’ leave to the State Controller’s Office payroll system to avert dual entry by university employees. Similarly, a modification was developed to create an interface between the university and the State Controller’s Office so payroll information from the State Controller’s Office could be downloaded into CMS each pay period. This payroll information is required in CMS to generate financial entries, link payroll transactions with funding sources, and charge labor costs in the CMS finance software application. The university expects to make some additional modifications to the CMS human resources software application. Although it developed a modification to eliminate double entry of certain timesheet information to the State Controller’s Office payroll system, campuses using the CMS human resources application must enter other employee information needed to ensure that employees are paid correctly, such as position changes or terminations, into both CMS and the State Controller’s Office personnel employment system. This double entry requires additional time to rekey the data and introduces the risk of data entry errors. The university decided to revisit this double entry issue in November 2002, and the senior director for the CMS project expects to modify the human resources application to resolve this issue and to make other changes and enhancements. The university also developed more than 50 modifications to the vendor’s 7.5 version of the finance application between August 1999 and September 2000, and about 40 modifications to the 8.0 version of the student administration application from late January 2001 through September 2002. 7722 California State Auditor Report 2002-110 California State Auditor Report 2002-110 7733 Compounding the time and costs for modifications, PeopleSoft periodically releases new versions of the CMS software, and the university intends to keep current with those releases. Thus, the university will need to reapply many of the CMS modifi- cations to the new releases, adding potentially significant maintenance costs in reapplying, testing, and implementing these modifications. For example, about 17 months after it developed the 7.6 version of the CMS human resources appli- cation, the university made 75 modifications to and released the newest human resources version 8.0 application. According to the director of software operations and support services, all these modifications were reapplications of the previous modifi- cations that still were necessary for the newest version to meet the university’s business needs. Reapplications of modifications also are likely to be needed when the university implements the next version, version 8.4, of the finance application, which the university began developing in June 2002 and plans to release in July 2003. In September 2002, we discovered that the university did not Because the university track the number of hours that employees and consultants did not track the hours worked to complete modifications, even though it estimates spent working on the number of hours needed to develop modifications and modifications, it does not considers these estimates when evaluating whether to approve know the costs associated modifications. It does not know the costs associated with these with these efforts. efforts because it did not track the hours spent working on modifications. After we brought this matter to its attention, the university informed us in November 2002 that it had begun to track and monitor both estimated and actual hours for developing modifications to the student administration application, and that it was planning to do so for all subsequent modifications to all three software applications. It appears that the costs of developing and implementing modifications could be significant. The chancellor’s office reported costs in the cost survey, described in Chapter 2, for CMS general operations, training, and development and implementation of the CMS software as of June 2002 at more than $34 million. According to the senior director for the CMS project, this includes costs associated with supporting the hardware and software for CMS that would have been incurred without modifying the vendor’s software. Nevertheless, it appears that a significant amount of the work associated with these costs was modifying the software to fit the university’s business needs and working with campuses that are implementing the modifications. 7722 California State Auditor Report 2002-110 California State Auditor Report 2002-110 7733 The senior director for the CMS project acknowledges that the chancellor’s office did not attempt to estimate the time or cost involved with making modifications before starting development of each software application. Nevertheless, she asserts that the time and cost in developing these modifications were not higher than anticipated for the student administration and finance applications. The senior director acknowledges that the human resources version 7.6 application required more modifications than might have been anticipated, but she points out that it was the university’s first development effort. However, although the university may have started to recognize the time and cost involved with making modifications once it started developing each software application, it had no basis to anticipate the extent of modification effort needed at the time it decided to purchase the software and implement CMS. Further, we question how the university could have measured its anticipation of the modifications’ costs when it did not track the actual costs associated with the modifications. The University Has Not Implemented CMS in a Manner That Will Maximize Systemwide Reporting The university intended CMS to meet the business objectives of providing ready access to current, accurate, and complete administrative information, as well as establishing standards Installing shared for common reporting processes. The July 1999 decision to software university-wide install the same suite of software university-wide created an could have created an opportunity for the university to achieve these objectives not opportunity to provide only within each campus, but also systemwide—using the the chancellor’s office and PeopleSoft software without maintaining its current software each campus direct access for systemwide reporting. Ideally, the university could have to information to produce chosen to use the same best business processes at each campus, current and complete supported by one shared copy of the software or several copies reports; however, the that focused on campus size and needs, where a data file was university did not stored each night for use the next day at a university-wide implement the software in data storage and retrieval facility (data warehouse). Then, this manner. individuals at the chancellor’s office or at each campus could have directly accessed this information to produce current and complete reports. This approach would have been one way to achieve savings through implementing a university-wide suite of administrative software as discussed in Chapter 1 and to enable all students, faculty, and staff to “interact with information resources from anyplace, to anyplace, at anytime” as described in the university’s information technology strategy. 7744 California State Auditor Report 2002-110 California State Auditor Report 2002-110 7755 However, the university did not implement its suite of software in such a manner. Instead of making its standard, “Are we maximizing the university-wide utility of the common administrative system while minimizing implementation and maintenance costs?,” it appears the university has been content to limit its standard to, “Are we implementing and maintaining the university-wide system at less cost than would have been incurred if each campus procured, implemented, and maintained its own suite of administrative software?” The University Is Not Installing Shared Databases As discussed in Chapter 1, IBM apparently built several potential models that varied from one shared copy of the Each campus’s database software to several copies that focused on campus size and must be individually needs. In addition, the university CIO informed the board of maintained and tested. trustees (board) that the university was identifying the costs and timing involved in moving to shared management information software for campuses. One method of sharing management information software would include sharing databases. However, the university has been installing separate and distinct databases for all campuses except the Sonoma and Maritime Academy campuses. As a result, the university must maintain each of these numerous databases separately, and each campus must perform testing of the changes to its distinct databases. We asked the CIO about information we received related to various campuses banding together to share databases. However, in January 2003, he informed us that little had been done in this area: “We are just beginning to look at how we might improve upon our hardware and operations set up in Salt Lake City based on the past year plus of experience. One item of review is that of the feasibility of campuses sharing databases, but we have not gotten into it very far at all yet. Our first step is a conference call with a few of the campus folks, that is just in the process of being scheduled.” Further, he indicated he could not even speculate on the benefits: “Primarily, we are looking for efficiencies and better utilization of resources that may be possible with the sharing of databases and/or database servers in Salt Lake City. There may be other benefits (and liabilities) involved, but until we actually have the discussions; I don’t want to speculate.” 7744 California State Auditor Report 2002-110 California State Auditor Report 2002-110 7755 Nevertheless, it is puzzling as to why the university did not focus on exploring shared databases when it designed the system. IBM raised the concept of shared software in 1996, the concept was communicated to the board, and a university business objective included minimizing costs to implement and maintain applica- tion software. Functionality Will Vary Across Campuses Implementing the CMS software throughout the university is resulting in a wide variation in functionality across campuses Only the Los Angeles because most campuses have not implemented and are not campus has plans to planning to implement all the modules or sub-modules implement all of the (functionality elements) purchased under the PeopleSoft functionality elements agreement. The university also has created additional function- available in the CMS ality elements for CMS in areas in which PeopleSoft did not applications. provide functionality. In fact, only the Los Angeles campus plans to implement all the functionality elements available in the finance, human relations, and student administration applications. This lack of uniformity raises the cost of imple- menting and maintaining the CMS software and limits its usefulness in producing systemwide reports. To assess the consistency of functionality across all campuses, we asked the university for a summary showing the functional- ity elements implemented at each campus. However, we found the university never defined at the beginning of the project what specific functionality comprised CMS. That is, the university did not identify which functionality elements each campus was to implement in return for what the university estimated to be the cost of the project. Further, the senior director for the CMS project acknowledges that the university has not established a minimum level of functionality that campuses are required to implement systemwide. Nevertheless, the university provided us a functionality matrix of what it subsequently understood campuses were implementing, although we noted instances in which the matrix was incomplete. Therefore, in October 2002 we asked the campuses to report the functionality elements they had implemented or planned to implement during the systemwide implementation effort. The functionality elements the campuses reported represent the level of functionality the campuses considered when reporting costs in the June 2002 campus cost survey. Table 8 gives the number of campuses related to the number of functionality elements they reported. 7766 California State Auditor Report 2002-110 California State Auditor Report 2002-110 7777 TABLE 8 Range of Functionality Elements Implemented by Campuses Number of Number of Functionality Elements Campuses Implemented or Planned* Human Resources Application† 1 41 0 36 to 40 6 31 to 35 2 26 to 30 4 21 to 25 9 16 to 20 1 11 to 15 1 6 to 10 0 1 to 5 Finance Application† 1 12 3 9 to 11 11 7 to 8 5 5 to 6 4 3 to 4 0 1 to 2 Student Administration Application‡ 1 22 8 18 to 21 4 14 to 17 5 10 to 13 4 6 to 9 0 1 to 5 Source: The Bureau of State Audits’ October 2002 Module Survey, which reflects the status of functionality elements as of June 2002. * Includes modules, sub-modules, and university-created functionality. † Number of campuses equals 23 campuses plus chancellor’s office (human resources and finance only). ‡ Channel Islands and chancellor’s office did not plan to implement student administration as of the October 2002 survey. Table 8 shows that wide variation exists in the number of campuses implementing or planning to implement functionality elements of the human resources, finance, and student administration applications. For example, four campuses plan to implement six to nine functionality elements of the student administration application, while eight campuses plan to implement 18 to 21 functionality elements. It is also important to note that CMS project managers explained that 7766 California State Auditor Report 2002-110 California State Auditor Report 2002-110 7777 these functionality elements do not have equal functionality. Variability can exist within each functionality element as well. For example, although all campuses have implemented or plan to implement the general ledger functionality element, its functionality still could vary by campus. The university is conducting or planning more than 20 unique implementations The university is conducting of the software, and this lack of uniformity increases or planning more than implementation costs. Also, maintenance costs increase because 20 unique implementations each campus must perform testing of its implementation of of the software, which software revisions that reflect modifications and upgrades. will increase the cost to implement the system. The university should expect additional costs from campuses that anticipate increasing functionality by adding functionality elements not considered in the previous cost projections. We asked campuses to report any functionality elements they anticipated implementing but had not included in the cost projections reported in the June 2002 cost survey. This informa- tion revealed that, similar to the results previously shown in Table 8, a great deal of variation exists in the anticipated functionality. To the extent that campuses actually implement these anticipated functionality elements, CMS project costs, currently estimated at $662 million including maintenance and operations through fiscal year 2006–07, will increase accordingly. Appendix E shows a list of the functionality elements that each campus and the chancellor’s office have implemented, plan to implement, or anticipate implementing for the systemwide CMS. With campuses implementing various functionality elements, various levels of functionality and data will exist at each campus, creating potential data gaps in systemwide reports the university might want to produce. Moreover, even if all campuses implemented the same functionality elements, the information could not be compiled into useful systemwide reports without additional manipulation because campuses currently structure their data in various ways. For example, while reviewing campus accounting records produced by CMS, we found an accounting table that had a column labeled “class field.” At one campus, the “class field” was left blank, or unused, while at another campus selecting the proper CMS class code in this “class code” field allowed us to capture the CMS-related expenditures at that campus. The university currently uses a system outside of CMS, known as the Financial Information Record Management System, to interpret this inconsistent campus coding for systemwide financial reporting. As a result, the feasibility of using CMS as it currently exists to produce 7788 California State Auditor Report 2002-110 California State Auditor Report 2002-110 7799 useful systemwide management reports without additional Information entered manipulation of the data may not be substantially better than it into CMS is not readily was previously. available to the chancellor’s office for Moreover, the university cannot use CMS, as the university systemwide reporting. designed it, to produce systemwide reports because CMS is a transaction-based operational system without a systemwide data storage and retrieval (data warehousing) operation. In other words, an employee enters information into the system, but that information is not readily available to the chancellor’s office for systemwide reporting. The university reportedly defined data warehousing as out of scope early in the CMS project’s life. We discuss the university’s recent efforts to address campus level data warehousing in Chapter 5. Systemwide Reporting Processes Will Remain Substantially Unchanged After CMS Is Implemented Currently, to prepare systemwide reports, campuses extract the necessary data from each of their electronic information systems, at the level the chancellor’s office prescribes. When it receives electronic data from the campuses, the chancellor’s office uses other systems, such as the Financial Information Record Management System, to manipulate and summarize the data into a format that allows systemwide reports. However, the chancellor’s office can produce only systemwide reports that stay within the parameters of the collected information. For example, when we were analyzing the increase in full-time-equivalent university personnel, the chancellor’s office could provide the number of information technology workers at a given campus, but it could not give information on these workers by certain functional departments. Instead, each campus had to provide the detailed data. The chancellor’s office collects systemwide data only for the tasks and decision making deemed necessary, so more detailed information than already is collected involves more effort by all campuses. This reporting process will remain substantially unchanged when CMS is implemented fully, except CMS will be the source of the data as opposed to previously existing information systems. For example, the currently existing corporate data warehousing operation will continue to contain its current level of financial data as defined in the Financial Information Record Management System reporting structure. As before, this data will not be current because campuses report the financial data quarterly. Also as before, student data (for state and federal 7788 California State Auditor Report 2002-110 California State Auditor Report 2002-110 7799 reporting on applications, enrollments, unit loads, and degrees granted) will not be current, but will continue to be gathered at the fall census dates or by semester. Although CMS Provides Some Functionality That Did Not Previously Exist, Some Campuses Have Lost Some Functionality As discussed in Chapter 1, the chancellor said the university implemented CMS, in part, because “improved administrative systems were necessary . . . to provide the level and type of One campus says it must services it required.” However, some campuses now have now manually perform less functionality for some processes than with the systems some functions and enter CMS replaced, increasing staff workload for these processes. the results into CMS For example, one of the three campuses we visited told us that its prior system had the CMS finance software lacks some functionality that the performed automatically. prior campus system had for automatically calculating and withholding taxes from employees and out-of-state vendors. The campus says it now must perform these functions manually and enter the results into CMS. Also, the campus tells us that its previous student administration application was more functional than CMS. For instance, the former student administration system automatically recognized students who applied to graduate but did not graduate as intended, and allowed them to register for the upcoming term. However, CMS cannot automatically recognize these students, so the enrollment staff must correct the student records in CMS manually to allow students to register. Campus staff say that in spring 2002 more than 20 percent of students who applied to graduate did not graduate as intended. In addition, campus staff say the former student administration system automated such processes as entering some test scores, correcting student data, and creating new terms for continuing and newly admitted students—all of which now require manual processing using CMS. Campus staff believe this manual processing has significantly increased their workloads and the time they need to complete their responsibilities. To change or add functionality to the CMS software, campuses must request the university to approve a modification. However, not all campus requests for modifications are approved because, in an effort to contain costs, the university approves modifications only if there is a “compelling justification” for the change. The university recognizes that some campuses have taken a step back with some of their functionality. However, the senior director for the CMS project believes these setbacks, 8800 California State Auditor Report 2002-110 California State Auditor Report 2002-110 8811 some of which will be improved by future software releases, are outweighed by the overall functionality across the system. For example, the senior director states that most campuses did not have a human resources system before CMS. She also says that several campuses have taken a step forward with their new CMS, citing Sonoma’s increased functionality in its student administration system. The university plans to add functionality elements to increase the CMS functionality available to campuses. Nevertheless, some campuses must continue to operate some processes with less functionality than before CMS. The CMS Project Has Encountered Other Problems in Work Quality and Information Security Finally, the university has experienced problems with fixing software errors and with information security. Although provid- ing updates and fixing some minor software errors to its newly modified CMS software is expected, the university also needed to make corrections and redistribute some of these CMS software updates and fixes. When the university takes more than once to provide complete updates or fix some errors, campuses must spend more time and money redoing their work or assume the risk of potential system errors. Also troubling is the university’s failure to address information security adequately, exposing confidential information of students and employees. Moreover, some campuses continue to operate the CMS software without adequate password management. Between October 2001 and November 2002, the university Between October 2001 distributed to campuses three releases each of the CMS software and November 2002, applications, totaling nine application releases. The university the university had to had to redistribute two of these within two weeks of their initial redistribute the modified distribution primarily because of errors in the initial releases. software twice because of During this same period, the university distributed 80 updates errors in its initial release. or fixes to the human resources application of the CMS software, but had to redistribute 14 of these. More recently, the university distributed 175 updates or fixes to the student administration application between June and December 2002, but had to redistribute 20. The university had to redistribute the updates and fixes largely because of errors, but also because it needed to change related unclear documentation. As a result, campuses that applied the erroneous updates or fixes either incurred additional expense in reapplying these updates and fixes, or assumed the risk of system errors if they decided not to reapply them. 8800 California State Auditor Report 2002-110 California State Auditor Report 2002-110 8811 Furthermore, the university has not fully addressed its information security needs for CMS. The lack of security around a search feature in the PeopleSoft software apparently allows employees access to the confidential information of other employees and students beyond what is needed to do their jobs. In designing CMS, the university recognized that the structure of the PeopleSoft software provided no security around its search feature other than the option of removing access to it. In 1999, the vice chancellor for human resources expressed concern that employees had access to confidential employee information when using the search feature. Although the university at the time had not begun implementing the student administration application, the human resources and student administration applications are integrated into one database. The integrated software would allow student administration users to access the confidential information of employees, and human resources users to access that of students. Therefore, to address the vice chancellor’s concern, the software was modified to restrict confidential information from being viewed by employees using the search feature. Nevertheless, when the university implemented its most recent Employees using the version of the human resources application, it chose not to CMS search feature have reapply this modification to the new version, as it was now access to confidential planning to implement the student administration application. information, namely Not reapplying this modification would allow employees using students’ and employees’ the search feature to access confidential information, namely Social Security numbers, students’ and employees’ full Social Security numbers, dates of dates of birth, and gender. birth, and gender information. According to a chancellor’s office analysis of the issue, the decision not to reapply the modification enabled the most effective searches. The benefits cited by the chancellor’s office for this decision were to minimize the number of duplicate records in the system, realize time and labor savings, and eliminate a modification that complicates the upgrade and maintenance processes. Also, the senior director for the CMS project states that the university determined that human resources and student administration users need access to the confidential information of all students and employees to do their jobs because otherwise they would not be able to use the capabilities in the CMS search feature to retrieve information about an individual who may have had one or even many prior roles at the university. Further, the senior director stated that, even though they may have access to confidential information of additional individuals, employees are expected to respect the 8822 California State Auditor Report 2002-110 California State Auditor Report 2002-110 8833 confidentiality of this information for any individuals for which they have access. However, this assertion raises the question of why the university initially implemented a modification to restrict the confidential information. When it decided not to reapply the first modification, the university noted that a second modification may have helped resolve the problem. It considered that modification to be costly and not supported by PeopleSoft, so the modification never was developed. Instead, when identifying alternate solutions, the chancellor’s office coupled the removal of the first modification with the implementation of a policy on “sensitive” information, which would place the responsibility for privacy on employees by requiring them to sign confidentiality agreements before getting access to the system. Although the chancellor’s office identified the need for a policy on sensitive information during the software design phase, it did not establish a policy requiring that campuses implement the use of confidentiality agreements for all employees with access to the system. According to the senior director for the CMS project, some campuses have chosen to require their employees to sign confidentiality agreements before giving them access to CMS. For example, staff at one campus we visited told us the CMS software allowed many campus employees access to the confidential information of other campus employees and students beyond what is needed to do their jobs. The project director believes that the campus mitigates any risks by requiring employees with this access to sign confidentiality agreements and attend training before access is granted to them. Although some campuses may have taken some steps to try to mitigate the security issue; nevertheless, because of the limitations of the software cited by the university, The university did not it is implementing an administrative system with less than ensure that the CMS optimal information security. software provided adequate password Likewise, the university did not ensure that the CMS software management. provided adequate password management, such as enforcing minimum password length, requiring frequent password changes, revoking the password after five failed log-in attempts, and not allowing reuse of previously used passwords. The university acknowledged that campuses cannot enhance CMS password management and purchased additional software security tools to mitigate that deficiency, but it had installed this additional software at only one campus as of November 2002. The university says the newest versions of the CMS software will incorporate many of the password management features that have been missing. Campuses can use these new features once 8822 California State Auditor Report 2002-110 California State Auditor Report 2002-110 8833 they install the newest version of the CMS software applications. As of December 2002, nine of the 11 campuses that installed the initial version of the human resources application had installed the newest version with enhanced password controls. However, 14 campuses have installed the initial version of the finance application and are not scheduled to use the newest version until 2004, leaving them without the enhanced password controls for their finance applications. THE UNIVERSITY’S WEAK INITIAL PLANNING AND LIMITED REPORTING EXPECTATIONS HAVE LED TO QUESTIONS ABOUT WHETHER THE PROJECT WILL ACHIEVE ITS BUSINESS OBJECTIVES Doubts about CMS fully accomplishing its business objectives and achieving the potential of a systemwide implementation can be traced to the university’s weak efforts early in the planning process and limited expectations with regard to systemwide reporting. Proper project management practices and planning is essential for success in IT project procurement, design, development, implementation, and final project quality. Although the university used some recommended project management practices during the project’s design and development phase, its initial planning did not adequately ensure that it would fully meet its CMS business objectives. The University Did Not Sufficiently Evaluate Its Business Processes Before Procurement Before purchasing the software, the university did not sufficiently evaluate its specific business processes to understand up front which business processes the potential vendors’ The university had no software products could accommodate and which software basis to anticipate products would require modification to meet its business the extent of software needs. Technical evaluations of potential software products modifications it would were not designed to gain an understanding of the extent of eventually make or the needed modifications. Thorough evaluations of its business loss of functionality processes before it purchased the software would have helped some campuses would the university make an informed decision about what software experience. product would fit its needs best and be the least costly to install. Failing to make these evaluations up front, the university had no basis to anticipate the extent of software modifications it eventually would make or the loss of functionality some campuses would experience. 8844 California State Auditor Report 2002-110 California State Auditor Report 2002-110 8855 The university is conducting its business process evaluations as it starts developing modifications to each PeopleSoft software application. Function teams compare campus needs against software capabilities to identify the software functionality that fits the campus business process requirements, as well as to identify the software functionality gaps. After these gaps are identified, a prototyping team designs the software modifications and proposes them to the university. If the university approves, a technical team develops the modifications to the vendor software to create the final CMS software. As discussed previously, the university has had to make about 200 modifications to the initial versions of the PeopleSoft software. The senior director for the CMS project contends that evaluat- ing the university’s business processes before purchasing the software would not have indicated the number or cost of the modifications to be made, as she believes this would have had to be done in the work sessions that occurred once the PeopleSoft software was purchased. She further states that doing so might A substantial amount have resulted in a less efficient process because the university of evaluation could is first looking at the processes in PeopleSoft to determine if have been performed by they would work for the university before modifying the soft- the university before it ware. Also, she believes that reviewing business processes before purchased the software to procurement might have diluted the opportunities to change better understand how its these processes. Finally, the senior director contends that the business processes fit with university would incur more consulting expense because it the software products it would have to evaluate its business process before and after was considering. purchasing the software, as she believes that simple reengineer- ing would not guarantee a fit with the PeopleSoft software. We recognize that some business process evaluation would need to take place after the university purchased its software. However, the university could have performed a substantial amount of evaluation to understand better how its business processes fit with the software products for the two finalist vendors that it considered during the procurement. Also, at the time it purchased the software, the university was planning to implement software for a voluntary collaborative of campuses rather than systemwide at all the campuses, as we discuss in Chapter 4. However, whether the project was a collaborative endeavor or a systemwide project such as CMS became, the university needed to have a clear understanding of how well the software it was purchasing would meet its needs. 8844 California State Auditor Report 2002-110 California State Auditor Report 2002-110 8855 Maximized Systemwide Reporting Was Not an Intended Benefit of CMS The university plans to continue to use existing processes for systemwide reports because it did not design CMS to replace these processes. Not achieving the full potential from installing a single university-wide suite of administrative software can be traced to the university’s conscious decision to limit this capability. According to the assistant vice chancellor for business planning and information management, “In a major planning effort It was a conscious conducted in 1995 and 1996, the senior management of the decision on the part of [university] endorsed the corporate data principle that ‘data the university to continue collected and maintained should be no more than that required to use existing processes to perform the functions established by senior management for for systemwide reports the chancellor’s office.’ ” Apparently, the university believes that rather than design CMS a single corporate data warehousing operation that provided to replace these processes. chancellor’s office employees access to systemwide information at the same level of detail collected at the campuses and at the same time that campus-level detail is available to campus employees would be inconsistent with this principle. The university chancellor informed us in a December 2002 letter as follows: “I believe it is essential that campus presidents and their management be held responsible for validating and ensuring the accuracy of their campus’ data. Direct access by systemwide staff to campus day- to-day data is not what I envisioned as part of the CMS project. I expect that campus data, validated by campus management, will come to the corporate data warehouse for use by the staff in the system offices.” However, as discussed previously, the corporate data warehouse is not intended to hold a type or level of information substan- tially different from the data it currently holds; the primary difference will be that the information will come from CMS as opposed to the prior systems. We recognize that each campus should be responsible for the accuracy of its data; however, we presume each campus is ensuring the accuracy of the data it is making available to the faculty, staff, and students and that the same level of accuracy would be sufficient for the chancellor’s office staff. Thus, it is unclear why data accuracy would be a reason to not have a corporate data warehousing operation for CMS systemwide reporting purposes. 8866 California State Auditor Report 2002-110 California State Auditor Report 2002-110 8877 In his December 2002 letter to us, the chancellor goes on to say the following: “CMS will (and does) ‘provide ready access to current, accurate and complete administrative information and the means to use it in an efficient manner’ on campuses as well as on a systemwide basis. Campus access is by The university’s current design more ‘ready’ in that campuses can directly access view on the “ready” campus data. ‘Ready’ access at the chancellor’s office as in any large collaborative system, does not require desktop access to information access to all individual campus information. ‘Ready’ in provided by CMS is this model includes the concept of rapid delivery of difficult to reconcile information from campuses in a common format, based with the university’s on common data definitions and elements. It also includes stated vision. the higher level of confidence that each campus can quickly generate comparable information using common reporting structures and procedures that will assure accurate, common data, using standard definitions.” Although an interesting perspective, the chancellor’s viewpoint seems difficult to reconcile with the university’s stated vision that “the [integrated technology strategy] will produce an integrated electronic environment that enables all [university] students, faculty, and staff to communicate with one another and to interact with information resources from anyplace, to anyplace, at anytime.” The chancellor also told us the following in his December 2002 letter: “In the past, campuses were able to report specific campus data; however, that data was often not comparable across campuses and was suspect in its accuracy. For example, we have been consistently incapable of deriving costs for technology expenditures with a high degree of confidence and accuracy. With CMS in full implementation, I expect that I will be able to request expenditure reports on technology and have the confidence that the data will be comparable from campus to campus and that it will be accurate based on the common set of definitions we have applied across the system.” However, the financial application currently being implemented will not give the chancellor the capability he describes. Nevertheless, we recognize that for additional cost, each subsequent version of the financial application can be installed at each campus and can bring more consistency than the current releases. 8866 California State Auditor Report 2002-110 California State Auditor Report 2002-110 8877 In fact, the university recently began to address some of the problems with uniform coding needed to meet systemwide reporting needs in the finance application. In May 2002, the university began addressing systemwide financial reporting for the CMS project, and in July 2002 it approved a modification for the next version of the finance application—release 8.4—that is under development. According to university documentation, this version will create an end-to-end audit trail, standard chart fields, and standard definitions. As of December 2002, the uni- versity expects that campuses will begin implementing this new release in July 2003 and expects it to be used at all campuses by July 2006. However, until campuses implement this new version of the finance application, the university will be without uni- form coding to meet its systemwide financial reporting needs. The University Did Not Establish an Effective Quality Assurance Function Another planning weakness of the CMS project is that the university did not establish an effective quality assurance function, which might have reduced the need to rework software fixes and improved information security. Recommended project management practices dictate a quality management plan to provide the overall framework to ensure project quality. Such a plan allows employees to understand, implement, and maintain quality at all levels of the information technology project. However, the university has not established a quality management plan for the CMS project. It does have processes to design, develop, and test software modifications After more than three according to plan specifications, and it has adequate change years of designing, control policies and procedures for the CMS project. However, developing, and until recently, the responsibility for these processes was assigned implementing CMS, the to the same team doing the work, without review by a quality university only recently assurance team. A quality assurance team is intended to help began establishing a identify and assess the significance of quality problems that quality assurance group. occur as the project progresses and to help minimize project risks by identifying potential problems earlier in the project development and implementation phases. After more than three years of designing, developing, and implementing the CMS project, the university only recently began establishing a quality assurance group. In November and December 2002, it hired two quality assurance analysts to focus specifically on development of software modifications. The university has never hired an independent oversight con- sultant to perform various quality assurance functions and to evaluate the progress of the project, nor does it plan to do so. 8888 California State Auditor Report 2002-110 California State Auditor Report 2002-110 8899 Independent oversight consultants help ensure that appropriate project management practices are implemented and followed throughout the project’s life. Hiring an independent oversight consultant may likely have assisted the university in identify- ing and addressing quality assurance and information security deficiencies earlier in the CMS project. THE UNIVERSITY’S PROCUREMENT APPROACH DID NOT SHARE PROJECT RISK WITH VENDORS AND CONSULTANTS The university’s procurement approach of identifying, procuring, and implementing its own solution caused it to assume substantially all the responsibility for the CMS project, sharing little if any project risk with vendors and consultants. As a result, it assumed the considerable financial and business risk involved in ensuring that the software meets its business needs and is implemented successfully at campuses. By procuring its own CMS solution through a piecemeal approach, the university effectively assumed nearly all responsibility for project success and for the risk of an unsuccessful project. The university’s procurement of software for the CMS project in September 1998 resulted in it agreeing to pay PeopleSoft $33 million for the right to use the software for the next eight years, and for an initial amount of training and consulting services. The contract was ultimately amended to $37 million for additional software products and maintenance. It then hired consultants on an hourly basis to help it identify campus business needs, to design and develop the modifications needed for the software, and to help implement this software at campuses throughout the university system. It believed that, without using consultants, it could not rapidly find The university could enough people with the skills and experience needed to ensure have entered into a successful implementation. contract that paid the firm only upon completion However, the university could have structured its procurement of key deliverables. so that, in return for a fixed fee, the winning firm would be responsible primarily for the successful implementation of whatever software product the university decided to use. The university then could have entered into a contract that paid the firm only upon completion of key deliverables, such as the successful modification of functionality elements within the software to meet the university’s needs. Structuring contracts to pay only after deliverables have been tested and accepted is 8888 California State Auditor Report 2002-110 California State Auditor Report 2002-110 8899 a recommended procurement practice. Instead, the university chose to purchase only the software, and it is conducting the substantial amount of work, with the assistance of consultants paid through additional contracts, necessary to ensure that the software is modified and implemented properly. The university concluded that it was best for it to modify and implement the software but it never performed sufficient analysis to determine that a university installation provided the best value. Further, the university has attempted to make a best-value implementation without the benefit of a diagram that shows The university has the relationship among the many database tables used by the attempted to implement PeopleSoft software. When we were performing our analysis of CMS without the benefit CMS costs, we asked the university for the entity relationship of a diagram that shows diagram that would show the relationship among tables and the relationship among how the information interrelates among certain tables. We the many database tables were informed that it does not have an entity relationship used by the software. diagram for the PeopleSoft system. When we inquired as to why the university does not have a copy of the entity relationship diagram, we were informed that there was an additional cost from PeopleSoft for this information. The university is assuming all the financial and business risk for the successful implementation of CMS, so a diagram showing its staff how the tables in the system interrelate with each other would seem like a prudent item to include in the purchase price of the software. Although we recognize that certain consultants may understand these relationships, the lack of the entity relationship diagram may impede the implementation of the system in the short term and may make the university more dependent on consultants in the long term. It also did not share risk when it procured consultants to assist in modifying and implementing the software. The chancellor’s office and the campuses we visited typically pay consultants on an hourly basis rather than for the delivery of accepted work products. For example, to adapt the software to meet its business needs, the chancellor’s office hired consultants to lead the process of examining the software to see if business processes can be accommodated or whether programming modifications will be required or business processes adjusted. Chapter 2 and Appendix D provide more details on the consultants employed by the university and their respective payments. Rather than have the consultants propose a fixed price for the work and pay only after accepting delivered work products, the chancellor’s office paid the consultants monthly for the hours billed at their preestablished hourly rates. The campuses we visited also 9900 California State Auditor Report 2002-110 California State Auditor Report 2002-110 9911 generally followed the practice of paying consultants on an hourly basis rather than at fixed prices for the various work products. The university exposes itself to the risk of paying for services that do not meet its needs when it pays consultants before testing and accepting work products. For one of the more recent CMS software development and implementation efforts, the student administration application, When the university pays the university shared some project risk with consultants consultants before testing by structuring the agreement so it contained a fixed-fee and accepting work component that linked payments to project milestones. The products, it exposes itself agreement’s fixed-fee component is paid through monthly to the risk of paying for payments that increase upon completion of two key milestones. services that do not meet Specifically, the chancellor’s office entered into an agreement its needs. with a consultant for developing and implementing the student administration application through a pilot project on two campuses and the chancellor’s office. The chancellor’s office structured the payment terms so the CMS software development was on a time-and-materials basis, while campus implementation services were on a fixed-fee basis. However, the effectiveness of the chancellor’s office risk-sharing concept may be diluted when campuses issue additional consultant work orders. For example, we noted that one pilot campus issued a work order that was on a time-and-materials basis to the same consultant during the same period for work outside the pilot project. Further compounding this problem is that time-and-materials orders can be more difficult to manage. For instance, because the consultant exceeded the $115,000 time-and-materials estimate and the time period stated in the work order, this same campus had to increase the work order to pay an $82,000 outstanding invoice for work the consultant performed beyond the terms specified in the initial order. The campus project director believes a material billing error is highly unlikely but acknowledges that there were no processes to ensure that the consultants working for the campus and the chancellor’s office were not billing both or that the hours detailed on the consultant invoice were for the appropriate work order. Nevertheless, the effectiveness of the chancellor’s office fixed-fee terms could have been diluted by campus work orders on a time-and-materials basis such as this one. 9900 California State Auditor Report 2002-110 California State Auditor Report 2002-110 9911 RECOMMENDATIONS To ensure that it minimizes the costs and time to implement and maintain its CMS software, the university should do the following: • Reassess the design of CMS and evaluate the economies that can be achieved by reducing the number of separate CMS databases throughout the university that currently must be tested separately when campuses implement software revisions that reflect modifications and upgrades. • Continue its recently established practice of tracking the actual hours spent to develop modifications to the CMS software. It should consider this information when esti- mating the costs and time associated with developing and applying future modifications to new versions of the vendor software, and when evaluating the associated maintenance costs in reapplying, testing, and implementing its current and future modifications. • Define the scope and associated costs of CMS by identify- ing the specific functionality that is necessary to achieve the university’s expectations expressed in the CMS project charter. Further, examine the costs associated with campuses’ plans to add functionality elements to increase functionality beyond this defined scope. Also, when procuring information technology systems or soft- ware in the future, the university should evaluate its specific business processes against vendor products before procurement, then select vendors that best accommodate the university’s specific needs. To ensure that it provides ready access to current, accurate, and complete administrative information and the means to use this information effectively as well as to establish stan- dards to share information for common reporting purposes, the university should determine how it could improve the design of CMS to report systemwide information. Additionally, it should establish a minimum level of functionality that all campuses must implement to facilitate this reporting. 9922 California State Auditor Report 2002-110 California State Auditor Report 2002-110 9933 To ensure that it adequately addresses CMS project quality and information security, the university should: • Establish a quality management plan and continue its efforts to establish an effective quality assurance function for the CMS project. • Consider hiring an independent oversight consultant to per- form various quality assurance functions and to evaluate the progress of the CMS project. • Establish a policy on sensitive information requiring that campuses implement the use of confidentiality agreements for all employees with access to CMS. To ensure that it uses recommended practices in its future pro- curements, the university should plan project procurements to share risk with vendors and consultants, such as allowing them to propose their solutions and structuring contracts to protect the university’s interests, including provisions to pay only after deliverables have been tested and accepted. n 9922 California State Auditor Report 2002-110 California State Auditor Report 2002-110 9933 Blank page inserted for reproduction purposes only. 9944 California State Auditor Report 2002-110 California State Auditor Report 2002-110 9955 CHAPTER 4 The Processes the University Used to Select the Software Vendor and Consultants on the Project Did Not Clearly Demonstrate Best-Value Procurements CHAPTER SUMMARY The California State University’s (university) processes to select the software vendor and consultants for the Common Management System (CMS) project did not clearly demonstrate best-value procurements that consider both quality of proposals and overall costs. The procurement process by which the university selected a single CMS software vendor raises questions about whether the university used a fair and objective competitive process. Although its solicitation document did not provide for a method to select only one vendor, the university decided late in the process that it needed such a method. When the selection narrowed to two vendors, the university did not use a quantitative scoring process to select a best-value vendor objectively. Likewise, the university could not demonstrate that it resolved issues the procurement evaluation teams raised. Further, the university based its procurement decision on a cost analysis comparison of the two vendors that did not compare costs for a systemwide implementation and that was based on a fraction of the actual maintenance and operations costs now estimated. Moreover, the university’s practice of employing consultants to work on the CMS project without appropriate competition raises more questions about the propriety of its business dealings. The university hired consulting firms under sole-source contracts for reasons that appear questionable. Finally, although it recommends discussions with consulting firms about scope of work and rates, the university does not require the solicitation of offers from more than one prequalified consultant with university-awarded master agreements. As a result, the university has not always solicited offers from multiple prequalified consultants before procuring their services and, therefore, cannot demonstrate that it procured best-value services. 9944 California State Auditor Report 2002-110 California State Auditor Report 2002-110 9955 THE UNIVERSITY’S SELECTION OF THE CMS SOFTWARE VENDOR WAS PROBLEMATIC When the objective of the CMS software procurement changed from selecting one or more vendors to selecting one vendor, the university continued with the procurement process, even though its solicitation document did not provide for a method to select only one vendor. Not restarting the process or formally modifying the process when its procurement objective and methodology changed raises questions about the propriety of the procurement process used in the CMS software vendor selection. The Procurement Objective for CMS Software Evolved from Identifying One or More Vendors to Selecting One Vendor A university task force, formed in 1996 to consider replacing financial systems, started the procurement process in 1997 with an objective of identifying one or more software vendors to pro- The original intent of the vide integrated systems that also included student and human CMS software procurement resources services. These vendors then could be selected by one was to identify one or or more campus collaboratives, voluntary groups of campuses more software vendors using shared resources to select the best administrative software that one or more campus suite for their respective campuses. Because campuses with vary- collaboratives could select. ing needs for administrative software preferred various software vendors, the university considered establishing contracts with various vendors from which campuses could choose. Therefore, in April 1997, the university issued a solicitation document, a request for qualifications (RFQ), that planned for a process to select one or more vendors for one or more collaboratives. Figure 6 provides a timeline of the events for the CMS software procurement process. Seven vendors responded in June 1997 to the university’s RFQ, and in July 1997 the university qualified three to proceed to the next procurement phase. Using various teams—finance, human resources, student administration, and technical—to evaluate the software of the three qualified vendors, the university passed all three vendors to the best-offer phase. In November 1997 the university requested the three vendors to provide their best offers that identified the benefits of multiple campus licenses. In December 1997, the university received these offers and began to analyze various cost scenarios with the objective of providing cost information to campuses to assist them in deciding on their participation in a collaborative. The university subsequently requested additional information and updated offers from ven- dors, receiving them in March 1998. 9966 California State Auditor Report 2002-110 California State Auditor Report 2002-110 9977 9966 California State Auditor Report 2002-110 California State Auditor Report 2002-110 9977 FIGURE 6 CMS Software Procurement Timeline ������������� ������������������������������ ������� ���������������������������� ����������������������������������������������� �������� ������������������������������ ������ �������������������������� ���������������������������������� ������������������������ ������� ������������������������������������ ���������������� ����������� ������������������������������������������ ������ ��� ���������������������� ������������������������������� �������������� ������������������������������� ���� ������������� ������������������������� ����������������������������� ���������������� ���� ���������������� ���������������� ������������������� ������ ��������������������� ������������ ����������� ���������������������� ��������������������� ���������������������� ������������ ������������������� ������������������ ���� ���� ���� ���� �������������� ������ ����������������������� ���������������������� ��������������� ������������������� ��������������������� �������������������������� ����������������������� ���������� ���������������� �������������������� �������������������������� �������� ����������� ������������������������������������������� �������������������� ���������������������������������������������� ���������������������� ���������� �������� �������������������������������� ����������������������������� ������������������������� ��������������������������� Source: The university’s contracting and procurement documents and committee meeting minutes and notes. However, during the final stages of the procurement, the university changed its objective from considering one or more vendors for one or more collaboratives to selecting only one software vendor for a single collaborative. By December 1997, the CMS task force (formerly the financial systems task force) began focusing toward creating a financial incentive for campuses for only one collaborative. In April 1998, the university decided to select a single software vendor for the CMS software. It acknowledged that multiple collaboratives still could form around different software vendors, but concluded there should be only one collaborative that received financial incentives from the chancellor’s office for ongoing operations. During the final stages In May, the CMS task force gathered more information by of the procurement, requesting additional clarifying information, attending the university decided additional vendor demonstrations of vendor systems, and to select a single conducting evaluation team visits to other university systems software vendor. that were using the vendors’ software. In July 1998 this lengthy procurement process resulted in the university deciding to negotiate a contract with PeopleSoft as the software vendor for the CMS project. Although its process had qualified two other vendors, the university never negotiated contracts with them. The university entered into contract negotiations and in September 1998 executed a $33 million contract, ultimately $37 million after amendments, with PeopleSoft for software licensing, training, and some consulting services. Effectively purchasing a systemwide license for software available to all campuses, the university thereby created a de facto standard for systemwide administrative software. Minutes from a CMS implementation steering committee meeting in December 1998 show the university had determined that all 23 campuses would implement CMS in five to seven years and that it now needed to determine the costs and resources necessary to make this happen. According to the chancellor, in July 1999 campus presidents unanimously endorsed the decision to move forward with a systemwide CMS. Not Restarting or Formally Modifying the Procurement Process When the Methodology Was No Longer Compatible With Its Objectives Raises Questions Not restarting or formally modifying the procurement process when its methodology was no longer compatible with its pro- curement objectives raises questions about the propriety of the university’s CMS software vendor selection process. When it became apparent that it needed to select a single vendor, a year 9988 California State Auditor Report 2002-110 California State Auditor Report 2002-110 9999 into the process, the university hastily created an additional step—a final evaluation process to identify and select a single software vendor for the collaborative that was to receive finan- cial incentives from the chancellor’s office. The university’s procurement process was not geared initially to identify a single vendor. Although the CMS solicitation document, the RFQ, outlined four phases—qualification, The university’s RFQ did discovery, best offer, and final negotiation—none provided not outline a process to for selecting a single vendor. Rather, the RFQ provided for an select a single vendor. evaluation of vendor responses to qualify vendors to participate in subsequent phases. Although one evaluation team performed some scoring as we discuss later, the RFQ did not provide a scoring method to select a single vendor. Also, its final phase outlined a process for discussions to finalize contracts with vendors who were successful in the previous phases, whether it was one or more vendors. This process was problematic because late in the process—April 1998, four months after it first received “best offers” from the vendors—the university decided to recommend a single vendor. Thus, the university needed an evaluation process that demonstrated it objectively surfaced the best-value vendor, considering both the quality of the vendors’ proposals and the costs associated with the vendors’ offers. However, the university did not restart the process by issuing a new RFQ that defined the evaluation process that was to be used to select a single vendor. Additionally, it did not formally modify the process by notifying the potential vendors in writing that it was changing the overall process outlined in the RFQ, nor did it inform them of the specific process by which it now expected to evaluate the vendors to select a single winner. Instead, it simply proceeded with an evaluation process to select a winning vendor. According to the executive vice chancellor and chief financial officer (executive vice chancellor), the university believes the evaluation and scoring conducted to select the CMS vendor was comprehensive and sufficient to meet any test of objectivity. The executive vice chancellor says the university evaluated vendors on criteria in the RFQ, as reported by the CMS task force in a document prepared in June 1998. The document focused only on PeopleSoft and the Systems and Computer Technology Corporation (SCT) because the university concluded that one of the three vendors being considered—Oracle—could not demonstrate that it could offer a student administration product. The document states that the significant potential for future opportunities and flexibility were the primary factors differentiating the two vendors; functionality and 9988 California State Auditor Report 2002-110 California State Auditor Report 2002-110 9999 ability to deliver were judged to be comparable. Although the technological foundation of both products was considered acceptable, several aspects of the PeopleSoft architecture, such as more tightly coupled modules providing for potentially better integration, were thought to have an advantage over that of SCT. Finally, the task force document concluded that, although both vendors were financially viable, PeopleSoft was more The university did not favorably positioned financially for the increasingly competitive use a weighted scoring market environment. process to select a best- value vendor objectively. However, the university’s evaluation process did not use a weighted scoring process to select a best-value vendor objectively. For the two finalist vendors, the CMS task force’s document simply placed positive and negative comments into plus and minus categories for various product dimensions and the vendor-client relationship. Because the university did not use a quantitative, weighted scoring evaluation process, we could not tell whether one factor carried more weight than another and to what degree one plus or minus comment was better or worse than another plus or minus comment. The University Could Not Show That It Resolved Evaluation Teams’ Concerns About Potential Vendors and Discounted Some Information Favoring the Vendor Not Chosen Further, the university could not demonstrate how it evaluated and resolved the evaluation teams’ various concerns and comments in the earlier discovery phase in fall 1997. Each evaluation team reported its evaluations and concluded that all three vendors (Oracle having not yet been excluded) should continue to the best-offer phase of the procurement process. However, along with positive comments, some teams reported concerns or negative comments for each software vendor. The university could not provide us with documentation demonstrating how it resolved, mitigated, or acknowledged acceptance of the risks associated with these concerns for any vendor being considered. Especially problematic, the university could not demonstrate it did this for the selected vendor. For example, the student administration function team reported several concerns about the PeopleSoft product that appeared to be significant. The team’s September 1997 report included comments about the PeopleSoft product such as, “the base product as shown is much more labor intensive than current system,” and, “[we have] serious concerns about meeting [university] functionality requirements,” and, “see no vision for the student systems other than to create a ‘completing piece’ 110000 California State Auditor Report 2002-110 California State Auditor Report 2002-110 110011 for their [human resources] and finance systems.” Further, although the RFQ did not have a scoring process, the student administration team scored SCT higher than PeopleSoft in 13 of the 15 categories evaluated. The financial team reported that SCT’s product basically met all the university’s requirements, whereas PeopleSoft lacked in accounts receivable, student revenue processing, grant processing, and student housing financial matters. In May 1998 the university gathered more information about the vendors and their software, observing vendor product demonstrations and performing site visits to out-of-state university systems. Although this additional information may have resolved or mitigated many of the Although additional evaluation teams’ concerns, without documentation, the information gathered university cannot show that it addressed them adequately. by the university may have resolved or The university also discounted other information that favored mitigated evaluation the vendor not chosen, thus raising more questions about its teams’ concerns, the procurement process. For each vendor asked for a best offer university cannot show in November 1997, the university provided survey results that the concerns were showing campus interest in the vendor. This survey shows addressed. that 13 campuses indicated a high interest for SCT, while only three indicated a high interest for PeopleSoft. The university’s executive vice chancellor said the university selected the CMS software vendor on a variety of factors, not just on factors where one vendor may have an advantage at one point in time. Also, the executive vice chancellor said the university’s selection criteria placed value on PeopleSoft being a visionary in the industry. In April 1997 a Gartner Group “industry scan” report, a university-commissioned study to evaluate educational software vendors, did identify PeopleSoft as a “visionary” and SCT as a “leader.” However, the Gartner Group reported that visionaries have “strong market vision but unproven execution capability,” whereas leaders have “strong visions about market trend and direction . . . and have successfully demonstrated their execution capabilities.” Therefore, according to the university- commissioned study, both vendors had strong vision, but SCT had proven execution capabilities. Although the university may have had valid reasons for deciding to select PeopleSoft, its inability to show how it resolved these discrepancies cast some doubt on the appropriateness of its CMS software procurement process. 110000 California State Auditor Report 2002-110 California State Auditor Report 2002-110 110011 The University Could Not Show How It Determined That Cost Differences Between the Competing Vendors Were Immaterial When asked whether the university compared overall costs of each vendor’s software, the executive vice chancellor pointed us to the June 1998 document where the CMS task force indicated that the cost differences between PeopleSoft and SCT were rela- tively immaterial. However, the university could not provide an analysis that demonstrated the differences were immaterial, nor explain the threshold it considered to be material. The universi- ty’s April 1998 estimates of each vendor’s costs, used during the procurement decision, are shown in Table 9. TABLE 9 The University’s Estimate of Campus Participation Costs for the Two Finalist Vendors in the CMS Software Procurement Campus Size PeopleSoft SCT Small One-Time* $4,758,272 $3,441,778 Annual† 181,403 182,410 Medium One-Time* 5,295,467 4,340,680 Annual† 259,431 257,254 Large One-Time* 5,947,177 6,203,910 Annual† 344,525 378,028 Source: The university’s April 1998 CMS Vendor Cost Summary. *The university calculated the estimated one-time cost assuming a 10-campus collaborative. † The university calculated the estimated annual recurring costs to campuses assuming a 10-campus collaborative. These annual costs are net of $408,500 in post-implementation costs, which were expected to be covered by the chancellor’s office through an incentive to support a team of university staff and consultants for maintaining the software. The estimates in Table 9 present the university’s estimated one-time and ongoing costs that a small, medium, or large campus should expect to incur for participating in a 10-campus collaborative with PeopleSoft or SCT. However, the university’s RFQ did not provide for a comparison of overall vendor costs, nor did the university compare total vendor costs for a 10-campus collaborative. In fact, it did not have a particular collaborative in mind. According to the chief of staff for information technology services, the university’s intention was only to provide individual campuses with cost information for deciding whether to join a collaborative or to understand the costs 110022 California State Auditor Report 2002-110 California State Auditor Report 2002-110 110033 involved for those campuses already committed to participation. The university’s analysis was not meant to estimate overall costs to the collaborative, let alone costs for a systemwide implementation of the vendor software. Thus, it is unclear how the university concluded cost differences between the two vendors were immaterial. The Procurement’s Cost Analysis Was Based on a Fraction of the Costs Being Incurred to Implement CMS The cost analysis on which the software procurement was based represents just a fraction of the one-time and annual costs of the CMS project that is being implemented. Therefore, at the time the university was considering the CMS software procurement, it did not use a realistic estimate of the complete costs to implement and maintain the software. Using the information from Table 9 and assuming eight large, eight medium, and seven small campuses, the one-time costs to implement the system using PeopleSoft software at 23 campuses would be $123 million and the annual maintenance and operating costs would be $16 million. These amounts do not include the $408,500 annual incentive each campus could expect from the chancellor’s office, which totals $9 million for 23 campuses. In comparison, based on the June 2002 cost survey, for the CMS project now being implemented, the one-time costs are expected to be The disparity between the $393 million and the annual maintenance and operating costs procurement and the later $65 million once CMS is installed at all campuses. estimates suggests the university either did not This disparity between the procurement and 2002 estimates consider or significantly suggests the university either did not consider or significantly underestimated project underestimated one-time costs and annual costs. Some of these costs. costs could not have been included because, as discussed in the previous chapter, the university did not sufficiently evalu- ate its business processes to understand the costs to modify and maintain the software for which it was seeking bids. Further, the university assumed that the costs of certain aspects of the system would be the same regardless of the vendor, even though it provided us with no analysis supporting this assumption. This unsupported assumption is especially bothersome because of the size of the $65 million anticipated for ongoing maintenance and the striking difference between that amount and the $16 million considered by the university in its analysis. 110022 California State Auditor Report 2002-110 California State Auditor Report 2002-110 110033 THE UNIVERSITY’S SELECTION OF CMS PROJECT CONSULTANTS IS TROUBLESOME AND DOES NOT ENSURE BEST VALUE The university’s practice of awarding CMS project contracts to consultants without appropriate competition is troublesome and Since early in the CMS does not ensure that the university receives the best value. The project, the university university’s policy manual for contracting and procurement says established a pattern of the university is to promote fair and open competition to the awarding sole-source maximum extent possible. The policy states that, except when consulting contracts necessary for the immediate preservation of the public health, for reasons that appear welfare, or safety, or the protection of university property or questionable. programs, sole-source contract procurement is permissible only upon determination that only one source exists for the required product or service. However, since early in the CMS project, the university has established a pattern of awarding sole-source con- sultant contracts for reasons that appear questionable. Further, the university has facilitated chancellor’s office and campus selection of consultants from university master agreements without requiring the solicitation of offers from more than one prequalified consultant to ensure best value. The University Effectively Hired PeopleSoft as a Sole-Source Contractor Early in the CMS project (October 1998), the university hired consultants from PeopleSoft without additional competition. Although the prior contract with PeopleSoft (the September 1998 CMS software vendor contract) provided for initial training and consultant time, the chancellor’s office entered into a second contract with this vendor for additional consulting services for the chancellor’s office and for campuses. However, the second PeopleSoft consulting contract was effectively sole-sourced. Further, PeopleSoft was excused from participating in all of a later qualification process for other consultants and allowed to continue with its previously sole-sourced contract. Such actions do not seem to promote the university’s policy of fair and open competition. As of June 2002, the chancellor’s office and campuses expected to spend about $12 million on PeopleSoft consulting services under this second contract. According to the university’s director of contract services and procurement, the process the chancellor’s office used to enter into this second PeopleSoft contract was based on the master agreement awarded to PeopleSoft a month earlier, as an outcome of the CMS software vendor RFQ effort. The director of contract 110044 California State Auditor Report 2002-110 California State Auditor Report 2002-110 110055 services and procurement stated that a separate contract governing consulting services was needed because the PeopleSoft master contract primarily addressed software licensing and maintenance. The director further stated that PeopleSoft was the primary source for technical expertise on its software at that time. Thus, the university effectively hired PeopleSoft to provide consulting services through a sole-source procurement. However, it subsequently identified numerous vendors with PeopleSoft expertise. In fact, as shown in Appendix D, non- PeopleSoft consultants are being paid for much of the CMS project work that the university now outsources. Moreover, when the chancellor’s office solicited for additional consultants in February 1999, less than four months after the October 1998 contract, PeopleSoft did not go through the same qualification process as the other consultants. The chancellor’s office scored, When qualifying based on technical and cost considerations, proposals from consultants to provide 13 firms, including PeopleSoft. PeopleSoft received the lowest services for the CMS score, other than four firms that then were dropped from further project, the university did consideration. However, the chancellor’s office did not require not require PeopleSoft PeopleSoft to continue through the qualification process. to go through the same Instead, the director of contract services and procurement said process as the other firms that PeopleSoft, in essence, already had been prequalified as a it considered. partner and that another agreement established under this later procurement would not be needed. The University Hired Io Consulting as a Sole-Source Contractor In April 1999 the chancellor’s office hired another firm, Io Consulting, through a questionable sole-source procurement, saying one individual at that firm was needed as project manager for the CMS human resources application. Now, after many contract amendments that expanded the sole-source contract’s scope, the chancellor’s office paid Io Consulting about $5.1 million for the services of not one, but 10 individuals, calling into question the fairness of the university’s procurement practices. The sole-source justification for the contract stated that, although the contract was to be with Io Consulting, the chancellor’s office actually needed the specific expertise of one individual in the firm, citing his unique qualifications needed for the CMS project. Based on her previous work experience with this consultant, the newly hired senior director for the CMS project recommended the sole-source contract and vouched for the consultant’s unique experience and knowledge. This arrangement expanded significantly, from a $350,000 sole-source contract for one consultant to a contract that continued until June 30, 2001, through seven amendments of the original—adding nine individuals and increasing the 110044 California State Auditor Report 2002-110 California State Auditor Report 2002-110 110055 total contract amount to about $5.1 million. The chancellor’s office relied heavily on Io Consulting to identify, design, and develop modifications for the CMS software, and to manage and Io Consulting’s sole- direct both contract and university staff. Although citing sole- source arrangement source justification of unique qualifications for Io Consulting, expanded through seven the chancellor’s office simultaneously began to identify and amendments from a qualify other firms to implement the CMS project. As previously $350,000 contract to a discussed, the chancellor’s office solicited for additional firms total contract of in February 1999 and in September 1999 selected seven of these about $5.1 million. firms to enter into master agreements—yet continued amending the sole-source contract with Io Consulting. Figure 7 gives a timeline of the events regarding Io Consulting. Although the initial contract with Io Consulting focused on the services of one individual, the chancellor’s office was well aware when it entered the initial contract that it was going to need additional consulting services. In fact, a project proposal incorporated into the contract stated that “additional resources from Io Consulting are available to begin work as early as May 1, 1999, on the functional and technical work that will be required for the successful completion of the project.” However, rather than competitively procuring the needed resources, the chancellor’s office simply brought in additional Io Consulting resources by continually amending the sole-source contract. Further, we noted that the chancellor’s office did not always promptly execute amendments for the Io Consulting resources it added, executing several contract amendments after the additional consultants began work. Moreover, when Io Consulting’s sole-source contract was not extended further, it continued to work under an arrangement that, considered in conjunction with the repeated amendments to the sole-source contract, gives the appearance that the university was attempting to circumvent competitive bidding requirements. After more than two years, the chancellor’s office did not extend the Io Consulting contract after the seventh amendment, and the sole-source contract ultimately ended on June 30, 2001. The university’s director of contract services and procurement told us the contract was not extended because she advised the CMS project staff that any further contracts with Io Consulting should be validated by competition. But because the chancellor’s office wanted to continue using Io Consulting on the CMS project, it had another consulting firm, KPMG, add Io Consulting, with higher hourly rates, as a subcontractor in July 2001. The university continued to use Io Consulting under this subcontracting arrangement through February 2002, paying approximately $684,000 for the services provided. 110066 California State Auditor Report 2002-110 California State Auditor Report 2002-110 110077 110066 California State Auditor Report 2002-110 California State Auditor Report 2002-110 110077 FIGURE 7 Timeline of the University’s Contracting With Io Consulting ������������������ ������������������������������� ����������������� ��������������������������� ������������������������ ��������������������������� �������������������� ���������������������������� ��������������������������� ����������������������������� ����������������������� ������������ ���������������������������������� ����������������������� ������������������ ���� ����������������������������� ��������������������������� ��������������������������������������� ���������������� �������������������������� ������������������������ ��������������������� ��������������� ����������������������� ��������������������������� ����������������������������� �������������������������� ������������������������� ������������������������������������� ����������������� ��������������������������� ���� ���� ���� ����������������� ��������� ������������������������ ������������������������������������� ��������������������� ������������������������������� ������������������������ ���������������������������������� ���������������������� �������������������������� ����������������� �������������������� ��������������������������������������� ���������������������������������������� ������������������������������������������ ����������� Source: University contract and payment documents. Meanwhile, the university executed a three-year master agreement with Io Consulting in November 2001 and continued to use Io Consulting under this master agreement. As of October 2002, Io Consulting had earned an additional $1.7 million under this master agreement at the chancellor’s office alone, not including earnings from services performed and paid at the campuses. Additionally, as of June 2002, campuses paid Io Consulting more than $9.8 million for services, such as helping campuses implement the CMS software Io Consulting helped design and develop. Finally, when performing a limited review of the remaining 10 consultants that the chancellor’s office paid more than $100,000 as of June 2002, we found four other consultants for which the chancellor’s office used sole- source contracts. The amount paid to these four sole-source consultants totaled more than $1.3 million as of June 2002. At the Long Beach campus, one of three campuses we visited, we found additional problems with the sole-source procurement Not enforcing its own of consulting services. For example, after completing a limited policy for sole-source comparison of Io Consulting against two other prequalified contracts leaves the consultants that had master agreements, Long Beach decided university vulnerable to select Io Consulting. Io Consulting did not have a master to criticism that it has agreement in place, so Long Beach applied to the chancellor’s not been equitable in office for sole-source approval in August 2000. Although the selecting consultants. director of contract services and procurement at the chancellor’s office denied the sole-source request in September 2000, it allowed the $1.8 million contract to proceed anyway by defining Long Beach’s limited comparison as a “competitive” process. However, because the selection process used by Long Beach was not open to all vendors, this award failed to meet the university’s procurement policy for open competition to the maximum extent possible. When it does not enforce its own policy for sole-source contracts, the university leaves itself vulnerable to criticism that it has not been equitable in selecting consultants. Further, Long Beach amended this contract in December 2001 to extend the term from July 2001 to October 2001 and to increase the amount by $318,000 to cover some of the work Io Consulting performed after the contract expired in July 2001. Long Beach continued to procure Io Consulting services through Io Consulting’s November 2001 master agreement. 110088 California State Auditor Report 2002-110 California State Auditor Report 2002-110 110099 Campuses Selected Consultants From University Master Agreements Without Ensuring That They Received the Best Value Campuses also did not always ensure that they obtained the best-value consultants when selecting firms from the university’s master agreements. The chancellor’s office established groups of master agreements in 1999 in which seven consulting firms were When campuses using selected, and again in 2001 when an additional 15 firms were master agreements do not selected. The chancellor’s office evaluated and scored the con- solicit offers from more sulting firms’ qualifications and proposals, and selected those than one consultant, they that met a minimum score. These master agreements facilitate cannot demonstrate that the campus selection of prequalified consultants according to they selected the best- preestablished terms and conditions outlined in each master value consultant. agreement. Although master agreements may have certain bene- fits, such as multiple campuses avoiding the time and labor costs to identify qualified consultants, prudent steps must be taken to ensure best value, considering both quality of the services and cost. According to the university’s master contracts bulletin, the master agreements only provide ceiling labor rates and campuses may negotiate more favorable rates to reflect campus require- ments. Therefore, unless they solicit offers from more than one prequalified consultant, campuses cannot be sure they obtained the best-value consultant to meet their needs. When campuses do not take additional steps to solicit offers from more than one consultant, they cannot demonstrate that they evaluated their options thoroughly and selected the best-value consultant. Further, although the university is exempt from the State’s requirements for using master agreements, the California Department of General Services encourages it to comply with the intent of its directives addressing master agreements that it began issuing in May 2002. These directives generally require state agencies to solicit at least three offers when using master agreements to procure information technology goods and services. The director of contract services and procurement states that the university does not believe the intent of this directive is relevant to its master agreements because it considers the agreements to be established through competition. However, establishing such a policy would be a prudent action and would be consistent with its guidance to campuses in February 2002. Although it did not make it a requirement, the chancellor’s office recommended in its February 2002 master contracts bulletin that campuses enter into discussions about the scope of work and labor rates with a sufficient number of firms to determine the one that best fits their requirements. 110088 California State Auditor Report 2002-110 California State Auditor Report 2002-110 110099 However, two campuses we visited that selected several consultants through master agreements did not always take The project director at these recommended steps to ensure that they obtained the one campus we visited best-value consulting firm. The Sonoma campus project director acknowledged that explained that he generally learned about consultants who generally no discussions were performing work on the project and simply hired them took place with other through the master agreements that the chancellor’s office had prequalified consultants put into place. However, the project director stated that for before selecting a the student administration pilot project, the Sonoma campus consultant from the participated with the chancellor’s office and the Fresno campus master agreement. in developing a request for consulting services, evaluating responses, and ultimately recommending the selection of Cedar (formerly the Hunter Group) as the project’s implementation consulting partner. Similarly, the Fresno campus typically did not take additional steps to determine best value when selecting consultants that had master agreements in place. The campus project director acknowledged that generally no discussions took place with other prequalified consultants before hiring from the prequalified list of consultants with master agreements. For example, the project director told us that in one instance the campus simply selected a consultant because the campus project director had observed the consultant’s work at the chancellor’s office and said the consultant was qualified. When campuses do not take additional steps to solicit offers from more than one consultant, they cannot demonstrate that they thoroughly evaluated their options and selected the best-value consultant. RECOMMENDATIONS To ensure that it uses recommended practices in its future procurements, the university should do the following: • Use the procurement process appropriate to the procurement objective. If the procurement objectives change during the process, it should restart the procurement using the appropriate process or formally modify the procurement process through appropriate written notification to potential vendors. • Establish a practice of using quantitative scoring to demonstrate clearly that it followed an objective evaluation process to identify a best-value vendor. It also should document the resolution of evaluation team concerns to demonstrate it considered and appropriately addressed or mitigated these concerns. 111100 California State Auditor Report 2002-110 California State Auditor Report 2002-110 111111 • Enforce its policy that prohibits the use of sole-source con- tracts when multiple vendors or consultants are available to provide the goods or services. • Establish a policy to require the solicitation of at least three offers for its prequalified vendor goods and consultant services master agreements. n 111100 California State Auditor Report 2002-110 California State Auditor Report 2002-110 111111 Blank page inserted for reproduction purposes only. 111122 California State Auditor Report 2002-110 California State Auditor Report 2002-110 111133 CHAPTER 5 Data Center Services Have Improved, but Data Warehousing Needs Remain CHAPTER SUMMARY Unlike its procurement of the Common Management System (CMS) software, the California State University (university) did use recommended procurement practices to select the outsourced data processing services needed to run CMS. The university conveyed its needs to potential vendors, asking them to propose solutions. The university also used an objective selection process with weighted criteria to evaluate potential vendors. Further, the university shared risk with the vendor by establishing contract terms aimed at holding the vendor accountable for meeting preestablished service levels. When it experienced inadequate service from the data center in the early months of the contract, the university used the procedures outlined in the contract to help raise the data center services to agreed levels. The service levels have improved in recent months. The vendor has achieved or come within one percentage point of achieving targets in the five months ending in November 2002. Although the university worked to address its CMS data processing needs and is implementing more efficient means for reporting, it only now is starting to address campus CMS data storage and retrieval (data warehousing) needs. The outsourced data center processes CMS transactions, but is not designed for data warehousing. Data warehousing can provide for optimum data storage and reporting, such as enabling the production of reports that contain historical analysis of university operations. Largely because of concerns over CMS project resources, the university reportedly removed data warehousing from the CMS project scope early in the project and made this important component a campus responsibility. Now, with some campuses expressing an interest in data warehousing services, the university is addressing the data warehousing needs for a voluntary consortium of campuses. 111122 California State Auditor Report 2002-110 California State Auditor Report 2002-110 111133 THE DATA CENTER PROCUREMENT FOLLOWED RECOMMENDED PRACTICES The university followed recommended procurement practices when it procured the data center services. It outlined its needs in its procurement’s request for proposal (RFP), allowing vendors to propose solutions, and then used a methodical and weighted evaluation methodology to determine which vendor would provide the best value. Further, the contract includes terms aimed at holding the vendor accountable, so the vendor shares responsibility and risk with the university. Also, the contract is structured to help ensure that the vendor is held responsible for providing many important functions, such as data security, quality assurance, and internal control. During the initial planning, the university determined that the In February 2000, the data center would be an outsourced endeavor. The university’s university issued its primary roles were to ensure that the data center was planned, request for qualifications procured, and implemented appropriately and to work with the for outsourced data contractor to ensure that the services are provided according to center services. the contract terms. The purpose of the data center procurement was to select a primary contractor for the CMS data processing operations that would provide the physical facilities, personnel, hardware, operating system software, network connectivity, and certain customer services. The university’s procurement formally began in February 2000 when it issued a request for qualifications (RFQ) and culminated in a contract signing in March 2001, 13 months later. The contract, ultimately executed with Unisys for approximately $60 million, covers a five-year period for the Unisys data center in Salt Lake City, Utah. The University Outlined Its Needs and Methodically Evaluated Vendors’ Responses The university followed recommended procurement practices by outlining its business problem and allowing vendors to propose solutions for services. For example, it allowed vendors to propose services at one location or multiple locations, and it encouraged them to propose additional services that could ben- efit overall operations. It used two solicitation documents—an RFQ in February 2000, followed by an RFP in April 2000. The university received four vendor responses in the RFQ process; it qualified two to move on to the RFP process. The procurement’s RFP outlined the university’s needs and allowed vendors to pro- pose solutions. Also, the RFP’s evaluation and selection criteria included noncost criteria such as technical competency, commit- ment and relationship, and service delivery and performance. 111144 California State Auditor Report 2002-110 California State Auditor Report 2002-110 111155 In reaching its procurement decision, the university sought to determine which solution provided the best value. In selecting a vendor, the university used a methodical and weighted evaluation methodology, separately weighting its evaluation criteria. For example, technical competency was assigned 29 percent of the possible points, service delivery and performance 14 percent, cost 29 percent, and so on. The university further detailed each criterion into factors that evaluation team members scored individually on a scale from 0 to 5. It used these team members’ scores and the associated weights to calculate each vendor’s score for each criterion. In August 2000, when the final scores were tallied, IBM emerged as the winning vendor. The university ultimately terminated negotiations with IBM in December 2000 when they could not agree on final scope and cost issues. In Chapter 2, we discuss the costs the univer- sity incurred as a result of the failed IBM negotiations. Because campuses already were scheduled to begin implementing CMS in early 2001, and because Unisys was the only other bidder that participated in the RFP process, the university began negotia- tions and executed a contract with Unisys in March 2001. The University Negotiated a Contract That Shares Risk With the Vendor The Unisys data center contract has terms aimed at holding the vendor accountable for providing the contracted services, allowing the university to share the responsibility and risk for the data center with the vendor. The contract outlines a process the university can use to resolve disputes should concerns about vendor services develop. Also, the contract provides for the assessment of penalties should the vendor fail to meet specified service levels. For example, after a 180-day grace period from the beginning of data center services, contract provisions allow the university to assess penalties on an escalating scale depending on service levels. These penalties generally are not to exceed 5 percent of the monthly base charge for a given month, but Contract provisions they could escalate to 15 percent of the monthly base charge allow the university to if there are consecutive months of failure to meet service assess penalties on an levels. For example, if the first year’s monthly base charge of escalating scale should $1.2 million was assessed a 5 percent penalty, the university the vendor fail to meet would assess a $60,000 penalty on the vendor’s subsequent specified service levels. monthly billing. The monthly base charges are reduced during subsequent years of the contract, reducing the five-percent penalty to $33,000 in the final year. 111144 California State Auditor Report 2002-110 California State Auditor Report 2002-110 111155 The university also structured the contract to help ensure that the vendor is held responsible for providing many important functions, such as data security, quality assurance, and internal control. According to the data center contract, the university pays the vendor the initial start-up fees up front for equipment being installed at the data center and for the services to get the project planned, the center put together, and the university staff trained. However, once the data center is operational, the university pays the vendor a fixed amount monthly for an agreed-upon level of service. The contract terms also hold the vendor responsible for ongoing operational efficiencies and require the vendor to assess the most effective and efficient practices, platforms, and software to provide the university. The contract says the vendor is to identify potential improvements through regular system analyses and perform periodic internal reviews to ensure quality. Thus, the university’s contract helps ensure that the university can hold the vendor accountable for providing the services. THE DATA CENTER HAS RAISED SERVICE LEVELS IN RECENT MONTHS Although it followed recommended procurement practices when procuring data center services, the university has not always received the level of service contractually required of the vendor. The university has not The university has worked with the data center vendor, and always received the level service has improved in recent months, as the data center has of service contractually met required service levels or fallen just short of achieving them. required of the data The university receives service level data from the vendor and center vendor. prepares monthly average data center service level reports, using these reports to assess penalties against the vendor for not meet- ing the service levels stipulated in the contract. The university and the vendor began tracking service levels in October 2001. Figure 8 presents the center’s service level history for October 2001 through November 2002. This figure shows that, although the data center met the contracted service level only once during October 2001 through June 2002, more recent months show improvement. During July 2002 through November 2002, the data center met service levels three out of the five months. The university’s director of hardware operations and support services asserts that after the first four months of production, the university continued to experience many of the data center performance problems on a daily basis that it encountered early in the contract. In particular, according to the director, 111166 California State Auditor Report 2002-110 California State Auditor Report 2002-110 111177 FIGURE 8 Monthly Data Center Service Levels Have Improved Recently ���� ����� ����� ����� ����� ����������� ����� ����� ����� ������������ ����� ����� ����� ����� ����� �� ����� ����� �� �� �� � � ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� �� �� �� �� �� �� �� �� �� �� �� �� �� ��� ��������������� Source: Monthly Unisys reports on service levels. * According to the senior director for the CMS project, November 2002 service levels are preliminary as of January 2003, pending further review by Unisys and the university. the university experienced problems caused by repeated vendor errors, which appeared to be operational errors that should have been simple to correct with adequate management. Some examples of these problems included computer outages and changes the vendor made without notifying campuses. Therefore, in November 2001, the director of hardware operations and support services informed the vendor that the university expected immediate and sustained improvements in upgrading campus services, improving computers and process schedulers, and monitoring production. Also, the university suggested better communication through daily reports and notification to campuses before performing maintenance on the computer system. Shortly thereafter, the university formally 111166 California State Auditor Report 2002-110 California State Auditor Report 2002-110 111177 escalated the ongoing performance issues with the data center When service levels were vendor through the dispute resolution process outlined in deficient, the university the contract. First, the university requested in writing that used the dispute the vendor act to resolve the problems, requiring the vendor resolution provisions to develop a quality assurance program. The next dispute in the contract and resolution step was for the vendor to meet with the university in assessed penalties December 2001 for a status briefing. According to the university, against the vendor. the vendor agreed at the meeting to work with the university to provide timelines for improvements, including developing revised implementation plans and implementing a new-hire training plan that included providing refresher courses to current staff members as appropriate. Service levels remained deficient, however, and the university used the provisions in the contract to assess penalties of $28,500 for substandard January 2002 data center services. It continued to assess penalties ranging from $4,800 to $58,500 for subse- quent months when the data center did not meet service levels. After these concerns over continuing data center issues, the university met with the vendor in March 2002, resulting in the vendor taking action to investigate the issues that month. The next month, the vendor replaced the service delivery manager and presented the university with a “Get Well Plan” that focused on areas for improvement. Shortly afterward in the same month, the university used the escalation procedures again because campuses could not install the software needed to schedule computer processing without the need for constant monitoring by campus staff. Partly in response to the university’s continuing complaints, the vendor added an additional project manager in May 2002 and replaced key personnel, such as the data center director in June 2002 and the account executive in August 2002. In the most recent five months shown previously in Figure 8 (as of November 2002), the data center has achieved the targeted service levels for three months. It missed the targeted level by less than one percentage point for two months. THE UNIVERSITY MUST ADDRESS NEEDS FOR CMS DATA WAREHOUSING In addition to working to address its CMS data processing needs, the chancellor’s office is implementing more efficient management reporting. However, it only now is starting to address campus CMS data warehousing needs. Data warehous- ing can provide for optimum data storage and reporting, such 111188 California State Auditor Report 2002-110 California State Auditor Report 2002-110 111199 as enabling the production of reports that contain historical analysis of university operations. Also, because the chancellor’s office considers data warehousing a campus responsibility, it has not included the costs associated with CMS data warehousing as a central cost of the CMS project. Further, we noted that only a few campuses included CMS data warehousing costs in the June 2002 cost survey. Potential CMS data warehousing costs are unknown now, but they could be significant. The university is implementing more efficient means for reporting CMS information. In December 2002, the director of business management systems told us that some campuses must use the CMS transactions databases and reporting tools at the data center to produce reports. Running reports from these databases can cause CMS transaction processing to slow down. The director added, however, that the data center is replicating the transactions databases to reporting databases, from which reports can be run instead. According to the director, as of December 2002, some campuses had their replicated databases, and the rest will be getting them. These reporting databases are included within the terms of the data center contract and are expected to eliminate the need to use the transactions databases for producing reports. However, the university recognizes that separate data warehous- ing operations could be designed for optimum reporting, and the chancellor’s office is starting to look into data warehous- ing options for some campuses. For example, data warehousing facilitates the cost-effective storage of data for very long periods Originally, the university of time. This data then can be used to conduct various busi- recognized the need ness analyses, such as producing reports that reflect historical for data warehousing, business trends that may not be possible or efficient without but it eliminated data data warehousing. Additionally, data warehousing provides the warehousing from the potential to integrate CMS data with data from other systems project scope early in to produce even more sophisticated analysis and management the project. reports. Originally, the university recognized the need for data warehousing for CMS and included it within the project’s scope, but in April 2002 the university added language to the project charter that removed this component from the scope. The senior director for the CMS project explained that the university eliminated data warehousing from the project scope early in the project and believes the university made it a campus responsibility largely because of concerns about CMS project resources. 111188 California State Auditor Report 2002-110 California State Auditor Report 2002-110 111199 The chancellor’s office is looking into data warehousing solutions because some campuses that do not have their own data warehousing operations expressed an interest for these services to provide them with reporting options that are not being met by the CMS data center. Although the current university position is that campuses are responsible for CMS data warehousing, the chancellor’s office is working, on a fee-for-service basis, with a voluntary consortium of campuses to address data warehousing needs. The director of business management services says the chancellor’s office is working on a pilot project currently funded by the chancellor’s office to design and develop a data warehousing model for campuses. Because campuses expressed an interest in these services, the chancellor’s office is asking each of the eight campuses currently participating in the pilot project to explain the reasons each campus needs these services. By early 2003, the chancellor’s office plans to estimate the fees to charge campuses that decide to use this model. The chancellor’s office also expects to have released its final version of the data warehousing model in early 2003. RECOMMENDATIONS To ensure that it continues to receive improved service levels from the data center vendor, the university should continue to monitor the data center services and promptly take action to resolve problems with the vendor, including problems in meet- ing service levels or providing and maintaining appropriate and sufficient facilities, equipment, and staff throughout the life of the contract. To ensure that it provides campuses with the means to effec- tively and efficiently store and retrieve data needed for useful management reporting, the university should expedite the CMS data warehousing project and include the CMS-related costs of campus data warehousing in its CMS project costs. n 112200 California State Auditor Report 2002-110 California State Auditor Report 2002-110 112211 CHAPTER 6 The University’s Oversight Over Potential Conflicts of Interest Needs Improvement CHAPTER SUMMARY In addition to its problems with procurements on the Common Management System (CMS) project discussed earlier in the report, the California State University (university) did not do enough to detect or prevent conflicts of interest by decision makers for CMS-related procurements. The university did not identify all necessary employee positions in its conflict-of-interest code as designated positions required to file annual statement of economic interests forms (Form 700s) and did not always retain and make available certain required filings of these forms. Additionally, the university did not require consultants on the project to file Form 700s, although they performed duties similar to employees in designated positions. Further, the university failed to provide for adequate disclosure processes to help ensure that individuals participating in the procurement process were free from conflicts. Also, the university did not provide appropriate guidance to employees to identify potential conflicts using the Fair Political Practices Commission (FPPC) process for determining conflicts. Finally, the university lacks a policy that spells out for employees what constitutes “incompatible activities,” such as accepting anything of value from anyone seeking to do business with the university, and does not require that employees in designated positions receive regular ethics training. Our review of CMS-related procurements found an employee who appeared to have a conflict of interest while participating in a procurement decision and an employee who possibly may have used nonpublic information to benefit personally. Conflicts of interest cast a shadow over the university’s reputation for fair and honest business practices and undermine public confidence in the university’s procurement decisions. Moreover, if an employee uses information not available to the general public for personal financial gain, it not only harms the university’s reputation but also is unlawful. 112200 California State Auditor Report 2002-110 California State Auditor Report 2002-110 112211 THE UNIVERSITY’S PROCESS FOR PREVENTING AND DETECTING CONFLICTS OF INTEREST WAS LACKING IN SEVERAL AREAS The university’s conflict-of-interest code left out some university positions that should have been designated for filing annual Form 700s. The university fails to comply with state statutes by not maintaining its Form 700s adequately. It also hinders Form 700s are used by the public’s ability to assess possible conflicts of interest by not certain state and local providing copies of certain individuals’ annual Form 700s. government officials and employees annually to The FPPC provides annual Form 700s for certain state and local publicly disclose their government officials and employees to publicly disclose certain economic interests. personal assets and income as required by the Political Reform Act of 1974 (Political Reform Act), codified in the California Government Code, sections 81000 through 91015. The Political Reform Act also requires every government agency to adopt a conflict-of-interest code. The code must list each position within an agency that makes or participates in the making of governmental decisions that may foreseeably have a material financial effect, as defined by state regulations, on the economic interests of the person filling that position. Individuals employed in these positions are called “designated employees” or “code filers.” Certain consultants to public agencies also qualify as “code filers” because they make or participate in making governmental decisions on the agency’s behalf. Although the university maintains its conflict-of-interest code centrally, it maintains the filing of annual Form 700s separately at the chancellor’s office and each campus. The University Did Not Include Some CMS Management Positions in Its Conflict-of-Interest Code and Did Not Retain All Form 700s The university did not include certain positions within its conflict-of-interest code, even though those positions required governmental decision making. These university employees are responsible for regularly overseeing and evaluating vendors and consultants. Therefore, filing annual Form 700s is important to ensure the disclosure of any potential conflicts of interest. The Political Reform Act requires the university to submit to the FPPC amendments to its code within 90 days after creating a new position that must be designated to file an annual Form 700. However, at the time of our review in summer 2002, the university had not submitted amendments for two chancellor’s office management positions on the CMS project 112222 California State Auditor Report 2002-110 California State Auditor Report 2002-110 112233 and certain CMS project directors at the campus level, even though the 90-day time requirement had passed. In fact, the university created most of the positions in 1999 and 2000. The associate director of human resource services for the chancellor’s office told us that the office requested a Form 700 for one management position in March 2002 because it became apparent after the approval of the university’s conflict-of- interest code that the position warranted being designated in the code. However, that employee did not submit the form by the due date of April 2002, and the university did not collect the form until six months later in October 2002. As a result of our inquiries, the chancellor’s office requested forms from this and the other management employee whose position had not been designated and furnished them to us. Also, the chancellor’s office added these positions to the amendments to its conflict- of-interest code that it submitted to the FPPC in October 2002. Additionally, each campus is responsible for designating its own positions for the university’s conflict-of-interest code. Campus CMS project directors are responsible for making or participating in making decisions that may foreseeably have a material financial effect on their economic interests and therefore should have been designated in the university’s code. For example, Some campus CMS the CMS project director’s duties and responsibilities include project directors with developing implementation plans, managing the project budget, significant decision- making recommendations regarding policies, negotiating with making authority vendors, setting standards and service levels, setting priorities, were not designated and hiring and supervising staff. In addition, the salaries for to file Form 700s. these positions indicate significant decision-making authority: four campuses told us their CMS project director salaries ranged from $7,700 to $10,400 per month. Campus responses as to why the project director positions were not designated in the conflict-of-interest code ranged from saying they overlooked the position and would be sure to designate it in the future, to asserting that the position does not entail purchasing decisions so they do not plan to designate it. However, as stated earlier, project directors are clearly involved with making or participating in making decisions that may foreseeably have a material effect on their economic interests, including purchasing decisions. Furthermore, the Political Reform Act requires the university to retain original filings of annual Form 700s for seven years, but the chancellor’s office and campuses could not locate these forms in a number of cases. Of the 244 forms we requested for 112222 California State Auditor Report 2002-110 California State Auditor Report 2002-110 112233 various years on 63 university employees, the chancellor’s office and campuses were unable to provide 39 forms because they could not locate them or because they failed to require certain individuals to complete them. In response to our initial request, the human resource services director for the chancellor’s office stated that he found upon assuming the position in March 2000 that the records kept by previous chancellor’s office personnel The university could directors were not complete for the full seven-year period that not produce 39 of they were required to be retained. In fact, many required forms the 244 Form 700s for calendar year 1999 and previous years were unavailable at that we requested. the chancellor’s office until the filing officer was able to locate them more than four months after our initial request. Even after the chancellor’s office later found these additional forms, some still could not be located. At the San Jose campus, the assistant to the associate vice president of human resources told us that staff could not locate any forms from 1995 to 2000. This indi- vidual stated that the staff member responsible for the annual statements during those years was no longer with the depart- ment, and the records were incomplete. The University Also Did Not Require CMS Consultants to File Form 700s In addition to omitting the CMS managers and project directors, the university did not require consultants who participated on the CMS project to file Form 700s. For example, the university used consultants to help it evaluate responses on the CMS data center procurement. The university’s director of contract services and procurement asserts that Form 700s were not required of these consultants because the university believes the consultants possessed no authority with respect to the award decision beyond providing information and advice to industry and technical expertise. The director of contract services and procurement stated that consultants are not required to file a Form 700 if the consultant prepares a product or performs services for a single specific matter versus performing ongoing services, or if there is significant intervening substantive review of the consultant’s recommendations before submission to the final decision maker. Although not all consultants are required to file Form 700s, the California Code of Regulations, Title 2, Section 18701, requires consultants who serve in a staff capacity with an agency, and in that capacity participate in making governmental decisions, to file Form 700s. According to these regulations, this participation includes advising or making recommendations 112244 California State Auditor Report 2002-110 California State Auditor Report 2002-110 112255 to the decision maker, either directly or without significant intervening substantive review, by methods that require the The university should have exercise of judgment and the purpose of which is to influence required consultants who a governmental decision. The regulations list the following served in a staff capacity methods: conducting research; making any investigation; or to file Form 700s. preparing or presenting any report, analysis, or opinion either orally or in writing. Finally, consultants should file Form 700s if they perform the same or substantially the same duties that would otherwise be performed by individuals holding a designated position within the university’s conflict-of-interest code. One consultant from Aligne, a firm with which the university contracted, worked to provide management services for the data center procurement on an ongoing basis and appears to have participated in making governmental decisions without signifi- cant intervening substantive review, but was not required to file a Form 700. This consultant provided project management services for about nine months for this procurement effort. The consultant’s duties and responsibilities included project manage- ment, reviewing and validating evaluation and selection criteria, participating as a member of the evaluation team, responding to vendor questions, developing the draft final evaluation report, and leading the negotiation team. Paid approximately $1,190 to $2,500 per day, the consultant’s duties were similar to those of positions that are designated positions within the university’s conflict-of-interest code. In addition, the university hired the principal consultant from Io Consulting as a project manager in 1999 and did not require him to file a Form 700. The university’s director of contract services and procurement stated that a Form 700 was not required because consultants from Io Consulting possessed no authority with respect to any award decision beyond providing information and advice related to industry and technical expertise. However, the principal consultant for this firm was given significant responsibility for the management of the project at that time. According to the contract, this consultant functioned as a project manager, assuming responsibility during part of the project for the management of all tasks necessary for the design, development, testing, and preparation of the human resources software. This consultant was responsible for managing and directing contract and university staff and had the duty of communicating and presenting the project’s status to relevant parties and groups, such as the senior director for the CMS project and the steering committee. Chapter 4 discusses the selection and the role of the Io Consulting firm in the CMS project. 112244 California State Auditor Report 2002-110 California State Auditor Report 2002-110 112255 Its Procurement Disclosure Forms Are Inadequate, and the University Could Not Provide These Forms for Many of the CMS Software Procurement Participants In addition to mishandling annual Form 700s, the university could not provide signed disclosure forms for many participants in the CMS software procurement. Individuals signing disclosure forms are certifying that they have no personal or financial interests incompatible with their participation in By not ensuring that the procurement process. Moreover, these disclosure forms procurement participants are themselves inadequate to allow participants to assess their sign adequate conflict-of- potential conflicts of interest. Thus, the university could not interest disclosure forms, demonstrate that all participants had assessed adequately the university leaves itself whether their personal interests were compatible with the open to criticism. procurement process and that these participants had been reminded of their duty to keep aspects of the procurement process confidential. By not ensuring that procurement participants sign adequate disclosure forms, the university leaves itself open to criticism that bidders did not get equal treatment and their information was not kept confidential. Disclosure forms the university used for the procurements we reviewed did not provide an effective conflict-of-interest disclosure process for individuals participating in the procurements. Although the university is not required by policy or statute to require such forms, its policy manual for contracting and procurement requires it to have procedures in place to ensure fairness and uniform treatment to all bidders and ensure that confidentiality of certain information received from bidders is maintained. Therefore, the university’s practice is to require individuals, before they participate in confidential procurement activities, to sign these forms, which address two main areas: conflicts of interest and confidentiality. Individuals signing the forms are certifying that they have no personal or financial interest incompatible with their participation in the procurement process, and that they will hold in strictest confidence certain information regarding vendors’ responses to the procurement. However, the disclosure forms we were able to review for selected procurements for the CMS project did not clearly indicate what constitutes a conflict of interest or even list potential bidders so procurement participants could assess their potential for conflicts of interest. Further, the university could not provide signed disclosure forms for 30 of the 94 individuals whom we were able to identify as participating in one or more phases of the CMS software procurement process. Therefore, the university could not demonstrate that all participants signed these forms. 112266 California State Auditor Report 2002-110 California State Auditor Report 2002-110 112277 The University Has No Employee Policy on Incompatible Activities and No Requirement That Employees Receive Regular Ethics Training Further, the university lacks a policy that formally communicates incompatible activities to employees and lacks a requirement for Without a published ethics training. Two examples of incompatible activities include incompatible activities employees using confidential information for private gain or policy, university advantage, or accepting anything of value from anyone seeking employees do not have to do business with the university. When the university does a basis to reasonably not take prudent steps to inform employees of incompatible know what is considered activities through widely published policy and ethics training, an incompatible activity. it leaves itself vulnerable to inappropriate employee actions. Although the university may make some efforts to communicate incompatible activities, without a published policy that communicates these statutory and constitutional requirements, university employees do not have a basis to reasonably know what is considered an incompatible activity. The deputy general counsel told us the university does not have an incompatible activities policy, similar to that addressed in Government Code, Section 19990, for state employees, because statutes and the state constitution spell out the incompatible activities, and current statutes no longer require the university to have such a policy. The university explained that it takes other steps to ensure that employees know their responsibilities. The assistant vice chancellor for human resources administration asserts that through orientation, supervision, and training the university advises employees that they are expected to maintain high professional standards in meeting their assigned responsibilities and that they are not to engage in activities that are incompatible with their university responsibilities. Further, the university believes its efforts to coordinate and communicate its conflict-of-interest code address incompatible activities issues. Finally, the assistant vice chancellor asserts that negotiating one incompatible activities policy would be difficult, if not impossible, because the university believes it would have to negotiate such a policy for the 10 different collective bargaining agreements that cover the majority of its employees. However, if the university had a policy for activities that are addressed by statute or the state constitution, we question how they would be negotiable issues during the collective bargaining process. The university also does not require its employees to receive ethics training regularly. Although Government Code, sections 11146 through 11146.4, require certain state officials to complete an ethics training course every two years, the 112266 California State Auditor Report 2002-110 California State Auditor Report 2002-110 112277 university’s deputy general counsel asserts the university is exempt from this requirement. The deputy general counsel cites the California Education Code, Section 66606.2, which states The university asserts the Legislature intends that the university not be governed by it is exempt from state any statute enacted after January 1, 1997, that does not amend requirements that a previously applicable act, unless the statute expressly provides employees receive that the university is to be governed by that statute. The ethics training on a biennial ethics training requirement was part of a 1998 statute regular basis. that did not specifically include the university. Nevertheless, recently enacted state law, Chapter 663, Statutes of 2002, which became effective January 1, 2003, demonstrates the Legislature’s commitment to ethics training by expanding these training requirements to include all employees of a state agency who are required to file statements of economic interest pursuant to the Political Reform Act. The deputy general counsel asserts that the university is exempt from this requirement as well. However, it would be prudent for the university to hold such training regardless of whether it is required by statute. Although some university training for employees may address “ethics,” the university could not demonstrate that it provided a focused ethics training process. When we asked the associate director for human resource services for the chancellor’s office about its practices to provide ethics training, she responded that the university offers several training modules that incorporate elements of “ethics,” such as dealing with conflict and violence in the workplace, sexual harassment prevention, and training for managers in employment practices. In addition, the director of contract services and procurement said that, although her department does not specifically provide ethics classes for its employees, ethics and public stewardship concepts are covered in certain sections of the training they do receive. Although the university may provide training that touches upon aspects of ethics, it does not provide employees with training that is focused on ethics such as that addressed in Government Code, sections 11146.1 through 11146.4. When the university is lax in its conflict-of-interest policies and practices, actual or seeming conflicts of interest may cast doubt over its procurement decisions. Also, it leaves itself vulnerable to improper employee actions when it does not take prudent steps to inform employees of incompatible activities through widely published policy and ethics training. In the following section, we discuss where our review of Form 700s indicated that an apparent conflict of interest existed with a university employee 112288 California State Auditor Report 2002-110 California State Auditor Report 2002-110 112299 who participated in a CMS procurement. We also discuss where an employee possibly may have used nonpublic information in an attempt to benefi t personally. FORM 700s INDICATE A UNIVERSITY EMPLOYEE HAD AN APPARENT CONFLICT OF INTEREST AND ANOTHER MAY HAVE TRIED TO BENEFIT PERSONALLY FROM NONPUBLIC INFORMATION Our review found a university employee who participated in a CMS procurement whose Form 700s indicate an apparent confl ict of interest, receiving income from a fi rm competing for and ultimately awarded a contract. Another university employee may have used information not available to the general public with the possible intent to benefi t fi nancially. The Political Reform Act generally holds individuals who violate the act liable rather than their employers. Nonetheless, with better practices in place, the university would have been better positioned to detect and perhaps avert the kinds of apparent confl icts identifi ed. Confl icts of interest undermine public confi dence in the university’s procurement decisions, opening the door for criticism of improper behavior and tarnishing the university’s reputation for fair and FPPC’s Eight-Step Analysis to Determine Whether an Individual Has a honest business practices. Disqualifying Confl ict of Interest Under the Political Reform Act Determining what constitutes a confl ict of interest and is therefore a violation of the Political 1. Determine whether the individual is a public offi cial. Reform Act is generally the purview of the FPPC. The California Code of Regulations, Title 2, 2. Determine whether the offi cial is participating in or attempting to infl uence Section 18700, provides a basic rule for what a governmental decision. constitutes a confl ict of interest under the Political Reform Act and outlines an eight-step analysis 3. Identify the public offi cial’s economic interests. (see textbox) that the FPPC uses to determine if a confl ict of interest exists. The basic rule states 4. For each economic interest, determine whether that interest is directly or that public offi cials may not make, participate in indirectly involved in the governmental making, or use or attempt to use their position decision. to infl uence a governmental decision in which 5. Determine if the effect is material. they know, or have reason to know, that the 6. Determine if the effect is reasonably decision will have a reasonably foreseeable material foreseeable. fi nancial effect on their economic interests. Public offi cials include employees and consultants of the 7. Determine if the effect is distinguishable from the effect on the public generally. university. Economic interests are defi ned with certain thresholds. For example, an economic 8. Determine if the offi cial’s participation is legally required. interest in a business entity exists if an investment is $2,000 or more; an economic interest exists regarding a source of income if it totals $500 or 112288 California State Auditor Report 2002-110 California State Auditor Report 2002-110 112299 more within the 12 months before the governmental decision. If a conflict exists, public officials must disqualify themselves from participation in the decision. According to state regulations, the university’s responsibility for reviewing potential conflicts disclosed on Form 700s includes It would be prudent for conducting what the FPPC calls a “facial” review of each the university to have a statement and conducting a full review of at least 20 percent process in place to resolve of the statements. Facial reviews include verifying that the potential conflicts. forms are complete. A full review includes additional steps such as reviewing attached schedules for required descriptive information and checking for consistency between schedules. Although state regulations do not require the university to use the FPPC’s eight-step process for testing potential conflicts, the university would be prudent to have a process to resolve identified potential conflicts. For example, the attorney general publishes guidance for government officials in complying with conflict-of-interest requirements. That guidance recommends that employees use the eight-step process and seek advice from in-house counsel when the potential for a conflict is recognized. Thus, the university should provide employees guidance on using the eight- step process and their need to seek the advice of counsel. A University Senior Executive Who Participated in the CMS Software Procurement Received Income From the Selected Vendor Although we recognize that the FPPC is the determining entity, our review of Form 700s identified financial interests that appeared to create a conflict of interest. A university senior executive participated in the CMS software procurement process although he reported on his Form 700 that he received income from the vendor the university selected. The CMS software procurement took place between April 1997 and September 1998; this senior executive received income from the bidder for facilitating meetings periodically from 1996 through 1998. For 1996, the senior executive reported more than $10,000 of consulting fees and between $1,001 and $10,000 in travel reimbursements from this vendor, and between $1,001 and $10,000 for meeting facilitation for both 1997 and 1998 in addition to travel payments of $4,400 for 1997 and $2,900 for 1998. The software procurement ultimately resulted in a $33 million software licensing, consulting, and training contract with the vendor, ultimately $37 million after amendments, and a subsequent agreement for additional consulting services that, as of June 2002, was expected to total $12 million. 113300 California State Auditor Report 2002-110 California State Auditor Report 2002-110 113311 The senior executive asserted that his role in the CMS procure- ment process did not include the selection of the vendor because he absented himself from every meeting in which decisions were made about the vendor or its competitors, including the final meeting during which the vendor was recommended. Although the senior executive may not have participated in this final meeting, we did find that he participated in key activities related to the vendor’s selection. For example, the senior executive participated in site visits to evaluate the quality of the potential vendors’ products and services and was on the final contract negotiations team. The senior executive asserts that his role in the site visits was as an observer, note taker, and staff support person, just as on all the integrated technology strategy initiatives and projects. The senior executive’s However, this minimization of his role to clerical or analyst assertion that he had support does not reconcile with the senior executive’s significant a minimal role in the responsibility for leading and managing complex multicampus procurement does information technology projects. As evidenced by the more than not reconcile with his $11,000 per month compensation received during the period of significant responsibility. the CMS software procurement, the senior executive presumably was being paid to make or participate in making decisions affecting the Integrated Technology Strategy plan and its CMS component. A description of the role and scope of the senior executive’s responsibilities during that time indicates he was responsible for managing and executing information resources and technology projects, fostering and brokering partnerships and alliances within the university and with external public and private organizations, and representing the university with various external organizations relative to the university’s integrated technology strategy. Although he did not perform similar services for any other vendor competing for the university’s CMS software business, the senior executive states that his supervisor at the time encouraged him to accept the vendor’s invitation in late 1995 because of his proven skills as a facilitator and because the position was another “positive” to the university’s growing national reputation as a leading institution. The senior executive’s agreement to facilitate the vendor’s higher education meetings may have added some value to the university; nevertheless, once the senior executive recognized the vendor had a serious interest in conducting business with the university, he was obligated to determine whether he had a potential conflict of interest before participating in any phase of the CMS software procurement process. The senior executive could have 113300 California State Auditor Report 2002-110 California State Auditor Report 2002-110 113311 determined whether a potential conflict existed had he used the FPPC eight-step process, sought advice from university counsel, or contacted the FPPC for a determination. While reviewing the 1998 Form 700s subsequent to the CMS software procurement, the former director of human resources for the chancellor’s office became aware of the potential for a conflict of interest and questioned the senior executive. However, the former director did not use the FPPC eight-step process and did not refer the individual to university counsel to establish whether a conflict of interest existed. As discussed previously, the university is not required to use the eight- step process. However, we used the eight-step process and found that a conflict of interest appeared to exist because the senior executive received, within 12 months before the senior executive’s participation in the procurement, more than $500 in income from a vendor competing for the university’s CMS software contract. A University Senior Executive May Have Tried to Use Nonpublic Information for Financial Benefit Finally, even though another senior executive apparently was not involved directly in the procurement process, this individual may have used nonpublic information regarding the university’s contracting for personal computers to try to benefit personally. Using nonpublic information in this manner would be a violation of university policy and Education A university senior Code, Section 89006, which prohibits employees from using executive reportedly nonpublic information to benefit personally. The senior purchased between executive reportedly purchased stock in a company one day $10,001 to $100,000 of before the university executed the contract with that company. a computer company’s According to the individual’s Form 700, the senior executive stock one day before the purchased between $10,001 and $100,000 of the stock on university signed February 24, 1999, in a company to which the university a major contract with awarded a major contract for computer products and services that company. the following day. This contract was a master agreement for the chancellor’s office and campuses to order computers and servers. Although the master agreement did not specify a total amount to be spent, the university’s request for proposal addressed the potential significance of the agreement by noting that the university operated at the time approximately 30,000 laptop computers, desktop computers, and servers with an estimated useful life of three years. 113322 California State Auditor Report 2002-110 California State Auditor Report 2002-110 113333 The senior executive asserts that her spouse purchased the stock, and that she cannot recall whether she knew the university even had a process under way to establish systemwide agreements with computer vendors, much less to award an agreement to the vendor in question. Although the senior executive cannot recall, meeting minutes indicate that the senior executive attended a mid-February 1999 meeting where the “signing” of a contract to acquire computer hardware at “very attractive prices” was discussed. Although the discussion was of a vendor that ultimately did not receive the contract, as we discuss further below, the senior executive was in a position to reasonably be aware that the university was involved in a procurement to establish systemwide agreements with computer vendors. Further, the senior executive has a high-ranking management position whose responsibilities included overseeing information technology services that support the chancellor’s office. Thus, we find it at least questionable that the senior executive The university advised was unaware of the university’s final decision to enter into the senior executive to the contract. refrain from participating in future decisions Interestingly, the public was most likely not apprised of the regarding this company; impending contract because the university’s notification yet the possible use of of intent to award regarding this procurement indicated a nonpublic information different vendor. The company that was ultimately awarded this was not discussed. contract was a secondary finalist, so it was not mentioned on the university’s notice of intent to award. Pending negotiation of a final agreement, the university issued a notification of intent to award on December 14, 1998, to the initial winning bidder. Subsequently, the university awarded the contract to this secondary finalist on February 25, 1999, one day after the senior executive (or spouse) had reportedly purchased between $10,001 and $100,000 of the second finalist’s stock. University documentation indicates that the human resources director at the time met with the individual in June 1999 to discuss the stock holdings identified on the individual’s Form 700. The documentation shows the former human resources director advised the senior executive to refrain from participating in any future decisions regarding this vendor, including making recommendations. However, the documentation failed to address the senior executive’s possible use of information not available to the public to benefit personally. The individual who was the human resources director at the time told us that he was not aware of the stock purchase timing until we brought it to his attention. 113322 California State Auditor Report 2002-110 California State Auditor Report 2002-110 113333 Apparent conflicts and incompatible activities such as these undermine confidence in the university’s procurement decisions, leaving the university vulnerable to allegations of improper decisions. Also, if an employee uses information not available to the general public for personal financial gain, it is not only unlawful, but it also may harm the university’s reputation for fair and honest business practices. RECOMMENDATIONS The Legislature should consider the following to ensure that the university takes appropriate action to prevent potential conflicts of interest in the future: • Require the university to provide designated university employees with periodic ethics training similar to that required for designated state employees by Government Code, sections 11146 through 11146.4. • Require the university to establish an incompatible activities policy for university employees similar to that addressed in Government Code, Section 19990, for state employees. To ensure that it adequately addresses potential conflicts of interest and prohibited use of nonpublic information, the university should: • Conduct periodic conflict-of-interest training, such as the ethics training required of state agencies, for designated employees to inform those in decision-making positions about required disqualification when potential conflicts of interest exist. This training should provide employees guid- ance on using the FPPC’s eight-step process as well as their responsibility to seek the advice of counsel. • Establish an incompatible activities policy that it formally communicates to employees. • Require all employees to sign disclosure statements before participating in the procurement process. • Enhance its disclosure form to indicate clearly what constitutes a conflict of interest, list all vendors participating in the procurement as they are known, and state that evaluators are prohibited from using nonpublic information to benefit personally. 113344 California State Auditor Report 2002-110 California State Auditor Report 2002-110 113355 • Update its conflict-of-interest code to classify all positions responsible for evaluating or overseeing vendors or contractors. • Require consultants that serve in a staff capacity and that participate or influence university decisions to file Form 700s. • Remind human resources personnel of their responsibility to collect, retain, and make available the filed Form 700s for the required seven-year period. • Remind employees of the prohibition against using informa- tion not available to the public to benefit financially, and discipline infractions if necessary. We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, ELAINE M. HOWLE State Auditor Date: March 11, 2003 Staff: Karen L. McKenna, CPA, Audit Principal Robert Cabral, CPA, CIA Grant Parks Randy Russell Leonard Van Ryn Loretta T. Wright 113344 California State Auditor Report 2002-110 California State Auditor Report 2002-110 113355 Blank page inserted for reproduction purposes only. 113366 California State Auditor Report 2002-110 California State Auditor Report 2002-110 113377 APPENDIX A Chancellor’s Office Positions and Workload Will Increase, but It Is Too Early to Determine the Impact of Project Implementation on Campus Positions and Workload The California State University’s (university) approach to implementing the Common Management System (CMS) depends heavily on consultants during the project’s initial stages. However, the university plans to replace most of these consultants with its own employees once they are trained adequately on CMS operations. As of December 2002, the university was still in the early stages of CMS implementation, with CMS being used at five campuses, 14 implementing the software, and four yet to begin the implementation process. As a result, it is too early to judge the impact of CMS implemen- tation on the number of employee positions on the campuses. Nevertheless, employee full-time-equivalent (FTE) data obtained during the audit suggests the chancellor’s office will experience a permanent increase in FTEs. The 11 “first wave” campuses to implement the new software also have experienced some FTE growth attributable to CMS. Given that the CMS project is still incomplete, it remains unclear whether the campus FTE increases represent permanent positions. Finally, it is too early to assess CMS’s impact on employee workload, given the current status of this project and the university’s lack of preestablished targets for its post-CMS information technology (IT) and non-IT staffing. As discussed in Chapter 1, the university did not estab- lish quantitative measures of business process efficiencies. As a result, it has not attempted to monitor changes in employee workload attributable to CMS. The Chancellor’s Office Likely Will Have Permanent Increases in Employee Positions Related to CMS The Bureau of State Audits (bureau) surveyed the chancellor’s office to obtain FTE totals (from 1997 through 2002) for IT and non-IT designated employees. 113366 California State Auditor Report 2002-110 California State Auditor Report 2002-110 113377 FIGURE A.1 CMS FTE Growth at the Chancellor’s Office ��� ��� ��� �� ��� �� �� �� �� �� �� �� �� �� �� � � � � ��� ��� ��� ��� ��� ��� ���� ���� ���� ���� ���� ���� ���� As Figure A.1 demonstrates, the chancellor’s office has realized FTE growth in CMS-related functions in both IT and non- IT designated positions. Between October 1997 and 2002, 119 FTE positions were added at the chancellor’s office in CMS-related functions—76 in IT positions such as functional analysts and 43 FTEs in non-IT positions such as managers and administrators. Between October 1997 and October 2002, total chancellor’s office FTE increased by 138 positions. Of this total increase, 119 related to finance, human resources, student administration, and general CMS functions. This latter category accounted for 98 positions. According to its June 2002 cost survey, the chancellor’s office expects to spend $14 million on personnel for maintaining CMS during fiscal year 2006–07, the final year of implementation. According to the chief of staff for information technology services, this $14 million relates to 110 employees expected to be working on CMS that year. 113388 California State Auditor Report 2002-110 California State Auditor Report 2002-110 113399 ��� ������ �� ������������������ �������������������� �������������������� ��������������������������� Source: The university’s June 2002 cost survey. Note: The FTE data presented above are based only on work functions that may include individuals working on the CMS project. These include human resources, finance, student administration, and general CMS work functions only. As Figure A.1 demonstrates, FTE growth at the chancellor’s office has been the most pronounced in IT designated positions, espe- cially beginning in October 2000, a few months before campuses started implementing the software. IT personnel hired during this time included CMS function analysts, technical analysts, and help desk personnel. The significant growth in IT designated positions at the chancellor’s office around October 2000 seems consistent with its support of campus CMS implementation. In their role under CMS, chancellor’s office personnel are respon- sible for the acquisition, development, and final distribution of the CMS software to the campuses. The CMS software request for qualifications was issued in April 1997, and the software was procured in September 1998. Since early 1999, the chancellor’s office has been modifying the software so it will meet the cam- puses’ collective needs more closely. Today, as more campuses begin the implementation process, chancellor’s office staff continue to work on the CMS software. Non-IT personnel hired at the chancellor’s office between October 1997 and 2002 were placed in various administrative and managerial positions that support the CMS project. The FTE data provided in Figure A.1 relates only to employee positions likely to be affected by CMS, namely finance, human resources, student administration, and general CMS positions. Although our survey obtained FTE data for other positions, such as data center and managerial staff, the survey’s instructions directed the chancellor’s office to report all CMS-related FTE information in the work categories used in Figure A.1. It Is Too Early to Assess CMS’s Impact on Employee Positions at Campuses Given that the CMS implementation effort is still in its early stages, assessing the impact of this new software program on employee positions is likely premature. Also, the permanency of any new positions is unclear given that the project is still incomplete. Nevertheless, to respond to the Joint Legislative Audit Committee’s request for this assessment, the bureau surveyed all 23 campuses to obtain FTE totals for IT and non-IT designated employees. The FTE data obtained through the survey was provided as of October of each year, covering 1997 through 2002. The results of the FTE survey for the 11 “first wave” campuses to implement CMS suggests these campuses have increased their IT and non-IT FTEs because of the CMS project. 113388 California State Auditor Report 2002-110 California State Auditor Report 2002-110 113399 FIGURE A.2 CMS FTE Growth at “First Wave” Campuses* ����� ����� ����� ����� ����� ����� ����� ����� ����� ��� �� �� ��� ��� ��� ��� � ��� ��� ��� ��� ��� ��� ���� ���� ���� ���� ���� ���� ���� Source: The university’s June 2002 cost survey. Note: The FTE data presented above are based only on work functions that may include individuals working on the CMS project. These include human resources, finance, student administration, and general CMS work functions only. * First 11 campuses implementing CMS. The FTE totals presented in Figure A.2 represent IT and non-IT designated positions only in work classifications likely to be affected by CMS during the period, namely in human resources, finance, student administration, and general CMS positions. As Figure A.2 demonstrates, non-IT designated employees increased by 254 FTEs, or about 17 percent between October 1997 and 2002. During this same period, IT designated employees also increased by 60 FTEs, or about 74 percent. It appears that most of these FTE increases can be attributed to CMS. For selected FTE increases during this period, we asked the first wave campuses of Sonoma, Fresno, and Long Beach to explain selected FTE increases and indicate whether those increases were due to CMS. Of the 78 FTE increases that we 114400 California State Auditor Report 2002-110 California State Auditor Report 2002-110 114411 ��� ������ �� ������������������ �������������������� �������������������� ��������������������������� asked these campuses to explain, 33 were attributed to the CMS project, with 21 in IT designated positions and 12 in non-IT positions such as managers and administrators. In instances in which campuses reported FTE increases unrelated to CMS, they cited needs for increased support at financial aid offices, student records, and enrollment services as the reasons for hiring more personnel. These campuses also reported adding some IT-designated employees as telecommunications and network analysts for reasons unrelated to the CMS project. The University Does Not Track Changes in Employee Workload Attributable to CMS As discussed in Chapter 1, the university has elected to measure CMS’s benefits qualitatively by surveying university employees about their level of satisfaction with the new software. Although the university hopes to achieve improved employee efficiency through CMS, it has not developed a means to monitor or record changes in employee workload attributable to the project. During our audit we asked the chancellor’s office and the three campuses we visited about their efforts to monitor and track employee workload changes attributable to CMS. The Long Beach campus did not respond to our question, and the chancellor’s office and the Sonoma campus indicated they do not track employee workload. The Fresno campus said it was too early to assess employee workload; however, it plans to measure workload in fiscal year 2002–03 when the effects of implementation subside. 114400 California State Auditor Report 2002-110 California State Auditor Report 2002-110 114411 Blank page inserted for reproduction purposes only. 114422 California State Auditor Report 2002-110 California State Auditor Report 2002-110 114433 APPENDIX B Projected Total Investment and Maintenance and Operations Costs of the Common Management System Project by the Chancellor’s Office and Campuses Table B.1 on the following page shows two broad cost areas—the projected total investment costs and maintenance and operations costs—for the chancellor’s office central costs and each campus. The source of the cost data is the California State University’s (university) June 2002 cost survey. First, the table shows the total one-time investment costs of the Common Management System (CMS) project covering the nine-year project period. This includes the costs for the project areas of implementation, integration, in-kind, and upgrades as discussed in Chapter 2. Next, the table shows the maintenance and operations costs, or the total of the recurring costs of the CMS project during the nine-year period. As discussed further in Chapter 2, the amount of maintenance and operations costs a campus reflects is influenced by how far along campuses are in the implementation process. 114422 California State Auditor Report 2002-110 California State Auditor Report 2002-110 114433 TABLE B.1 Projected Total Central Costs and Campus Costs for Fiscal Years 1998–99 Through 2006–07 Projected Total Projected Total Maintenance and Projected Total Investment Costs Operations Costs Costs Central* $ 83,855,717 $152,560,212 $236,415,929 Campus Bakersfield 6,611,545 1,852,256 8,463,801 Chancellor’s Office† 3,321,300 3,391,935 6,713,235 Channel Islands 514,880 346,787 861,667 Chico 14,815,073 3,631,067 18,446,140 Dominguez Hills 9,710,000 4,119,800 13,829,800 Fresno 10,656,526 7,725,600 18,382,126 Fullerton 17,978,070 1,345,563 19,323,633 Hayward 12,303,668 6,191,958 18,495,626 Humboldt 9,235,829 2,407,035 11,642,864 Long Beach 25,448,554 11,659,574 37,108,128 Los Angeles 8,769,400 4,651,100 13,420,500 Maritime Academy 1,303,614 3,334,571 4,638,185 Monterey Bay 11,984,613 3,399,024 15,383,637 Northridge 13,717,868 4,500,961 18,218,829 Pomona 16,154,608 4,082,831 20,237,439 Sacramento 16,469,300 2,803,200 19,272,500 San Bernardino 14,869,265 5,039,719 19,908,984 San Diego 34,661,458 2,934,900 37,596,358 San Francisco 21,844,500 11,437,500 33,282,000 San Jose 8,419,218 7,044,984 15,464,202 San Luis Obispo 26,622,700 7,029,374 33,652,074 San Marcos 7,733,700 3,088,600 10,822,300 Sonoma 8,567,704 12,217,710 20,785,414 Stanislaus 7,038,450 2,478,097 9,516,547 Campus Costs 308,751,843 116,714,146 425,465,989 Total CMS Costs $392,607,560 $269,274,358 $661,881,918 Source: The university’s June 2002 cost survey. * Central costs are the centralized costs for the development, implementation support, and operation of CMS systemwide. † Chancellor’s office “campus” costs represent the cost of implementing human resources and finance applications at the chancellor’s office. 114444 California State Auditor Report 2002-110 California State Auditor Report 2002-110 114455 APPENDIX C Projected Total Costs of the Common Management System Project by Type of Costs The projected total costs of the Common Management System (CMS) project as of June 2002, $662 million for fiscal years 1998–99 through 2006–07, are shown in both of these tables. These costs reflect the one-time investment costs and the recurring maintenance and operations costs. Table C.1 on the following page presents the systemwide costs in the areas of personnel, training, travel, consultants, hardware, software, and other costs by fiscal year. This table shows that costs are expected to climb until they reach $111 million in fiscal year 2004–05, when they start to decline. It is noteworthy that of the almost $85 million listed as the total for fiscal year 2006–07, more than $65 million represents the operations and mainte- nance costs that recur annually. Table C.2 provides a further breakdown of the same cost areas by campus. As shown in the table, campuses differ significantly in the amount and nature of the costs they are incurring. The source of the cost data is the university’s June 2002 cost survey. 114444 California State Auditor Report 2002-110 California State Auditor Report 2002-110 114455 114466 California State Auditor Report 2002-110 California State Auditor Report 2002-110 114477 1.C ELBAT raeY yb stsoC ediwmetsyS metsyS tnemeganaM nommoC detcejorP dna lautcA fo yrammuS 70–6002 hguorhT 99–8991 sraeY lacsiF morF detcejorP detcejorP detcejorP detcejorP detcejorP detcejorP lautcA lautcA lautcA lautcA stsoC latoT 70–6002 60–5002 50–4002 40–3002 30–2002 20–1002 10–0002 0002–9991 99–8991 stsoC fo sepyT 514,236,692$ 013,563,55$ 241,102,06 $ 448,423,45 $ 883,786,44$ 662,474,93$ 755,752,52$ 953,461,31$ 754,155,3 $ 290,606 $ lennosreP 537,444,91 877,832,1 617,247,2 854,853,3 263,974,3 476,843,3 283,827,2 603,212,1 486,326 573,217 gniniarT 768,702,11 095,160,1 227,453,2 107,594,2 900,115,1 342,784,1 463,437 536,296 157,417 258,551 levarT 815,586,661 857,236,8 121,826,52 730,803,23 916,466,22 323,295,12 392,541,52 361,456,12 534,852,8 967,108 stnatlusnoC 915,865,58 985,459,9 882,890,01 197,387,01 757,664,21 303,305,51 544,271,71 203,860,9 335,722 115,392 erawdraH 096,611,86 645,628,6 470,567,4 328,017,5 044,353,5 741,680,7 159,703,21 962,761,61 214,228,5 820,770,4 erawtfoS 471,622,41 661,876,1 668,298,1 453,567,1 276,779,1 898,035,2 864,344,2 402,872,1 040,874 605,181 rehtO 819,188,166$ 737,757,48$ 929,286,701$ 800,747,011$ 742,041,29$ 458,220,19$ 064,987,58$ 832,732,36$ 213,676,91$ 331,828,6$ stsoC latoT .yevrus tsoc 2002 enuJ s’ytisrevinu ehT :ecruoS 114466 California State Auditor Report 2002-110 California State Auditor Report 2002-110 114477 2.C ELBAT supmaC yb dna tsoC fo epyT yb stsoC tcejorP SMC 70–6002 hguorhT 99–8991 sraeY lacsiF rof latoT detcejorP stsoC stsoC rehtO stsoC erawtfoS stsoC erawdraH stsoC tnatlusnoC stsoC levarT stsoC gniniarT stsoC lennosreP 929,514,632$ 515,008,5 $ 961,300,44$ 189,984,67$ 230,462,13 $ 832,889,1 $ 099,551,3 $ 400,417,37 $ *lartneC supmaC 108,364,8 682,681 487,726 622,871 371,402,3 846,541 490,202 095,919,3 dlefisrekaB 532,317,6 000,35 538,72 004,09 001,286,1 000,4 005,532 004,026,4 †ecfifO s’rollecnahC 766,168 431,7 000,5 870,756 000,02 554,271 sdnalsI lennahC 041,644,81 139,896 418,295,2 202,908 519,176,4 045,328 756,284 180,763,8 ocihC 008,928,31 000,582 008,441 000,002 000,074,1 000,043 000,073,1 000,020,01 slliH zeugnimoD 621,283,81 005,772 000,679 008,873 007,268,3 625,802 004,111 002,765,21 onserF 336,323,91 000,964 593,239,1 492,55 005,339,6 524,392 086,586 933,459,8 notrelluF 626,594,81 092,175 004,482 007,782 076,026,7 005,876 001,115 669,145,8 drawyaH 468,246,11 931,744 931,521 000,681 324,289,2 496,534 785,359 288,215,6 tdlobmuH 821,801,73 389,050,1 747,205,3 812,815,1 900,964,01 473,893 812,798 975,172,91 hcaeB gnoL 005,024,31 006,15 007,182 008,45 000,440,1 003,003 004,621,1 007,165,01 selegnA soL 581,836,4 781,403 348,213 847,217,3 060,911 743,981 ymedacA emitiraM 736,383,51 366,518 251,305 301,685 670,503,4 000,28 005,215 341,975,8 yaB yeretnoM 928,812,81 112,121 025,422,1 000,081 498,983,7 832,222 065,985 604,194,8 egdirhtroN 934,732,02 393,801,1 390,753 812,533 526,814,7 736,214 870,546 593,069,9 anomoP 005,272,91 009,572 001,765,2 000,014 004,072,7 006,443 007,363 008,040,8 otnemarcaS 489,809,91 637,261 656,410,3 575,971 138,340,9 456,811 966,39 368,592,7 onidranreB naS 853,695,73 000,27 005,282,1 002,873,21 991,643,2 003,288,1 951,536,91 ogeiD naS 000,282,33 000,852 001,813,2 000,892,1 005,175,11 008,503 003,878 003,256,61 ocsicnarF naS 202,464,51 526,661 340,351,1 976,041 339,415,4 961,982 000,671,2 357,320,7 esoJ naS 470,256,33 309,104 697,122 681,955 841,592,21 767,974 377,490,1 105,995,81 opsibO siuL naS 003,228,01 007,551 003,001 004,620,1 009,874,2 004,072 006,897 000,299,5 socraM naS 414,587,02 215,876 759,033 823,412 529,735,6 382,383 230,042 773,004,21 ‡amonoS 745,615,9 782,811 963,332 665,27 837,609,1 518,712 052,922 225,837,6 sualsinatS 989,564,524 956,524,8 125,311,42 835,870,9 684,124,531 926,912,9 547,882,61 114,819,222 stsoC supmaC 819,188,166$ 471,622,41$ 096,611,86$ 915,865,58$ 815,586,661$ 768,702,11$ 537,444,91$ 514,236,692$ stsoC SMC latoT .yevrus tsoc 2002 enuJ s’ytisrevinu ehT :ecruoS .ediwmetsys SMC fo noitarepo dna ,troppus noitatnemelpmi ,tnempoleved eht rof stsoc dezilartnec eht era stsoc lartneC * .ecfifo s’rollecnahc eht ta snoitacilppa ecnanfi dna secruoser namuh gnitnemelpmi fo tsoc eht tneserper stsoc ”supmac“ ecfifo s’rollecnahC † .stsoc lennosrep sa defiissalc ew esoht ot ralimis stsoc detneserper ”stsoc rehto“ sti fo 000,46$ taht su demrofni supmac amonoS eht ,yevrus tsoc 2002 enuJ eht ot tneuqesbuS ‡ Blank page inserted for reproduction purposes only. 114488 California State Auditor Report 2002-110 California State Auditor Report 2002-110 114499 APPENDIX D Consultant Costs, Actual and Projected, for the Common Management System Project The California State University’s (university) reported cost information as of June 2002 for consultants working on the Common Management System (CMS) project is contained in the following tables. These costs reflect the one- time investment costs and the recurring maintenance and operations costs. The costs are divided into two categories: first, those amounts the university reported it paid to consultants through June 2002, and then those amounts the university projects it will pay to consultants from July 2002 through June 2007. All three tables include amounts that the university expects to spend for consulting services even if the specific consultant has not yet been determined. Table D.1 on the following page lists the consultants reported by the chancellor’s office for the project’s central costs. Table D.2 on page 151 is a summary of the consultants reported by the campuses. Table D.3 on page 152 details the consultants working at each campus. The source of the cost data is the university’s June 2002 cost survey. 114488 California State Auditor Report 2002-110 California State Auditor Report 2002-110 114499 TABLE D.1 Chancellor’s Office Central Consultant Costs Actual Costs Projected Costs July 1998 July 2002 Projected Consulting Firm Through June 2002 Through June 2007 Total Costs PeopleSoft $ 7,398,778 $ 137,308 $ 7,536,086 Not Yet Determined by Chancellor’s Office 7,050,000 7,050,000 Cedar/Hunter Group 4,083,292 2,261,100 6,344,392 Io Consulting 5,662,907 5,662,907 KPMG Consulting* 1,506,417 1,506,417 Lewis & Co 713,129 713,129 Monarch Information Technology Systems 514,795 514,795 Aligne 352,508 352,508 IBM 318,110 318,110 BIT/Digiterra/Ciber 268,976 268,976 California State University, Fresno 225,812 225,812 4GL Solutions 130,800 130,800 Mercury Interactive 124,788 124,788 Vista IT 91,338 91,338 Reboot 83,750 83,750 Price Waterhouse 57,500 57,500 Sunset Data Services 40,000 40,000 John G. Kelly 39,414 39,414 International Management 35,480 35,480 Korn/Ferry International 30,750 30,750 Academe Solutions 26,437 26,437 Technical Connection 19,500 19,500 GTC Systems 19,409 19,409 Michael W. Dula, Ph.D 16,508 16,508 Epeople 16,250 16,250 Bea Systems 16,040 16,040 ITprolink 15,600 15,600 John Miller Information 4,800 4,800 Ampco Systems Parking 2,471 2,471 Adver Services 65 65 Total Chancellor’s Office Central Consultant Costs $21,815,624 $9,448,408 $31,264,032 Source: The university’s June 2002 campus cost survey. * The actual amount reflects $683,612 paid to firm for Io Consulting as a subcontractor. 115500 California State Auditor Report 2002-110 California State Auditor Report 2002-110 115511 TABLE D.2 Campus Consultant Costs Actual Costs Projected Costs July 1998 July 2002 Projected Consulting Firm Through June 2002 Through June 2007 Total Costs Not Yet Determined by Campus $ 61,801,855 $ 61,801,855 Cedar/Hunter Group $ 9,393,876 23,231,129 32,625,005 Io Consulting 9,826,229 10,803,655 20,629,884 KPMG Consulting 6,479,099 6,479,099 PeopleSoft 3,451,590 1,527,000 4,978,590 Price Waterhouse 2,003,023 2,003,023 Sonoma State University 297,919 1,100,000 1,397,919 Monarch Information Technology Systems 123,472 1,054,680 1,178,152 EQV Consulting 227,783 795,200 1,022,983 BIT/Digiterra/Ciber 746,612 160,000 906,612 Signature 314,152 314,152 Bennett (SDB) 50,000 250,000 300,000 Financial Aid Services 226,331 226,331 Hershey Business Systems 23,430 175,000 198,430 Informed Decision 137,241 11,500 148,741 Independent–Jacobson 136,200 136,200 Fugatt 124,600 124,600 Walt Patterson 105,800 105,800 Carrera-Maximus Consulting 105,700 105,700 CSLink Consulting 100,000 100,000 IBM Consulting 91,800 91,800 Provista 81,789 10,000 91,789 San Jose State University 91,500 91,500 Deloitte Touche 71,383 71,383 Independent-Miller 71,348 71,348 Sargent 67,500 67,500 Information Management Systems Consulting 50,000 50,000 Sunset Data 50,000 50,000 Prism Computing 15,000 15,000 Page Consulting 14,950 14,950 CMC 9,000 9,000 Sierra Systems 6,500 6,500 Orion 5,000 5,000 EPNL 2,400 2,400 Amelia Assoc-Citrix Consulting 240 240 Total Campus Consultant Costs $34,044,036 $101,377,450 $135,421,486 Source: The university’s June 2002 cost survey. 115500 California State Auditor Report 2002-110 California State Auditor Report 2002-110 115511 TABLE D.3 Consultant Costs by Campus Actual Costs Projected Costs July 1998 July 2002 Projected Campus Through June 2002 Through June 2007 Total Costs Bakersfield Not Yet Determined by Campus $3,067,925 $3,067,925 Monarch Information Technology Systems $ 48,323 64,680 113,003 EQV Consulting 19,745 19,745 Io Consulting 3,500 3,500 Totals 71,568 3,132,605 3,204,173 Chancellor’s Office* Io Consulting 657,500 734,600 1,392,100 Not Yet Determined by Chancellor’s Office 257,000 257,000 Cedar/Hunter Group 33,000 33,000 Totals 690,500 991,600 1,682,100 Channel Islands Io Consulting 92,034 550,094 642,128 Page Consulting 14,950 14,950 Totals 106,984 550,094 657,078 Chico Not Yet Determined by Campus 3,900,000 3,900,000 Io Consulting 539,367 539,367 Independent–Jacobson 136,200 136,200 Independent–Miller 71,348 71,348 PeopleSoft 25,000 25,000 Totals 771,915 3,900,000 4,671,915 Dominguez Hills Not Yet Determined by Campus 1,470,000 1,470,000 Totals 1,470,000 1,470,000 Fresno Cedar/Hunter Group 1,865,300 776,600 2,641,900 BIT/Digiterra/Ciber 695,400 160,000 855,400 PeopleSoft 120,700 30,000 150,700 Monarch Information Technology Systems 52,900 70,000 122,900 IBM Consulting 91,800 91,800 Totals 2,826,100 1,036,600 3,862,700 Fullerton Not Yet Determined by Campus 5,933,500 5,933,500 Monarch Information Technology Systems 900,000 900,000 Hershey Business Systems 100,000 100,000 Totals 6,933,500 6,933,500 Hayward Not Yet Determined by Campus 4,630,000 4,630,000 KPMG Consulting 1,424,872 1,424,872 Io Consulting 166,398 1,070,000 1,236,398 Signature 96,400 96,400 San Jose State University 91,500 91,500 Informed Decisions 54,600 54,600 Price Waterhouse 51,700 51,700 Cedar/Hunter Group 30,200 30,200 Orion 5,000 5,000 Totals 1,920,670 5,700,000 7,620,670 115522 California State Auditor Report 2002-110 California State Auditor Report 2002-110 115533 Actual Costs Projected Costs July 1998 July 2002 Projected Campus Through June 2002 Through June 2007 Total Costs Humboldt Cedar/Hunter Group $ 58,173 $2,579,070 $2,637,243 Not Yet Determined by Campus 345,180 345,180 Totals 58,173 2,924,250 2,982,423 Long Beach Cedar/Hunter Group 2,032,665 3,169,400 5,202,065 Io Consulting 2,740,492 1,160,000 3,900,492 PeopleSoft 147,150 190,000 337,150 Bennett (SDB) 50,000 250,000 300,000 Financial Aid Services 226,331 226,331 Fugatt 124,600 124,600 Walt Patterson 105,800 105,800 CSLink Consulting 100,000 100,000 Hershey Business Systems 23,430 75,000 98,430 Informed Decisions 62,641 11,500 74,141 Totals 5,162,178 5,306,831 10,469,009 Los Angeles Not Yet Determined by Campus 900,000 900,000 Information Management Systems Consulting 50,000 50,000 Sunset Data 50,000 50,000 PeopleSoft 35,100 35,100 Sierra Systems 6,500 6,500 EPNL 2,400 2,400 Totals 137,500 906,500 1,044,000 Maritime Academy Sonoma State University 297,919 1,100,000 1,397,919 Io Consulting 295,450 910,000 1,205,450 Cedar/Hunter Group 1,059,379 1,059,379 KPMG Consulting 50,000 50,000 Totals 1,702,748 2,010,000 3,712,748 Monterey Bay Not Yet Determined by Campus 2,675,000 2,675,000 Io Consulting 555,200 1,007,376 1,562,576 Sargent 67,500 67,500 Totals 622,700 3,682,376 4,305,076 Northridge KPMG Consulting 3,423,000 3,423,000 PeopleSoft 2,692,894 2,692,894 Io Consulting 624,000 650,000 1,274,000 Totals 6,739,894 650,000 7,389,894 Pomona Cedar/Hunter Group 1,566,875 5,800,000 7,366,875 Price Waterhouse 51,750 51,750 Totals 1,618,625 5,800,000 7,418,625 Sacramento Not Yet Determined by Campus 4,623,800 4,623,800 PeopleSoft 363,100 1,287,000 1,650,100 EQV Consulting 201,300 795,200 996,500 Totals 564,400 6,706,000 7,270,400 continued on the next page 115522 California State Auditor Report 2002-110 California State Auditor Report 2002-110 115533 Actual Costs Projected Costs July 1998 July 2002 Projected Campus Through June 2002 Through June 2007 Total Costs San Bernardino Not Yet Determined by Campus $ 6,500,000 $ 6,500,000 Price Waterhouse $ 1,796,073 1,796,073 Io Consulting 378,025 378,025 Signature 168,472 168,472 Carrera-Maximus Consulting 105,700 105,700 BIT/Digiterra/Ciber 51,212 51,212 KPMG Consulting 23,000 23,000 CMC 9,000 9,000 Monarch Information Technology Systems 7,249 7,249 PeopleSoft 5,100 5,100 Totals 2,543,831 6,500,000 9,043,831 San Diego Not Yet Determined by Campus 12,378,200 12,378,200 Totals 12,378,200 12,378,200 San Francisco Not Yet Determined by Campus 11,571,500 11,571,500 Totals 11,571,500 11,571,500 San Jose Cedar/Hunter Group 541,185 2,513,815 3,055,000 Io Consulting 860,030 286,185 1,146,215 KPMG Consulting 261,968 261,968 Price Waterhouse 51,750 51,750 Totals 1,714,933 2,800,000 4,514,933 San Luis Obispo Cedar/Hunter Group 125,651 7,472,744 7,598,395 Io Consulting 2,150,528 2,274,900 4,425,428 Provista 81,789 10,000 91,789 PeopleSoft 37,546 20,000 57,546 Price Waterhouse 51,750 51,750 Monarch Information Technology Systems 15,000 20,000 35,000 Informed Decision 20,000 20,000 Prism Computing 15,000 15,000 Amelia Assoc-Citrix Consulting 240 240 Totals 2,497,504 9,797,644 12,295,148 San Marcos Not Yet Determined by Campus 1,250,000 1,250,000 Io Consulting 18,200 1,171,000 1,189,200 PeopleSoft 25,000 25,000 KPMG Consulting 14,700 14,700 Totals 57,900 2,421,000 2,478,900 Sonoma Cedar/Hunter Group 2,081,448 919,500 3,000,948 Io Consulting 745,505 989,500 1,735,005 KPMG Consulting 1,281,559 1,281,559 Not Yet Determined by Campus 399,750 399,750 Deloitte Touche 71,383 71,383 Signature 49,280 49,280 Totals 4,229,175 2,308,750 6,537,925 Stanislaus Not Yet Determined by Campus 1,900,000 1,900,000 EQV Consulting 6,738 6,738 Totals 6,738 1,900,000 1,906,738 Total Campus Consultant Costs $34,044,036 $101,377,450 $135,421,486 Source: The university’s June 2002 cost survey. * Chancellor’s office “campus” costs represent the cost of implementing the human resources and finance applications at the chancellor’s office. 115544 California State Auditor Report 2002-110 California State Auditor Report 2002-110 115555 APPENDIX E Common Management System Functionality at Each Campus As discussed in Chapter 3, implementing the Common Management System (CMS) software throughout the California State University (university) is resulting in a wide variation of functionality across campuses. The tables in this appendix illustrate the CMS functionality being implemented and planned on each university campus. CMS includes three applications whose functionality is presented in the tables of this appendix: Table E.1 on page 157 presents the finance application; Table E.2 beginning on page 158 displays the human resources application; and Table E.3 beginning on page 160 presents the student administration application. The chancellor’s office and the 23 university campuses are noted across the top of each table along with the version of the application currently in use and the version each campus plans to use in the future. Each application consists of functionality elements—modules, sub-modules, or university-created functionality—that represent different degrees of functionality for each CMS application. Because the more recent 8.x versions of the CMS applications were not available when the university’s campuses began implementing CMS, some campuses began their initial CMS implementation with the 7.x versions of the human resources and finance applications. The student administration application did not become available for use until the 8.x version was released. The version of each application and the CMS functionality elements being implemented, at the chancellor’s office and at all 23 campuses, is based on the university’s response to our October 2002 module survey. We divided the responses into two groups. The first group represents CMS functionality that each campus included in its June 2002 cost survey. The status of these functionality elements is defined as follows: • Implemented (I)—The campus already has implemented and is using this version of the application and this functionality element as of June 2002. 115544 California State Auditor Report 2002-110 California State Auditor Report 2002-110 115555 • Planned (P)—The campus plans to implement this version of the application and this functionality element after June 2002, and its projected costs in the June 2002 cost survey reflect these plans. • To Be Determined (TBD)—The campus considered this functionality element when it reported costs for the cost survey, but its planning process is incomplete, with a final decision yet to be made. The second group represents CMS functionality that was not included in the implementation costs when the campus com- pleted its June 2002 cost survey and is defined as follows: • Anticipated (A)—The campus anticipates implementing this functionality element at some future time and at an additional cost. • Blank—The campus did not indicate it has plans to implement this functionality element. 115566 California State Auditor Report 2002-110 California State Auditor Report 2002-110 115577 115566 California State Auditor Report 2002-110 California State Auditor Report 2002-110 115577 1.E ELBAT noitacilppA ecnaniF metsyS tnemeganaM nommoC eht fo ytilanoitcnuF ehT sualsinatS amonoS socraM naS opsibO siuL naS esoJ naS ocsicnarF naS ogeiD naS onidranreB naS otnemarcaS anomoP egdirhtroN yaB yeretnoM ymedacA emitiraM selegnA soL hcaeB gnoL tdlobmuH drawyaH notrelluF onserF slliH zeugnimoD ocihC sdnalsI lennahC ecfifO s’rollecnahC dlefisrekaB dnarG slatoT slatoT knalB DBT A P I noisreV noitacilppA 42 9 0 0 2 31 I I I I P I I I I I I I I I P )noisrev 5.7( SIF 42 0 0 0 42 0 P P P P P P P P P P P P P P P P P P P P P P P P )noisrev 4.8( SIF *stnemelE ytilanoitcnuF regdeL lareneG regdeL lareneG 42 0 0 0 11 31 P I P I I P P I P I I I I P I P I P I P I I P P lortnoC tnemtimmoC 42 9 1 0 41 0 P P P P DBT P P P P P P P P P P gnisahcruP 42 0 0 0 11 31 P I P I I P P I P I I I I P I P I P I P I I P P tnemerucorP selbayaP 42 0 0 0 11 31 P I P I I P P I P I I I I P I P I P I P I I P P elbayaP stnuoccA 42 3 0 1 02 0 P P P P P P P P P P P A P P P P P P P P P tnemeganaM tessA 42 2 1 1 02 0 P P P P P DBT P P P P P A P P P P P P P P P P selbavieceR 42 2 1 1 02 0 P P P P P DBT P P P P P A P P P P P P P P P P gnilliB 42 01 1 3 7 3 P I P P A A P P I P DBT P A I stegduB 42 41 1 6 3 0 A P A A P A P DBT A A yrotnevnI 42 01 1 9 4 0 A P P A A P A A P DBT A A A A stcejorP 42 9 2 11 2 0 DBT A P A A A A A A P DBT A A A A stnarG 42 31 4 6 1 0 DBT A DBT A DBT A P DBT A A A †sesnepxE 0 4 0 3 3 0 0 3 0 3 3 3 4 0 3 0 3 0 3 0 4 3 0 0 detnemelpmI stnemelE 8 4 11 5 4 4 9 3 6 3 6 0 4 21 4 7 3 6 4 7 4 0 3 7 dennalP stnemelE 2 0 0 1 0 3 1 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 denimreteD eB ot stnemelE 0 4 0 0 4 5 1 0 0 0 2 4 4 0 0 4 0 0 3 5 2 0 0 0 detapicitnA stnemelE 01 21 11 9 11 21 11 6 6 6 11 7 21 21 21 11 6 6 01 21 01 3 3 7 slatoT .2002 enuJ fo sa stnemele ytilanoitcnuf fo sutats eht stcefler hcihw ,yevruS eludoM 2002 rebotcO ’stiduA etatS fo uaeruB ehT :ecruoS .ytilanoitcnuf fo seerged suoirav niatnoc stnemelE * .ecfifo s’rollecnahc eht yb detroppus yllartnec si taht erawtfos SMC eht ni dedulcni ton si dna supmac a yb denwo si tnemele ytilanoitcnuf sihT † 115588 California State Auditor Report 2002-110 California State Auditor Report 2002-110 115599 2.E ELBAT noitacilppA secruoseR namuH metsyS tnemeganaM nommoC eht fo ytilanoitcnuF ehT sualsinatS amonoS socraM naS opsibO siuL naS esoJ naS ocsicnarF naS ogeiD naS onidranreB naS otnemarcaS anomoP egdirhtroN yaB yeretnoM ymedacA emitiraM selegnA soL hcaeB gnoL tdlobmuH drawyaH notrelluF onserF slliH zeugnimoD ocihC sdnalsI lennahC ecfifO s’rollecnahC dlefisrekaB dnarG slatoT slatoT knalB DBT A P I noisreV noitacilppA 42 31 0 0 0 11 I I I I I I I I I I I )noisrev 6.7( RH 42 0 0 0 61 8 P I I P I P P P P P I P I P P P P P I P I I P P )noisrev 0.8( RH *stnemelE ytilanoitcnuF secruoseR namuH ecrofkroW retsinimdA 42 3 0 0 9 21 P I I I I P P I P I I P I I I P P I P I P gnissecorP tnemyolpmE 42 6 1 2 8 7 P I P I I DBT P I P I P I P P I A A P stnemngissA ytlucaF_USC 42 5 1 0 8 01 P I P I I P P I P I P I I I DBT I P I P noitanimreT/tiurceR_USC 42 7 2 2 9 4 P I P P DBT P P P I P I P P I DBT A A drawA ydutS kroW_USC 42 1 1 1 01 11 P I P I I P P I P I P I P I DBT I P I P I I P A tnemyolpmE tnedutS_USC ot noitaborP ISG/ISS_USC 42 1 1 0 01 21 P I P I I P P I P I P I I I P I P I DBT I I P P tnenamreP 42 0 0 0 11 31 P I P I I P P I P I I P I I I P I P I P I I P P daoL eludehcS yralaS_USC 42 0 2 1 01 11 P I P I I DBT P I P I I P I P I P I P I DBT A I P P snoitalugeR tropeR ecrofkroW poleveD 42 11 3 4 4 2 DBT A P I DBT P P A A P DBT I A gniniarT retsinimdA 42 01 3 2 4 5 DBT I P I DBT P I I P DBT P A A I seicnetepmoC eganaM 42 0 1 1 01 21 P I P I I P P I P I P P I I I DBT I P I P I I P A snoitisoP eganaM 42 7 4 5 5 3 DBT A P I I DBT P P P A P DBT DBT I A A A stnevE ytlucaF eganaM 42 3 0 1 9 11 P I I I I P P I P A I I I P I P I P I P P ecrofkroW tiurceR 42 5 1 1 8 9 P I P I I P P P I P I I DBT P I I A I P ytlucaF yraropmeT_USC 42 6 1 1 9 7 P I P I I P P P P I P P DBT I I A I P noitalumuccA UTW FT_USC robaL & emiT 42 2 0 2 11 9 P I P I I P P I P I I A I P I P P I P A P P robaL & emiT edoC tnuoccA L&T_USC 42 4 2 2 8 8 P I P I I DBT P I P I A I P I DBT I P A P P edirrevO 42 1 1 0 21 01 P I P I I P P I P I I P I P I P P I P I DBT P P ecafretnI PSE_USC 42 6 2 1 8 7 P I P I I P DBT I P P I A I P DBT I P P lloryaP retsaM yfitreC_USC 42 2 0 2 11 9 P I P I I P P I P I I A I P I P P I P A P P ecafretnI PIP_USC sualsinatS amonoS socraM naS opsibO siuL naS esoJ naS ocsicnarF naS ogeiD naS onidranreB naS otnemarcaS anomoP egdirhtroN yaB yeretnoM ymedacA emitiraM selegnA soL hcaeB gnoL tdlobmuH drawyaH notrelluF onserF slliH zeugnimoD ocihC sdnalsI lennahC ecfifO s’rollecnahC dlefisrekaB 115588 California State Auditor Report 2002-110 California State Auditor Report 2002-110 115599 slatoT dnarG slatoT knalB DBT A P I *stnemelE ytilanoitcnuF lloryaP 42 61 0 0 4 4 P I I I P P P I lloryaP 42 0 0 0 21 21 P I P I I P P I P I I P I P I P I P I P I I P P noitubirtsiD tsoC robaL 42 7 4 1 7 5 P I P I I DBT DBT P A I P DBT DBT I P P P noceR yaP evitisoP_USC 42 01 3 1 5 5 P I DBT I I DBT P P I P DBT A P I ecafretnI lloryaP retsinimdA 42 21 4 4 4 0 DBT A DBT P DBT P A P DBT A P A stfieneB detamotuA retsinimdA 42 1 0 0 21 11 P I P I I P P I P I I P I I P I P I P P I P P stfieneB esaB retsinimdA 42 3 1 1 11 8 P I P I I P P I P P I I I P I P A DBT P P P ecafretnI SECA_USC 42 5 1 4 11 3 P A P I P P P A P I A P I P P P DBT A P gnitnuoccA evaeL_USC 42 01 2 3 6 3 P I P I DBT P P I P DBT P A A A ARBOC retsinimdA 42 41 1 3 4 2 P A I DBT P P A I P A ASF retsinimdA 42 41 1 4 5 0 DBT P P A P A P P A A noitartsinimdA stfieneB *stnemelE ytilanoitcnuF ecivreS-fleS stfieneBe 42 3 2 7 21 0 DBT A P P P P P A P P A A P DBT A P P P A A P stfieneBe tiurceRe 42 4 2 8 01 0 DBT A P P P P A A P A A P DBT A P P P A A P tnacilppA elfiorPe 42 3 1 7 21 1 P A P P P P P A A P P A P DBT A P P I A A P eeyolpmE 42 6 1 8 8 1 P A P P P A P A P A A P DBT A P I A A ytlucaF 42 31 2 5 4 0 DBT A P DBT P A P A P A A eenoitpO 42 21 3 3 6 0 DBT A P DBT P P A P DBT P P A noitasnepmoCe 42 31 2 5 4 0 DBT A DBT A P P A P P A A tnempoleveDe 42 61 1 1 6 0 DBT P P P P A P P yaPe 42 11 1 6 6 0 DBT A P P A P P A P A P A A reganaM 42 6 6 4 7 1 DBT DBT DBT P P P DBT A I A DBT P DBT A P P P A eeyolpme yb yrtne emit beW 0 42 2 42 62 0 0 61 0 81 21 0 42 01 61 0 11 0 42 0 11 01 0 0 detnemelpmI stnemelE 82 0 23 7 8 12 23 1 71 9 91 61 0 13 1 81 01 91 7 61 2 0 71 91 dennalP stnemelE 31 1 4 0 0 31 4 0 0 0 0 0 1 0 01 21 0 0 0 5 0 1 0 0 denimreteD eB ot stnemelE 0 31 0 0 0 4 0 1 6 2 1 01 31 0 0 8 0 0 5 51 02 0 0 5 detapicitnA stnemelE 14 83 83 13 43 83 63 81 32 92 23 62 83 14 72 83 12 91 63 63 33 11 71 42 slatoT .2002 enuJ fo sa stnemele ytilanoitcnuf fo sutats eht stcefler hcihw ,yevruS eludoM 2002 rebotcO ’stiduA etatS fo uaeruB ehT :ecruoS .ytilanoitcnuf fo seerged suoirav niatnoc stnemelE * 116600 California State Auditor Report 2002-110 California State Auditor Report 2002-110 116611 3.E ELBAT noitacilppA noitartsinimdA tnedutS metsyS tnemeganaM nommoC eht fo ytilanoitcnuF ehT sualsinatS amonoS socraM naS opsibO siuL naS esoJ naS ocsicnarF naS ogeiD naS onidranreB naS otnemarcaS anomoP egdirhtroN yaB yeretnoM ymedacA emitiraM selegnA soL hcaeB gnoL tdlobmuH drawyaH notrelluF onserF slliH zeugnimoD ocihC sdnalsI lennahC ecfifO s’rollecnahC dlefisrekaB dnarG slatoT slatoT knalB DBT A P I noisreV noitacilppA 42 1 0 1 91 3 P I P P P P P P P P P P I P P P P P I P P A P )noisrev 0.8( AS *stnemelE ytilanoitcnuF ytinummoC supmaC 42 1 0 1 91 3 P I P P P P P P P P P P I P P P P P I P P A P ataD tnedutS eganaM snoissimdA 42 2 0 0 91 3 P I P P P P P P P P P P I P P P P P I P P P tnedutS timdA 42 2 0 0 02 2 P I P P P P P P P P P P I P P P P P P P P P tnedutS llornE diA laicnaniF 42 1 0 1 91 3 P I P P P P P P P P P P I P P P P P I P P A P diA laicnaniF sdroceR tnedutS 42 1 0 1 91 3 P I P P P P P P P P P P I P P P P P I P P A P sdroceR tnedutS niatniaM slaicnaniF tnedutS 42 2 0 2 71 3 P I P P P P P P P P A I P P P P P I P P A P secivreS laicnaniF edivorP 42 31 1 1 7 2 P I P P A P I P P DBT P ytilanoitcnuF tfoSelpoeP–gnireihsaC tnemesivdA cimedacA 42 4 4 7 9 0 DBT P P DBT DBT A P P A A P P P A DBT P A P A A tiduA eergeD tnemecnavdA 42 21 0 8 2 2 A I A A P I P A A A A A snoitaleR rotubirtnoC *stnemelE ytilanoitcnuF ecivreS-fleS oiloftroP lanosreP 42 4 0 5 41 1 P A P P P A P P A P P A P P P P P I A P oiloftroP lanosreP sseccA ytinummoC 42 8 0 5 11 0 P A P P P A P A P P P P P A A P sseccA ytinummoC yrotceriD ytinummoC 42 8 0 7 9 0 P A P P P A P A A P P P A A A P yrotceriD ytinummoC hcaertuO 42 6 0 4 31 1 P A P P A P P P A P P P P P I A P P tnacilppA 42 7 0 2 31 2 P P P A P P P I P P P P P I A P P tcepsorP sualsinatS amonoS socraM naS opsibO siuL naS esoJ naS ocsicnarF naS ogeiD naS onidranreB naS otnemarcaS anomoP egdirhtroN yaB yeretnoM ymedacA emitiraM selegnA soL hcaeB gnoL tdlobmuH drawyaH notrelluF onserF slliH zeugnimoD ocihC sdnalsI lennahC ecfifO s’rollecnahC dlefisrekaB 116600 California State Auditor Report 2002-110 California State Auditor Report 2002-110 116611 dnarG slatoT slatoT knalB DBT A P I *stnemelE ytilanoitcnuF ecivreS-fleS secivreS renraeL 42 8 0 1 31 2 P I P P P P P P P P P P P I A P secivreS renraeL 42 5 0 2 61 1 P I P P P A P P P P P P P P P P A P P noitartsigeR ssalC 42 6 0 2 51 1 P I P P P A P P P P P P P P P A P P )claC noitiuT( lliB tnedutS tnemeganaM gninraeL 42 11 0 1 11 1 P A P P P P P P P P P I P tnemeganaM gninraeL 42 7 0 3 41 0 P A P P P A P P P P P P P P A P P )tnedutS( tropeR ssergorP eergeD 42 8 0 3 31 0 P A P P P A P P P P P P P A P P )ytlucaF( tropeR ssergorP eergeD 42 31 0 9 2 0 A A A P A P A A A A A tnemevlovnI 42 3 0 6 41 1 P A P P P A P A P P A A A P P P P I P P P latroP supmaC SP - latroP 0 11 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 11 0 0 0 0 0 detnemelpmI stnemelE 91 1 91 71 81 7 81 21 8 21 21 6 1 22 81 91 81 61 6 7 41 0 0 91 dennalP stnemelE 1 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 denimreteD eB ot stnemelE 1 9 0 0 2 21 1 1 3 0 2 3 6 0 0 3 0 0 4 31 3 5 0 3 detapicitnA stnemelE 12 12 91 81 02 02 91 31 11 21 41 9 61 22 81 22 91 61 12 12 71 5 0 22 latoT .2002 enuJ fo sa stnemele ytilanoitcnuf fo sutats eht stcefler hcihw ,yevruS eludoM 2002 rebotcO s’tiduA etatS fo uaeruB ehT :ecruoS .ytilanoitcnuf fo seerged suoirav niatnoc stnemelE * Blank page inserted for reproduction purposes only. 116622 California State Auditor Report 2002-110 California State Auditor Report 2002-110 116633 Agency’s comments provided as text only. The California State University Office of the Chancellor 401 Golden Shore Long Beach, CA 90802-4210 February 19, 2003 Ms. Elaine Howle* State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: The California State University welcomes the opportunity to respond to the draft audit of its Common Management Systems project. The Bureau of State Audits’ time and effort dedicated to the comprehensive review of the California State University’s Common Management Systems is appreciated. The auditors’ recommendations will be beneficial for the continued development and improvement of the Common Management System effort in the CSU. We have already implemented or begun to implement some of the recommendations and will be acting on the others. The attached document will provide responses to each of the auditors’ recommendations. With kind regards, Sincerely, (Signed by: Charles B. Reed) Charles B. Reed Chancellor Enclosure * California State Auditor’s comments begin on page 175. 116622 California State Auditor Report 2002-110 California State Auditor Report 2002-110 116633 The CSU appreciates the time and effort dedicated by the Bureau of State Audits to the comprehensive review of the CSU’s Common Management System. The auditor’s recommendations will be beneficial for the continued development and improvement of the Common Management System effort in the CSU. The CSU further appreciates this opportunity to respond to the draft audit of its Common Management Systems project. We have reviewed the draft and find that the facts are correctly reported in the audit. However, we do not always agree with the interpretation of those facts. The CSU agrees in nearly all cases with the auditor’s recommendations, although we do not agree fully with all the findings of the audit. As reported in the audit draft, we have already implemented or begun to implement some of the recommendations. We will be acting on the others. We have provided responses to each of the auditor’s recommendations and have organized those responses in the same order as they were presented in the audit. Projects of this size and complexity lend themselves to a variety of equally valid approaches. The standards and approaches suggested by the auditors are appropriate in some enterprise resource planning (ERP) implementations and are useful guidelines as we move to complete the CMS project. The CSU’s implementation approach to the CMS project is working and more importantly continues to be effective. In the final analysis, large ERP projects such as CMS are judged more on the outcomes they achieve than the implementation process they utilize. The CSU stands by the outcomes achieved by the CMS project. The majority of the 24 CSU locations (campuses and the Chancellor’s Office) are already successfully operational in CMS, 14 on finance, 15 on human resources, and 5 on student administration. By July of 2003, those numbers will increase to 15 on finance, 19 on human resources, and 6 on student administration. We are completing the CMS 1 implementation within our original budget estimates and on the schedule we proposed. Together, these achievements constitute a successful project by industry and higher education standards. Chapter 1 - CMS BUSINESS CASE ° RECOMMENDATION: To ensure that the university’s future IT projects are appropriate expenditures of state resources, the university should adopt policies and procedures that require a feasibility study before the acquisition and implementation of significant IT projects. Such a feasibility study should include at least a clearly defined statement of the business problems or opportunities being addressed by the project, as well as an economic analysis of the project’s life-cycle costs and benefits compared with the current method of operation. ° RESPONSE: Agree – The CSU will adopt policies and procedures that require a feasibility study prior to the acquisition and implementation of significant IT projects. The CSU is intending to complete a two-phase cost benefit analysis of the current CMS project to help demonstrate current and future benefits of the project to campuses and the system. The first phase will have been completed by March 15, 2003. ° RECOMMENDATION: To measure the benefits achieved through common management and business practices, the university should establish quantitative measures of increased business process efficiencies through CMS, in addition to any qualitative factors being assessed. Such measures should identify the increased productivity of staff, reduced operational costs, or both. ° RESPONSE: Agree –As part of our ongoing process to assess the successes of our CMS project the CSU will examine the establishment of quantitative measures of business process efficiencies. Comments on Chapter 1 Findings The CSU began the Information Technology Strategies (ITS) initiative in the mid 1990’s to improve the university’s application of technology to teaching, learning, and managing. The university implemented an extensive planning and development process that included an assessment of the current state of 116644 California State Auditor Report 2002-110 California State Auditor Report 2002-110 116655 technology, development of a future state vision, identification and prioritization of potential initiatives, and finally the development of the first wave of initiatives. The first wave initiatives were selected based on their ability to support the CSU’s mission, the value to stakeholders (students, faculty, staff, institutions), their potential cost efficiencies, and their support of institutional values. The CSU invested over two years in the development and refinement of the ITS. The CSU’s ITS has guided the universities information technology investments since March of 1996 when it was endorsed by the Board of Trustees. One of the first eleven initiative investments was the Common Management System. In 1996-97, campuses began the effort to develop a collaborative to replace their aging and limited administrative systems with systems to provide students, faculty and staff the level of administrative support required of a competitive university in the 21st century. The business case for the campuses 2 was compelling. The existing campus administrative systems were aging. They could not provide the integrated, comprehensive administrative services required to meet the universities needs. Vendors had indicated that they would no longer support the legacy software. The campuses understood the need to provide more comprehensive services to students, faculty and staff. Since so many campuses were looking to replace their systems the only question was to determine how and not whether to go about replacing those systems. 3 Cost benefit analyses and feasibility studies are necessary and valuable tools when a business entity has a choice in a course of action, but they are not as valuable or necessary when only one course of action is available. In 1996-97, the CSU was spending $100 million on faltering administrative systems and had no human resources system. With systems not meeting campus needs and soon to be unsupported by vendors, the CSU’s focus was on adopting the most cost-effective replacement strategy, not on studies to determine if replacement was necessary. To provide the services and management capacity required for the new century, the CSU had to replace its financial and student systems and implement a modern human resources system. While a comprehensive feasibility and cost benefit study was not needed to make a decision to move 4 forward, an understanding of the cost of new systems was important to campuses and the system. The system and each implementing campus did determine potential cost and develop a financial plan to manage the implementation. The CMS effort focused on the objectives for participating campuses of: • Minimizing the cost to implement and maintain application software • Minimizing the time to implement application software • Adopting best practices or a foundation for best practice, where possible during the implementation process • Establishing standards to share information for common reporting purposes • Promoting multi-campus cooperation and collaboration • Leveraging limited resources • Providing ready access to current, accurate, and complete administrative information and the means to use it in an efficient manner The opportunities for efficiency and effectiveness were compelling. The limitations of funding to provide systems were obvious. Many campuses could not afford to purchase and develop systems on their own. Further, they could not afford to redevelop or expand existing data centers. The campuses and the university collectively saw opportunities for cost avoidance and minimizing the cost of developing new systems through collaboration and cooperation among the campuses. It would be more economical to 23 purchase, implement and maintain a single software system collaboratively than to purchase, implement, and maintain twenty-three separate systems. The CSU focused on the efficiencies available in leveraging its size, sharing the work of selecting, developing, implementing, and operating administrative systems, and in negotiating the best deal through that size and collaboration. 116644 California State Auditor Report 2002-110 California State Auditor Report 2002-110 116655 Chapter 2 – PROJECT COSTS ° RECOMMENDATION: To ensure that it adequately monitors and controls project costs, the university should take the following actions: ° Establish a mechanism to collect and compile system-wide project cost information, including campus costs. The university should determine what level of quarterly cost information it needs from campuses to monitor the project. Further, the university should collect comprehensive cost information annually, including in-kind, upgrades, and integration costs. ° Compare the collected project costs against the approved system-wide project budget and publish this information in a quarterly status report. ° RESPONSE: Agree –The CSU will determine the campus quarterly and annual cost information necessary to monitor the project and will establish a mechanism to collect and report that data on a system-wide basis. ° RECOMMENDATION: Additionally the university should ensure that it includes all costs of the CMS project in its annual reports to the Legislature. ° RESPONSE: Agree - The CSU will implement annual reporting of campus and central expenditures in the annual Measures of Success (MOS) reports to the Legislature. The university welcomes the continued collaboration with the Legislative Analysts Office and the Legislature to refine the Measures of Success as our collective needs develop. As circumstances change, we are committed to change the MOS procedures, data, and reporting. ° RECOMMENDATION: Finally, the university should establish a system-wide funding plan for the CMS project that includes campuses. Further, before it begins any major information technology project in the future, it should ensure that it has a funding plan in place that covers the entire scope of the project. ° RESPONSE: Agree – The CSU has a funding plan in place for the central CMS project. The CSU will also ensure that its funding plan includes campuses. The CSU will ensure it has a comprehensive funding plan in place for future major information technology projects. Comments on Chapter 2 Findings 1 The current projected implementation cost of the CMS project is below the originally projected costs. The CSU projected implementation costs of $350 to $400 million over seven years when the project began. According to the cost areas defined in the initial CMS projections, the CSU is meeting budget projections. Consistent with the delegation of fiscal responsibility to campuses, each CSU campus has established and monitored budgets and expenditures related to CMS while the Chancellor’s Office has monitored budgets and expenditures for the central project. While we did not collect the costs centrally, campuses have monitored local costs. The CSU is now collecting campus expenditure information to report on a system-wide basis, although responsibility for management of campus expenditures will remain with the campuses. The university holds campuses accountable for their overall budgets and expenditures. The CMS project has been implemented consistent with this overall delegation of responsibility and authority. Chapter 3 – BUSINESS OBJECTIVES ° RECOMMENDATION: To ensure that it minimizes the costs and time to implement and maintain its CMS software, the university should do the following: ° Reassess the design of CMS and evaluate the economies that can be achieved by reducing the number of separate CMS databases throughout the university that currently must be tested separately when campuses implement software revisions that reflect modifications and upgrades. 116666 California State Auditor Report 2002-110 California State Auditor Report 2002-110 116677 ° RESPONSE: Agree – The CSU will evaluate the economies and benefits that could be achieved by alternative technology approaches. ° Continue its recently-established practice of tracking the actual hours spent to develop modifications to the CMS software. It should consider this information when estimating the costs and time associated with developing and applying future modifications to new versions of the vendor software, and when evaluating the associated maintenance costs in reapplying, testing and implementing its current and future modifications. ° RESPONSE: Agree – The CSU will continue to implement and improve its processes for managing modifications including tracking the actual hours spent to deliver modifications to the campuses. ° Define the scope and associated costs of CMS by identifying the specific functionality that is necessary to achieve the university’s expectations expressed in the CMS project charter. ° RESPONSE: Agree – The CSU will identify the specific functionality that is necessary to achieve the expectations documented in the CMS Project Charter. ° Further, examine the cost associated with campuses’ plans to add functionality elements to increase functionality beyond the defined scope. ° RESPONSE: Agree – The CSU will examine the cost associated with campuses’ plans to add PeopleSoft functionality beyond the defined scope. ° RECOMMENDATION: Also, when procuring information technology systems or software in the future, the university should evaluate its specific business processes against vendor products before procurement, then select vendors that best accommodate the university’s specific needs. ° RESPONSE: Agree – The CSU will, as part of the requirement development phase of a project, consider the impact of current business processes on vendor selection before procuring information technology solutions or software in the future in those situations that industry best practice would suggest warrant such a review. ° RECOMMENDATION: To ensure that it provides ready access to current, accurate and complete administrative information and the means to use this information effectively as well as to establish standards to share information for common reporting purposes, the university should: ° Determine how it could improve the design of CMS to report system-wide information, and establish a minimum level of functionality that all campuses must implement to facilitate this reporting. ° RESPONSE: Agree –The CSU will evaluate the need for improvements in system-wide reporting and define the minimal level of functionality that all campuses must implement to facilitate such system-wide reporting. ° RECOMMENDATION: To ensure that it adequately addresses CMS project quality and information security, the university should: ° Establish a quality management plan and continue its efforts to establish an effective quality assurance function for the CMS project. ° RESPONSE: Agree – The CSU will continue to plan and establish an effective quality assurance function for the CMS project. ° Consider hiring an independent oversight consultant to perform various quality assurance functions and to evaluate the progress of the CMS project. ° RESPONSE: Agree – The CSU will evaluate the need for an independent consultant to supplement the comprehensive oversight functions already performed through the CSU’s established information technology advisory structure. 116666 California State Auditor Report 2002-110 California State Auditor Report 2002-110 116677 ° Establish a policy on sensitive information requiring that campuses implement the use of confidentiality agreements for all employees with access to the CMS system. ° RESPONSE: Agree – The CSU will establish a policy on sensitive information requiring that campuses implement the use of confidentiality agreements for all employees with access to the CMS system, as has been the traditional CSU practice of handling sensitive information. Security is of prime concern with the CMS project. The CSU will continue to address security issues and develop solutions that provide optimal protection of individual and institutional information. ° RECOMMENDATION: To ensure that it uses recommended practices in its future procurements, the university should: ° Plan project procurements to share risk with vendors and consultants, such as allowing them to propose their solutions and structuring contracts to protect the university’s interests, including provisions to pay only after deliverables have been tested and accepted ° RESPONSE: Agree – The CSU will continue to use risk sharing with vendors, as it did with the shared data center contract (Unisys), when circumstances are consistent with industry best practices and when marketplace conditions make such an approach feasible, appropriate, and cost effective. Comments on Chapter 3 Findings In the limited cases where some functionality may be less in PeopleSoft than in the replaced legacy system, the functionality will likely be regained in future software releases. These temporary occurrences are far outweighed by the overall functionality improvement across the system. The addition of a comprehensive human resources system where there was none is one major example of this fact. ECAR corroborated this fact in its “Enterprise Resource Planning Systems in Higher Education” stating “Many institutions reported losing functionality and momentum in the early stages of implementation, only to recover old functions and gain new ones as they mastered new technologies and business practices. The assumption that institutions will gain value from an ERP implementation as soon as the system is installed is misleading. Most systems don’t reveal their value for several years. Not only must the institution go through a transition period, but many times business processes must be changed in order to reap the full benefits of the new systems.” (ECAR Research Bulletin, Volume 2002, Issue 22, November 12, 2002) Central CMS provides a variety of development and support functions for all three-project applications: finance, human resources, and student administration. These support functions are integral to project operations even without modification to the base software. They include planning, software review, software development, and implementation support for campuses. The software modification function is only one part of software development. The expenditures of $34 million through June 2002 include effort in all of these areas, all of which were required with or without CSU modifications to the PeopleSoft product. The CSU identified seven business objectives when it began the CMS project. We expect to achieve each of those objectives and are well on the road to that achievement. • Minimizing the cost to implement and maintain application software – The CSU’s implementation of a single system-wide version of PeopleSoft significantly minimizes cost by reducing the implementation and maintenance effort from twenty-three stand-alone campus efforts to one coordinated effort. The CSU further minimized cost by limiting campus variations to the common application. Some system-wide modification would be required for any modern administrative 5 system available on the market. Fewer modifications were required for the PeopleSoft software because of its flexible design. System-wide modifications were assumed to be necessary from the beginning, but have been carefully monitored and managed to further minimize cost. The CSU has managed the number of modifications required to meet our system and campus needs 116688 California State Auditor Report 2002-110 California State Auditor Report 2002-110 116699 to only 200, a level that we believe is far below industry averages for software modifications in Enterprise Resource Planning implementations. Furthermore, the CSU has managed to reduce the number of modifications over time as evidenced by the reduction of HR modifications from 100 to 75 during the recent software upgrade process. • Minimizing the time to implement application software – It takes far less time for the university 6 to implement one version of a single software package twenty three times than it would to separately develop, implement, and maintain twenty three separate applications. • Adopting best practices or a foundation for best practice, where possible, during the implementation process – The CSU has maintained the focus on best practices throughout the development of the CMS project. Campuses have collaboratively redesigned business practices for the betterment of the university throughout the development and implementation. • Establishing standards to share information for common reporting purposes – The CSU has established and is implementing standards for system-wide reporting. System-wide reporting begins with accurate campus data. Campuses are held accountable for providing consistent and accurate data to meet system-wide reporting. This methodology continues to provide campuses with autonomy while assuring that system-wide reporting can be done on a consistent and comparable basis. • Promoting multi-campus cooperation and collaboration – CMS was initially conceived as several multi-campus collaboratives. The objective of multi-campus cooperation and collaboration was not only to minimize time and cost to implement software, adopt best practices, establish standards, and leverage limited resources, but also to capitalize on and develop the resources available on individual campuses across the system. CMS embraced and expanded on these same objectives through its system-wide implementation. • Leveraging limited resources – The CSU is leveraging its limited resources to provide a modern, comprehensive and integrated administrative management system to each campus. Campuses provided staff time and expertise to develop and share business processes as an integral part of the CMS development. Campus staff continues to provide guidance to the central CMS effort. Campuses agreed to contribute funding to the central project through “off-the-top” assessments to fund a single development effort in lieu of multiple development efforts. The CSU developed a single data center to provide the needed technological support for the application software. A central CMS group was formed that guided the development to a common standard for all campuses. The development of such a comprehensive system would not have been fiscally possible without this leveraging. • Providing ready access to current, accurate, and complete administrative information and the means to use it in an efficient manner – Campuses implementing CMS have realized the ready access envisioned in the beginning of the project. CMS already provides campuses better 7 access to higher quality, more accurate data. Students have access to their own personal and academic data. Faculty and staff have better access to personal data and to data required to do their jobs. Managers have access to data to effect better decision making. CMS provides for commonly defined data in common formats that facilitate effective reporting on campuses and system-wide. Chapter 4 – CMS PROCUREMENT ° RECOMMENDATION: To ensure that it uses recommended practices in its future procurements, the university should: ° Use the procurement process appropriate to the procurement objective. If the procurement objectives change during the process, it should restart the procurement using the appropriate process or formally modify the procurement process through appropriate written notification to potential vendors. ° RESPONSE: Agree – The CSU will continue to use processes appropriate to the objectives of its procurements including restart and re-bid of procurements when conditions so warrant. 116688 California State Auditor Report 2002-110 California State Auditor Report 2002-110 116699 ° Establish a practice of using quantitative scoring to demonstrate clearly that it followed an objective evaluation process to identify a best-value vendor. It also should document the resolution of evaluation team concerns to demonstrate it considered and appropriately addressed or mitigated these concerns. ° RESPONSE: Agree – While the CSU did not use a weighting process for the application software procurement, it is the CSU’s general practice to utilize weights and scoring to identify best value as evidenced by the data center selection process. The applications software selection process considered each vendor against pre-stated selection criteria. Only two vendors, PeopleSoft and SCT met the minimum criteria. After extensive consideration of each vendor’s capabilities with regard to the pre-stated criteria, it was ultimately determined that PeopleSoft best satisfied those criteria. The CSU will further review its procedures for the resolution and documentation of concerns arising during evaluation processes. ° Enforce its policy that prohibits the use of sole source contracts when multiple vendors or consultants are available to provide the goods or services. ° RESPONSE: Agree – The CSU makes every effort to apply and manage sole sourcing consistent with law and policy. In retrospect the University could have done a better job of applying the sole-source policy in some of the CMS procurement efforts. Nevertheless, we believe the results achieved were appropriate. The CSU gained value through providing the best possible consulting at a reasonable price. ° Establish a policy to require the solicitation of at least three offers for its pre-qualified vendor goods and consultant services master agreements. ° RESPONSE: Agree – The CSU will require campuses to solicit at least three offers for its pre- qualified multiple-award master agreements. Comments on Chapter 4 Findings The CSU believes that the decision to continue the procurement process for applications software was appropriate when the evaluation committee determined to select only one applications software vendor. At the time of the selection process, there were only two qualified vendors that could potentially meet the CSU requirements for an integrated suite of management software for higher education. Even today, the market for an integrated suite of human resources, finance, and student systems is limited to the same two vendors. Restarting the process would not have brought more vendors to the table or 8 changed the vendors’ product offerings or pricing. The vendors were aware throughout the process that the final outcome could be a single vendor solution since the criteria said “one or more” vendors would be selected. Even the vendors contemplated one award and had already provided their best offers for a system-wide implementation. The choice of PeopleSoft was a long-term process. While a “CMS campus participation survey”, done early in the process, indicated a preference for one vendor over another, it was before complete information was known about the capabilities of the vendors. When more was known, there was a clear change in thinking on the part of most of the campuses that yielded the final outcome. The selection of application software was a long and thorough process and the CSU re-evaluated vendors and options as the project progressed. In the early stages, more was known about SCT since some campuses were on the SCT/IA products. Early opinions were based more on vendor familiarity that on any informed assessment of vendor characteristics. The selection of PeopleSoft was based on a variety of factors including cost. As in other ERP’s in higher 8 education, cost was not the highest-ranking factor. Other selection factors are of equal or greater importance such as feature/functionality best-fit requirements, architecture best fit with IT strategy/goals, the vendor’s reputation, and the vendor’s ability to provide a complete solution. While initial acquisition cost was a factor in the CSU’s selection process for application software, there were other equally or more important factors such as life cycle cost, flexibility and usability. 117700 California State Auditor Report 2002-110 California State Auditor Report 2002-110 117711 Chapter 5 – DATA CENTER ° RECOMMENDATION: To ensure that it continues to receive improved service levels from the data center vendor, the university should continue to monitor the data center services and promptly take action to resolve problems with the vendor, including problems in meeting service levels or providing and maintaining appropriate and sufficient facilities, equipment and staff throughout the life of the contract. ° RESPONSE: Agree – The CSU will continue to monitor the performance of its data center and take appropriate and prompt action to assure appropriate service levels. ° RECOMMENDATION: To ensure that it provides CMS campuses with the means to effectively and efficiently store and retrieve data needed for useful management reporting, the university should expedite the CMS data-warehousing project and include the CMS related costs of campus data warehousing in its CMS project costs. ° RESPONSE: Agree – While there is no current CMS data warehousing project, there is a data warehousing study under way to look at one option for data warehousing. The CSU will evaluate whether the inclusion of data warehousing within the scope of CMS is the best solution. If it is, then costs will be included within CMS. Comments on Chapter 5 Findings 9 The CSU appreciates the auditors focus on data warehousing, however it is important to reemphasize that data warehousing was not included within the scope of the CMS project. It was considered in the early development of the project, but was deferred in order to focus on the priorities of development of the operational applications. The CSU will assess how to best respond to campus and system needs for data warehousing. Solutions may range from campus-centric to centralized data warehousing designs. The current CMS data center environment, while not providing a data warehousing capability, does provide for optimum data storage, retrieval and reporting in support of the application functions. Chapter 6 – CONFLICT OF INTEREST ° RECOMMENDATION: The Legislature should consider the following to ensure that the university takes appropriate action to prevent potential conflicts of interest in the future: ° Require the university to provide designated university employees with periodic ethics training similar to that required for designated state employees by Government Code, Sections 11146 through 1146.4. ° RESPONSE: Agree - Should the legislature so require, the CSU will of course comply. ° Require the university to establish an incompatible activities policy for university employees similar to that addressed in Government Code Section 19990, for state employees. ° RESPONSE: Agree - California law currently regulates CSU incompatible activities. The CSU will distribute a comprehensive review of the law that governs CSU related incompatible activities. Should the legislature desire to enact a further expression, the CSU will of course comply. In fact the CSU is actively supporting related legislative action. Pending legislation supported by the CSU requires employees with outside employment to report that employment on an annual basis. This legislation would help make sure we are aware of any potential conflicts in the future with all the individuals that make decisions on services, programs and related areas for our students. ° RECOMMENDATION: To ensure that it adequately addresses potential conflicts of interest and prohibited use of nonpublic information, the university should: 117700 California State Auditor Report 2002-110 California State Auditor Report 2002-110 117711 ° Conduct periodic conflict-of-interest trainings, such as the ethics training required of state agencies, for designated employees to inform those in decision-making positions about required disqualification when potential conflicts of interest exist. This training should provide employees guidance on using the FPPC’s eight-step process as well as their responsibility to seek the advice of counsel. ° RESPONSE: Agree – The CSU will periodically provide conflict of interest training to its campus filing officers and other administrative personnel. It will also continue to provide the FPPC Form 700 with instructions to its designated personnel annually and answer any questions. The CSU will increase its training efforts. ° Establish an incompatible activities policy that it formally communicates to employees. ° RESPONSE: Agree – Enactment of an incompatible activities policy such as that appropriate to civil service employees in GC 19990 will eclipse the need to administratively establish such a policy. The CSU will communicate the current requirements under California law regarding incompatible activities and will communicate and comply with any enacted legislation addressing CSU incompatible activities. ° Require all employees to sign disclosure statements before participating in the procurement process. ° RESPONSE: Agree – The CSU will require all employees to sign disclosure statements before participating in the procurement process. ° Enhance its disclosure form to indicate clearly what constitutes a conflict of interest, list all vendors participating in the procurement as they are known, and state that evaluators are prohibited from using nonpublic information to benefit personally. ° RESPONSE: Agree – The CSU will enhance its disclosure form to clearly indicate what constitutes a conflict of interest and state that evaluators are prohibited from using nonpublic information to benefit personally. Where possible, all vendors participating in the procurement will be listed on the disclosure form. In addition, the CSU will ensure that all participants understand the scope of the procurement activity and the nature of the commitments made when signing the form prior to participating in a procurement activity. ° Update its conflict-of-interest code to classify all positions responsible for evaluating or overseeing vendors or contractors. ° RESPONSE: Agree – The CSU will continue to update and apply its conflict of interest code and designation of positions to reflect changes in decision-making responsibilities. ° Require consultants that serve in a staff capacity and that participate or influence university decisions to file Form 700s. ° RESPONSE: Agree – Consistent with existing policies, the CSU will require consultants to complete Form 700 when the consultants are hired to make or participate in making decisions that foreseeably will have a material effect in a personal financial interest. ° Remind human resources personnel of their responsibility to collect, retain, and make available the filed Form 700s for the required seven-year period. ° RESPONSE: Agree – The CSU will remind human resources personnel of their responsibility to collect, retain, and make available the filed Form 700s for the required seven-year period. ° Remind employees of the prohibition against using information not available to the public to benefit financially, and discipline infractions if necessary. ° RESPONSE: Agree – The CSU will remind designated employees of their rights and responsibilities under the conflict of interest code. The Chancellor has directed the Vice Chancellor for Human Resources to review any issues related to conflict of interest. 117722 California State Auditor Report 2002-110 California State Auditor Report 2002-110 117733 Comments on Chapter 6 Findings The CSU believes that its employees behaved appropriately and in accordance with policy and that there were no improprieties. The CSU has sought to comply with Conflict of Interest law, policies and procedures. Positions have been added to and amended on the list of designated positions when it was believed that those positions warranted inclusion and amendment. Staff have completed and submitted Form 700 conflict of interest statements. In the physical relocation of the Chancellor’s Office, some older filings were misplaced, but generally records have been conscientiously kept. 117722 California State Auditor Report 2002-110 California State Auditor Report 2002-110 117733 Blank page inserted for reproduction purposes only. 117744 California State Auditor Report 2002-110 California State Auditor Report 2002-110 117755 COMMENTS California State Auditor’s Comments on the Response From the California State University To provide clarity and perspective, we are commenting on the response to our audit from the California State University (university). The numbers correspond with the numbers we have placed in the university’s response. 1 Although the university contends it is completing the Common Management System (CMS) implementation within its original budget estimates, on pages 40 through 45 we discuss how the university’s updated cost projections for the CMS project exceed initial cost estimates and that these initial cost estimates did not include all related costs. Further, later in its response, on page 166, the university states that it projected implementation costs of $350 million to $400 million over seven years when the project began. However, as we note on page 40, it could show us no documentation to support these estimates. 2 The university indicates that the business case for CMS was compelling given that its prior administrative systems were aging, no longer meeting campus needs, and losing vendor support. On pages 30 through 35, we analyze these and other reasons cited by the university for implementing CMS. Although there is some degree of merit to the reasons, neither individually nor collectively do they justify proceeding with CMS without developing and documenting a business case that establishes the problem being solved as well as the expected benefits and associated costs to achieve those benefits. 3 The university states that cost-benefit analyses and feasibility studies are not as valuable or necessary when only one course of action is available. However, a feasibility study was necessary to determine if, in fact, only one course of action was available. Further, it was necessary to determine the most cost-effective method to implement the selected alternative. For example, the university could have prepared a cost analysis comparing the cost of developing, implementing, and maintaining one or a few databases with the cost to develop, implement, and maintain distinct databases at each campus (but one) as is currently planned. As indicated on page 21, had the university conducted 117744 California State Auditor Report 2002-110 California State Auditor Report 2002-110 117755 a feasibility study that mirrored the requirements of the State Administrative Manual (SAM), it would have conducted an economic analysis of each alternative considered and identified reasons for rejecting those not selected. Absent such an analysis, it is unclear how the university determined that today’s CMS project represents the most cost-effective replacement strategy. Further, although the university contends that it was spending $100 million on faltering administrative systems, it is unclear whether the $100 million figure cited by the university reasonably represents the university’s true cost to maintain the systems that were replaced by CMS. On page 24, we used the same data as the university and prepared an alternative analysis that indicates the annual cost to maintain these prior systems was approximately $43 million. Similarly, as we note on page 36, information provided by the university to the Legislature suggests that it spent roughly $42 million annually to maintain these systems. 4 The university states that the system and each implementing campus determined potential cost and developed a financial plan to manage the implementation. However, as we discuss on pages 59 through 61, the university does not have a systemwide funding plan for the project. Further, as we state on page 67, our funding survey of campuses revealed that only seven of 23 campuses were able to provide a funding plan that identified funding sources for their projected CMS costs. 5 Although the university contends that fewer modifications were required for the PeopleSoft software, as we discuss beginning on page 84, the university did not sufficiently evaluate its specific business processes before purchasing the software. Although we recognize that some business process evaluation would need to take place after procurement, it could have performed a substantial amount of this evaluation to better understand how its business processes fit the software products it was considering and which software products would require modification to meet its business needs. Failing to do so, the university had no basis to anticipate the extent of software modifications it would eventually make or the lost functionality some campuses would experience. 6 Although the university’s assertion that it takes less time to implement one version of a single software package 23 times than it would to separately develop, implement, and maintain 23 separate applications makes conceptual sense on the surface, 117766 California State Auditor Report 2002-110 California State Auditor Report 2002-110 117777 it implies more commonality than currently exists with the university’s CMS project. As we discuss on page 75, with one exception, the university has been installing and maintaining separate and distinct databases for campuses that must be tested separately at each campus. Moreover, as we state on page 76, the university did not define at the beginning of the CMS project what specific functionality comprised CMS, and it has not established a minimum level of functionality that campuses are required to implement systemwide. Thus, implementing the CMS software throughout the university is resulting in a wide variation in functionality across campuses. This lack of uniformity increases the overall costs to implement and maintain the CMS software and limits its usefulness in producing systemwide reports. 7 Although individuals at each campus may have better access to higher quality, more accurate data, it is important to recognize that, as we discuss on page 86, the university will continue to use processes that existed before CMS to produce systemwide reports because it did not design CMS to replace these pro- cesses. Not achieving the full potential from installing a single university-wide suite of administrative software can be traced to the university’s conscious decision to limit this capability. 8 The university states that vendors were aware throughout the process that the final outcome could be a single vendor selection since the criteria said “one or more” vendors would be selected. However, as we discuss on page 99, the university’s request for qualifications (RFQ) document did not provide for a scoring method to select a single vendor. Instead, the RFQ provided for an evaluation to qualify vendors to participate in subsequent phases. Further, although the university asserts that its selection of PeopleSoft considered other factors that it believes are equally or more important than initial acquisition costs, the university’s evaluation process never established the relative importance of the factors that the university may have considered to select a single vendor. Instead, as we discuss on page 100, the university’s evaluation document simply placed positive and negative com- ments into plus and minus categories, and we could not tell whether one factor carried more weight than another. 9 The university states that it is important to reemphasize that data warehousing was not included within the scope of the CMS project. However, as we discuss on page 119, although the university later removed data warehousing from the scope of the CMS project, data warehousing was originally included within 117766 California State Auditor Report 2002-110 California State Auditor Report 2002-110 117777 the scope of the CMS project charter. The university’s senior director for the CMS project explained that the university elimi- nated data warehousing from the scope early in the project and believes the university made it a campus responsibility largely because of concerns about CMS project resources. 117788 California State Auditor Report 2002-110 California State Auditor Report 2002-110 117799 cc: Members of the Legislature Office of the Lieutenant Governor Milton Marks Commission on California State Government Organization and Economy Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press 117788 California State Auditor Report 2002-110 California State Auditor Report 2002-110 117799