Summary

Student Aid Commission: Problems Continue With Its Automated Financial Aid Processing System Table of Contents Summary Introduction Chapter The Commission’s Automation Project Continues To Be Plagued With Problems Recommendations 3 Response to the Audit Student Aid Commission Summary  Has been plagued Results in Brief with problems since Audiitts H inicgehplitgiohnt.s ... T he California Student Aid Commission (commission) helps students achieve their postsecondary educational goals by  Stu d I e s n p t o A o id rl y C o d m es m ig is n s e io d n , ’s providing financial aid services. These services include (com c m o i m ss p io le n x ) , a utomated administering state and federal authorized grant and loan syste c m u , m F b A e P r S so : me, and programs. inefficient. In 1986, an outside consultant suggested that considerable cost  Faulty programming savings and improved service could be achieved through the caused incorrect commission’s increased use of automation. In 1987, the loan balances that commission began its procurement of the automated system, still exist. known as the Financial Aid Processing System (FAPS). After several years of development, the commission declared FAPS  System flaws operational in January 1993. However, FAPS has been continue to cause plagued with problems since its inception, and the continuation interruptions and of system problems led to the assessment of FAPS by Deloitte inefficiencies. & Touche (Deloitte) in November 1993. New management team In September 1994, Deloitte reported that the FAPS database is revised commission’s poorly designed; the FAPS application is complex, strategy for addressing cumbersome, and unable to function adequately; and the problems with FAPS. operation of FAPS is inefficient. Deloitte recommended the commission document the current system, perform only mandatory system changes, correct the problems with the accounting functions, add systems that are external to FAPS to expand services, and develop alternatives to FAPS. Chapter 303, Statutes of 1995, required the Bureau of State Audits to determine the extent to which the commission has addressed the concerns and implemented the recommendations Deloitte reported in September 1994. During our review, we found the commission has neither fully addressed the concerns nor fully implemented the recommendations. Although the 4 commission has made some improvements to FAPS, it has not corrected the fundamental design deficiencies of FAPS. According to the chief information officer, the commission has not corrected the design deficiencies because it would require a complete revision and rewrite of the FAPS database and application. As a result, the following conditions continue to exist:  The FAPS accounting application still does not allow for the proper reconciliation of cash deposits and FAPS cash transactions. In addition, FAPS cannot function adequately as the subsidiary ledger to the general ledger accounting system; therefore, accounting staff must continue to maintain subsidiary ledgers on personal computers.  FAPS processing of loan payments has improved; however, erroneous transactions still exist. Specifically, from January 1993 through December 1994, faulty FAPS processing caused inaccurate loan balances and some improper refunds. Although it corrected the faulty processing in December 1994, the commission has not identified and corrected all the erroneous loan balances caused by the faulty FAPS processing.  Despite some improvements in FAPS operations, system flaws continue to cause system interruptions and inefficiencies. In addition, poor system design make system documentation, corrections, and improvements costly and slow. Additionally, the commission has implemented some, but not all of the recommendations cited in the Deloitte report. The commission completed three of the five recommended types of documentation, but believes it is imprudent to continue with the documentation project because it is limited in its use and is not cost-effective. Also, as recommended by Deloitte, the commission is making only mandatory changes to FAPS. Specifically, the commission only makes changes to correct or modify FAPS when the system stops working, to comply with legal requirements, to make changes promised to outside parties, or to meet financial or legal liability to its clients. To correct problems with the system’s accounting functions, the commission is studying the feasibility of splicing an existing accounting system to FAPS. If the study indicates that adding an accounting package is not feasible, the commission plans to research other alternatives to FAPS. In the meantime, the commission has not completed its analysis of alternatives to FAPS, 5 such as redesigning FAPS or entering into a joint venture with another guarantee agency that has an efficient system. Finally, a new management team has revised the commission’s strategy for addressing the problems with FAPS. The team has developed a comprehensive strategy that includes both short- and long-term plans. The commission is implementing its short-term plan, which is premised on the outcome of the accounting system study; however, the direction of the long-term plan depends on the results of the short-term plan. Therefore, the commission’s direction in regards to FAPS will not be known until May 1996. Because the commission is still determining its approach to address the problems outlined in the Deloitte report, we could not fully assess the commission’s actions to correct the problems with FAPS. Recommendations In addition to implementing its comprehensive strategy, the commission should do the following:  Correct the financial and accounting problems with FAPS so that accounting staff can perform basic accounting functions, such as reconciling cash deposits to FAPS cash transactions; and  Identify and correct those incorrect loan balances that occurred from January 1993 through December 1994 because of FAPS processing errors. The Bureau of State Audits should conduct a follow-up review of the commission’s actions after the commission has determined its approach to address the problems with FAPS. Agency Comments The commission agreed with our report and recommendations. To address our recommendations, the commission states that it recognizes the need to correct the FAPS financial and accounting problems and will continue to make this a high priority issue. The commission also states that it anticipates correcting the loan balances affected by the FAPS programming error by June 1996. 6 [Blank page inserted for reproduction purposes only.] 7 Introduction T he mission of the California Student Aid Commission (commission) is to help students achieve their postsecondary educational goals by providing grant and loan services to students, schools, and lending institutions. The commission accomplishes its mission by administering federal and state authorized grant and loan programs for students attending California colleges, universities, and other postsecondary institutions. The commission consists of 15 members representing segments of the State’s postsecondary education community, students, and the general public. These commission members appoint an executive director, who is responsible for formulating commission policy and managing the internal and external daily business of the Financial Aid Grants Program and the California Loan Program. To accomplish its responsibilities, the commission receives funding from both the State and the federal government. The Financial Aid Grants Program is funded primarily by the State and provides students with grants; work-study aid; and other specialized financial aid, such as loan assumption programs for students pursuing careers in teaching. The California Loan Program is funded primarily by the federal government and assists students in meeting postsecondary education expenses by providing low-interest loans that the State guarantees and the federal government reinsures. During fiscal year 1994-95, the commission received approximately $229 million from the State’s General Fund; $334 million from the federal government; and $45 million from other sources, such as the State Guaranteed Loan Reserve Fund and reimbursements. In 1986, an outside consultant completed a study of the commission’s automated information systems and suggested that considerable cost savings could be achieved through the commission’s increased use of automation. The commission believed that automation would provide better management oversight and policy planning by lowering operating costs and eliminating its dependence on external contracting. 8 The commission’s long-term goal was to develop an integrated automated system that would support the commission’s grant and loan programs by providing the following benefits:  The ability to consolidate the existing grant and loan program systems spread across six separate systems;  An anticipated savings of millions of dollars annually after the initial expenditure of installing this automated system because of anticipated savings in external grant and loan processing contracts;  The ability to handle loans and collections that had been handled until then by a contractor;  Increased revenues from collections on defaulted loans; and  Additional benefits in terms of better services to students, schools, and participating lending institutions. Scope and Methodology Chapter 303, Statutes of 1995, mandated that the Bureau of State Audits conduct an audit to determine the actions the commission has taken to address the concerns and implement the recommendations reported by Deloitte & Touche on September 28, 1994. The report is entitled “An Independent Assessment of the Financial Aid Processing System.” During our review, we interviewed commission staff and management and reviewed various documents, such as internal and external reports, accounting records, commission minutes, and management plans. Chapter The Commission’s Automation Project Continues To Be Plagued With Problems Chapter Summary 9 T he California Student Aid Commission’s (commission) Financial Aid Processing System (FAPS) has been plagued with problems since its inception. Because problems with the system continued after installation was complete in January1993, the commission recommended that an independent consultant conduct an assessment of FAPS. In September 1994, Deloitte & Touche (Deloitte) completed its assessment and concluded that the system design was not adequate to support the commission today or in the future. Deloitte reported several concerns and recommendations, but the commission has not fully addressed and implemented them. Although some improvement has been made to correct the accounting functions of FAPS, significant problems still exist in the system itself. For example, the database design is still poor, the application design is complex and cumbersome, and the system still operates inefficiently. A recently established management team has changed the commission’s approach to addressing existing problems with FAPS. Specifically, the team has developed a comprehensive strategy that includes both short-term and long-term plans. In the short term, the commission is assessing whether an accounting package can be spliced onto FAPS; however, the assessment will not be completed until mid-December 1995. The direction of the commission’s long-term plan depends on the results of its accounting package assessment; therefore, if the assessment indicates that it is not feasible to splice an accounting package onto FAPS, the commission plans to implement a preferred solution from its alternatives analysis. However, the commission will not complete its assessment of alternatives to FAPS until May 1996. Because the commission is still determining the best approach to address the issues outlined in the Deloitte report, we could not fully assess the commission’s actions to address the problems with FAPS. Background In January 1987, the Department of Finance approved the commission’s plans to develop the new automated system, which would be operated using computers at Teale Data Center, one of the State’s shared computer facilities. In June 1987, the commission released a vendor solicitation to design, develop, and implement the system. In June 1988, the commission awarded the FAPS $5.2 million contract to Systemhouse. 10 Although FAPS was to be installed by April 1990, approximately two years after the contract award, continuing system performance issues delayed final installation. In March 1992, continuing delays in the development of the loan subsystem and increasing Operation of FAPS computer operating costs led the commission to replace began more than two and Systemhouse and Teale Data Center with E.D.S. Federal a half years behind Corporation (EDS). At that time, EDS was the existing loan schedule and $3.8 million processing contractor for the commission. EDS assumed the over the original contract. responsibility for installing FAPS onto its computers, converting the commission’s loan accounts to FAPS, and providing ongoing software maintenance and computer operation support for FAPS. The commission began operating FAPS in January 1993, more than two and a half years behind schedule and $3.8 million over the original contract for FAPS development services. When FAPS became operational and the commission took over the daily loan processing operations previously contracted to EDS, it continued to experience system problems. For example, the commission had to correct system flaws in the loan subsystem while continuing to perform day-to-day business operations. Additionally, the commission had to simultaneously improve the operating efficiency of the system and implement design changes to accommodate changing state and federal laws. In September 1993, the commission’s executive director recommended that an independent audit of FAPS be conducted amidst ongoing frustration by lenders and schools with the system’s performance. They were frustrated because FAPS was unable to properly process both lender premium fee payments for the guarantee of student loans and borrower loan payments. The commission decided that an independent firm should audit the entire FAPS to identify any undiscovered problems and to ensure that any previously identified problems had been resolved. Independent Consultant Identified Management and Performance Problems With FAPS In November 1993, Deloitte was retained to conduct an independent assessment of FAPS. Deloitte divided the assessment into two phases. The first phase of its assessment was a preliminary review of FAPS management and system performance to identify possible issues that could be reviewed in more detail during the second phase. The second phase focused on a detailed 11 analysis of FAPS’ effectiveness and whether FAPS can support the commission over the next decade. The Phase I report, dated March 1, 1994, identified three potential FAPS management and system performance areas that could be studied in more detail. Specifically, the report identified the potential need for improvement in the area of managing system changes to FAPS, such as the need for more detailed planning and Consultant concludes documentation of service requests and rigorous testing of system that system design is not changes. In addition, the report stated that in the area of adequate to support the organization and management, the commission could potentially commission today or in improve client services by striving for greater consistency and the future. quality in internal and external client communication and by strengthening FAPS contract management. Finally, the report indicated that in the area of system effectiveness, the poor performance of FAPS and the service deficiencies may stem from poor initial system and database design and inadequate system documentation. During the second phase of the assessment, the commission agreed that Deloitte should conduct a detailed analysis of FAPS to determine whether the system is adequate to meet the needs of the commission and adequate to respond to the changing student financial aid environment. In its Phase II report, dated September 28, 1994, Deloitte concluded that the FAPS database and application design are inadequate to support the commission today or in the future. Deloitte Report Indicates That FAPS Has Data, Application, and Operational Problems Deloitte organized its issues into three groups: data issues, application issues, and operations issues. Regarding FAPS data issues, Deloitte stated that the FAPS database is poorly designed, contains duplicate data, and does not allow for the efficient access of data. 12 In addition, Deloitte found that the FAPS application, a collection of 13 subsystems interacting with the database and each other, is complex and cumbersome and cannot adequately support the Consultant found that commission’s financial operations. Specifically, the accounting FAPS database is subsystem does not adequately record and track accounting complex, cumbersome, transactions, it does not process loan payments correctly, and it and does not adequately does not function adequately as the subsidiary ledger for the support financial commission’s general ledger accounting system. As a result, operations. accounting staff cannot reconcile daily cash transactions to FAPS and manual alternatives are necessary to address FAPS processing errors. Finally, Deloitte found that the operation of FAPS is inefficient because of needed system improvements. System flaws cause slowdowns and interruptions that lead to processing delays. Furthermore, since the existing system documentation is poor, system improvements are difficult to implement. As a result of these problems, the costs to maintain and operate FAPS are higher than necessary, the system’s availability to users is reduced, and the commission’s ability to respond to needed changes is limited. FAPS Database and Application Design Remain Poor We found the commission has made some improvements to FAPS since the Deloitte report, but has not corrected the fundamental design deficiencies of FAPS. As a result, the conditions reported by Deloitte continue to exist. According to the commission’s chief information officer, the database design is poor, and the database does not allow for the efficient access of data. Furthermore, he agrees that the application design is complex and Commission states that cumbersome. However, he states that the commission would have correcting the design to completely revise the database and rewrite the application to deficiencies would correct these problems. He stated that the commission has not require a complete completed its analysis of alternative solutions to FAPS and revision and rewrite of therefore, the commission has not revised or rewritten the database the FAPS database and and application. application. Although the commission has made some improvements to the FAPS accounting application, it is still experiencing problems with its reconciliations of cash deposits and FAPS cash transactions. Because FAPS does not always process the transactions correctly, accounting staff cannot always reconcile the cash deposits to the cash transactions reported by FAPS. Furthermore, we also found that FAPS still cannot function adequately as the commission’s subsidiary ledger to its general ledger accounting system. As a 13 result, accounting staff must continue to manually input information into the general ledger accounting system and maintain some subsidiary ledgers on personal computers. FAPS Processing of Loan Payments Has Improved; However, Erroneous Transactions Still Exist The commission has corrected the faulty FAPS processing of loan payments. However, it has not corrected all the erroneous loan balances that resulted from the faulty processing. Specifically, in July 1993, commission staff discovered that when corrections were made to loan payments, FAPS incorrectly processed the corrections, creating inaccurate loan balances. As a result, the commission issued refunds to borrowers who actually have outstanding loan balances. After discovering the faulty FAPS processing, approximately seven months after FAPS began operating, the commission developed a process to manually intervene and review the appropriateness of subsequent FAPS refunds before issuing refund checks to the borrowers. In December 1994, EDS corrected the faulty FAPS programming. Although EDS has corrected the faulty FAPS programming, the commission has not identified all the erroneous loan balances Commission has not caused by nearly two years of faulty operation. Specifically, the identified all inaccurate commission has developed a data processing solution that will loan balances caused by correct the erroneous loan balances; however, it has not two years of faulty implemented the data processing solution due to legal and operation. operational issues. For example, the commission is reviewing the legality of collecting on accounts that were previously noted as being “paid in full” and determining how it will ensure that FAPS is available to conduct daily business operations while correcting these errors. In the meantime, accounting staff must continue to manually verify and calculate refunds before issuing refund checks to borrowers. Although FAPS Operations Have Improved, Performance Remains Inefficient Despite some improvements in FAPS operations, the problems Deloitte identified continue to exist. System flaws continue to cause system interruptions and inefficient processing. In addition, poor system design make system documentation, corrections, and improvements costly and slow. Specifically, we determined that 14 although the number of times the system goes down has decreased over the last few months, it continues to terminate abnormally almost nightly and sometimes two to three times per night. As a result, an EDS system engineer must be available throughout the nightly system processing to monitor and restart the system whenever it terminates abnormally. According to EDS, it has prepared and submitted solutions to the commission to System flaws continue to permanently correct the processing problems that lead to system cause interruptions and shutdown, but the commission has not assigned the lengthy processing implementation of these solutions a high priority. According to times. the chief information officer, the abnormal termination of the system is dealt with on a case-by-case basis, deferring the correction of those problems that do not affect accuracy or swift processing. Furthermore, we determined that system performance has improved but is still inefficient. For example, we noted that for the most recent six months (May 1995 through October1995) FAPS was available as scheduled approximately 99.6 percent of the time compared to 97.1 percent for the previous six months (November 1994 through April 1995). However, FAPS was still unavailable during the end of the month because of long end-of-the-month system processing times. According to the chief information officer, other system performance improvements are being studied and implemented, such as purging of files; however, performance issues cannot be fully addressed without a complete rewrite of the system. Finally, the commission recognizes that the original system designer, Systemhouse, produced documentation with an unconventional format and structure and that the existing documentation has inconsistent quality and depth of detail. The chief information officer stated that a consulting firm was hired to properly document FAPS; however, the commission felt it was imprudent to continue with the project because the nonstructured format and constant changes to FAPS limit the value of extensive documentation that may not prove to be cost-effective. Some Recommendations Have Not Been Implemented In earlier sections of this report, we highlighted some of the commission’s improvements to FAPS; however, some of Deloitte’s recommendations have not been fully implemented. In its September 1994 report, Deloitte presented five overall recommendations: document the current system, perform only 15 mandatory changes to the system, correct problems with the accounting functions, add systems that are external to FAPS for expanded service, and develop alternatives to FAPS. As we discussed earlier, the commission began a process to document the system as recommended by Deloitte. By November 1994, the consulting firm hired to document FAPS completed three of the five recommended types of documentation. However, according to the chief information officer, the commission felt it was imprudent to continue the documentation project because it was not considered cost effective to document a system that is written in a nonstructured manner and is constantly changing. Further, the commission uses a service request ranking system to rank requested changes to FAPS. Currently, only service requests ranked as priority one are approved by the commission for implementation. Priority one service requests include requests to correct or modify FAPS when the system stops working, to modify the system to comply with legal requirements, to make changes promised to parties outside of the commission, and to meet financial or legal liability to its clients. Additionally, the commission did not act to immediately correct the system’s accounting functions as recommended by Deloitte. In June 1995, approximately nine months after the Deloitte report was issued, the commission entered into a three-way agreement A feasibility study to with the U.S. Department of Education and the Transitional replace the accounting Guarantee Agency to develop a plan to address unmet system sub-system should be requirements. As part of this agreement, the Transitional complete by Guarantee Agency arranged for a study to review the feasibility of mid-December. splicing an accounting package onto FAPS to replace the existing accounting subsystem. The study, conducted by McGladrey & Pullen, is expected to be completed by mid-December 1995. It involves evaluating the information and business requirements related to the commission’s general ledger accounting system, identifying interfaces to and from FAPS, and identifying potential software solutions and vendors. Regarding the external systems approach recommended by Deloitte, our review determined that in November and December 1994, the commission participated in a project to complete a national student loan database that used the external systems approach recommended by Deloitte. However, we found no other examples of similar projects using the external systems approach. 16 Finally, although the commission surveyed systems from other guarantee agencies in January 1995, it has not completed its analysis of alternatives to FAPS, such as redesigning FAPS or entering into a joint venture with another guarantee agency that has an effective and efficient system. According to the chief information officer, if the McGladrey & Pullen study indicates that adding an accounting package to FAPS is unfeasible, the commission will complete its alternatives analysis for a long-term grant and loan processing solution. New Management Team Revises Commission’s FAPS Strategy The commission hired a new chief financial officer in April 1995 and a new executive officer in July 1995. The new executive officer hired a chief information officer in August 1995 to assist in resolving the commission’s data processing system problems. This new management team has revised the commission’s strategy for addressing existing problems with FAPS. Although the commission had planned to reprocure the continued maintenance and operation of FAPS in September 1995, the new management team was not convinced that continuing with the current system is in the best interest of the commission and canceled the reprocurement. Management team has Instead, the new management team has changed the commission’s changed the direction regarding the future of FAPS and has developed a commission’s direction comprehensive strategy for providing technology to support the regarding the future of commission’s grant and loan processing. This comprehensive FAPS. strategy was presented to the commission in September 1995 and includes both short-term and long-term plans regarding FAPS. The commission’s short-term plan for FAPS includes the following steps:  Negotiate an extension of its contract with EDS and establish a cost containment team;  Complete the feasibility assessment of adding an existing accounting system to FAPS;  Provide funding and assistance for high-priority projects;  Establish a quality assurance and validation process; and  Defer any noncritical enhancements to FAPS. 17 The commission is negotiating a contract extension with EDS for continued FAPS maintenance and computer operations through December 1996 to allow the commission to properly assess its alternatives to FAPS. In addition, the commission plans to establish a team that will be responsible for addressing system inefficiencies. Furthermore, the commission is waiting for the results of the study being conducted by McGladrey & Pullen to determine the feasibility of adding an existing accounting system to FAPS. If the study identifies a system that could be added to FAPS, the commission plans to provide management oversight of the project. If this is not feasible, the commission plans to implement the preferred solution from its analysis of system alternatives. Finally, in an effort to contain the cost of operating FAPS, the commission is planning to establish a validation and verification process to review the reasonableness of service requests. The commission’s long-term plan consists of several steps: obtain computer operations support services, complete its analysis of alternatives to FAPS, and implement the preferred solution from the analysis. Specifically, the commission plans to separate Commission expects to computer operations from system maintenance services and is complete its analysis of waiting for a proposal from Teale Data Center in which the center alternatives to FAPS by estimates its cost for providing FAPS computer operation support May 1996. services. The commission expects the data center to complete its analysis and submit its proposal by the middle of December 1995. If the proposal is not cost-effective, the commission plans to solicit bids for computer operation support services. In addition, the commission plans to complete its analysis of alternative solutions for correcting the problems with FAPS by May 1996. Once the analysis is complete, the commission plans to issue a vendor solicitation document for the procurement of the preferred solution, which may include the development of a new system or the redesign of FAPS. Recommendations In addition to implementing its comprehensive strategy, the commission should do the following:  Correct the financial and accounting problems with FAPS so that the accounting staff can perform basic accounting functions, such as reconciling cash deposits to FAPS cash transactions; and 18  Identify and correct those incorrect loan balances that occurred from January 1993 through December 1994 because of the FAPS processing errors. The Bureau of State Audits should conduct a follow-up review of the commission’s actions after the commission has determined its approach to address the problems with FAPS. We conducted this review under the authority vested in the state auditor by Section 8543 et seq. of the California Government Code and according to generally accepted governmental auditing standards. We limited our review to those areas specified in the audit scope of this report. Respectfully submitted, KURT R. SJOBERG State Auditor Date: December 12, 1995 Staff: Elaine Howle, CPA, Audit Principal Robert Cabral, CIA George Alves 19 20 21 22 23