CSA

Summary

California State Auditor · 99028.2 · 1999-01-01

Read the report at California State Auditor ↗

Child Support Enforcement Program: The State Has Contracted With IBM to Develop and Implement the Major Component of the Statewide Automated Child Support System September 2003 99028.2 rotiduA etatS ainrofilaC S T I D U A E T A T S F O U A E R U B The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TTY (916) 445-0033 OR This report is also available on the World Wide Web http://www.bsa.ca.gov/bsa/ The California State Auditor is pleased to announce the availability of an on-line subscription service. For information on how to subscribe, please contact the Information Technology Unit at (916) 445-0255, ext. 456, or visit our Web site at www.bsa.ca.gov/bsa Alternate format reports available upon request. Permission is granted to reproduce reports. � � � �� September 24, 2003 99028.2 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As required by Chapter 479, Statutes of 1999, the Bureau of State Audits (bureau) presents its second audit report concerning the procurement of a single, statewide automated child support system by the Department of Child Support Services (department), with the Franchise Tax Board (board) acting as its agent. The statute requires the bureau to monitor the evaluation and selection process for any signs of bias or favoritism toward any bidder. In December 2002, we reported on the project team’s progress toward procuring the main part of the system, referred to as the child support enforcement system, through June 2002. That report discussed the project team’s evaluation of the single bid that had a proposed contract price of almost $1.2 billion received from the IBM Group. During our evaluation of the process the team used to score the proposal, nothing came to our attention to cause us to conclude that the project team deviated from the predefined evaluation criteria. This report discusses the procurement process through July 14, 2003, the date the State signed a contract with the IBM Group to design, develop, and implement the child support enforcement system. Though federal and state agencies approving the project expressed various concerns about the terms of the contract and the feasibility study developed by the project team, the project team received the required approvals after satisfying these concerns and obtained a contract price totaling $801 million for the system. During our monitoring of the negotiation sessions, nothing came to our attention that would lead us to believe that the negotiations resulted in significant changes in the contract that might violate the requirements in Chapter 479, Statutes of 1999, or the solicitation document. Our limited review of the contract found that it included the major business need areas that the child support enforcement system must address and the compensation method agreed to by the parties did not violate the law and was within the parameters of the solicitation document. Although the project team has now executed the contract for the child support enforcement system, it is still more than a year away from procuring a contractor for the state disbursement unit, a separate system for collecting, disbursing, and recording child support payments. Respectfully submitted, ELAINE M. HOWLE State Auditor �� CONTENTS Summary 1 Introduction 5 Audit Results The Negotiation Process Reduced the Project’s Cost and Resulted in a Contract That Appears to Meet the Criteria Outlined in the Solicitation Document 9 Despite Concerns, the Project Team Received Federal and State Approval to Execute the Contract 12 The Project Team Has Yet to Award a Contract for the State Disbursement Unit 17 Appendix A The Procurement Process 19 Appendix B Timeline of Key Events 23 Responses to the Audit Health and Human Services Agency, Department of Child Support Services 25 State and Consumer Services Agency, Franchise Tax Board 27 SUMMARY RESULTS IN BRIEF In 1988, Congress passed the Family Support Act (act) requiring each state to establish a single, statewide automated child support enforcement system to track and collect court- Audit Highlights . . . ordered child support payments and to locate nonpaying parents. Although California has made two attempts in the last 10 years Our continued review of the to meet the requirements of the act, its failures have resulted in Department of Child Support the federal government imposing penalties on the State that may Services and Franchise Tax Board’s (project team) cumulatively total approximately $1.2 billion by federal fiscal procurement of a single, year 2006. To relieve itself of the mounting penalties, in 2000 statewide automated child California began a project called the California Child Support support enforcement system Automation System (project) using a procurement approach revealed the following: intended to maximize vendor commitment to the project’s þ On July 14, 2003, the success and minimize the risk to the State. On July 14, 2003, the project team signed a State signed a contract with the IBM Group to design, develop, contract for $801 million with the IBM Group to and implement the major component of the project, the child design, develop, and support enforcement system, for a total of $801 million. implement the major part of the single, Chapter 479, Statutes of 1999, one of several laws passed by the statewide automated child support system. Legislature to restructure the State’s child support enforcement activities, assigned the responsibility for procuring, developing, þ Despite concerns, the implementing, and maintaining the single, statewide Federal Office of Child Support Enforcement automated child support enforcement system to California’s approved the State’s Department of Child Support Services (department), with request for funding, the Franchise Tax Board (board) as its agent. The legislation giving the project team also requires the Bureau of State Audits (bureau) to monitor permission to execute the contract between the the evaluation and selection process for any signs of bias or State and the IBM Group. favoritism toward any bidder. þ The State Department In 2000, the project team, made up of staff from both the of Finance placed certain conditions on its approval department and the board, began the procurement process by of the feasibility study, separating the project into two parts: a main system, referred to requiring, for example, as the child support enforcement system, and a state disbursement that the project team submit a benefits unit, a separate system for collecting, disbursing, and recording measurement plan within child support payments. In December 2002, we reported on one year following the the project team’s progress toward procuring the child support contract’s signing. enforcement system through June 2002. That report discussed continued on next page . . . the project team’s evaluation of the single bid it received, which was from the IBM Group and had a proposed contract price of almost $1.2 billion. According to the project team, this bid met the requirements outlined in the solicitation document. We also California State Auditor Report 99028.2 11 þ The project team is noted that during our evaluation of the process the team used to still more than a year score the proposal, nothing came to our attention to cause us to away from procuring a conclude that the project team had deviated from the predefined contractor for the state evaluation criteria. disbursement unit, a separate but integral part of the single, statewide The project team’s next step involved negotiating the terms automated child support of the contract. Because it had received only a single bid, it enforcement system. engaged a consulting firm to examine the reasonableness of that bid’s costs. The consulting firm reported that the IBM Group’s proposal might contain a premium of up to 25 percent for some of the contract costs to minimize its risks related to the project. The project’s negotiation team negotiated a reduction in the contract price from the proposed $1.2 billion to $900 million. During this stage of the process, nothing came to our attention to indicate that the project team deviated from its predefined negotiation process. In addition to observing the contract negotiations, we also compared the business requirements and compensation approach included in the contract to the terms outlined in the solicitation document. We did not identify anything during our review that would cause us to conclude that the project team deviated from the requirements for these areas of the solicitation document. Once the negotiating team and the IBM Group had agreed on a draft contract, the project team submitted a funding request to the federal Office of Child Support Enforcement (OCSE) in order to receive federal funding for the project at the maximum rate of 66 percent. The funding request included the draft contract and a feasibility study developed by the project team. Since the project team only received one proposal, it hired a consultant to develop an alternative for comparison purposes to select the child support enforcement system solution that presented the best value to California’s Child Support Program. This alternative solution involved modifying Texas’s child support enforcement system to meet California’s needs. After comparing this alternative to the IBM Group’s proposal, the project team concluded that the proposed solution from the IBM Group was the most cost-beneficial to the State. Upon reviewing the funding request, the OCSE outlined a few conditions before granting its approval to the project team to execute the contract with the IBM Group. Most significantly, the OCSE stated that it would not provide its share of funding on $98.8 million of the contract cost. This amount represented what it felt were high labor costs and duplicative overhead costs. As a result, the project team presented a contract priced at 22 California State Auditor Report 99028.2 California State Auditor Report 99028.2 33 $801 million to the IBM Group, which the IBM Group accepted. Once the project team resolved this and the other conditions imposed by the OCSE, the OCSE granted permission to the project team to execute the contract. In addition to seeking federal approval, the project team also submitted the feasibility study to the Department of Finance (Finance). Within Finance, the Technology Investment Review Unit (TIRU) is responsible for ensuring that expenditures for state information technology proposals represent a prudent investment of resources while meeting the State’s business needs. Like OCSE, TIRU placed certain conditions on its approval of the feasibility study, requiring, for example, that the project team submit a benefits measurement plan developed in conjunction with Finance within one year following the contract’s signing. As required by the Budget Act, Finance also notified the Legislature that it felt the project was ready to move forward. In response, the Joint Legislative Budget Committee directed Finance to keep the Legislature informed of significant project developments and to provide the Legislature with annual progress reports. Although the project team has now executed the contract for the child support enforcement system, it is still more than a year away from procuring a contractor for the state disbursement unit. By 2008, the project team anticipates full implementation of the single, statewide system. AGENCY COMMENTS The department and the board agree with the information contained in this report and feel that it accurately describes the progress made and the processes used for this procurement. n 22 California State Auditor Report 99028.2 California State Auditor Report 99028.2 33 Blank page inserted for reproduction purposes only. 44 California State Auditor Report 99028.2 California State Auditor Report 99028.2 55 INTRODUCTION BACKGROUND In 1988, Congress passed the Family Support Act (act), a legislative initiative directed at enforcing the payment of court-ordered child support. The act mandated that each state have an operational automated child support enforcement system and established an initial deadline of October 1, 1995, which Congress later extended to October 1, 1997. In the past 10 years, California Bureau of State Audits Reports on the has twice attempted to meet this federal mandate. Child Support System Development As we noted in our previous three audit reports, the 97116—Health and Welfare Agency: Lockheed State failed to do so. This included a $111 million Martin Information Management failure of the Statewide Automated Child Support Systems Failed to Deliver and the State Poorly Managed the Statewide System in 1997. Automated Child Support System (March 1998) California’s failure to develop a single, statewide 98025—Automated Child Support System: automated child support enforcement system by the Selection of Interim System Appears federal deadline has resulted in signifi cant fi nancial Reasonable (November 1998) consequences for the State in terms of reduced 99103—Child Support Enforcement Program: federal funding and fi nes. Specifi cally, the federal Without Stronger Leadership, California’s Child Support Program Will government reduced its funding rate for the costs Continue to Struggle (August 1999) of developing and implementing a system from 90 percent to 66 percent, the rate that the State is currently receiving. Further, in fi scal year 1998–99, the State began accruing federal penalties for having missed the deadline. These penalties started at $11.9 million and have grown to approximately $190 million for fi scal year 2002–03. The Department of Child Support Services (department) estimates that penalties may cumulatively total almost $1.2 billion by federal fi scal year 2006. In an attempt to address this situation, the Legislature passed several laws in 1999 that restructured the way the State conducts its child support enforcement activities. Chapter 478, Statutes of 1999, created the department and transferred responsibility for enforcing child support from the Department of Social Services to the new department. Chapter 479, Statutes of 1999, designated responsibility for procuring, developing, implementing, and maintaining the statewide automated system, now referred to as the California Child Support Automation System (project), to the Franchise Tax Board (board) as the agent for the department. The statute defi nes the roles of these two distinct organizations 44 California State Auditor Report 99028.2 California State Auditor Report 99028.2 55 as being that of an owner (department) and an agent (board). In other words, the department is responsible for procuring a system that meets federal requirements, with the board playing a major role on the department’s behalf. Staff from these two entities make up the project team. The act requires the single, statewide automated child support enforcement system to be an Certain Functional Requirements of the Child Support System Under automated data processing and information Federal Law retrieval system. The accompanying text box includes some, but not all, of the functional • Locating parents or alleged parents who are requirements of this system. The project team not fi nancially supporting their children. planned this procurement in two separate parts: • Establishing paternity through blood the procurement of the main system, known testing and court hearings and assessing the amount of fi nancial support owed. as the child support enforcement system, and a separate procurement for the state disbursement • Processing, tracking, and controlling cases after initiation. unit. The child support enforcement system procurement consists of the design, development, • Billing absent parents regularly for all and implementation of the system, including obligations and collecting, distributing, and disbursing payments. data conversion and system integration. Linked to the child support enforcement system, the state • Monitoring, tracking, and remedying cases with delinquent payments, through such disbursement unit will be a separate centralized means as attaching wages and intercepting system for collecting, disbursing, and recording tax refunds and other income. child support payments.1 • Generating various reports for federal, state, and county managers. For the procurement of the child support enforcement system, the project team used an alternative procurement method, described in detail in Appendix A. According to the board, this method reduces the risk associated with large procurements through risk sharing. For example, if the project does not provide the agreed-upon benefi ts, the State is not responsible for paying the business partner. The State can thus avoid paying for solutions that do not work. As part of its oversight of the procurement process, the federal Offi ce of Child Support Enforcement (OCSE) continues to provide the State with technical assistance in the form of conference calls, on-site visits, and document reviews. Additionally, the federal government required the State2 to have an independent verifi cation and validation team 1This report does not cover the procurement of the state disbursement unit. The project team currently anticipates awarding a contract for this part of the project in fall 2004. 2The federal government required an independent verifi cation and validation team for all states that failed to meet the requirements of the act for a single, statewide automated child support enforcement system. 66 California State Auditor Report 99028.2 California State Auditor Report 99028.2 77 (verification team) to review the processes the State uses to develop and implement the single, statewide automated child support enforcement system. Through its technical assistance and the work of the verification team, the OCSE has reviewed and concurred with or authorized a number of critical documents related to the procurement process. For example, the OCSE reviewed the procurement handbook prior to its release, authorized the State to release the project solicitation document, and approved the criteria used by the project team to evaluate the proposals. Chapter 479 requires the Bureau of State Audits (bureau) to monitor the process for evaluating and selecting a vendor for the State’s single, statewide automated child support enforcement system to determine whether the vendor or vendors were chosen according to the methodology and the criteria contained in the solicitation document. In December 2002, we issued a report on the progress of the project through June 2002, which was prior to completion of contract negotiations. That report described the obstacles the project team faced, including the withdrawal of all but one team of qualified vendors from the competition, resulting in a single proposal from the IBM Group, which consists of IBM, Accenture, and American Management Systems. In the December 2002 report, we concluded that nothing came to our attention during the process used to score the proposal to indicate that the project team had deviated from the evaluation criteria or that it had materially deviated from the predefined evaluation process in a way that might have resulted in unfair treatment of the potential vendors. Appendix B shows a timeline of the key events in the procurement of the child support enforcement system from July 2000 to July 2003, encompassing both the December 2002 report and this report. SCOPE AND METHODOLOGY Chapter 479, Statutes of 1999, requires the bureau to monitor the process of evaluating and selecting a vendor for the project to determine whether the evaluation is based on the criteria contained in the solicitation document. Further, it requires the bureau to monitor the process to determine whether the project team chose the vendor or vendors according to the methodology in the solicitation document and to determine whether it did so without bias or favoritism toward any bidder. Our previous report, issued in December 2002, focused on the main part of the system, referred to as the child support 66 California State Auditor Report 99028.2 California State Auditor Report 99028.2 77 enforcement system, and described our observations of the vendor qualification process, the review of the final proposal, and the non-confidential and confidential discussions between the qualified business partners and the project team. This report contains our observations of the remaining stages of the procurement process for the child support enforcement system. Because the statute does not require us to go beyond reviewing the procurement process, we did not evaluate or analyze areas outside of the defined scope, such as whether the State’s contract with the IBM Group will result in the best value for the State. To monitor the contract negotiations between the project team and the IBM Group, we attended negotiation sessions and reviewed the composition of the project’s negotiating team. To determine whether the contract deviated from certain requirements of the solicitation document, we compared the compensation approach agreed upon by the parties to the compensation approach outlined in the law and the solicitation document. Further, we contracted with a consultant to compare and trace a sample of the business requirements listed in the solicitation document to the contract and vice versa. Since the federal government— specifically, OCSE and its verification team—performed a detailed review of the State’s feasibility study of viable options, we relied on their work in this area. Finally, we followed the contract through state and federal approval processes. The procurement process for the state disbursement unit, the second part of the single, statewide automated child support enforcement system, is in the early stages. As required by the statute, we will continue to monitor the evaluation and selection process for the state disbursement unit. n 88 California State Auditor Report 99028.2 California State Auditor Report 99028.2 99 AUDIT RESULTS THE NEGOTIATION PROCESS REDUCED THE PROJECT’S COST AND RESULTED IN A CONTRACT THAT APPEARS TO MEET THE CRITERIA OUTLINED IN THE SOLICITATION DOCUMENT As discussed in the Introduction, in December 2002, we reported the State’s progress toward a single, statewide automated child support enforcement system prior to June 2002, when the California Child Support Automation System (project) team began to negotiate with the IBM Group for the procurement of the project’s main part, the child support enforcement system. After determining that the proposal might include a risk premium for the IBM Group, the project’s negotiation team negotiated a reduction in price from approximately $1.2 billion to $900 million. In our monitoring of contract negotiations, we found that the project team appeared to generally adhere to the predefined negotiating process outlined in its negotiation plan. Additionally, nothing came to our attention to indicate that the project team excluded from the contract the business requirements of the solicitation document. Furthermore, the compensation method specified within the contract did not violate Chapter 479, Statutes of 1999, and were within the parameters defined in the solicitation document. The Negotiations Resulted in a $300 Million Reduction of the Project’s Costs The project’s negotiation team consisted of a main negotiation team, an executive management team, and a support team, which included subject matter experts and legal counsel. The teams included individuals from the Franchise Tax Board (board) with prior experience in negotiating performance- based information technology contracts; staff from the board and the Department of Child Support Services (department) with backgrounds in information technology and the child support program; consultants who participated in the independent verification and validation review of the previous statewide automation efforts; legal counsel from an outside firm with expertise in governmental information technology contracts; and a representative of the counties that will use the single, statewide automated child support enforcement system. 88 California State Auditor Report 99028.2 California State Auditor Report 99028.2 99 As noted in our previous report, the only bid the project team received in response to its solicitation document was from the IBM Group, with a contract price of almost $1.2 billion. A cost reasonableness Following the example set in another competitive procurement study of the IBM Group in which the State received only a single proposal, the project proposal suggested that team hired a consultant to examine the reasonableness of the certain costs may contain cost set forth in the IBM Group’s bid. Although the consultant risk premiums of up to found that the proposed costs were sufficiently reasonable to 25 percent. begin contract negotiations, the consultant’s analysis suggested that some of the costs for development and implementation of the proposed system might contain risk premiums of up to 25 percent. Equipped with its consultant’s information, the project’s negotiation team used a negotiation strategy that resulted in a contract price of $900 million. The Project Team Did Not Deviate From the Requirements of the Solicitation Document in Its Negotiation Process, Business Requirements, and Compensation Plan In our monitoring of the negotiation sessions, we did not observe anything that would lead us to believe that the negotiations resulted in significant changes in the contract that might violate the requirements outlined in the law or the solicitation document. The negotiation team also appeared to generally adhere to the predefined process set for contract negotiations. Our conclusions are solely based on our observations during the negotiation process that included conversations and discussions. We also found that the contract included the business requirements—the major business need areas that the child support enforcement system must address—as outlined in the solicitation document. We compared a sample of the business requirements in the solicitation document to those in the contract, then reversed the process, comparing a different sample from the contract to those in the solicitation document. The items matched without exception, indicating that the contract included these requirements. Finally, we found that the compensation method agreed to by the parties was consistent with the law and was within the parameters defined in the solicitation document. Chapter 479, Statutes of 1999, describes the foundation of the compensation methodology that the project team must use for its contract with a business partner. The law required the State to develop a procurement plan that would specify business outcomes to be achieved, maintain maximum vendor commitment to the project’s success, and minimize risks to the State by 1100 California State Auditor Report 99028.2 California State Auditor Report 99028.2 1111 sharing it with the private sector. Moreover, it requires that the State base payments to the business partner primarily on the achievement of performance measures defi ned in its procurement plan. However, it also allows According to the solicitation document, the project team to compensate the business the child support enforcement system partner for agreed-upon deliverables and to make must address the following fi ve additional payments to the business partner for business problems: predefi ned levels of higher performance once the • Certifi cation—The State is not in system is operational. Thus, the law allows for compliance with the automated system both performance-based payments and deliverable- requirements of the Federal Family Support based payments, with an emphasis on achieving Act of 1988 and Personal Responsibility and Work Opportunity Reconciliation Act of 1996. predefi ned levels of performance. • Worker Effectiveness—The lack of timely, adequate, and accurate data; the lack of The project team incorporated a similar uniformity; and limited system functionality compensation methodology into its solicitation and automation are constraining worker document. The solicitation document, authorized effectiveness and accountability. for release by the federal government, stated that • Customer Service—The lack of timely, although qualifi ed business partners were expected adequate, accurate, and accessible data; clear communication of information; to use the specifi ed compensation method as a and uniformity combined with limited basis, they could propose alternative performance system functionality and automation are constraining customer service. measures for the project team to consider. The solicitation document also described the process • System Maintainability—Disparate the project team will use to review the deliverables systems, inadequate documentation, the lack of open system architecture, provided by the business partner to measure and business changes are constraining progress and performance. It stated that the team system maintainability. will consider one or more of the fi ve business • System Implementation—Disparate problems in the solicitation document (listed in systems, complex business rules, number of locations, and changes in the way the accompanying text box) solved when it accepts of doing business are constraining new specifi c deliverables and judges that the business system implementation in the State. partner has achieved corresponding performance levels. This approach ties performance measures to the strategic business problems as a method of assessing value, thereby allowing the State to base the business partner’s compensation on realized value. The solicitation document identifi ed three categories of performance measures: • Management performance—measures the business partner’s project management and system development activities. • Program performance—measures the business value added to the State’s Child Support Program and the achievement of specifi c business objectives. • System performance—measures the technical merit of specifi c system capabilities. 1100 California State Auditor Report 99028.2 California State Auditor Report 99028.2 1111 The solicitation document also provided examples of how to measure these performance categories. The project team can measure management performance through timely acceptance of all deliverables listed in the contract and program performance by certification and business improvements such as increases in establishment of paternity and the collection of child support. The project team can measure system performance by the timely and regular clearance of all significant data and operational defects. The contract generally includes the same compensation methodology outlined in the solicitation document. It bases compensation to the IBM Group on value realized by the State, through the performance measures discussed above tied to the five business problems. This should result in payment to the IBM Group only if and when it achieves the performance measures. In other words, instead of paying the IBM Group for developing a system that meets requirements or for producing project deliverables, the State should pay for the development and implementation of a child support enforcement system that solves business problems. DESPITE CONCERNS, THE PROJECT TEAM RECEIVED FEDERAL AND STATE APPROVAL TO EXECUTE THE CONTRACT Once the project team and the IBM Group had agreed upon a draft contract for the child support enforcement system, the project team submitted various documents and studies to the Although the federal federal Office of Child Support Enforcement (OCSE) and the state Office of Child Support control agencies to obtain funding and approval. In the process of Enforcement and the their reviews, both OCSE and the State Department of Finance State Department of (Finance) raised a number of concerns, including the accuracy Finance identified several of the project’s feasibility study, the compensation model in concerns regarding the the contract, and the terms of the child support enforcement feasibility study, both system’s warranty. Despite these concerns, the agencies felt felt that they were not that they were not significant enough to prevent the project significant enough to from moving forward. However, OCSE specifically disallowed prevent the project from federal financial participation funding on $98.8 million, or moving forward. 11 percent, of the total negotiated contract costs of $900 million, stating that this amount represented overly high labor costs and duplicative overhead costs. In response, the project team presented the IBM Group a contract price of $801 million, which the IBM Group accepted. Upon receiving the approvals from the OCSE and Finance, the project team signed the contract with the IBM Group on July 14, 2003. 1122 California State Auditor Report 99028.2 California State Auditor Report 99028.2 1133 The OCSE and the Independent Verification and Validation Team Identified Several Concerns and Observations In order for the State to receive federal funding annually, it must submit a funding request to the OCSE. To obtain the OCSE’s approval for the design, development, and implementation of the child support enforcement system, the project team submitted the contract and a feasibility study to the OCSE along with the request for funding. The purpose of the feasibility study, which the project team developed, was to clarify the business problems and solution requirements, to identify alternative solutions, and to perform a cost-benefit analysis to select the solution that presented the best value to California’s Child Support Program in California. As discussed in our December 2002 report, the project team expected the competitive procurement approach it used for the child support enforcement system to generate multiple viable alternatives for consideration in the feasibility study, but instead it received only the IBM Group’s proposal. Under the direction of the OCSE, the project team, therefore, considered another option for use in the feasibility study. The project team did not consider the State’s current system, under which the counties consolidate their automation efforts onto six federally approved The project team’s consortia-based systems, as an alternative for its study because it feasibility study concluded would not meet the federal certification requirements and would that the solution proposed result in the State being subject to the accumulation of penalties. by the IBM Group offered Instead, it hired a consultant to consider an alternative system more significant benefits for California by modifying an existing system from another to California, to the federal state—in this case, Texas—to address the requirements contained government, and to other in the solicitation document. After comparing the modified Texas states through increases in child support system to the IBM Group’s proposal, the project child support collections. team found that the latter offered more significant benefits to California, to the federal government, and to other states through increases in child support collections. Both the OCSE and the independent verification and validation team (verification team) reviewed the feasibility study to assess the accuracy, consistency, measurability, repeatability, and reasonableness of the methodology and data that the project team had employed in identifying the IBM Group’s proposal as more beneficial. Their reviews identified a number of errors in the mathematical calculations related to various elements of costs and benefits in the two proposed alternatives. Additionally, they questioned a number of assumptions and constraints in the study. For example, the verification team identified a logic error: The feasibility study included benefits realized from the IBM Group’s 1122 California State Auditor Report 99028.2 California State Auditor Report 99028.2 1133 proposed system immediately after deployment into the counties rather than gradually as users became familiar with it. The OCSE deferred to the project team to determine if any modifications to the feasibility study were necessary to address the verification team’s findings. In response, the project team performed an analysis of the verification team’s review that revalidated the feasibility study. Based on this analysis, the project team concluded that the feasibility study was valid, supportable, and established the best course for California’s statewide automated system. The verification team also observed that the compensation model in the contract, which we previously discussed at greater length, had changed significantly from the model found in the original solicitation document. While the verification team did not consider these observations direct findings or recommendations, it reported that the model went from one that relied heavily on measurable program and system performance to one that focuses on project tasks and deliverables. The verification team believes the negotiated compensation model also awards compensation much sooner than the model presented in the solicitation document. It contends that the delayed payment to vendors in the original solicitation document was a key reason for potential business partners to drop out of the bidding process, thus leading to a single bid proposal. In response to this concern, the project team states that it had made clear to the qualified business partners during confidential discussions that the model described in the solicitation document was provided as an illustration. The business partners could propose a compensation model that differed from the solicitation document; however, a proposal that was less performance-based would receive a lower score in the respective area. The OCSE Granted Approval With Certain Conditions The OCSE granted the team conditional approval on May 29, 2003, to execute the contract with the IBM Group. However, it did Because OCSE disallowed impose one major condition, disallowing federal financial federal financial participation funding on $98.8 million, or 11 percent, of the participation on total negotiated contract costs of $900 million. The OCSE $98.8 million of the identified this amount as resulting from high labor costs and contract costs, the project duplicative overhead costs. In response, the project team team presented and the presented the IBM Group a contract price reflecting the decrease, IBM Group accepted a or $801 million. The IBM Group accepted the reduced contract reduced contract price of price, which allowed the State to remain eligible to receive $801 million. federal financial participation of 66 percent on the entire contract costs. During these negotiations, the project team 1144 California State Auditor Report 99028.2 California State Auditor Report 99028.2 1155 stated that it did not reduce the scope of services in the contract. Furthermore, nothing came to our attention to indicate that the project team decreased the scope of services. The OCSE imposed other conditions on its approval as well. Specifically, it questioned the absence of a performance bond during the first 10 months of the contract even though To satisfy OCSE’s concern, the contract included payments to the IBM Group totaling the project team will approximately $100 million within the first 12 months. The provide a $25 million project team responded by indicating it would provide a performance bond to $25 million performance bond to cover the first 10 months of cover the first 10 months the contract, which satisfied the OCSE’s concern. The OCSE of the contract. also questioned the short warranty period—45 state business days—for such a complex system, particularly compared to other state projects that had warranty periods of one or more years. The project team responded that a one-year warranty period is appropriate in cases where a buyer is purchasing a product “sight unseen” or without an opportunity to test its performance prior to purchase. However, in this instance, the project team asserts that the contract does not require the State to pay the entire cost of the child support enforcement system until it achieves system and program performance measures and system defects have been cleared. Moreover, the 45-day warranty period begins upon acceptance of the system, which is after the end of the 24-month implementation period that takes the State from a successful pilot to statewide operation. After the project team satisfied the conditional approval items presented by the OCSE, the OCSE granted the project team permission on June 25, 2003, to execute the contract with the IBM Group. The State Department of Finance Concluded That the Project’s Benefits Outweighed Potential Areas of Concern In order to execute the contract, the project team needed to obtain state as well as federal approval. Within Finance, the Technology Investment Review Unit (TIRU) is responsible for reviewing information technology proposals to ensure that they represent a prudent investment of resources while meeting the State’s business needs. After reviewing the feasibility study for the project, TIRU identified a number of areas of concern. In a letter to the project team, it expressed concern about the initial use of the IBM data center and technical help desk rather than the State data center, the lack of definition in the plans for Customer Service Support Centers, the possibility of future costs related to 1144 California State Auditor Report 99028.2 California State Auditor Report 99028.2 1155 activities such as the cleanup of legacy data before conversion and the interface to other data sources outside of the department, and the inability to validate the estimated project benefits. TIRU also stated that it believed these concerns should not prevent the project from moving forward. Therefore, it approved the contract expenditures subject to several conditions. For example, the project team estimated that over the 10-year life of the contract, the system would yield benefits of approximately $5.1 billion. It estimated these benefits would result from increased child support collections, decreased operating costs, and the elimination of federal penalties. Because TIRU did not have an analytical basis to validate these estimated benefits, it is requiring the project team to submit a benefits measurement plan developed in conjunction with Finance within one year of the signing of the contract. In addition, TIRU is requiring the project team to submit within 24 months of signing the contract a transition plan for the State to assume responsibility for the maintenance and operations of the system at the end of the contract. Among other things, the plan must include the final location of the data center and the help desk. In addition to these technical aspects of the project, Finance identified concerns regarding the extent to which the terms The chief counsel of of the contract with the IBM Group protected the State. For Finance and the interim example, Finance was concerned with the appropriateness of director of General Services the performance-based compensation model, adequacy concluded that the of the performance bond, limitation on vendor liability, compensation provisions and sufficiency of the warranty period for the child support of the contract provide a enforcement system. As a result, Finance requested its chief satisfactory mechanism counsel and the interim director of the Department of for the State to manage General Services (General Services) to review the contract and performance risks as its provisions to examine the allocation of risk between the long as the project team State and the IBM Group. The chief counsel of Finance and conducts timely and the interim director of General Services concluded that the rigorous assessments of the compensation provisions of the contract provide a satisfactory performance requirements. mechanism for the State to manage performance risks as long as the project team conducts timely and appropriately rigorous assessments of the performance requirements. Moreover, they concluded that the warranty provisions of the contract are adequate to protect the State’s interests. 1166 California State Auditor Report 99028.2 California State Auditor Report 99028.2 1177 The Joint Legislative Budget Committee Requested That Finance and the Project Team Periodically Report on the Project’s Progress Control Section 11.00 of the Budget Act requires state agencies and departments to obtain approval from Finance and provide the Legislature at least a 30-day advance notification prior to executing information technology contracts that exceed $500,000 or more in total costs that meet certain criteria. Besides providing Finance with the feasibility study and related documents, the board, as the agent for the project, also provided Finance with a Control Section 11.00 application for the contract with the IBM Group. On June 13, 2003, Finance notified the Joint Legislative Budget Committee (committee) that the project was ready to move forward. In response, the committee instructed Finance to keep the Legislature apprised of significant project developments. Further, the committee requested the board, the department, and Finance to provide jointly prepared annual progress reports during the legislative budget hearing process until the project is completed. Finally, the committee urged Finance to ensure the project team fully justifies requests for additional funding by providing detailed information documenting the link between the request and successful, timely project development and implementation. THE PROJECT TEAM HAS YET TO AWARD A CONTRACT FOR THE STATE DISBURSEMENT UNIT While the project team has successfully solicited, negotiated, and executed a contract for the child support enforcement system, it is over a year away from awarding a contract for the state disbursement unit. As discussed in the Introduction, this unit, although linked to the child support enforcement system, will be a separate centralized system for collecting, disbursing, and recording child support payments. Currently, the project team plans to award a contract for the state disbursement unit in fall 2004 and anticipates full implementation of the single, statewide system in 2008. 1166 California State Auditor Report 99028.2 California State Auditor Report 99028.2 1177 We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, ELAINE M. HOWLE State Auditor Date: September 24, 2003 Staff: Debbie Meador, CPA Phillip Burkholder, CPA Gayatri Patel 1188 California State Auditor Report 99028.2 California State Auditor Report 99028.2 1199 APPENDIX A The Procurement Process Chapter 479, Statutes of 1999, requires the Franchise Tax Board (board) to serve as the Department of Child Support Services’ (department) agent in procuring and developing a single, statewide automated child support enforcement system, and it further required that the board employ techniques proven to be successful in its previous tax automation efforts. It also required the board to develop a procurement plan that included, but was not limited to, elements that accomplish the following: • Provide for full and open competition among qualified vendors. • Specify the goals the project needs to accomplish, not how to accomplish those goals. • Maintain maximum vendor commitment to the project success and minimize the risk to the State. • Use evaluation methods that select the best solution based on business performance measures, not necessarily the lowest price. • Consider the future ability of the selected system to provide enhancements that will improve the long-term effectiveness of child support management. • Base payments to the vendor or vendors primarily on achieving predefined performance measures. The project team addressed the key provisions in its procurement strategy by using a performance-based contracting approach, an alternative form of procurement that the board has used successfully in the past for tax automation projects. According to a 1998 presentation by the board, this procurement approach lessens the risk associated with large procurements through risk sharing. While both the traditional procurement approach and the performance-based approach are designed to solicit bids from multiple vendors, the performance-based contracting approach 1188 California State Auditor Report 99028.2 California State Auditor Report 99028.2 1199 differs from a traditional procurement in several ways. Figure A.1 illustrates the two approaches, using color to identify steps that are comparable. For example, according to the board, instead of focusing on business and technology requirements, the performance-based contracting approach seeks to obtain proven solutions to business problems and to identify new and innovative ways of achieving business goals. Further, the performance-based contracting approach results in solutions that provide the best value to the State rather than simply the lowest cost. This is achieved through an objective method that critically reviews and competitively scores each solution based upon the merits of the solution, the risk of implementing the new technology and work processes, and the net benefit produced. Moreover, under the performance-based contracting approach, a vendor receives payment only if and when it implements the solution and realizes its benefits. Theoretically, if a vendor does not receive payment until the system works and generates benefits, it is more likely to remain on task until the system achieves the desired business goals. Another difference between the two approaches resides in the development of the feasibility study. In the traditional model, a feasibility study is one of the first steps, and it is used to justify the approach the State will take on a project, generally serving as a basis for the request for proposals. The state agency normally performs the analysis and documents the requirements for the feasibility study in isolation, before receiving the vendors’ proposed solutions. In contrast, under the performance- based contracting approach, the project team develops the feasibility study after evaluating the proposals received from potential vendors. In other words, in the traditional approach the feasibility study generally drives the solution identified in the solicitation document, whereas in the performance-based contracting approach the solutions that vendors propose are used as viable options for the feasibility study. 2200 California State Auditor Report 99028.2 California State Auditor Report 99028.2 2211 FIGURE A.1 Comparison of Procurement Models Alternative Procurement Steps/ Traditional Procurement Steps/ Performance-Based Contracting Approach Stages (Request for Proposal) Stages (Solicitation for Conceptual Proposal) 1 Approval of feasibility study report 1 Approval of alternative procurement business justification 2 Advertise and request for interest 2 Advertise and request for interest 3 Release request for proposal 3 Identify qualified business partners (QBPs) 4 Receive intent to bid 4 Hold nonconfidential discussions 5 Hold bidder’s conference 5 Release solicitation document to QBPs 6 Negotiate contract 6 Hold QBP conference 7 Bidders submit conceptual proposal, detailed 7 Receive intent to respond technical proposal 8 Hold confidential discussions 8 Hold confidential discussions with QBPs 9 Bidders submit draft proposal 9 QBPs submit draft proposal 10 Bidders submit final proposal 10 Hold confidential discussions with QBPs 11 Evaluate technical response 11 QBPs submit final proposal 12 Cost opening 12 Evaluate technical response 13 Evaluate and select winning bidder 13 Cost opening 14 Issue letter of intent to award 14 Evaluate and select winning QBP 15 Award contract 15 Debrief partners on selection rationale 16 Debrief bidders on selection rationale 16 Issue notice of intent to enter into contract negotiations 17 Begin development and implementation 17 Negotiate contract 18 Approval of feasibility study 19 Issue letter of intent to award 20 Award contract 21 Begin development and implementation Source: California Child Support Automation System Project. 2200 California State Auditor Report 99028.2 California State Auditor Report 99028.2 2211 Blank page inserted for reproduction purposes only. 2222 California State Auditor Report 99028.2 California State Auditor Report 99028.2 2233 APPENDIX B Timeline of Key Events Figure B.1 on the following page is a timeline of key events and tasks completed during the three-year procurement process for the main part of the single, statewide automated child support enforcement system, referred to as the child support enforcement system. It starts in July 2000 and ends in July 2003 with the State signing a contract with the IBM Group to design, develop, and implement the child support enforcement system. 2222 California State Auditor Report 99028.2 California State Auditor Report 99028.2 2233 2244 California State Auditor Report 99028.2 California State Auditor Report 99028.2 2255 FIGURE B.1 Timeline of Key Events �� Source: California Child Support Automation System Project. Agency’s comments provided as text only. Health and Human Services Agency 1600 9th Street, Room 460 Sacramento, CA 95814-6404 September 15, 2003 Elaine M. Howle State Auditor 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: Thank you for your efforts in monitoring the procurement for the California Child Support Automated System procurement. This effort is of vital importance to the state, not only because of the significant penalties that the state pays because of the lack of a statewide system, but also because of the improvements in service to families that we can expect to achieve by implementing a new automated system. Your role in monitoring the procurement has served to ensure that the procurement is not delayed due to controversies over procedure that can arise large procurements. I am pleased that the Department of Child Support Services and the Franchise Tax Board have successfully completed the procurement effort for the Child Support Enforcement element of the system. I look forward to the successful completion of the State Disbursement Unit procurement. Sincerely, (Signed by: Grantland Johnson) GRANTLAND JOHNSON 2244 California State Auditor Report 99028.2 California State Auditor Report 99028.2 2255 California Department of Child Support Services P.O. Box 419064 Rancho Cordova, CA 95741-9064 September 10, 2003 Ms. Elaine M. Howle State Auditor 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: SUBJECT: CHILD SUPPORT ENFORCEMENT PROGRAM: THE STATE HAS CONTRACTED WITH IBM TO DEVELOP AND IMPLEMENT THE MAJOR COMPONENT OF THE STATEWIDE AUTOMATED CHILD SUPPORT SYSTEM We concur with the finding of your report on the procurement that we recently completed for the Child Support Enforcement component of the California Child Support Automation System. Your findings reflect accurately the complexity of the procurement, the extensive reviews and analysis the proposed contract has been subjected to, and the important decisions faced by the state in managing the procurement. The success of this procurement is critical to the state’s getting relief from federal penalties. Thank you for the extensive efforts that you and your staff have made in monitoring the procurement. Your staff have observed all of the important activities that we have undertaken. We have welcomed your careful monitoring of the process to provide independent validation of the fairness of the approach that the Franchise Tax Board and my department took in conducting this procurement. We look forward to your continued access to the remaining procurement effort, the selection of a contractor for the State Disbursement Unit. Your independent assessment of that effort will be important to the success of the effort. Sincerely, (Signed by: Curtis L. Child) CURTIS L. CHILD Director 2266 California State Auditor Report 99028.2 California State Auditor Report 99028.2 2277 Agency’s comments provided as text only. State and Consumer Services Agency Office of the Secretary 915 Capitol Mall, Suite 200 Sacramento, CA 95814 September 12, 2003 Elaine Howle, State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: Attached is the audit response prepared by the Franchise Tax Board on the Bureau of State Audits’ Report No. 99028.2 entitled Child Support Enforcement Program: The State Has Con- tracted With IBM to Develop and Implement the Major Component of the Statewide Automated Child Support System. If you have any questions or need additional information, please contact me at 653-4090. Sincerely, (Signed by: George Valverde) George Valverde Deputy Secretary Attachment 2266 California State Auditor Report 99028.2 California State Auditor Report 99028.2 2277 Franchise Tax Board PO Box 115 Sacramento, CA 95741-0115 M E M O R A N D U M To: Aileen Adams, Secretary September 11, 2003 State and Consumer Services Agency 915 Capitol Mall, Suite 200 Sacramento, CA 95814 From: Gerald H. Goldberg Subject: Child Support Enforcement Program: The State Has Contracted With IBM to Develop and Implement the Major Component of the Statewide Automated Child Support System, September 2003 We agree with the information contained in the above audit report issued by the Bureau of State Audits (BSA) and feel that it accurately described the contracting processes used for this procurement. We appreciate the BSA involvement in this procurement process. (Signed by: Gerald H. Goldberg) Executive Officer 2288 California State Auditor Report 99028.2 California State Auditor Report 99028.2 2299 cc: Members of the Legislature Office of the Lieutenant Governor Milton Marks Commission on California State Government Organization and Economy Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press 2288 California State Auditor Report 99028.2 California State Auditor Report 99028.2 2299