Summary

Department of Education: Lax Monitoring Led to Payment of Unsubstantiated Adult Education Claims and Changes in the Program May Seriously Impact Its Effectiveness July 1999 98113 The first copy of each California State Auditor report is free. Additional copies are $3 each. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TDD (916) 445-0255 x 248 OR This report may also be available on the World Wide Web http://www.bsa.ca.gov/bsa/ Permission is granted to reproduce reports. C S A ALIFORNIA TATE UDITOR KURT R. SJOBERG MARIANNE P. EVASHENK STATE AUDITOR CHIEF DEPUTY STATE AUDITOR July 20, 1999 98113 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders:, the Bureau of State Audits presents its audit report concerning the Department of Education(cid:146)s (department) management of the federal adult education program. This report concludes that the department(cid:146)s monitoring efforts failed to detect unsubstantiated claims for adult education funds submitted by community-based organizations. In addition, the department(cid:146)s award process yielded inconsistent decisions that leave it open to charges of favoritism. The department is making significant changes to the adult education program. However, the changes may not resolve past problems and could result in a drop in the number of students served. Respectfully submitted, KURT R. SJOBERG State Auditor BUREAU OF STATE AUDITS 660 J Street, Suite 300, Sacramento, California 95814 Telephone: (916) 445-0255 Fax: (916) 327-0019 CONTENTS Summary 1 Introduction 3 Chapter 1 The Department of Education Failed to Detect Cases Where Claimed Services Were Not Substantiated 9 Recommendations 17 Chapter 2 Changes the Department of Education Proposes to Correct Its Flawed Oversight of Adult Education Funding May Provide More Consistency, but May Also Diminish Services 19 Recommendations 30 Responses to the Audit California Department of Education R-1 California State Auditor’s Comments on the Response From the California Department of Education R-15 SUMMARY RESULTS IN BRIEF T he California Department of Education (department) has mismanaged oversight of the federal adult education program in the State. As a result of its ineffective monitor- ing practices, the department has paid community-based Audit Highlights . . . organizations (CBOs) for services these organizations cannot substantiate. Although the department claims it monitors its Our review revealed that the service providers, 8 of 10 CBOs we reviewed could not document Department of Education the number of class hours for which they received federal funds mismanaged its oversight of the federal adult education in at least one of the last five years, and none could consistently program. As a result, demonstrate gains in skill for the students they claimed to have several community-based educated. In addition, the department’s grant award process organizations could not yielded inconsistent decisions that leave the department open to support services for which they were paid. charges of favoritism. Furthermore, although it is The department is making significant changes to the adult making significant changes to the program, past education program for fiscal year 1999-2000; however, these monitoring problems may changes may not resolve past problems and could even create not be resolved and new new ones. The new award grading process, though an problems may be created. improvement on the old method, must still be implemented. Additionally, drafted monitoring procedures do not include steps to determine whether claimed services are documented. Meanwhile, new eligibility requirements and payment rates, intended to improve the accountability of service providers, could result in a drop in the number of students served— especially students with the least skills who most need these educational services. Finally, the department will need to reas- sess its new rate structure and award process because earmarking of federal adult education funds in the State’s fiscal year 1999-2000 budget is much more restrictive than the department expected. RECOMMENDATIONS To ensure that service providers maintain appropriate evidence to support their claims for payment, the department should do the following: C A L I F O R N I A S T A T E A U D I T O R 1 • Establish strict guidelines for service providers to document student testing and hours of instruction to deter easily falsi- fied evidence. • Place a high priority on developing a battery of interchange- able tests for measuring gains in skill to avoid falsification of evidence or teaching to the test questions. • Design monitoring procedures to test support for claimed services, including review of attendance records, summary documents, and tests showing attainment of benchmarks. To ensure that its award decisions are consistent and fair, the department should do the following: • Hold all applicants accountable for submitting required information, including audit reports, to qualify for funding. • Evaluate funding requests in light of prior-year performance and the size of the service provider before authorizing grant awards. • Review a sample of fiscal year 1999-2000 awards to ensure that decisions to award or deny funds are consistent and defensible. To ensure that it reaches the greatest number of students and maximizes the use of federal adult education funds, the department should evaluate the impact that changes in the program will have on students and service providers. If, as we anticipate, this evaluation shows that fewer students will be served, the department needs to develop strategies to encourage program expansion. AGENCY COMMENTS The California Department of Education expresses concern with the perspective presented in our report and believes it should be given more credit for changes it has undertaken since March 1998. It also takes exception with the accuracy of some of our findings. It does not, however, disagree with any of our recommendations. n 2 C A L I F O R N I A S T A T E A U D I T O R INTRODUCTION BACKGROUND S ince 1966, California has participated in a federal grant program to expand educational opportunities for adults under the Adult Education Act. The California Department of Education (department) administers the program, and local agencies provide instructional services. This program is distinct from the much larger state-funded adult education program, which the department also administers. The department devel- ops state plans to guide the use of federal program funds. During the early 1990s, the major components of the program were English as a second language (ESL) and adult basic education targeted to adults whose native language is English. Instruction focused on improving the computational, problem-solving, and language abilities of adults with skills below the ninth-grade level. During this period, school districts, community colleges, and job training programs served the vast majority of adult education students. During the late 1980s and early 1990s, the department also used State Legalization Impact Assistance Grants from the federal government to fund local agencies delivering instruction and services to help immigrants obtain legal status. Private, non- profit entities, referred to as community-based organizations (CBOs), furnished a portion of these services. When the grants were reduced, and then eliminated, in fiscal year 1992-93, many CBOs were forced to dramatically reduce their programs. In response, the state Legislature enacted Senate Bill 1764, which set aside $3.0 million of fiscal year 1992-93 federal adult education funds for CBO programs. Due to a federal provision requiring equal access for all organizations to any of these federal funds, the U.S. Department of Education refused to authorize a waiver allowing the department to implement this set-aside, and adult education programs remained unchanged. For fiscal year 1994-95, the Legislature earmarked $7.7 million in federal adult education funds for citizenship training and services. This time, however, it excluded language referring specifically to CBOs. In response, the department added a third major program component, ESL-citizenship instruction. After discussions with local agencies, it drew up administrative C A L I F O R N I A S T A T E A U D I T O R 3 requirements and a rate structure for the new component. It also gained the approval of the U.S. Department of Education to use federal adult education funds for ESL-citizenship instruction in fiscal year 1994-95. Although the department added numerous requirements for local agencies that offered ESL-citizenship instruction, it also set much higher funding rates than it did for the other two components. The department chose to pay $2.50 per hour of student attendance for the new component while it offered only 16 cents per hour for adult basic education and ESL instruction. These funding rates changed slightly, but they never exceeded 16 cents per hour and the ESL-citizenship funding rate remained at $2.50 per hour. Each year since fiscal year 1994-95, the state Legislature has continued to set aside adult education funds for citizenship instruction, and the department has gained yearly approvals from the U.S. Department of Education for technical changes to the state plan. The total earmarked for ESL-citizenship instruc- tion through fiscal year 1997-98 amounted to around $35.6 million, or 31 percent, of the $115.4 million in federal adult education funds paid to California adult education providers. The department in turn awarded 49 percent of funds earmarked for ESL-citizenship instruction to CBOs during these four fiscal years, as shown in Figure 1. CBOs, however, received only 11 percent of federal funds for other adult education components. 4 C A L I F O R N I A S T A T E A U D I T O R FIGURE 1 Allocation of Federal Adult Education Funds to CBOs Fiscal Years 1994-95 Through 1997-98 Other Federal Adult Education Program Components ESL-citizenship 69% 31% 11% CBOs CBOs 49% 89% 51% Other Providers Other Providers In the wake of the 1996 defeat of a long-term U.S. representative from Orange County, media reports linked a prominent CBO to voter fraud related to noncitizens. In turn, the Office of the Inspector General of the U.S. Department of Education and the Federal Bureau of Investigation (FBI) launched an investigation of 10 CBOs funded by the department. These investigations are still underway. Recent actions at both the state and federal levels will signifi- cantly impact the adult education program for fiscal year 1999-2000. Congress passed the Adult Education and Family Literacy Act in 1998, and these significant changes will take effect July 1, 1999. In response to the new act, the department toughened eligibility requirements for adult education service providers. It also changed the basis of payment from hours of attendance to achievement of standardized skill benchmarks and has flattened its fee structure so that all components are reimbursed at the same rate. C A L I F O R N I A S T A T E A U D I T O R 5 DEPARTMENTAL STRUCTURE The names of the department’s various units have changed since fiscal year 1994-95, but their basic structure remains the same, as shown in Figure 2. Office managers and their staff carry out day-to-day operations of the adult education program while the division director and branch deputy superintendent generally take part in award decisions and some payment approvals. FIGURE 2 Department of Education Units Involved in Administering the Federal Adult Education Program January 1999 Department of Education Legal and Audits (State Superintendent) (Chief Counsel) (Chief Deputy Superintendent) Audits and Education Equity, Access & Investigations Support Branch Division Deputy Superintendent (Manager) Education Support Systems Division Director Adult Education Office (Manager) Management of the adult education program experienced significant turnover in the last five years. In fact, two different deputy superintendents have managed what is now the Educa- tion Equity, Access, and Support Branch; four directors have managed what is now the Education Support Systems Division; and the Adult Education Office is on its second manager. The functionally independent Audits and Investigations Division (division) reviews the annual audit reports of service providers and conducts its own reviews and audits. This division is also responsible for internal audits of the department. In 1997, at the request of the Adult Education Office, the division con- ducted a review of award, monitoring, and policy-setting processes related to the adult education program. 6 C A L I F O R N I A S T A T E A U D I T O R SCOPE AND METHODOLOGY The Joint Legislative Audit Committee asked the Bureau of State Audits to evaluate the State’s administration of the adult educa- tion program. Specifically, we were requested to determine whether the department has effective processes for awarding and monitoring service providers and whether CBOs can prove they furnished services as claimed. We were also asked to deter- mine the extent to which state and federal agencies investigated allegations of the misuse of funds. Additionally, we were asked to review the appropriateness of changes the department has made to the adult education program in light of annual budget act requirements and the state plan for the program. Our audit focused on the department’s administration of federal adult education funds and on the CBOs’ use of these funds. We did not review the administration of the state adult education program since it does not include CBOs. To determine legal requirements governing the adult education program, we reviewed the applicable federal laws and regulations and the state plan. We also interviewed staff at the department to gain an understanding of how the department awards grants and monitors service providers. We visited 10 CBOs granted funds from fiscal years 1994-95 through 1998-99 and reviewed department records to determine if the department consistently followed award and monitoring procedures. We selected 5 of these from a group of 10 for which the department provided records under a federal subpoena related to the previously mentioned federal investigation. We refer to these 5 as “questioned CBOs.” We selected the other 5 from all other participating CBOs. Our sample represents a cross section of all CBOs according to size, geographic location, and ethnicity of population served. These 10 received 69.6 percent of all adult education funds awarded to CBOs between fiscal years 1994-95 and 1997-98, as shown in Figure 3. We reviewed their records to determine if they adequately supported the services they claimed to have provided and for which they were paid. C A L I F O R N I A S T A T E A U D I T O R 7 FIGURE 3 Proportion of Federal Adult Education Funds Allocated to Sampled CBOs From Fiscal Years 1994-95 Through 1997-98 Others Sampled CBOs 30% 70% We were able to review some of the documents subpoenaed, but we could not determine the extent or nature of federal investiga- tions due to lack of cooperation from the Office of the Inspector General of the U.S. Department of Education and the FBI. However, we did review the audit work the department’s inter- nal auditors performed related to the award and monitoring of federal adult education funds. To obtain an understanding of changes the department made to the adult education program, we interviewed department staff and reviewed related docu- ments. We assessed the appropriateness and effectiveness of these changes in view of federal laws, state budget requirements, the program’s state plan, the department’s past experience, and our observations throughout the audit. n 8 C A L I F O R N I A S T A T E A U D I T O R CHAPTER 1 The Department of Education Failed to Detect Cases Where Claimed Services Were Not Substantiated CHAPTER SUMMARY T he Department of Education (department) made pay- ments to community-based organizations (CBOs) for more services than the CBOs are able to substantiate. Eight of the 10 CBOs we reviewed—including all 5 of the CBOs named in a federal subpoena—could not adequately document claimed hours of instruction for at least one of the last five fiscal years. In addition, none of the 10 CBOs could consistently show that the students for whom hours were claimed demonstrated a gain in skill. These collective circumstances indicate the department paid for services that the CBOs may not have furnished. Although we found these problems to be widespread, the department rarely detected them during its site visits, giving CBOs no compelling reason to improve their record keeping. The department risks paying even more money for unsubstanti- ated claims in fiscal year 1999-2000 since it has drafted monitoring procedures for its revamped adult education program that do not ensure that claimed services are documented and it has not yet developed documentation requirements for service providers. Community-Based Organizations Often Lacked Evidence to Support the Hours of Instruction the Department Paid For It was evident that the CBOs we reviewed provided some level of instruction to eligible students, but most of them could not produce enough evidence to support claimed services. Deficien- cies ranged from an absence of attendance records and an insufficient number of recorded hours to basic attendance documents that appeared to be highly dubious. Further, most CBOs could not substantiate the instruction hours they claimed throughout the period of our review. To receive payments under the adult education program, the department requires service providers to submit claim forms. C A L I F O R N I A S T A T E A U D I T O R 9 Amounts claimed are calculated based on a predetermined rate Five “questioned CBOs” per hour. The department does not ask for evidence of student we reviewed could attendance with the claim, but it does require that service substantiate only providers maintain auditable records in a central location. The 30 percent of services department has never spelled out what constitutes auditable claimed, while the other records, but we believe it is reasonable to expect service provid- five substantiated ers to maintain basic attendance records, such as daily sign-in only 50 percent. sheets or class rolls, plus summary documents that accurately tabulate hours from the daily records. We further expect total hours in summary documents to equal those presented to the department on claim forms. We reviewed records at 10 CBOs to determine if they substan- tially supported the claims they submitted to the department over the last 5 years. We selected 5 of these CBOs from a group of 10 for which the department provided records under a federal subpoena (questioned CBOs). We discuss this subpoena in the Introduction. Three of the 5 CBOs received adult education funds in each of the fiscal years 1994-95 through 1998-99, and the other 2 received funding in 4 of those 5 years, which we calculate as a total of 23 funded years. We selected another five subjects from all other participating CBOs, four of whom received adult education funds in 4 of the 5 subject years. The fifth received funding in only 2 of those years, for a total of 18 funded years. Although we anticipated that questioned CBOs would have a harder time supporting their claims than the others, both groups fell far short of supplying full documenta- tion. The questioned CBOs substantially supported claims for only 7 out of 23, or 30 percent, of their funded years. The other 5, however, could only document claimed instruction hours for 9 out of 18, or 50 percent, of their funded years. Most of the CBOs We Reviewed Could Not Support Their Claims We summarized the results of our reviews in Table 1. Questioned CBOs are numbered 1 through 5. The conditions we found are divided into four categories. First are those situations where CBOs substantially supported their claim for a given year. We defined “substantial support” as a credible summary document whose total hours substantiate at least 90 percent of claimed dollars, plus credible daily attendance records that support at least 90 percent of the student hours we selected for further review. Depending on the CBO’s system of record keeping, we sampled either annual hours of instruction for individual students or monthly hours for individual classes. 10 C A L I F O R N I A S T A T E A U D I T O R TABLE 1 Instruction Hours Claimed Fiscal Years 1994-95 Through 1998-99 Substantially No Insufficient Supported Support Support Dubious Support CBO 1* 2 of 4 2 of 4 2 of 4 CBO 2 1 of 5 4 of 5 CBO 3 3 of 4 1 of 4 CBO 4* 4 of 5 1 of 5 1 of 5 CBO 5 3 of 5 1 of 5 1 of 5 CBO 6* 2 of 2 2 of 2 CBO 7 4 of 4 CBO 8 1 of 4 1 of 4 2 of 4 CBO 9 4 of 4 CBO 10 4 of 4 * These CBOs had multiple problems in the same year or years. Specifically, their records were both dubious and insufficient to support the hours claimed. The “insufficient support” column designates those CBOs that Summary attendance failed either of the 90 percent criteria above, and “no support” records for one CBO indicates CBOs that could not provide us with any attendance support at most 5 percent records. Finally, the “dubious support” column highlights cases of claimed hours for any where attendance records appeared highly questionable. In these of the last four years. cases, we did not credit the CBO with maintaining substantial support for its claim even if attendance records and summary documents numerically met the 90 percent criteria. Several CBOs lacked both summary documents and daily atten- dance records for the same time period. For example, CBO 2’s summary attendance records for fiscal years 1995-96 through 1998-99 support no more than 5 percent of claimed hours for any individual year while its daily attendance records only provide evidence for, at most, 85 percent of the student hours we reviewed in any single year. This lack of documentation is significant considering the department paid this particular CBO over $2.2 million from fiscal year 1994-95 through 1998-99, the second largest amount any CBO received during that period, according to department data. C A L I F O R N I A S T A T E A U D I T O R 11 In another example, CBO 7 could furnish daily attendance and summary documents to support, at most, only 27 percent of the hours it claimed for fiscal years 1995-96 and 1996-97. During the period, the department paid this CBO almost $283,000. Some CBOs failed to substantiate their claims either because underlying attendance records did not support summary docu- ments or because no summary documents were available. For example, hours of attendance listed on summary documents for CBO 4 equaled its claimed hours, but it could not produce daily attendance records for one of the five students whose annual hours we reviewed from its fiscal year 1995-96 summary. Simi- larly, CBO 8 kept adequate summary documents for fiscal years 1994-95 and 1995-96, but its attendance records for the five students we reviewed in the first year amounted to only 83 percent of the hours listed on the summary. Records in the second year substantiated only 24 percent of those hours. In addition, this CBO could furnish no attendance records for its largest teaching site in fiscal year 1994-95. In numerous other instances, CBOs had no attendance records to support their claims. CBO 3 was particularly deficient in this regard, failing to produce records for three of four years. CBO 3 explained that it discarded its fiscal year 1994-95 records and that its records for fiscal years 1995-96 and 1996-97 were not available because they had all been turned over to the federal government under a subpoena. However, when we gained access to the subpoenaed records through the Office of the Inspector General for the U.S. Department of Education, we found neither attendance records nor summary documents for those years. Some Existing Attendance Records Lacked Credibility Finally, an even more glaring problem—but one encountered Records for one CBO less frequently—was caused by highly questionable attendance show students attended records. Most teaching sites for CBO 1 used a weekly class roll to class on Labor Day, record daily attendance. Check marks typically indicated a Thanksgiving, Christmas, particular student was present on a given day. However, teachers and New Year’s Day, at one location used check marks to record absences; if a student while one student went to was present, the space was left blank. The more sensible way of class 3,406 hours— taking roll is to record when students are present. Additionally, equivalent to 9 hours if a teacher checks off absent students, one would expect to see each day of the year. corrections to the roll when students arrive late—yet we saw no such corrections. In fact, our analysis of one class roll for three consecutive months showed that only 2 of the 32 students were recorded as being absent on Independence Day and Labor Day. 12 C A L I F O R N I A S T A T E A U D I T O R We find it hard to believe that the CBO conducted classes on national holidays, and even if it did, that attendance would be so high. We found even more implausible evidence at CBO 6. It fur- nished us with pages from class rolls showing attendance for one of the students we reviewed. These pages showed every student in a particular class attending every weekday from August 1997 through April 1998. Supposedly, students even attended class on Labor Day, Thanksgiving, Christmas, and New Year’s Day. This same CBO’s February 1998 class rolls showed another student attending one class from 4 p.m. to 8 p.m. and a different class from 5 p.m. to 9 p.m. on the same days as well as concurrent classes from 8 a.m. to noon. Summary records showed another student we selected attending 3,406 hours in five different classes during fiscal year 1997-98. To accumulate such a total, this student would have to attend an average of over 9 hours of classes every day of the year. In all of these cases, we found the underlying scenarios impossible and concluded that the records were fabricated. Community-Based Organizations Usually Failed to Demonstrate Student Gains in Skill, a Requirement for ESL-Citizenship Classes Though we found testing documents for most ESL-citizenship students in our sample, the documents usually failed to demon- strate a gain in skill level, as required by the department. Despite these deficiencies, the CBOs we reviewed continued to claim hours for all students participating in the program. Prior to receiving adult education funds, service providers must agree to show a demonstrated gain in skill for all ESL-citizenship students whose hours they claim. Gain in skill is shown through an improvement in scores from pretesting to post-testing. We reviewed testing documents at the same 10 CBOs where we reviewed support for claimed hours. Again, we anticipated finding compliance problems at the 5 CBOs included in a federal subpoena, but all 10 CBOs had difficulty demonstrating gains in skill. Although the questioned CBOs could only demonstrate gains in skill for an abysmally low 25 percent of students we sampled, the other 5 could only document gains for 57 percent. C A L I F O R N I A S T A T E A U D I T O R 13 Specifically, we reviewed documents for 171 students at 9 CBOs The CBOs we reviewed and found evidence demonstrating a gain in skill for only 75, or could only demonstrate 44 percent, of the students. In addition, for three years, CBO 3 gains in skill for lacked documentation demonstrating gains in skill for any of its 44 percent of sampled students while CBOs 2 and 4 lacked this documentation for any students. of their students for one year. Despite the inability to show gains in skill, none of the 10 reduced accumulated hours accordingly. CBOs were unable to demonstrate gain in skill for a variety of reasons, ranging from inability to distinguish pretests from post- tests to highly questionable test results. For instance, the files of 1 of 10 students tested at CBO 10 contained no post-tests for fiscal year 1996-97, and another 3 students actually showed a drop in score from pretesting to post-testing. In another instance, CBO 5 did not even grade the tests of the 5 students we reviewed for fiscal year 1997-98. At one CBO, we found test records that appeared dubious. CBO 6 used standardized tests created by the Comprehensive Adult Student Assessment System (CASAS) to demonstrate gain in skill. Although its records showed skill improvement, the size of the gain made us question whether the testing was aboveboard. Specifically, all of the five students we reviewed registered gains ranging from 33 to 55 points after participating in the program for one to four months. In comparison, the department esti- mates that, at most, only 27 percent of students are expected to demonstrate a five-point gain on the CASAS test scale by the end of one year. The Department’s Monitoring Efforts Failed to Detect and Correct Inadequate Record Keeping The department claimed to have conducted site visits of the CBOs in our sample, but the reviewers rarely noted the significant problems we observed during our visits. We conclude that the department’s monitoring efforts were not effective in correcting inadequate record keeping at CBOs. Federal regula- tions required the department to perform a site review of at least 80 percent of all adult education service providers during the four-year period of the state plan. Between fiscal years 1994-95 and 1998-99, the department claimed to have made at least one site visit to all 10 of the CBOs we reviewed. However, while we found during our own review that CBOs only substantiated claimed hours for 39 percent of funding years we tested, and 14 C A L I F O R N I A S T A T E A U D I T O R only demonstrated gains in skill for 44 percent of tested students, department records show it detected similar compli- ance issues during just two of its site visits. Even in the two instances where the department noted prob- lems, it either did not follow up adequately or its reviews came too late to rectify CBO noncompliance. In its February 1997 visit to CBO 5, the department noted incomplete attendance records and missing post-tests. Nevertheless, our site review indicated that this same CBO continued to have problems substantiating hours claimed and demonstrating gains in skill through fiscal year 1998-99. At the second site visit, made to CBO 1 in August 1998, the department found even more significant noncompliance issues. It conducted a range of tests similar to ours to determine the sufficiency of attendance records and found that this CBO had made implausible claims. Unfortunately, the visit occurred more than two years after the department first became aware of serious noncompliance issues and several months after the end of the CBO’s last year of funding. Although the department poorly documented its site visits, we Although our review pieced together an understanding of why reviewers missed uncovered significant inadequately documented gains in skill and claimed hours. problems at all 10 CBOs, Review procedures and checklists simply did not adequately during its site visits the cover these two issues. For example, the checklist for the department detected October 1995 site visit at CBO 2 shows that a reviewer checked problems at only two. attendance records and test documents for as many as 143 students to determine whether pretests and post-tests were on file and whether students had attended at least 12 hours of class. The reviewer did not, however, analyze testing documents to determine if they demonstrated gains in skill or verify that summary documents contained accurate information and were properly compiled. Thus, reviewers missed opportunities to examine these two essential compliance issues. A 1997 review of the adult education monitoring system by the department’s Audits and Investigations Division (division) found further problems. In particular, the division found that site visits appeared to be cursory, with no compliance issues reported in 11 of the 14 documented reviews. The division drew this conclusion after finding little written evidence that review- ers actually examined any of the CBOs’ records, prompting it to recommend the department develop a guide to direct reviewers on properly conducting and documenting their site visits. C A L I F O R N I A S T A T E A U D I T O R 15 In September 1998, the division followed up to determine whether the department had carried out this recommendation. It found that the department had made some progress toward resolving monitoring issues but the department’s efforts needed continued follow-up. Because it did not develop adequate procedures for reviewing Because the inadequate service providers, the department failed to identify problems monitoring system during site visits. When it subsequently gave virtually all CBOs a essentially gave them clean bill of health, they had no compelling reason to improve clean bills of health, CBOs their systems for documenting gains in skill and the number of had no reason to improve student hours produced. The department concedes that its operations. monitoring in the past has failed to identify the magnitude of the problems we identified during our review. Plans for Monitoring Under the Revamped Program Could Result in Continued Problems Changes in federal law led the department to adopt a new state plan, new eligibility requirements, and new reimbursement rates for the adult education program. However, by the end of June 1999, the department had not yet established updated documentation requirements for service providers. In addition, its drafted monitoring procedures did not include tests of the documents supporting claims for payment. Given its weak monitoring efforts in the past, we feel that it is imperative that the department quickly develop a reliable monitoring system for this evidence in order to avoid paying unsupported claims. In particular, it must pay attention to the quality of evidence service providers use to support claimed benchmarks. As discussed further in Chapter 2, payments in fiscal year 1999-2000 will be based on claimed benchmarks. We have already mentioned one CBO that furnished dubious records of gains in skill based on CASAS tests. These are the same tests the department will require agencies to use in fiscal year 1999-2000 to demonstrate the number of benchmarks attained. We also note that CASAS test questions have remained unchanged for several years. Only 1 of 16 reading tests in two distinct series is less than four years old. The danger of repeated use of the same questions is that agencies may teach students only to answer the questions on the tests to assure success in meeting benchmarks. In discussions with the CASAS, we learned that it is developing two new tests due for release in fall 1999. Even with the addition of these tests, the depth of these two testing series is quite limited. 16 C A L I F O R N I A S T A T E A U D I T O R When we asked the department how it would deal with what appear to be incredible claims, staff said they would conduct a site visit and compare claimed student gains against reported hours of instruction to determine if the claims were valid. In the case of CBO 6, however, the hours of instruction claimed were as implausible as the gains in skill. This leads us to con- clude that the department must continue to require that service providers maintain attendance records and must review them for reliability. RECOMMENDATIONS To ensure that service providers maintain appropriate evidence to support their claims for payment, the department should do the following: • Establish strict guidelines for service providers to document student testing and hours of instruction to deter easily falsi- fied evidence. • Place a high priority on developing a battery of interchange- able tests for measuring gains in skill to avoid falsification of evidence or teaching to the test questions. • Design monitoring procedures to test support for claimed services, including review of attendance records, summary documents, and tests showing attainment of benchmarks. n C A L I F O R N I A S T A T E A U D I T O R 17 Blank page inserted for reproduction purposes only. 18 C A L I F O R N I A S T A T E A U D I T O R CHAPTER 2 Changes the Department of Education Proposes to Correct Its Flawed Oversight of Adult Education Funding May Provide More Consistency, but May Also Diminish Services CHAPTER SUMMARY T he inconsistent review and approval by the Department of Education (department) of applications for federal funding from community-based organizations (CBOs) raises questions regarding the fairness of its decisions. Specifi- cally, the department approved some deficient applications while rejecting others for the same deficiencies. In addition, it approved levels of funding for some CBOs that were unreason- able in light of their past performance. Further, the department awarded one grant that was unrealistically large given the CBO’s reported size. In addition, the department failed to react appropriately when it learned of significant concerns about this same CBO. The department is implementing a new award process that may result in more consistent award decisions, if it follows through with current plans; however, a new rate structure and more stringent eligibility requirements may also reduce services for the neediest students. Additionally, because the State’s fiscal year 1999-2000 budget earmarks federal funds for adult education much more restrictively than the department expected, the department will need to reassess its new rate structure and award process. The Department’s Treatment of Different Community-Based Organizations Was Inconsistent The department was inconsistent in approving and denying applications for funding. The inconsistencies we found related to its requirements for CBOs to submit audit reports before it awarded grants and to its evaluation of qualitative aspects of C A L I F O R N I A S T A T E A U D I T O R 19 CBO applications. As a result, the department has left itself vulnerable to questions of fairness. Further, in one significant case, the department continued to fund a CBO in spite of major negative findings noted in an audit report. The department, which put grant funds at risk in that case, is now trying to recover over $4 million from that CBO. As a condition for receiving grants for subsequent years, CBOs Before approving that applied for federal funds were supposed to submit audit funding, the department reports along with their applications if they had spent a did not consistently sufficient amount of federal funds in the past to require an require CBOs to submit annual independent audit. Up until fiscal year 1996-97, any audit reports. agency spending more than $25,000 yearly in federal funds needed to provide a financial and compliance audit report covering its entire operations for that year. Beginning in fiscal year 1996-97, federal regulations increased the dollar cutoff to $300,000. Each of the 10 CBOs we reviewed was subject to the audit requirement for at least one of the years in which they received a grant. However, the department did not consistently enforce this requirement. In 10 instances from fiscal years 1994-95 through 1998-99, the department withheld approval of grants to CBOs in our sample until it had received the required audit reports, but it awarded three grants to other CBOs before they submitted their reports. In specific examples, as recently as fiscal year 1997-98, the department awarded funds to CBOs 3 and 8 even though they had not yet submitted audit reports. In the same fiscal year, the department delayed awards to CBOs 4, 5, and 7 until they submitted their audit reports. This raises concern about whether the department treated the CBOs equally. Current program officials could not explain why the department had not consistently enforced requirements regarding the audit reports. However, it appears that the department more rigor- ously enforced this requirement for fiscal year 1998-99. The Department Measured the Qualitative Value of Applications Inconsistently Another area in which the department’s review and approval of applications was inconsistent was its evaluation of the quality of each applicant’s program. Each agency must describe its general procedures and program content as part of the application process. Using a rating sheet, the department evaluated the quality of the responses. In most years, it used as many as three raters to review each application. However, the limited evidence 20 C A L I F O R N I A S T A T E A U D I T O R available to us about the results of those reviews highlights inconsistencies in the process. For example, for 5 of the 10 appli- cations we examined for fiscal year 1997-98, at least one rater failed the application on the rating sheet. In three of these five cases, two raters failed the applications before a third passed them. For the other two, the first rater failed the applications but the second rater passed them. In all cases, the department eventually awarded a grant to the CBOs. We realize some judg- ment needs to be exercised in evaluating qualitative issues, so it may have been appropriate for the department to award the grants in these cases. However, the documentation shows inconsistency in how staff evaluated the applications. In another example, the department inconsistently evaluated Although one or more whether an agency’s adult education program duplicated raters often failed CBO services available to students under other programs. Specifically, applications, most failed two raters failed the fiscal year 1997-98 application of CBO 5 applications were because of concerns about duplication of services. The depart- ultimately passed, with ment eventually approved the organization’s grant for fiscal year no documented 1997-98 and provided another grant for fiscal year 1998-99. In explanation. contrast, it denied funding for CBO 7 in fiscal year 1998-99 based on one rater’s concern about duplication of services. Correspondence from CBO 7 explained how its services did not actually overlap those of an adult school in the area, and we saw no other evidence in the file to refute this claim. Nevertheless, the department did not revise its decision to deny this CBO’s application. Current program officials told us that in their efforts to more rigorously enforce requirements, they may have erred on the side of caution. However, the department awarded grants for fiscal year 1997-98 to two different CBOs even though their applications actually indicated an overlap in services. Specifically, CBO 6 requested funding for 3.5 million student hours, including 2.5 million for activities in two Southern California counties. The department eventually approved 1.4 million hours for CBO 6 in only one of the two counties. For the same year, CBO 1 requested funding for 22.4 million student hours, including about 1 million hours for activities in the same county the department approved for CBO 6. Ultimately, the department approved all but 760,000 student hours of CBO 1’s request. In addition to overlapping funding requests for the same county, other information in the applications led us to conclude that these CBOs duplicated services. For example, both of the CBOs’ C A L I F O R N I A S T A T E A U D I T O R 21 applications list the same person as the key contact in this The department awarded county. This individual is purported to be a member of the grants to two different board of directors of CBO 1 and the executive director of CBO 6. CBOs whose applications Further, the address listed in CBO 6’s articles of incorporation indicated an overlap of is the same as the address listed for CBO 1’s operations in services. that county. The administrator of the Adult Education Office told us that her staff discussed this matter with officials of CBO 6 and were satisfied that services would not overlap. However, based on the information in the CBOs’ applications and the fact that we saw no documentation indicating that services would not overlap, we believe that the department’s decision was inconsistent with its denial of funding for CBO 7 for a similar issue. It certainly appears to be unfair to CBO 7. The department’s Audits and Investigations Division (division) was also critical of the Adult Education Office’s grant application process. In its October 1997 report, the division reported that the adult education grant award process was not effectively planned or managed in fiscal year 1996-97. Based on a review of grant applications that initially failed, the division’s auditors found that some applicants failed for minor technical reasons, while others did not qualify for funding based on the informa- tion they provided. The auditors also noted that the initial decisions of application raters were arbitrarily changed based on new criteria introduced after the grants were first scored. Further, the auditors learned through interviews that the raters were frustrated with the process when the department changed or inconsistently fol- lowed established deadlines and pass/fail policies. The auditors recommended that a grant application process based on well- planned procedures and clearly communicated criteria be developed. However, in following up on this recommendation in September 1998, the auditors expressed continuing concerns about the application process used for evaluating the fiscal year 1998-99 grants and concluded that their concerns had not been sufficiently resolved. The Department Failed to Take Appropriate Action When It Learned of Significant Problems With One CBO The department was generally proactive in reviewing the audit reports it received from applicant agencies and took some action to follow up on issues of concern, but it had no clear policy 22 C A L I F O R N I A S T A T E A U D I T O R defining audit issues that would result in denial of funding. As a result, it continued to fund one of the questioned CBOs for two years despite serious audit concerns. Eventually, the department decided to stop funding the CBO and requested repayment for funds paid under previous annual grants. The department awarded significant grants to CBO 1 from fiscal years 1994-95 through 1997-98. The awards ranged from a low of about $428,000 in fiscal year 1994-95 to a high of about $3,500,000 in fiscal year 1997-98, the largest overall funding awarded to a CBO during these years. The audit report for CBO 1 covering fiscal year 1994-95 was due by July 1996; however, the department did not receive a report until April 1997, when the CBO submitted one that covered both fiscal years 1994-95 and 1995-96. Although it withheld funding for fiscal year 1996-97 until it had received the report, the department eventually approved CBO 1 for funding that year. It made this decision even though independent auditors issued a qualified opinion on the CBO’s financial statements and noted many concerns that could jeopardize its government contracts. For example, the auditors found that because its general ledger A CBO received multi- and supporting records were not accurately prepared and recon- million dollar awards ciled, the CBO had reported inaccurate financial information for despite evidence that it its grants. The auditors also noted that it had not paid payroll was financially unstable. taxes to the Internal Revenue Service and the Employment Development Department (EDD) even though it reported these as costs incurred on its reports to the department. In a final example, the auditors found there was insufficient evidence to support $300,000 in expenditures under a separate federal grant the CBO received from the U.S. Department of Health and Human Services. The department was well aware of CBO 1’s failure to remit payroll taxes to the EDD before it received the auditor’s report in April 1997. In October 1996, the EDD sent a letter requesting that the department offset the CBO’s grant by $171,000 to cover unpaid payroll taxes and interest. In January and February 1997, EDD representatives had several telephone conversations with department officials, including the director of the division that oversaw the Adult Education Office at the time. According to EDD records, on March 10, 1997, the department paid the EDD $178,250 to cover CBO 1’s tax obligations as of February 26, 1997. Clearly, the department had substantial notice that this CBO was financially unstable. C A L I F O R N I A S T A T E A U D I T O R 23 In spite of being aware of these problems, the department apparently believed it could work with the CBO to ensure it became fiscally responsible and compliant with program requirements. It took the department until June 1998 to demand repayment of over $4 million it had previously paid CBO 1, citing the significant fiscal concerns raised by the audit report it had received in April 1997. This amount represented the entire funding the department provided CBO 1 for fiscal years 1994-95 through 1996-97. Between April 1997 and June 1998, CBO 1 continued to miss deadlines for submitting required information and taking agreed-upon actions, and the department continued to monitor its actions. In spite of the significant problems noted in the April 1997 audit report, the department approved a fiscal year 1996-97 award of about $2.1 million for CBO 1 a month later. In October 1997, it approved yet another award for fiscal year 1997-98 totaling about $3.5 million, the largest grant CBO 1 received. As of June 1999, the department has been unable to recover any of the $4 million. The Department Approved Unreasonable Levels of Funding In reviewing applications for funding, the department did not Some of the department’s assess the reasonableness of student hours it awarded to CBOs decisions resulted in for ESL-citizenship instruction in light of their actual perfor- funding inequities. mance the prior year. Some of the department’s award decisions also resulted in funding inequities. Moreover, in one case, the department approved an award for services far beyond what the CBO could have possibly produced. In fiscal years 1994-95 through 1998-99, requests for ESL-citizen- ship funding exceeded the amount of federal funds available. Throughout this period, the reimbursement rate remained $2.50 for each student hour of instruction. Since the expressed need was greater than available funding, the department decided that it would only fund a certain percentage of the ESL-citizenship hours each service provider requested rather than adjusting the reimbursement rate. For example, it notified applicants approved for fiscal year 1994-95 funding that they would receive about 26 percent of their request. The deputy superintendent of the branch overseeing the Adult Education Office in January 1997 reported to the state superintendent of education that most agencies received only about 22 percent of their requests for funding in fiscal year 1996-97 due to the heavy demand for these moneys. 24 C A L I F O R N I A S T A T E A U D I T O R Although following this type of process appears to be fair, we found it resulted in funding inequities. For example, CBO 1, one of those named in the federal subpoena discussed in the Intro- duction, initially requested funding for 2,500,000 student hours for ESL-citizenship instruction in fiscal year 1996-97. This initial request was almost seven times the 327,000 hours it reported generating in fiscal year 1995-96. When the CBO reduced its request to 500,000 student hours, or about 20 percent of its initial request, the department approved the entire revised amount. While 20 percent seems consistent with the 22 percent funding rate the deputy superintendent reported above, this award represented a 53 percent increase over student hours the CBO produced in the prior year. Meanwhile, for the same funding year, CBO 10 initially requested 48,000 student hours for ESL-citizenship instruction, 95 percent more than the 24,640 hours it reported generating in fiscal year 1995-96. In its application, the CBO reduced its request to 12,000 hours, or 25 percent of its initial request, and the department then approved the entire revised amount. Again, 25 percent seems fairly consistent with the 22 percent funding rate reported by the deputy superintendent. However, 12,000 hours is a 51 percent decrease from the student hours the CBO produced in the prior year. Contrasted with the 53 percent increase that CBO 1 enjoyed, this award seems unfair to CBO 10. In addition, CBO 10 reported at the end of the year that it actually generated 42,200 hours in fiscal year 1996-97, more than 2.5 times the hours it was awarded. Had CBO 10 inflated its initial request, as CBO 1 did, it might have received a larger award. In a more recent example, the department awarded 125,000 hours to CBO 6 and 60,000 to CBO 9 for ESL-citizenship instruction in fiscal year 1998-99. These amounts represented 50 percent of each CBO’s request. However, CBO 6’s 125,000 hours were more than double the hours it reported for fiscal year 1997-98, while CBO 9’s award was only 9 percent higher than its prior-year reported production. Although pro- gram officials told us they reviewed requests for fiscal year 1998-99 funds to assess whether requests were reasonable based on prior year performance, another overstated request resulted in a larger award. C A L I F O R N I A S T A T E A U D I T O R 25 A third example in fiscal year 1997-98 involved CBO 1. The One CBO was awarded office awarded this agency far more hours than it could have 21 million student actually produced given its reported size. Specifically, the depart- hours—far more hours ment awarded CBO 1 over 21 million student hours for that than it could generate. fiscal year. Even presuming that students attended class every week during the year for 40 hours per week, this agency would have needed over 200 classes with 50 students in each all year long to produce that many hours. However, in its application for fiscal year 1997-98, CBO 1 indicated that it had only 42 teaching sites. We also found that some of this CBO’s attendance support was dubious, as described in Chapter 1. Therefore, we believe it was inappropriate for the department to approve such a large award. When the department allocates funds to CBOs that have not demonstrated that they can deliver a corresponding level of service, fewer funds are available for allocation to other service providers. According to the manager of the Adult Education Office, raters did not always consider past performance and information about the size of CBO programs when evaluating applications. By awarding grants without sufficiently scrutinizing the level of funding requested and adjusting awards to reasonable levels, the department has opened the door for allegations of favoritism and unequal treatment of grant recipients. New Procedures May Result in More Consistent Award Decisions, but Fewer Students May Be Served In response to the new federal Adult Education and Family Literacy Act, the department has made major changes in the award and payment processes for the adult education program. Based on its implementation plans, we expect the department to more consistently manage the awards process. We also expect, however, that new eligibility requirements and funding rates will reduce the number of service providers, especially CBOs. Consequently, the number of students covered by the program may decrease, and students with the lowest skills will probably bear the brunt of the reduction in services. We were unable to fully assess the potential problems because the department refused to provide us with the number of students that appli- cants hoped to educate and the number of benchmarks they estimated students would achieve in fiscal year 1999-2000. Changes the department has made to its awards procedures for fiscal year 1999-2000 may lead to more consistent award deci- sions and more reasonable funding levels. Specifically, it has 26 C A L I F O R N I A S T A T E A U D I T O R developed review criteria and attempted to build consensus among raters through review and discussion of sample applications. In the new process, two raters will review each application, with a third rater adding the tie-breaking assess- ment if the first two disagree. The department has also set a firm rejection policy for late or incomplete applications. Finally, it is developing a means to check requested benchmarks against general standards and against a service provider’s past perfor- mance. Although it is too early to tell how well the department’s new procedures are working, we believe they will significantly improve past performance if they are implemented as the department intends. Taking the new federal Adult Education and Family Literacy Act as a jumping-off point, the department now requires all applicants to meet minimum criteria in 12 program areas. For example, applicants for fiscal year 1999-2000 funds must provide automated evidence of gains in student skills related to New eligibility their past activities; describe how they coordinate their activities requirements and funding with those of other service providers; and explain how they use rates will reduce the technology to enhance student learning. In the past, the number of service department required applicants to describe various program providers. activities but almost never denied funding because of an inadequate response. Federal law states that these 12 criteria must be considered in making award decisions, but it does not require that service providers meet minimum standards for each. Nevertheless, the department has chosen to raise the bar for eligibility signifi- cantly above minimum federal standards. To meet several of the criteria, service providers must already have systems and procedures in place, a requirement that conflicts with current practices that allow service providers to operate with a minimum of administrative sophistication and documentation. On the whole, we believe the requirements will discourage or eliminate many service providers that have limited funds for noninstructional functions such as interagency coordination, management information systems, and classroom technology purchases. In fact, the number of agencies applying for funding in fiscal year 1999-2000, including CBOs, adult schools, and community colleges, fell by 164, or 42 percent, from 389 in fiscal year 1998-99. In addition to more stringent eligibility requirements, the department has changed its reimbursement structure in ways that are likely to reduce payments to CBOs significantly. Starting C A L I F O R N I A S T A T E A U D I T O R 27 in fiscal year 1999-2000, service providers will be reimbursed for the number of performance benchmarks achieved by individual students. The department currently plans to pay $50 per 5-point skill gain and $50 for each student that successfully progresses through two skill levels, as measured by standardized tests. Each skill level ranges from 10 to 30 points and varies by type of instruction, so students must gain between 12 and 41 points for the service provider to earn the additional $50. Using depart- ment estimates of the number of instructional hours needed for a student to reach one benchmark, we estimate that payments to CBOs would drop by 80 percent for similar services. For example, a CBO that taught ESL-citizenship classes was previ- ously paid $250 for the 100 hours of instruction normally required to attain one benchmark. It would only receive $50 for those 100 hours under the new system. Our above estimate generously assumes that students stay in the The department’s new program for the amount of time it takes to reach benchmarks, rate structure will greatly but previously, the department reimbursed service providers for reduce payments to CBOs students receiving as few as 12 hours of instruction. The reduc- for the same level of tion in funding would be even more precipitous if dropout rates service. were factored in. According to the department, it expects that 33 percent of students, at most, will meet one benchmark in fiscal year 1999-2000. Given the fact that the 10 CBOs we reviewed depend on the federal grant to cover a large portion of their adult education costs, many agencies will find it impossible to continue operating their programs under these conditions. In fact, only 40 CBOs have applied for funding in fiscal year 1999-2000 compared to 87 in fiscal year 1998-99. A large reduction in CBO participation is likely to have a negative effect on the department’s ability to reach students most in need of adult education services because CBOs have historically served groups with the least skills. According to the department’s latest report on student progress and goal attain- ment for the adult education program, the percentage of CBO students in the lowest skill level was much higher than that of adult schools or community colleges. Figure 4 shows the percentage of least-skilled students included in the client base of these three types of service providers. 28 C A L I F O R N I A S T A T E A U D I T O R FIGURE 4 Percentage of Students in Lowest Instructional Level by Provider Type CBO Adult School 80% Community College 70.3 70 66.7 60 48.5 50 40 30 20 18.6 20 11.6 10.4 11.2 10 7.7 0 Adult Basic ESL ESL-Citizenship Education Adult Education Program - Although CBOs served only about 5.2 percent of California’s 1.4 million students in adult education during fiscal year 1997-98, adult schools and community colleges will still need to expand or refocus their activities to pick up students no longer served by CBOs. The department has not, however, added any extra incentives or requirements to assure that local agencies continue to serve students with the least skills. According to the department, only 185 non-CBOs applied for fiscal year 1999-2000 funding, down 39 percent from fiscal year 1998-99 levels. We believe this drop in participation will make it difficult for adult schools and community colleges to maintain their current levels of service, let alone make up for the loss of CBOs’ instructional services. The department is facing another challenge created by language in the recently approved budget for fiscal year 1999-2000. Although the department developed its new rate structure based on the assumption that the budget language would be broad- ened to allow earmarked federal funds to be used for either ESL or ESL-citizenship instruction, this has not turned out to be the case. Specifically, the new budget continues to set aside $12.6 million (30 percent) of the $42.3 million in federal funds for “citizenship and naturalization services for legal permanent residents who are eligible for naturalization.” Even though the C A L I F O R N I A S T A T E A U D I T O R 29 department has already received and evaluated applications for fiscal year 1999-2000 based on its previous assumption, it will now have to reassess its funding structure and make award decisions carefully taking into consideration the specific ear- marking of funds. RECOMMENDATIONS To ensure that its award decisions are consistent and fair, the department should do the following: • Hold all applicants accountable for submitting required information, including audit reports, to qualify for funding. • Evaluate funding requests in light of prior-year performance and the size of the service provider before authorizing grant awards. • Review a sample of fiscal year 1999-2000 awards to ensure that decisions to award or deny funds are consistent and defensible. To ensure that it reaches the greatest number of students and maximizes the use of federal adult education funds, the depart- ment should evaluate the impact that changes in the program will have on students and service providers. If, as we anticipate, this evaluation shows that fewer students will be served, the department needs to develop strategies to encourage program expansion. 30 C A L I F O R N I A S T A T E A U D I T O R We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, KURT R. SJOBERG State Auditor Date: July 20, 1999 Staff: Ann K. Campbell, CFE, Audit Principal John F. Collins II, CPA Kris Patel James Sandberg-Larsen, CPA C A L I F O R N I A S T A T E A U D I T O R 31 Blank page inserted for reproduction purposes only. 32 C A L I F O R N I A S T A T E A U D I T O R Agency’s response provided as text only: California Department of Education 721 Capitol Mall P.O. Box 944272 Sacramento, California 94244-2720 July 12, 1999 Kurt R. Sjoberg California State Auditor 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Report 98113 Dear Mr. Sjoberg: Thank you for the opportunity to review and comment on your draft audit report on the California Department of Education’s (CDE’s) administration of the federal adult education program en- titled “Department of Education: Lax Monitoring led to Payment of Unsubstantiated Adult Educa- tion Claims and Changes in the Program May Seriously Impact Its Effectiveness,” dated July 1999 (Report 98113). CDE has concerns about the perspective provided and therefore, in some instances, the accuracy of the information provided in your draft audit report Enclosed are CDE’s comments and clarification, organized to parallel the structure contained in your audit report. If you have questions about the CDE’s response, please contact the CDE’s Audit Response Coordinator, Peggy Peters, at (916) 6574440. Sincerely, (Signed by: Leslie Fausset) Leslie Fausset Chief Deputy Superintendent Policy and Programs Enclosure R-1 California Department of Education Response to Audit Report 98113 INTRODUCTION Historical Perspective Throughout the past few year, the California Department of Education (CDE) has received considerable pressure from the State Legislature to fund community based organizations (CBOs) who provide English as a Second Language (ESL)-Citizenship services to legalized aliens. Beginning on June 30, 1992, the California State Board of Education (State Board) approved and submitted to the U.S. Department of Education (USDE), the Revised 1992-1995 California State Plan for Adult Basic Education (State Plan). The State Plan was a revision of an earlier state plan, adopted by the State Board for 1989-1993. This state plan identified that funds would be used to provide literacy services to eligible adult basic education (ABE) and ESL learners. This state plan also identified adult immigrants as a key population group to be served by the federal adult basic education funds. During the time period of 1992-96, approximately 3 million legal aliens in the United States, granted legal status during the Immigration Reform and Control Act (IRCA, Public Law 99-603) amnesty period in the late 1980s, became eligible for citizenship. Immigration advocates estimated that 1.8 million eligible legalized aliens in California would require supportive services before being able to independently complete the application and interview preparation process, including English language and citizenship classes. This number of legalized aliens far surpassed the number that could be served by the state-funded adult schools and community colleges. Many community-based organizations (CBOs) were established through the Attorney General’s Office as Qualified Designated Entities (QDEs) during IRCA and provided supportive services through State Legalization Impact Assistance Grants (SLIAG). The SLIAG grants provided funds to CBOs to cover their administrative and instructional expenses incurred in offering supportive services to legalized aliens. When SLIAG funds were exhausted many of these CBOs were interested in continuing services to legal aliens through the use of National Literacy Act resources, administered through the CDE. In contrast to the SLIAG money, the National Literacy Act funds are intended to be supplemental to state and local efforts to provide programs for ABE and ESL. 1 * In 1994, the State Legislature provided funding through a set-aside of $7.7 million of National Literacy Act funds specifically for CBOs to provide citizenship and naturalization services for legalized aliens. USDE clarified for California that all agency types must have access to the National Literacy Act funds, including adult schools, community colleges, library literacy programs, as well as the CBOs. Throughout the fall *California State Auditor’s comments on this response begin on page R-15. R-2 of 1994, CDE participated in discussions with members of the State Legislature and adult education providers about permissible education activities under federal Section 321 funds of the National Literacy Act. The State developed a new program area, called ESL-Citizenship, to provide funds for supportive services to adult immigrants as specified in the State Plan. Consequently, since 1995, State Budget Act Language has provided a large set-aside of $7.7 to $12.57 million of the federal National Literacy Act allocation for ESL- Citizenship. This allocation has become one-third to one-half of the local assistance money provided for ABE, ESL, and ESL-Citizenship programs. The significant amount of the set-aside resulted in a larger reimbursement rate for ESL-Citizenship ($2.50 per hour) compared to ESL ($.10 per hour) and ABE ($.13 per hour). Members of the State Legislature strongly encouraged CDE to support the CBOs in their efforts to provide federally-funded education services. Through the large set-aside, state legislators intended CBOs to receive sufficient funds to support an administrative structure for providing services to their clients, often the hardest to serve and functioning at the lowest literacy levels. However, federal regulations emphasized equal access for all agencies - adult schools, community colleges, library literacy programs, and CBOs - to receive the same reimbursement rate. The considerable difference in funding rates required greater accountability for providers receiving ESL-Citizenship federal funding. Provider agencies have kept separate records on student performance in order to receive the $2.50 per hour reimbursement rate. Agencies could submit student hours of attendance for reimbursement only if the student had attended a minimum of 12 hours of instruction and the agency had a student folder that demonstrated progress. Each agency has been required to maintain a student folder containing: (1) a completed INS-400 form, (2) pre- and post- test scores that show progress, (3) completion of a citizenship practice test, (4) dated quizzes, and (5) other assessments. In summary, ESL-Citizenship was offered in California as a result of a state mandate. CBOs lobbied for a higher reimbursement rate and larger set-aside throughout the years. The State Legislature supported their request by providing a large amount of funds for CBOs and strongly encouraging the CDE to fund CBOs with National Literacy Act Section 321 monies. CDE Structure CDE changed the entire management structure over adult education to improve administration. On March 1, 1998, a new Deputy Superintendent assumed responsibility for the administration of the Specialized Programs Branch, which has since been renamed the Education Equity, Access and Support Branch. The manager of the Adult Education Program in the Adult Education, Education Options and Safe Schools Division began on April 1, 1998, and a new Assistant Superintendent/ Director was appointed on April 22, 1998. All three individuals have been working very closely to improve the administration and accountability over federally funded programs. R-3 Specifically, this new management team has been working to establish new procedures for awarding National Literacy Act Section 321 funds to eligible CBOs, adult schools, community colleges, and library literacy programs. Scope and Methodology The sample selected by your auditors clearly did not represent a cross section of all 2 CBOs, according to size and ethnicity of population served and may, therefore, ¤ contribute to false conclusions. As indicated in Attachment A enclosed, the 10 CBOs visited by the Bureau of State Audits were from among the largest CBOs in the state. The chart demonstrates the wide range of funding levels among the 86 CBOs who received funds for ESL-Citizenship. The sample of 10 chosen by the Bureau of State Audits is a subset of the 18 largest CBOs that received funds from CDE, not representative of the entire group of CBOs. In addition, your sample does not well represent the ethnicity of the students served by the 86 CBOs. Nine of the 10 CBOs sampled served predominantly Hispanic students. Chapter 1 Evidence to Support Hours Claimed and Demonstrate Student Gains in Skill CDE recognizes that some CBOs showed deficiencies in providing evidence to support claimed services. The CBOs are often unsophisticated in the administration of their programs and focus primarily on providing services to as many clients as possible. Records demonstrate that huge numbers of students have been served by CBOs as indicated by the large number of instructional sites, the abundance of classes offered at various hours of the day and night, and the numerous student folders showing individual progress. CBO’s instructional staff are primarily volunteer and part-time, a combination that creates a formidable task for providing training and developing consistency in recordkeeping of student attendance and performance outcomes. The turnover in staff, both administrative and instructional, in the largely volunteer organizations provides challenges to CBOs attempting to serve a large number of clients and implement and sustain an adequate data collection system at the same time. CDE staff generally found CBOs passionate about the work they do for their community, well intentioned about keeping accurate records, and successful in helping their clients in the citizenship application process. CDE staff were impressed to see that several large CBOs developed close relationships with Immigration and Naturalization Services (INS) to provide the most comprehensive services possible. However, CDE staff became frustrated with the CBOs inability to comply with the recordkeeping functions demanded by the state grant process. Challenges faced by all ¤We have not included attachments in the report. However, with the exception of confidential information on adult education students included in the attachments, they are available at the California State Auditor’s Office. R-4 literacy providers in meeting the requirements were magnified at many CBOs, especially when they attempted to serve a community in more than one county. Because of the lack of infrastructure to implement CDE suggestions and strategies, some CBOs did not take full advantage of the technical assistance and compliance activities provided to them by CDE. In the meantime, CDE management faced the mandate to allocate a $7.7 - $12.57 million set aside of the National Literacy Act allocation, specifically for ESL-Citizenship, as directed in State Budget Act language. Legislative staff and many communities viewed CBOs as significant resources for addressing the critical need for services. As noted in the draft report on pages 10 and 11, CDE staff have conducted monitoring visits to CBOs. In fact, CDE made a concerted effort in 1995-96 to monitor the ability of the CBOs to keep the required individual student documentation of attendance and progress. CDE staff visited all CBOs and viewed a sample of student records. At each site, staff looked at the student folders to determine if they were complete. It is true that the CDE’s review of some agencies in 1995-96, utilizing a checklist process, did not indicate compliance issues regarding student gains in skills in some agencies. However, there is evidence among other CDE files of CBOs that some CDE staff did review and 3 document data on student gains in skills. Examples of this documentation are provided in Attachment B. ¤ Some CDE staff did note significant problems in the area of demonstrating student gains in skill. For example, in October 1995, an Adult Education Office consultant visited CBO 8 and determined the agency “out of compliance.” Reasons stated for non- compliance included the number of test scores that did not show a gain in skill level, completion of test preparation, and completed practice items. CDE marked the agency 4 as compliant in January 1996 after they provided the appropriate evidence. However, CDE’s continued monitoring efforts discovered that the agency did not remain in compliance. When different staff visited in December 1998, they again marked CBO 8 “out of compliance” for its assessment system and other programmatic issues. Due to the volunteer status of the staff at many CBOs, the number of outreach sites for instruction, and the sometimes-large geographical distance between the instructional sites, CBO volunteer staff at times become confused about utilizing appropriate test materials. All agencies that administer the Comprehensive Adult Student Assessment System (CASAS) test must send a representative to training on how to administer the test and utilize the results for planning of instruction and program improvement. No test forms are distributed to agencies without training. In many CBOs, the individual that received training for the agency is unable to train all of the volunteers that use the materials or leaves the agency for permanent employment. Then, mistakes in the administration of the test become common. ¤We have not included attachments in the report. However, with the exception of confidential information on adult education students included in the attachments, they are available at the California State Auditor’s Office. R-5 The new management team established within the Education Equity, Access and Support Branch of CDE has established training for Adult Education Office consultant staff to ensure that meaningful compliance reviews are conducted and technical assistance to recipient CBOs, adult schools, community colleges, and library literacy programs is provided. The manager of the Adult Education Office holds weekly staff meetings, teams of staff develop monitoring and technical assistance documents, and CASAS assessment experts provide training to CDE staff. CDE’s Monitoring Efforts CDE staff conducted site visits of the CBOs in the Bureau of State Audits sample. However, monitoring of programs appears to be inconsistent among consultant staff. Some staff are impressed with the community outreach and success with large number of the clients served and are lenient in their expectations of the recordkeeping capabilities. Other staff are highly critical of the CBOs, in general, and emphasize compliance rather than technical assistance during visits to the programs. The new manager of the Adult Education Office holds weekly staff meetings to provide opportunities for staff to discuss ways to consistently monitor programs and to develop criteria for ensuring consistency among compliance review efforts. Staff are beginning to document in more detail their visits to agencies and outline their areas of concern 5 and non-compliance. The areas of concern and non-compliance are tracked on a newly-established database to determine areas where technical assistance are needed and to track resolution of non-compliance items. Plans for Monitoring under the Revamped Federal Program Changes in federal law require the adoption of a new state plan under the Workforce Investment Act, Title II, Adult Education and Family Literacy Act (AEFLA). During 1998- 99, the Adult Education Office successfully prepared for adoption of a new state plan that addresses the intent of the new federal legislation: to help adults become literate and obtain educational skills needed for employment, provide parents with skills necessary for the education of their children, and assist adults complete a secondary school education. Because AEFLA significantly changes the previous federal requirements through the implementation of a performance accountability system, including core indicators of performance, CDE included in the new plan major reforms for allocating funds and documenting provider performance. New eligibility requirements and reimbursement rates for the adult education program are outlined in the new 1998-99 state plan. This first year of implementation is one of transition, in which CDE will provide USDE with an estimate of the number students statewide for which the new performance levels will be met using the new benchmarks. In each of the next two years, the estimated number of students for which performance levels will be met will increase. In year four, the CDE R-6 hopes that all California adult education students will meet the new performance levels. AEFLA provides both challenges and opportunities for developing continuous improvement processes. Consequently, CDE is developing a new monitoring instrument. Staff in the Adult Education Office began developing this instrument in mid-June. They are using input from the evaluation of the application review process to verify that the instrument will include all of the necessary compliance items to test support for claimed services, including review of attendance records, summary documents, and tests showing attainment of benchmarks. It is the intent of the Adult Education Office that the monitoring instrument also serve as a technical assistance and a self-review document for participating agencies. 6 Contrary to the claim on page 6 of the draft report that the Adult Education Office has not yet developed new monitoring procedures, on June 29, 1999 the CDE provided the Bureau of State Auditors with a copy of the first draft of the monitoring document as proof that CDE staff are developing the procedures. CDE’s deadline for publication of the document is July 22, 1999. The CASAS has a continuous process in place for ongoing development, field testing and dissemination of new, age appropriate, performance test items, and tests. CASAS validates performance test items with adult students to meet field needs for classroom assessment as well as to meet federal and state requirements for program accountability. Recent federal and state legislation, with the increasing emphasis on high stakes accountability at the local program level, are directing the current test development process. Several new reading tests were made available to agencies 7 during this past year. In addition, an ESL Level Completion Test for Beginning Level ESL was validated and disseminated last year. Additional test forms at each instructional level for reading, listening comprehension, and math are currently being field tested, as well as two additional ESL Level Completion Tests, and will be available during this school year. A committee of field practitioners met in June 1999 to prioritize ¤ additional assessment needs for the next two years. Attachment C lists all test forms currently available to the field and the release date of anticipated tests. Teaching to the test is always a concern in educational assessment. However, instructors are encouraged to teach students how to take a test and how to respond to different test formats. CASAS is also working to develop computerized assessment that will automatically adjust the test items to the appropriate curriculum items and level of the student. ¤We have not included attachments in the report. However, with the exception of confidential information on adult education students included in the attachments, they are available at the California State Auditor’s Office. R-7 Chapter 2 8 CDE Treatment of Different CBOs is Consistent CDE has taken action to ensure that each CBO is treated similarly. Until 1998-99, agencies had thirteen months after the end of a fiscal year in which to submit an independent audit if they received more than $25,000 in federal funds and agencies could submit their audits for two fiscal years. Consequently, an agency could receive a grant for two years before an audit report became due. This was problematic for CDE, especially if there were material findings in the audit report submitted. Therefore, in 1998-99 the new leadership for the Adult Education program included in the agency assurances for acceptance of the grant, a change in the deadline for submitting the audit report to six months after the end of the agencies’ fiscal year. Agency audit reports are now due annually. Some CBOs operated under a different fiscal year due to the requirements of their base funding. Historically, CDE made allowances to those agencies and extended the submission date to match the agency fiscal calendar. In these situations CDE staff visits and observations were the basis for funding the agencies. This was true of CBO 3 that 9 had an audit year from January 1-December 31 in the funding year 1995-96. Therefore, the audit report was not due until February 1, 1998, too late to influence the 1997-98 application process. If CDE was inconsistent in funding and de-funding agencies based on performance, it is due to an unsophisticated file-sharing process. Three analysts in the Adult Education Office collected and reported different aspects of the program reports. One handled fiscal allocations, one worked with student enrollment records, and one worked with the External Audits Office to evaluate agency audit reports. Data was kept individually by each of the analysts with no technical way to share or cross-reference the information. Similarly, each regional consultant kept individual agency files for follow up and monitoring. Except for the comprehensive overview of all programs serving ESL- Citizenship students in 1995-96, there was no time when CDE organized a comprehensive list of findings. CDE had no central file for sharing information and analyzing comprehensive problems. In 1998-99, the new management team directed a process to reformat the databases and network the computers so the Adult Education office staff has quick reference to all aspects of each agency’s report and status. CDE staff now record each agency visit on ¤ a Contact Report (Attachment D) and then enter the visit findings and issues into a master data base for tracking trends needed for technical assistance and identifying ¤We have not included attachments in the report. However, with the exception of confidential information on adult education students included in the attachments, they are available at the California State Auditor’s Office. R-8 programs with and without compliance issues. This database has been invaluable to quickly determine agency needs, changes in funding levels, and identification of compliance and audit issues. 0 Contrary to the draft report at page 14, CBO 8 was initially not awarded a grant in 1996- 97 because of its missing audit report for January 1994-June 1995, which was due August 1, 1996. When CDE received the audit report on November 28, 1997, it awarded 1996-97 funding to CBO 8. As mentioned above, CDE has requested that agencies annually submit an audit report within six months of the end of their fiscal cycle. Of the CBOs identified in the report, CDE has reviewed eight of the ten agencies’ audit reports for completeness. Adult Education Office staff continue to follow up on overdue audit reports and send billing requests to those organizations that are out of compliance with Office of Management and Budget Circular A-133 audit requirements. In keeping q with established application review policy, CDE did not accept as eligible for funding any application for 1998-99 and 1999-00 from an agency that did not submit a required audit report. Department Measured the Qualitative Value of Applications Consistently The draft report starting at page 14 questions the adequacy of CDE’s grant award process for 1998-99. However, Adult Education Office files indicate a grant award ¤ process in place for 1998-99 (Attachment E). CDE staff meeting minutes indicate that on May 18, 1998, the criteria and process for reading the application was reviewed and discussed with all staff. It should be noted that only where it was evident that CDE staff did not follow the review criteria did the manager of the Adult Education Office choose w to overturn the rating given a grant application. The manager did not overturn a decision where more than one rater “failed” an application. The CDE attempted to consistently review and rate each of the 1998-99 grant applications as evidenced by the June 22, 1998, CDE Staff Meeting minutes indicate that: 321 Applications continue to be read. Poorly written applications require technical assistance from the regional consultant if the agency is eligible to receive funds. If major revisions need to be made, we can hold the Letters of Encumbrance over the summer until revisions are submitted. Applications for those agencies who have not submitted the CASAS test scores and Intake/Update forms are being held. CBOs required submitting an audit (if Federal funds exceeded $25,000 in 1994-95 and 1995-96) will not be funded in 98-99 if their audits are not submitted with acceptable findings. ¤We have not included attachments in the report. However, with the exception of confidential information on adult education students included in the attachments, they are available at the California State Auditor’s Office. R-9 In addition, the draft report does not recognize the analysis and conclusion statement of the Internal Audit report of October 22, 1998 that states that the manager assigned in May 1998 “has been effective in implementing corrective action in response to audit issues. Of the nine outstanding audit issues, IMA staff recommends closing or clearing four issues, resulting in five remaining findings. These five findings remain open even though sufficient work has been done toward achieving corrective action but not closure. IMA staff intend to continue to provide assistance toward achieving complete closure.” Department Taking Appropriate Action to Address Significant Problems with One CBO e Contrary to statements on pages 16 and 17 of the draft report, CDE is taking appropriate action to address significant problems with CBO 1. CDE took proactive steps to investigate the audit findings of CBO 1. A team of Adult Education staff, the Division Director, and CDE auditors met with CBO 1 officials in May 1997. This meeting met legislative intent to support CBOs in their efforts to provide services, particularly in the case of CBO 1. The parties involved in the meeting reached significant resolution of the issues; CDE receiving assurance that the audit findings of CBO 1 could be resolved. This meeting and the assurances led CDE to authorize funding for 1997-98. However, CDE was not able to monitor CBO 1’s corrective action because of unexpected and dramatic CDE management staffing and personnel changes. CDE meetings during 1997-98 produced sufficient evidence to indicate that the assurances made by CBO 1 had not been completed and, therefore, CDE filed claims requesting that CBO 1 return its 1997-98 funding. CDE continues to determine the actual level of service provided by CBO 1 in 1997-98. The extent to which CBO- provided services met the full number of students claimed for funding is currently in litigation. Adult Education Office Verifies Reasonable Levels of Funding Contrary to claims made on pages 17-18 of the draft report, CDE did verify the reasonableness of requested hours claimed for FY 1998-99. As stated to the auditors on several occasions, and with documentation to back up the claims, the Adult r Education Office asked applicant agencies to substantiate that the numbers requested for the funding year were reasonable compared with the total number served by that agency in previous years. Adjustments to requests were made by the applicant agencies based on the CDE inquiries. The Adult Education Office is currently utilizing the same process to compare numbers of students served in past years and the reasonableness of the projected number of students meeting benchmarks before publicizing final numbers of projected students to be served in 1999-00. In contrast to the draft report claims that CDE refused to provide the number of students that applicants hope to educate in 1999-00, CDE recognizes that it is inappropriate to present inflated or undervalued numbers prior to concluding any application review process. New procedures for funding performance levels, rather R-10 than attendance and number of students, have challenged applicant agencies to estimate performance data not required in previous years. The Adult Education Office is in the process of evaluating the reasonableness of the numbers of students to be served and meeting the benchmarks. The draft report concludes on page 18 that when agencies are unable to utilize all authorized funds, fewer funds are available for allocation to other service providers. However, CDE has in place processes and procedures to reallocate unused funds. During the years 1994-95 and 1995-96, the Department of Finance granted CDE authority to redistribute unused funds during the funding year. The requests for ESL- Citizenship funds were incredibly high during this time and agencies were allocated only a percentage of funds requested in their original application. It was a relatively simple process at that time to allocate a greater percentage of funds in the final payment. The amount of funding available for local programs is outlined in Attachment F. This chart indicates that in each funding year there was an additional amount added which was the unused funds from the prior year. For example, in 1995-96, CDE had $11,202,000 to allocate to local programs for ESL-Citizenship, plus $3,533,000 from prior year funds. In that year, 51% of the available funds was allocated to ESL- Citizenship programs. In 1996-97, CDE had $7,925,000 to allocate to local programs for ESL-Citizenship and an additional $256,000 from prior year funds. In that year, 50% of the available money was allocated to ESL-Citizenship programs. However, in 1996-97, the Department of Finance no longer allowed CDE to request authority to spend unused funds during the current year. Therefore, since 1997-98, if CDE determines mid-year that agencies will not be able to utilize their total grant award amount, CDE must wait until the following funding year to augment grants. The new manager and the staff in the Adult Education Office addressed the issue of appropriate funding levels during the reviews of grant applications for 1998-99. Grant requests for funding were compared with the previous year’s data. Where the requested grant amounts were significantly more or less than in previous years, adult education staff made calls to the individual agencies to request clarification of their grant request. In most cases, the agencies responded by lowering the amount of the grant funds requested. However, even with this analysis of “reasonableness of the total number of student hours” awarded to agencies, unexpected changes in student access of services can occur. In 1998-99, for example, agencies noted a tremendous decrease in the number of ESL-Citizenship students. Numerous phone calls to CBOs and other literacy providers indicated three barriers to potential ESL-Citizenship students. First, and foremost, is the increase in INS fees to process citizenship applications. Many applicants cannot afford the $250 fee. Second, the wait for the actual INS interview is R-11 now up to three years in the highly populated regions of the state. Therefore, clients wait to take ESL-Citizenship classes until just prior to their interview. And, third, a predominantly older population of eligible aliens is currently in the system learning ESL skills before entering the ESL-Citizenship classroom. Generally, younger students tend to progress more quickly in the preparation classes and have already completed ESL- Citizenship instruction. The older students need more time to learn enough English to benefit from the ESL-Citizenship instruction. As a result of the decrease in the number of ESL-Citizenship students, several CBOs t are requesting less than their original allocation for 1998-99. For example, CBO 4’s mid-year report requests 37.5% less for 1998-99 and it did not apply for 1999-00 funding. CBO 9 requests 73% and CBO 6 will receive 80% of the original spending authority for 1998- 99. Statewide, agencies have notified the Adult Education Office that there has been an unexpected decline in need for ESL-Citizenship funds for this fiscal year but an increased need for funding in the ESL program. Therefore the management team with oversight of the Adult Education Office has determined an unallocated amount of over $4 million dollars for 1998-99. To utilize all of the available funds, management has augmented ABE and ESL funds for 1998-99. Reauthorization of Federal Adult Education and Literacy Programs: New Legislation and New Guidelines On August 7, 1998, President Clinton signed into law Public Law 105-220, the Workforce Investment Act (WIA). The intent of the legislation was to extend, coordinate, and consolidate federal programs for employment and job training, adult education and literacy, and vocational rehabilitation. Title II of WIA is the AEFLA, which repealed the Adult Education Act (AEA) and the National Literacy Act of 1991, Public Law 102-73, but amended and extended major provisions for adult education and literacy through fiscal year 2003. These programs, which originated in the 1960s, currently provide services to help adults become literate and obtain educational skills needed for employment, provide parents with skills necessary for the education of their children, and help adults complete their secondary school education. Most of the programs and activities that were authorized by the AEA and funded in fiscal year 1998 are continued by the AEFLA. However, the AEFLA significantly augments previous AEA requirements through the implementation of a performance accountability system, including core indicators of performance. This system is to be used to measure program effectiveness and progress at the state and local levels and to award state incentive grants; performance results are to be considered in making local awards. In addition, the AEFLA expands the purpose of the adult education and literacy programs formerly authorized by the AEA specifically to include assistance for parents to improve the educational development of their children. R-12 Under AEFLA, CDE must make competitive, multi-year grants or contracts to eligible providers to develop, implement, and improve adult education and literacy services within the state. Each provider must use the award to provide services or instruction in at least one of the following categories: adult education and literacy services - including workplace literacy services - family literacy services, and English literacy programs. In making awards, CDE is required to take into consideration 12 specific qualities regarding the eligible provider, including past effectiveness, a commitment to serve those most in need of literacy services, the use of research and technological advances, and coordination with other community resources. Each state then weights the level of importance of each of the 12 criteria in the selection process. In California, the new State Plan and application process place the greatest emphasis on identified need within the communities served by the agencies, the implementation of a management information system (MIS), and the past effectiveness of the agency in serving students. These priorities resulted from an extensive public hearing process. Other considerations include: (1) intensity and duration of the program that allow participants to achieve substantial learning gains; (2) use of instructional practices that are research-based; (3) learning in real life contexts to ensure that an individual has the skills needed to compete in the workplace and exercise the rights and responsibilities of citizenship; (4) staff that is well-trained, including instructors, counselors and administrators; and (5) the ability to provide a high-quality information management system. Not more than 5% of each award can be used for planning, administration, personnel development, and interagency coordination. Throughout the State Plan and application process for implementation of AEFLA, USDE clearly stated that benchmarks of student success are necessary for compliance to the new regulations. In addition, only those states with demonstrated gains will continue to receive funding. By insisting that benchmarks be met, California in the long run ensures fiscal support for agencies providing services to adult learners. Should we allow any agency to receive funding without the capacity to document student outcomes, CDE provides a disservice to funded agencies because of the potential loss of substantial funding in California. Also, by requiring the documentation of student benchmarks, CDE is responding to the implied message from federal legislators that state programs will not continue to receive funds if they cannot demonstrate student outcomes. Students are assisted to stay in programs long enough to become literate and obtain the knowledge and skills necessary for employment and self-sufficiency, obtain the educational skills necessary to become full partners in the educational development of their children, or complete a secondary school education. It is important to emphasize that states must achieve their estimated number of students meeting performance levels in order to continue to receive the federal funding. Local agencies participating in AEFLA receive the R-13 necessary MIS software system from CDE to assist their efforts to document student outcomes. California faces sanctions if performance goals are not met by the fourth year. The draft report (page 19) expresses concern that a decline in participation of CBOs under these new legislated requirements will make it difficult for adult schools and community colleges to maintain their current level of service and make up for that of nonparticipating CBOs. Historically, we note that in 1990, 170 agencies applied for the federal adult education program funding. By 1994-95, with a decline in SLIAG funding and the advent of the ESL-Citizenship set-aside in California, and the significant raise in reimbursement rate for ESL-Citizenship, CDE received over 375 applications. During the development of the new state plan in 1998-99, and as required by federal law, CDE conducted public hearings to assess the adequacy of the set aside and state rates of reimbursement to adult education programs. During those hearings, CDE gathered strong input from among those who testified and urged consistent reimbursement rates among funded programs in ABE, ESL, and ESL-Citizenship. As a result of that input, the new state plan, approved by USDE in June 1999, states that all programs will be funded at the same reimbursement rate and that programs will be reimbursed based on the number of student gains, not by hours of attendance. Eighty percent of the state allocation to local providers will be utilized for serving students most in need of adult education services, including those in ABE, ESL, and ESL-Citizenship. Currently, the new management team of the adult education program is identifying how to meet adult education needs in underserved areas of the state in 1999-00 which resulted from the decrease in the number of applicants for funding and the number of unfundable applications. The Adult Education Office plans to release a second application for agencies within these underserved areas. CDE staff will provide assistance to interested agencies in the application process. It is our understanding that present Budget Act Language is intended to broaden the y earmarked federal funds for either ESL or ESL-Citizenship instruction. Due to the significant decrease in the number of ESL-Citizenship students in 1998-99, CDE recognizes that the inclusion of ESL in the set-aside language is crucial. There is no disagreement with the draft report recommendation that CDE evaluate funding requests for 1999-00 in light of prior-year performance and the size of the service provider before authorizing grant awards, especially since that process worked well in the allocation of 1998-99 funds. R-14 COMMENTS California State Auditor’s Comments on the Response From the California Department of Education T o provide clarity and perspective, we are commenting on the California Department of Education(cid:146)s (department) response to our audit report. The numbers correspond to the numbers we have placed in the response. 1 As we stated in the report, the $7.7 million set-aside was not specifically for community-based organizations (CBOs), but rather for all agencies providing citizenship and naturalization services. 2 Our selection included CBOs with small, moderate, and large annual awards. In fact, the 10 CBOs we selected for testing received annual awards ranging from $24,000 to $3,500,000 between fiscal years 1994-95 and 1998-99. Moreover, 7 of the 41 annual awards we reviewed for these CBOs fell below $100,000. Regarding the ethnicity of students served by the selected CBOs, it is not surprising that our sample was heavily weighted toward groups serving Hispanic students. According to the department(cid:146)s latest Student Progress and Goal Attainment Report, 80.1 percent of CBO students were Hispanic, while White, Non-Hispanic, and Asian students made up 8.2 percent and 8.1 percent, respectively. We note that CBO 7 served Southeast Asian immigrants, while CBO 10 served Southeast Asian and Russian, as well as Hispanic, immigrants. 3 The documentation the department provided (Attachment B of the department(cid:146)s response) relates to reviews at school districts and community colleges, not CBOs. 4 We repeatedly asked the department to provide us with copies of reviews it had conducted at the 10 CBOs we visited. During our audit, the department did not provide us with any evidence of its reviews at CBO 8 beyond an internal memo from February 1996 indicating that staff approved payment of a claim based on the results of a site visit. R-15 5 As we noted in our report, the department(cid:146)s site visits to CBOs rarely noted compliance issues related to claimed hours or gains in skill. A database that tracks the results of compliance reviews will be useless until the department revamps its monitoring efforts to more critically evaluate claimed services. 6 After we received the department(cid:146)s draft monitoring procedures, we revised our report; however, as we state in the report, we believe the draft procedures are inadequate. 7 According to Comprehensive Adult Student Assessment System (CASAS) and the list the department furnished, only one reading test was published in the last year. Per CASAS, other reading tests disseminated in the last four years have either been for advanced students or were not designed to be used to measure progress through comparison of scores. As we stated in the report, CASAS only plans on releasing two new reading tests for release in 1999. No other reading tests to measure progress are currently in development. 8 We do not agree that the department ensures consistent treatment of each CBO and believe that the examples cited in Chapter 2 of our report demonstrate these inconsistencies. 9 CBO 3 received more than $25,000 in adult education funding for services it generated during calendar year 1995. It was therefore required to provide the department with an audit report covering that calendar year by February 1, 1997, early enough to influence the fiscal year 1997-98 application process. 0 The department is confusing funding years. Our finding relates to the department(cid:146)s award of funding to CBO 8 for fiscal year 1997-98, not 1996-97. q In Chapter 2, we gave the department credit for improving this part of its award process. w Our finding related to funding in fiscal year 1997-98, not 1998-99. Regardless of whether the manager or another staff member made the final decision, the department awarded funding to three applicants that were failed by two raters. R-16 e As we note in the report, the department was well aware of fiscal problems at CBO 1. Given the large size of this CBO(cid:146)s awards, the fact that department management and staff changed during the period does not seem a reasonable excuse for failing to undertake timely corrective action. In our view, the department(cid:146)s eventual actions are an example of too little, too late. r In our example from fiscal year 1998-99, the department inquired about the many hours CBO 6 requested and the CBO justified the request, saying that it needed to serve remote areas and the unem- ployed. The department allowed the request to stand but did not require the CBO to substantiate its ability to expand its program so dramatically. We believe this decision was unreasonable. t CBOs may be reducing their requests for ESL-citizenship funding in fiscal year 1999-2000 due to reasons other than those the department cites, such as the reduced funding rate. In fact, CBO 4 indicated to us that it was dissatisfied with the adult education program because the department(cid:146)s late funding decisions hindered its ability to deliver steady services. It did not indicate that the demand for its ESL-citizenship classes had declined. y The department needs to read the budget act more carefully. The language regarding earmarking in the enacted budget for fiscal year 1999-2000 is identical to the language contained in state budgets for the last five years. It specifically sets aside $12.6 million for (cid:147)citizenship and naturalization services for legal permanent residents who are eligible for naturalization,(cid:148) as we state in our report. We fail to see how the department can broaden its interpretation of this language to use the funds for general ESL classes without violating the law. R-17 cc: Members of the Legislature Office of the Lieutenant Governor Attorney General State Controller Legislative Analyst Assembly Office of Research Senate Office of Research Assmembly Majority/Minority Consultants Senate Majority/Minority Consultants Capitol Press Corps R-18