Summary

Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult June 2005 2004-120 rotiduA etatS ainrofilaC S T I D U A E T A T S F O U A E R U B The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TTY (916) 445-0033 OR This report is also available on the World Wide Web http://www.bsa.ca.gov/bsa/ The California State Auditor is pleased to announce the availability of an on-line subscription service. For information on how to subscribe, please contact the Information Technology Unit at (916) 445-0255, ext. 456, or visit our Web site at www.bsa.ca.gov/bsa Alternate format reports available upon request. Permission is granted to reproduce reports. � � � ��������� ���� ������ ������������� ������������������� ������������ ����������������������� June 16, 2005 2004-120 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the Bureau of State Audits presents its audit report concerning the administration and monitoring of state and federal supplemental English learner program funds by the Department of Education (department) and a sample of school districts. This report concludes that the department is charged with distributing program funds for three main programs that address the needs of public school students who are not yet fluent in English, known as English learners. The department, however, provides leeway to school districts in establishing certain criteria they use both to identify students as English learners and to redesignate them as fluent in English. Significant differences exist in the stringency of school districts’ criteria causing funding variances and a lack of comparability in performance results across the State. Moreover, school district and department monitoring of schools’ adherence to the redesignation process is inadequate, causing students who meet the criteria for fluent status to remain as English learners. In addition, the department provides school districts little guidance on documenting expenditures and performs limited monitoring of their expenditure of English learner program funds, resulting in some questionable and unallowable uses of these funds by school districts. Further, the department’s evaluation of the effectiveness of particular English learner programs is weak, and a recent independent evaluation of the English Language Acquisition Program has not provided decisive answers regarding that program’s effectiveness. Finally, although the department’s funding formulas are generally sound, the formula for Impact Aid needs updating. Respectfully submitted, ELAINE M. HOWLE State Auditor ������������������� �������������������������������������������������� ���������������������������������������� ������������������ CONTENTS Summary 1 Introduction 5 Audit Results School Districts Are Inconsistent in the Criteria They Use to Identify and Redesignate English Learners 18 Inadequate Monitoring of the Redesignation Process Causes Students Who Have Met School District Criteria for Fluency to Remain in the English Learner Population 25 Diverse Designation and Redesignation Criteria and Inconsistent Implementation of These Criteria May Cause Funding Variances and Hinder Comparisons of Performance Results 29 Minimal Monitoring of Expenditures Allows School Districts to Use Some Funds for Unallowable Costs 33 The Department Measures English Learner Progress in Language Proficiency and Academics, but Its Evaluation of the Contribution of Specific English Learner Programs Is Weak 39 Funding Formulas Are Generally Equitable, but a Poverty Statistic for Impact Aid Needs Updating 45 Recommendations 49 Appendix Measurable Achievement Objectives Established for Title III 53 Responses to the Audit California Department of Education 55 California State Auditor’s Comments on the Response From the California Department of Education 63 Anaheim Union High School District 67 California State Auditor’s Comments on the Response From the Anaheim Union High School District 69 Long Beach Unified School District 71 Los Angeles Unified School District 73 Pajaro Valley Unified School District 79 California State Auditor’s Comments on the Response From the Pajaro Valley Unified School District 81 Sacramento Unified School District 83 California State Auditor’s Comments on the Response From the Sacramento Unified School District 87 San Diego Unified School District 89 California State Auditor’s Comments on the Response From the San Diego Unified School District 95 San Francisco Unified School District 97 California State Auditor’s Comments on the Response From the San Francisco Unified School District 99 Stockton Unified School District 101 California State Auditor’s Comments on the Response From the Stockton Unified School District 105 California State Auditor Report 2004-120 11 SUMMARY RESULTS IN BRIEF The Department of Education (department) distributes the funds for three programs that address the needs of public school students who are not yet fluent in English, known Audit Highlights . . . as English learners. However, the approach the department and the school districts use to manage and monitor these Our review of the administration supplemental programs is inadequate, allowing for funding and monitoring of English variances, a lack of comparability in performance results learner programs by the between school districts, and the use of funds for unallowable Department of Education and questionable purposes. (department) and a sample of school districts found that: The department distributes funds for its three main English þ The department provides learner programs—federal Title III–Limited English Proficient school districts leeway in setting certain criteria they and Immigrant Students (Title III), state Economic Impact Aid use to identify students as (Impact Aid), and the state English Language Acquisition Program English learners and to (ELAP)—according to established criteria, and it measures the redesignate them as fluent. progress of English learners according to established standards. þ Differences in school However, it provides school districts leeway in establishing certain districts’ identification criteria they use to identify students as English learners and to and redesignation redesignate them as fluent in English. As a result, some school criteria cause funding variances and a lack districts have developed more stringent criteria than those included of comparability in in the department’s guidelines. In noting this fact, we are performance results. not concluding that a particular criterion or scoring standard is þ Sixty-two percent of the preferable to another, but rather that inter-district variation exists. 180 English learners we reviewed, who For example, one school district we reviewed requires students were candidates for to score at the early advanced level in all three language skill redesignation but had not been redesignated, met areas of the California English Language Development Test school districts’ criteria for (CELDT) in order to be considered fluent, while the department’s fluent status but were still guidelines recommend only intermediate scores. In fiscal counted as English learners. year 2003–04, this school district categorized 8 percent of the þ School district and students it tested for initial categorization as fluent. It would department monitoring have categorized an additional 19 percent as fluent if it had used of schools’ adherence to the department’s more lenient criteria. the redesignation process is inadequate. Moreover, some students remain in the English learner population continued on next page . . . after they have met the criteria to be recognized as fluent because school districts fail to monitor the student redesignation process. In addition, the department’s coordinated compliance review did not, until May 2005, include guidance for its consultants to review current English learners’ records to ensure that they California State Auditor Report 2004-120 11 þ Of 180 tested expenditures, are designated correctly. In fact, some schools we reviewed failed to eight were for unallowable initiate, complete, or adhere to their districts’ redesignation process. purposes and 43 were Of the 180 students we reviewed at eight school districts who questionable. were candidates for redesignation in fiscal year 2003–04 but had þ The department performs not been redesignated, 111 met their school districts’ criteria for limited monitoring of redesignation but remained in the English learner population. In school districts’ addition, 21 others had been redesignated as fluent by their schools expenditure of English learner program funds. but were still listed as English learners in their district’s database. þ The State’s evaluation of Because the number of English learners enrolled is a primary the impact of particular English learner programs factor in funding formulas for English learner programs, some is weak. school districts likely receive higher funding under both state and federal English learner programs than they would if their criteria þ The funding formula for were aligned more closely with those of other school districts or if Impact Aid is complicated and likely outdated. they did a better job of completing their redesignation processes. Further, school districts with tougher redesignation criteria retain a larger proportion of English learners who perform well on the CELDT. This appears to make it easier for those school districts to meet one of the three statewide performance objectives established by the department under the federal No Child Left Behind Act, of which Title III is a part. The total funding for the three largest English learner programs was roughly $605 million in fiscal year 2003–04, and the department distributed most of these funds to school districts. The majority of these funds at the eight school districts we sampled were spent on salaries and benefits for teachers and staff. However, the department provides little guidance to school districts on how to document their use of these funds, and it does limited monitoring of the districts’ expenditures. As a result, some school districts have inadequate documentation practices and sometimes spend funds for unallowable or questionable purposes. Of the 180 expenditures we tested, eight were for unallowable purposes and 43 were questionable. Most of the questionable expenditures related to purchases that had no contemporaneous documentation linking the expense to English learners or had documentation indicating that the purchased goods or services covered non-English learners as well as English learners. In addition, two of the eight school districts we reviewed spent ELAP funds at schools or on activities that are not covered by the grant award. One district spent $11 million in ELAP funds in fiscal year 2003–04 on an extended learning program that covered a range of underachieving students in kindergarten through eighth grade, even though ELAP funds are restricted to English learners in fourth through eighth grades. 22 California State Auditor Report 2004-120 California State Auditor Report 2004-120 33 Although the department measures school districts’ success in improving English learner progress in language proficiency and academics based on student performance on statewide tests, its evaluation of the contribution of specific English learner programs to this success is weak. The State appears never to have evaluated the effectiveness of the Impact Aid program in improving the academic performance of English learners, although it was established more than 25 years ago. With regard to ELAP, program evaluators hired by the department have been unable to reach decisive conclusions on the program’s value in part because school districts combine ELAP with other funding sources to pay for a variety of English learner services and because student performance results are not comparable across school districts. Finally, although the department’s formulas for distributing English learner program funds are generally sound, the funding formula for Impact Aid is complicated and likely outdated. The Legislative Analyst’s Office (legislative analyst) has observed that the complexity of the Impact Aid formula results in district allocations that are hard to understand based on underlying school district demographics and that the formula is weighted heavily toward poverty. Further, a key statistic used in the formula, the number of students in families receiving assistance under the California Work Opportunity and Responsibility to Kids (CalWORKs) program, has become less and less reflective of the population of students in poverty and is currently unavailable to the department. The governor vetoed a bill redirecting funds to study the Impact Aid formula, instead directing the Department of Finance and the Secretary of Education to work with the legislative analyst and the department to develop options for restructuring the formula. The department indicates that it will collaborate to develop a long-term solution for allocating Impact Aid funds, including determining an appropriate replacement for the CalWORKs data. RECOMMENDATIONS The department, in consultation with stakeholders, should establish required initial designation and redesignation criteria related to statewide tests that would provide greater consistency in the English learner population across the State. The department should pursue legislative action, as necessary, to achieve this goal. In addition, the department should require school districts to document redesignation decisions, including decisions against redesignating students who are candidates for fluent status. 22 California State Auditor Report 2004-120 California State Auditor Report 2004-120 33 School districts should ensure that their redesignation criteria include each of the four criteria required by state law for redesignating English learners to fluent status. They also should monitor their designation and redesignation processes more closely to ensure that schools actually complete the process and that school district databases accurately reflect all redesignations. The department should consider changing the annual objective that measures students’ annual progress in learning English to offer less incentive for school districts to maintain students as English learners. The department should perform the steps necessary to ensure the school districts we reviewed have taken appropriate action to resolve their unallowable expenditures of supplemental English learner program funds. The department should revise the documentation policy it provides to school districts to better ensure that expenditures are directed clearly at activities that serve the English learner programs’ target populations. School districts should implement documentation policies to ensure that expenditure files clearly demonstrate that supplemental English learner program funds are directed at activities that serve the law’s target populations. The department should continue to work with the Department of Finance, the legislative analyst, and the Legislature to revise the Impact Aid funding formula to include statistics that better measure the number of students in poverty. AGENCY COMMENTS Some school districts we reviewed are concerned that the report may be interpreted to imply that their more stringent redesignation criteria are inappropriate or that the results of the Bureau of State Audits’ testing of student records may be misapplied to the entire population of English learners who are candidates for redesignation. Nevertheless, the school districts generally indicate that they intend to implement our recommendations. The department believes that the law restrains it from establishing criteria that all school districts must follow. It also says that its monitoring efforts have been stronger than we indicate in the report. The department, however, intends to implement most of our recommendations. n 44 California State Auditor Report 2004-120 California State Auditor Report 2004-120 55 INTRODUCTION BACKGROUND Students in kindergarten through grade 12 with limited proficiency in English represent a significant portion of the State’s public school students. These students traditionally have been designated as “limited English proficient”; more recently, the State has adopted the term “English learners.”1 As Figure 1 on the following page shows, the State’s enrollment of English learners has grown almost fivefold over the past 24 years, and its proportion of the State’s total public school student enrollment has increased by more than threefold. Specifically, English learner enrollments have increased from about 326,000, or 8 percent, of the State’s 4.1 million public school students in fiscal year 1979–80 to roughly 1.6 million, or about 25 percent, of the State’s 6.3 million public school students in fiscal year 2003–04. After a rapid rise in the late 1980s to mid-1990s, the proportion of English learners has stabilized and represents about one-quarter of all students. Over the past 40 years, federal and state courts, Congress, the California Legislature, and the voters of California have considered the issue of how best to educate English learners. Generally, federal courts recognize that English learners have a right to equal access to education under the federal Civil Rights Act of 1964 (Civil Rights Act). 1For the purposes of this report, the term “English learners” includes students who were not born in the United States or whose native language is a language other than English or who come from an environment in which a language other than English is dominant, and who therefore may not meet the proficient level of achievement on state assessments. 44 California State Auditor Report 2004-120 California State Auditor Report 2004-120 55 FIGURE 1 English Learners as a Percentage of Total Public School Enrollment ��� ��������� �� �� �� �� ������� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� � � �� �� � � � � �� � � �� � � �� � � �� � � � � � � � � � � � � � � � � � � � � � � � � � � � ����������� Source: Department of Education’s California Basic Educational Data System database and annual language census. As shown in Figure 2, in 1974 the United States Supreme Court considered a class action suit originating from the San Francisco Unified School District, alleging that the school district’s failure to provide English language instruction and adequate instructional procedures to Chinese-speaking students violated the Civil Rights Act because it denied those students a meaningful opportunity to participate in the school district’s public educational program. In its decision, known as Lau v. Nichols, the court found that by failing to provide adequate English instruction, the school system was denying these students the opportunity to obtain the education received by other students in the school system. The court stated that “basic English skills are at the very core of what the public schools teach,” and found that the school district had an obligation to take affirmative steps to rectify the language deficiency so that its instructional program would be available to these students. 66 California State Auditor Report 2004-120 California State Auditor Report 2004-120 77 FIGURE 2 Significant Legislation and Court Decisions Related to English Learner Instruction �������������� ������������������� ���������������������������� ����������������������� ������������� ����������������� ������������������������ ��������������������� ���� ���� ���� ���� ���� �������������� ������������������������ ������������������� ��������������������� ������������������������� Subsequent to Lau v. Nichols, in the federal Equal Educational Opportunities Act of 1974 (EEOA), Congress defined “impermissible denial of educational opportunity” to include “the failure by an educational agency to take appropriate action to overcome language barriers that impede equal participation by students in an instructional program.” In 1981, a federal appellate court considered what obligations the EEOA placed on educational agencies. In deciding Castaneda v. Pickard, the court found that language remediation programs should (1) be based on sound educational principles or theories, (2) be implemented effectively, for example, through the availability of qualified staff to implement the program, and (3) include a system to evaluate their effectiveness in overcoming language barriers. Subsequent court decisions have affirmed this decision and have clarified that under the EEOA states have a responsibility to take appropriate action to overcome language barriers and cannot completely delegate this responsibility to local educational agencies. In California, local educational agencies include school districts, charter schools, county offices of education, special education local plan areas, regional occupational centers, and three state diagnostic centers. Although these various entities are all local educational agencies, the overwhelming majority of state and federal funding for English learners is administered by school districts, and thus we use the term “school district” rather than “local educational agency” throughout this report. 66 California State Auditor Report 2004-120 California State Auditor Report 2004-120 77 A Consent Decree Required Extra Monitoring by the Department In 1976, California enacted the Chacon-Moscone Bilingual- Bicultural Act (Chacon-Moscone Act) to establish transitional bilingual education programs to meet the needs of English learners. The Chacon-Moscone Act required school districts to teach English learners in a manner consistent with federal court decisions, laws, and regulations. It also called for native- language instruction for English learners as they made the transition to English fl uency. In 1979, a parent committee fi led suit against the state Department of Education (department), alleging that the department had failed to comply with the Chacon-Moscone Act and federal laws designed to ensure that English learners receive adequate instruction. In 1985, the parties agreed to settle that lawsuit under a court order that became known as the Comité Consent Decree (consent decree). The consent decree required the department to undertake specifi c steps in monitoring the Chacon-Moscone Act, including conducting coordinated compliance reviews (compliance reviews) at each school district in the State on a three-year cycle, performing audits of the English learner enrollments reported annually by school districts, and performing annual on-site follow-up reviews of at least 10 school districts to ensure that compliance review issues were being resolved. In 1987, the Chacon- Moscone Act expired because of a sunset provision, meaning that the act remained on the books, The Department of Education’s but school districts receiving funds were required compliance review includes procedures only to meet its general purposes. In view of the to assess whether school districts’ programs for English learners: sunset provision, the department asked the court to terminate the consent decree. The court granted • Are based on acceptable standards the department’s request to remove the enrollment and include a process for determining audit requirement, amended the original consent effectiveness. decree to lengthen the department’s compliance • Are based on student needs and refl ect review cycle from three years to four years, and acceptable educational practices. made other amendments to the consent decree. • Ensure equal access to educational services. • Are staffed by qualifi ed educators who In June 1998, California voters approved have access to professional training. Proposition 227, which expressed a strong • Involve parents and the community. preference for teaching English learners in English, except under special conditions. In view of the • Are managed and operated within legal requirements. requirements of Proposition 227, the department again asked the court to terminate the consent • Refl ect fi nancial plans and practices decree, and this time the request was granted. In an that meet legal requirements and school district priorities. unpublished decision issued in 2004, a California appellate court upheld the 2002 decision to terminate 88 California State Auditor Report 2004-120 California State Auditor Report 2004-120 99 the consent decree but expressed its very strong disappointment with the department’s efforts toward meeting the requirements of the consent decree. It reminded the department that it must continue to monitor school districts, and that lifting the decree simply meant that it no longer was required to monitor them in the manner specified in the consent decree. At the time the court terminated the consent decree, a number of school districts were subject to the on-site follow-up reviews previously described. As a result, the department decided to complete the follow-up reviews for these remaining school districts. As of April 2005, 18 school districts still were resolving their compliance review issues. Moreover, the department continues its compliance reviews of English learner programs and services on a four-year cycle for most school districts in the State. These compliance reviews ensure that school districts meet various requirements, and they are conducted by consultants who observe classroom lessons, ensure that only qualified teachers provide instruction, determine whether school districts provide adequate teacher training, and ensure that school district plans address curricula for English learners. STATE AND FEDERAL GOVERNMENTS HAVE ESTABLISHED A NUMBER OF SUPPLEMENTAL PROGRAMS FOR ENGLISH LEARNERS In fiscal year 2003–04, the department provided roughly $630 million in state and federal funding to school districts to supplement English learner programs. The bulk of the money was disbursed through three programs: state Economic Impact Aid (Impact Aid), federal Title III–Limited English Proficient and Immigrant Students (Title III), and the state English Language Acquisition Program (ELAP). As shown in Figure 3 on the following page, these three programs totaled $605 million in fiscal year 2003–04, or 96 percent of total supplementary English learner funding. This supplemental funding amounted to a little more than 1 percent of the total $56.8 billion in revenues for kindergarten through grade 12 education in fiscal year 2003–04. 88 California State Auditor Report 2004-120 California State Auditor Report 2004-120 99 FIGURE 3 Fiscal Year 2003–04 Budgets for Supplemental English Language Programs (Dollars in Millions) ��������������� ��������������� ���������������������������� ��������� ����������� �������������������� ��������������������������� Sources: 2003–04 Final Budget Summary and Department of Education budget data by school district. Impact Aid The Legislature created the Impact Aid program in July 1979 to support programs for educationally disadvantaged youth and bilingual education. This program provides school districts with funding to support additional programs and services for English learners and to offer compensatory education services for educationally disadvantaged students. School districts use their funding for a variety of purposes, including supplemental instructional services to English learners, training of teachers who instruct English learners, and supplementary educational materials. The allocation formula for Impact Aid is complex, and the money is divided among school districts based on two schedules that focus primarily on students whose families receive public assistance or live in poverty, and on English learners. According to the department’s records, in fiscal year 2003–04, school districts received an average of about $236 per disadvantaged student under this program. A further discussion of the Impact Aid formula appears in the Audit Results section of this report. 1100 California State Auditor Report 2004-120 California State Auditor Report 2004-120 1111 Title III In January 2002, the federal No Child Left Behind Act became law, providing financial support to school districts under Title III. The federal government awards the Title III grant to the State each year based on the number of English learners and immigrant students in the State. The department divides the State’s total award by the number of English learner and immigrant students to arrive at a per pupil funding rate. For fiscal year 2003–04, school districts received approximately $77 for each English learner and immigrant student, with $119.3 million set aside to provide services to roughly 1.6 million English learners statewide. The Limited English Proficient portion of Title III provides funds to improve the education of English learners by assisting them in attaining English proficiency and meeting state standards for academic content and student academic achievement. These funds must be used to provide supplementary services related to English language development instruction, enhanced instruction in core subjects, and professional development for teachers and other staff. Up to 15 percent of Title III funds may be earmarked for immigrant students. In fiscal year 2003–04, the department allocated roughly $13.5 million, or about 10 percent of the Title III award, to provide supplementary programs and services to just more than 175,000 immigrant students and their families. These funds pay for activities such as family literacy programs, community outreach, and instructional services that assist immigrant students in meeting the same standards as mainstream students. ELAP ELAP was established in July 1999, subsequent to the passage of Proposition 227. The purpose of this program is to improve the English proficiency of California’s English learners in grades four through eight and to better prepare them to meet state standards for academic content and performance. ELAP pursues this goal by providing funds to schools for conducting academic assessments, providing English language development instruction, and offering supplemental instructional support, such as summer school. In fiscal year 2003–04, the department allocated $53.2 million in ELAP funds for English learners in grades four through eight. The law states that schools that have implemented the California 1100 California State Auditor Report 2004-120 California State Auditor Report 2004-120 1111 English Language Development Test (CELDT) also may receive $100 on a one-time basis for each English learner enrolled in kindergarten through grade 12 who was redesignated as fluent English proficient (fluent). According to the administrator for the department’s language policy and leadership office, however, as of fiscal year 2004–05, the department has not distributed ELAP funds for this purpose because the funds have not been available since the establishment of the CELDT. PROGRAMS HAVE FEW RESTRICTIONS REGARDING ALLOWABLE COSTS There are few restrictions on how Title III funds can be used. The No Child Left Behind Act requires that Title III funds advance the education of English learners and supplement, not supplant, local expenditures. In other words, Title III funds cannot replace federal, state, or local public funds that would have been spent on English learners in the absence of Title III funds. Some examples of allowable activities for Title III funds specifically cited in the law include providing tutorials and academic or vocational education for English learners, providing community participation programs to improve the English language skills of English learners, and improving instruction of English learners by acquiring educational technology or instructional materials. Similarly, few requirements are placed on state Impact Aid and ELAP funds. The California Education Code requires that Impact Aid funds serve and assist English learners and supplement, not supplant, local expenditures. Likewise, the California Education Code requires only that ELAP funds supplement existing resources supporting language acquisition for English learners in grades four through eight. Among allowable activities for ELAP cited in the law are conducting academic assessments of English learners to ensure appropriate placement, providing instruction to assist pupils in meeting English language development standards, and providing supplemental instructional support such as intersession or summer school. ELAP funds are required to be allocated to specific school sites and directed to English learners in grades four through eight. 1122 California State Auditor Report 2004-120 California State Auditor Report 2004-120 1133 THE DEPARTMENT RELIES ON STANDARD TESTS TO DETERMINE ENGLISH PROFICIENCY AND MEASURE STUDENT ACHIEVEMENT To better assess the English language development of pupils whose primary language is a language other than English, in 1997 the Legislature required that the department create a test to identify and measure English language profi ciency. The department complied by approving and fi nancing the CELDT, which ranks students on a fi ve-tiered scale covering three skill areas. School districts are to use the CELDT for the initial identifi cation of English learners as well as for part of the annual process of evaluating these students’ progress in acquiring the skills of listening and speaking, reading, and writing in English, until they can be redesignated as fl uent under school district- specifi c criteria. The Audit Results section of this report further discusses the fl exibility school districts have in determining the English learner status of their students. The department collaborated with an outside contractor to develop the CELDT and the scoring levels shown in the text box for students in kindergarten through grade 12. The test design built upon an existing language assessment test administered throughout the United States. Although it was built upon an existing The CELDT evaluates selected students’ test, the CELDT underwent a series of internal language ability related to the following: and external reviews to ensure the reliability • Listening and speaking skills and validity of its test items. In addition to • Reading skills developing the test, the contractor is responsible for ensuring the security of the testing process, • Writing skills including requiring confi dentiality agreements • Overall profi ciency with its reviewers and security agreements with CELDT scores place tested students in its staff and subcontractors, and for developing the following levels for each skill area and implementing an extensive process for and overall profi ciency: administering, scoring, and reporting results for • Beginning the test. • Early intermediate • Intermediate In addition to requiring that the department • Early advanced develop a test to identify and measure English • Advanced profi ciency, in 1997 the Legislature required the California State Board of Education (board) to designate a single achievement test aligned with state academic content standards to measure how well students learn required academic skills. As a result, the department commissioned the California Standards Test (CST), which is administered annually under the Standardized Testing and Reporting program to measure students’ achievement in 1122 California State Auditor Report 2004-120 California State Auditor Report 2004-120 1133 meeting state content standards. These standards, adopted in 1997 and 1998 by the board, cover four major content areas: English language arts, mathematics, history/social science, and science. Unlike the CELDT, all students, including English learners, enrolled in grades two through 11 in California public schools are required to take the CST2 unless exempted by a guardian’s written request. The department uses the CELDT and the CST collectively to satisfy the accountability requirements in the federal No Child Left Behind Act of 2001. This act requires states receiving Title III funds to establish English language proficiency standards, identify or develop evaluation measures that assess English language proficiency, and use annual measurable achievement objectives (annual objectives) to monitor the progress of English learners in attaining proficiency and meeting academic content and achievement standards. In total, approximately 1.8 million students took the CELDT in fiscal year 2003–04. Of these, about 433,000 took it for the purpose of initial designation as English learners, and the remaining students took the test as part of their annual assessment. In addition, roughly 1.2 million English learners took the CST in fiscal year 2003–04, which represented 25 percent of the total student population who took the test in that fiscal year. SCOPE AND METHODOLOGY The Joint Legislative Audit Committee (audit committee) requested that the Bureau of State Audits review the administration and monitoring of state and federal English learner program (English learner) funds at the department and a sample of local recipients. Specifically, the audit committee asked us to examine the processes the department and a sample of local recipients use to determine the eligibility of students for the English learner programs, including an evaluation of the criteria used to determine eligibility for these programs and a determination of whether local recipients redesignate students once they become fluent in English. The audit committee also asked us to review the department’s processes for allocating program funds to local recipients and to determine whether the processes are equitable and based on established criteria. Our audit also included evaluating the department’s process for monitoring local recipients’ management and expenditure of program funds and, for selected 2Students with significant cognitive disabilities who are not able to take the CST are assessed using the California Alternate Performance Assessment. 1144 California State Auditor Report 2004-120 California State Auditor Report 2004-120 1155 local recipients, testing a sample of expenditures to determine whether they were used for allowable purposes. Lastly, the audit committee asked us to determine how the department measures the effectiveness of the English learner programs, including a review of any studies or evaluations that identify whether students with limited English proficiency show improvement as a result of receiving services under these programs. To identify English learner programs, we reviewed the Department of Finance’s fiscal years 2003–04 and 2004–05 Final Budget Summary and other departmental information, such as accounting records, the department’s Web site, and the Coordinated Compliance Review Training Guide. Further, we reviewed the relevant California Education Code sections and the federal No Child Left Behind Act of 2001. We identified the English learner programs with the largest amount of funds in fiscal year 2003–04. Based on this identification, we selected the State’s Impact Aid and ELAP programs, along with the federal Title III program, as the focus of our audit. We reviewed and evaluated the laws, rules, and regulations associated with each program, as well as court cases affecting English learner instruction. To choose our sample of local recipients, we identified school districts with a large number of English learners. We selected a total of eight school districts across the State with a range of redesignation rates, as identified by the department’s Web site. The eight school districts we selected using this method were Anaheim Union High School District, Long Beach Unified School District, Los Angeles Unified School District, Pajaro Valley Unified School District, Sacramento City Unified School District, San Diego City Unified School District, San Francisco Unified School District, and Stockton Unified School District. These school districts accounted for 453,000, or about 28 percent of California’s English learners in fiscal year 2003–04 and they redesignated roughly 28,000 students in that year. To determine the processes used by the department and our sample of school districts to determine whether students are limited in their English proficiency and whether school districts redesignate students once they become fluent in English, we interviewed staff at the department and the eight school districts. We also reviewed documentation relating to the reliability of the standardized tests used for determining language status. We obtained test score data from the test contractors and the eight school districts. We assessed the reliability of the test contractors’ data by performing electronic testing of critical data elements and by reviewing security agreements and affidavits. We assessed the reliability of 1144 California State Auditor Report 2004-120 California State Auditor Report 2004-120 1155 school districts’ data by performing electronic testing of critical data elements, by comparing the school district’s data to the test contractors’ data, and by comparing the number of English learners in the data files to the number reported to the department. We determined that the data from the test contractors and the school districts were sufficiently reliable for the purposes of our report. Based on this data, we determined that the eight school districts accurately track English learner CELDT scores and that their reported number of English learners materially agrees with the number of English learners in their databases. For seven of the eight school districts, we selected a sample of 20 students’ files from each district to determine whether the school districts adhered to their initial eligibility designation criteria and their redesignation criteria. We focused our testing on English learners who were candidates for redesignation in fiscal year 2003–04, but who had not been redesignated as fluent. For the eight school districts we reviewed, there were approximately 42,000 such English learners. Due to the size of the Los Angeles Unified School District, we selected 40 of the school district’s student files for testing. To determine the department’s processes for allocating program funds to local recipients and whether the allocations are equitable, we reviewed department documentation relative to Impact Aid, ELAP, and Title III funding. We also reviewed the Legislative Analyst’s Office Analysis of the 2004–05 Budget Bill: Economic Impact Aid. Further, we reviewed the department’s fiscal year 2004–05 awards and determined whether they adhered to the formulas for the various programs. To understand the department’s process for monitoring local recipients’ management and expenditure of English learner program funds, we interviewed department staff and reviewed the latest compliance review for each of our sampled school districts. We analyzed the school districts’ expenditures to obtain an understanding of their use of English learner funds and tested transactions for each selected school district to determine whether expenditures were for allowable purposes. To determine how the department measures the effectiveness of the English learner programs, we interviewed department staff members to identify any evaluations or reports that have been conducted relating to the effectiveness of these programs. We also performed an Internet search to determine whether other evaluations or reports had been completed relating to the effectiveness of the English learner programs, and we reviewed the reports we found. n 1166 California State Auditor Report 2004-120 California State Auditor Report 2004-120 1177 AUDIT RESULTS The approach that the Department of Education (department) and school districts use to manage and monitor supplemental English learner programs needs improvement. The department distributes funds for English learner programs according to established criteria and measures the progress of English learners according to established standards. However, the leeway it provides school districts in establishing certain criteria they use to identify and redesignate English learners as fluent English proficient (fluent), and the insufficiency of department and school district monitoring of the redesignation process and of program expenditures, allow for funding variances, lack of comparability in performance results between school districts, and the use of funds for unallowable and questionable purposes. Specifically, funding is skewed and performance results are not comparable across the State because some school districts use more stringent criteria to redesignate English learners as fluent. The failure of the department and school districts to monitor the student redesignation process also has led to some students remaining in the English learner population after they meet the criteria for fluency, as some schools fail to initiate, complete, or adhere to their district’s redesignation process. Further, the department provides little guidance on documenting expenditures and performs limited monitoring of school districts’ use of English learner program funds, so some school districts have inadequate documentation practices and sometimes spend funds on unallowable and questionable activities. The department measures school districts’ success in improving English learner progress in language proficiency and academics based on student performance on statewide tests, but its ability to evaluate the contribution of specific English learner programs is weak. Program evaluators have been unable to reach decisive conclusions as to the value of individual English learner programs in part because school districts combine funding sources to pay for English learner services, and because student performance results are not comparable across school districts. Finally, although the department’s formulas for distributing English learner program funds are generally sound, the funding formula for the Economic Impact Aid (Impact Aid) program is complicated and likely outdated, and has been criticized as obscure. 1166 California State Auditor Report 2004-120 California State Auditor Report 2004-120 1177 SCHOOL DISTRICTS ARE INCONSISTENT IN THE CRITERIA THEY USE TO IDENTIFY AND REDESIGNATE ENGLISH LEARNERS Although the department has provided guidance to school districts for establishing criteria to identify students as English learners and to redesignate them as fl uent,3 it has allowed the school districts some latitude in setting test score thresholds for redesignation. State law Required criteria for redesignation requires school districts to use California English include the following: Language Development Test (CELDT) results as the • Assessment of language profi ciency. primary indicator for their initial identifi cation of pupils as English learners, and as the fi rst of • Performance in basic skills demonstrating the ability to participate effectively in a four specifi c criteria for redesignating English curriculum designed for pupils of the same learners as fl uent. State law also requires the age whose native language is English. department, with the approval of the California • Teacher evaluation. State Board of Education (board), to use at least the four criteria defi ned in law and shown in the • Parental opinion and consultation. text box to establish procedures for redesignating English learners to fl uent status. In September 2002, the department published board-approved guidance for school districts to use in developing their initial and redesignation criteria. Because these are not regulations, school districts are not required to adhere to the department’s guidelines. However, according to the board’s chief legal counsel, the guidelines were based on an analysis of actual test data and developed with public input, so the board expects that school districts will pay great deference to them when making their initial identifi cation and their redesignation decisions. Nevertheless, these are only guidelines and school districts are allowed fl exibility in defi ning their criteria. District Criteria for the Initial Designation of English Learners Vary The department’s guidance on the initial identifi cation of students as English learners indicates that school districts should administer a home language survey (survey) to the guardians of new enrollees to determine a student’s primary language. The survey includes questions such as what language the student fi rst learned, what language is spoken most often by adults at home, and what language is spoken most often by the student 3Fluent students are redesignated according to criteria established by their school districts and demonstrate an English language profi ciency comparable to pupils of the same age whose native language is English. 1188 California State Auditor Report 2004-120 California State Auditor Report 2004-120 1199 at home. If the survey indicates that a new enrollee’s primary language is English, the student should be placed in a regular instructional program. If the survey indicates a primary language other than English, the student should take the CELDT to assess English proficiency. According to the department’s guidance, to be identified as fluent, new enrollees taking the CELDT should attain a score placing them in the early advanced or advanced proficiency categories overall, and should receive no score below intermediate in the areas of listening and speaking, reading, and writing (skill areas). The department recommends that students achieving such scores should be identified as fluent and placed in a regular instructional program. It advises that all students scoring below these levels should be designated as English learners and should receive English language instructional services from their school district. In reviewing the criteria and processes used by eight school districts for the initial identification of English learners, we noted that Three of the eight school five of the districts follow the department’s guidance, and the districts we reviewed remaining three impose more stringent standards on new enrollees. hold new enrollees to In noting this fact, we are not concluding that a particular scoring higher scoring standards standard is preferable to another, but rather that inter-district on the CELDT than the variation exists. The Anaheim Union High (Anaheim), Sacramento department’s guidance City Unified (Sacramento), and San Diego City Unified (San Diego) indicates for initial school districts hold new enrollees to higher scoring standards on identification as an the CELDT than the department’s guidance requires. Instead of English learner. using the intermediate designation to gauge fluency, these school districts require new enrollees to score at least early advanced in one or more of the skill areas described earlier. For example, Sacramento requires new enrollees to score at least early advanced in each of the CELDT skill areas to be identified as fluent. Similarly, San Diego and Anaheim require early advanced scores in at least two of the three CELDT skill areas while accepting a score of intermediate for the remaining skill area. Further, Anaheim requires students to attain certain minimum scores on the CELDT’s short written composition, a subsection of the writing skill area. In addition to its more stringent criteria for each of the CELDT skill areas, Sacramento requires new enrollees to achieve an overall score of advanced on the CELDT to be identified as fluent. When school districts set test score thresholds for fluency that are higher than those recommended by the department, they end up with larger English learner enrollments. For instance, Sacramento initially identified as fluent 8 percent of its new enrollees who took the CELDT during fiscal year 2003–04. If this 1188 California State Auditor Report 2004-120 California State Auditor Report 2004-120 1199 school district had mirrored the department guidelines for initial identifi cation, it would have identifi ed an additional 19 percent of its new enrollees as fl uent. As we discuss in more detail later, state and federal funding for English learner programs is affected by the relative size of school districts’ English learner enrollments, so varying criteria such as those just described can cause funding variances between school districts. In the case of Sacramento, the school district received about $74,000 in extra English learner program funds for the additional English learners. Redesignation Criteria Are Even More Variable Among School Districts School districts have even more discretion in establishing criteria for redesignating students from English learner to fl uent status. Because state law requires the department, with the board’s approval, to establish procedures The CST evaluates students’ mastery of for redesignating English learners as fl uent, the state-adopted content standards in the department has established four criteria for school following subjects: districts to use as guidelines in establishing their own criteria. These criteria, in accordance with • English/language arts state law, consist of student performance on the • Mathematics CELDT and the California Standards Test (CST) in English language arts (CST-ELA), as well as a • History/social science teacher evaluation of academic performance, and • Science parental opinion. CST scores range from 150 to 600 and place students in the following levels The department set the CELDT scores English for each skill area: learners should attain to be considered for redesignation at the same level it set for the initial • Far below basic identifi cation of English learners. The department’s • Below basic redesignation criteria also defi ne a range of scores • Basic—scores for this item range from English learners should achieve on the CST-ELA— 300 to 349. A score of 325 is referred to from basic to mid-basic as shown in the text box. as mid-basic School districts are given the discretion to require • Profi cient higher CELDT scores and are to choose a specifi c score English learners must attain within the • Advanced recommended CST range. The department guidance indicates that school district criteria should include teacher evaluation of student academic performance, as required by law. Some districts we reviewed have interpreted this to include grades in specifi c subjects, overall grade point averages, and performance on additional school district-specifi c assessments, while one set no requirements in this area. 2200 California State Auditor Report 2004-120 California State Auditor Report 2004-120 2211 According to the department, discretion was provided to school districts in keeping with the case law because it believed most districts had reasonable redesignation processes in place that addressed relevant local conditions, including beliefs about the level of proficiency needed to succeed in their respective regular curricula. Further, the department explained that school districts’ beliefs about the benefits of redesignation vary. Some believe that English learners benefit from more conservative redesignation criteria that ensure the continuation of needed English learner services; others believe that more liberal redesignation criteria encourage the mainstreaming of English learners as quickly as possible to allow students full access to the curriculum and instruction needed to ensure academic success. The department believes the current guidelines allow school districts a degree of flexibility in making these decisions in accordance with local conditions. Anaheim echoed the department’s statement regarding more conservative redesignation criteria. Its coordinator of English learner programs said that her district’s students, who are in grades seven through 12, face complex and sophisticated demands, such as those reflected in the high school exit exam, in a short time frame. As such, she said Anaheim wants to assure that English learners receive ongoing supplementary services and meet community expectations when they are redesignated as fluent. Although we recognize that school districts have varying Although we recognize perspectives regarding redesignation, it is also important that that school districts have they employ similar redesignation processes to ensure consistent varying perspectives funding and performance measurement across the State. As part regarding redesignation, of a five-year study of the implementation of Proposition 227, it is also important that the American Institutes for Research and WestEd (evaluators) they employ similar reported in 2003 that varying school district criteria and redesignation processes differing procedures appear to cause redesignation rates and to ensure consistent even the meaning of redesignation to vary widely across funding and performance school districts. (We discuss this study in more detail later in measurement across this report.) Greater standardization in the criteria related to the State. statewide tests, for example, could provide more consistency in the definition of fluency, while still allowing school districts discretion with regard to criteria related to teacher evaluations. 2200 California State Auditor Report 2004-120 California State Auditor Report 2004-120 2211 In reviewing eight school districts’ redesignation criteria, we noted significant variances from the department’s guidelines, as well as differences when compared to one another, as demonstrated in Table 1. As a result, some school districts have established additional or more rigorous criteria that their English learners must meet to attain fluent status when compared to other school districts. In noting this fact, we are not concluding that a particular criterion or Significant variances exist scoring standard is preferable to another, but rather that inter-district in the rigor of the criteria variation exists. For example, Sacramento requires English learners school districts require to attain scores of at least early advanced in the CELDT’s skill areas, English learners to meet to while San Diego requires scores of at least early advanced in two skill attain fluent status. areas for a student to be considered for redesignation to fluent status. Other school districts, including Stockton Unified School District (Stockton) and Los Angeles Unified School District (Los Angeles), require only intermediate scores. School districts with criteria requiring higher CELDT scores for redesignation appear to have larger proportions of English learners scoring at the early advanced and advanced levels. This is likely due to the retention of high scorers in these districts’ English learner populations. For example, 31 percent and 27 percent of the English learners in San Diego and Sacramento, respectively, met their school districts’ CELDT redesignation criteria in fiscal year 2003–04. However, if these school districts had followed the department’s guidance on CELDT scores, an additional 4 percent and 20 percent of English learners in San Diego and Sacramento, respectively, would have met the CELDT criteria for redesignation. As a result, these school districts appear to be maintaining larger proportions of English learners scoring in the upper levels on the CELDT. As we discuss later, these larger proportions of high-scoring English learners may allow school districts with more stringent redesignation criteria to achieve higher performance results. Moreover, two of the eight school districts we reviewed require English learners to score higher on the CST-ELA than others in order to be considered for redesignation. Specifically, Sacramento and San Francisco Unified School District (San Francisco) require scores of at least 324 and 325, respectively, on the CST-ELA, while the remaining six school districts require scores of at least 300. This can have a significant impact on a school district’s pool of redesignation candidates. For example, 29 percent and 30 percent of English learners who took the CST-ELA in Sacramento and San Francisco, respectively, scored at or above their school district’s required proficiency level. However, if these districts required the same CST-ELA scores as other districts, an additional 21 percent and 18 percent of English learners in Sacramento and San Francisco, respectively, would have met this portion of the redesignation criteria. 2222 California State Auditor Report 2004-120 California State Auditor Report 2004-120 2233 TABLE 1 Summary of Tested School Districts’ Redesignation Criteria, Ordered by the Percentage of English Learners Scoring Proficient on the California English Language Development Test (Based on Fiscal Year 2003–04 Redesignation Criteria) CELDT Scores California Standards Test Scores Teacher Evaluation Percentage of English Learners Scoring Listening Proficient on and English Language Other School the CELDT in Overall Speaking Reading Writing Arts§ Math Grades Other Evaluations District Criteria Fiscal Year 2003–04 Department Early Intermediate Intermediate Intermediate Between basic and No Student academic performance 40% of Education advanced or higher or higher or higher mid-basic established (all grades) (department) or higher guidelines 54% redesignation (grades 7 through 12) guidelines and state average* Anaheim Union † † Early † Proficient or higher † Grade point average of at least 2.0 Passing grade on district Students must be 54 High School advanced or for students in grade with grades of C or higher in current writing assessments or enrolled in the District higher seven. Basic or English courses and D or higher in California High School Exit district’s program at higher for students current core subjects Exam least one year in other grades Sacramento † Early Early Early Mid-basic or higher Mid-basic Teacher recommendation based on the Passing grades on Students must be in 47 City Unified advanced advanced advanced or higher teacher’s knowledge of the student’s assessments including grade three or above School District or higher or higher or higher concepts and skills writing and math San Francisco † † † † Mid-basic or higher † Grades of C or higher in core subjects Advanced levels or higher on Students must be in 43 Unified School district language and literacy grade three or above District assessment Los Angeles Unified † † † † Basic or higher † Grades of at least satisfactory or C in Passing grades on district 40 School District English and math courses English assessments for students in primary school Long Beach Unified † † † † Basic or higher Basic or Teachers should consider student performance in core subjects and on 36 School District higherll district English and Math assessments San Diego † Early Early Early Basic or higher † Teacher represents that he/she believes student will be able to perform Students must be in 35 City Unified advanced advanced advanced grade level work in core subject areas in a regular program grade two or above School District or higher‡ or higher‡ or higher‡ Pajaro Valley Unified † † † † Basic or higher Basic or Passing score on Students must be in 27 School District higher reading, writing, and English grade three or above assessments Stockton Unified † † † † Basic or higher§ † Does not require a teacher evaluation Students must be in 27 School District grade three or above Sources: School Districts’ 2003–04 redesignation criteria, and department’s CELDT data and redesignation guidelines. * The department also recommends that school districts notify guardians of their right to participate in the redesignation process, and that they not redesignate kindergarteners. All school districts we tested incorporated this guidance into their redesignation criteria. † The school district has established criteria for this category that emulates department guidance. ‡ Students may score intermediate in one skill area of the CELDT but must score early advanced or higher in the other skill areas. § The department’s guidelines allow school districts to determine whether factors other than English proficiency are responsible for an English learners’ low performance on the California Standards Test-English language arts. As a result, Stockton Unified School District uses other indicators for redesignation, including assessments in the areas of writing, language arts, and reading if student scores are below basic. ll Long Beach suggests that teachers consider student performance on this test when evaluating them for fluent status. 2222 California State Auditor Report 2004-120 California State Auditor Report 2004-120 2233 In another example, according to the coordinator of English learner programs at Anaheim, the school district changed its criterion for the CST-ELA from 300 in fiscal year 2003–04 to 325 in fiscal year 2004–05. According to the coordinator, Anaheim did so on the advice of the department consultant who conducted a compliance review in fiscal year 2003–04 and said the previous cutoff score was too low. She noted that this change in criteria eliminated 1,535 English learners from consideration for redesignation in fiscal year 2004–05 who would have met the old criteria. Further, although not included in the department’s guidelines on redesignation, Sacramento and Pajaro Valley Unified School District (Pajaro) require English learners to demonstrate a proficiency level ranging from basic to mid-basic on the CST mathematics exam. The remaining six school districts do not include a requirement for this exam in their redesignation criteria. Although the department has given school districts some flexibility with respect to the test scores needed for redesignation, the districts have complete discretion in establishing criteria for teacher evaluation of student academic performance. Five of the eight school districts we reviewed have established criteria in this area by requiring English learners to meet specific standards on additional school district evaluations, including writing and math assessments. These assessments probably create additional variances in the rate at which English learners attain fluent status. Most school districts we reviewed have added a number of extra assessments as part of their academic review, but Stockton does not require any teacher evaluation, which does not appear to conform to the law’s requirement. As such, this school district may redesignate students who have developed proficiency in English but have not yet demonstrated the ability to compete academically with pupils of the same age whose native language is English. According to the assistant director of curriculum and professional development, Stockton plans to include a teacher evaluation as part of the redesignation criteria beginning in fiscal year 2005–06, More stringent criteria pending approval of its school board in August 2005. result in some school districts reporting larger More stringent criteria result in some school districts reporting English learner enrollments larger English learner enrollments than they would if they than they would if they established criteria more comparable to those of other school established criteria more districts. English learner enrollments are a primary factor used in comparable to those of funding formulas for English learner programs, so some school other school districts. districts likely receive more funding than they would if their criteria were aligned more closely with the department’s guidance and criteria used by other districts. 2244 California State Auditor Report 2004-120 California State Auditor Report 2004-120 2255 INADEQUATE MONITORING OF THE REDESIGNATION PROCESS CAUSES STUDENTS WHO HAVE MET SCHOOL DISTRICT CRITERIA FOR FLUENCY TO REMAIN IN THE ENGLISH LEARNER POPULATION Although school districts generally appear to identify English learners appropriately when they enroll new students, they do not do as good a job of ensuring that English learners who meet minimum school district redesignation criteria are removed from the English learner population. In reviewing redesignations at eight school districts, we found that schools often failed to redesignate English learners who had met the district criteria. Moreover, when schools appropriately redesignated English learners to fluent status, we noted that the school districts did not always update their student information databases to reflect the change, and thus they continued to report some fluent students to the department as English learners. One factor contributing to these weak processes is the Partly because school inadequate monitoring effort school districts employ to ensure district and department that schools adhere to their redesignation processes. Another monitoring of schools’ factor is the department’s coordinated compliance review adherence to the (compliance review), which includes testing of fluent students to redesignation process is ensure that they meet redesignation criteria, but did not, until inadequate, schools fail May 2005, include guidance for its consultants to test current to redesignate all English English learners’ records to ensure that they are designated learners who meet the correctly. Without adequate monitoring, the school districts and criteria for fluent status. the department lack assurance that English learners who have met the criteria for fluency are redesignated consistently. The eight school districts we reviewed generally initiate the redesignation process by distributing to schools lists of English learners they have identified as candidates for fluent status, based on data including CELDT scores. It is up to the individual schools to complete the process by obtaining results for the remainder of the evaluation, such as teacher evaluations and parental input. This process differs from the initial identification of English learners at these school districts, which generally is initiated and completed at the school level when new students enroll. As we discussed previously, the department requires schools to administer the home language survey to guardians of new enrollees to determine students’ primary language. If the survey indicates a language other than English, the school must administer the CELDT4 to determine whether the student is an English learner or fluent. 4Before the CELDT, which was first administered in the State during May and October 2001, school districts administered another assessment to determine new enrollees’ English proficiency. For the purposes of this report, we will refer to the initial assessment as the CELDT because it is the only assessment currently used throughout the State to determine new enrollees’ English proficiency. 2244 California State Auditor Report 2004-120 California State Auditor Report 2004-120 2255 In testing whether various schools within the eight sampled school districts adequately adhered to their districts’ initial identification processes, we noted few exceptions, as shown in Table 2. Six of the 180 students we reviewed were identified incorrectly as English learners, while 13 other students did not have adequate documentation on file, such as a home language survey or CELDT scores, to confirm their English learner designations. Specifically, three of the six students we identified as incorrectly designated had met their school district’s initial identification criteria for fluency by scoring early advanced or higher on the CELDT, while another two students’ districts recognized them as native English speakers. The remaining student had been redesignated previously as fluent. For instance, two students in Long Beach Unified School District (Long Beach) had met the school district’s initial identification criteria for fluency by scoring advanced overall with no skill area score below intermediate; however, the school district designated them as English learners. According to the assistant director of program assistance for language minority students at Long Beach, these students were designated erroneously as English learners and should have been designated initially as fluent. She explained that such errors occurred because of the high turnover in school staff responsible for initial designations. Although the schools we reviewed consistently adhered to their districts’ initial identification processes, we noted that most of the same schools failed to fully complete, and in some cases Sixty-two percent of even begin, the process of redesignating English learners to the English learners we fluent status. Specifically, 111 (62 percent) of the 180 English reviewed met the school learners we reviewed met the school districts’ redesignation districts’ redesignation criteria but had not been redesignated to fluent in the school criteria for fluent status district records. We focused our testing on English learners who but were still categorized were candidates for redesignation in fiscal year 2003–04, but as English learners. who had not been redesignated as fluent. There were about 42,000 such students at the eight school districts we reviewed. For example, 19 of the 20 students we reviewed in San Francisco met the school district’s redesignation criteria for fluent status. San Francisco’s executive director of multilingual programs explained that the students we reviewed remained as English learners because the respective schools did not always begin or complete the redesignation process, as it was not a high priority. Similarly, 14 of the 20 students we reviewed at Pajaro met their school district’s redesignation criteria for fluency. However, according to the director of federal and state programs, their 2266 California State Auditor Report 2004-120 California State Auditor Report 2004-120 2277 schools either did not finish the redesignation process or, if the process was completed, did not document why the student was not redesignated. Pajaro does not monitor schools’ adherence to the process, so English learners who meet the criteria for fluency may not be redesignated. Moreover, 18 of the 20 students we reviewed at San Diego met the school district criteria for fluency but were not redesignated. At each of the San Diego schools we visited there appeared to be confusion about the redesignation process, as some schools did not adhere to the district’s redesignation policy. When we asked San Diego’s program manager of its biliteracy and English learner support department why such confusion existed, she explained that the school district provides school staff with training on its redesignation procedures, but much depends on the individuals handling these procedures at the schools. TABLE 2 Student Designation and Redesignation Exceptions in the Sample School Districts Students Students Meeting Redesignated School District Students as Fluent but Redesignation Incorrectly Listed as English Criteria but Designated as Learners in School Maintained as School District English Learners District Database English Learners* Anaheim Union High 1 0 6 Long Beach Unified 2 2 12 Los Angeles Unified 1 7 12 Pajaro Valley Unified 0 6 14 Sacramento City Unified 1 0 17 San Diego City Unified 0 0 18 San Francisco Unified 0 0 19 Stockton Unified 1 6 13 Totals 6 21 111 Percentage of total reviewed at all school districts 3% 12% 62% Note: Our testing focused on English learners who were candidates for redesignation in fiscal year 2003–04, but who had not been redesignated as fluent that year. There were about 42,000 such students at the eight districts we reviewed. * Our determination was based on a review of test results, academic records, and other documents in student cumulative files. For these exceptions, the files did not contain documentation explaining why tested students who met district criteria were not redesignated. 2266 California State Auditor Report 2004-120 California State Auditor Report 2004-120 2277 We also found that an additional 21 of the students we reviewed had been redesignated as fluent, according to documentation at their schools, but continued to be reported as English learners in the districts’ student databases. For example, seven of the 40 students we tested in Los Angeles had documentation in their files indicating that they had been redesignated, including letters Sometimes schools to the students’ guardians notifying them of the redesignation. redesignated students However, the school district’s student database had not been as fluent but failed to updated to reflect the change in designation, and the school change their status district continued to report these students to the department in school district as English learners. According to the coordinator of curriculum databases, leading to and compliance in Los Angeles’s language acquisition branch, an overcount of English schools are responsible for monitoring English learner progress, learners reported to the maintaining and updating their student information database, department. and identifying students eligible for redesignation. As such, the district lacks controls to ensure that schools consistently comply with the redesignation process and update the student database. We noted similar instances in three other school districts. When these databases overstate the number of English learners, school districts receive more funding than they are entitled to receive. State regulations require school districts to maintain in students’ records documentation of input from teachers, other certified staff, and parents regarding redesignation, so we expected to see teacher comments or evidence of parent consultation in the students’ records explaining why students who met school district criteria were still designated as English learners. However, we noted that almost none of the students we reviewed who had met school district criteria for fluency but had not been redesignated had such documentation in their records. Given this lack of documentation, as well as the many exceptions we noted in our testing, it appears that school districts lack adequate monitoring efforts to ensure that all schools comply with the redesignation process. Because most districts do not monitor schools’ efforts to complete the redesignation process, they do not have adequate assurance that they are adhering to applicable state and federal laws regarding redesignation. Specifically, six of the eight school districts we reviewed limited their monitoring to providing technical assistance, including training, to schools regarding English learner programs and services, while another had informal procedures to ensure completion of the redesignation process. Only Sacramento had a more formal monitoring process in place. However, given that we noted exceptions with 18 of the 20 Sacramento students we reviewed, it is likely that the school district is not implementing its process fully. Without adequate procedures to ensure that 2288 California State Auditor Report 2004-120 California State Auditor Report 2004-120 2299 schools comply with their redesignation processes, school districts cannot be assured that they are reporting the correct English learner enrollments to the department. Moreover, although the department’s compliance review process includes a review of school districts’ redesignation criteria and schools’ adherence to those criteria, the process is limited because the monitoring guidance it provides consultants only includes School districts’ monitoring a review of former English learners who have been redesignated of schools’ adherence as fluent. Specifically, the guidance instructs consultants to test to the redesignation fluent students’ records to ensure that they met school district process is limited, redesignation criteria and were redesignated appropriately. The and the department’s guidance does not instruct consultants to test current English compliance review did learners’ records to ensure that they are designated correctly. not, until May 2005, According to the manager of the department’s English learner include guidance that accountability unit, consultants generally check student files English learner records to determine whether English learners should have been should be tested to ensure redesignated. However, without including such a review in its appropriate designation. monitoring guidance, the department cannot ensure consistent practice among its consultants. For example, only one of the eight school district compliance review reports and checklists we reviewed indicated that such a review had occurred. The manager of the department’s English learner accountability unit agreed that including a review of current English learners’ records is a good idea. In fact, the manager stated that the department proposed, and in May 2005 the board approved, including such a review in its monitoring guidance. The manager explained further that school districts are under great pressure to redesignate students as soon as possible. However, given the results we found, it appears that school districts are not always redesignating English learners who achieve fluency. DIVERSE DESIGNATION AND REDESIGNATION CRITERIA AND INCONSISTENT IMPLEMENTATION OF THESE CRITERIA MAY CAUSE FUNDING VARIANCES AND HINDER COMPARISONS OF PERFORMANCE RESULTS School districts’ use of more stringent designation and redesignation criteria, and a failure to implement redesignation criteria, can positively affect their funding and the outcomes for one of the three annual measurable achievement objectives (annual objectives) the department has established in accordance with Title III of the federal No Child Left Behind Act of 2001. Taking in and retaining high-scoring English learners gives some school districts a funding advantage because funding formulas 2288 California State Auditor Report 2004-120 California State Auditor Report 2004-120 2299 are based on English learner counts. The inclusion and retention of more-advanced students also can be expected to make it easier for these districts to meet one of the annual objectives. As we discussed in the Introduction, Title III—Limited English Proficient and Immigrant Students (Title III) and English School districts with more Language Acquisition Program (ELAP) funding is linked directly stringent redesignation to English learner counts. Impact Aid funding also takes into criteria, and those which account the number of English learners. School districts that do not redesignate all opt for more stringent designation and redesignation criteria eligible students as fluent, increase their English learner counts and in turn increase their maintain higher English English learner funding. Furthermore, school districts that do learner counts and thus not fully implement their established redesignation criteria receive higher funding. and thus fail to redesignate all eligible students maintain higher English learner counts and receive higher funding than otherwise would be the case. However, we found varying designation and redesignation criteria, as well as numerous errors in the redesignation process, at all sampled school districts. Therefore, we cannot determine how much of an effect divergent criteria and a failure to implement these criteria have on English learner funding. In accordance with federal law, the board and the U.S. Department of Education approved three annual objectives to measure English learner progress recommended by the department. Beginning in fiscal year 2003–04, school districts receiving Title III funds are required to meet annual targets for these objectives. For example, in fiscal year 2003–04, 51 percent of English learners in each school district needed to meet the first annual objective; by fiscal year 2013–14, 64 percent must do so. The department holds school districts that do not meet the annual objectives accountable by announcing their failures publicly. It also may discontinue their Title III funding. Specifically, a school district that does not meet one or more of the annual objectives in any year must so inform the guardians of English learners in the district, and a school district that does not meet the annual objectives for two consecutive years must develop an improvement plan to ensure that it will meet the annual objectives. Moreover, if a school district does not meet the annual objectives for four consecutive years, the department will require it to modify its curriculum or will determine whether its Title III funding should be eliminated. Four years of accountability results will not be available until fiscal year 2007–08, so the department has not yet imposed such actions upon any school district. 3300 California State Auditor Report 2004-120 California State Auditor Report 2004-120 3311 The three annual objectives measure: 1. The percentage of students making annual progress in learning English. 2. The percentage of English learners who attained English proficiency on the CELDT.5 3. The percentage of students who score proficient or higher on assessment data, including the CST. The student population used to measure school district success in meeting annual objective 1 consists of almost all English learners. The student population for annual objective 2 consists of only a segment of English learners, and the student population for annual objective 3 consists of almost all English learners plus some fluent students. Because of the way the student populations have been defined for annual objectives 2 and 3, we do not believe that differences in redesignation criteria will significantly affect school districts’ results for these objectives. For a detailed description of the three annual objectives and their student populations, see the Appendix. School districts with relatively stringent initial designation and redesignation criteria may find it easier to meet objective 1’s target for progress in learning English because they tend to have higher percentages of students who have attained proficiency on the CELDT. According to objective 1, English learners attaining proficiency on the CELDT need only maintain their proficiency to meet the annual progress target, while those who do not attain proficiency must improve their proficiency level to meet the objective. As shown earlier in Table 1 on page 23, school districts with more stringent redesignation criteria generally have higher percentages of English learners attaining proficiency on the CELDT. For instance, Anaheim and Sacramento require English learners to score above the intermediate range on some or all CELDT skill areas, and Sacramento requires a mid-basic score on the CST-ELA and the CST math test, while Anaheim has additional requirements that other school districts do not use. At least 47 percent of English learners in these districts attained proficiency on the CELDT in fiscal year 2003–04—the highest scores in our sample. Relatively stringent initial designation criteria also may help increase this percentage, because they cause some proficient students to be designated as English learners who otherwise would be considered fluent. 5English proficiency on the CELDT is defined as an overall score of at least early advanced with no score below intermediate in the three skill areas. 3300 California State Auditor Report 2004-120 California State Auditor Report 2004-120 3311 As shown in Table 3, school districts with higher percentages of English learners attaining proficiency on the CELDT generally have higher percentages of English learners who meet annual objective 1. Furthermore, based on data provided by the administrator of the department’s language policy and leadership office, in fiscal year 2003–04, 77 percent of English learners who previously attained proficiency on the CELDT were able to maintain their proficiency level, while only 57 percent of English learners who had not attained proficiency on the CELDT were able to improve their overall proficiency level. Thus, it appears to be more difficult to gain than to maintain a proficiency level. Consequently, performance results for objective 1 probably are skewed by the varying redesignation policies, and it is questionable whether these performance results are really comparable across school districts. TABLE 3 Proficient English Learners and Those Meeting Objective 1 in the Sample School Districts Fiscal Year 2003–04 Percentage of English Learners Meeting Annual Percentage of English Measureable Achievement Learners Attaining Objective 1 in Proficiency on the CELDT Fiscal Year 2003–04 School District in Fiscal Year 2003–04 (Goal is at least 51 percent) Anaheim Union High 54% 69% Sacramento City Unified 47 62 San Francisco Unified 43 65 Los Angeles Unified 40 62 Long Beach Unified 36 55 San Diego City Unified 35 59 Pajaro Valley Unified 27 53 Stockton Unified 27 52 Source: Department of Education’s CELDT data and school districts’ Title III Accountability reports. 3322 California State Auditor Report 2004-120 California State Auditor Report 2004-120 3333 MINIMAL MONITORING OF EXPENDITURES ALLOWS SCHOOL DISTRICTS TO USE SOME FUNDS FOR UNALLOWABLE COSTS The majority of supplemental English learner funds at the eight school districts we sampled were spent on salaries and benefits for teachers and staff. As shown in Figure 4, about 75 percent of total English learner funds we reviewed were for salaries and benefits, with about 47 percent spent on certificated salaries. Certificated salaries relate to positions that require a credential or permit from the Commission on Teacher Credentialing, such as those for teachers and counselors. Figure 4 also shows that these school districts spent more on certificated salaries than on any other category for the two state programs, while they spent about 47 percent of Title III funds on books and supplies. FIGURE 4 Sample Districts’ Program Expenditures Fiscal Year 2003–04 ����� ���������������� ���������������� ������������������ �������������������� ���� �� �� �� �� � ����� ����������������� ��������������� ���������������� ������������������ ������������������ �������������������� ����������������� Source: Districts’ accounting records. 3322 California State Auditor Report 2004-120 California State Auditor Report 2004-120 3333 Although school districts must use English learner program funds for supplementary services and activities, including student instruction in core academic subjects and professional training of teachers and staff, the department performs limited monitoring to ensure that school districts spend these The department funds in accordance with applicable state and federal laws. performs limited Moreover, although the authorizing laws for the Title III and monitoring of school Impact Aid programs allow school districts to expend funds districts’ expenditures on supplemental activities that benefit English learners in all of supplemental English grades, school districts must use state ELAP funds to benefit learner program funds. English learners only in grades four through eight. Despite such restrictions, the department does not monitor school districts’ management and expenditure of ELAP funds to ensure compliance with the purposes of the program, and it performs minimal monitoring of their use of Title III and Impact Aid funds, thus increasing the risk that these funds may be used for unintended purposes. In fact, when we tested English learner program expenditures at the eight school districts we sampled, we found cases in which five school districts spent funds for unallowed purposes. We also questioned many other transactions, either because they did not appear to be a prudent use of public funds or because it was unclear whether the funds were spent on English learner services. Specifically, as shown in Table 4, we found that funds for eight of the 180 tested expenditure transactions were spent for unallowable purposes, while an additional 43 expenditures were questionable, with most lacking sufficient supporting documentation to demonstrate that the expenditures were related to English learner programs and services. For example, Los Angeles used Title III funds to make two separate purchases, totaling nearly $3.8 million, of mathematics materials for students in general instructional programs—an unallowed use of these funds. According to the school district’s associate general counsel, the school district believes its former acting director of mathematics and the publisher of the materials are at fault with regard to the purchase of the mathematics materials and the district has initiated a lawsuit to recover the funds. The budget director for Los Angeles said that the district has implemented controls to ensure that this is not repeated in the future, and is in the process of reimbursing Title III for the unallowed purchases by transferring the expenditures to general fund, unrestricted programs. In addition, Los Angeles used Title III funds to pay for several questionable items. For example, two high school staff members were paid almost $2,800 in overtime 3344 California State Auditor Report 2004-120 California State Auditor Report 2004-120 3355 for activities such as processing payroll and working on school budgets, which are questionable because they were not related specifically to English learner programs or services. TABLE 4 Expenditure Exceptions Questionable Expenditures Purpose of Not a Prudent Expenditure Unallowable School District Use of Funds Unclear Expenditures Anaheim Union High 0 0 0 Long Beach Unified 2 7 1 Los Angeles Unified 0 7 2 Pajaro Valley Unified 2 8 0 Sacramento City Unified 0 3 1 San Diego City Unified 0 10 3 San Francisco Unified 1 1 1 Stockton Unified 0 2 0 Totals 5 38 8 Percentage of total reviewed at all school districts 3% 21% 4% We also question several instances in which school districts purchased refreshments, meals, and rental equipment that do not appear to be reasonable and prudent uses of English learner program funds. For example, San Francisco spent $209 in Title III funds to cater lunch for 20 attendees of a one-day bilingual education meeting at the school district. According to San Francisco’s executive director of multilingual programs, the expenditure is justified because the attendees worked through the lunch hour and the meeting focused completely on addressing the educational needs of English learners. In addition, Pajaro used $1,100 in Impact Aid funds to purchase dinners for an English learner advisory committee meeting and spent an additional $400 to rent tables, chairs, and table covers for the same meeting. According to Pajaro’s director of federal and state programs, the school district is required to hold these meetings and has found that dinners have been an important incentive, encouraging parents to attend. Although these meetings were related to English learners, we question whether 3344 California State Auditor Report 2004-120 California State Auditor Report 2004-120 3355 using public funds to provide private individuals with meals and refreshments and to rent dining furniture truly furthers a public purpose. Moreover, we noted that school districts often did not have documentation demonstrating the purpose of various expenditures. Because state regulations require recipients of English learner program funds to maintain auditable records of expenditures to document compliance with federal and School districts’ files often state regulations, we expected expenditure files to contain lacked documentation documentation demonstrating how purchases relate to describing the purpose English learner services. However, we noted 38 transactions of expenditures or the totaling almost $189,000 that lacked such documentation. For student population they example, one elementary school within the San Diego school would benefit. district used Impact Aid funds to purchase cameras totaling approximately $61,000, but the expenditure files contained no contemporaneous documentation describing the purpose of the transaction or the student population it would benefit. When we requested supporting documentation, the school principal wrote a letter stating that this purchase was for English learners. However, the content of this letter did not provide adequate evidence to link the expenditure to English learners. In another instance, Long Beach used Impact Aid funds to purchase boom boxes at a cost of nearly $2,500, but the school district’s files did not include documentation demonstrating how the boom boxes would benefit English learners. Therefore, we were unable to determine whether these purchases complied with applicable state laws. When school districts fail to document the link between English learners and program expenditures, they cannot demonstrate that their spending benefits English learners and is appropriate and in accord with state and federal laws. School districts are required to provide certifications to the department in order to receive English learner program funds, and the department further requires school districts to give their entire ELAP award only to individual schools the department has authorized for specific funding amounts. Nevertheless, Stockton and Los Angeles spent ELAP funds otherwise. Specifically, Stockton allocated about $3,500 in ELAP funds to eight schools and a school district program not included in its award. Stockton also allocated about $53,500 in ELAP funds to its central office to provide support for primary language testing. According to Stockton’s assistant director of curriculum and professional development, the schools with the largest English learner populations are those needing the most testing. Although this activity is an acceptable use of ELAP funds, 3366 California State Auditor Report 2004-120 California State Auditor Report 2004-120 3377 Stockton did not track these expenditures by school site, and thus we could not tell if the testing was for the schools the department authorized to receive ELAP funds. Further, although ELAP must be used for English learners in grades four through eight, in fiscal year 2003–04 Los Angeles used roughly $11 million of its ELAP funds in combination with a small portion of unrestricted general fund monies to operate extended learning programs, such as after-school and summer school programs, targeting underachieving students in kindergarten through eighth grade. In fiscal year 2004–05, Los Angeles again used ELAP funds for the extended learning program, which it expanded to target students in all grades. According to the budget director for Los Angeles, the district used a single program code, which included ELAP resources, to facilitate instructional activities for English learners and at-risk students at its schools. The limited guidance and monitoring the department provides school districts on their use of English learner program funds appear to be contributing factors to the instances of poor documentation and questionable expenditures that we observed. The department lacks Specifically, the department lacks documentation standards to documentation standards guide school districts in substantiating the purposes for which to guide school districts in supplemental English learner funds can be expended. Although substantiating the purposes the department requires school districts to certify that they for which supplemental will maintain auditable records, it does not define the level English learner funds can of documentation necessary for such records. Without such be expended. guidance, the department cannot ensure that school districts have adequate documentation practices and standards to demonstrate that they spend funds for intended program purposes. In addition, the department performs only minimal monitoring of school districts’ expenditures of English learner program funds. The department’s compliance review process focuses its monitoring efforts almost exclusively on programmatic issues. It currently includes only a high-level review of how Impact Aid funds are used, for which the department proposed, and in May 2005, the board approved expanding in fiscal year 2005–06 to include Title III. The department does not receive funding to administer ELAP, although the law does require an independent evaluation of the program’s effectiveness. According to the manager of the department’s English learner accountability unit, compliance reviews verify that funds are used for English learners and that they supplement school 3366 California State Auditor Report 2004-120 California State Auditor Report 2004-120 3377 districts’ general fund obligations. She also said, however, that the consultants who conduct the reviews are not auditors and are not required to perform a detailed review of English learner program expenditures. The manager explained that these consultants do sometimes request supporting documentation for certain Impact Aid expenditures when it is unclear how the funds were used. In examining compliance review reports and checklists for our eight sample school districts, we noted only one report that indicated a detailed review of Impact Aid expenditures. The manager stated that the consultants are required to review several compliance items during their one-day site visit, so the time they can devote to reviewing expenditure detail is limited. According to the instrument used by the department’s consultants to conduct compliance Compliance review reviews, the consultants typically focus on making sure that reports and checklists for school districts meet other requirements, including observing the eight school districts classroom lessons, ensuring that only qualified teachers provide we sampled indicate a instruction, determining whether school districts provide detailed review of Impact adequate teacher training, and examining school district plans Aid expenditures occurred to be sure they address curricula for English learners. Although in only one case. all these procedures seem to be worthwhile, they do not adequately ensure that English learner expenditures are made only for allowable purposes. The State has some assurance that school districts receiving certain federal awards are complying with applicable federal requirements because they are subject to independent audits under the federal Single Audit Act of 1984. However, these audits do not always include a review of English learner programs. Specifically, the State Controller’s Office issues guidance to auditors, including a number of state compliance requirements that do not directly relate to Impact Aid or ELAP. In addition, although local audits may include a review of Title III, auditors need to review this program only if the school district receives large Title III funding amounts relative to other federal grants. In reviewing the eight sample school districts’ fiscal year 2003–04 independent auditor reports, we observed that only three indicated that Title III had been reviewed. Given that these audits do not always include a review of English learner programs and that only some school districts receive an audit of Title III, these additional mechanisms for monitoring state and federal programs give limited assurance that all school districts are expending these funds as intended. 3388 California State Auditor Report 2004-120 California State Auditor Report 2004-120 3399 THE DEPARTMENT MEASURES ENGLISH LEARNER PROGRESS IN LANGUAGE PROFICIENCY AND ACADEMICS, BUT ITS EVALUATION OF THE CONTRIBUTION OF SPECIFIC ENGLISH LEARNER PROGRAMS IS WEAK In accordance with federal law, the department has defined annual objectives to measure school districts’ success in increasing the percentage of English learners who develop and attain English proficiency. However, school districts inconsistently define their English learner populations, so it is difficult to compare one district’s success to another’s in meeting the targets for one of the annual objectives. Moreover, state law does not require program-specific evaluations of Impact Aid, and a recent independent evaluation of school districts’ implementation of ELAP has not provided conclusive evidence or reliable data on ELAP’s effectiveness. Without dependable program-specific evaluations, the State cannot isolate and measure the effectiveness of particular English learner programs. The department primarily uses its three annual objectives to measure the effectiveness of English learner instruction throughout the State. As we mentioned previously, the first two annual objectives measure progress in gaining English proficiency and are based on CELDT data, while the third measures academic success in English language arts and mathematics based on assessment data, including the CST. The Appendix provides additional detail on the annual objectives. The department first reported school districts’ success in meeting the annual objectives in fiscal year 2003–04, the first year in which two full years of CELDT data were available for comparison. According to its February 2005 data, the department indicates that in fiscal year 2003–04, 82 percent of the school districts receiving The department’s annual Title III funding met the target for the first annual objective and objectives measure 68 percent met the targets for all three annual objectives. students’ success on standardized tests, but As we discussed previously, school district achievement in they do not measure meeting the first objective is affected by the stringency of the effectiveness of a the district’s redesignation criteria. School districts requiring particular program. English learners to attain advanced levels on the CELDT have a competitive edge in meeting this annual objective because they retain high-scoring English learners in the test population. Moreover, because some school districts’ redesignation criteria contain additional requirements, such as English learners must achieve a mid-basic score on the CST, must earn a particular grade point average, or must pass a unique school district assessment, some students who score well on the CELDT may trip on one of 3388 California State Auditor Report 2004-120 California State Auditor Report 2004-120 3399 these additional hurdles and remain English learners. As a result, the comparability of performance between school districts is reduced significantly. Although the annual objectives were created to measure the effectiveness of school districts’ language instruction programs, they instead provide a high-level perspective on the achievement of English learners on standardized assessments. They do not measure the contribution of individual English It appears that the State learner programs. In fact, the State’s efforts to evaluate has never conducted individual program effectiveness are weak. For example, an evaluation of the although Impact Aid is the largest source of supplemental effectiveness of Impact funding for English learners and was established more than Aid in improving the 25 years ago, it appears that the State has never conducted an academic performance of evaluation of the program’s effectiveness in improving the English learners. academic performance of English learners. Although state law does not require an evaluation of the specific effectiveness of Impact Aid, according to the manager of the department’s English learner accountability unit, the department does monitor the effectiveness of English learner programs through its compliance review process, which includes a review of school districts’ use of Impact Aid funds, and through Title III annual objectives. However, the department’s compliance review does not measure specific program effectiveness, and as discussed previously, its review of Impact Aid expenditures is limited. Further, the annual objectives do not evaluate the effectiveness of particular English learner programs. State law required the department to hire independent evaluators to conduct a five-year study on the impact of Proposition 227 and to evaluate ELAP. However, the evaluators concluded that it is difficult to provide definitive answers regarding the degree of ELAP’s success. Although the evaluators could not provide decisive conclusions, they did provide meaningful insight and several recommendations regarding ELAP, based on school districts’ responses to a survey. We discuss certain of the evaluators’ findings and four of their recommendations, and the department’s perspective on those recommendations, in more detail in the remainder of this section. In July 2001, the contracted evaluators released the first of five annual reports titled Effects of the Implementation of Proposition 227 on the Education of English Learners, Kindergarten through Grade 12. These reports focused primarily on the implementation of Proposition 227 and included a minimal evaluation of ELAP. The evaluators reported that school 4400 California State Auditor Report 2004-120 California State Auditor Report 2004-120 4411 districts generally agree that ELAP helps fund programs and purchase materials that otherwise would be unaffordable, but they often were confused about the program’s purpose because of the minimal guidance provided by the department regarding how to use ELAP. Moreover, because school districts view ELAP as a funding source and not as a specific program, the evaluators reported that school districts believe it would be difficult to evaluate its effectiveness. According to the report, a number Evaluators concluded it of school districts reported combining ELAP with other grant is difficult to determine monies or funding sources to meet specific needs related to the effectiveness of ELAP, English learners, which, the report stated, adds to the challenge in part, because school of monitoring and assessing students receiving resources districts view ELAP as a through this program. The evaluators also reported that school funding source and not districts frequently said that tying ELAP funds to the number of as a specific program. English learners they have enrolled served as a disincentive to redesignate students to fluent status. In June 2002, in their second report, the evaluators reiterated their prior finding that ELAP is a difficult program to evaluate and recommended that the State and school districts review the incentives associated with the funding formulas for English learner programs. The evaluators concluded that school districts have no incentive to monitor the progress and success of English learner students adequately because funding for supplemental English learner programs is based on their English learner enrollments. Student achievement does not positively affect the receipt of these funds. The evaluators also reported that local school districts lose funding when students are redesignated to fluent status unless other English learners replace them. The evaluators recommended that the department consider funding these programs using some form of improvement-based model or use cumulative counts of English learners and redesignated students who attain and maintain grade-level performance in the school districts. When we asked for the department’s perspective regarding this recommendation, the administrator of its language policy and leadership office stated that there is an inherent contradiction between funding school districts based on need and providing incentives through additional funding. She further stated that if funding for these programs is used as an incentive, school districts with large enrollments of English learners may be denied full funding because of low academic performance results, while school districts with small English learner enrollments might receive more funding than they need. The administrator explained further that one possible way to provide incentives 4400 California State Auditor Report 2004-120 California State Auditor Report 2004-120 4411 would be to fund these programs based on school districts’ enrollment of both English learners and students who have been redesignated as fluent. She stated that the department will study this matter and, if appropriate, will consider possible amendments to current laws to address the issue after the final evaluation is submitted in October 2005. The law establishing ELAP requires school districts to evaluate their effectiveness in assisting English learners and to submit reports to the department by October 2003. However, the evaluators’ second report stated that school districts were finding ELAP difficult to evaluate in part because most districts do not specifically monitor or assess students participating in ELAP- funded programs. Moreover, many school districts combine ELAP funds with other funds, adding to the challenge of monitoring and assessing students receiving resources through this program. As a result, the evaluators recommended that the evaluation requirements for ELAP be bolstered and made a state, rather than school district, responsibility. When we asked the department for its perspective on this recommendation, the administrator of its language policy and leadership office explained that the department’s three Title III annual objectives currently hold districts accountable for ensuring that English learners are making progress in English, reading, and math proficiency. However, these annual objectives do not measure a particular program’s effectiveness, including the effectiveness of ELAP. In addition, because 70 percent of the responding school districts reported that restricting the use of ELAP funds to English learners in grades four through eight was a significant Although the department constraint, the evaluators recommended that the State consider indicates it supports giving school districts flexibility in the use of these funds while certain recommendations, holding them accountable for improved services and results. it has not sought In response, the administrator explained that the department amendments to law or supported this recommendation, but its implementation additional funding to would require amending state law and providing additional implement them. funds to hold school districts accountable. To the best of the administrator’s knowledge, the department has yet to seek such an amendment or additional funding. The evaluators’ third report, in October 2003, repeated many of the themes identified in the first two annual studies and included a discussion of school districts’ varying redesignation 4422 California State Auditor Report 2004-120 California State Auditor Report 2004-120 4433 criteria. The evaluators noted that the flexibility given to school districts in establishing their own criteria as permitted under state guidelines and the different procedures for redesignation appeared to cause redesignation rates and even the meaning of redesignation to vary widely across school districts. As we already noted, this variability makes comparisons of school districts’ performance difficult. The evaluators released their fourth report, English Language Acquisition Program Evaluation Report, in July 2004. This report, which focused specifically on ELAP, discussed, among other The evaluators concluded things, issues regarding the state-mandated ELAP evaluation that that no effect from ELAP school districts must conduct. Specifically, the evaluators reported could be inferred because that only seven, or approximately 1 percent, of the 518 school school districts’ analyses districts that completed their survey had conducted a formal did not include data from evaluation that explicitly provided data related to ELAP. These a comparison group. seven school districts based their evaluations of ELAP primarily on students’ performance on standardized tests. However, the evaluators concluded that, while these analyses attempt to use data to assess the progress of English learners as a result of ELAP, none included data from a comparison group, and thus no effect from ELAP could be inferred. The evaluators stated that these relatively meager results from the school districts’ evaluative efforts are understandable, given that isolating the impact of ELAP from the many other program initiatives and other outside factors is quite challenging, and likely daunting for individual school districts. For example, in its survey response, one school district said it was impossible to assess how much of an impact ELAP has had on English learners’ progress as measured using various standardized tests, including the CELDT, because so many other programs have been in play, including Impact Aid and Title III, that the results cannot be ascribed to any one program, especially a relatively small program such as ELAP. The evaluators stated that such results raise important questions about the State’s delegation of evaluative responsibilities for major funding initiatives to individual school districts. As a result of this survey, the evaluators recommended that the department consider providing an incentive to selected large school districts to collaborate on efforts to evaluate whether ELAP appears to have an impact on student performance and whether some uses of ELAP are more cost-effective than others. When we asked the department for its perspective on this recommendation, the administrator of the language policy and 4422 California State Auditor Report 2004-120 California State Auditor Report 2004-120 4433 leadership office explained that state funding is necessary to enable such case study evaluations, and that no administrative funds were allocated for this program. Because it did not seek additional funding for administration, the department did not attempt to implement this recommendation. Further, in reporting on school districts’ survey responses, the evaluators noted that one of the most common constraints school districts reported was their uncertainty about the level of ELAP funding. A significant percentage of school districts stated The evaluators reported that the department’s delay in allocating funds made it difficult that many school districts for them to hire staff. Several school districts reported that these indicated constraints delays also hindered their ability to plan ahead or implement in administering the programs. For example, the department did not mail the award program because the letters giving school districts the authority to spend ELAP funds department distributed for fiscal years 2003–04 and 2004–05 until January 2004 and ELAP funds late in the December 2004, respectively. According to an analyst in the school year. department’s language policy and leadership office, ELAP allocations are late primarily because processing applications is time-consuming and the department does not receive funding to administer the program. Additionally, she said that for fiscal year 2004–05, the governor’s proposed budget delayed allocations by two to three months. The analyst also stated that many of the school districts’ applications are submitted without the required signature certification by a member of the school board, and that by law the department cannot allocate ELAP funds until such certifications are received. Some survey respondents also noted a lack of guidance from the department as to how ELAP funds may be used, prompting one school district to suggest having a more complete and specific program guideline package that would enable the school district to align its evaluation to the goals of ELAP. Although the evaluators made no recommendations to the department regarding such issues, when we asked the analyst in the language policy and leadership office why there is limited guidance to school districts on program implementation, she explained that the authorizing law for ELAP does not provide specific guidance on program implementation and the department does not receive funding to administer the program. Finally, the evaluators performed an analysis of the impact of ELAP funds on student performance on standardized assessments, pointing out the limitations of such analyses and cautioning against inferring that ELAP funding is the cause of increased student achievement. The evaluators concluded 4444 California State Auditor Report 2004-120 California State Auditor Report 2004-120 4455 that, although they saw a statistically positive relationship between ELAP and selected student outcome measures, they could not say with confidence that ELAP had been the cause of these gains. The evaluators also reported that it is difficult to determine how ELAP affects redesignation, given the varying redesignation criteria used throughout the State and other factors affecting these rates. The evaluators also found it difficult to analyze possible relationships between ELAP and English learner academic performance without being able to match test scores to specific students. The evaluators indicated that this was not possible because statewide data do not contain individual student identifiers. However, according to the administrator of the department’s language policy and leadership office, beginning in fiscal year 2005–06, the department will require that each student in statewide testing programs be assigned an individual student identifier as part of its California Longitudinal Pupil Achievement System. She explained that the data from this system will provide better ways to monitor and report on the academic progress of English learners. FUNDING FORMULAS ARE GENERALLY EQUITABLE, BUT A POVERTY STATISTIC FOR IMPACT AID NEEDS UPDATING Although school district criteria for identifying and redesignating English learners can skew English learner counts and thus funding, the department’s funding formulas are based on established criteria and are designed to allocate funds equitably among school districts. The formula for Impact Aid, however, is complicated, and critics have noted that the way it allocates funds to school districts, based on various characteristics of their student populations, appears to be arbitrary and unpredictable. In addition, the formula uses a poverty statistic—the number of students in families receiving assistance under the California Work Opportunity and Responsibility to Kids (CalWORKs) program—that may not track the student population in poverty as well as it once did. The department also is currently having difficulty obtaining this statistic. In August 2004, the Legislature passed a bill that would have redirected $1 million to establish a task force to develop options for restructuring the Impact Aid formula; the governor vetoed the bill. The department allocates the State’s Title III award according to department-developed criteria that equitably spread funds across the program’s targeted populations—English learners and immigrant students. The funding formula for Title III is relatively simple. English learner funds are available to all 4444 California State Auditor Report 2004-120 California State Auditor Report 2004-120 4455 school districts on a per pupil basis. In accordance with Title III, immigrant student funds are allocated to school districts that have had a significant increase in their percentage or number of immigrant children and youth. The department has set a prior two-year average of 5 percent as the growth rate school districts must meet to receive immigrant student funds. The department determined the per pupil amounts for fiscal year 2004–05 based on the annual statewide census of English learners and the Student National Origin Report, which identifies immigrant students. After subtracting 5 percent of the grant for state administrative costs, the department provided school districts with about $86 per English learner and immigrant student in eligible school districts. To determine the exact number of students to include in the formula, the department identified the number of English learners and immigrant students for each school district that applied for funding. It then derived a per-pupil funding amount by dividing the State’s award by the total number of English learners and immigrant students The department allocates in these school districts. In fiscal year 2004–05, school districts Title III and ELAP primarily funded by Title III accounted for 100 percent of English learners based on school districts’ and 79 percent of immigrant students statewide. The remaining English learner enrollments. immigrant students were enrolled in school districts that did not apply for Title III funding or did not meet the 5 percent cutoff for average growth in their immigrant populations. School district allocations of ELAP funds are based on a formula established by the Legislature that also appears to be equitable. Like Title III funds, ELAP funds are allocated primarily based on the number of targeted students, in this case English learners in grades four through eight, in school districts that apply for the grant. State law, however, sets per-pupil funding at $100. In years when the appropriation is insufficient to cover all English learners, it requires that students at schools with the highest concentrations of English learners be given priority. The law thus gives a preference to schools with higher proportions of English learners, a presumably more challenging environment. In fiscal year 2004–05, the ELAP appropriation of $55 million was not high enough to cover the more than 556,000 English learners in school districts that applied for the grant. As such, about 6,500 English learners in schools with English learner concentrations below 2.4 percent did not receive funding. The formula for Impact Aid, first detailed in state law in 1977, is much more complicated than the formula for Title III or ELAP. Although the Legislature revisited the law in 1989 to add an equalization component that resulted in a greater focus on English learners, the formula gives primary weight to 4466 California State Auditor Report 2004-120 California State Auditor Report 2004-120 4477 poverty statistics in allocating funds. The department computes Impact Aid allocations through two schedules, pursuant to detailed procedures in the Education Code. The first schedule assesses a school district’s need for Impact Aid by comparing its concentration of ethnic, poor, and transient students to the State’s overall concentration of these students. The resulting factor, which favors school districts with high concentrations of these students, is multiplied by the average of a school district’s number of students living in poverty per the U.S. Census plus its number of students in families enrolled in CalWORKs (CalWORKs students), and is then multiplied by a set amount per student. In fiscal year 2004–05, the average amount of Impact Aid funding per student from the first schedule was $518. The second schedule distributes an equalization adjustment to school districts to ensure a minimum amount of funding for each CalWORKs student and for each English learner student who are referred to collectively as economically disadvantaged (disadvantaged). Allocations under this schedule are based on the number of disadvantaged students enrolled in a school district, multiplied by the prior year’s average rate of funding per disadvantaged student. In fiscal year 2004–05, the department used a rate of about $236 per student in the second schedule. We found that for fiscal year 2004–05, the department correctly implemented the Impact Aid formula established in law, a formula that appears to target the intended populations in need. Nevertheless, the formula has its critics. In February 2004, the Legislative Analyst’s Office (legislative analyst) observed that The Legislative Analyst’s the complexity of the Impact Aid formula results in district Office has observed allocations that are hard to understand based on underlying that the complexity of district demographics. It pointed to an example of two school the Impact Aid formula districts with almost identical numbers of English learners results in school district and CalWORKs students, but very different allocations. The allocations that are hard legislative analyst also noted that districts of similar size but to understand. differing proportions of English learners and CalWORKs students were receiving dissimilar allocations. It pointed to a school district that had a relatively high concentration of CalWORKs students but received more funds than another school district with a relatively high concentration of English learners. Further, the legislative analyst concluded that the Impact Aid formula creates unpredictable year-to-year results that complicate school district planning efforts. It noted that in fiscal year 2003–04, more than 300 school districts received increased funding even though their enrollments of disadvantaged students had declined, while 16 school districts received decreased funding despite increases in their disadvantaged student enrollments. 4466 California State Auditor Report 2004-120 California State Auditor Report 2004-120 4477 With regard to the Impact Aid formula’s weighting toward school districts with high numbers of students living in poverty, the legislative analyst noted a divergence between the population of CalWORKs students and the population of English learners in recent years. It pointed out that the number of English learners was about two and a half times the number of CalWORKs students in fiscal year 2001–02, but that school districts with large numbers of students living in poverty still receive far more than school districts with large numbers of English learners. According to statistics provided by the department, as well as U.S. Census data, there has been a steady decrease in the CalWORKs population, but this measure does not seem to be reflective of the overall population of students in poverty. As shown in Figure 5, the number of CalWORKs students has been consistently lower than the number of English learners, but the population of school-age children in poverty actually exceeded the English learner population until 1998, and their number saw an upswing in 2002, the latest year for which U.S Census data are available. FIGURE 5 School-Age Children in Various Categories 1995 Through 2002 (in Millions) ���� ������� ��������������� �������� ���� ���� ���� ���� ��� ��� ���� ���� ���� ���� ���� ���� ���� ���� Sources: The Department of Education’s Impact Aid annual summary and the U.S. Census Bureau’s Small Area Income and Poverty Estimates. �������������������������� 4488 California State Auditor Report 2004-120 California State Auditor Report 2004-120 4499 The department is also having difficulties in obtaining CalWORKs data. In December 2004, the Department of Social Services (Social Services) informed the department that because of heightened security concerns regarding confidential data, it no longer would supply or ask county welfare departments to supply child-specific data to the department. However, according to the administrator of the department’s school fiscal services division, Social Services will cooperate in providing all needed data. The Legislature recognized concerns regarding the Impact Aid formula, and in August 2004 it passed a bill redirecting $1 million in federal funds to establish a task force to conduct a yearlong investigation that would develop options for restructuring the formula. The task force was to consult with a diverse stakeholder group, determine the best alternatives to restructure the Impact Aid formula, and develop recommendations covering a number of topics, including the manner in which to measure the number of economically disadvantaged children. The governor subsequently vetoed the bill, saying he believed the work could be accomplished with existing resources. He directed the Department of Finance and the Secretary of Education to work in collaboration with the legislative analyst and the department to develop options for restructuring the Impact Aid formula. According to the administrator of the department’s school fiscal services division, the department will work with the Department of Finance, the legislative analyst, and the Legislature to develop a long-term solution for allocating Impact Aid funds. This could include determining an appropriate replacement for CalWORKs data, as well as revamping the entire Impact Aid formula. RECOMMENDATIONS • The department, in consultation with stakeholders, should establish required initial designation and redesignation criteria related to statewide tests that would provide greater consistency in the English learner population across the State. The department should pursue legislative action, as necessary, to achieve this goal. In addition, the department should require school districts to document redesignation decisions, including decisions against redesignating students who are candidates for fluent status. • School districts should ensure their redesignation criteria include each of the four criteria required by state law for redesignating English learners to fluent status. They also 4488 California State Auditor Report 2004-120 California State Auditor Report 2004-120 4499 should monitor their designation and redesignation processes more closely to ensure that schools actually complete the process and that school district databases accurately reflect all redesignations. • The department should consider changing annual objective 1 to offer less incentive for school districts to maintain students as English learners. • The department should perform the necessary steps to ensure the school districts we reviewed have taken appropriate action to resolve their unallowable expenditures of supplemental English learner program funds. • The department should revise the documentation policy it provides to school districts to better ensure that expenditures are directed at activities that serve the English learner programs’ target populations. • School districts should implement documentation policies to ensure that expenditure files clearly demonstrate that supplemental English learner program funds are directed at activities that serve the law’s target populations. • To ensure the State benefits from recommendations in reports on the effects of the implementation of Proposition 227 and ELAP, the department should review the evaluators’ recommendations, subsequent to submission of the final report in October 2005, and take necessary actions to implement those recommendations it identifies as having merit. • The department should continue to work with the Department of Finance, the Legislative Analyst’s Office, and the Legislature to revise the Impact Aid funding formula to include statistics that better measure the number of students in poverty. 5500 California State Auditor Report 2004-120 California State Auditor Report 2004-120 5511 We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, ELAINE M. HOWLE State Auditor Date: June 16, 2005 Staff: Doug Cordiner, CGFM, Audit Principal Jim Sandberg-Larsen, CPA Julianna N. Field Laura G. Kearney Benjamin L. Ward 5500 California State Auditor Report 2004-120 California State Auditor Report 2004-120 5511 Blank page inserted for reproduction purposes only. 5522 California State Auditor Report 2004-120 California State Auditor Report 2004-120 5533 APPENDIX Measurable Achievement Objectives Established for Title III In the Audit Results section of this report, we discuss the three annual measurable achievement objectives (annual objectives) for Title III of the No Child Left Behind Act of 2001. As shown in Table A on the following page, the annual objectives measure the percentage of English learners making progress on the California English Language Development Test (CELDT), the percentage of English learners attaining English proficiency on the CELDT, and the rate of English learners participating in and the percentage of English learners earning a proficient score on statewide assessments in English language arts and math. The criteria used to measure success on the first annual objective vary by the past proficiency level of the tested students, while the criteria used to measure the second and third annual objectives is the same for all tested students. The student populations used to measure the annual objectives include subgroups of the English learner population and vary with each annual objective. Annual objective 3 also includes a subgroup of English learners who have been redesignated as fluent. 5522 California State Auditor Report 2004-120 California State Auditor Report 2004-120 5533 TABLE A Summary of Title III Annual Measurable Achievement Objectives Fiscal Year 2003–04 Annual Measurable Annual Measurable Annual Measurable Achievement Objective 1 Achievement Objective 2 Achievement Objective 3 Objective At least 51 percent of English learners make At least 30 percent of English learners attain At least a 95 percent participation rate and the annual progress on the California English English proficiency on the CELDT. following proficiency percentages on statewide Language Development Test (CELDT). assessments for the English learner subgroup: Elementary School District–at least • 13.6 percent proficient in English language arts • 16 percent proficient in mathematics High School District–at least • 11.2 percent proficient in English language arts • 9.6 percent proficient in mathematics Unified School District–at least • 12 percent proficient in English language arts • 12.8 percent proficient in mathematics Criteria There are three ways for English learners to Attain an overall proficiency level score of Attain at least the proficient level on the California meet the annual growth target for CELDT early advanced or advanced, with each skill Standards Test, California Alternate Performance performance, depending upon what level area proficiency score at the intermediate Assessment, or California High School Exit Exam in they achieved on the previous CELDT. level or above. English language arts or mathematics. 1. Those at the beginning, early intermediate, or intermediate levels are expected to gain one proficiency level. 2. Those at the early advanced or advanced level overall who did not score at least intermediate in all skill areas—i.e., are not English proficient—are expected to become English proficient. Not all skill areas are tested for students below grade two. 3. Those at the English proficient level are expected to maintain that level. Student Population • English learners who took the annual • English learners at the beginning or early • English learners who are continuously enrolled in CELDT during the testing window and intermediate level in the prior year who the district from the prior year to the test date. have a valid prior score. have been in U.S. schools since spring 2000. • Students redesignated as fluent who have not • English learners from direct-funded • English learners at the beginning or early scored proficient or above on the California charter schools will be removed from intermediate level in the prior year who Standards Test in English language arts for three the sponsoring district or county office of entered U.S. schools after spring 2000 and years are included in the percent proficient education’s cohort. who met the English proficient level in 2003. calculation and the participation rate. The three • All English learners who were at the years must be after redesignation, but do not intermediate level the prior year. have to be consecutive. • English learners at the early advanced • English learners who are in their first year in a or advanced level overall who were not U.S. school are included in the participation rate English proficient the prior year. but not in the percent proficient calculation. Sources: Department of Education’s 2004 Accountability Progress Report Information Guide and the 2003–04 Title III Accountability Report Information Guide. 5544 California State Auditor Report 2004-120 California State Auditor Report 2004-120 5555 Agency’s comments provided as text only. California Department of Education 1430 N Street Sacramento, CA 95814 June 3, 2005 Elaine M. Howle, State Auditor* Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Mrs. Howle: Audit No. 2004-120 This letter and enclosed documents constitute the California Department of Education’s (CDE) response to the Bureau of State Audits’ (BSA) draft audit report entitled, “Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances to Make Comparisons of Performance Outcomes Difficult.” We appreciate the opportunity to comment on your draft report. Our response to each of your audit recommendations is enclosed. To provide clarity and perspective, the CDE is also commenting on the BSA draft audit report. In general, the report has many references to the CDE providing school districts the flexibility in establishing criteria used to identify students as English learners and to redesignate them as fluent in English. However, it is 1 current law that allows and requires this flexibility, not the CDE. In addition, the audit report states school district criteria are inconsistent from one district to another, but again, the inconsistency is consistent with the current law. If you have any questions regarding the CDE’s response or our clarification to the report, please contact Kim Sakata, Audit Response Coordinator, Audits and Investigations Division, at (916) 323-3560 or by email at ksakata@cde.ca.gov. Sincerely, (Signed by: Gavin Payne) GAVIN PAYNE Chief Deputy Superintendent of Public Instruction Enclosures * California State Auditor’s comments begin on page 63. 5544 California State Auditor Report 2004-120 California State Auditor Report 2004-120 5555 Enclosure 1 CALIFORNIA DEPARTMENT OF EDUCATION’S RESPONSE TO THE RECOMMENDATIONS IN THE BUREAU OF STATE AUDITS REPORT NUMBER 2004-120 Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult Recommendation 1: The CDE, in consultation with stakeholders, should establish required initial designation and redesignation criteria related to statewide tests that would provide greater consistency in the English learner population across the State. The CDE should pursue legislative action, as necessary, to achieve this goal. In addition, the CDE should require school districts to document redesignation decisions, including decisions to not redesignate students who are candidates for fluent status. CDE’s Response: 1 Current law does not grant the CDE the authority to establish specific criteria that all districts must follow. The CDE requires districts to include what the law requires, and encourages districts to follow the State Board of Education (SBE) guidelines. If the Legislature changes the law eliminating the existing flexibility given to the districts, the CDE will require the school districts to comply with the new legislative actions. The CDE will inform the school districts that they are required to document redesignation decisions, including decisions to not redesignate students who are candidates for fluent status. Recommendation 2: No response; this recommendation is directed to the school districts. Recommendation 3: The CDE should consider changing the annual measurable achievement objective (AMAO) 1 to offer less incentive for school districts to maintain students as English learners. CDE’s Response: The CDE does not believe that the AMAO 1 needs to be revised at this time. Although there is some advantage to districts with a higher percentage of students at the English proficient level on the California English Language Development Test (CELDT) in meeting AMAO 1, this does not help in meeting AMAO 2 and AMAO3. Additionally, a common scale is being developed for the 2007 annual CELDT; and at that time, the CDE plans to reexamine the growth metric to determine if the use of scale score growth rather than proficiency level gains should be recommended. 5566 California State Auditor Report 2004-120 California State Auditor Report 2004-120 5577 Elaine M. Howle, State Auditor Enclosure 1 June 3, 2005 Page 2 Recommendation 4: The CDE should perform the necessary steps to ensure the school districts we reviewed have taken appropriate action to resolve their unallowable expenditures of supplemental English learner program funds. CDE’s Response: After receiving information regarding the eight expenditures considered unallowable by the BSA, the CDE will require the districts to take the appropriate action. Recommendation 5: The CDE should revise the documentation policy it provides to school districts in order to better assure that expenditures are clearly directed at activities that serve the English learner programs’ target populations. CDE’s Response: The CDE will provide the school districts with information of the documentation needed to support expenditures charged to the English learner program. Recommendation 6: No response; this recommendation is directed to the school district. Recommendation 7: To ensure the State benefits from recommendations in reports on the effects of the implementation of Proposition 227 and English Language Acquisition Program (ELAP), the CDE should review the evaluator’s recommendations, subsequent to submission of the final report in October 2005, and take necessary actions to implement those recommendations it identifies as having merit. CDE’s Response: As stated in the report, the CDE will study the recommendations from the evaluation and, after the final evaluation is submitted in October 2005, will consider possible amendments to current laws to address the issue. 5566 California State Auditor Report 2004-120 California State Auditor Report 2004-120 5577 Elaine M. Howle, State Auditor Enclosure 1 June 3, 2005 Page 3 Recommendation 8: The CDE should continue to work with the Department of Finance, the Legislative Analyst, and the Legislature to revise the Economic Impact Aid funding formula to include statistics that better measure the number of students in poverty. CDE’s Response: The CDE agrees that an alternative funding method is necessary and has expressed interest in studying the issue. However, in November 2004, the Governor withdrew support for the study of a new funding formula. If resources are available in the final budget agreement for the study, progress will commence. 5588 California State Auditor Report 2004-120 California State Auditor Report 2004-120 5599 Enclosure 2 CALIFORNIA DEPARTMENT OF EDUCATION’S CLARIFICATION TO THE BUREAU OF STATE AUDITS REPORT NUMBER 2004-120 Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult Summary: Results in Brief 2 *Page 4: First paragraph regarding the Coordinated Compliance Review (CCR) guidance used by the consultants. Although the guidelines were revised in May 2005, the 2003-04 CCR monitoring instrument always contained a compliance item stating, in part, “Each English learner who meets the established redesignation criteria is reclassified as fluent English proficient.” Furthermore, the consultants have tested for this compliance item as evident of the noncompliant findings the consultants identified in this area. Page 5: The last sentence states, “With regard to ELAP, program evaluators hired by the department have been unable to reach decisive conclusions on the value of the program . . .” 3 However, this does not fully portray the independent ELAP report. Overall, ELAP results suggest a small but statistically significant increase in reading, math, and language policy and leadership office achievement scores in association with the program. Although the evaluators could not claim a causal relationship from these analyses, overall, the results from these analyses suggest a relationship between these two variables that is positive and statistically significant. Scope and Methodology Page 20: It should be noted that the auditors did not select their sample from the 453,000 English 4 learners, but from the relatively small population of those English learners who were possibly eligible for redesignation, but had not yet been redesignated. Moreover, the BSA did not review any files related to the other 28,000 students that had already been redesignated. Audit Results: Chapter Summary Page 23: First paragraph, “However, the leeway it provides school districts…and the insufficiency of department and school district monitoring…” is inaccurate; the CDE has a process for monitoring these issues. In fact, the CDE consultants identified redesignation and funding issues during their 5 monitoring reviews in fiscal year 2003-04. Of the 187 reviews conducted, the CDE found 68 districts noncompliant with redesignation (reclassification), and 20 districts noncompliant with the use of EIA-limited English proficient (LEP) funds. * Text refers to page numbers in earlier draft version of the report. 5588 California State Auditor Report 2004-120 California State Auditor Report 2004-120 5599 Elaine M. Howle, State Auditor Enclosure 2 June 3, 2005 Page 2 Page 23: Last sentence states, “Further, because the department provides little guidance and 6 monitoring to school districts on their use of English learner program funds…” is not a fair statement. The CDE conducts workshops at the Coordinated Compliance Review Regional Institutes throughout the state; presents presentations at various professional conferences and meetings; and provides technical assistance via telephone or email. Redesignation Criteria Are Even More Variable Among School Districts Page 30-31: Last paragraph and first paragraph. The way these paragraphs are written implies that it is inappropriate for Sacramento and San Francisco Unified School Districts to set their 7 CST-ELA scores at mid-basic, and for Anaheim to raise its CST-ELA score, when this is perfectly within the range allowed by the State Board of Education (SBE) guidelines. The report further indicates that if the lower/old score were used, then more students would have been able to meet the criteria. Again, this tends to mislead the reader that the CST-ELA scores are inappropriate at the mid-basic level. 7 Page 31: Middle paragraph. Again, having the report state that Sacramento, Pajaro, and Long Beach use the CST mathematics exam as part of the redesignation criteria, and other districts do not, implies that this is inappropriate. However, it should be noted that requiring math proficiency is allowable under current law, and up to the individual school districts. Page 31: Last paragraph. The report states that the CDE allows flexibility and grants even more 8 discretion in establishing criteria for teacher evaluation, when in fact it is the law that provides this flexibility and discretion to the school districts. Page 32: Middle paragraph. This paragraph implies that school districts are using more 9 stringent criteria to have a higher English learner enrollment, just to obtain more funding. However, districts could have more stringent criteria because they have an obligation to accurately identify students who need English learner services and ensure those students get the services to which they are entitled. Inadequate Monitoring of the Redesignation Process Causes Students who Have met School District Criteria for Fluency to Remain in the English Learner Population 0 Page 33: Although the CCR monitoring instrument did not previously contain specific guidance that consultants test current English learners’ records, the compliance item is the focus of the review, and states in part, “Each English learner who meets the established redesignation criteria is reclassified as fluent English proficient.” It is the responsibility of the consultants to ensure the schools meet this compliance item. Furthermore, the consultants have tested for this compliance item as evident of the noncompliant findings identified in this area. 6600 California State Auditor Report 2004-120 California State Auditor Report 2004-120 6611 Elaine M. Howle, State Auditor Enclosure 2 June 3, 2005 Page 3 Page 34: Last paragraph. The report identified three students as being incorrectly designated as English learner when the home language survey (HLS) indicated English as the primary language. q There are some instances where the HLS shows English as the primary language, but there are indications that a student is not proficient in English. In these cases, the district must test the student with the CELDT or some other method to identify whether the student is an English learner. Under federal law, the school districts must have procedures for identifying all English learners students in order to provide them with equal educational opportunity. Therefore, the initial designation is not solely based on the HLS. Page 35: Although the auditors identified some students in the various districts who might not have been redesignated, the CDE believes it is important to point out that nearly 28,000 students were redesignated in fiscal year 2003-04. The following table illustrates by district the number of students redesignated. District Number of ELs District Number of ELs Redesignated Redesignated 2003-04 2003-04 Los Angeles USD 13,531 Long Beach USD 5,373 San Diego USD 4,038 Anaheim HSD 931 San Francisco USD 1,662 Stockton USD 775 Sacramento City USD 903 Pajaro Valley USD 524 Page 37-38: The CCR monitoring instrument provides the consultants with compliance items that the consultants must check, and suggests examples on how to test for the compliance items. 2 Although the CCR monitoring instrument did not provide an example that the consultant review current English learners’ records to ensure correct designation, the consults perform tests as evident in the noncompliant findings the consultants identified in this area. In addition, the CDE provides training to the consultants and districts on this issue. Page 38: First paragraph, last sentence. The BSA’s sample was selected from a relatively small population of students who met redesignated criteria and were not redesignated (same comment w as for page 20). Some of these could have erroneously not been redesignated, but others might not have met all criteria to be redesignated and not contain the appropriate documentation supporting the non-redesignation as stated in your report on page 36. Page 39: Middle paragraph. The report focuses on more stringent designation and redesignation e criteria affecting the increase in English learner counts thereby increasing the English learner funds. However, the BSA did not review those students that had been redesignated and should not have been redesignated, and the affect that has on the number of English learners and the funding received. It is the CDE’s experience that a more frequent occurrence is that students are redesignated before they should be. 6600 California State Auditor Report 2004-120 California State Auditor Report 2004-120 6611 Elaine M. Howle, State Auditor Enclosure 2 June 3, 2005 Page 4 Minimal Monitoring of Expenditures Allows School Districts to use some Funds for Unallowable Costs Page 43: First paragraph. Although the report states that the CDE performs minimal monitoring r of the school districts’ use of Title III and Economic Impact Aid (EIA) funds, the CDE monitoring reviews found similar issues 20 times last year. It is an issue that is highly monitored by CDE staff. The Department Measures English Learner Progress in Language Proficiency and Academics, but its Evaluation of the Contribution of Specific English Learner Programs is Weak Page 49: In the first paragraph it state, “…independent evaluation of school districts’ implementation 3 of ELAP has not provided conclusive evidence or reliable data on ELAPs effectiveness.” As stated in our comment for page 5, the ELAP report does state that the overall results suggest a small but statistically significant increase in association with ELAP. Page 51: Second paragraph. “The evaluators reported…they [school districts] were often confused about the program’s purpose because of the minimal guidance provided by the department regarding how to use ELAP.” Although this is what the evaluators reported, the CDE’s website under FAQs for ELAP provides guidance to the school districts specifically on “What can the funds be used for?” and “What are the school and local education agencies basic responsibilities for accounting for these funds?” 6622 California State Auditor Report 2004-120 California State Auditor Report 2004-120 6633 COMMENTS California State Auditor’s Comments on the Response From the California Department of Education To provide clarity and perspective, we are commenting on the California Department of Education’s (department) response to our audit. The numbers below correspond to the numbers we have placed in its response. 1 The department has interpreted the law to provide school districts flexibility in establishing criteria used to identify students as English learners and to redesignate them as fluent in English. However, we believe that there are opportunities to provide more statewide consistency while still ensuring that each school district has the flexibility to meet its obligation to take affirmative steps to rectify language deficiencies so that its instructional program is available to English learners. More specifically, state law requires the department to establish procedures for redesignating English learners to fluent status, using at least the four criteria defined in law. We believe that the law anticipated some level of statewide consistency by requiring the department to establish these procedures. Furthermore, as we point out at page 21, the degree of flexibility that is currently provided allows for funding variances and lack of comparability in performance results between school districts. While two of the criteria established by law, teacher evaluation and parental opinion, are inherently subjective; the other two criteria, which rely on standardized tests, offer the opportunity for consistency. Consequently, we have recommended that the department, in consultation with stakeholders, establish required initial designation and redesignation criteria related to statewide tests that would provide greater consistency in the English learner population across the State and pursue legislative action, as necessary, to achieve this goal. 2 Although the department states that its monitoring instrument always contained an item related to the proper categorization of English learners, the evidence suggests that without specific guidance in this area, the department’s reviews have been inconsistent. At page 29 we describe that only one of the 6622 California State Auditor Report 2004-120 California State Auditor Report 2004-120 6633 compliance review reports and checklists for our eight sample school districts indicated a review of English learner records to ensure that they were designated correctly. 3 The department believes that our summary statement on the evaluators’ findings does not fully portray the report. Summary statements are necessarily concise. Further, the evaluators stated in their report that statewide data lack the attributes that would be needed to support more definitive statements of causality. That is, while the evaluators saw a statistically positive relationship between English Language Acquisition Program (ELAP) and selected student outcome measures, they could not say with confidence that ELAP has been the cause of these gains. Thus, we believe it is fair to say that the evaluators did not reach a decisive conclusion about the value of ELAP. 4 We neither state nor imply that we selected our sample from the entire English learner population or from redesignated students. At page 16 we clearly say that the focus of our testing was English learners who were candidates for redesignation in fiscal year 2003–04, but who had not been redesignated as fluent. For context, we added the number of such English learners from which we chose our samples at eight school districts at pages 16, 26, and 27. 5 The department says that it has monitoring processes in place. This is true, but we found them inadequate. At page 29 we describe that only one of the compliance review reports and checklists for our eight sample school districts indicated a review of English learners records to ensure that they were designated correctly. While the department states that it found 68 of the 187 school districts it reviewed in fiscal year 2003–04 to be noncompliant with regard to redesignation, given that the compliance review instrument only provided guidance to test redesignated students, the majority of these noncompliant findings are likely related to such students and not to English learners. 6 We have revised the text at page 17 to be more specific about the weaknesses we found in the department’s guidance and monitoring. 7 The department is incorrect when it says that the Bureau of State Audits implies that these school districts’ criteria for the redesignation of English learners are inappropriate. In fact at page 22 we state that we are not concluding that a particular criterion or scoring standard is preferable to another, but rather that inter-district variation exists. In the report we point out differences in criteria between districts and the effect those differences may have on program funding and performance results. 6644 California State Auditor Report 2004-120 California State Auditor Report 2004-120 6655 8 We revised the text on page 20 to clarify the department’s guidance for this criterion. 9 We disagree. At page 24, we simply point out the likely effect of more stringent criteria on funding. We do not impute a motive. 0 The department states that although the compliance review monitoring instrument did not previously contain specific guidance on testing English learners, its monitoring instrument always contained an item related to the redesignation of eligible English learners. The evidence suggests that without specific guidance in this area, the department’s reviews have been inconsistent. At page 29 we describe that only one of the compliance review reports and checklists for our eight sample school districts indicated a review of English learner records to ensure that they were designated correctly. q Home language surveys for three of the six students showed English as their primary language. However, this was not the only evidence that led us to conclude that these students should not have been initially identified as English learners. We have revised the sentence at page 26 to highlight these other pieces of evidence. w As we stated at page 27, our testing included a review of all available documents in students’ files. Students we noted as exceptions had no documentation indicating that they should be retained as English learners. Further, we shared the exceptions with the school districts and gave them ample opportunity to provide us with additional evidence to the contrary. e As we state at page 16, our scope covered English learners who were candidates for redesignation in fiscal year 2003–04, but who had not been redesignated as fluent. r The department states that its monitoring reviews found similar issues at 20 of the 187 school districts it reviewed in fiscal year 2003–04, and that this is an area that is highly monitored. However, as noted at page 38, the department said that its consultants are not auditors and are not required to perform a detailed review of English learner program expenditures. Further, compliance review reports and checklists for only one of our eight sample school districts indicated a detailed review of program expenditures took place. 6644 California State Auditor Report 2004-120 California State Auditor Report 2004-120 6655 Blank page inserted for reproduction purposes only. 6666 California State Auditor Report 2004-120 California State Auditor Report 2004-120 6677 Agency’s comments provided as text only. Anaheim Union High School District Education Division English Learner Program May 27, 2005 Elaine M. Howle, State Auditor* Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: I have reviewed the redacted draft of the report on the audit requested by the Joint Legislative Audit Committee entitled “California Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult.” I would like to share with you my views on the audit process, and the audit report recommendations. This audit began for my district in late January 2005 with a telephone call from a CDE consultant informing me that the Anaheim Union High School District had been selected for an audit, and that I would be contacted by the Bureau of State Audits about the details. Within three working days, a group of four auditors appeared and spent an entire week combing records, asking questions, meeting with staff and requesting data. When they left, my staff and I spent an additional ten days following up on the detailed list of information that they requested. Since that time, there has not been a week when additional data or some clarification statement has not been requested. The timelines for these requests have been very short, with responses expected to be immediate. This process has unfolded during the time that my office has been heavily involved with required data preparation for our state and federal reports and mandated staff development. In reviewing the report, I cannot fault what was said, but am concerned about two underlying assumptions. The 1 tone of the report suggests that the districts reviewed intentionally avoid redesignating English Learner students from limited- to fluent-English proficient, and that there is something inappropriate about the use of additional 2 or more rigorous redesignation criteria. As a secondary (grade 7-12) district, we feel a special responsibility and urgency to prepare our English Learners for real-world expectations. We believe that the standards we set and the programs that we implement reflect research-based, proven strategies that are educationally and linguistically-appropriate for our students. We implement our program with integrity and are concerned that our parents and community feel that their students’ academic success is our primary goal. To imply that students are placed in or held in programs that do not benefit them in order to collect additional funds is an implication that concerns us. Our additional criteria for initial assessment and redesignation have been discussed with the parents in our District English Learner Advisory Committee (DELAC), who have overwhelmingly supported our higher standards. Of course, as with any large organization, problems do arise. At the time of our initial audit visit, we acknowledged to the audit team that a unique situation at one school had resulted in a group of students not being considered for redesignation. The problem had been resolved, personnel changed and procedures refined prior to the visit. In the end, not all these students met our multiple redesignation * California State Auditor’s comments begin on page 69. 6666 California State Auditor Report 2004-120 California State Auditor Report 2004-120 6677 criteria. Still, this anomaly was the basis of some of the statements in the report, as was the identification 3 of a student as an English Learner who had been previously redesignated as an elementary student. In this case, we discovered that the student had been redesignated and then “re-identified” as an English Learner by two elementary districts prior to coming to our district as a seventh grader. As a secondary district with five elementary districts routinely sending us thousands of students each year, we are dependent on the timeliness and accuracy of the information that we receive from these districts prior to the annual student transition. In this case, neither the electronic data we received nor the student’s academic performance indicated that the student was anything other than an English Learner. We eventually redesignated the student, but were faulted in the report for not knowing sooner. Another concerning aspect of this report is the suggestion that there is something inappropriate or improper about the fact that we have initial identification and/or redesignation criteria that are higher than 4 the minimum set by the CDE. As you know, districts are required to use the minimum criteria outlined by the state, but are allowed to add their own local measures. In our case, that has meant the use of higher initial CELDT scores for initial fluent identification (I-FEP), and the inclusion of a 2.0 total grade point average (GPA) and higher California Standards Test English Language Arts (CST-ELA) scale scores for redesignation. Our underlying assumption is that the CELDT test is not a rigorous enough measure of English proficiency at the secondary level because the test weighs listening and speaking as heavily as reading and writing. More sophisticated academic literacy is necessary to succeed in high school and beyond. For this reason, we prefer to monitor closely new and potentially-redesignating students until we are sure that they are as competent as their peers, which is the expectation expressed in our Coordinated Compliance Review (CCR) documents. The report recommends that school districts should more closely monitor their designation and redesignation processes to assure that schools actually complete the process and the school district databases accurately reflect all redesignation. I agree with that assessment. In the Anaheim Union High School District, we centralized our initial assessment of students at our Language Assessment Center (LAC) to support the consistent initial identification and redesignation of English Learners. The LAC also monitors all types of EL data and works with the school sites to fill in missing information. I believe that when the CSIS data system is implemented state-wide in the coming months, it will be much easier for school districts to maintain and transfer correct and complete electronic information of all sorts to one another, including the initial assessment and redesignation data focused on in this audit. With almost 1.6 million English Learners in California and the current concerns about school funding, electronic data is the only feasible way to maintain and transmit required information. As for redesignation, it is a process with many steps, which if done correctly, relies on the hard work of teachers, counselors and parents. My staff and I continue to work systematically to monitor and support this site-based process at our twenty- three school sites. In closing, all districts in California look to the California Department of Education for guidance and clarification on program compliance. The CDE holds us accountable through the CCR process. I have never called the CDE for help or clarification and not been generously assisted. I respect the knowledge and expertise that the consultants possess and share with all the school districts in the state. They keep us focused while supporting us. I, for one, thank them for their support and know that whatever comes of this audit, the CDE and its wonderful staff will continue to be our partners in supporting and educating English Learners. Sincerely, (Signed by: Cheryl Quadrelli-Jones) Cheryl Quadrelli-Jones Coordinator, English Learner Program 6688 California State Auditor Report 2004-120 California State Auditor Report 2004-120 6699 COMMENTS California State Auditor’s Comments on the Response From the Anaheim Union High School District To provide clarity and perspective, we are commenting on the Anaheim Union High School District’s (Anaheim) response to our audit. The numbers below correspond to the numbers we have placed in its response. 1 Anaheim indicates that the Bureau of State Audits (bureau) assumes that the school districts reviewed intentionally avoid redesignating students. This is not true. At page 25 we say that inadequate monitoring of the redesignation process by school districts and the Department of Education contribute to the failure to redesignate students when they have met the criteria for fluency. We do not impute a motive. 2 Anaheim indicates that the bureau assumes that the use of additional or more rigorous criteria for the redesignation of English learners is inappropriate. This is not true. In the report we point out differences in criteria between districts and the effect those differences may have on program funding and performance results. At page 22 we state that we are not concluding that a particular criterion or scoring standard is preferable to another, but rather that inter-district variation exists. 3 The school cited by Anaheim as anomalous accounted for three of the district’s seven exceptions we note in Table 2 at page 27. However, the student incorrectly designated as an English learner was not enrolled in this school. Further, documents in this student’s cumulative file indicate that in 1999 his elementary school district redesignated him as fluent, and that as part of follow up of the student’s progress in 2000, the same district said that he showed no signs of academic deficiency due to his English language proficiency. 4 We do not suggest that Anaheim’s criteria for the initial designation and the redesignation of English learners are improper or inappropriate. In fact, on pages 19 and 22, we state that we are not concluding that a particular criterion or scoring standard is preferable to another, but rather that inter-district variation exists. 6688 California State Auditor Report 2004-120 California State Auditor Report 2004-120 6699 We do, however, point out differences in criteria between districts and the effect those differences may have on program funding and performance results. 7700 California State Auditor Report 2004-120 California State Auditor Report 2004-120 7711 Agency’s comments provided as text only. Palms Office Program Assistance for Language Minority Students Long Beach Unified School District Response to Recommendations “California Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult” 1. School districts should more closely monitor their designation and redesignation processes to assure that school actually complete the process and the school district databases accurately reflect all redesignations. Responsibility for the initial designation and redesignation processes was shifted from the Office of Program Assistance for Language Minority Students (PALMS) to the Office of Research, Planning and Evaluation during the 2004-2005 school year. Ongoing discussions between the two offices continue as more of the designation and redesignation processes are being automated. It is our belief that the automation will facilitate the additional monitoring recommended by the report. The PALMS Assistant Director/Acting Program Administrator and Research Administrative Assistant and Associate Research Analyst will continue to meet to review and refine the monitoring processes. The Research Office will communicate with school sites and the International Student Registration as needed to oversee the initial designation and redesignation processes. 2. School districts should implement documentation policies to ensure that expenditure files clearly demonstrate that supplemental English learner program funds are directed at activities that serve the law’s target populations. Currently, the PALMS Office requires all sites to submit a strategic plan listing the activities (i.e., after-school tutoring), supplemental materials and personnel for all categorical funds allocated. Additionally, a strategic plan is created for centrally-held funds. The PALMS Assistant Director/Acting Program Administrator and the Assistant Superintendent, Curriculum, Instruction & Professional Development, will develop more detailed requirements for expenditure files at all sites and offices to be implemented July 1, 2005. Submitted May 31, 2005 Pamela Seki, Assistant Director/Acting Program Administrator Program Assistance for Language Minority Students Long Beach Unified School District 1515 Hughes Way Long Beach, CA 90810 (562) 997-8031 pseki@lbusd.k12.ca.us 7700 California State Auditor Report 2004-120 California State Auditor Report 2004-120 7711 Blank page inserted for reproduction purposes only. 7722 California State Auditor Report 2004-120 California State Auditor Report 2004-120 7733 Agency’s comments provided as text only. Los Angeles Unified School District 333 South Beaudry Avenue, 24th Floor Los Angeles, California 90017 June 3, 2005 Elaine M. Howle, State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Elaine Howle, This is the Los Angeles Unified School District’s written response to the May 26, 2005 State Audit Report regarding District Title III, EJA-LEP and ELAP funds. The District’s response to the Auditor’s Report addresses the report’s recommendations and includes an explanation and plan to remedy the identified exceptions. The areas noted in the response were the monitoring of student redesignation, the maintenance of documentation to ensure student support and the targeted use of intervention funds. The enclosures are a cover letter with the narrative response and a diskette with a copy of these documents. If you need any further information please contact Jim Morris, Chief of Staff at (213) 241-1700. Sincerely (Signed by: Roy Romer) Roy Romer 7722 California State Auditor Report 2004-120 California State Auditor Report 2004-120 7733 LOS ANGELES UNIFIED SCHOOL DISTRICT This is the Los Angeles Unified School District’s response to the California State Auditor No. 2004-1120 titled “California Department of Education: School District’s Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult”. The District’s narrative addresses the two recommendations cited in this report. The following were areas of exception which were noted for response. 1. Monitor student designation and redesignation process 2. Maintain documentation which ensures support to English learners 3. Target use of ELAP funds for English learners I. CDE Recommendation (pg 64)*: Redesignation School districts should more closely monitor their designation and redesignation processes to assure that schools actually complete the process and the school district databases accurately reflect all redesignations. District Response: LAUSD adheres to the CDE initial designation of English learner guidelines and doesn’t impose more stringent standards on new enrollees. Likewise LAUSD adheres to the CDE redesignation criteria. However, LAUSD will assure that English learners who meet redesignation criteria are removed from the English learner population promptly 1) by ensuring the Student Information System (SIS) is updated automatically when students meet the criteria, and 2) by providing more structured central and local district monitoring procedures designed to ensure schools consistently comply with the district’s redesignation process. District Plan to Remedy: 1. Modify district redesignation procedures as described in policy Reference Guide #1416, Reclassification of English Learners, Elementary Schools and Reference Guide #1417, Reclassification of English Learners, Secondary Schools issued November 22, 2004. Currently district policy requires schools to wait until they receive a signed parent notification letter before manually updating SIS classification field to RFEP status. * Text refers to page numbers in earlier draft version of the report. 7744 California State Auditor Report 2004-120 California State Auditor Report 2004-120 7755 LOS ANGELES UNIFIED SCHOOL DISTRICT As a remedy, LAUSD will revise and reissue Reference Guides #1416 and 1417 at the beginning of the 2005-06 school year. The new policy will require SIS to automatically update the classification field from LEP to RFEP once the school prints the parent notification letter. This will avoid our dependence on manual SIS updates and should ensure that future State Auditors will not find students listed as LEP on SIS who have all the reclassification documents on file. 2. Develop more structured district redesignation monitoring and technical assistance procedures at the central and local district level. Currently the district requires local districts to submit a list schools that have submitted an updated SIS Reclassification Eligibility Roster and sample parent letters to them to the Language Acquisition Branch annually by January. Documentation remains on file at the local district EL Program office. District Plan to Remedy: (continued) As a remedy, LAUSD will revisit the annual EL Program Monitoring Notebook requirements to include the following evidence of structured professional development designed to support compliance with district’s reclassification process: Central Technical Assistance Local District Technical Assistance to Local District EL Program Staff to School EL Program Staff Insert additional Monitoring Notebook forms Submit required Monitoring Notebook for EL 1: Reclassification. (July 05) forms to LAB (June 06) Develop and provide professional Submit agenda and sign-in for development based on new training packet to reclassification policy professional support revised Reference Guides. (Aug/Sept) development. (Sept/Nov) Develop and provide professional Submit agenda and sign-in for development based on new training packet reclassification procedures and to support revised collecting and checking intervention services professional Reclassification Eligibility Rosters and development. (Nov/Dec) providing intervention services. (Oct/Nov) Revisit professional development based on Submit agenda and sign-in for 2nd training packet to support collecting and reclassification procedures professional checking updated Reclassification Eligibility development. (Feb/March) Rosters. (Jan/Feb) 7744 California State Auditor Report 2004-120 California State Auditor Report 2004-120 7755 LOS ANGELES UNIFIED SCHOOL DISTRICT II. CDE Recommendation (pg 65*): EL Documentation School districts should implement documentation policies to ensure that expenditure files clearly demonstrate that supplemental English learner program funds are directed at activities that serve the law’s target populations. District Response: LAUSD’s English learners have access to district general fund resources as well as to supplemental fund resources. However, the district recognizes the need to improve the monitoring of school English learner program expenditures. District Plan to Remedy: To ensure that expenditure files demonstrate that supplemental English learner program funds are directed at activities that serve the target population, LAUSD will do the following: 1. Review the district’s annually published Program and Budget Handbook to verify that documentation policies are clearly stated and outlined. 2. Design and include an additional workshop for the Annual Master Plan Institute that addresses supplemental English learner program fund guidelines and expenditure files. 3. Provide professional development to all Local District staff that provides technical assistance to schools on budget expenditures, such as the English Learner Program staff, the Fiscal Specialists and Fiscal Managers. 4. Include a budget component into the professional development provided to new school assistant principals and principals to emphasize district documentation policies and English learner program funding priorities and guidelines. 5. Organize monthly meetings to provide ongoing professional development and technical assistance to both Local District English Learner and Categorical Program staff, and Fiscal Specialist. 6. Revisit the Coordinated Compliance Self-Review process to improve the procedures on analyzing school level English learner program expenditures and verification of supporting documents to ensure that funds are used to meet the academic needs of English learners. 7. Hold regular School English Learner Program Coordinator meetings in each Local District to review, monitor, and provide technical assistance on English learner program school expenditures. 7766 California State Auditor Report 2004-120 California State Auditor Report 2004-120 7777 LOS ANGELES UNIFIED SCHOOL DISTRICT III. CDE Recommendation (pg 65*): ELAP School districts should implement documentation policies to ensure that expenditure files clearly demonstrate that supplemental English learner program funds are directed at activities that serve the law’s target populations. District Response The district has initiated its own intervention program for English Learners, among others, which are general fund supported. In addition, the district receives ELAP funds for eligible students in grades 4-8. When developing programs for these qualifying 4-8 grade students, the district has attempted to augment the scope of the ELAP program with the District Intervention program. The district recognizes the need of improving communication and implementation regarding ELAP guidelines (criteria, expenditures, and attendance accounting) with participating schools through district memorandum and meetings, and the need to monitor the program centrally. District Plan to Remedy: To ensure that expenditure files demonstrate that supplemental English learner program funds are directed at activities that serve the target population, LAUSD will do the following: 1. Provide all schools with a District Memorandum outlining specific guidelines for student participation, criteria for expenditures, budget worksheets, and attendance accounting procedures. 2. Establish a separate appropriation code for the ELAP program. 3. Provide budget assistance to schools through the local district fiscal support staff to ensure compliance. 4. Provide professional development to all Local District Personnel providing technical assistance to schools on budget expenditures and procedures for the various programs providing tutorial support such as Intervention Coordinators, English Learner Program staff, Fiscal Specialists and Fiscal Managers. 5. Provide professional development for Principals, Assistant Principals, English Language Coordinators, Title I Coordinators and School Administrative Assistants to assist schools in understanding the mandated guidelines and procedures of the ELAP Program. 6. Submit the entire ELAP award only to the individual schools the state department has authorized for specific funding amounts with the guidelines and procedures for implementing the ELAP Program. 7. Provide regular updates and reminders in each local district regarding the eligibility criteria and expenditure requirements for the ELAP Program thru the Superintendent’s monthly Principal’s Meetings. 8. Pursue records of 2003-2004 ELAP expenditures and accounting data at each site provided with ELAP funds. 7766 California State Auditor Report 2004-120 California State Auditor Report 2004-120 7777 Blank page inserted for reproduction purposes only. 7788 California State Auditor Report 2004-120 California State Auditor Report 2004-120 7799 Agency’s comments provided as text only. Pajaro Valley Unified School District Federal and State Programs May 31, 2005 TO: Jim Larsen/Bureau of State Audits* FROM: Cindy Cordova/Director of Federal and State Programs RE: Response to Report Received 5/27/05 I appreciate the opportunity to respond to the draft report titled: “California Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult”. As I mentioned during our phone conversation this afternoon, I suggest that the following revisions be made. While the document may be technically correct, it is very misleading. It leads the reader 1 to the incorrect assumption that grossly inadequate monitoring of the redesignation process has caused vast numbers of students to remain English learners. I suggest that you provide a context that is a fairer representation of your findings. For example, in Pajaro, out of 500 (plus or minus) students that met the redesignation criteria, based on their CELDT and CST data, over 450 were redesignated. Your study only sampled 20 of the 50 that remained English learners on our district database. For the sake of clarity, I suggest that you include a table which includes the number 2 of potential R-FEP’s, the actual number of students that were redesignated, and the number of potential R-FEP’s that remained English learners for each district you visited. On page 44†, the report states that districts purchased refreshments, meals and rental equipment.... Please note that there has never been any use of public funds for private purposes in Pajaro. We feel that involving parents in their students’ education, through our ELAC meetings, directly impacts student achievement in a positive way. Providing a light dinner to working parents facilitates productive interchange and yes, increases attendance. And for the record, there were no decorations purchased for the ELAC meetings; please make this correction. 3 We do appreciate the opportunity to improve our programs and services to English learners, and all students in the district, and have gained some important insights through this audit process. We have already put into place, as we shared with your auditor Ben Ward, systems that will greatly improve how we monitor the reclassification process so that every student, that is qualified to be redesignated, becomes redesignated. Again, thank you for the opportunity to provide feedback. * California State Auditor’s comments appear on page 81. † Text refers to page numbers in earlier draft version of the report. 7788 California State Auditor Report 2004-120 California State Auditor Report 2004-120 7799 Blank page inserted for reproduction purposes only. 8800 California State Auditor Report 2004-120 California State Auditor Report 2004-120 8811 COMMENTS California State Auditor’s Comments on the Response From the Pajaro Valley Unified School District To provide clarity and perspective, we are commenting on the Pajaro Valley Unified School District’s (Pajaro) response to our audit. The numbers below correspond to the numbers we have placed in its response. 1 Pajaro believes our report is misleading and suggests that we include a table listing the number of potential fluent students at each district, the number of these students who were redesignated, and the number of these students that remained English learners. In the course of the audit, we considered including such a table. However, since districts have different redesignation criteria and use differing subsets of these criteria to develop listings of redesignation candidates, their pools of redesignation candidates are dissimilar. Consequently, data showing the number of redesignation candidates and the number of those candidates who were not redesignated are not comparable between districts and could be misinterpreted as indicating relatively better performance by one district versus another. In discussing the draft report with staff from Pajaro, we told them that they could present information on their own district in their response if they wanted to do so. For context, we included the total number of English learners from which we selected our sample for the eight districts visited at pages 16, 26, and 27. 2 While it is true that we tested only 20 of the English learners who were eligible for redesignation in fiscal year 2003–04 but were not redesignated, it is interesting to note that subsequent to our visit, Pajaro investigated all such students. Pajaro told us it found errors in 49 of the 55 student files it reviewed. 3 As we told Pajaro while the district was reviewing the draft report, we removed the word “decorations” from the report. 8800 California State Auditor Report 2004-120 California State Auditor Report 2004-120 8811 Blank page inserted for reproduction purposes only. 8822 California State Auditor Report 2004-120 California State Auditor Report 2004-120 8833 Agency’s comments provided as text only. Sacramento City Unified School District Office of the Superintendent M. Magdalena Carrillo Mejia, Ph.D., Superintendent 5735 47th Avenue Sacramento, CA 95824 June 2, 2005 Elaine M. Howle, State Auditor* Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Ms. Howle, Thank you for the opportunity to respond to the redacted draft copy of your audit report, titled “California Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult.” Please find attached a copy of our response to your findings and recommendations as they relate to the Sacramento City Unified School District. Sincerely, (Signed by: M. Magdalena Carrillo Mejia) M. Magdalena Carrillo Mejia, Ph.D. Superintendent * California State Auditor’s comments begin on page 87. 8822 California State Auditor Report 2004-120 California State Auditor Report 2004-120 8833 Response to California State Audit Sacramento City Unified School District June 2, 2005 Pg. 26* 1 The district criteria for initial identification respond to the need to ascertain that students will be able to access the core instruction fully in English. For incoming students at the kindergarten and first grade levels, the CELDT only assesses listening and speaking. These assessments are not sufficient to indicate the students’ abilities to comprehend academic language and to write on par with English-only peers. Pg. 27 Sacramento, like other districts, is charged with ensuring that all students reach the proficient level 1 on the California Standards Tests. The identification criteria used in the district are intended to identify areas of need to be addressed. By identifying these areas of need, we are better able to ensure that students receive sufficient support to enable them to master standards and achieve proficiency on the standards assessments. Further, the district has established these criteria with the approval of our local Board of Education in line with current state laws and regulations. These criteria have been reviewed by the California Department of Education most recently in 2002-03 and found to be compliant. Pg. 29 - 32 The reclassification criteria in the Sacramento City USD are aligned with the cut-offs used to initially identify students as English Learners. The report does not substantiate the statement that the large 2 numbers of the English Learners at the higher levels is due to the “retention” of high scorers (on the CELDT.) The “larger number” of English Learners is due to enrollment patterns and the progress the students are making both on the CSTs and the curriculum-embedded assessments. In 2003- 04, 71% of students who were initially tested on the CELDT scored at the Intermediate or below levels. Only 29% scored at the Early Advanced or Advanced levels. In reviewing kindergarten students identified in the first year of the CELDT administration in 2000-01 as initial Fluent English Proficient (FEP) using the district’s criteria, only 59% met the CST levels of Proficient or Advanced in the 2003-04 administration of this assessment. While there are several contributing factors that could be discerned, even with identification criteria labeled as “more stringent” (Audit Report, page 26), 40% of the students identified failed to meet the CST standards in language arts. The reclassification process has as its purpose to establish that former English Learners have demonstrated “…English language proficiency comparable to that of the average native English speakers and can participate equally with them in the school’s regular instructional program.” (California Coordinated Compliance Review {CCR}Training Manual, 2002-03) Further, the CCR Training Manual lists the following multiple criteria that may be used “…for assessing English proficiency and academic achievement…including, but not limited to: * Text refers to page numbers in earlier draft version of the report. 8844 California State Auditor Report 2004-120 California State Auditor Report 2004-120 8855 Response to California State Audit Sacramento City Unified School District June 2, 2005 CCR Training Manual, 2002-03 (used SCUSD Adopted Criteria to determine compliance) • Teacher evaluation of the students’ • Input gathered during the English language proficiency and reclassification meeting at the curriculum mastery school site • Objective assessment of the student’s • CELDT English comprehension and speaking • Curriculum embedded reading proficiency assessment scores • Objective assessment of the student’s • CELDT English reading and writing skills • Curriculum embedded reading scores • Writing sample (secondary level) • Parental opinion or consultation • During a meeting at the school during a redesignation interview • Through a letter of notification • Objective data on the student’s • CST scores in language arts and academic performance in English mathematics using the State Board guidelines as a base • Other adopted criteria • none The district’s criteria for initial identification and for reclassification are aligned with the state and federal laws as defined in the CCR Training Manual and with State Board of Education guidelines. Pg. 34 3 In reference to the one student found in the district who was incorrectly designated as an English Learner in the district’s database, a notation existed in the student’s file acknowledging the error. Table 2 (no page number, located between pages 34 and 35) SCUSD is noted as having 17 “students meeting school district redesignation criteria but maintained as English Learners.” Records were found by district personnel and provided to the 4 auditors for 5 out of the 20 students on the list provided. Therefore, the correct number of students for whom records were not available is 15. It must be added that in 2003-04 the district reclassified 904 students out of a pool of 1030 signifying 88%. The schools referenced in the report reclassified 77% of potential students, significantly less than the district’s rate. The auditors had access to the district’s entire student database to review and from which to select its sample. The records of only 5 9 of 82 schools are referenced and the 9 schools are not representative of the schools in the district as 8 are secondary schools. 8844 California State Auditor Report 2004-120 California State Auditor Report 2004-120 8855 Response to California State Audit Sacramento City Unified School District June 2, 2005 At the schools listed in the audit report, the following number of students were reclassified in 2003-04: School Potential Reclassified Percentage 6 John Still 28 10 36% W. C. Wood 38 28 74% C. Chavez 18 12 67% Sutter Middle 13 8 62% Kennedy High 22 22 100% Rosemont High n/a (school was not open 0 (auditors reviewed 1 student n/a prior to 2003-04) reclassified in 2000) McClatchy High 17 16 94% H. Johnson High 20 19 95% C. Goethe Middle 23 22 96% TOTAL 179 137 77% Pg. 37 While the audit report acknowledges that Sacramento City USD has a formal monitoring process, 7 the report erroneously concludes that the district is not implementing its processes. This unsubstantiated conclusion is based on the records of 20 students drawn from a total population of English Learners of 15,110 and a population of 1010 students who met triggering criteria. Table 4 Two of the three expenditures were marked as unclear because the job classifications used in 8 the district are general. School Plans describe the functions that are specific for the employee being funded. The other exception involved training software for teachers on strategies for English Learners identified in the supporting documents for purchase authorization. Pg. 64 (only two visible recommendations in the redacted version received by the district) The district continues to conduct on-going monitoring of all of its compliance processes. In fact, the process has been strengthened throughout the 2004-05 school year. Each associate superintendent who supervises a cohort of elementary, middle, and high schools has received monitoring reports reflecting on-site verification of completion of all required processes. This involvement serves to reiterate to schools the importance of the established processes and criteria. Pg. 65 The district will continue to refine its documentation policies to ensure that expenditure files clearly demonstrate that supplemental English Learner program funds are used as required. 8866 California State Auditor Report 2004-120 California State Auditor Report 2004-120 8877 COMMENTS California State Auditor’s Comments on the Response From the Sacramento Unified School District To provide clarity and perspective, we are commenting on the Sacramento Unified School District’s (Sacramento) response to our audit. The numbers below correspond to the numbers we have placed in its response. 1 Sacramento seems to indicate that the Bureau of State Audits (bureau) believes that the district’s criteria for the initial identification of English learners are inappropriate. This is not true. However, in the report we do point out differences in criteria between districts and the effect those differences may have on program funding and performance results. At page 19 we state that we are not concluding that a particular scoring standard is preferable to another, but rather that inter-district variation exists. 2 Sacramento questions the connection we make between a district’s proportion of English learners who score in the upper proficiency levels on the California English Language Development Test and a district’s redesignation policies. We believe that the evidence, as presented in Table 1 at page 23, plus the examples at pages 19, 22, and 29 of the effect of more stringent criteria on the identification and retention of English learners, supports such a connection. 3 Sacramento states that a notation existed in the student’s file noting the designation error. This is true. Nevertheless, Sacramento continued to include the student on its list of English learners. 4 Sacramento indicates that it provided records for only five of 20 tested students. It is unclear how this statement ties to Table 2 on page 27. Sacramento provided us with cumulative student files for all tested students. Seventeen of the 20 students met district criteria for redesignation based on a review of documents available in these files, and we report this fact in Table 2. 5 Sacramento indicates that the bureau did not choose a representative sample. As we state at page 16, we focused our testing on English learners who were candidates for redesignation in fiscal year 2003–04, but who had not been 8866 California State Auditor Report 2004-120 California State Auditor Report 2004-120 8877 redesignated as fluent. As a result, it is not surprising that selected students were enrolled in schools that generally had redesignation rates that were lower than the district average. 6 Sacramento’s table shows data on the schools where our tested students where enrolled in fiscal year 2004–05. These are not necessarily the schools where these students were enrolled in fiscal year 2003–04 and where the redesignation decisions we tested were made. Sacramento’s table indicates that the bureau selected students from schools that had no redesignation candidates or that had redesignated all students. This is not the case. 7 Sacramento says that the bureau concluded that the district is not implementing its processes. This is incorrect. At page 28 we say that given the exceptions we noted, it is likely that the district is not fully implementing its monitoring process. 8 Sacramento seems to say that its documentation fully supported the charging of costs to English learner programs. This is not the case. Documentation, including school plans, indicated that non-English learners, as well as English learners, benefited from these three expenditures, which were charged to English learner programs. 8888 California State Auditor Report 2004-120 California State Auditor Report 2004-120 8899 Agency’s comments provided as text only. San Diego City Schools Eugene Brucker Education Center 4100 Normal Street San Diego, CA 92103-2682 June 1, 2005 Elaine M. Howle* State Auditor 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Ms. Howle: Thank you for the opportunity to review and respond to the draft report entitled “California Department of Education: School Districts’ Inconsistent IdentUlcation and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult.” The San Diego Unified School District’s written response to statements made by state auditors is attached, both in hard copy and on the disk provided by your office. Also included are pertinent district data and University of California Linguistic Minority Research Institute papers. It is our belief that these documents serve to lay a foundation of understanding for the complexity of issues inherent in learning English as a second language while at the same time maintaining academic advancement at a level commensurate with native speakers of English. In our view, in order to understand and measure the allegations regarding San Diego City Schools made or implied in this report, it will be essential for any reader of the report to consider complete and unbiased information and consider the perspectives of the districts in question. Therefore, I trust the report will either be modified to incorporate the information we have provided in our response, or that the report, if published as is, will include our reply. For our students, (Signed by: Leslie Fausset for) Alan D. Bersin Superintendent of Public Education Enclosures * California State Auditor’s comments begin on page 95. 8888 California State Auditor Report 2004-120 California State Auditor Report 2004-120 8899 SAN DIEGO UNIFIED SCHOOL DISTRICT DISTRICT RESPONSE May 26, 2005 Draft of the California State Auditor Report: “California Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult” June 1, 2005 LANGUAGE CONCERNING STATE REQUIREMENTS The California State Auditor Report titled “California Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult,” contains misleading language about state 1 requirements. The use of certain phraseology makes it appear as if the law mandates certain minimal levels at which a student can qualify for English Learner (EL) services, or be reclassified from EL to Fluent English Proficient (FEP) status. In fact, the state merely establishes certain minimal levels, and districts are allowed, indeed encouraged, to adjust those minimums according to the needs of language minority students in attaining full proficiency in English and meeting or exceeding grade-level standards. The word “should” is used throughout the draft report rather than the word “may,” implying that the district uses improper standards to identify or reclassify students. This implication is emphatically not the case. CRITERIA FOR THE INITIAL DESIGNATION OF ENGLISH LEARNERS On page 25* of the draft report, it states that the department has published guidelines which are board-approved for schools to use in developing both initial and redesignation criteria. It goes on to state, “school districts are not required to adhere to the department’s guidelines.” San Diego City Schools (SDCS) reviews English learner data on a regular basis and has set both initial and redesignation criteria to support students’ academic success. A comparison of 2003 CST English Language Arts performance among students who were new to the district in 2002-03 indicate that district criteria for initial identification supports academic achievement (see Attachment 1). * Text refers to page numbers in earlier draft version of the report. 9900 California State Auditor Report 2004-120 California State Auditor Report 2004-120 9911 District Analysis of Findings The comparison of 2003 CST ELA results across various English language proficiencies indicated that: • Newly enrolled students assessed with the CELDT, who earned an OPL of “advanced” and met the criteria for an IFEP designation (“advanced” IFEP), dramatically outperformed other groups, including newly enrolled native English speakers (English Only). • Overall, newly enrolled students assessed with the CELDT, who earned an OPL of “early advanced” and met the criteria for an IFEP designation (“early advanced” IFEP), outperformed newly enrolled students assessed with the CELDT who earned an OPL of “early advanced” and did not meet the criteria for an IFEP designation ( “early advanced” EL). This finding appears to support the district criteria on which IFEP status is assigned. Furthermore, research from University of California Linguistic Minority Research Institute (UC LMRI) has reviewed the CELDT results over a four-year period (see Attachment 2). “The CELDT results suggest that English proficiency is improving, but it is difficult to interpret these score gains because the state has never released any information on what these tests actually measure, or on their year-to-year consistency. In particular, we do not know to what extent the test accurately reflects the ability to use English as a tool for learning.” The goal for all students enrolled in SDCS is that each student graduates from high school and has access to college (Attachment 1). The draft report indicates on page 26 that SDCS holds new enrollees to higher scoring standards and on page 27 concludes that with higher standards districts end up with larger English learner 2 enrollments, however, the rationale for higher standards is to improve student achievement not to increase funding. The data presented in the UC LMRI analysis demonstrates the clear need for high standards (refer to Attachment 2). CRITERIA FOR REDESIGNATION OF ENGLISH LEARNERS The draft report verifies that school districts are given discretion in determining criteria for redesignation. While SDCS utilizes the four criteria recommended by the department, the report indicates that the district appears to be maintaining larger proportions of English learners due to higher standards for the CELDT criteria considered for redesignation. The goal for all English learners is to reach academic standards post-reclassification, maintain this level of performance and be able to compete with native English speakers in the district. In order for students to achieve grade level standards fluency with academic language is required. This means they must be able to read, speak, and write about more abstract — that is, less contextualized — concepts and topics and do so using the more formal language structures and functions associated with critical thinking (Cummins, 1991; Chamot & O’Malley, 1994). Therefore, SDCS higher CELDT criteria for redesignation are justified. 9900 California State Auditor Report 2004-120 California State Auditor Report 2004-120 9911 Page 30 of the draft report indicates that if SDCS followed the department’s guidance on CELDT scores for redesignation an additional 4 percent of English learners would have been considered 3 for redesignation. Although the CELDT represents a significant advance in the measurement of English language proficiency, the fact remains that no single, objective test can be used to determine whether any given student is sufficiently proficient in English to achieve at grade level standards in an instructional environment designed for native speakers of the language. The UC LMRI, September 2001, report titled “The Redesignation Dilemma,” page 20, states “The complex nature of what ELs must demonstrate in order to be reclassified as FEP is not widely understood by policy makers, by teachers who have not been trained to serve this population, and by much of the general public. The common notion is that students only need to learn English and their academic achievement will naturally follow. This misconception can hamper appropriate, effective, and timely support for EL students.” (See Attachment 3.) The 2002-04 data analysis for English learners enrolled in SDCS indicates that higher standards for redesignation in fact support student achievement. Students that were Early advanced and scored at the mid to high-end of basic on the CST-ELA maintained academic achievement (see Attachment 4 4). Therefore, the statement on page 32 of the draft report is misleading. “The use of more stringent criteria allows some school districts to report larger English learner enrollments than if they established criteria more comparable to those of other school districts and to the department’s guidance. Because English learner enrollments are a primary factor in funding formulas for English learner programs, some school districts likely receive more funding under both state and federal English learner programs than they would if their criteria were more closely aligned with the department’s guidance and criteria used by other districts.” In fact, the state provided guidance so individual districts could make decisions in the best interest of their English learner population. In the case of SDCS, the intent of using more stringent criteria should result in more students having access to college. MONITORING ENGLISH LEARNER PROGRAMS AND FUNDS On pages 35-37 of the draft report, San Diego is called out for not having documentation for 18 of the 20 students that “met district redesignation criteria but were not redesignated.” The Bureau of State Audits is assuming that all four criteria 5 were met but in fact based their assumption on only two criteria, CELDT and CST-ELA. The state auditors seem to be under the impression that districts are required by some law or regulation to keep a paper trail documenting reasons why a student meeting minimal standards on two of the four measures for reclassification is not reclassified, when, in fact, the Coordinated Compliance Review (CCR) Training Guide only calls for such documentation and monitoring of students who have been reclassified. 9922 California State Auditor Report 2004-120 California State Auditor Report 2004-120 9933 Expenditures for English learners are monitored by SDCS staff. Site administrators must approve all expenditures and the site budget analyst monitors expenditures from the central office. The CCR Training Guide does not require a paper trail of documentation in CCR item EL10b (EIA-LEP funding). In fact, the district was found to be in compliance with this aspect of the recent CCR (April 6 2005), which requires CDE reviewers to look at how we used the EIA-LEP funds in order to verify they were: (1) used for EL students, and (2) used in a supplementary manner. The state audit team was informed of policies the district has in place to ensure all categorical funds are used properly. On page 45, their report implied that a SDCS principal’s writing of a letter dated one year after an EIA-funded purchase of “cameras” for her site’s English learners 7 was somehow an attempt to cover-up wrongdoing. In SDCS document cameras comprise one component of the digital classroom – something our district has embraced as a fundamental tool for providing high quality English language development instruction and access to core curriculum for English learners. (See Attachment 5 for information about the digital classroom). In addition, the principal would not have written such a letter in the first place except for the fact that the audit team specifically requested that the district provide explanations for a set of sample purchases they selected during their review. The letter in question was written in response to that request. ♦ ATTACHMENTS: 1) San Diego City Schools Comparison of 2003 CST ELA Performance Levels Among Students Who Were New to the District in 2002–03 2) UC LMRI Winter 2005 Newsletter 3) UC LMRI Report: The Redesignation Dilemma 4) San Diego City Schools 2004 CST English Language Arts Performance of English Learners Reclassified at “Basic” in 2003–04 (Two-Year Cohort) 5) San Diego City Schools Digital Classroom Information ♦ We have not included attachments in the report; however, they are available for review at the California State Auditor’s office. 9922 California State Auditor Report 2004-120 California State Auditor Report 2004-120 9933 Blank page inserted for reproduction purposes only. 9944 California State Auditor Report 2004-120 California State Auditor Report 2004-120 9955 COMMENTS California State Auditor’s Comments on the Response From the San Diego Unified School District To provide clarity and perspective, we are commenting on the San Diego Unified School District’s (San Diego) response to our audit. The numbers below correspond to the numbers we have placed in its response. 1 San Diego contends that the Bureau of State Audits’ (bureau) report contains misleading language about state requirements for redesignation criteria. This is not the case. At page 18 we list the law’s four specific required criteria for redesignation and explain that the Department of Education (department) published guidelines for school districts to use. However, because these are not regulations, school districts are not required to adhere to them. 2 We do not conclude that San Diego set its criteria for the initial identification to increase funding. However, we point out differences in criteria between districts and the effect those differences may have on program funding and performance measurement results. At page 19 we state that we are not concluding that a particular scoring standard is preferable to another, but rather that inter-district variation exists. 3 San Diego misses the point we are making. The examples at page 22 illustrate the effect of varying redesignation criteria on student populations. We recognize that the law provides for the use of multiple criteria. In fact at page 18 we describe four specific criteria school districts must include in their redesignation criteria. 4 San Diego implies that the bureau believes that the school district’s criteria for the redesignation of English learners are inappropriate. This is not true. In the report we point out differences in criteria between school districts and the effect those differences may have on program funding and performance measurement results. At page 22 we state that we are not concluding that a particular criterion or scoring standard is preferable to another, but rather that inter-district variation exists. 9944 California State Auditor Report 2004-120 California State Auditor Report 2004-120 9955 5 San Diego indicates that the bureau only tested for two of its criteria. This is not true. We reviewed student files for all criteria and based our determination on the available documentation. As we state at page 28, we expected to see teacher comments or evidence of parent consultation in the students’ records explaining why students who met school district criteria were still designated as English learners. San Diego contends that there are no laws or regulations requiring school districts to maintain documentation describing the reasons why English learners who are candidates for fluent status are not redesignated. At page 28 we describe that state regulations require such documentation. San Diego also states that the department’s coordinated compliance review (compliance review) only requires documentation for students who have been redesignated. At page 29 we point out that the department’s guidance only covered redesignated students, a fact that may have lead San Diego to think that documentation for English learners was not necessary. We also note that in May 2005 the department changed its guidance to include English learners. Our report also recommends that the department should require school districts to document all redesignation decisions, including decisions against redesignating students who are candidates for fluent status. 6 San Diego implies that the department does not require documentation to detail the purpose of expenditures of English learner program funds. We recognize this problem at page 37, and recommend that the department revise the documentation policy it provides to school districts. The district also stated that it was found in compliance in its most current compliance review report relating to this area. At page 38 we point out the weaknesses of the compliance review process and note that only one of the compliance review reports for the eight districts we reviewed indicated a detailed review of expenditures had been conducted. San Diego was not that district. 7 We did not mean to imply any wrongdoing by the San Diego principal. We have revised the text at page 36 to avoid this interpretation. 9966 California State Auditor Report 2004-120 California State Auditor Report 2004-120 9977 Agency’s comments provided as text only. San Francisco Unified School District Mary Ellen Gallegos, Executive Director Multilingual Programs 1098 Harrison Street San Francisco, CA 94103 June 3, 2005 Elaine M. Howle, State Auditor* Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Ms. Howle: We have received the redacted draft copy of the report entitled “California Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult.” In response, we respectfully submit the following information. On page 30† of the report, it states that San Francisco Unified School District requires a score of at least 325 on the CST-ELA, while 1 some school districts require scores of at least 300. We feel strongly that this requirement is appropriate and within the range approved by the state. Making sure that students have acquired enough academic English to be able to successfully fulfill the necessary content and performance standards they will be required to meet in mainstream or general education classrooms is our ultimate goal. If students are redesignated too soon, the chances of them being successful are decreased. As a matter of fact, the overall redesignation rate for our district has been has been higher than the average for our state for the past three years. On page 11 of the redacted report (Table 4), the Expenditure Exceptions includes one questionable 2 expenditure under the column entitled “Purpose of Expenditure Unclear.” This expenditure was for a multifunded employee who was paid with EL funds for .25 FTE. English learners in that school were assigned to this employee for provision of English Learner services one quarter of every day during that school year, thus the .25 FTE. Please consider including this information in the final report. Sincerely, (Signed by: Mary Ellen Gallegos) Mary Ellen Gallegos * California State Auditor’s comments appear on page 99. † Text refers to page numbers in earlier draft version of the report. 9966 California State Auditor Report 2004-120 California State Auditor Report 2004-120 9977 Blank page inserted for reproduction purposes only. 9988 California State Auditor Report 2004-120 California State Auditor Report 2004-120 9999 COMMENTS California State Auditor’s Comments on the Response From the San Francisco Unified School District To provide clarity and perspective, we are commenting on the San Francisco Unified School District’s (San Francisco) response to our audit. The numbers below correspond to the numbers we have placed in its response. 1 As we state at page 22, we are not concluding that a particular criterion or scoring standard is preferable to another, but rather that inter-district variation exists. In the report we point out differences in criteria between districts and the effect those differences may have on program funding and performance measurement results. 2 San Francisco implies that the one expenditure the Bureau of State Audits lists as unclear in Table 4 on page 35 was appropriately documented. This is not true. Although the district provided documentation listing the employee’s position as multi-funded, for which one-quarter of the employee’s job description was to provide English learner services every day during the school year, the district could not provide timesheets or a documented allocation method to support the proportion of this employee’s salary that it charged to English learner programs. 9988 California State Auditor Report 2004-120 California State Auditor Report 2004-120 9999 Blank page inserted for reproduction purposes only. 110000 California State Auditor Report 2004-120 California State Auditor Report 2004-120 110011 Agency’s comments provided as text only. Stockton Unified School Distict Language Development Office 1503 St. Mark’s Plaza, Suite D-1 Stockton, CA 95207-6410 Date: June 3, 2005 To: Elaine M. Howle, State Auditor* From: Katarin Jurich, Assistant Director of Curriculum (Signed by: Katarin Jurich) RE: Response to California State Auditor report titled “California Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult.” In response to the California State Auditor Report please find enclosed the following: 1) One hard copy of the response from Stockton Unified School District to the State Audit Report and 2) An electronic version of the response filed on the floppy disk that was provided As per the request of the State Audit Office I am faxing a copy of my response to the State Audit office in order to comply with the timeframe of the original letter. Thank you for the opportunity to work with you to improve services to English Learner students. If you have any questions please feel free to contact me at the above number. * California State Auditor’s comments begin on page 105. 110000 California State Auditor Report 2004-120 California State Auditor Report 2004-120 110011 Stockton Unified School Distict Language Development Office 1503 St. Mark’s Plaza, Suite D-1 Stockton, CA 95207-6410 This is a response to the California State Auditor report titled “California Department of Education: School Districts’ Inconsistent Identification and Redesignation of English Learners Cause Funding Variances and Make Comparisons of Performance Outcomes Difficult.” We appreciate the thorough and collegial work that was conducted by the State Auditors office and have found the review supportive of areas we have already identified as in need of change. As such the process has been both helpful and instructive. I would like to respond to two particular areas of the report in order to provide clarification to the SUSD situation. First there are concerns regarding specific findings and statements surrounding redesignation process and procedures. The second has to do with fiscal concerns. Redesignation Concerns: The Report states that “…Stockton does not require any teacher assessment, which does not 1 appear to conform to the law’s requirements to have a teacher assessment” (Report, p. 37)*. This is not the case. The State “allows districts to define what constitutes teacher evaluation of student academic performance” (Report, p. 28). In SUSD each teacher who has a student who is identified as eligible for reclassification is notified of that eligibility. Site EL specialists meet with teachers and obtain teacher input as part of the reclassification process. There is no formal “evaluation” per se, rather an informal review of student work, their grades and performance in the classroom. At that point a teacher may suggest that an EL student is not, in her/his opinion, ready for mainstream instruction but would benefit from continued ELD and sheltered instruction. In actuality this is a very rare occurrence. In all reclassification cases though, there is a process for teacher input, as defined by review of student work, grades and classroom participation – all of which are considered reflective of teacher assessment. I think the report might be more accurate if it were noted that there is an informal process to include teacher assessment, but the process needs to be formalized and codified in the Master Plan. The process we have in place guards against the very issue raised in the Report that “this school district may redesignate students who have developed proficiency in English but have not yet demonstrated the ability to compete academically with pupils of the same age whose native language is English.” In another vein, an analysis of standardized test data for the district indicates that RFEP students consistently outperform many subgroups, including English Only students in several academic areas. I would posit that SUSD does have teacher assessment as part of the redesignation process. What is lacking is separate paper documentation of those meetings and reviews. A new form has been created and will be part of the Master Plan revision that goes to the district Board for approval this summer. (See Attachment Reclassification Form dated March, 2005). 2 The Report states that the monitoring of the redesignation process was limited to “providing technical assistance, including training, to schools regarding English learner programs and services” (Report, p. 37). SUSD was considered part of that group; however, there are additional * Text refers to page numbers in earlier draft version of the report. 110022 California State Auditor Report 2004-120 California State Auditor Report 2004-120 110033 efforts that are part of our process that extend beyond those indicated in the report. In SUSD there has been a concerted effort over the past three years to follow up on reclassification. Twice a year, District bilingual specialists visit schools over a two to three week period to work with EL deputies and teachers to support and guide them through the process of reclassification, review student work with teachers, discuss difficult cases, and help complete paperwork to assure that redesignation is complete and accurate. There is extensive support of sites in the process of reclassification. One of the major difficulties continues to be obtaining signed redesignation forms from parents. Forms are sent home, phone calls are made to parents to obtain permission for reclassification, and sometimes forms are sent home a second time. Despite those time consuming efforts there remains a small proportion of eligible students who do not get reclassified because the required parent approval cannot be obtained within the academic year, particularly when students are identified in the spring. In reviewing Table 2 of the report, it seems that the layout of data may be misleading. On cursory 3 glance it may appear that the Report is suggesting that 62% of EL students who meet the reclassification criteria do not get reclassified. What is not clear is that the population does not represent all eligible students, only those who were eligible and did not get reclassified during the 2003-04 academic school year. It would help to have a table that indicated to the reader the 4 number of students who were successfully reclassified in the same time frame. For example, considering only SUSD for the year 2003-04, 83% of all EL students eligible for reclassification were reclassified. Fiscal Concerns 5 Table 4 of the Report indicates that SUSD has two expenditures for which the purpose of expenditures is unclear. The report suggests that there was no documentation for expenditures and that the purchases may not have been related to EL services. ELAP monies during the 2003- 04 school year were all spent on supplemental ELD intervention materials and EL services. The intervention program was implemented that year in which EL students who were not progressing in their acquisition of English were given 90 minutes of intensive ELD through the use of state approved materials. There were several schools that were served that did not appear on the original list. Bush elementary was a new school whose boundaries were created by schools who were on the 2002-03 allocation list: August, Harrison, and Cleveland. During 2003-04 students from those feeder schools were served with ELAP funds. There were two K-8 schools, Rio Calaveras and Golden Valley, where only the elementary school appeared on the list and monies were allocated to grades 7 and 8 as well since all other middle schools were on the allocation list. The CDE does not differentiate the 7th and 8th grades of an elementary school as separate middle schools. When we get the allocation list it is assumed that when the school is named, that all EL students in grades 4-8 are eligible for support. It was suggested that CDE should have been contacted for approval of these expenditures. In the future CDE will be contacted. Since we have several new schools opening in the next few years, and we are moving to a K-8 configuration district wide, it will be important to have clarification regarding the expenditure of ELAP funds at grades 7 and 8 in K-8 school. Three other schools, Tyler, Stockton Skills, and Valenzuela were served and not on the list. The report stated that one district program “Home Instruction” received funding. EL students in that category are actually from regular schools sites and are only temporarily in “home instruction” status. The students were considered eligible for funding and were given support while they were at their home school assignment. 110022 California State Auditor Report 2004-120 California State Auditor Report 2004-120 110033 The other issue of concern was the expenditures for primary language testing. The fundamental issue here is that there is no direct tracking of expenses by site for the work done by the primary language assessors. Although we keep logs of all services provided and can submit all the logs for 2003-04 we did not identify in the logs the expenditures by funding source. In addition to primary language assessment the primary language assessors provide district offices and site based translations, and interpretation services for parents at IEP and expulsion hearings. The tracking of these details by funding source is extremely time consuming and prohibitive given the personnel resources available in our current staffing arrangements. There are several ways that it can be accommodated should we choose to continue to use ELAP funds for this purpose. We will be designing a tracking system for discrete expenditures of funds for primary language testing by site and arrange for additional time to account for these expenditures. In summary, it has been an instructive process, one that has helped reinforce some issues we have been working on to remedy in the past year, and other areas that need some dedicated attention and change in the coming year. It is helpful to have the careful eye of auditors to help refine our own approach to fiscal management, and to take time to review carefully the work that is done. I appreciate the conscientious and collegial approach of the State Auditor staff and commend them for their quick understanding of complex educational issues and processes. If you have questions regarding this response, please feel free to contact me. (Signed by: Katarin Jurich) Katarin Jurich, Ph.D. Assistant Director of Curriculum and Professional Development Stockton Unified School District (209) 933-7075 ext: 2432 110044 California State Auditor Report 2004-120 California State Auditor Report 2004-120 110055 COMMENTS California State Auditor’s Comments on the Response From the Stockton Unified School District To provide clarity and perspective, we are commenting on the Stockton Unified School District’s (Stockton) response to our audit. The numbers below correspond to the numbers we have placed in its response. 1 Stockton’s statement that it requires a teacher evaluation as part of its redesignation criteria is inconsistent with its earlier statements and written procedures. In fact, in e-mail communications with the Bureau of State Audits (bureau) in May 2005, Stockton’s assistant director of curriculum and professional development specifically stated that Stockton did not require a teacher evaluation as part of its redesignation process. Further, Stockton’s written redesignation procedures do not include teacher evaluation as a criterion. 2 Stockton says that it performs additional monitoring beyond that stated in the report. Despite the fact that we asked Stockton to confirm or revise our understanding of its monitoring process, this is the first time that the district has indicated this level of monitoring effort. Further, the noted activities are not included in Stockton’s Master Plan for English Learners, which details the persons responsible for student redesignation and their duties. Nevertheless, based on Stockton’s statement, we have reduced from seven to six the number of districts cited at page 28. 3 Stockton suggests that readers may misinterpret the information the bureau presents in Table 2 on page 27 on the number of redesignation exceptions. We have, however, made clear at pages 16 and 26 and in the note to Table 2 that we focused our testing on English learners who were candidates for redesignation in fiscal year 2003–04, but who had not been redesignated as fluent. 4 Stockton suggests that we include a table that lists the number of potential fluent students at each district and the number of these students that remained English learners. In the course of the audit, we considered including such a table. However, since 110044 California State Auditor Report 2004-120 California State Auditor Report 2004-120 110055 districts have different redesignation criteria and use differing subsets of these criteria to develop listings of redesignation candidates, their pools of redesignation candidates are dissimilar. Consequently, data showing the number of redesignation candidates and the number of those candidates who were not redesignated are not comparable between districts and could be misinterpreted as indicating relatively better performance by one district versus another. For context, we included the total number of English learners from which we selected our sample for the eight districts we visited at pages 16, 26 and 27. 5 Stockton’s statements infer that the two exceptions were related to the English Language Acquisition Program. This is not the case. These expenditures related to the Economic Impact Aid program and in both cases there was no documentation showing how the product or service benefited English learners. 110066 California State Auditor Report 2004-120 California State Auditor Report 2004-120 110077 cc: Members of the Legislature Office of the Lieutenant Governor Milton Marks Commission on California State Government Organization and Economy Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press 110066 California State Auditor Report 2004-120 California State Auditor Report 2004-120 110077