Summary

High School Graduation and Dropout Data California’s New Database May Enable the State to Better Serve Its High School Students Who Are at Risk of Dropping Out March 2012 Report 2011-117 Independent NONPARTISAN Accountability TRANSPARENT The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 916.445.0255 or TTY 916.445.0033 OR This report is also available on the World Wide Web http://www.bsa.ca.gov The California State Auditor is pleased to announce the availability of an on-line subscription service. For information on how to subscribe, please contact the Information Technology Unit at 916.445.0255, ext. 456, or visit our Web site at www.bsa.ca.gov. Alternate format reports available upon request. Permission is granted to reproduce reports. For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. CALIFORNIA STATE AUDITOR Elaine M. Howle State Auditor Doug Cordiner B u r e a u o f S t a t e A u d i t s Chief Deputy 555 Capitol Mall, Suite 300 Sacramento, CA 95814 916.445.0255 916.327.0019 fax www.bsa.ca.gov March 15, 2012 2011-117 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the California State Auditor presents this audit report concerning high school graduation and dropout data. Data reported by school districts to the California Longitudinal Pupil Achievement Data System (CALPADS) is used by the California Department of Education (department) to calculate graduation and dropout rates in accordance with state and federal requirements, including those established by the No Child Left Behind Act of 2001. This report describes how, through the use of a unique student identifier, CALPADS allows the department to track a student’s progress from when he or she enters high school to when he or she exits, and thus allows the department to calculate precise graduation and dropout rates for a particular cohort of students. Data from CALPADS compares favorably to older data, which was based on schools’ aggregate counts of graduates or dropouts. In August 2011 the department used CALPADS to report graduation and dropout rates for the first cohort of students tracked from the 2006-07 through the 2009-10 school year. In fact, this cohort graduated at an overall rate of 74 percent. The data, however, shows a divide in graduation rates between various demographic groups; specifically African-American students, Hispanic or Latino students, and students of lower socioeconomic status generally dropped out of school at rates higher than their peers. By establishing this baseline, we believe the rates generated from CALPADS will prove useful for encouraging data-driven decision making in meeting the needs of all students. We also conclude that there is room to increase the utility of CALPADS and improve the quality of data reported into CALPADS. School personnel varied in the extent to which they verified the reasons a student exited high school, in part because they did not consistently follow the department’s guidance. Additionally, several districts reported that the process of uploading data into CALPADS can be time-consuming and some districts stated that maintaining and updating CALPADS has required significantly more resources than previously required to report data to the State. Finally, because the funding for CALPADS is primarily focused on meeting state and federal reporting requirements, the State may risk missing opportunities to be more innovative in using its longitudinal data. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor California State Auditor Report 2011-117 vii March 2012 Contents Summary 1 Introduction 5 Audit Results The California Longitudinal Pupil Achievement Data System Appears to Measure Graduation and Dropout Rates Effectively 17 CALPADS’ Data Confirm That Graduation and Dropout Rates Vary Among Different Demographic Subgroups 19 Some Challenges Remain for the State in Ensuring That CALPADS Achieves Its Full Potential 26 Districts Employ a Variety of Strategies That May Assist Students at Risk of Dropping Out 33 Recommendations 36 Appendix School Districts’ Dropout Prevention Programs 39 Response to the Audit California Department of Education 45 California State Auditor Report 2011-117 1 March 2012 Summary Results In Brief Audit Highlights . . . The California Department of Education (department) designed the Our review of the high school California Longitudinal Pupil Achievement Data System graduation and dropout data (CALPADS) to gather student-level data from public schools highlighted the following: statewide so that it could comply with state and federal reporting requirements and more accurately calculate graduation and » The California Department of Education dropout rates. Unlike the California Basic Educational Data System (department) gathers student-level (CBEDS), which the department used in the past to extrapolate data from public schools statewide graduation and dropout rates, CALPADS allows the department through the California Longitudinal Pupil to track certain data, such as enrollment status, for individual Achievement Data System (CALPADS). students from the time they enter high school until they exit. Thus, » The State’s graduation rate for the it allows the department to calculate more precise graduation and 2009–10 cohort of students was dropout rates for a particular cohort of students by tracking them 74.4 percent, while the dropout rate through all four years of high school. Although CALPADS’ high was 18.2 percent. school graduation and dropout data are currently published only for the class of 2009–10, the department, school districts, and other • African-American students and stakeholders can use this first cohort’s data as a baseline against Hispanic or Latino students dropped which to compare future graduation and dropout rates. out at higher rates than their peers, as did students who were English In August 2011 the department published its First Annual Report on Language learners. Dropouts in California, which reported that the State’s graduation • White and Asian socioeconomically rate for the 2009–10 cohort of students was 74.4 percent and disadvantaged students were more that its dropout rate was 18.2 percent. (The remaining 7.4 percent likely to drop out than students of students were neither graduates nor dropouts.) CALPADS’ in those groups who were not data show that graduation and dropout rates varied for different socioeconomically disadvantaged. demographic subgroups. Specifically, African-American students and Hispanic or Latino students generally dropped out at • Students who failed the California higher rates than their peers, as did students who were English High School Exit Examination on their Language learners (English learners). Further, white and Asian first attempt were significantly less socioeconomically disadvantaged students were more likely to drop likely to graduate than students who out than white or Asian students who were not socioeconomically passed on their first attempt. disadvantaged; however, socioeconomic status did not have that same inverse relationship in the Hispanic or Latino student » School districts continue to face population. Finally, students who failed the California High challenges in implementing CALPADS. School Exit Examination on their first attempt were significantly • Schools have inconsistent practices less likely to graduate than students who passed on their first for verifying the reasons students exit attempt, although 94 percent of the cohort did eventually pass high school. the examination. • Because the degree to which Although CALPADS represents a significant improvement in personnel verified and documented California’s collection and reporting of graduation and dropout codes varied depending upon the data, school districts continue to face challenges in implementation. reasons that students left, the data Specifically, our review of student records from six school districts on some students may be less accurate suggests that schools have inconsistent practices for verifying than for others. the reasons students exit high school. The department requires that schools select from a standard list of exit codes whenever continued on next page . . . 2 California State Auditor Report 2011-117 March 2012 • Several districts reported that the students leave high school. However, the degree to which school process of uploading information personnel verified and documented these codes varied depending from their local systems into CALPADS upon the reasons that students left; consequently, the data on some can be time consuming. students may be less accurate than they are for others. Although the department has provided school districts with clear guidance » Because the funding for CALPADS is regarding the codes, including through the CALPADS Data Guide, primarily focused on meeting state and the districts have not always fully aligned their procedures with this federal reporting requirements, the guidance. In addition, several districts reported that the process of State may risk missing opportunities uploading information from their local systems into CALPADS can to be more innovative in using its be time-consuming. However, according to one district administrator, longitudinal data. this particular challenge can be mitigated by frequently updating the system with enrollment changes rather than waiting to send large batches of data at once. Finally, some districts stated that maintaining and updating CALPADS has required significantly more resources than previously required to report data to the State. Furthermore, we found that CALPADS’ current capabilities may not allow the system to fully carry out the Legislature’s goals of evaluating its educational progress and investments over time. Although the department provides data from CALPADS to researchers for certain projects, CALPADS currently does not provide the same robustness of data that certain other states’ systems offer. For example, some states’ systems can or will be able to track students’ success through college and even into the workforce. Because the funding for CALPADS is primarily focused on meeting state and federal reporting requirements, the State may risk missing opportunities to be more innovative in using its longitudinal data. Despite these limitations, CALPADS has long-term potential for enabling school districts to identify and develop effective strategies for reducing dropout rates. Although recent budget cuts have affected school districts’ ability to provide dedicated dropout prevention programs, we found that the six districts we visited all have programs and strategies in place to help students who are at risk of dropping out. CALPADS does not currently track student participation in specific programs; however, its data should enable school districts to identify those schools that are able to reduce their dropout rates over time through best practices and innovative programs. This should create an opportunity for schools throughout the State to allocate resources and adopt practices that may effectively serve their at-risk students. Recommendations To increase consistency, the department should remind schools and school districts of the importance of aligning their procedures for recording pupil enrollment and exit data with the CALPADS Data Guide. California State Auditor Report 2011-117 3 March 2012 To improve efficiency, the department should inform school districts of the value of frequently updating the data they transfer from their local student information systems to CALPADS. Also, to the extent that the department becomes aware of ways that schools and school districts can perform CALPADS-related activities more efficiently, it should provide written guidance to schools and school districts on these best practices. To improve the utility of CALPADS and fulfill the legislative intent of the system, the department should work with the Legislature, the State Board of Education, and the governor to identify priorities for building upon the system when funding is available. These priorities could include tracking student participation in dropout prevention programs or strategies to measure the effectiveness of those programs or strategies over time. Agency Comment The department agrees with the report’s recommendations and outlined its plans to implement them. 4 California State Auditor Report 2011-117 March 2012 Blank page inserted for reproduction purposes only. California State Auditor Report 2011-117 5 March 2012 Introduction Background Numerous studies have shown that graduating from high school can be an important predictor of future achievement in life. Research suggests that individuals who successfully complete high school are more likely to obtain stable employment and earn more than dropouts; conversely, dropouts tend to experience higher unemployment and have lower average earnings. As Figure 1 illustrates, the 2010 American Community Survey conducted by the U.S. Census Bureau identified average income levels for Americans with different levels of education. The data indicates that those who do not graduate from high school earn less than graduates nationwide; this income gap is evident in California. Research also indicates that dropping out of high school may be linked to health-related conditions, such as depression, and a higher risk of incarceration. Figure 1 Median Earnings by Educational Attainment, Among Wage Earners 25 Years of Age and Older United States Less Than High School Graduate California High School Graduate or Equivalent Some College or Associate’s Degree Bachelor’s Degree Graduate or Professional Degree $0 $10,000 $20,000 $30,000 $40,000 $50,000 $60,000 $70,000 $80,000 Source: U.S. Census Bureau’s 2010 American Community Survey, one‑year estimates. In addition, students dropping out of high school also leads to consequences for the education system in general and for the State as a whole. When students drop out, it affects the resources available to the schools they leave because the funding California public schools receive is based primarily on average daily attendance. Therefore, when students drop out it may decrease the amount of funding the schools receive. In addition, the education 6 California State Auditor Report 2011-117 March 2012 level of California’s citizens may have long-term consequences for the State in terms of its revenue base and potentially its desirability as a place in which to locate businesses. Graduation Rates and School Accountability California has long recognized the importance of tracking graduation rates as one measure of the performance of its public school system. For more than 20 years, the California Department of Education (department) computed graduation rates based on data it collected through its California Basic Educational Data System (CBEDS). Specifically, the department asked the school districts to report through CBEDS the number of dropouts they had in a given year. From these aggregate counts, the department derived four-year dropout rates. As shown in Figure 2, CBEDS provided some indication of the number of dropouts in California: from school years 1991–92 through 2008–09, the dropout rates based on this data fluctuated between approximately 11 percent and 22 percent. However, because the CBEDS data used aggregate counts rather than tracking individual students, these dropout rates did not effectively account for transfers, whether within the State or to other states and countries. This contributed in part to concerns that the CBEDS-based rates were too imprecise to be a good measure. More than a decade ago, California enacted sweeping education reforms through the Public Schools Accountability Act of 1999 (state accountability act), which requires that the department measure the academic performance of all schools using an Academic Performance Index (API). Under the act, the department uses a number of factors, including student performance on standardized tests and successful completion of the California High School Exit Examination (exit examination), to calculate and assign API scores to schools. The state accountability act also requires the inclusion of high school graduation rates in the API, contingent upon a determination by the state superintendent of public instruction (state superintendent) that the State’s graduation rate data are accurate. Because of concerns about the reliability of the data, the department did not include graduation rates derived from CBEDS data in the API scores. However, key changes to federal law in 2001 provided a strong incentive for California to implement a more effective system for measuring and reporting on graduation rates. Congress designed the federal Elementary and Secondary Education Act of 1965 (federal education act)—the historic law that made federal funding available to the states for education—to promote equal educational opportunity. It has since reauthorized the federal education act California State Auditor Report 2011-117 7 March 2012 several times, most recently through the No Child Left Behind Act of 2001 (No Child Left Behind), which made significant changes to the federal education act. In order to receive federal funding, each state must have a state plan that describes what constitutes progress toward various goals or targets related to academic achievement. No Child Left Behind also requires states to report certain data on students over time, regardless of where they attend public school, a concept known as longitudinal data. Figure 2 Statewide Graduation and Dropout Rates Based on the California Basic Educational Data System School Years 1991–92 Through 2008–09 80% Graduation rate Dropout rate 70 60 50 40 30 20 10 0 1991–92 1992–93 1993–94 1994–95 1995–96 1996–97 1997–98 1998–9 1 9 999–2000 2000–01 2001–02 2002–03 2003–04 2004–05 2005–06 2006–07 2007–08 2008–09 School Years Source: The California Department of Education’s DataQuest Web site. No Child Left Behind evaluates academic progress through a measure known as Adequate Yearly Progress (AYP). This measure reflects the extent to which schools and school districts meet their targets in various areas, including improving student scores on standardized tests and increasing the rate at which students graduate from high school. Federal regulations adopted in 2008 to implement No Child Left Behind require that each state establish annual graduation rate growth targets that reflect “continuous and substantial improvement.” The federal government requires that states subject schools that fail to meet AYP targets 8 California State Auditor Report 2011-117 March 2012 to increasingly severe consequences, which range from placing certain schools in Program Improvement status (which requires focused attention on fixing the problems) to subjecting them to state takeover. Thus, there is a strong impetus for California to have an effective means of tracking and reporting this information to the U.S. Department of Education. In response to the requirements of No Child Left Behind, the department issues AYP reports each year for all California public schools, school districts, and counties, and on a statewide basis. These reports reflect performance on a variety of metrics, including test scores, year-to-year improvement on California’s API metric, and high school graduation rates. The department also makes its AYP reports available on its Web site. Graduation and Dropout Rate Calculation Methods The department now uses data contained in its recently developed statewide education database, the California Longitudinal Pupil Achievement Data System (CALPADS), to measure statewide high school graduation and dropout rates. (We describe the development of CALPADS in more detail beginning on page 10.) Federal regulations generally require states to calculate the graduation rate according to a formula referred to as the four‑year cohort rate. California used this formula beginning with the graduating high school class of 2009–10. The department calculates the four-year cohort rate by dividing the number of students who graduate in four years or less with a regular high school diploma by the number of students who form the adjusted cohort for the graduating class. To determine the adjusted cohort over the four-year period, the department adjusts the number of students who enter the ninth grade by adding students who transfer into the cohort and subtracting students who transfer out, emigrate to another country, or die. In addition, state law requires the department to determine the dropout rate over four years by dividing the number of students who drop out by the number who remain in the adjusted cohort. Figure 3 visually depicts an example of the department’s calculation of graduation and dropout rates. It is worth noting that the number of students who drop out is not simply the difference between the number of students who graduate and the total number of students who remain in the adjusted cohort. Instead, the definitions of both dropout and graduate exclude students who pass the General Educational Development Test®, students who do not graduate at the end of four years but remain enrolled, and some special education students, as described in the following paragraph. The dropout rate includes students who California State Auditor Report 2011-117 9 March 2012 have permanently stopped attending school, as well as students who left school after attending four years of high school but did not meet graduation requirements. Figure 3 Example of Four‑Year Cohort Graduation and Dropout Rate Calculations Inbound Transfers STUDENT ACTION GRADUATES DROPOUTS COHORT Graduate Begin Ninth Grade +200 + 0 +200 Inbound Transfers +10 + 0 +10 Outbound Transfers – 11 – 0 – 11 Students Begin Dropout Stop Attending (Dropout) – 5 + 5 – 0 Ninth Grade Do Not Reenroll After Transfer (Dropout) – 1 + 1 – 0 Pass GED* – 2 – 0 – 0 Other Complete Special Education Certificate† – 4 – 0 – 0 Remain in School for Fifth Year – 8 – 0 – 0 Outbound Do Not Meet Graduation Requirements (Dropout) – 2 + 2 – 0 Transfers‡ Indicates the adjusted cohort. Totals 177 8 199 Graduation Rate Calculation Graduation Rate Percentage = Graduates 177 = 89% Cohort 199 Dropout Rate Calculation Droupout Rate Percentage = Dropouts 8 = 4% Cohort 199 Sources: California State Auditor’s analysis of state and federal laws and regulations, as well as documentation provided by the California Department of Education. * General Educational Development Test ® † These students did not graduate with a standard diploma. ‡ Students who are removed from the cohort also include those that emigrate to another country or die during the four‑year period. California students are generally required to pass the exit examination as a condition of graduating from high school. However, changes to state law that took effect beginning with the 2009–10 school year allow special education students who have not passed the exit examination but otherwise satisfy high school graduation requirements to be treated as graduates. Thus, some special education students—namely, those who were exempted from that requirement as of the 2009–10 school year—are treated as graduates in the first four-year cohort graduation rate that California reported. Prior to that, special education students who did not pass the exit examination could not earn a high school diploma unless they had obtained a waiver from that requirement from their local school board, and these waivers were granted only under specific, limited circumstances. A special education student who has satisfied the exit examination requirement by passing 10 California State Auditor Report 2011-117 March 2012 the examination, or by receiving either an exemption or a waiver and has met all other state and local graduation requirements, is considered a graduate. In our report we focused our attention on the four-year cohort rates based on the audit objectives (see Table 1) on page 13. However, students may sometimes complete high school in five or six years, rather than in four years. According to an education research and evaluation consultant within the department’s assessment and accountability division, the department is currently examining whether to use five- and six-year graduation rates, as allowed by federal The Legislative Intent Behind the California regulations, to demonstrate that schools are Longitudinal Pupil Achievement Data System meeting their AYP goals related to high school According to state law, the Legislature authorized California graduation. These rates are also factored into a Longitudinal Pupil Achievement Data System (CALPADS) school’s or district’s API score. In addition, some with the intention that the system would: students may drop out in middle school rather than in high school, and the department includes • Provide school districts and the California Department an eighth grade dropout rate in a middle school’s of Education (department) with access to the data necessary to comply with federal reporting requirements. or district’s API. • Provide a better means of evaluating educational progress and investments over time. CALPADS • Provide school districts with information they can use to improve pupil achievement. Recognizing that California needed longitudinal • Provide an efficient, flexible, and secure means of student data both to comply with No Child maintaining longitudinal statewide pupil‑level data. Left Behind and to assess the long-term value of its educational investments and programs, • Facilitate the ability of the State to publicly report data in the Legislature authorized the department to accordance with federal requirements. develop and implement CALPADS in 2002, as • Ensure that the department provides data access indicated in Figure 4. The text box further details to researchers only to the extent allowed by federal the Legislature’s goals in authorizing CALPADS. privacy laws. As we will discuss in the Audit Results, the most • Provide the ability to sort data by demographic element significant difference between CALPADS and collected from certain statewide standardized tests. CBEDS is that CALPADS collects information about individual students rather than relying on • Have the capacity to be expanded to include pupil aggregate information. This capability is a key achievement data from multiple years. component in satisfying the requirements of • Have the capacity to monitor pupil achievement on No Child Left Behind. CALPADS accomplishes certain statewide standardized tests from year to year this task by assigning all students unique and school to school. identifying numbers that ensure students are • Have the capacity to provide data to the State and school consistently tracked even if they transfer from districts upon their request. one California public school to another. School Source: California Education Code, sections 60900–60901. districts extract data from existing records contained in their student information systems and submit this data electronically to CALPADS. California State Auditor Report 2011-117 11 March 2012 Figure 4 Development and Implementation of the California Longitudinal Pupil Achievement Data System June June Start of 2006–07 school year School districts are required The Public Schools Accountability Act of 1999 takes effect. First cohort of students start to certify exit data for cohort. ninth grade. August January California Department of The No Child Left Behind Act of 2001 takes effect. End of 2009–10 school year Education (department) First cohort of students graduate. publishes first four-year cohort graduation and dropout rates. 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 June January CALPADS to begin International Business collecting student Machines Corporation (IBM) grades and credits, starts CALPADS development. October as well as additional CALPADS begins collecting student enrollment program participation data and exit records, as well as tracking information such as participation in some programs such as Title I services. special education. March October CALPADS to begin CALPADS begins collecting students’ course collecting students’ information and teachers’ assignments. standarized test scores. February Department puts IBM on notice for breach of contract. stnevE rojaM tnempoleveD SDAPLAC noitatnemelpmI dna September Legislature authorizes and the governor approves development and implementation of the California Longitudinal Pupil Achievement Data System (CALPADS). Sources: The Little Hoover Commission, the department, and federal and state laws. Initially, the development of CALPADS did not meet all expectations, raising concerns in the department, some districts, and the governor’s office that the system would not adequately meet the State’s needs. In 2008 the department began a contract to pay International Business Machines Corporation (IBM) about $15.3 million to build CALPADS. Approximately $3 million for this project came from a federal grant that provided funding for states’ development of longitudinal databases for education. Users and outside evaluators noted that early versions of the system—including the enrollment data component used to calculate graduation and dropout rates—appeared to work poorly, leading to doubts that CALPADS would ever operate effectively. Consequently, in February 2011 the department informed IBM that it was in breach of contract for failure to perform. An independent evaluator’s report released around the same time noted significant problems with CALPADS’ stability and performance.1 However, department officials state that IBM has since assigned better resources to CALPADS and is working to take various steps to address the concerns. The department believes that by the end of June 2012 the system will be fully built. Once completed, in addition 1 Sabot Technologies, a consulting firm, assessed CALPADS at the department’s request. 12 California State Auditor Report 2011-117 March 2012 to enrollment information, CALPADS will be able to store students’ standardized test scores, courses completed, and grades. State law also specifies that CALPADS should include attendance data when the department has the necessary federal funding to build that functionality into the system; however, the federal government has not yet made this funding available. How schools would use or report this attendance data is also unclear. According to the director of the department’s educational data management division, each school district and county office of education currently records individual student attendance in its student information systems and reports aggregate data to the state superintendent. After IBM finishes building the system, under the direction of the department, CALPADS will be operated by California School Information Services (CSIS), which is part of the Fiscal Crisis and Management Assistance Team, an agency administered by the Kern County Office of Education. CSIS already provides support to school districts across the State, including assisting them in efficiently reporting data to the department. Scope and Methodology We conducted this audit at the direction of the Joint Legislative Audit Committee, which approved the audit objectives listed in Table 1. Our fieldwork included work at the department and site visits to Delano Joint Union High School District, San Francisco Unified School District, Los Angeles Unified School District, Oroville Union High School District, Long Beach Unified School District, and San Diego Unified School District. Assessment of Data Reliability In performing this audit, we relied upon various electronic data files extracted from the information systems listed in Table 2 on page 15. We adhere to the standards of the U.S. Government Accountability Office, which require us to assess the sufficiency and appropriateness of computer-processed information. The table shows the results of this analysis. California State Auditor Report 2011-117 13 March 2012 1 elbaT sevitcejbO tiduA sserddA ot sdohteM DOHTEM EVITCEJBO TIDUA ,)edoc noitacude( edoC noitacudE ainrofilaC eht fo snoitces tnaveler deweiveR ,)dniheB tfeL dlihC oN( 1002 fo tcA dniheB tfeL dlihC oN eht sa hcus swal laredef .sevitcejbo tidua eht ot tnacfiingis snoitaluger dna ,selur ,swal eht etaulave dna weiveR 1 .snoitaluger dna swal rehto dna :sesylana gniwollof eht mrofrep ,stcirtsid loohcs egral dna llams fo elpmas a gnisU loohcS ainrofilaC ,rotcartnoc sti dna tnemtraped eht ta slaicffio deweivretnI • .secivreS noitamrofnI )tnemtraped( noitacudE fo tnemtrapeD ainrofilaC eht ecnatsissa dna ecnadiug tahw enimreteD enilno eht no dehsilbup esoht gnidulcni ,slairetam tnenitrep eht deweiveR • eht gnisu gnitroper dna noitcelloc atad tuopord gnidrager stcirtsid loohcs eht ot dedivorp a2 .latrop SDAPLAC ataD tnemeveihcA lipuP lanidutignoL ainrofilaC eht dna metsyS ataD lanoitacudE cisaB ainrofilaC stcirtsid loohcs eht ta slaicffio tnaveler htiw ecnadiug s’tnemtraped eht dessucsiD • .)SDAPLAC( metsyS .detisiv ew sloohcs hgih ta ,seiraterces ecnadnetta sa hcus ,ffats dna slaicffio tnaveler deweivretnI troper dna tcelloc ot desu stcirtsid loohcs eht serudecorp dna seicilop eht etaulave dna weiveR b2 .dedivorp yeht noitatnemucod yna deweiver dna stcirtsid loohcs xis ni detisiv ew .tnemtraped eht ot atad tuopord ”.tuopord“ fo noitinfied tnetsisnoc a deilppa stcirtsid loohcs eht rehtehw enimreteD c2 lanoitacude s’tnemtraped eht htiw ekops dna noitatnemucod ’SDAPLAC deweiveR sa stneduts gniyfissalc rof elbisnopser saw ,yna fi ,level lacol eht ta ytitne hcihw enimreteD tneduts niatrec sezirogetac tnemtraped eht woh enimreted ot ecffio scihpargomed rehtehw enimreted ,noitidda nI .noitacfiissalc siht edam ytitne eht woh enimreted dna stuopord d2 .stuopord sa sedoc tixe .stuopord sa stneduts fo noitacfiissalc reporp eht erusne dna rotinom ot tsixe slortnoc tixe dengissa yeht woh enimreted ot tcirtsid hcae ta sdrocer tneduts 02 denimaxE stcirtsid loohcs eht noitamrofni noitaudarg dna tuopord eht hcihw ot tnetxe eht enimreteD .snoitcurtsni s’tnemtraped eht ot demrofnoc yeht rehtehw dna sedoc :gniwollof eht tnuocca otni sekat tnemtraped eht ot detroper ,oslA .raey loohcs eht gnirud rehtona ot tcirtsid loohcs eno morf derrefsnart ohw stnedutS )i( .refsnart eht demrfinoc tcirtsid loohcs eht woh dna rehtehw enimreted enimreted ,oslA .smargorp noitacude evitanretla otni derrefsnart evah yam ohw stnedutS )ii( e2 .refsnart eht demrfinoc tcirtsid loohcs eht woh dna rehtehw yeht taht loohcs eht mrofni yllamrof ton did tub loohcs dnetta ton did ohw stnedutS )iii( eht demrfinoc tcirtsid loohcs eht woh dna rehtehw enimreted ,oslA .tuo deppord dah .sutats ’stneduts .evitcejbo siht mrofrep ot elbaliava ton erew atad yrassecen ehT eht no egaugnal dna ,erutluc ,ecar fo tcapmi eht ssessa ,elbaliava era atad tnetxe eht oT f2 .atad tuopord dna refsnart fo gnidoc dna noitcelloc gnidivorp sloohcs ot detaler stnemeriuqer s’edoc noitacude eht deweiveR • ,sdraob loohcs ,stnerap htiw setar tuopord rieht derahs stcirtsid loohcs rehtehw enimreteD .draC tropeR ytilibatnuoccA loohcS eht fo yaw yb noitamrofni ,.g.e( noitamrofni siht derahs yeht hcihw ni rennam eht enimreted ,os fI .cilbup lareneg eht dna g2 .noitamrofni detubirtsid stcirtsid eht woh weiver ot lennosrep tcirtsid htiw ekopS • .derahs yeht noitamrofni eht fo ecruos eht dna ).cte ,sgniliam ,setis beW ,sgniteem cilbup SDAPLAC ot atad gnittimsnart rof elbisnopser slaicffio tcirtsid loohcs deweivretnI ot dettimmoc secruoser dna seitilibapac etairporppa evah sloohcs rehtehw enimreteD h2 .slaicffio tnemtraped deweivretni dna .gnitroper dna noitcelloc atad tuopord dna noitaudarg .srotartsinimda tcirtsid loohcs dna srotacude deweivretnI • yeht spets tahw dna smargorp noitneverp tuopord reffo stcirtsid loohcs eht rehtehw enimreteD .elbaliava fi ,slairetam margorp noitneverp tuopord deweiveR • i2 .smargorp eseht ni ytinummoc eht dna ,stneduts ,stnerap ,srehcaet egagne ot ekat .tnemtraped eht ta slaicffio htiw ekopS • . . . egap txen no deunitnoc 14 California State Auditor Report 2011-117 March 2012 DOHTEM EVITCEJBO TIDUA dlihC oN sa hcus swal laredef ,edoc noitacude eht fo snoitces tnaveler deweiveR • .snoitaluger dna swal rehto dna ,dniheB tfeL sesoprup suoirav rof sreffid etar tuopord loohcs eht fo noitaluclac s’etatS eht rehtehw enimreteD noitaluclac etar tuopord a eriuqer ton seod dniheB tfeL dlihC oN taht denimreteD • oN dna )tca ytilibatnuocca etats( 9991 fo tcA ytilibatnuoccA sloohcS cilbuP s’etatS eht sa hcus 3 tnemtraped eht taht denimreteD .noitaluclac etar noitaudarg a eriuqer seod tub .reffid snoitaluclac eht woh ,os fi dna ,dniheB tfeL dlihC dniheB tfeL dlihC oN htob steem taht alumrof a gnisu setar noitaudarg setaluclac .tca ytilibatnuocca etats eht dna lanoitacude s’tnemtraped eht yb detareneg atad tuopord dna noitaudarg deweiveR neewteb yrav setar noitaudarg dna tuopord woh enimreted ,elbaliava era atad tnetxe eht oT .S.U eht morf sedoc noitacol ot sloohcs laudividni deknil taht ecffio scihpargomed 4 .stcirtsid loohcs larur dna ,nabrubus ,nabru .uaeruB susneC etis beW sti no dehsilbup tnemtraped eht atad tuopord dna noitaudarg deweiveR • gnoma yrav setar noitaudarg dna tuopord woh enimreted ,elbaliava era atad tnetxe eht oT vog.ac.edc.1atad ta hsilgnE ,level emocni dlohesuoh ,yticinhte ,redneg ,.g.e( seirogetac cihpargomed tnereffid 5 .SDAPLAC morf atad htiw sisylana lanoitidda demrofreP • .)secivres noitacude laiceps ,renrael egaugnaL eht( secivreS gnitseT lanoitacudE morf dna SDAPLAC morf atad dezylanA • noitanimaxE tixE loohcS hgiH ainrofilaC eht fo tcapmi eht ssessa ,elbaliava era atad tnetxe eht oT .)noitanimaxe tixe eht fo rotartsinimda pihsnoitaler eht ssessa ,elbissop tnetxe eht ot ,yllacfiicepS .setar tuopord no )noitanimaxe tixe( 6 fo tcapmi eht tuoba slaicffio rehto dna srolesnuoc loohcs hgih deweivretnI • .loohcs fo tuo gnippord dna noitanimaxe tixe loohcs hgih eht gniliaf neewteb .noitanimaxe eht ot stcirtsid rof segnellahc detaerc SDAPLAC hcihw ni syaw denimaxE • .noitamrofni troper rieht gnidrager setis beW ’setats rehto niatrec no noitamrofni deweiveR • fo gnitroper dna noitcelloc eht ot tnacfiingis era taht seussi rehto yna ssessa dna weiveR 7 enimreted ot sredlohekats fo yteirav a htiw ekops dna smetsys atad lanidutignol .atad tuopord loohcs ,srehcraeser rof ytilitu ’SDAPLAC evorpmi dluoc etatS eht hcihw ni syaw .srekam ycilop dna ,srotartsinimda noitatnemucod dna noitamrofni fo sisylana dna ,stnemucod gnipocs dna gninnalp s’rotiduA etatS ainrofilaC ,sevitcejbo tidua rof 711‑1102# tseuqer tidua eettimmoC tiduA evitalsigeL tnioJ :secruoS .dohteM deltit nmuloc elbat eht ni defiitnedi California State Auditor Report 2011-117 15 March 2012 Table 2 Methods to Assess Data Reliability INFORMATION SYSTEM PURPOSE METHOD AND RESULT CONCLUSION California Longitudinal Pupil To calculate dropout and • We performed data-set verification procedures and Undetermined Achievement Data System, graduation rates among electronic testing of key data elements and did not reliability for California Department of various demographic identify any issues. the purpose of Education (department) categories and by • We did not perform accuracy and completeness testing this audit. Data as of August 2011 California High School because the source documentation for this system is not Exit Examination (exit centrally stored at the department, thereby making such examination) result for the testing impractical. 2009–10 cohort. • Alternatively, we examined 20 student records from each of the six school districts we visited to determine how schools assigned exit codes and whether they conformed to the department’s instructions. See page 28. Department’s California To identify students’ • We performed data-set verification procedures and Undetermined High School Exit performance on their electronic testing of key data elements and did not reliability for Examination results first attempt at the exit identify any issues. the purpose of Data for the 2007–08 school year examination during the • We were unable to perform accuracy and completeness this audit. second half of grade 10. testing as required by government auditing standards The department contracted with because ETS’ subcontractor previously destroyed the Educational Testing Service (ETS) hard‑copy student answer documents in accordance with to provide for the development, the terms of its contract. administration, scoring, analysis, • Alternatively, we interviewed officials at the department and reporting of the exit and reviewed relevant documentation to determine examination. ETS subcontracted whether it conducted any work to verify that student for the design, printing, and essays were accurately scored. We learned that the scanning of hard‑copy student department contracted with Human Resources Research answer documents. Organization (HumRRO) to conduct an independent evaluation of the exit examination test results through September 2008. As part of its evaluation, HumRRO analyzed the consistency with which student essays were scored for the 2007–08 school year and found that the results were generally comparable to the 2006–07 school year and somewhat improved in comparison to previous years. • We also interviewed officials at the department, ETS, and ETS’ subcontractor and reviewed relevant documentation to determine whether they conducted any work to verify the accuracy of the electronic file of student multiple‑choice answers against the hard‑copy answer documents. Although ETS asserted to us that it performed quality control measures related to the accuracy of the data, it was not able to produce the detailed results of this analysis. We found that the level of detail ETS provides to the department when reporting the results of its quality control measures could be improved. This was communicated to the department in a separate management letter. Sources: Various documents and data collected from the department and its contractors. 16 California State Auditor Report 2011-117 March 2012 Terminology Related to Racial and Ethnic Groups, Socioeconomic Status, and English Language Proficiency Our data analysis grouped students by racial and ethnic subgroups based on how they self-identified when enrolling in school. We generally used the terminology the department included in the student records in CALPADS. As members of an ethnic group, Hispanic or Latino students may belong to any race. However, for clarity, no racial and ethnic subgroups referenced in our statistical discussions include students from Hispanic or Latino backgrounds, except for the Hispanic or Latino subgroup itself. In addition, we counted students who self-identified with more than one racial background as “two or more” and did not include them in the other racial and ethnic subgroups; such students comprise only about 1 percent of the 2009–10 cohort. About 1.6 percent of the cohort did not report their racial or ethnic background; we identify these students as “not reported” in our analysis. Additionally, our data analysis identified whether students were socioeconomically disadvantaged and whether they were English Language learners (English learners). CALPADS recorded students as being socioeconomically disadvantaged if they were eligible for a free or reduced-price lunch program, also known as the National School Lunch Program, or if neither parent had received a high school diploma. CALPADS collects several data elements related to English learners, including their English language acquisition status. A student’s status as an English learner may end during a student’s time in high school if the student is reclassified as fluent English proficient. However, for the tables and data in this report, we included every student who was designated as an English learner at any time during grades nine through 12. California State Auditor Report 2011-117 17 March 2012 Audit Results The California Longitudinal Pupil Achievement Data System Appears to Measure Graduation and Dropout Rates Effectively In our review, we found that the California Longitudinal Pupil Achievement Data System (CALPADS) generally appears to be a useful tool for measuring graduation and dropout rates and that it should allow policy makers and educators to effectively identify groups of students with the highest dropout rates. Because CALPADS collects information about individual students and contains codes indicating the reasons each student leaves high school, the California Department of Education (department) can use it to determine dropout and graduation rates for individual high schools and districts, as well as for the State as a whole, in a way that meets requirements of state and federal law. Unlike data from the California Basic Educational Data System (CBEDS), which did not include student identifiers of any kind, CALPADS’ data are longitudinal, meaning that the department can track individual student data from year to year. Further, because CALPADS avoids the aggregate reporting on which CBEDS depended, the precision in CALPADS data collection allows the department to more accurately calculate graduation and dropout rates. To determine graduation and dropout rates using CALPADS, the department uses the formula described in the Introduction. This formula determines the four-year cohort rates, which identify the percentages of students who graduate, drop out, or have some other outcome four years after they started high school. This formula also properly accounts for students who transferred to another public school or district, something that calculations using CBEDS data could not do. In August 2011 the department calculated and published the initial four-year cohort rates for the class of 2009–10, which we refer to as the first cohort. To ensure that schools and districts throughout the State utilize the same definition of dropout or graduate, the department established within CALPADS a set of student exit reason codes (exit codes) from which schools or districts can choose when identifying the reasons students leave school. Generally, when students leave school, school staff enter exit dates and exit codes in their school district’s student information system. The district then extracts this 18 California State Auditor Report 2011-117 March 2012 Reasons a Student Can Be Coded as a information and submits it to CALPADS.2 Depending on Dropout in the California Longitudinal Pupil Achievement Data System the assigned exit code, the department may classify individual students as graduates or dropouts for that high The California Department of Education caterogizes a student school or district, or, in some cases, it may remove them as a dropout when a school or district determines that one of from the cohort. As a result, the department’s ability to the following situations applies to that student and the accurately calculate graduation and dropout rates is California Longitudinal Pupil Achievement Data System dependent on how uniformly schools and districts use the (CALPADS) contains no subsequent enrollment for the exit codes. The exit codes in CALPADS allow the student for the same reporting year: department to employ a very precise definition of dropout. • The student withdrew from or left school, and no The text box explains the scenarios in which the evidence exists that he or she is in an academic program department considers students to be dropouts. leading toward a diploma or its equivalent. This includes students who leave school for jobs, marriage, etc. Because CALPADS assigns each student a unique • The student left school after being expelled and was identifier, a key function of the system is its ability to subsequently referred to another educational service track individual students, thereby leading to dropout rate institution but never showed up. The school’s or district’s calculations that are more accurate than those reported attempts to locate the student were unsuccessful. previously through CBEDS. For instance, according to • The student left school for unknown reasons or reasons the CALPADS coordinator at Long Beach Unified School not listed in any of the other codes. District (Long Beach), if a student claims to be transferring to another school district but CALPADS does not show • The student withdrew from or left school to enroll in an adult education program in order to obtain a General Education his or her subsequent enrollment in any public school Development Test ® certificate or high school diploma but district, the department counts the student as a dropout. subsequently dropped out of the adult education program. The coordinator explained that in the past, when school districts were reporting the number of dropouts in CBEDS, • The student withdrew from or left school to enter an these students would have been counted erroneously as institution that is not primarily academic (military, job corps, justice system, etc.), and the student is not in a transfers and not dropouts because the district lacked secondary program leading toward a high school diploma. the capability of verifying student enrollment in another public school district. He stated that, for this reason, the • The student completed an academic year at a school but dropout rates reported previously in CBEDS might have did not return to the same school the following year as been underestimated. expected, and no other exit code is appropriate. • The student withdrew from or left school for medical reasons. Another significant improvement over the previous • The student completed all local and state graduation system is a built-in verification process, known as anomaly requirements but failed the California High School resolution, that allows CALPADS to detect discrepancies Exit Examination (non-special education student). in student enrollment. Districts are required to reduce • The student completed grade 12 without completing the rate of certain anomalies to less than 2 percent graduation requirements. before they certify their enrollment data. Several types of anomalies are identified by CALPADS and displayed to • The student withdrew from or left for nondisciplinary reasons and transferred to another public school (within districts, including a student being enrolled in more than or outside the district) in California, and CALPADS one school simultaneously and one student being assigned contains no subsequent enrollment record for the student for the same reporting year. • The student withdrew from one school for disciplinary 2 Although districts may submit data to CALPADS on an ongoing basis, CALPADS reasons and transferred to another public school in collects snapshots of data, such as enrollment, on specific dates. It then uses these snapshots to create reports for districts to review and certify by specified California (within or outside the district), and CALPADS deadlines. In particular, CALPADS takes snapshots of districts’ enrollment, graduate, contains no subsequent enrollment for the student for and dropout data on its fall census day, the first Wednesday in October. By the same reporting year. mid‑December, districts must review and certify this data; however, districts may amend their data, if need be, until mid‑January. Once the districts certify the data, Source: California Department of Education. the department calculates the graduation and dropout rates for the State, as well as for each school and district. California State Auditor Report 2011-117 19 March 2012 more than one identification number. According to the administrator of the department’s educational demographics office, most of these anomalies arise because districts enter inconsistent information into CALPADS and, as a result, they need to coordinate with each other to resolve the anomalies. He explained that once the data are certified, the department takes into account discrepancies in student exit codes— such as when a student is coded as a transfer but is never subsequently enrolled in another California public school— adjusts the dropout counts accordingly, and then proceeds to calculate dropout and graduation rates for the State, districts, and schools. The process of anomaly resolution is an effective way to address some of the errors that would otherwise affect reported dropout and graduation rates. For example, CALPADS enables districts to review lists of students who may have had more than one identifier assigned to them and, where appropriate, to resolve the issue by selecting a single correct identifier to assign to each student. Uncorrected, this error may result in a lost transfer, which occurs when CALPADS loses track of a student during a transfer because he or she is assigned a new identifier upon arrival at a new school. Such errors increase the dropout count for the school the student transferred from. CALPADS can also identify whether a reported exit reason in a student’s record conflicts with the information that a district subsequently reported in the system, an anomaly that the department can use directly to modify the dropout and graduation rates. For instance, if one school reported that a student left school for no known reason (which would result in the student being counted as a dropout) but then the student subsequently enrolled in a different California public school, the department would adjust the dropout count so that the student was no longer counted as a dropout. CALPADS’ Data Confirm That Graduation and Dropout Rates Vary Among Different Demographic Subgroups As discussed, in August 2011 the department published graduation and dropout data from CALPADS for one four-year cohort, the class of 2009–10. The data related to this cohort suggest that graduation and dropout rates differ for certain demographic subgroups. Because the data are limited to this Because the dropout data from one cohort, the department, districts, and schools cannot yet CALPADS are limited to this use them to identify trends over time. However, the data do four‑year cohort, the department, illustrate the quality and variety of information that CALPADS districts, and schools cannot yet use can provide to educators and policy makers. In particular, the them to identify trends over time. department, districts, and schools can use these four-year 20 California State Auditor Report 2011-117 March 2012 graduation rates as the baseline against which they can compare future four-year graduation rates in order to develop strategies to demonstrate Adequate Yearly Progress as required by the No Child Left Behind Act of 2001, which we discuss in the Introduction. According to its 2011 Adequate Yearly Progress Report Information Guide, the State’s goal is to achieve a graduation rate of 90 percent. The State will require all schools and school districts with grade 12 students to meet this goal by 2019. The department’s First Annual Report on Dropouts in California, which it released in August 2011, reported that the State’s graduation rate for the 2009–10 cohort of students was 74.4 percent, and the dropout rate for this same cohort was 18.2 percent. The remaining 7.4 percent of students in the cohort were neither graduates nor dropouts; they included students who were still enrolled in high school after the fourth year, students who had left school but completed a General Educational Development Test®, and special education students who did not receive diplomas. Figure 5 presents the various graduation and dropout rates, sorted by race and ethnicity, gender, socioeconomic status, and program status, such as migrant education. It shows that dropout rates are higher in certain racial and ethnic groups. For example, the dropout rate reported by the department for African-American students was 30.1 percent; for Hispanic or Latino students, it was 22.7 percent. Socioeconomically disadvantaged We found that socioeconomically disadvantaged students were students were in general more likely in general more likely to drop out of high school than their peers. to drop out of high school than their In the 2009–10 cohort, the department reported that 21.8 percent peers—in the 2009–10 cohort, the of socioeconomically disadvantaged students dropped out, department reported 21.8 percent of compared to 12.9 percent of other students. However, when we socioeconomically disadvantaged disaggregated the data, we discovered that the degree to which students dropped out compared to socioeconomic status affected the likelihood that students might 12.9 percent of other students. drop out varied significantly among certain racial and ethnic subgroups. As illustrated in Table 3 on page 22, socioeconomically disadvantaged white and Asian students had higher dropout rates than white and Asian students who were not socioeconomically disadvantaged. However, socioeconomic factors did not appear to affect the likelihood that African-American and Hispanic or Latino students would drop out.3 3 The data shows that socioeconomically disadvantaged Hispanic or Latino students dropped out slightly less frequently than other Hispanic or Latino students. California State Auditor Report 2011-117 21 March 2012 Figure 5 Graduation and Dropout Rates by Race/Ethnicity, Gender, and Program Participation in the 2009–10 Cohort Graduation rate Dropout rate Students who were neither graduates nor dropouts* 0% 20% 40% 60% 80% 100% yticinhtE/ecaR redneG margorP Statewide Asian Filipino Two or more races White Pacific Islander Not reported Hispanic or Latino American Indian or Alaska Native African-American Female Male Migrant education Socioeconomically disadvantaged Special education English Language learners Source: August 2011 data from the California Longitudinal Pupil Achievement Data System, obtained from the California Department of Education’s DataQuest Web site. Note: In the Scope and Methodology section on page 16, we discuss our use of racial and ethnic terminology and our process for defining program participation. * Includes students who were currently enrolled, non‑diploma special education students, and students who passed the General Educational Development Test®. 22 California State Auditor Report 2011-117 March 2012 Table 3 Dropout Rates by Socioeconomic Status for Race and Ethnicity Subgroups in the 2009–10 Cohort STUDENTS WHO WERE NOT STUDENTS WHO WERE SOCIOECONOMICALLY SOCIOECONOMICALLY ALL STUDENTS WHO DISADVANTAGED AND DISADVANTAGED AND DROPPED OUT DROPPED OUT DROPPED OUT RACE OR ETHNICITY PROPORTION NUMBER PROPORTION NUMBER PROPORTION NUMBER Asian 7.7% 3,522 5.8% 1,358 9.7% 2,164 African‑American 30.1 12,976 28.1 3,550 30.9 9,426 Not reported 23.3 1,958 21.3 970 25.7 988 Filipino 8.4 1,159 7.5 618 9.8 541 Hispanic or Latino 22.7 54,033 23.6 11,124 22.4 42,909 American Indian or Alaska Native 23.8 1,061 18.1 347 28.1 714 Pacific Islander 20.9 729 19.3 272 22.0 457 Two or more races 10.7 573 6.3 213 18.1 360 White 11.7 18,301 8.1 8,866 20.0 9,435 Totals 18.2% 94,312 12.9% 27,318 21.8% 66,994 Source: California State Auditor’s analysis of data obtained from the California Department of Education’s California Longitudinal Pupil Achievement Data System as of August 2011. Note: In the Scope and Methodology section on page 16, we discuss our use of racial and ethnic terminology and our process for defining socioeconomic status. Also, as illustrated in Table 4, the dropout rate for English Language learners (English learners) was higher than for those students who were not designated as English learners. Because California law generally requires classes to be taught in English, lack of proficiency is a predictable barrier to academic success. The dropout rate for all English learners was 31.1 percent. Our analysis of the data indicated the dropout rate for English learners in the Hispanic or Latino subgroup was about 16 percentage points higher than the dropout rate for students in the Hispanic or Latino subgroup not designated as English learners. The dropout rate for English learners in the Asian subgroup was about 14 percentage points higher than the dropout rate for students in the Asian subgroup not designated as English learners. However, the dropout rate in the African-American subgroup did not vary all that much when considering English learner status. The data also show that schools in areas with very high or very low population densities generally had higher dropout rates than others. As illustrated in Figure 6, students attending high schools in cities dropped out more frequently than students in other locales. Students attending schools in rural areas were also slightly more likely to drop out. California State Auditor Report 2011-117 23 March 2012 Table 4 Dropout Rates by English Language Learner Status for Race and Ethnicity Subgroups in the 2009–10 Cohort STUDENTS WHO WERE NOT STUDENTS WHO WERE ENGLISH LANGUAGE ENGLISH LANGUAGE ALL STUDENTS WHO LEARNERS AND LEARNERS AND DROPPED OUT DROPPED OUT DROPPED OUT RACE OR ETHNICITY PROPORTION NUMBER PROPORTION NUMBER PROPORTION NUMBER Asian 7.7% 3,522 4.7% 1,668 18.3% 1,854 African‑American 30.1 12,976 30.0 12,689 30.7 287 Not reported 23.3 1,958 22.5 1,686 30.1 272 Filipino 8.4 1,159 7.2 868 16.8 291 Hispanic or Latino 22.7 54,033 17.3 27,771 33.5 26,262 American Indian or Alaska Native 23.8 1,061 23.2 1,001 42.3 60 Pacific Islander 20.9 729 19.4 581 30.1 148 Two or more races 10.7 573 10.5 549 18.6 24 White 11.7 18,301 11.5 17,552 20.8 749 Totals 18.2% 94,312 15.2% 64,365 31.1% 29,947 Source: California State Auditor’s analysis of data obtained from the California Department of Education’s California Longitudinal Pupil Achievement Data System as of August 2011. Note: In the Scope and Methodology section on page 16, we discuss our use of racial and ethnic terminology and our process for defining English Language learners. Figure 6 Graduation and Dropout Rates by Locale for the 2009–10 Cohort Graduates Dropouts Other outcomes* City Suburb Town Rural Area 0% 20% 40% 60% 80% 100% Source: The California Department of Education’s Educational Demographics Office, based on data from its California Longitudinal Pupil Achievement Data System compared to schools’ U.S. Census locale codes provided by the National Center for Education Statistics. * Includes students who were currently enrolled, non‑diploma special education students, and students who passed the General Educational Development Test®. In addition to completing all mandatory coursework and fulfilling other local graduation requirements, most public high school students must pass the California High School Exit Examination (exit examination) to receive a high school diploma. 24 California State Auditor Report 2011-117 March 2012 The examination contains two portions—an English-language arts assessment and a mathematics assessment—and most students must pass both sections to earn a high school diploma. The first opportunity students have to take the exit examination is in the second half of grade 10, although students who do not pass one or both parts of the exit examination in grade 10 can retake the part or parts they did not pass in grades 11 and 12, and in some adult education programs. Our review of the 2009–10 cohort’s test results shows that those students who passed the exit examination on their first attempt were much more likely to graduate within four years than those who did not. As illustrated in Figure 7, students who passed both the English-language arts and mathematics portions of the exit examination on their first attempt were much more likely to graduate from high school within four years than students who passed only one portion of the examination and failed the other on their first attempt. Similarly, students who passed only one portion and failed the other on their first attempt were more likely to graduate within four years than students who failed both portions of the examination. Figure 7 The Effect of Students’ California High School Exit Examination Results on Their First Attempt on Graduation and Dropout Rates for the 2009–10 Cohort Graduates Dropouts Other outcomes* Passed both portions (310,012 students) Passed one portion and failed the other (51,314 students) Failed both portions (45,293 students) All other results† (83,662 students) Student record not found‡ (28,996 students) 0% 20% 40% 60% 80% 100% Sources: California State Auditor’s analysis of data obtained from the California Department of Education’s California Longitudinal Pupil Achievement Data System as of August 2011 and the California High School Exit Examination (exit examination) for the 2007–08 school year. * This category includes students who were currently enrolled, or non‑diploma special education students, or students who passed the General Educational Development Test®. † This category includes students who had not yet entered the California school system as of the final exit examination administration date for the 2007–08 school year during the second half of grade 10, students who were absent on test day, students whose scores were invalidated, and students who attempted one section of the exit examination but not the other. ‡ Although these students were enrolled in the California school system at some point during the second half of grade 10, we found no record of the students attempting the exit examination during that time. California State Auditor Report 2011-117 25 March 2012 Despite the above-named result, the exit examination may not represent a substantial barrier to graduation for most students because the large majority eventually passed the examination. According to the department’s First Annual Report on Dropouts in California, CALPADS and other data suggest that the overall passage rate on the exit examination for the 2009–10 cohort was 94.4 percent by the end of the students’ fourth year of high school. According to the exit codes in CALPADS, failure to pass the examination precluded 4,175 students from receiving their diplomas, as illustrated in Figure 8. This number may shed only partial light on the impact of the exit examination on dropout rates, however, because we cannot determine if failing to pass the examination had other consequences for particular students. For example, students who dropped out without completing other graduation requirements may have also been motivated to drop out because of their poor performance on the exit examination. Figure 8 Exit Status for Students in the 2009–10 Cohort Who Did Not Graduate Non-special education students who completed all other local and state graduation requirements but failed the exit examination—4,175 Students who completed grade 12 but did not complete all graduation requirements—10,910 Students who stopped attending high Students who indicated that he school, and CALPADS did not contain or she transferred to another subsequent enrollment information California school, but CALPADS for another school—28,670 could not verify this*—61,022 Other†—33,994 Source: The California Department of Education’s (department) Educational Demographics Office’s analysis of data from its California Longitudinal Pupil Achievement Data System. * When a school reports that a student is expected to transfer within the State but no subsequent California public school ever reports enrolling the student, the department refers to the student as a lost transfer. † This category includes students who were still enrolled in high school, non‑diploma special education students, students who passed the General Educational Development Test®, and students who exited under various other circumstances, such as passing the California High School Proficiency Examination or departing school for medical reasons. 26 California State Auditor Report 2011-117 March 2012 Some Challenges Remain for the State in Ensuring That CALPADS Achieves Its Full Potential Although CALPADS has substantially improved California’s collecting and reporting of graduation and dropout data, school districts still face some challenges in implementing the system. Specifically, we noted inconsistencies in the school districts’ processes for applying, confirming, and documenting the reasons why students left high school. These inconsistencies may impact the accuracy of the exit codes the districts enter into CALPADS, potentially affecting their graduation and dropout rates. Additionally, several school districts asserted that certain technical limitations in CALPADS make the process of uploading student data to the system cumbersome, and some districts have also found that maintaining and updating CALPADS has required significant time and resources. In addition, CALPADS has yet to fully achieve all of the Legislature’s stated goals for the system. Specifically, although one of the goals of CALPADS is to provide a better means of evaluating the State’s educational progress and the effectiveness of its investments over time, the department’s primary objective during the initial development of the system was to ensure it could fulfill federal reporting requirements. Nevertheless, certain other states have, or are developing, databases that can produce and track more robust data than can CALPADS, potentially enabling these states to further improve their education outcomes. School Districts’ Efforts to Confirm and Document the Reasons Students Exited High School Varied Although CALPADS requires that districts use a standard set of exit codes when entering the reasons students leave high school, our review found that the extent to which districts verified and documented these reasons varied significantly. We reviewed a total of 120 records selected from listings of students from the 2009–10 cohort provided by the following school districts: Delano Joint Union High (Delano), San Francisco Unified (San Francisco), Los Angeles Unified (Los Angeles), Long Beach, Oroville Union High (Oroville), and San Diego Unified (San Diego). We found that many schools had no We found that many schools had no documentation or retained documentation or retained minimal minimal support for certain exit codes they assigned. However, the support for certain exit codes variations we noted revealed that school districts could, in some they assigned. cases, minimize inconsistencies in the data the department uses to calculate graduation and dropout rates by more carefully following the department’s guidance. California State Auditor Report 2011-117 27 March 2012 The degree to which school personnel verified and documented the reasons students left high school varied depending upon the related exit codes. When students graduate with a regular high school diploma, school personnel generally retained strong documentation in the form of transcripts showing that the students met all course and exit examination requirements. However, when students transferred to other California public schools, schools generally did not verify the exit code. In fact, of the student files we reviewed, only one contained evidence documenting that the student ultimately transferred. Federal regulations require schools to obtain official written documentation of a student’s enrollment in another California public school to confirm that the student transferred. Thus, we expected schools to retain documentation verifying such transfers; however, the remaining student files we reviewed contained documentation limited to withdrawal and transfer forms, a note in the student information system, or nothing at all. In general, for exit codes other than graduating, we found that some schools retained hard copy documents as their main source of support, some used notes in their student information systems, and some relied on a combination of the two. However, in several instances, schools retained little or no supporting documentation. In our review, we deliberately selected student records to ensure that we examined a full spectrum of exit codes across various schools within the school districts. Because we did not test a random sample, the results of our tests do not represent the overall accuracy of the data these school districts submitted to CALPADS.4 Nonetheless, we present the result of our review of the 120 files in Table 5 on the following page to illustrate the variation we saw in the accuracy of the different types of exit codes the schools assigned. Table 5 shows that it was unclear whether the schools appropriately assigned exit codes for 18 students. The schools may have assigned some of these codes incorrectly, which could potentially impact dropout and graduation rates. We also determined that there were 21 students to whom school staff assigned incorrect exit codes, but these errors generally did not affect the graduation and dropout rates; only three of these students were not correctly assigned to the dropout category, and another student was not correctly assigned to the graduate category. 4 Table 2 on page 15 in the Scope and Methodology section describes the reasons we did not perform accuracy and completeness testing of CALPADS. 28 California State Auditor Report 2011-117 March 2012 Table 5 The Appropriateness of Exit Codes That School Districts Assigned to a Selection of Students in the 2009–10 Cohort IS THE EXIT CODE CORRECT? TOTAL NUMBER STUDENT EXIT STATUS EXIT REASON YES NO NOT CLEAR OF EXIT CODES Graduated with a standard high school diploma 13 1 14 Graduated with a California High School Exit Examination Graduate 3 3 (exit examination) waiver Graduated with an exemption from the exit examination 4 1 5 Completed all local and state graduation requirements but failed the 4 3 7 exit examination (non‑special education student) Completed grade 12 without completing graduation requirements 8 8 Left school and there is no evidence of enrollment in an academic 5 2 5 12 program leading towards a diploma or its equivalent Other or unknown reason 6 1 1 8 Exited prior to the end of sixth grade 1 1 Dropout Exited during a temporary break such as summer vacation but was 1 1 2 expected to return to the same school after the break but did not Did not return to school the following school year when expected 7 2 9 to return Left school to enroll in an adult education program, but the transfer 2 2 to an adult school cannot be verified Transferred to an institution that is not primarily academic, such as the military and is not in a program leading towards a high 1 1 school diploma Transferred to another California public school 13 5 3 21 Removed from cohort or dropout Transferred to another California public school for 2 2 disciplinary reasons Was pre‑enrolled but never attended school 2 2 Removed from Transferred to a school in another state 2 2 cohort Transferred outside of the United States 2 2 Verified transfer to an adult education program 1 7 8 Still enrolled 1 1 Passed the California High School Proficiency Examination 1 1 Received a special education certificate of completion 2 1 3 Other Passed the General Educational Development Test® (GED) 2 2 Exited a special education transition program after previously receiving a certificate of completion, passing the proficiency exam, 2 2 or passing the GED Left school for medical reasons 1 1 2 Totals 81 21 18 120 Sources: File review conducted by the California State Auditor and the California Department of Education’s May 2011 California Longitudinal Pupil Data System Data Guide. Note: We judgmentally selected 20 student records for review at six school districts (10 student records at each of two schools within each school district). California State Auditor Report 2011-117 29 March 2012 We noted that some schools had trouble properly documenting transfers to adult schools. The department designated two codes for transfers to adult schools: a verified transfer, which the department does not count as a dropout, and an unverified transfer, which the department does count as a dropout. A high school may mark a student as a verified transfer if it determines the student is still enrolled in the adult school at the beginning of the following school year. However, several of the schools we visited appeared to identify adult-school transfers as verified without determining whether the students were enrolled as required. This verification is important because, according to the administrator of its educational demographics office (data administrator), the department does not require adult schools to report data to CALPADS. Thus, in order for the department to properly categorize adult-school transfers as either graduates or dropouts, the prior high school needs to verify directly with the adult school whether the student was enrolled, as required by the department’s guidance. The variation we noted in schools’ verification and documentation of Variation in schools’ verification some of the reasons students leave school appears to have stemmed and documentation of some of in part from the inherent challenges schools face when trying to the reasons students leave school obtain the necessary information, such as difficulties in reaching appears to have stemmed in part students who stop attending school. While some of these challenges from the inherent challenges in may be out of the schools’ control, the schools could achieve greater trying to obtain the necessary consistency in their use of exit codes, such as transfers to adult information—such as reaching schools, if their staff carefully followed the department’s guidance students who stop attending school. related to verifying and documenting exit codes. According to the data administrator, the department began requiring all school districts to report exit data beginning in the fall of 2005 and has since provided exit code descriptions to school districts that include high-level guidance related to verifying and documenting support for exit codes. In May 2011 the department released its most recent version of the CALPADS Data Guide, which includes more specific verification and documentation standards than earlier versions and appeared to us to be sufficiently instructive. However, in conducting our fieldwork at the six school districts between August 2011 and November 2011, we observed a potential information gap between the department’s guidance and instructions that school staff consult when selecting an exit code. In fact, with one exception, the school districts’ written procedures we reviewed during our fieldwork did not reflect the guidance from the department because it was either outdated or incomplete. For example, Long Beach provided its schools with general instructions for verifying certain exit codes without including the definitions of the exit codes to assist school staff in making the appropriate assessments. San Francisco’s procedures, on the other hand, although revised in September 2011, provided basic definitions but omitted other important clarifications about the various exit codes. 30 California State Auditor Report 2011-117 March 2012 At six school districts, we observed To reduce this information gap, the school districts should ensure that a potential information gap their procedures are fully aligned with the department’s most recent between the department’s guidance version of the CALPADS Data Guide. Further, they should revise their and instructions that school procedures as necessary each time the department issues an update. staff consult when selecting an exit code—the school districts’ Finally, we noted that schools often could not obtain the strongest procedures were outdated form of support to document student transfers to schools outside or incomplete. of the State or the country—types of withdrawals that result in the department removing the students from the cohort rather than counting them as dropouts. Although the department requires districts to obtain acceptable documentation of enrollment in another school when a student moves to another state, parents may not know which school their child will enroll in before they relocate. This may present a challenge for school staff in California, as they may not know which school to contact to request an official confirmation of enrollment.5 In such cases, a school’s reliance on less reliable evidence in the form of the parent’s written confirmation appears reasonable. In fact, the department allows schools to rely on a parent’s written confirmation to document a transfer outside of the country. Although such evidence leaves room for doubt about the accuracy of the exit code, the department’s instructions appear reasonable given that the cost and time spent to overcome language barriers in order to verify enrollment in schools in other countries, or the cost and time spent to verify enrollment in other states, most likely outweigh the benefits of having completely reliable documentation. Some School Districts Stated That CALPADS Can Be Unwieldy and May Require Significant Local Resources Although school districts generally noted that the department has addressed many of the issues they had with CALPADS in the earlier stages of its implementation, most stated that they continue to face technical challenges in uploading data to CALPADS. This appears to stem in part from the technical design of CALPADS and not from a lack of technical support, as nearly all of the school officials we interviewed expressed that they received good technical support from California School Information Services (CSIS). CSIS offers advice and help to the districts over the phone and through trainings and Webinars. As one example of CALPADS’ limitations, districts have to first extract information from their student information systems and then upload the data into CALPADS. The process would have 5 Although a similar challenge could present itself when a student moves within the State, CALPADS would generally detect the student’s subsequent enrollment in a new California public school. California State Auditor Report 2011-117 31 March 2012 been more efficient if the system had been designed to regularly communicate with the various school districts’ information systems and automatically harmonize the data between them. However, according to the director of the department’s educational data management division (data management director), this kind of communication between systems would not be possible unless all schools and school districts were to adopt a common data-sharing framework. According to the data management director, this would be a cost-prohibitive and potentially challenging project. Various district officials stated that sending or retrieving data from CALPADS can sometimes take several days or require multiple attempts, depending in part on the size of the files and the number of other districts accessing the system at the same time. The CALPADS coordinator at Long Beach noted that one way districts can avoid these problems may be to upload data more frequently. The frequency with which districts send updates to CALPADS varies widely—some update every few months while others update weekly or even daily, with those updating on a weekly or monthly basis reporting that they have limited resources or staff. According to the coordinator at Long Beach, CALPADS only requires school districts to upload data for student enrollments that have changed. He stated that by reporting changes regularly, such as on a daily basis, school districts can avoid uploading large batches of data. In addition, some school districts indicated that they have Some school districts indicated had to dedicate a substantial amount of time to maintaining that they have had to dedicate CALPADS. In particular, officials from both San Francisco and a substantial amount of time to Los Angeles explained that the State underestimated the burden maintaining CALPADS. that maintaining CALPADS would place on school districts. San Francisco stated that it had a team of four staff devoted to updating and maintaining CALPADS, whereas it previously employed only one person to maintain and report student data using CBEDS. The director of the school information branch at Los Angeles stated that budgetary pressures forced the branch responsible for maintaining CALPADS to cut a number of positions and as a result the workload for the remaining staff had become even heavier. She explained that currently Los Angeles relies on one full-time employee to serve as its CALPADS contact person plus five staff who work on the system in conjunction with their other responsibilities. According to a legislative representative in the government affairs division, the department acknowledges that the school districts lack adequate funding from the State for data management related to CALPADS. He stated that the department has regularly sought additional grant funding for the school districts but has yet to receive approval from the Legislature and the governor. 32 California State Auditor Report 2011-117 March 2012 CALPADS Can Provide Valuable Student‑Level Data to Researchers but Is Not as Robust or Innovative as Some Other States’ Systems In implementing CALPADS, the Legislature intended the system to do more than simply enable the State to comply with federal reporting requirements. Rather, its goals for CALPADS included allowing the State to evaluate its educational progress and to utilize its investments over time to increase student achievement. In other words, the Legislature intended CALPADS to assist educators in making data-driven decisions. To help in reaching this goal, the Legislature intended that the State would make CALPADS’ data available to outside researchers who have the resources to analyze and study it. However, CALPADS’ current capabilities as well as restrictions in state law may limit the State’s ability to do this in an efficient manner. The department has taken certain steps to make CALPADS data and other data public, such as publishing information on school performance, test scores, student demographics, and other topics through its DataQuest Web site.6 Nevertheless, the department releases only aggregate information to the general public in order to protect student privacy, which limits the data’s utility to outside researchers.7 To obtain and track individual students’ data as units of measurement, outside researchers must request it directly from the department, which appears to be time consuming for both researchers and the department. According to the data management director, the department evaluates each request for student-level data to determine whether the requestor’s research objectives are reasonable and whether the researcher has a genuine need for student-level data to accomplish these objectives. The data management director stated that recipients of student-level data must sign a confidentiality agreement, must not retain any copies of the data when their research is complete, can only use the data for the purposes the department authorizes, and must supply copies of their completed research products to the department. Although the basic data that CALPADS collects for students from kindergarten through grade 12 are valuable, some other states may be taking fuller advantage of the potential for longitudinal data to improve their educational systems. Based on our review of Web sites of other states as well as the National Center for Education Statistics, some states’ longitudinal databases are or will be capable of collecting more comprehensive data about students than CALPADS. For example, Massachusetts has an 6 The public can access the DataQuest Web site at data1.cde.ca.gov/dataquest. 7 The Family Educational Rights and Privacy Act, a federal law, as well as provisions of state law, generally prevent the department from releasing student records except under very limited circumstances. California State Auditor Report 2011-117 33 March 2012 early warning system that uses statewide data to assist local Other states may be taking fuller school districts to identify students who may be at risk of not advantage of the potential for graduating on time from high school. In particular, the system longitudinal data to improve their assigns incoming freshmen to one of five risk levels based on their educational systems. The Florida middle school academic performance and attendance record. In system, for example, links data addition, Florida has implemented a student-centric longitudinal concerning a variety of school system that stores various data, including information about information to information about student demographics, enrollment, courses, test scores, awards, student employment and earnings, and employment information. These data are longitudinally linked tracks students over time, and to information about students’ employment and earnings and performs trend analyses. the system provides capabilities to track students over time and perform trend analyses. Meanwhile, Texas is developing a system that it expects to longitudinally link information from the State’s pre-kindergarten, kindergarten through grade 12, higher education, and workforce systems. By comparison, according to California’s chief deputy superintendent of public instruction (chief deputy), when the department originally secured funding for CALPADS, the Department of Finance directed the department to limit the system’s functionality to fulfilling federal reporting requirements. As a result, he explained, CALPADS primarily produces reports that the federal government specifically requires. According to the chief deputy, although CALPADS has the potential to collect more data, the department has no funding to expand its data-collecting capabilities at this point. However, we believe that if the State cannot expand CALPADS to support initiatives similar to those of other states in the coming years, the State risks missing the opportunity to be more innovative in using its student longitudinal data. Districts Employ a Variety of Strategies That May Assist Students at Risk of Dropping Out One potential long-term benefit of CALPADS is that its graduation and dropout data should help districts to identify and develop effective approaches for reducing their dropout rates. School districts currently employ a variety of strategies to address and mitigate the risk factors that may cause students to drop out. Some of these strategies involve dedicated dropout programs, which for the purposes of this audit we define as either programs that districts offer to students they have specifically identified as at-risk or programs staffed by employees in positions dedicated to preventing or reducing dropouts. Other strategies may encompass efforts that target a wide range of students, including those who may be at risk of dropping out. Such efforts might involve various districts’ strategies to academically engage all students in order to help them succeed. 34 California State Auditor Report 2011-117 March 2012 Recent budget cuts may have affected the districts’ ability to dedicate resources to preventing students from dropping out. Although state law does not require districts to operate dropout prevention programs, in previous years the Legislature enacted a number of laws with the intent of preventing or reducing dropouts, including authorizing block grant programs for these purposes. However, as a result of budget cuts, the Legislature granted the districts greater discretion in choosing how to spend their funds, including those it previously provided for dropout prevention. As a result, some districts may have allocated funds to other needs that they previously used for dropout prevention. Nonetheless, we found that all the districts we visited have some dedicated dropout prevention programs and also conduct other efforts to engage their students. We summarize these programs and efforts in Table 6 and describe them in more detail in the Appendix, beginning on page 39. Table 6 School Districts’ Dropout Prevention Programs and Other Selected Strategies to Assist Students in Graduating OTHER SELECTED STRATEGIES THAT CAN ASSIST DROPOUT PREVENTION PROGRAMS* STUDENTS IN GRADUATING SYSTEMATIC IDENTIFICATION MENTORING AND COUNSELING ACADEMIC TOOLS THAT CAN AID COMMUNITY SCHOOL DISTRICT OF AT‑RISK STUDENTS FOR AT‑RISK STUDENTS AT‑RISK STUDENTS OUTREACH EFFORTS Specialized dropout • California High School Exit Regular meetings with prevention counselors Examination (exit examination) community to identify support classes assistance available Delano Joint • English Language learners to students Union High (English learners) • Remedial math and English classes • Continuation Education • Summer school Program to identify • Exit examination support classes and mentor at‑risk • English learners classes San Francisco Unified ninth graders • Remedial math and English classes • Continuation Education • Summer school Program to track and Specialized counseling for • Exit examination support classes identify at‑risk students selected at‑risk students • English learners classes and students who • Remedial math and English classes Los Angeles Unified may become at risk • Continuation Education based on risk factors • Summer school identified through a • Community day schools regression analysis Specialized dropout • Exit examination support classes prevention counselors; • English learners classes program to intervene with • Remedial math and English classes Oroville Union High at‑risk 9th graders • Continuation Education • Summer school • Community day school California State Auditor Report 2011-117 35 March 2012 OTHER SELECTED STRATEGIES THAT CAN ASSIST DROPOUT PREVENTION PROGRAMS* STUDENTS IN GRADUATING SYSTEMATIC IDENTIFICATION MENTORING AND COUNSELING ACADEMIC TOOLS THAT CAN AID COMMUNITY SCHOOL DISTRICT OF AT‑RISK STUDENTS FOR AT‑RISK STUDENTS AT‑RISK STUDENTS OUTREACH EFFORTS Program to track progress • Peer mentoring for • Exit examination support classes Committee with of students in meeting at‑risk students • English learners classes community members graduation requirements • Counseling for selected • Remedial math and English classes to assess the dropout Long Beach Unified and to allow users to view at‑risk students • Continuation Education issue and create a aggregate and individual • Summer school strategic plan for data for students who are • Beach High School for 9th and 10th assisting dropouts at risk of not graduating graders who are credit deficient Dedicated dropout • Exit examination support classes Partnership with prevention department • English learners classes local charities and overseeing various • Remedial math and English classes a state university to dropout prevention • Continuation Education provide mental health programs including • Summer school services on high the following: • Community day school school campuses San Diego Unified • Professional mentoring for at‑risk students • Task force that meets to discuss dropout solutions • Yearly push to re‑enroll students Source: Interviews and documentation review conducted by the California State Auditor. Note: We visited the six school districts listed and two high schools in each school district. For the purpose of this table, all information is presented at the school district level. * For the purposes of this audit, we use the term dropout prevention programs to refer either to programs offered specifically to students whom schools have identified as being at risk of dropping out or to programs staffed by employees in positions dedicated to preventing dropouts. State law does not require school districts to systematically measure the effectiveness of their dropout prevention efforts. However, as mentioned previously, one potential benefit of CALPADS is that its data should enable districts to better identify the programs and efforts that most effectively prevent students from dropping out. The districts can then allocate their limited resources accordingly. CALPADS does not currently capture data related to participation in dropout prevention programs, and given the numerous contacts a particular student may have with various educators, it could prove difficult to track all supports each student receives. However, expanding CALPADS to capture participation in certain efforts could prove beneficial in understanding the impact of these efforts. Even without these sorts of changes to the system, the eventual availability of CALPADS’ data for multiple cohorts should soon enable the department, the school districts, and other researchers to identify the school districts that have reduced their dropout rates through exemplary dropout prevention programs. 36 California State Auditor Report 2011-117 March 2012 Recommendations To increase consistency, the department should remind schools and school districts of the importance of aligning their procedures for recording pupil enrollment and exit data with the CALPADS Data Guide. To improve efficiency, the department should inform school districts of the value of frequently updating the data they transfer from their local student information systems to CALPADS. Also, to the extent that the department becomes aware of ways that schools and school districts can perform CALPADS-related activities more efficiently, it should provide written guidance to schools and school districts on these best practices. To improve the utility of CALPADS and fulfill the legislative intent of the system, the department should work with the Legislature, the State Board of Education, and the governor to identify priorities for building upon the system when funding is available. These priorities could include tracking student participation in dropout prevention programs or strategies to measure the effectiveness of those programs or strategies over time. California State Auditor Report 2011-117 37 March 2012 We conducted this audit under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives specified in the scope section of the report. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, ELAINE M. HOWLE, CPA State Auditor Date: March 15, 2012 Staff: Laura G. Boll, Project Manager Sean R. Gill, MPP Lisa Ayrapetyan Casey Caldwell Christopher P. Bellows Legal Counsel: Donna Neville, Associate Chief Counsel Stephanie Ramirez-Ridgeway, Senior Staff Counsel IT Audit Support: Michelle J. Baur, CISA, Audit Principal Sarah Rachael Black, MBA Richard W. Fry, MPA Lindsay M. Harris, MBA For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255. 38 California State Auditor Report 2011-117 March 2012 Blank page inserted for reproduction purposes only. California State Auditor Report 2011-117 39 March 2012 Appendix SCHOOL DISTRICTS’ DROPOUT PREVENTION PROGRAMS Research indicates that dropping out of school is often the result of a long process of disengagement that may begin before a child enters school and involves numerous risk factors that may adversely affect the student’s ability to graduate. Some of these risk factors relate to the student’s performance: Problems such as poor grades increase the likelihood that a student will drop out. Other risk factors originate in the student’s family: High family mobility and a lack of family commitment to education are both linked to higher dropout rates. In addition, research shows that poor attendance is a key indicator that a student is at risk of dropping out. Counselors and other school district personnel we interviewed generally agreed with this assessment. The Joint Legislative Audit Committee asked the California State Auditor to visit a selection of school districts to determine whether schools offer dropout prevention programs and, if so, what steps they take to engage teachers, parents, students, and the community in these programs. To address this objective, we defined dropout prevention programs as either programs that schools offer specifically to students they have identified as at risk of dropping out or programs staffed by employees whose positions are solely dedicated to preventing or reducing dropouts. As we indicate in Table 6 in the Audit Results on pages 34 and 35, we visited six school districts, each of which offered at least one dropout prevention program focused on goals such as systematically identifying at-risk students and preventing at-risk students from dropping out. The school districts also offered a variety of academic supports to aid at-risk students, such as credit recovery programs and small learning communities. We also found that the districts generally worked with their communities to engage students and help them succeed. The Figure on the following page shows the graduation and dropout rates of the districts we visited. Dropout Prevention Programs The majority of the dropout prevention programs at the school districts we visited involve specialized counseling or mentoring for at-risk students. For example, according to San Diego Unified School District’s program manager of dropout prevention, the dropout prevention department organizes a program called Check and Connect, which she described as a comprehensive mentoring program in 10 high schools that pairs approximately 300 10th grade students with mentors who will work with them through their anticipated graduation in 2014. The American 40 California State Auditor Report 2011-117 March 2012 Institutes for Research provided the necessary funding for this program. Likewise, in fall of 2011 at three partnering high schools, San Francisco Unified School District used a federal grant to launch Project Arrive, a program that targets incoming ninth graders who have a history of truancy with the goal of helping them successfully transition into high school. Additionally, Delano Joint Union High School District (Delano) and Oroville Union High School District indicated they employ counselors, known as outreach consultants, whose duties include identifying at-risk students and developing programs or safety nets to assist them. Figure The 2009–10 Cohort’s Graduation and Dropout Rates at Six School Districts Graduation rate 90% Dropout rate 80 70 60 50 40 30 20 10 0 Joi D n e t l U an n o ion Hig S h an Fr U an n c ifi is e c d o Los A U n n g i e fi l e e d s Long U B n e i a fi c e h d O U r n o i v o il n le High San U D n ie ifi g e o d School District Source: California Department of Education’s DataQuest Web site’s summary of data from the California Longitudinal Pupil Achievement Data System. Based on our fieldwork, we found that teachers, counselors, and school administrators most often identify and track students who meet at-risk criteria on a case-by-case basis. However, this method increases the risk of overlooking certain students. To address this problem, some of the districts we visited have begun implementing more systematic approaches. For example, Los Angeles Unified School District created the Student Alert System to identify at-risk students. According to the assistant administrative analyst who developed the system, the Student Alert System consists of two programs. In general, the first is an early warning system that California State Auditor Report 2011-117 41 March 2012 identifies students whose performance in academics, attendance, and other areas is declining, suggesting that intervention may be helpful. The second is an at-risk system that identifies students who may be at risk of dropping out because of low grade point averages, failure to pass the California High School Exit Examination (exit examination), below basic comprehension in mathematics or English, attendance problems, or a deficient number of credits towards graduation. The administrative analyst developed this system so that it uses the district’s own data about students to identify the key risk factors. Similar to this effort, the California Department of Education (department) is working with the California Comprehensive Assistance Center at WestEd, a nonpartisan, nonprofit research, development, and service agency based in San Francisco, and the National High School Center to pilot an early warning and intervention project in several districts throughout the State. The pilot began in June 2011 and is scheduled to end in June 2012. According to an education programs consultant within the department’s secondary, career, and adult learning division, the pilot project uses a free, downloadable spreadsheet that was developed by the National High School Center. He explained that this tool gathers data from the school districts’ student information systems including attendance and academic data, and where available, behavioral data, to identify students who are at risk. The consultant stated the department hopes to use the tool to influence and learn from school districts’ policies, thinking, and practices, and to improve early identification and intervention with at-risk students to help them graduate from high school. Academic Supports During our review, we found that school districts leverage the educational options that they already have in place to engage students and encourage them to succeed. Educators we spoke with explained that when students have access to curriculums that they find meaningful and relevant, they become more engaged and are therefore more likely to graduate. For example, some school districts we visited indicated that they employ small learning communities, which may take the form of a school within a school, a career-based academy, or a grouping of students who remain together throughout high school. These programs, which offer common classes and a career-oriented curriculum, can foster a sense of community among the students who participate, as well as other benefits. The small learning community model is similar to the department-sponsored California Partnership Academies, which began in 1984. The department’s research indicates that even though half of the students at the Partnership Academies are at risk, 42 California State Auditor Report 2011-117 March 2012 95 percent graduated at the end of 2009–10. Additionally, state law creates a State Student Attendance Review Board (attendance review board) and provides for the creation of district-based attendance review boards. The attendance review board coordinates statewide policy and assists with personnel training, while district attendance review boards intervene with chronically truant students. We found all the districts we visited had attendance review boards, though the director of research and professional learning communities (research director) at Long Beach Unified School District (Long Beach) indicated that the effectiveness of attendance review boards on older students is lessened, as students who have reached grade 10 are likely to have already established patterns of truancy. In addition, all six of the districts we visited offer specialized classes, supplementing their core curriculum, to students who require additional assistance. For example, the districts generally provide English Language learners with specialized instruction to increase their proficiency in reading and writing. The districts also offer classes to students who have not passed the exit examination or who are below proficiency in mathematics and English. Likewise, credit recovery classes allow students who have failed classes to retake them and get back on track towards graduation. Furthermore, all districts we visited offer nontraditional options for assisting at-risk students who either struggle in a traditional classroom setting or have fallen behind and thus are at risk of not graduating. For instance, some districts provide community day schools for students who have been expelled from school or have attendance or behavior issues. The districts tailor these schools to meet the needs of students who may fit poorly in traditional high schools. In addition, continuation education programs are offered to students who are 16 or older and are deemed at risk of not completing their schooling. In addition to the required academic courses for graduation, the program of instruction emphasizes occupational or career orientation or a work-study schedule and intensive guidance and counseling. Long Beach has also developed Beach High School to serve ninth and 10th grade students that have fallen seriously behind in meeting credit requirements. The school offers a hybrid model of instruction that fuses direct instruction, computer aided learning, and independent learning to enable students to engage more fully and take responsibility for their education. Currently, the high school serves approximately 300 students who may return to their school of origin and participate in graduation with the class with whom they began school. Finally, most districts we visited offer summer school to allow students to make up missed credits. California State Auditor Report 2011-117 43 March 2012 Community Involvement We found that some districts we visited have worked with their communities to decrease the number of students who drop out. This sort of community involvement can take a variety of forms. For instance, some districts provide services to students who do not relate directly to education. In addition to providing counseling and therapy, schools in these districts may work with local police and probation officers on the behalf of students. Schools may also act as a bridge to city or county services. For example, at Cesar E. Chavez High School in Delano, key administrators, counselors, and other educators meet with various representatives, including those from local nonprofits, county services, and clinics. The school psychologist there explained that in these meetings they discuss the interventions that are available, both generally and for specific students who may require help to remain in school. Aside from providing these sorts of services, some of the districts we visited explained that they solicit community involvement in developing dropout prevention policies and interventions. This can serve to increase the visibility of the issues that cause students to drop out and to engage the community in preventing dropouts. Notably, the research director at Long Beach informed us that during the 2008–09 school year, her district created a 55-member committee aimed at addressing dropout and graduation rates that included not only teachers, parents, and students but also personnel from community-based organizations and higher education. The committee met to discuss the reasons students dropped out and the available interventions. According to the research director, the committee also reviewed best practices and interviewed students who had dropped out of high school to gain an understanding of why they left and what could prevent them from dropping out. The research director explained that the committee proposed six recommendations to the Long Beach school board, which chose to include the proposed actions in the district’s strategic plan and now reports to its school board three times a year on the implementation of these recommendations. Finally, all of the districts we visited made at least some effort to communicate with their communities about their dropout rates. State law requires districts to publish School Accountability Report Cards that include schools’ graduation rates, teacher qualifications, testing results, and other data related to the Academic Performance Index and Adequate Yearly Progress reports. In addition, several of the districts we visited use electronic communication to engage parents and the community in ways that may enable schools to better prevent dropouts. For example, several districts we visited utilize School Loop. According to the Long Beach’s research director, School Loop is a Web-based program that allows students, 44 California State Auditor Report 2011-117 March 2012 parents, teachers, and counselors to view their specific grades, assignments, and attendance records. School Loop also allows school or district personnel to disseminate newsletters, bulletins, and other information. Methods such as School Loop may provide parents or educators with readier access to information about their students, giving them another way to help their students succeed. California State Auditor Report 2011-117 45 March 2012 (Agency response provided as text only.) California Department of Education 1430 N Street Sacramento, CA 95814 February 22, 2012 Elaine M. Howle, State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Ms. Howle: Subject: Response to the Bureau of State Audits’ Draft Audit Report No. 2011-117 This is the California Department of Education’s (Education) response to the Bureau of State Audits’ (BSA) report titled, High School Graduation and Dropout Data: California’s New Database May Enable the State to Better Serve Its High School Students Who Are at Risk of Dropping Out, Report No. 2011-117. Recommendation No. 1: To increase consistency, the department should remind schools and school districts of the importance of aligning their procedures for recording pupil enrollment and exit data with the California Longitudinal Pupil Achievement Data System (CALPADS) Data Guide. Education’s Comments and Corrective Actions: Education will issue a reminder to local educational agencies (LEAs) on the importance of following procedures that are consistent with the CALPADS Data Guide. The reminder will include reference and links to the relevant guidance and available training, along with contact information for follow-up questions. Recommendation No. 2: To improve efficiency, the department should inform school districts of the value of frequently updating the data they transfer from their local student information systems to CALPADS. Also, to the extent that the department becomes aware of ways that schools and school districts can perform CALPADS-related activities more efficiently, it should provide written guidance to schools and school districts on these best practices. Education’s Comments and Corrective Actions: Education will issue a reminder to LEAs on the importance of updating the data they transfer from their local student information systems to CALPADS. The reminder will include reference and links to the relevant guidance and available training, along with contact information for follow-up questions. 46 California State Auditor Report 2011-117 March 2012 Elaine M. Howle, State Auditor February 22, 2012 Recommendation No. 3: To improve the utility of CALPADS and achieve the full legislative intent of the system, the department should work with the Legislature, the State Board of Education, and the governor to identify priorities for building upon the system when funding is available. These priorities could include tracking student participation in dropout prevention programs or strategies to measure the effectiveness of those programs or strategies over time. Education’s Comments and Corrective Actions: Education will meet with the Legislature, the State Board of Education, and the Governor’s Office to determine priorities for building upon the system and to discuss CALPADS-related strategies. If you have any questions regarding this subject, please contact Kevin W. Chan, Director, Audits and Investigations Division, by phone at 916-323-1547, or by e-mail at kchan@cde.ca.gov. Sincerely, (Signed by: Richard Zeiger) Richard Zeiger Chief Deputy Superintendent of Public Instruction California State Auditor Report 2011-117 47 March 2012 cc: Members of the Legislature Office of the Lieutenant Governor Little Hoover Commission Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press