Comprehensive Review Pupil Achievement

PUPIL ACHIEVEMENT Summary of Findings and Recommendations From January 27 through 29, 2004, a study team conducted an on-site assessment of the progress the district has made implementing the Pupil Achievement recommendations in the Oakland USD Assessment and Recovery Plan, January 31, 2000 and the additional standards introduced in September, 2003, that reflected changes in legal and professional requirements affecting the district. The primary goal of the Pupil Achievement section of the original Assessment and Recovery Plan, January 31, 2000 was to improve student achievement through complete implementation of required programs and recommended strategies. To determine the extent of the district’s progress, the study team examined district documents and interviewed members of the administration, the faculty, and other key personnel directly involved with the design and delivery of curriculum in the district’s schools. Since school was not in session when the team visited the district in August 2003, the January visit focused on examining the classroom implementation aspects of the recommendations. The Oakland Unified School District has made significant progress toward implementing the Pupil Achievement recommendations of the Assessment and Recovery Plan, January 31, 2000. Notable gains have been made toward establishing an organizational foundation for future growth and improvement. Consequently, the results in some instances won’t be realized for years. However, much remains to be accomplished. Student achievement gains have been documented, but scores remain low. Major progress has been made toward the establishment of instructional program management based on the systematic use of data, and the budgeting of resources in the district promises to be curriculum-driven and results-based. The major strengths and weaknesses observed by the study team in the area of Pupil Achievement are summarized below. The findings have been grouped in five categories corresponding with the five major areas of investigation: 1. Control of Resources, Programs, and Personnel 2. Clear and Valid Objectives for Students 3. Internal Consistency and Rational Equity in Program Development and Implementation 4. Use of Assessments to Improve Programs 5. Improved Organizational Productivity Control of Resources, Programs, and Personnel The Oakland Unified School District Instructional framework has replaced the out-of- date and inadequate policy framework of the district. The Framework is designed to provide direction for district operations in the areas of assessment, professional development, Results-Based Budgeting, equity, and curriculum. Although the study team was not provided with a comprehensive set of board policies to guide curriculum design and delivery, staff has indicated that policies that provide for overall curriculum management are being revised and are due to be presented to the board by April 2004. The board has adopted the assessment part of the Instructional Framework – a plan for implementation of an assessment, data, and accountability model. The plan calls for assessment to be aligned with textbook adoptions in the core areas. The district’s Table of Organization has been revised and partially meets the requirements specified in the 2000 Recovery Plan. No substantial progress has been made to revise inadequate job descriptions. A district planning process has not been established to conform with the 11 criteria proposed in the Curriculum Management Audit that was conducted as part of the Assessment and Recovery Plan, January 31, 2000. However, the 2002-2007 Strategic Alignment Plan establishes high expectations and Twelve Core Strategies, and sound planning is happening in key areas. The board has adopted a policy that acknowledges long-range planning as an integral component of the growth and development of the district. Clear and Valid Objectives for Students The study team was not presented with a comprehensive curriculum management plan; however, staff indicated it is currently being developed and purportedly includes the quality components of a curriculum management plan as recommended in the Assessment and Recovery Plan, January 2000. In lieu of formal curriculum guides, the district administration decided to select and adopt several standards-based textbook series. The textbooks approved by the state are well aligned with the state’s standards and the state’s assessment instruments. The inherent lack of focus of a textbook curriculum has been addressed by the district staff by developing Pacing Guides to give teachers the necessary specificity for clear direction. The district’s need to rapidly reform its instructional operations made the adoption of a textbook curriculum a sound strategy. The study team visited the classrooms of 24 schools across the district and observed that the textbook curriculum strategy had apparent teacher support and was resulting in teaching directed at the California Standards. Adoption and subsequent districtwide implementation of the Open Court and High Point instructional programs, along with Harcourt Math and the comprehensive benchmark assessment systems, have at the elementary level notably addressed the lack of cohesion, feedback, and staff development described in the 2000 Assessment and Recovery Plan. 2 Internal Consistency and Rational Equity in Program Development and Implementation. The auditors found numerous potential compliance issues within the special education programs. Of special concern were the continuing high numbers of past-due yearly IEP reviews and triennial IEP reviews. The district has developed a Master English Language Learner Plan which has been accepted by the Comité with the California Department of Education. In addition, aspects of the ELL program have been incorporated in the Professional Development Plan adopted by the board. Progress has been made enforcing the provisions of the Voluntary Resolution Plan; however, no evidence was provided to indicate personnel were being held accountable through timely evaluations. Use of Assessments to Improve Programs The study team found continuing progress in the district’s data systems for disaggregating data by race/ethnicity, gender, socioeconomic factors, and language. The district provides these data in useable form to gauge operational performance and to improve instructional programs and decision-making. No progress has been made to expand the scope of local assessments to include all required subjects at all grade levels. The present focus is on reading/language arts and math. Presently the district has met 16 of the 20quality characteristics of a comprehensive program and student assessment plan as recommended in the Curriculum Management Audit conducted as part of the Assessment and Recovery Plan, January 2000. The effort to train principals is “still evolving.” All principals continue to be trained in the use of data and assessment with assistance from the executive directors. The study team was not provided with evidence that indicates that the training is effective in improving classroom instruction. Student achievement continues to be a major concern. None of the II/USP high schools met AYP targets, and twenty-four of thirty-four elementary schools did not meet their 2003 AYP targets. Improved Organizational Productivity The district is in the beginning phase of implementing a Results-Based Budgeting system. Principals are receiving training for their expanded role and are developing their first results-based budget. The study team reviewed the plans and concluded that the system has the potential to significantly increase the Oakland Unified School District’s educational productivity. 3 1.1 Curriculum Management—Policy Professional Standard The district through its adopted policy provides a clear, operational framework for the management of the curriculum. Progress on Implementing the Recommendations of the Recovery Plan 1. The district lacks a comprehensive set of board policies to guide curriculum design and delivery. A few plans, which district personnel identify as policies, have been adopted during the last three months, and a few more are in process. They include the following: a. A policy that provides for overall curriculum management is due to be presented to the Governing Board by April 2004. An Oakland Unified School District Instructional Framework has been developed that lays out basic assumptions about student achievement and the context for learning. This document sets some direction for the district. It specifies that state standards are the philosophical framework for curriculum design and that implementation is through state-adopted texts, professional development with follow-up coaching, and assessment data collection used to monitor program implementation. The district instructional framework is designed to provide direction in the following separate plans: assessment, professional development, Results-Based Budgeting, equity, and curriculum. b. Missing in this framework is a linkage to district goals and objectives in the Strategic Alignment Plan. Also missing is a system designed to monitor and review the curriculum cycle. Policy has not been developed that links organizational structure; job descriptions (roles and responsibilities) and the table of organization (accountability relationships). A current table of organization is in place. The district should develop a policy that links these areas. c. The Governing Board has adopted a policy that acknowledges long-range planning as an integral component of the district’s growth and development. d. The district has not adopted a policy that requires K-12 alignment among the curriculum goals and objectives (written), teacher delivery techniques and strategies in the classroom (taught), and districtwide assessments (tested). The district also lacks revised courses of study showing alignment among these components. The district should develop such a policy to ensure alignment among the written, taught, and tested curriculum. e. A policy requiring vertical and horizontal articulation between the grades and among the instructional levels has not been developed. The district should develop such a policy to ensure articulation and coordination of instruction and instructional resources. 4 f. The board has adopted as policy a plan on districtwide professional development for the certificated staff. It links staff development to the delivery of instruction. The district lacks a staff development plan for classified employees linking their role to instruction. A policy should be developed that addresses the classified staff’s professional development in its role of support for the delivery of curriculum. g. A policy establishing clear expectations for monitoring the curriculum at all levels has not been adopted. The administration has enacted practices for monitoring the implementation of specific adoptions such as Open Court, High Point and math texts across most grades and courses, but not for language arts as a whole, other curricular areas, or across the sites and the district. h. The board has adopted a plan, considered as policy, that implements an assessment, data, and accountability model. This plan, which is part of the implementation of the Instructional Framework, identifies the philosophical framework for the design of testing as assessment aligned to textbook adoptions combined with state assessment data. District formative assessments are identified within textbook adoptions for reading (K-5), mathematics (K-high school), English Language Development/Intervention (6-high school), and science grades 6 and 8. A time line is provided for further testing in science at the middle schools and social science at the high schools. The plan does not address comprehensive assessment of all subjects at all grade levels, or a process for the ongoing evaluation of the assessment plan. i. The current assessment plan should be revised to provide for the comprehensive assessment of student performance in all subjects at all grades, assessment of all levels of the system, and an evaluation of the effectiveness of the plan itself. 2. The district lacks a policy authorizing the Superintendent or the State Administrator to approve all administrative bulletins to enable more rapid response to organizational problems and to authorize top administrators to act more efficiently and effectively. Such a policy needs to be developed and implemented to promote efficient and effective administration of the district when the Governing Board assumes full control. 3. Executive directors and the district staff provide training for the administrative staff on policy implementation through area network meetings and management meetings. A plan should be established in policy that formalizes a training program for the staff on policy implementation at all levels of the district, including processes for holding administrators responsible for implementing policies and ensuring that the staff understands and follows policies. 4. The district lacks a plan that ensures the implementation of policy at all levels. Some processes are in place through the administrative evaluation protocol to hold administrators responsible for policy implementation and ensuring that the site staff 5 understands and follows relevant policies. There is an informal expectation that principals provide training for the site staff and monitor staff implementation. 5. Policies were not evaluated for adequacy or for the effectiveness of staff implementation. 5. The table of organization has been revised as follows: a. Current reporting relationships are reflected. Some elements are not clearly delineated. b. The chart does not show a staff member assigned to professional development for certificated staff within the Division of Student Achievement. c. The Technology Training Manager reports to the Chief Financial Officer for Business Services with no indication of the relationship between technology and the Division of Student Achievement. 7. Organizational changes are adequately communicated to all district personnel. The district has some job descriptions for the Department of Student Achievement. 8. Job descriptions have not been revised. The district should update all job descriptions for certificated and classified personnel to reflect the criteria presented in the Curriculum Management Audit Report (2000) and to reflect district operations. The district also should ensure that job descriptions include appropriate linkage to curriculum and instruction and match the organizational chart. 9. An instructional framework indicates that state standards are the district’s written curriculum. In-services have been provided for administrators on state textbooks in language arts (K-8) and mathematics (K-5). There is no professional development in other written curriculum. The district should continue to provide in-services for administrators and instructional facilitators to develop their skills continually in monitoring the delivery of the written curriculum in the classroom and for evaluating the instructional staff and providing feedback that emphasizes improved instruction. 10. District administrators have taken some steps to monitor the evaluations of the administrative and instructional staff to ensure that they provide feedback to improve the delivery of instruction. These steps include training for principals and instructional facilitators on textbook adoptions and instructional practices related to implementation. Through formal evaluations and informal collaboration, principals and teachers receive feedback aimed at improving the delivery of instruction. The district should establish a formal process to monitor the administrative and instructional staffs to ensure that they provide feedback to improve the delivery of instruction. 11. As part of the district initiative to implement Results-Based Budgeting, administrators performed a cost-benefit analysis of the teachers on special assignment (TSA) and developed a strategic reduction of centralized TSAs as new textbook implementations 6 become institutionalized. The role of some TSAs has evolved to providing feedback and modeling for teachers. A cost-benefit analysis has not been completed for the TSAs who support principals with issues related to compliance with state and federal regulations. Feedback from the principals indicates that the TSA role is valued. The district staff should continue to monitor the cost-benefit of TSAs and ensure that their role benefits delivery of improved instruction. Standard Implemented: Partially January 2000 Rating: 2 September 2003 Rating: 2 March 2004 Self Rating: 4 March 2004 New Rating: 3 Implementation Scale: 7 1.2 Student Objectives – Core Curriculum Content Professional Standard The district has clear and valid objectives for students, including the core curriculum content. Progress on Implementing the Recommendations of the Recovery Plan 1. The district plans to present its curriculum management plan to the board in April 2004. The district Associate Superintendent for Instruction indicated that the elements of a curriculum management plan recommended by FCMAT are to be included. These elements are: a. The district’s philosophical approach to the curriculum; b. A curriculum review cycle for all disciplines; c. A consistent curriculum format; d. Delineation of responsibilities for curriculum-related decision-making for district administrators, principals, teachers, district and school committees; e. Expectations for the delivery of the curriculum; f. Instructions for monitoring the curriculum that include specific procedures and criteria for principals and other staff; g. Timing, scope, team membership, and procedures for curriculum review and adoption; h. Selection procedures for instructional resources; i. A process for integrating technology into the curriculum; j. A process for communicating curricular revisions to the board, staff, and community. 2. The district’s curriculum philosophy, the Instructional Blueprint, includes an alternative to FCMAT’s recommendation to develop curriculum guides. The district wishes to adopt California standards-based textbooks as its curriculum, with pacing guides developed to address the inadequacies of a textbook curriculum. The district’s reasons for moving away from developing curriculum guides include the short time line for improving student achievement, lack of staffing expertise, and funding. FCMAT accepted the district alternative to the FCMAT curriculum monitoring process and has reviewed the district-adopted California standards-based textbooks and pacing guides using the five basic minimum guide criteria. When California standards-based textbooks were considered together with a district pacing guide, half of them reached the basic adequacy score of 12 points or higher. To determine quality, FCMAT examined 15 documents presented as curriculum guides by district personnel. These guides included district-adopted California standards-based textbook teacher editions and district-developed pacing guides. The teacher editions for each of the textbooks were reviewed and rated on whether they contained the elements of each of five criteria that support effective delivery of the curriculum. The criteria are listed in Exhibit 1.2.1 8 Exhibit 1.2.1: Curriculum Guide Audit Criteria Criteria Description One Clarity and Validity of Objectives 0 no goals/objectives present 1 vague delineation of goals/learner outcomes 2 states tasks to be performed or skills/concepts to be learned 3 states for each objective the what, when (sequence within course/grade), how actual standard is performed, and amount of time to be spent learning Two Congruity of the Curriculum to Testing/Evaluation 0 no evaluation approach 1 some approach of evaluation stated 2 states skills, knowledge, concepts which will be assessed 3 each objective is keyed to district and/or state performance evaluation Three Delineation of the Prerequisite Essential Skills, Knowledge, and Attitudes 0 no mention of required skill 1 states prior general experience needed 2 states prior general experience needed in specified grade level 3 states specific documented prerequisite or description of discrete skills/concepts required prior to this learning (may be a scope and sequence across grades/courses Four Delineation of the Major Instructional Tools 0 no mention of textbook or instructional tools 1 names the basic text/instructional resource(s) 2 names the basic text/instructional resource(s) and supplementary materials to be used 3 states for each objective the “match” between the basic text/instructional resource(s) and curriculum objective Five Clear Linkages (Strategies) for Classroom Use 0 no linkages cited for classroom use 1 overall, vague statement on linkage for approaching the subject 2 provides general suggestions on approach 3 provides specific examples on how to approach key concepts/skills in the classroom The documents were assigned values of zero to three on each of the five criteria. A maximum of 15 points is possible, and guides receiving a rating of 12 or more points are considered strong or adequate. The mean ratings for each criterion and the mean for the total guide ratings were then calculated. Exhibit 1.2.2 shows the team’s ratings of California standards-based textbook teacher editions and pacing guides. 9 Exhibit 1.2.2 FCMAT’s Rating of California Standards-based Textbook Teacher Editions and Pacing Guides in the Core Subject Areas Presented to FCMAT Grades K-12 Oakland Unified School District January 2004 Grades K-12 Textbook District Grade One Two Three Four Five Total Teacher Editions and Pacing Pacing Level (Obj) (Assess) (Pre) (Res) (App) Rating Guides Guide Open Court English Language K-5 Yes 2 2 2 3 3 12 Arts Open Court Foro Abierto Para K-5 Yes 2 2 2 3 3 12 La Lectura Hampton Brown High Point 6-12 Yes 2 2 3 3 3 13 Harcourt Social Studies K-5 No 1 1 2 3 3 10 Harcourt Mathematics K-5 Yes 2 2 2 3 3 12 Prentice Hall Algebra 1 9-12 Yes 2 2 1 3 3 11 Prentice Hall Pre-Algebra 8-12 Yes 2 2 1 3 3 11 McDougall Mathematics 6-8 No 2 1 2 3 3 11 Concepts and Skills Prentice Hall Focus on 6-8 No Teacher Editions Not Available at Time of Science (Life, Earth, Physical) Analysis Grades K-12 Mean Totals 1.88 1.75 1.88 3 3 11.5 Exhibit 1.2.2 shows the following: • While some core subject areas do not have adopted California standards-based textbooks and pacing guides, the district is taking steps to remedy this. Grades 6-8 English/language arts textbooks are scheduled to be adopted spring of 2004. • Prentice Hall Focus on Science (Life, Earth, Physical) textbooks were not reviewed because the teacher editions were not available at the time of analysis. • Four sets of textbook teacher editions and pacing guides reach the minimum basic adequacy score of 12 points or higher. Four of the sets analyzed do not contain enough information to provide teachers with complete and comprehensive work plans to guide their teaching. • The range of guide scores was from 10 to 13. • The strongest criteria across guides were the delineation of instructional tools and linkages for classroom use. The mean totals for each of these criteria were 3. The California standards-based textbook teacher editions provide specific examples on how to approach key concepts/skills in the classroom. The K-8 textbook teacher editions provide, as mandated by the State of California Department of Education, instructional strategies for English-language learners, students at risk in reading, and students who are ready for more challenging learning. 10 • The weakest criterion was congruity of the curriculum to testing/evaluation with a mean total of 1.75. None of the documents keyed each skill or standard to district and/or state performance evaluation. Standards should be clearly keyed to assessment on the district pacing guides. • The average curriculum guide scored 11.5 of a possible 15. 3. The district has produced several documents that begin to address the recommendations for developing, adopting, implementing, and monitoring programs and interventions that are aligned with the district’s priorities and student learning goals. a. The district’s effort to implement a results-based budget is a positive step toward aligning budget allocations with curriculum priorities so that initiatives can be sustained or terminated based solely on effects on student achievement. b. The district has designated at least three administrators from Student, Family and Community Services, the grants office and the Department of Research and Evaluation, Assessment, Data and Accountability to be responsible for programs and interventions. To ensure consistency, one district administrator should be assigned to coordinate the proposing, reviewing, adopting, implementing, and monitoring of programs and interventions throughout all departments. c. The district administrators cite the district’s data review procedures, the results- based budget, and the district professional development plans as mechanisms for a program intervention and screening process. However, there is no district- sanctioned formal process that includes the components of a screening process recommended by FCMAT. The district must develop and implement a formal districtwide screening process for all district programs. 4. The district has developed two Web site reports, the site services database being used by most intervention programs, and the grants office database that provides information throughout all externally funded programs to foster program alignment and ensure compatibility of focus as well as to facilitate program evaluation. These databases must include all the components recommended by FCMAT (see Standard 1.17). The district’s professional development plan, as described in the Instructional Blueprint and approved by the board, is an integral part of curriculum development, implementation, and assessment. District executive directors must communicate expectations to their principals that curriculum will be implemented, and curriculum delivery will be monitored. Standard Implemented: Partially January 2000 Rating: 2 September 2004 Rating: 4 March 2004 Self Rating 6 March 2004 New Rating: 5 11 Implementation Scale: 12 1.3 Allocation of Resources ________________________ Professional Standard: The district directs its resources fairly and consistently to accomplish its objectives. Progress on Implementing the Recommendations of the Recovery Plan 1. The district’s draft Budget Planning Manual 2004-2005 and the documents that comprise the Results-Based Budgeting plans require budget requests to be developed through analyses of instructional and support areas. 2. The documents previously cited describe the district’s programmatic budgeting process. a. There were no budgetary processes or criteria for evaluating the consequences of funding or not funding a program. b. Budget packages are built within each activity or program to provide for incremental delivery of goods and services to meet objectives. Directions to budget managers require them to configure their budgets at 100 percent and 75 percent of service delivery requirements. c. Broad participation in budget development is achieved through solicitation of community comment and through committees at school and district levels. d. Budget instructions require program leaders to prepare goal and outcome statements expressing the purpose of the program. Priority outcomes for the district are expressed as “key results.” e. The district has not addressed the recommendation that the staff gather data describing the cost and benefits of various programs and service levels. This is a critical omission, given the financial constraints imposed on the district. Since difficult decisions will be required, cost-benefit data are essential to support sound program choices. f. Guidelines for budget developers are contained in the draft Budget Planning Manual 2004-2005. g. The draft Budget Planning Manual 2004-2005 contained no instructions for ranking (or using) budget worksheets to describe individual programs. This recommendation was designed to help prioritize competing programs and service levels. h. The draft Budget Planning Manual 2004-2005 describes how historic income and expenditure data were used to build dollar allocation formulas. i. The draft Budget Planning Manual 2004-2005 and budgeting plans require that budgets be developed using projected revenues based on formula allocations. 13 They also describe the budget prioritization responsibilities of principals and the central office staff. j. Budgeting plans and the draft Budget Planning Manual 2004-2005 identify budget development activities and the associated time lines. These documents make it clear that budget development must support a decision-making process driven by student performance data and focused on eliminating resource inequities that inhibit student achievement. 3. Under the district’s new Results-Based Budgeting (RBB) formulas, the dollars per pupil allocated to schools are consistent and visible, as is the rationale for those allocations. Allocation formulas have been incorporated into decision-making in to identify and address resource inequities. If executed as planned, RBB can allocate financial resources more consistently, transparently, and equitably than in the past. However, to honor employee contracts and prevent unacceptable disruption of programs, inequities involving the distribution of the senior teaching staff and associated funding will be phased out. Therefore, some inequities will persist until this process is completed. 4. Dollar allocation formulas have been revised to support the provisions of Policy 1050: Philosophy and Goals of the Oakland Unified School District. Formulas are explained in the draft Budget Planning Manual 2004-2005. 5. The district leadership has not issued a directive specifically requiring the staff to comply with the deadlines of the Voluntary Resolution Plan. 6. Teachers on special assignment (TSAs) from the Office of Accountability regularly monitor compliance with the terms of the Voluntary Resolution Plan and other mandates. However, there is no clear link between compliance and noncompliance with deadlines and personnel evaluation results. 7. The recommendation to prohibit school-based decisions that cause inequities in course offerings, materials, and practices was not addressed directly. The recommendations were addressed indirectly through the formulas for allocating funds and equity goals contained in the “key results.” It remains to be seen whether the targeted inequities can be eliminated through these mechanisms. It should be noted that the district is in its first months (beginning phase) of attempting to implement Results-Based Budgeting. Principals were being trained for their expanded budgeting role and were developing their first Results-Based Budget. Budget guidance was in draft form. 14 Standard Implemented: Partially January 2000 Rating 0 September 2003 Rating: 2 March 2334 Self Rating: 5 March 2004 New Rating: 3 Implementation Scale: 15 1.4 Multiple Assessments – Program Adjustment Professional Standard: The district has adopted multiple assessment tools, including diagnostic assessments, to evaluate, improve, or adjust programs and resources. Progress on Implementing the Recommendations of the Recovery Plan: 1. An Assessment, Data and Accountability Plan, adopted as policy in December 2003, defines the district’s comprehensive assessment plan. The district is in the middle of a five-year process to build a comprehensive assessment, data and accountability system begun in 2000-2001 and designed to culminate in June 2005. The district remains deficient in four of the 20 characteristics of a comprehensive student assessment program (see Exhibit 1.4.1) a. Assessment is evidently not occurring at all levels of the system, even though significant progress has occurred. There are no districtwide formal processes to assess the organization’s effectiveness, its programs or interventions. Informal surveys, conversational feedback, and student assessment results are used, but without specific data correlation, comparatives or follow-up that is used over time. b. Although the district is providing assessment data to school sites to determine grade level and individual student achievement, assessments are neither analyzed nor used to determine curriculum and instruction effectiveness. Information is fragmented. The district lacks comparative school data by cohorts for specific skills, over time, which would show patterns, trends, and correlations to curriculum and instruction. c. Some progress has been made to align assessments to the curriculum. The district has formative assessment for student achievement associated with the state- adopted textbooks Open Court Reading (K-5), Harcourt Brace Mathematics (K- 5), High Point (6-12), and Prentice Hall, and McDougal-Littell Mathematics (6- 9). Alignment at the secondary schools is inconclusive, although math and science assessments are administered during the year. Assessments are not provided for all required subjects at all grade levels. d. The district lacks an ongoing evaluation of its Assessment, Data and Accountability Plan. Regulations for implementing the plan are missing. The plan does not mention how the assessment systems will be measured for effectiveness, who will be responsible for accessing the information, what procedures exist for accessing information, how often this will occur, or what will happen as a result (student outcomes/district goals). The following table summarizes the district’s progress toward the development of a comprehensive program and student assessment plan 16 Exhibit 1.4.1: Characteristics of a Comprehensive Program and Student Assessment Plan And FCMAT’s Assessment of District Plan Oakland Unified School District February 2004 FCMAT Rating Characteristic Adequate Inadequate 1. Describe the philosophical framework for the design of the program and students assessment plan (formative, X alignment, all subjects all grades, link to mission). 2. Gives appropriate direction through policy and X administrative regulations. 3. Provides ongoing needs assessment to establish goals of student assessment and program assessment. X 4. Provides for assessment at all levels of the system (organization, program, student). X 5. Identifies the multipurposes of assessment, types of assessments, appropriate data sources. X 6. Provides a matrix of assessment tools, purpose, subjects, type of student tested, time lines, etc. X 7. Controls for bias, culture, etc. X 8. Specifies the roles and responsibilities of the board, central office staff, and school-based staff. X 9. Directs the relations between district and state assessments. X 10. Specifies overall assessment procedures to determine curriculum effectiveness and specifications for analysis. X 11. Directs the feedback process; assures proper use of data. X 12. Specifies how assessment tools will be placed in X curriculum guides. 13. Specifies equity issues and data sources. X 14. Identifies the parameters of a program assessment. X 15. Provides ongoing training plan for various audiences on X assessment. 16. Presents procedures for monitoring assessment design and X use. 17. Establishes a communication plan for the process of student and program assessment. X 18. Provides ongoing evaluation of the assessment plan. X 19. Specifies facility and housing requirements. X 20. Describes budget ramifications, connections to resource X allocations. 2. The effort to train principals and teachers is still evolving. All principals continue to be trained in the use of data and assessment with assistance from the executive directors and follow-up support provided by data coaches and instructional facilitators. There is no indication that the training improves classroom instruction. 17 Standard Implemented: Partially January 2000 Rating: 2 September 2003 Rating: 5 March 2004 Self Rating: 6 March 2004 Rating: 5 Implementation Scale: 18 1.5 Preparation of Students—Expectations for Practices Professional Standard: Expectations and a practice exist to improve the preparation of students and to build a school structure that has the capacity to serve all students. Progress on Implementing the Recommendations of the Recovery Plan 1. Principals, assistant principals, and other educational administrators have been assigned to focus on maintaining and monitoring high quality curriculum and instruction through professional development on text adoptions and strategies for implementation, walk-through classroom visits with district personnel, collaboration among teachers, instructional facilitators, and principals, key results data dissemination, academic conferences, and administrative evaluation processes. This focus is on moving district administrators toward higher expectations, but this emphasis is not comprehensive or supported by a complete curriculum management plan. 2. The principals’ administrative duties have not been officially reduced to provide them with time to spend half the day in classrooms. Some extra support for compliance reporting, data analysis, and establishing the Single Plan for Student Achievement has been provided through network support teams. However, additional duties have been added with such initiatives as Results-Based Budgeting and mandatory redesign with school closures. 3. Training for administrators, instructional facilitators, and teachers on special assignment (TSAs) has focused on the following: coaching the delivery of instruction for the implementation of textbook adoptions, interpretation and instructional response to student performance data, professional development of teachers, and the design of student achievement goals and action steps in the site Single Plan for Student Achievement. 4. Communications to the district staff and the community regarding the district’s expectation for a high quality curriculum and instructional program have been accomplished through press releases and the Web site. There is no indication that all affected parties are involved and are committed to accomplishing the expectations established in recent plans adopted by the board related to student assessment and professional development. The district should establish processes to involve all affected parties in planning including the instructional and classified staffs, parents and community representatives so that everyone accepts that only high-quality curriculum and instruction are acceptable from teachers and educational administrators. 5. The Superintendent/State Administrator evidently do not provide regular reports to the board on planning activities and their results as they pertain to the attainment of district goals and objectives established in the 5-Year Strategic Alignment Plan. 19 6. Some training has been provided for administrators and the instructional staff pertaining to the development of the site Single Plan for Student Achievement. This process is not comprehensive since it does not adequately address the 11 critical components of an effective planning system as addressed in the Curriculum Management Audit Report (2000). • The district has not adopted a planning process that meets the 11 criteria. The 5-Year Strategic Alignment Plan and the Instructional Framework operate in parallel, but not as integrated plans with a singular focus on student achievement. • Reports to the board on planning activities related to the newly designed Instructional Framework have been scheduled from November 2003 to April 2004 through presentation of the Instructional Framework and its components: assessment, professional development, Results-Based Budgeting, equity and curriculum. • There is no linkage between this planning process and the activities to be addressed to achieve the goals and objectives of the 5-Year Strategic Alignment Plan. 7. A task force has not been established to incorporate all existing plans. District personnel indicated that a process has been adopted that involves cabinet members, instructional managers, and instructional TSAs in developing components that make up the Instructional Framework. There was no indication that this planning process will be incorporated in the 5-Year planning process. Standard Implemented: Partially January 2000 Rating: 0 September 2003 Rating: 3 March 2004 Self Rating: 5 March 2004 New Rating: 4 Implementation Scale: 20 1.6 Assessment Tools –Direction for Improvement . Professional Standard The assessment tools are clear measures of what is being taught, and they provide direction for improvement. Progress on Implementing the Recommendations of the Recovery Plan 1. The district has a multiyear assessment plan and is developing assessments that are aligned with the written and taught curriculum. Assessments are being administered in language arts and mathematics, and secondary science. Those for other areas must be developed. Full implementation of the assessment system is scheduled for the 2004-2005 school year. 2. The knowledgeable use of test guides and curriculum guides (in this instance, textbooks and pacing guides) has not been incorporated into the criteria for teacher supervision and principal evaluation. The intent of this recommendation was not to generate a body of “test prep” materials. Its purpose was to emphasize, at the school level, the need to align the content, procedures, and environment of routine classroom assessments with those of high-stakes tests, so that students are better prepared for these tests. Standard Implemented: Partially January 2000 Rating: 0 September 2003 Rating 1 March 2004 Self Rating: 4 March 2004 New Rating: 3 Implementation Scale: 21 1.7 Staff Development – Improved Instruction/Curriculum . Professional Standard Staff development provides staff with the knowledge and the skills to improve instruction and the curriculum. Progress on Implementing the Recommendations of the Recovery Plan 1. The district’s staff development policy includes only staff members involved in the design and delivery of curriculum. a. The policy includes the expectation that any school-based staff development will be aligned with district goals and priorities. b. There is no policy requirement for the periodic evaluation of staff development by analyzing quantitative and qualitative data from a variety of sources. The district has no procedure to determine whether the following are linked: the training a teacher receives, the correct application of that training in the classroom, and the impact of the application on student performance. c. The policy contains no requirement that evaluations of staff development progress be periodically reported to the State Administrator. d. It is unclear whether there are adequate resources to fund the staff development plan because there were no overall cost estimates for the strategies identified in the document. 2. The district staff development plan is essentially a concept paper. It lacks many of the requisite elements of a plan, such as a mission statement, planning and budgeting time line relationships, and action plans identifying strategies, strategy evaluation criteria, cost estimates, and those responsible for strategy execution. The training of new staff members was addressed briefly, but specific strategies to keep them updated in areas of interest to the district were not provided. Training administrators to supervise teachers in applying the skills acquired through staff development received some elaboration, but did not qualify as an adequate plan. 3. The staff development plan does not identify which staff members are responsible for various functions. a. The district has adopted the staff development plan, but the plan has not been communicated to all work sites. The plan specified how staff development would be coordinated across organizational elements. b. The district Web site lacks a current staff development calendar. c. Recommendations 3c through 3e are being addressed in varying degrees. As stated previously, with the exception of certain specific programs (e.g., High Point Reading), there is no formal link between teacher staff 22 development and the evaluation of teacher performance and student achievement. It should be noted that the district is in the initial stages of developing procedures, personnel requirements, budgets, and courses to implement the plan. Standard Implemented: Partially January 2000 Rating: 2 September 2003 Rating 3 March 2004 Self Rating: 6 March 2004 New Rating: 3 Implementation Scale: 23 1.8 Staff Development – Purpose, Goals, and Evaluation . Professional Standard Staff development demonstrates a clear understanding of purpose, written goals, and appropriate evaluations. Progress on Implementing the Recommendations of the Improvement Plan 1. The district has a carefully designed concept for development of staff members who have curriculum responsibilities. In that respect, the concept is well aligned with the district’s purposes and missions. The program to support that concept is in the initial stages of development. However, the concept is not comprehensive because it does not address staff development for all employees, but only those involved in curriculum design and delivery. 2. Teacher professional development is focused on acquiring basic teaching skills and providing quality instruction to all students. 3. Staff development documents contain no provisions evaluating program offerings to determine whether they were having the intended effect on student achievement. 4. There are no provisions in the program to examine the link between staff development and classroom changes that improve student achievement. Standard Implemented: Partially January 2000 Rating: 1 September 2003 Rating: 2 March 2004 Self Rating: 5 March 2004 New Rating: 3 Implementation Scale: 24 1.9 Evaluations – Improving Job Performance Professional Standard: The assessment tools are clear measures of what is being taught, and they provide direction for improvement. Progress on Implementing the Recommendations of the Recovery Plan: 1. The district personnel evaluation forms for certificated teachers and administrators have not been revised since August/September 2000. Although expectations for principal and teacher performance may be implied through the staff development plan, the use of Bins and Binders, and the accountability Web site, there are no formal procedures consistently used throughout the district about the use of assessment tools and improving job performance through the teacher and administrator evaluation processes. Little progress has been made to revise the teacher or administrator evaluation system by providing direction for improvement through the effective use of assessments aligned to the curriculum. a. There is little indication that executive directors in each network are providing the same kind of consistent and systematic feedback to principals. Nothing in writing explains the kind of feedback instruments that will be used, when this will happen, or who is responsible. Starting this year, some (not all) of the K-5 schools are using the Collaborative Inquiry Cycle with Open Court, and have started to use a process that includes recording forms for “Formal Site Visits” and “Grade Level Learning Team Meetings.” b. The verbiage “the knowledgeable use of test data, test guides, and curriculum guides,” which includes the use of pacing guides with standards-based textbooks, has not been formalized and added to the criteria for teacher supervision or administrator evaluations. 2. Principals, one school site to another, are not consistently and systematically providing the same kind of feedback to classroom teachers about the expected use of data and assessments. The district lacks written procedures describing how administrators are expected to use the accountability Web site or whether this will become a part of their performance evaluations Standard Implemented: Partially January 2000 Rating: 0 September 2003 Rating: 1 March 2004 Self Rating: 4 January 2004 Rating: 2 Implementation Scale: 25 1.10 Variety of Instructional Strategies– Student Diversity Professional Standard Teachers use a variety of instructional strategies and resources that address their students’ diverse needs. Progress on Implementing the Recommendations of the Recovery Plan 1. The board has approved a new professional development plan as part of the Instructional Blueprint and Framework (see Standards 1.7 and 1.8). Several opportunities for district-mandated staff development as well as site-based training have been made available to teachers at all levels. Some of these staff development activities include: High Point, Secondary ELD, Differentiating Curriculum, data analysis for grouping students based on skill need for instruction, and AVID. The district must develop a process assessing the effectiveness of these staff development opportunities. The district should follow up to ensure implementation and sustainability of varied instructional strategies that address the diverse needs of students in all classrooms. The district has budgeted resources and begun to focus on assessment feedback discrepancies as evidenced in the Collaborative Inquiry Cycle for grade level/learning teams at many schools. Steps must be taken to ensure that assessment feedback discrepancies are systematically addressed at every school. Standard Implemented: Partially January 2000 Rating: 0 September 2004 Rating: 1 March 2004 Self Rating: 4 March 2004 New Rating: 3 Implementation Scale: 26 1.12 California Standards for the Teaching Professional Professional Standard The standards set forth in the California Standards for the Teaching Profession are present and supported (California commission on Teacher Credentialing and California Department of Education, July, 1997). Progress on Implementing the Recommendations of the Recovery Plan 1. Although the district has adopted most practices and programs consistent with the California Standards for the Teaching Profession, policies supporting the California Standards for the Teaching Profession apparently have not been adopted by the board. Standard Implemented: Partially January 2000 Rating: 1 September 2003 Rating 5 March 2004 Self Rating 5 March 2004 New Rating: 5 Implementation Scale: 27 1.13 Instructional Plans – Modification and Adjustment . Professional Standard Teachers modify and adjust instruction plans according to student needs and success. Progress on Implementing the Recommendations of the Recovery Plan 1. The district staff has initiated numerous activities designed to achieve equity, but there is no policy to guide these efforts. There is a continuing need for an equity policy that defines the term “equity,” provides guidance for the staff, and specifies the data required by the state Administrator to monitor equity. An equity plan is scheduled to go to the board for consideration in March 2004. 2. Teachers and administrators reported a variety of instructional modifications based on student performance data. Those modifications included after- school/weekend tutoring, homework assistance, revising students’ schedules for more exposure to core curriculum, and the use of computer-assisted learning. 3. The Results-Based Budgeting plan addresses equity through new funding allocation formulas for school budget development. Equity strategies are also discussed, in varying degrees of detail, in the following plans: assessment, staff development, and the English Language Learner Master Plan. 4. No district regulations specified the staff’s equity roles and responsibilities. 5. The district leadership made a policy decision to use state-approved textbooks and pacing guides as curriculum guides. Those documents had been integrated into some, but not all courses. Where the approved textbooks were being used, they complied with the FCMAT recommendation to include a variety of strategies for teaching course objectives. A full assessment of the district’s efforts in this area cannot be completed until textbooks have been selected for all courses. (See Standard 1.2 for a detailed analysis of textbooks in use.) 6. The assessment plan requires the use of data to identify equity issues related to student performance. However, the assessment system generates student performance data for core courses only. A complete array of data will not be available until the system is fully operational in 2004-2005. Standard Implemented: Partially January 2000 Rating: 0 September 2003 Rating: 2 March 2004 Self Rating: 4 March 2004 New Rating: 3 Implementation Scale: 28 1.14 Learning Goals and Instructional Goals Professional Standard: There is evidence that learning goals and instructional plans and programs are challenging for all students. Progress on Implementing the Recommendations of the Recovery Plan: 1. The district Curriculum Management Plan is to be presented to the board in April 2004. That plan should include all components recommended by FCMAT. In addition, the plan should clearly communicate expectations that the curriculum be challenging for students at all levels and that the approach to curriculum development include a comprehensive needs assessment, a systematic review of each instructional program every 4 to 6 years, the development and revision of curriculum pacing guides, and the adoption of all curriculum (California standards-based textbooks) by the board (see Standard 1.2 above). The review team visited 24 schools (selected by the district Associate Superintendent of Instruction as a cross-section of the district) to assess whether the district-adopted California standards-based textbooks and pacing guides are being used by the teachers to teach the challenging California English/language arts and mathematics standards for the grade level. Exhibit 1.14.1 shows the results of the data collected during these visits. Exhibit 1.14.1 -- Results of Observations -- Oakland Unified School District -- January 2004 Calibration (Alignment) of Instruction to the California Standards CA Standards- Instruction Instruction Instruction Instruction Based Above At Below Not Classrooms # of ELA Textbook Grade Grade Grade California Where Students Grade API # of and Math Present Level Level Level Standard Were Off Task Level Color Schools Classes # % # % # % # % # % # % Span Code Visited Observed K-5 Blue 1 7 4 57% 0 0% 4 57% 0 0% 3 43% 0 0% Green 2 22 16 73% 1 5% 16 73% 4 18% 1 5% 0 0% Yellow 4 50 45 90% 7 14% 35 70% 8 16% 0 0% 0 0% Orange 3 27 13 48% 4 15% 5 19% 3 11% 5 19% 1 .04% Subtotals 10 106 78 74% 12 11% 60 57% 15 14% 9 8% 1 .01% 6-8 Green 1 10 3 30% 0 0% 5 50% 5 50% 0 0% 1 10% Yellow 3 36 15 42% 3 8% 16 44% 12 33% 5 14% 0 0% Orange 3 16 9 56% 1 6% 8 50% 7 44% 0 0% 0 0% Subtotals 7 62 27 44% 4 6% 29 47% 24 39% 5 8% 1 .07% 9-12 Yellow 2 23 5 22% 0 0% 9 39% 11 48% 3 13% 0 0% Orange 2 10 0 0% 0 0% 8 80% 2 20% 0 0% 3 30% No 3 10 4 40% 0 0% 6 60% 3 30% 1 10% 0 0% Code Subtotals 7 43 9 21% 0 0% 23 53% 16 37% 4 9% 3 .07% 24 211 114 54% 16 8% 112 53% 55 26% 18 9% 5 .02% Totals Exhibit 1.14.1 illustrates the following: • FCMAT observed California English/language arts and mathematics standards- based textbooks in 74 percent of the elementary school classrooms visited. California standards-based textbooks were evidenced in 90 percent of classrooms in elementary schools coded as yellow. Forty-four percent of the middle school classrooms and 21 percent of the high school classrooms visited used California standards-based textbooks. The discrepancy between elementary and secondary 29 schools can partially be attributed to the fact that only High Point, Algebra 1 and pre-algebra California standards-based textbooks have been adopted by the district for grades 6-12 at this time. Grades 6-12 English/language arts and some math courses do not have California standards-based texts. District adoption of California standards-based textbooks for all grade levels and core subject courses must be a priority. • When FCMAT compared English/language arts and mathematic instruction in district classrooms to the California standards, 53 percent of the instruction observed was at grade level, 26 percent below grade level, 8 percent above grade level, and 9 percent was not on a standard. The highest percentage of instruction at grade level (70 percent) occurred in classrooms of elementary schools coded yellow. The highest percentage of instruction considered below grade level (50 percent) occurred in classrooms of middle schools coded green with classrooms in high schools coded yellow having 48 percent instruction below grade level. The district should require administrators at all schools to monitor the level of instruction at all grade levels and facilitate opportunities for teachers to gain the expertise needed in adjusting instruction to ensure challenging curriculum for all students. 2. District teachers were asked how they use the district pacing guides. Many responded that they follow the pacing guide strictly to prepare the students for the district and state assessments. Others stated that they used the guides as suggested pacing and could vary it within the time lines of the district assessments. At the secondary level, there was some confusion as to the use of the term “pacing guide” and many called their pacing guide document a “scope and sequence.” Administrators were asked how they monitor the use of pacing guides, and they gave various answers such as analysis of test data, classroom observations, grade-level meeting discussions and requirements for teachers’ lesson plans to align with the pacing guides. 3. The district must continue to ensure that pacing guides are developed and implemented consistently for all core subject areas at all grade levels and that these pacing guides are revised often to reflect changing student learning needs. Standard Implemented: Partially January 2000 Rating: 0 September 2003 Rating: 2 March 2004 Self Rating: 3 March 2004 New Rating: 3 Implementation Scale: 30 1.15 Utilization of Assessment Information Professional Standard: The administration and staff utilize assessment information to improve learning opportunities for all students. Progress on Implementing the Recommendations of the Recovery Plan: 1. The board has not adopted policies to establish a framework for a comprehensive program of assessment. 2. The district continues to make progress in utilizing student assessment to improve instruction. The district’s Web site and in particular, the data portal, is in its first year of full implementation. The district has adopted an Assessment, Data and Accountability Plan as a part of the Instructional Blueprint Development Plan. Assessment and data are starting to be used by more principals and teachers “to help teachers find patterns of strengths and weakness in their teaching” (see Standard 1.4). More benchmark assessment data is accessible to school sites with support from the central office. Some schools are using data more readily than others. Not all sites access the information regularly. Those sites whose staff are downloading data and including it in regularly scheduled circuit meetings at the elementary level or department meetings at the secondary level are beginning to use data to help drive instruction. Executive directors have taken a more active role in helping principals use the data. The Department of Research and Evaluation provides support to schools through the use of data coaches and the posting of reports and assessment data that are maintained on the district accountability Web site. The data rosters, provided twice a year, are useful for overall test results with CELDT, CST and Open Court Reading benchmarks at grades 2-5. More detailed data is beginning to become more accessible for Open Court Reading, High Point, and CELDT. Assessments for ELD, the high school social studies and middle school science are beginning to be developed. By 2004-2005, the district intends to provide more detailed CST results, extensive rosters and summary reports. The data reports are in the beginning stages of showing longitudinal and comparative information that provides meaning to the data. 3. Significant progress has been made in upgrading administrator and teacher training in the use of assessment information. A 5-day training session is mandatory for all teachers and administrators on the use of the standards-based reading, language arts and math programs. Completion is required fall 2005. Schools are expected to follow up the professional development by reviewing the data and the standards-based programs. Executive directors and principals are receiving training in the review of formative and summative data. The district has divided the schools into four color bands based on their 2003 API (see Standard 2.a) to provide continuing professional development and site or district coaching. Academic conferences that include executive directors and principals, in a pilot stage, are expected to reinforce the training in the use of assessment data. 31 Standard Implemented: Partially January 2000 Rating: 0 September 2003 Rating: 2 March 2004 Self Rating: 6 March 2004 Rating: 4 Implementation Scale: 32 1.17 Goals and Grade-Level Performance Standards Professional Standard Goals and grade-level performance standards based on a common vision are present. Progress on Implementing the Recommendations of the Recovery Plan 1. The district utilizes data reporting protocols using color-coded rankings (red, orange, yellow, green, and blue) that provide uniformity of reporting. The Associate Superintendent of Instruction reviews the programs with the Governing Board, including a cost analysis, a description of how particular programs interact with other programs, and data on program effectiveness at the same time the components of the district Instructional Blueprint are presented. a. Appropriate funding has been provided. b. Ample program data are available. c. The reporting structure for data Web site reports and for benchmark assessment reports has been standardized. d. The district site services database is being used by most intervention programs with a move to programwide implementation to provide uniform data so that program effectiveness can be determined. The grants office database provides information across all externally funded programs to foster program alignment and ensure compatibility of focus as well as to facilitate program evaluation. However, the district must ensure that these databases include all the components of a database of supplemental programs recommended by FCMAT including: • The name of the program • The purpose and the district goals it supports • The number of students directly served • The funding source • The funds available • The allocation of funds within program • The costs of program per student • The expected stability of funding • The degree of learning achieved (or not achieved) • The resulting action for improvement, including program termination, if appropriate. Use of a consistent data protocol across programs will allow for more meaningful comparisons. 33 Standard Implemented: Partially January 2000 Rating: 0 September 2003 Rating: 3 March 2004 Self Rating: 4 March 2004 New Rating: 4 Implementation Scale: 34 1.21 Professional Development Linked to Evaluation Professional Standard Professional development is linked to personnel evaluation. Progress on Implementing the Recommendations of the Improvement Plan: 1. Professional development is not connected to staff appraisals in a consistent or systematic way that would result in improved student achievement. The evaluation form for certificated staff includes the phrase, “Developing as a professional educator.” A descriptor for the evaluation standard includes the phrase, “Reflecting on teaching practice and planning professional development.” Professional development is not formally tied to the district’s goal of improved student achievement through the evaluation process. The administrative Employee Accountability Appraisal Worksheet for Personnel and Professional Growth Management contains the following criteria: “Encourages participation in professional growth opportunities for self and staff,” and “Encourages staff participation in site’s improvement planning, professional growth opportunities and in student extra curricular activities (if applicable).” There is no indication that the effectiveness of a principal or teacher for improved instruction is a result of professional development. 2. The district has a Professional Development Plan that was approved on January 28, 2004. This is the first step in formalizing the linkage of staff development to evaluation. The Bins and Binders process requires that attendance at site professional development workshops be documented. This provides principals with data to include in the evaluation process. Based on the Professional Development Plan, principals and teachers are required to attend a 5-day training session tied to the adopted standards-based textbooks in key curriculum areas, and in differentiated instruction to make the curriculum accessible to all students. The district plans to provide differing levels of opportunities for professional development depending on the academic success of the school. The district is in its first year of providing beginning-of-the- year and monthly Network professional development workshops for principals and other people in leadership roles. Intensive support is planned for administrators at “orange” schools and AB75 is mandatory for the HPSG/II-USP administrators. Support for mentor principals coaching new administrators is planned for the start of the Oakland Leadership Academy in March 2004. 3. Beginning steps have been taken to link professional development to the evaluation process. Informally, executive directors are told that they are expected to evaluate principals’ participation at Network professional development workshops. District officials indicated that changes in the personnel evaluation documents are being incorporated into new negotiations with the bargaining units for administrators and teachers. The district states, “By 2007, the Oakland Unified School District will have in place a comprehensive professional development program for teachers and administrators as an articulated part of the district Instructional Blueprint.” Currently, 35 the district has not formalized the linkage between professional development and the employee appraisal process. Standard Implemented: Partially January 2000 Rating: 0 September 2003 Rating: 0 March 2004 Self Rating: 2 March 2004 New Rating: 2 Implementation Scale: 36 1.23 Initial Student Placement—Procedures Professional Standard Initial placement procedures are in place to ensure the timely and appropriate placement of all students with particular emphases being placed on students with special needs. Progress on Implementing the Recommendations of the Recovery Plan 1. Some progress has been made in implementing board policy on educational equity. The district has initiated a Results-Based Budget process in an effort to address resource and equity issues identified in the Voluntary Resolution. Initial stages of this new budget process were identified by the district as addressing differences in school size, allocations, etc. The new process was attempting to move away from only using an ADA process. 2. A mandated core curriculum (the district Curriculum Framework) and professional development (district Professional Development Plan) are being used to address inequity to the core curriculum and to address student achievement gaps and gains for sub populations. 3. The compliance office monitors the sites on course offerings and inequities through teachers on assignment (TSA) and a Bins and Binders process. A Master Plan for English Language Learners, adopted by the Governing Board and accepted by the Office of Civil Rights and Comité unit of the California Department of Education, addresses a component on student placement. Documentation of a process was demonstrated. Implementation in the actual buildings was still not evidenced. A Master Equity Plan was not presented at this time. Standard Implemented: Partially January 2000 Rating: 1 September 2003 Rating: 3 March 2004 Self Rating: 4 March 2004 New Rating: 4 Implementation Scale: 37 1.25 Instructional Materials – Student Accessibility Professional Standard The district will ensure that all instructional materials are accessible to all students. Progress on Implementing the Recommendations of the Recovery Plan 1. The district plans to adopt California standards-based English/language arts textbooks for grades 6-12 in spring, 2004, further ensuring that California standards-based core materials are accessible to all students. The district must make adoption of California standards-based textbooks and development of pacing guides a priority given the district philosophy supporting California standards-based textbooks and pacing guides as the district curriculum (see Standard 1.2).The district has just hired a teacher on special assignment to coordinate textbook purchases and institute a textbook recovery plan. The goal of this position is to have all textbooks purchased and delivered to sites prior to the end of the year so that any shortages can be identified in time for the start of the New Year. 2. Textbook purchases have provided for students with limited English proficiency. 3. Most teachers interviewed during site visits agreed that they had enough core subject area textbooks for all their students to take home. Standard Implemented: Partially January 2000 Rating: 5 September 2003 6 March 2004 Self Rating: 6 March 2004 New Rating: 6 Implementation Scale: 38 1.27 Planning Process---Focus and Connectivity Professional Standard The district planning process ensures focus and connectivity to increased student productivity. Progress on Implementing the Recommendations of the Recovery Plan 1. The board does not receive progress reports on planning activities and their results pertaining to the attainment of district goals and objectives established in the 5- Year Strategic Alignment Plan. Reports to the board that are scheduled relate to the Instructional Framework which is a newly developed set of plans that are not explicitly aligned to the goals and objectives of the 5-Year Strategic Alignment Plan. The reports on planning within the Instructional Framework show a focus on student achievement: Instructional Framework (November), Assessment, Data, Accountability Plan (December), Professional Development (January), Results- Based Budgeting Plan (January), Equity Plan (March), Curriculum Plan (April). 2. A planning process has not been developed that meets all the 11 criteria in the Curriculum Management Audit Report (2000). The 5-Year Strategic Alignment Plan meets several of the criteria, but does not address evaluation processes for action steps, plan integration with the newly developed plans within the Instructional Framework, monitoring of the plan’s effectiveness, or stakeholder commitment. The plans developed within the Instructional Framework are missing specific action plans and the same elements discussed previously. The district staff needs to develop a planning process that provides focus and connectivity to increase student productivity, and meets all the 11 criteria in the Curriculum Management Audit report (2000). 3. Some training has been provided for administrators and instructional staff pertaining to the development of the site Single Plan for Student Achievement. This process addresses some components of an effective planning system: analyzing data and setting goals for student achievement. This process is not comprehensive because it does not adequately address all the critical components of an effective planning system, or the capacity to assess the needs of the school as a whole. The district staff needs to provide training for all administrators and key instructional staff members pertaining to: • Understanding and adhering to the critical components of an effective planning system. • Building the capacity to address effectively the components of planning as they assess the needs of their individual sites, departments and the district. • Setting realistic goals and performance-based objectives. 39 4. A task force has not been established to incorporate all existing plans into one long- range district plan. The district should appoint a task force to incorporate all existing plans, grants, and endeavors into one long-range district plan. The district also should ascertain that this plan matches and meets the mission established in the 5-Year Strategic Alignment Plan and should align the Instructional Framework to achieve the goals and objectives of plan. Any practices that conflict with or drain resources from the districts’ mission and strategic goals should be eliminated. Standard Implemented: Partially January 2000 Rating: 0 September 2003: 3 March 2004 Self Rating: 5 March 2004 New Rating: 3 Implementation Scale: 40 1.28 Human Resources Practices Professional Standard Human resources practices support the delivery of sound educational programs. Progress on Implementing the Recommendations of the Recovery Plan 1. A board task force of district stakeholders has not been composed to draft strategic goal statements to address teacher absenteeism, turnover rate, and credentials. A new Assistant Superintendent for Human Resources is analyzing these issues and establishing systems to address achievement of these goals. Some steps have been taken to track teacher credentials and teacher absenteeism through a database. Reports on teacher absenteeism and credentials are made regularly to the cabinet and communicated to site managers. 2. Goals and objectives have not yet been established for the Human Resources Division that address teacher absenteeism, turnover rates, and credentials. Steps to address absenteeism are in the Results-Based Budgeting approach, where sites take on greater accountability for the relationship between absenteeism and site expenditures. Efforts to manage credentials are evident in the increased percentage of teachers who hold appropriate credentials for their positions. The Professional Development Plan within the Instructional Framework has established a goal to have all teachers meet the Highly Qualified Teacher requirements established by the No Child Left Behind Act (2001). District personnel acknowledged that since there is no current database to track turnover rates, a system would need to be developed before this issue can be resolved efficiently. The district should establish goals and objectives for the Human Resource Division that address teacher absenteeism, turnover rates, and credentials, and establish systems to achieve results. 3. There is no systematic data collection, interpretation, or use of information regarding systems attitudes or climate. The new Assistant Superintendent for Human Resources articulated a need to establish such a system. The district should assign the staff to establish and require systematic data collection, interpretation, and use of information regarding issues such as utilizing systems attitude surveys, climate studies, and exit interviews. 4. There is no collective-bargaining consultant retained by the district to address contractual impediments to goals and objectives. District personnel recognize a need for such a service, but acknowledge that a complete audit needs to be completed to clean up the multitude of job classifications, evaluate work rules, and establish alignment between roles, job descriptions, and district objectives. 41 Standard Implemented: Partially January 2000 Rating: 1 September 2003 Rating: 1 March 2004 Self Rating: 2 March 2004 New Rating: 1 Implementation Scale: 42 2.3(a) Management Professional Standard A process is in place to maintain alignment between standards, practices, and assessment. Progress on Implementing the Recommendations of the Recovery Plan 1. District administrators have indicated their support for the importance of maintaining alignment between standards, practices, and assessments. There is no board policy or administrative regulations for formally establishing an expectation that district administrators and teachers will maintain alignment between standards, practices, and assessments. 2. The district administrators are developing a Curriculum Management Plan consisting of the recommended FCMAT components, and it plans to present the Curriculum Management Plan to the board in April 2004 (See Standard 1.2). 3. As an alternative to creating a fully-developed written curriculum, the district has adopted California Standards-based textbooks as the district curriculum with pacing guides developed to address shortfalls of a curriculum driven solely by textbooks (See Standard 1.2). The district should implement a process that frequently reviews and revises pacing guides to maintain alignment of curriculum, instruction and assessment in meeting the learning needs of students. All pacing guides should include clear information for teachers indicating the state standards being taught and assessed. Priority should be given to aligning district-adopted textbooks that are not California standards-based to the state standards and assessments and to providing alignment information to teachers. 4. The Associate Superintendent of Instruction periodically reports to the board on the status of the alignment between standards, practices, and assessments through the presentation of the various sections of the district Instructional Framework. As reported in Standard 1.2, the Curriculum Management Plan is scheduled to be presented to the Board in April 2004. Standard Implemented: Partially January 2000 Rating: NA September 2003 Rating: 2 March 2004 Self Rating: 4 March 2004 New Rating: 4 Implementation Scale: 43 2.6(a) Standards Professional Standard The Governing Board has adopted and the district is implementing the California State Standards and assessments. Progress on Implementing the Recommendations of the Recovery Plan 1. The district lacks documentation that the K-5 Open Court English/language arts program was approved by the board. No documentation was available that the other California standards-based textbooks being used in district classrooms were, in fact, approved by the board (See Standard 1.2 for a listing of California standards-based textbooks purchased by the district). Plans are in place to adopt California standards- based 6-12 English language arts textbooks in spring 2004. The board must approve California standards-based textbooks for all core subject areas. 2. See Standard 1.2 for the district’s alternative to FCMAT’s recommendation to develop curriculum guides (adopting California standards-based textbooks as the district curriculum with pacing guides) and for a review of the textbooks and pacing guides based on five minimum quality criteria. Standard Implemented: Partially January 2000 Rating: NA September 2003 Rating: 2 March 2004 Self Rating: 4 March 2004 New Rating: 4 Implementation Scale: 44 3.1 English-Language Learners ________________________________________________________________________ Professional Standard The identification and placement of English-language learners into appropriate courses is conducted in a timely and effective manner. Progress on Implementing the Recommendations of the Recovery Plan 1. The district has developed and the Governing Board has adopted the English Language Learner (ELL) Master Plan. This plan was further accepted by the Comité of the Department of Education in California and by the Office for Civil Rights. This ELL Master Plan outlines the educational program for English-language learners, including their access to the core curriculum and placement procedures. The district further mandated the Curriculum Framework, which describes the district plan for providing equity and access to all multifunded students. Additional district-developed plans providing support for this recommendation include the Professional Development Plan and the Data, Assessment and Accountability Plan. The plans did not include specific strategies for equity in the schools. The overall plans are very general and action strategies for implementation were not provided. 2. The district Curriculum Framework provides for equity and access to core curriculum for multifunded students including English learners. Documentation of some professional development meetings indicated initial reviews of curriculum programs for English learners has started. A list of appropriate instructional materials for English learners was sent to the English-language teachers. However, there is no information providing direction on linkage of materials to specific English learner need. Further, a process for evaluation of ELL students for placement in appropriate courses is not well defined. 3. The district showed FCMAT copies of any updated and shared parent information regarding various student placement options and school choice. The updates were in process, but they were not fully completed. Documents were not available for all placement options. Interviews confirmed that parents lack understanding of information sent to them. 4. Counselors and teachers are not trained to help students better understand their educational options. 5. Some progress has been made regarding the use of Individual Student Profiles. Documentation and interviews confirmed the Individual Student Profile is used by Project SOAR [an after school and tutoring program]. There is no indication that these profiles are used for the regular program. 6. The district continues to use the Bins and Binders process to monitor equity in the schools as well as discuss equity at staff meetings. No focused equity checks occurred for courses and programs for the 2003-04 year. 45 7. The district has developed a Data, Assessment and Accountability Plan, which provides for monitoring the English Language Learner Program and the placement of the ELL learners in courses. However, paperwork holds up the reclassification at the secondary level. Interviews and documents substantiated that at the secondary level, English-language learners are placed in High Point two-block without assessment for placement. Parents are required to request a waiver for this not to automatically happen. Further, the English Language Learner Master Plan details the process for placing English-language learners in the core curriculum, but specific action steps were not included to implement the plan. Standard Implemented: Partially January 2000 Rating: NA September 2003 Rating: 3 March 2004 Self Rating: 4 March 2004 New Rating: 3 Implementation Scale: 46 3.2 English-Language Learners _______________________________________________________________________ Professional Standard Programs for English language learners comply with state and federal regulations and meet the quality criteria set forth by the California Department of Education. Progress on Implementing the Recommendations of the Recovery Plan 1. The district has developed a Master ELL plan accepted by the Comité with the Department of Education in California. Additionally, a Master Professional Development Plan has been developed by district leadership and adopted by the Governing Board to incorporate some aspects of the English Language Learner Program. Documents and interviews confirmed the overall plans are general, lacking specific action steps for implementing actual strategies for accessing the core curriculum. 2. A system for monitoring the overall aspects of the general district program is beginning; however this monitoring does not review articulation and continuous progress K-12. The data system in place is not used in a programmatic fashion for ELL. 3. The district leadership has developed a Results-Based Budget process that has been presented to leadership, but has not been implemented at this time. This process is designed to address equity and resource issues in the district. 4. Documentation, classroom observations, and interviews confirmed the lack of differentiation of instruction in the classroom to meet ELL needs. Training documents confirmed it was a topic on agenda meetings, but implementation could not be confirmed. 5. A monitoring process is used to review program compliance. These same monitoring documents indicate that there is minimal attention given to quality and time devoted to ELL issues and follow-up. Training agenda list this as a topic, but it could not be substantiated that this is a focus during classroom observations. 6. The district Curriculum Plan, the Professional Development Plan, and the ELL Master Plan specify ELL programs be supportive of the core curriculum. The district leadership identified High Point for secondary ELL curriculum and Open Court for Elementary ELL curriculum as aligned with the core curriculum. Training has been provided for these adopted series, as well as monthly informational meetings offered. A list of identified aligned ELL materials to use with the language arts has been provided to the teaching staff. Only the area of language arts has been thoroughly addressed. The district evidently has no core content areas for ELL materials. Further, there is no indication that the existing materials were selected and aligned with the needs of the EL students. There is no differentiation within these materials. 7. Training has been offered to the staff on the two adopted curriculum series, High Point and Foro Albierto. The agenda of professional trainings provide little indication 47 that additional methods of instruction are being discussed or introduced to address achievement gap issues between EL and general educational learners. Limited feedback occurs from limited monitoring visits to classroom teachers 8. Some new documents have been developed as a part of the ELL master plan. Site Council meetings and Parent Advisory Meetings are for providing information rather than for solving problems and making decisions together. Few parents are involved in the meetings. No date supports improved or increased parent understanding of curriculum programs and services. 9. The Results-Based Budget process addresses resource allocation, but provides minimal use in direct support of ELL access to the general curriculum. There is no indication that the monitoring process links use of dollars with needs of English- language learners and access to core curriculum, other than through the two adopted district language arts programs, High Point and Foro Albierto. 10. Site Council Meetings and parent advisory meetings are held, but the agenda indicate that there is presentation of information as opposed to shared decision making. The agenda of professional development training sessions addressed electing ELAC committees and holding meetings, but did not provide staff skill training in partnering with parents in shared decision-making. 11. Some training sessions have been offered on cultural awareness and understanding. This was not done in a comprehensive manner. No additional training has occurred. 12. A master plan to address the monitoring of multifunded students is being developed. Monitoring is occurring on a broad level, reviewing placement issues rather than curriculum issues. 13. The training agenda and interviews substantiated that teacher training has begun in counseling students to better understand their educational opportunities, but needs more implementation. 14. The training agenda and interviews confirm that workshops have been offered on a limited basis on classroom management skills and instructional strategies and management to noncredentialed teachers and paraprofessionals. Legal compliance training occurs. 15. Teachers on special assignment monitor the Bins and Binders of each school on a scheduled basis. No additional monitoring has been added since September. 16. A district assessment and accountability plan includes use of data to address equity concerns, but no specific action plans address use of data to improve ELD and EL achievement. 17. The district continues to use Individual Student Profiles only for Project SOAR and other after school tutoring efforts. There is no indication that the student profile is used in making course placement decisions for students. 48 18. The district does not use data rosters for individualization of student instruction. 19. Monitoring in the first two months of the year addressed some placement issues, but did not include a formal equity check for all courses and programs. 20. The Governing Board adopted a master professional development plan, but it did not address all the differentiated instruction and compliance issues necessary to improve achievement and reduce achievement gaps for English-language learners. The ELL program has several staff members working with it separately as opposed to holistically. 21. District leadership developed and the Governing Board adopted an Assessment and Accountability Plan. There is no indication that authentic program evaluation is occurring to improve student achievement. Standard Implemented: Partially January 2000 Rating: NA September 2003 Rating: 3 March 2004 Self Rating: 4 March 2004 New Rating: 3 Implementation Scale: 49 3.3 Special Education ________________________________________________________________________ Professional Standard Individual education plans are reviewed and updated on time. Progress on Implementing the Recommendations of the Recovery Plan 1. Procedures for identifying, evaluating and placing students in the special education program have not been reviewed and updated. 2. Assessment procedures have not been reviewed and updated to address initial identification and screening and triennials for special education students. 3. Principals and some teachers have been trained on the need for maintaining updated IEPs. Updated and current IEPs are identified as an evaluation issue for principals and teachers, but do not appear on the printed evaluation document. The district procedures manual has not been revised since September, so it contains no additional procedures to maintain updated IEPs. Documents support the continued practice of outdated IEPs, especially for the December child count. 4. A consultant has been employed to assist with the review of current policies and procedures. Few policies and procedures have been reviewed. The director’s action plan includes the intention to complete this. 5. The system currently uses a paper report to notify principals and teachers that their IEPs are due each month. There is minimal change to the internal process of schools assuming independent responsibility for updated IEPs for student instructional achievement. 6. A standalone, separate special education database is utilized by the Special Education Division. It is available only to the Director of Psychological and Mental Health Services and does not include all special education data. School sites do not have access to the database. The database is updated manually by one person through a cumbersome process with reports from the schools and manual entry at the central site. A paper report is mailed to the schools monthly regarding IEPs and triennials. This report is distributed to principals, directors, special education teachers, special education teachers on special assignment, psychologists, and other monitoring personnel. The district Assessment and Accountability System Plan did not include special education data needs. The district is addressing the outdated IEPs and triennials, but the following are still due [Boldface indicates the triennials are already overdue]: Month 2000 2001 2002/03 2003/04 50 Due August 5 September 2 24 October 2 41 November 1 4 73 December 6 45 January 7 82 February 13 117 March 10 123 April 17 119 May 1 1 20 191 June 1 22 167 July 1 Totals 1 4 103 280/717 7. Voluntary training has been offered by the district to the special education staff. The training agenda include several topics with no specific training focused on the development and review of IEPs. 8. Teachers on special assignment monitor IEPs and triennials. Corrective action plans for out-of-compliance sites were not implemented. 9. Twenty special education moderate to severe positions are staffed with noncredentialed teachers. A state waiver is being sought for the district to employ the teachers in this manner. Interviews confirmed the problem continues. The director is working with human resources to develop a different recruitment plan. Noncredentialed teachers without skills to develop and review IEPs contribute to the district’s out-of-date IEP problem for the district. 10. There is no policy development or initiative to address this recommendation. 11. A district parent advisory special education committee was created in October and meets monthly. The agenda indicate the committee meets, but does not function as an advisory committee. Standard Implemented: Partially January 2000 Rating: NA September 2004 Rating: 3 March 2004 Self Rating: 4 March 2004 New Rating: 4 Implementation Scale: 51 3.4 Special Education _______________________________________________________________________ Professional Standard Programs for special education students meet the least restrictive environment provision of the law and the quality criteria and goals set forth by the California Department of Education. Progress on Implementing the Recommendations of the Recovery Plan 1. Monthly reports on IEPS and triennials are provided to each principal, special education teacher, director, and psychologist. The district lacks a process to review and monitor the transition or transfer of special education students for overdue IEPs and/or triennials. 2. The Executive Director for Exceptional Children has been directed to examine the entire program for efficiency, costs analysis, and compliance. All aspects of the Program for Exceptional Children have not been addressed because the Executive Director established the priority areas of transportation, nonpublic services, and budget to focus on since September. 3. Principals have discussed overdue IEPs and triennials. However, the principal evaluation instrument has not changed. Executive directors discuss the overdue documents with principals, but there is no indication that any actions were taken beyond a conversations. 4. The psychologists meetings discussed time and production of evaluations. Monthly agenda indicate a focus on numbers of triennials due and comparison data of individual psychologists’ production of evaluations. The district lacks a written plan to address the backlog of evaluations. No evaluation of the current process of conducting evaluations has occurred. 5. The district has made no changes regarding report-writing software, IEP software and additional clerical support. Interviews indicated a software program was being investigated, but no systematic process for identifying the programs to investigate, the evaluation of the systems, the budget necessary, or other pieces of this plan was identified. 6. The special education staff was provided with training regarding the need for updated IEPs. The district has no guidance documents regarding triennial reviews and monitoring of psychologist and school performance. The district has datasheets and lists of overdue IEPs, but no systematic process in place. The district lacks a policy and procedures for special education in light of IDEA ’97. These would revise the triennial evaluation process by allowing for a committee to determine the need for such evaluation in the IEP review. 7. The district has data that includes comparison data among the psychologists and numbers of assessments. However, there is no plan to address low performances of psychologists. Guidance is not provided to psychologists performing below the norm. 52 8. There has been discussion on an IEP and special education management system. However, there was no plan with criteria and a system for investigation. Discussions regarding the topic had occurred, but no action was taken. 9. The district leadership discusses overdue IEPs at leadership meetings. The psychologists discuss the topic at their monthly meetings. The special education staff meets and discusses the topic with schools. The district recently employed some retired special education teachers to assist noncredentialed teachers in developing overdue IEPs. 10. The Executive Director has a prioritized list of special education action plans, and an examination of total district policies is a priority. To date, an analysis regarding instructional aides has not occurred. 11. The district has made no change regarding processing students entering from group homes or outside agencies. 12. The district has a monitoring process for paper reports on outdated IEPs. The district compliance office includes special education compliance issues unless they interface with another law. The Executive Director of Special Education has employed a consultant to assist with reviewing and developing systematic processes within special education. 13. Leadership team agenda, professional development agenda, memorandum, and reports indicate that the district has a priority on compliance. However, personnel action is not documented when an employee fails to meet established performance standards regarding compliance. Personnel evaluation instruments do not reflect this recommendation. 14. A monthly review of the special education program occurs with the executive directors and the cabinet on aspects of the special education program. A total program evaluation process has not been established. 15. A special education budget and expenditure spreadsheet has been developed by the Executive Director in working with the Finance Department. Written, integrated policies and procedures regarding district resources and special education usage were not available. 16. A SELPA budget was developed and shared. Integration of all budget systems for dissemination of accurate data was not confirmed. 17. A budget for special education with expenditure codes was developed. 18. A consultant has worked with the Executive Director of Special Education to review use of private transportation and nonpublic services. A plan for nonpublic service and for private transportation has been developed and shared with stakeholders in the district. 53 19. The district has made no change regarding work with juvenile justice. 20. The development of the district Professional Development Plan, Assessment and Accountability Plan, and district Core Curriculum Plan address core curriculum. There is no additional indication that the suggested strategies have been implemented. 21. The district has made no change on the collection of special education referral data. 22. Guidance documents have been developed, training provided, and information disseminated on alternatives to nonpublic school placements. Implementation of this plan has just started. 23. The district has made no change regarding the recommendation to eliminate some traditional practices in favor of innovative approaches. 24. No procedural changes have occurred since September. The district has made no change on shifts from special day to regular classrooms with support. 25. The Executive Director has started a process for budgeting based on caseload. This is an identified action plan item in the Special Education Department. The district has made no change regarding a final plan for allocating special education personnel based on multiple pieces of data. 26. The question of special education philosophy has not been addressed. 27. The Human Resources Department and the Special Education Executive Director are developing a recruitment plan for special education. To date, no plan is formalized. The district has no plan for retention of staff. 28. There is no review of the student study team process. 29. The Executive Director works with a CDE consultant for review of the budget process, review of nonpublic school placements, private transportation, and an initial look at least restrictive environment. Action plans are being developed by the Executive Director. Minimal staff members are involved in the process. The total program review is not in place at this time. 30. The Executive Director is working with a CDE consultant to review the budget and finance issues of special education. There is no plan, process, or data regarding the status of this recommendation. Standard Implemented: Partially January 2000 Rating: NA September 2003 Rating: 1 March 2004 Self Rating: 3 March 2004 New Rating: 2 54 Implementation Scale: 55 4.1 High School Exit Examination – Pre-Exam Intervention . Professional Standard A process to identify struggling students and intervene with the additional support necessary for them to pass the exit examination is well developed and communicated to teachers, students, and parents. Progress on Implementing the Recommendations of the Recovery Plan 1. The district staff has not prepared a plan to identify and provide effective interventions for students at risk of failing sections of the California High School Exit Examination (CHASEE) on their first attempt. 2. Since there is no written plan, analysis of the recommended components of the document was not possible. Resources planned or in place to prevent failure included the High Point Reading Intervention Program, a teacher professional development course focused on exit examination writing standards, and a restructured Algebra course to provide more time for students to master the standards. Also, optional CHASEE preparation courses were available for students. These measures were in addition to routine interventions provided for students who were not achieving academic objectives. Standard Implemented: Partially January 2000 Rating: NA September 2003 Rating: 0 March 2004 Self Rating: 2 March 2004 New Rating: 1 Implementation Scale: 56 4.2 II/USP – Measurement of Student Achievement Progress Professional Standard Grant recipients are collecting required data to measure progress of student achievement. Progress on Implementing the Recommendations of the Recovery Plan 1. The board lacks a policy that specifically focuses on low-performing schools (48 district schools are II/USP grant recipients). For the most part, the Assessment, Data and Accountability and Staff Development Plans mention teachers, schools and the district as a whole. One section in of the plan mentions “accountability, autonomy, and high stakes decision making” and has language that refers to the district criteria for “failing” schools. The document lists three criteria: “failure to move student achievement for all significant groups,” “failure to move individual student achievement” or “failure to provide a learning environment conducive to student achievement.” There are no specific procedures, time lines, or details for when a school becomes “failing” or the kind of “extra support” that will be provided. Reference is made to “details of the criteria for determination of failure” that are included in the “Board Policy on Failing Schools,” however, a policy has not been developed or adopted by the board. a. The district has a report without title or date that divides schools into four color bands based on their 2003 Academic Performance Index (API) scores (The report is also described in Standard 1.15). The Professional Development Plan states, “Orange schools that also fail to show progress on a range of other indicators are designated as ‘red’ and face mandatory redesign”. There are no “other indicators” or the kind of quantifiable data that would be used to measure the progress of student achievement. b. Low-performing schools are given the same data to measure the progress of student achievement as the other schools in the district. Data are not specifically generated any differently for schools performing below the green, yellow or orange bands (below 672 on the 2003 API) than the schools performing above the blue band (above 800 on the 2003 API). c. All principals in low-performing schools as well as all other principals are receiving training in the use of data, the data Web site, classroom roster, and data needed for the School Plan Self Analysis, and Quality School Portfolio. Six-week benchmark assessments for Open Court are accessible on the Web site. The Open Court “Collaboration Inquiry Cycle” is being implemented in the K-5 (yellow and orange) low-achieving schools “as a tool to ensure the application of assessment data to classroom practices.” 2. The district produces several reports with data colored by a basic green-yellow-red rubric to indicate levels of proficiency for student achievement. The district has CST English/Language Arts and Math Performance Reports for 2002 and 2003 for II/USP Schools, Cohorts 1 and 2. Green indicated improved scores; yet, in Cohort 2, 10 of the 32 schools, colored green, have more than 60 percent of their students performing 57 “far below” and “below basic proficiencies” on the 2003 CST for English/language arts. Reports such as this give a general, optimistic overview, but are not useful for diagnosing specific weaknesses in the core subject areas targeted for improvement by the low-performing schools. 3. Data reports do not delineate student achievement in the growth target areas specific to each school, according to their board-approved II/USP action plans. Grant recipients are receiving data reports that are generated on a generic basis rather than on a site specific, diagnostic basis. There are no specific district procedures, time lines or accountability requirements for the way II/USP schools monitor student achievement in the targeted areas specified by their II/USP action plans. 4. Data are not made available according to site specific, targeted areas such as trend data that could be used to monitor student achievement in the core subject areas for specific skills at the elementary and middle school levels (e.g. comprehension in reading or problem solving in math). Instead, all schools have access to the information the same way, with the same time lines. It is unclear the extent to which each site (especially at the high school level) is accessing data off the district’s data Web sites, the frequency, what graphs/tables, how they are being used, and whether it is having an effect on classroom instruction. Standard Implemented: Partially January 2000 Rating: NA September 2003 Rating: 5 March 2004 Self Rating: 6 January 2004 Rating: 5 Implementation Scale: 58 4.3 II/USP – Progress toward Meeting/Exceeding Goals Professional Standard: Grant recipients are meeting or exceeding goals as identified in action plans. Progress on Implementing the Recommendations of the Improvement Plan: 1. The district has limited information about the II/USP schools. The district staff indicated that due to the large number of low-performing schools, II/USP schools are not separated as a standalone segment of schools. “Rather the needs of the II/USP and other low-performing schools are being addressed through planning, development and implementation of intense support structures.” Sixty-eight schools are performing below expectations and/or below the 50th percentile on standardized state tests (41 schools are designated as yellow, below expectations; and 27 designated orange, far below performance expectations. 11 of the orange schools scored below a 500 on the API). Thirteen of the 48 II/USP schools (nine elementary and four high schools) did not meet the state’s 2003 API growth targets, either schoolwide or in all subgroups or both areas. None of the II/USP high schools met AYP targets and 24 of the 30-four elementary schools did not meet the 2003 AYP targets according to the California Department of Education, Phase I Report. a. Information has not been summarized in a way that identifies the needs per II/USP action plans for each of the II/USP schools. For example, no reports indicate the commonalities or differences between the green band II/USP schools and orange band II/USP schools other than the obvious difference in student achievement and the API ranking. b. Student achievement data (documentation) are not collected specific to the targeted goals for each II/USP school. Student achievement growth for the majority of II/USP schools has been minimal. Thirty-nine of the 48 II/USP schools remain in decile 1, 2, or 3 according to the California Department of Education 2002 API Base, List of Schools. These deciles identify schools performing significantly below the 50th percentile in the state’s standardized testing program. 2. Little progress has been made to produce the kind of comparative data that is useful to the grant recipient II/USP schools. Trend data and schoolwide subgroup item analysis for the core subject areas are not accessed consistently by all the underperforming schools. “Some schools are more sophisticated than others about using the data.” The executive directors, especially those with the most number of II/USP schools, are meeting more consistently to discuss the data and progress of the schools. The cumulative or progress reports of these meetings were unavailable. Standard Implemented: Partially 59 January 2000 Rating: NA September 2003 Rating: 1 March 2004 Self Rating: 4 March 2004 Rating: 3 Implementation Scale: 60 4.4 II/USP – Leadership for Underperforming Schools Professional Standard Principals and teachers in underperforming schools and/or in schools under mandated improvement programs are provided special training and support by the district; improvement plans are monitored. Progress on Implementing the Recommendations of the Recovery Plan 1. Some progress has been made toward developing a strategic plan to provide additional support to the lowest underperforming schools. Beginning this year, the district is assisting all its low-performing schools by “aggregating” all schools and adopting a color-coded ranking system (blue, green, yellow, orange, and red) denoting schools that exceed, meet, approach, and perform below expectations as well as “a small number that face district closure (red).” The 68 schools below the 678 score on the 2003 API (yellow, orange and red) are beginning to receive additional support, monitoring, and restrictions imposed by the district administration. Many low-performing schools are working with outside coaches through such providers as Performance Fact, Gibson and Associates, Dale Skinner from Region IV’s Staff Development Center (Alameda County Office of Education), and the Reading First support staff at the Sacramento County Office of Education. a. Some progress has been made toward having schools include goals and actions steps from their II/USP action plans with their single site plans. Principals are given assistance developing their single site plans through the district’s Web site and the district staff. Accountability TSAs review the “Bins and Binders” for compliance with state and federal regulations. Schools are monitored through the SBCP/school improvement process. Reports indicated there is limited use of quantifiable, comparative data in reports. Most did not include narrative findings and conclusions with illustrations, charts or graphs. b. The current reporting practices for evaluating school improvement are limited. Reports to the board are perfunctory. There are no current finding reports from the program quality reviews for individual II/USP schools, required by Administrative Bulletin #5002 Revised School Based Coordinated Programs and BP 6190. Similarly, the 2003 II/USP End-Of- The-Year district Evaluation/Impact Report was unavailable. c. The single site plans are used to establish school improvement goals, action steps and time lines. The district lacks summary information that would indicate whether sites are meeting their goals or receiving assistance that is effective in improving student achievement. d. This year, executive directors are meeting twice a month with principals – once a month for operational meetings and a second time for training and 61 follow-up. Academic conferences are taking place to assist all principals and their staffs with data analysis, staff development, the School Plan Self Analysis, and the district’s implementation of Results-Based Budgeting. Each network has II/USP schools working with an executive director. Of the three elementary Networks, one has 12 schools (11 are II/USP), a second has 23 schools (15 are II/USP), and a third has 22 schools (5 are II/USP). At the middle school level there is one Network, which includes 15 schools (9 are II/USP). At the high school level there is one Network and it includes 13 schools (4 are II/USP) – five of the high schools, ranked in decile 1 in 2002, scored between 409 to 484 on the 2003 API. Only one district high school in the orange band met schoolwide and subgroup state targets. There is improved progress toward identifying the schools that need additional or “special” support. e. The district lacks methods and/or significant data that help identify specific academic and intervention needs common to most of the underperforming schools. There is limited use of comparative data extrapolated from the single site plans and the districtwide assessments for the core subjects that indicate areas of greatest weakness. 2. There are no guidelines for principals that would indicate how progress is evaluated or whether the School Site Plan evaluations are tied to the administrative appraisal process. 3. Some assistance from outside consultants to the low-performing schools is provided. The district lacks a process for identifying needs and providing “extra support” to specific schools, prior to this year. FCMAT received limited information about specific intervention plans or programs for the low-performing schools at each level: elementary, middle or high school. There are no written guidelines related to the evaluation of this assistance or whether the support being given is specific to the identified needs of individual schools. 4. Through regularly scheduled meetings with the principals, Network executive directors are beginning to informally identify common needs and increase the opportunity for schools to align their priority academic goals with those of the district. Accountability TSAs are attending school site council meetings regularly. Procedures for aligning school goals with district goals are beginning to emerge through the SBCP process. Standard Implemented: Partially January 2000 Rating: NA September 2003 Rating: 3 March 2004 Self Rating: 4 March 2004 New Rating: 4 Implementation Scale: 62 63