Summary

rotiduA etatS ainrofilaC S T I D U A E T A T S F O U A E R U B Department of Education: Its Flawed Administration of the California Indian Education Center Program Prevents It From Effectively Evaluating, Funding, and Monitoring the Program February 2006 2005-104 The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TTY (916) 445-0033 OR This report is also available on the World Wide Web http://www.bsa.ca.gov The California State Auditor is pleased to announce the availability of an on-line subscription service. For information on how to subscribe, please contact the Information Technology Unit at (916) 445-0255, ext. 456, or visit our Web site at www.bsa.ca.gov Alternate format reports available upon request. Permission is granted to reproduce reports. C S A ALIFORNIA TATE UDITOR ELAINEM.HOWLE STEVENM.HENDRICKSON STATEAUDITOR CHIEFDEPUTYSTATEAUDITOR February 7, 2006 2005-104 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the Bureau of State Audits presents its audit report concerning the Department of Education’s (department) management of the California American Indian Education Center program (program). This report concludes that, despite established guidance, the department has not adequately administered the program and consequently cannot ensure that the program is successfully meeting the goals established in law or the needs of the communities it serves. For example, the department has not consistently collected all the data from the California Indian Education centers (centers) required by law to measure their performance and cannot fully justify either its basis for initially selecting centers to receive funding or for determining the annual amount of funding it grants each center. Further, the department has not always been prompt in disbursing funds to the centers; and it lacks a monitoring process to ensure that centers spend funds appropriately, pursue program goals, and report accurate data. Respectfully submitted, ELAINE M. HOWLE State Auditor BUREAUOFSTATEAUDITS 555CapitolMall,Suite300,Sacramento,California95814 Telephone:(916)445-0255Fax:(916)327-0019 www.bsa.ca.gov/bsa ConTenTS Summary 1 Introduction 5 Audit Results The Department of Education Does Not Know How the California Indian Education Center Program Is Performing 11 The Department Has Not Collected Complete or Consistent Annual Data as Required 11 Unable to Fully Justify How It Selects and Funds Centers, the Department Cannot Ensure That the Program Meets the Needs of the State’s American Indian Communities 18 The Department Has Not Always Paid Centers Within Its Established Time Frame 23 The Department Is Not Consistent in Its Program Monitoring and Communication 26 Recommendations 30 Response to the Audit California Department of Education 33 California State Auditor’s Comments on the Response From the Department of Education 41 SuMMArY ReSulTS in bRief Audit Highlights . . . Despite established guidance, the Department of Education (department) has not adequately Our review of the administered the California Indian Education Center management of the California Indian Education Center program (program) and consequently cannot ensure that the program (program) by the program is successfully meeting the goals established in law or Department of Education the needs of the communities it serves. To address the challenges (department) found that: facing American Indian students enrolled in California’s public  Because the department schools—low academic achievement at all grade levels, high has largely ignored dropout rates, and few students continuing their education the existing guidance beyond high school—the Legislature established the program for administering the program, it cannot in 1974. The legislation indicated that the California Indian ensure that the program Education centers (centers) should serve as educational resources is successfully meeting for American Indian students, their parents, and the public the established goals schools. In addition, to guide the operation of the centers, or the needs of the communities it serves. the Legislature established a set of goals, such as improving the academic achievement, self-concept, and employment  The department did not opportunities of American Indian students and adults. From its ensure that California Indian Education centers initial 10 centers funded by a total of $400,000 in grants, the (centers) reported all program has grown to comprise 30 centers that annually receive the annual data more than $4.4 million in total funding as of fiscal year 2005–06. required by law to If not reauthorized, the program is set to end on January 1, 2007. measure performance.  The department has no The department is required by state law to administer and record of the centers’ oversee the program and receives guidance from legislation assessments of needs called for by the as well as internal policies. For instance, state law requires the guidelines adopted by the department to collect data annually to measure the academic State Board of Education performance of the students the centers serve and how well the (board) and thus has no centers are meeting the goals established by law. Additionally, way of knowing whether the services the centers although no regulations govern the program, state law requires assert they provide are the State Board of Education (board) to adopt guidelines for those most needed by the selecting and administering the centers. The guidelines the populations they serve. board adopted in 1975 require, among other things, that  Though submitted centers design their programs after assessing the needs of their to the Legislature on respective communities. Internal guidance comes from the time, the department’s department’s 2001 Grant Administration Handbook (handbook), evaluation of the program lacks sufficient analysis which guides the administration of programs funded by to adequately support grants similar to those used in this program. The handbook its recommendations to stipulates that the department establish a competitive process to improve the program. objectively select grant recipients, a monitoring plan to ensure continued on next page . . . that grant recipients appropriately implement the program, California State Auditor Report 2005-104 11  The department is unable and a document retention and filing process to effect stable to justify its basis either program administration and clear communication between the for selecting centers to department and the centers. receive funding or for determining the annual amount of funding it However, the department has largely ignored the existing grants each center. guidance for administering the program and therefore has little  The department has means of determining program effectiveness. For example, not always promptly until 2005 the department did not ensure that centers reported disbursed funds to the the annual academic performance data of their students. centers. Further, the department has no record of the centers’ needs  The department lacks a assessments on file and thus has no way of knowing whether monitoring process to the services the centers assert they are providing are the services ensure that centers spend most needed by the populations they serve. The department funds appropriately, contends that its administrative shortcomings are the result of pursue program goals, and report accurate data. several factors, including staff turnover and limited resources. Nevertheless, it submitted an evaluation of the program to the Legislature by January 1, 2006, as required by state law. Because the department was slow to start collecting data for the report, however, the evaluation lacks sufficient analysis to adequately support its recommendations to improve the program. Another indication of the department’s flawed administration of the program is its inability to fully justify its basis either for initially selecting centers to receive funding or for determining the annual amount of funding it grants each center. According to the handbook, it should select grant recipients following a competitive process, which includes an objective scoring methodology and independent raters. However, the department could not demonstrate that it used a competitive process to select the most recent centers currently funded. Further, although program staff state that the department’s sole basis for computing the amount that each center receives is the amount granted in the previous fiscal year, it has not consistently followed that method. Without a documented selection and funding process, the department is vulnerable to criticism of inequitable treatment and cannot ensure that the program is effectively addressing the educational needs of American Indian students in the State. Further, the department has not always promptly disbursed funds to the centers. Despite the department’s informal policy that it would issue the first of three annual installment payments to centers with approved applications an estimated six to 10 weeks after the governor signs the state budget, in fiscal year 2003–04 the centers did not receive their first grant allocations until December—18 weeks after the budget was approved. Without the 22 California State Auditor Report 2005-104 expectation of receiving their first allocations within a relatively stable time frame, the centers may not be able to appropriately plan and provide services to their clients. Finally, the department lacks a monitoring process to ensure that centers spend funds appropriately, pursue program goals, and report accurate data. Without operating policies and procedures outlining how staff should consistently administer the program, the department may create confusion among the centers. The department indicates that it is attempting to improve its administration of the program by proposing more detailed legislation to reauthorize the program and by developing a plan for monitoring the centers, but these efforts are too preliminary for us to assess. ReCommendATionS To ensure that it administers the program clearly, consistently, and effectively, the department should develop operating policies and procedures specific to the program and train staff in their application. The policies and procedures should include the following: • A description of the data that centers must annually report to measure program performance and a standardized format for reporting to allow the department to effectively aggregate and consolidate the data for reports to the Legislature and other interested parties. Further, the department should outline the consequences for failing to submit the data. • An equitable process to select centers to receive grant awards and determine their respective funding amounts. • A set time frame that it adheres to for disbursing payments to the centers once their applications are received and approved. The time frame for the first payment can be expressed as a set number of weeks after enactment of the state budget for centers with approved applications. • A monitoring process and plan to ensure that reported fiscal and program information is accurate and complete, including a process for corrective action and departmental follow-up for noncompliance. California State Auditor Report 2005-104  AGenCY CommenTS The department generally agreed with our recommendations but provided additional information. n 44 California State Auditor Report 2005-104 InTroduCTIon bACkGRound To address low academic achievement, high dropout rates, and low postsecondary education among American Indian children attending California public schools, the Legislature authorized the California Indian Education Center program (program) in 1974. According to state law, California Indian Education centers (centers) must serve as educational resources in American Indian communities for American Indian students, parents, and the public schools and should be designed to carry out 12 program goals, such as improving the academic achievement, self-concept, and employment opportunities of American Indian students and adults. To those ends, centers may offer services such as libraries, computer access, tutoring, and various cultural activities. Some centers have established memorandums of understanding with public schools to enable American Indian students to access services either at their local schools or in their homes. The program began with a budget of $400,000, which initially funded 10 centers in 10 counties. According to law, the centers are operated by tribal groups or incorporated Indian associations. Since its inception the program has grown to 30 centers located in 23 counties with a budget of around $4.5 million coming from the State’s General Fund. For fiscal year 2004–05 centers received annual grant awards of between $102,650 and $240,550, with an average grant of about $138,450. Centers operate in urban, rural, and tribal environments and may receive funding from other sources, such as Tobacco Use Prevention Education grants. The Department of Education (department) administers the program through its Migrant, Indian, and International Education Office (office), which oversees various programs. The department administers and oversees the program using one consultant and one analyst, who are supervised by the manager of the office; however, the program is budgeted for only one-half of the consultant’s salary and benefits. For fiscal year 2004–05 the department budgeted approximately $72,000 for administration of the program, but this figure does not California State Auditor Report 2005-104 55 reflect the program’s true administrative cost.1 According to a department budget manager, the department uses part of the general operating funds allocated to it from the state budget to pay for the administration of programs, including this one, for which the Legislature does not specifically appropriate administrative funding. Although no state regulations govern the program, by law the State Board of Education (board) must adopt guidelines, as recommended by the superintendent of public instruction, to select and administer the centers. The board adopted guidelines in 1975 that include expectations regarding assessments that centers must conduct to identify their most pressing needs (needs assessments) and program applications centers must complete to be eligible for grant funds. The Legislature has reauthorized the program several times since 1974. The latest reauthorization, which took effect in October 2001, extended the program’s life from January 1, 2002, through January 1, 2007. The latest reauthorizing legislation added requirements for the department to collect certain annual data that measure the students served, services provided, the academic performance of students participating in the program, and the extent to which centers are meeting the 12 program goals. From the centers’ submission of yearly data and self-evaluations of their respective programs, the department is required by law to prepare a report consolidating the results of the program and recommending improvements. The department’s report was due to the Legislature on or before January 1, 2006. CenTeR fundinG Although it awards program grants to centers following an annual application process, the department approves the grant applicant for each center in multiyear cycles. In other words, if the department approves a center’s application at the beginning of a grant cycle and the center meets all the department’s subsequent requirements, the center can expect to continue receiving funding annually through the life of the cycle. Historically, the program’s grant cycles have comprised two or three years. The department started a new three-year cycle in fiscal year 2002–03 and has since extended it to five years to overlap with the current reauthorization of the program, which is scheduled to end on January 1, 2007. To receive funding in 1 This amount does not reflect more than $86,500 in combined staff salaries—excluding benefits—incurred administering the program by the consultant and analyst and charged to other programs in fiscal year 2004–05.  California State Auditor Report 2005-104 fiscal year 2002–03, the department required each center to submit a program application and budget. Before each new fiscal year within the grant award cycle, the department requires each center to submit a program update and budget. The department disburses funds to centers in three payments annually, apportioned at 50 percent, 40 percent, and 10 percent of the yearly grant allocations, respectively. The timing of the first disbursement depends on the date the state budget is enacted, when the department can be certain of the allocation amounts for the program, and the receipt and approval of centers’ applications. As Figure 1 illustrates, the department’s payment process allows it to issue first payments to centers with approved applications between six and 10 weeks after the state budget is enacted. Before releasing the second or third payments, the department requires centers to submit expenditure reports demonstrating that they have spent at least 80 percent of the funds they have already received. fiGuRe 1 Process of issuing first Grant Payments to Centers California Indian Department State Controller’s Education Centers of Education Office Send award notification letters (immediately after the state budget is signed) Sign and return award notification letters (1–2 weeks) Review signed award notification letters and process payments (2–4 weeks) Issue payments (3–4 weeks) Source: Department of Education. Note: This process applies to centers whose grant applications have been approved by the department by the date the state budget is enacted. California State Auditor Report 2005-104  SCoPe And meThodoloGY The Joint Legislative Audit Committee (audit committee) requested that the Bureau of State Audits review the department’s administration of the program, how it determines funding for the centers, and how it evaluates them. Specifically, the audit committee asked us to determine the department’s roles and responsibilities related to the centers and to review and evaluate the department’s existing policies, procedures, and practices for administering the program and monitoring the centers. The audit committee was also interested in any written procedures the department has developed to guide program administration. In addition, it asked us to review the department’s funding structure for the program and how it appropriates funds to administer the program. Further, the audit committee requested that we assess the reasonableness of the department’s uses of program funds; determine whether it has directed sufficient resources to the program in general and sufficient management attention to completing the program evaluation report that was due to the Legislature on January 1, 2006; and review the department’s document retention policies and practices. Finally, the audit committee asked us to review and evaluate the department’s process for allocating and disbursing funds to the centers. To determine the department’s roles and responsibilities related to the centers, we reviewed and evaluated the existing laws, guidelines, and other criteria significant to the program. We interviewed program staff and compared the department’s program files for fiscal years 2002–03 through 2004–05 with what we would have expected to find based on the existing guidance to evaluate the department’s existing policies, procedures, and practices for administering the program and monitoring the centers and retaining documentation; to assess the reasonableness of the department’s uses of the program funds; and to determine whether it has directed sufficient management attention to the program. To review the department’s funding structure for the program and how it appropriates funds for the program’s administration, we interviewed program, budget, and accounting staff and reviewed state budget appropriations. During our review, we noted that the department inappropriately used federal funds to pay for certain costs of administering the program. We are reporting this issue in a separate letter to the department. To determine whether the department has directed sufficient  California State Auditor Report 2005-104 management attention toward completing its evaluation of the program, we reviewed the evaluation report and the process the department used in its development. To review and evaluate how the department allocates and disburses funds to the centers, we interviewed current and former program staff as well as accounting staff and reviewed the department’s accounting records between fiscal years 2002–03 and 2005–06. Because the scope of this audit focused on the department’s administration of the program, we did not evaluate the centers’ performance or fiscal accountability. n California State Auditor Report 2005-104  Blank page inserted for reproduction purposes only. 1100 California State Auditor Report 2005-104 AudIT reSulTS The dePARTmenT of eduCATion doeS noT knoW hoW The CAlifoRniA indiAn eduCATion CenTeR PRoGRAm iS PeRfoRminG Although various forms of guidance require the Department of Education (department) to collect and maintain documentation regarding the performance of the California Indian Education centers (centers), the department lacks an understanding of the extent to which the centers are fulfilling the goals established in law for the California Indian Education Center program (program) to meet the needs of communities the centers serve. For example, since October 2001 state law has stipulated that centers cannot receive their annual funding until they report to the department various data, including the academic performance of students participating in the program. However, not until January 2005 did the department first request that centers provide such data, in the form of California Standards Test (CST) scores for fiscal year 2004–05. Moreover, guidelines adopted by the State Board of Education (board) require centers to conduct assessments to ensure that they are providing the appropriate services to meet the needs of the American Indian populations they serve, but the department could provide no evidence that any center completed a needs assessment. The department’s inadequate administration of the program results in part from staff’s lack of knowledge that the board had adopted guidelines and the department’s failure to establish internal policies and procedures specific to the program. Consequently, the department cannot provide an accurate depiction of the centers’ performance in meeting the program goals and fulfilling the needs of California’s American Indian population. The dePARTmenT hAS noT ColleCTed ComPleTe oR ConSiSTenT AnnuAl dATA AS RequiRed When the Legislature approved urgency legislation to extend the life of the program from January 1, 2002, to January 1, 2007, it revised the law to require that each center, as a condition of California State Auditor Report 2005-104 1111 receiving annual funding, collect and report to Goals of the California indian education the department site evaluation data measuring the Center Program established by law following: 1. Improve the academic achievement of American Indian students with particular • Number of students served. emphasis on reading and mathematics. • Services provided to students. 2. Improve the self-concept of American Indian students and adults. • Academic performance of students served. 3. Increase the employment of American Indian adults. • Extent to which the center is meeting the 12 program goals (see the text box). 4. Serve as a center for related community activities. Since October 2001 state law has directed the 5. Provide tutorial assistance to students in department to establish centers’ reporting of reading and mathematics. these data as a condition for receiving funding. 6. Provide individual and group counseling However, the department has collected only some to students and adults related to personal adjustment, academic progress, and data and did not provide guidance to the centers vocational planning. to ensure that the reported data were meaningful 7. Provide coordinated programs with the and comparable. As a result, the department’s public schools. data collection over the last several years has been incomplete to varying degrees and inconsistent 8. Provide a neutral location for parent- teacher conferences. with regard to all four required elements. For example, the department requests information on 9. Provide a focus for summer recreational sports and academic experience. the number of students served through reports it expects centers to submit at the end of each year. 10. Provide adult classes and activities. However, only 80 percent of centers had year-end 11. Provide college-related training programs reports on file for fiscal year 2002–03. Although the for prospective American Indian teachers. department’s documentation of year-end reports 12. Provide libraries and other related improved to 93 percent in fiscal year 2003–04, educational materials. as of November 30, 2005, only 67 percent of the year-end reports for fiscal year 2004–05 were on Source: Education Code, Section 33381. file. Thus, centers have received funding without meeting the requirements of the law. Moreover, the lack of clarity in the department’s directions for completing year-end reports can lead to inconsistent data reporting. For example, for fiscal year 2003–04 the department requested that centers report the number of students served by each of three service categories: academic, cultural, and leadership. However, the department did not require centers to report an unduplicated total number of students that received one or more of their services. Using the department’s method, a student who received multiple services would have been counted multiple times, one time for each service category the student received. Although it is important to obtain service counts as a means of assessing how heavily various services are 1122 California State Auditor Report 2005-104 being used, it is also important to know the size of the population of students who receive one or more services at a particular center to assess the overall need in that center’s community. Further, in some years the department instructed the centers to The inconsistent and report only students who received a certain number of service unclear guidance given hours, depending on the type of service, but in another year it to the centers on what did not request that information. Thus, in one year centers may data to report prevents not have included in the report a count of students receiving less the department from than the requested number of service hours, but in another year effectively consolidating the centers would have reported that count, giving the department the information, no consistent basis to compare the data. The inconsistent and identifying the actual unclear guidance given to the centers on what data to report number of students that prevents the department from effectively consolidating the each center serves, and information, identifying the actual number of students that each comparing equivalent center serves, and comparing equivalent data over time. data over time. Additionally, although the department’s year-end reports for two of the three fiscal years we reviewed contain a section for the centers to report on certain categories of services provided, the reports do not include a format for collecting measurable data on other services. For example, the centers are not required to report the services they provided to fulfill any of the 12 program goals they pursue; rather, the department requires centers to briefly describe their activities. Moreover, the department does not provide a uniform or meaningful format for effectively consolidating and evaluating the information on the services that centers provide. In January 2005 the department supplemented the information it collected by surveying the centers on the services they provide, and it modified the year-end report for fiscal year 2005–06 to include the survey. One item in the survey asked each center to estimate the percentage of operating time it spent providing services to meet each of the 12 program goals. However, the department has evidence of only 13 of the 30 centers (43 percent) responding to the survey, and some of those surveys may contain unreliable information. For example, one center indicated that it spent more than 100 percent of its time providing services that focus on meeting program goals. Although the year-end reports contain some subjective descriptions of academic performance and output, they do not include site evaluation data that measure academic performance or outcomes of the students involved with the program. For example, some centers highlighted their activities or the accomplishments of particular student groups in their California State Auditor Report 2005-104 11 year-end reports. However, no report included measurements over time or compared results against preestablished goals. Further, even though the statutory requirement to report academic performance data has been in effect since fiscal year 2002–03, the department did not request data from the centers until January 2005, in the form of 2004 CST scores for students receiving center services in fiscal year 2004–05. In January 2005 the department trained center staff and provided them with information regarding the CST test scores and how to report the collected data. However, the department’s files only contain such information for 23 of the 30 centers (77 percent). Moreover, the information was not always submitted in a uniform format and did not always contain all the data elements the department requested. For instance, many of the centers included only the language and mathematics test scores and did not include the required history or science portion of the CST. Additionally, three centers submitted the data in formats different from that requested by the department. Thus, the academic performance data the department received was not complete and was difficult to aggregate and assess. Finally, the department has not collected data to determine the extent to which each center meets the 12 program goals established by law. The year-end reports provide the opportunity for centers to elaborate on their successes but do not provide a means to present measurable outcomes relating to the goals that the centers initially identified in their applications. The program application, which each center completes when requesting program funding, requires the center to present its goals and The department has not objectives and state how it plans to evaluate them. However, the provided guidance or department has not provided guidance or a format for centers to a format for centers to use to present outcomes showing how well they are meeting their use to present outcomes identified goals. Although the program consultant indicates that showing how well the department’s new survey of services provided by the centers they are meeting their fulfills this statutory requirement, the survey measures only identified goals. outputs, such as the proportion of time spent on providing various services, not outcomes or achievements, such as an improvement in students’ test scores for reading or mathematics. The program consultant states that to ensure the centers submit all required data, fiscal year 2005–06 marks the first year centers will be accountable for all missing documentation, and centers will not receive grant allocations from the department until it receives all overdue reports. However, because the data required annually is due in August 2006, the department’s 1144 California State Auditor Report 2005-104 efforts were too preliminary for us to evaluate. By not ensuring that all centers submit all the information required in a uniform format, the department might not be able to understand or evaluate how well the centers are fulfilling the program’s goals. With Staff unaware of Guidelines Requiring needs Assessments, the department does not know if Centers have designed Their Programs to meet Community needs Guidelines adopted by the board in 1975 for the selection and administration of the centers require each center to assess and determine the needs of the local American Indian population and design its program according to those needs in the context of the 12 goals that state law establishes for the program. Ultimately, a center’s needs assessment should assist it in designing services that help American Indian students raise their academic achievement in reading and mathematics and that improve the self-concept of students and adults in the American Indian community the center serves. Specifically, the board’s guidelines require each center to conduct a needs assessment to determine the following: • Languages spoken in homes and the need for English language instruction. • Academic and vocational training needs. • Job skills and employment needs. • Strengths of existing educational and employment programs and how the center can incorporate and improve on those strengths to meet identified needs. • Plans to coordinate and adapt existing programs within the community to meet identified needs. Although the department has on file a version of the guidelines that requires each center to conduct an assessment to identify needed services within its community, both current and former program staff told us they were unaware that the board had adopted the guidelines. Staff further stated that the department lacks internal policies and procedures specific to the program. As a result, a former program consultant indicated that she had received so little guidance that she was concerned she would not know the answers when faced with questions from the centers regarding program administration. Lacking internal policies and procedures that informed program staff the board had adopted these guidelines, it appears that the department did not ensure that centers conducted needs assessments. California State Auditor Report 2005-104 1155 We expected that as the entity state law charges with administrative oversight of the program, the department would ensure that centers assess the needs of the communities they were selected to serve by requesting that centers submit individual needs assessments in compliance with the guidelines. In that way the department could position itself to fulfill the Legislature’s requirement that it study and identify the cultural and educational disadvantages affecting American Indian children in the existing public school system. However, our Without obtaining review of the department’s files indicates that it has no assurance and reviewing needs that any of its 30 centers conducted needs assessments, assessments, the either on entering the program or at any subsequent time, department cannot to determine the community’s needs and whether they effectively offer guidance continue to remain the same. In fact, although some centers to the centers or included general discussions of certain community needs in recommendations to the their applications for funding on file with the department, no Legislature regarding center’s file contains a formal needs assessment that fulfills whether the services that the requirements of the guidelines. Without obtaining and centers claim to provide reviewing needs assessments, the department cannot effectively actually respond to offer guidance to the centers or recommendations to the community needs. Legislature regarding whether the services that centers claim to provide actually respond to community needs. The department’s Report to the legislature is insufficient to Adequately evaluate the Performance of the Program The legislation reauthorizing the program in October 2001 required each center to submit an evaluation of its program to the department on or before July 1, 2005, along with any other information that the department requested. The department then had to report the consolidated results of the centers’ annual performance data and self-evaluations, as well as its recommendations for program improvement, to the Legislature by January 1, 2006. We reviewed the report and noted several flaws. The program consultant asserts that the department’s report includes the data that state law requires it to collect on an annual basis, but as discussed earlier, the data the department collected is inconsistent and incomplete, which limits its usefulness. For example, the department’s report includes data for only one year reported by less than 100 percent of the centers, which precludes the department’s ability to measure trends. Because the law requiring the report took effect in October 2001, if the department had actively sought the information, it could have included up to three fiscal years of data, allowing it to analyze and compare results over time. For 11 California State Auditor Report 2005-104 instance, the department could have compared the various achievements the centers made in each of the three years, and it could have compared the performance of American Indian students who received center services in those years with those who did not. Additionally, the department’s report makes no mention of the centers’ successes in meeting any of the 12 program goals and reports only on the types of services provided to indicate that the goals are being addressed. The department also failed to address the provision of the law that required the centers to submit a one-time self-evaluation by July 1, 2005. The program consultant contends that the centers’ year-end reports and the responses to the survey of services the department conducted provide the program information necessary to satisfy the requirements of the law. However, state law specifically requires each center to submit a self-evaluation to the department in addition to the annual data the department requests in the year-end report and survey of services. By not requiring them to complete self-evaluations, the department missed an opportunity to obtain centers’ perspectives on the program to include in the report to the Legislature. The department does include various recommendations in the report, but the recommendations are primarily based on insufficient center data, demographic studies of the American The department Indian population as a whole, and research studies conducted recommends that the by others, rather than on program-specific analyses. The program be reauthorized department recommends that the program be reauthorized and and expanded, and that expanded, and that it receive additional funding. While these it receive additional may be laudable recommendations, the department does not funding. While these provide measurable data to support them. The department also may be laudable recommends that guidelines for administering the program recommendations, the be updated and augmented to include training and technical department does not assistance processes. Although the department had begun provide measurable data drafting guidelines, the program consultant indicated that it has to support them. decided instead to ask for additional detail in the reauthorizing legislation, which the Legislature would likely consider before the program ends on January 1, 2007. In the meantime the department will operate under those provisions of the 1975 guidelines that have not been rendered unenforceable by subsequent changes to the law. The department’s report further recommends that a comprehensive process for monitoring the program be developed. Finally, it recommends that the Legislature appropriate sufficient positions and funds to support the California State Auditor Report 2005-104 11 American Indian education unit to provide administrative oversight of the American Indian education programs established by the State. However, the department’s report does not provide an analysis to support additional positions or funding. Currently, the program is budgeted for one half- time position, has a program consultant rather than a unit coordinator as specified in the law, and is not a separate unit but is located within the department’s Migrant, Indian, and International Education Office (office).2 According to a former manager of the office, the department’s only other American Indian education program is part of another department office.3 unAble To fullY juSTifY hoW iT SeleCTS And fundS CenTeRS, The dePARTmenT CAnnoT enSuRe ThAT The PRoGRAm meeTS The needS of The STATe’S AmeRiCAn indiAn CommuniTieS The department’s current approach to awarding program funds contravenes its own guidance that grants, such as those the centers receive, be awarded competitively in accordance with an objective process that includes a scoring methodology and independent scorers. The department’s failure to follow its own criteria may have prevented potential program recipients from applying for funding. Moreover, the department could not fully explain its approach for allocating funds to the centers. It has increased or decreased some centers’ requests for funding without a documented justification while allocating to other centers the exact amounts they requested. By apportioning program funds among the centers without objective justification, the department leaves itself vulnerable to criticisms of inequity and failure to identify and consider the needs of the American Indian community statewide. ignoring its Written Policies, the department has not used a documented and objective Process to Select Centers for funding The department’s 2001 Grant Administration Handbook (handbook), which it relies on for its administration of nonformula-based grants like the program’s, requires the use of a grant application review process that incorporates unbiased 2 Not reflected in the program’s administrative budget are some of the personal services costs of the consultant and an analyst incurred in administering the program but charged to other programs. For fiscal year 2004–05 these costs amounted to more than $86,500, excluding benefits. 3 The American Indian Early Childhood Education Program is part of the department’s Even Start Office. The statutes that established that program have been repealed and are no longer operative. However, the Legislature has continued to appropriate funds for its purpose. 11 California State Auditor Report 2005-104 grant readers, application screening, scoring criteria, and a scoring process.4 For example, a competitive grant review process could entail independent reviewers awarding points to applicants on the basis of various elements of the application, such as how closely the budget matches identified community needs, how well the goals and objectives relate to the needs, and the degree to which data will be used to measure success. According to a 1987 department report, the original 10 centers funded at the program’s inception were selected competitively from a pool of 26 applicants. Although a competitive selection process may have existed at certain times in the program’s past, we found no evidence that the department used a competitive process to select the centers following the program’s reauthorization in October 2001. As a result, the current funding process does not provide for the participation of other parties that, according to a former program consultant, have expressed interest in applying After telling a prospective for program funding. In fact, the department received an center that no funding unsolicited request for funding from one prospective center was available for new in December 2001. In response the department stated that no centers and that should funding was available for new centers and that should funding funding become available become available, the department would circulate a request it would circulate a for proposal (RFP) to all areas eligible for funds. Accounting request for proposal (RFP) records indicate that the department funded the new center the to all areas eligible for following fiscal year. Yet the department cannot explain how funds, the department it selected the center for funding or demonstrate that it had later funded the center circulated an RFP to any other possible centers. but cannot explain how it made the selection or By failing to document its justification for selecting the new demonstrate that it had center for funding, the department is vulnerable to accusations of circulated an RFP. inequity from potential applicants or current centers that believe their needs exceed the needs of the new recipient. Moreover, without adequate justification for funding the new center, the department cannot demonstrate how it considered statewide needs in making its determination to fund services for that area. The department’s inconsistent and undocumented funding Practices Could Prompt Criticism The department lacks evidence to demonstrate how it determines the amounts centers receive in grant funds. According to current and former program staff, the department 4 A formula-based program guarantees each recipient a certain funding level for each measurable unit without requiring the recipient to apply for the funding. A nonformula- based program, on the other hand, requires a potential recipient to apply for a grant and compete for funding by meeting whatever criteria the grantor requires for approval. California State Auditor Report 2005-104 11 has allocated annual funding amounts to centers based on the amounts they received in the prior year. A 1992 department study reports that the initial funds granted to the original 10 centers were determined based on the centers’ anticipated operational costs, but the department could not provide documentation demonstrating why it deviated from the operational costs identified by centers in the fiscal year 2002–03 budgets when either increasing or decreasing funding amounts. In our review of 29 center applications, we found that the department allocated to 11 centers more funds than they had requested. Additionally, seven centers received amounts less than they requested, and the department directed one center to resubmit its budget because the center had requested less than the department had allocated in the previous year’s grant. In an e-mail to the manager of the office at the time, a former program consultant indicated that she was uncomfortable with the department’s lack of justification for the funding amounts it granted. Without a documented and consistent approach, the department risks an appearance of inequity in its funding determinations. Further, the department cannot justify how it allocates increases in state budget appropriations. For fiscal year 2002–03, when the budget appropriated additional funding to provide for cost-of-living increases and an adjustment for the general increase in student enrollment statewide, our computations indicate that the department generally allocated both increases proportionately among the centers based on their respective allocation amounts at that time. According to the current program consultant, a former program consultant and a former office manager were responsible for apportioning the additional funds. Nevertheless, the department should be able to justify how it arrived at the amounts it provided to the centers for cost-of-living or other adjustments. Without appropriate justification for distributing funds added by the state budget, some centers may believe that they have been unfairly treated. In fiscal year 2004–05 the state budget appropriated additional The department failed to funding of $128,000 over the roughly $4.2 million appropriated allocate a $128,000 fiscal in the previous fiscal year for the program, but the department year 2004–05 budgetary failed to allocate the budgetary increase to any centers, and increase to any centers, the amount is no longer available to the program because and the amount is no the period to obligate those funds has lapsed. The program longer available to the consultant explained that a former consultant was responsible program because the for allocating the increase but could not explain why she did not period to obligate these perform the task. Given that the centers, on average, received funds has lapsed. about $138,450 in fiscal year 2004–05, the additional money would have represented a significant increase in funding for 2200 California State Auditor Report 2005-104 each center. Moreover, the department’s failure to allocate all the funds available may have prevented current centers from augmenting their funding of necessary services or blocked the opening of a new center. As a result, the department failed to maximize the funding intended for meeting the educational needs of American Indians living in the State. Finally, the department has increased certain centers’ allocations without providing explanations of its rationale to all the centers. According to an allocation worksheet in the department’s files, Other than asserting that in fiscal year 2002–03 the department awarded eight centers it made the centers aware a total of $18,000 above their regular allocation amounts. that some additional According to a memorandum from a former program consultant, funds would be available the department made the payments to defray the costs that to participating centers, directors at those centers incurred in participating with the the department could department in meetings of a focus group organized to perform not demonstrate that it various activities related to program administration. However, disclosed to all the centers other than asserting that it made the centers aware that some its process for paying the additional funds would be available to participating centers, focus group participants. the department could not demonstrate that it disclosed to all the centers its process for paying the focus group participants. This may have created the perception that some centers receive more favorable treatment than others. Some American indian Communities in the State may Remain unserved because the department’s Selection and funding Processes Are flawed A 1998 revision to state law indicates that the department is expected to allocate funds, at least in part, for establishing new centers to meet the needs of American Indians not yet served by the program. Figure 2 on the following page is a map indicating the locations of the current centers within California counties. As the figure shows, the department has not established centers in 11 counties in which 500 or more American Indian students are enrolled in public schools. Three of the 11 counties have more than 2,000 American Indian students enrolled. On the other hand, six counties with centers each have fewer than 500 American Indian students enrolled in public schools. Although a center is not limited to providing services only within the county in which it is located, our analysis suggests that the program might not serve American Indian communities in several counties. Recognizing that different centers may be delivering different mixes of services and that the service needs of American Indian students may be different in different California State Auditor Report 2005-104 2211 fiGuRe 2 American indian Student Population enrolled in Public Schools and California indian education Center locations DEL NORTE American Indian Crescent City Students Enrolled in SISKIYOU MODOC California Public Schools Alturas 0–500 Hoopa 501–1,000 Eureka Shasta Lake LASSEN 1,001–2,000 HUMBOLDT TRINITY SHASTA Susanville 2,000+ California Indian Education center TEHAMA Greenville Covelo PLUMAS Chico GLENN BUTTE SIERRA MENDOCINO R Va e l d le w y ood L L A a K k E eport COLUSA S U TTE YUBA NEVADA PLACER Sacramento R EL DORADO Markleeville YOLO O Placerville SONO M M A A RIN NAPA C SO O L N A T N R O A SACR J A M O EN S A T A Q N UIN AMA C D A O L R AVERAS TUOL A U L M PI N N E E MONO COSTA SAN FRANCISCO SAN MATEO O S A A a L N A k M T la A E n D d A STANISLAUS MARIPOSA CLARA MERCED MADERA San Bishop SANTA CRUZ Jose Auberry Big Pine Fresno SAN INYO BENITO FRESNO Lone Pine TULARE MONTEREY KINGS Porterville Bakersfield SAN LUIS OBISPO KERN SAN BERNARDINO SANTA BARBARA Santa Ynez VENTURA LOS ANGELES Fountain Valley RIVERSIDE ORANGE San Jacinto Valley Center IMPERIAL SAN DIEGO Alpine Winterhaven Campo Source: Department of Education. 2222 California State Auditor Report 2005-104 communities, using demographics to identify locations of enrolled American Indian students may not offer the best method of evaluating the equity of funding among current centers. However, given the department’s lack of knowledge of the community needs that centers are currently serving and the absence of justifications for selecting and funding centers, the department risks appearing inequitable and failing to ensure that the program serves the most needy areas. The program consultant told us she is currently in the process of preparing a competitive grant award application and process, which she expects to implement in fiscal year 2007–08 should the Legislature reauthorize the program for 2007. However, because the department is in the preliminary stages of this effort, it is too early for us to evaluate the application or process. The dePARTmenT hAS noT AlWAYS PAid CenTeRS WiThin iTS eSTAbliShed Time fRAme As discussed in the Introduction, the department’s time frame for processing the first grant payments to centers with approved applications is between six and 10 weeks after the governor signs the state budget. On at least one occasion, the department Although the department has communicated a time frame of at least six weeks to the cannot disburse any centers, but it has not consistently adhered to it. Although the payments until it receives department cannot disburse any payments until it receives approvable applications approvable applications from the centers and the state budget and the state budget is is enacted, the significant delay that occurred in one fiscal year enacted, the significant appears to have resulted from its own actions. The department’s delay that occurred in delay in disbursing funds it allocates to centers for the program one fiscal year appears can adversely impact the centers’ delivery of program services. to have resulted from its own actions. During the grant year the department pays centers in three installments of 50 percent, 40 percent, and 10 percent, respectively.5 Because a delay in the release of the first grant payment could hinder a center’s ability to meet current obligations, we computed the average dates for the program’s first payments in fiscal years 2002–03 through 2005–06. As Figure 3 on the following page shows, on average the department made the first disbursements in fiscal year 2002–03 within its payment-processing time frame, was more than eight weeks late 5 In fiscal year 2002–03 most centers received nearly the full amount of their allocation in their first payments, but in subsequent years the department has distributed the payments following the 50 percent, 40 percent, and 10 percent schedule. California State Auditor Report 2005-104 22 in fiscal year 2003–04, again stayed within its payment-processing time frame in fiscal year 2004–05, and was more than two weeks late with the payments issued through November 30, 2005, for fiscal year 2005–06. fiGuRe  Timing of first Payments to Centers 24 California State Auditor Report 2005-104 raeY lacsiF Date state budget signed Average date payment made 6 to 10 weeks after budget signed July 11 Oct. 5 2005–06 Sept. 14 July 31 2004–05 Aug. 2 Dec. 10 2003–04 Sept. 5 Nov. 11 2002–03 July 1 Aug. 1 Sept. 1 Oct. 1 Nov. 1 Dec. 1 Jan. 1 Sources: Budget Act for each indicated year, Department of Education accounting records, and payment processing timeline prepared by program staff. Note: We excluded first payments to four centers (five payments) in fiscal year 2002–03 and to one center in fiscal year 2004–05. These payments were issued considerably later than other payments, and we considered them outliers. The department was not able to explain why the payments were so late in fiscal year 2002–03. The delayed payment in fiscal year 2004–05 occurred because the center did not submit a certification required to receive funding until November 1, 2004. According to the program consultant, the department was late making the first payments to the centers in fiscal year 2003–04 because, as a result of a reorganization, a new division director for the program wanted to review the grant awards and understand the program before releasing the grant award letters that precede payments. Further, the program consultant indicated that, because some centers have not yet submitted applications for approval, their first grant allocations for fiscal year 2005–06 have not been released. In fact, 12 centers had not been issued their first allocations as of November 30, 2005. 24 In computing the average payment dates, we excluded centers that received their first allocations an average of nine weeks beyond the 10-week upper limit, thus avoiding distortion of the average payment dates for the other centers. Specifically, we excluded four centers in fiscal year 2002–03 and one in fiscal year 2004–05. When we asked the department to explain why it was so late in making the first payments to these centers, staff was unable to explain the late payments made to the four centers in fiscal year 2002–03. The department made one first payment in mid-December for fiscal year 2004–05 because it did not receive the center’s certification necessary to receive grant funds until November 1, 2004. Although the department has no control over when the state While the department has budget is enacted, the centers’ uncertainty about the timing no control over when the of their first payments could challenge their ability to plan for state budget is enacted, and deliver services to American Indian students. For example, the centers’ uncertainty in a September 2004 letter to the department praising the about the timing of their program staff’s promptness in making the initial payment to first payments could the centers for fiscal year 2004–05, one center director shared challenge their ability his past experiences as the director of three different centers that to plan for and deliver included very little program structure, inconsistent and arbitrary services to American department decisions, and late payments. The letter described Indian students. how the department’s late payments had forced some centers to lay off staff or shut down until their initial payments were processed and received in November or December—preventing the affected centers from starting up when school opened in August and early September. Some centers need to cover their operating expenses in the summer months, between the end of one fiscal year and the date on which they receive their first payments from the next fiscal year’s grant award. To accommodate these centers, the department instructed them to complete and submit the program extension request forms it provided. After submitting the forms, the centers could use the previous year’s grant funding to pay for services provided in the summer months of the new fiscal year. However, according to an internal communication, the department has been trying to discontinue this practice, in part because the Legislature may question why it is allowing centers to use grant funds appropriated in the prior fiscal year to pay for services provided in the current fiscal year. The departmental communication asserts that the centers are in the same situation as all state contractors and grantees: until the state budget is California State Auditor Report 2005-104 2255 passed, they cannot be sure they will be paid. According to the same document, centers should not provide services in the summer unless they know the funds needed to pay for them are available. Nonetheless, because the department was able to issue the first payments on average six to 10 weeks after the budget was enacted in both fiscal years 2002–03 and 2004–05, the department should be able to disburse the first payments to centers within this time frame every year, thereby affording centers the opportunity to more consistently plan for the delivery of services. The dePARTmenT iS noT ConSiSTenT in iTS PRoGRAm moniToRinG And CommuniCATion Although its handbook provides general guidance on the subject, the department has not effectively monitored the program. For example, according to the program consultant and our own observations, the department has not established a monitoring plan that includes requirements for the centers to report various information, the date such information is to be submitted, and the consequences for failing to meet reporting deadlines. Also, the department has not yet developed a consistent method for delivering technical assistance to centers or established a consistent process for conducting site visits. As a result, the department cannot be sure that centers are meeting the goals intended for the program and that the centers are submitting accurate and complete program and fiscal data. Additionally, the department has not retained complete documentation of the various reports and other program information that centers submit. Inconsistent monitoring and direction can contribute to the department’s uneven program administration and ineffective communication with the centers, particularly when combined with the frequent turnover of program staff. The department has not established a monitoring Plan for the Program The handbook suggests several monitoring techniques to use in overseeing grant recipients, such as collecting program and fiscal reports, providing technical assistance, and conducting site visits. However, the department has yet to develop a monitoring 22 California State Auditor Report 2005-104 plan that incorporates any of these components and thus Although the handbook cannot ensure that each center fully understands and complies suggests several with all program requirements. monitoring techniques to use in overseeing According to the program consultant, the department requires grant recipients, such as that centers submit periodic expenditure reports, annual audit collecting program and reports, year-end reports, and other program data but has no fiscal reports, providing method to verify the accuracy or completeness of the data it technical assistance, and receives. Further, the program consultant told us that in the conducting site visits, years before her tenure, because the department did not follow the department has yet up on missing documents, centers did not always submit them. to develop a monitoring The consultant stated that fiscal year 2005–06 will be the first plan that incorporates year that the department withholds payments when centers do any of these components. not submit the required reports. The department also lacks a system to provide consistent technical assistance to the centers. Despite department staff claims that they respond to telephone inquiries regarding the program, no documentation exists to verify who made the inquiries or what responses staff gave. If documentation of communications did exist, program staff could identify systemic concerns and ensure that the advice and training they provide is consistent. At times program staff have responded to the centers in writing, but the department has not centrally documented these communications to ensure that the centers receive a clear and consistent message. Rather, each staff member has maintained separate electronic files regarding technical assistance. Without centrally documenting the verbal and written technical assistance given, the department risks duplicating its efforts or responding to similar inquiries in dissimilar ways. The manager of the office told us that the office is working on implementing a centralized filing system to allow for easier access to common files. Although the manager did not specify what would be kept in the centralized files, we believe that documentation of the technical assistance provided to the centers should be among the items centrally filed to ensure that such assistance is consistent. The department conducts periodic meetings with center directors throughout the year, and the program consultant asserts that the meetings are an important means of imparting technical assistance. Nevertheless, the consultant acknowledges that, except for recaps of the meetings, the department has not recorded meeting minutes to ensure that each center director California State Auditor Report 2005-104 22 has an accurate record of the technical points covered and We attended a the agreements reached at all the meetings. We attended a directors’ meeting held directors’ meeting held in October 2005 and noted that the in October 2005 and program consultant’s recap of that meeting omitted certain noted that the program key topics such as a discussion of how to develop measurable consultant’s recap of objectives for the program. A center director who cannot attend that meeting omitted a particular meeting must rely on these recaps or notes taken by certain key topics such one of the attendees, which could result in the communication as a discussion of how of incomplete or incorrect information. to develop measurable objectives for the Finally, according to the program consultant, the department program. does not have a monitoring process it uses when conducting site visits of the centers. In fiscal year 2004–05 program staff conducted eight site visits, but lacking a program-specific monitoring method guiding the site visits, staff used the visits to provide technical assistance to the centers. The only form of documentation of these site visits are the personal notes of the staff performing the visits and follow-up letters sent to the visited centers that frequently did not correspond to the notes. Thus, the monitoring process may not be thorough and consistent, and centers may not receive guidance targeting their concerns. The program consultant told us that the department is planning to develop a formalized monitoring process to be implemented in fiscal year 2006–07. Without a clear plan for site visits, the department may not be consistently monitoring the centers to ensure that they are complying with the program’s requirements. The department’s Record keeping for the Program is Substandard The California Government Code and the State Administrative Manual provide criteria regarding document retention. The Government Code requires each department head to establish and maintain an active, continuing program for the economical and efficient management of the records and information collection practices of the agency. The State Administrative Manual requires each department to submit document retention schedules to the Department of General Services for approval at least every five years. According to a manager in the department’s personnel services division, the department expects each unit to prepare a record retention schedule. Finally, the handbook requires grant recipients to maintain records for at least three years from the date of the termination of the grant or the date the final reports are submitted to the department, whichever is later. 22 California State Auditor Report 2005-104 Based on our review of program files, the department does not Based on our review comply with either state or departmental document retention of program files, the requirements. The manager of the personnel services division department does not told us that neither the program nor the office has a record comply with either retention schedule on file. Moreover, our review of program state or departmental files revealed that the department does not have complete document retention documentation supporting the program. For instance, until requirements. October 2005 the department had not established an official grant program file, as required by the handbook, containing items such as awarding documents and papers describing the overall management and monitoring of the program. After we made program staff aware of the requirement, they created an official grant program file, although it still lacks documentation supporting how the centers are selected and how they are funded and monitored, and how they receive technical assistance. Further, the grant recipient files for some centers were missing expenditure reports, audit reports, or year-end reports that centers are required to submit, as well as the grant applications the centers completed. According to the manager of the office, program staff kept documentation relating to the program in their personal files, a practice that may have led to some documentation being inadvertently misplaced when staff members ended their service with the program. The office manager indicates that he is in the process of developing a filing system to ensure that all appropriate documentation is centrally located and retained in the future. The office manager partially attributes the missing documents related to the program to the physical moves of the department from one location to another several years ago and of the office from one floor to another approximately two years ago. Additionally, according to a list created by the fiscal analyst for the program, since January 2002 the turnover of program staff has included five consultants, two analysts, and five managers. The analyst’s list attributes the frequency of turnover to retirements, separations from state service, and transfers to other programs within the department or other state departments. Given its lack of program-specific policies and procedures and frequent staff turnover, the department must ensure that it retains all relevant documentation to avoid losing historical continuity within the program and communicating inconsistent information to the centers. California State Auditor Report 2005-104 22 ReCommendATionS To ensure that it administers the program clearly, consistently, and effectively, the department should develop operating policies and procedures specific to the program and train staff in their application. The policies and procedures should include the following: • A description of the data that centers must annually report to measure program performance and a standardized format for reporting to allow the department to effectively aggregate and consolidate the data for reports to the Legislature and other interested parties. Further, the department should outline the consequences for failing to submit the data. • An equitable process to select centers to receive grant awards and determine their respective funding amounts. • A set time frame that it adheres to for disbursing payments to the centers once their applications are received and approved. The time frame for the first payment can be expressed as a set number of weeks after enactment of the state budget for centers with approved applications. • A centralized filing system that contains all documents pertinent to the grant program, including documentation of the technical assistance provided to the centers. • A monitoring process and plan to ensure that reported fiscal and program information is accurate and complete, including a process for corrective action and departmental follow-up for noncompliance. • A set schedule indicating how long program records are to be kept. To ensure that centers use program funds effectively, the department should ensure that they periodically conduct needs assessments as required by the guidelines adopted by the board. If the Legislature decides to reauthorize the program, it should consider requiring annual or biannual reports from the department to monitor the progress of the program and supplement the report the department submitted to the Legislature by the due date of January 1, 2006. Alternatively, the Legislature might want to extend the life of the program in one- or two-year increments to augment the data available for evaluation. 00 California State Auditor Report 2005-104 We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, ELAINE M. HOWLE State Auditor Date: February 7, 2006 Staff: Doug Cordiner, Audit Principal Almis Udrys Julianna N. Field Toufic Tabshouri California State Auditor Report 2005-104 11 Blank page inserted for reproduction purposes only. 22 California State Auditor Report 2005-104 Agency’s comments provided as text only. California Department of Education 1430 N Street Sacramento, CA 95814-5901 January 19, 2006 Elaine M. Howle, State Auditor* Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Report No. 2005-104 Dear Mrs. Howle: This is the California Department of Education’s (CDE) response to the Bureau of State Audits’ (BSA) draft audit report entitled, “Department of Education: Its Flawed Administration of the California Indian Education Center Program Prevents It From Effectively Evaluating, Funding, and Monitoring the Program.” We appreciate the opportunity to comment on your draft report. Our response to each of your audit recommendations is enclosed (Enclosure 1). Additionally, the CDE is commenting on the BSA draft audit report to provide clarity and perspective (Enclosure 2). If you have any questions regarding the CDE’s response to the draft report, please contact Kim Sakata, Audit Response Coordinator, Audits and Investigations Division, at (916) 323-3560 or by email at ksakata@cde.ca.gov. Sincerely, (Signed by: Sue Hickel for Galvin Payne) GAVIN PAYNE Chief Deputy Superintendent of Public Instruction * California State Auditor’s comments begin on page 41. California State Auditor Report 2005-104  Enclosure 1 CALIFORNIA DEPARTMENT OF EDUCATION’S RESPONSE TO THE RECOMMENDATIONS IN THE BUREAU OF STATE AUDITS REPORT NUMBER 2005-104 Department of Education: Its Flawed Administration of the California Indian Education Center Program Prevents It From Effectively Evaluating, Funding, and Monitoring the Program Recommendation #1 To ensure that it administers the program clearly, consistently, and effectively, the CDE should develop operating policies and procedures specific to the program and train staff in their application. The policies and procedures should include the following: 1. A description of the data that the California American Indian Education Centers (Centers) must annually report to measure program performance and a standardized format for reporting to allow the CDE to effectively aggregate and consolidate the data for reports to the Legislature and other interested parties. Further, the CDE should outline the consequences for failing to submit the data. CDE’s Response: The CDE recognized that improvements were needed in this area, and to this end, designed a new End-of-Year report that was introduced to the directors in October 2005, with additional training planned for the directors during the January 2006 Directors meeting. This End-of-Year report is designed so that all required data can be collected, aggregated, and consolidated. The CDE completed its consolidated report for the Legislature based on the information submitted by the Centers and information gathered through site reviews throughout the year. The CDE will inform the Centers that they shall, as a condition of funding, collect and report site data including: the number of pupils served, the services provided to pupils, academic performance of pupils served, the extent to which the program goals, as set forth in Article 6 (commencing with Section 33380) of Chapter 3 of Part 20, are being met and other information as deemed necessary by the CDE for the purposes of evaluation. 2. An equitable process to select Centers to receive grant awards and determine their respective funding amounts. CDE’s Response: 1 Currently, the 2007-10 competitive application is being created. This application will require each submitting agency to include a complete and comprehensive needs assessment documenting the targeted community needs and supporting the services proposed in the application. The CDE will solicit applications from qualified agencies across California by announcing the availability of grant funds to all qualified agencies. 44 California State Auditor Report 2005-104 Enclosure 1 3. Adhering to a set time frame for disbursing payments to the Centers once their applications are received and approved. The time frame for the first payment can be expressed as a set number of weeks after enactment of the state budget. CDE’s Response: For the past two years the CDE has adhered to a set time frame. However, the CDE does not agree that a set time frame be linked solely to the enactment of the 2 state budget. It would be irresponsible to set a time frame that does not take into consideration the receipt of an approvable application. Thus, the set time frame must be expressed in a number of weeks after the receipt of an approvable application and the enactment of the state budget. The CDE has had this process in place for the past two years. The process has been explained to the Center directors and continues to be communicated. 4. A centralized filing system that contains all documents pertinent to the grant program, including documentation of the technical assistance provided to the Centers. CDE’s Response: The CDE has established and will refine a centralized filing system for this grant program. 5. A monitoring process and plan to ensure that reported fiscal and program information is accurate and complete, including a process for corrective action and departmental follow- up for noncompliance. CDE’s Response: Over the last 12 months, CDE staff conducted 20 on-site Center visits for the purpose of providing technical assistance and monitoring the Centers’ adherence to application and assurances requirements. The CDE will refine and formalize this monitoring process to ensure each Center’s programmatic and fiscal integrity. It is anticipated that this process will be fully implemented by the 2007-10 cycle; however, it is incumbent 3 on the Legislature to earmark the sufficient administrative funds to fully develop and implement this monitoring process. This monitoring process will place added administrative responsibilities on CDE staff and will require formal monitoring site visits. 6. A set schedule indicating how long program records are to be kept. CDE’s Response: The CDE is currently updating the existing records retention policy for this grant program. California State Auditor Report 2005-104 55 Enclosure 1 Recommendation #2 To ensure that Centers use program funds effectively, the CDE should ensure that Centers periodically conduct needs assessments as required by the guidelines adopted by the State Board of Education. CDE’s Response: The 2007-10 application cycle will require applicants to conduct a formal needs assessment. Recommendation #3 If the Legislature decides to reauthorize the program, it should consider requiring annual or biannual reports from the CDE to monitor the progress of the program and supplement the report the CDE submitted to the Legislature on January 1, 2006. Alternatively, the Legislature might want to extend the life of the program in one- or two-year increments to augment the data available for evaluation. CDE’s Response: 3 Should the Legislature enact these recommendations, sufficient administrative funds should be allocated to implement these requirements.  California State Auditor Report 2005-104 Enclosure 2 CALIFORNIA DEPARTMENT OF EDUCATION’S CLARIFICATION TO THE BUREAU OF STATE AUDITS REPORT NUMBER 2005-104 Department of Education: Its Flawed Administration of the California Indian Education Program Prevents It From Effectively Evaluating, Funding, and Monitoring the Program Summary: Results in Brief Third Paragraph: This paragraph states the California Department of Education (CDE) did not ensure the collection of data necessary to complete the required Legislative Report. The CDE 4 contends that the Legislative Report meets statute requirements. The statute states that the “State Department of Education shall report the consolidated results of the yearly evaluation data and self reviews . . .” The CDE completed its consolidated report based on the information submitted by the Centers and information gathered through site reviews throughout the year. Furthermore, this required report as well as other unfunded mandates affecting the CDE programs, were addressed in a September 2001 letter from the former State Superintendent of Public Instruction. This letter informed the Legislature that the CDE would be unable to comply 3 with the program’s evaluation and report requirements without the necessary state operation resources. Consequently, in subsequent years, not all data were submitted by the Centers, however, each did submit annual reports. In 2004-05, the CDE American Indian Education staff’s duties were modified to include the completion of the Legislative Report, which was delivered to the Legislature December 13, 2005. Fourth Paragraph: This paragraph contends the CDE cannot justify the basis or selection process for Centers funded in the 2002-07 cycle. The CDE agrees that this program must use a competitive process and will do so in the new cycle, 2007-10. Fifth Paragraph: The CDE must wait until the Centers submit approvable applications and 2 must wait for a signed state budget before the time frame of six to ten weeks can begin. The time frame for disbursement of payments cannot solely be based on a signed state budget. The CDE has a responsibility to hold funds until an approvable application is on file. Introduction: Background Third Paragraph: When the program was first implemented the state budget allocated12.5 percent for administration. Over the years the state operation allocation was removed from the state budget. For 2004-05, only $72,000 was available for the American Indian Education 3 Centers (AIEC) program administration. The CDE recommends that a reasonable percent be allocated for state operations from the amount authorized for this program in the state budget (12.5 percent equates to $562,506 for fiscal year 2006-07). Introduction: Funding 2 First Paragraph: The CDE contends that the Bureau of State Audits (BSA) ignores the requirement that funds cannot be disbursed without receipt of an approvable application. The submission of an approvable application and the passage of the state budget should trigger the time frame for disbursements. California State Auditor Report 2005-104  Enclosure 2 Scope and Methodology 2 Figure 1: The BSA fails to place in its chart the Centers’ responsibility to submit an approvable application. AUDIT RESULTS The Department of Education does not know how the California Indian Education Program is performing Page 14: In the first paragraph, the BSA states that the Department “did not provide guidance 4 to the Centers to ensure that the data reported were meaningful and comparable.” The CDE contends that the Legislative Report meets the legislative requirements. The statute requires that the “State Department of Education shall report the consolidated results of the yearly 5 evaluation data and self reviews . . .” This paragraph also states that some Centers received 2005-06 funding without submitting reports as required by law. This statement is misleading. The report fails to recognize that these Centers had approvable applications and were issued their first payment before their 2004-05 End-of-Year report was due. Page 14-16: The BSA is correct that the end of year reporting requirements have changed 6 over the years. However, the BSA fails to recognize the efforts of the CDE to address these problems. The current End-of-Year reporting requirements ask the Centers to provide an unduplicated count of students served, academic level of students served, types and duration of services, and staff time allotted to services. There are no plans to make changes during the current funding cycle to this report. The BSA indicates that the information regarding the 2004 California Standards Test (CST) scores was not completed by all the Centers and that in some cases, the Centers failed to include history or science scores. While this statement is accurate, the BSA failed to recognize 7 that these content areas are not tested at all grade levels. Consequently, test scores are not reported for tests that are not administered. Page 17: The first paragraph states that the Centers are not asked for program outcomes. While this was true in the past, the 2005-06 End-of-Year report does ask for outcomes. The 6 BSA fails to acknowledge the Department’s efforts to resolve this issue. Page 18: The CDE agrees that a needs assessment is crucial in designing AIEC programs. The 2007-10 funding cycle will be competitive and will require a comprehensive needs assessment. At this time this program is in year four of a five-year cycle. Therefore, a needs assessment is not feasible. The second paragraph states that, “As a result, the current program consultant stated that she 8 had to start from scratch to understand the program and how to administer it.” This statement is incorrect and should be stricken.  California State Auditor Report 2005-104 Enclosure 2 The Department’s Report to the Legislature is insufficient to adequately evaluate the performance of the program Page 20: The BSA states that the CDE “could have” done trend analysis and compared achievement gains for Centers over a three-year period. However, these types of analyses, while worthwhile, were not required by statute. The CDE contends it has followed California 4 Education Code 62004.14 that requires the CDE to report the consolidated results for the yearly evaluation data and self-reviews that were provided in the End-of-Year reports from the Centers. The evaluation also includes recommendations, as required. The BSA states the CDE could have compared the performance data of students served in the Centers to students who did not receive services. It is not feasible at this time due to the fact that the CDE does not collect data on American Indian students who are not receiving services. Page 22: The BSA states the Department does not provide analysis to support additional positions. However, the Legislative Report does discuss the statutory requirement for the existence of an American Indian Education Office. Also, the Legislative Report indicates the increase in formal monitoring and technical assistance will require greater administrative 3 resources. Furthermore, the BSA does not note that the current staffing is not adequate to meet the current responsibilities. Ignoring Its Written Policies, The Department Has Not Used A Documented And Objective Process To Select s For Funding 1 Page 22: The CDE agrees that an objective process to select AIEC grant recipients is necessary. Currently, the 2007-10 competitive application is being created. This application will require each submitting agency to include a complete and comprehensive needs assessment documenting the targeted community needs and supporting the services proposed in the application. The CDE will solicit applications from qualified agencies from across California by announcing the availability of grant funds to all qualified agencies. The Department Has Not Always Paid s Within Established Time Frame 2 Page 27-30: The BSA fails to stress the importance of the Centers submitting approvable applications in a timely manner. The CDE staff provides written technical assistance to the Centers that submit unapprovable applications, specifically noting the changes that need to be made. Using e-mail, American Indian Education staff works with staff to assure the 9 corrections are complete and acceptable before the resubmission. The BSA also fails to consider that the Centers are responsible for maintaining their fiscal solvency. The CDE funds agencies to provide services. It is the responsibility of each agency governing board to plan for fiscal short falls. It is the responsibility of the CDE to allocate grant funds in a fiscally responsible manner; this includes waiting for the enactment of the new state budget and the receipt of an approvable application. California State Auditor Report 2005-104  Enclosure 2 The Department Is Not Consistent In Its Program Monitoring And Communication Page 30-35: The CDE agrees that a formal monitoring process is necessary and is taking steps to create this process. This year, CDE staff began this process by conducting 20 on-site visits for the purpose of providing technical assistance and monitoring the Centers’ adherence to application and assurances requirements. Other General Concerns 0 Throughout the audit report, the BSA fails to acknowledge the steps the CDE has taken to resolve prior year issues. 4400 California State Auditor Report 2005-104 CoMMenTS California State Auditor’s Comments on the Response From the Department of Education To provide clarity and perspective, we are commenting on the response to our audit report from the Department of Education (department). The numbers correspond with the numbers we have placed in the department’s response. 1 We find this to be an encouraging first step. However, because the department did not provide any details concerning how it would develop the program-specific policies and procedures we recommended or how it would ensure equity in both the selection of new California Indian Education centers (centers) and the funding determinations made for new and existing centers, we look forward to learning more about these processes through the department’s periodic follow-up responses of its efforts to implement our recommendations. 2 We considered the department’s need to receive and approve centers’ applications before disbursing payments. We already noted this fact on pages 3, 7 (both in the text and in the note to Figure 1), 23, and 30. However, to address the department’s concern and to add clarity, we further reiterated this requirement on pages 2, 3, 7, 23, and 30. 3 As we state on page 18, while the department recommended in its evaluation report to the Legislature that additional positions and funds be appropriated to provide administrative oversight of the American Indian educational programs, it did not provide an analysis to support the need for additional positions or funding. 4 The department contends that its evaluation report to the Legislature meets the law’s requirements. For the reasons stated on pages 16 through the top part of 18, we disagree. Moreover, even if we agreed with the department’s interpretation that it only needed to consolidate and report the results of one year’s worth of the centers’ evaluation data, we would still find its report lacking. As we state on page 16, the department’s report includes data reported by less than 100 percent of the centers. For example, in discussing academic performance, the department’s report presents California Standards Test (CST) scores. However, as we point out on page 14, the department’s California State Auditor Report 2005-104 4411 files included CST test scores for only 23 of the 30 centers, and many of these centers did not include the required history or science portion of the CST. 5 The department appears to have misread our report. On pages 11 through 12 we state that the law that took effect in October 2001 requires that centers report certain evaluation data as a condition of receiving funding, and that some of this data is found in the centers’ year-end reports. However, as we point out, not all of the centers have submitted year-end reports as required. Thus, some centers have received funding without meeting the requirements of law. Also, contrary to the department’s assertion, we do not state “that some Centers received 2005–06 funding without submitting reports as required by law.” The law contains no such requirement. Instead, what we state on page 12 is the law’s requirement that centers must report certain evaluation data as a condition of receiving funding. 6 We recognize on page 13 that the department modified its year-end report to include a survey of center services. However, as we state on page 14, the survey will only measure outputs, such as the proportion of time centers spend providing services, not outcomes measuring how the services affect those who receive them. As for the other features of the new year-end report, as we further state on pages 14 and 15, the department’s efforts were too preliminary to evaluate. 7 While acknowledging that our statement is accurate, the department falsely implies that we failed to recognize that history and science is not tested at all grade levels. In fact, we considered test scores missing only when such test scores were required to be reported. On page 14 we clearly state that many centers did not include the required history or science portion of the CST. 8 This sentence is based on the program consultant’s signed statement attesting to its accuracy. However, we honored the department’s request and deleted it. 9 The department’s statement is misleading. We do not take issue with the department’s position regarding centers’ funding, which we present on pages 25 and 26. Moreover, the fiscal solvency of the centers is beyond the scope of the audit as stated on page 9. Nonetheless, we believe that the department should allocate funds to the centers as promptly as possible, because as 4422 California State Auditor Report 2005-104 we point out on page 23, the department’s delay in disbursing funds it allocates to centers for the program can adversely impact the centers’ ability to deliver services. 0 Where appropriate we acknowledge the department’s plans for change. On page 3 we included the department’s assertions that it is attempting to improve its administration of the program by proposing more detailed reauthorizing legislation and by developing a plan for monitoring the centers. Also, on page 13 we note that the department has modified its year-end report to include a survey of center services, on page 27 we reported that the Migrant, Indian, and International Education Office is working on a centralized filing system to allow for easier access to common program files, and on page 28 we acknowledge the department’s plan to develop a formalized monitoring process to be implemented in fiscal year 2006–07. California State Auditor Report 2005-104 44 cc: Members of the Legislature Office of the Lieutenant Governor Milton Marks Commission on California State Government Organization and Economy Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press 4444 California State Auditor Report 2005-104