Summary

California Children and Families Commissions: Some County Commissions’ Contracting Practices Are Lacking, and Both the State and County Commissions Can Improve Their Efforts to Find Funding Partners and Collect Data on Program Performance July 2004 2003-123 rotiduA etatS ainrofilaC S T I D U A E T A T S F O U A E R U B The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TTY (916) 445-0033 OR This report is also available on the World Wide Web http://www.bsa.ca.gov/bsa/ The California State Auditor is pleased to announce the availability of an on-line subscription service. For information on how to subscribe, please contact the Information Technology Unit at (916) 445-0255, ext. 456, or visit our Web site at www.bsa.ca.gov/bsa Alternate format reports available upon request. Permission is granted to reproduce reports. � � � ��������� ���� ������ ������������� ������������������� ������������ ����������������������� July 15, 2004 2003-123 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the Bureau of State Audits presents its audit report concerning our review of the California Children and Families Commission (state commission) and five local county commissions. The goal of the state and county commissions is to improve the early development of children from prenatal to age 5 using tax revenues generated from the November 1998 passage of Proposition 10. This report concludes that although the state commission consistently followed state contracting rules, some county commissions lacked well-defined or documented policies and practices for awarding contracts to service providers. In addition, we found that not all county commissions disclose to the public the noncompetitive nature of some of their funding decisions. However, other contracting practices of each county commission help to ensure a prudent allocation of funds. The June 30, 2003, fund balances of the county commissions are significant; however, most are already earmarked for a variety of purposes. In the future, a few major initiatives will reduce some county commissions’ financial reserves while others’ reserves will remain high. In addition, the investment practices of the state and county commissions help safeguard their financial resources. Although the state and county commissions acknowledge the important role funding partners can play, we found that they have received little funding from sources other than Proposition 10 tax revenues. In addition, some county commissions lack a clear commitment to limit their administrative spending. Finally, we found that the state and county commissions have only recently begun to evaluate program effectiveness and so far have mainly reported demographic and service output data rather than performance outcomes. Respectfully submitted, ELAINE M. HOWLE State Auditor ������������������� �������������������������������������������������� ���������������������������������������� ������������������ CONTENTS Summary 1 Introduction 7 Chapter 1 Not All County Commissions Follow Well-Defined Policies and Procedures When Allocating Funds 17 Recommendations 29 Chapter 2 As Most Fund Balances Grow, Some County Commissions Promise to Significantly Reduce Them 31 Recommendations 42 Chapter 3 Lacking Data on Program Outcomes, the Commissions Cannot Yet Tell Whether Funded Projects Have Had a Positive Impact 45 Recommendations 52 Responses to the Audit First 5 California, California Children and Families Commission 55 California State Auditor’s Comment on the Response From the First 5 California 63 First 5 El Dorado 65 California State Auditor’s Comment on the Response From First 5 El Dorado 69 First 5 Kern 71 California State Auditor’s Comment on the Response From First 5 Kern 73 First 5 Los Angeles County 75 California State Auditor’s Comment on the Response From the First 5 Los Angeles County 79 First 5 Commission of San Diego County 81 First 5 Santa Clara County 85 California State Auditor’s Comments on the Response From the First 5 Santa Clara County 89 California State Auditor Report 2003-123 11 SUMMARY RESULTS IN BRIEF In November 1998, Proposition 10 established the California Children and Families Act of 1998 (Act) to improve the early Audit Highlights . . . development of children from prenatal to age 5 and to ensure those childrens’ readiness to enter school. To implement these Our review of the state and goals, the Act established the California Children and Families five counties’ California Program (Children and Families Program) and created the California Children and Families Commissions funded by Children and Families Commission (state commission) as the lead Proposition 10 tax revenues agency. The Act requires participating counties to establish local revealed the following: county commissions that allocate Children and Families Program þ The state commission funds to local service providers for early childhood development consistently followed efforts that focus on community awareness, education, health care, contracting rules social services, and research. Current programs, such as the School applicable to all state Readiness Initiative, aim to improve the health and early learning agencies, but some county commissions lacked well- of some of California’s neediest preschoolers. The Children and defined and documented Families Program is funded by a tax on cigarettes and other tobacco policies and practices for products (Proposition 10 tax revenue), with $560 million available awarding contracts to for early childhood development programs in fiscal year 2002–03. service providers. þ To monitor service providers, In reviewing how the state commission and a sample of five county commissions county commissions have allocated this revenue, we found that require them to submit quarterly progress the state commission has consistently followed contracting reports as a condition of rules applicable to all state agencies. However, our review receiving payment. of some county commissions found a lack of well-defined and þ The county commissions documented policies and practices for awarding contracts to maintained significant community service providers, a deficiency that may confuse the fund balances as of public about the methods these commissions use to allocate funds. June 30, 2003, but have State contracting rules do not apply to the county commissions, earmarked most of these fund balances for and the Act requires only that each county commission spend its specific purposes. resources in accordance with a locally adopted strategic plan. Under the Act, each participating county must establish a local county þ Although the state and commission, either as an agency of the county or as a separate county commissions acknowledge the legal entity. Four of the five county commissions we reviewed importance of funding are separate legal entities, and our legal counsel has advised us partners, the commissions that, while under certain circumstances these commissions may have received little funding be required to follow the contracting rules of the counties they outside their Proposition 10 tax revenues. represent, the county ordinances for all of the county commissions we reviewed do not impose specific contracting practices on these continued on next page . . . commissions. Accordingly, to assess the county commissions’ practices for awarding contracts to community service providers, we examined their self-imposed contracting policies. California State Auditor Report 2003-123 11 þ Some county Although the Act emphasizes local allocation decisions, county commissions lack clear commissions need to establish and consistently follow clear, policies limiting their well-documented practices for awarding contracts to instill administrative spending. public confidence. However, some county commissions’ þ State and county decisions for allocating funds are clouded by a lack of commissions have only documentation or undermined by poorly defined policies. For recently begun to evaluate example, Kern County’s commission, known as First 5 Kern, program effectiveness and so far have mainly reported awarded contracts to service providers under its policy to use demographic and service any contracting procedure it deems to be in its best interest. output data rather than When clear policies for allocating funds do exist, they are not performance outcomes. always followed. First 5 Santa Clara ignored its policy of limiting the amount of an unsolicited grant to $15,000 by awarding a $1 million grant to a children’s museum. The public may also be confused by some county commissions’ allocation of funds through noncompetitive contracting practices that are not entirely open to public scrutiny, as it can raise concerns about whether service providers are competent and charge a fair price. Because state guidance and restrictions on noncompetitive contracts do not apply, the county commissions are free to allocate funds noncompetitively. Some county commissions create public confidence in their noncompetitive allocation processes by awarding contracts at regularly scheduled public meetings, providing the community a chance to express any concerns and scrutinize the choice of service provider. However, some commissions do not always disclose complete and accurate information to reassure the public that funds are being allocated appropriately. As another means of allocating resources to service providers, the state commission and the county commissions we visited also award contracts to service providers through the solicitation of proposals. This method helps to ensure a fair and appropriate allocation of funds through such techniques as independent review panels to make funding recommendations and requirements that potential service providers submit qualifications, budgets, and scopes of work. Regardless of the allocation method, the ultimate decision on whether to grant funds to service providers rests with the state commission and each locally appointed county commission. To address any potential conflicts of interest associated with these decision makers, the state and each county commission we reviewed have implemented conflict-of-interest policies and practices based upon existing state statutes. 22 California State Auditor Report 2003-123 California State Auditor Report 2003-123 33 To monitor their service providers, county commissions require them to submit quarterly progress reports as a condition of receiving payment. These reports allow the commissions to ensure that service providers are performing in accordance with their contracts. To show that they are spending funds in line with approved budgets, service providers must also submit quarterly financial summaries of their expenses. Having spent as little as 15 percent to as much as 67 percent of their revenues on early childhood development programs, the county commissions we reviewed maintained significant fund balances as of June 30, 2003. “Fund balance” refers to the difference between assets, primarily cash and investments, and liabilities. However, several county commissions have earmarked most of these fund balances for specific purposes, such as to pay or extend existing contracts. One county commission’s financial plan envisions reducing its fund balance significantly by spending the money on new preschool and children’s health programs already approved by its commissioners. Other county commissions’ financial plans intend to maintain substantial balances to sustain existing programs in the future as Proposition 10 tax revenues decline. Current investment practices adequately safeguard these funds, with cash balances invested in county pooled investment programs restricted by state law to conservative financial instruments. State and county commissions acknowledge the important role funding partners can play in supporting early childhood development programs. However, other than in-kind contributions, the commissions have received little funding outside of their state allocations. Although the Act grants the state and county commissions authority to apply for money and services from individuals, corporations, and foundations, efforts by the state and county commissions to create such partnerships have focused primarily on pooling their Proposition 10 tax revenues rather than on identifying and obtaining funds from outside sources. Although publicly committed to using every dollar possible for direct services to eligible children, some county commissions lack clear policies limiting their administrative spending. Only one county commission we reviewed has an ordinance limiting its administrative costs. Also, statements by county commissions defining what constitutes administrative spending differ. To compare the county commissions’ administrative spending, we developed a working definition of administrative costs, and, 22 California State Auditor Report 2003-123 California State Auditor Report 2003-123 33 using that definition, found that some commissions spend a greater proportion than others on such activities. Also, travel policies and reimbursement practices, while generally patterned after their respective counties, varied at the county commissions we reviewed, with some commissions spending a slightly larger percentage of their administrative costs than others on travel. The Act requires the state and county commissions to evaluate the impact of their respective programs, including how funds are spent, the progress toward goals and objectives, and specific performance outcomes, or effects, measured through appropriate indicators.1 However, the state and five county commissions show minimal reporting of program performance data, or outcome-based data, using reliable indicators. Instead, the state and county commissions have only recently begun to evaluate program effectiveness and so far have mainly reported demographic and service output data rather than performance outcomes. Having begun awarding funds in 2001, the state commission is in the early stages of gathering information for two long-term assessments of its programs by two external evaluators. The county commissions we reviewed have been gathering data from service providers, but external evaluators cite various limitations of those data. The evaluators observe that most of these data address outputs instead of specific performance outcomes and that some service providers are capturing little or no useful information. Reviews of county commission operations also do not always afford a comprehensive and objective look at performance. Although each county commission we visited undergoes an annual independent financial audit of its operations, following well-established and generally accepted standards, similar reviews of the county commissions’ performance are not occurring. RECOMMENDATIONS To ensure the appropriate use of program funds and instill public confidence, the Kern and Santa Clara county commissions should adopt and follow well-defined policies to guide their allocation efforts and should also maintain sufficient documentation to support their allocation decisions. 1As we explain later, performance outcomes are changes in behavior, knowledge, or situation accomplished by providing a service output. For example, an appropriately funded program might demonstrate that children who attend its classes on how to brush their teeth properly (output) have significantly less plaque on their teeth (outcome) than before they attended the classes. 44 California State Auditor Report 2003-123 California State Auditor Report 2003-123 55 To address the sustainability of their programs, the state and county commissions should continue to take action to identify and apply for any available grants, gifts, donations, or other sources of funding. To demonstrate its commitment to keeping administrative costs low, each county commission, which has not already done so, should define what constitutes its administrative costs, set a limit on the amount of funding it will spend on such costs, and annually track expenditures against this self-imposed limit. To ensure that county commissions are basing their funding decisions on outcome-based data, as required by the Act, they should address the concerns expressed by their evaluators to ensure that service providers are collecting these data. To provide a meaningful assessment of annual performance, the state commission should require each county commission to conduct an annual audit of its performance prior to any future revenue allocations. Such audits should be objective and should follow guidelines designed to critically assess each county commission’s performance. STATE AND COUNTY COMMISSIONS’ COMMENTS The state commission believes that its in-kind funding commitments from public and private entities deserve recognition as successful efforts by the state commission to leverage its funding and secure substantial financial contributions. The state commission believes it has in place the infrastructure to collect outcome data and assess over time the programs funded by the state and county commissions, and has been considering methods of strengthening the existing requirements of the Act relating to audits and data reporting. First 5 El Dorado stated it maintains a sustainability fund that it believes is a prudent way to protect currently funded programs for a two-year period, however, this fund is not separately reported in its audited financial statements as of June 30, 2003. First 5 El Dorado agrees that it has no written policies regarding administrative costs, and while it has taken various steps to control these costs, it will develop and adopt administrative cost policies. Finally, First 5 El Dorado stated that it continues to review the evaluation process to assure that it is effective and provides reliable data. 44 California State Auditor Report 2003-123 California State Auditor Report 2003-123 55 First 5 Kern agrees with all our recommendations and will change its process to ensure it maintains adequate documentation of its decisions regarding funding allocations to service providers. It will also ensure that any concerns expressed by independent evaluation committees will be resolved and the evidence retained. First 5 Kern will also continue to explore opportunities for obtaining other sources of funding and has or will continue to address the concerns of its external evaluator regarding the quality and quantity of performance outcomes collected from First 5 Kern’s service providers. First 5 Los Angeles is concerned that the working definition we developed for administrative costs includes costs that it considers to be program costs. First 5 Los Angeles stated its external evaluators are in the process of completing the analyses for two of its four initiatives and will provide summary reports in the Fall of 2004, while the evaluation of its other two initiatives are in the early stages and aggregate outcome data is not yet available. First 5 San Diego acknowledges that it has no set limit for its administrative costs but is committed to keeping them as low as possible. First 5 San Diego also cited three funding designations that occurred subsequent to those reflected in its June 30, 2003, audited financial statements that appear in our report. First 5 San Diego also provided information concerning claimed programmatic outcomes and added context regarding the length of time and expertise needed to gather and effectively analyze data intended to measure performance outcomes. First 5 Santa Clara believes that our report lacks an understanding of the extensive collaborative process it uses. It also asserts that our definition of administrative costs fails to include direct service costs and therefore does not account for First 5 Santa Clara’s employees that provide direct services to children and their families. n 66 California State Auditor Report 2003-123 California State Auditor Report 2003-123 77 INTRODUCTION BACKGROUND In November 1998, voters passed Proposition 10, the California Children and Families Act of 1998 (Act),2 intending to create an integrated, comprehensive, and collaborative system of information and services to achieve optimal early childhood development and ensure that children are ready to start school. The Act established the California Children and Families Program (Children and Families Program) to promote, support, and improve the early development of children, prenatal to age 5. The Children and Families Program aims to fulfill this mission through programs and resources that emphasize community awareness, education, nurturing, child and health care, social services, and research. To fund the Children and Families Program, the Act added a tax of 50 cents per pack on cigarettes and an equivalent tax on other tobacco products. The Act also created the California Children and Families Commission (state commission), now known as First 5 California, and allowed each county to create its own commission (county commission) to administer its programs. Counties may also join efforts to create joint county commissions. THE CALIFORNIA CHILDREN AND FAMILIES TRUST FUND The Act created the California Children and Families Trust Fund (trust fund) as the central repository for tax revenues it generates. However, not all of this revenue is available to the programs supported by the state and county commissions. Because additional taxes may reduce tobacco consumption and thus the tax revenue generated, the Act requires the trust fund to reimburse other programs funded with cigarette taxes established by prior legislation for their projected losses, on an annual basis. In November 1988, 10 years before Proposition 10, Proposition 99 levied a cigarette tax of 25 cents per pack to fund health and environmental programs and services. Further, state law created the Breast Cancer Fund and imposed a tax of 2 cents per pack on cigarettes beginning January 1, 1994, to finance research and early detection services for uninsured and underinsured women. Therefore, the Proposition 99 and Breast 2 Since its passage, the Act has been amended. Thus, when we refer to the Act throughout this report, we are referring to the Act as amended. 66 California State Auditor Report 2003-123 California State Auditor Report 2003-123 77 Cancer Fund programs must be reimbursed for any reduction in funding resulting from reduced tobacco consumption due to the tax imposed by the Act. The State Board of Equalization developed tobacco consumption models that compare actual tobacco consumption levels with levels projected to have occurred without the additional tax. Using these models, the trust fund paid $35.5 million generated by taxes imposed by the Act in fi scal year 2002–03 to the Proposition 99 and Breast Cancer Fund programs. After this reimbursement, $560 million was available for the state and county commissions’ early childhood development programs in that fi scal year. We refer to the amount available for the state and county commissions’ programs as Proposition 10 tax revenues. THE STATE AND COUNTY COMMISSIONS The Act requires the state commission to spend its 20 percent Proposition 10 tax Under the Act, the state commission serves as lead revenue allocation as follows: agency, receiving 20 percent of the Proposition 10 • Six percent for mass media tax revenues to provide technical assistance to the communication to the general public. county commissions, conduct program research • Five percent to ensure that children are and evaluations, manage public media campaigns, ready to enter school and for programs increase availability and access to child care relating to education. facilities, and establish educational programs. The • Three percent to ensure that children are state commission must spend its 20 percent share ready to enter school and for programs in accordance with guidelines imposed by the relating to child care. Act (see the text box). In defi ning responsibilities, • Three percent for research and the Act requires the state commission to adopt development of best practices and standards for early childhood guidelines for an integrated and comprehensive development programs and services. statewide program that promotes, supports, and improves early childhood development. It also • One percent for administrative costs. requires the state commission to defi ne the results • Two percent for any activity other than to be achieved by these adopted guidelines and to administrative costs. collect and analyze data measuring progress toward attaining those results. The state commission consists of seven voting members: three appointed by the governor, including the chairperson; two appointed by the speaker of the Assembly; and two appointed by the Senate Rules Committee. The secretary for education and the secretary of health and human services or their designees serve as ex offi cio, nonvoting members of the state commission. 88 California State Auditor Report 2003-123 California State Auditor Report 2003-123 99 Counties that develop and implement local early childhood development programs consistent with Proposition 10 tax revenue allocations during fi scal year 2002–03 for the fi ve counties the Act’s goals and objectives are entitled to a share we reviewed: of the remaining 80 percent of Proposition 10 tax • Los Angeles $134,058,535 revenues provided they adopt an ordinance with the provisions specifi ed in the Act. According to • San Diego $31,995,984 the state commission’s annual report for fi scal year • Santa Clara $23,094,970 2002–03, all 58 counties have established county • Kern $10,015,510 commissions and have received disbursements of Proposition 10 tax revenues. Each county’s • El Dorado $1,448,340 allocation is based on the proportion of births in that county compared to the total statewide Sources: County commissions’ audited births during the same period. The Act emphasizes fi nancial statements for fi scal year 2002–03. local decision making and allows each county commission to spend its revenue allocation for purposes authorized by the Act and in accordance with a locally adopted strategic plan. The text box shows the fi scal year 2002–03 Proposition 10 tax allocation received by each county commission we reviewed. Each county board of supervisors appoints a fi ve- to nine-member county commission that must include a member of the board of supervisors and at least two members from among the county health offi cer and those who manage county functions such as behavioral health services, social services, or tobacco prevention and treatment services. The remaining members can be drawn from recipients of services included in the strategic plan, county managers, or representatives of organizations that work in early childhood development—such as child care resource or referral agencies; community-based organizations; school districts; and medical, pediatric, or obstetric associations. Table 1 on the following page gives the composition of the county commissions we visited. The Act allows counties to establish these commissions either as county agencies or as separate legal public entities. Both of these types of county commissions are governed by the county ordinances that created them and by self-imposed bylaws. Four of the fi ve counties we reviewed—El Dorado, Kern, Los Angeles, and Santa Clara—established their commissions as separate legal entities. The Act contains various provisions that apply to this type of county commission and make it clear that it is a public entity generally to be treated like other public entities. In Chapter 1, we discuss this issue more fully. The fi fth county commission we reviewed, First 5 San Diego, was established as an agency of the county. 88 California State Auditor Report 2003-123 California State Auditor Report 2003-123 99 TABLE 1 Composition of the County Commissions We Reviewed Members of the county commissions represent the following areas: Los Angeles* San Diego Santa Clara* Kern* El Dorado* Member of the board of supervisors x x x x x County health officer x x x x x Representative of county childrens’ services x x Representative of county public health services x x x x Representative of county behavioral health services x x x x x Representative of county social services x x x Representative of county substance abuse prevention and treatment services x x Recipient of project services x Educator specializing in early childhood development x x x x x x x x Representative of a local child care resource or referral agency x Representative of local organization for prevention or early intervention for families at risk Representative of community-based organization that promotes nurturing and early childhood development x x x x x x x x Representative of local school districts x x x x x Representative of local medical, pediatric, or obstetric association or society x x x x Source: Commissioner résumés and/or applications to the county commission. *For these county commissions, the total number of members may appear to be more than nine, the maximum allowed by the Act, because some members represent more than one area. STRATEGIC PLANS AND FUNDING PRIORITIES The state commission adopts an annual strategic plan outlining its services and programs. According to its most recent strategic plan, the state commission has committed $207 million over the next four years to its signature School Readiness Initiative targeting the State’s underperforming elementary schools. This initiative is a state and county commission partnership that requires matching funds from county commissions for a total state and county commitment of $413 million. With this initiative, the state and county commissions hope to improve reading and academic achievement, reduce the rate of grade 1100 California State Auditor Report 2003-123 California State Auditor Report 2003-123 1111 retention and placement in special education, lower the rates of problem behavior and disciplinary problems, and ultimately improve high school graduation rates. While the Act specifies in what proportion and on what types of activities the state commission may spend its Proposition 10 tax revenues, no such guidelines apply to the county commissions’ expenditures. Instead, the Act permits the county commissions to spend their share of the remaining 80 percent for purposes authorized by the Act and in accordance with locally adopted strategic plans. The Act requires each county commission to adopt an adequate and complete county strategic plan to support and improve early childhood development within the county. The strategic plan must include a description of the goals and objectives the county commission will attain; a description of the programs, services, and projects the county commission will provide, sponsor, or facilitate; and a description of how the county commission will measure the outcomes of such programs, services, and projects using appropriate, reliable indicators. As the overarching result to be achieved, the Act specifies that children be ready to enter school. In its fiscal year 2002–03 annual report, the state commission reported that county commissions are targeting the following five strategies: • Direct services to children through a service provider or volunteer. These activities include family support, health, and child development services for children prenatal through age 5, their parents, and other family members. • Community strengthening through public outreach activities—community events, dissemination of written materials, media campaigns, and organizing of community networks—aimed at large groups of children, parents, and related community members. • Service provider support such as training, distribution of materials, incentives, and meetings that develop their skills and resources. • Infrastructure investments in facilities and capital improvements and the purchase of equipment and materials costing more than $5,000 to improve the quality of program services and make them more accessible and integrated. 1100 California State Auditor Report 2003-123 California State Auditor Report 2003-123 1111 • Improving the support systems that care for young children and their families through such activities as conducting evaluation and research efforts, engaging community members in county commission decisions, and developing universal health care and preschool programs. DEFINING OUTCOME-BASED ACCOUNTABILITY The Act requires the state and county commissions to employ outcome-based accountability to determine the appropriate use of their Proposition 10 tax revenue allocations. Because the Act does not define outcome-based accountability, we used the definition from the federal government’s General Accounting Office. Under that definition, state and county commissions should be required to base their program funding on measurable changes in behavior, knowledge, or situation (outcomes) accomplished by providing a good or service (output). Outcomes demonstrate how well a particular program or agency is moving toward its specific strategic goals or objectives. Outputs are the goods and services produced by a program or organization, without any assessment of how well the goods or services achieve the goals or objectives of the program. For example, an appropriately funded program might demonstrate that children who attend its classes on how to brush their teeth properly (output) have significantly less plaque on their teeth (outcome) than before they attended the classes. Using outcome-based accountability, the state and county commissions must document how well their programs have improved the lives of children rather than simply cite the number of children the program served. This task is either performed externally by evaluators hired by the county commissions or internally using county commission staff. SCOPE AND METHODOLOGY The Joint Legislative Audit Committee (audit committee) requested the Bureau of State Audits to review the state commission and a sample of county commissions. Specifically, the audit committee requested us to do the following: • Review and evaluate the policies and procedures the state commission and a sample of county commissions use to collect, deposit, distribute, and spend Proposition 10 tax revenues. 1122 California State Auditor Report 2003-123 California State Auditor Report 2003-123 1133 • Determine whether county commissions have surplus balances and what they intend to do with these funds. • Determine the extent to which county commissions have periodic internal or external reviews, such as performance or financial audits, of their operations. • Examine county commissions’ level of oversight of service providers, including the nature and extent to which service providers have standards and whether they report their progress to the county commissions. • Identify the amount county commissions spend on administration and travel, and determine whether the percentages spent on these activities are appropriate. • Determine whether county commissions have sought funding partners to leverage local funds through partnerships. • Evaluate the process county commissions use to select their chairpersons. To select a sample of county commissions for review, we obtained the state commission’s Annual Report for Fiscal Year 2002–03 to identify each county commission receiving Proposition 10 tax revenues. Using this information, we selected a sample of county commissions, considering both the northern and southern portions of the State, small and large counties, counties that reported spending a large portion of their Proposition 10 tax revenues, and counties reporting large fund reserves. We selected the following five county commissions for our testing: El Dorado, Kern, Los Angeles, San Diego, and Santa Clara. To evaluate the policies and procedures used to collect, deposit, distribute, and spend Proposition 10 tax revenues, we reviewed relevant laws, rules, and regulations associated with the operations of the state and county commissions, including those the Act imposes. Based on this review, and on interviews with state and county commission management and staff, we determined the roles of the state and county commissions as they relate to the development and implementation of early childhood development programs. Also, we reviewed the state commission’s process for allocating funds to a sample of county commissions and evaluated its practices for depositing allocated funds to ensure compliance with applicable legislation. To identify current program efforts and proposed programs to be 1122 California State Auditor Report 2003-123 California State Auditor Report 2003-123 1133 funded in the future, we obtained the annual strategic plans adopted by the state commission and each of the sampled county commissions. To determine whether the county commissions we selected have surplus balances and what they intend to do with these funds, we obtained independently audited financial statements from the sampled county commissions’ inception dates through June 30, 2003. Using this information, we determined the total amount of Proposition 10 tax revenues expended and identified any surplus funds. We then acquired the adopted financial plans of each county commission to identify proposed future expenditures. To determine the extent to which the county commissions we selected conduct internal or external reviews of their operations, we reviewed each county commission’s annual report of its operations submitted to the state commission as required by the Act. To address the oversight of service providers, including the nature and extent to which service providers have standards and whether they report their progress, we examined the methods county commissions use to select and evaluate their service providers. We obtained any state and county commission policies governing the contracting for services and reviewed a sample of contracts at each commission we selected to assess compliance with these rules. We also obtained quarterly reports from service providers and published reports of commission-hired evaluators to assess compliance with the Act’s requirements to implement outcome-based accountability. To determine the amount a sample of county commissions spends on administration and travel, we identified the various expenditure categories of the county commissions, using independently audited financial statements. We reconciled these amounts to each commission’s general ledger and used this information to determine how much money county commissions spent on administration. However, because the Act does not define administrative cost, we could not determine whether the percentages spent on activities were appropriate. Also, because county commissions’ definitions of such costs differ, we developed a working definition in order to compare them. We defined administrative costs as any money not spent on children or their families, either directly or through grants to service providers. We also reviewed each county commission’s policy on travel to identify reimbursement practices. 1144 California State Auditor Report 2003-123 California State Auditor Report 2003-123 1155 To assess whether a sample of county commissions has sought funding partners, we reviewed independently audited financial statements to identify any revenue from sources other than Proposition 10 tax revenues. To evaluate the process used to select county commission chairpersons, we reviewed county ordinances, county commission bylaws, and county commission membership for compliance with the Act’s guidelines. n 1144 California State Auditor Report 2003-123 California State Auditor Report 2003-123 1155 Blank page inserted for reproduction purposes only. 1166 California State Auditor Report 2003-123 California State Auditor Report 2003-123 1177 CHAPTER 1 Not All County Commissions Follow Well-Defined Policies and Procedures When Allocating Funds CHAPTER SUMMARY Proposition 10 established the Children and Families Act of 1998 (Act) to enhance early childhood development and ensure children’s readiness to start school. To carry out its goals and objectives, the Act requires each participating county to establish a local county commission, either as an agency of the county or as a separate legal entity. Of the five Children and Families County Commissions (county commissions) we reviewed, one is a county agency and four are separate legal entities. In seeking to determine applicable contracting requirements for these five county commissions, our legal counsel found no specific county or state requirements related to soliciting competitive proposals for the kinds of contracts they generally enter into with community service providers. Thus, to evaluate the county commissions’ policies and procedures for allocating funds, we reviewed the individual policies adopted by the county commissions themselves. These policies provide guidance for selecting service providers in several ways, including requests for proposals, consideration of unsolicited proposals, and collaboration with community members. To gain public credibility and confidence, county commissions need to consistently follow self-defined allocation practices that are clear and well documented. However, the commissions we visited sometimes lack well-defined policies and documented procedures for their funding allocations. The true nature of some county commissions’ funding decisions are uncertain due to a lack of documentation, and another’s is undermined by the lack of well-defined policies. For example, one county commission recommended contracts for approval under its policy to use any contracting procedure it deems to be in its best interest. Also, some county commissions did not disclose to the public the noncompetitive nature of their allocations of funds, raising concerns about whether funds are being allocated appropriately. The California Children and Families Commission (state commission), also established by the Act, consistently followed contracting rules applicable to all state agencies. 1166 California State Auditor Report 2003-123 California State Auditor Report 2003-123 1177 Regardless of the allocation method used, final approval to contract with county service providers rests with each county commission, and final approval to contract with state service providers rests with the state commission. Thus, the state and county commissioners have a public obligation to disclose any conflict of interest. To bolster public confidence in the allocation process, commissions have adopted conflict-of-interest policies based on existing state statutes. County commission contracts with service providers require the service providers to provide monitoring documents that detail their efforts to ensure that they fulfill the contracts’ terms. The contract terms require service providers to submit quarterly progress reports to the county commission that address each provider’s scope of work in the contract or grant prior to receiving their next payment. Service providers must also submit a quarterly financial summary of expenses to the county commission to confirm they are spending funds in line with their approved budgets. COUNTY COMMISSIONS RELY ON SELF-IMPOSED CONTRACTING POLICIES AND PROCEDURES Of the county commissions we reviewed, one is an agency of the county and the other four are separate legal entities. Our legal counsel has advised us that the four county commissions established as separate legal entities are local public agencies. Despite this status, county ordinances are not relevant in assessing how these county commissions allocate funds through contracts because, according to our legal counsel, the ordinances pertaining to the counties served by these commissions do not impose specific requirements on contracting practices for goods and services. Also, Neither state nor county provisions in state law related to purchasing goods and services are law set requirements on not generally applicable to local public agencies, such as counties. the county commissions’ With neither state nor county law setting requirements on the contracting practices. county commissions’ contracting practices, we reviewed the individual county commission’s self-imposed contracting policies to assess their funding allocation processes. 1188 California State Auditor Report 2003-123 California State Auditor Report 2003-123 1199 THE FUNDING ALLOCATION PRACTICES OF SOME COUNTY COMMISSIONS ARE NOT WELL DEFINED OR DOCUMENTED Two of the county commissions we reviewed maintain insufficient records of their funding practices and one lacks well-defined allocation policies. By requiring only that county To gain public credibility commissions spend their resources in accordance with locally and confidence, county adopted strategic plans, the Act emphasizes local decision commissions should making in allocating funds and provides local flexibility in consistently follow designing service delivery systems. However, to gain public self-defined allocation credibility and confidence, county commissions should practices that are clear consistently follow self-defined allocation practices that are and well-documented. clear and well documented. In spite of this, some county commissions lack necessary documentation to substantiate their allocation procedures, and one commission’s funding policies are poorly defined. Further, when well-defined policies do exist, another county commission did not always follow them. Two County Commissions Lack Documentation Supporting Their Funding Decisions Lacking proper records, two county commissions we reviewed placed themselves in the difficult position of defending their funding decisions without the information needed to substantiate their allocation processes. Santa Clara’s commission, First 5 Santa Clara, is unable to provide any records created at the time of its decision to allocate funds to certain service providers from a larger pool of interested applicants. First 5 Santa Clara did provide us its written recollection of the reason for not funding some of these applicants. Also, First 5 Kern allocated funds but cannot demonstrate that it resolved significant concerns expressed by a committee it selected to make funding recommendations. Without a well-documented allocation process, one cannot be certain as to how these two county commissions reached their funding decisions. Between January and May 2002, First 5 Santa Clara prepared three separate applications to participate in the state commission’s School Readiness Initiative targeting the county’s lowest-performing school districts. First 5 Santa Clara proposed dedicating $16.8 million over five years to provide several different services to children and their families, including pre-kindergarten and parental education. Each submitted 1188 California State Auditor Report 2003-123 California State Auditor Report 2003-123 1199 application described First 5 Santa Clara’s efforts to incorporate various community members—including school district personnel and parents—into its decision-making process by allowing them to identify their community’s needs. A collaborative process was one of several eligibility requirements for county commissions requesting matching funds under the School Readiness Initiative. First 5 Santa Clara verbally described but did not document the process it used to allocate the funding it received from the state commission under the School Readiness Initiative. To address the needs its community members identified, First 5 Santa Clara When we requested told us that it requested proposals from service providers documentation supporting interested in participating in its School Readiness Initiative. its allocation process for Staff stated that they then reviewed submitted proposals school readiness funding, and negotiated with all service providers who committed to First 5 Santa Clara stated it providing some level of in-kind contribution, successfully has no information beyond addressed program priorities, and met state school readiness an “institutional memory” criteria. An in-kind contribution represents a nonmonetary of its decisions to fund donation of property, services, materials, or equipment. First 5 certain service providers. Santa Clara also explained why it did not offer funding to some service providers: either they proposed preexisting services, which is not an allowed use of Proposition 10 funding, or they did not address the needs identified by community members. However, when we requested documentation supporting this described allocation process, First 5 Santa Clara stated that it has no information beyond an “institutional memory” of its decisions to fund certain service providers. Also, First 5 Kern could not locate information corroborating its funding decisions relating to approximately $4 million in allocations made to service providers. For example, in April 2000, the Kern County Network for Children (Kern Network) responded to First 5 Kern’s public request for proposals to integrate the services of various providers into one accessible and comprehensive delivery system. Kern Network proposed expanding the role of 21 local community collaboratives that link families with community services. Local community collaboratives are collections of organizations and people representing businesses, government, nonprofits, schools, and community residents. Established in September 1992, Kern Network is a nonprofit, community-based organization created by local government agencies and the Kern County board of supervisors. 2200 California State Auditor Report 2003-123 California State Auditor Report 2003-123 2211 Under its proposal, Kern Network would administer the local collaboratives and receive 3 percent of the $2.8 million requested, with the remaining 97 percent allocated among the lead agencies representing the local community collaboratives. In October 2000, an independent evaluation committee reviewed Kern Network’s proposal and identified several weaknesses, including that it did not clearly articulate how service integration would be improved, it was not specific about the collaboratives’ role, and that the proposal possibly supplanted existing funding. In accordance with the instructions provided to it by First 5 Kern, the committee recommended against awarding funds to Kern Network, but stated that it felt the proposal had merit and recommended that Kern Network submit a revised proposal resolving the committee’s First 5 Kern awarded concerns. Kern Network submitted a revised proposal later the Kern Network a one-year same month. contract for $3 million, but is unable to First 5 Kern subsequently awarded Kern Network a one-year demonstrate it resolved contract for $3 million in November 2000. However, First 5 Kern concerns expressed is unable to demonstrate that it resolved the concerns expressed by the independent by the independent evaluation committee. First 5 Kern could evaluation committee not document that it reviewed Kern Network’s revised proposal, that reviewed Kern indicating instead that the committee’s concerns were addressed Network’s proposal. through receipt of the revised proposal. The original contract with Kern Network has since expired. The Absence of Well-Defined Policies Undermines Funding Efforts Exercising its policy to use any contracting procedure it deems to be in its best interest, First 5 Kern awarded a contract to one service provider based on the undocumented recommendations made by two technical advisory committees that used no scoring tools to evaluate the proposal. Without clear policies for evaluating proposals and adequate documentation, the county commission loses credibility with the public because the basis of the technical committees’ recommendations cannot be demonstrated or evaluated. Specifically, in April 2001, First 5 Kern invited a select group of service providers to submit proposals to develop and implement an Internet-based data management system. The invitation informed participants that two technical advisory committees would review submitted information and make funding recommendations for the commission to consider. 2200 California State Auditor Report 2003-123 California State Auditor Report 2003-123 2211 We saw a reference to these advisory committees in the executive director’s recommendation for awarding $422,400 for the data management services contract. However, when we asked to review the advisory committees’ analyses and recommendations, First 5 Kern replied that the two committees did not use any scoring tools to evaluate the proposals submitted under the invitation and that no documentation of the advisory committees’ recommendations exist. Lacking such documentation, we were unable to review the evaluation and the basis for the two technical committees’ recommendations. One County Commission Did Not Always Adhere to Established Policies First 5 Santa Clara’s policy for funding unsolicited proposals limits awards to $15,000 per grantee and an aggregate of $50,000 per year. However, First 5 Santa Clara did not adhere Having issued no to this policy in one instance. Having issued no solicitation, in solicitation, First 5 August 2003, this county commission allocated $1 million to the Santa Clara allocated Children’s Discovery Museum to construct a developmentally $1 million to the appropriate exhibit. By not adhering to its established policies, Children’s Discovery First 5 Santa Clara is not maintaining an open and equitable Museum, even though allocation process. Although First 5 Santa Clara awarded this its policy for funding contract to the Children’s Discovery Museum at a regularly unsolicited proposals scheduled public meeting, the manner in which this contract limits awards to originated—according to the chair of the commission, a $15,000 per grantee representative from the museum approached him to discuss and an aggregate of possible First 5 funding—was not publicly disclosed. When $50,000 per year. we asked why the commission did not follow its public policy, this same county commissioner told us that First 5 Santa Clara viewed the contract as an opportunity to serve children prenatal to age 5, to invest in a capital improvement project that would benefit the community, and to take advantage of a good opportunity to inform the public about First 5 Santa Clara’s presence in the community. NOT ALL COUNTY COMMISSIONS’ DECISIONS TO AWARD FUNDS NONCOMPETITIVELY WERE OPEN TO PUBLIC SCRUTINY Because the Act specifies local control of funds, state restrictions on noncompetitive allocations do not apply to the county commissions, and no other regulations guide or restrict the commissions’ noncompetitive allocations. Three of the five 2222 California State Auditor Report 2003-123 California State Auditor Report 2003-123 2233 county commissions we reviewed used noncompetitive funding allocations for some contracts for which they believed they had identified the best or the only service provider available. Regardless of the justifications, some county commissions openly discussed their noncompetitive allocation practices, while at times some county commissions did not. Closing the allocation process to public scrutiny can raise concerns about whether service providers are competent and charge a fair price. Some Commissions’ Noncompetitive Allocation Practices Were Open to Public Scrutiny Some county commissions created confidence in their allocation processes by awarding noncompetitively allocated contracts at regularly scheduled public meetings, providing the public an opportunity to express any concerns and scrutinize the choice First 5 Los Angeles of service provider. First 5 Los Angeles and First 5 Santa Clara and First 5 Santa Clara noncompetitively allocated funds for their Healthy Kids noncompetitively allocated Initiative, a program providing health care to children who do funds for their Healthy not qualify under existing state and federal programs. Both of Kids Initiative, stating that these county commissions believed that a competitive allocation a competitive allocation process was unnecessary for these services, because they had process was unnecessary already identified the best available service providers. for these services, because they had already identified For example, First 5 Los Angeles contracted with L.A. Care, the best available a local nonprofit health maintenance organization (HMO), service providers. to serve as its program administrator at a cost of up to $1.3 million per month, and with the Los Angeles County Department of Health Services to conduct outreach services at a cost of $4.7 million per year. First 5 Los Angeles stated that L.A. Care—being one of the nation’s largest public HMOs and having extensive experience serving Medi-Cal and Healthy Families participants—could provide a seamless transition for children entering these state and federally subsidized health care programs when they no longer qualify for Healthy Kids. Likewise, First 5 Los Angeles contracted with the Los Angeles County Department of Health Services because it has extensive experience and a substantial record of implementing these types of activities. Both county commissions publicly discussed their intent to provide health care to uninsured children, specified the amount of funds they would use for this purpose, and publicly awarded the contracts. 2222 California State Auditor Report 2003-123 California State Auditor Report 2003-123 2233 The Noncompetitive Nature of Some Funding Decisions Was Not Publicly Disclosed Two county commissions we reviewed did not always publicly disclose information when awarding funds through a noncompetitive allocation process. For example, in 2000 First 5 Santa Clara solicited proposals to create a plan for an Early Screening, Prevention, Diagnostic and Treatment Center—noting that the plan would become the basis for a competitive process to allocate funds for services. However, the entity that received the initial planning grant funds subsequently received funds to provide services without participating in a competitive process. Moreover, when publicly approving this award for services, the county commission described it as a competitive award. When we asked why the commission referred to the award as a competitive allocation, staff indicated that the commission’s original intention was to allocate the funds noncompetitively but that it was confused regarding funding terminology. Furthermore, in awarding funds to another entity for a related element of this plan, the commission failed to openly award the contract to the service provider, leaving the public without clear information on First 5 Santa Clara’s process for allocating the funds. First 5 Santa Clara also noncompetitively allocated a $100,000 per year contract to an organization to provide information technology access to migrant families. Because this organization already held a subcontract from the county to provide services at a migrant camp, First 5 Santa Clara said it was best qualified to be the service provider. This award was part of a much larger initiative First 5 Santa Clara had presented to the public, approving a $2 million allocation over three years. However, when presenting the plan and identifying the service provider at a public meeting, First 5 Santa Clara did not disclose the allocation’s noncompetitive nature. Another county commission, First 5 Kern, allocated $600,700 for data evaluation services without soliciting competition, but did not publicly disclose the noncompetitive nature of this award. OTHER COUNTY COMMISSIONS’ CONTRACTING PRACTICES HELP ENSURE A PRUDENT ALLOCATION OF FUNDS In an effort to broaden their potential pools of service providers, two of the five county commissions we visited allocate funds noncompetitively when awarding contracts below a specified amount. Also, the state commission uses a noncompetitive 2244 California State Auditor Report 2003-123 California State Auditor Report 2003-123 2255 process for certain contracts, as allowed by state law. However, as another means of allocating resources, the state commission and each of the five county commissions use competitive practices that incorporate procedures to help ensure a fair and appropriate allocation of funds. Regardless of allocation practices, the ultimate funding decision rests with the state commission and each locally appointed county commission. Therefore, to address potential conflicts of interest, the state commission and five county commissions have adopted local policies based on state statutes. Noncompetitive Practices for Minor Awards Safeguard Resources While Allowing Small Community Organizations to Participate To broaden their potential pools of service providers, two of the five county commissions we reviewed allocate funds through noncompetitive methods when amounts awarded are below a To broaden their potential specified amount. These commissions believe that responding to pools of service providers, a competitive application process would be cost prohibitive for two of the five county some community service providers. Such allocation polices allow commissions allocate funds smaller service providers to participate in county commission through noncompetitive programs while safeguarding resources by keeping grant awards methods when amounts relatively low. awarded are below a specified amount. For example, First 5 Kern publicly released a mini-grant application to any service provider interested in reducing the incidence of childhood injury or death due to playground injury. Eligible service providers would be licensed child care facilities or family day care homes. This application—limited to a maximum grant award of $20,000 per grantee—solicited less information than a larger competitive allocation process would require. First 5 Santa Clara also awards small grants through a noncompetitive process, but its method is slightly different. Instead of releasing an application that focuses on one particular service, First 5 Santa Clara’s unsolicited request policy allows grantees to approach the commission for awards of up to $15,000 for a program or need they have identified. The State Commission Followed State Contracting Rules When Awarding Noncompetitive Contracts As a state entity, the state commission must follow state contracting rules, including those that govern the noncompetitive contracts the state commission awards for certain projects. Our review of the state commission’s contracting methods found 2244 California State Auditor Report 2003-123 California State Auditor Report 2003-123 2255 that it appropriately followed the State’s contracting rules When the state when awarding noncompetitive contracts. The State allows its commission awarded agencies to use noncompetitive contracting methods in certain approximately $2 million circumstances, such as between two or more state agencies to the University of or between a state agency and the University of California or California, Berkeley, California State University, when it is determined that a good or it followed the State’s service can be provided from only one source, or in emergency rules for awarding a situations, such as a fire or flood. For example, when the state noncompetitive contract. commission awarded approximately $2 million to the University of California, Berkeley, to evaluate the incentive strategies county commissions use to retain child care staff, it followed the State’s rules for awarding a noncompetitive contract. In this case, the state commission cited its justification for awarding a contract to the only provider that could supply the service and obtained the approval of the Department of General Services as required. A Variety of Competitive Practices Helps to Ensure the Proper Distribution of Funds As another means of allocating resources to service providers, the state commission and each of the five county commissions sometimes allocate funds by soliciting proposals. This competitive method helps to ensure a fair and prudent distribution of funds by requiring that potential service providers do the following: demonstrate their qualifications and abilities to achieve desired outcomes; submit budgets detailing program expenses such as staff salaries, fringe benefits, office expenses, and other indirect costs; and submit detailed scopes of work that are tied to specific outcomes, with indicators to measure success against these outcomes. Some applications also include an evaluation plan outlining efforts to collect the data needed to gauge progress. Lastly, the state and county commissions employ independent review panels to make funding recommendations. Some commissions require panelists to sign formal conflict-of-interest declarations, while others analyze panelists’ résumés to identify any concerns. The state commission and each of the five county commissions we visited use various terminology in their competitive contracting activities. For example, the state commission and First 5 Los Angeles make reference to “requests for proposals,” while First 5 Santa Clara uses the phrase “intention to negotiate.” Regardless of terminology, the process of selecting a service provider incorporates the practices just described. The few procedural differences that exist do not undermine 2266 California State Auditor Report 2003-123 California State Auditor Report 2003-123 2277 the competitive nature of these allocation efforts. For instance, each year some county commissions issue one “blanket” request that outlines the commission’s overall goals, leaving applicants to decide which goals they wish to address. Other commissions issue program-specific requests as needed. Also, some commissions weigh the criteria that independent review panels use to review proposals, while others use a “strength” and “weakness” format. Conflict-of-Interest Policies Add Credibility to the Contracting Process To address potential conflicts, the state commission and the five To address potential county commissions have adopted conflict-of-interest policies conflicts of interest, the based on existing state statutes. In awarding contracts, both state state commission and the and county decision makers should be free from any conflicts five county commissions of interest that create an unfair competitive advantage, impair have adopted conflict-of- effectiveness, or undermine public confidence when contracting interest policies based on for services. Several county commissioners are also service existing state statutes. providers of early childhood development programs, which is not specifically prohibited by the Act. However, conflicts do arise that prevent some commissioners from participating in commission business under certain circumstances. The state and county commissions’ conflict-of-interest policies adhere to the Political Reform Act of 1974 and other state laws outlining the actions and relationships that can constitute conflicts. Also, each county commission employs a standard state form for financial interest disclosure. However, laws that govern the state and county commissions differ in one respect. Existing statutes prohibit state commissioners from holding a financial interest in any contract made by the state commission, but county commissions are exempted from this law except when a commission member benefits financially and fails to recuse himself or herself from making, participating in making, or attempting in any way to use his or her official position to influence a decision. For example, upon identification of a potential conflict with several of its commissioners, First 5 Los Angeles circulated a memo informing other commissioners of the potential conflict. Commissioners with the potential conflict then recused themselves from participating in the decision-making process. As decision makers, commission members should also understand the needs of the population they serve, children prenatal to age 5, and the challenges facing early childhood development programs. 2266 California State Auditor Report 2003-123 California State Auditor Report 2003-123 2277 The Act addresses the need for qualified commissioners by specifying who may serve on the state and county commissions, including representatives of organizations working in early childhood development. (See pages 9–10 for details of these specifications.) Our review of the five county commissions we visited found that each had met these requirements. The Act also imposes term limits on state commissioners but not on county commissioners. First 5 Los Angeles is the only county we visited to specifically limit the number of terms a county commissioner may serve. In electing their chairperson, four of the five county commissions require a majority vote of the commission. First 5 San Diego, the fifth county commission we visited, is required by its ordinance to elect the commission chairperson. The way that First 5 San Diego chooses its chairperson is in accordance with a vote of its county commission to follow its bylaws, which state that the chairperson will be the representative from the county’s board of supervisors. Monitoring Can Ensure Adherence to Contract Provisions All five county commissions we visited have a process for monitoring their service providers’ performance, and county commission contracts state that service providers must submit quarterly progress reports before they can receive their next funding installments. County commissions use these All five county commissions quarterly progress reports to monitor compliance with program have a process for requirements and to ensure that the activities and/or services monitoring their service being performed are related to the scope of work approved in the providers’ performance, grant or contract. Also, service providers must submit a quarterly and county commission financial summary of their expenditures, to ensure that these contracts state that service funds are being spent according to the budget approved when providers must submit the grant or contract was awarded. Service providers may also be quarterly progress reports required to include in quarterly reports information on progress before they can receive their in achieving the sustainability plan they outlined in the original next funding installments. grant or contract. This requirement allows county commissions to monitor the service provider’s efforts to obtain outside resources, provide needed assistance, and help ensure that the program will continue after county commission funding ends. Some county commissions also use site visits to monitor how well service providers comply with the requirements of their grants or contracts. When visiting the service providers, some county commissions review the actual records that support the service providers’ quarterly progress reports. This 2288 California State Auditor Report 2003-123 California State Auditor Report 2003-123 2299 review reinforces the idea that the county commissions are committed to receiving an accurate picture of the service providers’ activities and expenditures. Site visits can also give county commissions an alternate perspective of the program that cannot be conveyed in the quarterly progress reports, and they also provide a more informal avenue for communications between the service providers and the county commissions. RECOMMENDATIONS To ensure the appropriate use of program funds and instill public confidence, the Kern and Santa Clara county commissions should adopt and follow well-defined policies to guide their allocation efforts and should also maintain sufficient documentation to support their allocation decisions. n 2288 California State Auditor Report 2003-123 California State Auditor Report 2003-123 2299 Blank page inserted for reproduction purposes only. 3300 California State Auditor Report 2003-123 California State Auditor Report 2003-123 3311 CHAPTER 2 As Most Fund Balances Grow, Some County Commissions Promise to Significantly Reduce Them CHAPTER SUMMARY Having spent as little as 15 percent to as much as 67 percent of their revenues on early childhood development, the five Children and Families County Commissions (county commissions) we reviewed maintain significant financial fund balances.3 The current financial plans of two of these county commissions promise to significantly reduce these fund balances, while other county commissions plan to maintain them. For example, First 5 Los Angeles plans to spend down its fund balance by launching three new programs, including a universal preschool initiative that will initially target 4-year-olds. In contrast, First 5 San Diego plans to maintain a significant fund balance to ensure that it can continue existing programs for 20 years and evaluate those programs’ long-term effects on children as they mature. In fact, most of the current fund balances for four of the five commissions are already earmarked for various purposes, including new programs and substantial amounts some commissions have set aside to sustain their programs in the future, anticipating a possible reduction in tobacco use that would decrease allocations. Facing this uncertainty, the California Children and Families Commission (state commission) and county commissions acknowledge the important role of funding partners; however, the commissions have received very little non-Proposition 10 funding. With limited revenue, the five county commissions we reviewed are publicly committed to spending every possible dollar on direct services to children. However, some county commissions do not express a clear commitment to a specific limit on administrative costs, and some spend proportionally more on administration than others. Only one county commission has an ordinance limiting its administrative costs. Other county commissions publicly state that they plan to limit their administrative 3 We use the term “fund balance” to mean the difference between a county commission’s assets and liabilities. The assets of the five county commissions we reviewed consisted primarily of cash and investments. Some of the fund balances are earmarked for future use by the county commissions. 3300 California State Auditor Report 2003-123 California State Auditor Report 2003-123 3311 spending, and all have differing definitions of administrative costs. Also, county commissions generally base their travel policies on their respective county’s travel practices. Travel spending varies, with some county commissions spending a slightly larger percentage of their administrative costs than others on this type of expense. ALTHOUGH MOST FUND BALANCES HAVE BEEN STEADILY GROWING, THE MAJORITY OF THEM ARE ALREADY EARMARKED Beginning as early as October 1999, county commissions began receiving periodic allocations of Proposition 10 tax revenues based on each county’s number of births in comparison to statewide totals. However, revenues allocated by the state commission have outpaced the actual spending patterns of the five county commissions we visited, resulting in significant fund balances as of June 30, 2003. Table 2 summarizes these fund balances. TABLE 2 Fund Balances of County Commissions We Reviewed From June 30, 2000, Through June 30, 2003 County Commission June 30, 2000 June 30, 2001 June 30, 2002 June 30, 2003 Los Angeles $259,443,618* $415,594,786† $548,112,701* $630,431,355* San Diego 68,524,419† 111,458,699† 139,305,323† 158,228,535* Santa Clara 41,796,338† 67,294,748† 87,003,505* 96,497,310* Kern 17,708,231† 25,159,120† 26,058,143† 21,359,105* El Dorado 2,628,390* 3,403,880* 3,479,169* 2,752,672* Source: County commission audited financial statements. *Reflects the accrual basis of accounting that recognizes expenses when incurred and revenues when earned. † Reflects the modified accrual basis of accounting that recognizes revenues when they become available and measurable and, with certain specific exceptions, recognizes expenses when the fund liability is incurred if measurable. As the table shows, the county commissions had accumulated significant financial balances as of June 30, 2003. Some of these funds were not yet slated for specific purposes. However, most county commissions had already earmarked substantial amounts for one of the following three purposes: (1) to ensure sufficient resources to pay for existing contracts; (2) to fund new, commission-approved programs; and (3) to ensure a future revenue source for existing programs with “sustainability” 3322 California State Auditor Report 2003-123 California State Auditor Report 2003-123 3333 funds that address an environment of uncertain or potentially declining revenues. Table 3 shows how much of the June 30, 2003, fund balances fall within these three categories and how much the county commissions consider undesignated and thus readily available to fund new programs. TABLE 3 Purposes for Which Fund Balances Are Earmarked as of June 30, 2003 Los Angeles San Diego Santa Clara Kern* El Dorado Totals Contractual obligations $ 27,168,914 $28,713,737 $10,543,817 $ 66,426,468 Designated for commission- approved programs $525,431,355† 6,072,859 27,460,000 $1,406,729 560,370,943 Sustainability funds 105,000,000 63,636,526 168,636,526 Undesignated funds 61,350,236 40,323,573 10,815,288 1,345,943 113,835,040 Totals $630,431,355 $158,228,535 $96,497,310 $21,359,105 $2,752,672 $909,268,977 Source: County commission audited financial statements for fiscal year 2002–03. * On July 1, 2003, First 5 Kern extended the terms of two of its contracts for a total amount of $2.5 million. Also, on July 2, 2003, First 5 Kern approved the continuation of 20 programs for fiscal year 2003–04 in the amount of $6.2 million. † We relied on the county commission’s audited financial statements to distinguish between programs that have contracts and those that have been approved by a county commission but for which no contract exists. The audited financial statements for First 5 Los Angeles did not differentiate between the two categories. Therefore, we grouped everything into the category labeled “designated for commission-approved programs.” As the table shows, four of the five county commissions we reviewed have already committed or plan to use the majority of their fund balances for a variety of purposes. Most county commissions we reviewed contract with service providers based on a multiple-year funding cycle. At the end of each year in this cycle, the county commission extends a service provider’s contract if the provider has fulfilled its obligations. Because service providers must meet certain requirements prior to receiving funds, the moneys in the second and subsequent years are not guaranteed and are not a liability of the county commission. However, in a contract’s first year the county commissions often set aside sufficient funds for the entire multiple-year period. In these cases, we relied on the county commissions’ auditors to identify the portions of the contracts that are legal obligations from those portions designated as approved for expenditure. 3322 California State Auditor Report 2003-123 California State Auditor Report 2003-123 3333 A FEW MAJOR INITIATIVES WILL REDUCE SOME COUNTY COMMISSIONS’ FINANCIAL RESERVES, WHILE OTHERS’ RESERVES WILL REMAIN HIGH Each of the five county commissions we visited maintains a financial plan, which is a flexible working document outlining the commission’s revenue and expenditure projections. For example, the financial plan adopted by First 5 Los Angeles First 5 Los Angeles reports its intention to focus its resources on a few major expects its total fund initiatives that, if carried out, will significantly reduce its balance to decline reserves. Specifically, First 5 Los Angeles expects its total fund from $630 million as of balance to decline from $630 million as of June 30, 2003, to June 30, 2003, to less less than $20 million by fiscal year 2008–09. In contrast, the than $20 million by financial plans adopted by three other county commissions fiscal year 2008–09. show their intentions to maintain considerable fund balances. For example, First 5 San Diego plans to maintain a significant reserve in order to sustain its existing programs in the event of a possible decline in Proposition 10 tax revenues. As shown earlier in Table 3, First 5 Kern has already contractually obligated roughly half of its fund balance as of June 30, 2003. First 5 Los Angeles intends to spend down its fund balance largely with three new programs. Starting in 2004, it has plans for a $600 million Universal Preschool Initiative. The county commission has already begun developing a 10-year implementation plan for this initiative, with a short-term goal of initially targeting 4-year-olds while developing planning partnerships and leveraging funds with other organizations to sustain and expand its operations. First 5 Los Angeles also has allocated $100 million to its Healthy Kids Initiative, which will make available medical, dental, and vision care to all Los Angeles County children up to age 5 who do not qualify for existing health care programs. It anticipates that this care will be provided at minimal or no cost to families, based on their income compared to the federal poverty level. Finally, First 5 Los Angeles has allocated $255 million to its Prenatal to Three Initiative, currently being developed to target prenatal to 3-year- old children. All of the county commissions we reviewed are concerned with sustaining their programs in the future, and therefore they either have or plan to maintain varying amounts of funds in reserve. First 5 Los Angeles has recently decided to spend the $105 million in its sustainability account. It intends to spend these funds beginning in fiscal year 2004–05 on its Prenatal 3344 California State Auditor Report 2003-123 California State Auditor Report 2003-123 3355 to Three Initiative, recognizing that these funds have more purchasing power now than in the future. It expects to fully exhaust these funds by fiscal year 2008–09. In contrast, First 5 San Diego intends to maintain significant financial reserves tied to program sustainability. The county commission’s audited financial statements as of June 30, 2003, First 5 San Diego intends reported close to $64 million in sustainability funds. However, its to maintain significant 20-year financial plan as of February 2004 shows a sustainability financial reserves tied to reserve of $125 million as of the same date. The $61 million program sustainability. difference between the two figures is reflected as undesignated Its 20-year financial plan funds in the county commission’s audited financial statements. shows a sustainability Using the balance of $125 million for sustainability, the 20-year reserve of $125 million as plan shows an initial $49 million drop in fiscal year 2003–04 of February 2004. and then a projected steady growth to more than $129 million in sustainability funds by fiscal year 2012–13. Although First 5 San Diego is currently completing studies on three initiatives, in health, preliteracy, and behavioral health, it has not yet determined what portion of its fund balance will be earmarked for these projects. The county commission established this sustainability account to stabilize its resources and maintain service levels for 20 years, including in this reserve any funds it does not allocate during the annual budget process. It believes that this long-term horizon will allow it to evaluate its programs’ effects on children as they enter adulthood. First 5 Santa Clara is also intent on maintaining program funding by having a sustainability reserve. In its strategic plan, it disclosed its intent to set aside 3 percent of its annual funds for this purpose. However, its most recent financial plan anticipates a one-time transfer during fiscal year 2004–05 of $30 million into a sustainability reserve, which it expects to maintain at least through fiscal year 2008–09. In addition to this reserve, the financial plan projects a separate $50 million fund balance as of June 30, 2005, an amount the county commission plans to spend by the end of fiscal year 2008–09. INVESTMENT PRACTICES ADEQUATELY SAFEGUARD THE FINANCIAL RESOURCES OF THE STATE AND COUNTY COMMISSIONS Investing their available cash in a county pooled investment program helps the county commissions safeguard these resources while maintaining sufficient liquidity to meet program needs. Each county commission we reviewed had established 3344 California State Auditor Report 2003-123 California State Auditor Report 2003-123 3355 a local trust fund to hold its allocation, as required by the Each county commission California Children and Families Act of 1998 (Act), and had we reviewed had elected to invest available cash balances in its respective county’s established a local pooled investment program. State law safeguards these cash trust fund to hold its balances by restricting the types of investments counties may allocation, as required acquire to such financial instruments as United States Treasury by the Act, and had notes, federal agency bonds, and highly rated commercial paper. elected to invest available To address the liquidity of these investments, arrangements cash balances in its between each county commission and its respective county respective county’s pooled allow for the immediate withdrawal of cash balances whenever investment program. needed to fund programs. While one county we visited solely administers all of the funds in its investment pool, the other counties turn over at least part of these cash balances to the State by participating in its local agency investment fund. The state treasurer administers a single pooled investment program comprising both the surplus balances of certain state agencies and the local agency investment fund. To safeguard these resources, state law imposes restrictions on the types of investments the state treasurer may make, similar to the restrictions on county-administered programs. Like those programs, the State allows participants to withdraw their cash balances as needed to meet program requirements. As a state agency, the state commission invests its available cash balances on deposit in its trust fund in the State’s pooled investment program. Both the state and county investment programs are included in the yearly entity-wide audits that also report on investment rules. However, one county commission, First 5 Santa Clara, plans to withdraw the available cash for its sustainability fund balance from its county’s pooled program and is currently requesting proposals from private firms to manage its investments. According to the request for proposals, any private firm under contract would still be governed by the same rules restricting investment options. EFFORTS TO OBTAIN FUNDING PARTNERS HAVE PRODUCED LITTLE NON-STATE FUNDING The Act grants the state commission and each county commission the authority to apply for gifts, grants, and donations to further a program of early childhood development. Although the state and county commissions acknowledge the important role funding partners can play in addressing early childhood development and sustaining ongoing programs, they have 3366 California State Auditor Report 2003-123 California State Auditor Report 2003-123 3377 received very little in funding from sources other than Proposition 10 tax revenues. For fiscal year 2002–03, only one county commission we reviewed had received any grant funding, which represented less than 1 percent of that commission’s total revenue, and the state commission received less than 7 percent of total revenue from contracts and interest on investments. However, funding partners are important because tax revenue from Proposition 10 may decline over time. As previously discussed, First 5 Los Angeles will significantly spend down its fund balance within five years and First 5 San Diego is currently setting aside a significant amount of funds to build up its sustainability fund. In addition, service providers are making in-kind contributions, which represent a nonmonetary donation of property, services, materials, or equipment. During fiscal year 2002–03, the state commission reported $121 million in revenue. However, less than $9 million, or less than 7 percent, resulted from contracts and income on investments, with 93 percent coming from Proposition 10 tax revenues. The contract revenue resulted from a contract agreement with the California Department of Health Services to produce and distribute informational kits to the county commissions aimed at new parents. During this same period, the only county commission we reviewed that received any grants from non-state sources was First 5 Santa Clara. Specifically, of the $27 million First 5 Santa Clara reported as revenue for fiscal year 2002–03, it received a total of $92,500 from four private foundations and an additional $62,000 from AmeriCorps, representing less than 1 percent of its total revenue. These private foundation grants included $75,000 from the San Francisco Foundation and $12,500 from the David and Lucille Packard Foundation. They came about not only because Proposition 10 tax revenue of the generosity of the private foundations, but also because continues to be the primary individuals who had knowledge of the availability of grant source of revenue for the funds contacted the foundations and completed the application county commissions we process. Proposition 10 tax revenue continues to be the primary reviewed, accounting for source of revenue for the county commissions we reviewed, 74 percent to 91 percent of accounting for 74 percent to 91 percent of their total revenue. their total revenue. Other revenue sources include interest earnings, which account for 3 percent to 9 percent, and state matching grants, which account for 2 percent to 17 percent. The state and county commissions focus primarily on leveraging each other’s funds through the matching of funds. Specifically, the state commission, in partnership with certain county commissions, has embarked on at least three jointly 3366 California State Auditor Report 2003-123 California State Auditor Report 2003-123 3377 funded initiatives: (1) the School Readiness Initiative, (2) Retention Incentives for Early Care and Education Providers (Retention Incentives Program), and (3) the Health Access for All Children project. To implement these programs, each county commission commits some of its money to receive a proportional share of matching funds from the state commission. The purpose of the School Readiness Initiative is to prepare children to enter school, and it targets each county’s lowest-performing school districts. The Retention Incentives Program aims to retain family child care providers and child care center staff by providing fiscal incentives to stay in the field and improve their education and professional development. Under the state commission’s recently announced Health Access for All Children project, county commissions will receive a match of $1 from the state commission for every $4 they spend to subsidize health insurance premiums for young children. Approved by the state commission in October 2003, the project addresses its goal to ensure that all children from birth to 5 years of age in the State are insured and have access to quality medical, dental, and vision care services. The state commission plans to spend $42.5 million over a four-year period to assist with the payment of insurance premiums for an expanded health insurance program to provide coverage for an estimated 48,000 of the State’s youngest uninsured children, regardless of immigration status, who meet specific requirements. Those requirements stipulate that the children must be ineligible for the Medi-Cal and Healthy Families programs (other state or federally subsidized health insurance programs) and must have a family income at or below 300 percent of the federal poverty level. This program is similar to the Healthy Kids programs of First 5 Los Angeles and First 5 Santa Clara that we discussed previously. County commissions often receive in-kind contributions from service providers during the contracting process. First 5 Santa Clara maintains a database to track its service providers’ in-kind contributions. Currently, this database reports in-kind contributions totaling $8.5 million for First 5 Santa Clara contracts paying $23 million in commission funds. This represents an in-kind contribution of 37 cents for each contract dollar invested in the community by First 5 Santa Clara. As shown in Table 4, our review of significant contracts at the four remaining county commissions identified levels of in-kind contributions that were lower than that of First 5 Santa Clara. 3388 California State Auditor Report 2003-123 California State Auditor Report 2003-123 3399 TABLE 4 In-Kind Contributions at the Five County Commissions We Reviewed Los Angeles San Diego Kern El Dorado Santa Clara In-kind contributions per contract dollar 10 cents 8 cents 15 cents 21 cents 37 cents Source: First 5 Santa Clara’s database for service provider contracts and a sample of significant service provider contracts for the remaining four county commissions we reviewed. Finally, some county commission contracts require service providers to solicit outside funding sources to help sustain their programs. For example, First 5 Los Angeles has asked service providers to have a formal plan that sets sustainability goals that includes resources such as matching funds, in-kind contributions, grants, endowments, and funding from private and/or public agencies. First 5 San Diego has asked service providers to identify specific steps to address sustainability, indicate the time frames to complete the steps, and identify who will be responsible for each identified step. SOME COUNTY COMMISSIONS LACK A CLEAR COMMITMENT TO LIMIT THEIR ADMINISTRATIVE SPENDING This audit request asked us to identify the amount county commissions spend on administration, including travel. Only one of the five Although the county commissions are publicly committed county commissions to using every dollar possible on direct services to eligible we visited, First 5 Kern, children, only one of the five county commissions we visited, had a county ordinance First 5 Kern, had a county ordinance limiting the amount that limiting the amount could be spent on administrative costs. Because each of the that could be spent on county commissions we visited had a differing opinion on the administrative costs. types of costs that constitute an administrative expense, for comparability we developed our own working definition and applied it to each county commission when calculating the percentage of administrative costs to ensure consistency among the county commissions. Also, all of the county commissions we visited base their travel policies on their respective county’s travel policies and vary somewhat in the amounts they spend on this activity. 3388 California State Auditor Report 2003-123 California State Auditor Report 2003-123 3399 Some County Commissions Spend Proportionally More Than Others in Administering Their Programs Applying our working definition of administrative costs to the activities of the five county commissions we reviewed, we found that some spend proportionally more on administration than others. The Act acknowledges the role of administrative functions in early childhood development programs and includes a spending limit of 5 percent for such costs at the state commission level. However, the Act is silent on the county commissions’ administrative costs, requiring only that they spend funds in accordance with locally-adopted strategic plans. Recognizing that a certain level of funding must be committed to administrative functions, four of the five county commissions we reviewed have expressed a commitment to keep such costs low. For example, in its fiscal year 2001–04 strategic plan, First 5 Los Angeles promised to spend only 5 percent of its revenues on operational and administrative costs. First 5 Santa Clara stipulated that it will set aside 5 percent to 10 percent of its annual funds for administrative purposes. Additionally, First 5 Kern is limited by county ordinance to spending no more than 8 percent of its annual funding allocation on administrative expenses. Two county commissions, El Dorado and San Diego, neither established an explicit maximum on the amount of administrative costs in their strategic plans nor had a maximum imposed by county ordinance. Moreover, county commissions may not be entirely consistent in the types of costs they consider to be administrative. For example, the First 5 San Diego strategic plan defines administrative costs as including the evaluation of funded activities, commission operations, and technical assistance provided to service providers; while First 5 Kern’s strategic plan defines administrative costs as including administrative infrastructure, planning, and monitoring. Because the Act does not define administrative costs and county commissions define them differently, we developed a working definition in order to compare them. Using our definition, as Table 5 shows, some county commissions spend a larger portion of their revenue or expenses than others on the administration of their programs. 4400 California State Auditor Report 2003-123 California State Auditor Report 2003-123 4411 TABLE 5 Administrative Percentages at the County Commissions We Reviewed Los Angeles San Diego Santa Clara* Kern El Dorado Costs as a percentage of total expenses 21% 10% 32% 10% 17% Costs as a percentage of total revenues 9 6 21 15 25 Source: Auditor calculated percentages are based on our working definition of administrative costs taken from the audited financial records of the county commissions. Note: Our percentages are based on a working definition of administrative costs for comparison purposes only. It is not our intention to suggest that ours is the only valid definition. *As previously stated, First 5 Santa Clara is the only county commission that received funding from private sources. Therefore, its total revenue will include these amounts. We defined all costs as administrative if the county commissions did not spend the funds directly on children or their families or grant the funds to service providers. As such, our definition of administrative costs includes the following: • Most county commission operations, including ongoing efforts to evaluate service provider performance, either by First 5 staff or external evaluators. • Professional development. • Consulting services. • First 5 staff salaries and benefits. Our definition may be more conservative than the definitions used by some county commissions. We have included costs for the evaluation of service providers, a mandated activity, which can represent a significant portion of a county commission’s costs. For example, 10 percent of First 5 Santa Clara’s administrative costs represent evaluation activities. Nonetheless, we believe our definition addresses the county commissions’ stated goal of focusing their efforts on providing direct services to children and their families and is sufficient for comparison purposes. However, we recognize that other valid definitions exist. County Commissions Base Their Travel Policies on County Practices and Some Spend Slightly Varying Percentages of Administrative Costs on Travel Each county commission we visited generally chose to implement its travel policy requirements based on preestablished county practices, with some commissions spending a slightly larger 4400 California State Auditor Report 2003-123 California State Auditor Report 2003-123 4411 percentage of their administrative costs on this type of expense than others, as shown in Table 6 on the following page. As we stated previously, the Act calls for local decision making to provide greater local flexibility in designing service delivery systems. As such, the day-to-day rules that govern the respective county commissions, including their administrative and travel costs, were delegated to the commissions. TABLE 6 Reimbursement Rates for County Commission Travel Effective in 2004 Fiscal Year County 2002–03 Travel Commission Lodging Food Mileage Incidentals Expenses Los Angeles Actual cost, no Per diem of $49.75 37.5 cents per mile Per diem of $52.50 $70,423 stated maximum (1 percent of administrative costs) San Diego Actual cost not to Actual cost not to 37.5 cents per mile Actual cost not to $30,228 exceed $99 per exceed: exceed $46 per day (1 percent of night Breakfast: $9 including meals administrative costs) Lunch: $11 Dinner: $24 Santa Clara Actual cost, no Actual cost not to 36 cents per mile Actual cost, no $37,973 stated maximum exceed $60 per day stated maximum (1 percent of administrative costs) Kern Actual cost not to Actual cost not to 37.5 cents per mile* Actual cost, $31,284 exceed $182 per exceed a range of reimbursement (2 percent of night $35 to $51 per day included with food administrative costs) based on the county allowance visited El Dorado Actual cost not to Actual cost not to 37.5 cents per mile Actual cost, no $10,568 exceed $125 per exceed $40 per day stated maximum (3 percent of night administrative costs) Source: County commission travel policies. Percentages were computed using the working definition of administrative costs we developed to allow for comparison among county commissions. * First 5 Kern’s executive director receives a monthly automobile allowance. As a result, First 5 Kern’s policy limits the director’s mileage reimbursement rate to 22 cents per mile. RECOMMENDATIONS To address the sustainability of their programs, the state and county commissions should continue to take action to identify and apply for any available grants, gifts, donations, or other sources of funding. 4422 California State Auditor Report 2003-123 California State Auditor Report 2003-123 4433 To demonstrate its commitment to keeping administrative costs low, each county commission, which has not already done so, should define what constitutes its administrative costs, set a limit on the amount of funding it will spend on such costs, and annually track expenditures against this self-imposed limit. n 4422 California State Auditor Report 2003-123 California State Auditor Report 2003-123 4433 Blank page inserted for reproduction purposes only. 4444 California State Auditor Report 2003-123 California State Auditor Report 2003-123 4455 CHAPTER 3 Lacking Data on Program Outcomes, the Commissions Cannot Yet Tell Whether Funded Projects Have Had a Positive Impact CHAPTER SUMMARY Evaluation efforts by the California Children and Families Commission (state commission) and each of the five Children and Families County Commissions (county commissions) we visited show minimal reporting of outcome-based data, as required by the California Children and Families Act of 1998 (Act). Having begun funding its program evaluation in 2001 and 2002, the state commission is in the early stages of capturing the information needed to assess the impact of its programs. The state commission has contracted with two outside evaluators. One is performing a three-year assessment of programs for retaining child care workers, and the other is working on a 33-month data collection and evaluation of other programs, including the School Readiness Initiative, the state commission’s largest active program. At the county level, county commissions have been gathering data from service providers, but service providers have collected scant performance-based outcome data. While one county commission’s outside evaluators have focused only on discussing various aspects of programs and have yet to measure program outcomes, other county commissions’ outside evaluators have expressed concerns that service providers are not capturing enough information to reasonably gauge program success. For example, the evaluator for First 5 Kern concluded that data from that county commission’s service providers yielded little results-based outcome data for analysis. Further, reviews of county commissions do not always provide a comprehensive and objective look at their performance. COMMISSIONS HAVE ONLY RECENTLY BEGUN EFFORTS TO MEASURE SUCCESS USING OUTCOME-BASED ACCOUNTABILITY STANDARDS The Act requires the state and county commissions to evaluate the impact of their programs, including how funds are spent, their progress toward goals and objectives, and the measurement 4444 California State Auditor Report 2003-123 California State Auditor Report 2003-123 4455 of specific outcomes through appropriate indicators. However, for the state commission and five county commissions, we Commissions have found only minimal reporting of program performance only recently begun outcomes using appropriate, reliable indicators. Instead, the to evaluate program commissions have only recently begun to evaluate program effectiveness using effectiveness using performance outcome-based data, and most performance outcome of the effort so far has resulted in the reporting of demographic data, and most of and service output data, such as the number of parental classes the effort so far has taught or the number of children enrolled in a program. As we resulted in the reporting explained in the Introduction, performance outcomes are changes of demographic and in behavior, knowledge, or situation accomplished by providing service output data, a service output. For example, an appropriately funded program such as the number of might demonstrate that children who attend its classes on how classes taught or the to brush their teeth properly (output) have significantly less plaque number of children on their teeth (outcome) than before they attended the classes. enrolled in a program. A Statewide Evaluation System Is Under Development Because the state commission is in the early stages of capturing information to assess the impact of its funded programs, at the time of our fieldwork there were no definitive data showing program success. The state commission’s efforts to measure program success rely on two external consultants who are responsible for data collection and evaluation. In April 2002, the state commission contracted with an outside evaluator for a three-year assessment of the relative effectiveness of various strategies used by programs that give incentives to retain child care workers. This evaluation will provide information for the state commission’s third largest active program, Retention Incentives for Early Care and Education Providers (Retention Incentives Program). In May 2002, the state commission contracted with another outside evaluator to create a statewide evaluation team that would collect statewide data during a 33-month evaluation of its other programs, including its largest active program, the School Readiness Initiative. In March 2001, the state commission announced the initial award of funds for the Retention Incentives Program, which addresses the low retention rate of family child care providers and child care center teaching staff by providing incentives for qualified staff to stay in the child care field and improve their education and professional development. To date, the outside evaluator has released two annual reports outlining its preliminary findings on this program. The most recent report, published in September 2003, provides initial findings related to program design and implementation; program participation; and program participants’ training, professional development, 4466 California State Auditor Report 2003-123 California State Auditor Report 2003-123 4477 and retention. This preliminary report stated that about 6 percent of the program’s participants leave the child care field within one year after receiving a retention stipend and that staff turnover at participating child care centers was around 22 percent overall, compared to the 30 percent turnover found in other studies. However, the evaluator warned that without a comparison group in the current study, improvements cannot definitely be attributed to participation in the Retention Incentives Program. The evaluator further stated that its final report, due in the fall of 2004, would provide a more comprehensive evaluation of the program and would include comparisons to a group of early care and education staff who have not participated in the Retention Incentives Program. The state commission awarded funds in July 2002 for the School Readiness Initiative, the state commission’s signature program and the primary means of achieving its overarching goal of ensuring that California’s children are ready for school. The statewide evaluation team developed 76 indicators to gauge progress toward the desired performance outcomes. The statewide evaluation Some of these indicators merely address service outputs such team developed as delivering cultural diversity training to providers, but many 76 indicators to gauge address performance outcomes, such as measuring the incidence progress toward desired of childhood obesity and malnutrition. performance outcomes, many of which address For nearly one-third of the 76 indicators, population-based performance outcomes data are fully or partially the source of the data used to gauge such as measuring the progress. The population-based data that the State includes incidence of childhood in its database reflects all children prenatal to age 5 living in obesity and malnutrition. California, rather than just those children participating in a First 5 program. These data are collected by other entities, such as the California Department of Health Services. The state commission’s director of research and evaluation (research director) said the state commission intends to use population- based data as a framework for providing an overall statewide view. The research director acknowledged that such data do not necessarily reflect the population of children served by First 5 programs, but indicated that such data are not intended to be used alone for funding decisions. When we asked the research director how funding decisions should be made using indicators based on population data, she stated that county commissions should make funding decisions using local data sources and use population data as a secondary source. To support data collection beginning with the School Readiness Initiative, the statewide evaluation team developed the Proposition 10 Evaluation Data System (PEDS) and is collecting data in PEDS 4466 California State Auditor Report 2003-123 California State Auditor Report 2003-123 4477 from four different data sources: (1) population-based data from public databases, (2) surveys of county commissions and surveys of other entities, (3) intake and follow-up interviews with certain program participants, and (4) developmental profiles based on family interviews and teacher assessments. In April 2004, the state commission issued a 2003 overview and initial statewide results of kindergarten profile data collected from family interviews and teacher checklists from a representative sample of 84 schools in 37 counties. The profile data are considered the first snapshot The state commission’s of children’s developmental competencies at the kindergarten research director level and will be repeated for 2004 to look at trends. The research estimated that it will be director estimated that it will be another two years before enough another two years before data will have been collected to report usable performance enough data will have outcomes. The state commission acknowledged that its indicators been collected and evaluation will not allow for definitive statements such as to report usable “First 5 programs are the sole cause for improved outcomes for performance outcomes. children and families at the program or community levels.” Instead, the indicators and evaluations will allow for statements such as “Positive changes are occurring in those areas First 5 is targeting with its activities.” According to Outside Evaluators, Some County Commissions’ Service Providers Have Collected Little Data on Performance Outcomes For some county commissions we reviewed, external evaluators’ published reports discuss concerns with their service providers’ data, citing various limitations of the data, including that most of the data address outputs instead of specific performance outcomes based on appropriate, reliable indicators. For example, in May 2001, First 5 Kern entered into an agreement with a foundation of the California State University, Bakersfield, to evaluate specific programs that the county commission funds. In its first annual report of findings, released in August 2003, this evaluator reviewed all available data from First 5 Kern’s service providers and concluded that the majority of the data are output-based, with only a small number of programs providing results-based outcome data for analysis. In addition, the evaluator reported severe limitations in drawing conclusions from comparisons of countywide baseline data to data provided by service providers. The evaluator stated that since the children being reported on are not from a randomly selected sample of children in the county, there is no guarantee that the children are representative of all children who receive First 5 Kern services or of the population of children in the county. In 4488 California State Auditor Report 2003-123 California State Auditor Report 2003-123 4499 addition, the evaluator stated that small sample sizes mean that reported trends may not be an accurate representation of the actual trends in the population, given a larger sample. In its December 2003 interim report, the evaluator provided a midterm report of data and information on the programs and initiatives supported by First 5 Kern. The evaluator reported that there was not a sufficient amount of information for definitive conclusions, but expects that this will change when it delivers its second annual report, due in August 2004. The evaluator believes that, overall, a good structure exists for data collection for almost all programs and is working with each provider to fully realize the data potential. Other county commissions are experiencing similar challenges. For example, in November 2003, First 5 Santa Clara’s external evaluator released a report stating that service providers would begin collecting short-term outcome data based on standardized indicators in the ensuing years. The external evaluator’s data for the previous years were largely either demographic or service output data collected from First 5 Santa Clara’s service providers. Some of First 5 Santa Clara’s difficulties with data collection may be attributable to its own actions. In April 2000, First 5 Santa Clara hired an external evaluator, only to terminate the contract prior to any formal report on performance outcomes. According to the acting executive director, First 5 Santa Clara hired a full-time evaluation director in September 2002 to evaluate the success of its programs in-house. However, after nine month’s effort failed to produce any reports, First 5 Santa Clara rehired its former external evaluator in June 2003 and hired a new evaluation director effective January 2004. First 5 El Dorado’s data are also limited in several respects. Citing wide variances in the evaluation instruments used by the external evaluator and the belief that the county commission would be in a better position to answer questions regarding gaps in service and to determine the differences funded dollars have made in the community, the county commission voted not to renew the evaluator’s contract, and as of July 1, 2003, is evaluating its programs through a combination of the program coordinator employed by the commission and state- funded evaluation systems such as PEDS. However, prior to this decision, the external evaluator published a report in June 2003 finding three major limitations with service provider data: the small number of children served compared to the number of 4488 California State Auditor Report 2003-123 California State Auditor Report 2003-123 4499 individuals living in the county, limiting any generalizations that might be made to a larger group; the small number of follow-up assessments, limiting the conclusions that might be drawn from the data; and the relatively brief time between pretests and posttests, allowing no conclusions about long-term change from short-term data. First 5 Los Angeles is another county commission with little recent reported performance outcome data. In its fiscal year 2001–04 strategic plan, First 5 Los Angeles outlined its intentions to use four county-wide indicators to measure progress toward preparing children for school. However, First 5 Los Angeles later decided that these four indicators would not adequately capture the level of performance outcome information the commission wanted to address. Therefore, in January 2003 First 5 Los Angeles established a workgroup charged with reaching a consensus on a new set of indicators. The commission later adopted 15 such indicators in June 2003. Early in 2004, First 5 Los Angeles reported preliminary data on these indicators and a baseline report is due out in September 2004. In addition, our review of evaluation reports specific to three significant initiatives sponsored by First 5 Los Angeles indicates that its current research efforts are focused mainly on the types of service providers receiving funds, the strategies they use, and the populations they serve. First 5 San Diego has yet to report substantial performance outcome data. Its former evaluator’s final report measured the data management capabilities of the county commission’s service providers but stated that many service providers did not collect data. In fact, First 5 San Diego provided only two reports from service providers with the capacity to collect outcome-based data on their programs for fiscal year 2002–03. A report from First 5 In August 2003 the county commission hired a different San Diego’s evaluator outside evaluator who has completed a report on certain service concluded that the system providers’ efforts to collect data for the statewide evaluation used to collect data pilot project. This report found that because service providers provided information on collected data from only a small percentage of eligible program services provided (outputs) participants, the results could not be projected to the entire but not on how these group. The report concluded that the system used to collect services affect changes data provided information on services provided (outputs) but in children and families not on how these services produce changes in children and associated with the county families associated with the county commission (outcomes). The commission (outcomes). county commission’s new external evaluator is collecting data and reports that certain outcomes will be included in its annual evaluation report due to the county commission in July 2004. 5500 California State Auditor Report 2003-123 California State Auditor Report 2003-123 5511 INTERNAL AND EXTERNAL REVIEWS OF COUNTY COMMISSION OPERATIONS FAIL TO ADEQUATELY ADDRESS PERFORMANCE Reviews of county commission operations do not always give a comprehensive and objective look at performance. Although each county commission we visited undergoes an annual independent financial audit of its operations, following well- established and generally accepted standards, similar reviews of the county commissions’ performance are not occurring. Instead, the county commissions’ annual reports to the state commission consist primarily of self-generated descriptions of their programs, planning efforts, and funding priorities. These reports lack an objective review of how the county commissions are managing their programs and also lack an assessment of how well county commissions are ensuring that they meet the Act’s goals and objectives. The Act authorizes the state commission to allocate revenues to county commissions, provided they first satisfy certain requirements, one of which is to conduct an audit and prepare a report on their performance during the preceding fiscal year. At a minimum, the audit is to address how each county commission spent its funds, the progress it made toward its respective goals and objectives, and the measurement of specific outcomes through appropriate and reliable indicators. To address this performance audit requirement, the state commission implemented a reporting template that solicits The state commission a variety of information, including sections that address does not believe the Act performance issues such as the reporting of major requires a performance accomplishments; the disclosure of program challenges; and audit in a technical sense. the status of local evaluation, reporting, and data collection Instead, it interprets the efforts. However, the county commissions we visited mainly Act to require county used these sections to describe their efforts, rather than to commissions to use critically review those efforts. Because the Act requires these indicators and other audits and reports before each county commission receives its measures of performance allocation of revenues each year, we asked the state commission outcomes to document about its efforts to ensure that the information meets the whether they are meeting Act’s requirements. According to its chief deputy, the state their goals and commission does not believe that the Act requires a performance achieving results. audit in the technical sense. Instead, the state commission interprets the Act to require county commissions to use indicators and other measures of performance outcomes to document whether they are meeting the goals of their strategic plans and achieving results. We believe the requirement calling 5500 California State Auditor Report 2003-123 California State Auditor Report 2003-123 5511 for an audit of county commissions’ annual performance does have a technical connotation. To be meaningful, these audits should be objective and should follow established guidelines designed to assess each county commission’s performance. Although county commissions are providing, through their annual reports, some level of data reporting on their respective service providers, these data are primarily service oriented (outputs) and address performance outcomes only on a limited basis. The source of much of these data is the previously discussed program evaluators; however, county commissions hired these evaluators to assess provider-submitted data, not to perform a critical review of their respective operations. As such, these consultants are almost exclusively focusing their reviews on service providers and therefore only indirectly address the performance of the county commissions. RECOMMENDATIONS To ensure that county commissions are basing their funding decisions on outcome-based data, as required by the Act, they should address the concerns expressed by their outside evaluators to ensure that service providers are collecting these data. To provide a meaningful assessment of annual performance, the state commission should require each county commission to conduct an annual audit of its performance prior to any revenue allocations for subsequent years. Such audits should be objective and should follow guidelines designed to critically assess each county commission’s performance. 5522 California State Auditor Report 2003-123 California State Auditor Report 2003-123 5533 We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, ELAINE M. HOWLE State Auditor Date: July 15, 2004 Staff: Doug Cordiner, CGFM, Audit Principal Theresa Gartner, CPA Theresa M. Carey, CPA, CFE Jonnathon Kline Susie Lackie, CPA 5522 California State Auditor Report 2003-123 California State Auditor Report 2003-123 5533 Blank page inserted for reproduction purposes only. 5544 California State Auditor Report 2003-123 California State Auditor Report 2003-123 5555 Agency’s comments provided as text only. First 5 California California Children and Families Commission 501 J Street, Suite 530 Sacramento, CA 95814 July 1, 2004 Ms. Elaine M. Howle* California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: Thank you for the opportunity to respond to the Bureau of State Audits draft report, “California Children and Families Commission (CCFC): While Some County Commissions’ Contracting Practices are Lacking, Both State and County Commissions Can Improve Their Efforts to Find Funding Partners and Data on Program Performance.” We appreciate the thoroughness of the audit, its sensitivity to our fundamental goals, and its recommendations for improving our work. There are two specific recommendations for which we are providing responses. The first deals with the state commission’s need to apply for more outside funding opportunities. The second deals with the issue of evaluation and county performance. First 5 California has documented success in receiving significant funding commitments from the foundation community, other private and public partners, as well as the state and federal governments. We have been successful, as you noted, in obtaining federal Medicaid funding for our Kit for New Parents, and those funds are reflected in our financial statements. In addition to this, however, First 5 California has entered into funding partnerships that total in excess of $53 million 1 from public and private partners and an additional $27 million through funding partnerships with other state agencies. Although philanthropic foundations and First 5’s other private and public partners have shown themselves to be ready to commit large amounts of funding to support the efforts of First 5 California, they are hesitant to directly donate funding for deposit in the CCFC fund. The hesitancy is borne of concerns about losing control over the use of the funding and the perceived slowness of the state contracting process. By directly funding needed activities, such as technical assistance * California State Auditor’s comment appears on page 63. 5544 California State Auditor Report 2003-123 California State Auditor Report 2003-123 5555 Ms Elaine M. Howle July 1, 2004 Page 2 on issues ranging from county commission start-up and strategic planning to school readiness project planning, and a significant portion of our Health Access Initiative, foundations have devoted significant resources to specific areas of need in an expeditious and targeted manner. In the absence of these donated funds, First 5 would have had to allocate over $53 million to provide these services. Because of this support, we believe that the partnerships we have forged should be recognized as successful efforts by the state commission to leverage our funding and secure substantial financial contributions from both the private and public sector. We will continue to pursue ways to secure additional funding to support our declining revenues; in fact, we anticipate federal matching funds for our new Health Access initiative. Our other response deals with the issue of evaluation and county performance. Chapter 3 of your report reflects the complexity of the state commission’s responsibility to evaluate not just the initiatives that are funded by the state commission, but to also evaluate the statewide effectiveness of Proposition 10 and to measure the progress of the state and county commissions toward achieving the goals of the Act. We appreciate the time and effort the auditors committed to understanding the many evaluation components being utilized by the state commission, as well as county commissions. Because the state commission’s evaluation processes are multifaceted, some further clarification may be needed to ensure our efforts are accurately represented. The First 5 California Children and Families Commission is conducting two distinctive types of evaluation: 1. Program evaluation of state commission funded initiatives (for example: Matching Funds for Retention Incentives for Early Care and Education Providers, Kit for New Parents, Early Oral Health Initiative, etc.). These evaluations are designed specifically to monitor program effectiveness and to measure progress toward identified outcomes. Most of these evaluations are completed through contracts with the UC system. 2. Statewide data collection and evaluation (for county funded programs and state/ county funded programs, such as the School Readiness Initiative). First 5 California is required to evaluate the impact of the First 5 California Children and Families funds at the state and county levels, including the manner in which funds were expended, the progress toward and achievement of goals and objectives, and the measurement of specific outcomes through the appropriate indicators. In April 2002, the Commission awarded a contract to SRI International for $23.5 million over three years, to design and conduct the Statewide Data Collection and Evaluation and the School Readiness Initiative Evaluation. 5566 California State Auditor Report 2003-123 California State Auditor Report 2003-123 5577 Ms Elaine M. Howle July 1, 2004 Page 3 The statewide evaluation is examining First 5 funded programs overall and the added value of School Readiness Initiative funded programs. The goal of the First 5 funded statewide evaluation is to evaluate state and county policies and programs to ensure that California’s children get the best start in life by: • Supporting the continuous improvement of local and state activities related to children from birth to age 5 and their families. • Supporting accountability to the public, State Legislature, and the Governor. • Producing reliable, high-quality information about: o Results for children and families (at the participant and community levels), disaggregated by key demographics (e.g., ethnicity, primary language) and for children with special needs. o Implementation of First 5 strategic plans and activities, disaggregated by key demographics. o Improvements in local systems of care. o Community context. o Promising practices. The statewide evaluation is collecting several types of data: • Annual report narratives and fiscal data on how funds are expended • Individual outcome and service data on intensively served clients • Aggregated service information on less intensely served participants • Population-based data for trend analysis • Kindergarten Entry Profiles: parent interviews and teacher surveys • Systems change surveys and interviews • Case studies • Special studies Although we believe we have in place the infrastructure to collect outcome data and assess over time the programs funded by the state and county commissions, we also have been considering additional methods of strengthening the existing requirements of the Act relating to audits and data reporting. In fact, we have been working with Assemblywoman Chan’s office during this legislative session in developing legislation (AB 380) that would clarify the Act’s requirement for conducting audits and reporting outcomes and other program data. We will use the results of this audit to help us craft that legislation and will report back to your office in the course of required follow-ups to provide you with the status of these efforts. We want to thank you and your audit team again for conducting a professional and thorough review and identifying areas where we can improve our efforts to secure additional funding and to improve our reporting on program performance and outcomes for children. Attached is our response to the recommendations made in the draft report. 5566 California State Auditor Report 2003-123 California State Auditor Report 2003-123 5577 Ms Elaine M. Howle July 1, 2004 Page 4 As you requested, our written response is enclosed in the envelope provided by your office. Also, to facilitate uploading of the final report and response, we copied our response, including the cover letter, on the enclosed diskette. If you have any questions or more information is needed, please contact Joe Munso at (916) 323-0056. Sincerely, (Signed by: Jane Henderson) Jane Henderson, Ph.D Executive Director First 5 California Children and Families Commission 5588 California State Auditor Report 2003-123 California State Auditor Report 2003-123 5599 The following First 5 California Children and Families Commission responses are provided for each of the auditor recommendations pertaining to the state commission included in the draft audit report. Auditor Recommendation No. 2 To address the sustainability of their programs, the state and county commissions should take action to identify and apply for grants, gifts, donations, or other sources of funding. CCFC Response: First 5 California has documented success in receiving significant funding commitments from the foundation community, other private and public partners, as well as the state and federal governments. We have been successful, as you noted, in obtaining federal Medicaid funding for our Kit for New Parents, and those funds are reflected in our financial statements. In addition to this, however, First 5 California has entered into funding partnerships that total in excess of $53 million 1 from public and private partners and an additional $27 million through funding partnerships with other state agencies. Although philanthropic foundations and First 5’s other private and public partners have shown themselves to be ready to commit large amounts of funding to support the efforts of First 5 California, they are hesitant to directly donate funding for deposit in the CCFC fund. The hesitancy is borne of concerns about losing control over the use of the funding and the perceived slowness of the state contracting process. By directly funding needed activities, such as technical assistance on issues ranging from county commission start-up and strategic planning to school readiness project planning, and a significant portion of our Health Access Initiative, foundations have devoted significant resources to specific areas of need in an expeditious and targeted manner. In the absence of these donated funds, First 5 would have had to allocate over $53 million to provide these services. Equally important is First 5’s commitment to and success in partnering with other state agencies to utilize and coordinate existing state and local government infrastructures and services. First 5 California has partnered with numerous state and county agencies to maximize efficiencies in providing needed services to young children, parents and caregivers, including collaborating with the State Department of Health in promoting tobacco cessation and with the State Department of Social Services to promote the Safely Surrendered Baby Campaign. All total, in excess of $27 million in additional funds for services has been accessed through funding partnerships with other governmental agencies. Because of this support, we believe that the partnerships we have forged should be recognized, as successful efforts by the state commission to leverage our funding and secure substantial financial contributions from both the private and public sector. We will continue to pursue ways to secure additional funding to support our declining revenues; in fact, we anticipate federal matching funds for our new Health Access Initiative. 1 5588 California State Auditor Report 2003-123 California State Auditor Report 2003-123 5599 Auditor Recommendation No. 3 To provide a meaningful assessment of annual performance, the state commission should require each county commission to conduct an annual audit of its performance prior to any revenue allocations. Such audits should be objective and follow guidelines designed to critically assess each county commission’s performance. CCFC Response: Chapter 3 of your report reflects the complexity of the state commission’s responsibility to evaluate not just the initiatives that are funded by state commission, but to also evaluate the statewide effectiveness of Proposition 10 and measure the progress of the state and county commissions toward achieving the goals of the Act. We appreciate the time and effort the auditors committed to understanding the many evaluation components being utilized by the state commission, as well as county commissions. Because the state commission’s evaluation processes are multifaceted, some further clarification may be needed to ensure our efforts are accurately represented. The First 5 California Children and Families Commission is conducting two distinctive types of evaluation: 1. Program evaluation of state commission funded initiatives (for example: Matching Funds for Retention Incentives for Early Care and Education Providers, Kit for New Parents, Early Oral Health Initiative, etc.). These evaluations are designed specifically to monitor program effectiveness and to measure progress toward identified outcomes. Most of these evaluations are completed through contracts with the UC system. 2. Statewide data collection and evaluation (for county funded programs and state/ county funded programs, such as the School Readiness Initiative). First 5 California is required to evaluate the impact of the First 5 California Children and Families Funds at the state and county levels, including the manner in which funds were expended, the progress toward and achievement of goals and objectives, and the measurement of specific outcomes through the appropriate indicators. In April 2002, the Commission awarded a contract to SRI International for $23.5 million over three years, to design and conduct the Statewide Data Collection and Evaluation and the School Readiness Initiative Evaluation. The statewide evaluation is examining First 5 funded programs overall and the added value of School Readiness Initiative funded programs. The goal of the First 5 funded statewide evaluation is to evaluate state and county policies and programs to ensure that California’s children get the best start in life by: • Supporting the continuous improvement of local and state activities related to children from birth to age 5 and their families. • Supporting accountability to the public, State Legislature, and the Governor. • Producing reliable, high-quality information about: 2 6600 California State Auditor Report 2003-123 California State Auditor Report 2003-123 6611 o Results for children and families (at the participant and community levels), disaggregated by key demographics (e.g., ethnicity, primary language) and for children with special needs. o Implementation of First 5 strategic plans and activities, disaggregated by key demographics. o Improvements in local systems of care. o Community context. o Promising practices. The statewide evaluation is collecting several types of data: • Annual report narratives and fiscal data on how funds are expended • Individual outcome and service data on intensively served clients • Aggregated service information on less intensely served participants • Population-based data for trend analysis • Kindergarten Entry Profiles: parent interviews and teacher surveys • Systems change surveys and interviews • Case studies • Special studies Although we believe we have in place the infrastructure to collect outcome data and assess over time the programs funded by the state and county commissions, we also have been considering additional methods of strengthening the existing requirements of the Act relating to audits and data reporting. In fact, we have been working with Assemblywoman Chan’s office during this legislative session in developing legislation (AB 380) that would clarify the Act’s requirement for conducting audits and reporting outcomes and other program data. We will use the results of this audit to help us craft that legislation and will report back to your office in the course of required follow-ups to provide you with the status of these efforts. 3 6600 California State Auditor Report 2003-123 California State Auditor Report 2003-123 6611 Blank page inserted for reproduction purposes only. 6622 California State Auditor Report 2003-123 California State Auditor Report 2003-123 6633 COMMENT California State Auditor’s Comment on the Response From First 5 California To provide clarity and perspective, we are commenting on First 5 California’s (state commission) response to our audit report. The number corresponds with the number we have placed in the state commission’s response. 1 The funding partnerships the state commission refers to in its response are for the commitments these entities have made to provide in-kind amounts; they do not represent separate revenue sources to be provided to the state commission. 6622 California State Auditor Report 2003-123 California State Auditor Report 2003-123 6633 Blank page inserted for reproduction purposes only. 6644 California State Auditor Report 2003-123 California State Auditor Report 2003-123 6655 Agency’s comments provided as text only. First 5 El Dorado Children and Families Commission 4111 Creekside Drive, Suite B Shingle Springs, California 95682 June 30, 2004 Elaine M. Howle* California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: This is the official response regarding the Redacted copy of the state audit preformed by the Bureau for First 5 El Dorado Children and Families Commission. Page 4 (Summary Text) 1 “Having spent little of their Proposition 10 allocations on early childhood development programs, the county commissions we reviewed maintained significant fund balances as of June 30, 2003.” First 5 El Dorado Response: First 5 El Dorado Commission funds programs for children prenatal and birth to age five and their families. The commission has assured, through monitoring, that all funds only serve children and families in this category. First 5 El Dorado estimates 63% of the funds have been allocated to programs in the county out of all Prop.10 funds collected. By June 2003, First 5 El Dorado had been funding programs for only two years. In efforts to assure that practices and procedures were in place, avoiding conflict of interest and allowing broad based funding opportunities, the Commission needed to establish funding policies. Each county commission was in a position of having unlimited funding needs and requests with limited funds. The Commission adopted funding cycles in an effort to assure that funds were allocated in a timely manner. * California State Auditor’s comment appears on page 69. 6644 California State Auditor Report 2003-123 California State Auditor Report 2003-123 6655 Chapter 2 Fund Balances First 5 El Dorado Response While First 5 El Dorado shows a decline in the amount of the fund balance in Table 2, please note that First 5 El Dorado does maintain a Sustainability fund to project current funded programs for a two-year period. The commission felt this was a prudent way to manage funds and protect programs providing services. The Sustainability Fund is not separated by the county Trust fund report and is not reported on as a category for the annual audit. Page 9-Efforts to Obtain Funding Partners Have Produced Little Non-State Funding First 5 El Dorado Response As this audit reported through June 2003, First 5 El Dorado agrees with the Auditors in terms of looking for other sources of revenue. First 5 El Dorado did receive a federal Early Learning Opportunities Act grant for $439,000 in October 2003. Since the revision of the Strategic Plan beginning July 1, 2003, Focus Area 1, Healthy Children, First 5 Commission coordinated efforts with the Master Settlement Tobacco dollars allocated to the county, partnered with the Public Health Department, participated in the HCAP grant and coordinated efforts to apply for additional federal funds with the Department of Mental Health. First 5 El Dorado sponsored and coordinated efforts with the Sacramento Regional First 5 Commissions’ Group to apply to the Packard Foundation to support a Training and Technical Assistance Grant for the Universal Health Regional Program. Since the First 5 El Dorado Commission was relatively new as an agency, it took time to begin to partner with community and governmental agencies. Page 12-County Commission Lacks Clear Commitment to Limit Their Administrative Spending First 5 El Dorado Response While First 5 El Dorado agrees there are no written approved Commission policies regarding administration costs, the Commission did follow general accounting rules, and in the first two years of releasing RFP’s the Commision did not allow programs applying for funding to charge administration and indirect costs separately. The Commission does not allow a contractor to charge administration and indirect costs for subcontracted work. First 5 El Dorado did not consider Professional Development and Evaluation as administration costs. The Commission will develop and adopt administrative cost policies. 6666 California State Auditor Report 2003-123 California State Auditor Report 2003-123 6677 Page 17-According to Outside Evaluators, County Commissions’ Service Providers Have Collected Little Data on Performance Outcomes First 5 El Dorado Response First 5 El Dorado continues to review the evaluation process. Commission staff and Commissioners have researched a variety of methods to assure the evaluation process is effective and provides reliable data. The commission is involved in the state process to gather data through the PED’s program and will continue to monitor and evaluate the process. If you need further information please do not hesitate to contact the Commission. Thank you for your time and services. Sincerely, (Signed by: ) Steven M. Thaxton Executive Director 6666 California State Auditor Report 2003-123 California State Auditor Report 2003-123 6677 Blank page inserted for reproduction purposes only. 6688 California State Auditor Report 2003-123 California State Auditor Report 2003-123 6699 COMMENT California State Auditor’s Comment on the Response From First 5 El Dorado To provide clarity and perspective, we are commenting on First 5 El Dorado’s response to our audit report. The number corresponds with the number we have placed in the response from First 5 El Dorado. 1 First 5 El Dorado is quoting a sentence in the redacted draft report that was revised. As shown on page 3, the sentence now reads “Having spent as little as 15 percent to as much as 67 percent of their revenues on early childhood development programs, the county commissions we reviewed maintained significant fund balances as of June 30, 2003.” 6688 California State Auditor Report 2003-123 California State Auditor Report 2003-123 6699 Blank page inserted for reproduction purposes only. 7700 California State Auditor Report 2003-123 California State Auditor Report 2003-123 7711 Agency’s comments provided as text only. First 5 Kern 2724 L Street Bakersfield, California 93301 July 6, 2004 Elaine M. Howle, State Auditor* Also Sent By Fax to: (916) 327-0019 Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Re: Audit Response – First 5 Kern Dear Ms. Howle: The purpose of this letter is to transmit to you our comments in response to the recommendations contained in the draft California Children and Families Act Audit Report. In accordance with your instructions, our comments are responsive only to the three recommendations identified as pertaining to First 5 Kern. Recommendation #1: To ensure the appropriate use of program funds and instill public confidence, county commissions should adopt and follow well-defined policies to guide their allocation efforts and should also maintain sufficient documentation to support their allocation practices. Response: We agree with this recommendation. Funding decisions for contractors are substantiated through a process that includes Conditions of Approval, revised Scopes of Work, and revised Project Budgets, all of which are intended as opportunities to address any concerns. In the case of the Kern County Network for Children contract cited in the Audit Report, such documentation was provided by the contractor and reviewed by First 5 Kern staff. In accordance with your recommendation, we will implement an additional step in our process to ensure that concerns expressed by the Independent Evaluation Committee are resolved and adequately documented. The contract for data evaluation services with California State University Bakersfield’s Applied Research Center was not intended to be competitive. Neither, as your Audit Report points out, was a competitive process required. The award of the contract was made in May 2001, at a properly noticed public meeting, after full disclosure and adequate opportunity for review and discussion by any interested party. In accordance with your recommendation, we will in the future ensure that the nature of all awards is clearly disclosed. County commissions at the time were new and exploring evaluation practices, including partnerships with universities and evaluation experts. First 5 Kern desired to utilize the expertise and evaluation model of our local State University, and no other potential evaluators were involved. The contractor for data management services was selected through an Invitation to Bid process. The contract with CS&O was approved by the Commission in June 2001, also at a properly noticed public meeting, after full disclosure and adequate opportunity for review and discussion by any interested party. As your Audit Report notes, proposals were received and reviewed by the Ad Hoc Evaluation Subcommittee and subsequently by the Technical Advisory Committee, which then made a recommendation to the Commission. Documentation was provided to Commissioners in the form of * California State Auditor’s comment appears on page 73. 7700 California State Auditor Report 2003-123 California State Auditor Report 2003-123 7711 Audit Response – First 5 Kern July 6, 2004 Page 2 a letter from the Executive Director supporting the recommendation for approval. In accordance with your recommendation, in the future we will consider the use of scoring tools, and we will ensure that the decision-making process is thoroughly documented. Recommendation #2: To address the sustainability of their programs, county commissions should take action to identify and apply for any available grants, gifts, donations, or other sources of funding. Response: We agree with this recommendation and will continue to explore opportunities for other sources of funding. We attend sustainability workshops, research foundation funding, and have applied for federal funding and grants for the School Readiness Initiative and Children’s Health Initiative. Each of our contractors is required to write their own Sustainability Plan as part of the contract for each project. We forward information about funding opportunities to our contractors as part of our technical assistance. We also maintain a Commission approved Five Year Financial Plan, which includes a $5M reserve for sustainability. We also recognize, of course, that many programs will not be sustained over time. Recommendation #3: To ensure that county commissions are basing their funding decisions on outcome-based data, as required by the Act, they should address the concerns expressed by their outside evaluators to ensure that service providers are collecting these data. Response: We agree with this recommendation and believe we have addressed and continue to address our evaluator’s concerns. In consideration of implementing a process of this magnitude, county commissions anticipated a long-term process to obtain baseline and client-level data to evaluate provider performance and, more importantly, to measure the impact of programs on children and families. Our local evaluator has worked extensively with contractors on data collection and submission. During FY2003-04, we obtained data to monitor contractor performance and to guide funding recommendations to the Commission. The evaluator’s annual report due in August 2004 is expected to document this improvement. Thank you for the opportunity to review the redacted version of the draft Audit Report provided to us last week. Although the time for review was limited, it did allow us a better opportunity to research issues and to prepare a more thoughtful response. Please also be advised 1 that we have not yet seen the final version of the Audit Report’s tables. While we do not believe any significant differences of opinion between your staff and First 5 Kern exist, we would like to reserve the right to comment further if the final tables raise any significant issues. We look forward to the opportunity to review the entire final Audit Report. If you have any questions or require additional information, please contact me directly at (661) 328-8889. Sincerely, (Signed by: Steven G. Ladd) Steven G. Ladd Executive Director 7722 California State Auditor Report 2003-123 California State Auditor Report 2003-123 7733 COMMENT California State Auditor’s Comment on the Response From First 5 Kern To provide clarity and perspective, we are commenting on First 5 Kern’s response to our audit report. The number corresponds with the number we have placed in First 5 Kern’s response. 1 At our exit conference with First 5 Kern held during the review period given to the county commission to review and respond to a redacted draft report containing information specific to the county commission, staff voiced a concern that Table 3 on page 33 was incomplete. Specifically, staff asserted that two amounts designated by its commission for approved programs were not reflected in the table. We based our table on the county commission’s audited financial statements as of June 30, 2003, that did not include these designations. We therefore requested that First 5 Kern provide us documentation supporting its assertions. As of the end of First 5 Kern’s review and response period, it had not furnished the evidence we would need to change the table. Therefore, Table 3 along with the other tables included in the redacted draft provided to First 5 Kern is the final version. 7722 California State Auditor Report 2003-123 California State Auditor Report 2003-123 7733 Blank page inserted for reproduction purposes only. 7744 California State Auditor Report 2003-123 California State Auditor Report 2003-123 7755 Agency’s comments provided as text only. First 5 Los Angeles County 333 S. Beaudry Ave., Suite 2100 Los Angeles, CA 90017 July 2, 2004 Elaine M. Howle* State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Ms. Howle: The following is the written response from First 5 LA to the State Auditor’s findings based on our meeting with representatives from your office (Doug Cordiner, Audit Principal and Theresa Carey, Auditor Evaluator II) on June 28, 2004 regarding the draft report of our county commission’s administrative and programmatic operations. Chapter 2: As Most Fund Balances Grow, Some Commissions Promise to Significantly Reduce Them, Section Administrative Spending Plan. First 5 LA, with resources generated from Proposition 10, addresses an urgent societal need to support and strengthen families with young children. The mission of First 5 LA, “…to optimize the development and well- being of children…”, is organized around specific initiatives. For example, there is a Healthy Kids initiative in which insurance coverage is provided for eligible children in Los Angeles County. Initiatives are generally carried out through grants and contracts with local providers. Administrative costs at First 5 LA are calculated as any costs that are not directly a part of an initiative. All staff salaries and direct operating expenses are considered administrative costs. First 5 LA’s administrative costs for fiscal year 2003 were 4.57 percent of total revenue. It is First 5 LA’s practice that administrative costs not exceed 5 percent of total revenue and 10 percent of total expenses. 1 Conversely, the California Bureau of State Audits defined “all costs as administrative if the county commissions did not spend the funds directly on children or their families or grant the funds to service providers”. This definition categorizes some costs directly associated with initiatives as administrative costs. For example, the evaluation of an initiative, which is designed to determine the effectiveness of that initiative, like the effectiveness of providing health insurance to eligible children, is considered an administrative cost by the California Bureau of State Audits, but is not considered an administrative cost by First 5 LA because it is a direct cost of the initiative. One of the functional expense categories that First 5 LA is required to disclose are the provider grants and contracts. By definition, provider grants/contracts need to be disclosed as a functional expense category in accordance with Paragraph 1800.17 of the Codification of Government Accounting and Financial Reporting Standards, which states, according to our audit firm, PriceWaterhouseCoopers, LLP, “Functional or program * California State Auditor’s comment appears on page 79. 7744 California State Auditor Report 2003-123 California State Auditor Report 2003-123 7755 Bureau of State Audits July 2, 2004 Page 2 classification provides information on the overall purposes or objectives of expenditures. Functions group related activities are aimed at accomplishing a major service or regulatory responsibility.” Additionally, the Codification further states in Paragraph G.107 that “governmental entities often receive grants and other financial assistance to transfer to or spend on behalf of a secondary recipient. These amounts are referred to as pass-through grants. All cash pass-through grants received by a governmental entity (referred to as a recipient government) should be reported in its financial statements. As a general rule, cash pass-through grants should be recognized as revenue and expenditures or expenses in a governmental, proprietary, or trust fund…”. This scenario applies to First 5 LA, and as such, this indicates that these grants should be recognized as an expense line item. If the total amounts of the provider grants were to be reclassified to the expense categories of First 5 LA, it would not be possible to determine the amount of funds disbursed to grantees or the amount utilized in the daily operations of First 5 LA from an assessment of the financial statements. First 5 LA’s calculation of administrative costs is in accordance with current accounting standards and is approved by our independent audit firm of PriceWaterhouseCoopers, LLP. Moreover, First 5 LA complies with the guidelines set forth by the California Children and Families Commission. In addition, First 5 LA follows the accounting standards of the Governmental Accounting Standards Board in the creation of its financial statements. First 5 LA also utilizes the American Institute of Certified Public Accountants Audit and Accounting Guide, “Audits of State and Local governments (GASB 34 Edition)” (AICPA Governmental Audit Guide) to ascertain that all fair and legal accounting practice standards have been adhered to. Chapter 3: According to Outside Evaluators, County Commissions’ Service Providers Have Collected Little Data on Performance Outcomes. In accordance with the California Children and Families First Act’s requirement that funding be based on outcome data, First 5 LA adopted a Results-Based Accountability (RBA) evaluation framework in October 2001. RBA begins with the selection of outcomes (conditions of child and family well-being) and the tracking of measurable indicators for those outcomes at the population (i.e., County) level. Initiatives are then developed based on evidence of their potential to impact population level outcomes. Finally, grantees within initiatives are held accountable for tracking both process and outcome data on their target populations to ensure that initiatives are having their intended effects. Implementation of Initiative and Program Level Accountability All First 5 LA grantees are required to collect both implementation (“output”) and impact (“outcome”) data as part of their program-level evaluations. First 5 LA initiative evaluations focus on aggregate analyses of programs’ output and outcome data, taking into consideration the county context. In the early part of an initiative, the evaluation focus is on implementation (“outputs”) more than impact (“outcomes”). There are two primary reasons for this: (1) significant impact on outcomes takes time and requires that programs be fully implemented, and (2) in order to interpret impact, one needs to document implementation. In order to understand why particular outcomes were or were not achieved, the initiative evaluation must look to how programs were implemented (for example, was the intervention implemented as intended? Did recipients receive enough intervention to make an impact on outcomes? Were there significant barriers to implementation that hindered the quality of the intervention?). As of June 11, 2004, the date the information gathering phase of the audit ended, the four initiative evaluators referred to in the audit report had not yet released aggregate analyses of grantee outcome data. Two of those evaluators (for the Child Care and Home Visitation initiatives) are currently in the process of completing these analyses. We anticipate that the summary reports will be disseminated in the Fall of 2004 and will be posted on our website. The other two initiative evaluations covered in the audit report – the Community-Developed Initiatives (CDI) and School Readiness Initiative – both are in the early stages of implementation and thus aggregate outcome data is not yet available. 7766 California State Auditor Report 2003-123 California State Auditor Report 2003-123 7777 Bureau of State Audits July 2, 2004 Page 3 Community Developed Initiative (CDI) Evaluation The CDI Initiative Evaluation began in September 2003. Each of the 54 CDI projects across the three funding cycles is required to conduct a project-specific evaluation that tracks both output and outcome data. As part of their contract, projects either hire an outside evaluator or have in-house staff to take the lead in planning and implementing a project-specific evaluation plan. Because the Community-Developed Initiatives is an open funding stream where communities on their own define priorities and appropriate interventions, the focus of the Initiative Evaluation is an examination of common outcome areas across all 54 grantees. In this first year, the CDI Initiative evaluators will focus on documenting the various implementation strategies and describing the range of projects. In addition, an organizational study will be launched to begin to understand how the grantee organizations function both internally and in the context of target communities. As requested by grantees, technical assistance will be provided to grantees around project-specific evaluation plans including data collection and management. The Year One evaluation report will be made available in October 2004. School Readiness Initiative Evaluation Like the CDI Initiative, the School Readiness Initiative is in the early stages of implementation. The School Readiness Initiative is unique in that it is a collaborative initiative between First 5 LA and First 5 California and thus includes an additional level of evaluation not included in any of our other initiatives. This additional layer of evaluation has posed some challenges and has required some shifting of our evaluation approach. While we have maintained the requirement that all programs design and conduct a program-level evaluation that tracks both output and outcome data and that focuses on continuous quality improvement, the state-level evaluation has added significant data collection burden for grantees. First 5 LA is committed to an initiative- wide evaluation of our School Readiness Initiative and we are in the process of designing that evaluation in a way that will complement both the program’s own evaluations as well as the state-level evaluation without adding too much additional data collection burden on our grantees. Audit Recommendation First 5 LA supports any efforts to improve the quality of outcome-based data collected by our partner service providers. In all of the independent evaluations conducted by First 5 LA, staff coordinates closely with our external evaluators to use available data to make mid course programmatic corrections. As a result of our early evaluation findings, First 5 LA has made substantive changes in the way we and our grantee partners operate. One concrete way in which First 5 LA has been working to improve the quality of outcome-based data collected by our service providers is through a series of comprehensive technical assistance sessions on topics ranging from the development of evaluation plans and data collection methodologies to data analysis techniques and report preparation. First 5 LA believes that evaluation is not just a tool for accountability, but acts as a tool for overall program improvement. Sincerely, (Signed by: Evelyn V. Martinez) Evelyn V. Martinez Executive Director 7766 California State Auditor Report 2003-123 California State Auditor Report 2003-123 7777 Blank page inserted for reproduction purposes only. 7788 California State Auditor Report 2003-123 California State Auditor Report 2003-123 7799 COMMENT California State Auditor’s Comment on the Response From First 5 Los Angeles County To provide clarity and perspective, we are commenting on First 5 Los Angeles’ response to our audit report. The number corresponds with the number we have placed in the response from First 5 Los Angeles. 1 As we clearly state on pages 40 and 41 of the report, we used a working definition for administrative costs to provide comparability among the county commissions we reviewed. We recognize there are other valid definitions. 7788 California State Auditor Report 2003-123 California State Auditor Report 2003-123 7799 Blank page inserted for reproduction purposes only. 8800 California State Auditor Report 2003-123 California State Auditor Report 2003-123 8811 Agency’s comments provided as text only. First 5 Commission of San Diego County 1495 Pacific Highway, Ste 202 San Diego, CA 92101 July 6, 2004 Ms. Elaine M. Howle State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Ms. Howle, On behalf of the First 5 Commission of San Diego, I am responding to the redacted draft copy of your office’s audit report entitled “California Children and Families Commission: While Some County Commissions’ Contracting Practices are Lacking, Both the State and County Commissions Can Improve Their Efforts to Find Funding Partners and Data on Program Performance.” There are a few items we discussed in our meeting on June 25th that we wish to confirm will be in the final report: 1. In Chapter 2, in the section pertaining to administrative spending, we discussed the difficulty in comparing the administrative costs of the audited commissions to each other. As we discussed, all 58 county commissions in California operate differently; for example, some have relationships with their county governments that may include paying for county government services (such as payroll) that support their commissions’ operations. It is valid that our Commission does not have a written ceiling for administrative costs but our commitment that: “Administrative costs will be kept as low as possible,” is directly stated in our Strategic Plan for 2003-2006 and our commitment has been demonstrated in practice by our low administrative costs. The other area discussed was the connection between growing evaluation needs and costs, and commissions’ administrative rates. Commissions are in their 5th year of operation and are at a developmental stage when they are growing and increasing the services they fund. As services increase, so will evaluation costs, and as evaluation costs increase, administrative rates may increase also. The audit report correctly reflects the tension between the need to expand evaluation while holding down administrative rates. 2. In the 4th paragraph of the Summary, Results in Brief, the report indicates, “the public may also be confused by some county commissions’ allocation of funds through noncompetitive contracting practices…price.” We appreciate that San Diego’s practices were not called out in the redacted report under this section as all our grants and contracts are competitively bid except in the rare case when there is only one vendor that can provide the needed service. In such situations, Commission staff must provide a strong sole source justification to the 8800 California State Auditor Report 2003-123 California State Auditor Report 2003-123 8811 Commission and obtain their approval before proceeding to contract with the provider. Our understanding is that, because our Commission was not specifically cited in the redacted version, the report reflects our practices are competitive. 3. As we discussed in our meeting, the information in Chapter 2, Table 3 concerning fund balances reflects data as of June 30, 2003 contained in our Commission’s audited report. We understand that Commission actions from FY 2003-04 are not reflected in the figures contained in the audit report. We are providing information that demonstrates that, subsequent to July of 2003, the Commission allocated $77,830,000 in funds. As referenced in Commission minutes, these include: (a) July 7, 2003 - $4 million for Intergenerational Capacity Building projects (see Item #7 on the agenda); (b) August 4, 2003 - $6 million for Healthcare Access projects (see Item #7 on the agenda); and (c) December 15, 2003 - $60 million for Capital projects (see Item #9 on the agenda); $2.85 million to extend for grants funded under RFGA #20055 for one year (developmental assessments and treatment) (Item #10); $780,000 for a primary care and oral health public education campaign (Item #10); and $4.2 for oral health screening, education and treatment projects (Item #10). We have a correction to the report, in Chapter 2, in the section regarding reserves. The report indicates that San Diego’s sustainability funds will grow to $130,000,000 by fiscal year 2012-13. The actual figure is $129,300,000. Another important point of discussion was in Chapter 3 concerning programmatic outcomes. As discussed in the meeting, we have included outcome data for several of the initiatives of the San Diego First 5 Commission in this response. The documented outcomes include: • a 76% increase in parenting skills, safety awareness and knowledge of development and growth patterns (Children’s Hospital and Health Center, The Welcome Home Baby Program, as measured by pre and post tests); The program tracks nine different outcomes including immunization, breastfeeding, and primary care versus emergency room visits. The 2002- 2003 report is included. These data continue to be tracked. • 45% of the 489 dental screenings conducted found abscesses and referred these children to treatment (North County Health Services, Project All Ready); • a statistically significant correlation between participation in the CARES program and retention of early childhood education staff. In addition, Harder & Co., the independent evaluation consultants for San Diego’s First 5 Commission, found the following organizational and systems outcomes: • increased input of key stakeholders; (leadership teams, other activities); • increased community input on commission activities (community engagement project, community conversations; • increased grantee capacity through development of logic models. 8822 California State Auditor Report 2003-123 California State Auditor Report 2003-123 8833 Recognizing that outcomes are a critical issue in this audit, we would like to add contextual information. There are two key factors that have affected the ability to identify, track and evaluate outcomes for Commission-funded programs: the timeframe for meaningful data to emerge and grantee expertise. For some Commission initiatives such as school readiness and literacy, outcomes cannot be measured for several years, i.e., when the children participating in the programs are in elementary school. Secondly, Commission grantees have different levels of experience in evaluation. For example, a number of healthcare grantees have more sophisticated levels of expertise and established systems for gathering and analyzing data, whereas early childhood education programs have generally required much more technical assistance throughout the entire process. This is a high priority area for the Commission. It is important to note that the trend is that the capacity of our grantees to gather data and the amount of outcome data is consistently expanding. This concludes our response to the draft audit report prepared by the Bureau of State Audits. If you desire additional information, please do not hesitate to contact me. Sincerely, (Signed by: ) Laura S. Spiegel Executive Director Attachmentst tWe have not included attachments in the report; however, they are available for review at the California State Auditor’s office. 8822 California State Auditor Report 2003-123 California State Auditor Report 2003-123 8833 Blank page inserted for reproduction purposes only. 8844 California State Auditor Report 2003-123 California State Auditor Report 2003-123 8855 Agency’s comments provided as text only. First 5 Santa Clara County 1150 S. Bascom Avenue, Suite 12 San Jose, California 95128-3509 July 6, 2004 Elaine M. Howle* State Auditor 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: FIRST 5 Santa Clara County understands the need for accountability of the expenditure of public funds. FIRST 5 Santa Clara County is responsive to community needs through an extensive civic engagement process and utilization of collaborative processes building upon existing community expertise and services in developing our strategic plan, procurement and grant awards. The Bureau of State Auditors’ (the Bureau) report discusses a number of deficiencies regarding documentation 1 of grantee selections. The Bureau’s report lacks an understanding of the extensive collaborative process used by FIRST 5 Santa Clara County. The Bureau also uses a definition of administrative costs that fails to include direct service costs in their calculations and does not account for FIRST 5 2 Santa Clara County employees providing direct services to children and their families. County Commissions Lack Documentation Supporting Their Funding Decisions The report questions our procurement methodologies. In June 2000, the Commission conducted an Allocation Workshop where various methodologies were identified. After the workshop, the Commission approved the Intention to Negotiate (ITN) model, which expands the collaborative, civic engagement process into a procurement methodology. The Commission has utilized ITNs, Request for Proposals, and sole source selection processes. Public documents concerning procurement awards have not routinely referenced the specific procurement method. All award recommendations are discussed at two public Committee meetings; and the Commission makes all award decisions in a public meeting. To address the Bureau’s concern, FIRST 5 Santa Clara County will clearly state the procurement method used to select a grantee or contractor. County Commission Did Not Always Adhere to Established Policies With respect to the Children’s Discovery Museum (CDM) award, the Bureau indicates that “FIRST 5 Santa Clara County did not adhere to … its Unsolicited Request Policy … in one instance” in 3 awarding the funds. This is an inaccurate statement. * California State Auditor’s comments begin on page 89. 8844 California State Auditor Report 2003-123 California State Auditor Report 2003-123 8855 FIRST 5 Santa Clara County’s Rebuttal July 6, 2004 Page 2 of 4 FIRST 5 Santa Clara County’s Unsolicited Request Policy refers to requests under $15,000 to address emergency and/or one-time need. All such requests are forwarded to the Program 3 Development and Communications Committee. The CDM was not an unsolicited request. The CDM project was unique. FIRST 5 Santa Clara County had established the Quality Early Learning Opportunities Collaborative (QUELO Collaborative), which evolved into the Preschool for All Initiative. The QUELO Collaborative included Commissioner Ferrer, the Interim Executive Director, and a wide variety of experts in early childhood development, including the Packard Foundation, City of San Jose, Local Child Care Planning Council, School District Superintendents, Social Services Agency, child care providers, and the local Information and Referral agency. The QUELO Collaborative identified the need to establish a professional, development training center for early care providers and to create a thematic, high-quality learning environment for children. When FIRST 5 Santa Clara County learned of the WonderCabinet project, it was a natural fit with the QUELO Collaborative strategy to establish a quality early learning environment for educating children and providers. The CDM proposal, which included the scope of work and budget for the project, was available to the public and distributed at each of the three public meetings when the project was discussed. The award was not specifically identified as a sole source award. CDM is a unique program which targets services to young children, is centrally located in Santa Clara County and receives support from various foundations. FIRST 5 Santa Clara County funds were used to expand the WonderCabinet project and the Water Ways learning environments to provide quality early learning opportunities for children through age 5 and a thematic learning laboratory for training child care providers. In the future all public documents will delineate the specific procurement method utilized in selecting the grantee or contractor. In addition, where noncompetitive awards are concerned, the documents will include the justification for selecting a sole source provider. Some Funding Decisions Were Not Publicly Disclosed All program funding decisions are presented to two Committees before they are presented to the 4 Commission for final approval. All Committee and Commission meetings are public. Items are identified on agendas and are presented for public input. When the Early Prevention Screening Diagnostic and Treatment (EPSDT) proposal was presented to the Commission, the procurement process was improperly identified on the Commission’s agenda. In order to avoid this problem in the future, all funding recommendations will identify the procurement process used to select the provider. The Bureau also questions the noncompetitive allocation to provide information technology to 5 migrant families. This provider was identified as the best qualified due to their presence onsite and their established relationship with families at the migrant camp. 8866 California State Auditor Report 2003-123 California State Auditor Report 2003-123 8877 FIRST 5 Santa Clara County’s Rebuttal July 6, 2004 Page 3 of 4 Administrative Percentages Vary at the County Commissions FIRST 5 Santa Clara County’s administrative and program expenses are based on a definition in accordance with GAAP (Generally Accepted Accounting Principles) from the Financial Accounting Standards Board (FASB) statement No. 117, which states: Program expenses are the activities that result in goods and services being distributed to beneficiaries, customers, or members that fulfill the purposes or mission for which the organization exists. Those services are the major purpose for and the major output of the organization and often relate to several major programs. Based on this definition, FIRST 5 Santa Clara County’s administrative costs for fiscal year 2003 2 were within the goal of 10% as directed by the Commission. The Bureau’s administrative figure is much greater, due to several faulty assumptions. For example, the Bureau assumed that all employees of FIRST 5 Santa Clara County perform administrative functions, and that only grants and a few miscellaneous expenses (e.g., child care, translation services) qualify as a program or direct service cost. The Bureau ignored the employees of FIRST 5 Santa Clara County who provide direct, program services to children and their families. In fiscal year 2003, the year addressed in the audit, FIRST 5 Santa Clara County employed 47 staff, of which 36 were dedicated to program services (per the GAAP definition). These employees were hired to avoid placing an additional burden on individual grantees and lead agencies (e.g. school district). If the positions were eliminated, each grant would increase because the grantee would provide the services directly. These positions are dedicated to the coordination and integration of FIRST 5 Santa Clara County programs and services in the community. The staff have daily contact with the children and families they serve. By providing services such as outreach and transportation, FIRST 5 Santa Clara County staff assist in building the capacity of each grantee who will eventually assume these responsibilities. The 36 staff’s salaries and benefits are all direct, program expenses. In addition, the Bureau ignored significant service and supply costs attributable to the Regional Partnership, Early Learning Initiatives (School Readiness), Family Court Initiative and Arts Enrichment Initiative directly impacting children and families. These costs include expenses for partnership meetings, mileage for transporting families to services, copying flyers, focus groups, community art events, and trainings for families and community-based organizations. All of these costs are direct, program expenses and not an administrative expense. (See attached Income and Expense Statement.)t When calculated in accordance with the GAAP definition, FIRST 5 Santa Clara County’s administrative costs are under 10%. The Bureau’s administrative expense definition penalizes FIRST 5 by not capturing all costs which have direct impact on children and families. tWe have not included the attachment in the report; however, it is available for review at the California State Auditor’s office. 8866 California State Auditor Report 2003-123 California State Auditor Report 2003-123 8877 FIRST 5 Santa Clara County’s Rebuttal July 6, 2004 Page 4 of 4 Conclusion FIRST 5 Santa Clara County is committed to fostering innovative programs that address the diverse needs of our community. Those needs are identified through civic engagement efforts and collaborative processes that provide the public with many opportunities for input into program design and grantee selection. In an effort to insure increased public awareness, FIRST 5 Santa Clara County will provide more detailed information in the documents presented to the Committees and Commission. Sincerely, (Signed by: Mary Jane Smith for Frederick J. Ferrer) (Signed by: Mary Jane Smith for Jolene Smith) Frederick J. Ferrer Jolene Smith Chairperson Interim Executive Director 8888 California State Auditor Report 2003-123 California State Auditor Report 2003-123 8899 COMMENTS California State Auditor’s Comments on the Response From First 5 Santa Clara County To provide clarity and perspective, we are commenting on First 5 Santa Clara’s response to our audit report. The numbers correspond with the numbers we have placed in the response from First 5 Santa Clara. 1 We do understand that First 5 Santa Clara sometimes uses a collaborative process. In fact, on page 20 we describe the collaborative process used by First 5 Santa Clara to participate in the state commission’s School Readiness Initiative targeting the county’s lowest-performing school districts. 2 As we clearly state on pages 40 and 41 of the report, we used a working definition for administrative costs to provide comparability among the county commissions we reviewed. We recognize there are other valid definitions. 3 Our statement is accurate and is based on a statement made by the chairperson for First 5 Santa Clara’s commission in which he described First 5 Santa Clara’s first contact with a representative from the Children’s Discovery Museum (museum). In his statement, the chairperson explained that when the museum needed funding for a new endeavor, First 5 Santa Clara was one of the organizations the museum’s representative approached. According to First 5 Santa Clara’s policies, this would qualify as an unsolicited proposal. However, the $1 million that was allocated to the museum exceeded First 5 Santa Clara’s policy limiting such unsolicited proposals to $15,000 per grantee. 4 First 5 Santa Clara is missing our point. As we discuss on page 24, when publicly approving this award for services relating to an early screening, prevention, diagnostic and treatment center, First 5 Santa Clara described it as a competitive award, yet made the funding allocation without competition. 5 Again, First 5 Santa Clara is missing the point. We do not dispute First 5 Santa Clara’s belief that the service provider was best qualified to be the service provider for the migrant camp, and we provide the county commission’s perspective on page 24. 8888 California State Auditor Report 2003-123 California State Auditor Report 2003-123 8899 However, when identifying the service provider at a public meeting, First 5 Santa Clara did not disclose the noncompetitive nature of the funding allocation. 9900 California State Auditor Report 2003-123 California State Auditor Report 2003-123 9911 cc: Members of the Legislature Office of the Lieutenant Governor Milton Marks Commission on California State Government Organization and Economy Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press 9900 California State Auditor Report 2003-123 California State Auditor Report 2003-123 9911