Summary

rotiduA etatS ainrofilaC S T I D U A E T A T S F O U A E R U B California K-12 High-Speed Network: The Network Architecture Is Sound, but Opportunities Exist to Increase Its Use January 2006 2005-116 The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TTY (916) 445-0033 OR This report is also available on the World Wide Web http://www.bsa.ca.gov The California State Auditor is pleased to announce the availability of an on-line subscription service. For information on how to subscribe, please contact the Information Technology Unit at (916) 445-0255, ext. 456, or visit our Web site at www.bsa.ca.gov Alternate format reports available upon request. Permission is granted to reproduce reports. C S A ALIFORNIA TATE UDITOR ELAINEM.HOWLE STEVENM.HENDRICKSON STATEAUDITOR CHIEFDEPUTYSTATEAUDITOR January 31, 2006 2005-116 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the Bureau of State Audits presents its audit report concerning our review of the California K-12 High-Speed Network (High-Speed Network) and whether or not the State is efficiently using its resources by supporting it. This report concludes that the State most likely spent less on the building and operation of the High-Speed Network by expanding the existing infrastructure used by the University of California (UC) and other higher education institutions than it would have spent for a separate network with comparable services. Furthermore, our technical consultant found no compelling technical or financial reason to abandon the existing High-Speed Network. However, although the Legislature shifted control of this project from UC to the California Department of Education, which selected the Imperial County Office of Education (ICOE) to act as lead agency, it still has not enacted legislation that clearly prescribes the goals to be accomplished with appropriations made to the project. Moreover, ICOE is still working to develop a process to measure the success of the network. From fiscal year 2000–01 through 2003–04, the Legislature appropriated more than $93 million to the High-Speed Network project. According to UC, it was directed by the former administration to use the Corporation for Education Network Initiatives in California (CENIC), a nonprofit, to implement the project. Under its agreement with CENIC, UC made advance payments because CENIC did not have the funds necessary to provide the services the former administration wanted. Although CENIC returned $10.8 million in unexpended funds to UC, as of June 30, 2005, CENIC still held $13.6 million in High-Speed Network funds. In fiscal year 2005–06, CENIC expects to receive an additional $3.6 million related to telecommunication discounts. Because the project was not funded in fiscal year 2005–06, some of these funds are being used to keep the network operating. Finally, opportunities exist for ICOE to strengthen its agreement with CENIC to better protect the State’s interest, such as including a provision to ensure the State’s ownership of tangible, nonshared assets. Respectfully submitted, ELAINE M. HOWLE State Auditor BUREAUOFSTATEAUDITS 555CapitolMall,Suite300,Sacramento,California95814 Telephone:(916)445-0255Fax:(916)327-0019 www.bsa.ca.gov/bsa TABLE OF CONTENTS Summary 1 Introduction 5 Chapter 1 From the Beginning, State Law Has Provided Limited Guidance and Oversight for the California K-12 High-Speed Network Project 15 Recommendations 31 Chapter 2 Some Issues Have Arisen With CENIC’s Fees and Its Use of High-Speed Network Funds, and Its Accumulated Surpluses Are Decreasing 33 Recommendations 49 Chapter 3 CENIC’s Network Architecture Is Sound, but It Is Difficult to Determine If the High-Speed Network Is Being Used as Originally Intended 51 Recommendations 70 Chapter 4 No Technical or Financial Reasons to Abandon the Existing High-Speed Network Are Apparent 73 Appendix A Glossary of Terms 87 Appendix B Key Service-Level Agreement Terms and Conditions That the California Department of Education Should Require Lead Agencies to Include in Contracts Relating to the California K-12 High-Speed Network Project 89 Appendix C Results of the Analysis of the California K-12 High-Speed Network Usage Data 93 Responses to the Audit University of California 103 California Department of Education 105 Corporation for Education Network Initiatives in California 115 California State Auditor’s Comments on the Response From the Corporation for Education Network Initiatives in California 117 SUMMARY RESULTS IN BRIEF This audit supports the basic conclusion that the State most Audit Highlights . . . likely spent less on the building and operation of the California K-12 High-Speed Network (High-Speed Network)1 Our review of the California by expanding the existing infrastructure used by the University K‑12 High‑Speed Network of California (UC) and other higher education institutions than it (High‑Speed Network) found that: would have spent for a separate network with comparable services. It is also clear that, based on a study conducted by our technical  The State most likely consultant in 2005, the current High-Speed Network provides spent less on the building ample bandwidth to support current applications used by the and operation of the High‑Speed Network by K-12 education community, but it is not overbuilt. Furthermore, expanding the existing our technical consultant found no compelling technical or infrastructure used by the financial reason to abandon the existing High-Speed Network. University of California and other higher education institutions The High-Speed Network connects the vast majority of than it would have spent kindergarten through 12th grade (K-12) schools, school districts, for a separate network and county offices of education statewide to each other, to with comparable services. California’s universities and community colleges, and to the  A study conducted by Internet. During fiscal years 2000–01 through 2003–04, the our technical consultant Legislature appropriated more than $93 million to UC for the in 2005 found that the High‑Speed Network has High-Speed Network. UC then contracted with the Corporation adequate bandwidth for for Education Network Initiatives in California (CENIC), a potential growth but is not nonprofit corporation that it helped create, to implement overbuilt. Furthermore, our the project. In fiscal year 2004–05, the Legislature switched technical consultant found no compelling technical the funding over to the California Department of Education or financial reason to (Education), appropriating $21 million in that year for the abandon the existing project. Education then selected the Imperial County Office High‑Speed Network. of Education (ICOE) to act as lead agency for the High-Speed  Because of the lack of Network, and ICOE also contracted with CENIC. For fiscal year specific performance 2005–06 the Legislature did not appropriate any additional measurements in funding to the High-Speed Network. state law and because the Imperial County Office of Education The Legislature provided no specific goals to UC in spending the (ICOE), which currently appropriations for the High-Speed Network project during fiscal administers the project years 2000–01 through 2003–04. Because both UC and CENIC is in the early stages of developing a suitable stated that they intended to enter into an agreement that was a plan for measuring the contract for services and not to acquire tangible equipment, UC success of the High‑Speed Network, it is difficult to determine whether the 1 The California K-12 High-Speed Network was originally named the Digital California Project: K-12 Statewide Network when it was started in fiscal year 2000–01. In fiscal year 2004–05, network accomplishes the when stewardship of the network was given to the Imperial County Office of Education, the Legislature’s goals. aspects of the network applicable to K-12 participants were then titled the K12HSN. We call it the High-Speed Network throughout this report. continued on next page . . . California State Auditor Report 2005-6   As of June 30, 2005, the did not include in its agreement with CENIC certain provisions. Corporation for Education For example, the agreement between UC and CENIC did Network Initiatives in not contain provisions to address the ownership of tangible, California (CENIC) the nonshared assets, such as the equipment located at the county nonprofit that built and currently operates the offices of education and school districts. Additionally, UC did network, held $13.6 million not include a provision to limit the use of the interest earned in High‑Speed Network on state appropriations for the High-Speed Network. Because Funds and it expects to receive an additional CENIC views its agreement with UC as a contract for services, $3.6 million related it considers the $1.5 million in interest earned on these funds to telecommunication available to use for its other activities. discounts in fiscal year 2005–06. These funds are being used to keep Although the Legislature shifted control of this project from UC the network operating in to Education and ultimately to ICOE, it still has not enacted fiscal year 2005–06 or are legislation that clearly prescribes the goals to be accomplished held for future equipment with these funds. Moreover, ICOE entered into agreements with replacement. CENIC under terms that were substantially similar to those  Opportunities exist for contained in UC’s agreement. Specifically, ICOE’s agreements ICOE to strengthen its continue to lack detailed service-level agreements, which would agreements with CENIC to better protect the State’s state the specific level of service CENIC is required to provide. interests. Specifically, its Additionally, its agreements do not ensure that ICOE retains agreements lack detailed ownership of tangible, nonshared assets, or that interest earned service‑level agreements, on advance payments that it makes to CENIC or funds that do not ensure that it retains ownership of CENIC holds on ICOE’s behalf accrue to the benefit of the tangible nonshared High-Speed Network. As of June 30, 2005, the amount of funds assets, and do not ensure available for the High-Speed Network, according to CENIC’s that interest earned on advance payments made accounting records, was $13.6 million. In addition, in fiscal year to CENIC or funds held by 2005–06, CENIC expects to receive an additional $3.6 million CENIC on its behalf accrue related to telecommunication discounts. to the benefit of the High‑ speed Network. The variability in the level of state funding for the High-Speed  Our legal consultant Network project has negatively affected the efforts of the ICOE reviewed the expenditure to expand network use in the K-12 education community. of funds by CENIC for the High‑Speed Network and Specifically, ICOE has been unable to fund its Advancing found that CENIC did not Network Uses Grant Program. Finally, although it states develop or acquire any that the 58 county offices of education, 887 school districts, assets that would have and 7,039 schools are currently hooked up to the High-Speed been eligible for protection under patent, copyright, or Network, ICOE’s and CENIC’s success in promoting network trade secret law. use is impossible to measure because neither entity has set up a process to do so. RECOMMENDATIONS To ensure that the High-Speed Network meets its expectations, the Legislature should consider enacting legislation that prescribes the specific goals and outcomes it wants from the High-Speed Network project. 22 California State Auditor Report 2005-6 To ensure that the High-Speed Network is appropriately managed, Education should ensure that ICOE includes the appropriate service-level agreements in its ongoing contracts with CENIC and other service providers for the High-Speed Network project. To ensure adequate protection of the State’s interest in tangible, nonshared assets, Education should direct ICOE to transfer ownership of these types of assets to the State. To ensure that the interest earned on advance payments made to CENIC, or funds that CENIC holds on ICOE’s behalf are used to benefit the High-Speed Network, Education should direct ICOE to amend its agreement with CENIC to stipulate the allowable use of the interest earned. Finally, Education should ensure that ICOE develops a method to measure the success of the High-Speed Network. AGENCY COMMENTS UC indicated that it is pleased with the overall conclusions of our report. Education and ICOE stated that they look forward to working with the other segment partners and CENIC to fully address the recommendations of the State Auditor. CENIC believes that the report provides useful information that highlights its value to the educational institutions it serves and that the report supports its belief that it can serve the unique needs of the education community more cost effectively than any other public or private organization. n California State Auditor Report 2005-6  Blank page inserted for reproduction purposes only.  California State Auditor Report 2005-6 INTRODUCTION BACKGROUND Until 2000, California’s kindergarten through 12th grade (K-12) schools, school districts, and county offices of education were on their own in their efforts to gain access to instructional, professional development, and data management resources from education networks and from the Internet. Beginning in fiscal year 2000–01, a cohesive effort was made to connect schools statewide. As a result of this effort, the California K-12 High-Speed Network (High-Speed Network)2 now connects the vast majority of K-12 schools, districts, and county offices of education statewide to the California Research and Education Network backbone,3 called CalREN. This backbone allows K-12 participants to connect to each other and to California’s universities and community colleges. Further, the CalREN backbone is also connected to various Internet service providers that provide access to the commodity Internet, the national Internet2 and various peer networks. Therefore, K-12 users also have access to these resources.4 The connection speed provided by the High-Speed Network is much faster than dial-up speeds and permits advanced services, such as videoconferencing, that are not possible when schools interconnect through the commodity Internet. Between fiscal years 2000–01 and 2003–04, the University of California (UC) received more than $93 million in state appropriations to expand Internet connectivity and its network infrastructure to K-12 schools and county offices of education. To assist with this task, UC contracted with a nonprofit public benefit corporation— the Corporation for Education Network Initiatives in California (CENIC)—that it created with other institutions of higher education. Specifically, UC, California State University (CSU), the California Institute of Technology, Stanford University, and the University of Southern California joined together in the late 1990s to form CENIC with the goal of achieving cost-effective, cohesive, 2 The California K-12 High-Speed Network was originally named the Digital California Project: K-12 Statewide Network when it was started in fiscal year 2000–01. In fiscal year 2004–05, when stewardship of the network was given to the Imperial County Office of Education, the aspects of the network applicable to K-12 participants were then titled the K12HSN. We call it the High-Speed Network throughout this report. 3 Backbone is defined, with other technical terms, in the glossary located in Appendix A. 4 Internet2 is a registered trademark of the University Corporation for Advanced Internet Development. California State Auditor Report 2005-6 55 advanced communications services. CENIC is governed by a board of directors (board) that is made up primarily of members that its charter associates appoint. Several committees, such as a technical advisory committee that helps in the planning and design of CENIC networks and technologies and a business advisory committee that reviews and proposes new rates and rate structures, advise the board on various issues. In fiscal year 2004–05, the Legislature transferred state oversight of the network to the California Department of Education (Education). It directed Education to select a lead county office of education to administer the High-Speed Network. In September 2004, Education, together with representatives from other entities, including the Legislative Analyst’s Office, the Department of Finance, and the Department of General Services, selected a consortium consisting of the offices of education for Imperial, Butte, and Mendocino counties and School Services of California, Inc., and led by the Imperial County Office Education (ICOE) to serve as the lead agency for the High-Speed Network. In December 2004, ICOE entered into an agreement with CENIC to continue to provide networking services to the K-12 education community. In August 2005, CENIC amended its bylaws to give California’s K-12 education community the same rights and privileges as it founding members. Specifically, CENIC gave the Imperial County superintendent of schools the authority to appoint three members to represent the K-12 education community’s interests on CENIC’s board. HISTORY OF THE HIGH-SPEED NETWORK In the late 1990s the State’s public colleges and universities were connected through two networks: CENIC’s CalREN-2, which connected California’s research universities including the UC campuses, and CSU’s and California Community Colleges’ 4CNet. CalREN-2 connected 36 universities throughout California to each other, to the commodity Internet, and to Internet2, which is a national initiative sponsored by the University Corporation for Advanced Internet Development to connect higher education institutions to each other using its Abilene network backbone. 4CNet was a high-speed wide area network connecting the respective campuses of CSU and the community colleges. CalREN-2 and 4CNet were connected at multiple locations and shared some circuits with each other, in effect creating a large statewide network for higher education. 66 California State Auditor Report 2005-6 In May 1999, Cisco Systems, Inc. (Cisco) wrote a paper entitled The California Educational Initiative, which brought forth the idea of connecting the State’s K-12 education community and its higher education institutions.5 CENIC, in collaboration with Cisco and other partners, developed a proposal to implement the Digital California Project: K-12 Statewide Network (project), which it presented to the former administration in early 2000. The project was conceived to extend CENIC’s CalREN-2 and CSU’s 4CNet into each of California’s 58 counties and had three specific objectives: (1) to strengthen the relationship between universities and colleges and K-12 schools in the areas of teacher preparation, professional development, curriculum development, student outreach, and information resource sharing; (2) to develop an integrated high-speed statewide network to expand K-12 schools’ connectivity to each other, to California’s universities and colleges, and to the commodity Internet; and (3) to identify and make accessible an array of learning content and information resources to K-12 teachers and students. The Budget Act of 2000 appropriated $32 million to UC to expand Internet connectivity and network infrastructure to the K-12 education community. In total, between fiscal years 2000–01 through 2004–05, the Legislature appropriated more than $100 million for this purpose. For fiscal year 2005–06 the Legislature did not appropriate any additional funding to the High-Speed Network. CENIC established a number of committees to assist with the development and management of the project. The program steering committee (steering committee) played a pivotal role in implementing the project. The steering committee was chartered in November 2000, and its responsibilities included coordinating applications to facilitate the use of professional development materials for teachers and multimedia-based curricula for K-12 students. Additionally, the steering committee was responsible for managing the financial aspects of the project. The steering committee’s members included representatives from entities such as multiple county offices of education, UC, and the former administration. UC also formed its own committee, the Digital California Project oversight committee, which was made up of several senior managers at UC. This group met regularly between January 2000 and December 2004. According to UC, CENIC managers were often present at these meetings during which policy issues were discussed and CENIC reports and work plans were reviewed. 5 Cisco and Cisco Systems are registered word marks of Cisco Technology Inc.. California State Auditor Report 2005-6  Additionally, CENIC’s network planning liaison team, which consisted primarily of representatives from the K-12 education community, provided ongoing leadership for the design and operation of the network. Further, CENIC’s video services oversight committee, which was a consortium of representatives from the K-12 education community, community colleges, CSU, and UC, was responsible for integrating a video over Internet protocol infrastructure on the network. Figure 1 shows a time line of significant events relating to the project. FIGURE  Time Line of Events November 2000 CENIC establishes a program steering committee to formulate and oversee the execution of specific strategies and tactics involved in planning and July 2004 implementing the project. The Legislature shifts the funding for the project from UC to the California May 1999 Department of Education and requires that a Cisco Systems, Inc. completes its paper lead county office of education be selected entitledThe California Educational Initiative. to administer the High-Speed Network project. July 1997 June 2004 Corporation for Education California State University’s and Network Initiatives in California California Community Colleges’ network, (CENIC) incorporates. 4CNet, is integrated into CalREN. 1997 1998 1999 2000 2001 2002 2003 2004 Early 2000 October 1998 CENIC presents the Digital January 2003 California Research and California: K–12 Statewide CENIC’s second-generation Education Network Network proposal to network backbone, now named (CalREN-2) begins service. the former administration. CalREN, is deployed. July 2000 September 2004 $32 million appropriated to the University of Education selected a consortium led by California (UC) to expand Internet connectivity and the Imperial County Office of Education (ICOE) to network infrastructure to the K–12 community. serve as lead agency of the High-Speed Network. September 2000 December 2004 CENIC signs an agreement with UC to provide CENIC enters into an agreement with ICOE to services for the High-Speed Network project. continue to provide K–12 networking services.  California State Auditor Report 2005-6 NETWORK ARCHITECTURE The initial network design, as outlined in a June 2000 project plan, was to develop and extend the architecture and infrastructure of the CalREN-2 and 4CNet networks into all 58 counties, thus enabling the K-12 schools, districts, and county offices of education to connect through a statewide network infrastructure. The proposal called for a point-to-point architecture that included at least 25 hub sites located across the State and as many as 200 county-based node sites. Each node site would have a single circuit that connected it to a hub on the backbone. Figure 2 illustrates this initial network architecture. FIGURE 2 Initial Concept for the Network Architecture Concept, Using a Point-to-Point Design CalREN County or 4CNet School District Backbone DCP node site 4 5 M b p s County or School District Cache DCP node 45Mbps server site ps b M 5 4 County or School District CalREN DCP node site County or 4CNet School District Backbone DCP node site California State Auditor Report 2005-6  spbM 54 To county schools To county schools Hub Site To county schools To county schools Source: Digital California Project: K-12 Statewide Network (DCP) Plan, dated June 30, 2000. By February 2001, according to CENIC, it elected to modify the network design to make it more reliable and to address concerns raised by the K-12 education community. Although the revised design still connects nodes to the hubs on the backbone, CENIC switched from a point-to-point architecture to a self-healing ring architecture. The point-to-point approach, by its nature, is unreliable in comparison to the self-healing ring architecture. First, there is only one route between a node and a hub on the backbone, and any failure of electronics or circuit on that route results in a service outage for that hub site. Second, in the point-to-point architecture, one hub typically serves as the center of the network, and all network traffic is routed through it. Therefore, a failure at a central hub site results in a network outage for all node sites. However, with a self-healing ring network architecture, data moves successfully around the network, even when there is an outage at some point. Data will typically travel in one primary direction around the network. However, with any outage, whether caused by an actual break in the circuit or fiber-optic cable or a failure of the network electronics at any hub site, the data will travel along the alternative path on the ring, avoiding the outage. This rerouting or self-healing function is transparent to the users. In the revised design, with very few exceptions, a minimum of two routes exist between the node sites and the backbone and between any two hub sites on the backbone. Figure 3 depicts two methods of connection: one in which a node site connects to two hub sites on the backbone, and one in which two node sites connect to each other and each node site has one connection to the backbone. FIGURE  Self-Healing Ring Network Architecture Backbone Hub A Hub B Node 1 B e a Node 3 o n c k b b k o c n a B e Node 2 Hub C Hub D Backbone Source: Bureau of State Audits’ technical consultant. 00 California State Auditor Report 2005-6 The new design also called for fewer hubs and nodes and a greater number of circuits. CENIC initially planned to install 12 hubs and 71 nodes and to add up to 60 additional secondary nodes later. It scaled back the second round of node installations due to budget concerns at the end of fiscal year 2001–02. SCOPE AND METHODOLOGY The Joint Legislative Audit Committee (audit committee) requested the Bureau of State Audits (bureau) to determine whether the State is efficiently using its resources by supporting the maintenance of the High-Speed Network. Specifically, the audit committee asked the bureau to determine the roles and responsibilities of the various entities involved since the inception of the High-Speed Network project, to identify the network’s funding sources and determine whether there are any limitations or restrictions on the use of this funding or on the disposition of unused funds, and to review the methods used to allocate the costs of the High-Speed Network to determine if they are reasonable. In addition, the audit committee instructed the bureau to review the cost, usage, and, to the extent possible, benefits of the High-Speed Network and to determine whether these costs and benefits are comparable to those of other Internet service providers. The audit committee also directed the bureau to examine any information the State, consortium, or other entity has used to determine whether the benefits of the network outweigh its costs. Further, the bureau was asked to evaluate the reasonableness of any options or plans the State or consortium of county offices of education considered to maximize the use of the High-Speed Network. Moreover, the audit committee requested that the bureau determine the ownership rights to purchases made or services related to the High-Speed Network, including but not limited to intellectual property rights and how the State may exercise those rights. Finally, the bureau was asked to review and evaluate the laws, rules, and regulations significant to the objectives stated above. To assist in conducting this audit, the bureau engaged the services of two consultants: a firm with broad telecommunications experience, including network design and evaluation (technical consultant), and a legal consultant with a background in engineering (legal consultant). These consultants performed technical reviews of our conclusions, provided technical or legal advice, and performed audit procedures as instructed by the bureau to achieve specific audit objectives. California State Auditor Report 2005-6  To ascertain the roles and responsibilities of the various entities involved since the inception of the High-Speed Network project, we reviewed relevant documentation from Education, ICOE, UC, and CENIC. Our review included contracts, policies and procedures, and project planning documents. We also reviewed membership in CENIC’s committees to identify other entities that provided input into the High-Speed Network Project. To identify the High-Speed Network’s funding sources and any applicable limitations or restrictions on the use or disposition of funds, we reviewed CENIC’s and ICOE’s financial records to identify the revenue they received from each source. We reviewed relevant budget language, laws, and regulations to assess any limitations on how the funds should have been spent. In addition, we interviewed representatives at ICOE, CENIC, and UC to determine their understanding related to limitations on the funds. Our technical consultant also reviewed CENIC’s process for applying for a federal telecommunications discount program for most schools and libraries, called E-rate, to ensure that K-12 schools are taking steps to maximize their receipt of federal assistance. Similarly, we reviewed CENIC’s process for applying for the California Teleconnect Fund, a state telecommunications discount program also for schools and libraries. To determine if the methods used in allocating costs were reasonable, we reviewed CENIC’s financial records, supporting documentation, and board minutes. We also interviewed managers at CENIC and ICOE. We found that CENIC recovers costs for aspects of the network that are shared by all of its participants by establishing fees. We reviewed the budget and underlying documentation used by CENIC to calculate its fees and spoke with representatives from CENIC as well as ICOE to gain the K-12 perspective on the equity of the fee structure. To obtain an understanding of the costs associated with the High-Speed Network, we reviewed relevant budget and financial information. We also performed testing of CENIC’s accounting records to determine the accuracy and completeness of CENIC’s information. To determine whether we should rely on CENIC’s audited financial statements for, certain information, we reviewed CENIC’s independent auditors most recent peer reviews. To examine usage of the network, we obtained network traffic data from ICOE. These data include all traffic traveling through High-Speed Network node sites. We tested these data and found 22 California State Auditor Report 2005-6 them to be reliable for the purposes of our audit. Using the data, our technical consultant analyzed network usage by node site and interpreted the data as described in Chapter 3 and Appendix C. To determine if any information was available to determine whether the benefits of the network outweigh the costs, we conducted interviews with representatives at Education, ICOE, and CENIC. Our technical consultant also reviewed the prices obtained by CENIC to construct the High-Speed Network to determine whether K-12 participation in the network was cost effective. Specifically, our technical consultant compared CENIC’s pricing for network components and circuitry to other available pricing information. To understand whether the State has ownership rights to the High-Speed Network and to determine how it could exercise any such rights, we consulted with our legal counsel regarding the legislative intent of the program. Our legal counsel also discussed with UC and CENIC their intent when entering into their agreement. Finally, our legal consultant reviewed the terms of the agreement between UC and CENIC as well as the agreements between ICOE and CENIC to determine if these contracts contained the appropriate language. To determine whether the State or the consortium have considered options or made plans for maximizing the use of the High-Speed Network, we reviewed relevant planning documents prepared by Education, ICOE, and CENIC. In addition, we conducted numerous interviews with representatives at each of these agencies. Because our technical consultant’s analysis of network traffic indicated that the network was not overbuilt, we focused on how CENIC, ICOE, and Education have worked to increase the number of applications available on the network. Finally, we evaluated the steps taken by CENIC and ICOE to measure the success of the High-Speed Network. To evaluate the alternatives to the High-Speed Network presented in MGT of America, Inc.’s (MGT) report titled Performance Evaluation of the K-12 High-Speed Network, our technical consultant evaluated MGT’s statements and recommendations. Our technical consultant’s conclusions can be found in Chapter 4. n California State Auditor Report 2005-6  Blank page inserted for reproduction purposes only.  California State Auditor Report 2005-6 CHAPTER 1 From the Beginning, State Law Has Provided Limited Guidance and Oversight for the California K-12 High-Speed Network Project SUMMARY During fiscal years 2000–01 through 2003–04, when state funds were appropriated to the University of California (UC) in the state budget to develop and implement the California K-12 High-Speed Network (High-Speed Network)6 by expanding the existing infrastructure used by UC and other higher education institutions, there was no specific guidance in the budget acts or elsewhere in state law regarding the specific goals UC was to accomplish using these funds. Due to the lack of clear legislative direction, UC had considerable discretion in implementing the High-Speed Network project. According to UC, it was directed by the former administration to contract with the Corporation for Education Network Initiatives in California (CENIC) to carry out the project. It passed the state appropriations on to CENIC in the form of quarterly prepayments. UC’s relationship with CENIC ultimately resulted in a technically reliable network that connects most kindergarten through 12th grade (K-12) schools, school districts, and county offices of education to each other and to California’s universities and community colleges, and that provides low-cost Internet connectivity. Although UC exercised oversight over this project through its participation on CENIC’s board of directors (board), its review of CENIC’s reports, participation on various steering committees, and other activities, we believe that it could have strengthened its oversight if the agreement between UC and CENIC had contained terms that provided greater protection to the State’s interests. 6 The California K-12 High-Speed Network was originally named the Digital California Project: K-12 Statewide Network when it was started in fiscal year 2000–01. In fiscal year 2004–05, when stewardship of the network was given to the Imperial County Office of Education, the aspects of the network applicable to K-12 participants were then titled the K12HSN. We call it the High-Speed Network throughout this report. California State Auditor Report 2005-6 55 In 2004 the Legislature transferred control of the High-Speed Network project from UC to the California Department of Education (Education), and required Education to select a lead agency to oversee the implementation of the project. The lead agency—the Imperial County Office of Education (ICOE)—has continued to use CENIC as the provider of services for the High-Speed Network. As this project moves forward and the Legislature appropriates additional funds for it, ICOE should seek to strengthen the terms of the agreement between it and CENIC to better protect the State’s interests. A MODEL APPROACH TO A SUCCESSFUL PROJECT BEGINS BY CLEARLY IDENTIFYING THE NEEDS OF THE K-2 COMMUNITY AND BUILDING THE NETWORK BASED ON THOSE NEEDS Our legal consultant has advised us that the generally accepted best practices in project management for a network project School districts, county of this type generally call for a five-step process that includes offices of education, the following: (1) determining the specific needs of the user; individual schools, (2) determining the services or software programs or technology and charter schools required to support those needs; (3) designing the engineering are the intended users aspects of the network to support those needs; (4) entering of the High‑Speed into appropriate service-level agreements between the provider Network services. of services and the recipient of the services to ensure that the desired level of service will be provided; and (5) continually monitoring the project and measuring performance to determine whether the user’s needs have been met. School districts, county offices of education, individual schools, and charter schools are the intended users of the High-Speed Network services. Thus, the design of a network for those users should begin by clearly identifying their educational and administrative needs. For example, the specific needs of K-12 schools might include the ability for all schools in the State to simultaneously participate in live videoconferencing or the ability for students in a particular classroom to participate in, via the network, an educational activity being held at a school in a remote location. Once these various educational and administrative needs have been clearly identified and defined, it is possible to determine which types of software programs and technology best support those needs. For example, live videoconferencing requires high bandwidth and a high level of reliability, whereas the ability to do basic Internet research requires minimal bandwidth and a lower level of reliability. This would influence the specific design 66 California State Auditor Report 2005-6 and bandwidth of the network. In short, the overall design of the network should take into account the identified needs of the education community as a whole and should be specifically tailored to meet those needs. Once the user and service provider have agreed upon the user’s needs and how they will be met, the contractual agreement between the two parties becomes the formal binding agreement that ensures that the services will be provided. As we discuss in more detail later in this chapter, it is customary in the telecommunications industry to include service-level agreements in contracts for telecommunications services that formalize the parties’ understanding and obligations related to the services that will be provided and how performance will be evaluated. CENIC coordinated a variety of High-Speed Network services on behalf of school districts by contracting with various software, hardware, and telecommunications service providers. Finally, the generally accepted best practices for a project of this type call for continually measuring and monitoring performance, and the service-level agreements typically prescribe the criteria to use and the consequences if the service provider does not meet the stated requirements. STATE LAW DID NOT PRESCRIBE THE GOALS OF OR IMPOSE SPECIFIC REQUIREMENTS ON THE HIGH-SPEED NETWORK PROJECT When the Legislature appropriated more than $93 million to UC for the High-Speed Network between fiscal years 2000–01 and 2003–04, the budget control language that made those The budget control appropriations stated only that the purpose of the funding was language that provided for “expanding Internet connectivity and network infrastructure appropriations for the for K-12.” This budget control language did not impose any High‑Speed Network more specific requirements or controls on the expenditure of did not impose specific these funds, nor did the Legislature enact legislation to further requirements or controls define the parameters of this project or what was meant by on the expenditure of “Internet connectivity and network infrastructure for K-12.” these funds. Therefore, it is difficult to determine if the Legislature got what it sought in appropriating the funds. When the Legislature makes an appropriation of funds for a new program or project in the annual budget act, often a “trailer bill” or other enabling legislation accompanies the appropriation and prescribes in some detail how those funds are to be expended. Although the Legislature has considerable latitude in deciding how much control to impose on the expenditure of public California State Auditor Report 2005-6  funds, enabling legislation may include a specific description of the goals and outcomes the use of the funds will achieve, the specific powers and duties the responsible agency possesses with respect to implementing the program or project, specific time frames for accomplishing the goals, and in some cases a requirement to submit progress reports to the Legislature, as well as any other specific requirements the Legislature deems appropriate. Our legal counsel has advised us that the Legislature clearly has the authority to enact legislation that prescribes the specific goals Specific statutory goals it sought to accomplish using these funds. Specific statutory will better ensure that the goals would better ensure that the K-12 education community K‑12 education community receives the intended benefits of this project. As we describe receives the intended more fully in Chapter 3 of this report, although most schools benefits of this project. are connected to one another and to the State’s universities and community colleges, and although the network appears to be adequately built, the lack of specific performance measurements in state law makes it difficult to determine whether the network accomplishes the Legislature’s goals. UC’S EXPENDITURE OF FUNDS FOR THE HIGH-SPEED NETWORK PROJECT WERE NOT SUBJECT TO CERTAIN CONTROLS THAT APPLY TO MOST STATE AGENCIES As a constitutionally created entity, UC is largely independent of executive branch control, and is not subject to the same budgetary and other oversight controls that apply when appropriations are made to a typical state agency. For example, UC is not subject to the provisions of the California Public Contract Code that govern competitive bidding or contract approval. In addition, UC is expressly exempt from the various provisions of state law relating to the centralized accounting of state funds to ensure the adequate protection and investment of state money. These provisions generally require state agencies to deposit all state money directly in the State Treasury to ensure that funds are accounted for through a centralized system and that they earn interest at the same rate as money in the State Treasury. Finally, state policy directs state agencies not to make advance payments for services that are under contract but have not yet been provided; however, this policy does not apply to UC. Although it is not generally subject to various state laws related to the control of public funds or public contracting, UC has adopted its own internal policies related to contracting for services. Those policies impose various requirements that are similar to those state agencies must adhere to under the  California State Auditor Report 2005-6 Public Contract Code. For example, UC’s policies require the solicitation of multiple price quotes before awarding a contract involving an expenditure of more than $50,000; prompt payment for services after they are received, rather than in advance of service; and legal review by its Office of the General Counsel as to the form of contracts that are unusually complex or are for high-value items or services. UC believes its policies relating to contracting for services did not directly apply to the agreement it formed with CENIC for the purposes of the High-Speed Network. Representatives in UC’s Office of the President have indicated that the circumstances leading to development of the agreement between CENIC Based on the somewhat and UC in 2000 were unique. Because UC was directed by the unique nature of its former administration to use CENIC to implement this project, agreement with CENIC, UC considered the policies that would have otherwise required representatives of UC competitive bidding inapplicable. We acknowledge the merit in also indicated that they UC’s position, and agree that UC was justified in not following believe its policy that its usual practice of soliciting bids for a project of this type. generally disallows advance payments for Based on the somewhat unique nature of its agreement with services also did not CENIC, representatives of UC also indicated that they believe apply to the agreement. its policy that generally disallows advance payments for services also did not apply to the agreement. It advanced more than $93 million to CENIC between fiscal years 2000–01 and 2003–04. When we asked why it made advance payments under this agreement, UC indicated that an advance payment was necessary because CENIC did not have the funds required to provide the services the former administration wanted. According to UC, at the time of the initial appropriation, CENIC had annual revenue of approximately $6 million. The Legislature allocated $32 million for the first year of the project, expressing the intention to provide similar amounts for several years thereafter. Given the expenditures necessary to create the network and the small amount of CENIC’s own funds, UC considered advance payments necessary. However, even if these advance payments were necessary, UC could have implemented measures to ensure that CENIC would not accumulate cash balances, such as making fund distributions that more closely aligned with CENIC’s expenditures or, if CENIC did accumulate excessive cash balances, ensuring that the interest earned on the funds would accrue to the benefit of the K-12 education community. Between fiscal years 2000–01 and 2003–04, the cash balances created by UC’s advance payments earned approximately $1.5 million in interest. Each year, the interest earned on the High-Speed California State Auditor Report 2005-6  Network funds was recorded in an account designated solely for the High-Speed Network and then transferred to CENIC’s general operations account. The transfer of these funds to CENIC’s general operations account makes it difficult to identify those funds belonging to the network. According to CENIC’s president and chief operating officer, the interest earned on these advance payments should not be considered High-Speed Network funds. He stated that CENIC’s agreement with UC did not specify that the interest earnings could only be used for High-Speed Network purposes. He further stated that although the agreement between UC and CENIC specifies that CENIC set up and use a separate financial account for the High-Speed Network funds and not to use that account to hold or dispense any other funds, the contract does not address ownership or use of interest earned on the fund balances. UC and CENIC have indicated to us that they do not believe that the interest on this account was required to remain with the account, and that as long as that interest provided some direct or indirect benefit to the High-Speed Network by furthering CENIC’s mission, they consider the interest earned to have been used appropriately. Although the parties to the contract believe that the interest earned on these advanced funds was used appropriately, if UC had included a provision in its contract that required the If UC had included a interest on those funds to remain with the principal, this would provision in its contract have provided greater assurance that the interest would have that required the interest directly benefited the High-Speed Network project. on those funds to remain with the principal, this UC’s legal counsel also stated that it is clear to UC that the would have provided High-Speed Network funds were spent only on the High-Speed greater assurance that the Network, that the funds were separately maintained, that interest would have directly unused funds were returned, and that the accomplishments of benefited the High‑Speed the program were notable. However, as we discuss more fully Network project. in Chapter 2, despite the degree to which UC was involved in CENIC’s operations, we found that almost $7.2 million was transferred out of an account designated solely for the High-Speed Network into CENIC’s general operations account. UC EXERCISED ITS OVERSIGHT OVER THE HIGH-SPEED NETWORK PROJECT THROUGH ITS PARTICIPATION ON THE CENIC BOARD AND ON VARIOUS STEERING COMMITTEES UC has indicated that it was directed by the former administration to undertake the High-Speed Network project using CENIC as the provider of services. Further, UC stated that the 2200 California State Auditor Report 2005-6 former administration was impressed with the successful implementation of another project that UC undertook in collaboration with CENIC and believed that UC could achieve similar success in implementing a network that would serve the needs of the K-12 education community by expanding the existing infrastructure to those users. In September 2000, UC entered into an agreement with CENIC to implement the High-Speed Network. This agreement stated that its purpose was “to enter into a contract with CENIC to perform the work outlined in the project plan” and made that plan a part of the agreement. The agreement required CENIC to annually provide UC with a written project plan for the coming year as well as a written detailed report of the implementation and operations of the High- Speed Network services during the prior year. Three representatives of UC hold positions on CENIC’s board, and a representative of UC participated in a program steering committee (steering committee) that was formed by CENIC. The purpose of this steering committee was to, among other things, The overarching purpose of the High-Speed focus on the goals identified in the project plan. Network project is to enrich learning for all K-12 students and the teaching of all K-12 educators. Those goals are shown in the text box. At board meetings, CENIC provided periodic reviews of The project has three foci: the High-Speed Network, including projected Strengthening the existing program and and actual expenditures as well as program goals service relationships between the CENIC universities, community colleges and K-12 and achievements. In addition, UC relied on its schools in the areas of teacher preparation, employees’ participation on CENIC’s executive professional development, curriculum committee, program steering committee, audit development, student outreach, and information resource sharing; committee, and business advisory committee to influence the implementation of the project. Developing a comprehensive/integrated high speed statewide network infrastructure that will expand K-12 schools’ connectivity UC believes that the representation provided by to each other, to California higher education institutions, and to K-12 schools, universities, three UC representatives on CENIC’s 15-member and other organizations across the globe; board, its participation on the steering committee formed by CENIC, and its review of various reports Identifying and making accessible a rich array of learning content and information provided by CENIC gave UC adequate control over resources from throughout the State and the this project. However, while serving on CENIC’s world that K-12 teachers and students will find to be of value in the teaching-learning process. board, the UC representatives have a fiduciary duty to protect the interests of the corporation as a whole Source: Digital California Project: K-12 Statewide and to make decisions that best serve the overall Network Plan dated June 30, 2000. mission of the corporation. In doing so, they must take into account CENIC’s charge to implement the High-Speed Network project as well as any other projects CENIC is currently undertaking. California State Auditor Report 2005-6 22 Although we acknowledge that UC’s participation on the CENIC board and its involvement in other activities allowed it to assert UC’s direct oversight influence over how CENIC implemented this project, we believe and control over this that UC’s direct oversight and control over this project could project could have have been strengthened if the agreement between UC and CENIC been strengthened if its had contained terms that were more protective of the State’s agreement with CENIC interests. Specifically, the agreement should have contained terms had contained terms that that required a specific level of performance and that gave UC were more protective of greater direct control over the project. If the agreement contained the State’s interest. these terms, UC would have been in a stronger position to compel performance if CENIC had failed to deliver. The agreement did not specify the expected level of performance that UC required of CENIC and its subcontractors. The September 2000 agreement stated that UC’s purpose was “to enter into a contract with CENIC to perform the work outlined in the project plan,” and made that plan a part of the agreement. The agreement itself did not explicitly prescribe the specific work or deliverables that CENIC was to provide. Section 3 of the agreement, entitled “Work to Be Performed by CENIC,” generally described the work as providing connectivity for K-12 users to the CalREN-2 network and its advanced services, but did not define what those advanced services included. The project plan referenced in the agreement between UC and CENIC stated that CENIC would “focus on identifying applications and facilitating making them accessible on the network.” Although it is not unusual for an agreement to make the requirements contained in another document, such as the project plan, a part of that agreement, our legal consultant advised us that the project plan failed to contain the level of specificity typically needed to justify the expenditure of millions of dollars. Specifically, the project plan did not clearly identify the deliverables that would be provided under the agreement. As we described earlier, best practices call for services to be specified and for a mechanism to be in place to monitor and measure the delivery of that service. The provisions of the agreement related to work performed by subcontractors lacked specificity and did not prescribe a clear standard of performance against which to measure subcontractors. For example, the section of the agreement entitled “Work by Subcontractors” required CENIC to make “every reasonable effort to ensure that work or services provided by subcontractors meets any requirements specified in this agreement or required in the [incorporated project plan].” However, neither the agreement nor the project plan provided any specific, standard requirements 2222 California State Auditor Report 2005-6 for any deliverables against which to measure the performance of subcontractors. This lack of specificity failed to ensure that CENIC required an appropriate level of service from each vendor or service provider, and that it had the ability to enforce those requirements. In addition, the terms of the agreement related to “Quality of the Services Provided” required that “all services provided under this agreement shall be operated and supported in a professional and reliable manner.” Our legal consultant has advised that this level of definition is inadequate to ensure that the intended services meet any form of generally accepted performance standards for an agreement of this type. The agreement gave UC no clearly identified standard against which to measure CENIC’s performance or to which it could hold CENIC accountable. Our legal consultant also advised us that UC should have included terms and conditions in the agreement that required CENIC to determine the specific services to be performed on behalf of the K-12 public school system as well as the expected level of performance by all subcontractors, otherwise known as a service-level agreement. A service-level agreement is a formal, negotiated agreement that identifies the specific level of services to be provided, and that clarifies and defines the responsibilities a service provider owes to its Applications that survey respondents in the customers. In this context, UC would be considered K-12 education community would use given the customer, acting as an agent or advocate on additional bandwidth: behalf of the K-12 educational community, and CENIC would be considered the service provider. County District Responses Responses In August 2000, at the same time that it Streaming video 85% 72% was developing the network design, CENIC engaged a consultant to help plan the project Video on demand 85 62 by surveying the K-12 education community in Originate remote all 58 counties. CENIC used the February 2001 instruction 81 51 readiness assessment and gap analysis report containing the survey results to understand the Receive remote instruction 77 70 existing infrastructure in different counties, to help design the network, and to determine Use teleconferencing 91 70 where it should place hub and node sites. The survey included questions such as which types Source: SAIC, which is a registered service mark owned of applications the county offices of education by Science Application International Corporation. and school districts would use if they had unlimited bandwidth. The text box summarizes the answers to this question. The report included responses from 57 of the 58 counties and 807 of the expected 1,006 school districts. California State Auditor Report 2005-6 22 The readiness assessment and gap analysis report indicated that bandwidth requirements should drive the architecture of the network. CENIC’s consultant’s bandwidth calculations allowed for the types of applications that the K-12 education community might use for streaming video, video on demand, remote The results of CENIC’s instruction, video teleconferencing, and increased business/ consultant’s analysis were administration data. However, the results of the analysis were not incorporated as legally not incorporated as legally binding requirements into the binding requirements into agreement with UC or into a revised project plan as a definition the agreement with UC of the specific needs of the K-12 community. or into a revised project plan as a definition of As the project progressed, CENIC’s board made decisions about the the specific needs of the implementation of the project. CENIC engaged in ongoing efforts K‑12 community. to build a network suited to K-12 and to identify applications and facilitate their use on the network. It created an applications coordination team, which included representatives such as local education agencies, Education, and UC. Although the team was successful in identifying the types of network applications needed, such as videoconferencing, and making them available on the network, CENIC and UC did not modify their agreement to require that these applications be made available. Finally, the agreement should have contained provisions that assured greater direct control by UC over the project. Although the agreement required that CENIC provide for “UC’s approval at the beginning of each annual anniversary . . . a written project plan for the subsequent year and a written detailed report of the implementation and operations of the DCP [Digital California Project] services during the year just concluded,” our legal consultant has advised us that a more desirable approach would have been to have a more detailed clause outlining project management requirements in the agreement with CENIC. For example, the agreement should have established much more frequent reporting periods, included the opportunity for UC to conduct a preliminary review of the proposed and final project design, and provided for regularly scheduled project implementation reviews and updates. UC AND CENIC BELIEVE THE AGREEMENT TO DEVELOP THE HIGH-SPEED NETWORK IS A SERVICE AGREEMENT THAT INCLUDES NO PROVISION FOR STATE OWNERSHIP OF ASSETS The agreement between UC and CENIC did not contain provisions that stated who would own the physical assets, such as the hardware, that CENIC purchased using state appropriations. 22 California State Auditor Report 2005-6 Both UC and CENIC stated that they intended to enter into an agreement that was a contract for Forms of Intellectual Property services and not to acquire tangible equipment. Consequently, UC believes that it paid CENIC to Patent—A property right that is granted to provide certain services under the contract, which an eligible inventor by the U.S. Patent and Trademark Office. It allows the patent holder to included developing the network infrastructure exclude others from “making, using, offering and providing network connectivity, but that it for sale, or selling” the invention in the United States or importing the invention into the did not acquire ownership rights to any of the United States for a period of years. This would physical equipment CENIC bought to provide include the development of a new technology that service. Our legal counsel advised us that or process. the mutual understanding of the parties to a Copyright—A form of protection provided to contract would likely prevail in any dispute related the authors of original works of authorship, including literary, dramatic, musical, artistic, to the contract. and certain other intellectual works, both published and unpublished. A copyright gives the owner the exclusive right to reproduce the copyrighted work, to prepare derivative CENIC DID NOT DEVELOP SIGNIFICANT works, to distribute copies or phonograph INTELLECTUAL PROPERTY RIGHTS USING records of the copyrighted work, to perform HIGH-SPEED NETWORK FUNDS the copyrighted work publicly, or to display the copyrighted work publicly. Software may be protected under copyright law. In When public funds are used to develop and addition, multimedia content, which typically implement an information technology project, the includes multiple authors, may be protected under copyright law. use of those funds may result in the development of intellectual property. Intellectual property is a Trade secret—Information that is not broad term that refers generally to intangible assets generally known in the industry, to the public, or to others who can realize economic value developed as a result of intellectual endeavor that from its disclosure or use; has independent reflect specialized knowledge, ideas, or processes. economic value, whether actual or potential, Intellectual property may take a variety of forms, that derives from its secrecy; and is the subject of efforts that are reasonable under such as a patent, copyright, trademark, or trade the circumstances to maintain its secrecy. secret, which are described further in the text box. “Information” includes, among other things, formulas, patterns, compilations, programs, Each form is subject to different laws that allow devices, methods, techniques, and processes. the holder of that intellectual property right to In essence, trade secrets create value because protect against inappropriate use by others and they are known only to the developer of the information. that may put the holder of that right in a position of economic advantage over competitors. Our legal Trademark—A word, name, symbol, or device that is used in trade with goods to indicate the consultant reviewed the expenditure of funds by source of the goods and to distinguish them CENIC for the High-Speed Network and found from the goods of others. A “service mark” is that CENIC did not develop or acquire any assets the same as a trademark except that it identifies and distinguishes the source of a service rather that would have been eligible for protection under than a product. The terms “trademark” and patent, copyright, or trade secret law. However, “mark” are commonly used to refer to both trademarks and service marks. Trademark rights some of the phrasing or symbols used by CENIC may be used to prevent others from using a to describe and represent the High-Speed Network confusingly similar mark, but not to prevent could be eligible for protection as a trademark. others from making the same goods or from selling the same goods or services under a clearly different mark. Our legal consultant’s review did not reveal the development of any “inventions” potentially eligible for protection under federal patent law. California State Auditor Report 2005-6 2255 To the extent that CENIC developed some novel process for implementing the High-Speed Network, that process might have been eligible for protection under patent laws. However, because the funds CENIC expended were for the development of a physical network infrastructure and connections to that infrastructure in a way that was not technologically unique or novel, they did not result in the development of a new technology or process that could be patentable. Similarly, our legal consultant’s review showed that CENIC’s expenditures did not result in the development of any original works protected by copyright laws. As this project moves forward, if CENIC or another entity does develop and deliver educational content on the network, the State should take steps to protect its intellectual property rights. Additionally, CENIC indicated to us that in developing and operating the network, it used publicly available, “open source” software and software that was otherwise available from commercial vendors rather than developing new software to operate the network. Consequently, our legal counsel concluded that CENIC did not develop any copyrightable software. Our legal consultant’s review also showed that the expenditure of High-Speed Network funds did not support the development of any asset that could have been protected as a trade secret. Few state agencies report owning trade secrets. By definition, a state agency must derive actual or potential economic value from a trade secret based on its not being generally known to the public or to other entities that could obtain economic value from its disclosure or use. Because it was not within CENIC’s mission to develop products or processes that might have been potentially marketable, it is not Just as UC and other likely that it would have developed trade secrets. governmental institutions protect their “brands,” Finally, our legal consultant’s review showed that CENIC did the State could have not necessarily generate trademark assets. Based on the original protected the phrase documents promoting development of a statewide High-Speed “Digital California Network to benefit education, it appears that Cisco, CENIC, and Project.” Such protection the State jointly coined the phrase “Digital California Project.” would allow it to use Just as UC and other governmental institutions protect their the phrase without issue “brands,” the State could have protected the phrase “Digital in any future business California Project.” Such protection would allow it to use development activities. the phrase without issue in any future business development activities. In addition, such protection would allow the State of California to prevent any misuse of the phrase. 2266 California State Auditor Report 2005-6 THE LEGISLATURE SHIFTED CONTROL OF THE NETWORK TO THE CALIFORNIA DEPARTMENT OF EDUCATION, BUT STILL DID NOT PRESCRIBE SPECIFIC GOALS TO BE ACCOMPLISHED In the Budget Act of 2004, the Legislature effectively transferred the responsibility for managing the Internet connectivity and infrastructure for K-12 educational institutions from UC to Education. The budget act appropriated $21 million to Education and required it to provide a grant to a lead county office of education, selected on a competitive basis, to implement the network. Education selected ICOE as the lead agency responsible for administering the project, as well as providing for the continuation and growth of network services and other support activities. Although the Legislature shifted control of this project from UC to Education and ultimately to ICOE, it still has not enacted legislation that clearly prescribes the goals to be accomplished using these funds. It did, however, state its intent in the Budget Act of 2004 not to make additional funds available for the project until legislation is in place that imposes specific Until legislation is programmatic requirements. In the regular legislative session for enacted, Education fiscal year 2005–06, legislation was proposed that would create cannot be certain the a high-speed interconnectivity program for the K-12 education High‑Speed Network’s community, under the administration of the superintendent design and use are of public instruction in consultation with an advisory board. achieving the Legislature’s However, this proposed legislation was not enacted in 2005. desired outcomes. Until legislation is enacted, Education cannot be certain that the design and use of the High-Speed Network are achieving the Legislature’s desired outcomes. THE CURRENT AGREEMENT BETWEEN ICOE AND CENIC COULD BE STRENGTHENED TO BETTER PROTECT THE STATE’S INTERESTS After its selection as the lead agency in 2004, ICOE entered into an agreement with CENIC under terms that were substantially similar to UC’s agreement. ICOE and CENIC executed two separate agreements. The first was executed December 1, 2004, and the second was executed June 24, 2005, and became effective July 1, 2005, after the first agreement expired. According to a manager at ICOE, CENIC was selected because its role as the current network services provider meant that it already had the resources in place to maintain the California State Auditor Report 2005-6 22 network, thus providing ICOE with a smooth transition as the new administrator of the High-Speed Network without disrupting current services. Nevertheless, both agreements continue to lack service-level agreements. Additionally, the agreements fail to contain provisions that fully address the issue of the State’s ownership of assets and that require CENIC to limit the use of interest earned on advance payments it receives related to the High-Speed Network. ICOE’s Agreements With CENIC Lack Service-Level Agreements The agreements between ICOE and CENIC continue to lack detailed service-level agreements. As we discussed, earlier, a service-level agreement describes the specific level of service a vendor is required to provide and typically provides a penalty if that level is not provided. The first agreement contained a single reference to quality of service requirements (QoS). These requirements are a measure of performance that reflects a system’s transmission quality and service availability. In the agreement, CENIC agrees to configure and maintain QoS on all backbone circuits and equipment, in accordance with CENIC policies governing QoS. However, the agreement does not state what the measure of performance is in other references to QoS in the agreement, nor does it refer to service-level agreements. The second agreement also fails to include specifics about how to measure service-level performance. In this agreement, CENIC agrees to operate and support services in a professional and reliable manner in accordance with the service specifications developed by CENIC and approved by its members. However, the only additional reference, which relates to outages, is a simple discussion of how outages will be handled that does not require a specific level of service, nor does it impose a penalty for not providing that level of service. Service-level agreements are commonplace in the telecommunications industry and are designed primarily to improve Service‑level agreements the quality of services provided to customers. They provide an are commonplace in objective way of monitoring contract performance and determining the telecommunications whether the agreed-upon level of services has been provided. industry and are designed Having a service-level agreement in place that clearly identifies the primarily to improve specific services that would be delivered to K-12 schools would offer the quality of services assurance that the needs of public school administrators, teachers, provided to customers. and students are met. The lack of a service-level agreement makes it difficult to monitor CENIC’s performance. Our legal consultant recommended the key elements that should be included in a service-level agreement for this type of project. These elements are listed in Appendix B. 22 California State Auditor Report 2005-6 ICOE agrees that service-level agreements are commonplace in the telecommunications industry. ICOE stated that its attempt to negotiate service-level agreements with CENIC was unsuccessful because it was told by CENIC that its agreement must be consistent with those of the other CalREN participants and that their agreements do not contain this type of provision. Furthermore, CENIC told us that it does not make sense for it to enter into service-level agreements, which it believes are essentially penalty contracts. Specifically, because CENIC is a membership organization, if a service-level agreement was enforced, members would be taxing themselves with higher fees to fund the penalty. CENIC also stated that because its members are involved in the design of its networks, if the network is not reliable they would discuss how to improve the network instead of seeking a refund. However, CENIC coordinates a variety of High-Speed Network services on behalf of the county offices of education and school districts by contracting with various service providers. CENIC enters into appropriate service-level agreements with the various providers, which, when viewed as a whole, would offer assurance that the network would meet the needs of the K-12 education community. CENIC could prepare a master service-level agreement that consolidates the service-level agreements that are contained in its contracts with its providers. CENIC could then include a reference to the master service-level agreement in its agreements with ICOE and other participants, which allows them to understand and discuss the types of recourse available to them from CENIC’s service providers if service levels are not met. Furthermore, because CENIC is a nonprofit, rather than directly penalize it for nonconformance, it should be expected to pass along appropriate penalties to its service providers in a manner that would benefit its network participants. ICOE Should Continue to Pursue Opportunities for the State to Claim Ownership of Tangible, Nonshared Assets In its first agreement with CENIC, ICOE took steps to ensure that the State maintains ownership of assets that are purchased specifically for the High-Speed Network. Specifically, the agreement required that if CENIC no longer provides network services for the High-Speed Network, node equipment purchased with state funds must be returned to the lead agency for the High-Speed Network. California State Auditor Report 2005-6 22 However, its second agreement with CENIC, did not include We believe that it is this provision. ICOE stated that its attempt to include this appropriate for ICOE provision was also unsuccessful for the reasons previously stated. to include a provision Additionally, ICOE stated that because it did not receive funding in its contract to ensure for fiscal year 2005–06, it lacked the leverage it needed to fully the State’s ownership of pursue this provision. We believe that it is appropriate for ICOE to tangible nonshared assets. include a provision in its contract to ensure the State’s ownership of tangible, nonshared assets such as the node site equipment at the county offices of education and school districts. According to CENIC’s president and chief operating officer, CENIC has been advised by its counsel that the agreement between it and an equipment provider prohibits “piggybacking.” In other words, any equipment pricing and discounts under the agreement are provided solely to CENIC and not to any other entity, which effectively prohibits CENIC from buying equipment on behalf of a State entity. Nevertheless, because the State spends a large amount of money for state-funded entities, including the High-Speed Network, to participate in CENIC’s CalREN, it seems appropriate for these entities to bargain for ownership of their respective tangible, nonshared assets. Finally, if CENIC, a nonprofit public benefit corporation, were to dissolve, the laws governing nonprofit public benefit corporations would govern the disposition of assets. Those laws generally require that the assets be distributed to some other person or corporation carrying out that same public purpose. ICOE’s Agreement With CENIC Does Not Stipulate the Use of Interest Earned on Advance Payments Like UC’s agreement, ICOE’s first agreement with CENIC allowed for quarterly advance payments. Specifically, the quarterly payments were due to CENIC no more than 14 days after ICOE received funding from Education for the High-Speed Network project. According to ICOE, this method of payment is inconsistent with its normal business practices. Further, ICOE stated that CENIC requested this method of payment and cited the need for ICOE’s agreement to be consistent with those of the other participants. However, ICOE took certain steps to ensure that High-Speed Network project expenditures are appropriate. For example, in its first agreement, ICOE required CENIC to provide it with, at a minimum, a final accounting of the amounts expended, with sufficient detail for certain services. In its second agreement, ICOE agreed to make a lump sum payment to CENIC for fiscal year 2005–06 services and required CENIC to submit an invoice. These types of steps ensure that the funding for the High-Speed Network is used appropriately. 00 California State Auditor Report 2005-6 Neither of ICOE’s agreements, however, include a provision that requires CENIC to limit the use of interest earned on the advance payments for the High-Speed Network. In its defense, CENIC points out that it does not charge interest to ICOE when it makes payments on ICOE’s behalf prior to receiving the K-12 funding for the project. Nevertheless, without this provision, the interest earned could be used to fund CENIC’s other activities. RECOMMENDATIONS To ensure that the High-Speed Network meets its expectations, the Legislature should consider enacting legislation that prescribes the specific goals and outcomes it wants from the High-Speed Network project. If future state appropriations are made for the development of applications and associated content delivery to the K-12 education community, the Legislature should require the responsible agency to develop policies and guidelines that protect ownership of any intellectual property associated with related software or content, and the relevant contracts should be amended to address those policies. Furthermore, to fully protect any intellectual property that may be developed in the future using state funds, the Legislature should require contract terms that vest ownership of all intellectual property developed under that contract using state funds in the State of California. To ensure that the High-Speed Network is appropriately managed, Education should ensure that ICOE does the following: • Develops a comprehensive and extensive set of service-level agreements based upon applications to be delivered via the High-Speed Network project. • Requests that CENIC provide a master service-level agreement for its review. • Includes the appropriate service-level agreements in its ongoing contracts with CENIC and other service providers for the High-Speed Network, using industry standards as described in Appendix B. To ensure adequate protection of the State’s interest in tangible, nonshared assets, Education should direct ICOE to transfer ownership of those types of assets to the State, to the extent that ICOE is able to bargain for the provision. California State Auditor Report 2005-6  To ensure that the interest earned on advance payments made to CENIC are used to benefit the High-Speed Network, Education should direct ICOE to amend its agreement with CENIC to stipulate the allowable use of the interest earned. n 22 California State Auditor Report 2005-6 CHAPTER 2 Some Issues Have Arisen With CENIC’s Fees and Its Use of High-Speed Network Funds, and Its Accumulated Surpluses Are Decreasing CHAPTER SUMMARY The State appropriated roughly $112 million for the California K-12 High-Speed Network7 (High-Speed Network) project between fiscal years 2000–01 and 2004–05. This represents the largest portion of the $122 million total received by the Corporation for Education Initiatives in California (CENIC) and the Imperial County Office of Education (ICOE) to build and operate the network. Most of the State’s contribution covered direct and shared costs of the High-Speed Network, which include such items as fees for access to the commodity Internet. ICOE’s June 2005 report indicated that CENIC provides access to the commodity Internet at a low rate. However, our review found that CENIC could further reduce its rate. In addition, CENIC received more in quarterly advance payments from its contract with the University of California (UC) than it spent on developing the project. In December 2004 CENIC returned $10.8 million in unexpended funds to UC, and it continues to hold $7.2 million of High-Speed Network money in its consolidated equipment replacement account. According to its president and chief operating officer, $2.2 million represents the High-Speed Network’s share of costs to replace CENIC’s California Research and Education Network (CalREN) backbone equipment and the remaining $4.9 million represents funds for the replacement of High-Speed Network specific equipment such as its node site equipment. However, CENIC could not provide us with a technology refresh plan to justify the need for the full $7.2 million or that only $4.9 million represents funds for the replacement of the High-Speed 7 The California K-12 High-Speed Network was originally named the Digital California Project: K-12 Statewide Network when it was started in fiscal year 2000–01. In fiscal year 2004–05, when stewardship of the network was given to the Imperial County Office of Education, the aspects of the network applicable to K-12 participants were then titled the K12HSN. We call it the High-Speed Network throughout this report. California State Auditor Report 2005-6  Network specific equipment. CENIC’s chief technology officer told us that it is now working on a refresh plan and expects to complete the plan by the end of fiscal year 2005–06. STATE APPROPRIATIONS HAVE BEEN THE LARGEST SOURCE OF REVENUE FOR THE HIGH-SPEED NETWORK PROJECT During fiscal years 2000–01 through 2003–04, CENIC received state appropriations totaling $93.3 million, 92 percent of its total revenues for the project ($101.5 million). We discussed in Chapter 1 the lack of limitations or restrictions on these funds because the budget control language that made the appropriations did not impose any more specific requirements or controls on the expenditure of these funds and because UC has significant latitude in how it spends public funds. In the Budget Act of 2004, the Legislature effectively transferred state oversight of the High-Speed Network project to the California Department of Education (Education) and directed it to select a lead county office of education to administer the project. ICOE was selected as the lead agency. Education’s grant award notification to ICOE contains numerous conditions and assurances that ICOE agreed to prior to receiving the state appropriations. For example, ICOE must use procedures to ensure the proper disbursement and accounting of these funds. Additionally, ICOE must repay any funds that Education determines through its audits have been misspent. Table 1 shows funding received by CENIC and ICOE for the High-Speed Network project during fiscal years 2000–01 through 2004–05. The second largest source of funding comes from E-rate and The second largest source California Teleconnect Fund (Teleconnect fund) discounts that of funding comes from CENIC receives from certain telecommunications carriers. E-rate— E‑rate and California or, more precisely, the Schools and Libraries Universal Service Teleconnect Fund discounts Support Mechanism—is a federal program that provides discounts that CENIC receives from to assist most schools and libraries in the United States to obtain certain telecommunications affordable telecommunications and Internet access. Eligible schools carriers. can receive discounts ranging from 20 percent to 90 percent, depending on the percentage of their students who are eligible to participate in the federally free and reduced-price school lunch program. The Universal Service Administrative Company, a not-for- profit corporation, was appointed by the Federal Communications Commission to administer the Federal Universal Service Fund. It ensures that the benefits of telecommunication services reach students and communities across the country. Funding for the  California State Auditor Report 2005-6 discounts come from the telecommunication industry through surcharges the carriers levy on telephone bills for all residential and commercial telephone users in the United States. TABLE  Revenues Received for the High-Speed Network Project by Type Fiscal Years 2000–0 200–02 2002–0 200–0 200–05 Totals State appropriations $31,639,500 $27,470,160 $20,900,000 $13,300,000 $17,616,250* $110,925,910 E-rate/California Teleconnect Fund 0 0 4,088,717 2,554,975 3,140,507† 9,784,199 discounts Interest and dividend 669,597 473,626 211,489 159,914 136,447 1,651,073 income Other income 0 0 0 0 63,011 63,011 Totals $2,0,0 $2,,6 $25,200,206 $6,0, $20,56,25 $22,2, Sources: CENIC’s and ICOE’s accounting records. Note: Generally there is a delay between the submission of the application for E-rate/California Teleconnect Fund discounts and the receipt of the discounts. Specifically, according to our technical consultant, E-rate applications for a given fiscal year must be submitted in January prior to the fiscal year. We count these revenues during the fiscal year in which they were received. * ICOE expects to receive an additional $1.5 million from the California Department of Education for fiscal year 2004–05. † According to CENIC’s accounting records, it accrued an additional $3.6 million in E-rate/California Teleconnect Fund revenue for fiscal year 2004–05. The Universal Service Administrative Company requires the entity that pays the bills for the services to apply for the E-rate discounts. Because CENIC pays for the telecommunication circuits that connect the node sites of the county offices of education and school districts to its CalREN backbone, it was necessary for CENIC to create a statewide consortium so that it could receive discounts on the behalf of the county offices of education and school districts. In September 2001, CENIC began to form such a consortium for this purpose. According to CENIC, it submitted its first application in early 2002 for fiscal year 2002–03. Each year, CENIC invites school districts and county offices of education to participate in the consortium by submitting letters of agency—a legal document whereby one agency gives another authority to act on its behalf—authorizing it to include them in the statewide E-rate application. The letter of agency requires CENIC to use any savings it realizes on behalf of the consortium members for the benefit of the members. ICOE, as the lead agency responsible California State Auditor Report 2005-6 55 for administering the High-Speed Network project, contracted with CENIC to continue to apply for E-rate discounts. Both ICOE and Education encourage school districts and county offices of education to submit a letter of agency to CENIC. ICOE encourages them to submit letters of agency because the discounts are set aside for the support and improvement of the High-Speed Network. According to CENIC, it had almost 700 letters of agency related to its application for fiscal year 2006–07. All customers eligible to receive E-rate discounts for telecommunication services can also receive discounts from the Because E‑rate and California Public Utilities Commission (CPUC), via the California California Teleconnect Teleconnect Fund program, which is aimed at advancing Fund discounts received telecommunication services by providing discounts to schools can vary from year and libraries. CENIC requests the telecommunication carriers to to year, and ICOE’s submit applications to the CPUC to receive the discounts. The conservative budgeting discounts are 50 percent and must be applied after deducting the practices do not permit E-rate discount. According to a CPUC representative, there are no budgeting these funds limitations or restrictions on the use of the California Teleconnect on an ongoing basis, Fund program discounts. Because the amount of E-rate and it targets the use of the California Teleconnect Fund discounts received can vary from funds for expenditures of year to year, and ICOE’s conservative budgeting practices do not a one‑time nature. permit budgeting these funds on an ongoing basis, it targets the use of the funds for expenditures of a one-time nature. STATE APPROPRIATIONS ARE USED PRIMARILY TO PAY FOR HIGH-SPEED NETWORK NODE SITES AND CENIC’S FEES FOR THE USE OF ITS CALREN BACKBONE The High-Speed Network generates both direct and shared costs. Direct costs are those specifically associated with a service, program, or department and thus are clearly identifiable to a particular function. The High-Speed Network’s direct costs are unique to the K-12 education community, such as the equipment and circuit leases used to connect their node sites to CENIC’s CalREN backbone and some administrative costs. The CalREN backbone itself is shared by the K-12 education community, UC, California State University, California Community Colleges, and three private universities. CENIC charges these entities fees that allow it to recover its annual expenses. For the purposes of this report, we refer to CENIC’s fees as the High-Speed Network’s shared costs. Table 2 presents the direct and shared costs charged to the High-Speed Network project from fiscal years 2000–01 through 2004–05. 66 California State Auditor Report 2005-6 TABLE 2 High-Speed Network Costs CENIC ICOE Fiscal Year Fiscal Year Fiscal Year Fiscal Year Fiscal Year 2000–0 200–02 2002–0 200–0 200–05 Total Direct Costs Circuit leases* $ 17 $ 5,537,259 $ 4,952,399 $11,312,209 $ 9,023,485 $30,825,369 Equipment 3,290,689 7,099,641 4,140,439 3,004,690 814,606 18,350,065 Other 2,185,581 3,871,666 2,819,411 2,391,607 3,442,459 14,710,724 Total direct costs 5,6,2 6,50,566 ,2,2 6,0,506 ,20,550 6,6,5 Shared Costs CalREN backbone 984,750 4,011,000 6,247,729 8,535,804† 5,075,229‡ 24,854,512 Network operations center 143,500 574,000 § 1,400,000 § 2,117,500 Commodity Internet service 0 468,468 764,220 748,364 596,505 2,577,557 Other 103,601 207,173 112,500 571,296 196,500 1,191,070 Total shared costs ,2,5 5,260,6 ,2, ,255,6 5,6,2 0,0,6 Total direct and shared costs $6,0, $2,6,20 $,06,6 $2,6,0 $,, $,626, Sources: CENIC’s and ICOE’s accounting records. Note: Costs include both actual expenditures and year-end accruals. Other direct costs include costs such as, consulting and management fees related to the design of the network. * In fiscal year 2001–02, CENIC prepaid a telecommunications carrier $13.4 million for certain circuit leases. However the full cost of the circuits are not shown in fiscal year 2001–02. Rather CENIC spread the cost out over fiscal years 2001–02 through 2003–04. † This amount includes $4,803,804 for CalREN backbone fees that represents a board-approved prepayment of the High-Speed Network’s share of CENIC’s capital costs to upgrade the CalREN backbone. ‡ This amount includes a $1.2 million reduction in the CalREN backbone fees due to the prepayment made in fiscal year 2003–04. § In fiscal years 2002–03 and 2004–05, CENIC combined its network operations center fees with its CalREN backbone fees. Shared costs are subject to some interpretation because the direct benefit to the K-12 education community is not clearly identifiable. CENIC’s previous fee-setting method for the CalREN backbone was based on the estimated number of node sites and circuits that connect to the backbone rather than actual use. CENIC’s new fee-setting methodology also is not based on actual use. Specifically, CENIC divides the costs evenly amongst the four largest participants: the K-12 education community, the UC, California State University, and the California Community Colleges. However, CENIC’s new method reduces the costs to the High-Speed Network project. Nonetheless, the fees that CENIC charges its customers for their access to the commodity Internet could be lower. California State Auditor Report 2005-6  CENIC Initially Charged Higher Fees to the High-Speed Network, but Its Revised Methodology for Setting Fees Appears Reasonable Although a majority of the High-Speed Network costs were for expenses directly attributable to the K-12 education community, fees CENIC charges to recover the shared costs of operating its CalREN backbone and network operations center (operations center) represent 29 percent of the total High-Speed Network costs incurred since fiscal year 2000–01. CENIC’s board of directors approves the fees relating to its CalREN backbone and operations center each year as part of its annual CalREN budget. Operations center costs cover a wide range of services, including network management, operational support, and problem resolution. CENIC does not set its fees based on the actual use of its CalREN backbone because it typically does not track data flowing over CENIC does not set its fees the backbone to determine the user. According to its chief based on the actual use technology officer, although it is technologically possible to of its CalREN backbone track the backbone use, it is very costly to do so and has the because it typically does potential to cause slowdowns of traffic across the backbone. not track data flowing He also stated that performing the sampling necessary to track over the backbone to backbone use would require additional equipment at each of determine the user. the backbone hub sites. He added that CENIC has not estimated the cost to add this equipment, since adding it would interfere with achieving the purpose for which the network was designed. According to a CENIC manager, before the network was expanded to include the K-12 education community, CENIC’s method for recovering its costs was to divide its total annual operating costs by the number of connections to arrive at a per-unit connection cost. Each customer would pay a fee equal to the number of its connections multiplied by the per-unit connection cost. According to CENIC, in fiscal year 2002–03, CENIC began charging fees using a new methodology. In general, it based its CalREN backbone fees on each participant’s proportion of circuit bandwidth in relation to the aggregate total of the circuit bandwidth that connects directly to the backbone. For example, CENIC estimated that the High-Speed Network would have 120 of the total 359 circuits connecting directly to the backbone. CENIC also estimated that the total bandwidth of the 120 circuits would equal 5,800 megabits per second (Mbps), which is 51 percent of its estimate of the bandwidth of 11,339 Mbps for the total 359 circuits. For fiscal years 2003–04 and 2004–05, CENIC stated that it used the fiscal year 2002–03 fee as a base amount for its calculation of the backbone fees before  California State Auditor Report 2005-6 adding its estimate of other costs. The fiscal year 2002–03 fee, or For fiscal years 2002–03 base amount, represents 51 percent of CENIC’s estimated annual through 2004–05, CENIC cost for its CalREN backbone for that year. stated that it used the fiscal year 2002–03 fee CENIC used a similar fee-setting methodology to determine its as a base amount for its operations center fees, basing these fees upon each participant’s calculation of backbone share of the total number of node sites and circuits. In fiscal year fees before adding its 2002–03, it estimated that the High-Speed Network would have estimate of other costs. The roughly 40 percent of the estimated total number of node sites fiscal year 2002–03 fee and circuits connecting to the backbone for all participants.8 For represents 51 percent of its fiscal years 2003–04 and 2004–05, CENIC continued to use estimated annual cost for the fiscal year 2002–03 fee as the base amount for its calculation its CalREN backbone for of the operations center fees. Thus, it continued to charge the that year. High-Speed Network at least roughly 40 percent of its fiscal year 2002–03 estimated annual cost to run its operations center. The High-Speed Network also paid fees to participate in Internet2. The University Corporation for Advanced Internet Development, which sponsors Internet2, assesses a sponsored education group participant (group participant) fee to CENIC and others for their use of Internet2 and the Abilene network backbone.9 The group participant program is intended to allow expanded access to the Abilene network by state and regional education networks through sponsorship by Internet2 university members. State and regional networks may include nonprofit and for-profit educational institutions, museums, libraries, art galleries, and some hospitals. CENIC allocates the group participant fee by dividing each participant’s total circuit bandwidth by the total aggregate of group-participant-eligible circuit bandwidth.10 Finally, CENIC charges fees to its charter associates (UC, the California State University, California Community Colleges, Stanford University, the California Institute of Technology, and the University of Southern California) to help cover its general administration costs. CENIC’s board of directors sets these fees. In setting its CalREN backbone fee for fiscal year 2002–03, CENIC developed estimates of its costs, excluding certain purchases of equipment. Our comparison of CENIC’s cost estimates to its actual costs for fiscal years 2002–03 through 2004–05 found the estimated 8 CENIC’s calculation was as follows: (1) number of participant node sites divided by the total number of node sites multiplied by 50 percent; (2) number of participant circuits divided by the total number of circuits connecting to the backbone multiplied by 50 percent; (3) the results of step 1 and step 2 were added together. 9 Internet2 is a registered trademark of the University Corporation for Advanced Internet Development. 10 CENIC applies this calculation to those users who do not participate in its High Performance and Research Network. CENIC also includes a portion of the cost of running this network in the group participant fees. California State Auditor Report 2005-6  costs used in the fee-setting process to be reasonable. However, we question the number of High-Speed Network node sites and circuits CENIC used in calculating the fees, because it did not update this information after its initial calculation in fiscal year 2002–03. For example, in its initial calculation, CENIC estimated that the High-Speed Network would have 94 node sites. However, according to CENIC, as of December 22, 2005, the High-Speed Network had only 74 node sites. Additionally, there were only 99 circuits as opposed to the 120 that it had estimated. Further, according to CENIC, the bandwidth of the 99 High-Speed Network circuits is 8,325 Mbps, which is 45.8 percent of the aggregate bandwidth of 18,185 Mbps for the participants’ 230 circuits. The president and chief operating officer of CENIC agrees that the High-Speed Network’s fees would have been lower had it based its fees on the actual number of circuits and node sites. However, he pointed out that because the fee-setting methodology did not include CENIC’s cost for purchasing certain equipment for its CalREN backbone, the fees paid by the High-Speed Network were actually lower than they would have been if it had included the equipment purchases. According to its chief technology officer, effective fiscal year 2005–06, CENIC changed how it set certain fees because it recognized that the previous methodology could be a disincentive Effective fiscal year for network participants to increase their circuit bandwidth or 2005–06, CENIC changed expand their networks, due to the higher fees that would result. He how it set certain fees. also stated that the network participants preferred to be billed using Although the methodology a methodology that would result in a more predictable rate. ICOE is still not based on also provided its perspective on CENIC’s decision to change how it participants’ actual set certain fees. According to an ICOE manager, he requested CENIC network usage, the change to provide a copy of its fee-setting methodology to ensure that the does represent a reduction High-Speed Network CalREN backbone fees did not exceed the in the annual shared amount stated in the agreement between ICOE and CENIC for fiscal costs to be borne by the year 2004–05. He further stated that his review of the methodology High‑Speed Network and raised a number of questions and indicated the High-Speed Network appears reasonable. fee should have been significantly lower for the K-12 education community. Therefore, he requested CENIC to provide an update of its methodology using accurate data. CENIC presented an update of its methodology as well as additional fee-setting models at its June 2005 business advisory committee meeting. Furthermore, according to the ICOE manager, the committee members used this information to test CENIC’s fee distribution to its other network participants, and found that each participant’s share, excluding the private entities, was roughly 25 percent.11 11 According to its president and chief operating officer, because the new fee represents a significant increase for the higher education institutions, CENIC plans to increase their fee over a two-year period beginning in fiscal year 2005–06. 00 California State Auditor Report 2005-6 CENIC’s new methodology subtracts fee amounts for the three private universities from its estimated annual costs for the four aforementioned services and divides the remaining total estimated annual cost equally among the four state-funded entities. Although this methodology is still not based on participants’ actual network usage, the change does represent a reduction in the annual shared costs to be borne by the High-Speed Network and appears reasonable. CENIC’s Charges for Commodity Internet Use Could Have Been Lower CENIC provides connections to Internet service providers, enabling High-Speed Network users to access the commodity Internet. Although the annual fees it charges for this access are lower than state negotiated pricing, it could further reduce the amount it charges users by consistently using funds left over from prior-year fees to offset the next year’s cost of providing the service. CENIC based its initial fee for commodity Internet service on a calculation that divided its estimate of the program’s annual CENIC’s commodity operating costs, such as CENIC’s charges from its Internet service Internet service, which providers, by each participant’s annual minimum commitment became effective during for Internet usage. According to CENIC, if participants did fiscal year 2002–03, not meet their minimum usage commitment, they would still has generated a be responsible for paying a fee that was equivalent to their surplus each year; as commitment amount, thus ensuring that CENIC would at least of June 30, 2005, this break even. CENIC’s commodity Internet service, which became surplus was $2.1 million. effective during fiscal year 2002–03, has generated a surplus each year; as of June 30, 2005, this surplus was $2.1 million. This surplus indicates that the fees CENIC charges do not reflect its actual costs to provide the service. According to a report published on November 1, 2005, by ICOE, based on a cost comparison of the High-Speed Network’s commodity Internet costs to the state-negotiated pricing, CENIC has been able to generate savings to the State for commodity Internet services. Additionally, according to the manager of the High-Speed Network at ICOE, because the fees for commodity Internet service are paid for as part of the High-Speed Network, none of the county offices of education, school districts, and schools connected to the network pay for their commodity Internet usage. According to its chief technology officer, CENIC has been able to offer low costs for commodity Internet usage because it represents the combined traffic volume of its participants and is a member of the Quilt, a project California State Auditor Report 2005-6  sponsored by the University Corporation for Advanced Internet Development that is open only to nonprofit regional network aggregators providing advanced network services in support of research and education and other select organizations. According to CENIC, as a Quilt member, it is able to obtain commodity Internet service at reduced costs and pass its savings to program participants. Nonetheless, although CENIC’s commodity Internet service fee is competitive, CENIC could reduce its fee. The commodity Internet service model approved by its board in June 2001 specifically states that the fixed rate charged per unit of commodity Internet usage should be set to enable CENIC to recover the entire cost of providing the services, should be reviewed semiannually, and should be adjusted downward if cost recovery is projected to be excessive. CENIC did use a portion of its fiscal year 2002–03 surplus revenues to reduce its per-unit rate in fiscal year 2003–04 by 38 percent. It was able to do this for several reasons: a reduction in its estimated annual costs, an increase in its minimum usage commitments for commodity Internet service, and its use of a portion of the surplus. For fiscal year 2004–05, however, although CENIC reduced its per-unit rate by a further 25 percent compared to its fiscal year CENIC did not use surplus 2003–04 per-unit rate, it did not use the surplus revenues to do revenues to reduce its fiscal so. It achieved its reduction by reducing its estimated annual costs year 2004–05 per‑unit rate and increasing the minimum usage commitments for commodity for commodity Internet Internet service for certain users. We believe that further reductions service. We believe that at would have been possible if CENIC had also used a portion of the a minimum, CENIC’s board surplus. Moreover, when it calculated its 2004–05 per-unit rate of could have reduced this $95 in May 2004, it did not include a minimum usage commitment per‑unit rate by the $14 amount in its calculation for the K-12 education community. proposed by CENIC staff. Because it divides its estimate of the program’s annual operating costs by each participant’s annual minimum commitment for Internet usage, when CENIC uses a lower minimum usage commitment amount its per-unit rate increases. According to CENIC’s chief technology officer, it did not do so because of the uncertainty surrounding the continuation of the K-12 education community’s participation in CENIC and the program. Once the contract between ICOE and CENIC was executed on December 6, 2004, CENIC staff did propose to CENIC’s board a per-unit rate reduction of $14 for fiscal year 2004–05 in January 2005. This proposed reduction was based on the inclusion of the K-12 education community’s minimum usage commitment into the fee calculation, not on surplus revenues. However, the board did not approve the rate reduction. 22 California State Auditor Report 2005-6 According to the chair of CENIC’s finance committee, this was due to the considerable uncertainty over continued K-12 participation in future years and the considerable financial stress this would create. He further stated that the board understood that by not lowering the per-unit rate in midyear, the Internet service program would generate a larger surplus. He also stated that it was entirely appropriate for the board to take this position in light of its fiduciary responsibility to the corporation. Although we agree that it is appropriate for the board to plan for contingencies, CENIC had already set aside more than $300,000 in reserves related to the commodity Internet service program for such contingencies and had accumulated an additional $971,000 in surplus funds from prior years. Therefore, we believe there was an opportunity for CENIC to reduce its rate. If the board had chosen to use the surplus revenues of $558,000 generated during fiscal year 2003–04 to offset CENIC’s fiscal year 2004–05 estimated annual cost, it could have reduced the 2004–05 per-unit rate from $95 to $64, a reduction of almost 33 percent. We believe that at a minimum, the board could have reduced its per-unit rate by the $14 proposed by CENIC staff because the contract with ICOE had already been executed, thus eliminating any uncertainty regarding the K-12 education community’s participation in the program. Instead, according to its president and chief operating officer, the remaining surplus revenues were consolidated into CENIC’s general operating revenue and were used as it determined best for the corporation as a whole. In June 2005, CENIC’s board approved removing the participants’ minimum usage commitments from the fee According to its president calculation because it felt this change would benefit all and chief operating participants. Its chief technology officer stated that CENIC officer, the remaining felt it had enough experience to fulfill its minimum usage surplus revenues were commitments to its commodity Internet service providers and consolidated into CENIC’s be able to charge its participants based on their actual usage. He general operating also stated that, given the increased use of peer networking, it revenue and were used as was likely that some participants may not meet their minimum it determined best for the usage commitments. Peer networking allows users connected corporation as a whole. to the CalREN backbone to access direct network connections to more than 60 other networks, including Google and Yahoo!, without using the commodity Internet. As a result, these users are not billed to visit those Web sites, and their actual commodity Internet usage is less. However, for fiscal year 2005–06, CENIC still has not lowered its per-unit rate. According to the chief technology officer, CENIC believes that a certain amount of risk exists without the minimum usage commitment levels. California State Auditor Report 2005-6  MORE FUNDS WERE APPROPRIATED TO THE HIGH-SPEED NETWORK PROJECT THAN WERE SPENT In fiscal year 2000–01, CENIC anticipated spending $16.2 million in High-Speed Network funds primarily for circuits and CalREN backbone fees, as well as $11.8 million for equipment. However, as of June 30, 2001, CENIC had expended only $4.3 million. In total, as of June 30, 2004, CENIC had expended $26 million less on the project than it had received in revenues. Because the High-Speed Network was not funded during fiscal year 2005–06, the balance of unused funds has decreased, as both CENIC and ICOE are using these funds to continue operating the network. Table 3 shows the funds available to ICOE for the High-Speed Network as of June 30, 2005. TABLE  High-Speed Network Funds and Credits Held by CENIC as of June 0, 2005 Funds Funds held at June 30, 2004* $25,989,965 E-rate/ California Teleconnect Fund discounts received during fiscal year 2004–05 3,140,507 E-rate/ California Teleconnect Fund discounts accrued during fiscal year 2004–05 3,547,248 E-rate/ California Teleconnect Fund discounts paid to ICOE during 2004–05 (3,300,000) Accounting adjustment† 134,341 Interest transferred to CENIC’s general operations account between fiscal years 2000-01 through 2003–04‡ (1,514,626) Funds returned to UC (10,808,580) Total funds available as of June 0, 2005 ,,55 Credits Prepayments for CalREN backbone fees 3,600,000 Total available funds and credits at June 30, 2005 20,788,855 Breakdown of available funds Equipment reserves 7,157,408 E-rate/California Teleconnect Fund discounts held for ICOE 10,031,447§ Total $,,55 Source: CENIC’s accounting records. * The $26 million is arrived at by subtracting the total expenditures for fiscal years 2000–01 through 2003–04 shown on Table 2 from the total revenue for the same period shown on Table 1. † The accounting adjustment is a reversal of a fiscal year 2003–04 expenses in fiscal year 2004–05. ‡ As discussed in Chapter 1, UC did not specify that the interest earnings could only be used for High-Speed Network purposes. Therefore, CENIC does not consider them to be High-Speed Network funds. Thus, it transferred the funds to its general operations account. According to CENIC’s accounting records, in fiscal year 2004–05, it earned an additional $136,447 in interest that is not reflected in this amount. § CENIC states that it anticipates returning $550,000 of the remaining E-rate discounts to the Universal Service Administration Company due to an overpayment.  California State Auditor Report 2005-6 CENIC Returned Some of the Unexpended Funds to UC On July 8, 2004, UC requested an audit of the unspent reserve held for the High-Speed Network project as of June 30, 2004. According to UC’s legal counsel, UC’s vice president of financial management met with a representative from CENIC and CENIC’s independent auditor prior to CENIC’s annual audit. During this meeting, the independent auditor was informed that a careful review of the High-Speed Network deferred revenue account was in order, since UC might request the return of these funds. Deferred revenue is money that CENIC has received but has not yet earned by providing services. The funds are held in a liability account until CENIC provides the services, at which time CENIC would then recognize them as revenue. On October 6, 2004, the board approved the return of the deferred revenue to UC upon CENIC’s signing of an agreement with ICOE and an external audit of the account balance. In a letter dated October 15, 2004, CENIC’s independent auditor stated that it believed the “unspent reserves” held for the High-Speed Network project as of June 30, 2004, deferred revenue of roughly $10.3 million, was fairly stated. CENIC later increased this amount to $10.8 million, primarily due to its reversal of certain High-Speed Network expense accruals. After CENIC signed its contract with ICOE on December 6, 2004, the CENIC board approved the return of the deferred revenues to UC, and a check totaling $10.8 million was issued on December 14, 2004. UC used these funds to offset its pending fiscal year 2003–04 midyear budget reductions. However, because UC directed CENIC’s independent auditor to account only for deferred revenues, the audit did not take into account the state appropriations held by CENIC as of June 30, 2004, related to the High-Speed Network reserves for equipment replacements. CENIC Has a Portion of the High-Speed Network’s Funds in Its Consolidated Equipment Replacement Account During its September 12, 2002 meeting, CENIC’s board approved the following three action items related to the High-Speed Network funds held by CENIC for equipment replacement: (1) the creation of a consolidated designated equipment replacement account as part of its CalREN account, the transfer of $2.6 million from its CalREN-2 account and $5.7 million in High-Speed Network funds from an account designated solely for the High-Speed Network into this new account, and the transfer of future High-Speed Network equipment replacement funds into this new account; (2) the transfer of all uncommitted interest income in the CalREN-2 account of $128,000 and $970,000 of the interest California State Auditor Report 2005-6 55 income in an account designated solely for the High-Speed Network into the consolidated designated equipment replacement account; and (3) the transfer of $6 million from the consolidated designated equipment replacement account into a one-year The board’s decision to certificate of deposit with a bank, the borrowing of $6 million include the High‑Speed from the same bank, and the use of the certificate of deposit as Network’s equipment collateral against the loan. According to CENIC’s accounting replacement funds in a records, on June 30, 2004, an additional $1.5 million was consolidated account placed into the consolidated designated equipment replacement appears inconsistent reserve account using state appropriations for the High-Speed with CENIC’s agreement Network. The board’s decision to include the High-Speed Network’s with UC, which requires equipment replacement funds into a consolidated account CENIC to set up and use a appears inconsistent with CENIC’s agreement with UC, which separate financial account requires CENIC to set up and use a separate financial account for for the High‑Speed the High-Speed Network funds and to not use that account to hold Network funds. or disperse any other funds. The purpose of establishing a separate financial account for the High-Speed Network funds is to ensure that these funds are being used to benefit the project. The transfer of these funds to CENIC’s consolidated account makes it difficult to identify those funds belonging to the High-Speed Network. According to its president and chief operating officer, as of January 12, 2006, CENIC still held the $7.2 million in High-Speed Network money in its consolidated equipment replacement account. He further stated that $2.3 million of this amount represents the High-Speed Network’s share of costs to replace CENIC’s CalREN backbone equipment and that the remaining $4.9 million represents funds for the replacement of equipment specifically for the High-Speed Network, such as its node site equipment. However, CENIC could not provide us with a technology refresh plan. An effective technology refresh plan establishes the points along the service life of a product or system at which it is optimal to change system components. According to CENIC’s chief technology officer, in the first two years of the High-Speed Network project, CENIC did not feel a need to have a refresh plan; developing a plan too early would not be useful, he stated, since neither its needs nor the technology can be predicted four years in advance. He also told us that CENIC’s general strategy was to create an equipment reserve so that at the end of the useful life of various pieces of equipment, it had funds available to refresh them. CENIC’s administrative policy and practices contain the useful lives for different types of assets, for accounting purposes. For example, CENIC assigned a useful life of four years for routers. It then uses the useful lives to calculate 66 California State Auditor Report 2005-6 how much it needs to set aside in equipment replacement reserves. For example, if a router cost $80,000, CENIC would place $20,000 in its equipment reserve each year for four years. Without a technology refresh plan, we do not believe CENIC can support its assertion that it needs the full $7.2 million, or that only $4.9 million represents funds for the replacement of equipment specific to the High-Speed Network. For example, our technical consultant found that certain components of the network, such as the primary router used at the node sites, should be able to support the network for many years to come, unless there is a need to upgrade them. Conversely, our technical consultant found that other components, such as certain routers at the CalREN backbone hub sites, are the subject of an end-of-life announcement from the manufacturer. The final date to receive service and support for these routers is February 2010. The manufacturer’s end-of-life announcement also provides a migration strategy to allow users to transition to other components within the same series family. CENIC’s chief technology officer told us that it is now working on a refresh plan and expects to complete it by the end of fiscal year 2005–06. In Chapter 1, we discussed our belief that it is appropriate for ICOE to transfer the ownership of tangible, nonshared assets from Until CENIC and ICOE CENIC to the State. If this transfer occurs, ICOE will be responsible establish a technology for maintaining sufficient funds in an equipment replacement refresh plan, it will be account and developing a technology refresh plan. Thus, it seems difficult for ICOE to reasonable that ICOE and CENIC should work together to develop determine if sufficient the technology refresh plan for the existing equipment and any funds have already been new purchases. Until they establish such a plan, it will be difficult set aside to replace K‑12 for ICOE to determine if sufficient funds have already been set node equipment before aside to replace K-12 node equipment before it advances CENIC it advances CENIC more funds. more funds. Finally, although CENIC is holding $7.2 million in High-Speed Network funds for equipment replacement, any interest earned on this money does not accrue to the benefit of the High-Speed Network. Specifically, as we discussed in Chapter 1, its agreement with ICOE does not contain a provision that limits the use of any interest earned on state appropriations to the High-Speed Network. By including this provision in its agreement, ICOE can ensure that the project benefits directly from any interest earnings. California State Auditor Report 2005-6  In the Absence of State Appropriations, CENIC Is Using the Remaining Unexpended Funds to Operate the Network For fiscal year 2005–06 the Legislature did not appropriate any additional funding to the High-Speed Network, because it required the Joint Legislative Audit Committee to conduct an audit of the High-Speed Network. In the absence of state funding, CENIC plans to use $8.1 million of the approximately $10 million in E-rate and California Teleconnect Fund discounts and $3.6 million in prepayments related to the CalREN backbone fees to offset its charges to ICOE for operating the High-Speed Network. In September 2002, the CENIC board of directors approved a $4.4 million charge to the High-Speed Network to prepay fees for using the CalREN backbone. The board subsequently increased the prepayment to $6 million. To adjust for the prepayment, CENIC applies a $1.2 million credit each year against the High-Speed Network’s CalREN backbone fees. CENIC first applied the credit in fiscal year 2003–04, leaving a prepayment amount of $4.8 million. In fiscal year 2004–05 CENIC applied another $1.2 million, leaving a prepayment balance of $3.6 million. On July 20, 2005, CENIC’s board approved using the remaining prepayment balance of $3.6 million to help fund the High-Speed Network’s operating costs through fiscal year 2005–06. Additionally, in accordance with their contract executed on December 6, 2004, ICOE and CENIC plan to use unspent E-rate and California Teleconnect Fund discounts to continue the operation of the High-Speed Network in fiscal year 2005–06. The contract states, “To the extent that program revenue balances generated by E-rate and California Teleconnect fund discounts from fiscal year 2002–03, or prior fiscal years, exist, such balances will be held by CENIC to help meet cash flow needs.” The contract further stipulates, “Such funds will be held in trust by CENIC for the benefit of the High-Speed Network and will not be expended without advance consultation with ICOE.” Finally, ICOE and CENIC agreed that any E-rate and California Teleconnect Fund discounts for fiscal year 2004–05 circuit expenditures received in that year shall be held by CENIC and applied against the network circuits, backbone fees, and related costs in fiscal year 2005–06. As of December 2005, according to CENIC’s estimate, a total of $10 million was available for use toward the 2005–06 High-Speed Network operational costs. However, ICOE’s agreement does not require CENIC to increase the amount that it holds on behalf of ICOE by any interest earned on the funds. A similar issue  California State Auditor Report 2005-6 arose regarding UC’s agreement with CENIC. As we discussed in Chapter 1, absent a specific provision in the agreement, CENIC views any interest earned on the High-Speed Network funds as accruing to its benefit. Until ICOE modifies its agreement with CENIC, the State will continue to lose the ability to use interest earnings to reduce High-Speed Network costs. RECOMMENDATIONS To ensure that CENIC’s per-unit rate for access to the commodity Internet is closer to its actual cost to provide the service, the California Department of Education (Education) should require ICOE to amend its agreement with CENIC to stipulate that to the extent possible, CENIC should use its surplus Internet service program revenues from each year to offset the per-unit rate that it sets the following year. ICOE should also stipulate in its agreement that if CENIC is unable to apply the surplus revenue due to a change in its financial position, CENIC should provide ICOE with documentation to support its inability to do so. To ensure that High-Speed Network equipment replacement funds are used to benefit the K-12 education community, Education should direct ICOE to request that CENIC reestablish a reserve for equipment replacement that is in an account solely for the High-Speed Network. Further, ICOE should amend its agreement with CENIC to stipulate that interest earned on the funds held in the High-Speed Network’s equipment replacement account accrues to the benefit of the High-Speed Network. Finally, Education should direct ICOE to amend its agreement with CENIC to stipulate that CENIC should use the funds held in the High-Speed Network equipment replacement account to purchase new equipment instead of requesting reimbursement from ICOE until the account is depleted. As CENIC purchases new equipment after advance consultation with ICOE, it should also consult with ICOE on the development of a technology refresh plan, which ICOE should use to establish its own equipment replacement funds for the High-Speed Network. To ensure that any interest earnings received for E-rate and California Teleconnect Fund discounts accrue to the benefit of the High-Speed Network, Education should direct ICOE to amend its agreement and require CENIC to credit any interest earnings to the High-Speed Network project. Additionally, ICOE should require CENIC to provide a detailed accounting of E-rate California State Auditor Report 2005-6  and California Teleconnect Fund discounts so that it can verify that it received the appropriate amount of interest. If CENIC does not agree to this provision, ICOE should consider requiring CENIC to remit all funds received from the telecommunications carriers, net of any agreed-upon administrative costs, to ICOE promptly upon its receipt of these funds. In the event that CENIC is unable to remit the funds promptly, any interest on the funds should accrue to the benefit of the High-Speed Network. n 5500 California State Auditor Report 2005-6 CHAPTER 3 CENIC’s Network Architecture Is Sound, but It Is Difficult to Determine If the High-Speed Network Is Being Used as Originally Intended CHAPTER SUMMARY As tudy conducted by our technical consultant in 2005 indicated that the California K-12 High-Speed Network (High-Speed Network)12 project has adequate bandwidth for potential growth but is not overbuilt. Most of the circuits are sized to support the current traffic load, but when the K-12 education community begins to use additional applications, many of the circuits will need increased bandwidth. However, actual usage of specific applications currently available on the network cannot be determined because neither the Corporation for Education Initiatives in California (CENIC) nor the Imperial County Office of Education (ICOE), the lead agency on the project, have provided a process to measure this. ALTHOUGH THE HIGH-SPEED NETWORK HAS SPARE BANDWIDTH, IT IS NOT OVERBUILT In April 2005, ICOE began using advanced network management software to collect meaningful usage data so that it can ensure the prudent administrative management of the High-Speed Network. Our technical consultant’s review of the High-Speed Network usage data for the period of April 6, 2005, through September 16, 2005 (study period) found that although the network clearly has spare bandwidth today on many circuits, it is not overbuilt.13 According 12 The California K-12 High-Speed Network was originally named the Digital California Project: K-12 Statewide Network when it was started in fiscal year 2000–01. In fiscal year 2004–05, when stewardship of the network was given to the Imperial County Office of Education, the aspects of the network applicable to K-12 participants were then titled the K12HSN. We call it the High-Speed Network throughout this report. 13 ICOE could not provide us with the data for the period May 15, 2005, through May 26, 2005, because the information was inadvertently discarded during its backup process. Also, the data include only usage on the K-12 node sites and circuits and do not include usage on the shared CENIC California Research and Education Network backbone, because CENIC does not track backbone traffic by customer. The data do not include weekends, because K-12 schools are not generally open for instruction on those days. However, the data do include the summer months because there is a reasonable amount of network traffic, although leaving the summer months in the calculation of the average tends to lower the average peak. California State Auditor Report 2005-6 55 to our technical consultant, a substantial majority of the circuits currently in place are appropriately sized to support today’s traffic loads. Also, many of the circuits that would appear to have excess bandwidth are in place primarily to support redundancy in the High-Speed Network’s self-healing ring architecture. Further, when the K-12 education community identifies and begins to use more applications that the network is designed to support, it is likely that many of the circuits will actually need to increase in bandwidth. Excess Bandwidth in Ethernet Connections Is Needed to Avoid Local Area Network Bottlenecks and Is Most Likely Not Costly According to our technical consultant, the standard for a local area network (LAN) has been Ethernet for several years. Ethernet is installed in virtually every office (and even in many homes today) to support communications among personal computers, printers, and other network devices. Over the years, the bandwidth of Ethernet LANs has increased steadily. Table 4 shows the Ethernet Hierarchy. TABLE  Ethernet Hierarchy Bandwith in Megabits per Second Ethernet Standard (Mbps) Bandwidth in Bits per Second (bps) 10Base-T 10 Mbps 10,000,000 bps 100Base-T (Fast Ethernet) 100 Mbps 100,000,000 bps 1,000BaseT (Gigabit Ethernet) 1,000 Mbps (1 Gbps) 1,000,000,000 bps 10,000Base-T (10 Gigabit Ethernet) 10,000 Mbps (10 Gbps) 10,000,000,000 bps Sources: B and B Electronics white paper titled Ethernet Basics issued in 2002; Intel Corporation white paper titled 10 Gigabit Ethernet Technology Overview issued in 2003. With regard to whether or not the High-Speed Network has excess network bandwidth, it is critical to understand that network bandwidth in an Ethernet environment must be obtained in one of these increments. If a company installs Fast Ethernet and finds that the bandwidth is inadequate, it must upgrade to Gigabit Ethernet. The increase in bandwidth is by definition tenfold, even if the actual growth requirement is minimal. According to our technical consultant, the bandwidth of the interface used to connect the High-Speed Network node equipment to the internal LAN of a county office of education (office) would 5522 California State Auditor Report 2005-6 usually be chosen based on the speed of the LAN. An interface allows two independent systems to meet and communicate with each other. If the office operates a Fast Ethernet LAN backbone, then a Fast Ethernet interface to the node would be adequate and appropriate because the LAN could never offer more traffic than its own bandwidth. However, if the office operates a Gigabit Ethernet LAN backbone, it would be reasonable to select a Gigabit Ethernet circuit to the node. Otherwise, a bottleneck could be created at the node, with more traffic being offered—up to 1,000 Mbps—than the 100 Mbps that the node interface could accept. This bottleneck would have a disruptive influence on the office’s LAN and would have the potential to slow down all LAN traffic. In Appendix C, we present the single and average peak usages for our study period, expressed as a percentage of the circuit’s bandwidth. It might be reasonable for the layperson to conclude Although the preferred that several of the Ethernet interfaces shown in Appendix C support percentage may vary, a substantially higher bandwidth than is required and therefore that our technical consultant these interfaces might be “overengineered.” However, according believes network to our technical consultant, it is important to note that accepted components should not industry practice purposely specifies that a network component exceed 70 percent of their commonly not reach its maximum bandwidth. Although the maximum bandwidth. preferred percentage may vary, our technical consultant believes that network components should not exceed 70 percent of their maximum bandwidth. Using that benchmark, in cases where the peak usage of a Fast Ethernet connection exceeds 70 percent, it would be appropriate to upgrade that connection to Gigabit Ethernet. When recalculated, the new Gigabit Ethernet connection would appear to have grossly excessive bandwidth, since a 70 percent usage rate of a Fast Ethernet would use only 7 percent of a Gigabit Ethernet and may therefore appear to be overengineered. Similarly, for a Fast Ethernet interface that shows only 7 percent usage, a reduction to a 10 Mbps circuit would be inadequate because that circuit would then be running at 70 percent of its bandwidth. Our technical consultant’s review of CENIC’s inventory of the equipment located at each node site found that in virtually every case, the equipment installed includes both Gigabit Ethernet and Fast Ethernet interfaces, allowing the office’s network managers to select the appropriate speed to support their LAN. This approach provides flexibility and results in a more standard installation at all node locations, which facilitates network management. Moreover, our technical consultant believes that the incremental costs for the interfaces that connect the Gigabit Ethernet versus the Fast Ethernet would be relatively small and a one-time expense. California State Auditor Report 2005-6 55 Most Digital Signal Level  Circuits With Low Usage Are Necessary to Support Network Traffic or Redundancy in the Self-Healing Ring The connections between node sites and CENIC’s California Research and Education Network (CalREN) backbone, as well as certain connections between node sites, are typically digital signal level (DS) 3 circuits. Table 5 illustrates the digital signal level hierarchy that, according to our technical consultant, has been the traditional network structure of all carriers in the United States for decades. TABLE 5 Digital Signal Hierarchy Bandwidth in Megabits per Second Signal Number of DS Circuits (Mbps) DS0 1/24 0.064 Mbps DS1 1 1.544 Mbps DS2 4 6.312 Mbps DS3 28 44.736 Mbps Sources: Bureau of State Audits’ technical consultant; Tektronix Inc. report titled SONET Telecommunications Standard Primer issued in 2001. Our technical consultant focused solely on DS1 and DS3 signals. According to the consultant, DS0 signals represent the equivalent of dial-up speeds and are inadequate even in most household applications today. The consultant believes that DS2, although defined as a standard within the hierarchy, is rarely, if ever, implemented, especially in an enterprise network. DS1 circuits provide adequate bandwidth for a small number of users. Their bandwidth is on the same order of magnitude as digital subscriber line (DSL) service (as supplied by telephone carriers) or cable modem service (as provided by cable television franchisees). Thus, DS1 circuits provide adequate bandwidth for home and small office environments. DS3 is the signal level typically used in the High-Speed Network to connect node sites to CENIC’s CalREN backbone. Unlike the Ethernet hierarchy, the digital signal hierarchy does permit multiple DS1 circuits to be aggregated to create a larger-bandwidth 55 California State Auditor Report 2005-6 circuit. For example, 10 DS1 circuits connecting two points could be installed to create a single circuit supporting 15.44 Mbps. However, according to our technical consultant, other factors must be reviewed to determine whether it is feasible to install multiple DS1 circuits versus a single DS3 circuit. These include the following: • The bandwidth of a DS3 circuit is 28 times that of a DS1 circuit. However, the cost to lease a DS3 circuit from a carrier is not 28 times the DS1 circuit cost. Typically, it becomes more cost effective to lease a DS3 circuit as compared to multiple DS1 circuits above a certain threshold. That threshold could be as low as five or up to as many as 10 DS1 circuits, depending on a number of factors that include mileage and length of commitment. This ratio explains why the DS2 signal level is irrelevant, because five DS1 circuits would exceed one DS2 circuit’s 6.312 Mbps. • The node equipment contains interfaces that connect the circuits to the LAN, the node sites, or the CalREN backbone. The equipment that CENIC typically uses has a maximum of five configurable interface slots to support various interfaces. Depending on the number of DS1 circuits to be installed, the available slot bandwidth could be exhausted, and the cost trade-off to use one DS3 circuit would be minimal. Figure 4 on the following page illustrates the break-even points where it becomes cost effective to use a DS3 circuit instead of multiple DS1 circuits. The figure shows that the break-even point is highly mileage sensitive, with DS3 circuits proving to be cost effective sooner when the mileage is greater. Further analysis of the data in Appendix C shows that 39 of the 105 DS3 circuits connect nodes directly to CENIC’s CalREN backbone. The maximum single-day usage for these circuits ranges from 13.4 percent to 163.3 percent, and their average maximum usage ranges from 4.2 percent to 70.7 percent.14 The remaining 66 of the 105 DS3 circuits connect nodes to each other. 14 According to our technical consultant, the traffic can exceed the DS3 circuit’s 44.21 Mbps capacity in short bursts. California State Auditor Report 2005-6 5555 FIGURE  Break-Even Points Between Multiple DS Circuits and One DS Circuit 20,000 One DS3 Circuit Ten DS1 Circuits Nine DS1 Circuits Eight DS1 Circuits 16,000 Seven DS1 Circuits Six DS1 Circuits Five DS1 Circuits 12,000 8,000 4,000 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 Miles Source: Bureau of State Audits’ technical consultant. Assuming a reasonable break-even scenario in which the cost for six DS1 circuits that are 45 miles long is greater than the cost for a single DS3 circuit, and considering the practice of not exceeding 70 percent of the actual circuit bandwidth, those DS3 circuits showing a usage of 15 percent or greater warrant the DS3 circuit. As shown in Appendix C, using the maximum single-day usage, this would represent 37 out of the 39 DS3 circuits. However, when considering the average maximum usage, only 25 of the 39 DS3 circuits exceed the 15 percent threshold. 5566 California State Auditor Report 2005-6 The maximum single-day usage for the 66 DS3 circuits connecting nodes to each other ranges from 0.2 percent to 151.3 percent, while the average maximum usage ranges from zero percent to 64.1 percent. Of the 66 circuits, 31 had a maximum single-day usage exceeding the 15 percent threshold, and 12 had an average maximum usage exceeding the 15 percent threshold. However, many of the 66 DS3 circuits between nodes support the redundancy function of the High-Speed Network’s self-healing ring architecture and are intended to support traffic only when another circuit fails. The argument could be made that, since this circuit serves almost solely as a backup and thus has lower usage, it might be feasible to install a lower-bandwidth, less expensive circuit. Our technical consultant suggested, however, that when a redundant circuit is required, it should be able to support the entire traffic load and should be transparent to the user. Therefore, the circuit should have the same bandwidth as the primary circuit. Most of the Optical Carrier Circuits Are Also Necessary to Support Network Traffic The High-Speed Network also includes 31 optical carrier (OC)-3 circuits. The OC-3 circuits, each with a bandwidth of 155 Mbps, are part of the Synchronous Optical Network (SONET) hierarchy, as shown in Table 6. The OC-3 circuit has the next largest bandwidth increment above DS3; 15 of these are used to link larger nodes to each other and 16 are used to link larger nodes to CENIC’s CalREN backbone. TABLE 6 Synchronous Optical Network Hierarchy Bandwidth in Megabits Optical Carrier (OC) Signal Level per second (Mbps) Digital Signal (DS) Level Equivalent OC-1 51.84 Mbps 28 DS1 or 1 DS3 OC-3 155.52 Mbps 84 DS1 or 3 DS3 OC-12 622.08 Mbps 336 DS1 or 12 DS3 OC-48 2,488.32 Mbps 1,344 DS1 or 48 DS3 OC-192 9,953.28 Mbps 5,376 DS1 or 192 DS3 OC-768 39,813.12 Mbps 21,504 DS1 or 768 DS3 Sources: Bureau of State Audits’ technical consultant; Tektronix Inc. report titled SONET Telecommunications Standard Primer issued in 2001. California State Auditor Report 2005-6 55 To calculate the point at which a DS3 circuit should be increased to an OC-3 circuit, note that an OC-3 circuit is the equivalent of three DS3 circuits. An OC-3 circuit at 33 percent of bandwidth usage is the equivalent of a DS3 circuit at 100 percent. An OC-3 circuit at 23 percent bandwidth usage is the equivalent of a DS3 circuit at 70 percent, which is our technical consultant’s preferred maximum usage rate. As shown in Appendix C, 20 of the 31 OC-3 circuits in the High-Speed Network have less than 23 percent average maximum usage. As the K‑12 education community adds new According to our technical consultant, although the network academic content, clearly has spare bandwidth today on these 20 circuits, the the patterns of actual nature of the applications and content used on the network network usage and are factors that can affect the available bandwidth. It should growth will become be anticipated that the traffic carried on the network would clear, and appropriate increase greatly as the use of available applications and modifications can be academic content increases. Finally, as the K-12 education made at that time. community adds new academic content, the patterns of actual network usage and growth will become clear, and appropriate modifications can be made at that time. THE HIGH-SPEED NETWORK APPEARS TO BE UNDERUSED A major weakness in the development of the High-Speed Network is both CENIC’s and ICOE’s inability to determine how successful they have been in increasing network usage. ICOE’s efforts to increase usage, such as granting funds to Web site content developers, have been hampered by the variability in the level of state funding. In addition, although CENIC and ICOE have made significant efforts to ensure that every county office of education, school district, and school in the State has access to the High-Speed Network, 2,470 schools and school districts are still not connected. The Effect of CENIC’s Efforts to Increase Usage of the High-Speed Network Cannot Be Determined The primary goal of CENIC’s applications coordination team was to demonstrate how online resources could positively affect teaching and learning. To fulfill that purpose, CENIC worked to launch a K-12 education community Web site in September 2003. However, the California Department of Education (Education) raised concerns about CENIC’s process for reviewing Web site content to make sure the content aligned with state standards, which, according to CENIC, limited the amount of material on the Web site. 55 California State Auditor Report 2005-6 CENIC’s Applications Coordination Team Worked to Identify Available Educational Content for Network Use CENIC formed an applications coordination team composed of representatives from various entities, including school districts, county offices of education, and Education, in December 2000. Its primary responsibility was to focus on identifying academic content for the K-12 education community, with an emphasis on demonstrating how online resources could make a difference in teaching and learning. The team’s principal roles were to act as liaisons to key stakeholder groups, provide access to represented organizations’ The vision for the High-Speed Network resources, provide advice on the ways and means was to enrich learning and teaching by providing access to the following: of acquiring resources, consult with staff, and act as “evangelists” for the High-Speed Network. • Student instruction that uses distance learning programs such as advanced The applications coordination team began by using placement courses or advanced courses the database of the California Learning Resources otherwise not available statewide. Network (CLRN) to identify academic content • Education collaboration without borders providers. Education contracted with the Stanislaus that uses videoconferencing to facilitate County Office of Education, as the lead agency, interaction between teachers, students, and experts in different locations. to administer the CLRN project. The goals of the CLRN include conducting a review of electronic • Teaching resources that enable instruction, learning resources, such as online resources, such as model lessons, provided via video software, and video, for alignment with the on demand. content standards adopted by the State Board of • Professional development or staff training Education (state board). Another goal is to develop that uses videoconferencing, video and maintain a Web information link database, teaching demonstrations and Web-delivered staff development. which is a collection of free primary source, secondary source, and reference Web sites that • Student assessment tools that educators are accessible through a standards-based search can use to analyze and apply student function. The CLRN’s supplemental Electronic assessment data. Learning Resources Review Criteria and Process • Library resources such as online (review process), approved by the state board, encyclopedia resources, art and history includes matching resources with the California video and picture archives, and access to primary research resources like the content standards, with the CLRN’s minimum California Digital Library. requirements, and with its legal compliance. Publishers of electronic learning resources may • Higher-speed and better-quality Internet submit their resource for review on a continual access. basis. Additionally, visitors to the CLRN Web site can recommend a resource by filling out an Sources: Digital California Project Plan: K-12 Statewide Network date June 30, 2000; Corporation for Education electronic form. Network Initiatives in California. Applications coordination team members also spoke with other entities to identify content in areas outside of the scope of the CLRN project, such as professional development. For example, according to the California State Auditor Report 2005-6 55 director of statewide initiatives at CENIC, in addition to CLRN, it worked with the California Commission on Teacher Credentialing, the California County Superintendents Educational Services Association, the California State Library, California State Parks, individual county offices of education that may have developed resources, and private vendors. In 2001, the applications coordination team undertook several efforts to inform the K-12 education community about available online academic content material that could be significantly enhanced by delivery over the High-Speed Network. The team published a document titled Applications Sampler, created a bimonthly electronic newsletter, and conducted a workshop. CENIC’s Applications Coordination Team’s Major Effort Was the Establishment of a K‑12 Education Community Web Site The applications coordination team’s largest effort involved establishing a network portal to inform the K-12 education community about online learning resources. Network portals can be used to link users to content that is on, as well as off, the network. For example, according to its Web site, the New York State Education Department established a Virtual Learning System to encourage the use of the Internet as a tool for teaching and learning and to assist classroom teachers in locating resources for instruction, including sample tasks, learning experiences, and lesson plans. In February 2002, CENIC contracted with the Sacramento County Office of Education at a cost of roughly $211,000 to According to CENIC’s develop the Web site. In early May 2002, CENIC held a High- director of statewide Speed Network implementation meeting to discuss, among other initiatives, UC agreed to things, the portal features needed to help educators, parents, dedicate a portion of state and students locate high-quality online resources and available funds for the High‑Speed online products. According to CENIC’s director of statewide Network to the creation of initiatives, this meeting was held at the mutual agreement of a K‑12 Web site or portal CENIC and the University of California (UC). She also stated that would serve as a that in response to a request from the former administration, “one‑stop” location for UC agreed to dedicate a portion of state funds appropriated for access to online resources the High-Speed Network to the creation of a K-12 Web site or for educators and students. portal that would serve as a “one-stop” location for access to online resources for educators and students. CENIC held its first meeting to develop a K-12 Web site on October 22, 2002, and there were numerous participants, including representatives from the former administration, Education, the state board, the California Commission on Teacher Credentialing, the Office of 6600 California State Auditor Report 2005-6 the Secretary of Education, and California Community Colleges. According to CENIC, it held numerous follow-up meetings between October 2002 and February 2003 to work on the Web site. A number of topics were discussed during the meetings, such as the appropriate method for identifying potential content and the review and approval process for establishing links to the content. The Web site development process took almost a year. Specifically, according to CENIC, the Web site was launched in September 2003 and was intended to be featured as part of the My California Web site. It was designed to allow for easy access to high-quality online resources, with an emphasis on resources closely aligned to California’s academic content standards. The main Web page directed visitors to the CLRN database to identify licensed instructional resources as well as information on online courses for students, virtual tours and field trips, and professional development for teachers. The Web site also includes information about the High-Speed Network. However, according to CENIC, the Web site was not as robust as was initially envisioned. Education raised concerns regarding CENIC’s process for reviewing resources to ensure that they were in alignment with state academic content standards and regarding how the resources identified by the CLRN project would be identified on the Web site. Education also had concerns about the review of resources that were outside the scope of the CLRN project, such as professional development resources and online full courses of study. The state board expressed similar concerns regarding the need to provide content that was aligned to the State’s academic standards. Although outreach efforts such as the creation of the Web site are beneficial in increasing the K-12 education community’s awareness and access to various types of academic content resources and applications available on the High-Speed Network, CENIC did not have a CENIC did not have a method for measuring the success of method to measure the the High-Speed Network project because it did not track K-12 success of the High‑Speed application use. According to CENIC’s chief technology officer, Network because it did not it did not track the type of applications the K-12 education track K‑12 application use. community was using, because it was not practical to do so. He further stated that the technology available would not have allowed CENIC to identify the type of content a High-Speed Network user had accessed, but would merely indicate that a K-12 user had accessed a Web site or an application that uses a Web browser as an interface. For example, traffic monitoring would not have provided any detailed information to allow CENIC to California State Auditor Report 2005-6 66 distinguish between different types of Web-based applications such as a math tutorial or Internet research. Therefore, even if CENIC did track the types of applications that K-12 users were using, the data would not be meaningful. The chief technology officer stated that content providers are in the best position to capture these data. Until the State is able to receive these data, it will be difficult to determine whether the network accomplishes the Legislature’s goals for the High-Speed Network. CENIC Established a Team to Evaluate Methods for Distributing Content on the Network CENIC also established a multimedia E-content committee to focus on identifying and assessing The Multimedia E-Content technology that could facilitate the delivery and Delivery Team’s Major Efforts Included These Tasks management of educational data resources across the High-Speed Network. Members from its applications • Evaluation of currently available products coordination team and its network planning team from manufacturers of “content charged with the planning, implementation, and distribution” products. ongoing management of the High-Speed Network • Development of a white paper on assisted the committee’s E-content delivery team. E-content delivery technologies and The text box lists the major tasks to be performed by strategies for the High-Speed Network. this team. • Development of a list of frequently asked questions and other informational In CENIC’s initial plan for the High-Speed documents to be place on the CENIC Network, it envisioned placing 25 cache servers15 Web site. on the network to reduce the duplication of • If appropriate, development of a competitive workload resulting from having academic content request for proposals resulting in the in many places throughout the State and to implementation of one or more products support the hosting and delivery of network- or technologies. intensive materials such as video clips. However, • Exploration of large-scale data warehouse the cache servers were eliminated from the solutions for centralized storage, potential CENIC High-Speed Network budget for fiscal year mirroring of large data stores, and management of such resources. 2002–03. In a letter to UC dated June 7, 2002, the former president of CENIC stated that due to • Research on and development of Internet2 the reduction in state appropriations, it would protocols and standards for global peer have to eliminate, among other things, a cohesive network services. statewide system of content distribution services. The former president also stated that the lack of Source: E-content delivery team status report dated October 31, 2001. these servers would mean less efficient resource sharing across the State, resulting in more network traffic during the school day, when demand is at 15 A common network content delivery approach involves placing cache servers on a network. The cache servers use a special routing code that redirects a Web page request to the closest server. When a Web user clicks on a Web page that is content delivery enabled, the content delivery network reroutes the user’s request away from the site’s originating server to a cache server closer to the user. Other than faster loading times, the process is generally transparent. 6622 California State Auditor Report 2005-6 its highest, and higher costs at the local level. The costs would increase because users would have to access content through the commodity Internet. According to the current president and chief operating officer of CENIC, another reason for eliminating the servers was that it was much more difficult to identify the type of content to place on them than envisioned in the original plan. However, he further stated that CENIC’s April 2005 agreement with a corporation that provides content, streaming media and applications delivery services over the Internet lessens the impact of eliminating the servers in 2002. Specifically, the agreement allows the corporation to physically locate and operate 20 servers on CENIC’s CalREN backbone. As part of its tasks, the multimedia E-content delivery team was charged with evaluating available content distribution products. According to CENIC’s president and chief operating officer, at the conclusion of the evaluation the team recommended the substitution of content hosting for the original cache servers. He further stated that after submitting its recommendation, the multimedia E-content delivery team had fulfilled its purpose and was not called upon again. On March 28, 2003, CENIC released a request for proposals (RFP) to identify qualified providers of content hosting services. The scope of this RFP was limited to the provision of facilities, power, services, and possibly equipment to support the hosting of digital resources on servers physically located within the High-Speed Network for use by network users. Thus, owners and According to its president developers of K-12 content that was aligned with state standards and chief operating and regulations would be able to deliver their resources to the officer, CENIC is no longer California K-12 education community without the need to traverse responsible for application the commodity Internet. CENIC identified five hosting sites located issues after June 30, 2004, at the county offices of education in Orange County, Placer County, because oversight of the San Luis Obispo County, Shasta County and at UC. According to its network was transferred president and chief operating officer, although CENIC identified the to Education in fiscal hosting sites, it no longer was responsible for application issues after year 2004–05. June 30, 2004, because as we discussed in the Introduction, oversight of the network was transferred to Education in fiscal year 2004–05. Although ICOE Has Worked to Increase Awareness of Content, It Postponed Awarding Grant Funds to Develop Content Hosted on the High-Speed Network As lead education agency for the High-Speed Network, ICOE is responsible for technical oversight of the project, financial and administrative services, collaboration and coordination with other agencies and projects, and the advancement of network California State Auditor Report 2005-6 66 uses. It established an executive management team to provide internal oversight and management of the High-Speed Network. Team members include the county superintendents of schools for Imperial, Butte, and Mendocino counties, various High-Speed Network staff, and a representative from School Services of California Inc. Additionally, a representative from Education regularly participates in executive management team meetings. ICOE also established an application coordination committee (application committee) in February 2005 to serve as an advisory committee to its executive management team. Some of the goals One of the goals and objectives of the application committee are to coordinate and objectives of the and assist in ICOE’s outreach activities and conferencing and application committee collaboration projects, to identify sources of content and is to identify sources of high-impact applications that should reside on the network, content and high‑impact and to work toward the successful implementation of ICOE’s applications that should Advancing Network Uses Grant program. Given that ICOE has reside on the network. been the lead agency for the High-Speed Network since September 2004 and that the project did not receive any funding for fiscal year 2005–06, the activities of its application committee related to increasing the usage of the network appear reasonable. ICOE Established Its Own Videoconference Services at No Cost to Participating Schools Through the work of the conferencing and collaboration advisory committee, a joint subcommittee of the application committee and the network implementation committee, ICOE currently provides certain videoconferencing services at no cost to schools in California that are connected to the High-Speed Network. Videoconferencing is a tool that connects two or more locations with interactive voice and video. The services ICOE provides to schools are scheduling of calls, recording of conferences, streaming that allows participants to view any conference live on their computer, and multiport service that allows three or more locations to participate in the conference. According to ICOE, schools use videoconferencing for teacher- to-teacher, student-to-student, and class-to-class collaborations and instruction; administrative uses; professional development; and virtual field trips. Although CENIC has a videoconferencing program, in fiscal year 2004–05, ICOE formed a joint subcommittee called the conferencing and collaboration steering committee to evaluate CENIC’s services and implementation model against other options. According to ICOE, it felt that the CENIC videoconferencing program focused on the needs of higher education and was not sensitive to the significant scope and scale 66 California State Auditor Report 2005-6 of the implementation across the K-12 education community. Additionally, according to ICOE, CENIC’s annual fee was not commensurate with the K-12 education community’s use of the service. The committee recommended that ICOE develop its own model that provides functionality, support, and scalability for the K-12 education community. Thus, instead of paying CENIC’s fee of $420,000, ICOE used these funds to purchase equipment and supplies. ICOE stated that its K-12 users have access to high-quality centralized services, and it believes that the scheduling portion of its videoconferencing program will be fully operational and available to all The California Department of Education schools in January 2006. Plays a Key Role in Education Technology for the K-2 Education Community ICOE’s Web Site Includes Examples of Uses of the State law requires Education to administer High‑Speed Internet the California Technology Assistance Project, which provides a regionalized network of In November 2004, ICOE began operating its own technical assistance to schools, and school High-Speed Network Web site that includes links districts, on how to implement education technology. The project provides, among and information related to learning resources. other things, technical assistance and The application committee’s content delivery information to support access, planning, and use of high-speed telecommunications subcommittee is responsible for outreach activities networks by school districts and county offices such as developing the High-Speed Network Web of education. presence and collecting and distributing examples of exemplary network uses. As part of its Web site, State law also requires Education to provide statewide education technology services. ICOE provides links to the CLRN database, online Education provides these services through the courses such as the UC College Preparatory Initiative, following projects: and the California Digital Library. In addition, ICOE • California Learning Resources Network— created a Snapshots of Innovation Web page that We discuss this program on page 59. shows exemplary uses of the High-Speed Network to positively affect teaching and learning. For example, • EdTechProfile—Provides educational it highlights a program titled English for All, which administrators with tools to guide their decisions about how to integrate is a multimedia system designed to teach English technology into classroom instruction to adults and older adolescents. English for All was and how to evaluate effective teacher technology training programs. developed as part of the Cyberstep Project funded by the U.S. Department of Education. Additionally, ICOE • TechSETS—Provides technical support highlights a distance learning algebra course used by staff in California schools with access to two schools in its own county via videoconferencing. training, support, and other resources. • TICAL—Technology Information Center Finally, as part of its Web site, ICOE provides for Administrative Leadership—Helps information and links to other state education administrators such as principals or district superintendents find technology resources to technology programs. As shown in the text box, assist in the day-to-day needs of their jobs. Education administers several education technology projects. In this role, among other things, it provides Source: Education’s Web site. statewide coordination, planning, and evaluation of education technology programs and resources, as well as advancing the use of technology in the California State Auditor Report 2005-6 6655 curriculum and in the administration of elementary and secondary schools. According to ICOE, both its application committee and its network implementation committee have representatives from each of these statewide projects. Additionally, Education requires, as part of the statewide projects’ approved plans, each of the administrators of these projects to work with ICOE on the High-Speed Network project. ICOE Is Working to Broaden Content Applications on the High‑Speed Network According to Education, as the lead agency, ICOE is responsible for identifying useful administrative and classroom management tools and professional development and classroom content applications, coordinating and communicating the identified applications to the K-12 education community, and facilitating their placement on the High-Speed Network. Education actively participates on ICOE’s executive management team and the application committee and provides guidance related to inventorying high-impact applications for learning resources, professional development, and administration. Additionally, ICOE is evaluating some methods related to linking with academic content from the adopted materials, ICOE plans to evaluate electronic learning resources, and electronic learning assessment CENIC’s previous work resources reviewed by the CLRN as supplemental to and aligned related to identifying with the California content standards for placement on the hosting sites and High-Speed Network. For example, ICOE plans to identify and incorporate those sites work with academic content providers to develop strategies as appropriate. for placing their content on the network. ICOE also plans to evaluate CENIC’s previous work related to identifying hosting sites and to incorporate those sites as appropriate. ICOE Issued an RFP to Develop Further Online Programming but Did Not Award Grants in Fiscal Year 2005–06 ICOE created the Advancing Network Uses Grant program to support the development and sharing of applications and learning resources that meet the critical needs of California’s schools and that make good use of the benefits of the High-Speed Network. One requirement is that all of the resources funded through the program must be made available to all California public schools for a period of at least 18 months. In May 2005, ICOE released an RFP for the program. Funding was available for learning resources, professional development, and data and management tools. Applicants were evaluated based on common criteria, including the 6666 California State Auditor Report 2005-6 extent of the impact the application would have on the classroom learning environment and how the application meets school and district needs related to student learning and achievement. The ICOE application committee and network implementation committee reviewed the proposals, and its executive management team approved nine potential recipients. For example, one potential recipient in the learning resources category would work with the California Department of Parks and Recreation’s Parks Online Resources for Teachers and Students program. The potential recipient would develop a model unit of study that includes a videoconference lesson enhanced by the use of an underwater remotely controlled vehicle located on the sea floor of Whales Cove at Point Lobos State Reserve. Students and teachers in the classroom would ICOE did not award be able to manipulate the movement of the vehicle and thus grant funds in fiscal year control what the camera captures. 2005–06 as planned because it was uncertain as The nine potential recipients would have received a total of to whether the High‑Speed roughly $650,000; however, ICOE did not award the grant funds Network would receive in fiscal year 2005–06 as planned because it was uncertain as to state funding in fiscal whether the High-Speed Network would receive state funding year 2005–06. in fiscal year 2005–06. Instead, it decided that all available funds should be used to ensure that the High-Speed Network remained operational. According to ICOE, should state funds be appropriated in the future, and provided enough funding exists, it will award funds to the winners of that previous grant competition. ICOE Is in the Early Stages of Developing a Suitable Plan for Evaluating the Success of the High-Speed Network Although Education requires administrators of certain education technology projects to work with ICOE on the High-Speed Network project, ICOE is in the early stages of developing a method to evaluate the statewide success of the High-Speed Network. According to ICOE, it is working closely with Education to obtain existing data from certain education technology projects and is evaluating these data to determine if they will assist it in tracking the types of applications the K-12 education community is using. Establishing a method to track K-12 network use is key to measuring the success of the High-Speed Network project. As we discussed previously, ICOE released an RFP for its Advancing Network Uses Grant program. As part of the application process, applicants were asked to briefly describe the extent of the impact their proposal would have on the classroom California State Auditor Report 2005-6 66 learning environment. Further, applicants were asked to include information as to whether the applications and academic content were already in use, as well as any data that had been collected to demonstrate their effectiveness. ICOE stated that if it had received state funding in fiscal year 2005–06, it would have been able to report on the specific impact of the applications and academic content used by the nine potential recipients of the grants. Additionally, according to ICOE, it is working with Education to obtain data from Education’s other education technology According to ICOE, it is projects, such as the Online Classroom Pilot Program and working with Education the Enhancing Education Through Technology Competitive to obtain data from Grant Program. Both of these programs include a requirement Education’s other to monitor and to report the impact they have on teaching education technology and learning. For example, the Enhancing Education Through projects. However, because Technology grants offer funding to schools for the technology the types of data that they use in the classrooms to improve students’ academic ICOE could obtain from achievement. In order to receive grant funding, each applicant these projects are limited must submit a technology plan. The plan should include, among to the specific projects and other things, a description of teachers’ and students’ current grant awardees, they do access to technology tools and of the school districts’ current use not provide a complete of hardware and software to support teaching and learning. picture of the statewide use Additionally, the plan should include a list of clear goals and a of the High‑Speed Network specific implementation plan for using technology to improve by county offices of teaching and learning by supporting the school district’s education, school districts, curricular goals and academic content standards. Finally, the plan and schools. should include a description of the process the applicant will use to monitor the results. However, because the types of data that ICOE could obtain from these projects are limited to the specific projects and grant awardees, they do not provide a complete picture of statewide use of the High-Speed Network by county offices of education, school districts, and schools. ICOE is in the early stages of establishing a process to measure the statewide success of the High-Speed Network project. Each year, Education conducts the California School Technology Survey (survey) to gather information to measure the progress of technology integration in California classrooms. Completion of the survey is a requirement for several education technology projects. According to ICOE, its application committee worked with Education to include a new section in the 2005 and 2006 surveys. For example, school districts are asked to identify their technology uses such as videoconferencing, online student courses, and virtual field trips, as well as the frequency of the use. ICOE stated that it has reviewed only the data collected from the 2005 survey and plans to use these data and the 2006 66 California State Auditor Report 2005-6 survey data to assist in its plans to measure the success of the High-Speed Network project. ICOE also stated that it would continue to work with Education to use other existing data collection systems, to modify those as necessary, and to create a system that provides information for decision-making at multiple levels. Moreover, ICOE plans to hire a consultant to assist with the development of a process to measure the success of the High-Speed Network project. Finally, as we discussed previously in this chapter, CENIC’s chief technology officer stated that content providers are in the best position to capture data about the type of applications the K-12 education community is using. According to ICOE, as part of their contracts with California school districts and county offices Until ICOE establishes a of education, some academic content providers can provide process to measure the detailed tracking information. ICOE is currently working with success of the High‑Speed one academic content provider to track the frequency and type Network that includes of academic content the county offices of education, school tracking the type of districts, and schools use. Preliminary data suggest that the applications the K‑12 frequency of their use increased from roughly 12,100 views education community is to 1.5 million views between calendar years 2001 and 2005. using, and the Legislature According to ICOE, it plans to work with additional academic establishes clear goals for content providers that are widely used throughout the State to the program, it is difficult gather the same type of data. Until ICOE establishes a process to determine whether the to measure the success of the High-Speed Network that includes network has achieved tracking the type of applications the K-12 education community the Legislature’s goals. is using, and the Legislature establishes clear goals for the program, it is difficult to determine whether the network has achieved such goals. Connecting the Remaining Schools Will Expand the Usage of the High-Speed Network Both CENIC and ICOE have made an effort to increase the usage of the High-Speed Network by assisting schools and school districts in connecting their LANs to existing node sites, which is commonly referred to as the last mile connection. In November 2001, CENIC established a last mile grant program to ensure that California schools had DS1 or better circuit bandwidth so that they could connect to the High-Speed Network and the commodity Internet. According to CENIC, it awarded 28 grants during fiscal 2002–03 to help defray the schools’ and school districts’ expenses associated with connecting to the High-Speed Network node sites. In total, recipients of CENIC’s last mile grants received almost $1.8 million over a three-year period. California State Auditor Report 2005-6 66 In some cases, the grants helped to increase the circuit bandwidth by providing funds for the school districts or schools to purchase DS3 circuits. For example, one school district was able to upgrade from a DS1 connection to a DS3 circuit, which it stated would then allow it to use streaming media and download large files. For fiscal year 2003–04, CENIC awarded last mile grant funds to 25 school districts for a total of 35 school sites. For fiscal year 2004–05, ICOE solicited applicants in April 2005 and selected 48 potential recipients in June 2005. However, in June 2005, given the uncertainty of the fiscal year 2005–06 budget, it decided to table the awarding of $1.1 million in last mile grants. Also on November 1, 2005, ICOE issued its report titled Connecting California’s Children: A Status of Connectivity to California Schools. It In November 2005, ICOE found that progress has been made every year on connecting the reported that although unconnected schools and school districts. According to ICOE, all 2,470 schools and 58 county offices of education are connected to the High-Speed school districts are still Network. In addition, 887 school districts, and 7,039 schools not connected to the serving 4,792,263 of California’s students are connected. Although, High‑Speed Network, 2,470 schools and school districts are still not connected to the roughly 1,960 are High-Speed Network, roughly 1,960 are connected to the Internet connected to the Internet using commercial Internet service providers. ICOE estimated using commercial that it would cost roughly $10 million to connect the remaining Internet service providers roughly 500 schools and school districts without any connection. and that it would cost It indicated that the reasons frequently cited by the schools and roughly $10 million to school districts for not connecting to the High-Speed Network are connect the remaining their obligations under existing long-term telecommunications roughly 500 schools contracts and financial concerns. It further stated that when funds without any connection. become available, it will determine how best to proceed with the last mile grant program. RECOMMENDATIONS To maximize the benefits of the High-Speed Network, the California Department of Education should ensure that ICOE does the following: • Continue its efforts to implement statewide videoconferencing. • Continue the efforts of its application committee to identify academic content and application uses to place on the High-Speed Network. 00 California State Auditor Report 2005-6 • Continue with its plans to fund the Advancing Network Uses Grant applicants. • Proceed with its last mile grant program. Education should ensure that ICOE develops a process to measure the success of the High-Speed Network that incorporates, but is not limited to, the following: • Continuing its efforts to gather data from the California School Technology Survey, as well as data collected as a part of other statewide education technology projects. • Hiring a consultant to assist with the development of a process to measure the success of the High-Speed Network. • Continuing to work with academic content providers to obtain statewide data on the K-12 education community’s frequency and type of academic content use. n California State Auditor Report 2005-6  Blank page inserted for reproduction purposes only. 22 California State Auditor Report 2005-6 CHAPTER 4 No Technical or Financial Reasons to Abandon the Existing High-Speed Network Are Apparent CHAPTER SUMMARY By leveraging its buying power and sharing costs with institutions of higher education through the Corporation for Education Network Initiatives in California (CENIC), the State likely spent less to build and operate the California K-12 High-Speed Network (High-Speed Network)16 than it would have spent for a similarly designed but separate network for the K-12 education community. Although we could not quantify these savings because the State would have to go through a bid process to price the various alternatives, it is apparent that CENIC has been able to negotiate competitive prices for network components such as fiber-optic cable and equipment. Further, our technical consultant compared the current High-Speed Network architecture to other alternatives, including a virtual private network. Our consultant found no compelling technical or financial reason to abandon the existing High-Speed Network. CENIC SUCCESSFULLY NEGOTIATED COMPETITIVE PRICES THAT RESULTED IN LOWER COSTS FOR THE HIGH-SPEED NETWORK By leveraging the State’s buying power, CENIC has been able to negotiate substantial discounts on network components such as fiber-optic cable for its California Research and Education Network (CalREN) backbone and circuits and equipment. If the State had chosen to use a virtual private network or to construct a private network specifically for the K-12 education community, it would likely have paid more. 16 The California K-12 High-Speed Network was originally named the Digital California Project: K-12 Statewide Network when it was started in fiscal year 2000–01. In fiscal year 2004–05, when stewardship of the network was given to the Imperial County Office of Education, the aspects of the network applicable to K-12 participants were then titled the K12HSN. We call it the High-Speed Network throughout this report. California State Auditor Report 2005-6  CENIC Obtained Competitive Pricing Arrangements for Circuits CENIC leases circuits from several service providers. These circuits connect its CalREN backbone hubs to node sites and connect node sites to each other. In Chapter 3 we discussed the specific circuits that CENIC leases. Most are either DS3 or OC-3 circuits. Our consultant compared the rates paid by CENIC to the schedule of rates published by one of its service providers and to the California Integrated Information Network, CALNET Master Agreement (CALNET). The CALNET, which serves state agencies and qualified local government agencies, is the result of a Department of General Services contract with two service providers on behalf of the State of California to provide a comprehensive array of telecommunications services to public entities throughout the State. It consists of state-of-the- art architecture that supplies the telecommunication transport necessary to provide critical communication and data services throughout California, including health and human services and public safety services such as the 9-1-1 program. Our consultant found that CENIC’s contracted rates are substantially lower than the service provider’s published five-year Our technical consultant term rates and are roughly 1 percent higher than the rates in the found that CENIC’s CALNET agreement. Given the volume of the CALNET contract contracted rates for ($300 million annually) and the duration of the agreement (seven circuits are substantially years with an option for three additional years), our consultant lower than one service believes it is reasonable that the CALNET agreement would contain providers published slightly lower rates. In addition, the CALNET agreement includes a five‑year term rates. clause stating that the “Contractor agrees that no other customer of [its two service providers], collectively or as individual companies, will receive better rates for a substantially similar suite of services offered under substantially similar terms and conditions when the volume of business from the other customer is equal to or less than the volume of business the State delivers under this Agreement.” Our consultant also compared pricing for selected circuits using CENIC’s invoices. A comparison of (1) the actual invoice amount, (2) the CENIC contract rate, (3) the CALNET agreement rate, and (4) the service provider’s published schedule of rates for a five-year agreement was made. The comparison for 53 DS3 circuits found that the amount shown on CENIC’s invoice was the lowest for 35 of the 53 circuits. For the remaining 18 DS3 circuits, the comparison found that the CALNET agreement rate was the  California State Auditor Report 2005-6 lowest. Similarly, the comparison for 12 OC-3 circuits found that the amount shown on CENIC’s invoice was the lowest for 7 of the 12 circuits and the CALNET contract rate was the lowest for the remaining 5 circuits. We cannot explain the variances between CENIC’s contract rates and the actual invoice. Nevertheless, it appears that CENIC has contracted for and received very competitive rates. CENIC Negotiated Competitive Prices for Its CalREN Backbone The primary components of CENIC’s CalREN backbone are related to its purchase of fiber-optic cable as well as the routers and the technology it uses to increase the bandwidth of a strand of fiber-optic cable by using different colors within the light spectrum to create multiple wavelengths. According to our technical consultant, CENIC has been able to obtain competitive pricing for its fiber and equipment. In January 2003, CENIC began deploying its fiber-based CalREN backbone. The CalREN backbone is made up of a single fiber pair and has three tiers. As of September 2005, CENIC was using its fiber-optic technology to operate two tiers by creating two different wavelengths for its two standard networks, the CalREN Digital California Network and the CalREN High-Performance Research Network. As shown in Figure 5 on the following page, the High-Speed Network uses only one tier, the CalREN-Digital California Network, which it shares with the institutions of higher education. Between April 2002 and July 2002, CENIC entered into agreements to purchase exclusive 20-year indefeasible rights to use (IRU) dark fiber—unactivated, or unlighted, fiber that is provided without electronic and/or optronic equipment—for the CalREN backbone from two telecommunications carriers. An IRU is an exclusive, nonrevocable right to use the specified property but does not convey title, ownership, or rights of possession of any real or personal property. According to CENIC’s accounting records, during fiscal years 2002–03 through 2004–05, it paid nearly $4.5 million for the fiber IRUs, which CENIC is amortizing over a 20-year period. CENIC also has to pay maintenance costs for this fiber (according to CENIC’s accounting records, this amount was more than $400,000 in fiscal year 2004–05). California State Auditor Report 2005-6 55 FIGURE 5 CalREN’s Three Tiers D E e N x v p e e l e t o w r p im o m r e k e n n t t a a l/ l r N es e e t a w r o ch rk er T s ier 1 Tier 2 R H es ig e h ar P ch er N fo e rm tw a o n r c k e Large ap u p se li r c s ations Tier 3 Digital California Network res A e l a l r K c – h 2 /e 0 d C u a c l a if t o io rn n i a users Network Type Users Source: CENIC. Note: Tier 1 is used by network researchers at sites such as the University of California Institute for Science and Innovation, the University of Southern California and its Information Sciences Institute, Stanford Linear Accelerator Center, and other major network research entities that collaborate with these researchers in California. Tier 2 connects to the Abilene network, which is an Internet2 high-performance backbone network. The High-Speed Network shares in CENIC’s participation fees and has access to Internet2. According to CENIC, the routing of traffic to Internet2 is done automatically. Our technical consultant’s comparison of the costs paid by CENIC for dark fiber to the costs for dark fiber applicable at about the time that CENIC negotiated its agreements found that CENIC obtained competitively low prices. According to our technical consultant, Our technical consultant’s because a published schedule of rates or charges does not exist, each comparison of the costs purchase is a new negotiation between the dark fiber provider and paid by CENIC for dark the purchaser. However, two sources provide insight into dark fiber fiber to the costs for prices. In 2001, the Phillips Group published a study titled Dark dark fiber applicable at Fiber USA – Technologies, Players, and Pricing in the USA Fiber Markets. about the time that CENIC According to our technical consultant, at the time of the 2001 negotiated its agreements report, the Phillips Group was an international telecommunication found that CENIC obtained consultancy composed of practices and research groups in law and competitively low prices. regulation, E-commerce, Internet, mobile, and pricing. The Phillips Group now operates under the name Infotech. The study found that dark fiber pricing was generally based on a 20-year IRU, that recent 66 California State Auditor Report 2005-6 industry transactions varied from $.93 to $6.20 per meter per fiber, and a price of $1.50 to $2.00 per meter per fiber could be used as a guide for standard pricing. Also, in March 2002 CANARIE Inc., a Canadian not-for-profit corporation that has developed high-speed networking throughout Canada, suggested that for budgetary purposes, costs of $2 to $3 per meter per strand pair can be used for a 20-year IRU for existing dark fiber. The study found that the prices were the same whether it was Canadian or U.S. dollars. According to CENIC, its CalREN backbone has approximately 4,300 miles of dark fiber at a cost of $4.5 million, which equates to 64 cents per meter per fiber.17 Based on this estimate, our technical consultant concluded that CENIC was able to obtain very competitive prices. Although CENIC spent $18.8 million on backbone equipment and equipment maintenance during fiscal years 2002–03 Our technical consultant through 2004–05, according to its accounting records, our concluded that CENIC technical consultant concluded that it obtained the best prices obtained the best prices that could have been expected. CENIC negotiated prices that on backbone equipment were a percentage discount off the vendor’s then current list and equipment prices. According to our technical consultant, many states, maintenance that could through state government or state university systems, have have been expected. statewide pricing agreements for equipment with the same vendor. Frequently, the negotiated discounts also extend to county and municipal governments within the states. However, our technical consultant found CENIC’s percentage discounts for equipment purchases to be substantially higher than other statewide pricing agreements. Also, CENIC’s percentage discount for equipment maintenance was higher. IT IS NOT LIKELY THAT ALTERNATIVES TO THE HIGH-SPEED NETWORK WOULD RESULT IN LOWER COSTS TO THE STATE The Budget Act of 2004 required the Imperial County Office of Education (ICOE), the lead county office of education responsible for administering the High-Speed Network, to contract with an independent consultant to report on, among other things, an estimate of the costs to provide the 17 CENIC’s agreements for dark fiber are confidential. Thus, to estimate the cost per meter per fiber, we converted the number of miles of fiber purchased by CENIC into meters. We then divided the cost of the dark fiber as reflected in CENIC’s accounting records by the number of meters. California State Auditor Report 2005-6  High-Speed Network’s current level of service through private vendors.18 MGT of America, Inc. (MGT) was the independent consultant selected, and it issued its report, titled Performance Evaluation of the K-12 High-Speed Network, on March 1, 2005. MGT concluded that it is impossible to conduct a direct comparison of the cost of staying with CENIC versus the cost of switching to a different service provider without issuing an RFP to solicit competitive bids. Instead, MGT presented in its report the factors that would drive a cost comparison, a preliminary estimate of the cost-effectiveness of CENIC’s services, and options and issues to consider when making a decision regarding which entity should provide K-12 education community network services in the future. Specifically, MGT reported on two possible alternatives: contracting for a virtual private network (VPN) and constructing a new K-12 network. A VPN, which provides customized connectivity via a shared public network infrastructure, would have a major impact on the delivery of services to the K-12 education community. Either a VPN or a newly built private network would likely cost the State more than the current High-Speed Network. Using a Virtual Private Network Would Significantly Change the K-2 Education Community’s Service According to our technical consultant, a VPN is a private data network that makes use of the public telecommunication infrastructure (the Internet), maintaining privacy through the use of a tunneling protocol and various security procedures. Customer access lines into the Internet may be digital subscriber lines (DSL), cable modems, or higher-speed lines that we discuss in Chapter 3, such as DS1 or DS3 circuits. These access lines 18 The Budget Act of 2004 states that if an audit of the High-Speed Network is not approved by the Joint Legislative Audit Committee by August 31, 2004, up to $300,000 of the total funding provided shall be used by the lead agency to contract with an independent consultant selected by the lead county office using competitive procurements in consultation with the Department of Finance and the Legislative Analyst’s Office. These independent consultants shall report to the fiscal and policy committees of the Legislature and to the Department of Finance by March 1, 2005 on the following: (a) a financial audit of the K-20 Internet system currently administered by CENIC, including the components serving the higher education segments, including an assessment as to the relative shares of cost borne by the various groups and alternative ways of assessing the costs in a fair manner, a detailed budget for the 2004–05 fiscal year, and detailed expenditure information on the Digital California Project since its inception. The audit shall identify any multi-year commitments that exist and any assets owned by the State, or any other public agency, or any nonprofit corporation in connection with the Digital California Project, (b) long-term projections of likely types of use and impacts on capacity usage and future costs, (c) its cost-versus-benefit analyses of current common K-12 uses including both academic and administrative uses, which include an estimate of the cost to provide the current service level through private vendors (d) identification of problems, and (e) recommendations for addressing problems.  California State Auditor Report 2005-6 can be obtained from any source. In the case of the High-Speed Network, access lines would probably be obtained from the VPN provider to ensure a higher level of reliability and accountability. If a VPN were used to replace the current High-Speed Network, a traditional Internet-based VPN could be provided. However, If a VPN were used our technical consultant expressed concern about the inherent to replace the current lack of reliability in this approach, because some applications High‑Speed Network, a will not tolerate the uncertainties of the commodity Internet. traditional Internet‑based For example, most applications that would support the use of VPN could be provided. videoconferencing, streaming media, or the transfer of large data However, our technical files would require the types of packet prioritization and other consultant expressed quality-of-service features that would be difficult to guarantee concern that certain on a traditional VPN. If a VPN were pursued, our technical applications will not consultant would envision one that uses multi-protocol label tolerate the uncertainties switching, a protocol that enhances the speed and performance of the commodity Internet. of Internet protocol networks. Each data packet is given a label that identifies the route that packets will take as they traverse the network, based on predetermined criteria such as bandwidth requirements, performance, or quality-of-service requirements, and network congestion. Additionally, according to our technical consultant, several large outsourcing firms, such as IBM Global Services and Perot Systems Corporation, in some cases develop their own private networks and provide networking services to their clients. Figure 6 on the following page depicts a simple VPN. According to our technical consultant, for practical purposes, in a VPN environment, the service provider network in Figure 6 on the following page would simply replace CENIC’s CalREN backbone. However, the actual configuration would almost certainly be significantly different than the CalREN design. For example, the service provider’s Internet provider network does not identify specific hub locations or fiber routes between hub and customer locations. Once a customer site is linked to the service provider’s Internet provider network, data may be routed on the network based on resource availability and other parameters that are at the discretion of the service provider. California State Auditor Report 2005-6  FIGURE 6 A Simple Virtual Private Network Internet Customer Customer site 1 site 8 Customer Customer site 2 site 7 Service provider Internet Protocol network Customer Customer site 3 site 6 Customer Customer site 4 site 5 Source: Bureau of State Audits’ technical consultant. This is in contrast to the CalREN backbone, which is based on the specific dark fiber routes CENIC was able to obtain through IRUs. In a VPN environment, a service provider would have its own routes available for customer sites to connect to its network. In the current High-Speed Network, school districts typically aggregate their traffic up to nodes at their respective county offices of education, and the nodes connect to the CalREN backbone using circuits leased from the local exchange carrier. In a VPN, according to our technical consultant, the current node locations would change substantially. School districts would be able to connect directly to the VPN service provider at a more local level, since the major VPN service providers have more widely distributed access points. The physical connections from a customer site to the service provider VPN could 00 California State Auditor Report 2005-6 be as local as the nearest telephone company’s central office. The leased circuit configuration and associated costs would also change significantly. Further, routing variations would lead to changes in the bandwidth of access circuits. This contrast between the VPN environment and the current CENIC design simply represents a difference in network architecture and does not favor a VPN over the current design. A true VPN is a completely outsourced, managed service. The service provider’s responsibility is to deliver a specific level of performance, but not necessarily specific equipment or traffic routes. Typically, fewer customer resources are required to design and manage the network, monitor performance, and respond to outages or maintenance problems. The responsibility for the management and performance of the VPN belongs to the service provider, and the expectations for performance (and the penalties for nonperformance) are detailed in service-level agreements, which are discussed in Chapter 1 and Appendix B. MGT Appears to Prefer a Virtual Private Network for the K-2 Education Community In its report, MGT recommended that ICOE conduct or commission research to define and specify K-12 wide-area data communication requirements for the future. Further, it recommended that, based on the requirements, ICOE should prepare and release a request for bid for a VPN. Thus, a VPN appears to be MGT’s preferred alternative to the High-Speed Network. Our technical consultant had a few concerns regarding MGT’s statements. First, MGT suggested that a VPN provider would “probably continue, at least in the near term, to use the node edge devices that CENIC has stated it would release to K-12, and to use the same access facilities and technology.”19 Our technical consultant reviewed the offerings of several VPN providers and noted that most will assume management responsibility of customer-owned routers, but with qualifications. However, as we discussed in Chapter 2, certain equipment is reaching obsolescence, so there is no guarantee that the equipment could be reused in a VPN. Perhaps the promise of a long-term agreement with such a high-visibility customer would prompt a VPN provider to support the existing equipment, but this would be the subject of contract negotiations. 19 The node edge devices referred to by MGT are the routers and associated equipment located at each node site. For purposes of our report, we refer to the edge devices as equipment. California State Auditor Report 2005-6  MGT also stated, “Given CENIC’s cost structure, it is likely that for the first few years, a VPN service could be provided at a price equal to or lower than the current K-12 contribution to CENIC.” According to our technical consultant, one advantage of VPN pricing is that the up-front cost is normally relatively low when compared to other options because the fiber is normally already in place, and the customer can lease the network equipment for the term of the service agreement. Also, a new VPN supporting only the K-12 education community would be entirely eligible for federal telecommunications services rebate discounts ranging from 20 percent to 90 percent. However, attempting to develop an estimate of the cost of a VPN solution is difficult. Thus, we cannot conclude that a VPN service could be provided at a cost equal to or lower than the K-12 funding CENIC has received for the project. As our technical consultant pointed out, the physical network shape and configuration would For a network of the size be significantly different than the CalREN backbone. Further, for and complexity of the a network of the size and complexity of the High-Speed Network, High‑Speed Network, the actual cost is dependent upon contract negotiations. The costs the actual cost of a of a VPN include the physical and technical network components VPN is dependent upon (circuits and routers, for example) plus the costs to monitor, manage, contract negotiations. and maintain the network, including the desired service-level agreement terms. According to our technical consultant, a VPN designed with comparable capabilities and bandwidth is the most expensive of the feasible alternatives, based on total life cycle costs. In a specific recent example, the cost for a VPN solution running Gigabit Ethernet from a Regional Bell Operating Company was nearly five times the cost of building a private network, when the projected 10-year life cycle costs were calculated. MGT stated, “It appears likely that over time, as the K-12 demand for and the cost of bandwidth rises, participation in CalREN will increasingly become the least costly alternative for HSN [High-Speed Network] access.” Our technical consultant assumed that the VPN envisioned by MGT would be a lower-bandwidth VPN, based on (1) MGT’s statement that the backbone network is “over-engineered for current levels of usage and therefore likely carries a higher cost in the short term than could be obtained by switching providers,” and (2) the fact that the VPN would support only K-12 traffic. It is not clear to our technical consultant how the K-12 education community might take advantage of perceived lower VPN costs in the short term and avail itself of CENIC’s network in the future. Moreover, as we discussed in Chapter 3, our analysis of network traffic showed higher usage than MGT’s analysis did. 22 California State Auditor Report 2005-6 Constructing a New Network Does Not Appear to Be a Sensible Alternative Although not recommended, MGT’s report also considered the construction of a new K-12 network as an alternative to the High-Speed Network. According to MGT, the K-12 community could choose to construct its own private network, similar to CENIC’s, with leased capacity rather than leased fiber. MGT also Building a private network stated that this would be extremely costly and would involve assumes that the entity a significant reinvestment of funds after the State has already begins from scratch contributed to constructing a statewide network for K-12 and constructs its own usage. MGT uses the term “construct its own private network,” network. This alternative but we do not believe that it is proposing the actual physical would be extremely costly construction of a private network. Building a private network and involve a significant assumes that the entity begins from scratch and constructs its reinvestment of funds own network. According to our technical consultant, the entity after the State has already would need to obtain rights-of-way to install its fiber-optic cable. contributed to constructing These rights-of-way may come in the form of pole attachment a statewide network for agreements negotiated with the telephone or electric utilities or K‑12 usage. routes following water utilities, gas pipelines, railways, or major highways. Government entities typically have rights-of-way, or ready access to them. For example, municipal governments typically have access to space on utility poles by regulation. They also normally negotiate conduits or space in conduits from cable television franchisees as part of a franchise agreement. If needed, they may also negotiate fiber strands and network services as part of such franchise agreements. Also, state governments, through their departments of transportation, have access to rights-of-way by virtue of their control over the roadways. Fiber routes are frequently negotiated along major highway corridors through the use of shared resource agreements with carriers. For entities with no such access, rights-of-way may take several months to negotiate. Once they are secured, the entity would then contract for the construction of the fiber infrastructure. The entity would own the fiber outright and could use it for any purpose, including reselling strands or wavelengths to other users. In certain cases, there would be regulatory implications to such resale of services. The entity would obtain separately and own all necessary components to complete the implementation of the network, and to manage and operate it. MGT defined this alternative as “leased capacity rather than a leased fiber.” Our technical consultant assumed that the term “leased capacity” refers to leasing lit fiber wavelengths, and that the network equipment necessary to create and define the California State Auditor Report 2005-6  bandwidth would be included in the lease. This is in contrast to leasing fiber, which would suggest that it would then be the responsibility of the K-12 education community to acquire the network equipment separately. Nonetheless, our technical consultant does agree with MGT’s assessment that implementing this alternative would be extremely costly and involve a “significant reinvestment of funds after the State has already contributed to constructing a statewide network for K-12 usage.” In general, it appears that MGT’s approach would involve outsourcing as much of the network as possible, assuming that CENIC would no longer be involved. Our technical consultant agrees that if CENIC is not involved, no entity within the current structure is capable of assuming management responsibility of the network. Therefore, if it is decided to terminate the relationship with CENIC, our technical consultant would agree that an Our technical consultant outsourced, managed solution would be appropriate. does not see a compelling technical However, our technical consultant does not see a compelling or financial reason to technical or financial reason to abandon the existing High-Speed abandon the existing Network. The network design is based on industry standards. It High‑Speed Network. has adequate bandwidth to support the current traffic and can be easily and inexpensively upgraded with additional bandwidth as traffic increases. CENIC obtained the components (fiber, network equipment, and circuits) at very competitive prices. Thus, our technical consultant believes CalREN will prove to be the least expensive choice in the long term.  California State Auditor Report 2005-6 We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, ELAINE M. HOWLE State Auditor Date: January 31, 2006 Executive Staff: Philip J. Jelicich, CPA, Deputy State Auditor Sharon Reilly, Esq., Chief Legal Counsel Donna L. Neville, JD, Senior Staff Counsel Staff: Joanne Quarles, CPA, Audit Principal Peter A. Foggiato III, CPA Heather Kopeck Sang Park Technical Consultants: RCC Consultants, Inc. Peter Berry, RCDD, Managing Director Jeanne Fleming, Senior Consultant Legal Consultant: Technology Protection, Development, Capitalization, and Commercialization Ken Murray Jr., JD California State Auditor Report 2005-6 55 Blank page inserted for reproduction purposes only. 66 California State Auditor Report 2005-6 APPENDIx A Glossary of Terms Backbone—The high-traffic density connectivity portion of a communications network. In the California K-12 High-Speed Network (High-Speed Network), the backbone is the main wire that connects the hub sites together. Bandwidth—(1) A range within a band of frequencies or wavelengths; (2) the amount of data that can be transmitted in a fixed amount of time. For digital devices, the bandwidth is usually expressed in bits per second (bps) or bytes per second. Bit—Short for binary digit, the smallest unit of information in a computer. A bit has one of two values: 0 or 1. Byte—In most computer systems, a byte is a unit of data that is eight bits long. Circuit—The path between two terminals over which one-way or two-way communications may be provided. In the High-Speed Network, a circuit is the main wire that connects node sites to each other or connects a node site to a hub site. Dark fiber—Refers to unused fiber-optic cable or fiber provided without electronic and/or optronic equipment to “light up” the fiber and transmit data. Fiber optics—A technology that uses glass (or plastic) threads (fibers) to transmit data. A fiber-optic cable consists of a bundle of glass threads, each of which is capable of transmitting messages modulated onto light waves. Fiber-optic cables have a much greater bandwidth than traditional metal cables; thus, they can carry more data. Gbps—Short for gigabits per second, a measure of data transfer speed that equals one billion bits. Hub—(1) A distribution point in a network; (2) a device that accepts a signal from one point and redistributes it to one or more points. California State Auditor Report 2005-6  Internet—A global network connecting millions of computers that is decentralized by design. Each Internet computer, called a host, is independent. Its operators can choose which Internet services to use and which local services to make available to the global Internet community. Internet2—A national initiative sponsored by the University Corporation for Advanced Internet Development to connect higher education institutions to each other using its Abilene network backbone. Internet service provider—A company that provides Internet access to companies or individuals. Kbps—Short for kilobits per second, a measure of data transfer speed that equals 1,000 bits. Local area network (LAN)—A computer network that spans a relatively small area. Mbps—Short for megabits per second, a measure of data transfer speed. One megabit is equal to one million bits. Network operations center (operations center)—The physical space from which a typically large telecommunications network is managed, monitored, and supervised. The operations center coordinates network troubles; provides problem management and router configuration services; manages network changes; and monitors the routers, switches, and hubs that keep the network operating smoothly. Node—A terminal of any branch of a network or an interconnection common to two or more branches of a network. In the High-Speed Network, the county offices of education typically serve as node sites that connect schools and school districts within the county to a hub site or to another node site. OC—Short for optical carrier, used to specify the speed of fiber-optic networks conforming to the SONET standard. Ring network—A topology of computer networks in which each user is connected to two other users, so as to create a ring. Router—A device that forwards data packets along networks. SONET—Short for synchronous optical network, a standard for connecting fiber-optic transmission systems.  California State Auditor Report 2005-6 APPENDIx B Key Service-Level Agreement Terms and Conditions That the California Department of Education Should Require Lead Agencies to Include in Contracts Relating to the California K-12 High-Speed Network Project Our legal consultant recommends that the service-level agreements with lead agencies contain certain key terms and conditions. The Imperial County Office of Education (ICOE) is the lead agency responsible for overseeing the California K-12 High-Speed Network Project (High-Speed Network). We believe the California Department of Education should require ICOE to include the following elements relating to service-level agreements as addenda to its contract. Moreover, the K-12 education community should be made a beneficiary of the service-level agreement (SLA) in the contract, although ICOE maintains responsibility for enforcing the SLA’s. 1. Introduction—Identification of parties to the SLA, the need for the SLA and the application or services the SLA supports. 2. Customer (K-12 education community) Requirements— How the customer will use the service and what requirements the customer has established for the service. 3. Service Overview—Description of the service, location of physical and logical interfaces between the parties, ownership, and any other information required to describe the service or product adequately. 4. Term—Period of validity for the SLA. 5. Responsibilities—Detailed description of the responsibilities of ICOE and the provider. This key section should clearly define the expectations of both parties in as much detail as required to satisfy ICOE’s expectations for performance and service. California State Auditor Report 2005-6  6. Service Details—Description of key quality indicators, enumerated later, associated with the service and the monitoring parameters as they will be reported to ICOE, including levels of acceptable performance and nonconformance and out-of-specification conditions. 7. Exceptions—Exceptions to service must be clearly documented in the SLA (for example, downtime for upgrades, routine maintenance or outages). 8. Sampling and Reporting—How often and in what form reports will be provided to ICOE. Sample report forms should be agreed on and included within the SLA document. 9. Penalties—Penalties for nonconformance should be detailed, since performance by a service provider may degrade, and since it is in the interest of ICOE to provide incentives to avoid performance degradation. Penalties help ensure proper and adequate service or product delivery. They should be structured to maximize the probability of conformance with service-level quality indicators. Penalties may include lost fees, repayment of fees, compensation for lost earnings, termination, or any combination of these items. Since the Corporation for Education Network Initiatives in California (CENIC) is a nonprofit, rather than directly penalize it for nonconformance, it should be expected to pass along appropriate penalties to its service providers in a manner that would benefit the K-12 education community. If CENIC’s performance becomes unacceptable, the SLA should include a provision for transferring responsibilities for the High-Speed Network to another entity satisfactory to the K-12 education community. 10. Dispute Resolution and Escalation—How any differences of opinion might be resolved concerning the SLA and associated compliance. 11. Change Requests—Detailed procedures to allow ICOE to institute changes necessary to support the evolving applications and services. 12. Termination—Method of terminating the contractual relationship with the service provider, in this case CENIC, if the level of service does not meet the stated commitments for performance and quality despite attempts to resolve any issues. 00 California State Auditor Report 2005-6 Key Quality Indicators The SLA for the K-12 education community relevant to various business applications or services could require any or all of several generic key quality indicators. Frequently, a standard set of key quality indicators may be used that are relevant to all applications, with additional indicators added as necessary. Following is a listing of the primary quality indicators that should be considered in a complete SLA, as they relate to the various business applications to be used by the K-12 education community. Generic key quality indicators recognized by the TeleManagement Forum1 include the following: • Availability • Speech/video quality • Response time • Round-trip delay • Delay • Latency • Jitter • Packet loss • Locking • Transaction rate • Goodput (carried) • Throughput (offered) • Idle time • Authorization • Confidentiality • Integrity • Non-repudiation • Disk space 1 The TeleManagement Forum is a nonprofit global organization that provides leadership, strategic guidance, and practical solutions to improve the management and operation of information and communication services. California State Auditor Report 2005-6  • Help desk • Training • Interoperability • Pickup time • Time to close • Hold time • Connect time • Graceful degradation • Revocation Example of Key Quality Indicators for Videoteleconferencing As an example of the elements of one SLA for a type of advanced service delivered to the K-12 education community, such as videoteleconferencing, the TeleManagement Forum recommends including the following quality indicators in an SLA: • Availability • Speech/visual quality • Response time • Round-trip delay • Delay • Confidentiality • Nonrepudiation • Help desk • Interoperability • Connect time Each of these quality indicators would be assessed by monitoring related service-level parameters, for example, jitter, latency, loss, and stability. 22 California State Auditor Report 2005-6 APPENDIx C Results of the Analysis of the California K-12 High-Speed Network Usage Data To examine the usage of the California K-12 High-Speed Network (High-Speed Network)1, we obtained the network traffic data for the K-12 education community from the Imperial County Office of Education (ICOE). Since April 2005, ICOE has had advanced network management software that polls the routers at each of the node sites to obtain and store traffic data for each node interface. These interfaces capture the traffic, measured in bits per second, for the K-12 education community.2 We assessed the reliability of the data we received from ICOE using criteria from the federal Government Accountability Office’s Assessing the Reliability of Computer-Processed Data. We gathered information regarding the network management software and interviewed staff at ICOE and the Corporation for Education Network Initiatives in California (CENIC) to understand the protocols used to poll the node routers and tabulate the data. We also performed electronic testing on relevant data fields to ensure that they were complete and logical. Finally, our technical consultant sampled the accuracy of the data files by comparing the maximum average bits per second as calculated by the software to real-time data. We determined that the High-Speed Network traffic data we obtained from ICOE was sufficiently reliable for the purposes of this audit. Using the data, our consultant analyzed network usage by node site and concluded that the network was not overbuilt, as described in Chapter 3. Specifically, according to our technical consultant, a substantial majority of the circuits currently in place are appropriately sized to support today’s traffic loads. Also, many of the circuits that would appear to have excess bandwidth 1 The California K-12 High-Speed Network was originally named the Digital California Project: K-12 Statewide Network when it was started in fiscal year 2000–01. In fiscal year 2004–05, when stewardship of the network was given to the Imperial County Office of Education, the aspects of the network applicable to K-12 participants were then titled the K-12HSN. We call it the High-Speed Network throughout this report. 2 The data include only usage on the K-12 node sites and circuits and does not include usage on the shared CENIC California Research and Education Network backbone because CENIC does not track backbone traffic by customer. California State Auditor Report 2005-6  are in place primarily to support redundancy in the High-Speed Network’s self-healing ring architecture. Further, excess bandwidth in Ethernet connections is needed to avoid local area network bottlenecks and is most likely not costly. Table C.1 presents the results of our technical consultant’s analysis. It shows the maximum single-day usage and the average maximum usage percents for each day of the study period of April 6, 2005, through September 16, 2005.3 The percentage of usage for the average maximum usage is calculated by dividing the maximum average number of bits per second transmitted and received during the study period by the bandwidth of the interface. Our technical consultant removed Saturdays and Sundays from the calculations because school is generally not in session on these days. However, our technical consultant did not remove the data for the summer months because there is a reasonable amount of network traffic during that time. Leaving these summer dates in the average calculations tends to lower the average peak. TABLE C Peak Usage of the High-Speed Network, by Node, From April 6, 2005 Through September 6, 2005 Ethernet Maximum Single Day Average Maximum Interface Name Bandwidth Percentage of Use Percentage of Use Alameda County Office of Education (COE) Gigabit Ethernet 1,000 Mbps 10.3% 4.1% Alpine COE Fast Ethernet 100 Mbps 16.90 2.0 Amador COE Fast Ethernet 100 Mbps 18.9 3.7 Bishop Union Elementary Gigabit Ethernet 1,000 Mbps 4.4 0.8 Butte COE Gigabit Ethernet 1,000 Mbps 5.3 1.9 Calaveras COE Fast Ethernet 100 Mbps 12.2 4.8 California Department of Education Fast Ethernet 100 Mbps 20.6 5.5 Chaffey Joint Union High School District Gigabit Ethernet 1,000 Mbps 9 3.3 Chowchilla Unified School District Fast Ethernet 100 Mbps 13.6 3.5 Colusa COE Fast Ethernet 100 Mbps 10.8 3.8 Contra Costa COE Gigabit Ethernet 1,000 Mbps 13.9 4.4 Del Norte COE Fast Ethernet 100 Mbps 0.1 0 Del Norte COE Gigabit Ethernet 1,000 Mbps 2.1 0.4 Dos Palos High School Fast Ethernet 100 Mbps 13.7 4.3 3 ICOE could not provide data for the period of May 15, 2005, through May 26, 2005, because information was inadvertently discarded during its backup process.  California State Auditor Report 2005-6 Maximum Single Day Average Maximum Interface Name Bandwidth Percentage of Use Percentage of Use El Dorado COE Fast Ethernet 100 Mbps 43.8 12.2 Fresno COE Gigabit Ethernet 1,000 Mbps 7.3 3.6 Glenn COE Fast Ethernet 100 Mbps 51.6 6.7 Humboldt COE Fast Ethernet 100 Mbps 12.2 6.3 Imperial COE Gigabit Ethernet 1,000 Mbps 9 4.9 Kern COE Gigabit Ethernet 1,000 Mbps 4.7 1.9 Kings COE Fast Ethernet 100 Mbps 1.8 0 Kings COE Gigabit Ethernet 1,000 Mbps 4 2.1 Lake COE Fast Ethernet 100 Mbps 27.3 9.7 Lake Tahoe Unified School District Fast Ethernet 100 Mbps 21.5 4.9 Lassen COE Fast Ethernet 100 Mbps 42.4 5.6 Los Angeles COE Gigabit Ethernet 1,000 Mbps 18.6 7.5 Los Angeles Unified School District Gigabit Ethernet 1,000 Mbps 23.6 13.8 Loyalton High School Fast Ethernet 100 Mbps 5 1.2 Madera COE Fast Ethernet 100 Mbps 20.9 9.6 Mammoth COE Fast Ethernet 100 Mbps 19.3 3.5 Marin COE Fast Ethernet 100 Mbps 51 19.1 Mariposa COE Fast Ethernet 100 Mbps 6.3 2.9 Merced COE Gigabit Ethernet 1,000 Mbps 5.1 2.1 Modoc COE Fast Ethernet 100 Mbps 20.8 2.8 Monterey COE Fast Ethernet 100 Mbps 54.5 25.8 Monterey Peninsula Community College Gigabit Ethernet 1,000 Mbps 4.6 1.2 Napa Valley Unified School District Fast Ethernet 100 Mbps 55.4 13.6 Nevada Joint Union High School District Gigabit Ethernet 1,000 Mbps 3.4 0.9 North Humboldt Unified School District Fast Ethernet 100 Mbps 6.5 2.7 Orange COE Gigabit Ethernet 1,000 Mbps 24.1 9.3 Placer COE Gigabit Ethernet 1,000 Mbps 5 1.5 Plumas COE Fast Ethernet 100 Mbps 59.1 26.5 Pomona Unified School District Fast Ethernet 100 Mbps 61.2 24.3 Red Bluff High School Fast Ethernet 100 Mbps 29.4 8.9 Riverside COE Gigabit Ethernet 1,000 Mbps 6.8 3.4 Riverside Indio Fast Ethernet 100 Mbps 59.8 28.7 Sacramento COE Gigabit Ethernet 1,000 Mbps 17.3 8.1 San Benito COE Fast Ethernet 100 Mbps 35.5 14.8 San Bernardino County Superintendent Schools (CSS) Gigabit Ethernet 1,000 Mbps 14.5 6 San Diego COE Gigabit Ethernet 1,000 Mbps 15.7 5.7 San Francisco COE Gigabit Ethernet 1,000 Mbps 3.9 2 San Joaquin COE Fast Ethernet 100 Mbps 34.7 13.8 continued on next page California State Auditor Report 2005-6 55 Maximum Single Day Average Maximum Interface Name Bandwidth Percentage of Use Percentage of Use San Luis Obispo COE Gigabit Ethernet 1,000 Mbps 3.8 1.7 San Luis Obispo COE to Hub Gigabit Ethernet 1,000 Mbps 4 1.7 San Mateo COE Gigabit Ethernet 1,000 Mbps 22.7 6 Santa Barbara COE Fast Ethernet 100 Mbps 45.4 13.8 Santa Clara COE Gigabit Ethernet 1,000 Mbps 9.5 4.5 Santa Cruz COE Fast Ethernet 100 Mbps 49.7 21.8 Shasta COE Fast Ethernet 100 Mbps 30.7 10.9 Sierra COE Fast Ethernet 100 Mbps 4.1 0.8 Siskiyou COE Fast Ethernet 100 Mbps 33.2 7.1 Solano COE Fast Ethernet 100 Mbps 21 10.4 Sonoma COE Fast Ethernet 100 Mbps 15.5 6.8 Stanislaus COE Fast Ethernet 100 Mbps 28.3 15.7 Sutter COE Fast Ethernet 100 Mbps 9 4.4 Trinity COE Fast Ethernet 100 Mbps 8.9 3.6 Truckee Donner COE Fast Ethernet 100 Mbps 7.6 3.9 Tulare COE Fast Ethernet 100 Mbps 70.5 31.5 Tulelake Basin Unified School District Fast Ethernet 100 Mbps 44.2 5.4 Tuolomne COE Fast Ethernet 100 Mbps 8.4 4.5 Ventura COE Gigabit Ethernet 1,000 Mbps 5.9 3.3 Victor Valley Community College Fast Ethernet 100 Mbps 27.6 14.7 Victor Valley Community College Gigabit Ethernet 1,000 Mbps 3.8 0.9 Yolo COE Fast Ethernet 100 Mbps 31.5 9.6 Yuba COE Fast Ethernet 100 Mbps 17 7.8 66 California State Auditor Report 2005-6 DS Circuits Node-to-Node Circuits Maximum Single Day Average Maximum Interface Name Bandwidth Percentage of Use Percentage of Use Amador COE to Calaveras COE 44.21 Mbps 28.5 5.7 Bishop Union Elementary to Mammoth COE 44.21 Mbps 34.4* 7.6 Butte COE to Glenn COE 44.21 Mbps 13.9 0.3 Calaveras COE to Amador COE 44.21 Mbps 28.4* 6.9 Calaveras COE to Tuolomne COE 44.21 Mbps 12.5 6.4 Colusa COE to Yolo COE 44.21 Mbps 13.7 0.6 Del Norte COE to Humboldt COE 44.21 Mbps 42.9* 9.9 Dos Palos High School to Madera COE 44.21 Mbps 0.4 0.1 El Dorado COE to Sacramento COE 44.21 Mbps 10.8 1.7 Eureka City Schools to Humboldt COE 44.21 Mbps 0.2 0.0 Glenn COE to Butte COE 44.21 Mbps 14.4 0.3 Humboldt COE to Del Norte COE 44.21 Mbps 42.7* 10.0 Humboldt COE to Eureka City Schools COE 44.21 Mbps 0.2 0.0 Humboldt COE to North Humboldt Unified School District 44.21 Mbps 14.0 6.2 Imperial COE to Riverside Indio 44.21 Mbps 151.3* 64.1† Kings COE to Tulare COE 44.21 Mbps 3.3 0.4 Lake Tahoe Unified School District to Trucker Donner COE 44.21 Mbps 17.0* 8.7 Lassen COE to Modoc COE 44.21 Mbps 40.8* 6.2 Loyalton High School to Plumas COE 44.21 Mbps 11.3 2.7 Loyalton High School to Truckee Donner COE 44.21 Mbps 0.7 0.1 Madera COE to Dos Palos COE 44.21 Mbps 0.4 0.1 Madera COE to Mariposa COE 44.21 Mbps 1.0 0.2 Mammoth COE to Bishop Union Elementary 44.21 Mbps 39.2* 8.0 Mariposa COE to Madera COE 44.21 Mbps 1.1 0.2 Mendocino COE to Sonoma COE 44.21 Mbps 12.0 0.3 Merced COE to San Joaquin COE 44.21 Mbps 134.4* 25.9† Merced COE to Stanislaus COE 44.21 Mbps 94.1* 26.4† Modoc COE to Lassen COE 44.21 Mbps 46.3* 6.3 Modoc COE to Tulelake Basin Unified School District 44.21 Mbps 1.8 0.2 Monterey COE to Monterey Peninsula Community College 44.21 Mbps 4.4 1.1 Monterey COE to San Benito COE 44.21 Mbps 12.3 9.7 Monterey Peninsula Community College to Monterey COE 44.21 Mbps 4.4 1.1 Monterey Peninsula Community College to Santa Cruz COE 44.21 Mbps 4.3 0.6 Napa Valley Unified School District to Solano COE 44.21 Mbps 137.5* 15.9† Napa Valley Unified School District to Sonoma COE 44.21 Mbps 113.8* 20.2† Nevada Joint Union High School District to Placer COE 44.21 Mbps 4.3 0.3 continued on next page California State Auditor Report 2005-6  Maximum Single Day Average Maximum Interface Name Bandwidth Percentage of Use Percentage of Use Nevada Joint Union High School District to Sierra COE 44.21 Mbps 9.1 1.8 North Humboldt Unified School District to Humboldt COE 44.21 Mbps 16.0* 6.2 Placer COE to Nevada Joint Union High School District 44.21 Mbps 5.2 0.3 Plumas COE to Loyalton High School 44.21 Mbps 11.3 2.7 Red Bluff High School to Shasta COE 44.21 Mbps 60.6* 12.6 Riverside COE to Imperial COE 44.21 Mbps 138.6* 63.9† Sacramento COE to El Dorado COE 44.21 Mbps 8.9 0.3 San Benito COE to Monterey COE 44.21 Mbps 12.4 9.8 San Bernardino CSS to Victor Valley Community College 44.21 Mbps 121.3* 42.7† San Joaquin COE to Merced COE 44.21 Mbps 94.7* 25.6† Santa Barbara COE to Ventura COE 44.21 Mbps 25.5* 0.8 Santa Cruz COE to Monterey Peninsula Community College 44.21 Mbps 4.2 0.6 Shasta COE to Red Bluff High School 44.21 Mbps 55.8* 12.5 Sierra COE to Nevada Joint Union High School District 44.21 Mbps 9.0 1.7 Siskiyou COE to Tulelake Basin High School 44.21 Mbps 98.0* 12.2 Solano COE to Napa Valley Unified School District 44.21 Mbps 87.7* 16.4† Sonoma COE to Mendocino COE 44.21 Mbps 18.1* 0.4 Sonoma COE to Napa Valley Unified School District 44.21 Mbps 94.4* 19.9† Stanislaus COE to Merced COE 44.21 Mbps 139.0* 22.1† Sutter COE to Yuba COE 44.21 Mbps 1.9 0.2 Truckee Donner COE to Lake Tahoe Unified School District 44.21 Mbps 21.9* 8.6 Truckee Donner COE to Loyalton High School 44.21 Mbps 0.6 0.1 Tulare COE to Kings COE 44.21 Mbps 3.3 0.4 Tulelake Basin Unified School District to Modoc COE 44.21 Mbps 1.9 0.2 Tulelake Basin Unified School District to Siskiyou COE 44.21 Mbps 131.8* 12.7 Tuolomne COE to Calaveras COE 44.21 Mbps 15.6* 6.4 Ventura COE to Santa Barbara COE 44.21 Mbps 25.5* 0.8 Victor Valley Community College to San Bernardino CSS 44.21 Mbps 82.6* 41.8† Yolo COE to Colusa COE 44.21 Mbps 13.3 0.6 Yuba COE to Sutter COE 44.21 Mbps 1.9 0.2 * 31 DS3 circuits connecting two nodes that had a maximum single-day usage exceeding the 15 percent threshold as described on page 57. † 12 DS3 circuits connecting two nodes that had an average maximum usage exceeding the 15 percent threshold described on page 57.  California State Auditor Report 2005-6 DS Circuits Node-to-Hub Circuits Maximum Single Day Average Maximum Interface Name Bandwidth Percentage of Use Percentage of Use Alpine COE to Sacramento Hub 44.21 Mbps 29.1* 4.2 Amador COE to Stockton Hub 44.21 Mbps 37.0* 11.7 Bishop Union Elementary to Los Angeles Hub 44.21 Mbps 92.6* 22.4† Butte COE to Corning Hub 44.21 Mbps 93.0* 44.1† California Department of Education to Sacramento Hub 44.21 Mbps 47.9* 12.2 Chowchilla Unified School District to Stockton Hub 44.21 Mbps 32.8* 7.8 Colusa COE to Sacramento Hub 44.21 Mbps 22.5* 8.8 Dos Palos High School to Fresno Hub 44.21 Mbps 34.4* 9.8 El Dorado COE to Sacramento Hub 44.21 Mbps 99.4* 25.7† Glenn COE to Corning Hub 44.21 Mbps 58.2* 14.8 Humboldt COE to Sunnyvale Hub 44.21 Mbps 45.5* 23.3† Kings COE to Fresno Hub 44.21 Mbps 87.7* 45.7† Lake COE to Oakland Hub 44.21 Mbps 45.5* 21.7† Lake Tahoe Unified School District to Sacramento Hub 44.21 Mbps 49.8* 16.3† Lassen COE to Corning Hub 44.21 Mbps 94.1* 14.8 Madera COE to Stockton Hub 44.21 Mbps 41.9* 21.1† Mariposa COE to Stockton Hub 44.21 Mbps 13.4 6.4 Mendocino COE to Sunnyvale Hub 44.21 Mbps 83.9* 31.5† Monterey COE to Soledad Hub 44.21 Mbps 106.5* 46.7† Monterey Peninsula Community College to Soledad Hub 44.21 Mbps 95.3* 26.3† Nevada Joint Union High School District to Sacramento Hub 44.21 Mbps 71.5* 20.6† Placer COE to Sacramento Hub 44.21 Mbps 120.5* 32.1† Plumas COE to Corning Hub 44.21 Mbps 13.5 6.8 Pomona Unified School District to Los Angeles Hub 44.21 Mbps 143.3* 53.1† Red Bluff High School to Corning Hub 44.21 Mbps 41.3* 7.6 San Benito COE to Soledad Hub 44.21 Mbps 85.3* 24.7† Santa Barbara COE to San Luis Obispo Hub 44.21 Mbps 103.5* 29.3† Santa Cruz COE to Sunnyvale Hub 44.21 Mbps 146.1* 49.0† Shasta COE to Corning Hub 44.21 Mbps 71.2* 32.5† Siskiyou COE to Corning Hub 44.21 Mbps 100.4* 22.4† Solano COE to Sunnyvale Hub 44.21 Mbps 93.6* 35.3† Sonoma COE to Oakland Hub 44.21 Mbps 134.8* 50.6† Sutter COE to Sacramento Hub 44.21 Mbps 20.2* 9.7 Trinity COE to Corning Hub 44.21 Mbps 20.0* 8.1 Tulare COE to Stockton Hub 44.21 Mbps 163.3* 70.6† continued on next page California State Auditor Report 2005-6  Maximum Single Day Average Maximum Interface Name Bandwidth Percentage of Use Percentage of Use Tuolomne COE to Stockton Hub 44.21 Mbps 23.9* 13.7 Ventura COE to Los Angeles Hub 44.21 Mbps 142.2* 70.7† Yolo COE to Sacramento Hub 44.21 Mbps 41.5* 18.4† Yuba COE to Sacramento Hub 44.21 Mbps 47.2* 17.6† * 37 DS3 circuits connecting nodes to hubs that had a maximum single-day usage exceeding the 15 percent threshold as described on page 56. † 25 DS3 circuits connecting nodes to hubs that had an average maximum usage exceeding the 15 percent threshold described on page 56. 0000 California State Auditor Report 2005-6 OC- Circuits Node-to-Node Circuits Maximum Single Day Average Maximum Interface Name Bandwidth Percentage of Use Percentage of Use Alameda COE to San Mateo COE 155 Mbps 30.5 15.1* Alameda COE to Santa Clara COE 155 Mbps 36.6 15.4* Chaffey Union High School District to San Bernardino CSS 155 Mbps 6.4 1.2* Contra Costa COE to Marin COE 155 Mbps 26.8 7.8* Los Angeles COE to Los Angeles Unified School District 155 Mbps 33.2 10.7* Los Angeles Unified School District to Los Angeles COE 155 Mbps 29.2 10.6* Marin COE to Contra Costa COE 155 Mbps 23.4 8.0* Marin COE to San Francisco COE 155 Mbps 20.3 5.7* Orange COE to Riverside COE 155 Mbps 6.8 0.1* Riverside COE to Orange COE 155 Mbps 6.7 0.1* San Bernardino CSS to Chaffey Joint Union High School District 155 Mbps 6.0 1.2* San Francisco COE to Marin COE 155 Mbps 20.3 5.8* San Luis Obispo COE ATM SONET† 150 Mbps 0.6 0.0* San Mateo COE to Alameda COE 155 Mbps 28.0 15.2* Santa Clara COE to Alameda COE 155 Mbps 32.7 15.9* Node-to-Hub Circuits Chaffey Joint Union High School District to Tustin Hub 155 Mbps 55.8 19.3* Contra Costa COE to Oakland Hub 155 Mbps 92.5 31.7 Fresno COE to Stockton Hub 155 Mbps 51.0 23.6 Imperial COE to San Diego Hub 155 Mbps 88.9 42.7 Kern COE to Bakersfield Hub 155 Mbps 24.7 12.3* Los Angeles COE to Los Angeles Hub 155 Mbps 101.1 40.4 Orange COE to Los Angeles Hub 155 Mbps 149.1 59.1 Riverside COE to Tustin Hub 155 Mbps 46.5 21.7 Sacramento COE to Sacramento Hub 155 Mbps 98.3 47.6 San Bernardino CSS to Los Angeles Hub 155 Mbps 98.2 41.5 San Diego COE to San Diego Hub 155 Mbps 83.5 36.7 San Francisco COE to Sunnyvale Hub 155 Mbps 31.4 16.8* San Joaquin COE to Stockton Hub 155 Mbps 30.8 14.5* San Mateo COE to Sunnyvale Hub 155 Mbps 62.0 26.9 Santa Clara COE to Sunnyvale Hub 155 Mbps 91.5 39.7 Stanislaus COE to Stockton Hub 155 Mbps 34.0 16.0* * There are 20 OC-3 circuits that have less than 23 percent average maximum usage as described on page 58. † This circuit is an anomaly because it is an Asynchronous Transfer Mode interface that operates at 150Mbps across a SONET OC-3 (155 Mbps) link. California State Auditor Report 2005-6 00 Blank page inserted for reproduction purposes only. 0022 California State Auditor Report 2005-6 Agency Comments provided as text only University of California Office of the President 1111 Franklin Street Oakland, California 94607-5200 January 13, 2006 Ms. Elaine M. Howle State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Ms. Howle: Thank you for the opportunity to review and comment on the audit report K-12 High-Speed Network: The Network Architecture is Sound but Opportunities Exist to Increase Its Use. The University of California (UC) is pleased with the overall conclusions of the Bureau of State Audit report, and we enthusiastically support the report’s findings that: • the K-12 High Speed Network architecture is sound; • it provides a cost-effective solution for K-12; • the network provides ample bandwidth to support the current and future applications used by the K-12 education community; • the network is not overbuilt; and • there is no compelling technical or financial reason to abandon the existing High Speed Network. When in 2000, the State of California requested that UC assist with K-12’s integration into the higher education network, UC felt privileged to work with the State to realize the vision of creating a new network-based information and communication infrastructure that would deliver rich multimedia, interactive digital educational and administrative resources and services to K-12 educators and administrators who were underserved by their current network services. It was well understood by UC and the State that UC would contract with the Corporation for Education Network Initiatives in California (CENIC) to extend the CalREN higher education backbone network to provide connectivity and advanced network services to the K-12 community. The K-12 community therefore benefitted from an existing, successfully managed solution for its network services requirements. The agreements among UC, CENIC, and the former administration reflected several fundamental premises, which were mirrored in both the general nature of the contract and original project plan between UC and CENIC and in UC’s oversight of CENIC. These premises included the following: California State Auditor Report 2005-6 00 Ms. Elaine M. Howle January 13, 2006 Page 2 • UC would allocate 100 percent of the State High Speed Network funds directly to CENIC without retaining any overhead for administrative services. • CENIC would design the K-12 network architecture and build the infrastructure, in a manner that leveraged and built upon the higher education network backbone for maximum cost effectiveness, and to enable electronic interchange of content among the various educational institutions within the state. • UC was contracting with CENIC for advanced network services provided to the K-12 community; UC therefore would judge CENIC’s effectiveness by the tangible outcomes achieved. • UC and CENIC would extend the network infrastructure to all 58 California County Offices of Education. CENIC was not asked to extend the network directly to districts and schools, nor was there a requirement or State funding to address specific uses of the network or content delivered over the network. • The network infrastructure would have the capacity, speed, flexibility and reliability required to provide excellent network-based services to the K-12 community. CENIC’s remarkable efforts to pursue the design and deployment of the K-12 High Speed Network concurrently with the ongoing development and refinement of the CalREN higher education network resulted in a very successful outcome. As indicated in the State Auditor’s Report, as of 2004, the vast majority of the kindergarten through twelfth grade (K-12) schools, school districts, and county offices of education in the State of California are now connected via a high-bandwidth, reliable network infrastructure to each other, to the Internet, to California’s universities and community colleges, and to peer institutions around the country and the world. This network access opened the door to significant opportunities to exchange and access new forms of content for use in the classroom and by administrators. UC oversight of the UC-CENIC contract for High Speed Network services involved many senior academic and administrative managers and technical experts throughout UC who were involved in CENIC in a variety of capacities as board members, business and technical advisory committee members, UC Office of the President oversight committee members, and program steering committee members. UC is proud to have contributed to the success of the High Speed Network. As the report has no recommendations for UC, no follow-up action is planned. Sincerely, (Signed by Robert C. Dynes) Robert C. Dynes 00 California State Auditor Report 2005-6 Agency Comments provided as text only California Department of Education 1430 N Street Sacramento, CA 95814-5901 January 12, 2006 Elaine M. Howle, State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Dear Mrs. Howle: Audit No. 2005-116 This is the California Department of Education’s (CDE) response to the Bureau of State Audits’ (BSA) draft audit report entitled, “K-12 High-Speed Network: The Network Architecture Is Sound But Opportunities Exist To Increase Its Use.” We appreciate the opportunity to comment on your draft report. This response expresses the views of both the CDE and the Imperial County Office of Education (ICOE). General Comments: Pursuant to the Request for Application (RFA) issued on August 2, 2004, by the CDE, the ICOE was selected as the lead education agency (lead agency) of the California K-12 High-Speed Network (High-Speed Network). The RFA was a direct response to the language in the Annual Budget Act of 2004 and required each applicant to adequately respond to several items related to goals, scope, and purpose for the program. The BSA draft audit report mentions on several occasions the lack of clear goals for the program and how it is difficult to measure success absent such goals. It is the position of the CDE and the ICOE that specific goals, while not defined by the Legislature, were in fact identified in the RFA as well as in the awarded response from the Imperial Consortium. These goals have either been met or have had adequate progress made toward their accomplishment. As the draft audit report describes, the CDE and the ICOE have worked closely together in all aspects of the program implementation and assert that the program has been successfully implemented given the circumstances faced by the High-Speed Network, most notably an absence of funding for Year 2 under the consortium’s leadership. Additionally, the Annual Budget Act of 2004 states that for fiscal year 2005-06, the program will be governed by statute that will provide that the program be overseen by a governance structure that accomplishes specified goals. It is important to note that no such legislation was enacted for fiscal year 2005-06, despite the CDE and the ICOE working collaboratively on proposed legislation that would have fulfilled this objective. Absent legislation for fiscal year 2005-06, the CDE and the ICOE have continued to implement the program using the goals, scope, and purpose identified in the original RFA. The CDE and the ICOE will work with the Legislature to more clearly identify these goals. California State Auditor Report 2005-6 0055 Ms. Elaine M. Howle, State Auditor January 12, 2006 Page 2 Many of the recommendations provided by the draft audit report direct the CDE to ensure that the ICOE performs various activities. These statements, taken on their face, could imply that the ICOE has not exercised the proper oversight or due diligence in the implementation of the High-Speed Network. Specifically, issues related to contract terms with the Corporation for Education Network Initiatives in California (CENIC) could leave the impression that the ICOE did not adequately address the needs of the State in its performance as lead agency. While the report goes on to explain the circumstances and reasons for the ICOE’s inability to negotiate specific terms, it is the position of the CDE that the ICOE has managed and operated the program in an exceptional manner with the highest integrity and with the State’s K-12 interests as the priority. Many of the recommendations related to contract terms require that the CENIC change the manner in which it deals with all education segments (UC, CSU, and Community Colleges), not just simply K-12, and we agree that a more comprehensive approach would promote greater accountability of state funds. The CDE and ICOE look forward to working with the other segment partners and the CENIC to fully address the recommendations of the State Auditor. A final point in regards to Chapter 4, the CalREN network, and more importantly the intersegmental participation and cost sharing that enable the network to exist, are exemplary uses of public resources and the individual segments’ determination to work collaboratively for public benefit. Recommendation #1 To ensure that the High-Speed Network meets its expectations, the Legislature should consider enacting legislation that prescribes the specific goals and outcomes it wants from the High-Speed Network project. CDE’s and ICOE’s Response: The CDE and the ICOE defined goals in the RFA, and the response to the RFA under which the ICOE was selected. The ICOE has been working with the CDE to meet those goals despite a lack of funding for the 2005-06 school year. The CDE and the ICOE look forward to working with the Legislature and the future governance structure of the High-Speed Network to further define goals and adequate measures of success for the program. 0066 California State Auditor Report 2005-6 Ms. Elaine M. Howle, State Auditor January 12, 2006 Page 3 Recommendation #2 If future state appropriations are made for the development of applications and associated content delivery to the K-12 community, the Legislature should require the responsible agency to develop policies and guidelines that protect ownership of any intellectual property associated with related software or content and the relevant contracts should be amended to address those policies. Furthermore, to fully protect any intellectual property that may be developed in the future using state funds, the Legislature should require contract terms that provide ownership of all intellectual property developed under that contract using state funds as vesting in the State of California. CDE’s and ICOE’s Response: The CDE and the ICOE will develop guidelines and policies that reinforce its position that State-funded assets belong to the State. The CDE and the ICOE will ensure contracts for content development reflect specified policies. Additionally, ownership of any intellectual property developed under any applications or content development agreements will vest in the State of California. Recommendation #3 To ensure that the High-Speed Network is appropriately managed, the CDE should ensure that the ICOE does the following: • Develops a comprehensive and extensive set of service-level agreement based upon applications to be delivered via the High-Speed Network project. • Requests CENIC to provide a master service-level agreement for its review. • Includes the appropriate service-level agreements in its ongoing contracts with CENIC and other service providers for the High-Speed Network using industry standards. CDE’s and ICOE’s Response: The CDE and the ICOE will ensure that the ICOE, as it contracts with the CENIC and other providers of K-12 services, secure terms that protect the State’s interests. These terms will include service-level agreements from content providers that are available on the High-Speed Network. Additionally, the CDE and the ICOE will work with the CENIC to compile a master service-level agreement developed from the multiple service-level agreements that the CENIC procures from its vendors and providers. Furthermore, as one of four public education segments participating in the CalREN network, the CDE and the ICOE will work with the other segments and seek to foster agreement with regard to the use of service-level agreements that match industry standards. California State Auditor Report 2005-6 00 Ms. Elaine M. Howle, State Auditor January 12, 2006 Page 4 Recommendation #4 To ensure the adequate protection of the State’s interest in tangible, non-shared assets, the CDE should direct the ICOE to transfer ownership of these types of assets to the State as purchases of new equipment are made. CDE’s and ICOE’s Response: The CDE and the ICOE agree that anytime there are assets to be purchased, the procurement process will be reviewed to protect the State’s assets and to obtain the best possible pricing. In circumstances where this consideration means that the CENIC will procure the tangible, non-shared asset, contract terms will be in place to vest title to the asset in the State. In cases of the ICOE making the purchase, title will be taken on behalf of the State. Recommendation #5 To ensure that the interest earned on advance payments made to the CENIC are used to benefit the High Speed Network, the CDE should direct the ICOE to amend its agreement with the CENIC to stipulate the use of interest earned. CDE’s and ICOE’s Response: The CDE and the ICOE agree that interest earned on advance payments, E-rate proceeds, California Teleconnect fund proceeds, and balances or reserves held by the CENIC should appropriately be credited to the benefit of the High-Speed Network and K-12. Therefore, the CDE and the ICOE will work with the CENIC to establish procedures that ensure accurate accounting and crediting of all K-12 interest revenues to the High-Speed Network and K-12. Additionally, the CDE and the ICOE will include a contract provision enabling it to adequately monitor the E-rate and California Teleconnect fund proceeds paid to CENIC on behalf of K- 12, and compare those receipts with the funds requested. The CDE and the ICOE continues to require a final accounting and reconciliation of amounts expended with sufficient detail to ascertain that quarterly pre-payments were used appropriately for services that benefit K-12. 00 California State Auditor Report 2005-6 Ms. Elaine M. Howle, State Auditor January 12, 2006 Page 5 Recommendation #6 To ensure that the CENIC’s per unit rate for access to the commodity Internet is closer to its actual costs to provide the service, the CDE should require the ICOE to amend its agreement with the CENIC to stipulate that to the extent possible, the CENIC should use its surplus Internet service program revenues from each year to offset the per-unit rate that it establishes for the following year. The ICOE should also stipulate in its agreement that if the CENIC is unable to apply the surplus revenue due to a change in its financial position, the CENIC should provide the ICOE with documentation to support its inability to do so. CDE’s and ICOE’s Response The CDE and the ICOE believe it is prudent to hold a reasonable amount of reserves for cost centers, such as Internet service, in order to manage program and funding uncertainties. The CDE and the ICOE will encourage the CENIC Business Advisory Committee, Board, and other segment representatives to use surplus Internet service program revenues from each year that exceed a reasonable reserve to offset the per-unit rate that it establishes for the following year. When the CENIC Board decisions are made to apply or not apply surplus revenues, the CDE and the ICOE, as well as other segments, expect to be provided with adequate documentation and rationale to support the Board decisions. The CDE and the ICOE will continue, to the fullest extent possible, to use their role with the Business Advisory Committee and by informing the K-12 representatives to the CENIC Board, to encourage cost model allocations and rate setting decisions that are appropriate to actual costs incurred and reasonable sharing of joint expenses among the participating segments. Recommendation #7 To ensure that High-Speed Network equipment replacement funds are used to benefit the K-12 education community, the CDE should direct the ICOE to request the CENIC to re-establish a reserve for equipment replacement that is in an account solely for the High-Speed Network. Further, the ICOE should also amend its agreement with the CENIC to stipulate that interest earned on the funds held in the High-Speed Network’s equipment replacement account accrues to the benefit of the High-Speed Network. Finally, the CDE should direct the ICOE to amend its agreement with the CENIC to stipulate that the CENIC should use the funds held in the High-Speed Network equipment replacement account to purchase new equipment instead of requesting reimbursement from the ICOE until after the account is depleted. As the CENIC purchases new equipment after advance consultation with the ICOE, it should also consult with the ICOE on the development of a technology refresh plan, which the ICOE should use to establish its own equipment replacement funds for the High-Speed Network. California State Auditor Report 2005-6 00 Ms. Elaine M. Howle, State Auditor January 12, 2006 Page 6 CDE’s and ICOE’s Response The CDE and the ICOE will amend its contract with the CENIC to require the CENIC to establish an equipment replacement reserve to fund the technology refresh plan developed for the program, and that these funds are held in a segregated account solely for the High-Speed Network. The CDE and the ICOE will ensure that the technology refresh plan utilizes the K-12 equipment replacement reserves appropriately, and ensure that the State’s interest in the assets is protected. As previously stated, the CDE and the ICOE will track balances, revenues, and interest earned to ensure that interest earned on behalf of K-12 is credited to the High-Speed Network and benefits K-12 specifically. In conjunction with the CDE, the ICOE will collect input from the CENIC when developing a technology refresh plan and, will negotiate jointly with CENIC for the purchase of equipment needed to fulfill it. Furthermore, the ICOE will authorize, as appropriate, expenditures from the equipment reserves for the purchase of said equipment to the point that the $7.2M in K-12 equipment reserves is exhausted. The ICOE will also budget for and maintain new equipment reserves in the future after those held by the CENIC are depleted in accordance with the technology refresh plan in order to retain a reasonable level of reserves for equipment replacement. Assets purchased with State funding are appropriately State assets, therefore, the ICOE will either take title to the equipment purchased using K-12 equipment reserves, both those held by CENIC currently and any future reserves held in trust for K-12, or work with CENIC to develop agreeable terms under which CENIC has the ability to manage the equipment while the State’s ownership rights are protected. The ICOE will hold title as an agent and fiduciary of the California Department of Education and the State of California and will consider all such assets as State-owned. Recommendation #8 To ensure that any interest earnings received for E-rate and California Teleconnect discounts accrue to the benefit of the High-Speed Network, the CDE should direct the ICOE to amend its agreement and require the CENIC to credit any interest earnings to the High-Speed Network project. Additionally, the ICOE should require the CENIC to provide a detailed accounting of E-Rate and California Teleconnect funds so that it can verify it received the appropriate amount of interest. However, if the CENIC does not agree to this provision, the ICOE should consider requiring the CENIC to remit all funds received from the telecommunications carriers, net of any agreed upon administrative costs, to the ICOE promptly upon its receipt of these funds. In the event that the CENIC is unable to remit the funds promptly, any interest on the funds should accrue to the benefit of the High-Speed Network. 00 California State Auditor Report 2005-6 Ms. Elaine M. Howle, State Auditor January 12, 2006 Page 7 CDE’s and ICOE’s Response: The CDE and the ICOE believe that interest earned on K-12 specific resources, whether pre-paid expenses, E-rate proceeds, California Teleconnect fund proceeds, or from any other source, is appropriately credited to K-12 and used for the benefit of K-12. The ICOE will monitor, track and credit interest earned from all sources and ensure that said interest revenues are appropriately credited to K-12 and used for the benefit of K-12. In order to accurately calculate interest and the appropriate tracking and crediting of revenues, the CDE and the ICOE will monitor the revenues received by the CENIC on behalf of the High-Speed Network and K-12 from any source, including E-Rate, and California Teleconnect funds, so that it can track interest on those amounts and verify that appropriate interest revenues are being credited. The ICOE will conduct periodic audits of the amounts expected from vendors based on discount applications, amounts credited, and interest accrued. If the CENIC is unable to provide a detailed accounting of E-Rate and California Teleconnect funds, and given the CENIC’s role as the agent of K-12 and the High-Speed Network in seeking E-rate and California Teleconnect fund discounts, the ICOE will require that the funds be immediately remitted to the ICOE. Recommendation #9 To maximize the benefit of the High-Speed Network, the CDE should ensure that the ICOE does the following: • Continue its effort to implement statewide videoconferencing. • Continue the efforts of its application committee to identify academic content and application uses to place on the High-Speed Network. • Continue with its plans to fund the advancing network uses grant applicants. • Proceed with its last mile grant program. California State Auditor Report 2005-6  Ms. Elaine M. Howle, State Auditor January 12, 2006 Page 8 CDE’s and ICOE’s Response: The CDE and the ICOE believe that videoconferencing is a critical application for K-12 today and into the future. An investment has been made that enables schools throughout California enhanced learning opportunities, professional development opportunities and administrative savings. To date a critical dependence on videoconferencing has emerged. The CDE and the ICOE acknowledge that the activities of the application committee are necessary to the development of the wide variety of resources required to meet the diverse needs of teachers and students in California. The application committee will also identify and promote coordinated uses of the network that leverage other resources and stretch the investment in technology being made at the local and Federal levels. The Advancing Network Uses Grant Program that was ranked for funding, until the High- Speed Network was removed from the 2005-06 State Budget, would have provided valuable resources to the entire state free of charge. The CDE and the ICOE intend to set the nine projects into motion once funding of the High-Speed Network is restored in the State Budget. The Last Mile Grants meet the needs of schools and districts that have encountered barriers to connecting. As with the Advancing Network Uses Grant Program, the CDE and the ICOE are positioned to release Last Mile Grants that will respond to the unconnected sites and those that need innovative solutions or increased capacity. Recommendation #10 The CDE should ensure that the ICOE develops a process to measure the success of the High- Speed Network that incorporates, but is not limited to, the following: • Continuing its efforts to gather data from the California School Technology Survey, as well as data collected as part of other statewide education technology projects. • Hiring a consultant to assist with the development of a process to measure the success of the High-Speed Network. • Continuing to work with academic content providers to obtain statewide data on the K-12 education community’s frequency and type of academic content use. 22 California State Auditor Report 2005-6 Ms. Elaine M. Howle, State Auditor January 12, 2006 Page 9 CDE’s and ICOE’s Response: The CDE and the ICOE will continue to work together to utilize existing data collection tools including the California School Technology Survey and data from other statewide education technology projects to monitor and report on the impact of the High-Speed Network program on education in California. The CDE and the ICOE will work together to identify and hire consulting services to assist in the process of establishing measures of success for the program. The CDE and the ICOE will ensure that any entities agreeing to develop applications and associated content delivery establish measures of success for their efforts. Any applications or content development funded through grants issued by the ICOE will include specific goals and outcomes by which the success of those projects may be evaluated. The CDE and the ICOE will continue to work with content providers to jointly develop data collection requirements and tools that will assist the program in determining educational uses and impact of network use as reflected in the aforementioned measures of success. If you have any questions regarding the CDE’s and ICOE’s response to the draft report, please contact Kim Sakata, Audit Response Coordinator, Audits and Investigations Division, at (916) 323- 3560 or by email at ksakata@cde.ca.gov. Sincerely, (Signed by Gavin Payne) GAVIN PAYNE Chief Deputy Superintendent of Public Instruction California State Auditor Report 2005-6  Blank page inserted for reproduction purposes only.  California State Auditor Report 2005-6 Agency Comments provided as text only Corporation for Education Network Initiatives in California 5757 Plaza Drive, Suite 205 Cypress, CA 90630 January 13, 2006 Elaine Howle* State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Ms. Howle, Thank you for the opportunity to comment on the audit of what is referred to as the “High Speed Network” (i.e. funding to permit the K-12 to use CENIC’s CalREN network). As you know, in conducting the audit the Bureau requested considerable information from CENIC, our vendors, and business partners. To the best of our knowledge, CENIC responded quickly and fully to each request for information and spent many hours in meetings and phone calls with the Bureau. We believe the audit report provides useful information that highlights the value CENIC provides to the educational institutions we serve. CENIC offers services slightly different than those provided by the industry. It also operates with strong controls and focuses on responding to our users’ needs as quickly and efficiently as possible. We believe our structure (i.e. all education segments united under a nonprofit organization) makes us better able to deliver services focused specifically on the unique needs of the education community, and to deliver them more cost effectively than any other public or private organization. We believe the results of the audit bear this out. Among the issues and conclusions of the audit are: 1 • The Bureau looked into CENIC’s cost allocation methods and the fees CENIC charged the K-12 based on those methods and found them to be reasonable. • The Bureau conducted a significant analysis of our costs and costs of comparable 2 organizations. It found that our fees are lower than would be available through other means. • The Bureau has validated the design of CENIC’s network and its capacity. It is both appropriately designed and is not overbuilt. 3 • The Bureau found that the state saved large sums of money by CENIC purchasing the equipment used to connect K-12 entities to CENIC’s network. Indeed, since those entities are not allowed to “piggyback” on CENIC’s attractive pricing, the equipment must continue to be purchased and owned by CENIC. The suggestion that each K-12 entity should own the equipment instead of CENIC would therefore result in additional costs to those entities in the many millions of dollars. * California State Auditor’s comments begin on page 117. California State Auditor Report 2005-6 55 4 • The Bureau found no evidence of service or responsiveness problems. 4 • The Bureau found no evidence of operating control problems or weaknesses • The Bureau found less specificity in the objectives given to CENIC for this project than it would have liked to have seen. Nevertheless, the Bureau found that CENIC complied with the terms of the project and that CENIC has been highly successful in constructing and operating a well designed network providing K-12 entities with better quality at less cost than could be achieved elsewhere. • The Bureau has suggested that the K-12 lead entity would have more control over problem resolution if there was a service level agreement in place with the K-12 lead entity. In raising this suggestion, the Bureau only notes the hypothetical nature of a service level issue-the suggestion, therefore, is not made in response to performance or service issues. CENIC does not believe service level agreements are needed in member type organizations as the “customers” are on the Board of Directors and are able to directly influence service levels. However, we are willing to do everything reasonable to be responsive to the suggestions of the audit. CENIC is proud of its work in supplying California educational institutions with first rate networking at reasonable costs. We look forward to continuing our on going relationship. Sincerely, (Signed by Jim Dolgonas) Jim Dolgonas President and COO 66 California State Auditor Report 2005-6 COMMENTS California State Auditor’s Comments on the Response From the Corporation for Education Network Initiatives in California To provide clarity and perspective, we are commenting on the Corporation for Education Network Initiatives in California’s (CENIC) response to our audit. The numbers below correspond to the numbers we have placed in its response. 1 CENIC overstates our conclusion. Specifically, on page 41 of the report we conclude that CENIC’s new methodology for setting certain fees, although still not based on participants’ actual network usage, represents a reduction in the annual shared costs to be borne by the High-Speed Network and appears reasonable. However, we did not conclude that CENIC’s prior methodology was reasonable. Specifically, as discussed on page 40 of the report, we question the number of High-Speed Network node sites and circuits CENIC used in calculating the fees for fiscal years 2002–03 through 2004–05, because CENIC did not update this information after its initial calculation in fiscal year 2002–03. Additionally, we also point out on pages 41 through 43 that CENIC could further reduce the amount it charges users to access the commodity Internet by consistently using funds left over from prior-year fees to offset the next year’s cost of providing the service. 2 CENIC’s statement is inaccurate. We did not compare CENIC’s costs to those of comparable organizations. Rather, we compared CENIC’s pricing arrangements for circuits, fiber-optic cable, and equipment to industry pricing or other state-negotiated pricing arrangements. However, as we state on page 73, by leveraging the State’s buying power, CENIC has been able to negotiate substantial discounts on network components such as fiber-optic cable for its California Research and Education Network (CalREN) backbone and circuits and equipment. 3 CENIC mischaracterizes our conclusions. Specifically, as stated on page 73, we could not quantify the savings because the State would have to go through a bid process to price the various California State Auditor Report 2005-6  alternatives. Further, as stated on page 30, it is our belief that because the State spends a large amount of money for state-funded entities, including the High-Speed Network, to participate in CENIC’s CalREN, it seems appropriate for these entities to bargain for ownership of the respective tangible, nonshared assets. Finally, our report does not suggest that each kindergarten through 12th grade entity should own the equipment instead of CENIC. Rather, as stated on page 30, we believe that it is appropriate for the Imperial County Office of Education to include a provision in its contract with CENIC to ensure the State’s ownership of tangible nonshared assets. 4 CENIC’s statements are inaccurate. Specifically, on page 11 we present the scope of our audit. The Joint Legislative Audit Committee did not request the Bureau of State Audits to review the operational aspects of the network. Therefore, we did not conclude on whether or not there were any service or responsiveness problems or operating control problems or weaknesses.  California State Auditor Report 2005-6 cc: Members of the Legislature Office of the Lieutenant Governor Milton Marks Commission on California State Government Organization and Economy Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press California State Auditor Report 2005-6