FCMAT
Yuba County Office of Education Report
data management review
Read the report at Yuba County Office of Education ↗
Yuba County Office of Education
Data Management Review
May 31, 2016
Joel D. Montero
Chief Executive Officer
Fiscal crisis & ManageMent assistance teaM
May 31, 2016
Josh Harris, Superintendent
Yuba County Office of Education
935 14th Street
Marysville, CA 95901-4149
Dear Superintendent Harris:
In December 2015, the Yuba County Office of Education and the Fiscal Crisis and Management
Assistance Team (FCMAT) entered into an agreement for a data management study. Specifically, the
agreement states that FCMAT will perform the following:
1. Review the COE’s practices for gathering, maintaining, analyzing and reporting
student and staff data stored in Pupil Records Online Management Information
System (PROMIS) and/or reported to the California Longitudinal Pupil
Achievement Data System (CALPADS), California Basic Educational Data
System (CBEDS) or Online Public Update for Schools – County District School
(OPUS-CDS) to determine if the COE’s data management practices need to be
modified to increase accuracy and/or efficiency. The team will evaluate the current
workflow and distribution of data management responsibilities in the COE for
PROMIS and CALPADS.
This component will include reviewing documentation such as board policies and
administrative regulations; desk or procedure manuals related to gathering data;
entering data into PROMIS, CALPADS, CBEDS and/or OPUS and reviewing
data for accuracy; job descriptions for positions with responsibility for gathering,
maintaining, auditing, using or reporting data from PROMIS, CALPADS,
CBEDS or OPUS; and enrollment forms and other forms used to gather data,
including but not necessarily limited to forms related to National School Lunch
Program/income eligibility, discipline and inter-district transfers. Additionally,
FCMAT may interview other COE or site staff to determine the efficiency and
effectiveness of data management responsibilities.
2. Review staff assignments with respect to data management for PROMIS,
CALPADS, CBEDS and OPUS, including both staff with primary responsibilities
and staff who are cross-trained to assist in the event of a staff shortage such as
a sudden absence due to a serious health issue. Make recommendations, if
appropriate, to modify the distribution of workload among staff members
and/or reporting relationships to improve data management practices.
3. Review training and professional development programs for data staff
members responsible for PROMIS, CALPADS, CBEDS and OPUS and
make recommendations as applicable.
This final report contains the study team’s findings and recommendations in the above areas of
review. FCMAT appreciates the opportunity to serve the Yuba County Office of Education, and
extends thanks to all the staff for their assistance during fieldwork.
Sincerely,
Joel D. Montero
Chief Executive Officer
Fiscal crisis & ManageMent assistance teaM
i
TABLE OF CONTENTS
Table of Contents
Foreword .................................................................................................iii
Introduction ............................................................................................1
Background ......................................................................................................1
Study and Report Guidelines .....................................................................2
Study Team.......................................................................................................3
Executive Summary ..............................................................................5
Data Management Best Practices ....................................................7
Findings and Recommendations ...................................................13
Practices and Procedures .........................................................................13
Data Management Staff Assignments ..................................................15
Training and Professional Development ..............................................17
Appendices ............................................................................................21
Yuba CountY offiCe of eduCation
ii
Fiscal crisis & ManageMent assistance teaM
iii
ABOUT FCMAT
About FCMAT
FCMAT’s primary mission is to assist California’s local K-14 educational agencies to identify,
prevent, and resolve financial, human resources and data management challenges. FCMAT
provides fiscal and data management assistance, professional development training, product
development and other related school business and data services. FCMAT’s fiscal and manage-
ment assistance services are used not just to help avert fiscal crisis, but to promote sound financial
practices, support the training and development of chief business officials and help to create
efficient organizational operations. FCMAT’s data management services are used to help local
educational agencies (LEAs) meet state reporting responsibilities, improve data quality, and
inform instructional program decisions.
FCMAT may be requested to provide fiscal crisis or management assistance by a school district,
charter school, community college, county office of education, the state Superintendent of Public
Instruction, or the Legislature.
When a request or assignment is received, FCMAT assembles a study team that works closely
with the LEA to define the scope of work, conduct on-site fieldwork and provide a written report
with findings and recommendations to help resolve issues, overcome challenges and plan for the
future.
FCMAT has continued to make adjustments in the types of support provided based on the changing
dynamics of K-14 LEAs and the implementation of major educational reforms.
Studies by Fiscal Year
90
80
70
60
50
40
30
20
10
0
92/93 93/94 94/95 95/96 96/97 97/98 98/99 99/00 00/01 01/02 02/03 03/04 04/05 05/06 06/07 07/08 08/09 09/10 10/11 11/12 12/13 13/14 14/15
Yuba CountY offiCe of eduCation
seidutS
fo
rebmuN
FCMAT also develops and provides numerous publications, software tools, workshops and
professional development opportunities to help LEAs operate more effectively and fulfill their fiscal
oversight and data management responsibilities. The California School Information Services (CSIS)
division of FCMAT assists the California Department of Education with the implementation
of the California Longitudinal Pupil Achievement Data System (CALPADS) and also maintains
DataGate, the FCMAT/CSIS software LEAs use for CSIS services. FCMAT was created by
Assembly Bill (AB) 1200 in 1992 to assist LEAs to meet and sustain their financial obligations.
AB 107 in 1997 charged FCMAT with responsibility for CSIS and its statewide data management
work. AB 1115 in 1999 codified CSIS’ mission.
iv
ABOUT FCMAT
AB 1200 is also a statewide plan for county offices of education and school districts to work
together locally to improve fiscal procedures and accountability standards. AB 2756 (2004)
provides specific responsibilities to FCMAT with regard to districts that have received emergency
state loans.
In January 2006, Senate Bill 430 (charter schools) and AB 1366 (community colleges) became
law and expanded FCMAT’s services to those types of LEAs.
Since 1992, FCMAT has been engaged to perform more than 1,000 reviews for LEAs, including
school districts, county offices of education, charter schools and community colleges. The Kern
County Superintendent of Schools is the administrative agent for FCMAT. The team is led by
Joel D. Montero, Chief Executive Officer, with funding derived through appropriations in the
state budget and a modest fee schedule for charges to requesting agencies.
Fiscal crisis & ManageMent assistance teaM
1
INTRODUCTION
Introduction
Background
Yuba County is located approximately 45 miles north of Sacramento and is the fifth-smallest
county in California by total area. There are five school districts in Yuba County. As of the
2014-15 census day, 13,976 students were enrolled in the county’s schools, including approxi-
mately 504 students served by the county office schools. According to data from the California
Department of Education (CDE), enrollment in the county office schools has fluctuated during
the past several years including a high of 636 students in 2010-11.
Yuba COE serves a highly mobile student population with academic, physical and behavioral
challenges at the following schools and programs:
• Yuba County Career Preparatory Charter Academy, a county-authorized, technology-
focused charter school serving students in classroom-based and independent study
programs.
• Harry P.B. Carden Court School for incarcerated students.
• Thomas E. Matthews Community School.
• Regional moderate/severe Special Education programs, including Virginia School (a
stand-alone Special Education site), Agnes Weber Meade, Anna Bell Karr, Spring Valley
and other Yuba COE classrooms serving moderate/severe students (pre-K through 22) on
host district sites.
• Yuba Environmental Science Charter Academy (not included in the study scope).
Yuba COE has undergone multiple changes in the last year, including several changes in key
staff positions, with a new superintendent and new principals at the charter and community
schools. Yuba COE uses a variety of information systems. At the beginning of the 2015-16
school year, Yuba COE replaced its student information system, Aeries, with the Pupil Records
Online Management Information System (PROMIS). Yuba COE also uses the Special Education
Information System (SEIS) for online access to IEPs, managing Special Education data, service
tracking, and required reporting of special education students. Both PROMIS and SEIS are
hosted, maintained and supported by San Joaquin County Office of Education.
PROMIS was designed for county programs and students (alternative education, court, juvenile
and community schools, regional occupational programs, and special education). SEIS is
California’s most widely used Special Education software program. SEIS helps to standardize
individualized education program (IEP) forms, goal writing, and transferability (over 3.5
million electronic student record transfers have occurred between districts in SEIS). The Yuba
COE’s charter and community schools use Illuminate for assessment data and gradebooks, and
Schoolwide Information System (SWIS) as a supplemental system to track behavioral inter-
ventions and discipline. The charter and community schools also use Renaissance Reading as a
student learning and evaluation resource.
In addition to the student data systems, Yuba COE uses Escape for its financial and human
resources information system and several FileMaker Pro and Access databases for maintaining
credential and employee placement data.
Yuba CountY offiCe of eduCation
2
INTRODUCTION
Implementation of Aesop, a web-based system for staff attendance tracking, substitute requests
and absence calling, is planned.
PROMIS SEIS Illuminate SWIS
All COE Students Special Ed Alt Ed Students Alt Ed Students
CALPADS
Demographics
Enrollment
Attendance
Grades
Special Programs Demographics Demographics
Discipline Demographics Gradebook Positive Behavior
Staff Info IEPs Assessments Interventions
Escape FileMaker Pro Access Aesop
County Office County Office County Office County Office
(not implemented)
Financial Staff Attendance
Human Resources Credentials Credentials Sub Requests
Study and Report Guidelines
FCMAT visited the Yuba County Office of Education on February 22-24, 2016 to conduct
interviews, collect data and review documents. This report is the result of those activities and is
divided into the following sections:
• Executive Summary
• Data Management Best Practices
• Findings and Recommendations
• Appendix A: Data Management Best Practice Resources
• Appendix B: Sample Job Description
Although the scope of this study is limited to issues related to PROMIS, CALPADS, CBEDS,
and OPUS, some of the recommendations may apply to the other information systems in use at
the county office.
In writing its reports, FCMAT uses the Associated Press Stylebook, a comprehensive guide to
usage and accepted style that emphasizes conciseness and clarity. In addition, this guide empha-
sizes plain language, discourages the use of jargon and capitalizes relatively few terms.
Fiscal crisis & ManageMent assistance teaM
tnedutS
CBEDS
OPUS-CDS
ssenisuB
State
Reporting
3
INTRODUCTION
Study Team
The study team was composed of the following members:
Nancy Sullivan Martha Friedrich
CSIS Chief Operations Officer CSIS Client Services Officer
Sacramento, CA Sacramento, CA
Lisa Hayes Andrea Dodson Alvarado
CSIS Intervention Specialist FCMAT Management Analyst
Modesto, CA Bakersfield, CA
Laura Haywood
FCMAT Technical Writer
Bakersfield, CA
Each team member reviewed the draft report to confirm the accuracy and to achieve consensus
on the final recommendations.
Yuba CountY offiCe of eduCation
4
Fiscal crisis & ManageMent assistance teaM
5
EXECUTIVE SUMMARY
Executive Summary
FCMAT conducted a review of the Yuba County Office of Education data management practices
relative to PROMIS, CALPADS, CBEDS and OPUS to identify if changes needed to be made
to increase accuracy and/or efficiency; make recommendations, if appropriate, to modify the
distribution of workload among staff members and/or reporting relationships to improve data
management practices; and if applicable, make recommendations regarding training and profes-
sional development for data staff members responsible for PROMIS, CALPADS, CBEDS and
OPUS.
The COE’s data management practices were evaluated against best practices found in FCMAT/
CSIS’ experience to yield accurate, reliable data that is available when needed for local use as
well as state and federal reporting. These are articulated in the Data Management Best Practices
section of the report.
The team conducted interviews regarding processes for gathering, maintaining, analyzing and
reporting data to OPUS-CDS, CBEDS-ORA and CALPADS. No problems were reported with
OPUS-CDS and CBEDS-ORA reporting, and no concerns were expressed about data quality,
efficiency or effectiveness related to required reporting in those systems. Numerous problems
were reported with the COE’s practices for gathering, maintaining, analyzing and reporting
data to CALPADS. The COE lacks organization-wide procedures that ensure enrollments in
CALPADS are consistently maintained. No formal process exists to have CALPADS reports
reviewed and signed off as accurate by county office administrators, site administrators or
program coordinators prior to certification. Staff and administrators interviewed reported
little, if any, organizational focus on developing standardized data management procedures.
Many staff reported they had written their own desk procedures, but they also indicated their
procedures were likely not current for the new student information system (SIS). The COE
has no procedures for developing common forms for use in data collection, nor is there a
process for reviewing forms each year to determine if they need to be updated as state or federal
requirements change. Development and implementation of consistent practices for collecting,
maintaining and reporting data will help ensure the COE can gather, report and use accurate,
reliable data.
No formal processes have been established to systematically analyze the information systems
for missing or erroneous data, or to verify the consistency of data stored in multiple systems.
Those interviewed reported the COE does not hold submission debriefs or conduct data quality
audits. No other processes are used to identify common problems so these issues can be addressed
globally, nor is there a common process to identify priorities for changes or enhancements in the
student information system. Practices such as submission debriefs and/or data quality audits will
help the COE identify and prioritize changes needed to improve its current processes.
The job descriptions for key data management positions were out of date at the time of the
review, and roles and responsibilities for data management are not clearly defined. It was
unclear from the interviews who is responsible for ensuring data is submitted to CALPADS in
a timely manner, reviewed for accuracy, and certified as accurate. Some of the data collection
and reporting is duplicative and/or contradictory. School site administrators and staff work
independently at their sites on data management tasks. Individual users have created solutions for
some of the challenges they encounter, but their processes are not always shared with other users
who might benefit from their knowledge. Clearly defining roles and responsibilities and assigning
Yuba CountY offiCe of eduCation
6
EXECUTIVE SUMMARY
one person as responsible for coordinating CALPADS work will facilitate local efforts to collect
and report accurate data.
County office and school site staff interviewed indicated little communication and coordination
regarding data management activities. No regular meetings are held to coordinate work or
resolve issues. County data management staff do not visit school sites and there is no designated
COE data management team. Creating a COE-wide data management team facilitated by the
employee responsible for CALPADS will help improve communication and resolve key issues
identified by staff. The team could develop and use data management calendars to coordinate
data collection for required state and federal reporting. It could also evaluate cumbersome
processes or inaccurate/incomplete data and develop improvement plans to resolve these issues.
Many staff reported training for the new student information system last fall was rushed and they
might benefit from follow-up training and support. At the time of the interviews in February the
COE had no formal mechanism for ongoing or intermittent follow-up training, nor was there
an effort to identify unmet training needs. Cross training is not a focus across the organization,
and staff expressed concern about how essential data management responsibilities would be
completed if key staff members were unexpectedly absent for an extended period of time. Staff
reported that in the past they had ongoing learning opportunities to develop their expertise in
the student information system and learn about upcoming changes in state and federal reporting
requirements. At the time of the review staff indicated they were not sure there would be these
types of opportunities with PROMIS. The COE should identify where staff continue to struggle
with PROMIS and provide targeted follow-up training as well as ongoing learning opportunities
for staff to develop expertise in PROMIS and data management best practices.
Fiscal crisis & ManageMent assistance teaM
7
DATA MANAGEMENT BEST PRACTICES
Data Management Best Practices
Data management best practices are those repeatable practices that support the collection, main-
tenance and use of accurate, timely, reliable and secure data. They can be categorized into three
components: clear procedures, well-trained people and the technical and physical infrastructure.
For the purposes of this report, infrastructure includes hardware, software, network resources and
services required for the existence, operation and management of the organization’s information
technology environment used to gather, maintain, report and use data. Best practices include
aligning data management practices and priorities with the organizational mission and vision
so reliable data are available to support the priority work of the organization. This alignment
also informs decisions involving weighing the best educational solution with the best technical
solution and the associated costs.
Those best practices as well as considerations for establishing local practices are described in this
section of the report.
Procedures
Local data management processes and procedures describe the tasks and detailed steps the local
education agency (LEA) staff complete to collect, maintain, store and report data. Procedures
address how staff members interact with data, data systems in use at the LEA and other staff
members on data-related issues. Procedures also describe how staff maintain confidentiality and
keep the data secure. Local data needs and priorities, as well as state and federal laws, guide what
data must be reported and how they are reported.
Developing sustainable practices for populating and maintaining data involves consideration of
factors such as the purpose of the data collection, when and how in the normal business process
the data are collected, when the data are needed for reporting or local decision making, issues
encountered with data quality in the past, and the staff roles involved with the data. Staff roles
and responsibilities are addressed in the People section of this report. Best practices for gathering
data include entering new or changed information into the system of record for that data element
at the time it is collected or as soon as possible afterward. Using software features to prevent
invalid or missing entries facilitates the collection of accurate, complete data. When a new
student information system (SIS) is adopted, enrollment forms and other forms used by the LEA
should be reviewed to ensure the flow of the form is consistent with the order in which the data
elements will be entered into the system.
Practices for maintaining data include planning for how updated information (e.g., student exits,
course changes, academic year rollover) will be gathered and entered into the SIS in a timely
fashion. The use of data entry checklists and calendars with due dates also helps ensure accurate,
complete data are available when the LEA needs it. Student enrollments should be kept current
in CALPADS as students enter and exit schools. The California Department of Education
(CDE) recommends updating CALPADS at least every two weeks, or more frequently for highly
mobile student populations like Yuba COE’s (see CDE’s Statewide Student Identifier (SSID) and
Enrollment Procedures, Chapter 3, http://www.cde.ca.gov/ds/sp/cl/systemdocs.asp). Best practice
for LEAs with highly mobile populations is to capture these changes daily. In addition, mainte-
nance activities include having program coordinators and site administrators review reports for
accuracy and sign off on certifications when they agree reports are accurate. Another important
consideration is ensuring the privacy and confidentiality of student data in accordance with the
Family Educational Rights and Privacy Act and other state and federal privacy laws. Best prac-
Yuba CountY offiCe of eduCation
8
DATA MANAGEMENT BEST PRACTICES
tices also involve examining the work environment of those involved in data entry and analysis to
minimize the distractions that lead to errors.
Another best practice is to require each staff member to develop and maintain step-by-step
directions or desk procedures that describe how to complete his/her essential data management
tasks and to save the procedures so others in the organization may access them. These directions
should include detail that allows others to complete the task in a manner consistent with that of
the primary staff member. Including screenshots with the directions when possible enhances the
understandability of these procedures. Current desk procedures are invaluable to the organization
if the staff member who normally performs the tasks is unexpectedly absent. They also facilitate
the training of new staff. In addition, the documentation of procedures often surfaces gaps in
internal processes and enables the organization to develop methods to address these gaps before
staff members unfamiliar with the work are left to determine how to do so.
Infrastructure
The overwhelming majority of LEAs in California use automated commercial student informa-
tion, human resources and financial systems to manage data about their students, programs, staff
and operations. The implementation of a new SIS or any other software application for student
or staff data involves multiple considerations:
• Identifying the business requirements.
• Determining and allocating resources for all operations and maintenance costs, including
purchase/licensing fees, the infrastructure needed to use the software, professional
development and staff time maintaining and using the system.
• Planning how the data are initially populated.
• Developing sustainable local practices for populating and maintaining data.
• Supporting the system users.
All information systems should facilitate effective and efficient business processes.
Defining requirements for and implementation of student information systems includes identi-
fying data needed to support daily operational needs and state and federal reporting, as well as
making decisions about the data to be captured in the context of the LEA. For example, will the
system capture attendance by the day, period, or simply positive attendance? Does the system
need to accommodate alternative schedules, block scheduling, or independent study? How are
grades (traditional progress reports and report cards) reported? Decisions must be made related
to high school requirements (tracking graduation requirements, producing transcripts, and calcu-
lating multiple types of GPAs), as well as requirements for tracking special student populations,
local goals, special programs and local initiatives. This activity includes data needed to develop or
monitor the Local Control and Accountability Plan (LCAP) goals and priorities (e.g., identifying
targeted populations, providing services, evaluating effectiveness, and quantifying progress).
Although most student information systems are quite comprehensive, it is very common for the
selected SIS to meet most, but not all, of the local needs. Implementation, therefore, includes
making plans to address functionality that is not in the selected system or that requires a change
in local business practices. Local actions include identifying gaps, establishing the importance/
weight of the missing features and then weighing the costs associated with one or more optional
modules offered by the SIS vendor, costs of custom features in the SIS to address the gap, the
Fiscal crisis & ManageMent assistance teaM
9
DATA MANAGEMENT BEST PRACTICES
costs of staff workarounds, and/or the cost and tradeoffs associated with using another software
program to meet needs. A total cost of ownership approach to the cost analysis that includes all
initial and ongoing costs of the system(s), staff time, training, etc. will ensure the accurate consid-
eration of the long-term costs of each option. If the LEA is considering purchasing a separate
software application to address gaps in the SIS, the analysis should include whether or not the
other software program includes an automated data sharing protocol so staff are not required to
maintain basic information (e.g., student names, identifications numbers, demographics, etc.) in
both systems. Manual data entry in multiple systems requires staff time and typically results in
inconsistent data and additional time spent resolving discrepancies between systems. When data
elements are stored in multiple software applications, a system of record (the authoritative data
source) should be identified for each data element so users know which system to use if they need
that data.
Selection and implementation of the SIS includes deciding whether the hosting and technical
implementation and maintenance will be done locally or by the SIS provider. Decisions include
who will install and maintain the software and hardware, perform backups, convert the data,
set up reference tables, establish user accounts and security roles, etc. Factors influencing these
decisions include cost, availability/cost of qualified staff, and attributes of the SIS, including
adaptability, scalability, the potential for customization, and ease of use.
Planning for the initial population of data may involve decisions regarding converting the data in
the former SIS and/or in current data systems. Typically multiple conversions are done with an
analysis of the results each time so routines can be refined until they accurately bring all needed
data from the old system into the new SIS. Analysis of options associated with using the vendor
to convert the data or having staff do this work must take staff availability and expertise into
account. If the LEA chooses not to convert all of the data in the old SIS, plans must be made to
address gaps and communicate to staff how to access this data when needed. If staff will manually
enter some of the data into the new SIS, specific time for data entry and for validating its accu-
racy needs to be built into their schedules. Many LEAs in this situation consider augmenting staff
hours or bringing in temporary help.
People
Best practices for data management relative to LEA staff involve roles and responsibilities,
communication and coordination, and ongoing professional learning.
Roles and Responsibilities
Clear assignment and communication of roles and responsibilities builds a common under-
standing about who is responsible for data and reporting and establishes expectations and
accountability for how these roles will be fulfilled. Staff roles, such as system administrator
(responsible for establishing accounts, ensuring backups are made, etc.), support/help desk staff
(responsible for responding to user questions about system functionality and issues with its use),
data stewards (responsible for the management and fitness of data elements) and data owners
(those ultimately accountable for a particular data set or student set) need to be articulated
and shared. Users need to know who to contact if they have questions or encounter problems.
They also need to understand who to contact with questions about a particular data element or
process. Best practices include having one point of contact for questions and using a knowledge
base or help desk system to capture questions and responses so they can be used for continuous
improvement.
Yuba CountY offiCe of eduCation
10
DATA MANAGEMENT BEST PRACTICES
Best practice is to document roles and responsibilities in duty statements/job descriptions so all
staff know the scope of each person’s responsibilities. Including in the job description the data
elements or sets of data each data owner/steward oversees as well as activities such as submitting
or reviewing data per the established calendar, coordinating data submissions, trainings and
communications clarifies expectations and provides the basis for holding staff members account-
able. Job descriptions should also state the requirements for creating and maintaining desk
procedures and for keeping data secure and confidential. Specifying a lead and a backup for each
activity clarifies who is ultimately responsible and accountable for the work and who is assigned
to help if the employee with primary responsibility needs assistance or is out of the office when
a critical question/issue arises. The use of a Responsible, Accountable, Consulted or Informed
(RACI) matrix may help in communicating roles and levels of involvement. See Appendix A for
more information about the RACI matrix.
All involved in data management activities should meet regularly to coordinate the many activ-
ities and roles involved in collecting and maintaining accurate and reliable data and to establish
the data management schedule/calendar for the year, as well as to make recommendations to
management about needed changes, such as what data to collect, how to collect it, who owns
it and how it is accessed and shared. Data management team meetings build a common under-
standing of how the data are used, the importance of privacy and confidentiality, and the conse-
quences of poor quality data. This team helps build organizational buy-in for needed changes and
mitigates risks associated with change since those involved have the opportunity to help create
new procedures before the changes are implemented.
Another best practice is to have the data management team conduct a debrief following each
major data collection or data event to document what worked well, issues that need resolution
and opportunities for improvement. Holding these debriefs soon after a major effort is complete
helps to capture useful information so that needed changes/improvements can be discussed,
approved and implemented before the event occurs again. See Appendix A for a sample agenda
for a submission debrief.
Some organizations also find it helpful to perform data quality audits, which are reviews that
assess how the organization’s data fits a given purpose. The benefits of data quality audits are
twofold: they typically result in improved data accuracy and can also promote an increased
sense of responsibility or ownership of the data. Most often, data quality audits involve running
embedded system processes or queries to determine if critical fields are blank or contain invalid
entries, such as a birth year being greater than the current year. These audits may focus on a
limited data set, such as data needed for CALPADS Fall 1 reporting, or might focus on one or
more specific student subgroups (e.g., students receiving special education services) to explore
whether all of the needed data are collected and if issues impacting data quality can be iden-
tified and resolved. Data quality audits can also involve selecting a sample of student records
and reviewing the source documentation in the SIS to determine if data were completely and
correctly captured. LEAs doing these types of audits typically focus on high-stakes data such as
that used to determine Local Control Funding Formula supplemental and concentration grants
to ensure the LEA has the proper documentation prior to the formal LEA audit. Page 33 of the
2015-16 Guide for Annual Audits of K-12 Local Education Agencies and State Compliance
Reporting provides the state process for auditing this data. LEAs often use this section in devel-
oping their own data quality audit. The audit guide may be downloaded from http://eaap.ca.gov/
audit-guide/current-audit-guide-booklet/. Best practice is to identify the source of the error and
to require the person who entered the data to correct it so they are encouraged to improve prac-
tices for collecting and entering quality data.
Fiscal crisis & ManageMent assistance teaM
11
DATA MANAGEMENT BEST PRACTICES
Data management leaders play a critical role through their actions as they communicate the
relative importance of data and data quality to the organization, advocate for consistent data
management practices, and hold staff accountable for successfully completing their responsibil-
ities. Leaders work with the local board to establish policies and then consistently implement
them across the LEA. Leaders also ensure staff members responsible for data management have
the time and resources to complete their work, including opportunities to learn and collaborate,
and access to tools such as a collaborative workspace and a knowledge base or support ticket
system. Leaders also recognize staff commitment and successful completion of data management
activities.
While all leadership team/cabinet members can and should help reinforce the importance
of good data management to the organization, organizations involved in the early stages of
improving their data management practices typically designate one leadership team member
to be responsible for guiding the effort. Having the primary data coordinator report to that
leadership team member facilitates communication, accountability and resolution of issues about
high-stakes data, including data used for Local Control Funding Formula (LCFF) calculations,
transcript data, and graduation and dropout rates. The designated leadership team member
should be responsible for looking at data and data management organization-wide, not from a
particular section of the organization. Leadership for data management also includes holding
program coordinators and principals accountable for regularly reviewing data reports for accuracy
and approving submission reports prior to their certification for state reporting.
Communication and Coordination
Good communication and coordination play an essential role in data management best prac-
tices. Individuals and the data management team should understand the role clear and timely
communication plays in building common understanding, setting expectations, reducing
confusion and frustration, reinforcing the importance of data to the organization, and building
the sense of connection with other team members and the overall mission and goals of the
organization. Staff members and leaders should be held accountable for clearly communicating
with others involved in the collection, maintenance and use of data. Those responsible for state
and federal reporting, and those involved in reporting data for local needs (e.g., a Local Control
and Accountability Plan stakeholder meeting) need to be committed to and held accountable
for clearly communicating what needs to be collected, by when, and why. Highlighting recent
changes in requirements will help all those involved develop and maintain a common under-
standing of requirements. Creating and sharing calendars and reminders of what is needed for
each new school year or submission will help ensure certain tasks are not overlooked, especially if
there is a staff change or responsible staff are juggling many competing priorities. These calendars
should allow for ample time to correct input validation or certification errors/warnings, and time
to send reports to program coordinators and site principals for review and correction prior to the
deadline. Practices that facilitate good communication and coordination include providing staff
with help desk systems, shared drives or collaborative workspaces to facilitate sharing of proce-
dures, and formalizing processes so all involved have a common understanding about what will
occur and the roles of those involved.
Professional Learning
Staff and leadership involved in data management need opportunities to master the tools
involved in meeting their job expectations/responsibilities, gain expertise in data management
best practices, and develop an understanding of the evolving role and use of data in improving
Yuba CountY offiCe of eduCation
12
DATA MANAGEMENT BEST PRACTICES
K-12 education. For those involved with using software applications such as the SIS, best practice
is to provide initial training upon hiring or implementation of a new SIS and then provide
follow-up training several months after implementation. This ensures individuals can get answers
to questions and issues encountered during the first few weeks of using the system. Best practice
allows for periodic or targeted training based on individual needs. In addition, sending those
responsible for data collections to annual training through the software vendor, a user group and/
or government agencies involved in supporting the collection (e.g., CALPADS trainings provided
by FCMAT/CSIS) ensures individuals are aware of upcoming changes and have the time to
adjust local practices as needed. These annual trainings also allow staff to learn concepts they may
have missed in the initial training and to explore helpful software features not currently in use
at the LEA. Staff members may also benefit from joining a listserv so they have access to others
performing the same work. These listservs offer a community support group that can provide
assistance or examples of procedures to resolve potential issues and may raise staff members’
awareness of potential threats and how to avoid them.
Professional learning should also include opportunities for cross training staff so that the desig-
nated backup staff understands how to perform the duties of the primary staff.
See Appendix A for resources on data management best practices.
Fiscal crisis & ManageMent assistance teaM
13
FINDINGS AND RECOMMENDATIONS
Findings and Recommendations
Practices and Procedures
No problems were reported with OPUS-CDS and CBEDS-ORA, and no concerns were
expressed about data quality, efficiency or effectiveness related to required reporting in those
systems.
Numerous problems were reported with the COE’s practices for gathering, maintaining,
analyzing and reporting data to CALPADS. The COE lacks organization-wide procedures
that ensure enrollments in CALPADS are consistently maintained. Staff reported that because
enrollments and exits are not always uploaded into CALPADS timely, the COE has a number
of students who appear to have issues with their statewide student identifiers (SSID), such as
two students who have the same SSID or one student with multiple SSIDs. Those interviewed
reported different understandings of the comprehensive requirements for annual reporting.
No formal process exists to have CALPADS reports reviewed and signed off as accurate by
county office administrators, site administrators or program coordinators prior to certification.
The principals and county office administrators interviewed indicated they did not review and
approve CALPADS Fall 1 reports in fall 2015 prior to certification, although some mentioned
reviewing CALPADS reports in prior years. Certification of the Fall 1 submission was completed
by the same person at level one and level two. Those interviewed indicated they feared the lack
of a consistent process had resulted in missing and/or incorrect data, potentially causing a loss of
funding to the county office.
All Yuba COE staff are required to review and sign acceptable use policies every year that include
guidelines for ensuring student data privacy and confidentiality. Employees interviewed indicated
they understood that practices needed to be in place to keep student data secure and confidential.
No concerns were raised regarding data privacy.
Staff and administrators interviewed reported little, if any, organizational focus on developing
standardized data management procedures. Many staff reported they had written their own
desk procedures, but they also indicated their procedures were likely not current for the new
SIS. There were no guidelines or consistent format. Individual procedures are stored on local
drives and thus inaccessible to other users should the need arise. There is no systematic process
for updating or reviewing desk procedures. All schools received timelines and checklists for
attendance reporting at the beginning of the school year. Other than the attendance procedures,
the COE has not developed or used local calendars or checklists of tasks to be completed daily,
weekly, monthly or yearly to hold staff accountable for completing tasks on time.
The Special Education and alternative schools use a standard county office enrollment form. The
charter school uses a different form that collects the same information. Staff reported this form
has not been reviewed to determine if the order of questions is consistent with the data entry
flow for PROMIS. Staff indicated they are unaware of any annual process to review the form and
ensure it is updated as state or federal requirements change. A student protocol form/checklist
was developed for the previous system to enroll students in the SIS and ensure all required infor-
mation was collected and entered. No similar form was found for use with PROMIS. Special
Education has a checklist used in IEP meetings to obtain all required data for SEIS.
National School Lunch Program (NSLP) applications are generally provided to parents/guardians
with the enrollment packets. Marysville Joint Unified School District processes the applications,
Yuba CountY offiCe of eduCation
14
FINDINGS AND RECOMMENDATIONS
identifies eligible students and provides meals to the Yuba COE alternative schools and students.
Wheatland Elementary processes and provides meals for the Virginia School students. The NSLP
application process was not communicated adequately to the charter and Special Education
schools, which may not have obtained applications from each parent. Those interviewed
indicated that when the Marysville staff changed the information regarding students eligible
for free and reduced price meals, it was not consistently provided back to the schools serving
those students. As a result, the information was not updated in the SIS or reported to the state,
resulting in a potential loss of funding under the LCFF and other sources that use information
about low-income students to qualify schools for state and federal grants.
No formal processes have been established to systematically analyze data in the information
systems for missing or erroneous data, or to verify the consistency of data stored in multiple
systems. Those interviewed reported the COE does not hold submission debriefs or conduct data
quality audits. No other processes are used to identify common problems so these issues can be
addressed globally, nor is there a common process to identify priorities for system changes or
enhancements. Multiple issues with CALPADS data and attendance reporting and reconciliation
were reported. Due to the highly mobile population and ease of use, some attendance data is
maintained in supplemental systems and manually entered into PROMIS at the end of the day.
School site staff indicated attendance reports are reviewed daily and school secretaries work with
teachers to resolve any missing or problematic data.
Issues also were reported with course and staff data. Staff reported since roles and responsibilities
are not clear (see the Data Management Staff Assignments section for more information), data
reconciliation is inefficient as multiple people perform essentially the same tasks with little or
no communication or coordination. Those interviewed indicated that at times one staff member
will correct data and then another staff member will edit that data again, making it inaccurate.
Staff and COE leadership team members indicated that county administrators expect issues to
be resolved at the staff level. Staff reported they sometimes have different understandings of the
best way to proceed. When this occurs individual staff members do what they think is best even
if it results in inconsistent approaches. These issues are not brought to the attention of individual
supervisors for help in resolving the issues and building consistent practices.
Infrastructure
The transition from Aeries to PROMIS occurred quickly. Not all data was converted from Aeries
to PROMIS, which created problems with historical data. The county office maintains a server
with the historical Aeries data, but it is not readily accessible to users on PROMIS. Staff and site
administrators interviewed indicated considerable time had been spent keying historical data into
PROMIS so it is available for use.
Although PROMIS can be integrated with SEIS for an additional cost, Yuba COE had not opted
to purchase that service at the time of the interviews. Data cannot be extracted from PROMIS
without vendor support. The charter school worked with the vendors of Illuminate and PROMIS
to establish data transfers between the two systems. This functionality is only available at the
charter school and other sites are not aware of it. Credential and authorization data are used indi-
rectly for state reporting to determine whether teachers are highly qualified for the course content
they teach. Credential data is stored in multiple systems, including Escape and an internal data-
base, and it was unclear which is the authoritative source or system of record. Data is manually
extracted from the database and then loaded into PROMIS for CALPADS reporting.
Fiscal crisis & ManageMent assistance teaM
15
FINDINGS AND RECOMMENDATIONS
Multiple problems were reported with the initial implementation of PROMIS, such as teachers
not having accounts or having multiple accounts, and problems with entering data. Staff reported
laborious and cumbersome processes re-entering transcript histories. Many critical problems
appear to have been resolved by the time of the review. For example, issues with no accounts and
multiple accounts had been resolved. Additional issues occurred at the court school when it was
split between two campuses. Although the problems were unrelated to PROMIS, the system was
limited in transferring students, so it had to be completed manually.
Staff identified shortcomings with PROMIS that could be attributed to training and/or imple-
mentation issues. For example, one employee indicated she needed to get grades out of PROMIS,
but didn’t know how. Employees also stated navigation for entering attendance and discipline
data was unfamiliar and frustrating. Another mentioned it was challenging to make grade level
changes. Finally, one employee indicated all emergency contact phone numbers had to be entered
in two places in PROMIS.
Supplemental systems are used to address the limitations with grade reporting and attendance
tracking at the alternative schools. Several daily operational functions were reported as laborious,
including enrolling students, discipline reporting, and producing report cards. Charter school
report cards can only be produced 50 at a time and can take several days to generate. The
report cards can no longer be accessed if the student has been exited from the system. Manual
intervention by a system administrator is required if the student re-enrolls in another school that
uses PROMIS. Updating and producing transcripts for the highly mobile juvenile hall students
requires multiple steps and manual intervention from one of the staff supporting PROMIS to
change access rights to update and to complete the process.
The school sites use data to analyze and assess student performance and progress, but it is not
available as a global resource to inform instruction, monitor progress, and analyze all students
enrolled in the county office schools. Sites would like to have data for developing LCAPs and
analysis, but it is not easily accessible or in a form that is readily usable. Staff reported technical
limitations impacted the ability to do this type of analysis, and there are no plans or initiatives to
address this gap. Cabinet members expressed interest in having student achievement data to help
improve student performance and services provided to students.
Data Management Staff Assignments
San Joaquin COE is responsible for the technical support, maintenance and hosting of PROMIS
and SEIS. San Joaquin COE also provides user support via a help desk. Yuba COE staff are
responsible for collecting, maintaining, reporting and using the data stored in PROMIS and
SEIS.
Data management assignments are distributed across the organization. The Human Resources
director is responsible for CBEDS and OPUS reporting. Two Yuba COE staff members share the
primary data management responsibilities for PROMIS and CALPADS with school site staff.
The coordinator/secretary to the Special Education assistant superintendent and the coordinator
CALPADS/credential analyst in Human Resources share responsibility for supporting PROMIS
and CALPADS, splitting the responsibilities between Special Education schools and students
and alternative education schools and students, and related data. The coordinator CALPADS/
credential analyst previously served as the credential analyst. The same staff share responsibilities
for maintaining and reporting requirements for CALPADS. A third staff member, the budget
analyst in the Fiscal Services division, helps with attendance reporting and enrollment related
Yuba CountY offiCe of eduCation
16
FINDINGS AND RECOMMENDATIONS
questions. School site staff are responsible for entering much of the student information needed
by CALPADS into PROMIS. The CALPADS requirements for student reporting did not appear
to be well understood by county office or site staff.
The job descriptions for these positions were out of date at the time of the review. The job title
for the coordinator CALPADS/credential analyst was recently changed to add CALPADS respon-
sibilities, but the job description was blank under the heading of Coordinator for CALPADS.
The job description for the coordinator/secretary to the Special Education assistant superin-
tendent also does not provide a clear description of the CALPADS/PROMIS responsibilities.
Because not all responsibilities are clearly defined, some of the data collection and reporting is
duplicative and/or contradictory. Both positions have other responsibilities that may compete
with data management tasks.
It was unclear from the interviews who is responsible for ensuring data is submitted to
CALPADS in a timely manner, reviewed for accuracy, and certified as accurate. Staff interviewed
indicated this lack of clarity results in inefficiencies. For example, the coordinator/secretary to
the Special Education assistant superintendent and the coordinator CALPADS/credential analyst
reported they submitted files to CALPADS without coordinating with one another, resulting in
duplicate efforts and in some cases, contradictory data.
School site administrators and staff work independently at their sites on data management tasks.
Individual users have created solutions for some of the challenges they encountered with the
implementation of PROMIS, but the processes are not always shared with other users who might
benefit from their knowledge. School site staff reported confusion as to the appropriate person
to contact at the district office if they have questions or problems. Some contact the coordinator
CALPADS/credential analyst and then the coordinator/secretary to the Special Education
assistant superintendent if their questions are not answered by the time they want to act on the
answer. Others contact the coordinator/secretary to the Special Education assistant superinten-
dent and bypass the coordinator CALPADS/credential analyst. No process exists to collect all
questions in a central repository or to routinely route all questions to the other coordinator so
that both are aware of all questions being raised.
The COE leadership provides little specific direction to these staff members with respect to data
management and coordination. Staff are told to work out unclear or overlapping assignments.
COE leaders expect the coordinator CALPADS/credential analyst and the coordinator/secretary
to the Special Education assistant superintendent to work together to resolve issues, but since
there is no clear articulation of roles and responsibilities and since both are busy with their varied
assignments, each reported working independently to complete tasks they considered to be most
critical. The process for identifying responsibilities for collecting, entering, and verifying data has
been not formalized and, as a result, there is no clear accountability for data quality. In addition,
the organization has not designated a primary employee as responsible for overall coordination
and communication regarding data management practices, nor has it designated a lead for data
management on the leadership team. Administrators and site staff interviewed indicated the lack
of clarity about roles and responsibilities contributes to staff and school sites working in isolation
to resolve their own data challenges. Those interviewed indicated time, resources and opportuni-
ties for improvement were lost as a result of this isolation.
Staff and administrators interviewed indicated the coordinator CALPADS/credential analyst
certifies CALPADS reports at both level one and level two.
Fiscal crisis & ManageMent assistance teaM
17
FINDINGS AND RECOMMENDATIONS
Staff and administrators interviewed indicated there has been little focus on designating and
training staff to serve as backups in the event of a serious illness or unexpected absence. All
expressed a desire to have designated staff who were trained and could assist in filling a critical
data management role should the need arise.
Communications and Coordination
County office and school site staff interviewed indicated little communication and coordination
regarding data management activities. No regular meetings are held to coordinate work or resolve
issues. County data management staff do not visit school sites and there is no designated COE
data management team.
Support and training is requested through direct phone calls and emails to internal staff or to
San Joaquin COE. No clear policy describes when to contact the San Joaquin COE help desk
and when users are to first ask the question within Yuba COE. Requests of Yuba COE are not
formally tracked. Site users are not always certain who to contact and reported that requests for
support do not always get responses. As a result, users often ask both the coordinator CALPADS/
credential analyst and the coordinator/secretary to the Special Education assistant superinten-
dent. This informal process for fielding and responding to support questions and problems has
led to frustration for the users, duplication of effort, inefficiencies, and conflicting responses.
Requests for user accounts and assigning roles is an informal process using email that has led to
conflicts over who can update which data. The lack of a support tracking system means the COE
cannot quantify the support workload or hold staff accountable.
A COE-wide team could help to resolve key issues such as developing uniform forms for
collecting student data across all school sites and reviewing the forms annually to ensure they are
up to date with state and federal requirements.
The team could develop and use data management calendars to coordinate data collection for
required state and federal reporting. Members could evaluate data that was in Aeries but not
converted to PROMIS and develop plans to convert or re-enter any additional historical data
found to be necessary. They could also evaluate cumbersome processes or inaccurate/incomplete
data and develop improvement plans to resolve these issues.
This team would include the coordinators of CALPADS work, program staff, data entry staff,
assessment/accountability staff, pupil records staff, and COE and site leadership staff.
Training and Professional Development
San Joaquin COE provided a few days of training on PROMIS to county office and site staff
shortly before the start of the 2015-16 school year. Many staff reported they felt the training was
rushed and they might benefit from follow-up training and support. At the time of the interviews
in February the COE had no formal mechanism for ongoing or intermittent follow-up training,
nor was there an effort to identify unmet training needs.
Staff reported that in the past they had ongoing learning opportunities to develop their expertise
in student information systems and to learn about upcoming changes in state and federal
reporting requirements through vendor conferences or user groups. At the time of the review staff
indicated they were not sure there would be these types of opportunities with PROMIS.
Yuba CountY offiCe of eduCation
18
FINDINGS AND RECOMMENDATIONS
Recommendations
1. Assign clear roles and responsibilities for data management, including
designating a primary staff member who is responsible for 1) ensuring all
required data for CALPADS are accurately populated in the local SIS; 2)
keeping CALPADS up to date, either by uploading files, troubleshooting data
problems, and performing the initial review of the reports, or overseeing the
development of repeatable, consistent processes for site staff to perform these
tasks; 3) sharing reports with administrators and site staff responsible for data
entry and working collaboratively to identify and resolve global data chal-
lenges; 4) clearly communicating deadlines and changes in reporting require-
ments to site and program staff; and 5) working with the SIS vendor and the
CALPADS Service Desk staff as needed to obtain answers to questions from
COE staff and to communicate local needs.
2. Given the size of the COE, designate the lead for CALPADS additionally as
the LEA CALPADS administrator, with responsibility for setting up local user
accounts and assigning roles, and training and supporting users to whom they
have assigned accounts. Assign the LEA administrator to contact the CALPADS
Service Desk on behalf of local users for any questions they are unable to answer.
3. Identify staff members who will temporarily fulfill data management respon-
sibilities in the event an employee with lead responsibilities leaves the county
office or is out of the office when tasks need to be completed. Train these
individuals so they can fulfill these backup responsibilities if necessary.
4. Identify a member of the COE leadership team to be responsible for guiding
the development and coordination of data management roles, responsibilities
and processes across the organization; helping to establish priorities for data
management; and resolving conflicts. If necessary, change reporting relation-
ships so the employee with primary responsibility for CALPADS reports to
this member of the leadership team.
5. Update job descriptions to accurately describe those roles/responsibilities.
See Appendix B for a sample job description for the lead staff for CALPADS
created from best practices and job descriptions from five county offices of
education with enrollment and school types comparable to Yuba COE.
6. Establish standard data management procedures to use with PROMIS,
CALPADS, CBEDS and OPUS for consistent results from each school site,
including requiring program coordinators and site administrators to review
and sign off on CALPADS certification reports prior to certification. Include
a procedure to assign someone other than the lead CALPADS staff person
to certify CALPADS reports at level two. This would be the superintendent
or designee and should be a different employee than the CALPADS lead to
ensure an additional opportunity to verify the accuracy of reports before certi-
fication. Review and update the procedures annually. Make the procedures
accessible on a central server or in a physical location.
Fiscal crisis & ManageMent assistance teaM
19
FINDINGS AND RECOMMENDATIONS
7. Develop a process to both create uniform forms for collecting student data
(e.g. student enrollment form) across all school sites, and review the forms
annually to ensure they are up to date with current state and federal require-
ments.
8. Develop and use data management calendars to coordinate data collection for
required state and federal reporting.
9. Hold staff and administrators accountable for fulfilling their data manage-
ment responsibilities and for coordinating with others as needed to build
consistent data management practices across the COE.
10. Establish a COE-wide data management team to manage and monitor data
collection, verify reports, conduct submission debriefs and/or data quality
audits, and make recommendations for improving local practices. Include on
the team the employee responsible for coordinating CALPADS work for the
COE; program staff responsible for data entry for students receiving services,
such as special education, homeless and English learners; data entry staff to
provide insight into site practices and challenges; assessment/accountability
staff; pupil records staff and leadership staff.
11. Develop a data management help desk ticket system or determine whether
the Technology Department help desk ticket system can be used to track
requests for data management support, training, new accounts, and enhance-
ments.
12. Implement a process to track and prioritize support and enhancement/
functionality requests for PROMIS before contacting the San Joaquin COE
help desk. Coordinate with other county offices that use PROMIS to identify
common high-priority improvements. Communicate with San Joaquin COE
to determine if these changes can be incorporated in the next release or if
Yuba COE or a consortium of COEs will need to contract for custom work.
Weigh the costs of custom work against the costs of taking no action before a
decision is made.
13. Establish support expectations with site users to reduce frustration; e.g.,
communicate the expected response time when a question is submitted to the
CALPADS coordinator.
14. Determine whether staff continue to struggle with understanding how to
complete certain functions in PROMIS. If so, schedule targeted follow-up
PROMIS training(s) taught by a Yuba COE employee (if an in-house expert
exists) or by an external expert.
15. Identify ongoing learning opportunities to develop expertise in PROMIS
or data management practices and allow staff to participate in these oppor-
tunities, e.g., attend the annual Center for Educational Development and
Research (CEDR) conference sponsored by the San Joaquin COE and/or
other conferences focused on data management practices.
Yuba CountY offiCe of eduCation
20
FINDINGS AND RECOMMENDATIONS
16. Provide annual refresher training for all staff using PROMIS to include new
requirements, any changes in policies, and a review of the privacy and confi-
dentiality guidelines included in the acceptable use policy.
17. Encourage new administrators supervising staff with CALPADS responsibili-
ties (e.g., principals as well as county office administrators) to attend the State
Reporting Overview for Administrators provided by FCMAT/CSIS through
self-paced training at http://csis.fcmat.org//Pages/K12-State-Reporting-
Overview-for-Administrators-SP.aspx.
18. Require staff with responsibilities for CALPADS to attend CALPADS train-
ings and information meetings sponsored by the CDE and FCMAT/CSIS so
they can understand requirements, common problems and how to address
them, and upcoming system changes.
19. Coordinate training and user support with other county offices of education
that use PROMIS.
20. Use the FAQs embedded in PROMIS and/or use the contact list of other
COEs using PROMIS to get assistance from others with more expertise.
Fiscal crisis & ManageMent assistance teaM
2211
APPENDDRICAEFST
Appendices
Appendix A - Data Management Best Practice Resources
Appendix B - Sample Job Description
Appendix C - Study Agreement
Yuba CountY offiCe of eduCation
2222
DARPPAEFNTDICES
Fiscal crisis & ManageMent assistance teaM
2233
APPENDDRICAEFST
Appendix A: Data Management Best Practice Resources
Data Quality Resources
Data Quality Program Handbook: Establishing a Culture of Data Quality
Massachusetts Department of Elementary and Secondary Education http://www.doe.mass.
edu/infoservices/data/quality/handbook-extract.pdf
Why Data Quality is Important [video]
Massachusetts Department of Elementary and Secondary Education
https://www.youtube.com/watch?v=SDCXLE8fwnY
Data Quality Certification Program Recommended Best Practices
Kansas Department of Education
http://community.ksde.org/LinkClick.
aspx?fileticket=kho4scKTJVY%3d&tabid=3413&mid=7558
National Forum on Education Statistics (2004). Forum Guide to Building a Culture
of Quality Data: A School & District Resource (NFES 2005–801). U.S. Department of
Education, Washington, DC: National Center for Education Statistics http://community.
ksde.org/LinkClick.aspx?fileticket=1vlw2dU5yII%3d&tabid=3413&mid=8447
Student Data Privacy Resources
Data Privacy
Laws, policies, and best practices related to both general and student data privacy at the
California Department of Education. Provides data privacy information for parents, teachers,
local education agencies, and the general public.
http://www.cde.ca.gov/ds/dp/
Privacy Technical Assistance Center (PTAC)
The U.S. Department of Education established the PTAC as a resource for education stakeholders
to learn about privacy, confidentiality, and security practices related to student data.
http://ptac.ed.gov/
Data Privacy Guidebook, Fagen, Friedman & Fulfrost
This downloadable PDF provides guidelines and practical tips for complying with student
data privacy laws and best practices.
http://www.f3law.com/resources.php?id=155&rs_id=68
RACI Matrix Resources
RACI Matrix*
University of Calgary
http://www.ucalgary.ca/research/researchers/forms-and-resources/project-management-
resources/raci-matrix
*Includes link for a Word template
Yuba CountY offiCe of eduCation
2244
DARPPAEFNTDICES
RACI Chart Tool
http://www.tutorialspoint.com/management_concepts/raci_chart_tool.htm
FCMAT/CSIS Client Services RACI Matrix
This matrix is included below. The step numbers refer to the steps in the documented
procedures.
Table 3.3: RACI Matrix for Service Desk
Step Service Desk
# Roles and Activities Within Process
Fiscal crisis & ManageMent assistance teaM
snaicinhceT
troppuS
tneilC
stsylanA
ataD
troppuS
dleiF
tcetihcrA
ecivreS
SC
fo
rotceriD
tsilaicepS
noitatnemelpmI
dna
snoitacinummoC
stcejorP
/kseD
ecivreS
lanretnI
snoitarepO
krowteN
tseT
dna
noitarugfinoC
SI
)T/C
SI(
maeT
tnemeganaM
stnemeriuqeR
6.1 Service Operations
6.1.1 Request Fulfillment R R R A C R
6.1.2 Incident Management R R R A C R R
6.1.3 New Service Requests R A C R R
6.1.4 Provide Intake for Internal Service Desk R A C
6.1.5 Process CTC Requests R A
6.1.6 Website Password Reset R A
6.1.7 Audit Tickets for Quality R A
6.1.8 Manage File Submissions R A C
6.1.9 Identify New FAQs C C C A C C
6.1.10 SIS Vendor Engagement R R A
6.2.1 Ensure Student Confidentiality R R R R A R
6.2.2 Manage Secure Upload Site R R A
6.3.1 Update support voicemail/email/Footprints for holidays R A I
6.3.2 Voicemail Management R A
6.3.3 Maintain LEA Admin Contact Data R R R A R R
6.3.4 Listserv Management R A
6.3.5 FootPrints Quick Communications C C C A C C
6.3.6 CALPADS Portal Messages R R A
6.3.7 Known Issues R A C
6.3.8 Release Update R A C
2255
APPENDDRICAEFST
Sample Data Submissions Debrief Agenda
Data Submission Debrief
[Insert organization’s logo here]
Date - Time
Location
Attendees:
Purpose: To improve data quality and processes.
Purpose & Objective: Objective: To review what worked well and challenges encountered in the [insert name] data submission and identify
action items to improve data quality and processes.
Gather your thoughts and notes on experiences during the submission and identify what you thought worked well and
Prerequisites your pain point during the submission. Bring these with you to the meeting.
Item Presenter(s) Time
Review Agenda & Meeting Purpose
• Review the purpose and objective for the meeting
• Review the norms, including agreement that any off topic ideas will be placed in the Parking Lot for
later discussion
• Review agenda Meeting Facilitator 5 minutes
What worked well?
• Record points identified All 5 minutes
What where the challenges or pain points?
Brainstorm without judgment or discuss problems encountered that need to be addressed – focus on
the problems and issues, not the solutions All 10 minutes
Possible solutions
• Brainstorm possible solutions – these need to be concrete, doable actions All 15 minutes
Development of action items
• Review results of brainstorming and come to consensus on specific actions that can be taken to
address the challenges identified
• Determine what actions need to be sent to the leadership team for discussion/approval and what
can be implemented immediately
• Assign actions to individuals
• Review and document action items and agreements All 20 minutes
Closing
What worked and what might need to be improved in terms of the debrief process and group interac-
tions for the next debrief?
If time, parking lot items may be discussed or the team may review the list and agree where items
should be referred. All 5 minutes
MEETING ACTION ITEMS
Item Assignee(s) Due Status
1.
2..
3.
4.
5.
Yuba CountY offiCe of eduCation
2266
DARPPAEFNTDICES
AGREEMENTS
1.
2.
Fiscal crisis & ManageMent assistance teaM
2277
APPENDDRICAEFST
Appendix B: Sample Job Description
The job descriptions of the COE’s data staff involved in coordinating data management and
reporting to CALPADS were compared with best practices and the job descriptions from
five comparable county offices of education identified using the Ed-Data website, Ed-Data.
org, a collection of demographic, performance and financial data reported to the California
Department of Education (CDE) by school districts and county offices of education. Ed-Data is
a collaborative effort of the CDE, EdSource and FCMAT. Comparable county offices of educa-
tion are those with a somewhat similar number of students and a similar number and type of
schools. Information about the five comparable county offices is provided in Table 1.
Table 1: Comparable County Offices of Education
County Office Calaveras Humboldt Imperial Sonoma El Dorado
2014-15 COE Enrollment 562 265 585 615 959
Number of Schools 3 6 3 4 5
SIS Product Infinite Campus PROMIS PROMIS Aeries Aeries
Has Data Management Advocate on
Leadership Team Y Y Y Y ?
CALPADS Lead (also known as
CALPADS Coordinator) Supports SIS
Users at County Schools Y Y Y Y N
1 (site staff 1 (site staff
Number of CALPADS Administrators 1 1 2 upload data) upload data)
COE Staff Supports Districts with Some districts
CALPADS Reporting (not all) Y Y Y Y
A sample job description was prepared for Yuba COE using the statements from the five compa-
rable counties’ job descriptions that are most aligned with best practices. That sample is included
below. Wording in italics is intended to be replaced with language appropriate to Yuba COE’s
decisions about best practices implementation. If the sample job description is used, it will need
to be modified to conform to the COE’s standard format and include all legal elements such as
essential duties, physical requirements, etc.
Yuba CountY offiCe of eduCation
2288
DARPPAEFNTDICES
Sample Job Description
Data Coordinator
Definition:
Under the general direction of [insert title of the Cabinet member/Leadership Team member
who is the designated manager for this position], the Data Coordinator would plan and coor-
dinate the collection, maintenance and reporting of data; coordinate, train, provide ongoing
support related to school information systems such as student information system (SIS); confer
with administrators and staff to provide support, documentation, training, data extraction/
reporting to COE staff and governmental agencies; troubleshoot problems with the above
systems; resolve reports and data issues; provide software quality assurance by testing new or
modified systems and perform other related accounting duties as assigned. Provide expert and
advisory guidance to county staff in the use of integrated school systems and support staff in
maintaining the accuracy and reliability of the systems. Perform other duties as assigned.
Distinguishing Characteristics:
This is a management classification that requires a thorough understanding of relational database
management; and the internal operations of student information systems from the user’s perspec-
tive related to attendance, registration, grades, test results, parent portals, learning management
systems and other application systems as assigned. This position provides expert and advisory
guidance to COE staff in the use of integrated school systems and supports staff in maintaining
the accuracy and reliability of the systems.
Supervision Exercised and Received:
Employees in this class receive limited supervision within a broad framework of policies and
procedures. The incumbent receives direction and supervision from the designated leadership
team member.
Examples of Duties and Responsibilities:
Duties and responsibilities may include, but are not limited to, the following:
• Supports county office users of student information systems (SIS) for the purpose of
collecting, organizing, analyzing and reporting all student information (e.g., attendance,
grading, academic progress, test results, enrollment, transfer, etc.) required by county,
state and federal agencies;
• Ensures proper linkage among related software systems (e.g. grading, attendance,
transfers, parent portal) for the purpose of maintaining system integrity and accuracy;
• Establishes, assigns, maintains and manages local security controls, local codes, and user
access.
• Maintains a wide range of data related to program and student files, demographics,
attendance, or other according to department specialization.
• Receives, verifies, enters data into required programs or formats.
Fiscal crisis & ManageMent assistance teaM
2299
APPENDDRICAEFST
• Retrieves data from files and/or records for the purpose of generating a variety of required
reports and statistical information.
• Prepares and transmits data files from information provided by individual schools or
programs, ensuring accuracy of data prior to submission and monitoring integrity of
databases.
• Documents and distributes information regarding data inconsistencies, errors, and
changes in variable definitions or coding that are discovered in the course of completing
requests and coordinates with data entry and administrative staff to identify process
changes needed to prevent future errors.
• Coordinates with data entry staff, administrative staff, and data requestors to develop and
maintain local data management procedures.
• Complies with state and federal privacy laws to protect the confidentiality of data.
• Keeps current on changes in regulations and procedures; attends in-services and/or
meetings to keep current with regulations and procedures. Communicates those changes
to staff involved in the entry, use and reporting of data.
• Assists end user with the creation of custom reports using a variety of software and
database tools including standard query language (SQL).
• Researches and investigates CALPADS (California Longitudinal Pupil Achievement Data
System) requirements including attending trainings, consulting with other districts and
researching CDE websites;
• Conducts workshops/seminars for district and county office personnel; assesses training
needs of users; develops workshop curriculum; creates written documentation and
workshop materials; schedules and executes training sessions and reviews evaluation
forms of participants; and trains one-to-one as needed;
• Acts as an information resource to employees concerning software applications;
researches and resolves problems; incorporates repeated questions into training program;
• Plans, implements and supervises projects; works with others on special projects to
design and format software programs to achieve desired result; performs other special
projects as assigned;
• Troubleshoots software problems, consults with school sites and other divisions to solve
data problems and takes corrective action;
• Acts as liaison between software vendors and county office; coordinates installation of
updates to applications;
• Attends workshops and seminars to remain current on software applications;
disseminates information to other employees and districts;
• Coordinates CALPADS submissions, report reviews and correction of errors;
• Maintains and monitors special project development and implementation;
• Composes own correspondence; maintains records related to area;
• Performs related duties as assigned.
Yuba CountY offiCe of eduCation
3300
DARPPAEFNTDICES
Employment Standards
Knowledge of:
• Policies, procedures, functions and principles to maintain complex database and
reporting systems;
• Industry standards and procedures of SIS database application and maintenance, student
attendance;
• Grading, registration, and class scheduling standards, practices, and procedures;
• Mathematics to perform calculations and statistical computations;
• Report design and formatting for data extracted from database systems;
• Troubleshooting and diagnostic techniques for identifying and resolving database and
reporting errors;
• Standard Query Language (SQL);
• Standard English usage, spelling, grammar and punctuation;
• Interpersonal skills using tact, patience and courtesy;
• Safe work practices;
• Effective training and facilitation techniques.
Ability to:
• Respond promptly to requests of internal and external clients; provide needed
information, assistance, training, materials and resources;
• Plan, organize and prioritize work to meet deadlines and accomplish assigned tasks
within established timelines including maintaining accurate records and files;
• Maintain an orderly work environment and perform tasks in prescribed and safe manner;
• Establish and maintain cooperative working relationships with those contacted during
performance of job duties;
• Maintain and improve professional skills and knowledge;
• Be flexible and receptive to change;
• Understand and apply principles, techniques and procedures required for effective job
performance;
• Install and maintain complex database systems;
• Troubleshoot and debug database applications and software problems;
• Train and assist others in the use and functions of assigned software;
• Write effective database queries using SQL and other software tools;
• Test and evaluate software applications;
• Effectively implement problem-solving techniques;
• Prepare and deliver clear and concise oral and written instructions;
Fiscal crisis & ManageMent assistance teaM
3311
APPENDDRICAEFST
• Prepare complex technical written material in an effective and comprehensive manner;
• Communicate effectively both orally and in writing, including developing and executing
oral presentations for small and large groups;
• Facilitate meetings and build consensus on continual improvement actions;
• Work in an independent manner with little direct supervision.
Minimum Qualifications
Education: Bachelor’s degree in computer information, computer science or equivalent. Work
experience may be substituted for the degree on a year-by-year basis.
Experience: Five years progressively responsible related experience with relational database
systems applications utilizing a variety of productivity software to gather, transform and transfer
data and three years related experience/training with systems application support (work experi-
ence may be concurrent).
Desirable: Previous work experience in a school district or county office of education.
Yuba CountY offiCe of eduCation
3322
DARPPAEFNTDICES
Fiscal crisis & ManageMent assistance teaM
3333
APPENDDRICAEFST
Appendix C
FISCAL CRISIS & MANAGEMENT ASSISTANCE TEAM
STUDY AGREEMENT
December 16, 2015
The Fiscal Crisis and Management Assistance Team (FCMAT), hereinafter referred to as the
team, and the Yuba County Office of Education, hereinafter referred to as the COE, mutually
agree as follows:
1. BASIS OF AGREEMENT
The team provides a variety of services to school districts and county offices of education
upon request. The COE has requested that the team assign professionals to study specific
aspects of the COE’s operations. These professionals may include staff of the team,
county offices of education, the California State Department of Education, school
districts, or private contractors. All work shall be performed in accordance with the terms
and conditions of this agreement.
The final report will be published on the FCMAT website.
2. SCOPE OF THE WORK
A. Scope and Objectives of the Study
Yuba County Office of Education contacted FCMAT to request a review of the
COE’s data management processes, including those related to the following
systems:
i. California Longitudinal Pupil Achievement Data System (CALPADS)
ii. California Basic Educational Data System (CBEDS)
iii. Online Public Update for Schools – County District School (OPUS-CDS)
iv. California Special Education Management Information System software
(CASEMIS)
v. Special Education Information System (SEIS)
vi. Special Education Personnel Data Report
vii. Civil Rights Data Collection
viii. Pupil Records Online Management Information System (PROMIS)
ix. School Accountability Report Card (SARC)
x. California Assessment of Student Performance and Progress (CAASPP)
1
Yuba CountY offiCe of eduCation
3344
DARPPAEFNTDICES
xi. Test Operations Management System (TOMS)
xii. Physical Fitness Test (PFT)
xiii. Principal Apportionment Data Collection (PADC)
xiv. Consolidated Application (ConApp)
The COE and the team have agreed to narrow the scope of this review to
PROMIS, CALPADS, CBEDS and OPUS as specified in items 1 – 3 below. This
study can then be used by the COE and the team to determine if additional study
agreements should be created to address the systems not included in the scope of
this agreement.
This study will include the following:
1. Review the COE’s practices for gathering, maintaining, analyzing and
reporting student and staff data stored in PROMIS and/or reported to
CALPADS, CBEDS, or OPUS to determine if the COE’s data
management practices need to be modified to increase accuracy and/or
efficiency. The team will evaluate the current workflow and distribution of
data management responsibilities in the COE for PROMIS and
CALPADS.
This component will include reviewing documentation such as board
policies and administrative regulations; desk or procedure manuals related
to gathering data; entering data into PROMIS, CALPADS, CBEDS and/or
OPUS and reviewing data for accuracy; job descriptions for positions with
responsibility for gathering, maintaining, auditing, using or reporting data
from PROMIS,CALPADS, CBEDS or OPUS; and enrollment forms and
other forms used to gather data, including but not necessarily limited to
forms related to National School Lunch Program/income eligibility,
discipline and inter-district transfers. Additionally, FCMAT may interview
other COE or site staff to determine the efficiency and effectiveness of
data management responsibilities.
2. Review staff assignments with respect to data management for PROMIS,
CALPADS, CBEDS and OPUS, including both staff with primary
responsibilities and staff who are cross-trained to assist in the event of a
staff shortage such as a sudden absence due to a serious health issue. Make
recommendations, if appropriate, to modify the distribution of workload
among staff members and/or reporting relationships to improve data
management practices.
3. Review training and professional development programs for data staff
members responsible for PROMIS, CALPADS, CBEDS and OPUS and
make recommendations as applicable.
2
Fiscal crisis & ManageMent assistance teaM
3355
APPENDDRICAEFST
B. Services and Products to be Provided
1. Orientation Meeting – The team will conduct an orientation session at the
COE to brief COE management and supervisory personnel on the team’s
procedures and the purpose and schedule of the study.
2. On-site Review – The team will conduct an on-site review at the COE
office and at all school sites.
3. Exit Report – The team will hold an exit meeting at the conclusion of the
on-site review to inform the COE of significant findings and
recommendations to that point.
4. Exit Letter – Approximately 10 days after the exit meeting, the team will
issue an exit letter briefly summarizing significant findings and
recommendations to date and memorializing the topics discussed in the
exit meeting.
5. Draft Report – Electronic copies of a preliminary draft report will be
delivered to the COE’s administration for review and comment.
6. Final Report - Electronic copies of the final report will be delivered to the
COE’s administration and to the county superintendent following
completion of the review. Printed copies are available from FCMAT upon
request.
7. Follow-Up Support – If requested, FCMAT will return to the COE at no
cost six months after completion of the study to assess the COE’s progress
in implementing the recommendations included in the report. Progress in
implementing the recommendations will be documented to the COE in a
FCMAT management letter.
3. PROJECT PERSONNEL
The study team will be supervised by Michael H. Fine, Chief Administrative Officer,
Fiscal Crisis and Management Assistance Team/California School Information Services
(FCMAT/CSIS), Kern County Superintendent of Schools Office. The study team may
also include:
A. Nancy Sullivan FCMAT/CSIS Chief Operations Officer
B. Martha Friedrich FCMAT/CSIS Client Services Officer
C. Lisa Hayes FCMAT/CSIS Implementation Specialist
D. Andrea Alvarado FCMAT Management Analyst
Other equally qualified staff or consultants will be substituted in the event one of the
above individuals is unable to participate in the study.
3
Yuba CountY offiCe of eduCation
3366
DARPPAEFNTDICES
4. PROJECT COSTS
The cost for studies requested pursuant to E.C. 42127.8(d)(1) shall be as follows:
A. $500 per day for three staff members while on site, conducting fieldwork at other
locations, preparing and presenting reports, or participating in meetings. The cost
of independent FCMAT consultants will be billed at their actual daily rate.
B. All out-of-pocket expenses, including travel, meals and lodging.
C. The COE will be invoiced at actual costs, with 50% of the estimated cost due
following the completion of the on-site review and the remaining amount due
upon the COE’s acceptance of the final report.
Based on the elements noted in section 2, the total estimated cost of the study
will be $12,800.
D. Any change to the scope will affect the estimate of total cost.
Payments for FCMAT’s services are payable to Kern County Superintendent of
Schools - Administrative Agent.
5. RESPONSIBILITIES OF THE COE
A. The COE will provide office and conference room space during on-site reviews.
B. The COE will provide the following:
1. Policies, regulations, procedures and forms that address the study scope.
2. Current or proposed organizational charts.
3. Job descriptions for all positions with responsibility for data management.
4. Any documents requested on a supplemental list. Documents requested on
the supplemental list should be provided to FCMAT only in electronic
format; if only hard copies are available, they should be scanned by the
COE and sent to FCMAT in electronic format.
Documents should be provided in advance of fieldwork; any delay in the receipt
of the requested documents may affect the start date of the project. Upon approval
of the signed study agreement, access will be provided to FCMAT’s online
SharePoint document repository, where the COE will upload all requested
documents.
C. The COE’s administration will review a preliminary draft copy of the report
resulting from the study. Any comments regarding the accuracy of the data
presented in the report or the practicability of the recommendations will be
reviewed with the team prior to completion of the final report.
4
Fiscal crisis & ManageMent assistance teaM
3377
APPENDDRICAEFST
Pursuant to EC 45125.1(c), representatives of FCMAT will have limited contact with
pupils. The COE shall take appropriate steps to comply with EC 45125.1(c).
6. PROJECT SCHEDULE
The following schedule will be established upon the receipt of a signed study agreement:
Orientation: to be determined
Staff Interviews: to be determined
Exit Meeting: to be determined
Preliminary Report Submitted: to be determined
Final Report Submitted: to be determined
Board Presentation: to be determined, if requested
Follow-Up Support: if requested
7. COMMENCEMENT, TERMINATION AND COMPLETION OF WORK:
FCMAT will begin work as soon as it has assembled an available and appropriate study
team consisting of FCMAT staff and independent consultants, taking into consideration
other jobs FCMAT has previously undertaken and assignments from the state. The team
will work expeditiously to complete its work and deliver its report, subject to the
cooperation of the COE and any other parties from which, in the team’s judgment, it must
obtain information. Once the team has completed its fieldwork, it will proceed to prepare
a preliminary draft report and a final report. Prior to completion of fieldwork, the COE
may terminate its request for service and will be responsible for all costs incurred by
FCMAT to the date of termination under Section 4 (Project Costs). If the COE does not
provide written notice of termination prior to completion of fieldwork, the team will
complete its work and deliver its report and the COE will be responsible for the full costs.
The COE understands and agrees that FCMAT is a state agency and all FCMAT reports
are published on the FCMAT website and made available to interested parties in state
government. In the absence of extraordinary circumstances, FCMAT will not withhold
preparation, publication and distribution of a report once field work has been completed,
and the COE shall not request that it do so.
8. INDEPENDENT CONTRACTOR:
FCMAT is an independent contractor and is not an employee or engaged in any manner
with the COE. The manner in which FCMAT’s services are rendered shall be within its
sole control and discretion. FCMAT representatives are not authorized to speak for,
represent, or obligate the COE in any manner without prior express written authorization
from an officer of the COE.
5
Yuba CountY offiCe of eduCation
3388
DARPPAEFNTDICES
Fiscal crisis & ManageMent assistance teaM