Recommendations

Los Angeles Community College District Personnel Commission Its Inconsistent Practices and Inadequate Policies Adversely Affect District Employees and Job Candidates, Leading to Concerns About the Fairness of Its Decisions May 2021 REPORT 2020‑111 CALIFORNIA STATE AUDITOR 621 Capitol Mall, Suite 1200 | Sacramento | CA | 95814 916.445.0255 | TTY 916.445.0033 For complaints of state employee misconduct, contact us through the Whistleblower Hotline: 1.800.952.5665 Don’t want to miss any of our reports? Subscribe to our email list at auditor.ca.gov For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255 This report is also available online at www.auditor.ca.gov | Alternative format reports available upon request | Permission is granted to reproduce reports Elaine M. Howle State Auditor May 6, 2021 2020-111 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As directed by the Joint Legislative Audit Committee, my office conducted an audit of the Los Angeles Community College District’s (District) Personnel Commission (Commission). The Commission establishes and administers the District’s merit system for classifying, hiring, and promoting nonacademic employees. The following report details how the Commission’s inadequate policies and practices led to inconsistent qualification decisions, inconsistent examination scoring, and delayed compensation for employees performing duties beyond their job classifications (higher-level duties). The Commission’s duties in administering the examination process include screening applicants for minimum qualifications, overseeing raters who score candidates’ examinations, and compiling ranked lists of candidates for District interviews. However, the Commission made inconsistent decisions when screening examination applicants because it failed to define or disclose the definition of key terms it used in these decisions and used past applications and outside information sources inconsistently. It also lacks sufficient guidelines for scoring examinations, and some raters did not adequately justify the scores they gave candidates. We identified scoring inconsistencies for nearly one-third of the candidates we reviewed, a pattern that directly affected some candidates’ abilities to get job interviews and increased the risk that the District did not interview and hire the most qualified candidates. We also found that the Commission’s practices failed to ensure prompt compensation for employees performing higher-level duties. As a result, in five of the six cases we reviewed, employees did not receive payment until five to 11 months after they began performing those duties. To help the Commission alleviate District employees’ concerns about inconsistent practices and unfair application of its rules, it should alter certain policies and increase transparency. For example, it should provide additional information about why applicants are disqualified, how it defines terms used in minimum qualification requirements, and how employees may request intermittent payments for performing higher-level duties. Respectfully submitted, ELAINE M. HOWLE, CPA California State Auditor 621 Capitol Mall, Suite 1200 | Sacramento, CA 95814 | 916.445.0255 | 916.327.0019 fax | www.auditor.ca.gov iv Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2020-111 v May 2021 CONTENTS Summary 1 Introduction 5 Inconsistent and Unjustified Qualification Decisions and Examination Scores Raise Concerns About the Impartiality of the Commission’s Decisions 11 The Commission’s Policies and Practices Do Not Ensure Prompt Compensation for All Out-of-Class Work 25 The Commission Does Not Adequately Address All Complaints and Protect Complainants 33 Other Areas We Reviewed 39 Appendix Scope and Methodology 51 Response to the Audit Los Angeles Community College District Personnel Commission 57 California State Auditor’s Comments on the Response From the Los Angeles Community College District Personnel Commission 81 vi Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2020-111 1 May 2021 SUMMARY The Los Angeles Community College District (District) is the largest community college district in the country with nine colleges located throughout Los Angeles County. This audit focuses on the District’s Personnel Commission (Commission), to which state law assigns personnel-related responsibilities for the District’s classified, or nonacademic, employees. The Commission’s primary purpose is to establish and administer a merit-based system for hiring, promoting, and classifying these employees. Classified employees, representative labor groups, and administrators have expressed concerns that the Commission has engaged in inconsistent practices and failed to apply its rules fairly, consistently, and in accordance with state law. Prompted in part by these concerns, this audit reviews the Commission’s application and examination processes for hiring and promotion; its processes for making fair personnel decisions; and its processes for reviewing applicants’ and employees’ complaints, grievances, and appeals, among other things. This report draws the following conclusions: Inconsistent and Unjustified Qualification Decisions and Page 11 Examination Scores Raise Concerns About the Impartiality of the Commission’s Decisions The Commission made inconsistent decisions when it screened examination applicants for minimum qualifications. A lack of clear definitions for key terms and the Commission staff’s inconsistent use of past applications and information from other sources contributed to these inconsistent decisions. These inconsistencies highlight the need for the Commission to notify applicants of the specific reasons they are disqualified. The Commission has also neither ensured that its raters—subject matter experts who assess the examination performance of candidates who are applying for District jobs—scored candidates’ performance consistently, nor have they always adequately justified the candidate scores they awarded, likely in part because the Commission has not established guidelines requiring them to do so for all candidates. By allowing such wide latitude to its raters, the Commission increases the risk that they will base their scores on factors other than a candidate’s performance on the examination. The Commission’s Policies and Practices Do Not Ensure Prompt Page 25 Compensation for All Out-of-Class Work Supervisors may assign work to employees outside their assigned job duties (known as out-of-class work), in which case the employees can file a claim to receive additional compensation. However, the Commission’s practice is to approve payment for these claims after 2 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 an employee has completed the higher-level job duties. As a result, for most out-of-class claims we reviewed, employees did not receive payment until five to 11 months after they began performing the higher-level duties. In addition, during the three-year period we reviewed, the Commission required that employees submit claims for additional compensation within 100 days of beginning their out-of-class work even though the Commission generally does not process these claims before the employee finishes the work. In two of the six cases, the Commission reduced the compensation when the employees did not submit by the deadline. Further exacerbating this problem, the Commission shortened the 100-day period to only 45 days in April of 2020. In contrast, California state agencies may reimburse claims for out-of-class work submitted up to a year after employees begin performing additional duties. The Commission Does Not Adequately Address All Complaints and Page 33 Protect Complainants The Commission does not track all complaints it receives from employees, and it could not demonstrate that it had adequately addressed some complaints. Although the Commission receives complaints through several methods, during the period we reviewed, it tracked only complaints made during public meetings. The Commission adequately responded to complaints raised in public meetings, but it could not document that it responded to five of the 21 email complaints that we reviewed. Further, the Commission’s rules allowed the former director inappropriate access to allegations against her, exposing these whistleblower complainants to potential retaliation. CALIFORNIA STATE AUDITOR | Report 2020-111 3 May 2021 Summary of Recommendations To ensure that it makes consistent decisions when assessing applicants’ minimum qualifications, the Commission should define for its staff and for applicants key qualification-related application terms and provide disqualification notices to applicants that describe its reasons for disqualifying them. The Commission should establish a method to determine candidates’ overall examination scores based on its raters’ evaluations of their individual skills, and it should require raters to use this method in determining overall scores. It should also require raters to provide justification for the scores they award. The Commission should compensate employees each month for the out-of-class work they perform. In addition, it should allow employees at least 100 days, rather than 45, to file a claim before it reduces their compensation. The Commission should amend its rules to create a formal process for addressing all complaints, should establish that complainant information may not be shared with the subject of a whistleblower complaint, and should direct whistleblower complaints to the District’s general counsel, who will have the responsibility of designating an appropriate party to respond to such complaints. Agency Comments The Commission disagreed with a number of our conclusions, objected to some of the phrasing in our report, and criticized our methodologies and staff expertise. Nonetheless, it agreed with many of our recommendations and stated that it will implement most of them. We address the Commission’s response beginning on page 81. 4 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2020-111 5 May 2021 INTRODUCTION Background The Los Angeles Community College District (District) is one of 73 community college districts in California and has nine colleges located throughout Los Angeles County. The largest community college district in the United States, with almost 230,000 students enrolled during the 2019–20 school year, its mission is to foster student success for all individuals seeking advancement by providing equitable and supportive learning environments. The District’s Board of Trustees (Board) establishes rules and regulations for the government and operation of the District’s colleges. The Board consists of seven members whom District voters elect and one nonvoting student member whom the associated student organizations select. The elected members serve four years in office, while the student member serves one year. The Board generally meets twice a month to make decisions regarding the District’s governance, such as approving its budget and educational programs and establishing student fees. State law assigns responsibilities related to certain District employees to its Personnel Commission (Commission), which is the focus of this audit. According to the Commission, it is an independent body composed of three commissioners who serve staggered three-year terms and who District Employee Groups can be reappointed indefinitely. The Commission’s Classified Employees personnel director (director) and 14 other positions Employees who work in nonacademic positions, including (Commission staff) advise the commissioners custodians, accountants, and administrative analysts. on personnel decisions. Figure 1 identifies the Four unions represent most of the District’s 2,300 classified Commission’s responsibilities, which we describe employees; of them, the American Federation of Teachers in more detail below. College Staff Guild (staff union) represents about 1,300 District classified employees. The Commission’s Roles and Responsibilities Academic Employees Employees who teach students, provide library and counseling The District organizes its 6,600 employees into services to students, and provide supervision of instructional three groups, as the text box describes. Each and student services. They include faculty members, librarians, employee group has its own employment policies, counselors, and administrators. Academic employees are the processes, and procedures. The Commission’s single largest employee group in the District. statutory responsibilities relate to the District’s Unclassified Employees classified employees and include categorizing All employees not included in the classified or academic those employees into classified positions and groups, including part-time student workers. recommending salary schedules (salaries) for the classified employees to the Board. When classifying Source: The Commission’s classified employee handbook, eligibility lists, and website; the staff union website; and the employees, the Commission uses a classification District’s collective bargaining agreements. description, which it updates about every five years. This description includes the classification’s title, 6 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 typical duties, and minimum qualifications. Commission rules state that job positions in the same classification must require the same level of education and experience and be paid according to the same salary range. Figure 1 The Board, District, and Commission Have Distinct Roles and Responsibilities The Board The District The Commission COMPOSITION Consists of seven members Consists of nine Consists of elected at large and community colleges and three commissioners one student representative District office. and the staff members selected by the associated who support them. student organizations. MISSION Establishes rules and Fosters student success Prescribes merit system regulations for the and provides supportive rules for classified government and operation learning environments. employees and administers of the community colleges the District’s merit system. within the District. RESPONSIBILITIES Adopts the District’s Hires employees. Authorizes merit system final budget. examinations and Provides its view on the establishes eligibility lists. Establishes salaries and Commission’s budget and benefits for all District monitors its expenditures. Recommends for Board employees. approval salaries for Pays for the Commission’s classified employees. Approves, amends, or expenses out of its general rejects Commission funds. Decides on requests for recommendations related reclassifications and to salaries for classified approves compensation employees. for out-of-class work. Reviews and sustains or denies appeals related to certain decisions. Source: State law, Board and Commission rules, District and Commission documents, and Commission staff interviews. The Commission is also responsible for administering the District’s merit system, which guides the selection, retention, and promotion of classified employees through competitive examinations. To implement the merit system, state law requires that the Commission prescribe BACHELORS DEGREE MASTERS DEGREE CALIFORNIA STATE AUDITOR | Report 2020-111 7 May 2021 rules as may be necessary to ensure the efficiency of the service and the selection and retention of employees upon a basis of merit and fitness. These include rules for examinations and for classifying positions and employees. The Commission uses several terms to describe the individuals whom it screens through the examination process, as the text box describes. The Commission’s Terms for Individuals Who According to the assistant personnel director (assistant Participate in Its Examination Process director), the Commission’s involvement in this process Applicant: A person who has filed an application to take a generally begins when a college’s personnel office merit system examination. notifies it of a vacancy that the District’s chancellor and budget office have authorized to be filled. The Candidate: A person who has taken one or more portions Commission’s involvement ends when it provides a list of a merit system examination. of eligible candidates (eligibility list) to the college’s Eligible Candidate: A person whose name appears personnel office, which interviews qualified candidates on a merit system eligibility list, which is a list ranking and makes hiring decisions. Figure 2 describes the persons who have qualified in all parts of a merit system different steps in the examination process. examination. An examination may include several sections, such as an Source: Commission rules. evaluation of a candidate’s training and experience, as well as written, performance, and oral examinations. Although an examiner oversees the examination process and screens all applicants to determine whether they meet the minimum qualifications, the examiner does not evaluate candidates’ performance on the examination. The examiner instead selects raters to score candidates’ performance, as the text box describes. According to the assistant director, generally two or more raters score each candidate during in-person evaluations or interviews, and the Commission averages the overall scores from each rater to determine the candidate’s final score for that examination section. Individuals Involved in Conducting The Commission then ranks the candidates on the Commission Examinations eligibility list based on their overall score from the examination sections, seniority points from previous Examiner: A Commission employee who selects existing District employment, and other factors, such as their sections of previous examinations or develops new sections status as a military veteran. to make up the examination, selects raters, and oversees the examination process. A candidate’s performance during the Commission’s Rater: A District employee or individual from outside the examination process only earns him or her a position District—typically from another public agency—with on the eligibility list; it does not determine the ultimate technical expertise or knowledge of the position who selection of whom the District interviews or hires reviews the candidates’ performance on the examination from among the candidates. State law requires these and assigns a numerical score. hiring managers to hire eligible candidates from the Assistant director: The Commission employee who first three ranks on an eligibility list who are ready supervises each examiner’s decisions, including approving and willing to accept the position. According to the an examination’s content, selection of its raters, and its list District’s human resources department executive of eligible candidates. assistant (executive assistant), hiring managers interview candidates in the top three ranks and can Source: Commission’s financial aid technician rater orientation, interview candidates in lower ranks if eligible candidates Commission staff interviews, and the Commission’s Selection Process Procedure Manual. in higher ranks are not ready and willing to accept the position. Managers base their decisions to hire 8 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 individuals on those interviews rather than on the candidates’ examination performance. Because the Commission is not involved in the interview stage of the District’s hiring process, we did not review this stage. Figure 2 The Commission and the District Each Have Roles in the Hiring Process The District conducts this step. A hiring manager notifies a college’s personnel office of a vacancy. The personnel office contacts The Commission the Commission. If there is not an active eligibility conducts this step. list, the Commission proceeds with a job recruitment and examination.* The assistant director selects an examiner for the Preparing examination. Using feedback from hiring managers, the examiner selects existing—or develops new—examination sections. The assistant director or director approve examination content. The examiner publishes a job recruitment announcement for the upcoming examination on the District's website and publishes advertisements in journals and on other websites. Applicants apply to take the examination. The examiner screens all applicants to determine whether they meet minimum qualifications and disqualifies those who do not. Examining The examiner selects raters and oversees the provision of the examination(s) to candidates. With the exception of certain written examinations, raters evaluate and score candidates on their examination performance. The examiner generates an eligibility list ranking Ranking candidates based on a number of factors, including their seniority and the scores they received for each examination section. The District hiring managers use the eligibility list to select eligible candidates to interview. Hiring The District hiring managers hire candidates based on interviews. Source: Commission rules, District and Commission websites and documents, Commission staff interviews, and the Commission’s Selection Process Procedure Manual. * The Commission sometimes self-initiates job examinations after a reclassification, if Commission staff identify unclassified District staff performing classified work, or if the Commission expects vacancies in lower-level positions because of expected promotions. BACHELORS DEGREE MASTERS DEGREE CALIFORNIA STATE AUDITOR | Report 2020-111 9 May 2021 Appointment of the Commissioners Commissioners are generally appointed to three-year terms, and historically they have been reappointed multiple times. Neither the state law governing personnel commissions nor the Commission’s rules establish requirements that limit the number of terms a commissioner can serve. As of January 2021, the Commission chair had been a member since 2001 and the vice chair had served since 2007. The third commissioner who served during our audit period was a member of the Commission for 11 years but left in February 2020 when she was not reappointed. The state chancellor of community colleges (state chancellor) appointed a new commissioner in March 2020. Until recently, a seven-member committee recommended nominees for commissioner to the Board. The Board then recommended a nominee to the state chancellor, who formally appointed each commissioner. State law allows the District’s classified employees to petition the Board to submit to an election the question of how personnel commission members are appointed. In September 2020, the Board announced that the employees had voted to change the process. Under the new process, the Board and the union that represents the largest number of classified employees each nominate one member to be appointed by the Board. The Board—rather than the state chancellor—formally appoints these commissioners, who then jointly appoint the third commissioner. The staff union provided notice of its first commissioner nomination—who will replace the current vice chair—in a public Commission hearing in December 2020. However, as of March 2021, the Board had not yet appointed this nominee. District Employee Concerns Regarding the Commission’s Practices Classified employees, representative labor groups, and administrators have expressed concerns that the Commission has engaged in inconsistent practices and failed to apply its rules fairly, consistently, and in accordance with state law. Further, when the staff union conducted a November 2019 survey of classified employees, many indicated that they had lost confidence in the Commission’s ability to function as a fair and impartial body. According to the union newsletter, the survey revealed that many classified employees were frustrated with the Commission for what they perceived as a lack of upward mobility, limited feedback, inconsistent practices, and improper interpretation of state law. As a result of this survey, the union petitioned the Board to change the way commissioners are selected, as we describe above. In addition, the survey—along with other concerns—prompted this audit. 10 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 When we reviewed responses to the union-administered survey, which included one open-ended question about employee experiences with the Commission, we found that 117 of the 130 employees who responded to that question about the Commission reported having negative experiences on a variety of issues, including eligibility determinations and classifications. However, only three of the 117 employees who reported having a negative experience with the Commission had a record of appealing a Commission decision, although some District employees separately told us that they feared retaliation or retribution if they questioned Commission practices. Table 1 Employee Responses to a 2019 Union Survey Describe a Variety of Negative Experiences With the Commission Total survey responses 975 Responses related to the Commission* 130 Positive comments 7 Topics outside the Commission’s purview 23 Negative comments 117 Topics Described in Negative Comments:† Eligibility and qualifications 35 Classifications and reclassifications 26 Promotions 22 Examinations 25 Compensation 17 Working out of class 14 Other 48 Source: Analysis of the Commission-related responses to the union’s 2019 survey and the union’s count of the total number of responses. These responses were identified by the staff union. * The survey included one open-ended question about experiences with the Commission, and responses included both negative and positive comments. Some responses had both negative and positive comments, and some included responses related to the Commission and topics outside of the Commission’s purview; thus, the counts of negative responses, positive responses, and topics outside the Commission’s purview total more than 130. † Many responses addressed more than one topic; thus, the counts of individual topics total more than the number of negative comments. CALIFORNIA STATE AUDITOR | Report 2020-111 11 May 2021 Inconsistent and Unjustified Qualification Decisions and Examination Scores Raise Concerns About the Impartiality of the Commission’s Decisions Key Points • The Commission has inconsistently evaluated some of its applicants, revealing weaknesses in its examination processes that highlight the need for the Commission to notify applicants of the reasons they were disqualified. Out of a selection of 119 candidates that examiners disqualified for failure to meet the minimum qualifications, we identified inconsistencies in the evaluation of five applicants. • In addition, raters provided inconsistent scores of the performance of some of those candidates taking examinations. For 25 candidates whom we reviewed who met minimum qualifications and advanced to the examination process, raters provided inconsistent scores for nine. These included candidates whose scores on individual evaluation factors were equivalent or higher than those given to a second candidate, who nevertheless received a higher overall score than the first candidate. These scoring inconsistencies suggest that raters either based their scores on factors other than those defined for the examination or they weighted factors differently for different candidates; as a result, the eligibility lists the Commission provided to District hiring managers may not have included all of the most qualified candidates. • Raters also provided minimal or no justification to explain the scores they had assigned to some candidates. The Commission’s guidance encourages but does not require raters to justify all scores, decreasing the Commission’s ability to identify whether raters are being consistent in their ratings or if raters are basing candidates’ overall scores on factors other than the established criteria. Commission Staff Made Inconsistent Decisions in the Examination Process, and Used Ambiguous Terms to Describe Minimum Qualifications The Commission made inconsistent decisions concerning the minimum qualifications of some applicants. To assess whether the Commission’s examination process was consistent and fair, we reviewed applications for 13 of the 319 employment examinations the Commission administered and applications for two provisional positions that the Commission posted from July 2016 through September 2020.1 We judgmentally selected up to 10 applicants for each of the examinations and provisional positions we reviewed, for a total of 144 applicants. The Commission disqualified 119 of these applicants based on minimum qualification screening, and it passed the remaining 25. Although the Commission had appropriately disqualified the majority of the applicants we reviewed, 1 Provisional positions are vacancies that the District can temporarily fill if an eligibility list does not exist or there are insufficient names on the existing list. 12 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 we identified concerns with five disqualified applications that revealed inconsistencies in the Commission’s processes, as demonstrated in the examples below. In one instance, an examiner cited her own knowledge of an applicant’s current District position as grounds for disqualifying the applicant (Applicant A) even though Applicant A described having more relevant experience than another applicant who was deemed qualified. As Figure 3 shows, the position in question required applicants to have experience related to Equal Employment Opportunity (EEO) or related areas. The examiner explained to us that she determined—based on her knowledge of Applicant A’s job with the District and past applications— that Applicant A did not meet the minimum experience requirements. However, Applicant A described in their application having dealt with complaints related to discrimination and harassment, which can relate to EEO, as well as having experience with civil rights investigative and compliance matters. In contrast, Applicant B only described experience investigating labor practices, primarily related to wage and contractor compliance. However, the Commission accepted Applicant B’s application, despite the fact that Applicant B did not specifically reference experience with EEO or related areas. Notably, the examiner was familiar with Applicant A, who was a current District employee, while Applicant B was an external candidate. An examiner cited her own knowledge of an applicant’s current District position as grounds for disqualifying the applicant. The Commission also appears to have made some qualification decisions based on internal applicants’ District job titles rather than the experience they described in their applications. For one examination the Commission disqualified multiple applicants who did not have jobs that it considered to be on the promotional path for the classification for which they were applying, stating that their current positions and experience were not “professional level,” an undefined term used in describing the minimum qualifications for that classification. In one case, the Commission noted that when it contacted the supervisor of the applicant it disqualified, the supervisor supported the applicant’s description in their application of the duties they performed. Nevertheless, the Commission disqualified the applicant. In contrast, it determined that another applicant, whose current job classification’s duties were similar to those the disqualified applicant described having performed, did meet the minimum qualifications for the classification, noting that the applicant’s job was on the promotional path. CALIFORNIA STATE AUDITOR | Report 2020-111 13 May 2021 Figure 3 A Commission Examiner Evaluated Two Similar Applicants’ Minimum Qualifications Inconsistently Experience Minimum Qualifications: Investigating complaints, appeals, and grievances related to equal employment opportunity or related areas. Applicant A Applicant B Selected Experience Selected Experience Complex investigations of student Monitored, investigated, and audited grievances, and employee-employer the labor practices of contractors to relations determine compliance with wage laws and regulations Coordinator for employee and student complaints of alleged Conducted investigations of alleged discrimination based on protected contractor violations uncovered class, retaliation, and/or non-sexual from worker complaints workplace harassment Investigated and processed labor law Extensive experience investigating complaints, appeals, and grievances sensitive and confidential matters from an employee of the company Handled investigative and civil rights compliance matters Discrimination/Harassment/ Retaliation complaint coordinator Relevant Degree(s) Relevant Degree(s) Graduate Degree and Bachelor’s Degree Bachelor’s Degree Relevant Experience (2 years required) Relevant Experience (4 years required) At least 11 Years At least 15 Years Experience with an educational institution (desired) Experience with an educational institution (desired) YES NO External or Internal Applicant External or Internal Applicant Internal External Commission decision Commission decision Disqualified Qualified Reason for disqualification: Determination that the applicant’s experience is not relevant—based on the examiner’s knowledge of applicant’s current and past applications. Source: Job applications, job announcement, and interviews with Commission staff. Note: The number of years of qualifying experience required varies depending on the applicant’s education level. BACHELORS DEGREE MASTERS DEGREE 14 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 When we asked why the Commission did not consider each of these applicants to be qualified, the assistant director referenced the classification descriptions for the District jobs the applicants held rather than addressing the duties they described performing while holding those jobs. However, basing a qualification decision on the general duties defined for a classification, rather than the experience the applicant describes in his or her application, particularly when the applicant’s supervisor confirms the applicant’s experience, does not allow for the possibility that the applicant has performed duties other than those generally defined for the classification. This practice does not appear to align with requirements that the merit system select employees on the basis of merit and fitness. If the Commission truly believes that internal candidates working only in specific positions have the necessary experience for a given position, it should describe time spent working in those positions—or equivalent experience—as minimum qualifications rather than using a more subjective description of “professional-level” experience. Similarly, the Commission disqualified one applicant for an executive assistant examination, in part because it determined they did not have experience providing assistance to an executive as specified in the minimum qualifications, but it accepted the applications of other applicants who likewise did not provide assistance to an executive. Specifically, it disqualified the applicant because it did not consider their supervisor, an office administrator, to be an executive or high-level administrator. The examiner and assistant director explained that the Commission was looking for applicants with experience as assistants to vice presidents, assistant directors, deans, or similar level positions. The assistant director also referenced the fact that the disqualified applicant performed work for an entire office rather than for a specific executive or administrator. However, the Commission accepted as qualified a number of applicants who described providing administrative support for offices rather than an individual, or reported to a store manager or plant operations supervisor—rather than an executive or a high-level administrator. The Commission has used ambiguous terms in its descriptions of minimum qualifications, which likely confused applicants and increases uncertainty in the qualification process. As described above, the Commission has used ambiguous terms in its descriptions of minimum qualifications, which likely confused applicants and increases uncertainty in the qualification process. Specifically, for six of 10 examinations whose minimum qualifications CALIFORNIA STATE AUDITOR | Report 2020-111 15 May 2021 we reviewed, the Commission used general terms such as “professional-level” and “recent” to describe the experience required for an applicant to qualify. However, the Commission did not define the meaning of these terms. Similarly, for one of these positions, the Commission stated that applicants must have experience in “a medium to large public agency or corporation.” The Commission did not define in the job announcement we reviewed what “medium to large” meant. Because the Commission did not define these terms, some individuals applied for jobs for which they may have believed they met the minimum qualifications but did not. For example, the Commission disqualified two applicants for a position requiring experience with a “medium to large organization” because it determined their former or current employers had fewer than 500 employees. However, this specific numerical standard was not shared with these applicants. The Commission is taking steps to address its ambiguous language. For example, the assistant director stated that the Commission revised minimum qualifications for The State’s Definition of Professional Employee examinations that required applicants to have experience working for “medium to large” The term “professional employee” means the following: organizations after receiving numerous inquiries from applicants asking how the Commission defined Any employee engaged in work: this term. Similarly, after our inquiries regarding its • predominantly intellectual and varied in character as definitions of “professional-level” and “recent,” the opposed to routine mental, manual, mechanical, or physical work; assistant director informed us that in March 2021 • involving the consistent exercise of discretion and it added definitions of these terms for its staff in its judgment in its performance; Selection Process Procedure Manual (manual). • of such a character that the output produced or the The assistant director further asserted that the result accomplished cannot be standardized in relation Commission trained its staff on the definitions of to a given period of time; these terms in the past although we could not review • requiring knowledge of an advanced type in a field the definitions because she stated this training was of science or learning customarily acquired by a verbal. However, the lack of a formal written prolonged course of specialized intellectual instruction definition in the past raises questions about the and study in an institution of higher learning or a consistency with which the Commission has applied hospital, as distinguished from a general academic education or from an apprenticeship or from training these terms. In contrast, although the State provides in the performance of routine mental, manual, or state agencies with a general definition of professional physical processes; experience by defining the term “professional employee,” as described in the text box, the State or Personnel Board recommends that state agencies Any employee who: consider revising minimum qualifications to remove • has completed the courses of specialized intellectual the term professional experience and include a instruction and study described in the fourth bullet definition of the specific required experience. If a above, and state agency continues to use the term professional • is performing related work under the supervision of experience, the State Personnel Board recommends a professional person to qualify himself to become a that it modify the minimum qualifications to provide professional employee as defined above. a specific definition, clarify acceptable and Source: State law. unacceptable experience, and add clarification to the examination bulletin for applicants. 16 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 The Commission’s practices for assessing minimum qualifications have had a number of negative consequences. By inconsistently disqualifying applicants and failing to define the terms it has used to describe minimum qualifications, the Commission may have reduced some applicants’ trust in its examination process. Responses to the union survey that the Introduction describes include allegations that qualification decisions were made off of a predetermined list and that the Commission engaged in preferential treatment and discrimination. Further, during the course of this audit, a number of District employees told us that the Commission had disqualified them for positions for which they believed they met minimum qualifications. Regardless of whether these employees’ perceptions are justified or misplaced, a lack of trust in the Commission’s decisions is harmful to employee morale and the District’s ability to attract and retain qualified employees. The Commission’s Process for Notifying Applicants of Disqualification Does Not Promote Transparency The Commission’s method of informing examination applicants that it has disqualified them does not promote transparency and trust in its process or align with the practices of similar entities. Although the Commission may disqualify applicants for a number of reasons, the notification it sends them does not disclose the reason. Instead, the notices state only that some applicants met the Commission’s needs more than others. Although the Commission’s application and examination system tracks the general reason for applicants’ disqualification, the assistant director stated it would be time-consuming for staff to enter the disqualification reason into each applicant’s notice and that staff will explain the reason applicants were disqualified if they call the Commission to ask. However, the notices do not advise applicants that they can call to obtain this information. Disqualification notices do not advise applicants that they can call to obtain the reason for their disqualification. Informing applicants of the basis for their disqualification would increase transparency and might reduce District employees’ perception that the Commission is abusing its discretion or arbitrarily disqualifying applicants. We reviewed documentation showing that a number of applicants had contacted the Commission to request information about why they were disqualified or to appeal the disqualification decision. In two of those instances, applicants had to reach out multiple times or speak at a Commission meeting before CALIFORNIA STATE AUDITOR | Report 2020-111 17 May 2021 the Commission provided a specific reason for disqualification, and in one instance Commission staff initially stated that the Commission does not provide explanations to candidates regarding why they do not meet minimum qualifications. For a third applicant, the former director explained only that there was no appeal process and that the applicant did not meet minimum qualifications. After we brought these issues to the attention of the Commission, they provided us several examples where staff responded appropriately to questions about disqualifications. However, the Commission should consistently respond to all applicants. In contrast, three other entities we spoke with that are responsible for administering merit systems all stated that they either provide the reason for a disqualification in the notice they send to applicants or inform applicants of how to obtain the reason. Two of these entities allow applicants a five-day period to contact them if they believe they have been disqualified in error and want to appeal the decision, and the third allows applicants 10 days to submit additional information and 30 days to appeal a decision. Although Commission rules state there is no appeal process for failure to meet minimum qualifications, establishing such a process would likely strengthen applicants’ trust in the Commission’s examination process and reduce the likelihood of the Commission inappropriately disqualifying applicants by providing them an opportunity to clarify information that they provide. The Commission Lacks Sufficient Guidelines for Scoring Its Examinations, Allowing Raters to Score Candidates’ Performances Inconsistently The Commission could also increase trust in its examination process by taking steps to better ensure that its raters are consistent as they score candidates. State law requires that examinations be administered objectively and it gives the Commission the authority to prescribe rules as may be necessary to ensure the selection and retention of employees upon the basis of merit and fitness. In addition, the Commission’s rules state it will select tests that assure objective testing procedures. However, the Commission generally allows raters to assign their own weights to individual evaluation factors, and it does not require them to determine a candidate’s overall score using the candidate’s ratings on the individual factors. This discretion increases the possibility of raters’ assigning candidates overall scores based on subjective factors. Indeed, when reviewing the score sheets for the 25 candidates in our selection whom the Commission determined were qualified, we found that raters’ overall scores often did not align with the ratings they assigned for the individual evaluation factors. For example, as Figure 4 shows, a rater provided Candidate A with the same or higher ratings than Candidate B for each evaluation factor; however, the rater then gave Candidate B a higher overall score. We describe these instances as scoring inconsistencies. 18 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 Figure 4 Two Candidates’ Overall Scores Did Not Align With Their Ratings on Individual Evaluation Factors Single Rater Candidate A Candidate B Evaluation Factors Training & experience Strong Strong Judgment Strong Strong Supervisory ability Strong Acceptable Organization & planning skills Strong Acceptable Computer skills Acceptable Acceptable Communication skills Strong Strong Interpersonal skills Strong Strong Overall Score 80 82 Source: Commission rating sheets, Commission rater orientation materials, and Commission staff interviews. Note: The Commission allows raters to assign their own weights to individual factors and does not require them to base overall scores on the ratings for individual evaluation factors. These scoring inconsistencies call into question the impartiality of the Commission’s examinations and the validity of the eligibility lists it provides to hiring managers. We found that individual raters inconsistently scored the scoring sheets for more than one-third (nine) of the 25 candidates we reviewed. When we asked the Commission to explain these inconsistencies, the assistant director did not address the specific cases. Instead, she generally asserted that raters are subject matter experts and that the Commission therefore gives them the discretion to assign weights to individual evaluation factors. She further stated that overall scores do not need to be an arithmetical average of individual factors and that a candidate who scored highly on the most important factors could receive a higher overall score than one who scored highly on less important factors. However, this explanation is not logical given the nature of the inconsistencies we identified. As Figure 4 shows, Candidate A received the same or higher ratings on each evaluation factor but BACHELORS DEGREE MASTERS DEGREE CALIFORNIA STATE AUDITOR | Report 2020-111 19 May 2021 received a lower overall score than Candidate B. Similarly, Figure 5 shows the individual evaluation factor ratings and overall scores that a single rater gave to three candidates for the same examination. Although the rater gave Candidate D the lowest evaluation factor ratings, the rater gave this person an overall score only one point less than Candidate E’s and eight points higher than Candidate C—whose ratings were equivalent to or higher than Candidate D’s. These scoring inconsistencies suggest that raters made mistakes in calculating overall scores, do not have sufficient guidance to calculate overall scores consistently, scored candidates on factors other than those defined in the examination, or weighted factors differently for different candidates. Figure 5 A Comparison of Candidates’ Individual Evaluation Factor Ratings Suggests That Raters Based Their Overall Scores on Other Factors Single Rater Evaluation Factors Candidate C Candidate D Candidate E Factor 1 Strong Acceptable Strong Factor 2 Acceptable Acceptable Acceptable Factor 3 Strong Marginal Acceptable Factor 4 Acceptable Acceptable Strong Factor 5 Acceptable Acceptable Strong Factor 6 Marginal Marginal Exceptional Overall Quality Acceptable Acceptable Exceptional Overall Score 75 83 84 Source: Commission rating sheets, Commission rater orientation materials, and Commission staff interviews. Note: The Commission allows raters to assign their own weights to individual factors and does not require them to base overall scores on the ratings for individual evaluation factors. Inconsistent scoring had a direct impact on these applicants’ job prospects. Based in part on the raters’ overall scores, the Commission assigned Candidate C an eligibility list rank of 5, which was not high enough to merit an interview for an open position. Generally, state BACHELORS DEGREE MASTERS DEGREE 20 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 law governing personnel commissions requires all vacancies be filled with eligible candidates whose scores place them in the first three ranks on the eligibility list and who are ready and willing to accept the position. The executive assistant explained that hiring managers can only interview eligible candidates in lower ranks if eligible candidates in higher ranks are not ready and willing to accept the position. According to Commission rules, if there are fewer than three eligible candidates in the first and second ranks, hiring managers must interview all eligible candidates in the first, second, and third ranks. The Commission states on its website that one of its goals is to select employees on the basis of merit after fair and open competition. However, by allowing scoring inconsistencies between applicants, the Commission is not ensuring that raters treat all candidates fairly, and it increases the risk that in some cases the District is not hiring the most qualified candidates because they incorrectly receive overall scores that are too low to merit consideration for an interview. Further, the Commission’s practices and guidelines do not identify or prevent such scoring inconsistencies. According to the assistant director, examiners cannot require raters to change their scores, but they can ask two or more raters of the same candidate to discuss the candidate further if they differ by more than 10 points in their overall scores. Similarly, the Commission’s guidance to raters generally focuses on inconsistencies between different raters on a panel. For example, the guidance suggests that if raters’ scores for a candidate differ by more than 10 points, they may be evaluating the candidate based on different criteria and further discussion may be necessary. The guidance does not address scoring Rating Definitions inconsistencies by the same rater or steps the From Commission Scoring Sheets raters should take to avoid these inconsistencies. • Weak: Did not demonstrate basic competence in this area; lacks critical aspect(s) of this factor. Other entities that are responsible for administering • Marginal: Lacks some aspect(s) of this factor; you would merit systems more clearly define how raters must not expect to see much effectiveness in this area. calculate scores, which helps prevent inconsistencies • Acceptable: Shows good, basic competence in this and reduces the risk of subjectivity. Three entities area. It is neither a particular strength nor a detriment to we spoke with all provide their raters with detailed performance. definitions for rating candidates on each evaluation • Good: Demonstrates above average competence in this factor (scoring benchmarks), which they require area. It is an area of particular strength. raters to use. In contrast, the Commission’s rating • Strong: Shows great expertise/capability in this area sheets provide only brief, general definitions, as and would demonstrate considerable effectiveness the text box shows. In addition, one agency stated in performance. that its rating sheets establish the weight of each Source: Commission examination scoring sheet for the position individual evaluation factor in the calculation of of general foreman. a candidate’s overall score, while the other two Note: The Commission’s rating definitions contain minor entities define the number of points awarded for phrasing differences for some examinations. candidates’ accomplishing specific tasks or providing specific answers. Establishing more detailed CALIFORNIA STATE AUDITOR | Report 2020-111 21 May 2021 procedures for its examination process, such as scoring benchmarks and weights, would give the Commission greater assurance that its raters are scoring candidates consistently and objectively and, thus, that the Commission is ranking candidates on the eligibility list fairly and impartially. Some Raters Provided Minimal Comments to Justify the Scores They Awarded, Limiting the Commission’s Ability to Ensure They Based the Scores on the Established Examination Factors Some raters did not adequately justify their scores with comments explaining the basis for those scores. Although the Commission’s rules do not require raters to leave comments, its guidelines for raters indicate that their ratings of individual evaluation factors and written comments should support or justify the overall score they assign to candidates. Further, the rating sheets for four of the five examinations that we reviewed directed raters to add an explanation if they recommended that a candidate not be hired. Commission guidelines also instruct raters to avoid generalizations and to be specific if they note critical shortcomings. Nevertheless, as Figure 6 shows, some raters provided insufficient justification of their scores on rating sheets for 19 of the 25 candidates we reviewed. Based on the Commission’s guidance, a rater who fails a candidate— or assigns them less than a passing score—should describe specific areas of concern and explain why the candidate is not a good fit for the job. Nonetheless, one rater who failed a candidate and scored them as weak or marginal in three evaluation factors and as acceptable—the middle-level score—in four evaluation factors commented on the rating sheet that the candidate was “not quite ready” for the position, but did not provide specific reasons to explain why they assigned the candidate an overall score that was not high enough for the candidate to proceed to the next stage of the examination process. In addition, we identified two raters who each failed a candidate but did not provide a single comment to explain why, although the directions for the rating sheet explicitly direct raters to explain the reasons why they would not hire the candidate for the job. The Commission failed to adequately enforce its guidelines instructing raters to justify their scores for candidates they fail. The Commission’s instructions for examiners require them to review each completed rating sheet. When we asked the assistant director why the Commission did not identify the instances in which raters did not follow the guidelines, she suggested that the raters might not have had enough time to leave comments or might have felt that their evaluation factor ratings provided enough detail. However, she agreed that examiners should have instructed raters who failed 22 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 candidates to leave comments justifying their decisions. Further, the director stated that he has instructed staff not to use any rater who demonstrates an inability to comply with the guidelines. Comments also serve additional purposes. For example, one candidate appealed her examination results because of concerns that the raters interviewing her were biased by her late arrival. The examiner stated in his appeal decision that based on his review of the raters’ notes, he was able to determine that they made their decision based on factors unrelated to the candidate’s arrival time. When raters do not leave comments, the Commission’s ability to adequately defend its scores against an appeal is weakened. Moreover, one of the other entities with which we spoke instructs raters to leave comments to serve as developmental feedback should a candidate review the ratings. Figure 6 The Raters Provided Minimal or No Justification to Support Their Ratings for 19 of the 25 Candidates We Reviewed Minimal Justification for Some Ratings and No Justification for Others Minimal No Justification Justification for Ratings for Ratings 6 3 10 Source: Commission rating sheets. Although the Commission also retained raters’ informal notes from some candidate interviews, the majority of the notes we reviewed described candidates’ answers rather than assessed their performance. Further, the notes do not improve the transparency of the rating process or allow candidates to review and improve their performance. The assistant director stated that the Commission provides candidates with only the comments that raters make on the rating sheets. CALIFORNIA STATE AUDITOR | Report 2020-111 23 May 2021 The Commission does not require raters to leave comments for the candidates that they pass. Although the Commission’s orientation materials encourage raters to leave comments for all candidates, the Commission does not require raters to leave comments for the candidates that they pass. However, other entities we reviewed emphasize the importance of such explanations. One of the entities specifically directs raters to “take lots of notes” because they are useful to improve both the applicants’ future performance and the validity of the raters’ scoring. As we describe in the previous section, several Commission raters provided inconsistent scores to candidates they reviewed, which makes justification of scores even more critical. Because the Commission allows its raters such wide latitude in deriving an overall score from the individual rating factors, written justifications for those scores are necessary to understand how raters reached their decisions, regardless of whether the candidate failed the examination. In addition, they are an important safeguard to reduce the likelihood that raters based their scores on factors other than a candidate’s performance on the examination. Recommendations To increase the objectivity and transparency of its minimum qualification requirements, when possible the Commission should create qualification requirements based on time spent working in District job classifications or equivalent experience, rather than using ambiguous terms such as “professional-level.” To increase transparency and ensure that it makes consistent decisions when assessing applicants’ minimum qualifications, the Commission should establish a rule for its examiners by October 2021 that defines the key terms it uses when reviewing applications for minimum qualifications, such as “professional-level” and “recent.” To ensure that its examination process is fair and evaluates all candidates consistently, the Commission should establish the following rules by October 2021: • Require examiners to provide disqualification notices that describe their reasons for disqualifying an applicant. 24 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 • Require examiners to create detailed scoring benchmarks that provide raters guidance on how to rate individual evaluation factors. • When creating examinations, establish a method for determining candidates’ overall scores based on the ratings of the individual evaluation factors. • Require raters to provide written comments on rating sheets for each candidate, explaining the basis for the score they awarded. • Require examiners to review scoring sheets to determine if raters have followed the Commission’s candidate evaluation guidance, and if the raters have failed to follow the guidance request that the raters review their evaluation of the candidate. To promote transparency in its application process and to minimize the perception that it arbitrarily disqualifies applicants, the Commission should revise its rules by October 2021 to implement an appeal process for applicants who it has determined do not meet minimum qualifications for a position. CALIFORNIA STATE AUDITOR | Report 2020-111 25 May 2021 The Commission’s Policies and Practices Do Not Ensure Prompt Compensation for All Out‑of‑Class Work Key Points • The Commission generally does not approve employees’ claims for work performed out of class until the employees have completed their out-of-class assignments. As a result, employees may perform higher-level duties for many months before receiving compensation for performing those duties. • Although the Commission generally does not approve employees’ out-of-class work claims until their higher-level assignments have concluded, it denies a portion of the employees’ compensation if they do not submit their claim within a set number of days after beginning their out-of-class work. The Commission’s Practices Delay Compensation for Employees Temporarily Performing Higher-Level Responsibilities District supervisors sometimes assign work to an employee that does not fall within that employee’s job classification. Under certain circumstances, employees who are required to perform duties inconsistent with those assigned to their position may submit a claim to the Commission to determine whether they are entitled to an appropriate increase in compensation. Generally, under state law and Commission rules, employees required to perform duties inconsistent with those of their position on more than five days of a 15-day period may have their compensation increased for the higher-level duties performed. To claim this compensation, the employees must—among other things—identify the specific time period during which they performed the higher-level duties and describe in detail the duties which they believe to be inconsistent with or not reasonably related to the duties of their regular class. We evaluated the Commission’s decisions for six of the 22 out-of-class work claims it considered from July 1, 2016, through June 30, 2019, and found that it consistently applied its rules. However, the Commission’s rules do not set requirements for when it should process or approve out-of-class claims, and its practice is to approve payment for the entire assignment after the conclusion of the out-of-class work, as Figure 7 shows. 26 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 Figure 7 The Commission Typically Does Not Approve Any Payments for Out-of-Class Work Until the Assignment Is Concluded Employee is required to perform duties inconsistent with those assigned to his or her class. Employee submits claim forms to supervisor for his or her certification that the duties were assigned and performed. Supervisor and college president or division head approve the claim and forward it to the Commission. Commission staff review information provided by the employee and/or supervisor to determine if the duties are at a higher level than the employee’s regular class. Generally, once the employee has completed the out-of-class work, Commission staff determine an appropriate difference in compensation. The Commission staff’s determination is submitted to the Commission for approval. If approved, the determination is submitted to the Board of Trustees for authorization of payment. District payroll department processes approved payments. Source: Commission rules, out-of-class claims, out-of-class claim form, and interviews with District staff. As a result, for five of the six out-of-class claims we reviewed, employees did not receive payment until five to 11 months after they began the higher-level work, as Figure 8 shows. Some of this delay is related to the time needed to process these claims. For example, for each of the out-of-class claims, the Commission evaluates whether the duties reported and certified were at a higher level, determines an appropriate difference in compensation between the employee’s classification and the classification normally assigned the out-of-class duties, and submits the claim to the commissioners for approval. BACHELORS DEGREE MASTERS DEGREE CALIFORNIA STATE AUDITOR | Report 2020-111 27 May 2021 Figure 8 The Commission’s Approach to Compensating Out-of-Class Work Results in Months of Payment Delays MIALC SSALCFOTUO MONTHS 0 2 4 6 8 10 12 1 2 3 4 5 6 Time working out of class Commission approval Time between end of out-of-class work and payment Payment Source: Analysis of Commission out-of-class work claim records, Commission meeting minutes, and payroll documentation. Note: As discussed later in this section, the Commission made multiple payments to the employee who submitted Claim 5. The Board’s approval process and the District’s subsequent processing of the payment added an average of two months to the time frame for the six claims we reviewed, and these steps took more than four months in one case. However, the remaining delay was largely attributable to the fact that the Commission typically does not approve claims until an employee has submitted all claims after an out-of-class work assignment ends. As the text box describes, the State uses a different method of calculating out-of-class compensation. That method improves its ability to make timely payments but does not result in additional compensation if employees spend a greater proportion of time working out of class than the minimum necessary to qualify for payment. 28 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 The Commission’s rationale for processing Compensating Work Out of Class: employees’ compensation in its current manner is Two Approaches questionable. According to the assistant director, it is more efficient for Commission staff, the Board, The District’s approach: Its out-of-class compensation and the District’s payroll department to approve all reflects the level and nature of the assigned responsibilities, the difference in compensation between the employee’s of an employee’s claims at once instead of each classification and the classification that normally performs claim individually. The assistant director stated that the assigned responsibilities, and the percent of time the processing claims more regularly would require employee performs the assigned out-of-class duties during additional work for staff to compile the reports and a given pay period. As a result, Commission staff cannot place the issues on the Commission’s meeting calculate the compensation amount for a given pay period agendas, for management to review the reports, and until that pay period ends and the employee submits a claim. for the commissioners to review the claims. However, the assistant director could not estimate The State’s approach: Its payments for out-of-class work are generally based on the employee’s salary, the nature the average number of hours it takes to process a of the out-of-class duties, and the number of days spent work-out-of-class claim, instead citing factors that performing those duties, but they do not vary based on can vary by claim. As a result, it is not possible to the proportion of the employee’s time spent working out estimate how much additional work would result of class each day. Generally, an employee is considered to from processing multiple payments for each claim. be working out of class if he or she performs the full range However, the Commission’s handling of Claim 5— of duties and responsibilities of another classification for which we describe in more detail below—suggests more than 50 percent of the time over the course of at that it can process subsequent payments for the least two consecutive weeks. When he or she is found to same claim more efficiently. It processed multiple be working out of class, he or she receives the rate of pay payments to this employee for her out-of-class work for the higher classification level. Thus, if the employee and processed her final claim and provided it to the meets the 50 percent requirement, the compensation is not commissioners for approval less than a week after affected by spending a higher percentage of time on duties outside his or her classification. Although the employee’s the employee finished performing the compensation will not increase if he or she spends a greater out-of-class work. proportion of time performing out-of-class work, the State’s payments for out-of-class work can be made in a more The Commission could alter its practices to timely manner than those made by the Commission. approve employees’ compensation for working out of class in a more timely manner, which would Source: Interviews and analysis of state and Commission policies and rules related to out-of-class work. encourage employees to submit their claim forms as soon as they are able to do so. The Commission could process payments monthly to coincide with its monthly Board meetings. The Commission considered only 22 out-of-class claims in the three-year period we reviewed, which suggests that the additional work necessary to process these claims each month would not dramatically increase the Commission’s workload. By processing out-of-class payments monthly, the Commission would likely decrease the time between employees’ performing out-of-class work and submitting their claims. This would provide two benefits. The assistant director and two personnel analysts indicated that the less time that passes between when an employee works out of class and when that employee submits a claim, the more information staff and supervisors are generally able to recall about the work performed. Additionally, it would reduce some District employees’ frustrations with the lengthy amount of time that they must wait to receive their pay for working out of class. CALIFORNIA STATE AUDITOR | Report 2020-111 29 May 2021 Two respondents to a Commission employee experience survey specifically expressed concerns about the amount of time it takes to be paid for out-of-class work. The Commission sometimes approves partial payments while an employee completes an out-of-class work assignment rather than waiting until after the work is complete. Claim 5 in Figure 8 involved an employee for whom the Commission processed and approved intermittent payments. According to the assistant director, the Commission felt it would have been harmful to the employee to withhold payment for the work because the campus was not following through with the hiring process to fill the vacant position for which the employee was performing duties. When we asked why it was harmful to withhold payment in this case but not in others, she stated that the Commission staff use their professional judgment but would process a claim in this way if an employee specifically requests it or if extenuating circumstances exist, particularly if those circumstances are related to the administration or management. However, the Commission does not proactively inform employees of this option, nor does it clearly describe this in its rules or on the out- of-class claim form. Two respondents to a Commission employee experience survey specifically expressed concerns about the amount of time it takes to be paid for out-of-class work. Other claims we reviewed involved longer time frames and larger amounts of money than Claim 5 yet did not result in intermittent payments. Of the six claims we reviewed, four were for larger amounts and, in one instance, an employee worked out of class for nearly six months before the Commission approved his out-of-class compensation of $12,000, which he did not receive until four months later. Although the Commission may not have been able to predict the duration or value of these out-of-class assignments when they began, its practices resulted in excessive delays in compensation and inconsistent treatment for some employees. These practices do not appear to be in the District’s best interests. Research shows that employees’ perceptions of equity or fairness have a significant relationship to absenteeism and turnover. Requiring most employees to wait to receive additional pay until their out-of-class assignment has concluded while some receive pay intermittently would likely increase employees’ perception that they are being treated unfairly. 30 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 The Commission Did Not Approve Full Compensation for the Out-Of-Class Work of Employees Who Missed Deadlines for Submitting Claims During the period we reviewed, the Commission’s rules generally required employees to submit claims for out-of-class work no later than 100 days after the first day they performed higher-level duties. We refer to this requirement as the 100-day limit. Generally, classified employees must identify the specific time period during which the higher duties were performed and if the work was performed intermittently, the employee must have performed the duties for five or more working days within each 15 calendar-day period. Generally, if the Commission receives the first of these claims more than 100 days after the employee began working out of class, it processes the claim but does not include compensation for the out-of-class work the employee performed that occurred more than 100 days before the date it received the claim. The Commission’s rules allow for exceptions to the 100-day limit in certain circumstances, such as when there is credible documentation of “a mistake, inadvertence, surprise, or excusable neglect by an employee or supervisor.” However, the exception specifically excludes instances in which a lack of knowledge of the out-of-class rule caused the untimely filing of the claim. The out-of-class claim form describes the 100-day limit; however, the form did not disclose the exception until February 2021, after we had questioned the Commission about this issue. For two of the six out-of-class claims that we reviewed, the Commission received the employees’ first claim more than 100 days after the employees began the out-of-class work. As a result, the Commission excluded from both individuals’ compensation the additional work they performed that occurred more than 100 days before the Commission received their claims. Although the Commission did not calculate the value of this work, we estimated the employees would have been compensated 5 percent or 13 percent more, respectively, based on the amounts they received for the periods for which they were compensated. As we describe previously, the Commission’s general practice is to approve payment for the entire assignment after the conclusion of the out-of-class work. However, based on the 100-day limit, the Commission did not approve payment for a portion of the out-of-class work these two employees performed, despite the fact that they were still performing out-of-class work when they submitted their initial claims. In fact, one employee was still working out of class for more than two months after the Commission received his initial claim. In April 2020, the Commission further restricted the time allowed to submit out-of-class claims by amending its rule to require that employees submit them within 45 days of beginning the out-of-class work. According to a document describing the rationale for this CALIFORNIA STATE AUDITOR | Report 2020-111 31 May 2021 change that the assistant director sent to the commissioners, the Commission had encountered problems with the late submission of claims and when claims are submitted late, employees have difficulty recalling or are unable to recall necessary details of work performed. The assistant director stated that 45 days should provide sufficient time for employees to submit a claim because out-of-class assignments are meant to be short in duration. However, limiting the time available to submit a claim for compensation has no influence on the length of an out-of-class assignment. The document also stated that other organizations have instituted similar time frames for employees to submit claims. The assistant director provided us with evidence of one other community college personnel commission that requires an employee to begin the claim process within 45 days of beginning out-of-class work. However, the other organizations we spoke to all provided time frames of one to three years for employees to submit out-of-class claims. For example, the Los Angeles Unified School District compensates employees for out-of-class work they performed up to three years before filing a claim, and the law allows the State to reimburse some employees for the duties outside the scope of their classifications they performed up to one year before filing a claim. The other organizations we spoke to all provided time frames of one to three years for employees to submit out-of-class claims. Further, the Commission’s implementation of the claims process raises questions about the practicality of changing the time limit from 100 to 45 days. For the purposes of determining compliance with the time limit, the Commission uses the date it receives the out-of-class claim form as the date of submission. According to the assistant director, this date is more reliable because employees could backdate their signature. However, as Figure 7 shows, a supervisor and division head or college president must approve the claim before sending it to the Commission. As a result, the employee cannot control when the Commission will receive the claim. In the six out-of-class claims we reviewed, an average of 17 days elapsed between the date of the employee’s signature and the date of the Commission’s receipt of the claim—more than a third of the 45-day time frame an employee now has to submit the claim. The Commission’s practice of basing the time limit on the date it receives the claim places an unfair burden on employees for processes outside their control. The time between the employee’s signature on the form and the date of the Commission’s receipt varied widely for the six claims we reviewed—from one to 49 days. Because employees have no control over 32 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 this aspect of the process, penalizing them for such delays by limiting their compensation is unreasonable. Although the reason for a specific delay may qualify a claim for the exception to the time limit that we describe earlier in this section, the assistant director stated that the employee must request such an exception. However, the Commission did not describe this exception on the claim form before February 2021, and the form still does not specify that the employee must request the exception. The Commission’s practice of basing the time limit on the date it receives the claim places an unfair burden on employees for processes outside their control. Recommendations To ensure that employees are aware that they can request intermittent payments while performing out-of-class work assignments, the Commission should immediately revise its claim form to include this option. To ensure that employees receive prompt compensation for the higher-level duties they perform, the Commission should revise its rules by October 2021 to process employees’ compensation for out-of-class work each month. To ensure that employees are fairly compensated for the entirety of the out-of-class work they perform, the Commission should amend its rules to do the following by October 2021: • Allow employees at least 100 days to submit their out-of-class work claims before limiting their compensation. • Require employees to submit a copy of their out-of-class claim form to the Commission at the same time as they submit it to their supervisors, and use the date the Commission receives this copy of the form as the date of submission. CALIFORNIA STATE AUDITOR | Report 2020-111 33 May 2021 The Commission Does Not Adequately Address All Complaints and Protect Complainants Key Points • The Commission lacks a defined process for addressing all employee complaints, and it could not document that it adequately addressed all of the complaints it received. Further, the commissioners provide limited oversight of the complaint process because the Commission staff generally does not notify them of complaints. • The Commission’s former director received details about whistleblower-type complaints made against her, despite the fact that sharing such details increased the risk of retaliation against those complainants. The Commission Does Not Adequately Track and Address All Employee Complaints When District employees or examination applicants are dissatisfied with the Commission, they have a number Primary Methods of Submitting of options for submitting complaints. They can submit a Complaint to the Commission complaints to the Commission itself using several different methods, as the text box shows. However, • Raise concerns during the public comment period of Commission meetings. employees may also submit complaints about the Commission to other entities both within and outside • Email complaints to Commission staff. the District, including the District’s human resources • Call Commission staff to voice complaints. division, the State’s Department of Fair Employment and • Complete the feedback survey on the Housing, and the State’s Public Employment Relations Commission’s website Board. Each of these entities is responsible for addressing Source: Commission website and interviews with the complaints it receives. Employees covered by the Commission staff. terms of the collective bargaining agreement may also make complaints through the union’s grievance procedure about applicable Commission rules. The Commission’s practices for documenting both complaints and the actions it takes to resolve them are inconsistent and vary based on how the complaint is submitted, in part because it has not clearly defined what it considers to be a complaint nor established a specific process for managing all of them. According to the assistant director, the Commission receives questions, inquiries, and complaints on a spectrum from minor, which are routine and simple to resolve, to serious allegations that require more formal action. She stated that the vast majority of inquiries and concerns are requests for information and for assistance in solving problems and Commission staff address these issues as they arise, often resolving them immediately. For example, if staff receive a complaint regarding a Commission process, such as a reclassification study, they document it in that study’s file. In contrast, the Commission records all complaints made at Commission meetings in a log that includes a description of the actions the Commission took to address them. However, the Commission’s rules 34 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 do not clearly define what it considers to be a complaint nor do they establish a formal process for managing all complaints. The assistant director asserted that it is unreasonable for any business to document every inquiry it receives and that the Commission tracks large issues, which she described as the formal appeals and issues raised at Commission meetings.2 Despite this assertion, as we describe below, we identified a number of complaints the Commission received via email that it failed to address. This may be the result of the lack of clear criteria in the Commission rules defining what represents a complaint or the lack of a formal process for managing those complaints. We identified a number of complaints the Commission received via email that it failed to address. Because the Commission does not consistently document and track complaints, it could not provide a comprehensive list of complaints it received from fiscal years 2016–17 through 2018–19. Instead, we reviewed a log of the comments that the public raised during Commission meetings (speakers log) to identify concerns raised in those meetings, and we searched for specific terms in emails sent to selected Commission accounts.3 We found that the Commission most consistently tracked complaints from the public comment periods of its meetings. The Commission’s executive assistant is responsible for maintaining the speakers log for these meetings, which lists 13 comments for fiscal years 2016–17 through 2018–19. However, public comment is not restricted to complaints, and as we describe above, the Commission’s rules do not provide clear criteria for what it considers to be a complaint. The summaries for the 13 comments include four that appear to be complaints about unfair personnel decisions. We determined that in each of these instances, the Commission responded appropriately. Although the Commission adequately addressed the complaints made in public meetings that we reviewed, it did not consistently do so for the complaints we identified through our search of 2 As a result of the concerns we raised about adequately handling complaints, the assistant director stated that in December 2020 she instructed staff to begin keeping a log of inquiries and concerns submitted by employees through the Commission’s website and general email. 3 It is likely that the Commission received additional complaints in its other email accounts, including the accounts it uses for job postings or general inquiries. Additionally, it also may have received additional complaints by phone. However, for the purposes of this audit, we reviewed only two of its email accounts. CALIFORNIA STATE AUDITOR | Report 2020-111 35 May 2021 its email. When we searched for specific terms in emails sent to two Commission addresses from July 1, 2016, to June 30, 2019, we identified 21 complaints from applicants and District employees, many of which related to concerns we describe elsewhere in this report. The Commission could not provide documentation that it appropriately addressed five of these 21 complaints. Although two of the five fell outside its purview, the Commission was unable to provide assurance that it referred these issues to the appropriate entities. The Commission’s Current Rules Concerning Whistleblower Information Create an Unnecessary Risk of Retaliation According to the Commission’s rules, requests for an investigation of personnel problems related to alleged violations of merit system laws or Commission rules must be made in writing. The director will conduct an investigation into the allegation although every effort must be made to resolve the matter informally. If informal resolution is not possible or the findings require formal action, a report with findings and recommendations is presented to the commissioners. The rules also specify that if the allegations implicate the director, the Commission must appoint an independent investigator. We identified two complaints the Commission received alleging misconduct by a director who has since retired (former director). The District’s general counsel sent both of the complaints to the former director and the commissioners. The emails included the complainants’ names as well as details of their allegations about the director. Commission rules do not address whether the director should be notified of requests for an investigation involving them; however, sharing that information creates a risk of retaliation against the complainant. The complaints against the former director could be considered whistleblower complaints because—as described by the nonprofit National Whistleblower Center—they describe alleged wrongdoing to those within the organization with the authority to correct that wrongdoing. Federal best practices describe methods for protecting whistleblowers that include providing a method of reporting the issue outside a complainant’s chain of command, establishing an independent complaint review process, and providing protection and confidentiality to those who submit complaints. Requiring an independent investigator for complaints involving the director is an important aspect of ensuring that they are evaluated objectively. However, notifying the director of the details of the complaint and the complainant’s identity before the investigation occurs is incompatible with maintaining complainant confidentiality, which is necessary to prevent retaliation and ensure an impartial investigation. The Commission’s failure to adequately 36 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 maintain complainants’ confidentiality in these cases increases the risk of retaliation and reduces the likelihood that future complainants will inform the Commission of potential wrongdoing. Notifying the director of the details of the complaint and the complainant’s identity before the investigation occurs is incompatible with maintaining complainant confidentiality. Further, the Commission processed only one of the two complaints against the former director in accordance with its rules. For that complaint, the commissioners delegated the issue to the District’s Office of General Counsel, which contracted with an outside party to conduct an investigation. However, the Commission did not provide evidence that it appointed an independent investigator to investigate the other complaint as required by Commission rules or took any other action. According to the Commission chair, the District’s general counsel and the vice chancellor decided that the complaint did not allege a violation of merit system rules, and therefore the rule requiring an independent investigation did not apply. However, the complainant alleged among other things that under the former director’s oversight, the Commission had not properly evaluated employee classifications, had revised job descriptions to prevent upward mobility, had created Commission rules not in accordance with state law, and had inconsistently applied standards and rules—all of which appear to violate the merit system’s rules. Involving the commissioners in the complaint process could help ensure that the Commission handles complaints appropriately and could prevent potential conflicts of interest. Currently, commissioners have little to no involvement in complaints. According to the assistant director, Commission staff generally do not have a reason to notify commissioners of complaints. In contrast, the State Center Community College District (State Center) Personnel Commission rules define a multi-level process for addressing complaints in which a complainant may submit a complaint to its commissioners if the complaint is not resolved at earlier levels. This ensures that all complaints are addressed and it elevates to the commissioners only those complaints unresolved at lower levels. Based on the assistant director’s assertion that serious complaints are infrequent, directing formal complaints that are not resolved by Commission staff to the commissioners would not significantly increase their workload. Further, including a provision CALIFORNIA STATE AUDITOR | Report 2020-111 37 May 2021 for submitting whistleblower complaints to the District’s general counsel—who is already generally responsible for representing the Commission in all legal matters—could help ensure that all complaints are addressed and reduce the risk of retaliation against complainants. Recommendations To ensure that it consistently identifies and responds to all complaints and to reduce the risk of retaliation against complainants, by October 2021 the Commission should amend its rules to do the following: • Clearly define complaints and create a formal process for addressing all complaints, including a process to elevate to the commissioners those complaints that are not resolved at lower levels. • Include a provision for submitting whistleblower complaints directly to the District’s Office of the General Counsel and assign it the responsibility of designating an appropriate party to respond. • Establish that complainant information may not be shared with the subject of a whistleblower complaint. 38 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2020-111 39 May 2021 OTHER AREAS WE REVIEWED To address all of the audit objectives approved by the Joint Legislative Audit Committee (Audit Committee), we reviewed eight issues in addition to those we discuss previously. Two of these issues resulted in additional recommendations to the Commission. The Commission’s Rules State law imposes some requirements as to how personnel commissions must operate, but it also gives them the authority to prescribe rules as may be necessary to insure the selection and retention of employees on a basis of merit and fitness. We reviewed a selection of the Commission’s merit system rules governing discipline, examinations, and classifications and found that they are generally consistent with state law. For example, state law requires reasonable notice be given to the exclusive bargaining unit representatives of any proposed classifications or reclassifications that would affect the classified employees they represent, and the Commission’s rules establish a policy of providing advance notice to exclusive bargaining unit representatives and consulting with them about a proposed classification or reclassification that could affect the bargaining unit or its members. In addition, the Commission’s rules are consistent with the elements included in the District’s bargaining agreement with the staff union, which represents the majority of the District’s classified employees. The collective bargaining agreement sets out the conditions of employment for classified employees and generally references and defers to the relevant Commission rules or processes for appealing discipline and examination decisions. The agreement also recognizes the Commission’s role in classifications and references its rule for out-of-class claims. For example, the agreement states that employees may file a request for reclassification with the Commission, that the Commission must notify the union when it receives such a request, and that the rules regarding reclassifications are contained in the Commission’s rules and state law. The Commission’s Debarment and Notification Practices During the course of this audit, District employees alleged to us that the Commission inconsistently denied them the opportunity to participate in the examination process with the District for a specific period of time, an action known as debarment. To investigate this allegation, we assessed the Commission’s 40 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 procedures for identifying facts that could lead to a debarment and reviewed pertinent documents for a selection of 10 individuals debarred in fiscal years 2017–18 through 2019–20. These debarments were the result of the Commission’s determining that candidates had made false statements in their applications or related documents, omitted material facts from their applications or related documents, or practiced deception or fraud to pass an examination or to secure employment. The Commission’s policies state that in most cases, deception is grounds for permanent debarment although the assistant director stated that the Commission rarely permanently debars current employees. Commission staff explained that debarments typically stem from their review of applications. During this review they sometimes identify items that appear vague, evasive, or incorrect. When they identify such a concern, they review past applications from the individual, if available, to look for inconsistencies. Commission staff whom we spoke with described identifying inconsistencies between the application they are assessing and previous applications from the same individual as a significant aspect of their method for identifying false statements or deception. They described using their professional judgment to determine which inconsistencies are significant. In some cases, they take additional steps to confirm the inconsistency, such as asking an applicant for clarification, verifying actual duties with a supervisor, or reviewing the specifications of the applicant’s current position. However, examiners have the discretion to determine when to contact applicants for additional information, which the applicant has three days to provide. We observed that examiners generally used the Commission’s electronic job application system to record any additional verification steps they took—such as obtaining information or documentation from a previous employer—but they did not consistently retain documentation of the information they obtained in the system. Because Commission staff do not always obtain additional information to verify whether inconsistencies constitute a false statement, applicants may face differing levels of scrutiny. In five of the 10 cases we reviewed, the examiners did not take additional steps before determining that debarment was the appropriate course of action. In three of those five cases, the examiners noted that it appeared that the applicants attempted to meet a position’s minimum qualifications by including additional experience they had not listed on past applications. Although such an addition may be grounds for concern, it is also possible that the applicants omitted those jobs or duties from past applications because they were not relevant to the positions for which they were previously applying. Because debarment has significant consequences, we expected the Commission to first take additional steps, such as contacting current or past supervisors, to verify the information CALIFORNIA STATE AUDITOR | Report 2020-111 41 May 2021 on an individual’s current application. Additionally, the Commission did not provide the applicants the opportunity to submit additional information regarding the issue for which they were debarred in any of the 10 cases we reviewed. We question the Commission’s decision to impose such significant repercussions without consistently allowing applicants to provide clarifying information. Recommendations To ensure that it treats applicants consistently when considering whether to debar them in the case of false statements or deception, the Commission should establish rules to require that examiners do the following: • Independently verify the reason for inconsistencies between applications. • Provide applicants with an opportunity to address the inconsistencies. • Document the steps taken to verify the disputed information and retain relevant supporting documentation. The Commission’s Accountability Measures Although the Commission is responsible for supervising staff implementing the Commission’s rules, our review indicates that it could improve its oversight of its staff. As the Introduction describes, three commissioners are appointed to staggered three-year terms. The Commission staff review and make determinations, which are submitted to the commissioners for approval, such as a recommendation that the Board approve a new job classification. Although the Board may approve, amend, or reject certain Commission recommendations, the commissioners are responsible for prescribing, amending, and interpreting rules subject to the merit system as set forth in state law, and supervising those activities of employees that are performed as part of the functions of the Commission. The commissioners should exercise greater oversight of the Commission’s operations. For example, the examination scoring inconsistencies we describe previously are not surprising, given the broad discretion the Commission affords its raters. None of the other organizations responsible for administering merit systems that we spoke with allow such discretion. However, the assistant director asserted that the Commission did not agree with our characterization of these inconsistencies, and she reiterated that the Commission allows raters discretion in determining overall scores. The Commission staff’s inability to recognize that its processes are not adequate to ensure consistent overall scores or the importance 42 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 of such consistency indicates that the commissioners must take a more direct role in overseeing the Commission’s practices. To minimize the risk of such inadequate processes, the commissioners should periodically require staff to compare the Commission’s practices to those of other merit systems and report the results to the commissioners. This will allow the Commission to examine the potential value of differing practices from comparable entities and to consider aligning its processes with these entities’ best practices. Further, the commissioners should require more information of Commission staff when considering proposed revisions of rules and other actions. As we describe previously, the Commission revised its rules concerning work out of class to reduce the 100-day limit for submitting a claim to 45 days. One of the stated reasons for this change was that staff had found that other entities have instituted similar time restrictions. However, the staff did not provide the details of their comparison to the commissioners, and the three entities we reviewed each provide one to three years for employees to submit such claims. If the commissioners require details of other entities’ practices when considering rule changes, they will have a better understanding of the context for those changes and awareness of potential best practices the Commission could adopt. Recommendation To ensure that the Commission’s practices align with the mission of the merit system, the commissioners should establish rules that require staff to periodically report to them on how its practices compare to those of other entities with merit systems, along with any recommendations for improving the Commission’s practices. Other Allegations We Received During the Course of the Audit During the course of the audit, a number of District employees alleged that the Commission had engaged in improper conduct. To determine whether the Commission had engaged in the alleged behaviors, we performed some additional audit procedures; we did not perform procedures for every allegation we received because some were not within the scope of our audit and it was not possible or cost-effective to objectively assess others. We generally limited these additional procedures to the period from 2018 through 2021. We did not attempt to determine whether the Commission engaged in the alleged behavior before the time periods we reviewed because we were primarily concerned with determining whether the behavior was an ongoing issue. Table 2 lists the allegations we assessed that we have not addressed in previous sections of this report, the methods we used to assess them, and the results of our analysis. CALIFORNIA STATE AUDITOR | Report 2020-111 43 May 2021 Table 2 Other Allegations We Received During the Course of the Audit ALLEGATION METHOD USED TO ASSESS THE ALLEGATION FINDING The Commission We reviewed the minimum qualifications for We were unable to substantiate this allegation. Although arbitrarily changed 10 job classifications, including eight for which the the Commission made the minimum qualifications for the minimum qualification Commission changed the minimum qualifications eight classifications more rigorous, these changes related to requirements. or class qualifications during 2017 through 2019. the duties of the positions, were applied consistently, and For each change, we determined whether the either reduced ambiguity in the qualification description or change made the qualifications more rigorous better tailored the requirements to the job classification or and obtained the Commission’s rationale for the District’s operations. For example, in some instances, the the changes. Commission included requirements for a bachelor’s degree rather than “graduation from a college or university.” The Commission We reviewed the minimum qualifications of We were unable to substantiate this allegation. The five established inconsistent five director-level job classifications, including classifications we reviewed had a number of similar minimum minimum qualifications the position to which the allegation pertained. qualifications. All five required similar levels of education. for a director position We also reviewed changes the Commission made The changes the Commission made to four of the classifications in order to discriminate to the qualification requirements for these job in 2018 made the positions’ minimum qualifications more against one applicant. classifications in 2018. consistent with each other. According to the assistant director, the Commission made these changes to make the minimum qualifications more clear and consistent after some applicants raised concerns. The Commission altered We interviewed relevant District staff and reviewed We were unable to fully substantiate this allegation. Although qualification requirements, documents to identify any instances in fiscal we found instances in which the Commission changed which eliminated years 2017–18 through 2019–20 in which the classifications’ educational requirements and eliminated the salary differentials. The Commission altered educational requirements that salary differentials for some employees as a result, we found Commission then required resulted in the elimination of salary differentials, no instances in which the Commission required employees to employees to repay their and we assessed whether these changes were repay their salary differentials during the period we reviewed, salary differentials for reasonable. We requested documents from the and we found the changes to the educational requirements to past periods. District for any instances in which individuals in be reasonable. these classifications were required to repay funds. The Commission sent We reviewed a selection of 20 rejection notices We were unable to substantiate this allegation. All 20 rejection notices that stating that applicants were debarred that the of the rejection notices we reviewed included the did not disclose required Commission sent from fiscal years 2017–18 through required information. information, including 2019–20 and determined whether they included the the right to appeal. three elements that the Commission’s rules require: Based on the documentation provided in support of the 1. The reason for the rejection. allegation, we concluded that the individuals making this allegation misunderstood the Commission’s rules. Specifically, 2. The length of time the applicant is ineligible what they believed to be rejection notices were actually for District jobs. disqualification notices.* The Commission’s rules do not require 3. Information about the right to appeal specific information to be included in disqualification notices. the rejection. Because a member of To determine whether the Commission canceled We were unable to substantiate this allegation. We did find the Commission staff examinations or eligibility lists, we reviewed reports multiple classifications with more than one eligibility list or disliked individuals of canceled job examinations, and eligibility lists examination within a 12-month period. In some instances, the on certain eligibility from fiscal years 2017–18 through 2019–20 to District did not hire individuals on these new lists. For example, lists, the Commission identify the number of examinations conducted for some eligibility lists there were an insufficient number of canceled those eligibility multiple times within 12 months. We then reviewed qualified candidates, and on others eligible candidates declined lists or examinations documentation and interviewed staff to identify the job offers. We also found that the District hires for several and conducted new reason for generating multiple eligibility lists and entry-level classifications on a regular basis and thus has a examinations. conducting multiple examinations. legitimate need to generate multiple lists within 12 months. Thus, the multiple eligibility lists the Commission generated during the period we reviewed appear to be appropriate. continued on next page . . . 44 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 ALLEGATION METHOD USED TO ASSESS THE ALLEGATION FINDING The Commission We requested the “black book” from the We determined that the Commission does have a document maintained a “black Commission and reviewed relevant information that it describes as a “black book.” However, according to book” that contained from the Commission’s manual. the Commission’s manual, the black book contains a list of a list of names that applicants and candidates that the Commission has debarred, the Commission or denied, from participating in its examinations. We identified discriminated against some entries on the list that lacked information regarding during examinations. the reasons for and lengths of the individuals’ debarments. The Commission asserted that these records were outdated and the individuals were not removed from examinations. We determined that the Commission had established procedures requiring notifications to be sent to individuals who were removed from the examination process due to debarment. This notification provides a safeguard as it would allow such applicants an opportunity to dispute their removal from an examination if they had not been properly debarred or otherwise excluded from the process. Further, we reviewed the status of applicants in the Commission’s electronic job application and examination system and confirmed that the system indicated that the Commission informed all applicants of their application status or allowed them to progress to the next stage of the process. The Director stated that the Commission has since removed all records from the list that lack necessary details. Source: Interviews with Commission staff, Commission rules and policies, and Commission documentation. * A disqualification generally occurs when an applicant does not meet the minimum qualifications of the position for which they have applied. In contrast, a rejection generally indicates the applicant is ineligible to apply for any District job for a specific period of time. The Commission’s Appeal Process The Commission’s rules for appeals are consistent with those of other comparable organizations, and it adhered Who Decides Examination to these rules for processing various types of appeals and Eligibility Appeals during our audit period. As Figure 9 illustrates, appeals are distinct from complaints. The Commission’s appeals Candidates may appeal the results of any part of an examination. The first level of appeal is to the processes are substantially similar to those of the other examiner, and if the examiner denies the appeal, personnel commissions whose policies we reviewed, the candidate may appeal to the director. except that the Commission does not allow applicants Applicants, candidates, and eligible candidates may to appeal minimum qualification determinations. appeal rejections (eligibility appeal). The first level Candidates may appeal any part of an examination of appeal is to the director. and eligibility determination, and permanent classified If the director denies an examination or eligibility employees may appeal disciplinary actions to the appeal, the appellant may appeal to the Commission. The Commission’s rules establish different commissioners. requirements for different types of appeals. As the text box shows, applicants, candidates, and eligible Source: Commission rules. candidates for examination may appeal rejection determinations to the Commission director. CALIFORNIA STATE AUDITOR | Report 2020-111 45 May 2021 Figure 9 The Commission Has Established Different Requirements and Processes for Managing Complaints and Appeals Exam & Disciplinary Complaints Eligibility Appeals Appeals REQUIREMENTS None Must be made in writing Must be written and and made within 5 working employee has 14 days days of receipt of exam or from receipt to file appeal. eligibility results. Must be based on Must contain specific procedural error, grounds, generally based on protected characteristic, procedural error, abuse of abuse of discretion, or discretion, unlawful an action taken that was discrimination, or violation not in accordance with of law or Commission rules. the facts. IS THERE A SET PROCESS TO FOLLOW? No Yes Yes WHO CAN SUBMIT? Members of the public, Current and potential Current and former including current and employees permanent employees potential employees WHO MAKES THE DECISION ON THE OUTCOME?* The entity responsible for Examiner, director, or Commission addressing a complaint Commission varies based on the nature of the complaint and the entity to which the individual submitted the complaint (Commission, District, union, or outside entity)† Source: Commission rules, staff union agreement, and staff interviews. * Commission decisions may be appealed to the Superior Court. According to the assistant director, no appellants have further appealed to the Superior Court in the more than 20 years she has been with the Commission. † For example, according to the assistant director, she or the director would generally handle a complaint made during the public comment portion of a Commission meeting. BACHELORS DEGREE MASTERS DEGREE 46 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 We reviewed a selection of 10 eligibility and examination appeals from fiscal years 2016–17 through 2018–19 and found that the Commission generally followed those of its rules that we assessed and appropriately addressed the appeals. Table 3 shows the outcomes of those appeals. Although the former director reversed an original decision to disqualify one applicant who appealed and the commissioners reversed the decision for a second, we did not identify any procedural errors that led to the initial rejections, such as Commission staff incorrectly assessing the applicant’s information. In both cases, the applicants submitted additional information during the appeal process that negated or sufficiently addressed the reasons for the rejection. For example, the Commission rejected one applicant because a previous employer had dismissed her for cause. However, the applicant explained in her appeal that she had misunderstood and incorrectly answered the question that led the Commission staff to that conclusion, and she provided documentation proving that she was still employed in that position. Table 3 The Director and Commission Upheld the Original Decision in Most Appeals We Reviewed OUTCOME UPHELD REVERSED TOTAL DECISION DECISION Exam Appeals to Director 3 3 0 Further Appealed to Commission 0 – – Eligibility Appeals to Director 5 4 1 Further Appealed to Commission 2 2 0 Eligibility Appealed Directly to Commission 2 1 1 Source: Analysis of 10 appeals the Commission received. The Commission also appropriately handled the disciplinary appeals we reviewed. Certain employees may appeal suspensions, demotions, and dismissals to the Commission; however, the Commission delegates disciplinary appeals to a third party. Specifically, for these appeals, its rules allow the Commission to authorize hearing officers—which are independent contractors, according to the assistant director—to conduct hearings or investigations. The hearing officer submits a report with conclusions and recommendations to the Commission, and the commissioners vote to accept or reject the recommendation. We identified four disciplinary appeals during fiscal years 2016–17 CALIFORNIA STATE AUDITOR | Report 2020-111 47 May 2021 through 2018–19 that the Commission decided, and determined that in each case it followed the specific rules and procedures for addressing disciplinary appeals that we reviewed. In each of the four appeals, the hearing officer recommended upholding the disciplinary action and the commissioners voted to adopt the recommendation. Although the Commission generally failed to meet the timelines described in its rules for investigating these cases and holding hearings, according to the assistant director these delays were caused by, among other things, a scarcity of hearing officers. Oversight of the Commission’s Budget and Expenditures The District has sufficient safeguards over the Commission’s budgets and expenditures. Although the Commission states that it is an independent entity, the District does process and monitor the Commission’s expenses. District rules state that the expenses of the Commission are to be paid out of the general funds of the District. For fiscal year 2018–19, the Commission’s budgeted expenditures totaled $2.2 million, of which it spent about 92 percent on employee salaries and benefits. In total, the Commission’s budget for fiscal year 2018–19 was approximately .04 percent of the District’s $5.7 billion budget. The District can provide input on the Commission’s budgets. According to the District’s director of budget and management analysis (budget director), the budget director and the District’s chief financial officer meet annually with the Commission’s director and assistant director to discuss the Commission’s proposed budget. State law requires the Commission to prepare a budget for a public hearing to be held no later than May 30 each year to which the Commission must invite the Board and District administration representatives to present their views. We found that the Commission generally adhered to this requirement. During the meetings at which it adopts its budget, the Commission schedules time to receive comments from the Board, the District’s chancellor, the staff union, and the public. After the Commission adopts its budget, the Commission then forwards its proposed budget to the county superintendent of schools (county superintendent) who may approve or reject it. During this approval process, the Board informs the county superintendent whether it concurs with the Commission’s proposed budget. The District also monitors and reviews the Commission’s expenditures throughout the year. Although the Board’s rules allow the Commission sole direction over expenditure of the funds appropriated for its operation, according to the District’s chief financial officer, the District treats the Commission in the same 48 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 manner in which it treats other District departments. The budget director explained that the District monitors the Commission’s expenditures through the District’s financial system, which allows it to ensure that the Commission does not spend funds in excess of its budget. The District processes the Commission’s payroll, and the District’s accounting manager for accounts payable and disbursements (accounting manager) explained that the accounts payable staff review and process payments for the Commission’s purchase orders, invoices, and receipts. The accounting manager stated that if the accounts payable staff identify a concern with a Commission expenditure, they follow up with the Commission to determine whether the expenditure was approved. If the Commission is unable to alleviate the concern, the staff refers the issue to the District’s internal auditor for additional review. The District’s rules also require the Commission to submit monthly financial reports to the chancellor as of the last day of each month showing the current status of the Commission’s expenditures in relation to its budget. Impact of the Commission’s Decisions on the District’s Budget The Commission has chosen not to analyze the financial impact of its decisions because the Board has final approval over them. As the Introduction describes, some of the Commission’s decisions relate to employee salaries and thus affect the District’s budget. Specifically, state law requires the Commission to recommend salaries for the District’s classified employees to the Board, which has the authority to approve, amend, or reject the Commission’s recommendations. The chancellor believes that recommendations to classify and reclassify positions are the Commission decisions that have the largest impact on the District’s budget. However, the Commission’s chair explained that he believes that because the Board has the responsibility to implement the Commission’s recommendations to alter salaries, the Board is also responsible for determining the impact of those changes on the District’s budget. Because the Commission classifies and reclassifies employees, which includes the preparation of job descriptions and recommendation of salaries to the Board, delegating consideration of the fiscal impact of its recommendations to the District helps increase its impartiality. The District’s chancellor explained that although the Commission’s decisions related to reclassifications and salaries have a fiscal impact on the District, the District believes that those costs are necessary to ensure that it retains its employees and pays them appropriately and fairly. The budget director could not recall CALIFORNIA STATE AUDITOR | Report 2020-111 49 May 2021 any decisions that resulted in excessive or unexpected costs for the District, and the chancellor stated that he does not have any significant concerns about the Commission’s decisions having a negative impact on the District’s budget. Commission Turnover and Work Environment District employees and former Commission employees described concerns about the Commission’s culture and stated that they feared retaliation by the former director. This culture appears to have contributed to the Commission’s turnover during the last five years. Specifically, from fiscal years 2014–15 through 2019–20, 10 staff members—including the former director—left the Commission. Based on the number of Commission employees during those years, its annual turnover rate averaged about 18 percent. We attempted to determine why these employees left the Commission; however, the assistant director stated that the District does not conduct exit interviews. Former Commission employees we spoke with stated that the behavior of the former director influenced their decisions to leave the Commission. Both current District and former Commission employees stated that they feared retaliation or retribution from the former director if they questioned Commission practices. Some District employees explained that they feared that if they spoke out against her the former director would remove their job position through a classification study. Former Commission employees also stated the former director did not allow them to speak with the commissioners. The former director retired in December 2019 and the Commission hired a new director in April 2020. In addition, as the Introduction describes, the process for selecting commissioners has changed, and a new commissioner was appointed in 2020 for the first time in 11 years. However, because we finished performing procedures for this audit in March 2021, it was not possible to determine the impact of these changes on the Commission’s culture and practices. 50 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 We conducted this performance audit in accordance with generally accepted government auditing standards and under the authority vested in the California State Auditor by Government Code sections 8543 et seq. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on the audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, ELAINE M. HOWLE, CPA California State Auditor May 6, 2021 CALIFORNIA STATE AUDITOR | Report 2020-111 51 May 2021 APPENDIX Scope and Methodology The Audit Committee directed the California State Auditor’s Office to conduct an audit of the Commission’s decision-making practices and its implementation of the merit system. The audit scope included nine objectives. The table below lists the objectives that the Audit Committee approved and the methods we used to address them. Audit Objectives and the Methods Used to Address Them AUDIT OBJECTIVE METHOD 1 Review and evaluate the laws, rules, and Reviewed relevant laws, regulations, Commission rules, a collective bargaining agreement, regulations significant to the audit objectives. and other background materials applicable to the Commission and the District. 2 Determine whether the Commission has • To identify best practices, reviewed relevant rules and other documentation and adhered to fair decision-making practices that interviewed representatives from three human resources organizations that perform are consistently applied in its processes used to similar functions: the State Personnel Board, the State Center, and the Los Angeles make personnel decisions. Unified School District Personnel Commission (LAUSD Commission). • Compared the Commission’s rules for making personnel decisions related to issues such as classifying and disciplining employees to the best practices that we identified to identify potential changes the Commission could make to better ensure the fairness of its decisions. • Judgmentally selected and reviewed 20 decisions that the Commission made from fiscal years 2016–17 through 2018–19. We examined reclassifications, out-of-class claims, and disciplinary appeals to determine whether the Commission made the decisions in compliance with its rules. • Because the Commission does not comprehensively track all complaints, reviewed its speaker logs for Commission meetings from fiscal years 2016–17 through 2018–19 to identify complaints made in those meetings. • Identified four complaints from the speaker logs and 21 email complaints, and reviewed the related documentation to determine whether the Commission handled these complaints adequately. • Judgmentally selected and reviewed 12 Commission meetings from fiscal years 2016–17 through 2018–19 to determine whether the Commission documented each non-agenda speaker during those meetings. 3 Review the accountability measures that the • Interviewed Commission staff and assessed the Commission’s accountability measures Commission uses and determine whether they to determine if it can ensure that it makes personnel decisions fairly and consistently. are sufficient for a public hiring agency. • Interviewed representatives from the State Personnel Board, State Center Commission, and LAUSD Commission to identify any accountability measures they use to assess performance that may be appropriate for the Commission to consider adopting. continued on next page . . . 52 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 AUDIT OBJECTIVE METHOD 4 Review the Commission’s candidate application • Reviewed lists of the examination and eligibility appeals the Commission received and examination process and determine and examinations it conducted from fiscal years 2016–17 through 2018–19. From whether it is fair, relevant, and structured in a these lists, we judgmentally selected and reviewed five examinations for which way that allows employees equal opportunities at least one applicant filed an examination or eligibility appeal. We selected 10 to promote. applicants per examination—five that the Commission determined were eligible to take the examination and five that the Commission determined were not eligible— for a total of 50 applications. We also selected an additional 94 applications that the Commission determined were not eligible—for a total of 144 applications of all types—to better assess whether the Commission appropriately disqualified applicants. We describe this work in more detail in Objective 9. • Attempted to review additional records concerning job examinations but were unable to do so because of the circumstances described in the section below titled Scope Limitation. • Evaluated key steps in the Commission’s examination application process, such as screening for minimum qualifications, to determine whether the process is fair, consistent, and structured in a way that allows the Commission to select and advance employees on the basis of merit after fair and open competition. We determined the Commission did not verify high school transcripts. As a result, the Commission did not determine whether 13 candidates met minimum qualifications. However, we verified the Commission required applicants to demonstrate full-time, paid work experience relevant to the job for which the examination was being held. • Evaluated key steps in the Commission’s examination process, such as conducting examinations and creating eligibility lists, to determine whether that process is fair, consistent, and structured in a way that allows it to select and advance employees on the basis of merit after fair and open competition. • Assessed whether the Commission’s implementation of its application and examination processes was fair by determining whether it adhered to key rules and steps in its processes. • Collected and reviewed best practices related to the application and examination process from the State Personnel Board, State Center Commission, and LAUSD Commission to identify potential opportunities for the Commission to improve its processes. 5 Review the Commission’s merit system rules • Compared the relevant state law and the Commission’s rules related to and determine whether they are consistent with reclassifications, examinations, out-of-class claims, appeals, and discipline for state law and the applicable memorandums classified employees. of understanding. • Assessed the Commission’s rules to determine whether they are consistent with the bargaining agreement between the staff union—which represents a majority of classified staff—and the District. 6 Determine whether there are sufficient • Interviewed a commissioner, the assistant director, and District staff to assess management controls for the Commission’s the District’s oversight of the Commission’s budget and whether it is sufficient to budgets and expenditures. minimize the risk of fraud, waste, and abuse related to the Commission’s budget and its expenditures. • Reviewed budget documentation from the District and Commission from fiscal years 2016–17 through 2018–19 to determine the effect of the Commission’s expenditures on the District’s budget. 7 Identify the Commission’s process for • Interviewed a commissioner and the assistant director to determine the extent determining and considering how its decisions to which the Commission has a process to assess how its decisions affect the affect the District’s budget. District’s budget. We interviewed District staff to obtain their perspectives on the Commission’s decisions and their effect on the District’s budget. • Reviewed documentation that demonstrates how the Commission’s decisions affect the District’s budget. CALIFORNIA STATE AUDITOR | Report 2020-111 53 May 2021 AUDIT OBJECTIVE METHOD 8 Review the Commission’s appeal process and • Reviewed the Commission’s rules for reviewing and processing appeals. determine whether it adequately considers • Reviewed rules and other documentation related to appeals and grievances from the employee grievances. State Center Commission, LAUSD Commission, the State Personnel Board, and the California Department of Human Resources. Compared these to the Commission’s rules to identify potential opportunities for the Commission to better ensure that it adequately considers employee appeals and grievances. • Judgmentally selected 10 appeals the Commission decided on from fiscal years 2016–17 through 2018–19, including some appeals related to the examinations we selected to review for Objective 4. • Evaluated the appeal decisions to determine whether the Commission followed its rules and time frame when addressing the appeals. Determined that the Commission made its decisions in a reasonable time frame. • Obtained and reviewed emails sent to and from the former director and assistant director from fiscal years 2016–17 through 2018–19 to determine whether Commission staff received complaints and addressed them appropriately. 9 Review and assess any other issues that are • After receiving allegations from several District employees about the Commission significant to the audit. using inconsistent practices and unfairly disqualifying applicants from examinations, selected 10 additional examinations posted from July 2017 through September 2020, and up to 10 disqualified applicants for each, for a total of 94 applicants. We assessed whether the Commission appropriately disqualified these applicants and used consistent practices when doing so. We also selected 20 individuals whom the Commission debarred during the same time period and reviewed relevant documentation and obtained Commission perspective to assess whether the Commission communicated required information and timely informed these individuals. For 10 debarments, we also assessed whether the Commission used a consistent approach when deciding to debar the individual. We performed various procedures to address the other allegations we received, as Table 2 outlines. • Judgmentally selected four Commission staff based on their length of employment at the Commission and their job positions and interviewed them about the Commission’s work environment. • Reviewed the Commission’s organization charts for fiscal years 2014–15 through 2019–20 to assess the extent of its staff turnover. Source: Analysis of the Audit Committee’s audit request number 2020-111 and audit workpapers. Scope Limitation To review the Commission’s candidate application and examination process as Objective 4 requires, we initially selected five examinations and obtained 10 applications associated with each of the examinations. We attempted to obtain an additional selection of the same number of examinations and associated applications; however, the Commission’s director stated that, because of the COVID-19 pandemic, Commission staff were unable to access the Commission’s physical location and hard copy records. As a result, our review was limited to the 50 applications we had initially obtained rather than the 100 applications we planned to review. After further discussion with Commission staff, we obtained remote access to its application and examination system and were 54 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 able to review an additional 94 applications. However, because this system only contains documentation from fiscal year 2017–18 and later, and it does not contain certain documentation—such as rating sheets—it was not possible to fully perform our intended review. For this reason, our review of the additional applications was limited to assessing the Commission’s screening of applications for minimum qualifications. Although this limitation affected the scope of our review, we obtained sufficient evidence in total to support the findings and conclusions we present in this report. Assessment of Data Reliability The U.S. Government Accountability Office, whose standards we are statutorily obligated to follow, requires us to assess the sufficiency and appropriateness of computer-processed information we use to support our findings, conclusions, or recommendations. In performing this audit, we relied on the following data and systems: Examinations We relied on summaries of the Commission’s eligibility lists to identify the number of examinations the Commission offered multiple times within a 12-month period and to make a selection of those examinations for further review. Because we used these data solely for selecting examinations, we reviewed key elements to ensure that they contained logical data and performed completeness testing and data-set verification procedures. We did not identify any issues. To verify the completeness of these summaries, we compared the total number of examinations the Commission reported in its annual report for each fiscal year of the audit period to the total number of examinations on the summaries. We determined that these data were sufficiently complete for the purpose of selecting items for further review. Public Complaints We used the Commission’s public meeting speaker log to identify the number and nature of complaints it received from fiscal years 2016–17 through 2018–19. To assess the completeness of these data, we judgmentally selected 12 Commission meetings that occurred during those fiscal years and reviewed the meeting minutes to determine whether each speaker from the meeting whose comment required Commission action or follow-up was described in the log. We determined that the Commission included CALIFORNIA STATE AUDITOR | Report 2020-111 55 May 2021 all relevant speakers from these meetings in its log and that these data were sufficiently reliable for the purpose of identifying non-agenda speaker complaints from Commission meetings. Appeals We relied on a list of the appeals the Commission received from fiscal years 2016–17 through 2018–19 to select appeal cases and examinations for further review. The Commission maintains hard copy appeal files, but due to the COVID-19 pandemic, we were unable to haphazardly select items from those files to compare to the list. Therefore, to assess the completeness of this list, we compiled a list of appeals from our review of certain employee emails and determined whether the appellant described in the email was listed on the appeals logs. We found that the appeals list was incomplete, as one of the seven appeals we identified in emails was not listed on the appeals log. According to the assistant director, this was due to a clerical error. Although the population of appeals from which we selected items was incomplete, this list was the best source of such data available and there is sufficient evidence in total to support our findings and recommendations. Commission’s Application and Examination System We used data from the Commission’s application and examination system to make a selection of rejected applicants, candidates, and eligible candidates for further review. We also obtained documents from this system related to applications and examinations. Due to the COVID-19 pandemic, we were unable to obtain source documents and could not perform data reliability testing on this system; therefore, the data are of undetermined reliability. However, there is sufficient evidence in total to support our findings and recommendations. Number of District Employees We used data from the California Community Colleges Chancellor’s Office’s management information systems data mart to determine the District’s total number of employees, number of classified employees, and number of enrolled students. Because we used these data solely for background or contextual information that does not materially affect findings, conclusions, or recommendations, we determined that a data reliability assessment was not necessary. 56 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2020-111 57 May 2021 LOS ANGELES COMMUNITY COLLEGE DISTRICT ____________________________________________________________________________________ 770 Wilshire Boulevard Los Angeles, California 90017 PersComm@Laccd.edu PERSONNEL COMMISSION April 14, 2021 Members of the Commission DAVID H. IWATA DIVA SANCHEZ TREVINO Personnel Director Elaine Howle * RONALD DELAHOUSSAYE California State Auditor 621 Capitol Mall, STE 1200 Sacramento, CA 95814 RE: Response to State Auditor’s Report Pertaining to the Los Angel es Com m u n ity 1 Personnel Director College District’s Personnel Commission RONALD DELAHOUSSAYE Dear Ms. Howle: Thank you for giving us the opportunity to respond to the audit report prepared by your office and provided to the Personnel Commission on April 7, 2021. Although we appreciate the time the State’s audit team spent on preparing this report and some of the suggestions made to improve processes, we do not believe that your audit staff adequately absorbed how a merit system-based Personnel Commission functions in the State of California or operates in the true spirit of the law. First, we would like to note that we believe the title of the report inadequately portrays our policies and practices and makes unwarranted conclusions based on the evidence. The Personnel Commission acknowledges that there are opportunities for change and improvements that can allow us to better service District employees and job candidates. We don’t believe it is accurate to take a few cases where mistakes may have been made or where improvements may be needed and conclude that our processes and policies are inconsistent. The Personnel Commission makes every reasonable effort to apply practices and policies consistently and fairly and do not believe the title of this report accurately reflects reality. Furthermore, we do not agree with the methodology your staff applied in their analysis of the data they collected over the course of a 12-month period. All conclusions drawn by the audit team in this report are based on data that represents only a very small percentage of cases and do not fairly represent the processes that work for the overwhelming majority of employees and job applicants served by the Personnel Commission. The Personnel Commission staff diligently and promptly provided detailed information and explanations for all areas reviewed or questioned by the audit team over the course of the audit. Yet, in the conclusions drawn in this report, it became apparent that in many instances this information and our explanations were not considered or accepted by your audit team even when supported by sound and credible evidence. Preliminary conclusions reached by your audit team remained on their list of final findings even after we provided credible evidence that refuted or clarified them. This led to the portrayal of the functions fulfilled and quality of service provided by the Personnel Commission of the Los Angeles Community College District that is neither accurate nor supported by solid and undisputable evidence. Below you will find the Personnel Commission of the Los Angeles Community College District’s specific responses to the three conclusions noted in the California State Auditor’s 1 * California State Auditor’s comments begin on page 81. 58 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 report and its subsequent recommendations as well as to the other areas that were reviewed as part of the audit and its recommendations. State Auditor’s Conclusion 1 Regarding the Screening of Job Applications and Oral and 2 Performance Examination Processes The Personnel Commission disagrees with the conclusion noted for this area and contends that it is not warranted given the evidence provided. With respect to the State auditor’s conclusion related to the screening of job applications, the Personnel Commission notes that the State auditors cited as supporting evidence that they found concerns with 5 out of 119 disqualified job applications they reviewed. This represents a percentage of less than 5% of all applications they reviewed. The State auditors acknowledged that the majority of applications were screened appropriately by the Personnel Commission. The State auditors provided the Personnel Commission with the names of those 5 applicants whose applications they had concerns with and the Personnel Commission provided detailed explanations as to why those applicants were disqualified. It included a combination of cases where clerical employees claimed to have performed higher level duties outside the scope of their job classifications, work experiences were described inaccurately with the intent to make them look qualifying, work experiences were not closely aligning with the minimum requirements, etc. The State auditors also stated that it appears that the “Commission made some qualification decisions based on internal applicants’ District job titles, rather than the experience they described in their applications. The Personnel Commission holds that its examiners decisions on whether or not to qualify an applicant are methodical, evidence driven, and based on our examiners’ shared understanding of the minimum entrance qualifications established for positions as well as their knowledge of the job classifications that exist in the classified service. The job history of applicants undergoes a comprehensive review by which the duties of all positions are carefully reviewed, additional research on an employer may be conducted to fully understand the level of an applicant’s position within the organization, and the final assessment of experiences is not based on job titles listed in the job application. Furthermore, all examiners undergo extensive and closely supervised on-the-job training covering all exam processes. Minimum entrance qualifications are carefully discussed with the supervisor and peer examiners and instructions are provided on how to interpret them. All examiners are required to follow the steps outlined in the JobAps applicant tracking system manual in the administration of their examinations, which includes definition of the terms “professional-level” and “recent”, which are commonly used terms in the minimum entrance qualifications of job descriptions used by public agencies at large, including the State Auditor’s Office. The Personnel Commission’s examiners make great efforts to ensure that the standards for the minimum qualifications are applied consistently and justly to all applicants. If clarification on specialized job classification qualifications is needed, subject matter experts are also consulted by the examiners. Contacts with employee supervisors are also made if the duties listed by an employee on the application represent duties that are inconsistent with the class concept of their current job classification and therefore require follow up and validation. The Personnel Commission receives well over ten thousand applications every year and, while an error may occasionally be made in a decision, such errors are rare and whenever they are discovered they are promptly corrected. The State auditors cite a few examples, some of which we would dispute are errors at all, but nonetheless, they represent a very small sample of applicants and fail to demonstrate that this is a consistent problem in our screening process. For the State auditors to say that we are inconsistent and unjustified in our qualification decisions 2 CALIFORNIA STATE AUDITOR | Report 2020-111 59 May 2021 is undeservedly harsh and does not accurately portray the Commission’s effort to ensure a fair and consistent process for all. With respect to the State Auditor’s conclusion related to oral and performance examination processes, which were the only examinations reported on in their conclusion, the Personnel Commission notes the following. The State auditors cited in their report that raters provided minimal or no justification to explain the scores they had assigned to 19 of 25 qualified candidates they reviewed. The Commission has processes in place wherein we provide detailed instructions and guidance to raters to ensure that fair scores are provided. Additionally, raters who have a close relationship with or have supervised a candidate are not permitted to rate that candidate which further helps to ensure a fair process. While, auditors did note a few instances where a rater provided a final score that was incongruent with a candidate’s individual factor ratings, these instances almost always result from a situation when a rater updated their final score based on discussion with other raters, but neglected to go back and update individual factor ratings corresponding to the change in final score. Additionally, the issues cited by the auditors represent raters’ scores that are averaged with at least one or two other raters’ scores thus offsetting any differences that may exist between raters. In many cases, there are multiple exam parts that comprise a score, thus lowering the impact on the final score even further. However, as we will note later differences between raters’ final scores are minimal and well within acceptable levels prescribed by psychometric experts. To conclude that this clerical oversight proves bias or unjustified scoring is unfair and exaggerates the issues cited by the State auditors. It is also important to note that despite the incongruency between individual factor scores and final score, the exams in question display a high-level interrater reliability/agreement. This level of agreement underscores the fact that the final ratings were closely agreed upon by all raters, which would suggest that these scores are consistent, justified, and impartial. Ultimately, it is this final rating that determines a candidate’s exam score. The individual factors ratings, while in some rare instances are incongruent with the final score, do not directly factor into the calculation of the final score. Therefore, inconsistencies between factor ratings and final ratings do not mean that a candidate’s score ought to have been different. In fact, the high levels of interrater reliability/agreement for the final scores for these exams indicates that raters had high levels of agreement on both the final scores and the rank ordering of candidates. Thus, in these cases it is unlikely that these incongruencies affected whether or not the candidate placed in the top 3 ranks. The Personnel Commission would also like to note that only some of 9 examples cited by the auditors actually represent a clerical oversight on our part. Others reflect factors that rater’s might have weighted more heavily because they are more important (e.g., Training & Experience). Given the uneven weighting, one cannot look simply at the number of each type of ratings a candidate received (e.g., # of weak, # of strong, etc.) and the relative weighting given by the raters must be considered to gain a full understanding of a candidate’s score. Furthermore, it is possible that there are slight variations in candidates’ skills within a rating category that are being factored into a candidate’s final score. For example, it could be the case that some candidates in the “Strong” category for a given factor may be slightly stronger than others and rater’s may choose to account for such nuances when assigning their final scores (as may be the case in comparing “Candidate A” & “Candidate B” in Figure 4 in the audit report). The Personnel Commission would also like to note that the 5 exams reviewed by the auditors included 327 ratings sheets for either performance or oral exams and the 9 examples cited represent only 2.8% of the rating sheets in these exams and therefore, these issues are outliers. The Personnel Commission’s low exam appeal rate further supports our assertion that our exams are fair. The percentage of candidates who participate in an examination and appeal their examination results in any given year amounts to less than 1% of all candidates. This highlights that candidates likely perceive this process as fair, otherwise we 3 60 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 would see greater number of candidates take advantage of the formal appeal process that is being offered to them through a well-publicized due process. Additionally, we would like to note that while raters may have not provided written comments on their rating sheets for some passing candidates to say their ratings are unjustified is misleading. The ratings a rater provides on a rating sheet in the vast majority of cases are in fact justified by the rating they provided on the individual factors and the final score categories are defined. Finally, this conclusion should reflect the fact that the auditor’s conclusion only refers to oral exams and one performance exam and not all exams conducted by the Commission. There are a multitude of other tests administered by the Commission such as standardized written tests, standardized computer software tests, and training and experience evaluations. The State auditors review did not yield any conclusions regarding our other test types. In conclusion, the Commission recognizes that our processes will always have room for improvement as technology develops and knowledge grows and that in a very few instances human error can occur, but the conclusions drawn by the State auditor harshly overstate the problems making them appear more widespread than they really are. The Personnel Commission would also like to note that it always strives to improve current processes and develop new methods and processes for the evaluation of candidates’ qualifications in our classified examination process. Staff frequently exchange information regarding testing methods and practices with other merit system agencies in an effort to stay current and innovative and attends professional conferences and presentations on topics related to employee recruitment and selection. Since the beginning of this year the Personnel Commission has been working on modifications to our interview rating sheet and has begun to pilot test this rating sheet in exams. These modifications have been prompted by two things. First, over the past year analysts were forced to begin doing exams virtually due to the Covid- 19 pandemic and this served as an opportunity to re-think our methods and how to adapt them to a virtual environment. As part of these changes, we have already begun to improve various methods to better align them with best practices as part of these improvements we have begun to implement. Second, comments from the State auditors have underscored a few areas for improvement which the commission has taken under consideration in this effort. Below is the summary of the changes made to the oral interview rating sheet and explanations for how it addresses issues raised by the State auditors. In addition to factor definitions which already exist, the Personnel Commission will add a definition for each point on our rating scale that is unique to each factor. These definitions will further ensure that raters are using same criteria to assign ratings and should further enhance inter-rater agreement. Finally, this will provide more specific guidance to raters when compared to the former rating sheet. The Personnel Commission will also be removing the final rating that is independent of the factors scores. Now raters’ scores will be calculated by summing all factor rating scores. More important factors will be assigned a heavier weight than factors that are less important. This will make the appearances of inconsistency between factor scores and total scores impossible. The new rating sheet will have a mandatory field for “Comments” where raters are required to provide written comments about a candidate’s overall strength and weaknesses. This will apply to all candidates regardless of whether they pass or fail the examination. 4 CALIFORNIA STATE AUDITOR | Report 2020-111 61 May 2021 Taken together these changes should significantly improve our rating sheet and create a more transparent process. State Auditor’s Recommendations: 1)When possible, the Commission should create qualification requirements based on time spent working in the District job classifications or equivalent experience rather than ambiguous terms such as professional-level. 2)The Commission should establish a rule for its examiners by October 2021 that defines the key terms it uses when reviewing applications for minimum qualifications such as “professional-level” and “recent”. 3)The Commission should require examiners to provide disqualification notices that describe their reasons for disqualifying an applicant. 4) The Commission should require examiners to create detailed scoring benchmarks that provide raters guidance on how to rate individual evaluation factors. 5) The Commission should establish methods for determining candidates’ overall scores based on the ratings of the individual evaluation factors. 6) The Commission should require raters to provide written comments on rating sheets for each candidate, explaining the basis for the score they awarded. 7) The Commission should require examiners to review scoring sheets to determine if raters follow the Commission’s candidate evaluation guidance, and if the raters fail to follow the guidance requests that they review their evaluation of the candidate. 8) The Commission should revise its rules by October 2021 to implement an appeal process for applicants that it has determined do not meet minimum qualifications for the position. Personnel Commission’s Response to State Auditor’s Recommendations: 1) The Personnel Commission contends that there is already a sound process in place for how class descriptions, including class qualifications and minimum requirements, are developed. All job descriptions developed and maintained by the Personnel Commission are based on solid research by Personnel Commission staff. Information is gathered on comparable jobs from the U.S, Department of Labor data base, other public agencies, and internally related job classes within the Los Angeles Community College District. Position duties are prescribed by the governing board and minimum qualification requirements that reasonably relate to the duties are prepared and approved by the Personnel Commission (see Personnel Commission Rule 522). Furthermore, all class specification reviews follow a standardized process that is outlined in the Revision Flow Chart for class specifications which is publicly posted on the Personnel Commission website. All constituent groups are afforded an opportunity to participate in this process. This includes administrators, supervisors, incumbents, and collective bargaining unit representatives. 2) Auditors stated that Personnel Commission does not provide definitions for key terms included on our job specifications (e.g., professional). This is not accurate, while these definitions are not part of any public materials, they are a part of on-the-job training that all examiners in our department receive and therefore, examiners all have a shared understanding of the terms used on our job specs. This shared understanding allows all examiners, to a reasonable degree, to come to the same conclusions about the qualifications of similar/same applicants in the overwhelming majority of cases. Additionally, key terms are now formally defined in the JobAps examiner manual and all examiners are required to follow the instructions provided in this manual in their examination administrations. Human error may occur, because of the sheer volume of applications we review, but these are rare exceptions and when they occur, they are corrected and explained to applicants. 5 62 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 3)The Personnel Commission accepts this recommendation and will amend the disqualification notification provided to applicants to include the basic reason(s) why an application was not found qualified in an effort to enhance transparency. 4) The Personnel Commission accepts this recommendation and improvements to rating sheets that are already underway will provide specific and detailed definition for each rating on the rating sheet. These definitions will further ensure that raters are using same criteria to assign ratings and should enhance inter-rater agreement. Finally, this will provide more specific guidance to raters when compared to our former rating sheet. 5) The Personnel Commission accepts this recommendation that candidates’ overall scores will be calculated by summing all individual factor rating scores. More important factors will be assigned a heavier weight than factors that are less important. This will make the appearance of inconsistency between factor ratings and total scores impossible. 6) The Commission accepts this recommendation. The new rating sheet will have a mandatory field for “Comments” where raters are required to provide written comments about a candidate’s strength and weaknesses. This will apply to all candidates regardless of whether they pass or fail the examination. Examiners will review rating sheets to ensure that comments are sufficient. 7) The Personnel Commission already has a process in place that was implemented this year by which multiple examiners review ratings sheets to ensure that the Commission’s evaluation guidance is being followed. 8) The Personnel Commission notes that the applicant tracking system (JobAps) that was implemented in 2017 provides detailed instructions to applicants on how to fill out a job application. Those instructions specifically advise applicants to provide complete and accurate information that substantiates that they meet the minimum entrance qualifications noted on the job announcement. However, in light of the fact that some applicants, including promotional candidates, have difficulty following these instructions, the Personnel Commission agrees to provide an applicant who is rejected for not meeting the minimum entrance qualifications one opportunity to provide supplemental information, documentation, or evidence necessary to meet the entrance qualifications. A rule amendment to Rule 600 reflecting this new process is scheduled to be placed on the Personnel Commission meeting agenda in May of 2021. Excerpt from the Jobaps System: Instructions to Job Applicants: 6 CALIFORNIA STATE AUDITOR | Report 2020-111 63 May 2021 State Auditor’s Conclusion Regarding Compensation for Temporary Work-out-Class 3 Cases Personnel Commission’s Response to this Conclusion: First, the Personnel Commission would like to note that temporary work-out-of -class cases are not very common transactions in the Personnel Commission. There are typically less than 10 cases in any given year. With respect to the State auditor’s conclusion implying that work-out- class work is not compensated on a prompt basis, the Personnel Commission notes that, as discussed with the audit staff on numerous occasions, there are multiple factors that may cause delays in the processing of claims and a fair amount of them are not within the control of Personnel Commission staff. The variation of the time it takes to compensate employees for out-of-class work is not only contingent on the length of time an employee performs the work, but there are several other factors that add time to the processing of claims. These factors include incomplete/missing information on the claim form and the time it takes to receive the requested incomplete/missing information from the employee; time reports to verify and calculate warranted compensation as the longer the work out-of-class claim period the more time it takes to verify and calculate the warranted compensation; work logs/descriptions of work performed take longer to evaluate depending upon the complexity of the work as well as the length of the work out-of-class period; Subject Matter Experts may need to be consulted with to verify and evaluate work performed for specialized job areas. With respect to the processes in place for temporary work-out-class cases compensation, the Personnel Commission notes that it has compensated employees in a consistent and fair manner in accordance with the established applicable Personnel Commission Rule. Since the deadline for an employee to submit a claim form time limit changed from 100 days to 45 days last year on April 29, 2020, only one temporary work out of class case was processed and the employee did submit all of her claims in a timely manner within the deadline of 45 calendar days from the first date of working out of class. State Auditor’s Recommendations: 1)The Commission should immediately revise the Work-out-of Class Form to include an option for employees to request intermittent payments while performing out-of-class work assignments. 2)The Commission should revise its rules by October 2021 to process employees’ compensation for out-of-class work each month. 3)The Commission should amend its rules by October 2021 to allow employees at least 100 days to submit their out-of-class work claims before limiting their compensation. 4)The Commission should require employee to submit a copy of their out-of-class claim form at the same time they submit it to their supervisor, and use the date the Commission receives this copy of the form as the date of submission. Personnel Commission’s Response to State Auditor’s Recommendations: 1) The Personnel Commission accepts this recommendation. The Work-out-of-Class Claim Form will be amended to include an option for employees to request intermittent payments while performing out-of-class work assignments. 2) The Personnel Commission does not agree with the recommendation to automatically process compensation for work-out-class claims on a monthly basis. As stated consistently in interviews with auditing staff, this would not be considered an efficient approach and would increase the work for the Commission, the Board of Trustees, and the Payroll 7 64 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 Department to approve payments for work out of class on a monthly basis rather than waiting until the employee is no longer working out of class. This would be additional time for the Personnel Commission staff to prepare and filter time reports, preparation of the Personnel Commission and Board of Trustees reports, distribution of the Personnel Commission report to the employee, supervisor, administration, and union, and time spent on responding to questions/feedback received which may result in amendments to the temporary work out-of-class report. Likewise, this would also increase the workload of the Board of Trustees’ Office to place this on their agenda as well as the Payroll Department since this is a specialized payment in addition to the regular payroll that is run on a semi- monthly and monthly basis. Additionally, the employee and supervisory feedback survey data for work out-of-class cases routinely gathered by the Personnel Commission over the last four fiscal years indicated that the overwhelming majority of employees and supervisors are satisfied with our current process. This information was provided to the audit staff. However, in recognition of the fact that compensation for some work out-of-class cases have taken months due to aforementioned reasons that can cause delays, the Personnel Commission agrees to process claims intermittently if the employee makes the request on the Claim Form. 3)The Personnel Commission accepts the recommendation to amend the applicable Personnel Commission rule to allow employees at least 100 days to submit their out-of- class claim form in acknowledgement that there have been instances where District administrators did not forward claims to the Personnel Commission in a timely manner. This rule amendment is scheduled to be placed on the Personnel Commission meeting agenda in May of 2021. 4) The Personnel Commission does not agree with the recommendation to amend Personnel Commission rules to require employees to copy the Commission when submitting out-of-class claims to their supervisor and have the recognized date of submission be changed to the date that the employee submits his/her claim forms to their supervisor. The duly authorized supervisor must certify that the out-of-class work was assigned prior to processing any out-of-class claims since assignment of duties is within the purview of management and not the employee (see Ed Code 88095). Additionally, Personnel Commission rules already address situations where the untimely filing of a claim may be excused under certain circumstances and increasing the allowable days to submit a claim to 100 days will also facilitate in maximizing compensation for employees who do not submit out-of-class claims in a timely manner. 4 State Auditor Conclusion 3 Regarding Methods for Addressing Complaints Personnel Commission’s Response to this Conclusion: The Personnel Commission contends that this conclusion ignores how employee inquiries/complaints/grievances are addressed in a merit system environment. Furthermore, the Personnel Commission believes that the State auditors made unsound inferences based on the evidence cited below from their report: 1) “The Personnel Commission could not provide documentation that it appropriately addressed 5 out 21 emails that the auditors gathered for a three-year period (July 1, 2016 to June 30, 2019) from the email server of the Los Angeles Community District.” First, the Personnel Commission notes that it had provided evidence that it had responded to the majority of the emails. Second, 21 emails represent a very small number of cases in a three-year period, an average of 7 emails in a year, particularly in light of the fact that the Personnel Commission processes thousands of transactions in a year where 8 CALIFORNIA STATE AUDITOR | Report 2020-111 65 May 2021 employees or applicants could have raised a question or concern. Thirdly, 6 out of the 21 emails compiled by the auditors were emails addressing areas that were either outside the purview of the Personnel Commission’s authority to take action, covered a procedural question raised by an employee, provided general comments with no specifics that did not require further action, or where anonymously submitted and staff was not able to further research the matter and respond. Fourthly, the Personnel Commission explained to the audit staff that the retrieval of emails for documentation purposes had been problematic due to email migration issues when the District switched email servers in 2019, which prevented the Personnel Commission from securing documentation that would have otherwise been available to be downloaded. 2) “The Commission rules do not provide clear criteria for what it considers to be a complaint”. The Personnel Commission notes that employee issues in a merit system environment relate to formal appeals, grievances, or are tied to the outcome of of a formal study/case. Both appeal and grievance terms are formally defined in Rule 500 and specific rules have been developed that outline the processes. Additionally, there are processes in place where employees have multiple opportunities to provide feedback on studies/job descriptions, etc. that involve their positions, and are ultimately able to address the Personnel Commission in a public meeting before a final action is taken on their case. Furthermore, all case studies are shared with an employee’s collective bargaining representative to provide them an opportunity for input and to get their concerns addressed before any actions are taken by the Personnel Commission. Monthly union consultations take place with the Personnel Director where collective bargaining unit reps can bring up any employee issues that fall within the purview of the Personnel Commission and the Personnel Commission staff will aid in resolving these matters promptly. Personnel Commission Rule 513, outlines the processes for how requests for investigations of personnel problems related to alleged violations of the Merit System Laws or Personnel Commission Rules are to be handled. This rule also addresses how requests should be handled when the Personnel Director is implicated in an allegation. The applicable rule provision reads as follows: A.4.: “If the allegations in a request for investigation implicate the Personnel Director, the request shall be referred to the Personnel Commission who shall appoint an independent investigator to perform the duties of the Personnel Director in carrying out the provisions of Paragraph A1. Through 3..” Furthermore, processes for this area are formally prescribed in Personnel Commission Rules 600, 624, 735, and 893, which are publicly posted on the Commission’s website. Therefore, the illustration in the audit report of the Complaint Column in Figure 9 under the description of the Commission’s Appeal Process is also incorrect. A detailed description of the Personnel Commission’s processes relevant to this area was provided early on during the audit (see below) and was consistently reiterated by the Personnel Commission staff in the interview notes that were provided to the State auditor as well as substantiated by the numerous logs provided by the Personnel Commission as supporting evidence, i.e. non-agenda speaker log, employee outreach log, appeal log, employee satisfaction survey summaries, etc. It is the Personnel Commission’s position, that the “complaint process” is not a stand-alone isolated process and that the core business transactions of the Personnel Commission are not handled through an independent “complaint process” but through processes and procedures outlined in Personnel Commission Laws and Rules. By definition, it is a guiding principle for all Personnel Commissions. Each rule, policy, and procedure have avenues and outlets for all 9 66 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 stakeholders to complain, disagree, and provide feedback. The Personnel Commission staff processes for conducting exams, classification studies, and rule revisions include multiple opportunities for direct communication, consultation, and collaboration with impacted constituents. Ultimately any constituent, stakeholder, or member of the public can bring any inquiry/issue/complaint directly to the Personnel Commissioners for consideration in a public meeting or closed meeting session if appropriate. The Commission is beholden to the Brown Act and all decisions are publicly noticed with measures built in to allow those affected to have their concerns heard and addressed. Opportunities to provide feedback, including “complaints”, is incorporated into Personnel Commission Operations via several additional mechanisms including general and special area Personnel Commission email addresses (one for general inquiries; another for examination related inquiries); online feedback surveys, individual feedback surveys (for employee and supervisor experience that went through a class study or work out of class process); and new employee satisfaction surveys. INFORMATION SUMMARY ON HOW EMPLOYEE INQUIRIES/ISSUES ARE ADDRESSED (developed by the Personnel Commission and provided to audit team on May 19, 2020) CONTEXT STATEMENT Within the Los Angeles Community College District, there are several offices that handle employee-related inquiries/issues/ “complaints”, which includes Human Resources Operations, Employee and Labor Relations, Employee Benefits, Payroll, Diversity, Equity, and Inclusion, and the Personnel Commission. The Personnel Commission is frequently the first point of contact for employees, union representatives, and District administration, managers, and supervisors seeking information and answers regarding HR matters. This has happened in part for the following reasons: • Accessibility – We are available. Examples: Dedicated office staff that answers the phone during regular business hours; email addresses dedicated to general inquires and job and exam information that are attended to by staff on a daily basis; the Personnel Commission Field Representative/Outreach Program; direct, unencumbered access to the Personnel Commissioners at public meetings; our level of participation with administrative and union groups. • Expertise - We have qualified and broadly experienced staff both in technical HR/PC matters and in District-specific HR/PC operational/procedural matters. As such we can provide comprehensive answers and instructions to employees and job applicants. • Navigating the Organization: Often classified employees need support and information from the Personnel Commission as to which District Office department may be the appropriate office to contact to have their concerns or complaints addressed. The Commission staff regularly refers classified staff to our Human Resources counterparts when they have concerns about benefits, their assignment, hours, quotas, pay, or equity related issues. 10 CALIFORNIA STATE AUDITOR | Report 2020-111 67 May 2021 COMPLAINT MANAGEMENT During interviews conducted by the auditors there have been recurring questions regarding our “Complaint Process”. According to widely recognized industry standards, an effective complaint management process typically includes the following elements. • Policies and Procedures • Accessibility • Clear Channels of Communication • Record Keeping • Investigation/Analysis/Research Process • Response • Corrective Action, when necessary • Employee Education Questions/Inquiries/Complaints exist on a spectrum from minor, which are routine and simple to resolve, to serious allegations that require more formal attention and action. The Personnel Commission receives questions/inquiries, phone and email, from employees, supervisors, managers, administrators, and the public on a continuous basis. The vast majority of contacts are: requests for information; requests for explanation/clarification; status requests, and requests for assistance in resolving problems. These inquires/issues are addressed by Personnel Commission staff who are trained to provide direct technical support and assistance which most often resolves the issue immediately. Complaints, i.e. more serious objections/allegations that something is unfair, unacceptable, or otherwise not up to normal standards, are sporadic and do not happen frequently. At the employee/public level, they often stem from misinformation, misunderstanding, or disagreement with a decision/ transaction/outcome regarding the individual. From District representatives, they often stem from new, unexperienced, untrained supervisory personnel (high turnover). These “complaints” are addressed by Personnel Commission staff who are trained to provide direct technical support and assistance which most often resolves the issue immediately. If the “complaint” cannot be resolved at that level, the matter is referred to the Assistant Director or Director for resolution, sometimes through a rule-based formal appeal process. In situations where the matter involves a formal appeal related to an examination, disciplinary action, or a debarment from employment, the appeal process outlined in the applicable Personnel Commission rules 600, 624, or 735 as well as in the Personnel Commission procedural summaries posted on the website is followed. It is our position, that the “complaint process” of the Personnel Commission is not a stand- alone isolated process. By definition, it is a guiding principal for all Personnel Commissions. Each rule, policy, and procedure have avenues and outlets for all stakeholders to complain, disagree, and provide feedback. The Personnel Commission processes for conducting exams, classification studies, and rule revisions include multiple opportunities for direct communication, consultation, and collaboration with impacted constituents. Ultimately any constituent, stakeholder, or a member of the public can bring any inquiry/issue/complaint directly to the Personnel Commissioners for consideration in a public meeting or closed meeting session if appropriate. The Commission is beholden to the Brown Act and all decisions are publicly noticed with measures build-in to allow those affected to have their concerns heard and addressed. 11 68 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 The following segments illustrate the principals of the merit system. For each of our processes/procedures/practices, we make reference to how it relates back to the essential elements of effective complaint management. POLICIES AND PROCEDURES The functional areas of the Personnel Commission are: Job Classification, Compensation, Policy/Rule Development, Recruitment and Selection, Assignment Processing/Audit, and Disciplinary Appeals. Each functional area has prescribed rules, processes, procedures, forms, and other tools which identify for individuals, employees, supervisors, managers, administrators, and executives how “transactions/complaints” within the purview of the Personnel Commission will be addressed from start to finish. Personnel Commission Laws and Rules The Personnel Commission Rules identify policy, decision makers and their roles, processes, procedures, and forms to be used in pursing all studies and transactions under the purview of the Personnel Commission. • Personnel Commission Laws and Rules are based in large part on provisions of Federal Law, the State of California Education Code with emphasis on the Merit System Article, the State of California Government Code, and many other laws, regulations, ordinances too numerous to specify here. The applicable law is cited at the beginning of each rule for reference and education of the reader When rules are created or amended the process requires at least two readings, one tentative, during which the Personnel Commission accepts and considers comments from interested parties, and one final where the rule is placed on the meeting agenda for final approval which affords interested parties another opportunity to comment before a final adoption occurs. (Communication/Education) • The Personnel Commission determines how it intends to put the various laws into effect following a process which highly encourages the participation of employees, supervisors, managers, administration, and the public during at least two public meetings prior to final adoption. (Accessibility/Communication) • Union representatives are provided additional opportunities for comment during regular consultations and special meetings in advance of adoption of Rules by the Personnel Commission. (Accessibility/Communication) • The Personnel Commission Rules can be amended at any time based on need, but are also on a regular review schedule to respond to any issues that have arisen in daily application of the rule, ensure adherence to current law and best practices, and educate/reeducate constituents regarding available channels to pursue concerns and interests. (Analysis/Corrective Action/Education) • Personnel Commission Rules are published and available on the Personnel Commission Website in their most up-to-date form at all times. (Accessibility/Communication) 12 CALIFORNIA STATE AUDITOR | Report 2020-111 69 May 2021 Overall Process and Resource Information In addition to the information available through Personnel Commission Rules, the Commission’s Website provides more “hands-on” process information for employees, supervisors, managers, administration, and the public on various matters. The information available includes: • Process Flow Charts and Timelines - Examples: Class Description Development and Review, Reclassification Study Process; Rule Revision Process; Disciplinary Hearing Process. It is of note that most of these processes/charts were developed and adopted in partnership with our classified union representatives and District administrative representatives in response to questions/inquiries/”complaints”. (Analysis/Communication/Education) • Informational Bulletins – These bulletins provide information and answers to the most often asked questions employees, supervisors, managers, and administrators pose during a study or processing of transactions. They were developed following our analysis of the questions/inquiries/ “complaints” asked over time. (Analysis/Corrective Action) • Career Ladders Guide – This guide was developed in response to questions/inquiries/ “complaints” voiced over time by employees seeking opportunities for promotion. It is of note that the Personnel Commission does not make hiring decisions for District departments, but often hears complaints from frustrated employees who are looking for opportunities to promote within the District. This guide helps all involved to understand the process and better prepare employees for promotional opportunities. This document combined with the opportunity for individual career counseling through our Outreach Process is highly valuable to employees who wish to promote or change careers and need guidance in achieving their career goals. (Analysis/Corrective Action/Communication/Education) • Employee Handbook – This handbook was developed in response to questions/inquiries/ “complaints” voiced over time by employees seeking general information about District and Board employment policies, procedures, services, and benefits. This document serves as a good starting point for new employees and as a quick reference guide for existing employees. (Analysis/Corrective Action/Communication/Education) Study Initiation The core business transactions of the Personnel Commission are not handled through an independent “complaint process” but through processes and procedures outlined in Personnel Commission Laws and Rules. Hence, a telephone call from an employee saying, “I am being asked to perform duties outside of my classification" is screened by staff with a few clarifying questions, providing the employee/complainant with basic information, directing them to PC Rules for more in depth information, and advising them of the correct form/procedure to request/initiate a formal study. Although this might be characterized as a “complaint” we consider it a request for a study. The person may also be directed to their union for assistance. Studies of a larger scale that are not necessarily focused on an individual employee, i.e. series 13 70 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 reclassifications, new classes, salary studies, reorganizations are typically initiated through administrative channels or the union via a written request to the Personnel Commission. (Policy/Procedure/Accessibility/Communication) Study Screening and Management Requests for a study submitted through the required request forms are logged in, reviewed by the Director or Assistant Director, assigned to an analyst, and a working file is created. (Process/Record Keeping) In an individual study, the employee is notified of receipt of their request and provided procedural information regarding the upcoming process. (Communication) Studies involving groups or broader issues, rather than individuals, are logged in, reviewed by the Director and Assistant Director, and assigned to an analyst. Required research, consultations, and meetings follow before any study is finalized. (Communication/Research/Investigation) Study Process In the classification study process, employees, supervisors, managers, and administrators are all given multiple opportunities to communicate anything they wish during studies involving job classification, reclassification, work-out of classification, etc. Such opportunities involve in-person desk audits, phone audits, submission of work samples, escalation of technical issues to higher-level analysts, involvement of supervisors, etc. Furthermore, union reps are provided opportunities to consult on studies before any action is taken by the Personnel Commission. The examination process begins with a “Start-Up Questionnaire” forwarded to hiring authorities wherein their input on many issues is proactively sought before an examination proceeds. After an examination has been administered, there is a formal three-level appeal process for exam participants who have been unsuccessful in a test. Administrative “complaints” regarding examination are addressed directly to the Personnel Commission. Study Conclusion All study requests are brought to conclusion with a written staff report which summarizes the analyst’s findings, analysis, and recommendations. All staff reports are reviewed by the Director and Assistant Director. The report is widely shared for comment with involved parties before being placed on the Personnel Commission open meeting agenda. All agenda reports are publicly posted on the Personnel Commission website at least 72 hours before the public meeting. All involved parties (i.e. employees, supervisors, union reps) are encouraged to attend the Personnel Commission meeting. (Accessibility/Communication/Response) The Personnel Commission makes the final decision to approve, amend, or reject staff's recommendations at the applicable open session of the Personnel Commission meeting. There are also instances when the Personnel Commission postpones action on an item based on a request of the employee, administrator, or union representative. This may include cases where the employee wishes to submit additional information, the union asks for more time to consult, the employee is not available to attend the meeting where the item is on the agenda, etc. After addressing the voiced concerns/requests, staff places the item back on the next available 14 CALIFORNIA STATE AUDITOR | Report 2020-111 71 May 2021 Personnel Commission agenda. (Accessibility/Communication/Investigation/Corrective Action) Outreach Efforts In an effort to help resolve personnel issues at the local level and increase access to Commission staff, the Personnel Commission initiated its Personnel Commission Field Representative Program in 2001. The program had two full-time field representatives who rotated between the colleges on a regular schedule to meet with employees, supervisors, managers, and administrators to answer questions, facilitate resolutions to issues, conduct workshops on topics of interest to classified employees and supervisors, write monthly informational bulletins, and offer career counseling sessions. In 2017, the program had to be downsized due to financial hardships within the District and the retirement of the last field representative. Currently, the Personnel Commission has one analyst dedicated to serve as a contact person for classified employees, supervisors, managers, and administrators to address any inquiries/issues they may have. This person, Patrick Sung, is listed on the Personnel Commission website under Employee Outreach. (Accessibility/Communication/ Response/ Education) Solicitation of Feedback Opportunities to provide feedback, including “complaints”, is incorporated into Personnel Commission operations via several additional mechanisms. • Address the Personnel Commissioners: Callers with concerns are always advised of the opportunity to address the Personnel Commissioners at a meeting or in writing. If in writing, the Personnel Commission receives the document as correspondence and provides direction to staff to address the concerns in a timely manner. Open public meetings are bi-monthly with time and location posted on the Personnel Commission’s website. • Special Personnel Commission Email Addresses: One for general inquiries. The second specifically for examination related inquiries. These email inboxes get frequently checked and inquiries are responded to in a timely manner. • Feedback Survey: Individuals can complete a small survey and provide comments through the Personnel Commission website under the following link: http://laccd.edu/Departments/PersonnelCommission/Pages/We-Want-Your- Feedback.aspx • Individual Feedback Survey: Following completion of a classification study, the subject employee and supervisor are given the opportunity to complete a brief survey regarding their experience. • New Employee Satisfaction Survey: Following a new hire, the hiring authority is asked for feedback regarding the new employee’s likelihood of job success. 15 72 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 State Auditor’s Recommendations: 1)The Commission should amend its rules by October 2021 to clearly define complainants and create a formal process for addressing all complaints, including a process to elevate to the Commissioners those complaints that are not resolved at lower levels. 2)The Commission should amend its rules by October 2021 to include a provision for submitting whistleblower complaints directly to the District’s general counsel and assign it the responsibility of designating an appropriate party to respond (Note: Auditors were considering changing this recommendation). 3)The Commission should amend its rules by October 2021 to establish that complainant information may not be shared with the subject of a whistle-blower complaint. (Note: Auditors were considering changing this recommendation) Personnel Commission’s Response to State Auditor’s Recommendations: 1)As previously noted, it is the Personnel Commission’s position that the “complaint process” is not a stand-alone isolated process and that the core business transactions of the Personnel Commission are not handled through an independent “complaint process” but through processes and procedures outlined in Personnel Commission Laws and Rules. Rule 513 addresses how requests for investigations of personnel problems related to alleged violations of the Merit System Laws or Personnel Commission Rules are to be processed, and Rules 600, 624, 735, and 893 outline appeal or adjustment processes for examination results, disciplinary actions, debarments from employment, and grievances of unrepresented employees, which represent the core areas of employee issues in a Personnel Commission environment. All key terms listed in Commission rules are defined in Rule 500. This includes definitions for “appeal”, “grievance”, “adjustment procedure”, etc. The term “complainant” is not a term that independently exists in a merit system environment for reasons already explained. The Personnel Commission believes that these rules, taken together, sufficiently address the few complaints that are made regarding our practices and policies. 2)The Personnel Commission notes that Whistleblower complaints do not fall within its purview. They are regulated by the Government Code. These types of complaints are handled by the Internal Audit Department of the Los Angeles Community College District (see Flyer below which is posted on the LACCD website). The audit team made reference to a whistleblower email in their audit report that was erroneously forwarded by the General Counsel’s Office to the Personnel Commissioners and cc’d to the former Personnel Director. Furthermore, as Rule 503 (C.1 and C.2) states the Personnel Commission recognizes that no set of rules can contemplate all possible combination of circumstances affecting particular cases. These rules are to be applied with consideration of their intent. Interpretations regarding the meaning, intent, or applications of the rules shall be made by the Personnel Commission. 3)The Personnel Commission notes that whistleblower complaints do not fall within its purview. They are regulated by the Government Code. These types of complaints are handled by the Internal Audit Department of the Los Angeles Community College District. (see flyer below). Furthermore, as Rule 503 (C.1 and C.2) states the Personnel Commission recognizes that no set of rules can contemplate all possible combination of circumstances affecting particular cases. These rules are to be applied with consideration of their intent. Interpretations regarding the meaning, intent, or applications of the rules shall be made by the Personnel Commission. 16 CALIFORNIA STATE AUDITOR | Report 2020-111 73 May 2021 17 74 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 5 Other Areas Reviewed By Auditors • Area Involving Commission’s Debarment and Notification Practices State Auditor’s Recommendations: 1) The Commission should establish rules to require that examiners independently verify the reason for inconsistencies between applications. 2) The Commission should establish rules that examiners provide applicants an opportunity to address the inconsistencies. 3) The Commission should establish rules that examiners document the steps they take to verify the disputed information and retain relevant supporting documentation. Personnel Commission’s Response to State Auditor’s Recommendations 1)The Personnel Commission contends that it already has a sound process in place for examiners since they are asked to summarize information on inconsistencies between job applications relevant to a debarment case. This summary is independently compiled and attached to the debarment notice that is sent to the applicant. 2)The Personnel Commission notes that only applicants who intentionally create major inconsistencies on their job applications that are based on making false statements, omitting material facts, and/or practicing deception or fraud to pass an examination or to secure employment are subject to debarment. This includes examples such as where an applicant intentionally leaves off positions he/she was terminated from, changes significant dates of employment for positions to increase the years of experience, adds new jobs that were never listed on prior job applications for the single purpose to list experience that will qualify them for the new position, etc. All applicants before they submit an application are provided instructions in the JopAps application system on how to fill out an application. The instructions specifically ask applicants to provide accurate and complete information on their employment history for the last ten years and educational background. Applicants are also alerted in the instructions to be consistent with the information across their job applications. Therefore, the Personnel Commission does not agree with the statement made by the auditors in the report that “applicants may have omitted jobs or duties from past applications because they were not relevant to the positions.” While applicants may have omitted jobs or duties that appear on previous applications because they did not feel they were relevant to the position now being applied for, this is in direct opposition to our very explicit instructions for completing an application and, unfortunately, leads to the suspicion that the applicant is being dishonest. The Personnel Commission notes that debarments of applicants due to inconsistencies that appear on their job applications are very rare (less than 1%) when comparing it with the total number of applications received each year by the classified examination unit, which typically ranges well over 10,000. In the event that an applicant is debarred due to providing significant inconsistencies on his/her job applications, there is due process in place by which he/she can appeal the examiner’s decision in accordance with Rule 600. During this appeal process an applicant will be able to provide additional clarification on the inconsistencies on their job applications. 3)The Personnel Commission notes that there is already an established process for how examiners document the information they obtained for debarment cases in the JobAps 18 CALIFORNIA STATE AUDITOR | Report 2020-111 75 May 2021 system. The Personnel Commission agrees to enhance the JobAps examiner’s manual to further outline the documentation process. • Area Involving Commission’s Accountability Measures The State auditors noted in their report that the “Personnel Commission should exercise greater oversight of Commission operations” and used as an example the examination administrations conducted by Commission staff. The Personnel Commission contends that Commissioners by law do not have regular work schedules like the Commission’s managerial, professional, and clerical staff. Their schedule is based on meetings (see Ed Code 88071) that occur twice a month during which they rule on agendized items, discuss appeal cases, and hear any concerns from non-agenda speakers. Hence by definition of their role, it not reasonable to assume that the Commissioners should get involved in the day-to- day transactions and examination administrations handled by the Commission’s Office. It is the role of the Personnel Director and its professional and clerical staff to attend to business transactions, administer examinations, and resolve routine complaints/inquires at the staff level. Furthermore, it would also constitute a problematic approach since the Commissioners represent a final level of appeal for applicants and employees and therefore it is important that they maintain their objectivity when hearing an appeal or a grievance, which would not be possible if they are involved in the day-to-day transactions of the Personnel Commission Office. State Auditor’s Recommendation: 1) The Commission should establish rules to require staff to periodically report to them on how its practices compare to those of other merit system entities’, along with any recommendations for improving the Commission’s practices. The Personnel Commission’s Response to State Auditor’s Recommendation 1) The Personnel Commission notes that there is a sound rule review process in place that goes beyond just using other agency’s’ practices when reviewing their own processes. It includes conducting regular annual legislative bill reviews related to human resources areas to determine if they have an impact on rules and processes, which are reported to the Commission in a detailed report, the review of comparable rules from other merit system Districts for comparative purposes, and the review of applicable provisions of collective bargaining unit agreements representing classified employees for consistency purposes. The Personnel Commission agrees to have staff add details on other agencies practices when rule amendments are presented to them, if applicable. • Area Involving Commission Turnover and Work Environment In the audit report it was alleged that a negative work culture existed in the Personnel Commission that resulted in employee turnover during the past five years. The evidence that was cited in support of this allegation included high employee turnover in 17/18 and interview comments from current and former employees. 19 76 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 The Personnel Commission’s Response to Turnover and Work Environment Statements made by State Auditors The following is presented in reaction/response to the methodology, analysis, and conclusions related to this allegation. The following is the “turnover” information the State auditors requested and relied on in their analysis (note: employees names and titles were removed). Employee FY Employee A 14/15 Employee B 15/16 Employee C 15/16 Employee D 16/17 Employee E 16/17 Employee F 17/18 Employee G 17/18 Employee H 17/18 Employee I 18/19 Employee J 18/19 From this information and organization charts, the auditors computed a turnover rate of 21% for l7/l8 and concluded that this was an unusually high turnover rate. Fiscal Year Total PC Employees Employee Turnover Turnover % 14/15 12 1 8% 15/16 14 2 14% 16/17 15 2 13% 17/18 14 3 21% 18/19 13 2 15% The impact of a small sample on the statistics is obvious and does not appear to have been considered in the auditors’ conclusion of a “unusually high turnover rate.” However, there is an additional issue. By definition, employee turnover rate refers to the proportion of employees who leave the employer during a certain time period. This rate includes both voluntary and involuntary separation and excludes internal movements (promotions and transfers) and employees who are on furlough or leave of absence. Therefore, to accurately compute even a basic turnover rate, knowing the reason an employee left is needed. 20 CALIFORNIA STATE AUDITOR | Report 2020-111 77 May 2021 The reasons for PC employees leaving during the audit period are as follows: Fiscal Yrs. of PC Employee Reason for Leaving Year Service Promotion; employee later promoted back to 14/15 Employee A 7 Yr 3 Mo the Personnel Commission Transfer to a College which was closer to the 15/16 Employee B 1 Yr 10 Mo employee’s home 15/16 Employee C Release during probation 4 Mo 16/17 Employee D Transfer: Career Change Opportunity 2 Yr 4 Mo 16/17 Employee E Retirement 33 Yr 17/18 Employee F Transfer to Vice Chancellor of HR Office 8 Yr 8 Mo 17/18 Employee G Promotion 1 Yr 7 Mo 17/18 Employee H Resignation due to medical/health Issue 6 Yr 5 Mo Voluntary Demotion to a college which was 18/19 Employee I 2 Yr 11 Mo closer to the employee’s home Involuntary Demotion following employee- 18/19 Employee J 15 Y 1 M employer relations matter By applying a different and more appropriate methodology to the Personnel Commission data, a completely different picture is painted: turnover is well within acceptable HR metrics (chart below); within the turnover record there is nothing to credibly support a conclusion that employees left because of a poor work culture; years of service with the Personnel Commission, all under the same leadership, as well as the reasons noted for turnover do not support a poor work culture finding. Fiscal Year Total PC Employees Employee Turnover Turnover % 14/15 12 0 0% 15/16 14 1 7% 16/17 15 1 7% 17/18 14 1 7% 18/19 13 0 0% INTERVIEWS WITH CURRENT AND PAST PC EMPLOYEES The State auditors reported that some current and past employees were interviewed and made comments about the working conditions and past leadership within the Personnel Commission. From the interviews alone with no specific examples to cite of retaliation or negative classification actions, the auditors alleged that a negative work culture existed in the Personnel Commission. Our concern is that no one in Personnel Commission leadership was contacted to provide context and management perspective to what may have been said. Workplace dynamics are very complex and without hearing all sides of the story, it is possible to come to an inaccurate conclusion. 21 78 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 The Personnel Commission leadership is dedicated to fostering a healthy and positive work environment and would have been interested in knowing what specific events, situations, circumstances, and statements lead to the auditors negative work culture conclusion. How the credibility of the employees they spoke with was established would have also been of interest. For example, there were unfounded discrimination claims made by a former employee who was discredited after the investigation was completed by an independent outside investigator. Finally, if leadership would have been informed of these claims, we would have been able to evaluated them and respond accordingly. • State Auditor’s Statements Related to Concerns Regarding the Commission’s Practices The State auditors noted in their report that they received survey information from one of the collective bargaining unions of the District which conducted a survey in November of 2019 and alleged that 117 out of 975 total survey responses included negative comments about functions performed by the Personnel Commission. The negative comments included areas such as eligibility and qualifications, classifications and reclassifications, promotions, examinations, compensation, and working out of class. The State auditors also noted in their report that “some district employees told them that they feared retaliation or retribution if they questioned Commission practices”. The Personnel Commission’s Response to those Concerns The Personnel Commission notes that there are approximately 2200 regular classified employees in the Los Angeles Community College District. The 117 survey responses cited by the state auditors represent approximately 11% of all regular classified employees that the Personnel Commission serves. The Commission notes that those survey results were neither shared with them nor were specific cases provided as examples so that the Commission would have been given the opportunity to research and respond to the allegations. Therefore, this information represents a one-sided, completely subjective view on Personnel Commission operations from a small percentage of employees that was not supported by any verifiable evidence. These negative survey responses were most likely driven by an outcome on a study or examination that was not satisfactory to the employee and cannot be used as a measure to substantiate valid concerns about the Commission practices and processes. The Personnel Commission also notes that it not aware of any case in its entire history where employees had evidence that they were retaliated against by the Personnel Commission if they questioned practices. Appendix Scope and Methodology Personnel Commission Comments related to Audit Scope 4 The State auditors noted in the fourth bullet in this section that the Personnel Commission did not verify high school transcripts and as as result the Commission did not determine whether 13 candidates met minimum qualifications. The Personnel Commission notes that it used to verify high school transcripts more than a decade ago but since it had become an exceedingly hard burden on job applicants to obtain their high school records from their high schools that they attended years or decades ago in the United States or foreign countries, the Commission made a policy decision to no longer require applicants to provide proof for an educational 22 CALIFORNIA STATE AUDITOR | Report 2020-111 79 May 2021 requirement that only represented proof of a basic education with no focus on an occupational specialty that would have been critical to the qualifications of a position. Furthermore, the Personnel Commission typically administers written tests for those classified positions which require a high school diploma as the minimum education requirement that cover the basic skills taught in high school, i.e. reading, arithmetic, spelling, grammar, etc. This concludes the summary of our responses to your State Audit report. Should you have any questions or wish to discuss our comments further, please do not hesitate to contact us. Sincerely, David H. Iwata Chair, Personnel Commission _________________________________________________________________________________________ LOS ANGELES COMMUNITY COLLEGES East Los Angeles College Los Angeles City College Los Angeles Harbor College Los Angeles Mission College Los Angeles Pierce College Los Angeles Southwest College Los Angeles Trade-Technical College Los Angeles Valley College West Los Angeles College 23 80 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 Blank page inserted for reproduction purposes only. CALIFORNIA STATE AUDITOR | Report 2020-111 81 May 2021 COMMENTS CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM THE LOS ANGELES COMMUNITY COLLEGE DISTRICT PERSONNEL COMMISSION To provide clarity and perspective, we are commenting on the Commission’s response to our audit. The numbers below correspond to the numbers we have placed in the margin of the Commission’s response. The Commission disagreed with a number of our conclusions, objected 1 to some phrasing in our report, and criticized our methodologies and staff expertise. Despite these criticisms, the Commission failed to provide support for its assertions, contradicted itself, mischaracterized some of our conclusions and recommendations, and presented flawed arguments. Furthermore, the Commission states that it has begun to improve its processes and intends to implement most of our recommendations. Rather than comment on all of the areas of its response that we believe are deficient or misleading we have summarized our comments according to the respective sections of our audit report. Application and Exam Process 2 In its response to the section of our report that pertains to the application and examination process for hiring and promotion, the Commission states that our conclusions overstate the problems making them appear more widespread than they really are. Contrary to the Commission’s assertion, our audit conclusions were based on our assessment of the Commission’s processes, and the examples we use in the audit report highlight and strengthen our conclusions regarding the shortcomings of those processes. In addition, as we describe throughout the report, we obtained sufficient evidence to support our conclusions and recommendations. Although we acknowledge that in its response in most cases the Commission agrees to fix many of the deficiencies we identified, it simultaneously attempts to minimize their effect. For example, by focusing on the number of cases we identified in which the deficiencies affected particular candidates, the Commission overlooks the fact that it should have processes in place to treat all candidates, rather than most candidates, fairly and consistently. 82 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 Despite the Commission’s attempt to minimize the deficiencies we identified, we are pleased to note that the Commission asserts that it has recently implemented our recommendation regarding the need to increase transparency and ensure that it makes consistent decisions when assessing applicants’ minimum qualifications by defining key terms such as “professional-level” and “recent.” Furthermore, the Commission asserts that it has updated some of its processes based on issues we raised during the audit and will implement nearly all of the remaining recommendations to improve its application and examination processes for hiring and promotion. We look forward to receiving the Commission’s 60-day, 6-month and one-year responses so that we can assess its progress in improving its processes and practices. 3 Working Out of Class The Commission’s response to the section of our report regarding compensation for employees temporarily working out of class fails to address the importance of the issue as described in our report. Specifically, the Commission’s assertion that it has compensated employees working in out-of-class assignments in a consistent and fair manner is not correct. As we describe on page 26, of the six out-of-class claims we reviewed, five employees did not receive payment for between five and 11 months after they began performing the higher-level work. In contrast, the remaining employee received multiple intermittent payments during their out-of-class assignment. In addition, the Commission’s assertion that our recommendation to revise its rules to process employees’ compensation claims for out-of-class work monthly would not be efficient and would increase the work for the Commission, Board and the Payroll Department is disingenuous. As it states in its response, “There are typically less than 10 cases in any given year.” Further, as we state on page 28, during the three fiscal year period we reviewed, the Commission only had to consider 22 out-of-class claims. As such, the additional work required to process monthly payments to employees working out-of-class is neither inefficient nor burdensome. Finally, with respect to our recommendation to require employees to submit a copy of their out-of-class claim to the Commission at the same time they submit it to their supervisors, the Commission’s response is misguided. Contrary to the Commission’s attempt to suggest that our recommendation would require the Commission to begin processing the claim prior to the supervisor and district administrator certifying it, the purpose of our recommendation is to ensure that the Commission has an accurate record of when the employee submitted the claim. CALIFORNIA STATE AUDITOR | Report 2020-111 83 May 2021 Addressing Complaints 4 The Commission’s response to our conclusions and recommendation regarding its processes for addressing complaints and protecting complainants is disappointing. Specifically, the Commission’s statement that the “complaint process” is not a stand-alone isolated process and that core business transactions are not handled through an independent “complaint process” fails to acknowledge the importance of a formal complaint process. In addition, the document that the Commission includes in its response on pages 66 through 71 was created by the Commission in responding to questions by auditors from our office during the course of the audit. The document itself is not part of the Commission’s rules or policy manuals; however; its length illustrates the complexity and number of processes employees and applicants must navigate to express a concern. Generally, this document makes a number of assertions about how the Commission addresses all complaints. However, our review revealed that the Commission had not established for all types of complaints several of the elements it describes as typical aspects of a complaint management process. Moreover, as we describe on page 33, the Commission lacks a defined process for addressing all employee complaints, and it could not document that it adequately addresses all complaints it received. Finally, we disagree with the Commission’s assertion that whistleblower complaints do not fall within its purview. Although state law outlines whistleblower complaint processes, it does not preclude the Commission from establishing rules to clarify how whistleblower complaints are to be handled—as the District has. As such we stand by our conclusions and recommendations and believe that the Commission must implement them to ensure that it consistently identifies and responds to all complaints and reduces the risk of retaliation against complainants. Other Areas We Reviewed 5 In the Commission’s response to the section of our report concerning its debarment and notification practices, the Commission does not address the deficiency we identified—that the Commission does not verify information used for debarment. Summaries of inconsistencies between job applications, even if they are compiled independently, do not provide adequate assurance that an examiner has reached an accurate conclusion. In addition, debarment is a significant consequence, and although discrepancies in applications may be a cause for suspicion and may warrant further follow-up, suspicion— absent evidence of actual dishonesty—should not be grounds for debarment. Moreover, eliminating a candidate from an examination and requiring them to avail themselves of the appeal process in order to defend themselves places an undue burden on the applicant. 84 Report 2020-111 | CALIFORNIA STATE AUDITOR May 2021 The Commission’s response regarding our report section on the Commission’s accountability measures exaggerates the level of commissioner involvement that we recommend. We did not recommend that the commissioners involve themselves in the day-to-day transactions of the Commission. Rather, we recommend that the commissioners more closely review the Commission’s policies and procedures, in accordance with their intended role. Doing so would not compromise their objectivity. Similarly, the Commission’s response does not address our recommendation. Rather than questioning the rule review process, we are recommending that the commissioners apply it more broadly and review existing rules and Commission practices. This proactive review would ensure that Commission practices are kept up to date and aligned with the best practices of other entities. The Commission’s response to the section of our report concerning Commission turnover and work environment is inaccurate in that we do not allege that a negative work culture existed at the Commission; rather, we state that District employees and former Commission employees described concerns about the Commission’s culture and stated that they feared retaliation by the former director. In addition, former Commission employees told us that the behavior of the former director influenced their decision to leave the Commission. The Commission’s suggestion that we should have provided it the opportunity to address specific employees’ perspectives, which would have required us to identify those employees to the Commission, illustrates its failure to grasp the basic tenets of confidentiality and retaliation prevention. Regarding the Commission’s statements about calculations of employee turnover, the Commission’s assertion about the information and approach we used for our calculation is incorrect; we shared both with the Commission during the course of our audit and it did not dispute them at that time. We also note that the Commission’s calculation of employee turnover is for a different time period than ours and that it inappropriately excluded some employees that we included in our calculation. It is also important to note that contrary to the Commissions’ assertion, we do not conclude in our report that the Commission had an unusually high turnover rate. Finally, the Commission’s response raises concerns about how we used information from a 2019 union survey that is included in the introduction of our report. Our purpose in reviewing this information was to assess District employees’ perceptions of the Commission, not to determine whether those perceptions were valid. Rather than reviewing the validity of individuals’ perceptions, we objectively assessed the adequacy of the Commission’s processes and independently reached our own conclusions, as described throughout the report. Nevertheless, the number of negative comments from the survey indicates that some of the District’s classified employees do have a poor perception of the Commission’s practices and the fairness of those practices.