Recommendations

Board of Registered Nursing It Has Failed to Use Sufficient Information When Considering Enrollment Decisions for New and Existing Nursing Programs July 2020 REPORT 2019‑120 CALIFORNIA STATE AUDITOR 621 Capitol Mall, Suite 1200 | Sacramento | CA | 95814 916.445.0255 | TTY 916.445.0033 For complaints of state employee misconduct, contact us through the Whistleblower Hotline: 1.800.952.5665 Don’t want to miss any of our reports? Subscribe to our email list at auditor.ca.gov For questions regarding the contents of this report, please contact Margarita Fernández, Chief of Public Affairs, at 916.445.0255 This report is also available online at www.auditor.ca.gov | Alternative format reports available upon request | Permission is granted to reproduce reports Elaine M. Howle State Auditor July 7, 2020 2019‑120 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As directed by the Joint Legislative Audit Committee, my office conducted an audit of the Board of Registered Nursing (BRN) to assess its oversight of prelicensure nursing programs (nursing programs). The following report details our determination that BRN has failed to use sufficient information when considering the number of students new and existing nursing programs propose to enroll. BRN’s governing board (governing board) both approves new nursing programs in the State and makes decisions about the number of students that existing nursing programs are allowed to enroll (enrollment decisions). Two of the key factors that should influence BRN’s enrollment decisions are the forecasted supply of nurses that the State will need to fulfill demand and the available number of clinical placement slots—placements at a health care facility for students to gain required clinical experience. BRN’s 2017 forecast of the State’s future nursing workforce indicated that the statewide nursing supply would meet demand; however, it failed to identify regional nursing shortages that California is currently experiencing and is expected to encounter in the future. BRN’s governing board also lacks critical information about clinical placement slots when making enrollment decisions, which hampers its ability to prevent nursing students from being displaced because other nursing programs took their clinical spots. BRN does not gather and share with the governing board information about the total number of placement slots that a clinical facility can accommodate annually or how many slots the programs that use the facility will need each year. Without this key information, BRN cannot properly gauge the risk of such student displacement— reported to have affected 2,300 students in academic year 2017–18—when its governing board makes enrollment decisions. Finally, we found that some of BRN’s requirements for nursing programs overlap with standards imposed by national nursing program accreditors (accreditors). As part of the Legislature’s 2021 review of BRN, it could consider the appropriateness of restructuring BRN’s oversight to leverage portions of the accreditors’ review in order to reduce duplication and more efficiently use state resources. Respectfully submitted, ELAINE M. HOWLE, CPA California State Auditor 621 Capitol Mall, Suite 1200 | Sacramento, CA 95814 | 916.445.0255 | 916.327.0019 fax | www.auditor.ca.gov iv California State Auditor Report 2019-120 July 2020 Selected Abbreviations Used in This Report ACC American Career College ACEN Accreditation Commission for Education in Nursing BRN Board of Registered Nursing CCNE Commission on Collegiate Nursing Education OAL Office of Administrative Law OSHPD Office of Statewide Health Planning and Development RN Registered nurses California State Auditor Report 2019-120 v July 2020 Contents Summary 1 Introduction 5 Audit Results BRN’s Forecasts of the Supply of Qualified Nurses Have Not Included Key Information 15 BRN’s Process for Assessing the Availability of Clinical Placements Is Inadequate 19 BRN’s Process for Approving Nursing Programs Partially Overlaps With the Work of Accreditors 29 Other Areas We Reviewed 32 Recommendations 34 Appendix Scope and Methodology 37 Response to the Audit Board of Registered Nursing 41 California State Auditor’s Comments on the Response From the Board of Registered Nursing 47 vi California State Auditor Report 2019-120 July 2020 Blank page inserted for reproduction purposes only. California State Auditor Report 2019-120 1 July 2020 Summary Audit Highlights . . . Results in Brief Our audit of BRN’s oversight of nursing programs highlighted the following: In addition to its other duties as the state agency that regulates » BRN does not gather and use sufficient the practice of registered nurses (RNs), the Board of Registered data to make decisions about the number Nursing (BRN) oversees California’s prelicensure nursing programs of students nursing programs can enroll. (nursing programs), which prepare students to practice as entry‑level RNs. BRN’s governing board (governing board) both • It determined the State’s overall approves new nursing programs in the State and makes decisions nursing supply and demand was about the number of students that new and existing nursing balanced, but did not identify programs are allowed to enroll (enrollment decisions). Two of the California’s current regional key factors that should influence BRN’s enrollment decisions are the nursing shortages. forecasted supply of nurses that the State will need to fulfill demand and the available number of clinical placement slots—placements • BRN’s governing board does not at a health care facility, such as a hospital, that nursing programs have needed information about must secure for students to gain required clinical experience. In clinical placement slots when this audit, we found that BRN has failed to gather and use sufficient making enrollment decisions—in data related to both of these factors to appropriately inform its academic year 2017–18, nursing enrollment decisions. programs reported that more than 2,300 students were affected by Specifically, BRN’s 2017 forecast of the State’s future nursing clinical displacement. workforce needs indicated that the statewide nursing supply would meet demand; however, it failed to identify the regional nursing • BRN uses inconsistent and incomplete shortages that California is currently experiencing and is expected information to assess the availability to encounter in the years ahead. Although BRN’s methodology for of clinical placements because it has determining the State’s overall nursing supply and demand was not provided guidance to its nursing reasonable, it did not measure regional variations that would have education staff about what to provide identified regional nursing shortages. Given the size and diversity the governing board to aid it when of California, regional forecasts would provide critical information making enrollment decisions. For to inform enrollment decisions and other actions by BRN’s example, it does not gather and share governing board. information about the total number of placement slots available at a facility. BRN’s governing board also lacks critical information about clinical placement slots when it considers enrollment decisions. When » Some of BRN’s requirements for nursing making these decisions, the governing board should consider programs—such as those related to the available number of clinical placement slots. If the governing approval of faculty and curriculum— board’s enrollment decisions allow for more enrolled students than overlap standards set by accreditors and, the number of clinical placements available in the region, nursing thus, some of BRN’s oversight could be programs end up having to compete for clinical space for their duplicative of what accreditors review. students. During the 2017–18 academic year, nursing programs reported that more than 2,300 students were affected by this clinical displacement—an insufficient supply of clinical placement slots. Nearly half of those programs reported that students from another program displaced their students, while many programs also reported losing clinical placements slots because facility staff workloads were too great to allow time for supervising nursing students. When displacement occurs, the nursing program 2 California State Auditor Report 2019-120 July 2020 losing placement slots must find new placement slots for its displaced students in order to provide the required clinical experience to its students. BRN uses inconsistent and incomplete information to assess the availability of clinical placements because it has not provided guidance to its nursing education consultants (nursing education staff), who are employees of BRN, about the information they should provide to the governing board to aid it in considering enrollment decisions. Our review of 15 enrollment decisions found that BRN nursing education staff did not consistently provide to the governing board the information the staff had on the availability of clinical placements, such as how a proposed increase in enrollment would affect facilities that the requesting program planned to use for clinical placements. Some of BRN’s governing board members have also expressed concern that BRN’s existing process for assessing clinical displacement is not clear. Additionally, BRN does not gather and share with the governing board information concerning the total number of placement slots a clinical facility can accommodate annually and how many slots the programs that use the facility will need each year. Without this key information, BRN cannot properly gauge the risk of displacement when its governing board is making enrollment decisions. To further enhance its information about clinical placement slots, BRN should require nursing programs to annually update information about the clinical facilities they use for student placements. With this information, BRN would be able to identify the types of facilities that programs most frequently use. Compiling this information and comparing it with other publicly available information about existing clinical facilities would also allow BRN to identify clinical facilities that programs do not currently use for placements, which could help nursing programs find additional facilities with capacity for their students. Lastly, some of the nursing programs that BRN oversees are accredited by national nursing program accreditors (accreditors). Accreditors are private educational associations that verify whether programs meet and maintain acceptable levels of quality. We found that some of BRN’s requirements for nursing programs— specifically those related to approval of faculty, curriculum, and continuing compliance with state requirements—overlap with the standards imposed by accreditors. As part of the sunset review process, during which the Legislature evaluates the efficiency of certain state agencies, the Legislature should consider whether it would be appropriate to restructure any of BRN’s oversight to reduce duplication with accreditors while still achieving BRN’s mission to protect the public. California State Auditor Report 2019-120 3 July 2020 Selected Recommendations The Legislature should amend state law to require BRN’s forecasts of the nursing workforce to incorporate regional analyses. BRN should specify in policy the information its nursing education staff must present to the governing board for each enrollment decision it considers. To better inform its enrollment decisions, BRN should gather information concerning the total number of placement slots a clinical facility can accommodate and how many slots the programs that use the facility will need. As part of the sunset review process, the Legislature should consider whether it would be appropriate to restructure any of BRN’s oversight of nursing programs that might overlap with accreditation. Agency Comments BRN generally agreed with the recommendations we made to it. However, it raised concerns over the feasibility of some of the time frames for implementation. 4 California State Auditor Report 2019-120 July 2020 Blank page inserted for reproduction purposes only. California State Auditor Report 2019-120 5 July 2020 Introduction Background The Board of Registered Nursing (BRN) is a state regulatory entity within the Department of Consumer Affairs (Consumer Affairs). State law establishes a nine‑member governing board (governing board) that serves as the governing body of BRN. It is composed of four members of the public and five registered nurses (RNs).1 The governing board appoints an executive officer who has the overall responsibility for managing BRN’s resources and staff, overseeing BRN’s regulatory requirements, and interpreting and executing the intent of board policies for the public and other governmental agencies. In February 2020, BRN’s executive officer resigned, and the governing board appointed an acting executive officer who it subsequently appointed as executive officer in June 2020. BRN had about 240 total authorized staff positions and operated with a budget of about $55 million in fiscal year 2019–20. BRN’s Mission and Functions BRN’s stated mission is to protect and advocate for the health and safety of the public by ensuring the highest quality of RNs in the State of California. The Legislature created BRN in order to regulate and oversee the practice of nursing by implementing and enforcing the Nursing Practice Act, which specifies that protecting the public must be BRN’s highest priority in exercising its functions. Some of these functions relate to nursing education programs, and the licensure, practice, and discipline of RNs. BRN approves two types of nursing education programs: prelicensure programs and advanced practice programs. Prelicensure programs focus on preparing students to practice as entry‑level RNs, while advanced practice programs are for RNs who want to advance their education by earning further certifications, such as nurse practitioner, nurse anesthetist, or clinical nurse specialist. RNs practice nursing by providing direct and indirect patient care, including administering medication and therapeutic agents necessary to implement treatments ordered by licensed physicians. Our review focused specifically on BRN’s oversight of prelicensure nursing programs (nursing programs) located within the State. 1 The five registered nurses include two direct patient care nurses, an advanced practice nurse, a nurse administrator, and a nurse who is an educator or administrator of a nursing education program. The Senate Committee on Rules and the Speaker of the Assembly each appoint a public member, and the Governor appoints the remaining seven board members. State law provides that all appointments are for a four‑year term. Members can be reappointed, although no member can serve more than two consecutive terms. 6 California State Auditor Report 2019-120 July 2020 State law requires BRN to adopt regulations that establish educational requirements for nursing programs. BRN ensures that nursing programs meet these educational requirements as part of its process for approving new nursing programs and inspecting existing programs, which includes verifying that programs provide required courses and hands‑on, clinical experience. Ultimately, BRN’s governing board approves nursing programs if they comply with these regulations. Nursing Programs in California Students graduating from a board‑approved nursing program must pass a national licensing examination in order to become licensed RNs in California. As of 2019, there were 145 board‑approved nursing programs in California. Of those programs, 105 are public schools—community colleges and public universities—and 40 are private schools. Admission to a nursing program can be competitive: in academic year 2017–18 the programs received more than 38,000 qualified applications, but only about 14,000 new students were able to enroll.2 All nursing programs must offer at least the minimum curriculum required by regulation, including specific numbers of coursework units in select areas, such as the science of nursing, related natural sciences, and behavioral and social sciences. Nursing programs can meet these curriculum requirements by offering a variety of degree programs: associate’s, bachelor’s, and entry‑level master’s degrees in nursing. Table 1 lists the types of nursing degrees offered by public and private schools in the State. Table 1 Number of Nursing Programs by Type As of September 2019 TYPE OF PROGRAM PUBLIC PRIVATE TOTAL Associate’s–Typically takes two to three years to complete. Graduates earn an associate’s degree in nursing, and are prepared to provide 79 13 92 nursing care. Bachelor’s–Typically takes four years to complete. Graduates earn a bachelor’s degree in nursing and are prepared to provide nursing care 21 20 41 and to move to administrative and leadership positions. Entry‑level Master’s–Typically takes one to two years, depending on how many nursing course prerequisites the student has completed. Graduates earn a master’s degree in nursing. Designed for individuals 5 7 12 who have a bachelor’s degree in another field and wish to become registered nurses. Graduates are prepared for advanced‑practice nursing careers in research, leadership, and patient care. Totals 105 40 145 Source: BRN’s website and director’s handbook and nursing program websites. 2 An individual can apply to multiple nursing programs, so qualified applications could be greater than the number of individuals. California State Auditor Report 2019-120 7 July 2020 To graduate from a nursing program, students must complete units in both theoretical coursework and hands‑on, clinical experience in five content areas—medical/surgical, obstetrics, pediatrics, mental health/psychiatry, and geriatrics, as Figure 1 shows. To provide the required clinical experience, nursing programs must acquire placements (clinical placements) for students at clinical facilities, such as hospitals. Once a student completes the required coursework and clinical experience and graduates, she or he can apply to BRN to receive a nursing license and take the National Council Licensure Examination (licensure exam) and, upon passing, becomes an RN. Nursing programs in California must maintain a pass rate on the licensure exam of 75 percent for first‑time test takers, though they generally have higher pass rates. On average, 92 percent of first‑time test takers in California pass the exam. As of November 2019, BRN had 11 staff members who are responsible for overseeing nursing programs. Nine of these were nursing education consultants and two were supervising nursing education consultants (nursing education staff). These staff members visit proposed and existing nursing programs to help ensure that they are using approved curricula to prepare competent RNs, as well as to ensure compliance with regulations. BRN generally divides staff assignments geographically into Northern California and Southern California areas, with a supervisor over each area. Each nursing education staff member oversees a group of between six and 20 nursing programs. 8 California State Auditor Report 2019-120 July 2020 Figure 1 Nursing Program Students in California Must Complete Both Classroom and Clinical Units to Become RNs Accepted applicants enroll in a nursing program at a public or private school. Classroom (cid:31)(cid:30)(cid:29)(cid:28)(cid:27)(cid:26)(cid:27)(cid:29)(cid:25)(cid:28)(cid:30)(cid:24)(cid:27)(cid:28)(cid:23)(cid:30)(cid:22)(cid:29)(cid:28)(cid:30)(cid:21)(cid:28)(cid:20)(cid:19)(cid:18)(cid:17)(cid:17)(cid:28) (cid:16)(cid:15)(cid:14)(cid:27)(cid:13)(cid:28)(cid:24)(cid:22)(cid:29)(cid:17)(cid:15)(cid:24)(cid:12)(cid:28)(cid:11)(cid:29)(cid:30)(cid:12)(cid:29)(cid:18)(cid:14)(cid:28) (cid:17)(cid:16)(cid:22)(cid:10)(cid:27)(cid:24)(cid:16)(cid:17)(cid:28)(cid:14)(cid:22)(cid:17)(cid:16)(cid:28)(cid:20)(cid:30)(cid:14)(cid:11)(cid:19)(cid:27)(cid:16)(cid:27)(cid:28) (cid:16)(cid:23)(cid:29)(cid:27)(cid:27)(cid:28)(cid:23)(cid:30)(cid:22)(cid:29)(cid:17)(cid:28)(cid:30)(cid:21)(cid:28)(cid:20)(cid:19)(cid:15)(cid:24)(cid:15)(cid:20)(cid:18)(cid:19)(cid:28)(cid:16)(cid:15)(cid:14)(cid:27)(cid:9) Medical/Surgical Obstetrics Pediatrics Clinical Mental Health/psychiatry Geriatrics nursing program Graduates must apply for licensure and pass the licensure exam. Registered nurses join the nursing workforce. Source: State law and BRN’s website. California State Auditor Report 2019-120 9 July 2020 BRN’s Approval of Nursing Programs and Enrollment Levels Nursing programs must receive approval from BRN in three circumstances: to establish a new nursing program (new program approval), to continue the nursing program following a review that takes place every five years after new program approval (continuing approval), and to make a substantive change. As a part of the new program approval process, a new nursing program must complete a feasibility study that demonstrates, Key Requirements for a Self‑Evaluation among other things, a sustainable budget, evidence of availability of clinical placements for students, A proposed nursing program must submit a self‑evaluation and information on the program’s applicant pool that includes the following items: and sustainability of enrollment. If the governing • Application for approval of a nursing program. board accepts the feasibility study, the proposed nursing program must appoint a nursing director • Total curriculum plan that lists all courses of the program, and complete a self‑study—a self‑evaluation by the including general education courses. nursing program that demonstrates how it plans to • Documentation of curriculum BRN requires for licensure, comply with BRN rules and regulations and such as courses related to nutrition and cultural diversity. provides additional details about the program • Narrative describing how the program will comply with (self‑evaluation), as the text box shows. BRN’s rules and regulations related to the following: nursing education staff members use the self‑evaluation to conduct an on‑site approval visit. – Faculty qualifications and changes to faculty. During this visit, nursing education staff members – Required curriculum. do an in‑depth evaluation of the proposed nursing – Clinical facilities. program to assess compliance with state law. When the governing board approves a new nursing – Licensing exam pass rate standard. program, it also approves how many students Source: State law and BRN forms. that program may enroll. New nursing programs must pay an approval fee to BRN of $40,000. In addition, nursing programs must periodically demonstrate continued compliance with state law. BRN’s policy is to conduct site visits of nursing programs every five years to determine whether they are complying with state law. Ahead of such on‑site visits, a nursing program must provide another self‑evaluation, similar to that required for initial approval. Nursing programs established after January 1, 2013 must pay a continuing approval fee of $15,000 every five years to BRN. If BRN finds that a nursing program did not comply with one or more of its rules and regulations, the program must respond to the findings at a meeting of the governing board’s Education and Licensing Committee (education committee), which consists of a subset of board members. According to BRN’s director’s handbook, in such instances, the education committee will recommend to the full governing board that it “defer action to continue approval” to give the program time to correct the violations. The program may remain in this deferred action status for no more than one year. If the school continues to be noncompliant, the governing board may place the program on “warning status, with intent to close the nursing program.” 10 California State Auditor Report 2019-120 July 2020 Furthermore, when a nursing program desires to Information Required When Submitting a make a major change to its curriculum, such as Request to Increase Enrollment changes in the program philosophy and goals or objectives, it must first receive governing board A letter of explanation on the nursing program’s letterhead, approval. BRN also considers an enrollment increase including descriptions of the following: to be a major curriculum change and, therefore, a • The proposed change. nursing program must request governing board • The reason for the change. approval before increasing its enrollment. BRN charges a processing fee of $2,500 that must • How the change will improve the education of students. accompany a proposal for a major curriculum • How the proposed change will affect clinical facilities. change. When a nursing program wants to make such a change, BRN policy requires the program to Source: BRN’s director’s handbook. submit a letter of explanation that includes specific required information, which we list in the text box. Generally, for enrollment increases we reviewed, this information included the number of students by which the program requested to increase its enrollment. Our audit focused on the governing board’s decisions to approve new nursing programs and enrollment increases. We refer to both new nursing program approval and the approval of an enrollment increase to an existing nursing program as enrollment decisions because both increase the number of enrolled nursing students. To inform these decisions, nursing education staff members review the information in the required self‑evaluation or letter of explanation from the nursing program that is making the request to determine whether the program has met the applicable requirements. The nursing education staff members then present their findings to the governing board’s education committee. The education committee advises and makes recommendations to the governing board regarding nursing program requests. Representatives from nursing programs requesting initial approval must appear at the education committee meeting to be available for questions. The governing board can approve, deny, or defer a nursing program’s request. Factors Related to Enrollment Decisions This report highlights two key factors related to the governing board’s enrollment decisions. The first factor is the number of RNs working in the State—the supply of nurses. In making decisions related to the number of students nursing programs can enroll, the governing board affects the flow of new nurses into the State’s nursing workforce, which can help alleviate or exacerbate shortages of nurses. In fact, state law enacted in 2002 requires BRN to collect and analyze nursing workforce data for future workforce planning. During an informational legislative hearing in 2001 on a nursing shortage—held before this law was introduced—various representatives from the nursing profession demonstrated to the Legislature that gathering California State Auditor Report 2019-120 11 July 2020 more complete data on the nursing workforce would better enable researchers and policymakers to identify, and find solutions to, nursing shortages in California. The law requires BRN to produce reports on nursing workforce data at least every two years. To meet these requirements, BRN has contracted with the University of California, San Francisco (UCSF) (contractor) since at least 2005 to publish a biennial statewide nursing workforce forecast (forecast). The second factor we highlight that influences the governing board’s enrollment decisions is the availability of clinical placement slots. When BRN evaluates a request to approve a new nursing program or increase enrollment in an existing nursing program, it considers whether the requesting program has secured sufficient clinical placement slots to accommodate the increase in students. Clinical placements are based on a written agreement with a clinical facility that has provided assurance of the facility’s availability to accommodate the program’s nursing students. Before a nursing program can use a facility for clinical placements—as a new program or for increased enrollment—the program must first obtain approval from BRN. The nursing program must complete and submit a clinical facility approval form (facility approval form) on which a facility representative attests that the program’s use of the facility will not displace students from other nursing programs currently using the facility to gain clinical experience. BRN nursing education staff members document their approval of the facility on the facility approval form, and BRN keeps records of these forms digitally in its network drive. State law requires all students to complete 864 hours of clinical experience to ensure that they are competent to serve the public when they become licensed nurses. Given a two‑year nursing program with 16‑week semesters, students might spend on average 12 to 15 hours per week meeting the State’s clinical experience requirement. California is not alone in requiring clinical experience for a student’s nursing education. In fact, 42 state boards of nursing require nursing programs to include clinical experience for their students. However, only 12 states have a required number of clinical hours. Clinical placement slots are a limited resource. Not all clinical facilities have the capacity or the desire to offer placement slots. The number of clinical placement slots available to a program can constrain the number of students the governing board will allow the nursing program to enroll. Clinical displacement occurs when a program loses placement slots that it is currently using to provide required clinical experience to students because a clinical facility decides to discontinue those placements for some reason. Although clinical displacement can happen for several reasons, including a change in facility staffing levels or emergency situations, such as the COVID‑19 pandemic in spring 2020, perhaps the reason of most 12 California State Auditor Report 2019-120 July 2020 interest to BRN occurs when students are displaced because other nursing programs took their clinical spots. When displacement occurs, the nursing program losing placement slots must find new placement slots for its displaced students, either on a different shift in the same facility or at another facility, in order to provide the required clinical experience to its students. This can be disruptive to nursing students and may hinder their ability to complete their required clinical experience. As a possible approach to alleviating some of the enrollment constraint caused by limited clinical placement slots, nursing programs and other stakeholders in health care and government have sought to increase the portion of clinical experience hours that students can fulfill through simulation labs. Simulation is an activity or event replicating clinical practice using scenarios, high‑fidelity manikins, standardized patients, role playing, skills stations, and computer‑based critical thinking simulations. State law allows students to meet their clinical experience requirements with up to 25 percent indirect patient care, which includes simulation labs. However, in response to the COVID‑19 pandemic, Consumer Affairs issued a waiver on April 3, 2020, that allowed nursing students to complete their clinical experience with up to 50 percent indirect patient care, which could include simulation labs. Consumer Affairs set this waiver to expire after 60 days and then extended the expiration date to August 1, 2020. Although the scope of this audit did not include an evaluation of simulation labs as a reasonable substitute for in‑person clinical experience, we believe it is an area that could be considered as an approach to alleviating the constraint that the requirement for in‑person clinical placements might have on nursing programs’ ability to enroll more students. Concerns Among Nursing Programs and Other Stakeholders Stakeholders have called into question certain aspects of BRN’s authority to make enrollment decisions and whether portions of BRN’s director’s handbook constitute underground regulations. For example, in October 2018 the California Association of Private Postsecondary Schools petitioned the Office of Administrative Law (OAL) asserting that BRN had no legal authority to restrain the enrollment levels of approved nursing programs, that BRN’s exercise of this authority was based on certain guidelines in BRN’s director’s handbook that BRN had issued without complying with state law, and that these guidelines constituted an underground regulation. If a state agency issues, uses, enforces, or attempts to enforce a guideline or other rule without following the Administrative Procedure Act when it is required to do so, the rule is called an “underground regulation.” State law prohibits state agencies from enforcing guidelines or rules that constitute underground regulations. If a party believes a state agency has issued California State Auditor Report 2019-120 13 July 2020 an underground regulation, that party may submit a petition to OAL seeking a determination of whether that guideline or rule is an underground regulation. Because BRN certified to OAL that it would no longer use or enforce the guidelines in question, OAL suspended the review it had initiated of the petition mentioned above. In July 2019, West Coast University also filed a petition with OAL claiming that BRN was continuing to use and enforce some of the guidelines in question despite certifying to OAL that it would not. However, because BRN had already filed the certification stating it would not enforce the guidelines, and because a nursing program filed a lawsuit related to the guidelines in April 2019, OAL declined to take action on the matter in accordance with its regulations. OAL’s director stated that OAL is considering amending its regulations to allow for it to continue its inquiry and make a determination in cases in which an agency or department has filed such a certification, but parties assert that the department or agency is continuing to use and enforce underground regulations. In addition, American Career College (ACC), a Los Angeles private college that offers nursing associate’s degrees, filed a lawsuit in April 2019 asking the court to find that BRN does not have the authority, power, or purview to determine the total number of nursing students that ACC may enroll. BRN has opposed the lawsuit because it believes it is authorized to regulate the number of students a nursing program is permitted to enroll. As the question of whether BRN has authority to make enrollment decisions regarding the number of permitted enrollments had been brought before the court, we made no such determination in this report regarding this issue because audit standards prohibit us from doing so. Instead, our report focuses on the actions BRN has taken in the recent past. Additionally, in September and October 2018, multiple stakeholders from academia, health care providers, labor groups, and government participated in seven regional summit meetings (stakeholder summits) at different locations across California to discuss issues surrounding clinical education capacity, particularly the availability of clinical placements for nursing students. The resulting report identified six priorities for action that all seven regions agreed upon. Five of these priorities are related to clinical experience or placements: • Seek to standardize requirements for nursing curricula, credits, and clinical hours. • Encourage nursing programs and clinical facilities to participate in groups, consortiums, and scheduling systems related to clinical placements. 14 California State Auditor Report 2019-120 July 2020 • Seek to standardize the requirements for licensing and accreditation of clinical facilities, as well as the onboarding and orientation process for students and faculty. • Facilitate increased use of nonacute, community‑based, and ambulatory clinical sites statewide. • Seek to enable students to use simulation for up to 50 percent of their clinical practice requirements. The sixth priority involved establishing structures to encourage communication, collaboration, cooperation, and decision making among senior‑level nursing program and clinical facility staff. Recent Developments Prior to the completion of this audit, the California State Auditor (State Auditor) received a whistleblower complaint alleging that BRN executives in the enforcement division intentionally manipulated data and delivered a falsified report to the State Auditor to satisfy a recommendation the State Auditor had made during a 2016 audit of the enforcement division. In response to the complaint, the State Auditor launched an investigation and substantiated that BRN executives violated state law when they carried out a plan to artificially decrease caseloads for BRN investigators before delivering a falsified report to the State Auditor. The plan involved temporarily reassigning some of the BRN investigators’ cases to other employees who should not have had cases assigned to them. The investigation found that within 10 days of the State Auditor reviewing the falsified report and concluding that BRN had fully implemented the recommendation, BRN managers reversed the reassignments, increasing caseloads to their original level. A copy of investigative report I2020‑0027, Board of Registered Nursing: Executives Violated State Law When They Falsified Data to Deceive the State Auditor’s Office, can be found on our website at www.auditor.ca.gov. The audit team became aware of the investigation during this audit and re‑evaluated the risk assessment it conducted for the audit to ensure it could rely upon the documentation provided by BRN for this audit report. We determined that the documentation we obtained was reliable. California State Auditor Report 2019-120 15 July 2020 Audit Results BRN’s Forecasts of the Supply of Qualified Nurses Have Not Included Key Information An adequate supply of nurses is critical to health care. BRN has an impact on the supply of nurses through its enrollment decisions, putting it in the unique position of being able to directly respond to and mitigate nursing shortages. BRN’s contractor explains in its 2017 forecast that nursing shortages generate significant challenges because the level of nurse staffing in hospitals and other care facilities can affect patient outcomes.3 As described in the Introduction, state law requires BRN to analyze data and produce reports on the nursing workforce in California to help researchers and policymakers find solutions to nursing shortages. However, the conclusion from BRN’s 2017 forecast that supply is adequate is inconsistent with other similar studies. This inconsistency has caused some confusion about whether the State will experience a nursing shortage. BRN’s forecast includes high and low estimates of supply and demand, but it indicates that the supply of and demand for RNs will be fairly well balanced across the State over the next 10 years, if current enrollment patterns and migration patterns of nurses into and out of the State remain stable. In contrast, various other studies and reports on the nursing workforce in California project a nursing shortage in the State or in areas within the State, although the studies differ as to the magnitude of the projected shortages. In particular, the projected statewide shortages range from none at all, according to BRN’s 2017 forecast, to a shortage of approximately 141,000 nurses by 2030, according to “United States Registered Nurse Workforce Report Card and Shortage Forecast: A Revisit” (RN Workforce Report Card), a study published in the May/June 2018 issue of the American Journal of Medical Quality. Table 2 shows five recent studies we identified and the key differences among them, such as their scope and how they measured supply and demand, that likely contributed to the different projections. The methodology that BRN’s contractor used in its 2017 forecast is reasonable, but BRN could have asked for a more robust analysis. The contractor measured the supply of nurses statewide by reviewing the number of RNs entering, departing, and choosing to participate in the workforce. Specifically, the contractor considered factors such as the number of newly graduated nurses, the 3 BRN published its more recent 2019 forecast in May 2020, near the completion of our audit. Therefore we refer to conclusions cited in the 2017 forecast. The 2017 and 2019 forecasts are largely similar in their scope and methodology. The 2019 forecast projected that a small surplus of RNs statewide could emerge in the future. 16 California State Auditor Report 2019-120 July 2020 migration of nurses to and from other states, and the number of RNs with active licenses in the State. In fact, the model that BRN’s contractor used to measure supply is similar to those used in other health care studies that we identified. Table 2 BRN’s Workforce Study Does Not Account for Regional Differences PUBLISHING DATE TIME STUDY SCOPE SUPPLY MODEL DEMAND MODEL CONCLUSION ENTITY RELEASED FRAME Forecasts of Estimated the number Estimated future the Registered of RNs entering, demand based on Supply and 2017 to Nurse UCSF for BRN June 2017 Statewide departing, and choosing current hospital demand are 2035 Workforce in to participate in the utilization and balanced California* workforce staffing patterns† Regional Estimated the number Estimated future Forecasts of of RNs entering, demand based on Large differences Healthforce December 2018 to the Registered Regional departing, and choosing current hospital across regions of Center at UCSF 2018 2035 Nurse Workforce to participate in the utilization and staffing the State. in California workforce patterns† United States Estimated the number Estimated number of Registered National of individuals in a region jobs needed to meet Nurse Workforce study that or state who are likely to population needs American Journal 2016 to Shortage of Report Card May 2018 provided work as a nurse based on based on the 2015 of Medical Quality 2030 141,348 nurses and Shortage statewide estimated populations national mean average Forecast: information over a 10‑year period of jobs per 100,000 A Revisit (2006 to 2015) people Estimated number Estimated the number of jobs needed to Supply and National of RNs entering, provide a level of Demand U.S. Department of study that 2014 to departing, and choosing care consistent Shortage of Projections of Health and Human July 2017 provided 2030 to participate in the with the baseline 44,500 nurses the Nursing Services statewide workforce and other year—2014—based Workforce information factors, such as wage rates on hospital utilization and staffing patterns California’s Office Registered of Statewide Actual number of Actual current hospital 28 counties are Nurse Shortage Health Planning June 2019 2017 County registered nurses in a and long‑term care RN shortage Areas Update and Development county facility utilization areas (OSHPD) Source: Studies as listed in table. * BRN published its 2019 forecast in May 2020, near the completion of our audit. The 2017 and 2019 forecasts are largely similar. In its 2019 version, BRN again reported a forecast of the nursing workforce on a statewide basis that did not include a regional analysis. It also generally used the same methodology as its 2017 forecast and projected that a small surplus of RNs statewide could emerge in the future. † OSHPD data was used to create these demand models. Similarly, the contractor’s method for measuring demand is generally reasonable. Specifically, it identified the demand for nurses at hospitals and other health care facilities in California by reviewing the staffing patterns of RNs—in particular, the number of RN hours worked per day that a patient was in the hospital (patient day)—and data on hospital usage. BRN’s contractor also California State Auditor Report 2019-120 17 July 2020 considered information that state law requires BRN to analyze, such as the number of RN hours worked, age‑specific demographics, and number of patient days. These factors are different from those used in the RN Workforce Report Card study, which defines RN demand as the estimated number of RN jobs needed to meet population needs. The section of law that requires BRN to analyze workforce data does not require BRN to collect and analyze information on the health care needs of California residents or the number of health care facilities that exist in California. The 2017 forecast has a limitation that it acknowledged: it represents the State as a whole and does not reflect the fact that one region of California may experience a shortage while another faces a surplus Because BRN’s forecast does not of RNs. Because BRN’s forecast does not measure regional variations measure regional variations in in supply and demand, it obscures regional shortages that currently supply and demand, it obscures exist and those projected to exist in the future. Thus, BRN’s forecast regional shortages that currently does not provide information that would help it respond to and exist and those projected to exist in mitigate regional nursing shortages. the future. BRN can influence the supply of nurses through its enrollment decisions. In fact, BRN’s contractor recommends in its 2017 forecast that policymakers continuously monitor factors that could influence regional shortages, such as the number of graduates from RN education programs and the interstate migration of nurses. According to BRN’s 2017 forecast, the solution to a nursing shortage in 2005 was in part to increase the number of graduates from California nursing programs, which led to a stable workforce. Additionally, the forecast indicates that if future numbers of student enrollments and graduates decline, a shortage could reemerge. Given the size and diversity of California, we believe a regional forecast would provide critical information to inform the governing board’s enrollment decisions and other actions to address identified shortages. BRN officials agreed that a regional analysis would provide valuable information. Only two of the five studies we reviewed measured shortages on a more local level. Specifically, the 2018 Regional Forecasts of the Registered Nurse Workforce in California (2018 regional forecast) by the Healthforce Center at UCSF, and a 2019 report by OSHPD titled Registered Nurse Shortage Area Update (OSHPD report) employ a more localized analysis. In fact, the 2018 regional forecast, which was prepared by the same entity with which BRN contracts for its forecast and, using generally the same method for measuring supply and demand, identified and measured regional differences in the need for RNs within California. The 2018 regional forecast concludes that all regions except the Central Coast appear to have had nursing shortages that year and that by 2035 the Central Valley, Central Coast, and San Francisco Bay Area will experience or continue to experience nursing shortages. Figure 2 shows the counties that are 18 California State Auditor Report 2019-120 July 2020 included in each of the eight regions defined in the 2018 regional forecast and indicates whether the regional forecast projects a shortage, a surplus, or balanced supply and demand for each region in 2035. Similarly, the OSHPD report used patient day data and BRN’s active nurse licensee data from 2017 to classify 28 counties as having had a shortage of RNs in that year. Figure 2 Some Regional Nursing Shortages Are Projected to Continue Within California 2018 2035 (cid:8)(cid:29)(cid:28)(cid:27)(cid:30)(cid:24)(cid:28)(cid:21)(cid:23)(cid:2)(cid:26)(cid:17)(cid:22)(cid:1)(cid:29)(cid:28)(cid:21)(cid:22)(cid:26) -6.6% +38.2% (cid:31)(cid:26)(cid:13)(cid:28)(cid:26)(cid:7)(cid:24)(cid:21)(cid:27)(cid:29) -15% +23.6% (cid:31)(cid:26)(cid:21)(cid:23) (cid:28)(cid:26)(cid:21)(cid:13)(cid:22)(cid:20)(cid:13)(cid:29) (cid:143)(cid:26)(cid:6)(cid:23)(cid:157)(cid:28)(cid:24)(cid:26) -10.9% -6.4% (cid:2)(cid:24)(cid:21)(cid:27)(cid:28)(cid:26)(cid:17)(cid:23)(cid:127)(cid:26)(cid:17)(cid:17)(cid:24)(cid:6) -17.3% -19.5% -11.1% (cid:2)(cid:24)(cid:21)(cid:27)(cid:28)(cid:26)(cid:17)(cid:23)(cid:2)(cid:29)(cid:26)(cid:20)(cid:27) (cid:129)(cid:21)(cid:17)(cid:26)(cid:21)(cid:12)(cid:23)(cid:141)(cid:7)(cid:18)(cid:22)(cid:28)(cid:24) -23.4% -10.4% +17% (cid:144)(cid:29)(cid:20)(cid:23)(cid:157)(cid:21)(cid:25)(cid:24)(cid:17)(cid:24)(cid:20) -9.1% (cid:31)(cid:29)(cid:19)(cid:27)(cid:30)(cid:24)(cid:28)(cid:21)(cid:23)(cid:143)(cid:29)(cid:28)(cid:12)(cid:24)(cid:28) (cid:31)(cid:30)(cid:29)(cid:28)(cid:27)(cid:26)(cid:25)(cid:24)(cid:23)(cid:28)(cid:24)(cid:25)(cid:22)(cid:29)(cid:21)(cid:20) x.X% (cid:11)(cid:24)(cid:28)(cid:13)(cid:24)(cid:21)(cid:27)(cid:26)(cid:25)(cid:24)(cid:20)(cid:23)(cid:20)(cid:30)(cid:29)(cid:10)(cid:23)(cid:27)(cid:30)(cid:24)(cid:23)(cid:24)(cid:9)(cid:27)(cid:24)(cid:21)(cid:27)(cid:23)(cid:27)(cid:29)(cid:23) (cid:31)(cid:19)(cid:28)(cid:18)(cid:17)(cid:19)(cid:20)(cid:23)(cid:28)(cid:24)(cid:25)(cid:22)(cid:29)(cid:21)(cid:20) (cid:10)(cid:30)(cid:22)(cid:13)(cid:30)(cid:23)(cid:16)(cid:8)(cid:23)(cid:12)(cid:24)(cid:7)(cid:26)(cid:21)(cid:12)(cid:23)(cid:24)(cid:9)(cid:13)(cid:24)(cid:24)(cid:12)(cid:20)(cid:23) (cid:16)(cid:24)(cid:25)(cid:22)(cid:29)(cid:21)(cid:20)(cid:23)(cid:22)(cid:21)(cid:23)(cid:28)(cid:24)(cid:17)(cid:26)(cid:27)(cid:22)(cid:15)(cid:24)(cid:23)(cid:14)(cid:26)(cid:17)(cid:26)(cid:21)(cid:13)(cid:24) (cid:20)(cid:19)(cid:18)(cid:18)(cid:17)(cid:6)(cid:23)(cid:5)(cid:20)(cid:30)(cid:29)(cid:28)(cid:27)(cid:26)(cid:25)(cid:24)(cid:4)(cid:23)(cid:29)(cid:28)(cid:23)(cid:16)(cid:8)(cid:23)(cid:20)(cid:19)(cid:18)(cid:18)(cid:17)(cid:6)(cid:23) (cid:16)(cid:24)(cid:25)(cid:22)(cid:29)(cid:21)(cid:26)(cid:17)(cid:23)(cid:14)(cid:29)(cid:28)(cid:12)(cid:24)(cid:28) (cid:24)(cid:9)(cid:13)(cid:24)(cid:24)(cid:12)(cid:20)(cid:23)(cid:12)(cid:24)(cid:7)(cid:26)(cid:21)(cid:12)(cid:23)(cid:5)(cid:20)(cid:19)(cid:28)(cid:18)(cid:17)(cid:19)(cid:20)(cid:4)(cid:3) Source: Analysis of UCSF’s 2018 Regional Forecasts of the Registered Nurse Workforce in California. Note: The supply and demand numbers for the regions include adjustments to account for RNs commuting between regions, advanced‑practice RNs not working in RN jobs, and the number of RN hours worked by contract staff at hospitals. If BRN’s forecast identified regional shortages and surpluses, it would be able to provide the governing board better information to consider the reasons that nursing programs assert for expanding their programs. We reviewed governing board meeting minutes and corresponding materials between 2017 and 2019 and found that California State Auditor Report 2019-120 19 July 2020 18 of the 35 requests from nursing programs to increase enrollment or open a new nursing program cited nursing shortages as a reason for requesting an enrollment increase. For example, in a June 2019 letter to BRN, Unitek College provided additional information to BRN about its proposal to start a registered nursing program at its Bakersfield campus. Unitek College cited community nursing workforce shortages and data from the 2018 regional forecast on the migration of RNs out of the Central Valley region as causes for concern. However, BRN’s forecast did not include relevant regional information that would allow its nursing education staff to verify those assertions. BRN officials stated that if BRN’s forecast identified more specific and concrete data on regional shortages, it would give the governing board better information to consider the assertions that nursing programs make for expanding their programs, such as nursing shortages that exist in their areas. Regularly collecting information on California’s regional nursing Regularly collecting information workforce would also give BRN the information it needs to on California’s regional nursing identify shortage areas and take action to mitigate those shortages. workforce would also give BRN the The Nursing Practice Act does not require BRN to address any information it needs to identify identified shortages. However, BRN’s mission, in part, is to advocate shortage areas and take action to for the health and safety of the public. As part of this advocacy, mitigate those shortages. BRN should develop a plan to support increases in enrollment at existing nursing programs or new programs in areas with shortages, such as providing programs with information that they could use to identify additional clinical placements, as we discuss later. BRN’s Process for Assessing the Availability of Clinical Placements Is Inadequate The number of available clinical placement slots affects the number of student enrollments the governing board should approve and the eventual supply of nurses in the State. This information is also crucial to understanding the risk of clinical displacement. However, BRN does not track or consistently report this information to its governing board. In fact, it has not established what information its nursing education staff must provide to the governing board when it is considering enrollment decisions. We found that nursing education staff provided inconsistent information to the governing board, hampering its ability to properly gauge the risk that its decisions might displace students from their clinical placement slots. If BRN augmented information it collects about the number of clinical placement slots at facilities and stored that information in a database, it could better analyze the data and present to the governing board more robust and objective information to consider in making its enrollment decisions. Additionally, BRN could 20 California State Auditor Report 2019-120 July 2020 compare the facility information in its database with OSHPD’s health care facility data to identify additional facilities with potential clinical placement slots. BRN Uses Inconsistent and Incomplete Information to Assess Whether an Adequate Number of Clinical Placement Slots Is Available Another key factor that should influence the governing board’s enrollment decisions is the availability of clinical placement slots. Because the availability of clinical placement slots has an impact on the number of student enrollments the governing board should approve for a nursing program and the eventual supply of nurses in the State, having this key information is crucial for the board. However, BRN has not established a policy for its nursing education staff members that specifies the information they must provide to the governing board for each enrollment decision, such as the number of available clinical placement slots in a facility where a program plans to place students. We found that, for the 15 enrollment decisions made between January 2015 and September 2019 we reviewed (five requests for new nursing programs and 10 requests for enrollment increases at existing programs), nursing education staff did not consistently present to the governing board the information that nursing programs must submit regarding clinical placements, as Figure 3 shows. Specifically, for eight of the 15 decisions, nursing education staff did not present all the clinical placement information that nursing programs must provide. For example, for the five requests Nursing education staff did not for new programs, nursing education staff did not present present information about the information about the number of students the programs intended number of students the programs to have in classroom nursing courses or the facilities they planned intended to have in classroom to use for the associated clinical experiences. Consequently, the nursing courses or the facilities they governing board could not properly assess the risk of clinical planned to use for the associated displacement for these programs. Nevertheless, the governing clinical experiences. board approved all but one of the requests. To help ensure that the governing board bases enrollment decisions on complete and consistent information in the future, BRN should establish a uniform format and structure for information that nursing education staff must provide to the governing board for each enrollment decision. California State Auditor Report 2019-120 21 July 2020 Figure 3 BRN’s Lack of Guidance Results in Staff Presenting Inconsistent Information to the Governing Board Nursing Programs (cid:31)(cid:30)(cid:29)(cid:28)(cid:27)(cid:26)(cid:25)(cid:24)(cid:23)(cid:27)(cid:26)(cid:22)(cid:28)(cid:21)(cid:24)(cid:27)(cid:22)(cid:23)(cid:21)(cid:20)(cid:28)(cid:19)(cid:27)(cid:18)(cid:20)(cid:27)(cid:17)(cid:16)(cid:22)(cid:28)(cid:15)(cid:18)(cid:28)(cid:22)(cid:24)(cid:14)(cid:16)(cid:23)(cid:15)(cid:28)(cid:13)(cid:26)(cid:27)(cid:15)(cid:17)(cid:23)(cid:21)(cid:28)(cid:23)(cid:21)(cid:12)(cid:18)(cid:27)(cid:16)(cid:17)(cid:15)(cid:23)(cid:18)(cid:21)(cid:28)(cid:17)(cid:14)(cid:18)(cid:24)(cid:15)(cid:28)(cid:15)(cid:11)(cid:26) (cid:17)(cid:10)(cid:17)(cid:23)(cid:9)(cid:17)(cid:14)(cid:23)(cid:9)(cid:23)(cid:15)(cid:8)(cid:28)(cid:18)(cid:12)(cid:28)(cid:13)(cid:9)(cid:23)(cid:21)(cid:23)(cid:13)(cid:17)(cid:9)(cid:28)(cid:19)(cid:9)(cid:17)(cid:13)(cid:26)(cid:16)(cid:26)(cid:21)(cid:15)(cid:22)(cid:28)(cid:15)(cid:18)(cid:28)(cid:21)(cid:24)(cid:27)(cid:22)(cid:23)(cid:21)(cid:20)(cid:28)(cid:26)(cid:7)(cid:24)(cid:13)(cid:17)(cid:15)(cid:23)(cid:18)(cid:21)(cid:28)(cid:22)(cid:15)(cid:17)(cid:6)(cid:28)(cid:5)(cid:11)(cid:26)(cid:21)(cid:28)(cid:27)(cid:26)(cid:25)(cid:24)(cid:26)(cid:22)(cid:15)(cid:23)(cid:21)(cid:20) (cid:17)(cid:28)(cid:21)(cid:26)(cid:5)(cid:28)(cid:19)(cid:27)(cid:18)(cid:20)(cid:27)(cid:17)(cid:16)(cid:28)(cid:18)(cid:27)(cid:28)(cid:26)(cid:21)(cid:27)(cid:18)(cid:9)(cid:9)(cid:16)(cid:26)(cid:21)(cid:15)(cid:28)(cid:23)(cid:21)(cid:13)(cid:27)(cid:26)(cid:17)(cid:22)(cid:26)(cid:4) (cid:3)(cid:29)(cid:30)(cid:2)(cid:1)(cid:1)(cid:127)(cid:3)(cid:29)(cid:129)(cid:28)(cid:141)(cid:29)(cid:143)(cid:30)(cid:3)(cid:144)(cid:157)(cid:3) (cid:29)(cid:3) (cid:28) (cid:30)(cid:2)€(cid:30)(cid:144)(cid:127) (cid:31)(cid:30)(cid:29)(cid:28)(cid:27)(cid:26)(cid:25)(cid:24)(cid:26)(cid:23)(cid:22)(cid:21)(cid:23)(cid:20)(cid:21)(cid:19)(cid:23)(cid:18)(cid:21)(cid:24)(cid:19)(cid:30)(cid:21) (cid:11)(cid:14)(cid:28)(cid:26)(cid:16)(cid:26)(cid:24)(cid:10)(cid:21)(cid:14)(cid:22)(cid:17)(cid:21)(cid:25)(cid:27)(cid:23)(cid:9)(cid:27)(cid:14)(cid:15)(cid:21) (cid:6)(cid:7)(cid:15)(cid:5)(cid:30)(cid:27)(cid:21)(cid:23)(cid:20)(cid:21)(cid:29)(cid:24)(cid:7)(cid:17)(cid:30)(cid:22)(cid:24)(cid:29)(cid:21)(cid:26)(cid:22)(cid:21)(cid:30)(cid:14)(cid:28)(cid:19)(cid:21) (cid:25)(cid:27)(cid:23)(cid:25)(cid:23)(cid:29)(cid:30)(cid:17)(cid:21)(cid:30)(cid:22)(cid:27)(cid:23)(cid:16)(cid:16)(cid:15)(cid:30)(cid:22)(cid:24)(cid:21)(cid:26)(cid:22)(cid:28)(cid:27)(cid:30)(cid:14)(cid:29)(cid:30)(cid:21) (cid:8)(cid:30)(cid:27)(cid:26)(cid:20)(cid:26)(cid:28)(cid:14)(cid:24)(cid:26)(cid:23)(cid:22)(cid:29)(cid:21)(cid:24)(cid:19)(cid:14)(cid:24)(cid:21)(cid:22)(cid:23)(cid:21)(cid:28)(cid:16)(cid:26)(cid:22)(cid:26)(cid:28)(cid:14)(cid:16)(cid:21) (cid:22)(cid:7)(cid:27)(cid:29)(cid:26)(cid:22)(cid:9)(cid:21)(cid:28)(cid:23)(cid:7)(cid:27)(cid:29)(cid:30)(cid:21)(cid:14)(cid:22)(cid:17)(cid:21)(cid:24)(cid:19)(cid:30)(cid:21) (cid:18)(cid:26)(cid:16)(cid:16)(cid:21)(cid:14)(cid:13)(cid:30)(cid:28)(cid:24)(cid:21)(cid:28)(cid:16)(cid:26)(cid:22)(cid:26)(cid:28)(cid:14)(cid:16)(cid:21)(cid:20)(cid:14)(cid:28)(cid:26)(cid:16)(cid:26)(cid:24)(cid:26)(cid:30)(cid:29)(cid:12) (cid:17)(cid:26)(cid:29)(cid:25)(cid:16)(cid:14)(cid:28)(cid:30)(cid:15)(cid:30)(cid:22)(cid:24)(cid:21)(cid:18)(cid:26)(cid:16)(cid:16)(cid:21)(cid:23)(cid:28)(cid:28)(cid:7)(cid:27)(cid:12) (cid:20)(cid:14)(cid:28)(cid:26)(cid:16)(cid:26)(cid:24)(cid:26)(cid:30)(cid:29)(cid:21)(cid:7)(cid:29)(cid:30)(cid:17)(cid:21)(cid:20)(cid:23)(cid:27)(cid:21)(cid:24)(cid:19)(cid:30)(cid:21) (cid:14)(cid:29)(cid:29)(cid:23)(cid:28)(cid:26)(cid:14)(cid:24)(cid:30)(cid:17)(cid:21)(cid:28)(cid:16)(cid:26)(cid:22)(cid:26)(cid:28)(cid:14)(cid:16)(cid:21)(cid:30)(cid:4)(cid:25)(cid:30)(cid:27)(cid:26)(cid:30)(cid:22)(cid:28)(cid:30)(cid:12) BRN Education staff (cid:31)(cid:30)(cid:29)(cid:28)(cid:7)(cid:18)(cid:26)(cid:22)(cid:28)(cid:21)(cid:18)(cid:15)(cid:28)(cid:11)(cid:17)(cid:10)(cid:26)(cid:28)(cid:19)(cid:18)(cid:9)(cid:23)(cid:13)(cid:23)(cid:26)(cid:22)(cid:28)(cid:15)(cid:11)(cid:17)(cid:15)(cid:28)(cid:20)(cid:18)(cid:10)(cid:26)(cid:27)(cid:21)(cid:28)(cid:5)(cid:11)(cid:17)(cid:15)(cid:28)(cid:26)(cid:7)(cid:24)(cid:13)(cid:17)(cid:15)(cid:23)(cid:18)(cid:21)(cid:28)(cid:22)(cid:15)(cid:17)(cid:6)(cid:28)(cid:16)(cid:24)(cid:22)(cid:15)(cid:28)(cid:22)(cid:24)(cid:14)(cid:16)(cid:23)(cid:15) (cid:15)(cid:18)(cid:28)(cid:15)(cid:11)(cid:26)(cid:28)(cid:20)(cid:18)(cid:10)(cid:26)(cid:27)(cid:21)(cid:23)(cid:21)(cid:20)(cid:28)(cid:14)(cid:18)(cid:17)(cid:27)(cid:7)(cid:28)(cid:27)(cid:26)(cid:20)(cid:17)(cid:27)(cid:7)(cid:23)(cid:21)(cid:20)(cid:28)(cid:19)(cid:26)(cid:21)(cid:7)(cid:23)(cid:21)(cid:20)(cid:28)(cid:26)(cid:21)(cid:27)(cid:18)(cid:9)(cid:9)(cid:16)(cid:26)(cid:21)(cid:15)(cid:28)(cid:7)(cid:26)(cid:13)(cid:23)(cid:22)(cid:23)(cid:18)(cid:21)(cid:22)(cid:4) (cid:31)(cid:30)(cid:29)(cid:28)(cid:27)(cid:26)(cid:25)(cid:24)(cid:23)(cid:22)(cid:30)(cid:21)(cid:20)(cid:19)(cid:27)(cid:18)(cid:26)(cid:24)(cid:28)(cid:19)(cid:20)(cid:17)(cid:24)(cid:27)(cid:26)(cid:21)(cid:18)(cid:26)(cid:28)(cid:27)(cid:28)(cid:19)(cid:23)(cid:26)(cid:19)(cid:16)(cid:15)(cid:24) (cid:14)(cid:29)(cid:23)(cid:28)(cid:23)(cid:26)(cid:19)(cid:23)(cid:22)(cid:24)(cid:27)(cid:26)(cid:13)(cid:18)(cid:29)(cid:12)(cid:20)(cid:19)(cid:27)(cid:18)(cid:26)(cid:24)(cid:19)(cid:18)(cid:24)(cid:19)(cid:11)(cid:23)(cid:24)(cid:25)(cid:18)(cid:10)(cid:23)(cid:29)(cid:26)(cid:27)(cid:26)(cid:25)(cid:24) (cid:9)(cid:18)(cid:20)(cid:29)(cid:22)(cid:8)(cid:24)(cid:7)(cid:18)(cid:29)(cid:24)(cid:23)(cid:27)(cid:25)(cid:11)(cid:19)(cid:24)(cid:18)(cid:13)(cid:24)(cid:19)(cid:11)(cid:23)(cid:24)(cid:6)(cid:5)(cid:24)(cid:22)(cid:23)(cid:21)(cid:27)(cid:28)(cid:27)(cid:18)(cid:26)(cid:28)(cid:24)(cid:12)(cid:20)(cid:22)(cid:23)(cid:24) (cid:9)(cid:23)(cid:19)(cid:4)(cid:23)(cid:23)(cid:26)(cid:24)(cid:3)(cid:20)(cid:26)(cid:30)(cid:20)(cid:29)(cid:15)(cid:24)(cid:2)(cid:1)(cid:6)(cid:5)(cid:24)(cid:20)(cid:26)(cid:22)(cid:24)(cid:127)(cid:23)(cid:14)(cid:19)(cid:23)(cid:12)(cid:9)(cid:23)(cid:29)(cid:24)(cid:2)(cid:1)(cid:6)(cid:129)(cid:24) (cid:4)(cid:23)(cid:24)(cid:29)(cid:23)(cid:10)(cid:27)(cid:23)(cid:4)(cid:23)(cid:22)(cid:141)(cid:24)(cid:26)(cid:30)(cid:29)(cid:28)(cid:27)(cid:26)(cid:25)(cid:24)(cid:23)(cid:22)(cid:30)(cid:21)(cid:20)(cid:19)(cid:27)(cid:18)(cid:26)(cid:24)(cid:28)(cid:19)(cid:20)(cid:17)(cid:24)(cid:22)(cid:27)(cid:22)(cid:24)(cid:26)(cid:18)(cid:19)(cid:24) (cid:14)(cid:29)(cid:23)(cid:28)(cid:23)(cid:26)(cid:19)(cid:24)(cid:19)(cid:11)(cid:23)(cid:24)(cid:21)(cid:16)(cid:27)(cid:26)(cid:27)(cid:21)(cid:20)(cid:16)(cid:24)(cid:14)(cid:16)(cid:20)(cid:21)(cid:23)(cid:12)(cid:23)(cid:26)(cid:19)(cid:24)(cid:27)(cid:26)(cid:13)(cid:18)(cid:29)(cid:12)(cid:20)(cid:19)(cid:27)(cid:18)(cid:26)(cid:24) (cid:26)(cid:30)(cid:29)(cid:28)(cid:27)(cid:26)(cid:25)(cid:24)(cid:14)(cid:29)(cid:18)(cid:25)(cid:29)(cid:20)(cid:12)(cid:28)(cid:24)(cid:12)(cid:30)(cid:28)(cid:19)(cid:24)(cid:14)(cid:29)(cid:18)(cid:10)(cid:27)(cid:22)(cid:23)(cid:8) BRN governing board Source: Analysis of state law, governing board meeting minutes, materials, and BRN’s director’s handbook. 22 California State Auditor Report 2019-120 July 2020 One possible unintended consequence of BRN’s enrollment decisions is the clinical displacement of students. Since at least 2009, BRN has been performing an annual survey of schools with nursing programs, a portion of which relates to clinical displacement. It asks responding nursing programs whether in the past year they lost clinical placement slots, how many students were affected, and the perceived reason that clinical placement slots were not available. BRN publishes the annual survey report on its website. As Figure 4 demonstrates, nursing programs reported in the most recent survey that more than 2,300 students were affected by a loss of clinical placement slots in academic year 2017–18—an amount generally similar to previous years. Most notably, nearly half of the nursing programs that lost a clinical placement reported that it occurred because other nursing programs took their clinical spots. Figure 4 Summary of Survey Responses Related to Clinical Displacement 75 140 of (cid:31)(cid:30)(cid:29)(cid:28)(cid:27)(cid:26)(cid:25)(cid:24)(cid:23) nursing programs 2,300 nursing students affected (cid:29)(cid:28)(cid:22)(cid:30)(cid:29)(cid:26)(cid:28)(cid:21)(cid:27)(cid:20)(cid:19)(cid:18)(cid:23)(cid:18)(cid:20)(cid:24)(cid:19)(cid:27)(cid:21)(cid:18)(cid:17)(cid:22)(cid:19)(cid:24)(cid:20)(cid:28)(cid:16)(cid:28)(cid:23)(cid:26) (cid:31)(cid:30)(cid:29)(cid:28)(cid:27)(cid:26)(cid:25)(cid:30)(cid:24)(cid:23)(cid:27)(cid:29)(cid:22)(cid:24)(cid:21)(cid:20)(cid:19)(cid:18)(cid:17)(cid:19)(cid:16) Examples of reasons programs lost clinical placement* 37 programs 47 programs 17 programs (cid:24)(cid:23)(cid:30)(cid:26)(cid:25)(cid:28)(cid:29)(cid:27)(cid:22)(cid:29)(cid:30)(cid:15)(cid:29)(cid:24)(cid:16) (cid:14)(cid:24)(cid:20)(cid:18)(cid:19)(cid:18)(cid:26)(cid:13)(cid:27)(cid:17)(cid:26)(cid:24)(cid:12)(cid:27)(cid:11)(cid:30)(cid:29)(cid:10)(cid:27)(cid:30)(cid:9)(cid:28)(cid:29)(cid:19)(cid:30)(cid:24)(cid:21) (cid:20)(cid:19)(cid:30)(cid:17)(cid:8)(cid:29)(cid:28)(cid:27)(cid:30)(cid:29)(cid:27)(cid:22)(cid:24)(cid:29)(cid:26)(cid:18)(cid:24)(cid:19)(cid:27)(cid:20)(cid:19)(cid:30)(cid:17)(cid:8)(cid:29)(cid:28) (cid:21)(cid:18)(cid:17)(cid:22)(cid:19)(cid:24)(cid:20)(cid:18)(cid:23)(cid:15)(cid:27)(cid:26)(cid:25)(cid:28)(cid:16) (cid:30)(cid:29)(cid:27)(cid:18)(cid:23)(cid:17)(cid:8)(cid:12)(cid:18)(cid:20)(cid:18)(cid:28)(cid:23)(cid:26)(cid:27)(cid:7)(cid:8)(cid:24)(cid:19)(cid:18)(cid:14)(cid:18)(cid:28)(cid:21) (cid:30)(cid:14)(cid:27)(cid:20)(cid:19)(cid:18)(cid:23)(cid:18)(cid:20)(cid:24)(cid:19)(cid:27)(cid:14)(cid:24)(cid:20)(cid:18)(cid:19)(cid:18)(cid:26)(cid:18)(cid:28)(cid:17) (cid:17)(cid:26)(cid:24)(cid:12)(cid:27)(cid:24)(cid:26)(cid:27)(cid:14)(cid:24)(cid:20)(cid:18)(cid:19)(cid:18)(cid:26)(cid:18)(cid:28)(cid:17) Source: BRN’s 2017–18 Annual School Report. * Nursing programs can report more than one reason for clinical displacement. California State Auditor Report 2019-120 23 July 2020 To identify potential clinical displacement, BRN asks programs that are seeking initial approval or enrollment increases to contact nearby nursing programs and obtain statements indicating their support or opposition to the proposed change. BRN does this despite the fact that it requires the clinical facilities to assert, on the facility approval form that programs are required to submit to BRN, that a program’s use of a facility will not displace the students of other programs. The nursing education staff members then generally provide a summary of the statements to the governing board. According to BRN’s assistant executive officer, this practice first occurred in October 2016, when the education committee requested that Azusa Pacific University obtain statements from nursing programs potentially affected by its proposed enrollment increase. Since 2016 programs have continued to provide these statements to BRN. However, BRN has never established a process for handling these statements, such as promulgating a regulation to govern this process. For instance, the governing board approved requests for new programs and increased enrollment for several nursing programs despite existing statements of opposition. BRN does not require its nursing education staff to independently verify the nearby nursing programs’ assertions in these statements. For example, when the statements present significant disagreement, such as the seven statements of opposition and five statements of support provided to BRN regarding a proposed enrollment increase, BRN policy does not require nursing education staff to contact the programs and investigate the discrepancy. Nearby nursing programs might compete with the new nursing programs for clinical placement slots, and thus they have no clear incentive to support increasing enrollment for another nursing program. Further, the nearby nursing programs do not always provide responses to the requesting program. For example, according to the governing board meeting materials, 25 of 38 programs did not respond to Concordia University Irvine’s June 2017 enrollment increase request. All of these factors call into question the validity and usefulness of the practice of soliciting the statements, and thus BRN should immediately discontinue its practice of asking nursing programs to seek statements of support or opposition from neighboring nursing programs. Some governing board members and stakeholders agree that the Some governing board members existing process for assessing clinical displacement lacks clear and stakeholders agree that the direction and robust information. During the September 2019 existing process for assessing board meeting, some governing board members echoed this clinical displacement lacks clear sentiment as they made decisions involving enrollment increases. direction and robust information. During this meeting, two governing board members acknowledged that the governing board had not provided its staff with clear direction on what information it needs when assessing clinical displacement. Stakeholders also voiced their displeasure with 24 California State Auditor Report 2019-120 July 2020 BRN’s current method of assessing clinical displacement during the stakeholder summit meetings in the fall of 2018. For example, the resulting summit report describes an interest in replacing BRN’s existing approval process with “reliable processes that provide sufficient evidence of clinical capacity/clinical placement.” BRN’s executive officer stated that gathering more information about clinical placement slots would help the governing board and BRN education staff better understand clinical capacity. Without accurate clinical placement information, BRN cannot consistently and confidently prevent current nursing students from being displaced. BRN Is Not Collecting and Analyzing Useful Information Regarding Clinical Placement Slots and Capacity Although BRN has a database with some information about the clinical facilities that nursing programs use (nursing program database), it does not track the number of available clinical placement slots or the total number of students placed at a clinical facility. Consequently, BRN cannot effectively analyze and report the risk of displacement to its governing board when it is considering enrollment decisions. As we mention in the Introduction, nursing programs must get BRN approval before using a clinical facility. BRN documents its approval on a facility approval form, on which the facility and program attest that the program’s clinical placements at the facility will not displace students from other nursing programs. The form also includes the program location and the content area for which the program is using the facility. Therefore, BRN should have a record of all facilities that nursing programs are using for clinical placement slots. BRN compiles some of the information captured in the facility approval form in its nursing program database. According to BRN, the database is intended as a tool for nursing education staff to hold information on nursing programs. Yet, BRN does not gather certain critical information about available clinical placement slots in its nursing program database. In particular, BRN does not collect on its facility approval form or track the total number of students—or clinical placement slots—a clinical facility can accommodate annually or how many slots the programs that use the facility will need each year, as Figure 5 shows. BRN does not track the total As a result, BRN’s governing board lacks key information it needs number of students a clinical facility to make enrollment decisions. For example, knowing the number can accommodate annually or how of placement slots that a facility can accommodate would allow many slots the programs that use the governing board to determine whether a program’s request the facility will need each year. to increase enrollment by using that facility would exceed that capacity and risk displacing students. California State Auditor Report 2019-120 25 July 2020 Figure 5 BRN Is Not Taking Full Advantage of Its Nursing Program Database BRN DOES not … …use facility approval forms …ensure that its …require nursing programs …analyze and report to gather key information: nursing program to submit updated facility key findings related to database is complete approval forms for any clinical placements to Total number of clinical placement and accurate. changes to facility use after the governing board and slots a clinical facility can BRN has approved the use of stakeholders via its accommodate annually. a facility. website. Total number of slots each program needs annually. FACILITY APPROVAL FORM Source: Analysis of state law and BRN’s data and documents. As it is, the database is incomplete and unreliable because BRN has not added information for all the facilities where nursing programs have clinical placements. Some of the facility approval forms on file, as well as entries in the database, are over a decade old and include outdated and incomplete information because BRN does not require nursing programs to submit updated facility approval forms once a facility is approved. Consequently, if a nursing program does not submit an updated facility approval form, BRN may be unaware of changes to facility use, and therefore the governing board may not have current and complete information to assess how any changes could affect its enrollment decisions concerning that facility. To ensure that it maintains up‑to‑date information on the number of available clinical placement slots at facilities, BRN should revise its regulations to require nursing programs to report to it, using a facility approval form, anytime they make changes to their use of clinical facilities, as well as to report annually if they have made no changes. BRN should use these forms to update the information contained in its database. If BRN’s database were complete and up to date, it could have used the data to analyze the risk of displacement related to a program’s request for an enrollment decision and informed the governing board of the results of its analysis. In fact, we tested this idea for the 16 nursing programs located in five Bay Area counties (Alameda, Contra Costa, Marin, San Francisco, and San Mateo). For these 26 California State Auditor Report 2019-120 July 2020 programs, we compiled the data from hundreds of facility approval forms BRN had in its files into a list, and we analyzed the data by program, facility, and content area. We found that, according to BRN’s records, the 16 programs reported using certain facilities for clinical placement slots far more frequently than others. For example, 11 of the 16 nursing programs we reviewed reported using UCSF Children’s Hospital in Oakland for their students to get their pediatric clinical experience. According to the executive officer, BRN agrees that it should compile and analyze data related to clinical placement slots, and she indicated that BRN would be able to assign administrative staff or a data expert to do so. The executive officer also asserted that although BRN does not track clinical capacity and displacement on a statewide systematic basis, it has been gathering information related to clinical displacements through its annual school survey Although the survey gathers for several years. Although the survey gathers valuable information, valuable information, it does not such as the number of students that nursing programs reported had capture statewide or regional lost clinical placement slots and the nursing program’s perceived information on clinical capacity. reason that clinical placement slots were not available, it does not capture statewide or regional information on clinical capacity. Capturing in its database the total number of placement slots a clinical facility can accommodate and how many slots the programs that use the facility utilize and then publishing this information on its website, would allow BRN and other key stakeholders to begin to understand the capacity for clinical placement slots on a regional and statewide basis. We acknowledge that the number of available clinical placement slots changes over time, and multiple factors can affect a facility’s ability to predict the exact number of its annual placements. However, even if there are changes throughout the year, collecting annual estimates of clinical slots from facilities across the State will allow BRN to make better informed enrollment decisions that affect the State’s nursing supply. BRN should revise its facility approval form to collect the total number of students that a clinical facility can accommodate annually as well as the number of students the program needs to place annually. BRN Is Forgoing Opportunities to Help Nursing Programs Identify Facilities With Potential Clinical Placement Slots BRN could also analyze and share information that could foster additional clinical placement opportunities, which in turn could enable some nursing programs to increase enrollment and educate new nurses. Specifically, OSHPD has a downloadable list on its California State Auditor Report 2019-120 27 July 2020 website of state health care facilities.4 If BRN had a complete and up‑to‑date database with information related to the facilities each nursing program is using, it could compare this information to OSHPD’s list of health care facilities and publish its comparison on its website. This comparison could assist nursing programs in identifying clinical facilities that other nursing programs are not using at all for clinical placement slots or that only a few are using. In fact, using OSHPD’s information, we identified many facilities that, according to BRN’s records, are not currently placing students, and some of these facilities potentially could be sources for clinical placement slots. Using the information we compiled from BRN’s facility approval forms for the 16 nursing programs in five Bay Area counties we described earlier, we compared the facilities these programs used with OSHPD’s list of health care facilities in those same counties.5 We found that the 16 nursing programs were using 121 of the 708 facilities on OSHPD’s list, or 17 percent. This means that there are hundreds of clinical facilities in those five counties that nursing programs are not currently using for clinical placement slots, representing a possible untapped source of additional clinical placement slots. We also found from this analysis that nursing programs have clinical placements at most acute‑care hospitals but are not currently using nonacute facilities, such as home health agencies, hospice facilities, and clinics nearly as much. Specifically, the programs in the Bay Area we reviewed are using 82 percent of the acute‑care hospitals in OSHPD’s list, but are using only 10 percent of the clinics. In fact, this analysis helps identify possible additional nonacute facilities for placements, which was a priority for action from the stakeholder summits. Figure 6 illustrates the number of used and unused facilities in the five counties by facility type. In addition, we determined the content areas for which nursing programs were using each type of facility, as Figure 6 also shows. For example, skilled nursing facilities can accommodate several content areas and, while 34 of those facilities are currently being used, 107 are currently unused. 4 According OSHPD’s website, this is a listing of facilities that are licensed by California Department of Public Health. 5 The counties are Alameda, Contra Costa, Marin, San Francisco, and San Mateo. 28 California State Auditor Report 2019-120 July 2020 Figure 6 Facilities Not Used by Nursing Programs for Clinical Placements Could Be a Source of Additional Placements Content areas Medical/Surgical Obstetrics Pediatrics Used Mental Health/psychiatry unused Geriatrics 46 34 21 3 5 0 2 3 1 2 4 1 0 8 12 11 11 64 64 107 141 168 home hospice other* free dialysis community other skilled other acute acute health nursing psych care agency home health clinic long-term hospital agency/hospice care facility type of facility Source: Analysis of BRN’s documents and OSHPD’s data for programs in the Bay Area. * Because no programs currently use other clinical facilities, we could not determine the content areas that would apply. California State Auditor Report 2019-120 29 July 2020 It is important to note that just because a nursing program is not using a facility does not necessarily mean the facility is available for use or willing to provide clinical placement slots for nursing students. For example, a facility might not have enough staff to support student learning or might have other concerns. BRN and nursing programs would need to do additional work to contact currently unused facilities to gauge their interest in providing clinical placement slots. However, we believe such a comparison and the necessary follow‑up would provide valuable information to help identify additional clinical placement slots and alleviate some of the possible constraints on enrollment for nursing programs in areas experiencing a nursing shortage. BRN agreed that comparing its data from the facility approval forms with OSHPD data could be helpful in identifying facilities that might provide additional clinical placement slots. BRN’s Process for Approving Nursing Programs Partially Overlaps With the Work of Accreditors Some of BRN’s requirements for approving nursing programs are similar to accreditation standards. National Nursing Program Accreditors (accreditors) are private educational associations that assess whether nursing programs meet and maintain acceptable levels of quality. As part of their evaluation of nursing programs, accreditors verify that course content is consistent with contemporary nursing practices, instructors are using teaching methods that support expected student outcomes, and schools are meeting the needs of nursing students by providing adequate resources and support services. Although BRN approval is required for nursing programs in California, accreditation is optional. BRN reported that roughly half of the nursing programs in the State were accredited as of fiscal year 2017–18. Of those that are accredited, nearly all are accredited by the Commission on Collegiate Nursing Education (CCNE) or the Accreditation Commission for Education in Nursing (ACEN). Both of these accreditors are recognized by the U.S. Department of Education as reliable authorities on the quality of nursing education. BRN’s approval of nursing programs has similarities to accreditation in both its approval process and the standards it requires nursing programs to meet. For instance, both review processes involve an initial approval in which accreditors and BRN verify that nursing programs meet their standards; a cycle of periodic continuing approvals; and the requirement that nursing programs report substantive changes, such as enrollment increases or curriculum changes. For continuing approval, both processes require a program to conduct a self‑evaluation that provides similar information, such as licensure exam pass rates and faculty 30 California State Auditor Report 2019-120 July 2020 qualifications. BRN requirements for nursing program approval are found in state law. These requirements are similar to accreditation standards in many categories. For example, as shown in Table 3, the accreditors’ standards overlap with BRN’s requirements in each of the following areas: administrator and faculty qualifications and responsibilities, program resources, curriculum requirements, and testing standards. For certain areas, one accreditor verifies that nursing programs are meeting the same state requirements that BRN verifies. In fact, eight ACEN accreditation standards specifically require accreditors to verify that nursing programs are in compliance with state requirements or policies for the applicable area under review. Table 3 Accreditors’ Standards Are Similar to Some of BRN’s Requirements ACCREDITORS SELECTION OF BRN’S REQUIREMENTS FOR ACEN CCNE NURSING PROGRAM APPROVAL Nursing program faculty and administrators are qualified and have   relevant experience.   Nursing program has sufficient resources for students and faculty. Curriculum is comprehensive and includes concurrent   clinical experience. Nursing program maintains a minimum pass rate for the   licensure exam. X X The majority of clinical hours are completed in direct patient care. Nursing program considers clinical displacement when selecting X X a new clinical facility to use. Source: Analysis of state law and accreditors’ documents. The requirement is present in the accreditor’s standard. The requirement is not present in the accreditor’s standard. However, there are some important differences between BRN oversight and accreditation. According to the National Council of State Boards of Nursing (National Council), a state board’s mission is protecting the public and ensuring that nursing programs meet state requirements, whereas accreditors focus on quality and program effectiveness.6 The National Council points out that boards of nursing also understand nursing education issues in their specific jurisdictions. Accreditors do not have statutory authority 6 The National Council is a nonprofit organization whose members consist of the nursing regulatory bodies in the 50 states, the District of Columbia, and four U.S. territories. Its mission is to empower and support nursing regulators in their mandate to protect the public. California State Auditor Report 2019-120 31 July 2020 to close nursing programs that do not meet standards, while boards of nursing do have that authority. The National Council also states that boards of nursing can act right away when they identify problems with nursing programs; accreditors cannot act as quickly. Additionally, continuing approval visits by ACEN and CCNE may occur less frequently than BRN’s—up to every eight to 10 years for the accreditors compared to every five years for BRN. Also, BRN approves nursing program faculty prior to employment, whereas accreditors do not. BRN’s executive officer strongly opposes the prospect of reducing BRN’s involvement in reviewing and approving nursing programs. She stated that accreditation reviews are too infrequent and are not focused on ensuring that nursing programs comply with BRN regulations. She added that BRN has identified noncompliance even at accredited programs, such as unapproved curriculum changes and insufficient resources. She also echoed the point made by the National Council that accreditors do not have statutory authority over nursing programs. She believes that maintaining BRN’s oversight and implementation of the review process is the only way to ensure consistent program review for all prelicensure nursing programs and that relying on accreditation does not enable BRN to achieve its mission of protecting the public and nursing students. Finally, she stated that reducing BRN oversight could result in registered nursing students and graduates not having sufficient educational preparation and opportunities to obtain the requisite knowledge, skills, and abilities needed to safely and competently perform required nursing functions. Nevertheless, aligning state review with accreditation is not uncommon. We identified several California healing arts boards that rely on accreditation in place of or in conjunction with state review: the Medical Board of California, the Osteopathic Medical Board of California, the Physician Assistant Board, and the Dental Hygiene Board of California. This is not the case for California nursing programs: the State does not require accreditation for these nursing programs, and only half of them have chosen to become accredited. However, the State does require accreditation Although the State requires for nurse practitioner programs located in California, which are accreditation for nurse practitioner advanced‑practice programs. The National Council recommended programs located in California, it in 2012 that all state boards of nursing require nursing programs to does not require it for prelicensure be accredited by 2020. As of March 2020, a total of 26 U.S. states and nursing programs. territories require accreditation, according to the National Council. Additionally, collaboration between states and accreditors is encouraged. Although BRN specifically states that it will not accept reports prepared for accrediting bodies, ACEN indicated that it welcomes the opportunity to cooperate with state regulatory agencies for nursing with the goal of increasing efficiency and 32 California State Auditor Report 2019-120 July 2020 decreasing workload while maximizing outcomes. In addition, the National Council recommends that boards of nursing work toward harmonizing their approval process with accreditors. Given the differences in the purposes of BRN’s approval and national accreditation, we are not suggesting that accreditation We believe policymakers should is an exact replacement for BRN’s oversight. Rather, we believe consider whether it would be policymakers should consider, as part of their sunset review, appropriate to restructure any whether it would be appropriate to restructure any of BRN’s of BRN’s oversight to reduce oversight to reduce duplication with accreditation agencies while duplication with accreditation still achieving its mission to protect the public. Sunset review is agencies while still achieving its a process intended to identify and eliminate waste, duplication, mission to protect the public. and inefficiency in government agencies. The purpose of sunset review is for a legislative committee to conduct a comprehensive analysis on a periodic basis to determine whether the subject agency is still necessary and cost‑effective. As a part of this process, the committee considers recommendations for changes and reorganization to help the agency better fulfill its purpose. Given that some of BRN’s oversight of nursing programs might be duplicated by accreditors, we believe the upcoming sunset review in 2021 would be an appropriate setting to consider whether the State would be better served by having BRN revise its regulations to leverage portions of the accreditors’ reviews in order to reduce duplication and more efficiently use state resources. For example, it could consider restructuring continuing approval requirements for nursing programs that are accredited and maintain certain high performance standards for consecutive years (for example, licensure exam pass rates, program completion rates, and job placement rates). Other Areas We Reviewed BRN’s Conflict‑of‑Interest Code Is Adequate, and Members of the Governing Board Recused Themselves Appropriately BRN’s conflict‑of‑interest code (code) incorporates the terms of the Fair Political Practices Commission’s standard code and appropriately identifies positions within BRN that must report economic interests. State law requires that every agency adopt and promulgate a code. It also requires that, in their codes, agencies must specifically designate positions that involve the making of or participation in the making of decisions that may have a foreseeable effect on any financial interest for individuals in those positions, and the types of financial interests that those individuals must report. Additionally, agencies’ codes must contain provisions that outline circumstances under which designated employees must recuse themselves from participation in decision making. California State Auditor Report 2019-120 33 July 2020 To report their economic interests, designated BRN employees file a Statement of Economic Interests—known as a Form 700—that the Fair Political Practices Commission publishes. Based on our review, every individual at BRN who is significantly involved in the approval process for nursing programs filed a Form 700 for each year from 2017 to 2019. However, two people filed two of their forms late after we found that they were missing and discussed it with a filing officer at Consumer Affairs. We found that governing board members appropriately recused themselves from decisions regarding nursing programs in which they had reported an economic interest during the audit period. Nursing Education Staff Members Responsible for Reviewing Nursing Programs Are Adequately Qualified BRN’s nursing education staff members are appropriately qualified to perform their oversight responsibilities. To assess their expertise, we reviewed the minimum qualifications of nursing education staff members as defined by their job classifications and compared each staff member’s most recent application file to those minimum qualifications. We also determined that the minimum qualifications appeared appropriate for the type of oversight work that nursing education staff perform. Nursing education staff members must have an active, valid California license as an RN and at least five years of nursing experience, which must include three years as a teaching nurse faculty member; or three years as a clinical specialist, nurse practitioner, or in‑service educator in a hospital, clinic, or private‑practice setting, and a master’s degree in nursing or a related field. Supervising nursing education staff members must have two years of experience performing the duties of staff‑level nursing education staff or five years of nursing experience, including three years as a teaching nurse faculty member and two years of experience in nursing administration. All of the 11 currently employed nursing education staff members meet or exceed the minimum education qualifications; in fact, six of the staff have a doctoral degree. 34 California State Auditor Report 2019-120 July 2020 Recommendations Legislature To better inform stakeholders and the governing board’s decision making, the Legislature should amend state law to do the following: • Require BRN to incorporate regional forecasts into its biennial analyses of the nursing workforce. • Require BRN to develop a plan to address regional areas of shortage identified by its nursing workforce forecast. BRN’s plan should include identifying additional facilities that might offer clinical placement slots. As part of BRN’s sunset review in 2021, the Legislature should consider whether the State would be better served by having BRN revise its regulations to leverage portions of the accreditors’ reviews to reduce duplication and more efficiently use state resources. For example, it could consider restructuring continuing approval requirements for nursing programs that are accredited and maintain certain high performance standards for consecutive years (for example, licensure exam pass rates, program completion rates, and job placement rates). Additionally, the Legislature should consider whether and how BRN could coordinate its reviews with accreditors to increase efficiency. To ensure that BRN and stakeholders have an understanding of clinical placement capacity in California, the Legislature should amend state law to require BRN to annually collect, analyze, and report information related to the number of clinical placement slots that are available and the location of those clinical placement slots within the State. BRN To better ensure that California has an appropriate number of nurses in the future, BRN should do the following by January 1, 2021: • Revise the scope of work of its contract for workforce forecasting services to direct the contractor to incorporate regional analyses. • Ensure that the governing board’s enrollment decisions and other actions adequately take into consideration the regional analyses in BRN’s future workforce forecasts. Specifically, it should amend its policies to require that when its staff present information to California State Auditor Report 2019-120 35 July 2020 the education committee and the governing board to inform them on pending enrollment decisions, staff should include relevant information related to BRN’s most recent forecast of the nursing workforce. To ensure that nursing education staff members provide complete information to the governing board when it is considering enrollment decisions, by January 1, 2021, BRN should establish in policy the specific information that its staff should present to the education committee and governing board, including data about clinical facilities that nursing programs use for placements, the content areas for which the programs use those facilities, and the total number of available placement slots and the risk of clinical displacements at the facilities. To ensure that BRN is using up‑to‑date, accurate, and objective information to inform the governing board’s enrollment decisions and to assess clinical capacity for student placements, by April 1, 2021, BRN should do the following: • Update its clinical facility approval form to capture annual capacity estimates from clinical facilities, as well as annual clinical placement needs of programs. • Revise its regulations to require nursing programs to report any changes they make to their use of clinical facilities within 90 days of making a change and report annually if the program has made no changes. • Compile and aggregate the information from the facility approval forms into a database and take reasonable steps to ensure that the information is accurate and current. • Annually publish clinical capacity information on its website for public use. • Immediately discontinue its practice of having nursing programs seek statements of support or opposition from neighboring nursing programs when considering requests for new programs or increased enrollment at existing programs. To identify additional facilities that might offer clinical placement slots, by October 1, 2021, and annually thereafter, BRN should compare its nursing program database with OSHPD’s list of health care facilities. BRN should share the results of its comparison with nursing programs by publishing this information on its website. 36 California State Auditor Report 2019-120 July 2020 We conducted this performance audit in accordance with generally accepted government auditing standards and under the authority vested in the California State Auditor by Government Code 8543 et seq. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on the audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Respectfully submitted, ELAINE M. HOWLE, CPA California State Auditor July 7, 2020 California State Auditor Report 2019-120 37 July 2020 Appendix Scope and Methodology The Joint Legislative Audit Committee (Audit Committee) directed the State Auditor to examine BRN’s oversight of nursing programs. Specifically, we reviewed BRN’s process for approving new nursing programs or programs seeking to expand and its efforts to analyze the nursing workforce in California. The Table lists the objectives that the Audit Committee approved and the methods we used to address them. Audit Objectives and the Methods Used to Address Them AUDIT OBJECTIVE METHOD 1 Review and evaluate the laws, rules, Reviewed relevant laws, rules, and regulations. and regulations significant to the audit objectives. 2 Determine whether BRN is appropriately • Objective 2 asked us to assess whether BRN’s policies and procedures comply with state law. reviewing and approving nursing We found that in matters not related to enrollment, BRN’s policies and procedures were in programs, including the following: compliance with the Nursing Practice Act and BRN’s regulations. We made no determination a. Whether BRN’s policies and as to whether BRN has authority to determine the total number of students a nursing procedures for approving, denying, program may enroll or whether any of BRN’s policies and procedures constitute underground deferring, or revoking its approval of regulations in violation of the Administrative Procedures Act as these issues were in litigation nursing programs comply with laws during our fieldwork and audit standards prohibit us from interfering with litigation. With and regulations. respect to these issues, our report simply focuses on the actions BRN has taken in the recent past. b. Whether the factors that BRN uses when considering a request from a • Reviewed BRN’s director’s handbook, which describes the information nursing programs school to expand its nursing program must provide to BRN when requesting to expand the program. are reasonable. • Identified governing board decisions approving new programs and expanding existing c. Whether BRN consistently and programs from January 2015 through September 2019, and reviewed related governing objectively applied these factors as board meeting minutes and materials. a part of its decision‑making process for a selection of requests. • Reviewed five requests for new programs and 10 requests to expand existing programs that the governing board decided between January 2015 and September 2019 to determine if the governing board’s decision making was objective and consistent. 3 Review petitions of regulatory violations Obtained and reviewed OAL’s list of petitions for regulatory violations regarding BRN and related to nursing programs filed against summarized outcomes. BRN with OAL over the last three years and summarize the outcomes of the complaint process. continued on next page . . . 38 California State Auditor Report 2019-120 July 2020 AUDIT OBJECTIVE METHOD 4 Determine whether there are adequate • Interviewed key staff at BRN and Consumer Affairs to identify relevant laws, regulations, conflict‑of‑interest rules or policies for policies, and documentation related to Consumer Affairs’ conflict‑of‑interest code and governing board members, executive statements of economic interest. management, and nursing education • Identified and assessed whether Consumer Affairs’ conflict‑of‑interest code that applies to staff who work on the oversight of BRN is sufficient and appropriate. nursing programs. Further, to the extent possible, identify whether BRN’s staff or • Identified governing board members, executive management, and nursing education governing board members appropriately staff required to file a Form 700, collected and reviewed each of those Form 700s for recused themselves from decisions 2017 through 2019, and determined whether those individuals had any pertinent regarding nursing programs with which economic interests. they may have had a conflict of interest. • Reviewed meeting minutes for each governing board meeting from January 2015 through September 2019 to determine whether governing board members recused themselves appropriately if their reported economic interests were the subject of board action. 5 Identify the process BRN uses to evaluate • Interviewed key staff at BRN and determined that BRN does not evaluate clinical placements clinical displacement and whether it across the State. We could not assess the clinical displacement that might have resulted from consistently and objectively uses that BRN’s enrollment decisions because it does not track this information at that level. process across all nursing programs. • Reviewed BRN’s annual school survey and the stakeholder summit report to determine the For a selection of requests for increased extent of clinical displacement. enrollment or new nursing programs, assess the factors BRN evaluated in • Assessed the factors BRN evaluated as part of our review under Objective 2, including when making its decisions and the resulting applicable, information about clinical displacement. clinical displacement. • Reviewed BRN’s database to identify the clinical facility information it has. Determined BRN’s database to be incomplete and unreliable. 6 Determine whether BRN’s oversight • Compared BRN’s oversight requirements to national accreditation standards and processes. of nursing programs is appropriate, Reviewed National Council documents related to state boards of nursing and national including the following: accreditation. a. Whether BRN is duplicating oversight • Interviewed key nursing education staff about documentation and processes related to their of nursing programs conducted by review of nursing programs. other entities, including state and federal entities, as well as nursing • Determined that nursing education staff are primarily responsible for evaluating the school accreditors. curricula of nursing programs. b. An assessment of the expertise • Compared the hiring applications for each nursing education staff member hired after BRN relies on when it evaluates the December 2014 with California Department of Human Resources’ minimum qualifications curricula of nursing programs. for those positions. • Assessed the type of oversight nursing education staff perform and available documentation of the various processes related to BRN’s approval of nursing curricula. 7 Determine whether BRN’s analysis • Interviewed key staff at BRN to understand the process BRN uses to develop and publish of California’s nursing workforce is studies on California’s nursing workforce forecast. reasonable and consistent with the • Identified recent studies related to the nursing workforce in California. scope and breadth of current and future health care workforce needs as identified • Reviewed key elements of the studies, including their methodologies and conclusions. by similar analyses. • Compared the methodology and findings of BRN’s nursing workforce forecast to those of other studies. 8 To the extent possible, identify the time • Interviewed key staff at BRN and Consumer Affairs to identify and understand BRN’s spent and resources used by BRN on budgeting practices. We could not identify the time spent and resources used by BRN on each of its programs. each of its programs because BRN is a single payroll reporting unit, which means it budgets and reports expenditures as a single unit. It does not track time and resources by program or organizational units. For example, its expenditures for salaries are recorded as one amount, even though BRN has staff dedicated to different units. • Reviewed documentation related to BRN’s budget, including its latest budget augmentation. California State Auditor Report 2019-120 39 July 2020 AUDIT OBJECTIVE METHOD 9 Review and assess any other issues that • Reviewed facility approval forms for 16 nursing programs in five counties in the are significant to the audit. San Francisco Bay Area and compared the clinical facilities associated with the 16 nursing programs with OSHPD data of registered health care facilities from the same five counties to identify facilities not currently used by the 16 nursing programs. • Prior to the completion of this audit, the State Auditor received a whistleblower complaint alleging that BRN executives in the enforcement division intentionally manipulated data and delivered a falsified report to the State Auditor to satisfy a recommendation the State Auditor had made during a 2016 audit of the enforcement division. In response to the complaint, the State Auditor launched an investigation and substantiated that BRN executives violated state law when they carried out a plan to artificially decrease caseloads for BRN investigators before delivering a falsified report to the State Auditor. The plan involved temporarily reassigning some of the BRN investigators’ cases to other employees who should not have had cases assigned to them. The investigation found that within 10 days of the State Auditor reviewing the falsified report and concluding that BRN had fully implemented the recommendation, BRN managers reversed the reassignments, increasing caseloads to their original level. A copy of investigative report I2020‑0027, Board of Registered Nursing: Executives Violated State Law When They Falsified Data to Deceive the State Auditor’s Office, can be found at www.auditor.ca.gov. The audit team became aware of the investigation during this audit and re‑evaluated the risk assessment it conducted for the audit to ensure it could rely upon the documentation provided by BRN for this audit report. We determined that the documentation we obtained was reliable. Source: Analysis of the Audit Committee’s audit request number 2019‑120, and information and documentation identified in the column titled Method. Assessment of Data Reliability In performing this audit, we relied on electronic data files that we obtained from OAL related to petitions it received and from OSHPD’s website related to health care facilities. The U.S. Government Accountability Office, whose standards we are statutorily obligated to follow, requires us to assess the sufficiency and appropriateness of computer‑processed information we use to support our findings, conclusions, and recommendations. We used the data from OAL to verify that it had received two petitions related to BRN over the last three years. OAL performed for us multiple queries of its system to identify petitions related to BRN, and each query identified the same two petitions; therefore, we determined that the data were sufficiently reliable for our purpose. We also downloaded from OSHPD’s website the list of health care facilities. We used the data to identify clinical facilities that nursing programs are not currently using for clinical placements. We verified that the data included logical information; however, we did not perform completeness testing because the supporting documentation is maintained at the facilities, making such testing impractical. We concluded that the data are of undetermined reliability. Although we recognize that this limitation may affect the precision of the numbers we present, there is sufficient evidence in total to support our audit findings, conclusions, and recommendations. 40 California State Auditor Report 2019-120 July 2020 Blank page inserted for reproduction purposes only. California State Auditor Report 2019-120 41 July 2020 BUSINESS, CONSUMER SERVICES, AND HO USING AGENCY • GAVIN NEWSOM, GOVERNOR BOARD OF REGISTERED NURSING PO Box 944210, Sacramento, CA 94244-2100 P (916) 322-3350 | www.rn.ca.gov June 11, 2020 Elaine M. Howle, State Auditor * California State Auditor’s Office 621 Capitol Mall, Suite 1200 Sacramento, CA 95814 RE: Response from California Board of Registered Nursing to Audit 2019-120 - Oversight of Pre-Licensure Nursing School Programs Dear Ms. Howle, The California Board of Registered Nursing (Board) appreciates the time and effort you and your staff have dedicated to evaluating our oversight of pre-licensure nursing school programs and making recommendations to refine and improve the Board’s processes. The Board sets a high standard for itself and is always interested in identifying opportunities to better fulfill its mission of protecting California consumers. We are keenly aware of the critical role of registered nurses in maintaining the health and safety of Californians. Thus, we are committed to ensuring that our nurses receive a quality education that prepares them for the incredibly important jobs that they have in our communities. We thank you for your recommendations in the audit report, and respectfully submit the attached responses. Should you have any questions or require anything else, please do not hesitate to contact the Board’s Assistant Executive Officer, Evon Lenerd Tapps at (916) 574-7610. Sincerely, Michael D. Jackson, MSN, RN, CEN Loretta Melby, RN, MSN President Executive Officer California Board of Registered Nursing California Board of Registered Nursing Attachment * California State Auditor’s comments begin on page 47. 42 California State Auditor Report 2019-120 July 2020 The California Board of Registered Nursing (BRN) Responses to the California Bureau of State Audits (BSA) Findings June 11, 2020 Audit Name Board of Registered Nursing – Oversight of Pre-Licensure Nursing School Programs Audit Number 2019-120 BSA Recommendations to BRN and BRN Responses Recommendation 1: To better ensure that California has an appropriate number of nurses in the future, BRN should do the following by January 1, 2021: • Revise the scope of work of its contract for workforce forecasting services to direct the contractor to incorporate regional analyses. • Ensure that the governing board’s enrollment decisions and other actions adequately take into consideration the regional analyses in BRN’s future workforce forecasts. Specifically, it should amend its policies to require that when its staff present information to the education committee and the governing board to inform it on pending enrollment decisions, they include relevant information related to BRN’s most recent forecast of the nursing workforce. BRN Response 1: 1 BRN collects data which assists in determining if California has the appropriate number of nurses in the future. This includes, but is not limited to, information gained from the 2018 Regional Nursing Summits (Summit)1, the raw data which the University of California, San Francisco (UCSF) collects on behalf of BRN, and information collected from pre-licensure nursing programs through their “written plan for evaluation of the total program” that includes, among other things, evaluation of the performance of the school’s graduates in meeting community needs. (16 CCR §1424(b)(1).)2 On or about January 1, 2021, to better ensure California has an appropriate number of nurses in the future, BRN will: • BRN has a current contract for workforce forecasting services in place with an end date of June 30, 2021, and work has already been performed for this contract period. BRN will request the contractor to include a regional analysis within the report ‘Forecasts of Registered Nurse Workforce in California’ that is published on the BRN website. BRN will ensure that the scope of work for future 1 The goal of these Summits was to examine clinical capacity in more detail with the intent to address clinical capacity issues and associated factors in a collaborative and transparent manner. The data collected during the Summits included regional workforce differences and other regional data. Although this data is not typically presented by NECs, it is used by the governing board when making enrollment decisions. If future Summits occur, BRN will seek to participate in these Summits to address ongoing clinical 1 capacity and collaborate with other stakeholders, as appropriate. 2 This data is typically collected and evaluated during the five-year Continuing Approval Visit. BRN does not have regulatory authority 2 to require a plan for evaluation of the total program on an annual basis, and to require that it include regional nursing workforce forecast data. Therefore, in order for BRN to require nursing programs to submit their written plan for evaluation on an annual basis, BRN would need to pursue a change to regulations, which would not feasibly be promulgated on or before January 1, 2021. However, BRN will consider revising its regulations to require nursing programs to submit their written plan for evaluation on an annual basis on or before October 1, 2021. On or before January 1, 2021, BRN will request nursing programs to submit their written plan for evaluation for their total program on an annual basis. BRN will also provide training to all impacted staff. 1 California State Auditor Report 2019-120 43 July 2020 contracts for workforce forecasting services will incorporate regional data and analysis, in alignment with the data in the 2018 Summit report currently relied upon by the governing board. 1 • Amend its policies, as appropriate, to require that relevant information related to BRN’s most recent forecast of the nursing workforce, and other relevant regional data, be included in Agenda Item Summaries (AIS), presentations by Nursing Education Consultants (NEC; referred to as nursing education staff in the audit report), and supporting documentation, so that they may be taken into consideration when making enrollment decisions. These items may also include, but will not necessarily be limited to, the school’s report on how their graduates will be meeting community needs, which sometimes includes regional nursing workforce data. Recommendation 2: To ensure that nursing education staff provide complete information to the governing board when it is considering enrollment decisions, by January 1, 2021, BRN should establish in policy the specific information that its staff should present to the education committee and governing board, including data about clinical facilities that nursing programs use for placements, the content areas for which the programs use those facilities, and the total number of available placement slots and the risk of clinical displacements at the facilities. BRN Response 2: Through discussions with BSA during the audit process, BRN initiated meetings and process improvement efforts to ensure consistency and uniformity with AIS and supporting documentation requirements when presenting to the ELC and governing board. BRN will continue to work with the ELC, the governing board, and the NECs to establish and implement a uniform format and reporting structure which informs the ELC and the governing board of appropriate information for enrollment decisions for pre-licensure nursing programs. On or before January 1, 2021, the information will include data about clinical facilities that nursing programs use for placements and the content areas for which the programs use those facilities. However, BRN can only include data relating to the total number of available placement slots and the risk of clinical displacements at the facilities once that information can be collected and analyzed, which will be after January 1, 2021. 3 BRN agrees that the available data on clinical placements can be enhanced; therefore, BRN has researched and discussed regional consortiums as a way to identify every student placement in all clinical settings, provide a transparent system for resolving clinical placement conflicts, and document problem areas. There are currently limited consortiums available in California and they are not uniform nor are they located in every region, and participation in the consortiums is voluntary. Without legislative and regulatory 4 authority, BRN cannot implement a statewide consortium with a regional focus and require all clinical settings and academic institutions to participate. Such a system could ensure that data relating to the total number of available placement slots and the risk of clinical displacements at the facilities will be collected and analyzed. A statewide consortium with regional focus would provide a complete and accurate representation of available clinical placement slots. Recommendation 3: To ensure that BRN is using up-to-date, accurate, and objective information to inform the governing board’s enrollment decisions and to assess clinical capacity for student placements, by April 1, 2021, BRN should: • Update its clinical facility approval form to capture annual capacity estimates from clinical facilities, as well as annual clinical placement needs of programs. • Require nursing programs to report any changes they make to their use of clinical facilities within 90 days of making a change and report annually if the program has made no changes. • Compile and aggregate the information from the facility approval forms into a database and take reasonable steps to ensure that the information is accurate and current. • Annually publish clinical capacity information on its website for public use. 2 44 California State Auditor Report 2019-120 July 2020 • Immediately discontinue its practice of having nursing programs seek statements of support or opposition from neighboring nursing programs when considering requests for new programs or increased enrollment at existing programs. BRN Response 3: As mentioned in the responses for recommendations one and two, effective March 2020, BRN initiated meetings and process improvement efforts to amend its policies related to the AIS, the NEC presentation, and supporting documentation, which will ensure that the information presented to the ELC and the governing board is up-to-date, accurate, and objective, and provides sufficient information for the ELC and the governing board to assess clinical capacity for student placements in connection with enrollment decisions; additionally, BRN will take the following actions: • On or before April 1, 2021, BRN will update the clinical facility approval form to capture annual capacity estimates from clinical facilities as well as annual clinical placement needs of programs. • In order for BRN to require nursing programs to report any changes they make to their use of clinical facilities within 90 days of making the change and report annually if the program has made no changes, regulation sections including, but not limited to, CCR sections 1427 and 1432 will need to be revised. It is not feasible that a regulatory change could be promulgated on or before April 1, 2021. However, 5 BRN will consider revising its regulations to require nursing programs to report any changes they make to their use of clinical facilities within 90 days of making the change and report annually if the program has made no changes. • In order for BRN to require nursing programs to submit the facility approval form, a regulatory change will need to be promulgated. It is not feasible that a regulation package could be promulgated on or before April 1, 2021. However, BRN will consider revising its regulations to require nursing programs to 5 submit a facility approval form on or before October 1, 2021. On or before April 1, 2021, BRN will develop a policy to compile and aggregate the information from the facility approval forms into a database and take steps to ensure it is accurate and current.3 This information will be used to assess the risk of clinical displacement when gathering information related to enrollment decisions and will be reported to the ELC and the governing board in its newly developed uniform reporting format and structure. BRN will also provide training to all impacted staff. • On or before April 1, 2021, BRN will commence the process to analyze clinical capacity information that is available to BRN for the purpose of publishing it on the BRN website for public use on an annual basis. • As of March 11, 2020, BRN discontinued its practice of requiring nursing programs to seek statements of support or opposition from neighboring nursing programs when considering requests for new programs or increased enrollment at existing programs. BRN will update the 2020 Director’s Handbook with this information. Recommendation 4: To identify additional facilities that might offer clinical placement slots, by October 1, 2021, and annually thereafter, BRN should compare its nursing program database with OSHPD’s 3 BRN agrees that collecting and analyzing clinical information is necessary; therefore, BRN has researched and discussed regional consortiums as a way to identify every student placement in all clinical settings, provide a transparent system for resolving clinical 4 placement conflicts, and document problem areas. There are currently limited consortiums available in California and they are not uniform nor are they located in every region, and participation in the consortiums is voluntary. Without legislative and regulatory authority, BRN cannot implement a statewide consortium with a regional focus and require all clinical settings and academic institutions to participate. Such a system could provide a complete and accurate representation of available clinical placement slots and ensure that information presented to the ELC and the governing board to assess clinical capacity for student placements is up-to-date, accurate, and objective. 3 California State Auditor Report 2019-120 45 July 2020 list of health care facilities. BRN should share the results of its comparison with nursing programs by publishing this information on its website. BRN Response 4: To identify additional facilities that might offer clinical placement slots, on or before October 1, 2021, and annually thereafter, BRN will compare its aggregated data in its nursing program database with OSHPD’s list of health care facilities and will share the results of the comparison by publishing to the BRN website. As stated by BSA in the audit report, OSHPD data will not show the clinical settings that do not have the capacity 6 or the desire to offer placement slots; therefore, such a comparison might produce information that could be used to locate unused clinical sites, however it would not be an accurate representation of available clinical placement slots for nursing students. As previously stated, a statewide consortium with a regional focus would 4 provide a complete and accurate representation of available clinical placement slots for nursing students. BRN needs legislative and regulatory authority to develop and implement a statewide consortium with a regional focus and require health care facilities and academic institutions to participate in the statewide consortium, which will ensure that BRN has accurate and current data on clinical placement slots. BSA Recommendations to the Legislature and BRN Responses Legislative Recommendation 1: To better inform the governing board’s decision making and stakeholders, the Legislature should amend state law to do the following: • Require BRN to incorporate regional forecasts into its biennial analyses of the nursing workforce. • Require BRN to develop a plan to address regional areas of shortage identified by its nursing workforce forecast. BRN’s plan should include identifying additional facilities that might offer clinical placement slots. BRN Response to Legislative Recommendation 1: Business and Professions Code section 2717 requires BRN to collect and analyze workforce data from its licensees for future workforce planning. BRN collects and analyzes this data via its contractor, the University of California, San Francisco (UCSF). However, BRN has not requested the regional information from UCSF for purposes of publishing to its website. BRN does not oppose the development of a plan to identify regional areas that are underserved and collaborating to identify options to address those underserved areas, including but not limited to finding additional facilities that may offer clinical placements to students. Legislative Recommendation 2: As a part of BRN’s sunset review in 2021, the Legislature should consider whether the State would be better served by having BRN revise its regulations to leverage portions of the accreditor’s review to reduce duplication and more efficiently use state resources. For example, it could consider restructuring continuing approval requirements for nursing programs that are accredited and maintain certain high performance standards for consecutive years (for example, licensure exam pass rates, program completion rates, and job placement rates). Additionally, the Legislature should consider whether and how BRN could coordinate its review with accreditors to increase efficiency. BRN Response to Legislative Recommendation 2: BRN is not opposed to identifying and addressing any duplicative efforts involving third party accreditation entities and BRN’s statutory and regulatory oversight of pre-licensure nursing programs. However, this 7 recommendation being addressed to the Legislature does not consider BRN’s ability and willingness to address any concerns regarding duplicative efforts. BRN is in the unique position to take the lead and 1) assess the roles of the accreditation entities and its current processes; 2) identify areas of overlap and areas of improvement; 3) incorporate feedback of the Deans and Directors of currently accredited ADN and/or BSN pre- licensure nursing programs; 4) implement enhancements to its processes; and 5) conduct continuous quality 4 46 California State Auditor Report 2019-120 July 2020 improvement assessments and implement revisions based on the data. BRN could report the progress and accomplishments of reducing these duplicative efforts during its sunset review for evaluation and additional input. BRN affirms its interest in ensuring that its processes are evidence based and that we continue to offer the highest level of protection to consumers, patients, nursing students, and licensees. Legislative Recommendation 3: To ensure that BRN and stakeholders have an understanding of clinical placement capacity in California, the Legislature should amend state law to require BRN to annually collect, analyze, and report information related to the number of clinical placement slots available and the location of those clinical placement slots within the State. BRN Response to Legislative Recommendation 3: BRN supports advancing the understanding of clinical placement capacity and supports working in collaboration with other stakeholders, including but not limited to, hospitals and other health care facilities eligible to offer clinical placements to nursing students, for the purpose of collecting, analyzing and reporting 4 information related to the number and location of clinical placement slots available in California. BRN believes that a statewide consortium with a regional focus could accomplish this. In order to implement such a statewide consortium and require health care facilities and academic institutions to participate, BRN needs legislative and regulatory authority. Such a system could ensure that data relating to the total number of available placement slots and the risk of clinical displacements at the facilities can be collected and analyzed. This would allow for identification of every student placement in all clinical settings, provide a transparent 4 system for resolving clinical placement conflicts, and allow for documentation of problem areas. In the absence of legislative authority for a statewide consortium, BRN believes that OSHPD and/or the California Department of Public Health (CDPH) are in a better position to annually collect information on clinical placement slots, as they have statutory authority over health care facilities. BRN will analyze and report clinical placement slots for nursing students based on the data that OSHPD and/or CDPH collect. 5 California State Auditor Report 2019-120 47 July 2020 Comments CALIFORNIA STATE AUDITOR’S COMMENTS ON THE RESPONSE FROM THE BOARD OF REGISTERED NURSING To provide clarity and perspective, we are commenting on BRN’s response to our audit. The numbers below correspond to the numbers we have placed in the margin of BRN’s response. Notwithstanding the other information that BRN asserts its 1 governing board members consider, the nursing education staff do not typically present regional workforce data to the governing board. Further, as we note on page 19, nursing programs have cited nursing shortages as a reason for requesting an enrollment increase and referenced other forecasts to support their requests. However, BRN’s forecasts do not include relevant regional information that would allow its nursing education staff to verify those assertions. Thus, BRN should ensure that the forecasts it is paying its contractor to develop every two years include regional variations in the projected supply and demand of nurses, to better inform the governing board’s enrollment decisions. We recommended that BRN revise the scope of its contract for 2 workforce forecasting services to incorporate regional analyses and ensure that the governing board’s enrollment decisions and other actions adequately take into consideration those regional analyses in future forecasts. We did not recommend that BRN require nursing programs to provide a plan for evaluation of the total program on an annual basis. BRN misunderstands the time frames of our recommendations. 3 We recommended that by January 1, 2021 BRN establish in policy the specific information its staff should present. As for the time frame for collecting the information, we recommended that BRN compile and aggregate the information by April 1, 2021. Although BRN expressed some concern in its response about promulgating regulations by April 1, 2021, we expect BRN to take actions to implement our recommendations and provide us documentation of its progress as part of its 60‑day, 6‑month, and 1‑year responses. BRN does not describe how the consortium—a group of nursing 4 programs and health care facilities that work together to address clinical placement issues—it mentions in its response would function to address our recommendations. Moreover, we believe BRN can implement our recommendation without using a consortium to identify clinical placements as BRN suggests. Specifically, BRN is well‑positioned to gather and analyze data regarding clinical placements. As we state on page 24 of our report, nursing programs must get BRN approval before using a clinical 48 California State Auditor Report 2019-120 July 2020 facility and BRN documents that approval on a facility approval form. Therefore, BRN should already have a record of all facilities that nursing programs are using for clinical placement slots. We believe that BRN can and should collect on the facility approval form the total number of clinical placement slots a clinical facility can accommodate annually and how many slots the programs that use the facilities will need each year. 5 We believe that it is imperative that BRN implement our recommendations to ensure its governing board has complete information about clinical placements when it is considering enrollment decisions. We look forward to BRN’s 60‑day, six month, and one‑year response to our audit report, which should include documentation demonstrating the actions it is taking to implement our recommendations. 6 To clarify, we note on page 29 of our report that just because a nursing program is not using a facility does not necessarily mean the facility is available for use or willing to provide clinical placement slots for nursing students. However, we believe such a comparison and the necessary follow‑up could identify additional clinical placement slots, thereby alleviating potential constraints on enrollment for nursing programs in areas with nursing shortages. 7 Nothing in our recommendation to the Legislature precludes BRN from taking the actions it identifies in its response. In fact, we believe these actions, if taken, would facilitate the Legislature’s implementation of our recommendation.