CSA

Summary

California State Auditor · 2004-123 · 2004-01-01

Read the report at California State Auditor ↗

California Public Employees’ Retirement System: It Relied Heavily on Blue Shield of California’s Exclusive Provider Network Analysis, an Analysis That Is Reasonable in Approach but Includes Some Questionable Elements and Possibly Overstates Estimated Savings March 2005 2004-123 rotiduA etatS ainrofilaC S T I D U A E T A T S F O U A E R U B The first five copies of each California State Auditor report are free. Additional copies are $3 each, payable by check or money order. You can obtain reports by contacting the Bureau of State Audits at the following address: California State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 (916) 445-0255 or TTY (916) 445-0033 OR This report is also available on the World Wide Web http://www.bsa.ca.gov/bsa/ The California State Auditor is pleased to announce the availability of an on-line subscription service. For information on how to subscribe, please contact the Information Technology Unit at (916) 445-0255, ext. 456, or visit our Web site at www.bsa.ca.gov/bsa Alternate format reports available upon request. Permission is granted to reproduce reports. � � � �� March 29, 2005 2004-123 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, California 95814 Dear Governor and Legislative Leaders: As requested by the Joint Legislative Audit Committee, the Bureau of State Audits presents its audit report concerning the May 19, 2004, decision by the board of administration (board) of the California Public Employees’ Retirement System (CalPERS) to discontinue contracting with certain hospitals through the Blue Shield of California (Blue Shield) health maintenance organization (HMO) provider network. This report concludes that Blue Shield’s analysis is reasonable in approach, but includes some questionable elements such as using non-CalPERS claim data. Blue Shield’s original savings estimate did not incorporate financial contract terms with a health system that were expected to produce substantial savings in 2005 only if the board did not adopt the exclusive provider network. However, the board chose to adopt the exclusive provider network, which resulted in the health system’s financial terms no longer applying in 2005 and 2006. In addition, Blue Shield’s savings estimate of $31.4 million does not consider the impact of members leaving its HMO provider network and joining other health-care plans. Our consultant estimated that the impact on the Sacramento area from member movement could drop Blue Shield’s $5.5 million savings estimate to between $1.7 million and $3.5 million. Further, according to our consultant, Blue Shield’s savings attributable to the exclusion of certain hospitals could drop from $20.6 million to $8.9 million if the model-review actuary’s emergency room assumptions were used. Finally, the CalPERS board, health benefits committee, and health benefits branch staff relied primarily on Blue Shield’s summary of its analyses and its presentations in deciding to approve the exclusive provider network. CalPERS did not fully consider all of the findings and recommendations made by an independent health actuary hired by Blue Shield to review its models prior to the board’s adoption of the exclusive provider network. Respectfully submitted, ELAINE M. HOWLE State Auditor �� CONTENTS Summary 1 Introduction 7 Audit Results Blue Shield Developed an Analysis to Help Control CalPERS’ Health-Care Costs 15 CalPERS Relied Heavily on Blue Shield’s Analyses and Did Not Consider Carefully an Independent Actuary’s Review 16 Although We Found No Evidence of Material Errors in the Claim Data, the Inclusion of Non-CalPERS Claims May Adversely Affect the Accuracy of the Analysis 22 Blue Shield’s Cost Model Appears Reasonable, but in One Instance Blue Shield Did Not Follow Its General Rule for Excluding Hospitals 31 Blue Shield’s Estimate of CalPERS Savings Resulting From the Exclusive Provider Network Is Possibly Overstated 37 Recommendations 43 Appendix A An Overview of Blue Shield’s Methodology for Its CalPERS Exclusive Provider Network Analysis 45 Appendix B Trend Information for Four Sample Hospitals Over a Three-Year Period 53 Responses to the Audit State and Consumer Services Agency, California Public Employees’ Retirement System 63 California State Auditor’s Comments on the Response From the State and Consumer Services Agency and the California Public Employees’ Retirement System 71 Blue Shield of California 79 California State Auditor’s Comments on the Response From the Blue Shield of California 83 California State Auditor Report 2004-123 11 SUMMARY RESULTS IN BRIEF In an effort to control health-care costs, the board of administration (board) of the California Public Employees’ Retirement System (CalPERS) voted on May 19, 2004, to Audit Highlights . . . approve an exclusive provider network1 for the 427,000 CalPERS members2 in the Blue Shield of California (Blue Shield) health Our review of the decision maintenance organization (HMO). The approval excluded 38 by the California Public hospitals. Subsequently, as a result of further negotiations, Employees’ Retirement Blue Shield permitted some of the hospitals to remain in the System (CalPERS) board of network, and the Department of Managed Health Care, which administration (board) in May 2004 to approve an is responsible for regulating health-care service plans in the exclusive provider network State, denied the exclusion of four others. As of January 1, 2005, for CalPERS members in the 24 hospitals were excluded from the Blue Shield HMO provider Blue Shield of California (Blue Shield) health network. Blue Shield estimated the hospital savings resulting maintenance organization from this network to be $20.6 million and the savings associated (HMO) found the following: with some medical groups to be $10.8 million, for a total of þ Our consultants found $31.4 million in savings to CalPERS in 2005.3 that many components of Blue Shield’s analysis In establishing this network, Blue Shield expected to save CalPERS appear reasonable money by excluding high-cost hospitals and medical groups but some questionable elements exist such as that admit only to those hospitals. An actuary Blue Shield hired using claim data from to review its models (model-review actuary), at the request non-CalPERS sources. of CalPERS, found that its original savings estimate did not þ Blue Shield’s original incorporate recent financial contract terms with a health system savings estimate did not that were expected to produce savings only if CalPERS retained incorporate a health the full provider network. According to the model-review actuary, system’s financial terms Blue Shield should have factored these savings into the baseline that were expected to produce substantial for the full provider network before projecting the savings savings in 2005 only if the CalPERS would realize by switching to the exclusive provider board did not adopt the network. The new contract between Blue Shield and the health exclusive provider network. system contained a clause with certain financial terms that would þ Blue Shield’s estimate of be available to CalPERS if it did not adopt an exclusive provider $31.4 million in savings does not take into consideration the impact 1 In this report, we refer to Blue Shield’s health maintenance organization provider of members leaving its network for CalPERS members as the exclusive provider network. HMO provider network and joining other health- 2 In this report, we refer to CalPERS members, retirees, and their survivors and beneficiaries collectively as members. care plans. 3 As of May 19, 2004, when the board made its decision to exclude 38 hospitals, Blue continued on next page . . . Shield estimated hospital savings to be $27.7 million and savings associated with some medical groups to be $8.6 million for a total of $36.3 million savings to CalPERS in 2005. For the purposes of our report, unless otherwise stated, the term savings estimate refers to Blue Shield’s estimate of $31.4 million that relates to the exclusion of 24 hospitals. In Appendix A we present an overview of Blue Shield’s analysis. California State Auditor Report 2004-123 11 þ Blue Shield did not network. If the CalPERS board approved the exclusive provider adequately address a network, the financial terms would not apply in 2005 and 2006. recommendation to Blue Shield estimated that the financial terms would result in investigate differences substantial savings to CalPERS in 2005. in emergency room assumptions for one health system. According to our In response to the model-review actuary’s recommendation, Blue consultant, Blue Shield’s Shield stated that due to the private and confidential nature of its hospital savings estimate of $20.6 million could drop hospital agreements, it was unable to present to the public or in to only $8.9 million if the open board or health benefits committee (committee) meetings model-review actuary’s the hospital’s contract savings. However, Blue Shield did present an assumptions were used. estimate of the impact of the savings to CalPERS’ committee during þ The CalPERS board, health a closed meeting held on March 16, 2004, and did present to the benefits committee, and board the impact of the health system’s financial terms during a health benefits branch closed meeting held on May 11, 2004. staff relied primarily on Blue Shield’s summary of its analyses and its During an open session at the board’s May 11, 2004, meeting, presentations in deciding Blue Shield presented two options. One option was to maintain to approve the exclusive the full network under the following conditions: accept the provider network. health system’s financial offer, provide higher-cost hospitals an þ Although a model-review opportunity to bring their costs closer to the industry average, actuary was hired to, and maintain the option of adopting an exclusive provider among other things, review Blue Shield’s cost network in the future (for example, January 1, 2006). The other savings projections, he option was to adopt the exclusive provider network effective was unable to express an January 1, 2005. On May 19, 2004, the board approved the opinion on the savings latter option, which resulted in the health system’s financial estimate of $36.3 million related to the 38 hospitals; terms no longer applying in 2005 and 2006. According to the thus, his report could not current deputy executive officer for benefits administration, the provide a credible basis board chose to proceed with the exclusive provider network for the CalPERS board because it sought to generate savings beyond the health system’s to evaluate the savings estimate. contract period and to initiate structural reform in the health care industry. This statement is consistent with our review of the þ In one instance, our transcripts of meetings prior to the board’s approval. consultant found that Blue Shield deviated from its original criteria for However, Blue Shield’s estimate of $31.4 million in savings for excluding hospitals from the exclusive provider network does not take into consideration the network. the impact of members leaving its HMO provider network and joining other health-care plans. A preliminary analysis prepared by Blue Shield, using CalPERS data, estimated that it lost 34,000 members during CalPERS’ December 2004 open enrollment period. According to an analysis prepared by CalPERS in February 2005, almost 16,000 Sacramento-area members left the Blue Shield HMO provider network during the open enrollment period.4 There could be a number of reasons why members chose to leave Blue Shield’s HMO provider network. However, as 4 CalPERS includes the counties of Sacramento, Placer, and Yolo in its analysis. 22 California State Auditor Report 2004-123 California State Auditor Report 2004-123 33 a result of this member movement, according to our consultant, Blue Shield’s savings estimate of $5.5 million for the Sacramento area could drop to between $1.7 million and $3.5 million.5 CalPERS’ analysis did not include similar information for other areas of the State. Therefore, we are unable to quantify the full effect that member movement has had on Blue Shield’s savings estimate. Blue Shield also did not adequately address a recommendation made by the model-review actuary to investigate differences in the emergency room assumptions for one hospital system. According to our consultant, Blue Shield’s hospital savings estimate of $20.6 million could drop to only $8.9 million if the model-review actuary’s assumptions were used. Given the sensitivity of the hospital savings estimate to differences in emergency room assumptions, Blue Shield should have reconciled the two analyses to ensure that its estimate of emergency room costs was reasonable. According to Blue Shield, if the savings from the exclusive provider network differ materially from its original estimate, it will adjust CalPERS’ future premiums. Specifically, a provision in Blue Shield’s contract with CalPERS requires it to compare the actual cost of health care to its projected costs. If the difference falls outside a certain range, Blue Shield will adjust for the difference when calculating the projected health-care costs for the following year, which could potentially increase CalPERS’ premiums. Blue Shield’s savings estimate are based on an analysis that consists of three distinct models: the Milliman USA, Inc., RBRVS for HospitalsTM Relative Value Unit Fee Schedule (Milliman model); the cost model; and the savings model. Blue Shield also developed a fourth model to estimate savings from financial terms for calendar years 2004 and 2005 that it negotiated with one health system after its initial analysis was completed. Our consultants found that many components of Blue Shield’s analysis appear reasonable but it contains some questionable elements such as using non-CalPERS claim data sources. 5 Information was not available on the cost of providing care to the members who leave Blue Shield and therefore this was not considered in the analysis. If the members moved to health plans that use the high-cost hospitals excluded from the Blue Shield HMO provider network then the lost savings may not be realized by CalPERS. If they move to health plans that do not include these high-cost hospitals or use them less frequently, then CalPERS may realize some of the lost savings. However, without knowledge of the rates these hospitals charge other health plans, it is not possible to further refine the effect that member movement has on Blue Shield’s estimate. 22 California State Auditor Report 2004-123 California State Auditor Report 2004-123 33 Our review found that the CalPERS board, committee, and health benefits branch staff relied primarily on Blue Shield’s summary of its analyses and its presentations in deciding to approve the exclusive provider network. A provision of the contract between CalPERS and Blue Shield prohibits Blue Shield from disclosing information that would breach the terms of contracts—including payment rates—with its provider hospitals. As a result, CalPERS did not have access to either hospital rates or Blue Shield’s cost model and was therefore unable to verify the model’s accuracy. Health benefits branch staff confirmed their reliance on Blue Shield to prepare technical analyses and on an independent actuary to verify the results of Blue Shield’s analysis. Although the model-review actuary concluded that Blue Shield’s general method of analysis was reasonable, his report indicates that time constraints prohibited him from fully addressing some important assumptions and elements of the analysis. For example, although the model-review actuary was hired to, among other things, review Blue Shield’s cost savings projections for the exclusive provider network, he was unable to express an opinion on Blue Shield’s savings estimate of $36.3 million that related to the exclusion of the 38 hospitals. Thus, his report could not provide a credible basis for the CalPERS board to evaluate Blue Shield’s savings estimate prior to adopting the exclusive provider network and excluding certain hospitals. Although the board and committee discussed Blue Shield’s savings estimate, our review found that the transcripts and meeting notes for the board’s and committee’s closed and open meetings held prior to the board’s approval of the exclusive provider network did not discuss the actuary’s inability to conclude on the savings estimate or all of his remaining findings and recommendations and their impact on CalPERS’ decision. CalPERS staff also did not investigate the model-review actuary’s other findings and recommendations that Blue Shield did not address fully but may have a significant impact on its analysis. Without addressing the model-review actuary’s concerns, CalPERS had no assurance from an independent source that Blue Shield’s analysis was accurate. A review of the Milliman model by our consultant found that it provides a reasonable basis for comparing relative reimbursement levels across hospitals. In addition, our consultant found no evidence of material errors in the claim data underlying the Milliman model. However, Blue Shield’s inclusion of non-CalPERS claim data, while arguably necessary to increase its sample size may adversely affect the accuracy of the analysis, due in 44 California State Auditor Report 2004-123 California State Auditor Report 2004-123 55 part, to varying reimbursement rates under different contracts. In deciding which hospitals to include in its network, Blue Shield assigned each hospital a quality-adjusted relative cost factor. Our consultant found that eight hospitals that had cost factors above the exclusion threshold (the cost factor threshold above which hospitals were subject to exclusion from the provider network) when the additional claims were used had costs below the threshold using only HMO claim data.6 Conversely, nine hospitals that had costs below the threshold when the additional claims were used had costs above the threshold using HMO claim data alone. These differences may be important, since these 17 hospitals provided more than 10 percent of hospital services to HMO enrollees as measured by relative value units, a measure of the resources needed by hospitals to provide services. However, our consultant concluded that, in as much as there was no ideal source of claim data to use in its analysis, Blue Shield’s decision to include the additional data does not appear unreasonable. Furthermore, our consultant found that in one instance Blue Shield deviated from its original criteria for excluding hospitals from the network. Specifically, in one geographic area, or cohort, rather than evaluating each hospital in a particular system separately against the cohort average, using HMO and preferred provider organization data, as called for by its rules, Blue Shield evaluated all the hospitals in one system as a group, using a different set of claim data. Since the system as a whole met the threshold, Blue Shield included all hospitals in the system in its exclusive provider network, even though one of them would not have met the threshold if it had been evaluated separately. RECOMMENDATIONS The Legislature should consider enacting legislation that would allow CalPERS, during its contract negotiation process, to obtain relevant documentation supporting any analyses it will use to make decisions that materially affect the members of the health benefits program established by the Public Employees’ Medical and Hospital Care Act. To ensure that its decisions are in the best interest of CalPERS members, CalPERS should require its health benefits branch staff to evaluate fully the findings and recommendations of third-party reviews and present their results to the board and committee. 6 In Appendix A we present an overview of Blue Shield’s analysis, including its inclusion threshold. 44 California State Auditor Report 2004-123 California State Auditor Report 2004-123 55 AGENCY COMMENTS Both CalPERS and Blue Shield disagree with our conclusion that Blue Shield’s estimate of cost savings related to the exclusive provider network may be overstated. In addition, both disagree with our conclusion that without addressing the concerns raised in the model-review actuary’s report, CalPERS had no assurance that Blue Shield’s analysis was accurate. Finally, Blue Shield believes that the elements identified as questionable in our report had no material effect on the exclusive provider network analysis on which CalPERS relied. Our comments follow each response. n 66 California State Auditor Report 2004-123 California State Auditor Report 2004-123 77 INTRODUCTION BACKGROUND The State established the California Public Employees’ Retirement System (CalPERS) in 1932. Its mission is to advance the financial and health security of participants in the system. CalPERS participants include members, retirees, and their survivors and beneficiaries. For the purposes of this report, we refer to these participants collectively as members. CalPERS is administered by a board of administration (board) containing 13 members, of which six are elected by CalPERS members, three are appointed either by the governor or jointly by the speaker of the Assembly and the Senate Committee on Rules, and four are designated by statute. State law requires the board, its officers, and employees to perform their duties solely in the interest of CalPERS members by providing benefits; defraying reasonable expenses of administering the system; minimizing employers’ costs of providing benefits; and investing with the care, skill, and diligence that a prudent person in a like capacity would use. Nine of the 13 board members also serve on the CalPERS health benefits committee (committee), which oversees the administration of the Public Employees’ Medical and Hospital Care Act (act). The act, which became law in 1962, authorized CalPERS to establish a health benefits program (program) for state employees. Subsequent amendments to the act expanded the program to include employees of public agencies and schools.7 The CalPERS health benefits branch oversees the program. The branch consists of four offices, including the Office of Health Plan Policy and Administration and the Office of Decision and Program Support Services. Among other things, these two offices play a role in negotiating health plan premiums. The program offers CalPERS members health-care coverage through four health maintenance organizations (HMOs) and four preferred provider organizations, as shown in the text box on the following page. According to CalPERS, its program provided health coverage to 1.2 million members as of January 31, 2005. Nearly 859,000, or 72 percent, of these 7 CalPERS’ definition of schools includes school districts, charter schools, county offices of education, and community colleges. 66 California State Auditor Report 2004-123 California State Auditor Report 2004-123 77 members are covered by the HMOs. Blue Shield of California (Blue Shield) has been providing services CalPERS offers the following health- under CalPERS since 1988 and currently provides care plans to its members: coverage to roughly half of the 859,000 members. Health Maintenance Organizations CalPERS members constitute roughly 31 percent of the 1.4 million members Blue Shield serves through • Blue Shield of California its HMO provider network. • Kaiser Health Plan Foundation, Inc. • Western Health Advantage • Health Net–California Correctional In response to concerns about rising health-care Peace Offi cers Association* costs, the board approved, on May 19, 2004, Preferred Provider Organizations a motion to support an exclusive Blue Shield HMO provider network for CalPERS employees. • PERSCare Specifi cally, the board approved the exclusion of • PERS Choice • California Association of Highway 38 hospitals from the Blue Shield HMO provider Patrolmen Health Benefi ts Trust* network. CalPERS stated publicly that the exclusive • Peace Offi cers Research Association of provider network would generate savings of California* up to $36 million in calendar year 2005 and $50 million per year thereafter. Subsequent to the Sources: Department of Managed Health Care, CalPERS Web site, and evidence of coverage with board’s approval, Blue Shield allowed 10 of the the PPOs. 38 hospitals to remain in its network after they met * Participation in the plan is limited to members in its cost and quality criteria. these organizations. Because Blue Shield is licensed by the Department of Managed Health Care (DMHC) as a full-service health plan, its action to modify the HMO provider network available to CalPERS members was subject to the approval of DMHC, which is responsible for regulating health-care service plans in the State. A full-service health plan provides at least six basic health-care services to its members, including physician care, hospital inpatient and ambulatory care, and emergency health care. State law requires that, when a licensed health plan intends to implement material modifi cations to its plan or operations, the health plan must give notice to the director of DMHC. Material modifi cations include mergers and acquisitions, service area expansions, and product withdrawal from a market. State law further requires the director of DMHC to issue an order to approve, disapprove, suspend, or postpone the effectiveness of a health plan’s material modifi cation within 20 business days from the date of a health plan’s notice, or longer if the plan specifi es additional time. Of the 28 hospitals presented to it for exclusion, DMHC denied the exclusion of four on August 5, 2004, citing concerns about CalPERS members’ access to care. However, DMHC approved the exclusion of the remaining 24 hospitals and their 88 California State Auditor Report 2004-123 California State Auditor Report 2004-123 99 10 related medical groups from Blue Shield’s HMO provider network effective January 1, 2005.8 Blue Shield estimated that the exclusion of these hospitals and their related medical groups would save CalPERS $31.4 million in 2005. As of January 1, 2005, more than 300 hospitals and 200 medical groups remain available to CalPERS members in Blue Shield’s HMO exclusive provider network. Figure 1 on the following page shows the locations of the 24 hospitals that were excluded from the Blue Shield HMO provider network. In addition to approving the exclusion of 24 hospitals and denying the exclusion of four hospitals, DMHC ordered Blue Shield to do the following: • Require six medical groups in four counties in the greater Sacramento area to provide access for CalPERS members to specialty services in accordance with DMHC’s access standards, monitor CalPERS members’ complaints and take appropriate corrective action, and monitor the first available appointment for a new patient consult for the first 12 months following implementation of the exclusive provider network and quarterly thereafter. • Authorize admissions to and provide full benefits for CalPERS members for medically necessary admissions at three hospitals that are otherwise excluded from the provider network, and authorize and provide full benefits for CalPERS members for outpatient radiology services at one hospital. • Upon a member’s request, offer affected CalPERS members who would qualify for continuity of care the ability to continue care with an excluded provider. Finally, according to DMHC, it took additional steps to ensure the smooth transition of CalPERS members. Specifically, DMHC stated that it began monitoring complaints to identify any problems, including problems with members selecting new providers, and communicating with Blue Shield to discuss and resolve any concerns. DMHC also stated it conducted an audit of all continuity of care denials for serious and chronic conditions and required Blue Shield to submit bi-weekly reports on continuity of care requests. 8 Catholic Healthcare West sold to Kaiser Permanente one of the 24 excluded hospitals, St. Dominic’s Hospital, effective November 1, 2004. Blue Shield included the hospital in its calculation of cost savings. 88 California State Auditor Report 2004-123 California State Auditor Report 2004-123 99 FIGURE 1 Hospitals Excluded From Blue Shield HMO Network Effective January 1, 2005 �� Source: CalPERS. 1100 California State Auditor Report 2004-123 California State Auditor Report 2004-123 1111 SCOPE AND METHODOLOGY The Joint Legislative Audit Committee (audit committee) requested that the Bureau of State Audits (bureau) examine the CalPERS decision to discontinue contracting with certain hospitals through the Blue Shield HMO provider network. Specifically, the audit committee directed the bureau to examine the information the board used to make its decision to exclude hospitals from the provider network and to estimate the resulting cost savings. In addition, the audit committee instructed the bureau to determine whether the information, including any analyses, data, and methodologies, is valid and provides a clear case for conclusions drawn. Further, the audit committee asked the bureau to select a sample of hospitals or hospital systems affected by the CalPERS board decision and, to the extent possible, identify trends for at least a three-year period in the following areas: profit margins and prices; the systems’ market share of physicians and hospital services; the percentage of revenue going to patient care, administration, and profits; expenditures for charity care and other community benefits; operating characteristics; and executive compensation. To obtain an understanding of CalPERS’ role in providing health benefits for its members, we reviewed relevant laws. In examining the information the board used to make its decision to exclude certain hospitals from the Blue Shield HMO provider network, we reviewed agendas, minutes, and transcripts of the CalPERS committee and board meetings. We also reviewed presentations made by Blue Shield to the CalPERS committee and the board about the decision as to whether to use an exclusive provider network beginning in 2005. Further, we reviewed the contract CalPERS entered into with Blue Shield, effective for the three-year period from January 1, 2004, through December 31, 2006, to identify the limitations it placed on access to private and confidential information. Finally, we interviewed key CalPERS health benefits branch staff. To determine whether the analyses, data, and methodologies presented by Blue Shield to CalPERS are valid and provide a clear case for the decision to exclude hospitals from its HMO provider network, we hired two consultants: a firm with broad experience in analyzing health-care costs and benefits, pricing strategies, and models (consultant) and an actuary. Our consultant interviewed both key Blue Shield and Milliman USA, Inc., (Milliman) staff. In addition, our consultant reviewed the analyses, data, and methodology relating to Blue Shield’s models that were used to support its recommendation of the hospitals 1100 California State Auditor Report 2004-123 California State Auditor Report 2004-123 1111 and medical groups to exclude.9 However, our consultant did not review the underlying claim data or the Milliman USA, Inc., RBRVS for HospitalsTM Relative Value Unit Fee Schedule (Milliman model), although the consultant did review and perform sensitivity analysis related to the output of the Milliman model. In evaluating the cost model, our consultant reviewed Blue Shield’s calculation of cost factors for inpatient and outpatient services at each hospital in its provider network. In addition, our consultant reviewed Blue Shield’s calculation of average cost factors for hospitals in geographic regions, called cohorts, and analyzed Blue Shield’s adjustments to each hospital’s relative cost factor for quality, efficiency, and rate increases. In evaluating the savings model and a separate model used to estimate savings for one health system, our consultant reviewed Blue Shield’s calculations for estimating the savings from excluding given hospitals and medical groups and conducted tests of the data. In Appendix A we present an overview of Blue Shield’s analysis. Our consultant also reviewed the report issued by an actuary hired by Blue Shield to conduct a third-party review of its models (model-review actuary). Our consultant interviewed the model-review actuary and examined the documentation supporting the actuary’s findings and recommendations. Finally, our actuary reviewed the Milliman model and performed a peer review of our consultant’s analyses. Our actuary believes that the Milliman model, as used in the Blue Shield analysis, should produce reliable results. Because state law requires DMHC to approve material modifications to the plan or operations of health-care service plans, we reviewed laws and regulations related to its role in regulating health plans and the procedures DMHC used in its review of Blue Shield’s material modification. To assess whether DMHC’s decision was consistent with its statutory authority, we judgmentally selected a sample of three hospitals—two hospitals that DMHC approved to exclude and one hospital it required Blue Shield to retain in its HMO provider network. For each hospital, we reviewed DMHC’s documentation and interviewed key staff about its decision. For the three hospitals we reviewed, 9 The analyses, data, and methodologies used in Blue Shield’s analysis of the exclusive provider network include private and confidential information, such as the terms of its contracts with providers. Thus, the bureau is prohibited from disclosing specific information about hospitals or hospital systems. 1122 California State Auditor Report 2004-123 California State Auditor Report 2004-123 1133 we found that DMHC’s decision was based on CalPERS members’ access to care and continuity of care, consistent with its statutory responsibility. To identify certain trends over a three-year period, we judgmentally selected a sample of four hospitals—two that had been excluded from the Blue Shield HMO provider network and two that were potentially affected by the exclusion of other hospitals. We considered a hospital to be affected by the exclusion when it was proposed by Blue Shield to receive CalPERS member patients being covered by excluded hospitals in the same geographic area. Depending on their reporting period, we requested information from the four hospitals for either the three calendar years from 2001 through 2003 or for the three fiscal years from 2001–02 through 2003–04. In Appendix B we present our methodology and the trend information requested by the audit committee. n 1122 California State Auditor Report 2004-123 California State Auditor Report 2004-123 1133 Blank page inserted for reproduction purposes only. 1144 California State Auditor Report 2004-123 California State Auditor Report 2004-123 1155 AUDIT RESULTS BLUE SHIELD DEVELOPED AN ANALYSIS TO HELP CONTROL CALPERS’ HEALTH-CARE COSTS At the request of the board of administration (board) of the California Public Employees’ Retirement System (CalPERS), Blue Shield of California (Blue Shield) developed its analysis for the exclusive provider network10 (analysis) to help control CalPERS’ health-care costs. The objective of the analysis was to reduce the growth in hospital costs while maintaining quality and minimizing member-physician disruptions within an exclusive provider network approved by the Department of Managed Health Care. Blue Shield’s analysis was based on three distinct models it uses within the course of its normal business and a fourth model it created specifically for CalPERS. Although we present an overview of Blue Shield’s analysis in Appendix A, below is a summary of the four models: • Milliman USA, Inc., RBRVS for HospitalsTM Relative Value Unit Fee Schedule (Milliman model), which adjusts for differences among hospitals’ inpatient populations and the types of services hospitals provide. • Cost model, which determines the cost of each hospital relative to the costs of other hospitals in the same geographic area, or cohort. • Savings model, which estimates the hospital and physician savings that would be realized by excluding high-cost hospitals and associated medical groups from the network. • Savings-in-base model, which estimates the savings from financial terms for calendar years 2004 and 2005 that Blue Shield negotiated with a health system after it completed its initial analysis. 10 Blue Shield has conducted multiple versions of the analysis. Unless otherwise stated, the version referred to in this report is Blue Shield’s Network Choice V (NC5). 1144 California State Auditor Report 2004-123 California State Auditor Report 2004-123 1155 CALPERS RELIED HEAVILY ON BLUE SHIELD’S ANALYSES AND DID NOT CONSIDER CAREFULLY AN INDEPENDENT ACTUARY’S REVIEW CalPERS health benefits branch (branch) staff, health benefits committee (committee) members, and board members relied on Blue Shield’s summary of its analyses and presentations when making the decision to exclude 38 hospitals from the Blue Shield health maintenance organization (HMO)11 provider network available to its members. For the purposes of this report, we refer to this network as the exclusive provider network. The contract between CalPERS and Blue Shield does not allow CalPERS access to certain information about Blue Shield’s HMO provider network that is subject to confidentiality obligations with third parties or otherwise protected from disclosure by law. Because Blue Shield could not provide CalPERS with access to the specific underlying data, at CalPERS’ request, it hired an actuary to review its models. However, the CalPERS board apparently did not consider carefully all of the actuary’s findings and recommendations in its deliberations before it voted to approve the exclusive provider network, a decision that could potentially disrupt existing health care provider relationships for 47,000 CalPERS members. CalPERS Relied Primarily on Blue Shield’s Summary of Its Analyses and Presentations in Making the Decision to Exclude Hospitals Because the terms of the A provision of the contract between CalPERS and Blue Shield contracts between Blue specifies that Blue Shield cannot disclose information to CalPERS Shield and providers in that would cause it to breach the terms of any contract to which its network specifically it is a party. According to Blue Shield, the terms of the contracts prohibit the disclosure between it and providers in its network specifically prohibit the of certain information, disclosure of certain information, including rates of payment. CalPERS health benefits Consequently, CalPERS health benefits branch staff did not branch staff did not have have access to hospital rates, nor could they review Blue Shield’s access to hospital rates, cost model. As a result, CalPERS staff were unable to verify the nor could they review accuracy of Blue Shield’s cost comparison data. Blue Shield’s cost model. According to CalPERS, its health benefits branch staff participated in meetings and conference calls with Blue Shield on five days between July 2003 and October 2003. Staff 11 The Blue Shield health plan available to CalPERS members consists of an HMO and an exclusive provider organization, but for simplicity we refer to this as an HMO. 1166 California State Auditor Report 2004-123 California State Auditor Report 2004-123 1177 stated that they reviewed Blue Shield’s analyses of hospitals’ relative costs compared to statewide and regional averages, its methodology for comparing cost and quality, and its assessment of the lack of correlation between hospital costs and quality, as well as Blue Shield’s data sources. Further, CalPERS stated that during a four-hour working session on February 6, 2004, Blue Shield shared two scenarios for eliminating hospitals from the HMO provider network—including its proposed methodology, expected savings, and the expected disruption to CalPERS members—with two health benefits branch staff managers, who were comfortable with the methodology and with putting the scenarios before the board. However, we are unable to reach any conclusion regarding the depth of the staff’s review of the data, methodologies, and analyses because, according to staff, Blue Shield requested that they return their notes and any handouts provided to them. After the February 6, 2004, working session, CalPERS health benefits branch staff and executive staff spent time following up on concerns raised by the board and providing feedback to Blue Shield on draft presentations to be made at board and committee meetings. However, branch staff also confirmed their reliance on Blue Shield to prepare the technical analyses and on an independent actuary to verify the methodology and results due to the private and confidential nature of the information. Similarly, the board and the committee relied heavily on the branch staff agendas and on Blue Shield’s presentations at board and committee meetings. However, these agendas and presentations provided only a high-level description of Blue Shield’s methodology and aggregate information relating to Although it requested the hospitals, without touching on the underlying data. For more detail regarding the example, Blue Shield’s presentation at the February 18, 2004, analysis, CalPERS stated committee meeting briefly described its methodology for that Blue Shield had comparing hospital costs, choosing quality indicators, assessing responded with as much member access to services, and the potential savings for various detailed information as network scenarios. According to CalPERS, although it requested it was able to provide, more detail, Blue Shield had responded with as much detailed given the confidential information as it was able to provide, given the confidential nature of the information. nature of the information. Without full access by CalPERS staff to all of the detailed information in Blue Shield’s methodologies, data, and analyses, board and committee members had to rely almost exclusively upon Blue Shield’s assertions about the validity of its information relating to the exclusive provider network. 1166 California State Auditor Report 2004-123 California State Auditor Report 2004-123 1177 CalPERS Did Not Fully Consider All of the Findings and Recommendations Made by the Actuary Hired to Perform a Third-Party Review Prior to Approving the Exclusive Provider Network A few hospitals and health systems in Blue Shield’s HMO provider network expressed concerns about the methodology, data, and analyses Blue Shield used to determine the exclusive provider network. In fact, one health system presented to CalPERS and Blue Shield its own analysis of its hospital charges, which had different results than Blue Shield’s analysis. A primary factor in the differences was the data used in the analyses. Specifi cally, the health system’s analysis was prepared using Offi ce of Statewide Health Planning and Development (OSHPD) data, while Blue Shield used its claim data. Because of the controversy surrounding its analysis, in a discussion at the April 8, 2004, CalPERS board meeting, a Blue Shield representative pointed out that a provision in the contract between it and CalPERS permits CalPERS to use an independent health actuary to audit the data and methods Blue Shield uses to establish rates and payments. According to the current and former deputy executive offi cer for benefi ts administration, the board directed Blue Shield contracted with an actuary CalPERS staff to proceed with a third-party review to to perform the following services: resolve the differences between Blue Shield’s and the • Review the Milliman USA, Inc. analysis of health system’s analyses. Blue Shield’s reimbursement by hospital. • Review a health system’s analysis of its According to Blue Shield, CalPERS benefi ts branch charges. staff directed it to hire an independent health actuary (model-review actuary). On April 12, 2004, Blue Shield • Reconcile, to the extent possible, the conclusions from the health system’s notifi ed an actuary of its need for his services to analysis to the conclusions drawn from the conduct a third-party review of its analysis for Milliman analysis. CalPERS (see the text box). According to Blue Shield, • Review cost savings projections for the one of the primary factors in hiring the actuary it exclusive provider network. chose was the actuary’s experience working with the • Prepare a written summary of observations, Milliman model used by Blue Shield and the actuary’s comments, and recommendations working relationship with the health system that regarding the analyses and conclusions drawn from each. proposed the alternative analysis. The actuary was to complete the review by April 19, 2004. However, • Participate in a CalPERS committee meeting to be held on April 20, 2004. the model-review actuary’s report indicates that this short time frame did not provide him suffi cient time Source: Reden and Anders, Ltd., a health actuarial to conduct a thorough, detailed review. The model- consulting fi rm. review actuary conducted his review and provided a draft report to Blue Shield and CalPERS branch staff within three days. Blue Shield planned to have the model-review actuary describe the results of his review at a committee meeting on April 20, 2004. However, the meeting transcript does not include a discussion of the actuary’s review. 1188 California State Auditor Report 2004-123 California State Auditor Report 2004-123 1199 The model-review actuary issued his final report to Blue Shield and CalPERS branch staff on April 26, 2004. He concluded that it is inappropriate to use OSHPD data as the sole source for hospital comparisons because OSHPD data are not available to the public in sufficient detail to allow a credible hospital-specific, payor-specific cost analysis. He also concluded that in general Blue Shield’s method of analyzing hospital costs, which includes using its claim data and explicit consideration of a significant number of cost elements, is a reasonable basis for comparing hospital costs and should provide a credible basis for projecting the financial impact of the exclusive provider network. In addition to reconciling the conclusions from the health system’s analysis to the conclusions drawn from the Blue Shield analysis, the model-review actuary was hired to review Blue Shield’s cost savings projections for the exclusive provider network of $36.3 million.12 The model-review actuary concluded that Blue Shield’s key assumptions and variables in its cost model were used appropriately and, as a result, that the cost savings calculated should be reasonable unless there is a flaw in the underlying data The model-review or in the creation of the base evaluation criteria. actuary did not express an opinion regarding Although the model-review actuary concluded that Blue Shield’s Blue Shield’s savings general method of analysis was reasonable, time constraints did estimate, and thus his not allow him to fully address some important assumptions and report could not provide elements relating to Blue Shield’s savings estimate, as well as a credible basis for other aspects of the analysis. Consequently, the model-review the CalPERS board to actuary did not express an opinion regarding Blue Shield’s evaluate Blue Shield’s cost savings estimate, and thus his report could not provide a savings projection prior credible basis for the CalPERS board to evaluate Blue Shield’s to its decision to adopt cost savings projection prior to its decision to adopt the the exclusive provider exclusive provider network and exclude certain hospitals. network and exclude certain hospitals. Because of the time constraints, the model-review actuary disclosed in his report some other limitations to his review that are key to Blue Shield’s analyses for the exclusive provider network. For example, he did not review the Milliman model or any details used in the calculation of the model. Instead, he 12 As of May 19, 2004, when the board made its decision to exclude 38 hospitals, Blue Shield estimated savings to be $27.7 million and savings associated with some medical groups to be $8.6 million for a total of $36.3 million savings to CalPERS in 2005. For the purposes of our report, unless otherwise stated, the term savings estimate refers to Blue Shield’s estimate of $31.4 million that relates to the exclusion of 24 hospitals. In Appendix A we present an overview of Blue Shield’s analysis. 1188 California State Auditor Report 2004-123 California State Auditor Report 2004-123 1199 relied on the opinions of others about the relative value units (RVUs), which are used to measure the resources needed by the hospitals to perform procedures. In another example, the model-review actuary did not review the information used to develop the quality submodel. Also, although he compared each hospital’s results with and without the quality adjustment, he did not test the impact of varying the percentages Blue Shield used in its adjustment. Furthermore, the model-review actuary reported that his review did not include assessing the capacity, availability, and accessibility of services that must be addressed when limits are imposed on patient access in provider networks. He stated that these assessments were important because provider capacity would likely have the biggest influence on out-of-network use and the resultant cost savings. The model-review actuary made numerous findings and recommendations, some of which he either did not specifically identify as having a material impact or was silent on the course of action Blue Shield should take. According to Blue Shield, it addressed the three recommendations it identified as potentially material. Specifically, it revised the cost model to address the model- review actuary’s finding that the model should account not only for emergency visits at excluded hospitals but also additional services related to the emergency visits. Blue Shield stated that, in response to another finding, it reviewed the capacity at hospitals included in the provider network. Further, Blue Shield stated that it followed up on the recommendation to investigate a health system’s assertion that a certain percentage of revenue resulted from emergency Although the board and visits, which affects one of the assumptions Blue Shield used in its committee discussed cost model and thus could have a significant effect on the savings Blue Shield’s savings estimate. Finally, Blue Shield stated that both it and Milliman estimate, our review reviewed the model-review actuary’s report and believe that all of the transcript found other recommendations, as presented, would not have a significant that they did not discuss impact on the overall savings. all of the model-review actuary’s findings Although the board and committee discussed Blue Shield’s and recommendations savings estimate, our review of the transcripts found that or their impact on they did not discuss all of the model-review actuary’s findings CalPERS’ decision in the and recommendations or their impact on CalPERS’ decision meetings held before in the meetings held before the board voted to approve the the board voted to exclusive provider network on May 19, 2004. Our review of the approve the exclusive transcript for the May 18, 2004, committee meeting found no provider network on mention of the model-review actuary’s report, and the report May 19, 2004. was mentioned only briefly in the May 11 and May 19 board meetings. Specifically, a Blue Shield representative commented 2200 California State Auditor Report 2004-123 California State Auditor Report 2004-123 2211 at the May 11 board meeting that Blue Shield had gone through a third-party review of its hospital cost model since the previous board meeting. Also, at the May 11 board meeting, a board member stated incorrectly that Blue Shield’s model-review actuary had certified that the health system’s costs were 60 percent higher than those of Northern California hospitals and 80 percent higher than those of Southern California hospitals. The model-review actuary’s report does not contain this statement. At the May 19 board meeting, a representative for the city of Tracy stated that he was provided a document showing that the health system’s charges were in the middle of the pack. A board member stated that he thought the document was prepared by the health system and that CalPERS had an independent third-party review it. During the May 11 and May 19 board meetings, Blue Shield did present to the board a summary of its savings estimate for the full and exclusive provider networks. However, we found The former deputy no mention in the transcripts of the model-review actuary’s executive officer for inability to render an opinion on this savings estimate. We benefits administration discuss the board’s action regarding the savings estimate more stated that the board fully later. was satisfied with the model-review actuary’s According to the former deputy executive officer for benefits conclusion that in general administration, he personally briefed the board on the results of Blue Shield’s method the model-review actuary’s report, although this briefing is not of analyzing costs is reflected in the transcripts. CalPERS’ notes for the April 20, 2004, a reasonable basis for closed committee meeting indicate that his briefing was limited to comparing hospital the model-review actuary’s conclusions regarding the differences costs and should provide between Blue Shield’s and the health system’s analysis. According a credible basis for to the former deputy executive officer for benefits administration, projecting the financial the board was satisfied with the model-review actuary’s conclusion impact of the exclusive that in general Blue Shield’s method of analyzing costs is provider network. a reasonable basis for comparing hospital costs and should provide a credible basis for projecting the financial impact of the exclusive provider network. However, as we stated previously, the model-review actuary disclosed in his report some limitations to his review, due to time constraints, that are key to Blue Shield’s analysis for the exclusive provider network. Without fully addressing all of the concerns raised by the model- review actuary, CalPERS had no assurance from an independent source that Blue Shield’s savings estimate, as well as other aspects of the model, were accurate. We discuss some of the model-review actuary’s other findings and recommendations 2200 California State Auditor Report 2004-123 California State Auditor Report 2004-123 2211 that Blue Shield did not address fully but could have a significant impact on its analyses and savings estimate more fully later. ALTHOUGH WE FOUND NO EVIDENCE OF MATERIAL ERRORS IN THE CLAIM DATA, THE INCLUSION OF NON-CALPERS CLAIMS MAY ADVERSELY AFFECT THE ACCURACY OF THE ANALYSIS Our consultant’s analyses found no evidence of material errors in the claim data underlying Blue Shield’s analysis of the exclusive provider network. The consultant did, however, find that Blue Shield’s use of non-CalPERS claims in its analysis may either produce inaccurate results or materially affect the relative cost rankings of some hospitals. The Model-Review Actuary Did Not Review the Underlying Claim Data Used in Blue Shield’s Analysis The model-review actuary stated that he did not review or audit any of the underlying claim data, nor did he audit Blue Shield’s procedures for assuring that the data submission and capture were accurate and represent fairly the services provided by each hospital. According to the model-review actuary, Blue Shield and Milliman stated that they had reconciled the underlying claim data to other Blue Shield financial documents, but he did not independently verify their assertions. According to Blue Shield, its analysis was performed using data taken from health-care claims submitted to it for hospital services incurred from July 1, 2002, through June 30, 2003. Blue Shield stated that it extracted the claim data using the same database and extraction criteria that it uses in the normal Neither Blue Shield nor course of business. Blue Shield also stated that it has been using Milliman performed a its extraction method successfully for several years to perform detailed review of the financial analyses relating to pricing, provider negotiations, claim data to confirm and setting its reserves, and it has no reason to believe the that the data agreed with information is incorrect or deficient. Although Blue Shield Blue Shield’s financial reports using this method of data extraction frequently, neither documents and were it nor Milliman performed a detailed review of the data to accurate and complete. confirm that the data agreed with its financial documents and were accurate and complete. Reconciling the data is important because if the data are incorrect, the conclusions of the analysis will not be reliable. 2222 California State Auditor Report 2004-123 California State Auditor Report 2004-123 2233 In an attempt to validate its claim data, our consultant asked Blue Shield to provide evidence that the allowed amounts used in the analysis were equal to the total payments made by Blue Shield to each hospital. Blue Shield stated that it could not produce such a reconciliation because the timing and nature of its payments to hospitals can vary, allowed amounts include payments from members, and the number of months allowed to process claims after the close of the period can vary. Without a reconciliation of the underlying claim data to other Blue Shield financial documents, we have no assurance that the conclusions of the analysis are correct. The Inability to Assign RVUs for All Outpatient Claims Does Not Indicate a Significant Problem With the Claim Data Blue Shield estimated that fewer than 10 percent of hospital outpatient claims had insufficient information to allow for the proper assignment of RVUs and that only a nominal amount of inpatient claims were missing an assignment. RVUs are assigned based upon procedures, diagnoses, and other information reported on each claim. If some of this information is missing, it may not be possible to assign RVUs. Although the model-review actuary did not review the underlying claim data, he stated that Blue Shield’s estimate of the percentage of insufficient information should not cause a material bias in the analysis results unless a substantial amount of information was missing for a single hospital, or unless procedure codes submitted or captured are incomplete. Our consultant informed us that generally, the inability to assign RVUs to claims has the potential to bias the results of the analysis and may indicate a problem with the underlying data. For example, if all of the unassigned claims are in a small number of hospitals, a systematic problem with the data may be the cause. Similarly, a high percentage of unassigned claims occurring in all excluded hospitals would cause concern that hospitals are being excluded because of a problem with the claim data unrelated to cost. However, if the unassigned claims are distributed randomly across all hospitals, it is unlikely that the failure to assign RVUs for all claims will compromise the results. Our consultant conducted four analyses to investigate the inability to assign RVUs to all outpatient claims. Figure 2 on the following page presents the percentage of outpatient claims to which Blue Shield was able to assign RVUs. It shows that for 249 hospitals, or more than 80 percent of the 298 hospitals, at 2222 California State Auditor Report 2004-123 California State Auditor Report 2004-123 2233 FIGURE 2 Percentage of Outpatient Claims to Which RVUs Were Assigned in 298 Hospitals �� least 90 percent of outpatient claims received RVU assignments. These findings are consistent with unassigned claims being randomly distributed across hospitals. Figure 2 also shows that only three hospitals assigned RVUs to fewer than 70 percent of outpatient claims. None of these hospitals were excluded from the network. The two hospitals with the lowest percentages have RVU totals below 5,000. Blue Shield did not exclude any hospitals with fewer than 5,000 RVUs, because cost estimates for these hospitals were considered unreliable due to the small number of RVUs. Our consultant performed a comparison of the percentage of outpatient claims assigned an RVU and the difference between the quality-adjusted relative cost factor and Blue Shield’s exclusion threshold. In deciding which hospitals to include in its network, Blue Shield assigned each hospital a quality-adjusted �� Sources: Blue Shield analysis and Bureau of State Audits’ consultant’s analysis. Note: The 298 hospitals represent the hospitals Blue Shield included in its exclusive provider network analysis. 2244 California State Auditor Report 2004-123 California State Auditor Report 2004-123 2255 relative cost factor. A hospital’s cost factor can be thought of as the cost per unit of output, where output is measured using RVUs. Blue Shield calculates a hospital’s relative cost factor by dividing the hospital’s cost factor by the average of the cost factors for other hospitals in the same cohort. It then adjusts a hospital’s relative cost factor for quality, using quality scores, to produce a quality-adjusted relative cost factor.13 Those hospitals with quality-adjusted relative cost factors above a certain threshold, called the exclusion threshold, were subject to exclusion from the HMO provider network. Once our consultant removed the three hospitals with low percentages of assigned RVUs from the analysis, the average percentage of outpatient claims assigned an RVU did not appear to differ materially between those hospitals above and those below Blue Shield’s exclusion threshold. Another analysis performed by our consultant limited the previous comparison to only those hospitals with quality- adjusted relative cost factors within ±0.05 percentage points of the threshold. This analysis also found that the average percentage of claims assigned an RVU did not differ for hospitals above or below the threshold. Additionally, the lowest percentage of claims assigned an RVU was roughly 75 percent. Our consultant’s fourth analysis found that the excluded hospitals all had a relatively high percentage of claims that were assigned RVUs. All four analyses indicate that the inability to assign RVUs to outpatient claims is not concentrated among any groups of hospitals that would raise concern and indicate a problem with the underlying claim data. Hospitals With Low Ratios of Allowed-to-Billed Charges Are Also Not Indicative of a Problem With the Underlying Claim Data The model-review actuary recommended that Blue Shield investigate hospitals with an allowed-to-billed ratio (ratio) of less than 10 percent, which is lower than the California average of 28 percent. This ratio represents Blue Shield’s contractually allowed reimbursement to the hospital as a percentage of the hospital’s billed, or list, charges. According to our consultant, an exceptionally low ratio may indicate a problem with the underlying claim data. The model-review actuary was concerned that high ratios on some claims or services and low ratios on others could signify a problem with the underlying claim data. Figure 3 presents the ratio for the 13 Blue Shield also adjusts each hospital’s relative cost factor for efficiency and rate increases, which we discuss more fully later. 2244 California State Auditor Report 2004-123 California State Auditor Report 2004-123 2255 FIGURE 3 Allowed-to-Billed Ratio for Combined Inpatient and Outpatient Services �� Source: Blue Shield analysis and Bureau of State Audits’ consultant’s analysis. Note: The 298 hospitals represent the hospitals Blue Shield included in its exclusive provider network analysis. 298 hospitals included in the analysis. As the figure shows, one hospital had a ratio of less than 10 percent, and an additional 19 hospitals had a ratio between 10 percent and 15 percent. The results do not indicate a material problem with the underlying claim data. By creating a scatter plot of the ratio to the RVUs of each of the 298 hospitals, our consultant found that hospitals excluded by Blue Shield were not overrepresented among those hospitals with low ratios. Our consultant’s final analysis was to investigate the characteristics of hospitals that had a low ratio at the service category level.14 For this analysis, our consultant reviewed all 14 Outpatient service categories are (1) surgery, (2) radiology, (3) pathology, (4) emergency room, and (5) other. Inpatient service categories are (1) medical, (2) surgical, (3) maternity, and (4) mental health/substance abuse. 2266 California State Auditor Report 2004-123 California State Auditor Report 2004-123 2277 hospitals with a ratio less than or equal to 15 percent at the service category level. Overall, 70 percent of the hospitals with ratios at or below 15 percent are owned by one hospital system. According to our consultant, because this hospital system is known to have very high charges, it is not surprising that it would have low ratios. Thus, the results of our consultant’s investigation of the concern by the model-review actuary reveals that low ratios are not indicative of a problem with the underlying claim data. The number of hospitals with low ratios is small, and hospitals with low ratios are not more likely to be excluded than other hospitals. In addition, there appears to be a valid explanation for the majority of the hospitals with low ratios. Differences Between CalPERS and Non-CalPERS Claims May Materially Affect Some Hospitals, However, Using These Claim Data Does Not Appear Unreasonable In performing its analysis, Blue Shield had the option of using claim data from three sources: CalPERS HMO members’ claims, Blue Shield non-CalPERS HMO enrollees’ claims, and Blue Shield preferred provider organization (PPO) enrollees’ claims. As we describe later, each source presents a unique drawback. The claims used in Blue Shield’s analysis included claim data from all three sources. Specifi cally, according to Blue Shield, prior to calendar year 2003 the number of CalPERS members in its network was less than 125,000. However, since calendar year 2003 the number of Inaccuracies in using non-CalPERS members has grown to more than 400,000. Blue HMO data may exist due to the following factors: Shield stated that claims other than those for CalPERS members were included in the analysis to • The claims experience of CalPERS members increase the sample size and statistical signifi cance may be different from that of non-CalPERS HMO members due to demographic of the hospital-specifi c estimated cost factors. factors. According to our consultant, larger samples are • The claims experience of HMO enrollees may be different from that of PPO preferred to smaller samples because the precision enrollees due to demographic or other with which cost factors are estimated increases factors, including self-selection of older, less healthy enrollees into the PPOs. as the sample size increases. However, including claims of non-CalPERS HMO members and PPO • The allowed amount at a given hospital can be different for identical HMO and enrollees can potentially distort the results of the PPO claims because Blue Shield has analysis and reduce the accuracy of the estimated separate HMO and PPO provisions within cost factor, as shown in the text box. Given the its contracts with some hospitals that specify different reimbursement rates. potential for inaccuracy, there is a trade-off in increasing sample size by adding non-CalPERS HMO and PPO claims. Thus, our consultant Source: Bureau of State Audits’ consultant. conducted additional analyses to determine the likely signifi cance of these potential distortions. 2266 California State Auditor Report 2004-123 California State Auditor Report 2004-123 2277 According to our consultant, large differences between the demographic profile of CalPERS enrollees and those of the other types of enrollees used in Blue Shield’s analysis would heighten concerns about possible differences in claim experience that could adversely affect the accuracy of the analysis and the resulting savings estimates. Our consultant found that CalPERS HMO enrollees are more similar demographically to Blue Shield PPO enrollees than to Blue Shield non-CalPERS HMO enrollees. Specifically, the weighted average age and sex factors for the three categories are as follows:15 Age/Sex Factor16 CalPERS HMO 1.1823 Non-CalPERS HMO 1.0317 PPO 1.1792 This comparison suggests that differences in the demographic According to our profile of PPO enrollees relative to CalPERS HMO enrollees may consultant, the inclusion not introduce significant distortions into Blue Shield’s analysis.17 of PPO claims may also However, there does appear to be a substantial difference produce inaccurate between the demographic profile of CalPERS HMO enrollees and results because rates that of non-CalPERS HMO enrollees. This difference means that paid by Blue Shield to CalPERS members may use a different mix of hospital services hospitals under PPO than non-CalPERS HMO enrollees. Significant differences in contracts, in some the cost factors of different services within the same hospital instances, are different may affect the overall hospital cost factor when it is calculated than rates paid under including claim data for non-CalPERS HMO enrollees, as HMO contracts. opposed to when it is based only on the mix of services used by CalPERS HMO enrollees.18 Therefore, an analysis that includes both CalPERS and non-CalPERS HMO claim data may produce inaccurate results. 15 Blue Shield calculates its weighted average age and sex factor by weighting the institutional age and sex factor for each of the 28 age and sex categories by the number of member months in each category. A single member enrolled for an entire year represents 12 member months. 16 Includes Blue Shield’s commercial HMO members with facility capitation and self- funded large group, as well as national account PPO business, but excludes its Senior HMO, Senior Medicare Supplement product, and Federal Employee Program PPO. 17 According to our consultant, although the overall age and sex factor for CalPERS HMO enrollees is similar to that for Blue Shield’s PPO enrollees, the mix of services used by the two groups may still differ. 18 It is not uncommon for the cost factor for one category of service within a hospital to differ by more than 25 percent from the cost factor for another category of service within the same hospital. For example, for one hospital, the cost factor for inpatient medical services was $161, while the cost factor for maternity services was $94. 2288 California State Auditor Report 2004-123 California State Auditor Report 2004-123 2299 Although the demographic profi le for the CalPERS HMO members does not differ substantially from that of PPO enrollees, according to our consultant the inclusion of PPO claims may also produce inaccurate results because rates paid by Blue Shield to hospitals under PPO contracts, in some instances, are different than rates paid under HMO contracts. Our consultant reviewed contracts for a sample of 13 hospitals and concluded that PPO rates were different in some of the contracts. These differences may have an impact on the fi nal quality-adjusted relative cost factors (see the The Impact of Including PPO Claim text box) and the resulting estimate of savings Data on Cost Factors from the exclusive provider network. However, the magnitude of any potential inaccuracies • Cost factors will be infl ated for hospitals with higher PPO rates because allowed introduced by this type of problem depends on amounts for some claims at these hospitals several factors, including the number of hospitals will refl ect higher levels of reimbursement under PPO contracts than under HMO with different HMO and PPO rates and the contracts for CalPERS members. magnitude of the differences between the rates. • The relative cost factors for hospitals in the cohort will be affected because a hospital’s At our request, Blue Shield reran portions of the relative cost factor depends on the cost analysis using only data from HMO claims and factors of other hospitals in the cohort. provided us with a summary of the results.19 Our consultant made the following observations. First, Source: Bureau of State Audits’ consultant. for some hospitals the change in the relative cost factor using only HMO claims was signifi cant. For example, the relative cost factor for 16 hospitals increased by 19 percentage points or more, while the relative cost factor for eight hospitals decreased by 17 percentage points or more. Further, the changes in the relative cost factors resulted in a total of 17 hospitals that would potentially change status, from excluded to included or vice versa.20 Figure 4 on the following page shows the two cost factors for each of the 17 hospitals: one calculated using HMO and PPO claims, the other calculated using HMO claims only. As the fi gure shows, nine hospitals would potentially be excluded rather than included, while eight would potentially be included rather than excluded. It is important to note that the net change in results from using only HMO claim data may be caused by differences in HMO and PPO contract rates, differences in the 19 The analysis was performed by Blue Shield using claim data from a later period and a later version of its analysis (NC6) than the one presented to CalPERS when the exclusive provider network decision was made. Our consultant’s analysis assumes that Blue Shield’s exclusion threshold remains the same. 20 Our consultant’s evaluation did not include whether some of these hospitals would have changed status due to capacity, access to tertiary services, or other considerations. 2288 California State Auditor Report 2004-123 California State Auditor Report 2004-123 2299 mix of services used by HMO and PPO enrollees, and random variation across the two samples of claims. Therefore, although using only HMO claim data produces different results for some hospitals than using combined HMO and PPO claims, it does not necessarily produce more accurate results. FIGURE 4 Comparison of 17 Hospitals’ Relative Cost Factors Using HMO/PPO Claims Versus Only HMO Claims �� Sources: Blue Shield analysis and Bureau of State Audits’ consultant’s analysis. These results indicate that Blue Shield’s decision to include PPO claim data in its analysis materially affected the relative cost ranking for some hospitals compared to their ranking using data for HMO claims only. Although the number of hospitals affected is relatively small compared to the total number (17 of 298, or roughly 6 percent), the results are potentially important because these hospitals provide more than 10 percent of services to HMO enrollees. 3300 California State Auditor Report 2004-123 California State Auditor Report 2004-123 3311 A potential concern with limiting the sample to HMO claims only is that the number of claims, or similarly the number of RVUs, at any particular hospital may become too small to produce a reliable estimate of hospital costs. Under Blue Shield’s reliability standards, this concern does not appear to be a serious problem. Blue Shield considers cost factors for hospitals with fewer than 5,000 RVUs to be unreliable, and results for hospitals with between 5,000 and 10,000 RVUs to be usable but volatile. Compared to using both HMO and PPO claims, using only HMO claims increases the number of hospitals that fall below Blue Shield’s minimum 5,000 RVU standard from 38 to 86.21 However, because these are among the smallest hospitals, the services they deliver (as measured by HMO RVUs) represent less than 1 percent of total hospital services. Moreover, less than 2 percent of hospital services are provided at facilities with between 5,000 and 10,000 HMO RVUs. This means that, even using only HMO claim data, over 97 percent of hospital services are provided at facilities with sufficient data to produce reliable cost factors, based on Blue Shield’s reliability standards.22 Given that there was no ideal source of claim data to use in its Given that there was no analysis, Blue Shield’s decision to include data from all three ideal source of claim data sources does not appear unreasonable. However, as additional to use in its analysis, claims for CalPERS members become available, Blue Shield Blue Shield’s decision to can consider performing an analysis using only CalPERS HMO include data from all claims. If the number of CalPERS claims is still too small to three sources does not produce reliable results, it should supplement these claims with appear unreasonable. a select subset of non-CalPERS HMO and/or PPO claims that are most similar to CalPERS HMO claims with respect to member demographics and hospital contract rates. BLUE SHIELD’S COST MODEL APPEARS REASONABLE, BUT IN ONE INSTANCE BLUE SHIELD DID NOT FOLLOW ITS GENERAL RULE FOR EXCLUDING HOSPITALS Our consultant’s review of Blue Shield’s cost model found that many of the components it used appear reasonable when comparing the relative costs of hospitals. However, in 21 Our consultant’s calculation of 48 additional hospitals is based upon applying the change in total cohort relative cost (HMO only minus HMO/PPO) provided by Blue Shield to the quality-adjusted relative cost factor from the NC5 model for each hospital. 22 In at least one cohort, Blue Shield used data from HMO claims only to determine which hospitals would be excluded from its network. Thus, it appears that Blue Shield also considered results using only HMO claims to be reliable, at least in some circumstances. 3300 California State Auditor Report 2004-123 California State Auditor Report 2004-123 3311 one instance Blue Shield did not consistently apply certain calculations when determining whether to remove hospitals from the exclusive provider network. As a result, it may have chosen to include some hospitals that the general rule would have excluded or vice versa. Many Components of Blue Shield’s Cost Model Appear Reasonable As we discussed previously, Blue Shield uses a cost factor to determine a hospital’s cost relative to the cost of other hospitals in the cohort. According to our consultant, the use of cost factors is a reasonable method for correcting for interhospital variation in case mix and severity.23 Without using cost factors or some other method that corrects for such variation, meaningful comparisons among hospitals are not possible. Blue Shield appears to have calculated and applied cost factors in a sound manner. In addition, our consultant found that Blue Shield’s method for establishing its cohorts also appears reasonable. According to Blue Shield, its 31 cohorts were originally developed in the normal course of business, and its use of them predates its analysis for the CalPERS exclusive provider network. Blue Shield’s rationale for using cohorts is that if a hospital is excluded from the network, patients in most cases will switch to nearby hospitals. Therefore, hospitals should be compared to their geographic peers. Most cohorts are composed of one or more counties, but because of the large size of a few counties, they were split into multiple cohorts. Blue Shield’s method of defining cohorts, while it Our consultant believes it is reasonable to compare hospitals may be an unacceptably to others in the same geographic area, since most hospitals do crude measure of hospital not compete with faraway facilities for a significant share of markets, is likely to their business.24 Cohorts defined by county boundaries may be represent an acceptable an unacceptably crude measure of hospital markets for some trade-off between simplicity purposes. However, in a modeling exercise such as Blue Shield’s and lack of precision. analysis, defining markets in this way is likely to represent an acceptable trade-off between simplicity and lack of precision. Our consultant’s examination of maps that show the location, 23 Case mix indicates the mix of patients treated at the hospital, as measured by factors such as age, gender, or patient diagnosis. Severity is a measure of the statistically “expected” outcome (e.g., mortality, morbidity, efficiency of care) of a disease in a particular patient. 24 According to our consultant, an exception may be hospitals with a large share of highly complex or specialized services. 3322 California State Auditor Report 2004-123 California State Auditor Report 2004-123 3333 size, and relative efficiency of hospitals within and around the cohorts that contained excluded hospitals indicates that movement of patients between hospitals in different cohorts was not likely to have a material effect on estimated savings. Furthermore, Blue Shield’s quality adjustments have had a If the quality adjustment minimal effect on its decision to exclude hospitals from the had not been applied, network. The amount of these adjustments may be thought of four hospitals that were as reflecting the hospital’s cost of implementing initiatives that above the exclusion are generally recognized as possibly leading to improved quality, threshold with the such as a computerized physician order entry system to reduce adjustment would move prescribing errors. Our consultant found that the impact of below the threshold. Only Blue Shield’s quality adjustment does not appear to have a strong one of these four hospitals influence on the results of its analysis. If the quality adjustment was actually excluded had not been applied, four hospitals that were above the exclusion from the exclusive provider threshold with the adjustment would move below the threshold. network. Only one of these four hospitals was actually excluded from the exclusive provider network. According to Blue Shield, the other three hospitals were retained because of their coverage, capacity, or tertiary services. Five hospitals that were below the threshold after the quality adjustment would otherwise be above the threshold. Despite being below the threshold, one hospital was excluded due to certain contract terms. Blue Shield’s efficiency adjustment also does not appear to materially influence the results of the analysis. Blue Shield includes an efficiency adjustment in its calculation of hospitals’ cost factor to account for differences in the length of stay between hospitals.25 To quantify the impact of the efficiency adjustment, our consultant calculated the quality-adjusted relative cost factor with and without the efficiency adjustment. The difference between the two values was calculated and the results are displayed in Figure 5 on the following page. The average efficiency adjustment is 0.04. As Figure 5 shows, for 10 hospitals the impact of the efficiency adjustment changed the quality-adjusted relative cost factor by more than 15 percentage points. For nine of these 10 hospitals, the efficiency adjustment made no difference in whether they were included or excluded. 25 The data to calculate the efficiency adjustment are included in the Milliman model inpatient input into the cost model. The remaining calculation for the efficiency adjustment occurs in the cost model. 3322 California State Auditor Report 2004-123 California State Auditor Report 2004-123 3333 FIGURE 5 Distribution of the Efficiency Adjustment Impact on Hospitals’ Final Cost Factor �� Our consultant’s second assessment of the impact of the efficiency adjustment identified the number of hospitals that changed their classification when the adjustment was not applied. In total, 10 hospitals changed classification. Five of the 10 hospitals had scores above the threshold with the adjustment and were subject to exclusion, but their scores would have been below the threshold if the adjustment were not applied. Only one of the five was actually excluded from the network. According to Blue Shield, three of the other four were not excluded due to the need to provide coverage, capacity, or tertiary services, while one was not excluded because it was part of a system that was evaluated as a system rather than as individual hospitals. The remaining five of the 10 hospitals had scores at or below the threshold with the adjustment and would have had scores above the threshold without the adjustment. Because the RVU assignments depend on length of stay, �� Sources: Blue Shield analysis and Bureau of State Audits’ consultant’s analysis. 3344 California State Auditor Report 2004-123 California State Auditor Report 2004-123 3355 which varies, a clear need exists for the effi ciency adjustment. Blue Shield’s methodology for determining its effi ciency adjustment appears reasonable. Finally, the method used by Blue Shield in its rate increase submodel to estimate the price increases for individual hospitals appears reasonable. Blue Shield’s submodel used data from July 1, 2002, through June 30, 2003, to forecast hospital prices as of December 31, 2004. Blue Shield then used the forecast prices to calculate the relative cost factors for calendar year 2005. To avoid penalizing hospitals with rate increases scheduled to take effect immediately prior to January 1, 2005, Blue Shield gave less weight to these rate increases. Alternatively, it could have forecast weighted average prices for the entire calendar year 2005 rather than prices as of December 31, 2004. Although Blue Shield’s is one of several possible methods, we have no reason to believe that another method would consistently provide a more accurate estimate of hospitals’ price increases. In One Instance Blue Shield Did Not Apply Its Threshold and Cohort Average Calculations Blue Shield’s General Rule for Excluding Consistently Hospitals From Its Network As we mentioned previously, in order to determine Predetermined quality-adjusted relative cost which hospitals to exclude from the HMO provider factor threshold unless one of the following conditions is present: network, Blue Shield established a threshold for the quality-adjusted relative cost factor, above which • Hospital is needed to provide adequate coverage of the area it services. hospitals were subject to exclusion unless they were needed in the network for one of the reasons shown • Hospital is needed to ensure adequate in the text box. We recognize that exceptions to capacity. this general rule may have been needed to handle • Hospital is needed to provide specifi c and/ unanticipated situations. Nevertheless, one such or tertiary services. exception resulted in the inconsistent application of • Hospital did not have suffi cient RVU the rule for excluding hospitals. volume to allow cost factors to be estimated reliably. A hospital relative cost factor of 1 indicates that • Decision arising from hospital contract negotiations. a hospital’s costs were at the average among the hospitals in its cohort. According to Blue Shield, Source: Blue Shield. the main reason it established the exclusion threshold above 1 was that the hospital-specifi c cost factors are updated every six months using new claim data. Individual hospital cost factors have a natural and random variation due to changes in the claim mix. If it set the threshold at 1, hospitals with costs that were actually below the average could exceed the threshold because of random variations in the data and thus would be 3344 California State Auditor Report 2004-123 California State Auditor Report 2004-123 3355 subject to exclusion. Blue Shield believes that with a threshold greater than 1, any hospital that is excluded is highly likely to have above-average costs relative to the other hospitals in the cohort, even accounting for random variation. Blue Shield also stated that another reason for the higher threshold was that a lower threshold could result in CalPERS incurring higher costs. This would occur when an expensive hospital was retained for access or other reasons and a more cost-effective hospital was excluded. A third reason provided by Blue Shield was that each successive hospital dropped from the network as the threshold was lowered to the average score of 1 would reduce member access but provide an ever-decreasing amount of savings. Although we requested documents to support its assertion regarding the need to use a threshold higher than 1, Blue Shield was unable to provide such documents, stating that they were deleted in the transition from its previous director of finance to the current director. Thus, our consultant cannot conclude on Blue Shield’s specific threshold, but believes Blue Shield’s rationale for selecting a threshold higher than 1 appears reasonable. Our consultant found that in one instance Blue Shield appears to have inconsistently applied its general rule for excluding In one cohort, rather hospitals from its network. Specifically, in one cohort, rather than evaluating each than evaluating each hospital in a system separately against the hospital in a system cohort average, using HMO and PPO data, Blue Shield evaluated separately against the all the hospitals in one system as a group, using a different set cohort average, using of claims. Since the system as a whole met the threshold, all HMO and PPO data, hospitals in the system were included in the provider network, Blue Shield evaluated even though one of the hospitals would have exceeded the all the hospitals in one threshold if it had been evaluated separately. system as a group, using only HMO data. Blue Shield’s stated reason for deviating from the original criteria and evaluating the system as a whole was that unique circumstances required the system to be evaluated as a system rather than as individual hospitals. According to Blue Shield, these unique circumstances included, but were not limited to, the types of services offered at the hospitals within this system, the nature of the affiliations between the hospitals and certain physician groups, the relative cost of services at other hospitals in this cohort, and that retaining all the hospitals in the system led to relative increase in savings to CalPERS. Blue Shield stated that it analyzed the other hospital systems in each cohort in the State and that no other systems met the exceptions. However, when asked to provide copies of the analyses, Blue Shield was unable to produce any contemporaneous analyses supporting this assertion. 3366 California State Auditor Report 2004-123 California State Auditor Report 2004-123 3377 Blue Shield also stated that another reason for deviating from the original criteria was due to circumstances it claims are unique to this system, and that using a different claim data set created a fairer and more accurate cost comparison of that system within its cohort. However, our consultant’s review of Blue Shield’s contract with another hospital found that the circumstances were not unique to the hospital system that received special treatment from Blue Shield. Yet, when this other hospital requested an evaluation using a different claim data set, Blue Shield refused, even though the second hospital had a relative cost factor that was only marginally above the threshold. Blue Shield stated that its investigation of the impact of using the different data set in evaluating the hospital found that the impact was not material and was just as likely, if not more likely, the result of factors other than differences in the data sets. A review of the hospital’s contract rates by our consultants, however, revealed that use of the different data set would almost certainly have moved the second hospital below the threshold. Therefore, it appears that although exceptions to Blue Shield’s general rule may have been warranted in the case where they were applied, these exceptions were not applied consistently in other cohorts. BLUE SHIELD’S ESTIMATE OF CALPERS SAVINGS RESULTING FROM THE EXCLUSIVE PROVIDER NETWORK IS POSSIBLY OVERSTATED Blue Shield estimated that in 2005 CalPERS’ hospital savings would be $20.6 million and the medical group savings would be $10.8 million, for a total of $31.4 million in savings from switching to the exclusive provider network.26 This estimate did not incorporate the terms of a new contract actually signed between Blue Shield and a health system in early 2004. The new contract contained a clause with certain financial terms that would be available to CalPERS only if it did not adopt the exclusive provider network. If the CalPERS board approved the exclusive provider network, the financial terms would no longer be in effect. Blue Shield estimated that the financial terms would save CalPERS a substantial amount in 2005. During a closed meeting held on May 11, 2004, Blue Shield presented to 26 As of May 19, 2004, when the board made its decision to exclude 38 hospitals, Blue Shield estimated hospital savings to be $27.7 million and savings associated with some medical groups to be $8.6 million for a total of $36.3 million savings to CalPERS in 2005. The savings estimate in this section refers to Blue Shield’s estimate of $31.4 million that relates to the exclusion of 24 hospitals. In Appendix A we present an overview of Blue Shield’s analysis. 3366 California State Auditor Report 2004-123 California State Auditor Report 2004-123 3377 the board the impact of the health system’s financial terms. On May 19, 2004, the board approved the exclusive provider network, which resulted in the health system’s financial terms no longer applying in 2005 and 2006. Its estimate of $31.4 million did not consider the impact of members leaving the Blue Shield HMO provider network and joining other health-care plans. We were unable to quantify the full effect of Blue Shield’s omission of this assumption from its savings estimate. However, using data from an analysis prepared by CalPERS, we determined that the impact resulting from member movement in the Sacramento area would drop Blue Shield’s estimate from $5.5 million to between $1.7 million and $3.5 million.27 Finally, Blue Shield did not thoroughly investigate concerns about the uncertainty of emergency room assumptions for one health system, which may cause Blue Shield’s hospital savings estimate to CalPERS of $20.6 million to drop to only $8.9 million. According to Blue Shield, if the savings from the exclusive provider network differ materially from its original estimate, it will adjust CalPERS’ future premiums. Specifically, a provision in Blue Shield’s contract with CalPERS requires it to compare the actual cost of health care to its projected costs. If the difference falls outside of a certain range, Blue Shield will adjust for the difference when calculating the projected health-care costs for the following year, which could potentially increase CalPERS’ premiums. Blue Shield’s Savings Estimate Does Not Incorporate Financial Terms Contained in a New Contract According to the model- review actuary, the proper Blue Shield expected to save CalPERS money by excluding from baseline for projecting its HMO provider network high-cost hospitals and medical the change from the full groups that admit only to those hospitals. The model-review network to the exclusive actuary found that the original estimate did not include recent provider network would negotiations with hospitals that were expected to produce be the incremental savings savings for the full network. According to the model-review above the savings expected actuary, the proper baseline for projecting the change from the from the full network, full network to the exclusive provider network would be the which should take into incremental savings above the savings expected from the full consideration any new network, which should take into consideration any new hospital hospital reimbursement reimbursement agreements. agreements. 27 CalPERS includes the counties of Sacramento, Placer, and Yolo in its analysis. 3388 California State Auditor Report 2004-123 California State Auditor Report 2004-123 3399 Blue Shield’s $31.4 million savings estimate is based on information available as of December 2003, before a new contract had been negotiated with one health system. The savings estimate did not incorporate the terms of a new contract actually signed between Blue Shield and the health system in early April 2004. The new contract created two options for CalPERS: a full and a limited provider network, each of which had different financial consequences. Blue Shield estimated that the financial terms would save CalPERS a substantial amount in 2005, if the CalPERS board chose the full provider network. Assuming CalPERS had chosen not to proceed with the exclusive provider network, the baseline for the savings estimate should have adjusted downward substantially. Blue Shield stated that it reviewed the model-review actuary’s finding. However, Blue Shield also stated that due to the private and confidential nature of its hospital reimbursement agreements, it was unable to present to the public or in open board or committee meetings the health system’s contract savings. However, on March 16, 2004, Blue Shield did present an estimate of the impact of the system’s contract savings to the committee during a closed meeting, so that CalPERS could understand the overall incremental savings. Additionally, during a closed board meeting held on April 8, 2004, Blue Shield informed the board that the health system’s financial terms would generate savings. Blue Shield and the board had further discussions concerning the exclusive provider network savings during a closed session held at the May 11, 2004, board meeting. During an open session at the same board meeting, Blue Shield presented two options to the board. One option was to maintain the full network under the following conditions: accept one health system’s financial offer, provide higher-cost hospitals an According to the current opportunity to bring their costs closer to the industry average, deputy executive officer for and maintain the option of adopting an exclusive provider benefits administration, network in the future (for example, January 1, 2006). The other the board chose to proceed option was to adopt the exclusive provider network effective with the exclusive provider January 1, 2005. On May 19, 2004, the board approved the latter network because it sought option, which resulted in the health system’s financial terms for to generate savings the full provider network no longer applying in 2005 and 2006. beyond the health system’s contract period and to According to the current deputy executive officer for benefits initiate structural reform in administration, the board chose to proceed with the exclusive the health-care industry. provider network because it sought to generate savings beyond the health system’s contract period and to initiate structural reform in the health-care industry. This statement is consistent with our review of the transcripts of meetings 3388 California State Auditor Report 2004-123 California State Auditor Report 2004-123 3399 prior to the board’s approval. Since then, the board has taken steps toward the structural reform it is seeking. Specifically in its February 2005 meeting, the board approved two initiatives: CalPERS’ hospital reimbursement project, which is aimed at advancing performance transparency and managing hospital reimbursements, and CalPERS’ plans to establish an ongoing process to review and evaluate hospital performance and identify those hospitals that will remain in the CalPERS network. According to Blue Shield, if the savings from the exclusive provider network differ materially from its original estimate, it will adjust CalPERS’ future premiums. Specifically, a provision in Blue Shield’s contract with CalPERS requires it to compare the actual cost of health care to its projected costs. If the difference falls outside of a certain range, Blue Shield will adjust for the difference when calculating the projected health- care costs for the following year, which could potentially increase CalPERS’ premiums. Blue Shield’s Savings Estimate Does Not Account for Members Leaving Its Provider Network The model-review actuary found that the savings forecast model he reviewed did not explicitly project the potential impact on savings (or the added expense) of the possible shift of Using CalPERS data, enrollment from Blue Shield’s HMO provider network to other Blue Shield estimated coverage options available to CalPERS members. According to that during CalPERS’ a preliminary analysis prepared by Blue Shield, it experienced a December 2004 net loss of 29,200 members as a result of members transferring enrollment period almost between its HMO provider network and other coverage options 34,000 members were during CalPERS’ December 2004 enrollment period. Specifically, transferring out of its using CalPERS data, Blue Shield estimated that almost 34,000 provider network, while members were transferring out of its provider network, while only roughly 4,800 were only roughly 4,800 were transferring into the network. transferring into the network. Although there could be a number of reasons why members chose to leave Blue Shield’s HMO provider network, a substantial number of members leaving the network were from the Sacramento area, which was estimated to be the area most affected by the creation of the exclusive provider network, in terms of the number of hospitals and medical groups that were excluded. According to an analysis prepared by CalPERS health benefits branch staff in February 2005, almost 16,000 Sacramento area members left the Blue Shield HMO provider 4400 California State Auditor Report 2004-123 California State Auditor Report 2004-123 4411 network. Of these, some are now paying lower premiums but According to our did not retain provider continuity, some are paying higher consultant, Blue Shield’s premiums and did retain provider continuity, some are paying saving estimate includes lower premiums and did retain provider continuity, and $5.5 million for the others left the CalPERS benefits program. According to our Sacramento area, which consultant, Blue Shield’s saving estimate includes $5.5 million could drop to between for the Sacramento area. This amount could drop to between $1.7 million and $1.7 million and $3.5 million as a result of members’ movement $3.5 million as a result of to other coverage options.28 CalPERS’ analysis does not provide members’ movement to similar information for other areas in the State. Thus, we other coverage options. are unable to quantify the full effect that members leaving Blue Shield’s provider network has on its savings estimate. Blue Shield Did Not Thoroughly Investigate Concerns About the Emergency Room Diversion Assumptions Used in Its Savings Estimate To calculate the savings from excluding high-cost hospitals, Blue Shield redistributed a portion of the RVUs from the excluded hospitals to included hospitals in the same cohort. This diversion of RVUs assumes that some costs that would have been incurred at an excluded hospital will instead be incurred at an included hospital. Blue Shield’s assumptions about the percentage of RVUs to divert varied depending on the type of service and the hospital system. For one health system, Blue Shield did not assume the diversion of any emergency room RVUs, but for other hospitals it assumed a 40 percent diversion of their total emergency room RVUs. Emergency room RVUs include ancillary services such as laboratory and radiology. In contrast, for inpatient medical and surgical (medical/surgical) RVUs, Blue Shield assumed an 85 percent diversion for the one health system and a 90 percent diversion for other hospitals. Inpatient medical/surgical RVUs are assumed to be retained both because of continuity of care requirements and to provide support for emergency room visits. For other services such as maternity, mental health/substance abuse, and nonemergency outpatient services, Blue Shield’s diversion assumption was 28Information was not available on the cost of providing care to the members who leave Blue Shield and therefore this was not considered in the analysis. If the members moved to health plans that use the high-cost hospitals excluded from the Blue Shield HMO provider network then the lost savings may not be realized by CalPERS. If they move to health plans that do not include these high-cost hospitals or use them less frequently, then CalPERS may realize some of the lost savings. However, without knowledge of the rates these hospitals charge other health plans, it is not possible to further refine the effect that member movement has on Blue Shield’s estimate. 4400 California State Auditor Report 2004-123 California State Auditor Report 2004-123 4411 100 percent for all hospitals. Blue Shield stated that the higher diversion assumptions for the one health system’s emergency room RVUs was because the system would likely seek to retain a higher percentage of its patients. The model-review actuary recommended that it investigate the difference between the results of its analysis and the hospital system’s statement that more than 25 percent of the revenue for CalPERS members at its excluded hospitals originates from emergency room visits. The model-review actuary calculated that it would take 100 percent of the hospital system’s outpatient emergency room payments plus 28.5 percent of its inpatient medical/surgical payments to generate the system’s revenue estimate for excluded hospitals. The model-review actuary’s estimate of 28.5 percent for inpatient medical/surgical payments, which equates to a similar percent of RVUs, is almost double Blue Shield’s estimate of 15 percent of RVUs to be retained by the excluded hospital system used in Blue Shield’s savings model. Blue Shield stated that it reviewed the percentage of inpatient payments resulting from emergency room visits at the system’s excluded hospitals in 2003 and that the analysis showed inpatient payments resulting from emergency room visits were close to 10 percent. According to our consultant, Blue Shield’s documentation for this analysis appears to support the 10 percent number, but the documentation did not provide confidence that Blue Shield’s analysis was correct and that the health system was incorrect. A more convincing analysis by Blue Shield would have reconciled its analysis with the health system’s analysis to determine how the two organizations could reach such different estimates using what appear to be similar methodologies. Determining an accurate estimate for the health system’s emergency room diversion assumptions is especially important given the sensitivity of the model to the assumptions. Depending on whose assumption is used, the hospital savings estimate changes by more than 50 percent. Using Blue Shield’s assumption of 15 percent29 of medical/surgical RVUs yields a hospital savings estimate to CalPERS of $20.6 million. However, using the 28.5 percent calculated by the model-review actuary based on the system’s estimate, the savings estimate drops to only $8.9 million. However, given the sensitivity of the savings estimate to differences in emergency room diversion 29 The medical/surgical assumption of 15 percent includes both RVUs resulting from emergency room visits, estimated by Blue Shield to be close to 10 percent, and those required to meet continuity of care requirements. 4422 California State Auditor Report 2004-123 California State Auditor Report 2004-123 4433 assumptions for the health system, Blue Shield should have reconciled the two analyses to ensure that its estimate of emergency room costs was reasonable. RECOMMENDATIONS The Legislature should consider enacting legislation that would allow CalPERS, during its contract negotiation process, to obtain relevant documentation supporting any analyses it will use to make decisions that materially affect the members of the health benefits program established by the Public Employees’ Medical and Hospital Care Act. To ensure that its decisions are in the best interest of CalPERS members, CalPERS should require its health benefits branch staff to evaluate fully the findings and recommendations of third-party reviews and present their results to its board of administration and committee. We conducted this review under the authority vested in the California State Auditor by Section 8543 et seq. of the California Government Code and according to generally accepted government auditing standards. We limited our review to those areas specified in the audit scope section of this report. Respectfully submitted, ELAINE M. HOWLE State Auditor Date: March 29, 2005 Staff: Joanne Quarles, CPA, Audit Principal Russ Hayden, CGFM Cameron Swinko John J. Romero Consultants: Analysis Group, Inc. Bruce Strombom, Ph.D., Managing Principal Mark Gustafson, MPP, Manager Anthony Mak, Senior Analyst Health Actuaries, LLC James N. Roberts, F.S.A., M.A.A.A. 4422 California State Auditor Report 2004-123 California State Auditor Report 2004-123 4433 Blank page inserted for reproduction purposes only. 4444 California State Auditor Report 2004-123 California State Auditor Report 2004-123 4455 APPENDIX A An Overview of Blue Shield’s Methodology for Its CalPERS Exclusive Provider Network Analysis Blue Shield of California (Blue Shield) developed the exclusive provider network analysis (analysis) to help control the health-care costs of the California Public Employees’ Retirement System (CalPERS). The objective of Blue Shield’s analysis was to reduce the growth in hospital costs while maintaining quality and minimizing member-physician disruptions within an exclusive provider network approved by the Department of Managed Health Care (DMHC). Blue Shield worked with Milliman USA, Inc. (Milliman) to create its analysis. Blue Shield used three distinct models in the analysis: the Milliman USA, Inc., RBRVS for HospitalsTM Relative Value Unit Fee Schedule (Milliman model), the cost model, and the savings model. Figure A on the following page presents a flowchart that summarizes the mechanics of the analysis from the raw Blue Shield claim data to the final savings estimate. The elements of the analysis that make up each of the three models are highlighted. The Milliman model is highlighted in red, the cost model in blue, and the savings model in orange. The various parts of the flowchart are explained in the discussion of the models that follows. 4444 California State Auditor Report 2004-123 California State Auditor Report 2004-123 4455 A ERUGIF sisylanA krowteN redivorP evisulcxE )dleihS eulB( ainrofilaC fo dleihS eulB �� .dleihS eulB :ecruoS �� 4466 California State Auditor Report 2004-123 California State Auditor Report 2004-123 4477 Milliman Model The Milliman model was created by Milliman to compare and benchmark hospital contracts by adjusting for differences in the characteristics of given populations, such as case mix and severity across hospitals.30 According to Milliman, the model is used widely by both insurers and hospitals. To account for differences in patient populations, the Milliman model assigns relative value units (RVUs) to every hospital inpatient and outpatient procedure. An RVU is used to measure the resources needed by hospitals to perform a procedure. A chest x-ray, for example, will have fewer RVUs than magnetic resonance imaging of the brain because fewer resources are needed to provide a chest x-ray. The number of RVUs assigned to each procedure is determined by Milliman. According to Milliman, the RVUs are updated at least once per year to reflect changes in the relative resources of the underlying procedures. Milliman uses public and proprietary data sources, the federal Centers for Medicare and Medicaid Services’ (CMS) fee schedules, feedback from clients, and clinical and actuarial review in their update process. The inpatient portion of the model was developed in 1994 and is calculated using 3MTM All Patient Refined Diagnosis Related Groups (APR-DRGs) software for inpatient hospital stays. Within an APR-DRG, differences in severity of illness and risk of mortality are accounted for using four severity levels that range from minor to extreme. Additionally, because inpatient RVU assignments are based on the length of stay, the Milliman model uses an efficiency adjustment to account for differences in the length of stay between hospitals.31 Thus, for inpatient stays, RVU assignments can vary within APR-DRGs. The outpatient portion of the model was developed in 1999 and is calculated using the CMS’ Current Procedural Terminology and Healthcare Common Procedure Coding System codes for outpatient services. In addition to accounting for differences across hospitals’ inpatient populations, the Milliman model also accounts for differences in the types of services each hospital provides. According to our consultant, failing to account for such differences would penalize tertiary facilities and other hospitals 30 Case mix indicates the mix of patients treated at the hospital, as measured by factors such as age, gender, or patient diagnosis. Severity is a measure of the statistically “expected” outcome (e.g., mortality, morbidity, efficiency of care) of a disease in a particular patient. 31 The data to calculate the efficiency adjustment are included in the inpatient drop. The remaining calculation for the efficiency adjustment occurs in the cost model. 4466 California State Auditor Report 2004-123 California State Auditor Report 2004-123 4477 that provide more complicated and, therefore, more expensive services.32 If the model did not account for these differences, our consultant believes that for such hospitals, simple metrics for comparing hospital costs such as cost per admission or cost per bed day would tend to indicate incorrectly that tertiary facilities are more expensive than community hospitals simply because tertiary facilities provide more complicated services. According to Blue Shield, data for the period of July 1, 2002, through June 30, 2003, taken from health-care claims for both CalPERS and non-CalPERS members in both health maintenance organizations (HMOs) and preferred provider organizations (PPOs), were used as inputs to the Milliman model. RVUs and the dollar amounts hospitals agree to accept for payment according to their contracts with Blue Shield for each procedure (allowed amount), as reflected in the Blue Shield claim data, were then summarized by category of service for each hospital.33 The summarized information was used as an input into the cost model. The summarized data are presented in Figure A on page 46 as the boxes titled “Inpatient Drop” and “Outpatient Drop.” Cost Model The purpose of Blue Shield’s cost model is to determine the cost of each hospital relative to the costs of other hospitals in the same cohort. A cohort includes all hospitals that Blue Shield considers to be in the same geographic market. Blue Shield has divided the State into 31 cohorts. In addition to the output of the Milliman model, the cost model uses output from two submodels: the quality submodel and the rate increase submodel. The quality submodel accounts for each hospital’s efforts in implementing measures that are generally recognized as possibly leading to improved quality. Blue Shield’s stated intention in including a quality adjustment is to provide an incentive for hospitals to participate in various quality initiatives and to avoid penalizing hospitals that have higher 32 Tertiary facilities are typically large medical centers that have sophisticated technological and support facilities and offer highly specialized medical and surgical care for unusual and complex medical problems. 33 The allowed amount can be contrasted with at least two other sums: billed charges and paid amount. Billed charges are the gross or list charges of the hospital before any discount. The paid amount is the amount paid by Blue Shield. The difference between the allowed amount and the paid amount includes member co-payments; deductibles, which are the members’ annual amount of out-of-pocket medical expenses that must be paid before Blue Shield begins paying for expenses; and co-insurance, which is the members’ cost of sharing hospital or medical expenses at a specified rate. 4488 California State Auditor Report 2004-123 California State Auditor Report 2004-123 4499 costs because they are implementing these initiatives. The quality submodel uses only publicly available information from a number of independent quality assessment organizations. A hospital’s performance on the various quality metrics was converted into an aggregate hospital quality score that was then used to calculate a quality adjustment based on the hospital’s score relative to the scores of all other hospitals in its peer group. The quality adjustments are represented in Figure A on page 46 as the box titled “Quality Drop.” The rate increase submodel was used to adjust for increases in hospital contract rates occurring between July 1, 2002, and December 31, 2004. Claim data from July 1, 2002, through June 30, 2003, were used to project hospital charges as of December 31, 2004. The summarized projected rate increases are presented in Figure A on page 46 as the box titled “Rate Increase Drop.” Using the input from the Milliman model and the rate increase submodel, the cost model calculates a cost factor for each hospital. According to our consultant, the hospital cost factor can be thought of as the cost per unit of output, where output is measured by RVUs. For example, at a hospital with a hospital cost factor of 1.25, the cost to perform a procedure requiring 100 RVUs would be $125. At a hospital with a hospital cost factor of 1, the cost would only be $100. Blue Shield calculates a hospital’s relative cost factor by dividing the hospital’s cost factor by the cohort average, which it calculates by excluding the target hospitals and all the hospitals in the same system as the target. The hospital’s relative cost factor is then adjusted for quality, using the scores calculated in the quality submodel, to produce a quality-adjusted relative cost factor. Those hospitals with a quality-adjusted relative cost factor that exceeded a specific threshold were subject to exclusion from the network. Using the results from the Milliman and cost models, Blue Shield then compiled a preliminary list of hospitals that were candidates for exclusion and retained those hospitals that it believed should be included in the network for reasons such as member access to tertiary services. DMHC reviewed Blue Shield’s proposed list of excluded hospitals and required it to include four hospitals that otherwise would have been excluded. For the purposes of our report, the 24 hospitals that were approved for exclusion by DMHC are designated as excluded hospitals and all others are designated as included hospitals. 4488 California State Auditor Report 2004-123 California State Auditor Report 2004-123 4499 Savings Model The savings model was designed by Blue Shield to estimate the hospital and physician savings that would be generated from excluding high-cost hospitals and associated medical groups from the network. The savings estimate consists of two components: hospital savings and medical group savings. The hospital savings are savings from excluding hospitals, and the medical group savings are the savings from excluding the medical groups that admit only to excluded hospitals. The hospital savings from the exclusive provider network were assumed to be the difference between the reduction in costs resulting from the redistribution of RVUs to the included hospital and the excluded hospitals’ costs for their undistributed RVUs. The hospital savings were calculated by redistributing a portion of the RVUs from the generally higher-cost excluded hospitals to included hospitals in the same cohort. RVUs diverted from excluded hospitals were distributed to included hospitals in the same cohort based on their market shares and were priced using the cost factor of the receiving hospital.34 Some RVUs were assumed to be retained at the excluded hospital. For most excluded hospitals, the cost per RVU for retained RVUs was unchanged. The medical group savings were calculated by allocating members to new primary care physicians based on the receiving medical groups having physicians who were open to new members, the new members’ ability to obtain a physician with the same specialty and hospital affiliation as their previous primary care physician, and proximity of the new physician’s office to the members’ previous primary care physician’s office. Once CalPERS members were reallocated, the cost of providing care at the new medical groups was calculated considering both the fixed amount paid for each person regardless of the actual services provided (capitated costs) and the specific amount paid for each procedure (fee-for-service costs). Medical group savings are assumed to be the difference between the cost at the excluded medical groups and the projected cost at new medical groups. The final savings estimate for moving to the exclusive provider network was the sum of the hospital savings and medical group savings. 34 The actual calculation is different but mathematically identical. In the actual calculation, the average of the cost factors for included hospitals weighted by RVUs was multiplied by the sum of diverted RVUs. 5500 California State Auditor Report 2004-123 California State Auditor Report 2004-123 5511 Savings-in-Base Model Distinct from the three models used in the analysis and summarized in the previous sections, Blue Shield prepared a fourth ad hoc model to estimate savings from financial terms for calendar years 2004 and 2005 that were negotiated with one health system after the initial analysis was completed. The model was developed in or around March 2004 and was used during negotiations to quantify the financial impact of the terms. 5500 California State Auditor Report 2004-123 California State Auditor Report 2004-123 5511 Blank page inserted for reproduction purposes only. 5522 California State Auditor Report 2004-123 California State Auditor Report 2004-123 5533 APPENDIX B Trend Information for Four Sample Hospitals Over a Three-Year Period The Joint Legislative Audit Committee (audit committee) directed the Bureau of State Audits (bureau) to select a sample of hospitals or hospital systems affected by the California Public Employees’ Retirement System board of administration’s decision to discontinue contracting with certain hospitals through the Blue Shield of California (Blue Shield) health maintenance organization (HMO) provider network and, to the extent possible, to identify certain trends for the hospitals for at least a three-year period. In particular, the audit committee directed the bureau to identify for the sample hospitals trends in profit margins and prices; systems’ market share of physicians and hospital services; percentages of revenue going to patient care, administration, and profits; expenditures for charity care and other community benefits; operating characteristics; and executive compensation. To accomplish this objective, we judgmentally selected a sample of four hospitals: two hospitals excluded from Blue Shield’s HMO provider network and two hospitals potentially affected by the exclusion of other hospitals. Our definition of affected hospitals includes those hospitals in the same geographic area as the excluded hospital that could potentially receive CalPERS member patients. We requested certain data from the four hospitals for either the three calendar years from 2001 through 2003 or for the three fiscal years from 2001–02 through 2003–04, depending on the hospitals’ reporting period. The tables in Appendix B present the trends for these four hospitals. Although federal law requires that some of the underlying information that we relied on to produce these results must be made publicly available by the hospitals, other information is confidential and the hospitals have no legal obligation to publicly disclose it. Consistent with the bureau’s legal obligation to protect confidential information, we have identified these hospitals using only the generic labels—Hospital A, Hospital B, Hospital C, and Hospital D—and we have generalized or aggregated the confidential data so that it presents general trends among hospitals and does not reveal confidential information. We were unable to generalize or aggregrate confidential data relating to the hospitals’ contract prices or operating characteristics in a 5522 California State Auditor Report 2004-123 California State Auditor Report 2004-123 5533 manner that would both protect the confi dentiality of the data and be meaningful to the reader. Therefore, we do not present this information. Net income margin is calculated as the excess of revenue over expenses divided by the total operating and nonoperating We have received the cooperation of the revenue. hospitals in gathering and presenting this Operating income margin is calculated as information, but the bureau did not perform any operating revenue minus operating expenses testing to ensure the validity or reliability of divided by operating revenue. the data. Thus, the bureau does not present any Source: Bureau of State Audits. fi ndings and has not drawn any conclusions related to these data. Profi t Margins To identify the trends in profi t margins, we reviewed the hospitals’ annual fi nancial statements. If the hospital was part of a system, we verifi ed that its fi nancial statement information materially agreed with the amounts shown in the system’s audited fi nancial statements. Table B.1 shows trends in the hospitals’ net income margins and operating income margins. TABLE B.1 Hospitals’ Net Income Margin and Operating Margin Trends 2001* 2002* 2003* Net Income Operating Net Income Operating Net Income Operating Hospital Margin Income Margin Margin Income Margin Margin Income Margin Hospital A 5.2% (2.1%) 3.4% (4.0%) 3.4% (5.1%) Hospital B 2.9 3.0 4.2 4.2 7.0 6.7 Hospital C 6.7 2.9 10.6 9.5 12.5 11.4 Hospital D 1.0 1.0 1.3 2.0 1.0 1.7 *The reporting period represents either the hospitals’ fi scal year or the calendar year. 5544 California State Auditor Report 2004-123 California State Auditor Report 2004-123 5555 Number of Physicians in the Same Cohort as the Sample Hospitals In determining the trends in the number of physicians in the same cohort as our sample hospital, we obtained unaudited data from the Medical Board of California (medical board) on physicians throughout the State for fiscal years 2001–02 through 2004–05.35 Using the cohorts established by Blue Shield of California (Blue Shield) and the medical board’s unaudited data, we identified the Blue Shield provider hospitals and other hospitals in a cohort with which Blue Shield does not contract. From that data, we determined the number of licensed physicians in each cohort. We also obtained from the four hospitals a roster of physicians who are eligible to admit patients to the hospital and calculated the number of physicians. Using the number of physicians in each of the four hospitals and in their cohorts, we calculated the physicians in the hospitals as a percentage of physicians in the cohort. Table B.2 presents the trends in physician data for the four hospitals. TABLE B.2 Trends in the Number of Physicians in the Same Cohort as the Four Sample Hospitals Sample Hospital 2001–02* 2002–03* 2003–04* 2004–05* Hospital A Not available 10.6% 10.4% 10.9% Hospital B 14.6% 14.0 14.7 Not available Hospital C Not available Not available Not available 18.3 Hospital D Not available 8.5 Not available 8.8 Sources: The Medical Board of California’s unaudited data and hospitals’ physician rosters. *The number of physicians per sample hospital represents a specific point in time that may or may not be representative of the entire fiscal year. 35 The data we obtained from the medical board are subject to limitations. According to the medical board, it tracks whether or not a physician possesses a current license to practice medicine in the State of California. Thus, we cannot determine the number of actively practicing physicians. In addition, the address the physician provides to the board is only a mailing address, which may be the physician’s business or residence. Thus, in our analysis, the address may place the physician in a cohort with our sample hospitals, but the physician may practice medicine outside the cohort. Finally, the data does not account for physicians with admitting privileges at more than one hospital or at no hospital. 5544 California State Auditor Report 2004-123 California State Auditor Report 2004-123 5555 Market Share of Hospital Services Licensed beds are those stated in the license To compile the hospitals’ trends in their respective at the end of the reporting period, excluding market shares of hospital services, we used statewide beds placed in suspense and nursery bassinets. hospital annual fi nancial data from Offi ce of Statewide Health Planning and Development Available beds are the average daily number of beds (excluding nursery bassinets) (OSHPD). Using the OSHPD data, we identifi ed the physically existing and actually available for hospitals in the same cohort as each of the four overnight use, regardless of staffi ng levels. hospitals and extracted the licensed and available acute care, psychiatric care, chemical dependency, Source: Offi ce of Statewide Health Planning and Development. rehabilitation, long-term care, and residential care bed data (see the text box) for all hospitals in each cohort. We also verifi ed that the four hospitals’ licenses matched OSHPD’s licensed-bed data, and determined each cohort’s total number of licensed and available beds for the six categories of beds. Table B.3 shows the hospitals’ trends in their market shares. TABLE B.3 Hospitals’ Trend in Their Market Share of Their Cohort’s Licensed and Available Beds Market Share 2001* 2002* 2003* Available Available Available Hospital Service Licensed Beds Beds† Licensed Beds Beds† Licensed Beds Beds Hospital A 5.1% 5.6% 5.3% 5.6% 5.3% 5.7% Hospital B 10.5 10.8 10.5 10.6 10.4 10.7 Hospital C 20.1 18.1 20.1 18.4 19.6 18.5 Hospital D 3.9 4.2 3.9 4.3 4.0 4.3 Source: Offi ce of Statewide Health Planning and Development. *The reporting period represents either the hospitals’ fi scal year or the calendar year. Comparison of Licensed-Bed Occupancy Rates We compiled the hospitals’ trends in their licensed-bed occupancy rates for the acute care, psychiatric care, chemical dependency, rehabilitation, long-term care, and residential care bed types using the OSHPD data. We divided each hospital’s patient days by the number of bed days, which is the number of days in the reporting period times the number of licensed beds at the end of the reporting period. We performed the same calculation for the hospitals in each of the four hospitals’ 5566 California State Auditor Report 2004-123 California State Auditor Report 2004-123 5577 cohorts to calculate the cohorts’ occupancy rate. OSHPD defines patient days as the number of days that all formally admitted inpatients spend in a hospital during a reporting period. Table B.4 presents the trends in the hospitals’ and cohorts’ licensed-bed occupancy rates. TABLE B.4 Hospitals’ Licensed-Bed Occupancy Rate in Comparison to the Cohort’s Occupancy Rate 2001* 2002* 2003* Hospital Cohort Hospital Cohort Hospital Cohort Hospital A 86.7% 59.4% 85.2% 61.9% 86.9% 62.6% Hospital B 59.1 64.3 53.1 65.6 52.3 65.1 Hospital C 63.7 64.3 68.0 65.6 70.5 65.1 Hospital D 68.0 59.4 70.5 61.9 69.8 62.6 Source: Office of Statewide Health Planning and Development. *The reporting period represents either the hospitals’ fiscal year or the calendar year. Patient Care, Administration, and Net Income To calculate the percentage of total revenue going toward each hospital’s profit, or excess of revenue over expenses, we reviewed the hospitals’ annual financial statements. In addition, we reviewed the direct costs the hospitals reported to OSHPD or prepared for us. Using either the hospital’s direct costs reports or OSHPD data, we calculated the percentages of the hospitals’ total revenue going toward patient care and administration costs, to the extent possible. Although the OSHPD Accounting and Reporting Manual for California Hospitals has a separate reporting category for administrative services, we also included in our calculation its fiscal services category, which covers services such as general accounting, patient accounting, and 5566 California State Auditor Report 2004-123 California State Auditor Report 2004-123 5577 credit and collection. Patient care includes the OSHPD categories of daily hospital, ambulatory, ancillary, and purchased services. Tables B.5 through B.8 present our results. TABLE B.5 Hospital A’s Trend in Patient Care, Administration, and Net Income as a Percentage of Total Revenues 2001* 2002* 2003* Patient care 46.3% 44.4% 45.3% Administration 16.0 16.6 16.0 Net income 5.2 3.4 3.4 Source: Hospital A’s financial statements and cost center reports. * The reporting period represents either the hospital’s fiscal year or the calendar year. TABLE B.6 Hospital B’s Trend in Patient Care, Administration, and Net Income as a Percentage of Total Revenues 2001* 2002* 2003* Patient care 63.7% 66.5% Data unavailable Administration 15.7 13.6 Data unavailable Net income 2.9 4.2 7.0 Sources: Hospital B’s financial statements and Office of Statewide Health Planning and Development data. *The reporting period represents either the hospital’s fiscal year or the calendar year. 5588 California State Auditor Report 2004-123 California State Auditor Report 2004-123 5599 TABLE B.7 Hospital C’s Trend in Patient Care, Administration, and Net Income as a Percentage of Total Revenues 2001* 2002* 2003* Patient care 53.8% 51.2% 51.0% Administration 15.4 15.4 16.0 Net income 6.8 9.9 12.6 Source: Office of Statewide Health Planning and Development data. *The reporting period represents either the hospital’s fiscal year or the calendar year. TABLE B.8 Hospital D’s Trend in Patient Care, Administration, and Net Income as a Percentage of Total Revenues 2001* 2002* 2003* Patient care 64.7% 71.7% Data unavailable Administration 13.4 14.1 Data unavailable Net income 1.0 1.3 1.0% Sources: Hospital D’s financial statements and Office of Statewide Health Planning and Development data. *The reporting period represents either the hospital’s fiscal year or the calendar year. Expenditures for Charity Care and Other Community Benefits Charity care is the cost of services that hospitals provide to patients who have demonstrated an inability to pay for the services. State law requires private not-for-profit hospitals to adopt community benefits plans and file reports on their plans and activities with OSHPD. State law defines community benefits as a hospital’s activities that are intended to address community needs and priorities primarily through disease prevention and improvement of health status. Examples of community benefits are health-care services given to vulnerable populations, financial support of public health programs, and the promotion of health education and prevention of disease. To identify the hospitals’ trends in the cost of providing charity care and other community benefits, we reviewed each sample hospital’s financial statements, including the related notes. In 5588 California State Auditor Report 2004-123 California State Auditor Report 2004-123 5599 addition, we reviewed the hospital’s community benefits plan, and other relevant information. Although one hospital does provide both charity care and community benefits, it does not track the value of its community benefits. Thus, we do not present any information in the table for this hospital. Table B.9 presents the hospitals’ trends in expenses for charity care and other community benefits as a percentage of their net patient revenues. TABLE B.9 Hospitals’ Trends in Charity Care and Other Community Benefits Expenses as a Percentage of Net Patient Revenues 2001* 2002* 2003* Hospital A 9.0% 9.5% 8.9% Hospital B 3.6 8.3 9.6 Hospital C 19.6 12.6 19.3 Source: Individual hospitals. *The reporting period represents either the hospitals’ fiscal year or the calendar year. Executive Compensation To identify the hospitals’ trends in executive compensation, for the four hospitals we totaled the amounts reported on their Internal Revenue Service (IRS) form 990 or reviewed information prepared for us. The IRS requires each tax-exempt organization to report, among other information, the compensation paid to its officers, directors, trustees, and key employees. The IRS defines a key employee as any person having responsibilities or powers similar to those of officers, directors, or trustees, such as the chief management and administrative officials of an organization. Executive compensation includes salary; fees; bonuses; severance pay; pensions; other benefits such as medical, dental, and life insurance; deferred compensation; and expense accounts. Table B.10 presents the hospitals’ trends for executive compensation. 6600 California State Auditor Report 2004-123 California State Auditor Report 2004-123 6611 TABLE B.10 Hospitals’ Trends for Executive Compensation 2001* 2002* 2003* Executive Number of Executive Number of Executive Number of Compensation Employees Compensation Employees Compensation Employees Hospital (in Millions) Included (in Millions) Included (in Millions) Included Hospital A $8.1 24 $3.4 7 $4.1 6 Hospital B† — — 1.1 5 1.4 7 Hospital C 0.7 2 0.7 2 0.5 1 Hospital D 2.7 10 2.8 10 2.2 9 Source: IRS form 990s or other relevant information. *The reporting period represents either the hospitals’ fiscal year or the calendar year. † Hospital B’s executive compensation includes taxable income and deferred compensation, but does not include other benefits. 6600 California State Auditor Report 2004-123 California State Auditor Report 2004-123 6611 Blank page inserted for reproduction purposes only. 6622 California State Auditor Report 2004-123 California State Auditor Report 2004-123 6633 Agency’s comments provided as text only. State and Consumer Services Agency 915 Capitol Mall, Suite 200 Sacramento, CA 95814 March 22, 2005 Elaine Howle, State Auditor* Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Dear Ms. Howle: Enclosed is the response prepared by the California Public Employees’ Retirement System to the Bureau of State Audits’ Report No. 2004-123 entitled, California Public Employees Retirement System: It Relied Heavily on Blue Shield of California’s Exclusive Provider Network Analysis Which Is Reasonable in Approach but Includes Some Questionable Elements and Possibly Overstates Savings. A copy of the response is also included on the enclosed diskette. If you have any questions or need additional information, please contact me at (916) 653-4090. Sincerely, (Signed by: Fred Aguiar) Fred Aguiar, Secretary Enclosures * California State Auditor’s comments begin on page 71. 6622 California State Auditor Report 2004-123 California State Auditor Report 2004-123 6633 CalPERS Executive Office P.O. Box 942701 Sacramento, Ca 94229-2701 March 21, 2005 Frank Aguiar, Secretary State and Consumer Services Agency 915 Capitol Mall, Suite 200 Sacramento, CA 95814 Subject: Response to Draft BSA Report on Exclusive Provider Network Decision Dear Mr. Aguiar: Enclosed is our response to the Bureau of State Audits (BSA) draft report titled California Public Employees Retirement System: It Relied Heavily on Blue Shield of California’s Exclusive Provider Network Analysis Which Is Reasonable in Approach but Includes Some Questionable Elements and Possibly Overstates Savings (March 2005, Report No. 2004-123). Please note that the Agency response (in hard copy and on diskette) is due to BSA by close of business March 22, 2005. Please contact me or Jarvio Grevious if you have any questions about our response or need further information. Sincerely, (Signed by: Fred Buenrostro) Fred Buenrostro Chief Executive Office California Public Employees’ Retirement System 6644 California State Auditor Report 2004-123 California State Auditor Report 2004-123 6655 March 21, 2005 Elaine M. Howle, State Auditor Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, CA 95814 Subject: Response to Draft Report on Exclusive Provider Network Decision Dear Ms. Howle: We appreciate the opportunity to formally respond to the Bureau of State Audits (BSA) draft report titled California Public Employees Retirement System: It Relied Heavily on Blue Shield of California’s Exclusive Provider Network Analysis Which Is Reasonable in Approach but Includes Some Questionable Elements and Possibly Overstates Savings (March 2005, Report No. 2004-123). Before proceeding with the details of our response, we must offer the following comments relative to the report as a whole: • BSA’s draft report does note that the CalPERS Board of Administration implemented the 2005 Blue Shield CalPERS exclusive provider network not only to realize cost savings, but 1 also “to initiate structural reform in the health care industry.” However, the report focuses nearly exclusively on the issue of whether the cost savings were overstated. Throughout the deliberation process that led to the Board’s decision, the Board, health program staff, and Blue Shield staff focused on whether the exclusive provider network would reduce cost while also maintaining appropriate quality of care, minimizing member and physician disruption, and meeting the regulatory requirements of the Department of Managed Health Care. We believe the BSA’s report should more appropriately reflect the Board’s focus in its decision-making process not only on cost, but also on quality, stability, and regulatory compliance. • We are dismayed that we were not provided with a complete copy of the draft audit report and believe this hampered our ability to make a full and comprehensive response to the BSA’s finding and recommendation. We also are concerned that our comments may not be made in the appropriate context of the full report. We reserve the right to supplement or amend our response based upon our review of the full draft audit report. • We have been further restricted in our response by your direction that we not discuss the Blue 2 Shield portion of the draft report with Blue Shield staff who, as our contractual agent, worked very closely with CalPERS on the exclusive provider network issue. We do not believe that 3 Government Code section 8454.1, which you cite as an authority, supports this directive. The intent of section 8545.1 is to prevent the public disclosure of restricted information or information that could compromise an audit. Since Blue Shield provided much of the information on which CalPERS based its decision, any discussion between CalPERS and Blue Shield about the draft report would not cause the disclosure of restricted information. Although we have honored the BSA’s request, we do not concede that we are legally prohibited from conferring with Blue Shield. We reserve the right to supplement or amend our response based on any additional information that comes to our attention prior to the release of the final report. 6644 California State Auditor Report 2004-123 California State Auditor Report 2004-123 6655 -2- March 21, 2005 Overview of CalPERS Response As indicated in the title of the draft audit report, the BSA has concluded that CalPERS relied heavily on Blue Shield’s analysis related to the impact of the implementation of the exclusive provider network in 2005. While acknowledging that Blue Shield’s analysis was “reasonable in approach,” the BSA also concludes that the analysis contained some questionable elements and that CalPERS did not carefully consider the review by an external actuarial consultant. Further, the BSA concludes that Blue Shield’s analysis possibly overstated the savings that could result from the network change. I will first address these conclusions in general and then proceed with more specific comments: 4 • Our reliance on Blue Shield’s analysis of contracted hospital rates relative to both cost and quality criteria was appropriate and consistent with the terms of our contract with the plan. Section 7.14.1 of Blue Shield’s agreement with CalPERS for 2004-2006 specifies that Blue Shield is not obligated to provide any information to CalPERS that would cause the plan “to breach the terms of any contract to which Blue Shield of California is a party.” As noted in the BSA’s report, Blue Shield’s contracts with its providers “specifically prohibit the disclosure of certain information, including rates of payment.” This information is generally not made available by health plans to purchasers. 5 • Bringing due diligence to this important decision, we arranged for outside actuarial consultants to validate the methodology used by Blue Shield to determine which hospitals to recommend for exclusion from the exclusive provider network for CalPERS. Health program staff reviewed 6 hospital-specific cost and quality data with the Board, which included providing confidential Blue Shield information to the Board. • Health program staff independently validated that the projected savings from the exclusive 7 provider network were included in the final premiums negotiated with Blue Shield for 2005. As part of this validation, staff compared total annual premium, premium savings, and rate increase percentages for Blue Shield for 2005 with the standard network (without excluding any hospitals or medical groups), with the narrow network, and with an assumption that implementing the exclusive provider network would provide Blue Shield with further leverage in negotiating hospital contracts that were still open for 2005. 8 • The report’s conclusion that savings from the exclusive provider network were overstated is unwarranted. All our discussions of the exclusive provider network – whether held in public or in closed session – estimated a range of savings. The range of $25-$50 million was conservative, taking into account “worst case” scenarios related to the costs of providing continuity of care and paying for out-of-network emergency room usage. Our best estimate 9 was $36 million, savings which have already been realized for California taxpayers in the 2005 premiums we negotiated with Blue Shield. We were very conservative in our projections and so far haven’t seen any areas of concern materializing. We will continue to monitor this as part of our ongoing review of Blue Shield’s monthly financial reports, which include trend projections. 6666 California State Auditor Report 2004-123 California State Auditor Report 2004-123 6677 -3- March 21, 2005 • The report’s conclusion that savings estimates did not “consider the impact of members leaving 0 the Blue Shield HMO provider network and joining other health care plans” is incorrect. We arranged for Mercer Human Resource Consulting to evaluate the potential impact of member movement resulting from the change in Blue Shield’s provider network for CalPERS members. This analysis considered the impact of potential member movement on rates not only for Blue Shield, but also for our self-funded Preferred Provider Organization plans (PERS Choice and PERSCare) since our “worst case” assumption was that exiting members would move to the PPOs in order to retain their current physician and that many of the members moving would be high utilizers of services. Final 2005 rates for both Blue Shield and PERS Choice reflected this analysis. Specific Report Responses 8 1. Title – The phrase Possibly Overstates Savings in the title is unwarranted as noted above and below. 2. The report indicates “…transcripts of board and committee meetings do not indicate that CalPERS used the report generated by the model-review actuary in its decision-making process.” The report also indicates “Without addressing the model-review actuary’s concerns, CalPERS had no assurance that Blue Shield analysis was accurate.” This is incorrect. CalPERS had a substantial basis for relying on Blue Shield’s analysis of the relative standing of hospitals and the savings that would result if specified ones were eliminated from the hospital, as follows: 5 • CalPERS used the services of three qualified health actuaries in conducting its analysis and review – Blue Shield of California, Milliman, and Reden and Anders. All three actuaries concluded that the analytical approach and method used was sound and reasonable. Further, all three concluded that the methodology and data used for the CalPERS analysis q was substantially superior to the alternative approach using OSHPD data. 5w • Two of the three actuaries endorsed the methodology used for the savings forecast and concluded that the forecast was reasonable and credible. The third ran out of time. 7 • CalPERS staff independently verified that the savings estimate was included in the development of the 2005 final rate proposal for the Board. • Preliminary estimates related to the 2006 rate renewal indicate that the narrow network e savings estimated for 2005 are sound and, if anything, may be slightly understated due to lower than anticipated out-of-network emergency room use. r 3. The statement “…staff participated in meetings and conference calls with Blue Shield on five days between July 2003 and October 2003” implies that these were the only discussions held. The five meetings and conference calls were the “working sessions” we were able to document for the BSA. Numerous other meetings and discussions were held with Blue Shield during this time period, as warranted by such an important decision. 6666 California State Auditor Report 2004-123 California State Auditor Report 2004-123 6677 -4- March 21, 2005 4. The report states: “…According to CalPERS, although it requested more detail, Blue Shield responded with as much detailed information as it was able to provide, given the confidential nature of the information.” It should be noted that the Blue Shield report referenced in this discussion was mailed by CalPERS to all Board members on March 10, 2004. A copy of this report was provided to the BSA on August 23, 2004. 5. The report states that “the model-review actuary did not opine on Blue Shield’s savings 5w estimate and, thus, his report could not provide a credible basis for the CalPERS Board to evaluate Blue Shield’s cost savings projection…” As stated above, we had ample other basis for using the Blue Shield estimate. 6. The report indicates that Reden and Anders’s review “did not include assessing the capacity, availability, and accessibility of services…” This leaves the impression that these issues were not addressed. The Board spent countless hours in public and private sessions reviewing analyses of these issues to ensure that there was adequate alternative capacity and access to services. Your report notes later in the same paragraph, “Blue Shield stated that…it reviewed the capacity at included hospitals.” Public documents used in the April and May 2004 Health Benefits Committee and Board meetings reflect these considerations. t 7. The report states that the Health Benefits Committee and the Board “did not discuss all of the actuary’s findings and recommendations and their impact on CalPERS’ decision in the meetings held before the board voted to approve the exclusive provider network on May 19, 2004.” It should be noted that the Board discussed the Reden and Anders review in depth at a closed session meeting on April 20, 2004. Transcripts were not made for this closed session, but we have provided staff notes for this discussion to the BSA. 8. The report indicates that “The model-review actuary’s report does not contain this statement” with regard to a board member’s incorrect statement regarding the actuary certifying that “the hospital system’s costs were 60 percent higher than northern California hospitals and 80 percent higher than southern California hospitals.” It should be noted that the actuary did opine clearly that the use of the OSHPD data was inappropriate for this purpose and that the approach used to determine the relative standings of hospitals was reasonable. The latter point was the basis for the board member’s statement. t 9. The report indicates that the BSA found “no mention in the transcripts of the model-review actuary’s inability to render an opinion on the savings estimate.” There is no reference to this in the transcript for the open session, but the model-review actuary’s findings and inability to render an assessment of savings were discussed in closed session on April 20, 2004. 10. The report expresses concern that the savings estimate could be overstated “because Blue Shield did not account for members leavings its HMO provider network.” This point is inaccurate in this 0 context as member movement had no effect on Blue Shield’s narrow network savings estimate. As indicated earlier, when finalizing the 2005 HMO and PPO rates, the Board considered the potential impact of members moving from Blue Shield to a PPO (primarily PERS Choice) in order to retain their current provider. This shift would actually result in a savings to the HMO and a commensurate cost to the PPO due to the assumed higher risk of this population. Adjustments were made to the final 2005 Blue Shield and PERS Choice premiums to provide for this. 6688 California State Auditor Report 2004-123 California State Auditor Report 2004-123 6699 -5- March 21, 2005 Response to Recommendations The draft report contains two recommendations: • The first is that legislation be enacted to …”allow CalPERS, during its contract negotiation process, to obtain relevant documentation supporting any analyses it will use to materially affect 5 the members in the health benefits program…” Although CalPERS exercised all due diligence in its decision to implement the Blue Shield exclusive provider network in 2005, we could support legislative changes that would make additional data available to CalPERS in the future. • The second recommendation is that CalPERS “…require its health benefits branch staff to evaluate fully the findings and recommendations of third-party reviews and present their results t to the Board and the Health Benefits Committee.” CalPERS staff did evaluate fully the findings and recommendations of third-party reviews and presented our results to the Board and the Health Benefits Committee, and we will continue to do so. In conclusion, I want to thank the BSA for the opportunity to respond to the report’s conclusions and recommendations. We hope the final report will reflect our input, particularly the fact that the Blue Shield CalPERS exclusive provider network was implemented not only to achieve cost savings, but also to take a significant step towards achieving greater transparency of and accountability for the cost of healthcare paid for by CalPERS members and employers and ultimately by all California taxpayers. Please don’t hesitate to contact me or Jarvio Grevious if you have any questions about our response or need further information before the final audit report is released. Sincerely, (Signed by: Fred Buenrostro) Fred Buenrostro Chief Executive Officer California Public Employees’ Retirement System cc: CalPERS Board of Administration Jarvio Grevious 6688 California State Auditor Report 2004-123 California State Auditor Report 2004-123 6699 Blank page inserted for reproduction purposes only. 7700 California State Auditor Report 2004-123 California State Auditor Report 2004-123 7711 COMMENTS California State Auditor’s Comments on the Response From the State and Consumer Services Agency and the California Public Employees’ Retirement System To provide clarity and perspective, we are commenting on the California Public Employees’ Retirement System (CalPERS) response to our audit. The numbers below correspond to the numbers we have placed in its response. 1 CalPERS’ opinion regarding the appropriate focus for our audit is inconsistent with the Joint Legislative Audit Committee’s (audit committee) directive to the Bureau of State Audits (bureau). Specifically, as we state on page 11, the audit committee directed the bureau to examine the information the CalPERS board of administration (board) used to make its decision to exclude hospitals from the Blue Shield of California (Blue Shield) health maintenance organization (HMO) provider network and to estimate the resulting cost savings. In addition, the audit committee instructed the bureau to determine whether the information, including any analyses, data, and methodologies, is valid and provides a clear case for conclusions drawn. Thus, the focus of our report is aimed appropriately at fulfilling the audit committee’s directive. 2 CalPERS states correctly that it did not receive a complete copy of the draft report. It is the bureau’s customary practice to provide the agencies that it audits with a draft report for their review and comment before the report is made publicly available. The bureau makes every effort to provide agencies with as much information as possible so that they are able to respond in a meaningful way to our audit findings and recommendations. The bureau does this so that it treats the agencies that it audits fairly, ensures its audit reports are factually accurate, and complies with generally accepted government auditing standards. This audit presented a somewhat unusual set of circumstances in that during this audit the bureau was able to obtain highly confidential information from various private parties, including 7700 California State Auditor Report 2004-123 California State Auditor Report 2004-123 7711 Blue Shield, that CalPERS had not been able to review prior to making its decision related to the exclusive provider. CalPERS did not have access to this confidential information because the parties with whom it was negotiating had contractual agreements with other third parties that prohibited them from disclosing this information. The statutes that the state auditor operates under allow the state auditor to receive and review confidential information, but prohibit the state auditor from disclosing that information if some law prohibits disclosure or allows that information to be withheld from public disclosure. Based on that authority, the various private parties involved, including Blue Shield, worked cooperatively with the bureau to allow access to this highly confidential information, with the clear understanding that it would not be disclosed, either publicly or to any other party who did not have the legal authority to obtain this information. Consequently, some of the information that the bureau reviewed and analyzed during this audit could not be shared with any other party, including, in some cases, CalPERS. To comply with its legal obligation not to improperly disclose confidential information, the bureau worked very closely with the various parties involved during the review process. The bureau provided Blue Shield with a portion of the draft report relating to the information it received from Blue Shield. The bureau also provided CalPERS with a portion of the draft report, which did not initially include those portions that were sent to Blue Shield for its review. The bureau did this because it was necessary to ensure that the portion of the report that relied on confidential information obtained from Blue Shield did not improperly disclose confidential information. Once the bureau had resolved Blue Shield’s concerns related to confidentiality, it provided CalPERS with an opportunity to review a more complete version of the draft report. Although CalPERS did not receive this more complete version of the report in time to respond to that portion in its written comments, it is important to point out that those portions of the report that CalPERS did not have at the beginning of the review process related primarily to aspects of Blue Shield’s analysis that CalPERS did not have access to when the board of administration made the decision to adopt the exclusive provider network. 3 CalPERS is correct that the primary purpose of Section 8545.1 of the Government Code, which generally prohibits disclosure of audit findings prior to the time they are made public is 7722 California State Auditor Report 2004-123 California State Auditor Report 2004-123 7733 designed to prevent public disclosure of information that could compromise the integrity of the audit. However, as discussed previously, this audit presented a somewhat unusual situation that required the bureau to resolve the various private parties’ concerns related to confidentiality before it could share all of its findings and conclusions with CalPERS. Once the bureau resolved those concerns, it agreed that the parties could consult with one another concerning their response to the report and made this known to all parties. 4 Although CalPERS states correctly the limitations imposed by section 7.14.1 of its contract with Blue Shield, it fails to mention another provision in the contract that permits CalPERS to use an independent health actuary to audit the data and methods Blue Shield uses to establish rates and payments. In early April 2004 CalPERS benefits branch staff directed Blue Shield to hire an independent health actuary (model-review actuary). However, as we discuss on pages 16 through 22, CalPERS did not fully consider the findings and recommendations made by the model-review actuary prior to approving the exclusive provider network. Thus, we disagree that CalPERS’ reliance on Blue Shield’s analysis was appropriate and consistent with its contract terms. 5 CalPERS statement that it brought due diligence to this important decision is questionable. CalPERS states that it arranged for actuarial consultants to validate Blue Shield’s methodology for excluding hospitals from its exclusive provider network for CalPERS members. Specifically, according to CalPERS, it used the actuarial services of Blue Shield, Milliman USA, Inc. (Milliman), and the model-review actuary in conducting its reviews of Blue Shield’s analysis. However, because Blue Shield developed the methodology and Milliman assisted with the development of certain portions of the methodology, we question their ability to render an independent conclusion concerning the accuracy of Blue Shield’s methodology. Moreover, although it had an opportunity to benefit from an independent third-party review, as we discuss on pages 16 through 22, CalPERS did not fully consider the findings and recommendations made by the model-review actuary prior to approving the exclusive provider network. Thus, without fully addressing all of the concerns raised by the model-review actuary, CalPERS had no independent assurance that Blue Shield’s analysis was accurate. To address CalPERS’ concern, we modified the text on pages 4 and 21 to add the phrase “from an independent source.” 7722 California State Auditor Report 2004-123 California State Auditor Report 2004-123 7733 6 CalPERS’ response regarding the level of review that occurred is misleading. CalPERS fails to mention that its health benefits branch staff reviewed “summaries” of the data relating to hospital-specific cost and quality data. As we state on page 16, CalPERS health benefit branch staff did not have access to hospital rates, nor could they review Blue Shield’s cost comparison data. 7 CalPERS states that its staff independently validated that the projected savings from the exclusive provider network were included in the final premiums for 2005. However, when we requested documentation to verify its statement, CalPERS did not have this information readily available. Although CalPERS plans to provide this information to us, we were unable to verify its statement prior to the release of our report. 8 We disagree with CalPERS that our conclusion is unwarranted. As we discuss on pages 37 through 43, Blue Shield’s savings estimate does not account for members leaving its provider network. Blue Shield also did not investigate thoroughly concerns about its emergency room assumptions. In its response, CalPERS states that the range of savings was $25 million to $50 million. This was the estimate presented to the CalPERS board in open session on May 11, 2004, and, we understand, was based upon the exclusion of 38 hospitals before consideration of financial terms from one health system. On that same day, in a closed session presentation to the board, Blue Shield estimated savings with a range that was substantially lower due to the system’s financial terms. However, the model that Blue Shield provided to our consultant for review indicated a savings estimate of $31.4 million, not $36 million as presented in its May 11, 2004, presentation. Further, our consultant did not receive information relating to the assumptions or calculation of either of the ranges of estimated savings. Therefore, we have no opinion about what the ranges were intended to represent or whether these estimates are accurate or reasonable. 9 CalPERS states that the $36 million in savings has already been realized for taxpayers in the 2005 premiums. However, when we requested documentation to verify its statement, CalPERS did not have this information readily available. Although CalPERS plans to provide this information to us, we were unable to verify its statement prior to the release of our report. 7744 California State Auditor Report 2004-123 California State Auditor Report 2004-123 7755 0 CalPERS is missing our point. As we state on pages 40 and 41, the model-review actuary found that Blue Shield’s savings forecast model did not explicitly project the potential impact on savings (or the added expense) of the possible shift of enrollment from Blue Shield’s HMO provider network to other coverage options available to CalPERS members. CalPERS refers to an analysis prepared by Mercer Human Resource Consulting (Mercer) as evidence of its consideration of member movement. However, when we asked CalPERS if Mercer issued a formal report, it stated no. As support for Mercer’s analysis we were given a copy of the slide presentation Mercer presented to the board on May 11, 2004, that did not provide sufficient information to allow our consultant to evaluate the basis for or reasonableness of Mercer’s conclusion. The slide presentation addressed the movement of high cost members from the HMO to PPOs and concluded that “One Rate Decrease Offsets Other Rate Increase.” This conclusion seems to be inconsistent with assertions Blue Shield made to our consultant that benefit differences between HMOs and PPOs would result in additional savings to CalPERS. Nevertheless, the larger issue is that CalPERS fails to recognize that Blue Shield did not account for member movement in its analysis, in effect it assumed no member movement, which we believe is an unrealistic assumption. q CalPERS is overstating the model-review actuary’s conclusions. Specifically, as stated on pages 18 and 19 the board directed CalPERS to proceed with a third-party review to resolve the differences between Blue Shield’s and a health system’s analyses. A primary factor in the differences was the data used in the analyses. The hospital system’s analysis was prepared using Office of Statewide Planning and Development (OSHPD) data, while Blue Shield used its claim data. The model-review actuary concluded that it is inappropriate to use OSHPD data as the sole source for hospital comparisons because OSHPD data are not available to the public in sufficient detail to allow a credible hospital-specific, payor-specific cost analysis. He also concluded that in general Blue Shield’s method of analyzing hospital costs, which includes using its claim data and an explicit consideration of a significant number of cost elements, is a reasonable basis for comparing hospital costs and should provide a credible basis for projecting the financial impact of the exclusive provider network. However, although the model-review actuary concluded that Blue Shield’s general method of analysis was reasonable, his report indicates that time constraints did not allow him to fully address some important assumptions relating to the analysis. 7744 California State Auditor Report 2004-123 California State Auditor Report 2004-123 7755 w CalPERS is attempting to downplay the role of the model-review actuary. As we state on page 19, in addition to reconciling the conclusions from the health system’s analysis to conclusions drawn from the Blue Shield analysis, the model-review actuary was hired to review Blue Shield’s cost savings projections for the exclusive provider network. Thus, CalPERS’ statement that the model-review actuary merely “ran out of time” does not depict accurately the fact that time constraints prevented him from fulfilling a key provision in his contract with Blue Shield. Consequently, the model-review actuary did not express an opinion regarding Blue Shield’s savings, and thus his report could not provide a credible basis for the CalPERS board to evaluate Blue Shield’s cost savings projections prior to its decision to adopt the exclusive provider network and exclude certain hospitals. e CalPERS states that its preliminary estimates related to the 2006 rate renewal indicate that the savings estimates for 2005 are sound and may be slightly understated. However, when we asked for documentation to verify its statement, CalPERS did not have this information readily available. Although CalPERS plans to provide this information to us, we were unable to verify its statement prior to the release of our report. r CalPERS states incorrectly that our report “implies” that these were the only discussions held. The characterization of events that took place prior to CalPERS’ February 6, 2004, working session was provided to us by CalPERS on December 13, 2004. Specifically, CalPERS stated that prior to February 6, 2004, it held meetings and/or conference calls on July 25, 2003, August 8, 2003, September 11, 2003, September 16, 2003, and October 3, 2003. Additionally, on March 7, 2005, CalPERS reiterated to us that these were the only dates and did not provide us with any evidence indicating that additional meetings or conference calls took place. Thus, as we state on page 16, according to CalPERS, its health benefits branch staff participated in meetings and conference calls with Blue Shield on five days between July 2003 and October 2003. t CalPERS states correctly that it provided us with its notes for the April 20, 2004 closed board meeting. However, the notes do not indicate that the board spoke in-depth about all of the model-review actuary’s findings and recommendations. Specifically, according to the notes, the former deputy executive officer for benefits administration informed the board that “the third party neutral actuary looked at costs between health 7766 California State Auditor Report 2004-123 California State Auditor Report 2004-123 7777 care facilities, Blue Shield 98 percent credibility and hospital system credibility 5 percent, supports Blue Shield study.” These statements are consistent with those made by the former deputy executive officer that we present on page 21. Further, although we would expect to see some discussion in the meeting notes regarding the model-review actuary’s inability to express an opinion regarding Blue Shield’s savings, the notes do not indicate that this important topic was discussed. 7766 California State Auditor Report 2004-123 California State Auditor Report 2004-123 7777 Blank page inserted for reproduction purposes only. 7788 California State Auditor Report 2004-123 California State Auditor Report 2004-123 7799 Agency’s comments provided as text only. Blue Shield of California March 23, 2005 Elaine M. Howle, State Auditor* Bureau of State Audits 555 Capitol Mall, Suite 300 Sacramento, California 95814 Re: Blue Shield of California’s Response to Report No. 2004-123 Dear Ms. Howle: On behalf of Blue Shield of California (Blue Shield), thank you for allowing us to submit this response to the redacted drafts of the Bureau of State Audits (BSA) Report No. 2004-123 provided to us on March 16 and 21, 2004 (Report). We appreciate that your staff already has considered and incorporated some of our earlier comments. Accordingly, this response is limited to those issues that we understand will remain in the final Report. As indicated in the Report’s title, the BSA has concluded that the California Public Employees Retirement System (CalPERS) relied heavily on Blue Shield’s “exclusive provider network analysis which is reasonable in approach but includes some questionable elements and possibly overstates savings.” Blue Shield is pleased that the BSA found our analysis “reasonable in approach.” This response is focused on the two tenets that Blue Shield’s analysis, according to the BSA, includes “some questionable elements” and “possibly overstates savings.” In summary, Blue Shield believes 1 that the savings forecasts are not overstated and, based on preliminary data from January and February 2005, may in fact be understated. We are also confident that the elements the BSA has identified as questionable had no material effect on the exclusive provider network analysis on which CalPERS relied. Please consider the following as our response on these issues. 1. The Elements Identified by the BSA As “Questionable” Do Not Have A Material Effect on Blue Shield’s Exclusive Provider Network Analysis. According to the Report and BSA staff, the elements deemed questionable are: (a) Blue Shield “did not adequately address a recommendation made by its model-review actuary to investigate differences in the emergency room assumptions for one hospital system”; (b) in addition to CalPERS HMO claims data, Blue Shield used claims data from all of its commercial business in the cost and savings models for its exclusive provider network analyses; and (c) Blue Shield did not reconcile the claims data used in the modeling against actual claims paid. Below, we address each of these three elements. * California State Auditor’s comments begin on page 83. 7788 California State Auditor Report 2004-123 California State Auditor Report 2004-123 7799 Emergency Room Assumptions: 2 • We believed at the time we completed the analysis and we believe now that the Emergency Room assumptions used in the analysis are reasonable and credible. If anything, it is possible that the savings forecast associated with Emergency Room assumptions is understated. The hospital system in question asserted that the diversion rate should be 75% or lower (a higher diversion rate increases savings to CalPERS and a lower rate would decrease savings to CalPERS). This stands in stark contrast to CalPERS’ own historical claim data which indicate a diversion rate of approximately 90%. In the savings estimate we assumed an 85% diversion rate, and the claims data for the first two months of 2005 indicate the diversion rate is higher rather than lower than our forecast. 3 • At the time we made the savings forecast we also reviewed our Emergency Room assumptions with Milliman, an independent actuary, who found them to be reasonable and credible. • In all situations, we presented our savings to CalPERS as a range around a best estimate. In our presentation to the board in open session on May 11, 2004, our report specifically highlighted the difficultly of making accurate Emergency Room assumptions and concluded: 4 “Consequently, it is best to think of the savings potential as a range instead of a specific number.” Our best estimate was approximately $36 million; however, the range was $25-$50 million. Even if we made the most extreme Emergency Room assumption highlighted in the Report, the savings would be at the bottom of our proposed range, although the most current data suggest the savings may be higher than we originally forecasted. CalPERS HMO claim data: • We agree with the BSA’s conclusion that: “Given that there was no ideal source of claim data to use in its analysis, Blue Shield’s decision to include data from all three sources [CalPERS HMO claims, other HMO claims, and PPO claims] does not appear unreasonable.” Reconciling data: 5 • We understand the BSA’s desire to ensure all of the numbers reconcile. Our understanding is that one of the major concerns that prompted this finding is the discrepancy between the total dollar amounts for the claims pulled by Blue Shield as compared to the total dollar amount run through Milliman’s model to do the hospital cost comparison. While we agree there are some steps we can take to improve the reconciliation process, it is important to note that the total dollar discrepancy between these two amounts is 1-2% of the total claims and as such is highly unlikely to have a material impact on the analysis. Also, the methods used to pull claims data are tested and used extensively in all aspects of our business. Page 2 of 4 8800 California State Auditor Report 2004-123 California State Auditor Report 2004-123 8811 2. Contrary to the BSA’s Conclusion, Preliminary Data Indicates Blue Shield May Have Underestimated CalPERS’ Savings from the Exclusive Provider Network. As we understand the Report, there are two factors leading to the conclusion that Blue Shield’s analysis “Possibly Overstates Savings.” 1.) BSA’s opinion that Blue Shield “did not adequately address a recommendation made by its model-review actuary to investigate differences in the emergency room assumptions for one hospital system.” 2.) “Blue Shield’s Savings Estimate Does Not Account for Members Leaving Its Provider Network.” Emergency Room Assumptions: 12 • We have addressed this issue with our comments in the previous section. 34 Members Leaving Blue Shield: 6 • Blue Shield and CalPERS did consider the potential savings impact of members leaving Blue Shield for another health plan. Specifically, CalPERS asked its benefits consultant, Mercer, to opine on the potential impact of members leaving Blue Shield for the PPO. Mercer concluded that due to the PPO products having deductibles and higher out-of-pocket member payments for hospital costs, CalPERS should experience savings commensurate with Blue Shield’s forecast even if the members change plans. As a result, we agreed to continue assuming that all members stayed with Blue Shield when forecasting the savings to CalPERS. • Blue Shield’s own updated savings forecast for Sacramento, using the most recent claims information, indicates the savings to CalPERS will be $4.7 million as compared to the original 7 $5.5 million. This does not include the additional savings CalPERS will accrue due to the benefit differences mentioned above. When these additional savings are added, it is quite possible that the total savings to CalPERS in the Sacramento area will be higher rather than lower than the original estimate. Other Responses • Use of OSHPD data as an alternative cost comparison analysis is inappropriate: We agree with the model-review actuary’s conclusion cited in the Report that: “OSHPD data are not available to the public in sufficient detail to allow a credible hospital-specific, payor-specific cost analysis.” We also agree with the conclusion of the BSA’s consultant that “the use of cost factors is a reasonable method for correcting for interhospital variation in case mix and severity.” 8 The alternative analysis highlighted in the Report that used OSHPD data did not use such cost factors and did not do any case mix or severity adjustment. Therefore, while OSHPD can serve as a valuable source of information to answer some questions, it is not a credible alternative to completing this type of analysis. 9 · Assurances to CalPERS: We believe the BSA’s conclusion that “Without addressing the model-review actuary’s concerns, CalPERS had no assurance that Blue Shield’s analysis was accurate,” is overstated. While the model-review actuary’s report may not have provided an independent validation of the savings, Blue Shield has highly qualified actuaries and had a Page 3 of 4 8800 California State Auditor Report 2004-123 California State Auditor Report 2004-123 8811 reasonable process for projecting the savings. We gave assurances to CalPERS, including a signed letter from our CEO regarding the 2005 rates, that the savings forecasts were our best estimates. Furthermore, we worked closely with Milliman USA, an independent actuary that found the approach as well as the savings estimates to be sound and credible. 0 • Applying the Threshold and Cohort Average Consistently: We believe that the changes made in the claims data set and treatment of one set of hospitals as a group was appropriate in that it maximized CalPERS’ savings and was the most fair and reasonable approach for the providers in question. It is important to note that we did analyze other groups of hospitals in similar circumstances and concluded that applying this approach universally would not change any individual hospital’s status. It is also worthy of note that the hospitals in question represented approximately 1.3% of CalPERS’ total claims. In conclusion, Blue Shield remains confident that CalPERS was fully justified in relying on our exclusive provider network analysis and that our assumptions, including, but not limited, to those regarding savings, are reasonable and sound. Blue Shield appreciates the opportunity to have this response considered and included in the final Report. Please feel free to contact me regarding any questions you may have. Yours sincerely, (Signed by: Paul Markovich) Paul Markovich SVP & Chief Executive Blue Shield of CA Page 4 of 4 8822 California State Auditor Report 2004-123 California State Auditor Report 2004-123 8833 COMMENTS California State Auditor’s Comments on the Response From the Blue Shield of California To provide clarity and perspective, we are commenting on the Blue Shield of California (Blue Shield) response to our audit. The numbers below correspond to the numbers we have placed in its response. 1 Blue Shield is referring to data that was not available to the California Public Employees’ Retirement System (CalPERS) board of administration (board) when making its decision to adopt the exclusive provider network. Additionally, Blue Shield did not provide its analysis of claim data for the first two months of 2005 to our consultant in time or with sufficient supporting documentation to allow us to determine whether it supports the emergency room assumptions used in the analysis. Nevertheless, based upon the information that was provided, our consultant has the following concerns with the analysis. • It is incomplete in that it only addresses emergency room related admissions and does not reflect continuity of care cases. The emergency room assumption was intended to address both. If continuity of care cases were included, the estimated diversion rate would be lower than indicated, which would tend to reduce estimated savings. • It appears to be based upon only 22 percent of inpatient claims resulting from emergency room visits for the first two months of 2005. This limited amount of data is effectively increased by a factor of over 27 times in order to project annual 2005 claim costs. The limited amount of data upon which the forecast is based, and the method of extrapolating to a full-year estimate, raise serious concerns about the reliability of the results. • A critical assumption in the analysis is the estimated total claim cost in 2005 that would have occurred had the exclusive provider network not been adopted. Blue Shield’s information does not state clearly how this estimate was derived and whether it properly reflects the significant reduction in enrollment experienced by Blue Shield in 2005. If it does 8822 California State Auditor Report 2004-123 California State Auditor Report 2004-123 8833 not fully reflect the reduction in enrollment, then the true diversion rate would be lower than indicated and the savings estimate would be lower. 2 Blue Shield’s statement that its emergency room assumptions are reasonable and credible causes us concern. Specifically, as we state on pages 41 through 43, Blue Shield did not, as recommended by the model-review actuary, thoroughly investigate concerns raised by one health system and did not determine how it and the system could have reached such different estimates using what appear to be similar methodologies. Determining an accurate estimate for the system’s emergency room diversion assumptions is crucial because the savings estimate is particularly sensitive to this assumption. Depending on whose assumptions are used, Blue Shield’s or the model-review actuary’s, the hospital savings estimate varies by $11.7 million from $20.6 million to only $8.9 million, a difference of more than 50 percent. 3 We disagree with Blue Shield’s characterization of Milliman USA, Inc. (Milliman), as an independent actuary. Given Milliman’s integral involvement with the development of Blue Shield’s analysis, we do not consider them to be independent with respect to this analysis. 4 In its response, Blue Shield states that its “best estimate (of savings) was approximately $36 million; however, the range was $25-$50 million.” This was the estimate presented to the CalPERS board in open session on May 11, 2004, and, we understand, was based upon the exclusion of 38 hospitals before consideration of financial terms from one health system. On that same day, in a closed session presentation to the board, Blue Shield estimated savings that was substantially lower due to the system’s financial terms. The importance of the emergency room assumption is even greater given this updated, lower savings estimate. However, the model that Blue Shield provided to our consultant for review indicated a savings estimate of $31.4 million, not $36 million as presented in its May 11, 2004, presentation. Further, we did not receive information relating to the assumptions or calculation of either of the ranges of estimated savings. Therefore we have no opinion about what the ranges were intended to represent or whether these estimates are accurate or reasonable. 8844 California State Auditor Report 2004-123 California State Auditor Report 2004-123 8855 5 Blue Shield has mischaracterized our finding relating to data that we present on page 22. Specifically, there were two aspects of our finding related to the reconciliation of data. The first is that Blue Shield made no attempt to ensure that the data transferred to Milliman were input into the model accurately. The second is that the data were not reconciled to other Blue Shield financial reports to ensure that the data used were comprehensive and complete. Blue Shield’s response is related solely to the first aspect and does not address the intent of the finding. The fact that there was a small discrepancy between the total dollar amount of claims pulled by Blue Shield as compared to the total dollar amount run through the Milliman model to do the hospital comparison is irrelevant. Our concern was that if a mistake had been made, neither Blue Shield nor Milliman were likely to discover it at the time the claims were run through the Milliman model. The fact that Blue Shield’s methods for extracting claims are tested and used extensively in its business reduces, but does not eliminate the possibility of a mistake. By comparing control totals Blue Shield and Milliman could have ensured that any mistake made in the data transfer process was identified. Our larger concern related to the reconciliation of data was the failure to reconcile the data used in the model with other Blue Shield financial reports. It was the understanding of the model- review actuary that the data used in the analysis had been reconciled to other Blue Shield financial data. When we asked Blue Shield for the reconciliation, it was unable to produce one. 6 Blue Shield’s statement that it and CalPERS did consider the potential savings impact of members leaving its provider network is misleading. As we state on page 40, the model- review actuary found that Blue Shield’s forecast model did not explicitly project the potential impact on savings (or the added expense) of the possible shift of enrollment from Blue Shield’s HMO provider network to other enrollment coverage options available to CalPERS members. Blue Shield’s consideration of member movement appears to be based on CalPERS’ benefits consultant’s, Mercer Human Resource Consulting (Mercer), conclusions that CalPERS should experience savings commensurate with Blue Shield’s forecast even if the members change plans. However, when we asked CalPERS if Mercer issued a formal report, it stated no. As support for Mercer’s analysis we were given a copy of the slide presentation Mercer presented to the board on May 11, 2004, that did not provide sufficient information to allow our consultant to evaluate the basis or 8844 California State Auditor Report 2004-123 California State Auditor Report 2004-123 8855 reasonableness of Mercer’s conclusion. The slide presentation addressed the movement of high cost members from the HMO to PPOs and concluded that “One Rate Decrease Offsets Other Rate Increase.” This conclusion seems to be inconsistent with Blue Shield’s assertion that benefit differences between HMOs and PPOs would result in additional savings to CalPERS. Nevertheless, the larger issue is Blue Shield’s failure to account for member movement in its analysis, in essence it assumed no member movement, which we believe is an unrealistic assumption. 7 Blue Shield states that using the most recent claim information, which is Network Choice VI (NC6) data, the savings to CalPERS in Sacramento would be $4.7 million as compared to the original $5.5 million. However, our concern is that these results are not comparable to the savings estimate for Sacramento that we discuss on pages 37 through 43, which is based on its Network Choice V (NC5), because the data used in each is different. For example, at one hospital the conversion factor fell by 38 percent and the allowed-to-billed ratio changed by 25 percent when NC6 data were used in place of NC5 data. In order to be comparable, the assumptions used in Blue Shield’s savings estimate of $4.7 million would need to be applied to the model using the same data that were used in our estimate. The results based on NC6 data are also not relevant because our report focuses on the information used by the CalPERS board in its decision to pursue the exclusive provider network. Since the NC6 results were not available when the board made its decision, they are not relevant to the report. 8 Blue Shield describes incorrectly our discussion of the model- review actuary’s conclusions relating to the alternative analysis. Specifically, as stated on pages 18 and 19, the board directed CalPERS to proceed with a third-party review to resolve the differences between Blue Shield’s and a health system’s analyses. A primary factor in the differences was the data used in the analyses. The system’s analysis was prepared using Office of Statewide Health Planning and Development (OSHPD) data, while Blue Shield used its claim data. The model-review actuary concluded that it is inappropriate to use OSHPD data as the sole source for hospital comparisons because OSHPD data are not available to the public in sufficient detail to allow a credible hospital-specific, payor-specific cost analysis. Our report does not highlight or offer any discussion on whether or not the alternative analysis did or did not do any case mix or severity adjustments. 8866 California State Auditor Report 2004-123 California State Auditor Report 2004-123 8877 9 We disagree with Blue Shield’s statement that our opinion is overstated. CalPERS states that it arranged for actuarial consultants to validate Blue Shield’s analysis for excluding hospitals from its exclusive provider network for CalPERS members. Specifically, according to CalPERS, it used the actuarial services of Blue Shield, Milliman, and the model-review actuary in conducting its review of Blue Shield’s analysis. However, because Blue Shield developed the methodology and Milliman assisted with the development of certain portions of the methodology, we question their ability to render an independent conclusion concerning the accuracy of Blue Shield’s analysis. Moreover, although it had an opportunity to benefit from an independent third-party review, as we discuss on pages 16 through 22, CalPERS did not fully consider the findings and recommendations made by the model-review actuary prior to approving the exclusive provider network. Thus, without fully addressing all of the concerns raised by the model-review actuary, CalPERS had no independent assurance that Blue Shield’s analysis was accurate. To address Blue Shield’s concern, we modified the text on pages 4 and 21 to add the phrase “from an independent source.” 0 We disagree. As we state on pages 36 and 37, we found that in one cohort, Blue Shield inconsistently applied its general rule for excluding hospitals from its network. We concluded that while exceptions to Blue Shield’s general rule may have been warranted in the case where they were applied, these exceptions were not applied consistently in other cohorts. Blue Shield’s treatment in the cohort in question was inconsistent in two respects: It analyzed all hospitals in one system as a group rather than individually, and it used a different set of claims to perform the analysis. Although Blue Shield asserts that it analyzed other groups of hospitals in similar circumstances, as we state on page 36, it was unable to produce any documentation that would allow us to verify that the analysis had been performed or that the conclusions that were drawn were reasonable or correct. Further, in its response, Blue Shield claims that it “concluded that applying this approach universally would not change any individual hospital’s status.” As we discuss on page 37, our consultant reached a different conclusion for one hospital that requested to be evaluated by Blue Shield using a different set of claims data. Specifically, our consultant found that the hospital would most certainly have changed status from excluded to included if the different set of claims data had been used. 8866 California State Auditor Report 2004-123 California State Auditor Report 2004-123 8877 cc: Members of the Legislature Office of the Lieutenant Governor Milton Marks Commission on California State Government Organization and Economy Department of Finance Attorney General State Controller State Treasurer Legislative Analyst Senate Office of Research California Research Bureau Capitol Press 8888 California State Auditor Report 2004-123