CCC and Uc: An Evaluation of Best Value Procurement Pilot Programs

CCC and UC: An Evaluation of Best Value Procurement Pilot Programs MAC TAYLOR LEGISLATIVE ANALYST DECEMBER 21, 2017 Summary State Created Best Value Procurement Pilots in 2012. Chapter 708 of 2012 (Pavley) created pilots authorizing the California Community Colleges (CCC) and the University of California (UC) to use a best value (BV) approach when procuring goods and services. The BV pilots allowed CCC and UC to consider noncost factors—such as quality and experience—when selecting vendors, rather than having to select the lowest-cost bidder. Chapter 708 required community college districts and UC to develop BV policies and report information about contracts procured during the pilot period. It further required our office to evaluate the pilots and recommend to the Legislature whether to continue CCC’s and UC’s BV authority after the January 1, 2019 sunset. This reports fulfills that requirement. CCC Did Not Report—Recommend Extending Pilot and Clarifying Statute. Many CCC districts had considered noncost factors in the procurement of services prior to the pilot and continued these practices after the pilot began. Because CCC did not change its practices, it did not believe these practices constituted BV under the pilot. It therefore did not report any contract information as required by Chapter 708. It also cited other reasons for not reporting, such as overly burdensome reporting requirements. Without contract data, we could not assess CCC’s procurement practices. Nonetheless, we recognize BV can have benefits in certain instances. Accordingly, we recommend the Legislature extend the CCC pilot program and clarify statute to indicate that consideration of noncost factors in any procurement constitutes participation in the pilot. We also recommend the Legislature simplify reporting requirements and require CCC to develop systemwide BV policies to promote the use of best practices among districts. UC’s Use of BV Generally Reasonable—Recommend Making BV Authority Permanent. Somewhat similar to CCC, UC considered noncost factors in the procurement of goods and services before the pilot. Prior to the pilot, UC used an alternative procurement method called “Cost Per Quality Point” (CPQP). During the pilot period, UC continued to use CPQP, using it more frequently than BV. Specifically, CPQP accounted for 42 percent of reported contracts during the period, while BV accounted for 14 percent of such contracts. Though UC did not heavily rely on BV during the pilot period, we believe UC’s use of it was reasonable. Having BV authority generally provided UC the flexibility to select vendors that met its needs without greatly increasing up-front costs relative to the lowest-cost bidder. We also believe UC developed a reasonable set of BV policies, but some of its guidance to campuses could be improved. Accordingly, we recommend the Legislature make UC’s BV authority permanent but require UC to include additional BV best practices in its procurement policies. We also recommend the Legislature have UC phase out CPQP, which is a less common approach than BV, and one for which the effect of price on the final award decision is hard to understand. analysis full gutter AN LAO REPORT INTRODUCTION State Created Two Pilot Programs in 2012. directed our office to assess (1) the advantages and Chapter 708 of 2012 (SB 1280, Pavley) created disadvantages of the BV approach compared to the pilot programs authorizing the use of best value (BV) LRB approach, (2) the number and resolution of bid evaluation in the procurement of goods and services protests, (3) the BV policies developed by CCC districts at the California Community Colleges (CCC) and the and UC, and (4) the overall cost of contracts awarded University of California (UC). The CCC pilot program during the period. It further directed us to recommend applied to all 72 community college districts and the UC to the Legislature whether to continue CCC’s and UC’s pilot program applied to all 10 of its campuses, 5 medical BV authority. This report fulfills these program evaluation centers, and the UC Office of the President (UCOP). requirements. Under these BV pilot programs, CCC and UC can select We Analyzed Contract Data, Conducted vendors of goods and services based on an evaluation Interviews, and Reviewed Available Studies. To of cost as well as noncost factors—such as quality evaluate the pilot programs, we examined contract data and experience. Prior to Chapter 708, statute in most provided to our office. (As described later in our report, cases required CCC and UC to select vendors of goods UC provided contract data, but CCC did not). We also and services based solely on whether they offered the conducted interviews with procurement staff associated lowest cost, as long as they met certain other minimum with UC (UCOP and five UC campuses) as well as qualifications. This is typically known as the lowest CCC (four districts and CollegeBuys—an organization responsible bidder (LRB) approach to vendor selection. that handles systemwide procurements for CCC). Chapter 708 Required Program Evaluations Prior Additionally, we spoke to procurement staff at numerous to January 1, 2019 Sunset. Chapter 708 required other state agencies and departments to learn about CCC and UC to provide our office with information their procurement practices and use of BV. Finally, we about contracts awarded between 2013 and 2015. It reviewed various studies about procurement and BV. BACKGROUND In this section, we provide general background on solicit bids before selecting a vendor. Statute sets the the state’s use of two main approaches to procuring competitive bidding threshold at $50,000 for CCC. goods and services—LRB and BV. We then provide Pursuant to statute, CCC’s level is adjusted annually for background specific to CCC’s and UC’s procurement inflation and is currently at $88,300. Statute sets the of goods and services prior to the enactment of threshold for UC at $100,000. Below these competitive Chapter 708. (Construction services are treated bidding thresholds, entities are typically authorized to separately in statute and are not covered in this report.) negotiate with potential vendors and/or solicit bids on a less formal basis. For the remainder of this report, we State Procurement focus on contracts that must be competitively bid. Statute Requires Competitive Bidding When State Uses Two Main Approaches to Procure Contracts Exceed Certain Monetary Thresholds. Goods and Services. Figure 1 outlines these When procuring goods and services, the state seeks two approaches. Under one approach, LRB, the to promote fair and open competition that is free state issues a solicitation seeking bidders willing to from bias and favoritism. To this end, statute includes provide the requested goods or services. The state various requirements for the procurement of goods then verifies that the bidders meet the solicitation’s and services, particularly those of significant monetary minimum qualifications—specifically, that they are both value. Specifically, statute sets certain monetary responsible and responsive. In this context, responsible thresholds above which agencies generally must means the vendor has the capability to do the work use a competitive bidding process to advertise and in terms of such factors as financial resources and 2 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT experience. Responsive means the vendor’s bid meets procurement because it is thought to protect the public all the requirements identified in the solicitation. Vendors from the misuse of public funds (since the state pays are not rated on these requirements—they either do the lowest price offered by vendors) and guard against or do not meet them. Finally, the state awards the favoritism, fraud, and corruption (since the award contract to the qualified bidder offering the lowest price. determination is objectively based on who offered the The other main approach to competitive bidding is BV. lowest bid). Under the BV approach, the state issues a solicitation . . . But LRB Has Some Limitations. The LRB for proposals, typically referred to as a Request for approach works well in many instances. For example, Proposal (RFP). Similar to LRB, the state then verifies it is well-suited to situations where the features of the that the bidders are responsible and responsive. Unlike desired good or service are easy to specify and there LRB, the proposals are then evaluated and scored is little variation in observed quality across vendors. For based on cost as well as other criteria. The BV criteria example, procurement of routine goods purchased by must be specified in the RFP and can include factors the state—such as basic office supplies—is typically such as lifetime costs, use of sustainable materials or conducted most appropriately using LRB. However, practices, experience, timeliness, terms and conditions, LRB can sometimes be an inflexible approach when or economic benefits to the community. The bidder features or quality of a desired good or service are with the highest score (not necessarily the lowest bid) hard to define or the up-front price does not reflect the receives the contract. longer-term costs. These limitations have led some Most State Procurement of Goods and Services state agencies to request authority to consider other Historically Has Been Based on LRB . . . State factors in addition to price. agencies have typically selected vendors using the LRB State Has Granted Some Public Agencies BV approach. LRB has been the default approach in state Authority. Recognizing the limitations of the LRB Figure 1 Unlike Lowest Responsible Bidder, Best Value Entails Scoring Proposals Lowest Reject bidders Establish minimum Responsible that fail to meet Award contract to qualifications and minimum the lowest bidder. Bidder: solicit bids. qualifications. Score proposals by Establish minimum assigning points to qualifications, evaluation Reject bidders that Award contract to Best Value: nonprice and price criteria, and point system fail to meet minimum the bidder with the factors. (Lowest price for nonprice and price qualifications. most points. receives highest factors. Solicit proposals. number of price points.) www.lao.ca.gov 3 analysis full gutter AN LAO REPORT approach, the state in 1983 began allowing some of ways. For example, if a customer needs a supplier to entities to use the BV approach for some purchases develop a public relations or educational campaign, it (see Figure 2). Notably, the state has authorized the may wish to consider various creative approaches that Department of General Services to use this approach may be difficult to specify in advance. In these cases, to procure services for various state departments. BV allows customers (that is, the staff at the state Additionally, it also authorized the California Department department making the purchase) to ask more nuanced of Technology to use the BV approach for procurement questions of suppliers and to score their proposals of information technology (IT) goods and services based on criteria detailed in the RFPs. Additionally, (including telecommunications services) for various procurement staff we interviewed noted that BV has state departments. It further gave the California allowed them to avoid having to award contracts to State University (CSU) BV authority, which CSU poor-quality suppliers who meet minimum criteria and uses to procure goods and services. Finally, it has routinely understate their cost in order to be the low provided BV authority to certain other entities such as bidder. In these latter cases, BV potentially can yield municipal utilities and various regional and local transit, long-term state savings while avoiding the hassle of transportation, and highway districts. In some cases, hiring vendors unlikely to perform adequately. the authority includes both goods and services, while in . . . Although It Also Has Some Drawbacks. others it covers only services or only goods. State procurement staff note some drawbacks to BV State’s Experience to Date With BV Generally relative to LRB. First, the up-front cost of the contract Positive . . . According to procurement staff at various could be higher as the contract may not necessarily state departments, BV has provided much needed go to the lowest bidder. Second, procurement staff flexibility, particularly when it comes to complex and customers need training on how to carry out a BV purchases. While price remains a required consideration procurement successfully because the steps involved under BV, other factors are sometimes as important, in developing the RFP (including evaluation criteria) or more important, than price. The state’s experience and in reviewing and evaluating vendor proposals are has confirmed that BV can be a valuable tool in cases generally more complex than for LRB. Also due to their where it is difficult to define up front the quality or complexity, BV procurements are typically more time features of a good or service to be procured. This may consuming to conduct than LRB procurements. Finally, occur, for example, with complex services that require BV uses a more subjective evaluation process than LRB innovative solutions or that can be delivered in a variety to award contracts. Accordingly, a potentially greater Figure 2 Several Public Agencies in California Have Had Best Value (BV) Authority State Entity Competitive Bid Thresholda Used For . . . Statutory Authority California State University $50,000 Goods and services Chapter 219 of 2001 (AB 1719, Committee on Higher Education) Department of General Services $5,000 Services Chapter 1231 of 1983 (SB 129, Boatright) Department of Technology None Goods and services Chapter 1106 of 1993 (AB 1727, Polanco)b Sacramento Municipal $50,000 Goods and services Chapter 665 of 2001 (AB 793, Cox) Utility District Various transit, transportation, Varies (either $100,000 or Goods Chapter 814 of 2006 (SB 1687, Murray) and highway districtsc $150,000) Chapter 408 of 2009 (AB 116, Beall) Chapter 460 of 2009 (AB 644, Caballero) Chapter 220 of 2012 (SB 1068, Rubio) a If the cost of a good or service exceeds this threshold, it must be competitively bid. For the Department of Technology, virtually all contracts must be competitively bid. b Although the original authority to use a BV-like approach for information technology purchases existed prior to this law, Chapter 1106 clarified how the process could be used. c Includes Alameda-Contra Costa Transit District, Golden Gate Bridge Highway and Transportation District, Los Angeles County Metropolitan Transit Authority, Monterey-Salinas Transit District, Sacramento Regional Transit District, San Mateo County Transit District, Santa Clara Valley Transportation Authority, San Francisco Bay Area Rapid Transit District, and Golden Empire Transit District. 4 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT chance of bias exists, and thus additional steps need to services. Statute excludes these types of services from be followed to ensure that solicitations are fair and free the LRB requirement without specifying the alternative from bias. procurement approach (such as BV) to use. In practice, Many Other States Use BV. BV is a widely used CCC and UC have used an approach like BV, where practice in private industry and is commonly used factors beyond price are considered and scored. by public agencies in other states. According to a Some Community College Districts Considered 2016 survey by the National Association of State Noncost Factors in Procuring Certain Services. Procurement Officials, 41 of the 47 states that Prior to the BV pilot, some of the state’s 72 community responded (including the District of Columbia) reported college districts considered noncost factors to procure that their central procurement offices have BV authority. a variety of services. Some of these services appear to BV is also a common practice among public institutions be professional services, which, as described above, of higher education. In a 2009 survey conducted are excluded from the requirement to use LRB. It by the American Association of State Colleges appears that CCC also used an approach like BV to and Universities and the National Association of procure services that were not clearly professional Educational Procurement, 77 percent of the responding services, such as food and transportation services. procurement officers at public four-year universities and UC Also Considered Noncost Factors in Its public university systems (representing 37 states and Procurement Practices. Prior to the BV pilot, all UC the District of Columbia) reported that state law allows campuses and UCOP considered noncost factors in them to consider noncost factors in procurement. the procurement of a wide range of goods and services (not only professional services) through its development CCC and UC Procurement of an alternative procurement approach called Cost CCC and UC Did Not Have Specific BV Authority Per Quality Point (CPQP). As discussed in the nearby Prior to Chapter 708. Prior to Chapter 708, statute box, UC indicated it believes CPQP complied with the generally required CCC and UC to use the LRB method requirement that it use LRB. This approach, however, to procure goods and services. The exceptions to the allowed UC to evaluate and score quality criteria, a requirement to use LRB included procurement of IT process that goes beyond the typical definition of LRB goods and services and professional or other services and is more like BV. In the wider procurement industry, where special training is required—such as insurance, CPQP is a lesser-known approach, and UC campuses financial, economic, accounting, engineering, or legal indicated they often have to educate vendors on how Some UC Procurement of Goods and Services Based on CPQP UC Developed Alternative Procurement Approach. UC developed its own approach called Cost Per Quality Point (CPQP) before it had the authority to consider noncost factors under the best value (BV) pilot and when it was still required to select the lowest responsible bidder (LRB) for goods and many services. The CPQP approach to procurement establishes a set of “quality” evaluation criteria, which are described in the solicitation. Under CPQP, evaluators score every responsible bidder’s proposal on each criterion within a predetermined range of points. Unlike BV, a bidder’s proposed price is divided by the total number of quality points to reach a CPQP. (For example, if a bidder offers a price of $100,000 and receives 80 out of 100 quality points for factors such as experience, financial stability, and key personnel, then its CPQP would be $1,250.) The bidder with the lowest CPQP is awarded the contract. CPQP Arguably More Similar to BV Than LRB Process. UC maintains that CPQP conforms to the statutory LRB requirement because the award goes to the bidder with the “lowest cost per quality point.” Yet CPQP scoring systems can produce contract winners that differ from the result of a traditional LRB process. In our assessment, CPQP is more like BV than LRB because it scores noncost criteria and the award may not go to the LRB. www.lao.ca.gov 5 analysis full gutter AN LAO REPORT it works. In particular, the effect that price will have on the criteria described in the solicitation, (2) evaluation the final decision is especially difficult to understand. criteria conform to statute, and (3) bidders that do not Additionally, no industry standards exist for CPQP receive the contract award be notified in writing. because the process is largely unique to UC. CCC and UC Also Were Required to Report Chapter 708 Authorized CCC and UC to Use Certain Contract Information. As a condition of BV Through 2018. Chapter 708 established BV pilot participating in the BV pilot, Chapter 708 also required programs for CCC and UC for the procurement of CCC and UC to provide the following information to our goods and services. The main rationale for Chapter 708 office: was to provide the segments with additional flexibility • A copy of adopted BV policies. to purchase goods and services in a cost-effective • A list of contracts awarded during the period manner. Accordingly, Chapter 708 authorized CCC (above the competitive bid threshold) and whether and UC to use BV when they believe they could the contract was awarded using BV or LRB. attain long-term savings or other economic benefits. Additionally, statute specifies contract evaluation criteria • For each contract, (1) a brief description of the that can be considered, including total cost, operational good or service, (2) the name of the vendor cost, added value, quality, availability, supplier financial awarded the contract, (3) the total contract cost stability and experience, and other economic or for CCC and the total contract expenditures for environmental benefits. UC (referred to as the contract “volume”), and (4) a summary of any written bid protests and Chapter 708 Required CCC and UC to Establish how they were resolved. CCC was also required BV Policies. Chapter 708 directed CCC and UC to to provide the bid award announcement and the create BV policies prior to using their BV authority scored ratings for each bidder. under the pilot programs. Specifically, statute required the governing board of any CCC district deciding to use • For each BV contract, (1) information about the BV to adopt BV policies and required the UC Board of evaluation criteria, (2) the reasons the award Regents to adopt and publish systemwide BV policies. winner was selected, (3) whether there were The legislation required that, in adopting BV policies, any additional economic benefits beyond the CCC districts and UC were to focus on reducing their good or service itself, and (4) identification of any overall operating costs and supporting their strategic comparable previous contracts awarded using efforts to increase purchasing efficiencies and leverage the LRB method. (In identifying comparable their purchasing power. It also required them to contracts, CCC was required to provide more establish processes for bid protests and resolution detailed information than UC about the previous of protests. In addition, statute required that for both contracts, such as the solicitation materials, the CCC and UC (1) bidders only be evaluated based on bid award announcements, names of vendors selected, and the amount of the awards.) CCC PILOT PROGRAM In this section, we provide our findings and districts instead continued their previous procurement recommendations relating to the CCC pilot program. practices. For many districts, this meant considering noncost factors—just like BV—for the purchase of FINDINGS various types of services. Notably, it appears these purchases were not limited to professional services. In CCC Districts Continued Former Procurement a review of select RFPs on the websites of CollegeBuys Practices During Pilot Period. CCC reported that no and several college districts, we found districts districts participated in the BV pilot program. Based were using a BV-like approach to procure services on conversations with CCC districts and CollegeBuys, such as digital imaging services, food services, and 6 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT printing services. Staff at one district said it was their are 72 districts, many of which likely lack experience understanding that all services (not only professional or expertise about how to use the standard BV services) were exempt from the LRB requirement. It is procurement method. unclear whether any district used the BV-like approach to procure goods. The four districts to whom we spoke RECOMMENDATIONS said they limited consideration of noncost factors to the procurement of services and always used the LRB Below, we make three recommendations relating to method to procure goods. the CCC BV pilot. No Contract Information Provided for Program Extend CCC BV Pilot and Clarify Statute. Evaluation. As no district participated in the pilot Because we did not receive any information about CCC program, CCC did not report any contract information procurement contracts, we could not assess whether to our office. Lacking this information, we cannot CCC districts used BV appropriately or effectively. determine: how widespread the practice of using However, we recognize that BV procurement can noncost factors in procuring goods and services have notable benefits in some cases. Accordingly, was among districts, how long districts have been we recommend the Legislature extend BV authority employing this practice, or whether districts’ governing to CCC on a pilot basis (for example, extending it for boards adopted policies for considering noncost five additional years, from 2019 through 2023, with factors. Furthermore, given the lack of reported CCC required to report certain information to the state information, we were unable to assess the number of in 2022 and program evaluation required in 2023). In bid protests or evaluate overall costs of BV, as required addition, we recommend the Legislature clarify that by statute. consideration of noncost factors in any competitive procurement constitutes participation in the pilot. CCC Identified Three Primary Reasons for Not Retaining the pilot and clarifying statute would provide Reporting. CCC provided three primary reasons for districts additional time to develop BV policies while not reporting the statutorily required information to our providing the Legislature an opportunity to receive data office: on how CCC is using BV. Based upon data submitted • Since none of the districts reported changing over the next few years, the Legislature could assess procurement practices to use their new BV whether CCC is using BV in an appropriate and authority, they believed it was not necessary to cost-effective manner before deciding whether to make report contract information. that authority permanent. • Districts found the reporting requirements too Revise CCC Reporting Requirements. The cumbersome, particularly collecting information amount of information CCC was required to report about past comparable LRB contracts. (which was somewhat greater than that required of • Some districts lacked the staff, expertise, and UC) appears to have created a disincentive for districts data systems necessary to compile the required to participate in the pilot. Accordingly, we recommend information. revising reporting requirements in collaboration with CCC to ensure adequate information is available to Developing Systemwide BV Policies and Best conduct a basic analysis of districts’ use of BV while Practices Could Be Particularly Helpful for CCC reducing the burden of the reporting requirements. For Districts. Currently no systemwide policies govern example, districts could forgo reporting on previous community college districts’ use of BV. Given the comparable contracts and focus on providing basic relative complexity and potentially greater subjectivity information about each new contract. For each new of BV compared to LRB, we think having systemwide contract, districts could identify the procurement BV policies is particularly important. Should CCC method used and the price offered by each bidder. For use BV (or BV-like) procurement approaches in the BV contracts, districts also could list the evaluation future, a set of well-understood systemwide policies criteria and each bidder’s score. and best practices would help reduce the risk of bias Require CCC to Adopt Systemwide BV Policies. and ensure that each district follows a minimum set of We recommend the Legislature require CCC adopt a requirements. This is particularly important given there www.lao.ca.gov 7 analysis full gutter AN LAO REPORT set of systemwide BV policies prior to participating in particularly important given the number and diversity of an extended BV pilot program. These policies should districts, some of which have small procurement staff be based on best practices for BV procurement. A and limited BV expertise. set of systemwide policies reflecting best practices is UC PILOT PROGRAM In this section, we share our findings, assessment, large part a result of some campuses having been more and recommendations relating to the UC pilot program. accustomed to CPQP. We also compare and contrast UC’s use of BV, CPQP, UC Used BV and CPQP Primarily to Purchase and LRB procurement approaches. Services. About 80 percent of UC’s BV contracts and 70 percent of CPQP contracts were for services. By FINDINGS contrast, a much smaller share of LRB contracts— about 25 percent—were for services. We note that the UC Used BV for Relatively Few Purchases services procured using BV and CPQP tended to be During the Pilot Period. From 2013 through 2015, somewhat more complex than services procured using UC reported 339 contracts of $100,000 or more. LRB. For example, one campus used BV to procure As Figure 3 shows, UC used BV evaluation for the services of a firm to conduct a large-scale research 49 (or 14 percent) of these contracts. Nearly three survey but used LRB to procure beverage pouring times as many contracts (143 or 42 percent) were rights. awarded using UC’s older BV-like approach, CPQP. BV and CPQP Purchases Were Larger on The remaining contracts (147 or 43 percent) were Average Than LRB Purchases. BV and CPQP awarded using LRB. Based on our discussions with UC contracts tended to be much higher value contracts campuses, the low utilization of BV appears to be in than LRB contracts. Specifically, among the 339 contracts reported, the average expenditure per contract was Figure 3 $1.34 million. Notably, five contracts were particularly large—two BV Best Value Accounted for a Relatively Small Number of UC Contracts contracts (both for IT systems), two CPQP contracts (one for Number of Contracts (2013 Through 2015) dining services and one for solar 120 energy), and one LRB contract (for 107 general laboratory supplies). These Goods 98 100 particularly large contracts ranged Services from $17 million to $94 million. 80 Excluding these contracts, Figure 4 shows that the average expenditure 60 45 per BV contract ($855,000) and 40 40 40 CPQP contract ($929,000) were still more than twice the average spent 20 on LRB contracts ($382,000). 9 More Campuses Used CPQP Best Cost Per Lowest Than BV. As noted previously, both Value Quality Point Responsible approaches were used in large part Bidder to procure services and average spending per contract was similar 8 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT between the approaches. Seven Figure 4 campuses, one medical center, and UC Used Best Value and Cost Per Quality Point UCOP used BV. (Two campuses and For Higher Value Purchases four medical centers did not.) By Average Expendituresa (2013 Through 2015) comparison, all ten campuses and five medical centers used CPQP. $1,000,000 $929,000 (Only UCOP did not.) While some $855,000 campuses used a combination of 800,000 both approaches, others relied more heavily on one or the other. The 600,000 reporting period ended in 2015, and it is our understanding that some $382,000 400,000 campuses are now moving toward BV and away from CPQP. 200,000 ASSESSMENT Best Cost Per Lowest Value Quality Point Responsible Below, we assess UC’s BV Bidder policies and evaluation criteria, BV contract costs, and bid protests. a Calculations of average expenditures exclude five particularly large contracts, which ranged from $17 million to $94 million. The average of all remaining contracts was $680,000. Figure 5 summarizes the key points of our assessment in each of these areas. inform in writing the bidders not awarded contracts. The definition of BV evaluation criteria also includes Policies and Evaluation Criteria a reference to the statutory guidelines about the UC’s Procurement Policies, Which Include types of criteria that may be used. Furthermore, UC’s BV, Are Generally Reasonable . . . UC’s written policies include the following provisions that, while not procurement policies, including those that cover required by statute, are supported by studies on BV overall procurement as well as BV, are consistent with procurement: the requirements of Chapter 708. Specifically, the • For all procurement approaches, bidders may not overall procurement policy notes the ways in which correct an error in their proposal after submission procurement efforts should seek to reduce UC’s costs if it would give them a material advantage. and fulfill UC’s strategic sourcing goals. Additionally, • For BV and CPQP, proposals will typically be the policies on BV discuss the requirement to include scored by an evaluation team. evaluation criteria in the solicitation materials and Figure 5 Summary of Assessment of UC Best Value (BV) Pilot Policies and Evaluation Criteria • UC’s overall procurement policies and specific BV policies are generally reasonable. • UC’s policies should include some additional guidance about BV best practices to help campuses implement the approach. • UC generally selected BV vendors using a core set of reasonable evaluation criteria. Costs • Majority of BV contracts awarded to the lowest bidder. • Remainder of BV contracts cost more up front but likely have long-term savings or qualitative benefits. Bid Protests • Bid protests are rare. www.lao.ca.gov 9 analysis full gutter AN LAO REPORT • For BV and CPQP, the total number of points and the quality and effectiveness of the goods or services, points per criterion must be determined by the which might include a firm’s technical capabilities, evaluation team before opening any bids. (For BV, vendor experience, sustainability practices, and this includes the weight for price.) warranties. In addition, in several instances, campuses considered staffing capabilities and expertise. For . . . But Should Include Additional Guidance the contracts we reviewed, the criteria selected by About BV Implementation. Although we believe campuses generally appeared reasonable for the types UC’s policies are generally reasonable and consistent of goods and services being purchased. with Chapter 708 requirements, UC’s policies lack certain guidance about implementing the BV approach Costs that could be particularly beneficial to campuses. Majority of BV Contracts Awarded to Lowest Specifically, UC’s policies do not incorporate various Bidder. Based on additional information we requested best practices, such as the following: from UC about BV procurements, we found that • Separating bidders’ cost sheets from their two-thirds of the BV contracts (32 of 49) were awarded narrative proposals (to minimize biases that may to the lowest bidder. occur from reviewing the narrative with knowledge Remainder of BV Contracts Cost More Up Front. of the bidders’ prices). When UC did not select the lowest bidder in a BV • Conducting blind reviews when possible procurement, the contract award cost on average (removing the name of the bidder from the bidding 16 percent more than the lowest bid. Whether such materials). an additional up-front cost is warranted depends upon • Preventing evaluators from changing their scores the Legislature’s evaluation of the qualitative benefits or throwing scores out. and long-term fiscal effects of the contracts. In the case of these UC contracts, some qualitative benefits • Preventing evaluators from seeking outside likely were achieved. For example, it appears that information about a bidder or bidder’s proposal winning bidders scored higher than other bidders on (an exception is contacting references provided quality (such as account management, reporting, and by the bidder). warranty), technical capabilities (such as innovative • Providing all bidders with the answers to solutions and design), and previous experience. In questions asked by any single bidder about the some cases, these contracts also could result in solicitation. long-term savings. For example, UC may be able to • Including people from various departments (or avoid future costs in cases where the winning bidder campuses if the procurement is systemwide) offered a better service warranty than the lowest bidder. on the evaluation team to minimize personal, departmental, or campus biases. Bid Protests Though some campuses have instituted some of these Bid Protests Are Rare. One concern about BV best practices on their own, UC has not endorsed them is that bidders will be more likely to protest contract systemwide. Additional systemwide guidance in these award decisions given the more subjective nature of the areas would help ensure procurement staff across all evaluation. This is because protests typically happen the campuses conduct the BV process as fairly as when a bidder believes there is something improper possible. about the solicitation or award. Chapter 708 required BV Vendors Generally Selected Based on Core reporting and evaluation of bid protests during the pilot Set of Reasonable Evaluation Criteria. Based on to determine whether BV led to more protests than our review of contract information provided by UC, LRB. Per UC policy, protests must be made in writing the campuses that used BV generally appeared to within two calendar weeks of the solicitation (if the evaluate prospective suppliers’ proposals using criteria protest is about the RFP) or award announcement. UC that are consistent with the pilot’s statutory direction reported just five protests among the 339 contracts on the criteria that may be considered. For example, awarded during the pilot. While all of these protests in addition to cost, campuses commonly considered were associated with BV procurements, the rarity of 10 LEGISLATIVE ANALYST’S OFFICE analysis full gutter AN LAO REPORT protests makes it difficult to draw definitive conclusions Require UC to Develop More Comprehensive about which approach results in more bid protests. BV Policies Based on Best Practices. Prior Additionally, the five protests reported by UC do to making BV authority permanent for UC, we not raise major concerns. Based on the information recommend the Legislature require UC to develop provided, they appear to have been either Public a more comprehensive written set of policies that Records Act (PRA) requests or resolved amicably by reflect best practices for BV procurement. To promote campus staff. The staff to whom we spoke said that transparency in procurement, this revised policy should bidders will often make PRA requests as part of their be available on UC’s website for the vendor community competitive research—that is, to learn more about and public to access. The more comprehensive set competitors’ pricing—rather than to protest an award. of policies should provide campuses with additional implementation guidance to help ensure campuses use RECOMMENDATIONS BV as consistently, fairly, and impartially as possible. The expanded guidance should include some or Below, we make three recommendations relating to all of the best practices mentioned earlier, such as the UC BV pilot. conducting blind evaluations and sharing information Provide UC With Permanent BV Authority. Based about any questions asked during solicitation. on the results of the UC pilot, we recommend the Have UC Phase Out CPQP. We believe UC’s former Legislature provide UC with permanent BV authority for procurement practice that evaluated noncost factors— the procurement of goods and services. We believe that CPQP—is simply a form of BV that is no longer needed UC used BV in an appropriate manner. Importantly, BV if UC has permanent BV authority. Moreover, CPQP appears to give UC the flexibility to select vendors that is not an industry-recognized practice, such that it meet its needs. Moreover, UC often selected the lowest could be serving to discourage certain prospective bidder even when using the BV method. Finally, making bidders from competing for awards. It also is an inferior BV authority permanent would allow UC to work more approach, in that the effect of price on the final award easily with private vendors who are more accustomed desision is much harder to understand. For these to BV than CPQP. reasons, we recommend that in granting BV authority, the Legislature have UC phase out CPQP. CONCLUSION Different Conclusions Reached for the Two for UC. Though we think permanent authority might Pilot Programs. The state currently does not have someday be appropriate for CCC too, CCC to date consistent policies for how public agencies may has not provided data to the Legislature demonstrating procure goods and services. Whereas the state has that it knows how to use BV and would use the granted several public agencies permanent authority procurement method appropriately. For these reasons, to use BV for procuring goods and services, it granted we recommend the Legislature extend the duration such authority to CCC and UC only on a pilot basis. of the pilot for CCC. Once more contract data are Based upon our review of UC’s use of BV authority for available, the Legislature could revisit the issue of the procurement of goods and services, we have no whether to make BV authority permanent for CCC. major concerns with making this authority permanent www.lao.ca.gov 11 analysis full gutter AN LAO REPORT LAO PUBLICATIONS This report was prepared by Sonja Petek and reviewed by Helen Kerstein and Jennifer Kuhn. The Legislative Analyst’s Office (LAO) is a nonpartisan office that provides fiscal and policy information and advice to the Legislature. To request publications call (916) 445-4656. This report and others, as well as an e-mail subscription service, are available on the LAO’s website at www.lao.ca.gov. The LAO is located at 925 L Street, Suite 1000, Sacramento, CA 95814. 12 LEGISLATIVE ANALYST’S OFFICE