Publications

THE CULTURAL REVOLUTION

4th Annual Acquisition Research Symposium
May 16-17, 2007
Monterey, CA

Download Article (PDF)

Authors

Nathan Chong, U.S. Army Medical Command, 2050 Worth Road, Suite 22, MCFA, Ft Sam Houston, TX 78234-6022, PH: 210-221-7938, eMail: nathan.chong@us.army.mil

Kenneth Sullivan, Marie Sullivan, Dean Kashiwagi, PhD, PE, Performance Based Studies Research Group, Arizona State University, PO Box 870204, Tempe, AZ 85287-0204, PH: 480-965-4273, eMail: Dean.Kashiwagi@asu.edu

Biographies of Presenters

Dean T. Kashiwagi
Director, Professor, Professional Engineer (in 13 states)
Arizona State University, Performance Based Studies Research Group (PBSRG)

Dr. Kashiwagi is a professor at Arizona State University's Del E Web School of Construction and the Director of the Performance Based Studies Research (PBSRG) since 1992. PBSRG is the worldwide leader in improving construction performance and efficiency. Kashiwagi has developed a "hands off" approach to managing contractors/vendors in the construction industry or any industry. His concept is contrary to traditional price-driven construction procurement. It forces the contractor/vendor to be accountable in other words minimizes risk for the facility owner. The technology has been tested over 450 times totally $521M in construction projects with a 98% success rate (on time, on budget, and high quality). This is one of the few documented processes that brings better value for the owner and maximizes the profit of the contractor. It is currently being tested in other professional areas outside of construction.

Nathan B. N. Chong
Chief, Facility Life Cycle Management Division
US Army Medical Command

Nathan has been involved with over $1Billion worth of Army medical and research renovation, and new construction projects around the world for the past 22 years. He has a M.S. in Environmental/Civil Engineering from George Washington University and a B.S.C.E. in Civil Engineering, Purdue University. Nathan is a license Professional Engineer in the State of Hawaii, Project Management Professional (PMP) Certified, and Acquisition Level II Certified - Facilities Engineering with the U.S. Department of Defense. Facilitate the implementation of Performance Information Procurement System (PIPS) within the Army Medical Facility acquisition system. His career objective is to improve the Army medical design and construction processes.

Abstract

The culture within the Federal Government Acquisitions is based on the Federal Acquisition Regulations (FAR) and its interpretation, often placing organizations/agencies in the cultural environment of the price based environment. In the healthcare system, clients depend on the qualifications and expertise of the design and construction team to meet their specific needs and requirements. The hiring criteria of these experts have been primarily based on low bid or relationships, and have continuously resulted in poor performance. The US Army Medical Command (MEDCOM) (contracting approximately $100M in medical renovation awards per year) partnered with the Performance Based Studies Research Group (PBSRG) at Arizona State University, to create and test an information environment to assist in alleviating some of its cultural inefficiencies. The developed information environment minimized the flow of information, forced the contractors to concentrate on value and the assumption of risk, and stimulated an atmosphere of accountability. Through the system, the client's internal bureaucratic resistance was minimized and without controlling the various contract/procurement processes, MEDCOM leadership has gained control of the performance of their infrastructure revitalization program by implementing a cultural environment of information.

Introduction

The Performance Based Studies Research Group (PBSRG), experts in the area of performance information, has used the Construction Industry Structure (CIS) diagram to define and compare the characteristics of the high competition environments: Price Based and Value Based (Figure 1). The best value environment focuses on securing the best value vendor for the owner, and transferring all project risk to the outsourced expert. It considers the vendor's past performance, ability to identify and minimize risk, preplanning foresight, and project knowledge. It requires the contractor to use their expertise to complete a project that fulfills the intent of the owner, and minimize controlled project risk at the beginning of the project. It forces accountability between all parties, and benefits vendors with foresight, experience, skill, and efficiency. It provides an environment that maximizes contractor profit, while minimizing owner resources.

In contrast, the price based environment focuses on using minimum standards to define the requirement of the contract, in order to ensure that the minimal requirement is met. Due to its concentration on price, it encourages the contractors and vendors to translate the minimum requirement to a maximum, in attempts to lower the quality of the delivered product to gain the competitive advantage. This penalizes the high performers who can see the prospective project from a visionary view, from beginning to end, and seeks to minimize change orders by pricing in items which will need to be done that are missing from the specifications. It discourages the use of expertise, and asks the contractor/vendor to price only what is written, ignoring the owner's intent. It promotes project dependency on change orders when the unspecified or client "unexpected" events occur during project execution. The price based environment gives inexperienced contractors the competitive advantage over experienced contractors, thus driving the experienced contractors to move from a position of minimizing risk to a position of ignoring project risk. This culture results in the contractor relying on the client to manage, direct, control, and inspect, and become reactive instead of proactive. This trend, which is clearly demonstrated in Figure 2, has the following ramifications:

1. Penalizes contractors who carefully preplan, understand the scope of the project, and price out the project.
2. Motivates contractors to take the low price at the last minute, not knowing whether they are meeting the specifications, and when receiving the award, further price shopping to ensure that they can meet the specifications for the lowest possible price.
3. Encourages all manufacturers to ensure that their products meet only the lowest possible quality to get the largest possible volume sales of contractors trying to get the lowest price.
4. Promotes the bypass of education and personnel training in the industry, leading to a critical shortage of trained personnel.
5. Leads to poor construction performance (not on time, not on budget, and not meeting the client's expectations).

Relative Analysis of the Two Environments

A relative analysis of the two environments (price based and best value) leads to the following deductive conclusions. Performance can only occur when risk is transferred to an entity that has the capabilities of minimizing the risk. The best value environment ensures high performance by transferring risk to the best value contractor who can verify past performance, send their best personnel, identify risk that they control and do not control, and develop a plan to minimize the risk that they do not control. The owner's resulting risk in this environment (assuming that the client did pick the best value - highest performer for the lowest price), is the interface or the seam between the client and the contractor, or in other words the risk that the contractor does not control (Figure 3). Alternatively, the price based environment passes risk to the contractor with the lowest price, without ensuring that they can minimize the risk; hence the poor performance of the construction industry.

When risk is transferred to a party unable to minimize the risk, the party must be managed, inspected, and controlled. In consequence, the owner's risk in the low price environment is the potential that the minimally trained, managed, and directed contractor / vendor may not do what they are directed to do (Figure 3). The price based environment has reflected this, in a heavy overhead for transaction costs relating to management, direction, control, inspection, and communications that would be eliminated if the client's process were more efficient. This has also translated to a higher requirement of people needed to maintain the system (due to inefficiency of the process). Accordingly, there is more confusion in the price based environment due to the management, decision making, unrealistic expectations, attempts to control others, use of leverage (making a party do free work or work that they are incapable of doing), and the lack of performance information of key individuals, contractors, and the client's personnel. Without simple, easy to understand measurements, that consider the vendor's capabilities, the price based environment is adversarial; where every participant, regardless of who they work for, protects themselves, before they protect the company they work for, or the client/user the construction is being built.

The above characteristics are supported by documentation of construction delivered by the price based environment (AGC, 2005; Butler, 2002; CII, 2005; Doree, 2004; Fitz-Gibbon et al., 2006; Guo 2006; Markus, 1997; NDU, 2005; Post, 1998; State of Hawaii, 2002). The characteristics identify management as the key component to the price based environment, and leadership, or the alignment of resources that can minimize risk, as the key component to the best value environment. The understanding of the Construction Industry Structure (CIS) and the impact of the price based bidding identifies the following leadership/management characteristics as well as the projected goal of personnel who are trying to move from the price based environment (management and control) to the best value environment (leadership):

Management

1. Focus on relationships
2. Lack of performance information on critical elements of the contractor's team (relative ability to finish on time, on budget, and meet the client's expectations)
3. Direction/Control by client's personnel
4. Decision making performed by client's personnel instead of contractor personnel
5. Maximum communication/documentation passed between client and contractors
6. Duplication of cost estimates quantities, and approvals
7. Passing of risk without regard to relative ability to minimize risk
8. The most important person in the process becomes the client's procurement agent although they have no technical expertise, and take no responsibility for cost or time overruns

Leadership

1. Performance information used to minimize decision making
2. Transfer of risk to those that can minimize risk
3. Replacement of the client management, direction, and inspection with contractor/vendor self documentation/regulation
4. Minimization of client decision making, documentation, and flow of information
5. Process installed to ensure that the best value contractors/vendors know how they will minimize risk
6. Contractors/vendors address performance and risk in terms of value chain, supply chain, and overall transaction costs

Information Measurement Theory and Best Value Test Results

PBSRG has been testing best value procurement using an information based Performance Information Procurement System (PIPS). PIPS is built on a foundation of principles outlined by the Information Measurement Theory (IMT). IMT is a set of deductive logic models which predict future results based on relative measurements. The following are the major concepts or models (Kashiwagi, 2004):

1. Decision making requires an individual to use their subjective bias and experience to solve a situation where they have insufficient information to predict an outcome.
2. Decision making brings risk.
3. Decision making is minimized when the decision maker has accurate information.
4. Dominant information is information that will minimize the need for decision making.
5. It is difficult for one organization/person to control the actions of another individual.
6. People and organizations are predictable with enough information.
7. Past performance and future capability to perform on unique events can be predicted.
8. Experienced personnel can identify future activities in an event before it happens. They can identify and prioritize risk and they will have a plan to minimize the risk before it happens.

The following test results and measurements of PIPS have validated the above concepts:

1. Duration of testing: 13 years.
2. Research Funding: $6.2M
3. Number of tests: 480
4. Construction volume/scope of tests: $500M
5. Largest projects: $100M City of Peoria Wastewater Treatment DB project (2007) and the $53M Olympic Village/University of Utah Housing Project (2001)
6. Performance of contractors in tests (on time, on budget with no contractor generated cost change orders, meeting client's expectations): 98%
7. Surprise factor of nonperformance: Less than 1%
8. Management effort of client's construction managers: minimized by 80 to 90% (University of Hawaii (2000) and University of Minnesota (2006), and the ability of project managers to deliver 10 times the amount of projects (State of Hawaii (1997-2001)
9. Awards: 2005 Corenet Global Innovation of the Year Award for testing at Harvard University, and the 2000 Tech Pono Award for the testing at the State of Hawaii
10. Clients in the Public Sector: FAA, US Army Medical Command, USCG, States of Washington, Wyoming, Utah, Georgia, Hawaii, and Missouri, City of Peoria, AZ, City of Miami Beach, and Universities (University of Hawaii, Arizona State University and University of Minnesota)
11. Clients in the Private Sector: General Dynamics, Raytheon, Schering Plough, United Airlines, Motorola, Honeywell, IBM, Boeing, Intel, and International Rectifier
12. Risks and Reason for Stopping PIPS: the champion/expert of Best Value/PIPS moves or retires, political change, someone in the organization feels threatened and stops process, organization is too inefficient, ineffective, and bureaucratic to make process work

Theoretical Concepts of Best Value

The Best Value/PIPS process (shown in Figure 4) is composed of three primary steps:

1. Selection Phase (Filter 1-4): Identification of the Best Value
2. Pre-Planning /Quality Control Phase (Filter 5): Forcing the best value to preplan and minimize risk that they control and do not control through a PIPS Quality Control Plan or Risk Plan and schedule
3. Risk Management Phase (Filter 6): Management of the construction project through risk minimization

The selection phase attempts to differentiate the performance and expertise of competing vendors. This is done through the collection of each contractor's past performance information (from key individuals as well as the general contractor and critical subcontractors), risk assessment/value added plan, and interview ratings. It is important to note that if the contractors cannot differentiate themselves through their past performance, identification of project risk out of their control, plans to minimize uncontrolled risks, value added options, and interview, there is nothing wrong with awarding the project based on the best price (as the contractors proved that they are all the same). The client should not make decisions to assist any contractor to become competitive. No contractor should be assisted by being given a second chance, redoing their cost estimate, or given information from other contractors that could possibly make them more competitive. In best value, every contractor is competitive, and every contractor has a chance to differentiate themselves without biased assistance from the client's representative.

The Best Value/PIPS process forces the best value contractor to take their price, risk assessment/value added plan and interview statements into the second phase of Preplanning/Quality Control. In the preplanning/PIPS Quality Control Phase, the contractor concentrates on minimizing the project risks. A schedule listing the major milestones in the project is developed. A QC plan is also compiled by the contractor which includes a list of risks out of the contractor's immediate control in conjunction with a detailed plan to minimize each risk. Technical risks are not included, as the contractor minimizes risks that they control by meeting the requirements of the specification. Only after the owner is satisfied with the pre-planning performed by the contractor will the contract be awarded.

Once the project has been initialized, the contractor enters the Risk Management Phase. Every week during project execution, the contractor is required to submit a Quality Assurance (QA) plan and Weekly Report to the owner. The QA plan is a checklist of the risks identified in the previous phase that ensures that each risk is being monitored and minimized according to the directives included in the QC plan. If the risk cannot be minimized according to client pre-approved QC efforts, the risk is reported on the weekly report along with unforeseen risks adversely impacting the schedule or budget. The client is then obliged to pay for additional time and effort (Figure 5). This process transfers the risk to the contractor, who then uses the mechanism of the QC plan to make all parties accountable thorough communication, coordination, and preplanning.

The selection process ensures the procurement of the best value contractor and the transfer of all project risk to the contractor. The QC plan and the weekly risk report then defend the high performing contractor by identifying risks out of the contractor's control and the contractor's limited abilities to impact the risk. Because it is reviewed by the client's representative, the information is usually very accurate. The QC plan, QA checklist, weekly report, and schedule, also help to regulate the contractor's work. All the elements are incorporated into the contractor's contract on award. At the end of the project, the contractor is rated by the owner, and the rating modifies the past performance rating of the contractor by 50% (Figure 6).

Information System

The owner is able to compile a group of individual contractor weekly reports (spreadsheets submitted to the client weekly) into a Director's Report (Figure 7) which can give a Facility/Construction director valuable measurements of risk/performance for the organization as a whole, as well as a prioritization of the risks. The Director's Report also allows for the comparison of contractors, project managers, project integrators, inspectors or design professionals involved in the projects. The report provides accurate, timely risk/performance information that disables bureaucracy and identifies where risk is being created. For the first time, it gives a director a simplistic information system with minimal maintenance that deters nonperformance by highlighting nonperformance quickly and accurately.

Implementation of Best Value/PIPS in the US Medical Command (MEDCOM)

In order to implement Best Value/PIPS into the US Medical Command (MEDCOM), the system was adjusted in order to assure compatibility with the FAR and AFARS. As a result, the following changes were made:

1. The pre-planning quality control period is performed during a preconstruction period after the award of the contract.
2. The technical and non-technical concerns of the client/user are given to all contractors.

The Best Value/PIPS process can be implemented on Design-Build, CM@Risk, Design-Bid-Build, IDIQ contracts, or on designers. The US Army Medical Command has implemented it on IDIQ contracts, and directed the contractors to use a best value process that includes preplanning, Quality Control that minimizes risk that the contractor does not control, weekly risk reporting, and client identification of contractor performance.

Once the IDIQ contractors had been educated in the Best Value process, they began to realize that it would assist them to be more efficient, make the client's/user's representatives accountable, and maximize their profits without charging more (win-win). The contractors began to realize that the process was a very successful enterprise model that used best business practices, motivated their personnel to improve, and measured their own performance. The majority of the contractors began requesting their personal performance measurements.

Federal Acquisition Regulation (FAR)

The intent of the Federal Acquisition Regulation (FAR) (2002) is to bring the government the "best value." In the delivery of construction, the FAR recommends sealed bids. However, in FAR 36.103b and 6.04(b)(1), it states that if the use of sealed bids cannot effectively deliver the best value, the request for proposal process using other criteria than price can be used. There is ample evidence that implies the delivery of construction cannot be treated as a commodity.

The term best value is mentioned 34 times, and the term "low bid" is mentioned 19 times in the FAR. Best Value means the "expected outcome of an acquisition that, in the estimation of the Government, provides the greatest possible benefit to the requirement (FAR 2.1)." The benefits include (FAR 102.2(b)):

1. Satisfy the customer in terms of cost, quality, and timeliness of the delivered product or service by, for example-

   • Maximizing the use of commercial products and services;
   • Using contractors who have a track record of successful past performance or who demonstrate a current superior ability to perform; and
   • Promoting competition;
2. Minimize administrative operating costs;
3. Conduct business with integrity, fairness, and openness

FAR 1.102.2-1 (b) states:

(b) Vision. All participants in the System are responsible for making acquisition decisions that deliver the best value product or service to the customer. Best value must be viewed from a broad perspective and is achieved by balancing the many competing interests in the System. The result is a system which works better and costs less.

FAR 1.102-2(b)(2) states:

(2) The System must provide uniformity where it contributes to efficiency or where fairness or predictability is essential. The System should also, however, encourage innovation, and local adaptation where uniformity is not essential.

FAR 1.102-4 (e) further states that:

(e) The FAR outlines procurement policies and procedures that are used by members of the Acquisition Team. If a policy or procedure, or a particular strategy or practice, is in the best interest of the Government and is not specifically addressed in the FAR, nor prohibited by law (statute or case law), Executive order or other regulation, Government members of the Team should not assume it is prohibited. Rather, absence of direction should be interpreted as permitting the Team to innovate and use sound business judgment that is otherwise consistent with law and within the limits of their authority. Contracting officers should take the lead in encouraging business process innovations and ensuring that business decisions are sound.

The FAR addresses the use of the PIPS filters of past performance (FAR 15.305(a)(2)), the risk assessment plan/value added plan (FAR 15-305(a)(3)(i)), and the interview (FAR 15.102). The FAR addresses prioritization of alternatives in 15.305 Proposal evaluation:

Proposal evaluation is an assessment of the proposal and the offeror's ability to perform the prospective contract successfully. An agency shall evaluate competitive proposals and then assess their relative qualities solely on the factors and subfactors specified in the solicitation. Evaluations may be conducted using any rating method or combination of methods, including color or adjectival ratings, numerical weights and ordinal rankings. The relative strengths, deficiencies, significant weaknesses, and risks supporting proposal evaluation shall be documented in the contract file.

However, the Army FARS has the following:

5115.304 -- Evaluation Factors and Significant Subfactors (iv) Must be qualitative. Numerical weighting (i.e., assigning points or percentages to evaluation factors and subfactors) is not an authorized method of expressing the relative importance of these factors and subfactors. Evaluation factors and subfactors must be definable in readily understood qualitative terms (i.e., adjectival, colors, or other indicators, but not numbers) and represent the key areas of importance to be considered in the source selection process. The direction of this subparagraph is not waivable, either on an individual or class basis, as an AFARS deviation.

Therefore, when selecting a contractor, AFARS Best Value process must not use either weights or the ratings on any of the evaluation factors must have qualitative ratings. The impact of this policy is the lack of transparency but also prevents protests due to the inability or difficulty to challenge a subjective, nontransparent system. The downside to this type of system is that it motivates owner representatives to make decisions instead of allowing the contractors to determine who gets the project based on a preset system that is very predictable. However, Best Value/PIPS can still be run using qualitative ratings on past performance, risk assessment/value added submittal, and interviews.

The major contribution of Best Value/ PIPS is in the Preplanning/QC phase and the Risk Management phase. These two phases can be written into the IDIQ specification. The phases provide the user with relevant information related to the contractor's performance. These components are not a procurement issue, but a client's requirement to ensure wise usage of their funding. This responsibility must be fulfilled and is periodically checked by auditors to ensure the funds are receiving the best value for the government.

Resistance to Change

The movement to Best Value is threatening to the status quo due to the following:

1. The contractor becomes the expert, the center of the universe, the most important component of the value chain, replacing the perch the procurement officer has in the low bid environment.
2. Best Value/ PIPS forces the government to release control to the outsourced vendor. It is difficult for procurement agents to release control.
3. If services are outsourced, and the process becomes more efficient and effective, the government will need less personnel to make it work. This threatens the procurement community, who may feel that their jobs are going away.
4. A change in culture brings fear to the government community. This fear is exhibited in many ways: resistance to change, not being open to logic/best practices, using the FAR and AFARS and interpreting new concepts as illegal, increasing transaction costs of other participants in the delivery chain, and not acting in the best interest of the government.

Resistance to using the Best Value/PIPS process has included:

1. Stating that Arizona State of University could not be used to replace the COE procurement process. This is a fear that is totally unjustified. ASU is a research/education group. ASU does not participate in the procurement selection.
2. Best Value/PIPS will only quantify contractors with past performance. This is incorrect. If a contractor has no past performance ratings, the client is instructed to rate them an average rating. This rating is called "I don't know." The contractor could be a very high performer based on the owner's past experience, however the client should not bias the system by making a decision and helping a contractor get in.
3. The past performance is only good past performance. In order to win a true best value RFP, the contractor must send their best people, and therefore their best past performance. In this case, the best past performance is indicative in the way the best value will operate. This minimizes client decision making, forces the competition among performers, and allows the transfer of risk to the best performer.
4. Risk assessment plans are general and ambiguous. As contractors get grounded into the system of identifying the risks that they do not control, the risk plans will get better. However, they can only add to the contractor's preplanning and thinking and do not act as a detriment.
5. Interviews should focus on technical matters. In order to rate contractors on technical matters, the government has to be the expert, make decisions, and therefore absorb the risk. This system has proved to be unsuccessful. Contractors who can answer concerns simply, in non-technical language, are high performers. Poor performers cannot simplify.
6. There is a lack of competitive range and discussions. The competitive range should be identified by the best values.
7. The scope of the project should not be published in terms of budgets. Best Value / PIPS encourages giving the budget to the contractors so that they may determine risk. However, giving the budget is not mandatory. Not giving the budget oftentimes helps the low bidders who gain the advantage with low price. High performers always minimize risk, and if any information is withheld, the risk increases and the high performers increase their prices.
8. Sharing of offerors risk plans with other contractors. This is not required.

Conclusion

Best Value/PIPS is a cultural revolution for the government. The process/structure has been well tested out over an extended period of time. Best Value/PIPS requires the procurement agents to release control, minimize decision making, minimize the creation of transaction costs, and change their thinking. Education is the key. The government needs to become more efficient and effective.

References

AGC (2005/2006) Associated General Contractors of America. Accessed on March 7, 2007, at http://www.agc.org

Butler, J. (2002) Construction Quality Stinks, Engineering News Record (ENR), 248 [10], pg. 99.

CII (2005/2006) Construction Industry Institute. Accessed on March 7, 2007, at http://www.construction-institute.org/scriptcontent/Index.cfm

Doree, A.G. (2004) Collusion in the Dutch construction industry: an industrial organization perspective. Building Research & Information (March-April, 2004), 32(2); Accessed on March 7, 2007, at http://www.tandf.co.uk/journals/

Federal Acquisition Regulations (FAR) (2002) Accessed on March 7, 2007 at http://www.acqnet.gov/far/current/html/Subpart 14_4.html

Fitz-Gibbon, Jorge and Golding, Bruce (2006) Low bidder for Tappan Zee repair has history of overruns, litigation. The Journal News. Accessed on March 7, 2007, at http://www.thejournalnews.com/apps/pbcs.dll/article?AID=/20060402/ NEWS02/604020373/1029/news13

Guo, Y. (2006). Cash Flow and Risk Analysis: Best Value vs. Low Bid. Master's Thesis, Arizona State University, 2006.

Kashiwagi, Dean T. (2004). Best Value Procurement: How to use Information System to Minimize Risk, Increase Performance, and Predict Future Success, Performance Based Studies Research Group, Arizona.

Markus, Edward (1997) Low bid alternatives. American City and County, Accessed on March 7, 2007, at http://americancityandcounty.com/mag/government_low_bid _alternatives/ index.html

NDU (2005) Government Goals and Rules: Industry studies. Accessed on March 7, 2007, at http://www.ndu.edu/icaf/industry/2000/construction/construction.htm

Post, Nadine M. (1998, May 11). Building Teams Get High Marks. Engineering News Record, 240 [19], pg. 32-39.)

State of Hawaii Office of Administrative Hearings, Department of Commerce and Consumer Affairs (2002, June 28) Hearings Officer's Findings of Fact, Conclusions of Law, and Decision.