Prediction of Project Performance-Fluor [PDF]

Zitiervorschau

PREDICTION OF PROJECT PERFORMANCE DEVELOPMENT OF A CONCEPTUAL MODEL FOR PREDICTING FUTURE PERFORMANCE OF AN OG&C PROJECT IN EPC ENVIRONMENT.



Naresh K. Kaushik Delft University of technology



®



PREDICTION OF PROJECT PERFORMANCE

Development of a prediction model for predicting future performance of an O&C project in EPC environment Thesis report Public version

Naresh Kaushik Student Number – 4141555

Master of Science thesis System Engineering Policy Analysis and Management Faculty of Technology, Policy and Management

®

®

PROJECT DETAILS Author: Naresh K. Kaushik Student Number: 4141555 Email: [email protected] This report is for thesis graduation project for:

Study program: System, engineering policy analysis and management (SEPAM) Graduation section: System Engineering Faculty of Technology, Policy and management Delft University of technology

Graduation Date:

5th of April, 2013

This research is performed in collaboration with

FlUOR B.V, Haarlem Department: Project controls

Graduation committee:

Chair: Prof. dr. ir. Alexander Verbraeck Section: Systems Engineering, Faculty of Technology, Policy, and Management

First supervisor: Dr. Mamadou D. Seck Section: Systems Engineering, Faculty of Technology, Policy, and Management

Second supervisor: Dr. W.W. Veeneman Section: Policy, Organization, Law and Gaming Faculty of Technology, Policy, and Management

External supervisor: Robert V. Velzen E&C Global Leader for Project Controls/Estimating – Fluor Corporation

External supervisor: Erik J. Groeneweg Project controls manager – Fluor Corporation



2

®

PREFACE This thesis report on development of performance prediction model is the result of my graduation thesis for master program System engineering policy analysis and management at Delft University of technology. I performed this thesis research as graduation intern at Fluor Corporation at their Haarlem Office. The past 7 months of my master thesis has been a great learning experience academically, professionally and personally. The research topic turned to be quite complex and resulted into lot a large scope for research. However, I enjoyed every bit of this research. At the conclusion of my research, I convey my warm thanks to my supervisors at TU delft: Mamadou Seck, Wijnand veeneman and Alexander Verbraeck for their continuous support and encouragement. My special thanks to Mamadou seck for extra support in form of frequent meetings and discussion that help my research. I would like to thank Robert V. Velzen for providing me this opportunity to conduct this research at Fluor and his invaluable role as my supervisor. In addition, I would like to thank Erik for his continuous guidance and feedback on my research. Furthermore, I would like to thank everybody at Fluor Haarlem that contributed to my research in form of semi-structured interviews and informal discussions. Finally, I would like to thank Kees Berends, Professor Hans Bakker from shell and Ted Ong from Exxon for providing their useful insights during interviews. I hope you all enjoy reading the results Naresh Kaushik Delft, March 2013

3

®

EXECUTIVE SUMMARY The projects in oil and chemical (O&C) industry often experience problems during their execution, because of those problems, some of the project ends with large cost and schedule overruns. The poor performance of projects not only affects the strategic objective of project’s owner but also poses a dual threat to engineering and construction (E&C) companies. They negatively affect their profit margins and their business objectives. Given the strict budget constraints imposed by the present global economic situation, owners and stakeholders expect their projects to be delivered cost effectively and efficiently. Therefore, it is important for E&C companies to strive for improvement in their project management practices. The current thesis research is a step in direction to introduce a new concept for improvement in performance management practices. For that purpose, the research introduces “early detection of project problems” as the main instrument and uses the quantitative information from past project to develop a body of knowledge and first conceptual model to predict the future performance of projects at their early stages. The research is conducted in five phases, the first phase of the research explores O&C project and their performance management practices. Based on the gathered knowledge via literature study and available information, the main research question is formulated as “How can future problems and performance of a current O&C project be predicted at early stages using knowledge and experience from past projects in an EPC environment?” Thereafter, a series of sub questions were formulated aimed to answer the above-mentioned research question. The later part of the first phase developed a structured research approach and research methods. In the second part of the research, efforts were directed to find the so-called “early warnings” of problems. To identify the early warnings, two main sources were explored, literature and experts from O&C project industry. Each investigation into respective sources resulted into number of early warnings. Each identified early warning was evaluated on selection criteria with three selection parameters. After the careful evaluation, the following ten early warnings were selected. ID LES PTE COC NCO CCO FED PH PS CE

Early warning indicator Lack of understanding of project execution strategy among project team Project team lacks experience required for the project Conflicts between owner and E&C contractor Numbers of change orders Cost impact of changes Percentage of missing information in FEED package Growth in process man-hours Delay in process engineering Change in concurrency level between process and piping engineering

4

®

DPO Delay in issuance of purchase orders

The selected early warnings were carried to the third phase of the research, in which four detailed case studies were performed to have observatory evidence. The case studies in this phase consisted of four project with different performance levels. The difference in performance levels of case projects set the contrast in which the predictive capability of early warnings could be observed. The case study investigation found that there is a relationship between early warnings, project problems and project performance. After obtaining the observatory evidence, the fourth phase of the research adopted a purely quantitative approach and studied the behavior of early warnings in a relatively larger set of past projects. Subsequently correlation analysis was performed to find correlations between early warnings and final project outcomes (which collectively asses the project performance). The quantitative analysis did present interesting and encouraging results. The main results are mentioned as follows: I. Early warnings do behave differently in case of poor and good performance projects, few in terms of their absolute value and few in their incremental changes. II. Correlations do exist between EWI and project outcomes, however not all the EWI found to be correlated with all project outcomes. III. The EWI indicators does show a dynamic quantitative relationship with project outcomes over engineering duration of the project Using the results from quantitative analysis, an attempt is made in the last phase of this research for the development of prediction model, which can predict the future performance of projects. The results of pilot prediction model were analyzed and compared with forecasts made via traditional forecasting methods. The comparison of forecasts found that prediction model does make prediction that is more accurate. However, there are errors with-in prediction models. In addition, the external validation of model suggested limited reliability and accuracy of pilot model. The dataset used for quantitative analysis and building of prediction model is relatively small and limit the generalization of findings. Therefore, to have a more accurate prediction in good projects, a dataset is required which contains a balance of Successful and less than successful performance projects. Despite the smaller dataset, the findings and approaches presented in this research can be used to build a useful model and subsequently applied in O&C project industry. A set of insights and recommendations (short term and long term) has been made for Fluor to implement the findings of this research to develop an operational performance prediction system. The research possibly has following main contributions to scientific and industry. Contribution to scientific community I. A shift from reactive project management to proactive project management

5

®

II. A new and constructive role of past projects Contribution to O&C project industry I. An approach, which facilitate the early detection of future potential problems II. An approach to capitalize on past projects to improve project performance management Note: The confidentially apply to the part of attachments, therefore are not attached with this report.

6

®

TABLE OF CONTENTS 1

Introduction .....................................................................................................................12

2

Research description .......................................................................................................16 2.1 Summary ................................................................................................................16 2.2 Overview of oil & chemical project execution.......................................................16 2.3 Research problem ..................................................................................................21 2.4 Research questions ................................................................................................25 2.5 Research goals and deliverables ...........................................................................26 2.6 Relevance...............................................................................................................26

3

Research design ...............................................................................................................28 3.1 Summary ................................................................................................................28 3.2 Research scope ......................................................................................................28 3.3 Fundamental approach..........................................................................................28 3.4 Research methods ..................................................................................................32

4

Literature study................................................................................................................34 4.1 Summary ................................................................................................................34 4.2 Author affiliations..................................................................................................34 4.3 Concept of project success.....................................................................................35 4.4 Concept of early warnings.....................................................................................40 4.5 Conclusions and discussions .................................................................................45

5

Early warnings in projects...............................................................................................47 5.1 Summary ................................................................................................................47 5.2 identification of early warnings.............................................................................47 5.3 Selection criteria of early warnings ......................................................................48 5.4 Selection of Early warnings...................................................................................49 5.5 Early warnings from literature..............................................................................50 5.6 Early warnings from experts .................................................................................53 5.7 Early warning indicators.......................................................................................58 5.8 Discussion and conclusion ....................................................................................62

6 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 7

Case Studies.....................................................................................................................63 Summary ................................................................................................................63 Case study design ..................................................................................................63 Case study selection...............................................................................................64 Case 1 (Less than successful project) ....................................................................65 Case 2 (successful project) ....................................................................................68 Case 3 (successful project) ....................................................................................71 Case 4 (Less than successful project) ....................................................................73 Cross case analysis................................................................................................77 Discussion and conclusion ....................................................................................79 Quantitative analysis .......................................................................................................81

7

®

Summary 81 7.1 Analysis approach .................................................................................................81 7.2 Exploratory data analysis......................................................................................82 7.3 Quantitative analysis .............................................................................................85 7.4 Correlations over engineering duration ................................................................89 7.5 Discussion and conclusions...................................................................................93 8

Development of prediction model ....................................................................................97 8.1 Requirements and guidelines for performance prediction model..........................97 8.2 Selection of prediction Methodology .....................................................................98 8.3 Prediction model development approach ..............................................................99 8.4 Development of pilot Prediction model ...............................................................102 8.5 Model evaluation methods...................................................................................104 8.6 Analysis of predictions.........................................................................................105 8.7 External validation ..............................................................................................110 8.8 Final evaluation...................................................................................................114 8.9 Integration of project problems with prediction model .......................................115 8.10 Discussion and conclusion ..................................................................................117

9

Insights and recommendations for implementation......................................................119 9.1 Insights and recommendations ............................................................................119 9.2 Recommendations for implementation ................................................................120

10

Conclusions and reflections...........................................................................................125 10.1 Revisiting research questions ..............................................................................125 10.2 Answer to the main RESEARCH question ...........................................................129 10.3 Discussion on research goals and deliverables...................................................130 10.4 Contribution to scientific community...................................................................130 10.5 Contribution to O&C project industry ................................................................131 10.6 Final reflections...................................................................................................132 10.7 Future research opportunities .............................................................................134

11

References......................................................................................................................135

8

®

LIST OF FIGURES Figure 1 Success and failure of O&C projects........................................................................................... 13 Figure 2 Phases of OG&C projects............................................................................................................ 17 Figure 3: The generic control cycle............................................................................................................ 19 Figure 4: Conceptual procedure for controlling of projects ..................................................................... 21 Figure 5: Cost of reactive approach........................................................................................................... 23 Figure 6: Existing knowledge gaps............................................................................................................. 24 Figure 7: The wheel of science (Wallace, 1971) ........................................................................................ 29 Figure 8: Fundamental research approach................................................................................................ 31 Figure 9: Iron triangle of projects.............................................................................................................. 36 Figure 10: Project success criteria............................................................................................................. 37 Figure 11: 95 % engineering completion milestone .................................................................................. 39 Figure 12 Potential benefits of EWI ........................................................................................................... 45 Figure 13: Early warning selection criteria............................................................................................... 48 Figure 14 Classification of early warnings from literature by source ...................................................... 51 Figure 15 Early warnings from literature by sub-category ....................................................................... 52 Figure 16: Early warning from experts by sub-category........................................................................... 56 Figure 17 Early warning mentioned by numbers of experts ...................................................................... 57 Figure 18 Framework for mapping the relationship between early warnings, project problems, and project outcomes.......................................................................................................................................... 64 19-27 Confidential Figure 28: Quantitative analysis approach................................................................................................ 82 28-37 Confidential Figure 38 Significant correlations between EWI and project outcomes at each prediction moment...... 91 Figure 39 Significant correlations of project outcomes with EWI over engineering duration................. 92 Figure 40 : Step approach for development of prediction model ............................................................ 101 Figure 41 Predictive capability comparison of traditional method and developed prediction tool....... 105 Figure 42 : Errors in prediction of final TIC ........................................................................................... 108 Figure 43 Errors in prediction of ESI....................................................................................................... 108 Figure 44 Errors in prediction of MHI..................................................................................................... 109 Figure 45 Errors in prediction of MCI ..................................................................................................... 110 Figure 46: Model validation: prediction of TIC....................................................................................... 111 Figure 47 Model validation: prediction of MCI....................................................................................... 112 Figure 48: Model validation- prediction of ESI ....................................................................................... 113 Figure 49: Usability of prediction Model................................................................................................. 121

9

®

Figure 50 future project problems and project outcomes associated with NCO .................................... 128 Figure 51: synthesis of answer to main research question ...................................................................... 129 Figure 52: Data collection moments ........................................................................................................ 153

10

®

KEY ABBREVIATIONS O&C

Oil and Chemicals

EPC

Engineering, Procurement and Construction

E&C

Engineering and Construction

E&P

Energy and petroleum

BOD

Basis of Design

BDP

Basic Design Package

CII

Construction Industry Institute

FEED

Front End Engineering Design

IPA

Independent Project Analysis

PEP

Project Execution plan

EVM

Earned Value Management

EWI

Early Warning indicator

ESI

Engineering Schedule Index

TIC

Total Installed Cost

MCI

Mechanical Completion index

COP

Cost of Problems

11

®

1 INTRODUCTION The oil and chemical owner companies rely heavily on engineering and construction (E&C) companies to meet their strategic objectives such as building of a new assets, expansion and performance improvement of existing assets. Moreover, given the strict budget constraints imposed by the present global economic situation, owners and stakeholders expect their projects to be delivered cost effectively and efficiently. E&C companies are working hard to match these expectations by changing their project management methods, tools and the way they execute projects. However, there are sufficient examples of projects, where E&C companies face problems in meeting their as sold cost estimates, agreed upon schedules and desired quality requirements. The number of project, that fail to meet their stated objectives vary significantly per industry, mainly due to the difference in complexity, the industry’s market dynamics, the type of stakeholders and their influence levels. Many researchers investigated the reasons for poor performance of projects (Flyvbjerg & Bruzelius, 2003; Morris & Hough, 1987; Turner, 1999; Thamhain & Wilemon, 1986). For example, handbook of project-based management by Turner mentions several reasons for project’s poor performance such as poor project establishment in terms of priorities, bad initial planning, inefficient control procedures and many more (Turner, 1999). Flyvbjer and Bruzelius (2003) suggested that in projects decision-making, planning and management are typically multi-actor processes with conflicting interests and therefore, projects are often faced with mistrust, violation of good project governance practices, ambiguity and poor collective decision-making (Flyvbjerg & Bruzelius, 2003). The above-mentioned behaviors of stakeholders penetrate through the permeable boundaries of project plans and can lead a project to high cost and schedule overruns. In this respect, projects in the oil and chemical (O&C) industry are no exception. Although the performance of O&C projects seems to be better than that of civil or mining projects, there are still ample examples of poor performing projects. Mckenna, Wilczynski and Vandersee (2006) estimated that about 30-40 % of capital project in O&C industry suffer from a budget and/or schedule overrun larger than 10%. Figure 1 shows the result of a study conducted by Independent Project Analysis (IPA). The study includes 318 projects across the O&C industry. Out of those projects, only 50% can be categorized as successful. The other 50% incurred either 33% cost overrun and/or schedule overrun of more than 30% (Merrow, 2012). Two third of the projects even failed to meet the production schedule or targets, thus affecting the profitability of its investors. The above results definitely are of serious concern for both the E&C and the owner companies.

12

®

Figure 1 Success and failure of O&C projects (Source: IPA, 2012)

From the above discussion, it can be concluded that O&C projects had experienced problems in the past and might encounter problems and challenges in future. There have been multiple attempts by the academic as well as industry experts to explore potential areas of improvement such as risk management, stakeholder management, benchmarking practices, and project control practices to improve the situation. Accordingly, there have been achievements such as development of value improvement practices (VIP), industry’s best practices, and front end loading (FEL) to mention a few. However, the majority of the research has been focused in the frond end phase of the projects. Prominently, the importance of the FEED phase for improving project performance is suggested over the years (Artto, Lehtonen, & Saranen, 2000; Thamhain & Wilemon, 1986) and little focus has been given to the execution phase of the project, where the problems actually surface and affect the project performance. The control mechanism of project execution phase (EPC) has seen little advancement and is still relying on the principles defined in 50-60’s such as principle of “deviation management” (Vanhoucke, 2011; Nikander, 2002). Why the “deviation based” traditional control mechanisms would not be suitable for successful control of project execution? There are two major problems with traditional deviation based control methods. First, the deviations are reported on aggregated level therefore, the poor performance in one part of the project is masked by good performance in other part of the project (Vanhoucke, 2011; Nikander, 2002). Secondly, even if the localized deviations are observed, they are seen in limited manner. The cascading effect of localized deviations on other activities is neither reported nor anticipated by these methods. Therefore, the accuracy of future forecasts of project performance based on the deviations is somewhat debatable. As a consequence to above mentioned fallacies in deviation management principles, often problems in a project are not visible until they are already manifested and degraded the project performance. The corrective strategy to manifested problems can be termed as

13

®

reactive approach, as the mangers act to correct what has already gone wrong. This reactive approach brings additional schedule requirements and incurs substantial cost, thus add into the cost and schedule overrun of the projects. In addition, forecasting of future project performance based on traditional methods is vulnerable to optimism bias. The mangers are often seen as very optimistic against the localized deviations and do not consider them as potential risk to future activities. Interestingly, forecasters never mention optimism bias as a main cause of inaccurate forecasts. Then the question arises, where to look then for the improvement in the project control management? A guided investigation of poor and good performance of past O&C project could provide us an answer to this question. If problems could not be eliminated from projects, can we predict problems, allowing longer correction time at lesser cost? This capability will allow for their pro-active management with considerably less cost and schedule impact. The aim of this thesis is to take a first step in creating a scientific understanding of prediction of problems via early warnings. Using this understanding, an attempt is made within this research to build a quantitative model to predict the future performance of project based on selected identified early warnings. Looking at the different chapters that build this thesis, chapter 2 provides the background for conducting this research by defining important concepts and delineating the main research problem. The problem delineation guides the formulation of research questions. Furthermore, research goals are introduced, the relevance of these goals is explained and the main deliverables are defined. In chapter 3, the design of this research is presented. The fundamental approach is described with logical sequence of research phases. Subsequently, the employed research methods and tools are explained and coupled with the goals set in chapter 2. A literature study regarding project performance of O&C projects is provided in chapter 4. The adopted measures of O&C project performance are presented. In this literature study, the concept of early warnings is explored and relevant literature is reviewed. The chapter also highlights the potential benefits of operationalizing early warnings in projects. Chapter 5 describes the identification of the early warnings from literature and from experts from O&C project industry. Furthermore, the selection criteria for selecting key early warnings are formulated, by focusing on the main objective of research. Each early warning is evaluated on selection criteria and few were selected for further analysis. In chapter 6, in-depth case studies are performed with an objective to have the preliminary evidence of relation between early warning, problems and their relation with project performance. In addition, Individual case conclusion and cross case analysis is performed and presented. In chapter 7, quantitative analysis is performed to 1) analyze the dynamic behavior of EWI over engineering duration of the project to map the behavior with successful or less than

14

®

successful projects. 2) Correlation analysis of EWI with specific project outcomes and 3) Longitudinal correlation analysis to find suitable EWIs for development of prediction model at each prediction moment In chapter 8, based on the past project data, early warnings are assigned quantitative indicators and an effort is been made to build a performance prediction model. The results of developed pilot model are analyzed and external validation is performed. Insights and general recommendations are provided in chapter 9. In addition, a short term and long term implementation strategy is presented in chapter 9. Finally, chapter 10 concludes the research by revisiting the research questions and their answers and evaluating the contribution to scientific and O&C project industry. Reflections have been made towards research approach, adopted methods, and results of the research.

15

®

2 RESEARCH DESCRIPTION 2.1 SUMMARY The main objective of the current chapter is to understand the research problem, its context and the research questions, which needs to be answered in order to find a solution to the main research problem. In addition, the scientific and social relevance of research had been provided. This objective has been achieved sequentially by understanding the 1) execution of O&C projects 2) their controlling mechanisms. With the understanding of context, a critical review of current practices enabled the delineation of research problem and existed knowledge gaps. The problem delineation helped in forming the main research question. Furthermore, the main research question has been broken into sub questions that need to be answered to obtain the solution to main research problem. The section 2.3.1 provides the overview of oil and chemical projects. Section 2.2.2 provides information on the subject of controlling mechanism of projects. Section 2.2.3, integrates the above two sections and shift the attention specifically on current project controlling mechanism employed in O&C projects. Section 2.3 provides a critical overview of the current controlling mechanism and describes the research problem. Based on the defined research problem, research questions are formulated in section 2.4. Section 2.5 describes the research goals and main research deliverables followed by relevance of research in scientific, social and business domains (section 2.6).

2.2 OVERVIEW OF OIL & CHEMICAL PROJECT EXECUTION 2.2.1

Oil & Chemical projects

O&C plants are also addressed as process plants, mainly due the fact that they have chemical processes at their heart. The chemical process convert the input (Crude oil, chemicals) into other chemicals with higher economic value (Fuels, industrial chemicals) by means of mechanical equipments, auxiliary facilities and the infrastructure to support the whole plant. The process plants are strategic assets of major petrochemical companies and are fundamental to their business. Furthermore, the O&C chemical projects should not be seen just as economical assets, they do contribute significantly in meeting the rising demands for energy of society as a whole. Although an increase in production of renewable energy is expected, experts still believe that the O&C industry will play an important role as energy producer, at least in near future.

16

®

O&C projects are capital intensive and do require systematic economic and project planning to deliver their intended results. Therefore, almost every (O&C) project is executed in systematic phases and its project life cycle encompasses the total time between identification of the project need to its completion. The different phases in project life cycle are (sub) projects in themselves and are separated by gates or decision points. The gated project lifecycle means that at certain points in the life cycle of project, the evolving design or plant concept and associated parameters (e.g. cost, schedule, and environmental impact) must pass through certain decision/review gates. The gated process allows for the evaluation of options based on the intended objectives of its stakeholders and consequently the selection of optimal option. In this sense, the gated process for a project allows for the structured way of decision-making. In addition, due to the comprehensive reviews, the project stakeholders are more informed about the deficiencies and/or risks in the project at a certain gate. The figure 2 shows the stage gated project life cycle of a typical O&C project from scope definition phase to its completion. Harpum, in his article in book titled: “The Wiley guide to managing projects” defined the basic rules for a project to pass through these gates (Harpum, 2004). However, the specific rules and passing requirements differ according to the individual company’s procedures and criticality of a project.

Figure 2 Phases of OG&C projects (Adapted from: The Wiley guide to project management, 2004)

17

®

Furthermore, in practice, the strictness of these gates also depends upon many other factors such as capital expenditure, urgency of project, contracting philosophy of owner to name a few. The following paragraphs describe each phase of an O&C project in brief manner. At scope definition level, the requirements of owner are identified and “what has to be done” is defined on a broader level. At the conceptual phase, basic functional characteristics of a project are described as a system in terms of input(s), throughput(s), outputs and major equipments required to achieve the desired production. In addition, the major interconnections between subsystems of a project are determined based on the process philosophy of the project (CII, 2004). Subsequently, the preliminary design is performed to provide basic design information i.e. process flow sheets, general design specifications, preliminary equipment specifications and their arrangements, preliminary plot plans, preliminary estimates and preliminary project execution strategy. In oil and chemical industry, conceptual and preliminary engineering phase together are called front-end engineering design (FEED) and a key deliverable at the end of FEED phase is the basic design package (BDP) (CII, 2004). In the detailed engineering design phase, the BDP is detailed further as engineering disciplines initiates detailed engineering in their respective domains. The main deliverables of this phase are technical, procurement and construction documents. Table 1 shows the main engineering disciplines typically involved in typical O&C project and their associated main deliverables. Table 1 Main engineering deliverables of detailed engineering phase Engineering disciplines

Key deliverables in detailed design

Process engineering

Process and instrument diagrams (P&ID), equipment and Instrument requirement list, control and relief valve specs

Mechanical

Equipment data sheets and equipment bid evaluation

Piping engineering

Plot plan, Piping design, stress calculations, Iso metrics and plant 3D model

Civil, structural Architectural

and

Electrical and control systems

Foundations drawings, Structural steel drawings Power system design, instrument data sheets, DCS specification

In procurement phase, the buying process is initiated based on the design specifications of equipment, instruments and materials. Later the contracts for civil works and installation of mechanical, electrical, instruments and piping materials are awarded.

18

®

During the construction phase, the facility is constructed according to the drawings and specifications prepared during detailed design phase using material and equipment obtained via procurement. In the start-up phase equipment are subjected to testing and inspection, both individually and in combination to validate the proper functioning of the facility. The phases detailed engineering, procurement and construction phase together are commonly known as EPC phase of project (CII, 2004).

2.2.2

Controlling of projects

“Controlling is the measurement and correction of performance in order to make sure that enterprise objectives and the plans devised to attain them are accomplished.” - Harold Koontz (1909-1984)

By definition, the control in project execution is exercised by measurement and comparing of “what was planned” with “what is being done” i.e. finding the deviation between the planned (known as baseline) and the actual. Figure 3: The generic control cycle Shows the generic control cycle employed in a project. The deviations could be caused by internal sub optimal performance and/or by influences from external environment penetrating the permeable boundaries of project. Fundamentally, control tries to make sure that the project stays on course to meet its predefined objectives and goals. By definition, good monitoring and control mechanism provides a better performance management over a project.

Figure 3: The generic control cycle (Source: Brandon, 2004)

In the control cycle, “What to measure” varies with the type of project and the perspective of the organization managing the project. The same is true for “how to measure.”

19

®

Corrective actions are management prerogatives that are available to project manager based on the type of organization and authorities of the project manager. “Taking action to improve the performance” refers to the corrective action necessary to bring deviation to a minimum level. Various examples of corrective action employed in project are fast tracking, adding additional resources, scope reduction, trade-offs, increasing risks and disciplinary actions and so on. Moreover, a specific corrective action is depending on the type of problem causing the deviation.

2.2.3

Controlling of O&C projects:

Having defined the control mechanisms, the project execution control of O&C projects could be seen in similar manner except the variables to be measured and tools could vary in accordance with O&C projects. The section 2.2.2 implies that for controlling, the first requirement is to establish a baseline against which we could measure the deviation and actual performance of project. To establish a project baseline for an O&C project, the following project information should be in available. I. Overall cost estimates (-10%/+20% variation) II. Work scope (refers to activities need to be accomplished to achieve the project objectives) III. Cost breakdown structure (Cost associated with activities i.e. services, equipments, overheads, contingency) IV. Project approved schedule (Milestones dates, activity durations) V. Comprehensive risk analysis along with accepted risks, planned mitigation strategies and actions VI. Commercial baseline: As sold pricing, time bound revenue and margins. The above documents act as basis for baseline developments. The final baselines for scope, schedule and cost are established along with control strategies and parameters to identify the deviations from the baseline.

20

®

Figure 4: Conceptual procedure for controlling of projects (Source: Fluor Corporation)

The Figure 4 above shows the applied concept of control cycle, specific to O&C projects. As soon as the project proceeds into detailed engineering execution, progress and performance are measured and monitored. In addition, the risks are monitored and dealt with during the course of execution. The progress in engineering, procurement and construction is monitored through earned value1 (EVM) concept with visualization via progress curves (cost progress and schedule progress). The primary instruments of project control are deviations between planned value of work (PV), earned value of the work performed (EV), actual cost (AC). Performance ratios are calculated at project level, phase level and discipline level, signifying the performance at respective levels. Based on the deviations and performance ratios, the required resources and cost for the balanced scope of work is forecasted along with incorporation of any strategy to recover the deviations (Vanhoucke, 2011). Along with cost and schedule performance ratios, multiple key performance indicators (KPI) such as safety performance, quality performance are monitored.

2.3 RESEARCH PROBLEM

1

Earned value management is a concept, in which progress is measured via integration of scope, cost and schedule. (For more information on EVM, please refer Christensen, 1998; Lipke et.al, 2009.)

21

®

2.3.1

Problem in current controlling practices

During project execution, projects are evaluated periodically using above described parameters such as earned value, performance ratios, KPI’s and variances from baseline. Such conventional methods are based on the principle of “deviation management.” At a certain moment in time, aggregated deviations reflect two aspects of project execution 1.) How much project is deviating from its baseline 2) given the deviations, how the project is performing i.e. performance of project? When aggregated deviations in the project are visible and are regarded as significant, it implies that there is/are problem(s) that has already manifested and degrading the project performance: the problem can no longer be avoided. After identifying deviations and nondesirable performance ratios, backward analysis is performed to search for the problems and strategies to manage the impacts of the problem(s). It should be noted that the deviations in project are mostly seen on aggregated level and the impacts of deviations within an area are often seen as limited that area, as their impact on total project performance is not clear. These localized problems become more critical if they have significant effect on downstream parts of the project. However, in current practices, these localized problems are not seen as problems but overlooked by aggregated performance of project might be still in acceptable limits. Furthermore, when localized problems develop into project problems, their delayed identification leads to additional cost. Figure 5 below explains this current problem more explicitly. The additional cost due to reactive approach called as “cost of reactive approach” which could be significant based on the nature of the problem and the timing of problem detection. Generally, this cost of reactive approach contributes significantly to cost overrun on projects. In addition, if the reporting of deviations is delayed due to any reason the cost to fix those problems will increase significantly, driving the project cost and schedule way off the baseline. The key to manage a project with predictability and certainty is to manage the problems before they affect the project outcomes. In other words, acting proactively based on the symptoms of problems (termed as early warnings) rather than reactively to the problems.

22

®

Figure 5: Cost of reactive approach

If these localized problems appear in the early phase of a project, given the interdependent nature of project activities in O&C projects it is almost certain that will have negative effect of downstream activities. However, the cascading effect of these problems at aggregated level performance reporting is likely delayed. Therefore, the future forecasts and project performance based on aggregated current performance is inaccurate. The above paragraphs clearly indicate that the current controlling and performance management practices lack the capability to detect the problems early enough and are always somewhat late. In addition, the forecast based on these traditional methods might not capture the change in dynamics of project due to localized problems. Thus, rather than minimizing the cost and schedule overrun in projects they add to it by providing inaccurate picture of project performance. However, if the localized problems can be measured as early warnings in projects and proactive management of these early warnings could minimize their impact and could significantly reduce the cost and schedule overrun in projects. In addition, having a more focused proactive approach can predict the future performance of project with more certainty, But how can E&C companies can achieve that is still to be discovered. Despite the vast body of literature covering the topic of project control and project performance, there is still no clear knowledge regarding early detection of problems and performance prediction based on the early warnings of problems (Vanhoucke, 2011; (Nikander & Eloranta, 2001). Most of the literature either focuses on quantifying deviations, diagnosis of deviation cause or corrective action decision making signifying a clear knowledge gap.

23

®

Diagrammatically, the existing knowledge gap can be presented by Figure 6 below.

Figure 6: Existing knowledge gaps

2.3.2

Under-utilization of past projects

Although in the domain of project management, each project is considered unique, certain types of projects are characterized by large amount of similarities, especially in types of deliverables, work sequence and procedures. For example, building of a O&C facility follows the same sequence of engineering design. Engineering deliverables flow through a logical path across these disciplines thus having standardized engineering work processes (CII, 2004; Fluor, 2012). Similarly, dependencies exist between engineering, procurement and construction. The above dependencies are not completely one directional rather a cycle of reviews and revisions occur before a design is finalized. As a result, the dependencies multiply the negative effects of a local problem and significantly influence the project performance. A detailed analysis of past projects can provide us with valuable quantitative and qualitative information regarding relationships between early warnings of problems and their impact on project performance. In the EPC phase of a project, the engineering cost is significantly lower compared to procurement and construction costs. However, actions and decisions during engineering influence the cost of a project much more than during procurement and construction, therefore detection of potential problems in the engineering phase can avoid large amount of rework/problems that ultimately cost/schedule overruns. However, at an O&C project industry, this knowledge is not captured in practice and only limited information is extracted from past projects. The current practices can be seen as under-utilization of past project information. Almost none of the available literature focus on capturing the actual quantitative relationship between early warnings and project performance. Rather they stress on capturing more qualitative information as lesson learned

24

®

or limited quantitative data for improving their benchmarking database (Barber, 2004; Williams, 2004). This fallacy in past project analysis is another identified knowledge gap, which this proposed research intends to fill. The fulfillment of above two identified knowledge gaps can be seen as complement to each other towards the development of performance prediction model.

2.4 RESEARCH QUESTIONS Having provided a background of the topic and description of the problem that the proposed research intends to tackle, the main research question is formulated as follows:

“How can future problems and performance of a current O&C project be predicted at early stages using knowledge and experience from past projects in an EPC environment?”

In order to find the answer to this main research question, it is necessary to proceed systematically through a series of sub questions. The first set of sub-questions will investigate performance assessment criteria employed in O&C projects and the concept early warnings of potential future problems.

RQ.1 What constitutes project success and what are performance assessment criteria of O&C projects? RQ.2 What do we understand by early warnings of project problems? The second set of sub questions will focus on identifying the early warnings in project execution in general followed by identifying early warnings that are specific to O&C projects. After having a set of early warnings the efforts will be directed to search the early warnings that can be used in an accurate performance prediction model. RQ.3 What early warnings can be identified in project execution? RQ.4 Which early warnings can be operationalized to build a performance prediction model. The third set will use the identified early warnings in RQ.4 and investigates their detection, problem prediction capability and their relation with project performance RQ.5 What are the dynamics between early warnings and project performance?

25

®

The final set of sub questions investigates the development of prediction model for predicting the probable future performance of O&C projects. RQ.6 What early warnings are indicative of deviation in project performance? RQ.7 How performance prediction model could predict the future performance of O&C projects?

2.5 RESEARCH GOALS AND DELIVERABLES Having described the research problem and main research question, this thesis ultimately aims to achieve following goals I. To provide a new scientific base for understanding and analyzing the early detection of project problems in capital O&C projects II. To present a new scientific approach which facilitates more constructive utilization of knowledge from past projects and exploring the power of prediction modeling for successful performance management of capital O&C projects In order to meet the above-mentioned goals, this thesis intent to deliver I. An overview of early warnings to predict future problems in projects derived from both academic literature and industry leaders, with observatory and quantitative evidence from real past projects. II. A methodology for analyzing early warning indicators in projects, their associated future project problems and project performance III. A conceptual performance prediction model systematically derived from quantitative information from past projects. Apart from this thesis, a set of recommendations will be presented along with conceptual performance prediction model to Fluor Corporation

2.6 RELEVANCE The relevance of the research results presented in this thesis is both scientific and social.

2.6.1

Scientific relevance

This thesis will contribute to scientific knowledge on project management, with a specific focus on project execution of O&C projects, by I. Exploring and gathering industry specific knowledge regarding early detection of future project problems

26

®

II. Providing a methodology to utilize past project information by pointing out the early warnings and their relations to project performance III. Exploring usefulness of performance prediction modeling techniques in project management The points mentioned above can act as a starting point for future research in project management, marking a shift from traditional methods of project control to more enhanced performance prediction. In addition, the content of this thesis will highlight the usefulness of past project data, beyond their current use as estimation and planning benchmarks.

2.6.2

Social relevance

The insight gained from this research can be used to improve controlling practices in O&C projects. The systematic process of early problem detection and development of prediction model will be most important contribution, which can be applied to other industries. The concept can be extended to other industry such as offshore facility development or civil infrastructure. More realistic predictions could lead to more proactive and informed decision making and ultimately to better project performance. In a world of projects, where the capital investments are high and efficient capital utilization is a prerequisite for development of new projects, an improved project performance can provide strategic certainty in capital planning.

27

®

3 RESEARCH DESIGN 3.1 SUMMARY As indicated in chapter 2, the main objective of this research project is to develop a model to predict the future performance of projects at early stages by capitalizing on the knowledge from past projects. The main instrument argued in chapter 2 for development of such capability is the early warnings of problems in projects. To direct the research efforts towards the achievement of the objective, a clear research approach has been designed and is presented in this chapter. The section 3.2 illustrates the scope of the research together with the argumentation for its selection. Subsequently, the chapter provides a blue print of the research’s fundamental approach (section 3.3). The section 3.4 aims to provide an overview of the research methods, tools and data collection methodology. The chapter concludes by discussing the possible limitations of the adopted research approach.

3.2 RESEARCH SCOPE To tackle the research problem efficiently, it is wise to limit the scope of research project around relevancy of existed knowledge gaps. The present research focuses on EPC phase of the project, which means phases between “start of detailed engineering” and “mechanical completion.” More specifically, the research is focused on project within O&C industry, which consists of either refineries or petrochemical processing plants and exclude offshore projects. In present research, the perspective of main engineering and construction (E&C) contractor is adopted, mainly due the fact that throughout the EPC phase of the project, E&C contractor is the main custodian and has primary responsibility to deliver the project as per agreed term and conditions. In addition, the present research has been conducted with significant support from Fluor Corporation, which is a renowned multinational E& company and main stakeholder in this research. The research is performed with in project controls department at Fluor Corporation at their Haarlem office. Fluor Corporation is one of the largest multinational E&C contractors and executed many O&C projects since its inception 100 years ago. The past projects executed by Fluor Corporation are the primary sources of industry’s project execution practices and past project data.

3.3 FUNDAMENTAL APPROACH In a research approach, two main methods of logic can be distinguished: deductive and inductive reasoning. These are described in a well-known “wheel of science” (Wallace, 1971). The starting point of the present research is deductive in nature; Theory of weak signals or early warnings is explored analogically in project management domain. This is done by exploring the relevant scientific and professional literature. To compensate for the

28

®

practical deficiencies in literature on early warnings, expert interviews are conducted to gain in depth knowledge of early warnings and project performance in O&C project execution. Based on the literature sources and expert interviews the main hypothesis is formed and defined as “early warning indicators have a (in) direct relationship with project performance. Consequently, the early warnings have the ability to predict project performance.” In the subsequent deductive phase, the hypothesis has been put to test via indepth case studies. Case studies based on past projects are performed to have observational evidence of the hypothesis. Subsequently, in the induction phase of the research, quantitative analysis is performed on a larger set of past projects to have an empirical evidence of prediction ability of early warning indicators. Based on the finding of quantitative analysis, a conceptual prediction model is build and has been validated. In the last part of the research, conclusions are formed based on results obtained from model and its validation. In the final section, recommendations are made for implementation of conceptual model and future research work.

Figure 7: The wheel of science (Wallace, 1971)

The fundamental approach has been illustrated in figure 8 with a detailed description in following paragraphs along with the research processes. The research methods are described in more detail in section 3.4. In the first phase of the research, the concept of project performance is explored and criteria for measurement of project performance are defined. The second part of this phase includes exploration of early warnings concept, its application in project management and its potential benefits during control of project execution. In the second phase of the research, relevant literature is explored and experts are interviewed to find out to what early warnings can be detected during execution of O&C projects. Semi-structured interviews are held with experts in the O&C industry. The majority of expert interviews are conducted within Fluor Corporation, along with some experts from owner companies (to get the perspective of project owners). The second phase is concluded by consolidating the early warnings from both literature and interviews,

29

®

followed by the selection of those early warnings that are used in further investigation. The employed selection criteria are based on main objective of the research project. Third phase of the research is focused on the in-depth explanatory case studies. The cases are selected from projects executed by Fluor in the past. Choosing different projects within one company reduces the variations in execution procedures of projects, as all the projects were executed with more or less same standard of project execution processes. The selected projects include both Successful and less than successful performance projects to set the contrast in which the differences can be visible. In observatory sense, this phase is used as a reality check of our hypothesis and at the same explained the relationship between early warnings, project problems and project outcomes (performed in subsequent sections). As a result, this phase has a more explanatory character. The fourth phase of the research is purely quantitative in nature and investigates the quantitative data from past projects with an objective to establish the predictive relationship between early warnings and project performance. The quantitative data from eight O&C projects is collected via available project documentation such as close out reports, project status reports and detailed monthly progress reports. The final phase of the research explores the methodology for building the prediction model and presents the model itself. In this phase, several quantitative prediction methods are presented and discussed, followed by selection of stepwise multi-regression as adopted method. The developed model has been evaluated with a new past project (different from projects those used to develop the model). The research is concluded at two levels, I. Presenting a set of recommendations and implementation strategy for Fluor Corporation to adopt the model in their project control processes II. Discussing the results of each phase and drawing conclusions from them and subsequently integrating the parts of research to provide answer the main research question.

30

Figure 8: Fundamental research approach

3.3.1 Limitations Having provided the detailed overview of research problem and adopted fundamental research approach. It is necessary to realize the limitations of research approach and methods. Quantitative analysis is highly depended on availability of data corresponding to early warnings. The early warnings, for which the past data is not available, will be excluded from quantitative analysis. This in turn, will affect the quality of research and subsequently, development of prediction model. Another identified and more critical limitation is that the past project data is very limited therefore could limit the accuracy and reliability of prediction model. Furthermore, the data is specific to Fluor Corporation. Thus, the data will likely be product of the standard and practices of Fluor rather than O&C industry as whole.

3.4 RESEARCH METHODS 3.4.1

Bibliographic and desk research

The proposed research project consists of an evaluation of the existing knowledge on the concepts of project management and primarily on early detection of problems in projects. Relevant literature from scientific and professional domains was studied. The main aim of this part is to understand the tools and procedures applied in management of O&C projects. Project performance and success are defined based on the academic, professional literature study and Fluor’s measurement standards. The concept of early warnings was defined by the study of available literature by academicians, professional organizations such as CII, IPA, and PMI along with expert interviews.

3.4.2

Expert interviews

Identifying early warnings relevant to O&C projects is an important task of the proposed research. For that, the concept of early warning is defined upfront. Experts from O&C industry were asked to provide potential early warnings based on their experience. The expert interview is selected as suitable method because there is little or no literature is available regarding early warnings, especially during the execution of O&C projects. Past project could be seen as potential source of selecting early warnings. Nevertheless, the time required for analysis of vast project data does not fit into the available timeframe, yet the past projects played as role for observatory evidence and provider of past data to build the prediction model. Interview base include experienced project directors, project managers and project control managers within Fluor Corporation and from some external owner companies. Interviewees were asked to provide potential early warning along with their possible measurement criteria. Furthermore, the interviewees were asked to provide additional information such as associated

®

future problems. A measurable quantitative attribute were attached to each identified early warning and will be termed as EWI.

3.4.3

Data collection

To obtain the understanding of relationship between early warnings and project performances, data had to be collected and analyzed. Fluor Corporation is the primary source of past project data. Due to the time constrain, date from past eight projects is used for quantitative analysis, However the each project will provide 8 data collection point, collected at 0% (baseline), 15%, 30%, 45%, 60%, 75% and 95% of actual engineering duration to normalize the project with different durations. The primary objective of collecting multiple data within one project is to understand the dynamic relationship between early warning and project performance and to develop a dynamic prediction model. Apart from quantitative analysis, part of past projects are studied as case studies understand the relationships and to differentiate between coincidences and causality of early warning and project performance.

3.4.4

Performance prediction model development

Exploratory data analysis was performed before establishing statistical relationship between identified early warnings and project performances. R Project for Statistical Computing will be used to perform the statistical analysis due to its capability of customization the statistical techniques and graphical outputs. Relationship of EWI with project outcomes was established through collection and analysis of past project data via stepwise multi regression. The conceptual prediction model was validated using past project data (which were not included in training) and current projects. The results of the validation test will be analyzed to form recommendations and conclusions.

33

®

4 LITERATURE STUDY 4.1 SUMMARY First step in collecting the available literature on the topic of project performance management and early warning is database research. Various search phrases were used to find the relevant literature. Google scholar was used as primary internet search tool. For all the relevant literature that could be identified, an attempt was made to get access. Further references of many sources were searched to get the more specific literature regarding O&C industry. The purpose of this chapter is to investigate how project management literature treats the detection of early warnings during project execution. The chapter first defines the project success and project performance, followed by adaption of project performance from current research perspective. The chapter then proceeds to define the concept of early warnings from theoretical perspective, followed by reflecting on their benefit in project execution control.

4.2 AUTHOR AFFILIATIONS When the preliminary literature research was performed, it seems logical to describe the affiliations of respective authors because the different affiliations are strongly related to the mental framework from which the literature was written. The different groups of authors, their interests, and assumption that might underlie their respective literature are presented below: Construction Industry Institute (CII): Established in 1983, the construction industry institute (CII) based at the University of Texas at Austin, is a consortium of over 100 owner, engineering-construction contractors and suppliers. Its aim is to improve the business effectiveness of its member organizations and cost effectiveness of capital projects through research, related initiatives and alliance among organizations. The research by CII focus on 14 knowledge areas of engineering and construction industry such as design optimization, project organization and planning and project controls are to name a few. Their primary focus of research in CII is the current practices employed by industries. For each knowledge area, CII identified best practices (methods or processes, which lead to enhance project performance), other practices (methods or processes that are not proven to enhance project performance) and research information (which are neither method nor processes). (CII, 2012) Consulting companies: Professional consulting companies such as Independent Project Analysis (IPA), schlumberger business consulting (SBC) have published their company’s perspective and experience on project management systems, project performance of engineering and construction projects. Especially, IPA focuses primarily on project development and execution through its project evaluation system (PES).

34

®

The professional consulting companies serve their customers globally thus representing their findings on projects all over the world, along with publishing region based reports. These consulting companies published mainly through their official publications. Academic researchers: Academic research group involves authors from academic institutions like technical and business universities, research schools and sponsored academic research by organizations. The focus of authors is to enhance theoretical and scientific knowledge base regarding overall project management and specific domains of project management. Their research explores the science and engineering to delineate unknown causes and potential solutions of practical problems faced by industry and to find their theoretical solutions. The most applicable findings are further explored and tested by industries before adopting them as practices. The present research investigates (but do not limit itself) academic publications relevant to early detection of problems in projects, problems in execution and their performance management. Limitations of the review: Terminology in project management is not uniform for early warning indicators of problems. Some describes them as leading indicators, symptoms, early warnings, problem causes. Moreover, many authors see actual problems as potential indicators of future problems. The diverse approaches and many implicit mentioning of early warning indicators could make the literature study a time consuming activity. Therefore, it is logical and necessary to adhere to the discussion of more relevant literature, which explicitly deals with early warnings in context of project execution. This approach will result in only a part of literature that could possibly be relevant and might bring the risk of leaving “block of literature” which might result in extra concepts.

4.3 CONCEPT OF PROJECT SUCCESS 4.3.1

Project management view:

Having a view on what O&C project are, what are their phases and their performance management, a natural question arises “what are successful projects,” in other words “How do we perceive success of a project.” Before answering this question, it is necessary to understand the concept of project success. The Figure 9 shows the best-known and most used representation of project success i.e. iron triangle with time, cost and scope (or performance/quality) on its corners (See e.g. Freeman & Beale, 1992, Larsen & Gobeli, 1989, Might & Fischer, 1985) and Oisen, 1971). From perspective of cost, time and scope, the green colored triangle is seen as a successful project, whereas the dotted red triangle can be termed as unsuccessful project, due to overrun in three dimensions of success. Although, this approach has been seen as too narrow and often criticized. See: (Atkinson, 1999), (Raz & Dvir, 2002) and (W.Hughes, Tippett, & Thomas, 2004) To widen the concept of project success, Morris and Hough defined three dimensions of project success (Moris & Hough, 1987):

35

®

I. Project functionality: to what extent does the project perform financially and or technically in the way expected by the project sponsors? II. Project management: how close is the implementation of the project to budget, schedule and technical specification? III. Contractors’ commercial performance: did the contractors have a commercial benefit in either short or long term?

Figure 9: Iron triangle of projects

The project success dimensions comprehend project success from different perspectives of; the customer, project execution contractors and sub contractor and other stakeholders. However, in reality the perspectives differ more than they look. A project could be delayed, but can be termed as a commercial success from client perspective given changes in his strategic financial goals. This indicates that whether a project is success depends largely on the perspective from which the project is viewed (Lientz & Rea, 1995). (Lim & Mohamed, 1999) addressed the differences in perspective of stakeholders and defined project success into two criteria: project completion criteria and satisfaction criteria. At macro perspective the criteria involves both the project completion and satisfaction criteria. On the other hand, the micro perspective only involves completion criteria. This is shown below in Figure 10 below.

36

®

Figure 10: Project success criteria (Source: Lim and Mohamed, 1999)

As mentioned in the scope of this research project, (see section 3.2) the present research adopted the perspective of E&C contractors as they have the key responsibility of EPC phase of project. In other words, the research is focused upon micro level criteria as defined by Lim and Mohamed (1999). However, it would be wrong to assume that the perspective of client and subcontractors are ignored, because to achieve the sustainable success, an engineering and construction contractor has to work collaboratively with its customer and suppliers by integrating their perception of success into its own to the possible extent. The success of a project is determined by evaluating its performance against success criteria (Wit, 1988), which implies that performance needs to be measured to determine the successfulness of a project. Another noteworthy point regarding project performances is that intermediate project performance varies with the time during project execution. A bad performing project could be turned around by making necessary strategic changes or a good performing project could turn into a poor performing project due to multiple reasons. However, final project performance is static and determines the success or failures of a project.

4.3.2

Project performance measurement in O&C projects

Having adopted the micro level success, the next step is to develop performance measurement criteria to measure the success. Menches and Hanna (2006) developed a performance measurement index with the following six project outcomes: I. Percentage budget overrun, II. Percentage schedule overrun, III. Actual percentage profit, IV. Change in work hours

37

®

V. Number of change orders VI. Communication between project team The change in work hours is more of a factor rather than an outcome and contributes to the final cost overrun of the project. Therefore, the use of numbers of change orders and change in work hours as project performance criteria could be debated (Atkinson, 1999), (Shenhar & Dvir, 1996) and (Hughes, Tippet, & Thomas, 2004). The actual percentage profit also seems to be contradictory with definition of project success, as it is highly dependent of the perspective of the stakeholder and type of contract. For example in reimbursable contracts, the percentage profit for E&C contractor may increase with scope and delay, whereas on contrary the project cost and schedule performance will decrease. With the adopted perspective of E&C contractors, it seems logical to limit and translate the measures of project success to following project outcomes. Knowing that this is very limited view on project success, yet they are the most commonly used across industry, therefore the availability of actual data for these indicators are higher than other indicators.

I) Mechanical completion schedule of plant Mechanical completion (MC) of plant is defined as “The checking and testing of equipment and construction to confirm that the installation is in accordance with drawings and specifications and ready for commissioning in a safe manner and incompliance with project requirements’ (Norwegian Technology Standards Institution, 2009). The scope of MC includes construction validation, testing of equipments (dynamic and static) and handover for start-up to owner. However, the testing phase could be excluded based on prior agreed upon scope between E&C contractor and owner (Fluor Corporation, 2012). MC can be seen as an important milestone from E&C contractor perspective as well as owner perspective. For the E&C contractor, incentives or liabilities are attached with MC milestone. Moreover, for an owner, completeness of MC marks as an indicator that plant is ready for startup. Delay in MC could negatively affect its production plans and prior agreements with buyers, in other words its revenue generation (Choi, Anderson, & Kim, 2006). II) 95 % engineering complete As explained in section 2.2.1, detailed engineering phase takes BOD as input (from FEED phase) and transform conceptual engineering into detailed engineering documents. It provides an input to procurement and construction. Although, the average engineering cost is only 20% of the project cost (CII, 2012), but it has significant influence on the rest of 80 % cost. In O&C project, the key deliverables of detailed engineering are as follows (Fluor Corporation, 2012): Input to procurement:

Input to construction:

- Equipment data sheets

- Process and instrument diagrams

38

®

- Instrument data sheets

- Plant plot plan

- Bulk material take-offs

- Civil foundation drawings

- Technical bid reviews

- Structure fabrication drawings - Pipe routing drawings (UG/AG) - Piping isometrics - Electrical single line diagrams - Installation procedure and manuals

The milestone for 95 % engineering complete signifies the completion of all major engineering activities including final issuance for key deliverables (Issue for construction). In other words, marks the completion of “E” phase of EPC project. The rest 5 % of engineering is designated to miscellaneous construction and start-up support, which could extend until completion of construction or MC (Fluor Corporation, 2012). Therefore, in industry practice 95% engineering completion is seen as finish of engineering efforts. Figure 11 shows the 95 % engineering milestone on EPC progress curves.

Figure 11: 95 % engineering completion milestone Adapted from Fluor Corporation, 2012

III) Total installed cost of project Total installed cost (TIC) by definition means that it is the total cost of installing a plant. TIC includes the cost of engineering efforts, cost of all equipments, materials and construction and other costs such as contingency, services fee, and escalation. The most cost effective project execution is the one allowing lowest TIC consistent with as sold estimates and owner requirements. TIC is an important project outcome for both E&C

39

®

contractor and the owner due to the simple fact that TIC is the important determinant factor in net present value (NPV) of a plant. In contractual terms, the dynamics of TIC on project economics of owner and E&C contractor can be illustrated by following model (Berends, 2007):

Where: P = Actual E&C profit Pt = Target E&C profit α = E&C sharing cost related profit; 0 ≤ α ≤ 1 (Based on contract type) Ct = Target/as sold TIC C = Actual TIC cost Cc = Owner contract cost From Equation 3, it is evident on higher level that growth in TIC (C) will shrink the profit margin for E&C contractor and at the same time will increase the cost for owner. The cost performance in terms of TIC as project outcome can be assessed as follows: IV) Engineering man-hours The amount of engineering man-hours in a project can be seen as an indicator of engineering efforts required in a project. Although from cost perspective, the cost of engineering efforts is quite small as compared to the cost of equipments and construction (on average varies between 10-15 % of TIC). In addition, the maximum engineering cost could be as high as 31% of TIC and as low as 8 % (Bakker, 2012). However, from project execution perspective engineering is the most important activity. As the engineering set the basis for equipment, purchase documents and construction drawings (see section 2.2.1). Any significant variation or a change in engineering man-hours has direct effect of procurement and construction activities. Therefore, from project performance perspective, a project has high chances of being a good performing project and successful project, if it consume more or less the same hours as estimated.

4.4 CONCEPT OF EARLY WARNINGS “The secret of all victories lies in the organization of non-obvious” -Marcus Aurelius (Roman emperor, AD 161-180)

40

®

Marcus Aurelius’s philosophy from days of Roman Empire suits well in the context of project execution management. However, a balance between obvious and non-obvious needs to be identified and monitored. In the present research, early warnings are the mixture of nonobvious and obvious, who’s monitoring and subsequent appropriate actions based on them could prevent manifestation of many problems. The scientific relation of early warnings of problems goes back to 1970’s theory of weak signals proposed by Professor Igor Ansoff. His primary aim was to improve the strategic planning methods in business environment. As per his proposition those methods did not work satisfactorily when faced with sudden changes or unanticipated problems. Moreover, the traditional strategic planning tools of trend monitoring and basing future planning on these were not efficient neither successful in time of turbulence. Further, he emphasized that the potential strategic surprises give advance information of themselves through weak signals (Ansoff, 1984). He defined weak signals as follows: “An imprecise early indicator about impending impactful events…all that is known is

that some threat or opportunity will undoubtedly arise, and their shape and nature and source are unknown (Ansoff, 1984; p. 22).” The Ansoff theory of weak signals was advanced by many researches in a wide range of areas such as communications research (Aberg, 1989), military science and research on international security (Betts, 1982; Herman, 1996) and predicting bankruptcies (Morris R. , 1997). However, the occurrence of weak signals in project environment was first addressed by Lientz and Rea; they suggested that the potential problems have prior symptoms, which are visible in the project environment (Lientz & Rea, 1995). They presented weak signals as “symptoms of problems.” However, these symptoms are implicit in nature and seem to be embedded in the project environment. Their identification relies heavily on the skills and proactiveness of project manager and team members. There are other literature sources, which presented implicit discussions about early warnings regardless of the term used in original text. Lewis presented a long list (10 pages) called “checklist for managing projects” (Lewis, 1993, Ch. 24). The absence of elements mentioned in this list could act is a cause of potential problems in projects, in other words those elements could act as potential indicators. Kerzner, Cleland and Honko (Kerzner, 1995; Cleland 1995, Honko, 1982) mention the similar lists or causes. The first comprehensive and explicit study of early warning phenomenon in project management was published by Nikander in the dissertation “Early warnings: A phenomenon in project management”. His research contrasted on the deficiencies of traditional project performance and control mechanisms for correct assessment of project performance status. He went further on the forecasting techniques and questioned their efficacy by highlighting their incapacity to accommodate possibly changing project circumstances. He proposed that the potential problems during project execution could be detected beforehand via their early warnings. The early detection of problems facilitates more informed decision-making and

41

®

provides an opportunity for the project manager to act proactively. He defined an early warning as: “An early warning is an observation, a signal, a message or some other item that is can

be seen as an expression, an indication, a proof, or a sign of the existence of some future or incipient positive or negative issue. It is a signal, omen, or indication of future development (Nikander, 2002; p. 49).” The research conducted by Nikander marks a stepping-stone in the direction of early detection of problems however, lacks the quantitative nature and ability to forecast project performance in light of early warnings. In addition, the majority of early warnings identified within the research composed of feeling and behavior of the project team and its stakeholders. Their detection largely depends upon the experience and intuition of project manager. Another relevant research titled “Leading indicators to project outcomes” was conducted by CII to identify the leading indicators (beyond the conventional methods or standard practices used to evaluate the status of projects) which may have a significant impact on project outcomes. The research defines leading indicators as: “Leading indicators are fundamental project characteristics and/or events that reflect or

predict project health. Reveled in timely manner, these indicators allow for proactive management to influence project outcomes” (CII, 2006; p. 20). The research had two objectives: identification of leading indicators and development of tool to assess the health of a project based on those leading indicators (CII, 2006). Unlike Lientz and Nikander, this study was more focused on industrial practices and relied heavily on the inputs from the owner and E&C companies. The study emphasized on leading indicators from industrial perspective and how they anticipate their impacts on project outcomes. However, the research is purely qualitative in nature and the respective tool utilizes user subjective information (based on perception) for reflecting on the status of a project. IPA also provided the O&C project industry with its services through its project evaluation system (PES). The underlying premise behind the PES is that a relationship exists between project drivers and the project's outcomes (IPA, 2012). The service by IPA is quantitative in nature and utilizes the vast database of past projects. The IPA performs the project audits primarily at FID and at project completion, which are the battery limits for project execution phase; the dynamics with in project execution is ignored. The IPA findings at FID could serve as early warnings for project execution i.e. if the project is not well defined and basis of design is not of good quality. It could act as first warning for the project manager and the deficient areas of project will most probably have problems during execution, if no remedial action is taken. IPA being a service provider, their activities is commercial in nature and only the outcomes and their illustration is known to the project owners.

42

®

In the present research, the larger view focuses primarily on the management of project performance with in project execution; therefore, audit key parameters from IPA could be used as early warnings for continuous monitoring.

4.4.1

Potential value addition of early warnings:

By definition, early warnings are indicators of future development. Following the reasoning of Nikander (2002), if early warnings of potential project problems are detected at early phases of project, many options are still available at the discretion of project manager. How early warnings can contribute to the cost effectiveness of project can be understood by following simple mathematical expressions:

Where: COP = Cost of problems in projects/unanticipated risks C = Actual cost of project Ct = Target cost Cm = Cost due to unavoidable events The cost of problems can be measured as the cost incurred to mitigate the negative impacts of manifested problems. In the context of present research, the COP can be seen in two different perspectives: COPR = Cost associated with reactive approach (cost of corrective strategy after problem has manifested) COPA = Cost associated with active approach (Early warning detected and corrective action taken) Applying the reasoning of cost influence curve, explained via Figure 5, it can be concluded that the cost of reactive approach is more significant as compared to the cost of proactive approach.

43

®

With the identification of early warnings in projects, we can minimize the COP to COPA and subsequently minimize the cost over-rum. In other words, EWI can help project in achieving following state:

Figure 12 Potential benefits of EWIillustrates the relation between early warning and project success more explicitly. As the project success is a function of project outcomes, undesired project outcome can be interpreted as unsuccessful projects. Figure 13(i) show a situation in which the potential problems was not anticipated well before and when it became visible. As a reaction to the existence of problem, a corrective strategy was formulated to contain the negative effects of problem, and subsequently implemented. The implementation of corrective strategy, required additional cost and schedule requirements, which were may not be anticipated while baseline finalization (COPR). Moreover, it might cause non-satisfaction among owners, project team or suppliers. Figure 13 (ii) shows a similar situation where the early warning indicated the presence of same potential problem in future course of project execution. The project manager has additional time to make a proactive decision, inform the stakeholders, and form preventive strategies and to implement the chosen strategy. The preventive strategy utilizes much less resources, thus keeping the cost and schedule close to desirable project outcomes (COPA). Furthermore, it vitalizes the sense of confidence and predictability in project manager and project team.

44

®

Figure 12 Potential benefits of EWI

4.5 CONCLUSIONS AND DISCUSSIONS The presented chapter aimed to explore the relevant literature and answer the first two research questions i.e. “What constitutes project success and what are performance assessment criteria of O&C projects?” and “What do we understand by early warnings of project problems?” it was observed that the project management literature sees project performance and success in a broader sense. Lim and Mohamed comprehend the different views at two levels micro and macro level. The present research adopted the project success criteria at micro level; the decision was taken mainly because of two reasons 1) The research aims at EPC phase of the project and micro level criteria (time cost and scope) is the most widely used criterion across E&C companies. 2) The present research relies heavily on the data from past project and availability of data relevant to micro level criteria was higher than other mentioned criteria. Based on the micro level criteria, following four project outcomes (PO) are defined to determine the project performance I. Mechanical completion of the project (Referred as MCI) II. Total installed cost of the project (Referred as TIC) III. 95% engineering completion schedule (Referred as ESI) IV. Engineering man-hour of the project (Referred as MHI)

45

®

The first two PO (MCI and TIC) signifies the final cost and schedule of the project and the rest two (ESI and MHI) can be seen as intermediate project outcomes, yet from an E&C contractor perspective they do represent an important aspect of projects. Review of the project management literature advocates the presence of similar notations to early warnings in projects and addressed as “symptoms of problems” leading indicators” and even as “checklist for managing projects.” However, there is a limitation of explicit pragmatic studies, which provides the evidence of early warnings engaged in management of projects. The study by Nikander titled “Early warning: phenomenon in project management” lacks the utilization capability in actual project work as the study focus on early warnings which are either purely qualitative or composed of feelings and behaviors of project team. Another research was conducted by CII titled “Leading indicators to project outcomes” identified the leading indicators which might influence the project performance. However, this research is also qualitative in nature and respective tool depends on user-subjective information. The methodology adopted by IPA is of limited use, as it only asses static project health at FID and project completion and does not provide active control mechanism during project execution. Section 4.4.1 illustrated the potential benefits of early warnings. The detection of early warnings and forecasting project performance based on them does provide valuable information and additional time to project manager to correct the problem before its manifestation. By the means of information and additional time, the reactive approach could be converted into active approach, thus reducing the cost and schedule overrun of the projects.

46

®

5

EARLY WARNINGS IN PROJECTS

5.1 SUMMARY The primary objective of this chapter is to find a set of early warnings in O&C projects, which could be operationalized to predict the project performance. To achieve this objective, the present chapter explores the early warnings from various sources (literature, experts) followed by assessment of identified early warnings based on selection criteria (formulated from present research perspective) to select fewer early warnings, which will to be carried forward for further investigation. The section 5.2 discuses the importance of measurability of early warnings and provides a formal definition of early earnings used in present research. Section 5.3 presents the early warning selection criteria with selection parameters and provides the argumentation for selecting respective parameters. Section 5.4 discusses the further classification and potential uses of early warnings after their evaluation on section parameters. Further section 5.5 and Section 5.6 provides the overview of early warnings found in literature sources and via expert interviews respectively. The comprehensive lists of early warnings are attached as appendix A. Section 5.7 presents the final list of early warnings as “EWI” after analyzing and rearranging the selected early warnings from respective sources.

5.2 IDENTIFICATION OF EARLY WARNINGS Policy and decision makers are heard often saying, “You can not achieve what you can not measure” and/or “What get measured gets managed.” This implies that if you are not able to formulate a goal/objective in terms of a set of quantitative indicators, then it has not been defined either in sufficient detail or still there is lack of census over its meaning. Bossel (1999) described indicators as “Indicators summarize complex information of value to the observer. They are our link to the world. They condense its enormous complexity to a manageable amount of meaningful information, to a small subset of observations informing our decisions and directing our actions (Bossel, 1999)” This is also true in case of early warnings, as they need to present in measurable terms, so that project manager or stakeholders can take decisions that are more informed. Therefore, the early warnings as indicators altogether should provide the decision makers with a sufficiently complete and accurate image of their status to enable them to decide whether proactive action is required or not. For the identification of early warnings and practical issues, following formal definition based on the aggregated academic and industrial research is determined. “Early warning indicators are inherited measurable project characteristics and/or observations that indicate towards the development of potential problems in future course of project execution.”

47

®

With reference to the type of information presented, early warnings may be separated into two categories: quantitative and qualitative. The former provides information in the form of hard data extracted from system state (i.e. numbers) and latter are based on the subjective data based on some one’s perception (i.e. ordinal numbers or qualitative judgments such as good, sufficient, bad). The information, which can be treated as early warning in projects is available in both forms i.e. qualitative (qualitative assessment) and quantitative (variations, percentage delays). However, there is another set of early warnings, which is expressed as feeling, inter-personnel behaviors and communications.

5.3 SELECTION CRITERIA OF EARLY WARNINGS The selection criteria are formulated while keeping the main objective of the research in focus, i.e. building of a prediction model. The collection and analysis process is explained and is shown in Figure 13.

Figure 13: Early warning selection criteria

Measurable: The early warnings that are measured quantitatively or at least have the potential to be measured Early in project - The definition of “early” for identification of early warnings is somewhat subjective. The screening of early warnings gives an indication that some of the early warnings could exist all along the life cycle of projects which include early phases of projects.

48

®

There is no reference found in literature that could define the early phase of an O&C project. Therefore, expert opinion was sought and based on the discussions; the period from start of detailed engineering, up to 2/32 model review was defined as early. This choice seems quite reasonable and can be argued based on design definition at 2/3 model review stage. Data availability – To establish the statistical relationship between early warnings and project performance, it is of utmost importance that we can find the data in past projects corresponding to selected early warnings. However, there might be few early warnings, which might not have quantitative data attached to them, yet the written sources of information in project reports could provide an opportunity to make an inference regarding their existence. Such qualitative early warnings will be accepted only for analysis during case studies.

5.4 SELECTION OF EARLY WARNINGS Having defined the selection criteria for early warnings, each individual early warning was assessed on the selection criteria. The primary objective was to select the early warnings, which can be carried over the next phase for further analysis and for building of the prediction model. The early warnings, which are termed as important3 and cannot be selected for further analysis, will be presented as “recommendations.” The rest of the early warnings will be presented as “for information.” As a result, the selection process delivered three types of early warnings Early warning as indicators (EWI): To be a EWI, a particular early warning should qualify on all three criteria. The EWI are further explored via case studies in chapter 6, and ultimately acted as variables in the building of the performance prediction model. There are few exceptions made for early warnings, which do not satisfy all three-selection criteria, but are accepted as early warnings in case studies mainly due importance given to them by both literature and interviewees. Early warnings as recommendations: The early warnings which match any two criteria and are either measurable or have potential to be measured. Apart from measurability, they should occur at early stage in project execution and can provide useful information for project management or future research. Early warning “for information”: This category also includes early warnings, which are either purely qualitative in nature and are not seen early enough in project. The main argument for their selection as “for information” is that from the analysis perspective, its quite difficult to quantify them and assign indicators, thought they can still provide useful information to project manager to his capacity as the leader on the project. The PM could identify these early

2

Define 2/3 Model review

3

Important here refers to the importance given by the sources i.e. multiple literature or interviewee mention the same early warnings (Frequency)

49

®

warning and can take necessary steps to make project environment more positive and constructive.

5.5 EARLY WARNINGS FROM LITERATURE As mentioned in section 4.4, project management literature does include few examples that can be interpreted as early warnings. However, many of the examples are quite implicit and difficult to operationalize for forecasting project performance. The deficiency emerged due to the limited research and lack of consistency regarding “what is an early warning” and “when it is identified.” Therefore, the effort was made to explore not only the academic database, but also the publications by professional organizations.

5.5.1

Academic literature

The academic literature discusses the early warnings from different viewpoints such as symptoms of problems, problems themselves, complexity factors or non-fulfillment of project standard procedures (Lientz, 95; Honko, 82; Cleland, 84; Kerzner-95; Bosch-rekveldt, 2011). From the analysis of early warnings found in this class, it could be readily observed that all of them contain a clear indication of their warning like character. Explicit or implicit, all the relevant quotes or elements are mentioned in appendix A.1 (L-1 to L-30).

5.5.2

Early warnings from CII

The CII study titled “Leading indicators to project outcome” investigated the leading indicators (LI) of potential problems in projects. Given the accepted definition in chapter 5, the mentioned LI probably can be seen as early warnings. The study concluded with 43 LI, which were presented after three levels of evaluations. All 43 LI were analyzed and after the preliminary screening, 15 top indicators were selected based on the importance (in terms of score) given to them by E&C contractors. The early warnings from CII are mentioned in appendix A.1 (L-31 to L-45)

5.5.3

Early warnings from IPA

As mentioned in section 4.4 IPA audit findings at completion of front-end design could serve as early warnings for project execution. IPA project audits are purely quantitative and could serve as excellent early warning indicators as per our adopted definition. IPA has defined its own metrics and project database to validate those metrics. Apart from rating the project based on its characteristics and metrics, they provide a benchmarked comparison of project with other more or less similar projects. In addition, IPA findings on post project evaluation could also provide key early variables, which makes difference between good performance and bad performance of projects and can be seen as early warnings. In the present research, the underlying principle of IPA PES is of interest, attempts could be made to find the relevant data from past project and conclusions could be drawn based on the findings. The early warnings from IPA PES are mentioned in appendix A.1 (L-45 to L-49)

50

®

5.5.4

Analysis of early warnings from literature

Based on the characterization and typology of early warnings found in literatures, it was observed that the majority of the early warnings are in qualitative in nature (21 No’s, 45 % of total). In addition, there were 17 no’s of quantitative early warnings including 4 no’s of traditional performance indicators which could be seen as early warnings. The classification of early warnings by literature source could be illustrated by following graph. Early warnings from literature 30 Feeling, behavior Quantitative 25

Qualitative

8

20 7 15

1

6

10 13 5

8 6

0 Acedemic Literature

CII

IPA

Figure 14 Classification of early warnings from literature by source

The early warnings can be further classified into sub categories. Most of the sub categories consist of various early warnings primarily related to that sub categories. Figure 16 shows the classification of early warnings into sub categories.

51

®

Early warnings by sub categories 30 Feeling, Behavior Quantitative

25

Qualitative 9

20 3 15

10 15 4

5

7

0

1

1 2

4 Project Team development & integration

Change mgmt

Early Engineering

Procurement

3 Concurrency in projects

Figure 15 Early warnings from literature by sub-category4

Note that the quantitative nature of early warning reflects the potential to measure and does not necessary mean that it is measured. For example, early warnings number L-35, which could be measured by monitoring the experience (Based on the grade), personnel need date of actually deployed personnel against the planned requirements. However, not every project or E&C Company measures the experience and skills of the project team quantitatively. There are 27 early warnings that belong to quality of project team and its integration, which in a sense reflects the importance of team in a project. The noteworthy point from above analysis is that majority of early warnings in category “team development and integration” (24 out of total 27) are either qualitative in nature or are feeling/behavior of team members. Based on the selection criteria, following early warnings are selected for further analysis Table 2: Early warnings from literature sources Criteria

4

ID

Early warning

L-1

Monitoring of actual allocation of resources against the plan can provide early warning of lack of work

Measura bility

√

Early in project

√

Data Availability

√

Remarks

The early warning is accepted for early discipline. Project level reporting may not reveal the actual picture due to aggregation factor

Here it should be noted that the quantitative nature of early warning reflects the potential to measure.

52

®

L-2

Noticeable change in performance level for a team or individual members reflects problems.

√

√

√

L-18

High rate of changes in part prompted by errors in assumptions and mistakes

√

√

√

√

√

√

Adapted as cost of changes and no’s of changes in project

√

√

√

Adapted as cost of changes and no’s of changes in project

√

Accepted due to the importance stated by experts and literature Data could be found as write-up in internal reports

L-30

L-32

L-34

Owner and contractor is requesting an excessive number of contract changes during project execution Project is experiencing a high level of engineering/design/ specifications changes The project team is lacking in the necessary expertise, experience, breath and depth to successfully execute the project

√

√

L-46

Poor front end definition

√

√

√

L-48

Poor team development

√

√

√

L-49

Major design changes at later stage in design

√

√

√

The early warning is accepted for early discipline. Project level reporting may not reveal the actual picture due to aggregation factor Segregation of change orders from past projects at required level of detail is difficult, therefore the early warning is mentioned as no’s of changes

Accepted due to the importance stated by experts and literature Data could be found as write-up in internal reports Accepted due to the importance stated by experts and literature Data could be found as write-up in internal reports Changes after 1/3 model review in project are considered as late changes.

5.6 EARLY WARNINGS FROM EXPERTS This part of research included 1) semi-structured interviews 2) preliminary analysis and 3) selection of early warnings. The preliminary analysis included refining the interview findings by understanding the early warning in context of O&C project execution. Furthermore, interview transcripts were rewritten to understand how early warnings affect project outcomes; the final selection is made on the criteria described in section 5.3. The main objectives of interviews were defined as follows: I. To find out whether information akin to early warnings can be detected in project execution of O&C projects II. If early warning can be detected, to form a description and understanding the mechanism behind those early warnings

53

®

III. How EWI relate to project outcomes? IV. To get practical example, (if exist) relating suggested early warnings with project problems and project outcomes. The group selected for interview was small, but at the same time, it was widely representative of actual project experience with the professional field of capital O&C project execution.

5.6.1

Interview planning and execution:

Due to the unavailability of literature on early warnings in O&C project execution, a development of redefined research questions was a challenging task. Therefore, semistructured interview approach was selected; all the interviewees were provided with a brief introduction about the research with definition of early warnings and project outcomes. In addition, the context, approach and possible research outcomes were explained to the interviewees. The interviews were conducted in October and November 2012, extending over period of 3 weeks. All the interviews were conducted face-to-face except one, which was conducted over the phone. The interview lasted 1.5 hours on an average with maximum duration of 2 hours and minimum of 1 hour. The interviewees’ base had been selected in consultation with supervisors at Fluor and mainly comprised of project directors, senior management (with prior PM experience) and project control managers (with prior PM experience). A complete list of interviewees has been attached as appendix C.2. Before the interview: Before the actual interview, the interviewee was emailed a preparatory document explaining the background of the research project, key definitions, proposed agenda of the research and the main practical matters. The document consists of brief introduction, aim of the interview, relevant project outcomes and further steps in research and attached as appendix C.1. During the interview: The steps for the interview were as follows: I. Personal introduction II. Introduction to the research project III. Setting the stage: types of EWI their basic characteristics, attributes, and potentially affected project outcomes. IV. Questions to understand the mechanism behind the suggested EWI to understand, how and to which problem they reflect. V. The measurability of suggested EWI and any practical project example VI. Closing remarks

54

®

After a brief personal introduction, the interviewee was explained the conceptual design of research, the concept of early warning indicators, EWI’s intended use in present research and the expected outcomes of the research project. The interviewee was asked to describe the key early warning indicators in project execution, which they believe leads to the project problems in project at later stage. Then further questions were asked to better understand of how the particular EWI influences the project outcomes or key deliverables of the project and how it enables the project manager to predict future problems. After the discussing the particular early warning, the interviewee was asked to delineate the time or instance on the project timeline at which this early warning can be identified along with the possibility to measure this early warning quantitatively or qualitatively. In addition, any example from interviewee’s practical experience was sought. The interview was concluded with a discussion of a few practical matters on project execution and key parameters to be kept in notice from a project manager point of view. All interviews were taped after consent and notes of important points were taken during the interviews. After the interviews, preliminary analysis was done. The final analysis was conducted after all the interviews had been completed.

5.6.2

Early warnings, the results of the interview

The detailed transcripts provide the overview regarding each early warning’s description, which project outcome it affected, and how it could be measured. Based on the interviews the list of early warnings from practical perspective is attached as appendix A.2:

5.6.3

Analysis of early warnings from experts

Based on the typology of early warnings found from expert interviews, it should be noted that there is no early warning mentioned as feeling or behavior, which might be explained by the fact that prior to interviews, every interviewee was made aware of the final objective of the research i.e. building of quantitative model. Another explanation could be that the interviewee base included people who had experience as PM or experience in project leadership, (PM, PCM, Sr. mgmt) and being leaders, they might perceive feeling/behavior aspects as qualitative to have meaningful interpretations from them rather than just behavior or feelings (Prati et al, 93; Pescolido, 02).

55

®

Early w arning from experts by sub-category 20 18

Quantitative

16 14

Qualitative 1

12 10

16

8 14 6

11 7

4

4 2 2

1

1

Early Engineering

Procurement

Concurrency in projects

0 Project Team development & integration

Change mgmt

Figure 16: Early warning from experts by sub-category

On the contrary, to the early warnings from literature, experts see “early engineering as most prominent category of early warning (31%). In addition, the majority of early warnings in sub category “project team development and integration’ are qualitative in nature (14 out of 15), similar to early warnings from literature. Apart from the classification of early warnings, it should be noted that the early warnings related to sub category “change management” and “early engineering” were mentioned by all interviewees irrespective of the nature of early warning, implies towards the possible importance of these two categories to determine project success. The following figure presents the graphical representation of early warnings (category wise) as mentioned by interviewees.

56

®

Early w arnings mentioned by nos of experts (Sub category w ise) 14

12

13

13

11

10

9 8

8

6

4

2

0 Project Team development & integration

Change mgmt

Early Engineering

Procurement

Concurrency in projects

Figure 17 Early warning mentioned by numbers of experts

Based on the selection criteria, following early warnings are selected for further analysis: Table 3: Early warnings from Experts

ID

Early warning

I-3

Large amount/Numbers of scope changes (additional items or details) by clients

I-13 I-15 I-16 I-18 I-20

High Concurrency level in projects High Level of client involvement Late scope change Overrun in process engineering manhours Delay in process engineering

Criteria Measurab ility

Early in project

Data Availability

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

Remarks Segregation of change orders from past projects at required level of detail is difficult, therefore the early warning is mentioned as no’s of changes Data could be found as write-up in internal reports

I-23

High No’s of arguments between E&C contractor and owner

√

√

√

Accepted due to the importance stated by experts and literature Data could be found as write-up in internal reports

I-25

Low level of team integration

√

√

√

Accepted due to the importance stated by

57

®

experts and literature Data could be found as write-up in internal reports I-26

I-27 I-31

I-35

I-40 I-43 I-52

Lack of understanding regarding project objectives among project team Delay in purchase order placing No’s of changes After P&ID's are issued for design due to rework Changes in major project objectives Early engineering discipline crossing late curve Higher numbers of changes in project at early stages Incomplete Front end design

√

√

√ Same as I-25

√

√

√

√

√

√

√

√

√

√

√

√

Accepted due to the importance stated by experts and literature Data could be found as write-up in internal reports Same as I-20

√ √

√ √

√ √

Same as I-3 Data could be found as write-up in internal reports

5.7 EARLY WARNING INDICATORS After the evaluation, several early warnings were found to be similar in terms of their meaning and were integrated as one early warning. The similarities are mentioned in the remark section of selected early warnings. The exercise to integrate the early warning resulted into a smaller numbers of early warnings. Following is the list of early warnings, which are selected for analysis via case studies, each early warning is defined in terms of its ID, name, measurability, source documents and the ID of other early warnings, which has been integrated with it. EWI related to Project team development and integration: From the analysis of early warnings from literature (mentioned by academic, CII and IPA) and experts (mentioned by eleven experts), it become quite evident that quality of the project team does play an important role in making a difference between successful and unsuccessful project. Yet the early warnings are qualitative in nature and no quantitative information could be found in past projects for development of prediction model, it seems necessary and logical to include them at-least in case study analysis. In addition, the past project literature does provide information in form of “write-ups” regarding the quality of team and its integration. Therefore following EWI were selected.

58

®

New ID LES

Early warning indicator Lack of understanding of project execution strategy among project team

PTE

Project team lacks experience required for the project Conflicts between owner and E&C contractor

COC

Measurability

Possible sources

Qualitative

Project review meeting, Project surveys

Qualitative

Deficiency in required and actual deployed resources Client survey, review meetings, contractual disagreements

Qualitative

Integrated early warnings I-26, L-48

L-34

I-15, I-25

Note: In the perspective of present research, the term conflict has been used in negative sense i.e. the conflicts which manifest the project problems. However, it should be noted that in conflict is not always dysfunctional or destructive, though it also play constructive role by generating new norms, and stimulating the engaged parties towards economic gains and technological advancement (Coser, 1957) EWI related to change management: The changes during project execution do affect the project outcomes (often negatively) and this is confirmed by both literature and experts. One of the basic principles of efficient work processes is that work is done once and done correctly. However, there might be situations in the project where changes are not avoidable due to safety, functionality or standards. Ideally, in an EPC phase of an O&C project any change after completion of FEED stage should be avoided. Any proposed change should pass the following criteria before implementation (Bakker, 2012): I. If the current design is not safe II. If the current design does not work III. If the current design is not as per standard Changes are generally detrimental to the health of a project, as they often have major impact on cost and schedule. It is also well documented that the cumulative impact of numbers of small changes is much greater than sum of individual impacts. This is mainly due the fact that the changes not only brings new addition of work, but also produces out of sequence activities and disruption effect on ongoing activities.

59

®

Another noteworthy point is that impact of disruption and out of sequence activities is often difficult to assess before implementing the change and often become visible at the later stage of project. The following EWI’s were selected based on the selection criteria. New ID NCO CCO

Early warning indicator Numbers of change orders Cost impact of changes

Measurability

Possible sources

No’s of changes in project Cost of changes/estimated cost of project

Change order log Change order log

Integrated early warnings L-32, L-34, L-49, I-3, I43 L-30, L-32, L-34, L-49, I-3, I-43

EWI related to early engineering: The early engineering in the EPC phase of an O&C project can be seen in two perspectives: 1) Completion of FEED 2) Process engineering of EPC phase The FEED phase of a project produces the basis of design (BOD) for detailed engineering. Efforts at the FEED phase defines the quality of design basis (BOD), The BOD forms the starting point for detailed engineering and the project baseline estimate. Therefore, an incomplete FEED could not only disturb well-defined start of the detailed engineering design but also affect the accuracy of estimate and resources for the project. However, if the incompleteness of front-end design has been addressed by project team, it is of utmost importance that incompleteness has been communicated well with in project team as well as has been reflected in estimates, management reserves and contingency plans. Process engineering works on BOD and set the design criteria for further discipline deliverables. Process engineering delivers the critical documents, which act as the base documents for subsequent engineering. In a case where process engineering is forecasting significantly higher numbers of hours due to any reason (changes, missing items, inefficiency etc,) it implies that the critical documents require additional efforts. The growth in process hours is an indicator that the subsequent engineering discipline also might need additional hours (to complete the added or modified work) and additional cost is required for material and services. The delay in process engineering implies that the critical documents will be issued with delay, thus affecting the downstream disciplines in terms of work front and delay in their respective deliverables. In addition, the delay in process engineering impacts the procurement cycle for the project and this could result into late delivery of material and ultimately construction delay. Any early problems in these two areas should be seen as an early warning. Following EWI were selected in this category

60

®

New ID

Early warning indicator

FED

Percentage of missing information in FEED package

PH

Growth in process manhours

PS

Delay in process engineering

Measurability IPA FEL index Non-conformities and observations in Gate-C Review Forecasted or actual hours / Baseline hours Actual % completion – Planned % completion

Possible sources

Integrated early warnings

IPA audit report Gate C review

I-52, L-46

Progress reports

I-18, L-2

Progress reports

I-20, I-40, L-1

EWI related to concurrent engineering: At the start of EPC phase, engineering planning is carried out based on certain level of concurrency (baseline concurrency levels) via information available from BOD, schedule and cost targets and PEP. From the analysis of early warnings by literature and experts, it could be concluded that the concurrent engineering execution is directly proportional to the amount of assumptions being made in project. A high concurrency level means that an engineering discipline is either working on assumptions or sufficient reliable information is available beforehand. In either case, there exists a risk of assumed information proved wrong at later stage, which will produce rework not only with in a specific discipline but also with the associated discipline activities, bulk quantities and changes in vendor’s scope. From an early warning perspective, it seems logical to identify the change in concurrency levels early in the detailed engineering to have a check, if design is being carried on high level of assumptions. The engineering discipline process and piping has been identified as indicator of concurrency in project. The choice has been made because of the following reasons I. The engineering activities of the piping discipline constitute a significant part of early engineering efforts are highly dependent on the inputs from process engineering. II. Piping engineering constitutes of a substantial part of engineering and material cost. III. The rework in piping could lead to the high impact of total project cost and schedule. Following EWI were selected in this category: New ID CE

Early warning indicator

Measurability

Change in concurrency level between process and piping engineering

Change in area between process and piping progress curves

Possible sources Discipline progress curves

Integrated early warnings I-13, L-32

61

®

EWI related to procurement: The procurement is of strategic importance as the equipment and materials are not only required for construction, but they also form a critical part of detailed engineering design. Many engineering deliverables are based on the information to be provided by equipment vendors. Delay in issuing of purchase orders can have direct impact on schedule of vendor information required for design. Based on the analysis of early warnings related to procurement and selection criteria, the following EWI were selected for analysis. New ID

Early warning

Measurability

DPO

Delay in issuance of purchase orders

Actual procurement progress - Planned procurement progress

Possible sources Procurement progress curves

Integrated early warnings I-27

5.8 DISCUSSION AND CONCLUSION The primary aim of this chapter was to find a set of early warnings that can be operationalized to build the performance prediction model, thereby answer the research question “What early warnings can be identified in project execution” and “Which early warnings can be operationalized to build a performance prediction model.” The aim was achieved sequentially via investigation of academic literature and expert interviews followed by selection based on defined criteria. The chapter presented a selection criterion, which is composed of three parameters “early in project”, “data availability” and “measurability.” For an early warning to be converted into EWI, all of these criterions need to be satisfied. It was found that literature provides a balance of quantitative and qualitative early warnings, whereas experts were found to be quantitative focused. The difference among sources was mainly found due to the reason that the interviewees were well informed regarding the aim of this research (I.e. the quantitative nature). Each early warning was evaluated on the selection criteria and few early warnings were selected. Due to the similarity in nature of many early warnings, few of them were integrated to form final ten early warnings. The ten early warnings are defined in detail, via how do they affect the project performance and project outcomes. Moreover, they were assigned new ID and defined in terms of their measurability, potential sources of information.

62

®

6 CASE STUDIES 6.1 SUMMARY The primary aim of this chapter is to assess the presence and predictive capability of selected early warnings (in section 5.7) in real life projects. To achieve this goal, a case study approach is adopted. A multiple cases embedded design is used, in which each case represents a completed project (Yin, 2002). The case study approach was chosen, mainly because the present research is focused on contemporary real life situation. Therefore, it seemed necessary to understand, how the suggested early warnings can be identified in projects, what problem they can predict and how they influence the project outcomes. In case studies, an effort has been made to recognize the link between early warnings, their associated project problems and final project outcomes. In other words, the intention here is to check and understand if an early warning provides a correct picture and illustration of their intended purpose. Section 6.2, discusses the basic design of case studies, including the possible data sources. Section 6.3 describes the factors, which were considered while selecting the projects for case studies followed by a brief overview of the selected cases. Section 6.4 to 6.7 discusses the individual case studies by discussing the presence of early warnings, their associated project problems and final project outcomes. Each case study is concluded by proving an overall case summary. The final section 6.8 provides a cross case analysis to argument the prediction capabilities of early warnings in the contrast set by Successful and less than successful performing case projects.

6.2 CASE STUDY DESIGN The chosen unit of analysis for this part of research is a completed project in the O&C industry. The case study design refers to the different sub units of analysis: early warnings, major problems in projects and project performance. A project is taken in following definition: it covers all activities after the start of EPC phase up to the engineering completion for identifying the presence of early warnings; this choice was made due to adopted definition of EWI (see section 4.4). However, the potential future problems, which could be identified by EWI, were not limited to engineering phase only. The case studies involved a study of written project archives, which involved closeout reports, project monthly status reports and other relevant project archives. The data collection method for case studies was uniform across all case studies. The following sources of information were searched: 1) Project closeout reports 2) Monthly progress report’s (MPR) narrative 3) Project deviation log, 3) Project change log 4) FEED project closeout reports (if available) 5) Gate C reviews (if

63

®

available) 6) IPA audit reports (if available) 7) Discipline progress curves 8) Procurement progress curves. Wherever possible, an effort has been made to investigate more than one source for particular evidence. On the top of above mentioned sources, informal talks with project team members were conducted with an intention to understand the itinerary of project execution. The information from project members also helped in identifying any special or abnormal events that happened during the project execution, which might have influenced the project outcomes. The relationship between detected early warnings, manifested project problems and their impact on project outcomes has been mapped by following framework.

Figure 18 Framework for mapping the relationship between early warnings, project problems, and project outcomes

In the above framework, the early warnings indicate the possible manifestation of specific project problem, which cause an undesired growth in project outcome i.e. affects the project performance. The relationship for each warning in all four case studies is mapped and has been attached as appendix D.

6.3 CASE STUDY SELECTION The cases selected for the study, covers both good performance projects and poor performance projects defined in terms of meeting their schedule and “as sold” cost estimates (at micro level, see section 4.3.1). The case projects were selected with intent of having a balance of good performance and poor performance projects. The good performance project can provide contrast in which the prediction capability of early warnings is more visible. Under this assumption, the early warnings in a less than successful project should be clearly visible or at-least have quantitative values different from their values in successful project.

64

®

Having said that, three of the four selected case study projects were executed between 2008 and 2012. One case project was executed in year 2004 and 2005, which was selected due to its extremely successful execution, and therefore could provide useful information while comparing the cases. In terms of project characteristics, the case projects are summarized in following table: Case project Case project 1

Cost of the project (Estimated) Confidential

Case project 2

Confidential

Case project 3

Confidential

Case project 4

Confidential

Project type

Contract Type

Chemical-green field Refinery upgradebrown field Chemical-Green field Chemical-green field

Cost-reimbursable fixed Fee Cost-plus fixed Fee Lump sum Cost-reimbursable fixed Fee

Year of EPC execution 2008-2011 2004-2005 2011-2013 2010-2012

The case number one and four did experience growth all project outcomes. However, the growth in final project outcomes i.e. TIC and MCI has been relatively low as compared to the MHI and ESI. The good performance project did not show any significant growth in final project outcomes i.e. TIC and MCI. The case project number two finished on schedule with cost under. The case project three finished on schedule with less than 1% growth in TIC. The case project three did experience growth in MHI, at the same time finished the detailed engineering ahead of the schedule.

6.4 CASE 1 (LESS THAN SUCCESSFUL PROJECT) 6.4.1

Brief project description

The objective of discussed case project was to construct a new fully continuous integrated chemical. The E&C contractor worked under fixed fee reimbursable contract.. The period for EPCm execution of project was from 2008 to 2011. From project documentation, it was concluded that during project execution, the project drivers were often changed. This is evident by the multiple cost review and scope reduction exercises. The final project experienced both cost and schedule overrun. Apart from cost and schedule, the performance of the project was very good in terms of functionality and safety.

6.4.2

Project evaluation

An argumentative evaluation is presented in each section to understand how selected early warnings could have predicted the project problems and their relation with the project outcomes. 1) Team development and integration: The project execution team involved E&C contractor team, client team, an external major sub-contractor and work share offices. The summary of the findings on project team development and integration is given in following table:

65

®

Early warning

Presence

Lack of understanding regarding project execution strategy among project team

Yes

Project team lacks experience required for the project.

Yes

Conflicts between owner and E&C contractor

Yes

Moment of detection 4 months after EPC start 2 months after EPC start 2 months after EPC start

EWI mentioned as

Source

A Subcontractor did not perform in line with the requirements. Few Inexperience issues identified within project team Extreme involvement by few stakeholders raised conflicts

Project status report Project status report Project status report

Analysis of the project status reports indicated that the project team integration was a challenge during project execution. The project team (E&C, Client) had some inexperience issues. Extreme involvement by few stakeholders amplified the decision-making problems. A sub-contractor did not seem to be aligned with the project execution strategy, which prompted the E&C contractor to take corrective actions that resulted into some rework. The analysis of project reports suggests that the early warning in team development and integration domain has substantial potential to predict the problems at late engineering and construction stage. The link between early warnings, their associated future project problems, and their impact on final project outcomes is attached in appendix D.1.1: 2.) Change management: A strict change management process was placed since the start of EPC phase, although project execution went through changes both in scope addition/reduction and in detailing of design. The numbers of change orders were seen as extremely high by the E&C contractor team and issues were raised. In addition, the frequency of change orders was quite high, there were 70 change orders between 15 to 30 % engineering. The significant numbers of change orders not only affected the engineering efforts, but also affected proper assessment of their impacts. The status of early warnings is shown in following table. Early warning

Presence

Detection time in project

Early warning mentioned as

Source

Number of changes

Yes

7 Months after EPC start

No’s of agreed change Orders – 132

Change order Log

Cost impact of changes

Yes

7 Months after EPC start

Cost of Changes/TIC of plant = 6 %

Change order Log

The relationship between early warnings related to project changes, project problems and their impact on final project outcomes is illustrated with figure in attachment D.1.2.

66

®

It was observed from the study of project archives that such high magnitude of changes introduced many problems during project execution. The manifested problems include engineering, cost and schedule related problems. 3.) Early engineering: The assessment of FEED package and process engineering is explained further below. I) Completion of FEED – As explained in Figure 2 when a project moves from the FEED phase to EPC, generally a review should be conducted (known as “gate review”). Furthermore, the quality of a gate review should be thorough, rather than a “tick the box” approach. In the present case project, the gate between FEED and EPC is known as Gate C. The Gate C review of the FEED package determines the quality of BOD and ideally, all issues at this stage should be resolved before moving into full scale EPC. Gate C review of the project indicates that the E&C team identified non-conformities and observation from organization’s standards including few related to design required for downstream disciplines. Furthermore, due to schedule pressure, the project team made a plan to deal with non-conformities; however, it was mentioned in project documentation that project moved into EPC too early. In summary, the real implications of missing information at FEED stage only became visible at EPC phase and resulted late engineering problems and enforced E&C contractor to work on concurrent execution. Early warning

Presence

Detection time in project

EWI mentioned as

Source

Percentage of missing information in FEED package

Yes

Start of EPC

Non-conformity Observations

Gate C review

II) Process Engineering: As per interdisciplinary logics of engineering design, any design problem in process engineering produces a negative effect on downstream engineering disciplines. The case project has few issues with deliverables in process engineering. The extra efforts made, which had to be made by process engineering to meet with deficiencies from previous phase enforced few scheduling problems within the discipline and for downstream disciplines and affected some of the deliverables of the project. The total project team worked cohesively to deal with problems, thus caused some overrun in manhours. The status of early warnings at 2/3 model review stage is shown in following table: Early warning

Presence

Growth in process manhours

Yes

Delay in process engineering

Yes

Detection time in project 7 Months after EPC start 7 Months after EPC start

Early warning mentioned as Growth in process hours Process engineering behind schedule

Source Project status report Project status report

67

®

The relationships between early warnings, project problems (discussed in above paragraphs) and project outcomes are shown D.1.3 below. 4) Concurrency: In the case of project 1, the concurrency level in process and piping discipline varied throughout the engineering execution. It can be concluded that 15 % to 30 % engineering duration, process forecasted growth in hours, and was behind schedule. There was downstream effect in piping and to meet the schedule constrains piping engineering made assumptions. Later, when the piping engineering was not able to sustain the assumption and could not ignore the risk of high amount of rework, it re-optimized its schedule and forecasted some delays thus concurrency level decreased, going beyond the planned concurrency. However, the concurrent execution of piping engineering in the early stage of the project (up to 45 % engineering completion) resulted into few problems related to cost and schedule. The re-optimization and some rework affected bulk quantities and their subsequent purchase; in addition to some delay in project milestones. 5. Procurement: The case project did not experience much delay (4 %) in issuance of purchase orders for equipment as due to favorable economic conditions (Buyer market) at early 2008, and major equipment were already purchased (50 % PO issued at start of EPC). However, from the project execution perspective the strategy met with some challenges, mainly due to the changes in the project. The E&C contactor made corrective strategies to deal with the problem and had some cost and schedule impact on project. Early warning

Presence

Delay in PO

Yes

Detection time in project – 2/3 Model review 7 Months after EPC start

EWI mentioned as

Source

Delay in PO

Project status report

From the perspective of EWI, the delay in PO did not manifest any major problem during the project. However when coupled with late changes in the project it did contribute to the cost overrun.

6.5 CASE 2 (SUCCESSFUL PROJECT) 6.5.1

Brief project description

The case project 2 stared in January 2011 and the planned to be complete in august 2013. The objective of case 2 project is to build a chemical plant. The project is a part of larger project, aimed to construct a chemical complex owned by consortium petrochemical companies. The EPC contract between E&C contractor and owner is on lump-sum basis. The EPC project duration is 29 months with 14 months for detailed engineering. The peak engineering staffing

68

®

for project was 160 full time equivalents (FTE). Although the project is still in end of construction phase, the project performance so far has been very good in terms of cost, schedule, quality and HSE.

6.5.2

Project evaluation

As explained in section 6.1 (case study design) project evaluations are performed with an intention to identify the presence and prediction potential of EWI. The present case project is of lump-sum nature and still in the last phase of execution; therefore, project reporting was selective and regarded as highly confidential. 1) Team development and integration: The project execution team consisted of owner and E&C contractor and work-share office within E&C contractor. The project management team consisted of an integrated management team (IMT) with owner and E&C contractor with an intention to gather the experience of operations from owner. The IMT approach was adopted since the FEED phase of the project. The IMT approach did work well and no significant issues were identified in the team development and integration. 2) Change management: The changes implemented during the detailed engineering phase are considered significantly low for project of such magnitude. The cost impact of implemented changes was only 0.6% of estimated TIC, with the largest change being 0.24 % of TIC in magnitude. The early changes did produce some rework during engineering, but the project was capable of absorbing the impact. However, the schedule impact of changes was contained within schedule and the 95% engineering was achieved ahead of schedule. In addition, no period in the project experienced relatively high numbers of changes. The well executed “no change” policy helped the project in initial stages by not only minimizing the impact on engineering efforts, but also to assess the impact of changes more precisely. The figure 25 below underlines the above observations. The status of EWI related to changes is as follows: Early warning

Presence

Number of changes

Yes

Timing of changes

Yes

Cost impact of changes

No

Potential direct impact of changes on engineering Hours

No

Detection time in project – 2/3 Model review 7 Months after EPC start 7 Months after EPC start 7 Months after EPC start 7 Months after EPC start

Early warning

Source

No’s of change Orders – 20 No’s of changes after 1/3 model review – 20 Cost of Changes/TIC of plant = 0.4 %

Change order Log Change order Log Change order Log

CO man-hours/original budget = 0.3 %

Change order Log

The mapping of EWI, associated project problems (discussed in above paragraphs) and their impact on project outcomes has been illustrated in appendix D.2.1.

69

®

I) Completion of FEED: The Gate C review of the project was conducted 3 months prior to the start of EPC phase with an objective to assess if the BOD and the associated work were ready to proceed into EPC phase. The overall assessment suggested that the BOD was sufficiently defined to move into EPC phase. The results of Gate C review indicated few non-conformity and observations, however the time available between the FEED and EPC phase allowed for the resolution of all the observations. All non-conformities were rectified by E&C project team. Early warning

Presence

Detection time in project – Gate C Review

EWI mentioned as

Source

Percentage of missing information in FEED package

No

Start of EPC

Non-conformity – 0 Observations – 4

Gate C review

II) Process engineering: The process engineering execution for case project went extremely well with ahead of schedule and with small person-hour growth. The process deliverables were well on schedule with no or minimum scope changes at later stage, that could be validated with only 0.82 % man-hour growth in process engineering caused by changes. In addition, there were no significant project problems identified. The status of EWI related to process engineering is shown below. Early warning

Presence

Detection time in project – 2/3 Model review

Growth in process manhours

No

7 Months after EPC start

Delay in process engineering

No

7 Months after EPC start

EWI mentioned as Growth in process hours – 3% (negative) Process engineering ahead of schedule (3 %)

Source Project progress data Project progress data

4) Concurrency: The case project did experience increase in the concurrency between process and piping engineering. However, the increase in concurrency did not manifest into any major project problem. The status of CE has been shown in following table. Early warning

Presence

Detection time in project – 2/3 Model review

EWI mentioned as

Source

Change in concurrency level between process and piping engineering

Yes

7 Months after EPC start

Increase in concurrency by 20%

Discipline progress curves.

5. Procurement: The case project did experience a delay in issuance of purchase orders for equipment during early stages. However, the delay was recovered soon enough and purchase orders were issued well in time to receive the required vendor information. From the

70

®

perspective of early warning, the delay in PO did not manifest any major problem during the project.

Early warning

Presence

Detection time in project – 2/3 Model review

EWI mentioned as

Source

Delay in PO

Yes

7 Months after EPC start

Delay in PO –0.1 %

Project status report

6.6 CASE 3 (SUCCESSFUL PROJECT) 6.6.1

Brief project description

The case three project was aimed at up-gradation of a fuel refinery to produce cleaner fuels. The project was brown field in nature and involved tie-in with existing operation plant. The contract between E&C contractor and project owner was EPCm under cost reimbursable with incentive arrangement. The period for project EPC execution was 2004 through 2005 with 13 months for detailed engineering and 19 months to MC. The project performed quite extraordinary with under run in TIC and on time completion of plant. Furthermore, the project performance in terms of safety and quality was in line with client’s expectations.

6.6.2

Project evaluation

1) Team development and integration: The case project involved a Fluor project team, client team, and Fluor work share offices. The study of internal status review documents suggested that were no issues at all with client and internal interfaces. On few instances, the project literature did mention conflicts with the client during initial stages, but they were limited to negotiations. Later the contract negotiations concluded with satisfaction of all parties concerned. The E&C team was open to client audits and there were no major disagreement issues identified. Early warning

Presence

Lack of understanding of project execution strategy among project team

Detection time in project

EWI mentioned as

Source

No

No issues identified

Project status report, IPA audit

Project team lacks experience required for the project

No

The project team is well integrated & experienced

Project status report, IPA audit

Conflicts between owner and E&C contractor

No

No issues identified

Project status report

2) Change management: The project change log reveals that there were no significant changes in the project due to the “no change” culture from client and E&C contractor. The

71

®

successful implementation of a “No change philosophy was possible due to the strong and clear implementation of FEED package, management support and regular feedback to the team regarding success of “No change” philosophy. The changes implemented in EPC phase resulted in net cost saving. The engineering hours in change orders show 4 % growth with respect to original budget, and the total TIC impact shows 1 % saving. The following observation is shown by graph. The EWI identification: Detection time in project – 2/3 Model review 7 Months after EPC start

Early warning

Presence

Number of changes

Yes

Timing of changes

Yes

7 Months after EPC start

Cost impact of changes

No

7 Months after EPC start

Hour impact of changes

Yes

7 Months after EPC start

EWI mentioned as No’s of change Orders – 14 No’s of changes after 1/3 model review – 14 Cost of Changes/TIC of plant = -0.87% CO manhours/original budget = 4 %

Source Change order Log Change order Log Change order Log Change order Log

3) Early Engineering: The early engineering in EPC phase O&C project can be seen in two perspectives: I) Completion of FEED: The quality of FEED for case project was significantly high. The audit by IPA suggested that the FEL index for FEED was 4.5, which is considered as “best in class.” The project moved into EPC phase with just three non-conformities (As per Gate C review). The gate-C audit report of the project concluded that the design is ready to move into EPC phase. However, the report did recommend that project should wait for 2 weeks for the start of few activities to include the information in BOD rather than working on assumptions. No information could be found regarding project action on the recommendations of the audit report. Early warning Percentage of missing information in FEED package

Presence

Yes

Detection time in project – Gate C Review

EWI mentioned as

Source

Start of EPC

Numbers of Nonconformities FEL Index – 4.5 “Best in class”

Gate C audit report IPA Audit report

II) Process engineering: The case project had an exceptional well-executed process design. As mentioned earlier, this is due to “well defined FEED package” and “No change” philosophy employed in the project. The process engineering deliverables experience no major changes and timely process deliverables enabled downstream disciplines to execute work as planned. Furthermore, the process man-hours showed under run of 9 %. This statement could

72

®

be verified by the zero growth in process man-hours and on schedule execution of process engineering.

Early warning

Presence

Detection time in project – 2/3 Model review

EWI mentioned as

Source

Growth in process manhours

No

7 Months after EPC start

Growth in process hours under-run by 16 %

Project status report

Delay in process engineering

No

7 Months after EPC start

Process engineering behind schedule – 0 %

Project status report

4) Concurrency in engineering: The concurrency level between process and piping engineering did not change much during project execution, only at one instance the concurrency increased to 20 % level at 45 % engineering complete, which could be explained be a slight delay in process engineering and its projection of optimistic schedule. Yet the project was able to absorb the increase in slight concurrency in concurrency level without compromising schedule or cost. From the gathered information, no major problem could be identified which was evolved due to concurrent engineering. 5) Procurement: The project did not experience any significant delay in issuance of purchase orders, and no problems were mentioned in the project documentation that was a result of delay in purchase orders. Early warning

Presence

Delay in PO

Yes

Detection time in project – 2/3 Model review 7 Months after EPC start

EWI mentioned as

Source

Delay in PO – 2 %

Project status report

6.7 CASE 4 (LESS THAN SUCCESSFUL PROJECT) 6.7.1

Project brief description

The case four project was aimed at construction of a chemical processing facility. The project was green field in nature and was extension of an existing chemical complex. The contract between E&C contractor and project owner was for EPCm under cost reimbursable with liabilities. The schedule was the main project driver followed by cost and quality. The period for project EPC execution was 2010 through 2012 with 11 months for detailed engineering and 18 months to MC. The project did not perform as per expectations with over run in TIC

73

®

and schedule overrun. Furthermore, the project performance in terms of safety and quality was in line with client’s expectations.

6.7.2

Project evaluation

1) Team development and integration: The project execution team involved E&C contractor, client team, and work share offices with in Fluor. The project adopted an integrated project execution team led by client’s project manager. The summary of the findings on project team development and integration is given in following table: Early warning

Presence

Lack of understanding regarding project execution strategy among project team5

Yes

Project team lacks experience required for the project.

Yes

Conflicts between owner and E&C contractor

Yes

Detection time in project 2 months after EPC start 2 months after EPC start 2 months after EPC start

EWI mentioned as Different perception about project execution strategy The project required new software system for piping design, on which project team had limited experience. Different underlying interests of stakeholders. Less synergy on impacts of proposed changes on project

Source Project status report Project status report Project status report

Analysis of the project status reports indicated that the project team integration had a good start with establishment of integrated project management team, but soon challenges emerged due to different underlying interest of stakeholders. At few instances, intervention level by stakeholders was seen as significantly high and was a point discussion among project team. Moreover, the schedule and cost impact of almost all changes were challenged by project team with an intention to only accept necessary changes, which ultimately affected relations and timing of change implementation. The link between EWI, above discussed project problems and their impact on final project outcomes is illustrated in attachment D.4.1: 2.) Change management: The basis for development of aggressive project schedule was “no change during engineering,” however the project experienced introduction of heavy changes since the start of EPC phase. Within 5 months of EPC, 51 change orders were issued. These changes were in direct violation of assumptions made in schedule development. The possible impacts of such high changes were highlighted by project E&C project management. At intermediate model review, there were 125 approved changes in the project with a cost impact of 4 %. The summary of the relevant EWI is presented in following table:

5

However, there is no specific reference to the difference in understanding of project execution strategy among project team; the study of project document does suggest that there were differences in confidence level of team members regarding highly concurrent execution model.

74

®

Detection time in project – 2/3 Model review 7 Months after EPC start

Early warning

Presence

Higher number of changes

Yes

Timing of changes

Yes

7 Months after EPC start

Cost impact of changes

Yes

7 Months after EPC start

Potential direct impact of changes on engineering Hours

Yes

7 Months after EPC start

EWI mentioned as

Source

No’s of change Orders – 125 No’s of changes after 1/3 model review – 125 Cost of Changes/TIC of plant = 4 % CO manhours/original budget = 23 %

Change order Log Change order Log Change order Log Change order Log

The analysis revealed that the project did acknowledge the impact of changes and their effect of final engineering hours yet was a point of conflict. In addition, the frequency of change orders was quite high as there were on average 26 change orders between 0 to 60 % engineering. The significant numbers of change orders not only affected the engineering efforts, but also brought inefficiency in assessing the impact of changes. Moreover, relationship between EWI related to project changes, the project problems, and their impact on final project outcomes is illustrated by attachment D.4.2 3.) Early engineering: I) Completion of FEED – The Gate C review conducted by project team indicates that the FEED package had non-conformities and observations of deviations from the standards of E&C organization. The literature shows there were deficiencies with the received BOD and was not of good quality to proceed into EPC phase. However, due to schedule requirements the project continued for a certain time with project team simultaneously improving BOD. In summary, the real implications of missing information at FEED stage only became visible at EPC phase and resulted late design changes and concurrency in design disciplines. Early warning

Presence

Incomplete front end engineering design

Yes

Detection time in project – Gate C Review Start of EPC

EWI mentioned as

Source

Non-conformity Observations

Gate C review

II) Process Engineering: The P&IDs during early design went through major changes as soon as project moved into EPC phase. Because of the incompleteness in the BOD, the activities had to be reworked. Moreover, the discovery of new work during process engineering schedule and cost related problems in project. It was observed that the E&C process engineering experienced & acknowledged the problems at very early stages, and forecasted growth, which produced cascading effect into downstream discipline and ultimately total engineering.

75

®

Early warning

Presence

Growth in process manhours

Yes

Delay in process engineering

Yes

Detection time in project – 2/3 Model review 7 Months after EPC start

EWI mentioned as Growth in process hours

Project report

status

7 Months after EPC start

Process engineering behind schedule

Project report

status

Source

The relationships between early warnings, project problems, and their impact on project outcomes are shown in attachment D.4.3: 4) Concurrency: In case of project 4, a concurrent project execution model with high concurrency level in process and piping discipline was planned. An interesting observation was made that the concurrency level did not change significantly throughout the project engineering duration. It implies that the during the project execution the E&C team was proactive and acknowledge the risk of concurrent execution and piping engineering forecasted the realistic progress plans based on behavior of process engineering. However, at the later stage, the concurrency level reduced further because frequent changes forced piping engineering re-optimize its planning. From EWI perspective, the change in concurrency did not reveal any significant problem mainly because the pre-set level of concurrency very initially high and as the detailed engineering progressed in the project, the process and piping discipline addressed the early problems and reforecast their realistic growth. 5. Procurement: The case project experienced delays in issuance of purchase order for equipment. The delay in PO did affect the downstream disciplines regarding required vendor information for their design. Early warning

Presence

Delay in PO

Yes

Detection time in project – 2/3 Model review 7 Months after EPC start

EWI mentioned as

Source

Delay in PO

Project status report

Delayed vendor design information produced a cascading effect and ultimately induced schedule and cost related problems, which in turn affected the availability of work front for mechanical and piping contracts at site. From the perspective of EWI, the delay in issuance of PO resulted few project problems and ultimately contributed to the growth in project outcomes. The relationship of early warning, project problems, and project outcomes is shown in attachment in D.4.4.

76

®

6.8 CROSS CASE ANALYSIS This section presents a cross case analysis to build a contrast by comparing the four case projects. The cross case analysis is conducted by collective discussion of early warnings across all four case studies. Project team development and integration: All three early warnings (LES, PTE, COC) in this category were identified in case of project 1 and case project 4, that were “less than successful” projects. Early warnings LES PTE COC

Case 1

Case 2

Case 3 X X X

√ √ √

Case 4 X X X

√ √ √

Red Tick “√” = Early warning detected and contributed to project problems, Green tick “√” = early warning detected but did not result into any problem, X = no early warnings was detected The effect of COC was observed to be important in all case projects. The conflicts in project environment not only reduce the effectiveness of decision-making process, but also affect the management of changes. From efficiency and communication perspective, the early warning LES did play an important role during execution of case projects. In case of project two and three, the positive synergy on project execution strategy among project team members did improve the communication and enhanced the overall efficiency. In addition, the understanding of project execution strategy among project team does result into a focused approach towards project’s main driver and help the project to adapt in case the project driver changes (due to external or internal environment). In addition, it was found that incase of projects with less than successful performance, lack of experience among few domains of project team members did induced problems. Change management: All four case projects did declare the use of “No change policy” at the start of EPC phase. The “no change” prohibits the introduction of any change unless and until the design is either unsafe, does not function or violate any procedure (Bakker, 2012). However, almost all the projects did implement changes during their execution. Early warnings NOC CCO

Case 1

Case 2 √ √

Case 3 √ √

Case 4 √ √

√ √

“√” = Early warning detected and contributed to project problems, “√” = early warning detected but did not result into any problem, X = no early warnings was detected In case of projects with less than successful performance, (case project one and four) the changes were seen as extremely high and did contribute to the manifestation of project problems. The major problems found to be associated with changes were rework (In E&C domain and associated contracts and vendors), out of sequence activity execution. Furthermore, the high no’s and frequency of changes negatively affected the assessment and management of changes.

77

®

The case project two experienced some rework due to early changes in the project that resulted into growth in engineering man-hours and made small impact on the TIC. However, the small no’s of changes allowed proper assessment and management of those changes. In case of project 3, the no’s of changes were seen as small by project. In addition, the majorities of the changes were small in their cost impact and intended towards cost savings and no major problems were found to be associated with implemented changes during project execution. Early engineering: The early warnings FED, PH and PS did show a distinct behavior in successful and less than successful projects. The results of early warnings in this category are shown in following table. Early warnings FED PH PS

Case 1

Case 2 √ √ √

Case 3 X X X

Case 4 √ X X

√ √ √

“√” = Early warning detected and contributed to project problems, “√” = early warning detected but did not result into problems, X = no early warnings was detected Significant non-conformities and observations were found in FEED design of case project one and four that subsequently helped in manifestation of project problems. One of the major problems found to be associated with FED was additional rework to improve the quality of FEED, which was earlier not accounted in the EPC estimate. The second problem was discovery of new work in form of changes that brought engineering problems and to a certain extent affected the procurement efforts. The both early warnings in process engineering domain i.e. PS and PH were detected in case of project one and four, and contributed to project problems. From schedule perspective, the delay in deliverables of process engineering did affect the activities of downstream disciplines and key project milestones such as intermediate model review. In addition, the delay also resulted into to concurrent engineering to maintain the project schedule. The case projects two and three did not experience any significant growth in process hours or delay in process engineering. Thus not only avoided the above-mentioned problems however supported the downstream discipline with on schedule deliverables and concrete information. Concurrency: The concurrent engineering design between process and piping engineering was observed in case project one, two and three. Early warnings CE

Case 1

Case 2 √

Case 3 √

Case 4 √

X

“√” = Early warning detected and contributed to project problems, “√” = early warning detected but did not result into problems, X = no early warnings was detected However, the amount of change in concurrency differs with each project. The case project one experienced a 20% increase in concurrency until 45% engineering completion followed by reduction of 40% from baseline concurrency level. Such high change in concurrency level represents the level of assumptions adopted by piping engineering. The subsequent reduction

78

®

in concurrency level was observed due to the high risk associated with assumptions thus delaying the deliverables or actual information found to be different from assumptions, which forced the piping discipline to forecast delay in their schedule. In case of project two and three, the change in concurrency level was less significant as compared (20%) to project one. The projects were able to absorb the changes and did not face any major problems. The ability to absorb the increase in concurrency level could also be explained by number of changes in these projects, as the project did not experience significant amount of changes therefore it could be concluded that the assumptions made by piping engineering did not change significantly. From change in concurrency perspective, the case project four proved to be an exception. The project was seen as extremely concurrent by project team members, therefore the piping engineering discipline was proactive in forecasting its schedule with reference to schedule of expected inputs from process engineering. The joint change in progress curves of these two disciplines resulted into low change in concurrency level. Procurement: The early warning, DPO was detected in all four case projects. Early warnings CE

Case 1

Case 2 √

Case 3 √

Case 4 √

√

“√” = Early warning detected and contributed to project problems, “√” = early warning detected but did not result into problems, X = no early warnings was detected In case of project one, majority of PO were issued before the start of EPC (due to favorable market conditions) and project did not experience any significant delay in rest of the PO. However, the adopted approach faced challenges due to the high number of changes. The successful projects two and three did experience slight delay in issuance of PO. However, their delay is limited to 30 % engineering completion. After which they delay was recovered and did not contribute to any major problems. The case project four did experience delay in issuance of PO. The delay was found to associate with few major problems during project execution. The schedule of detailed engineering was impacted due to delayed vendor design information. Consequently, the estimation of bulk materials was delayed followed by delay of constriction drawings and work at construction site.

6.9 DISCUSSION AND CONCLUSION In this chapter the relationship between early warnings, project problems and project outcomes were presented. It was investigated that whether the identified early warnings help the manifestation of project problems or they themselves turn into problems and affect the project performance. The four in-depth case studies showed the dynamic between early warnings and project performance in form of project problems. From overall research perspective, this chapter contributed to answering the research question “What are the dynamics between early warnings and project performance?” Understanding the dynamics between early warnings and project performance: Through considerable analysis of project documentation, each early warning was mapped with the

79

®

specific project problems and project problems were analyzed to map their affect on project performance. By looking at the time aspect of relationship, it was concluded that the early warnings could predict the project problems and specific project problems manifest because of problems in domain of these early warnings. Observatory evidence to predictive capability of early warnings: The contrast set by project with different performance project played a significant role in highlighting the predictive capability of early warnings. The differences in detection of early warning and their associated states (quantitative and qualitative) validate the importance of early warnings during project execution. It can be concluded that if these early warnings are monitored and interpreted as symptoms of potential problems, they could provide advance and crucial information for proactive actions.

80

®

7 QUANTITATIVE ANALYSIS SUMMARY In the previous chapter, it was observed that in case of good performance projects, most of the early warnings were absent or had quantitative values within acceptable limits (In the view of project team). The projects with significant cost/schedule overrun exhibited early warnings with significant high quantitative values. The difference in performance of projects in light of early warnings became clearer during cross case analysis in section 6.8. In the present chapter, an attempt has been made to explore the set of early warnings over a larger number of projects than in chapter 6. As explained in section 5.7, it is not possible to include qualitative early warnings (LES, COC and PTE) in development of the quantitative prediction model without introducing user specific subjectivity. In addition, the early warning “completion of FEED” could not be implemented, as it is static in nature and provides information only at the start of EPC phase. Furthermore, if the deficiencies in FEED are not corrected before diving into EPC phase, the other early warnings such as process engineering should show warning like behavior. Section 7.1 discuses the analysis approach at three levels, longitudinal behavior analysis of early warnings, correlation analysis at 95% engineering completion and longitudinal correlation analysis of the project. The description of data collected for past projects along with an explicit description of early warnings as explanatory variables and project outcomes as dependent variables has been attached as appendix G. Later in the chapter, section 7.2.1 analyzes the longitudinal behavior of early warnings over engineering duration. For this purpose, each EWI is plotted over the engineering duration of the project. Subsequently to have the quantitative evidence of relationship between EWI and project outcomes, the correlation analysis at 95 % engineering completion is performed in section 7.3, followed by longitudinal correlation analysis over engineering duration. In the end of this chapter, section 7.5 provides the discussion on three levels of analysis and conclusions of this chapter.

7.1 ANALYSIS APPROACH The quantitative analysis is performed at three levels. The first level of analysis investigates the longitudinal behavior of early warnings over engineering duration. The second level of analysis investigates the correlations of EWI with project outcomes at 95% engineering completion. The third level of analysis combine the above two level and establish the correlations between EWI and project outcomes longitudinally over engineering duration. The third level of analysis will lay the foundation for development of dynamic prediction model. The analysis approach is shown in Figure 19: Quantitative analysis approach and their relation is explained in following paragraphs.

81

®

Figure 19: Quantitative analysis approach

Based on the finding of cross case analysis in section 6.8, a hypothesis is made that the EWI do behave differently in case of successful and less than successful projects. The first level of analysis tests this hypothesis by studying the longitudinal behavior of EWI over engineering duration of past eight projects. If the results of the analysis are found to be affirmative towards stated hypothesis, then the findings could be seen as first empirical evidence of predictive capability of EWI. In addition, the results could validate the correlations found in subsequent analysis and supports the causality between EWI and project performance. The second level of analysis, i.e. correlations at 95 % engineering completion could be seen as complementary to level one analysis and at the same time take the analysis further by mapping the relationship of each EWI with parameters of project performance i.e. project outcomes. The main question to be answered by this analysis is that “which EWI relates to which project outcome.” As it could be observed in chapter 6 that not all the EWI contribute to all the project problems and not all the project problems, affect all project outcomes. Therefore, the second level analysis makes an effort to understand the relationship between EWI and project outcomes. The third level performs a longitudinal correlation analysis over the engineering duration. This analysis could be seen as a statistical combination of first two analyses. In addition, it provides the dynamic statistical relationship between EWI and project outcomes that will be used for building of prediction model in subsequent chapter.

7.2 EXPLORATORY DATA ANALYSIS To build the prediction model, data relevant to six early warnings and four project outcomes is collected from past eight projects. The dataset constitutes a relatively balance type of project in terms of size, cost and type. The overview of the past project has been provided in appendix F. The final project outcomes of eight past projects are described in appendix E.

82

®

Note: Moreover, looking at the project outcomes in four projects (in appendix E) with reference to the study presented by Merrow6 in chapter 1, it can be inferred that the project execution by Fluor Corporation are well executed. (As no project experienced growth of 30% in either TIC or MCI) It could be observed from the figure that the data include a mixed performance projects. For the classification purposes, the present research consider a project with less than successful performance, if and only if the project has 15 % growth in either final cost (TIC) or schedule delay of more than 10% in completion of project (MCI) and combination of both. The scatter plot matrix was plotted and has been attached in appendix E. It is observed that the there is a strong relationship between TIC and MHI; this implies that a significant growth in engineering man-hours leads to the growth in TIC. From cost perspective, the observation seems obvious as engineering manhours has cost associated with them. As explained in section 4.3.2 the cost of engineering efforts is quite small as compared to the cost of equipments and construction (on average varies between 10-15 % of TIC). Therefore, even a 100 % growth in manhours could only affect 10 to 15 % of TIC. However from logical perspective the above observation implies that the impacts of problems in engineering design is not limited to engineering itself, in addition they do affect the activities in procurement and construction, therefore affect total project cost. The above statement could be validated by found relationships between EWI, project problems and project outcomes in chapter 6. The above observation is also true for MCI and ESI i.e. delay in completion of detailed engineering leads to the delay in mechanical completion of the project. As discussed in section 4.3.2 engineering prepare and deliver key documents for procurement and construction activities, therefore a relationship between MCI and ESI validate that the delay in issuance of these key documents do impact the progress of procurement and construction activities. Another important observation is that the growth in engineering manhours does not necessarily result in improvement of schedule. This could be observed by absence of negative relationship between MHI and MCI. Therefore, it can be concluded that in most of the past projects the growth in man-hours was not caused by schedule acceleration, rather rework or other problems.

7.2.1

Behavior of EWI over engineering duration

The next step in exploratory analysis was to study the behavior of each early warning to check, if there exists a distinct behavior for projects that performed well and others whose performance was not as per expectations. To achieve the above goal, each early warning was plotted over the engineering duration of the project and results are attached as Appendix F and discussed in following paragraphs. Process schedule (PS): It is observed that PS showed a distinct behavior in good or average performance projects and less than successful performance projects. An interesting

6

O&C industry’s 50% project experience growth more than 30% in either cost or schedule.

83

®

observation is that in most of the projects, which did not perform well, process engineering was not able to recover from delay until 60 % of the engineering duration. After which, almost all the projects did show improvement. This could mean that the delay in early deliverables of process engineering have more impact on the schedule and performance of downstream disciplines and total project performance7. Process hours (PH): The results are similar as PS. The PH had significantly low growth values in good or average performance and high values in less than successful performance projects. It could be observed from the figure that process engineering start projecting significant growth after 30% engineering completion. When combined with delay in process schedule, it can be concluded that process engineering forecast growth in manhours after actual delay in early deliverables ( average 7.5 % delay in process at 30% engineering completion) , which might be due to the changes or incomplete FEED. Number of change orders (NOC): The analysis found that the good performance project did experience less numbers of changes as compared to the less than successful performance projects. The analysis indicated that the behavior of cumulative NOC in case of successful projects (Green and blue) is significantly different as compared to less than successful projects (Red). However at the early stages in the project, no clear distinction could be found between projects (Except two), therefore the incremental number of changes were also explored and difference became clearer as the less than successful performance projects did experience relative higher number of changes within a short time span during their engineering execution. Therefore, it can be concluded that changes combined with high incremental changes could lead to bad performance of a project. Cost of changes (COC): The costs of changes adopted during project execution do reflect differences in projects, at-least after 45 % engineering completion. From zero to 45% engineering duration, it is difficult to segregate between successful and less than successful performance of the projects (same as NOC). However, the projects, which kept on introducing changes even after 45 % engineering, ultimately resulted in undesired performance as compared to the project that experienced little or no additional cost due to changes Delay in purchase orders (DPO): The behavior of early warning DPO did not show a clear distinction between the Successful and less than successful performance projects. Along with successful projects, some less than successful performance project did not experience any significant delay in issuance of purchase orders. However, a deeper view on the projects provided more insights into the behavior of DPO. Some less than successful performance projects issued their PO before the start of engineering phase of the project, due to either favorable market conditions or the equipments had long lead times. Despite the project strategic decisions, it can be concluded that the delayed issuance of

7

For more explanation, please see section 5.7

84

®

purchase order does affect the project performance as the related vendor information might be delayed8. In addition, the strategic advantage gained by early issuance of PO could be lost if changes during detailed engineering impact the vendor equipments or services because that will cause additional cost and schedule claims by the vendor and contribute to the cost and schedule overrun of a project. Concurrent engineering (CE): The early warning CE reflects the change in concurrency level between process and piping engineering, i.e. what is the relative change in behavior of piping engineering progress w.r.t process engineering. The analysis suggests that the in case of less than successful performance projects (Shown in Red) the piping engineering progress curve did came close to process engineering curve during early engineering, which implies that they worked on concurrent engineering model9. However later, three out of five less than successful performance projects were not able to sustain the concurrency level and piping engineering subsequently projected schedule delay.

7.3 QUANTITATIVE ANALYSIS The analysis and exploration of data in previous section did provide us with few encouraging results. The analysis suggested that the six early warnings did behave differently in case of successful and less than successful performance projects. In this section, the quantitative analysis is performed to find the correlations between EWI and project outcomes. The data from eight projects was collected at baseline, during detailed engineering (six data points) and at end of the project10. The sample size contained eight projects, which is not considered as significant for quantitative analysis; therefore, to obtain firm quantitative results more data would be required. Since the data contained non-normally distributed variables, spearman’s rho correlation was used. The following “classification” for the correlation strength was used (Cohan and Holiday, 1982) to classify correlation strength. 0.9 to 1: Very strong correlation 0.7 to 0.9: Strong, high correlation 0.4 to 0.7: Moderate correlation 0.2 to 0.4: weak, low correlation 0 to 0.2: weak to negligible correlation

8

The conclusion is drawn on the basis of information provided by interviewees and explained in section 5.7

9

High concurrency implies that the piping engineering has to work on assumptions due to non-availability of information from process engineering. For detailed explanation please see section 5.7

10

For detailed explanation see section 0

85

®

Table 4 shows the correlations between project outcomes and PH at 95% engineering completion. A strong and significant correlation was found between PH and ESI (rs = 0.74, p = 0.0366), Very strong correlation between PH and MCI (rs = 0.97, p = 0.001), and between PH and TIC (rs = 0.69, p = 0.058). A moderate correlation with 85 % significance was found between PH and MHI (rs = 0.55, p = 0.15). Table 4: Correlation between growth in process engineering and project outcomes EWI

Project outcomes

Correlation strength (rs)

Significance (P)

ESI

0.74+++

0.03

Growth in progress hours results into delay in 95 % engineering.

MCI

0.97++++

0.001

Growth in progress hours results into delay in mechanical completion of the project.

MHI

0.550*

0.15

Growth in progress hours results into growth in total engineering manhours.

TIC

0.69+++

0.05

Growth in progress hours results into growth in TIC of the project.

PH

Meaning of correlation

“++++” significant at 0.01 level, “+++” significant at 0.05 level, “++” significant at 0.1 level, “*” significant at 0.15 level, N = 8 The presence of these correlations with high strength and significance suggested that growth in process engineering hours indeed affect the project outcomes and ultimately project performance. Table 5 below shows the correlations between delay in process engineering (PS) and project outcomes. The negative sign in correlations means that delay in process results into growth in project outcomes. A high and significant correlation was found between PS and ESI (rs = 0.68, p = 0.15) and between PS and TIC (rs = -0.75, p = 0.03). Very high and significant correlation was found between PS and MHI (rs = -0.93, p = 0.007). However no significant correlation was found between PS and MCI (rs = -0.52, p = 0.18) Table 5: Correlation between delay in process engineering and project outcomes EWI

Project outcomes

Correlation strength (rs)

Significance (P)

ESI

-0.68++

0.06

MCI

-0.520

0.18

MHI

-0.93++++

0.00

Delay in process engineering results into growth in engineering man-hours.

TIC

-0.75+++

0.03

Delay in process engineering results into growth in TIC of the project.

PS

Meaning of correlation Delay in process engineering results into delay in 95 % engineering completion. Delay in process engineering results into delay in mechanical completion of project.

86

®

“++++” significant at 0.01 level, “+++” significant at 0.05 level, “++” significant at 0.1 level, “*” significant at 0.15 level, N = 8 The above correlation suggests that delay in process engineering does affect the project outcomes (3 out of 4) and ultimately project performance. The absence of significant correlation between PS and MCI can be interpreted as that delay in process engineering might be resulted into deployment of additional resources, which resulted into high manhours and high TIC of the plant but did not impacted the MCI. Table 6: Correlations between number of changes and project outcomes EWI

Project outcomes

Correlation strength (rs)

Significance (P)

ESI

0.79+++

0.02

MCI

0.83+++

0.01

MHI

0.6+

0.10

Higher number of change orders resulted in growth in engineering manhours

TIC

0.520

0.18

Higher number of change orders resulted into growth in TIC of the project.

NCO

Meaning of correlation Higher number of change orders resulted into delay in completion of 95 % engineering Higher number of change orders resulted into delay in mechanical completion of project.

“++++” significant at 0.01 level, “+++” significant at 0.05 level, “++” significant at 0.1 level, “*” significant at 0.15 level, N = 8 The above Table 6 provides the relationships between number of change orders and project outcomes. The correlation analysis suggests that there is strong and significant correlations between NCO and ESI (rs = 0.79, p = 0.02) and NCO and MCI (rs = 0.83, p = 0.01). Moreover, moderate, low significant correlation between NCO and MHI (rs = 0.6, p = 0.10) and NCO and TIC (rs = 0.52, p = 0.18). The correlations strength and values suggests that the high number of change orders does affect the schedule outcomes i.e. 95 % engineering completion and mechanical completion of the plant. However, does not have significant relationship with growth in manhours and growth in TIC of the plant. The Table 7 below shows the correlations between cost of changes and project outcomes. In case of COC strong and significant correlation was found between CCO and TIC of the plant (rs = 0.69, p = 0.05) and moderate and low significant correlation between CCO and MHI (rs = 0.52, p = 0.15). The correlation implies that the cost of changes does affect the project outcomes associated with cost i.e. MHI and TIC of the project. In addition, it implies that cost of changes does include additional engineering man-hour cost and cost does not only include material cost. Table 7: Correlations between cost of changes and project outcomes EWI

Project outcomes

Correlation strength (rs)

Significance (P)

Meaning of correlation

87

®

ESI

0.240

0.57

Changes with high cost results into delay in completion of 95 % engineering

MCI

0.350

0.39

Changes with high cost results into delay in mechanical completion of the project

MHI

0.550*

0.15

Changes with high cost results into growth in total engineering manhours

TIC

0.69++

0.05

Changes with high cost results into growth in TIC of the project

CCO

“++++” significant at 0.01 level, “+++” significant at 0.05 level, “++” significant at 0.1 level, “*” significant at 0.15 level, N = 8 The Table 8 below shows the correlations between delay in purchase orders and project outcomes. The negative sign in correlations implies that the delay in purchase orders results into growth in project outcomes. The analysis found a very strong and highly significant correlation between delay in DPO and MHI (rs = -0.99, p = 0.001). A strong and significant correlation was found between DPO and TIC (rs = -0.73, p = 0.03). Moderate the low significant correlations exits between DPO and ESI (rs = -0.50, p = 0.20) and between DPO and MCI (rs = -0.49, p = 0.21). The correlations strength and significance suggests that delay in issuance of purchase orders resulted into growth in cost aspects of the project i.e. MHI and TIC, which could imply that the projects intend to stay on schedule by either working on assumption, due to delayed vendor information. Subsequently, then spend additional manhours in re-verifying the assumption or re-working in case the assumptions were wrong and accepted high growth in manhours and final TIC of the plant. Table 8: correlations between delay in purchase orders and project outcomes EWI

Project outcomes

Correlation strength (rs)

Significance (P)

ESI

-0.50

0.2039

MCI

-0.490

0.2199

MHI

-0.99++++

0.001

TIC

-0.73+++

0.0396

DPO

Meaning of correlation Delay in purchase orders resulted into delay in completion of 895 % engineering. Delay in purchase orders resulted into delay in mechanical completion of the project. Delay in purchase orders resulted into growth in total engineering manhours of the project Delay in purchase orders resulted into growth in TIC of the project

“++++” significant at 0.01 level, “+++” significant at 0.05 level, “++” significant at 0.1 level, “*” significant at 0.15 level, N = 8 The Table 9 below shows the correlations between the final changes (at completion) in concurrency between process and piping engineering and The positive sign in the correlations means that positive change (low planned concurrency level) results into growth in project outcomes.

95 % engineering project outcomes. concurrency w.r.t A moderate and

88

®

significant correlation was found between CE and MHI (rs = 0.64, p = 0.08). This means that the delay in progress of piping engineering due to delay in inputs from process engineering ultimately results into growth in engineering man-hours. Table 9: Correlation between Change in concurrency and project outcomes EWI

Project outcomes

Correlation strength (rs)

Significance (P)

ESI

0.140

0.73

MCI

0.20

0.62

MHI

0.64++

0.08

TIC

0.380

0.35

CE

Meaning of correlation Significant change in concurrency between process and piping engineering results into delay in completion of 95% engineering. Significant change in concurrency between process and piping engineering results into delay in mechanical completion the project Significant change in concurrency between process and piping engineering results into growth in total engineering man-hours. Significant change in concurrency between process and piping engineering results into growth in final TIC of the project.

“++++” significant at 0.01 level, “+++” significant at 0.05 level, “++” significant at 0.1 level, “*” significant at 0.15 level, N = 8

7.4 CORRELATIONS OVER ENGINEERING DURATION The analysis of correlations over engineering duration indicates that not all the EWI have correlations with all the project outcomes at all prediction moments. The summary of significant correlations w.r.t each prediction moment is provided in Figure 20. Furthermore, the correlations per project outcome over engineering completion are provided in Figure 21 . The following paragraphs discuss the analysis of correlation in Figure 20 and Figure 21. The overview of the significant correlations in figure 44 indicates that the number of correlations increased as the engineering progress i.e. minimum correlations at 15 % engineering and maximum correlations at 75% engineering completion. This observation could be explained, based on the behavior of EWI over engineering duration. As explained in section 7.2, the difference in behavior of EWI becomes more significant in successful and less than successful performance projects as the engineering progress. The correlations did seem to be in accordance with the behavior of EWI over engineering duration, for example PS & PH found to be highly correlated start from 15% engineering and NCO become significant, after 45% engineering completion. Another interesting observation is that throughout the engineering duration CCO, CE and DPO were not found to be significant for schedule outcomes i.e. ESI and MCI. This could imply the schedule impact of changes was considered less than actual impact (conservative estimate) and additional efforts were made to absorb the schedule impact of higher concurrency and delay in purchase orders. The effects of conservative schedule estimate of changes and optimistic planning reflected in increase in MHI and TIC (Which found to be correlated with CCO, CE and DPO after 45% engineering).

89

®

Looking at the incremental changes in EWI, the time duration before and after 45% engineering completion seemed to be critical. Significant correlations were found between project outcomes, PH, and CCO. This could imply that most of the project with less than successful performance did has significant incremental growth in PH and CCO around 45% engineering duration. The figure 45 provides the overview of significant correlations per project outcome over engineering duration. It was observed that the cost related project outcomes i.e. TIC and MHI have more correlations in general as compared to schedule project outcomes i.e. ESI and MCI. The phenomena could imply that either the schedule impact of changes, delay in purchase orders and high concurrent engineering was seen optimistically or ignored until 60 % engineering completion or schedule was given priority over cost. The ESI found to be consistently correlated with PH. This could lead to the inference that the problems in process engineering do produce downstream cascading effect and delay the subsequent engineering disciplines. The change in concurrency only become significant after 60 % engineering, And found to be positively highly correlated with MHI. This could mean that until 60% engineering, the piping engineering tried to work on assumptions and keep its schedule as planned. However, when it could not sustain the concurrency due to changes in assumptions (and need rework) and/or need concrete information to issue documents it project the growth in man-hours.

90

Significant correlations among Project outcomes and EWI EWI

ESI

15% Engineering completion MCI MHI TIC

30% Engineering completion ESI MCI MHI TIC

0.79+++

0.64++

CCO

ESI

45% Engineering completion MCI MHI TIC 0.64++

ESI

60%Engineering completion MCI MHI TIC

0.83+++

0.67++

ESI

75% Engineering completion MCI MHI TIC

0.74+++

0.67++

0.59+

CE -0.76+++

DPO

-0.69++

PH

-0.56+

0.6+

PS

-0.74+++ -0.81+++

0.62+

-0.9++++

-0.89++++

0.83+++ -0.75+++

0.66++

0.79+++

-0.66++

-0.86++++ -0.9++++

0.64++

-0.6+

-0.67++

0.65++

0.74+++

0.62+

0.62+

0.83+++

0.88++++ 0.71++

0.83+++

0.62+

0.74+++

0.87++++ 0.57+

NCO

0.74+++

-0.64++

-0.69++

0.62+

0.74+++

-0.62+

0.62+ -0.57+

Significant correlations among Project outcomes and incremental change in EWI EWI

ESI

15% Engineering completion MCI MHI TIC

ESI

30% Engineering completion MCI MHI TIC

ESI

45% Engineering completion MCI MHI TIC 0.64++

CCO

ESI

60% Engineering completion MCI MHI TIC

0.88++++ 0.74+++

ESI

75% Engineering completion MCI MHI TIC

0.8+++ 0.76+++

CE -0.6+

DPO

0.69++

0.77+++

0.55+ 0.8+++

NCO 0.57+

PH -0.86++++

PS ++++ +++ ++ +

0.001