Evaluation of Delphi Method - Coursework Example

Running Head: Evaluation of Delphi Method [Name] [Professor Name] [Course] [Date] Table of Contents Executive summary 2 1.0 Introduction 3 1.01 The Delphi Method 3 1.02 History of the Delphi Method 4 1.02 Application of Delphi 5 2.0 Key Characteristics of the method 6 3.0 The Delphi Process 8 4.0 Advantages and disadvantages of the Delphi method 10 4.01 Advantages 10 4.02 Disadvantages 11 5.0 Examples of Delphi Method 13 6.0 Conclusion 14 References 15 Executive summary The Delphi method set out with the formation of set of open-ended questions examining a specific issue. The questions are afterwards dispersed to various experts in the area of study. The responses to the questions made by these experts are then analyzed as a set of the second questions and then sent until eventual stability in results is reached. The purpose of this report is to evaluate the Delphi method with the hope of improving the performance of the company as well as saving money. The report examines the history of the method, the applications of the method, the main features of the method, the Delphi processes, the advantages of the Delphi method, the disadvantages of the Delphi method and finally, two example of Delphi method. 1.0 Introduction 1.01 The Delphi Method Most Delphi applications entail creative and reliable investigation of ideas aimed at ensuring the generation of information for informed decision-making. Typically, the Delphi Method is rooted on a structured process essential for collection and extraction of knowledge from a panel of experts using a succession of questionnaires containing controlled feedback or opinions (Rowe and Wright, 1999). Delphi Method signifies a constructive communication instrument among a ‘group’ of experts and hence aids in the creation of a ‘group’ judgment. It can further be described as a mono-variable technique for exploration of ideas that is helpful on forecasting. Technically, it was developed with the aim of facilitating discussions among experts while limiting particular social interactive behavior that is often evident in normal group discussions and that inhibits objective formation of opinion. In this method, decision-makers are forced to solely depend on their personal insights and expert opinions (Hsu and Sandford, 2007). The experts answer questions in two or several rounds and after each successive round, an unspecified review of the forecasts from the experts in the previous round and the rationale behind each forecast is read out by a facilitator. At this juncture, the experts are at liberty to review and modify their earlier responses based on the answers issued out by other experts within the group. At this point, the range of answers is expected to reduce while the groups involuntarily join to endorse the right answer. In the end, the process is halted after a pre-designed stop criterion, such as based on the stability of the results, a consensus or a specified number of rounds. The results are finally determined by the median or mean scores (Günaydin, n.d.). This technique, which is based on the rule of decisions or forecasts, has generally been used in business forecasting. From the above outline, it can be concluded that Delphi method relied on the postulation that group decisions or insights are more valid compared to individual ones.  1.02 History of the Delphi Method Originally, it was called Oracle of Delphi, but the name Delphi was later settled on where the term Oracle of Delphi was in return derived from a revered Oracle in the traditional Greek legends. Within the legends, advises and forecasts were sought from gods. Technology forecasting studies attributed to the creation of the Delphi method began in 1944. The need for the studies was prompted during the Second World War when General Arnold directed that technology forecasts for future military technological eminence be prepared for the US Army Air Corps. In 1946, an aircraft company (Douglas Aircraft in Santa Monica California) was commissioned to form a research and development (RAND) project called Project RAND. A range of approaches, including quantitative approaches, theoretical methods and extrapolation of trend, were explored over the next 13 years and in 1959, to combat the shortcomings associated with these explored methods, Project Rand researchers led by Olaf Helmer recommended the Delphi method (Günaydin, n.d.). This technique distinguishes human judgment as a proper and useful device essential for generation of forecasts. In all, it attempts to claim superiority over single expert decisions, which can often be subjected to bias. 1.02 Application of Delphi The Delphi method is applicable in situations where there is very limited or no historical data. From the outset, Delphi method found application in forecasting science and technology. Its objective was to integrate expert opinions on the possibility and predictable development time of a certain technology. Among such first reports to indicate its objectives were published by Helmer and Gordon and examined the enduring as well as future trends in the field of science and technology. As part of development in technologies, it examined possible scientific c breakthroughs, space technology progress, and weaponry and war prevention. Other areas of forecasting included robotics, broadband networks, roads network and intelligent internet (Hsu and Sandford, 2007.). Delphi method can be applied in the making of government policies in the areas of education, health and economic trends. It has been initially applied in business forecasting where it has also proved effective. For instance, it was able to predict the sale of new products within the first two years following its implementation, with inaccuracy of about 4 percent even as the conventional unstructured methods of forecasting had errors of nearly 20 percent. Delphi method has further been applied an instrument for implementing multi-stakeholder models for participatory policy-making. For instance, some countries have applied the technique as an open-ended private-public sector model to determine the most severe economic and social challenges (Günaydin, n.d.). The application of Delphi method has found relevance in policy-making, in the end introducing a range of methodological innovations such as the need for investigation of various aspects of an item, such as item forecasting, item options, item issues. It has also led to the introduction of various evaluation scales including the feasibility, probability and desirability (Rowe and Wright, 1999). Delphi method finds application in the evaluation of long-term issues. Given that it majorly entails a procedure for identification of topics applicable for the future, it has the capacity to reduce the unspoken and complex information to a simple statement. It also makes it possible to compare expert opinions. Conversely, it is effective in scenarios where complex issues with irreducible themes that require extensive thinking and discussions are often less effective. 2.0 Key Characteristics of the method The Delphi method is characterized by a group communication strategy among a group of experts based in different geographical regions. Using the technique, experts are allowed to deal methodologically with complex tasks, without a doubt, the technique is generally simple and clear-cut. It involves distributing questionnaires successively to a pre-determined panel of experts. This prevents the possibility of undue effects of face-to-face group discussions. The questionnaires are created to draw out individual responses to the issues of concern and to allow the experts to review their answers based on responses made by other experts as the group’s tasks progresses based on specified tasks. These successive review of responses or forecasts ensures that the experts do not stick to preconceived opinions and that they conform to established opinions based on opinions made by others (Stuter, n.d.). The underlying concept behind the technique is to prevail over the shortcomings of conventional group decision. The technique is characterized by controlled feedback, statistical responses and anonymity. In this case, the interaction of the experts is kept anonymous, meaning the originators of the forecasts, the comments and expressions are not identified although they are provided to the penal by a panel director or a facilitator in a way that ensures optimal anonymity. Accordingly, this prevents domination by some participants over the others. It also prevents participants from exercising their personal biases thus allowing for an open and objective critique. Generally, the Delphi method is essential for answering single dimension questions compared to determination of complex forecasts that involved a range of factors. It can however be pointed out that likely complex model building is suitable for quantitative approaches with Delphi results that serve as inputs. Therefore, the Delphi method, since it is based on a collection of expert opinions, forms the possibility that feedback on the forecasted ideas may fail to benefit from optimal consideration. However, this can be improved through integrating the Delphi method with a cross-impact matrix approach. 3.0 The Delphi Process Ten steps are involved in the Delphi method. An issue to be addressed is determined. A team of facilitators, directors or monitors is then selected to conduct, monitor and evaluate the Delphi on a specified subject. The team will organize, oversee and coordinate the Delphi technique. This will include selection of the experts and facilitation of the data collection and analysis of data. To ensure that the exercise remains transparent, the facilitator should be independent. In addition, stakeholders or some other participants may be included (Rowe and Wright, 1999). The experts in the specified subject or field being investigated are selected to participate in the exercise. The experts are confined within a panel or a group. They are often considered experts in the field and have vest experience. However, some instances have also been witnessed where some lay people are also included since they can bring in new ideas. The questionnaires for the first round of the Delphi method are created. It is important that the time the participants are expected to take in answering the questions be specified for easier planning. The questionnaires are examined for errors, ambiguities or proper wording. This is important in ensuring clarity of the questions and to avoid instances where the participants are misled into answering questions incorrectly. A typical question should examine an issue and enable the participants to brainstorm as well as to detail out their ideas. However, they do not have to justify their answers. The first round questionnaires are passed or dispersed to the panelists. The response forms should be coded to enable the facilitator to know who has participated and which participant should be followed for response (Stuter, n.d.). The responses in the first round of the Delphi method are analyzed. Analysis should include identification of common or contradicting viewpoints The second round questionnaires are prepared and tested. The second questionnaire should be based on the analysis of their responses. The objective of the second questionnaire is to attain stability in results or a shared understanding of how the experts view the issue. At this juncture, the participants could be requested to rank or sort their responses. The second round questionnaires are also passed or transmitted to the panelists. The second round responses are analyzed, steps ‘g’ and ‘h’ can thereby be repeated as necessary to attains stable results. Thus, the questionnaires should be sent until it is determined that no significant idea can be created by the participant any longer. A report is prepared by the analysis team to explain the findings in the Delphi method. The report should as well be sent to the participants. Here, the names of the participants should be withheld, in any case, no one who participated in the study should be revealed. In the report, the underlying assumptions on the decisions elected should be emphasized. Within the ten processes outlined, the fundamental issue is the need to understand the objective of the Delphi method by the participants without which the participants may give improper responses or lose interest. Also, the experts selected should be well informed in the specified subject of interest. The number of the participants should depend on the study design although in an ideal circumstance, a minimum of four experts is appropriate (Stuter, n.d.). 4.0 Advantages and disadvantages of the Delphi method 4.01 Advantages A major advantage of the Delphi method is the anonymity of participants and application of questionnaires. Both of these avoid the issues identifiable with group interviews, such as possible persuasions, dominance, the bandwagon effect and verbal expressions. The method has also proved effective and very popular as given the number of participant experts, the quality and quantity of data and the reality that divergent opinions are partially overlapped by the main converging ones. The Delphi technique also ensures the panel of experts reflects on reasoned opinions, as it compels the participants to contemplate the issue under study logically and to offer a written feedback. In which case the consensus reached by the panel of experts depicts a seasoned and reasoned opinion (Rowe and Wright, 1999). The technique also applies group decision-making techniques through enabling the experts in a field of specialization to participate. The decisions by the groups convey greater authority and validity compared to those made by individuals. Since it may be difficult to assemble the experts at any one time, Delphi method enables reaching these experts at their specific areas of work, thus expert opinions can be collated from a panel of experts residing in different geographical regions. In addition, the isolated creation of ideas can result in high quality of validated data. Physical perusal and reading the questions in the questionnaire enables the experts to take their time and offer high quality responses, as compared to face-to-face methods. Since the expert participants are compelled to consider a solution from their personal angle, they cannot react to responses contributed by others within the group. Thus the nature of search is proactive. The responses or forecasts made in the Delphi study permits setting of priorities, rankings and analysis. Compared with other formalized methods, Delphi method compels the participants to contemplate the future. It further offers the participants an opportunity to think consciously and deeply, and to collate information over successive rounds (Rowe and Wright, 1999). 4.02 Disadvantages Several shortcomings are associated with the Delphi method. First, this technique is vulnerable to delays as it is significantly time consuming. It is also labour intensive and assembling the required number of experts may take time. They also require significant external and internal preparations. In summary apart from being time-consuming, they can also be expensive. Often, the group effect has to be treated with precaution. Typically, in a panel or group of experts, the ideas issued as responses will often reflect the kind of the participants. A constricted set of criteria may miss out on significant sources of information or misleading opinions. Individual opinions that might have significant value are often assembled and ignored. As a result, only the selected results will be published under the conditions of anonymity. It is also possible that some participants in the Delphi study may drop out when the process is still ongoing, particularly after the first round. Additionally, even as further qualitative study of Delphi inquiry may generate useful information, the process is often neglected due to time constraints. In most cases, convincing the experts to answer the same questionnaires more than once may be difficult. The dropout rate may further increase in the next round thus limiting the study to few rounds. In which case, the stability in the results may not be attained. Given that the procedure to generate the topic of study is necessary, Delphi survey is practically a mixture if methods. Since the participants do not meet, there is the difficulty in appraising and leveraging the expertise of the panel extensively. Greater dependence might also be placed on the responses than it might be appropriate. It is hence crucial to point out that a consensus may not fully guarantee the correct answer, rather, the technique and the results should be applied to structure group discussions and to raise issues of debate (Günaydin, n.d.). 5.0 Examples of Delphi Method a) Tourism Demand Forecasting: South Australia Delphi technique can serve as an integrative approach in forecasting the trends of tourism. The Delphi study has been used in the evaluation of the results of tourism forecasting in South Australia between the period 1996 and 1998. The technique was significant in forecasting the tourism demand in the domestic and global market. The study used three techniques to generate time-series predictions. A group of 26 experts from the tourism industry were selected to respond to a set questionnaire, after which comments in successive group meeting processes were made. The ultimate forecasting technique was based on the responses of the Delphi process. The grouped forecasting trends for the international tourism proved accurate although the forecasts for segments, for instance visitors from specific countries, were less accurate. Overall, Delphi method in determining the demand forecasting proved accurate, thus confirming results from previous studies. b) Technology Infrastructure Forecasting in West Virginia High Schools Delphi method was used to investigate the technology infrastructure of West Virginia High Schools, and whether it will have an impact on the school’s education system, and whether there was the need to rehabilitate the existing school infrastructure of construct new one. In the first round, the questionnaires were sent to 21 practitioners representing West Virginia’s public education via email (Carman, 1999). While responding to the first round, the panel of experts generated 21 varied technology infrastructures, which they believed could be integrated into the construction of a new school or rehabilitation of the current structure. In the second round, the participants were asked to rate the life-span of the existing infrastructure using the Likert-like scale. In the second round, only 18 experts responded. In the third round, the facilitator sent a copy of the results indicating a determined consensus. They were further issued with a copy of their answers and asked to either disagree or agree with the consensus, or alternatively, specify a reason justifying their initial answers. Later, scenario was formulated through the time-frames compared to Round 3 and Round 2. The end conclusion of the study was that the technology infrastructure that was viewed as the most enduring and affordable was the development of staff. In the study, the education and business experts reached a consensus on the conclusion. The Delphi method appeared to have established that despite all the technology the school could implement into its program, the most important of them all was training of individuals (Carman, 1999). 6.0 Conclusion In conclusion, the Delphi method can find application at all levels of business decision-making processes. The technique is rooted on a structured process essential for collection and extraction of knowledge from a panel of experts using a succession of questionnaires containing controlled feedback or opinions. Its advantages and superiority over other data collection and decision-making methods has made it prove effective and very popular in the decision making and forecasting processes. References Carman, W.(1999). An Application of The Delphi Method Of Forecasting To The Future Of Technology Infrastructure In West Virginia High Schools. West Virginia: West Virginia University. (Online). Retrieved from: [http://wvuscholar.wvu.edu:8881//exlibris/dtl/d3_1/apache_media/L2V4bGlicmlzL2R0bC9kM18xL2FwYWNoZV9tZWRpYS8yMDY0Mg==.pdf] Günaydin, M. (n.d.) The Delphi Method. (Online) Retrieved from: [http://web.iyte.edu.tr/~muratgunaydin/delphi.htm Stuter, Lynn. (n.d.). The Delphi Technique: How to achieve a workable consensus within time limits. (Online) Retrieved from: [http://www.seanet.com/~barkonwd/school/DELPHI.HTM] Hsu, C. & Sandford, B. 2007. “The Delphi Technique: Making Sense Of Consensus.” Practical Assessment, Research & Evaluation, Vol 12, No 10. Retrieved from: [http://pareonline.net/pdf/v12n10.pdf] Rowe, G. & Wright, G. (1999). “The Delphi technique as a forecasting tool: issues and analysis”. International Journal of Forecasting 15, 353–375 . (Online) retrieved from: [http://www.academia.edu/214235/The_Delphi_technique_as_a_forecasting_tool_Issues_and_analysis] Tideswell, C., T. Mules and B. Faulkner. 2001. "An Integ.rative Approach to Tourism Forecasting: A Glance in the Rearview Mirror." Journal of Travel Research 40 (November 2001):162-171. Retrieved from: [http://www.uk.sagepub.com/chaston/Chaston%20Web%20readings%20chapters%201-12/Chapter%205%20-%2012%20Tideswell%20et%20al.pdf] Read More