The Delphi method is based on structural surveys and makes use of information from the experience and knowledge of the participants, who are mainly experts. It therefore yields both qualitative and quantitative results and draws on exploratory, predictive even normative elements.
A single Delphi methodology exists but the applications are diverse. In the most common form, the opinions sought concern the particular developments that are likely to take place. Such Delphis have been widely used in Technology Foresight studies and other exercises. But there are many other types of Delphi exercise possible. They may focus on different topics – on social developments, for example. Instead of trying to forecast the time scales of particular developments, Delphis can be constructed to help identify and prioritise policy goals, for example.
There is agreement that Delphi is an expert survey in two or more ’rounds’ in which, in the second and later rounds of the survey the results of the previous round are given as feedback (Cuhls 1998). Therefore, the experts answer from the second round on under the influence of their colleagues’ opinions, and this is what differentiates Delphi from ordinary opinion surveys. The idea is that the respondents can learn from the views of others, without being unduly influenced by the people who talk loudest at meetings, or who have most prestige, etc. Ideally, significant dissenters from a developing consensus would be required to explain their reasons for their views, and this would serve as useful intelligence for others. Unfortunately, to carry this out is time consuming, and is quite often missed out from Delphi studies.
Thus, the Delphi method is a ‘relatively strongly structured group communication process, in subjects, on which naturally unsure and incomplete knowledge is available, are judged upon by experts’, write Häder and Häder (1995, p. 12). Giving feedback and the anonymity of the Delphi survey are important characteristics. Wechsler describes a ‘Standard-Delphi-Method’ in the following way: ‘It is a survey which is steered by a monitor group, comprises several rounds of a group of experts, who are anonymous to each other and for whose subjective-intuitive prognoses a consensus is aimed at. After each survey round, a standard feedback about the statistical group judgement calculated from median and quartiles of single prognoses is given and if possible, the arguments and counter arguments of the extreme answers are fed back…’ (Wechsler 1978, pp. 23f.).
Characteristics of Delphi are therefore specified as (see e.g. Häder and Häder 1995):
• Delphi studies always tackle issues formulated in statements about which uncertain and incomplete knowledge exists. Otherwise there are more efficient methods for decision-making.
• Delphi involves making judgments in the face of uncertainty. The people involved in Delphi studies only give estimates.
• The experts involved need to be selected on the basis of their knowledge and experience so that they are able to give a competent assessment. They have the opportunity to gather new information during the successive rounds of the process.
• The Delphi method stresses the psychological processes involved in communication rather than mathematical models (Pill 1971 p. 64; Dalkey 1968 and 1969; Dalkey, Brown, Cochran 1969; Dalkey, Helmer 1963; Krüger 1975).
• Delphi tries to make use of self-fulfilling and self-destroying prophecies in the sense of shaping or even ‘creating’ the future.
Resources
Usually, the goal (and the result) of a Delphi study is to organise a debate, to collect and synthesise opinions and to achieve a degree of convergence. It is a valuable tool for communication and for exchanging opinions on a topic, making experts’ tacit knowledge of the future more explicit. It is also useful for longer-term assessments where extrapolations make no sense. It can help to gather the opinions of a larger group of experts and in fields where there is not a lot of evidence about the developments and where experts often do not dare to explain their real opinion. It is conducted anonymously in order not to let anyone lose face in the event of a change of opinion. The methodology is designed to avoid domination by particular individuals.
Normally, the number of respondents is small. Delphis are not intended to produce statistically significant results. In fact, the outcomes provided by a panel do not predict the response of a larger population or even a different Delphi panel. The outcomes represent the view of a particular group of experts.
The Delphi method is mainly used when long-term issues (up to 30 years) have to be assessed. It is a useful means of looking at emerging developments where there is no empirical database, where external factors are likely to have a determining effect and where social arguments may dominate economic or technical considerations. As it implies identifying statements (topics) that are relevant for the future, it reduces the tacit and complex knowledge to a single statement and makes it possible to judge. On the other hand, in more complex issues, when the themes cannot be reduced that much or when thinking about and discussing alternatives are the major target, Delphi is not the method of choice. It is also suitable if there is the (political) will to involve a large number of people in processes (Eto 2003).
The Delphi method seems to have managed to withstand the pressures of the changing challenges of the past 50 years. It has been able serve different perceptions of forecasting or Foresight and was probably understood by the users as being an appropriate way to tackle technical perspectives, organisational perspectives, and also personal perspectives. The individual could express a distinctly different opinion as compared to the group perspective and this to a differing degree between the technical details under scrutiny. As multiple perspectives are recommended for decision-making (Linstone/Mitroff 1994; Linstone 1998), the Delphi technique seems to have an appeal in quite diverse situations at the the longer-range end of the scale. As controlled scientific experiments have shown that Delphi estimates are no better than those of other consensus-oriented methods (Woudenberg 1991), it must be the communicative force of Delphi approaches that facilitates the switching between different perspectives. Indeed, Delphi is no longer used to create consensus but to test if there is already consensus about the shape of things to come. What users like in particular are the sets of data about the future that are gathered. Writing down future topics seems to have an immense psychological effect because it transfers implicit and tacit knowledge to more visible, explicit, and therefore transferable knowledge.
The major users are companies, particularly strategy departments. In the case of national Delphi studies, the target audience is usually defined as anyone interested in information about the future so, along with companies, the major users tend to be research institutions, ministries, journalists, teachers, students and pupils. This formalised and traceable method has credibility with policy-makers. The panel of respondents includes experts from business, government, research, associations and other persons competent in the field of subject, but usually the term expert is used in a relatively broad sense.
Approach
Selection of the topic: The subject should be one where there is a lack of hard data on future trends. In some cases, one thematic field is enough, in many cases the aim is to get an overview so that more fields are decided on and handled in a flexible way. There is always the possibility to add, remove or re-name fields.
Definition of the procedure: The whole procedure has to be fixed in advance. Will panel meetings be set-up or will the teams work on-line? Is the questionnaire an electronic or a paper one? This means that logistics (whether setting up the website or typing in results from the paper versions) have to be organised. Will there be follow-up workshops, interviews, presentations? If so, these also have to be organised and prepared. Printing of brochures, leaflets, questionnaire, reports also have to be considered.
When designing the questionnaire, it is important to consider from the beginning how to give feedback to the participants during the second round. The usual way is to provide percentages or graphics from the accumulated data in a similar way as in the first round questionnaire. However, there is much room for creativity, especially with online questionnaires.
Formulation of the statements: Respondents are often asked their opinion in relation to given statements, which can be derived either from desk based research or from previous workshop or working groups. This is a time-consuming process. It is often necessary to filter the topics twice or even three times because often, the experts in working groups add topics instead of reducing the number. The last step is formulation of the questions.
Formulation of the questions: The questions derive from the objectives of the Foresight exercise in general and have to be adapted to them. They should be clearly defined, possible to answer, and match the statements made. The statements have to be formulated in a way that the criteria or questions can be judged on the basis of them. Other questions may be related to the possible constraints (economical, technological, social and political) to the occurrence of event or development.
Selection of the panel of experts: Care is needed in recruiting the panel and the criteria for selection should be set out (problem of bias, which is present in every panel group). Before an expert agrees to take part in a Delphi inquiry, he/she should understand the purpose of the inquiry and should be aware that his/ her expertise should be made available in different rounds of the inquiry. The Delphi method has an iterative nature. If the exercise is to maintain its credibility the tendency for panel members to drop out after the first round should be minimised.
Pros and cons
Advantages:
• The formalisation of the methodology, the amount of data, the number of experts involved and the fact that diverging opinions are partially hidden behind the main converging one, make it popular and credible approach for policy makers.
• As with other well-formalised methods, it forces people to think about the future.
• It gives participants the opportunity to think in more depth and gather further information between the rounds (psychological effect).
• It highlights clearly whether there is consensus on an issue or not.
• There is a psychological effect and a communication effect in being forced to express ideas in a clear and concise way.
• The judgements allows for analyses, rankings and priority-settings.
• The output is in a form which is operational for many actors including policy makers.
• Even oriented towards action, Delphi surveys allow for longer-term thinking.
Disadvantages:
• Delphi studies are difficult to perform well. They are fairly time-consuming and labour intensive and require (external) expert preparation. They are therefore expensive.
• The consensus obtained in the second round is often artificial.
• However, there is a danger of regarding results as facts.
• Single opinions that might be of special value are also pooled and normally ignored. Only the accumulated results are published to preserve anonymity. It is difficult to find out reasons for dissenting answers later on, as this anonymity has to be respected.
• A poorly designed Delphi will provoke antagonism and elicit poor quality information. It may fuel criticisms of the overall Foresight activity with which it is associated. Therefore, a great deal of attention must be given to the choice of participants; the questionnaire must be meticulously prepared and thoroughly tested to avoid ambiguity.
• Care has to be taken over group effects. As in all panels or expert groups, the opinions will reflect the set of participants involved: a narrow set of criteria for these may lead to unrepresentative views or miss out important sources of knowledge.
• Some participants drop out during the process (especially after the first round). In addition, although further qualitative assessment of Delphi inquiry may produce useful information, this step is often not carried out due to lack of time.
• It is often difficult to convince people to answer a questionnaire twice or more and incentives may be needed (e.g. that the experts receive the results). The dropout-rate increases after the second or third round, so most current studies are limited to preparation and two rounds.
• A Delphi survey is actually always a mix of methods because a topic generation procedure is needed.
• It is not applicable in all fields or cases, because the statements have to be formulated relatively quickly. Even when it is applicable, this short formulation reduces the complexity.
Variations
Expert panel, Critical & Key Technology Study