Posts Tagged ‘learning effects’

EFP Brief No. 236: Assessing Dutch Defence Needs Follow-up

Friday, December 21st, 2012

Under the influence of (inter)national technological, political and economic developments, the Dutch defence industry is increasingly intertwined with and developing towards a civilian industry. Consequently, the political responsibilities, atti-tudes and criteria are changing for both the Ministry of Defence and the Ministry of Economic Affairs. An analysis of the Dutch defence industry helped to determine the main opportunities for innovation in the industry and to identify the com-plementary technological competences needed to make the most of them. A strategic vision, including options for innova-tion policy, was developed as well. In this follow-up brief, we reiterate the background, approach and results of the initial foresight study and describe its impact in the years to follow.

Transition of Defence

Historically, “defence” supports national strategy, in which nations have built their own forces, defence industry and knowledge infrastructure. Consequently, within nations there arose a demand driven chain with a solid and confidential relationship between the parties in a closed chain, also discerning the industry from ‘civil’ industries. However, technological, political and economic developments in the last twenty years are changing defence radically. Issues such as the end of the Cold War, decreasing budgets, international cooperation, international organization of forces, industries and knowledge infrastructure, growing use of civil technologies, civil industries and civil markets, ‘the war on terrorism’, and homeland defence have entered the stage. Consequently, the political responsibilities, attitudes and measurements are changing for both the Ministry of Defence and the Ministry of Economic Affairs, while the defence industry and knowledge infrastructure is increasingly intertwined and developing towards a civil industry and knowledge infrastructure. This critical transition of the defence chain demands timely strategic information and a vision to anticipate effectively. For ministries this means a clear view on responsibilities, effective investment strategies for a capable future force and an effective industry and innovation policy. The defence industry increasingly has to deter-mine their most favourable innovative possibilities.

Developing a New Strategic Vision

As a result, the ministries wanted to assess four is-sues/developments and formed working groups to prepare the strategy. Four groups were formed to

– Inventory the relevant international developments,

– determine success factors of international cooperation in procurement,

– determine priority technological areas for the defence industry which are for interest for the domestic market, and

– policy instruments to strengthen the strategic vision.

The third question concerning the identification of priority technological areas was the core issue in this project and divided into four sub questions:

  1. What are the current strengths of the Dutch defence industry?
  2. What are international opportunities for innovation in the defence market?

Structural Approach Based on Clusters

The challenge of the exercise was to systematically translate the four sub questions into perspectives on technological clusters or innovation opportunities. This makes the outcomes comparable. Every perspective was analysed and then translated into a codified taxonomy of technologies developed by the Western European Armaments Group (WEAG); this WEAG-classification on defence technologies is generally accepted within the defence sector. This taxonomy includes technology, products and intelligence or as they are called ‘underpinning technologies’, ‘systems-related technologies’ and ‘military assessments, equipment and functions’.

Additionally, the WEAG-classes were checked for interrelation such that priority clusters are formed and interpreted, which seem to combine specific technologies with products and intelligence. Finally, these priority clusters are compared such that a final reflection is made from the four different perspectives (see figure 1).

For determining the strengths of the defence industry, companies were analysed and a computer aided workshop including the industry was organized (Group Decision Room). The innovative opportunities were inventoried based on desk re-search and interviews with leading parties. Future needs of the military forces were inventoried and weighted based on al-ready planned investments by the Ministry of Defence. Finally, the civil market was assessed by experts based on most relevant societal challenges.

Below the analysis on current strengths is elaborated. For foresight purposes, the results on innovative opportunities are also included.

Outcomes: New Paradigm of Effectiveness

Military operations are increasingly operations other than war, such as peace operations, foreign humanitarian assistance and other military support to civil authorities. Consequently, governments turned their focus on the ultimate goal of ‘effect-based [security] operations’. In practice, effect-based operations imply a joint and combined cooperation between different armies and forces resulting in a transformation of a plat-form-centric force into a network-centric force. The term “network-centric warfare” or “network enabled operations” broadly describes the combination of emerging tactics, techniques, and procedures that a fully or even partially networked force can employ to create a decisive advantage. On the whole, the defence sector still innovates on platforms, weaponry and increasingly on intelligence. Figure 3 below shows all innovation themes which are on the agenda of the defence sector.

Innovation themes are divided into underlying innovative opportunities, translated in the WEAG-classification and finally clusters are identified. The main clusters are C4I, sensor systems and integrated system design and development.

Information Based Services

The clusters arising from the four perspectives are compared with each other to identify the main clusters. Table 3 below shows the synthesis.

Type 1 clusters can be regarded as broad, strong clusters, with a good industry base and market potential in domestic, inter-national and civil markets. This first type of cluster represents information based services for the Dutch industry. Type 2 clusters cover a couple of interesting niche markets. Finally, type 3 clusters are fragmented but might have some niches.

Original Brief Impact Discussion

In the 2007 brief, some of the impact of the foresight study was already visible and described:

The project was on a highly political trajectory, where the interests of industry and the ministries of Defence and Economic Affairs were intertwined. Also being a part of a broader process and the project delivering the content for just one of four working groups led to intensive discussions within the interdepartmental group before the results could be used as input to the national strategy for the defence industry. This, together with the change of government, considerably prolonged the finalization of the strategy.

About one year after the finalization of the project, the ministries determined their Defence industry strategy. The results of the project were largely integrated into the strategy and therefore had a high impact. The technological priorities stated were fully accepted and provided the backbone to the suggested defence innovation policy. The strategy was discussed in Parliament and will be part of the national policy on the defence industry.

A Follow-up Foresight Study

As noted, the results from the 2006 foresight exercise were integrated in the Dutch Defense Industry Strategy of 2007. However, since 2007 the strategic context in which this industry sector operates has changed significantly. New forms of conflict arise, that demand new kinds of response (e.g. cyberdefense), closer cooperation with coalition partners requires further integration of systems, the financial crisis has had an impact on defense budgets, and finally there is a clear movement to an open and transparent European defense market.

These strategic changes has prompted the Dutch Defense Ministry to evaluate the Defense Industry Strategy that was formulated in 2007. A key part of this evaluation is a follow-up foresight exercise to the foresight exercise of 2006 described earlier in this brief. In the original foresight exercise, research was done on three questions with regards to the Dutch Defense Industry: (1) what is the Dutch Defense Industry good in? (2) What does the market need? (3) What does Dutch Defense need? Questions 1 and 2 were sufficiently answered, but changes in the strategic context require an update to these answers. The answer to 3 was less detailed, and still required a more extensive study.

This follow-up foresight exercise is planned for 2012, and will be performed by the Hague Centre for Strategic Studies and TNO. It aims to examine whether the identified technology clusters are still relevant, whether they need to be adjusted to extended, considering the developments in the last 5 years. The approach is mostly similar to the one of the previous foresight exercise.

Several other forward looking activities in the past 5 years provide key input for the follow-up foresight study, including an exploration to the Dutch Defense force of the future (Dutch Ministry of Defense, 2010), and a NATO study into the future of joint operations (NATO, 2011).

The follow-up foresight study will be shaped along three main topics:

Needs: the future needs of the Dutch defense are investigated, including innovation characteristics of (new) required capacities, attention to the speeding-up of the lifecycle of innovations and capacities, and the role of defense in this lifecycle of capacities and innovations.

Strengths: the strengths of the Dutch defense industry are analyzed using datasets gathered yearly by other organizations using interviews and surveys with industry organizations.

Opportunities: in interviews and focus group sessions the estimates that the Dutch defense industry make about their own future opportunities are analyzed. This analysis is accompanied by an international comparison and a separate analysis by the organizations performing the follow-up foresight exercise.

In a synthesis phase, representatives from ministries, industry and knowledge institutions will be brought together in a workshop session, in which the final conclusions and recommendations of the study will be formulated.

Conclusions

The foresight exercise described in the original brief had a high level of impact in a specific area: the Dutch Defense Industry Strategy. The study results have proven to be useful in formulating a defense industry strategy by the relevant ministries. This usefulness is further illustrated by the fact that a follow-up study was requested and has been initiated, which is expected to provide input for an update to the defense industry strategy.

Authors: Bas van Schoonhoven                                   bas.vanschoonhoven@tno.nl

Annelieke van der Giessen                 annelieke.vandergiessen@tno.nl

 
Sponsors: Dutch Ministry of Economic Affairs and Dutch Ministry of Defence  
Type: Single foresight exercise  
Geographic coverage: National (Netherlands)
Organizer: TNO – The Netherlands Organization for Applied Scientific Research (www.tno.nl)
Duration: Jan/Jul 2006 Budget: € 150,000 Time Horizon: 2015    
Date of original brief: Oct. 2007     Date of follow-up brief: Oct. 2012    

 

Download EFP Brief No. 236_Assessing Dutch Defence Needs_Follow-up.

Sources and References

Butter, M, J.H.A. Hoogendoorn, A. Rensma and A. van der Giessen (2006), “The Dutch Defence Outlook”, TNO.

Hoogendoorn J.H.A., Rensma A., Butter M., van der Giessen A., (2007), “Opportunities in Innovation for the Dutch Defence Industry”, EFMN Foresight Brief No. 120, available online at
http://www.foresight-platform.eu/briefs-resources/

(Dutch) Dutch Ministry of Defense, 2010, Eindrapport – Verkenningen: Houvast voor de krijgsmacht van de toekomst
http://www.defensie.nl/actueel/nieuws/2010/03/29/46153012/strategische_verkenningen_bij_defensie_afgerond

NATO, 2011, Joint Operations 2030 – Final Report
http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA545152

EFP Brief No. 234: Learning Effects of a Foresight Exercise: An Accompanying Social Research Study

Friday, December 21st, 2012

The purpose of the accompanying social research study to the Freightvision exercise (Brief No. 226) was twofold: First, we wanted to introduce a concept for accompanying social research of a large participatory foresight process in order to grasp immediate learning effects. Secondly, we analysed immediate learning effects in the course of a large participatory foresight process. The research questions guiding the empirical analysis were: How can we operationalise and measure learning in the context of a large foresight process? Learning thereby involves different levels of learning: individual learning, group learning, organisational learning, system-level learning etc. And how can we operationalise and measure networking, i.e. the establishment of personal ties that enable the exchange of information and hence learning in a large foresight process?

The Foresight Case Freightvision in Focus

The foresight case in focus intended to integrate new knowledge, perspectives and stakeholder groups into an established field. Creating channels for communication between participants from business, policy, civil society and R&D to overcome sectoral boundaries was an explicit goal from the beginning. Stakeholder participation in this case was defined as inviting representatives of research, business, policy and civil society explicitly as “experts” who take part in a strategic dialogue on long-term issues. The expertise of participants was sought as deliberative input and shaped the content and tangible results of the foresight process, leading to robust scenarios, recommended action plans, visions and background reports.
 
Given the large scale of the foresight exercise (up to 90 participants in four fora, budget >3 m EUR, duration > 3 years), deliberative participation was guaranteed through four large and highly interactive fora using large group intervention techniques derived from organisational development theory (world café, open chair discussion rounds, interactive poster sessions etc.). Methodologically, the Freightvision foresight assessed here relied on an overall architecture and methods of organisational development (OD) that focus particularly on changing the thinking and actions of stakeholders. The application of OD concepts and instruments throughout all phases of the foresight exercise was assumed to maximise interaction, collaboration and learning among stakeholders in this foresight system.

Methodologies of the Accompanying Social Research

Learning effects of foresight processes can occur in various dimensions, which we tried to capture in our accompanying social research study: i) the acquisition of social capital (e.g., establishing new contacts, building networks), ii) the acquisition of factual knowledge and understanding (new insights derived from discussions and multiple perspectives), and iii) the development of strategic alternatives (Amantidou & Guy, 2008). Following Lewin (1953), Schein (1995) and Grossman et. al. (2007), we distinguished and applied three different approaches of accompanying social research to analyse and assess the immediate learning effects of foresight. The three approaches were the practitioner model of field research, qualitative interviewing and content analysis.

Practitioner Model of Field Research

The accompanying research to evaluate the effects of the foresight process on participants and stakeholders was conducted by AIT – Foresight and Policy Development Department. The process involved 165 individual participants coming from private enterprises, interest groups representing the various transport modes, infrastructure providers, trade unions, environmental NGOs, research organisations and administration. Participation in Forum 1 to 4 was between 96 and 75 individuals.
 
In moderated workshops, we conducted a survey and several discussions as part of the foresight process. Within this foresight project group, organisational development (OD) researchers acted as counsellors trying to intervene in social systems in order to provoke change (Schein, 1995; Grossman, et al., 2007). In the context of these moderated workshops, the foresight counsellors and the foresight project group evaluated their roles during the stakeholder fora as well as other impacts by (1) reflecting on and adapting their own observations and patterns of intervention, (2) by evaluating the process as a whole and (3) by carrying out a qualitative survey of the project group in the moderated workshops after each stakeholder forum. The questions addressed mutual learning processes, short-term effects and the evaluation of the overall design and process of the stakeholder fora.
 

Participatory Ethnographic Research

According to Schein (1995) and Grossman et al. (2007), researchers participate in the day-to-day life of social systems yet try to minimise influence or set interventions. To capture various kinds of immediate impacts from the foresight case, telephone interviews were carried out after each of forum. Around 20% of the participants were interviewed by the research team, resulting in 71 interviews all in all (the interviews took 15-20 minutes each). Qualitative content analysis was applied to extract information from the interviews. The post-forum telephone interviews showed that participants were positive about the methodology. They were particularly positive about the high levels of interaction during the fora (working intensively in a productive atmosphere, using creative methods including wild cards and visualisation of the freight transport system in 2050), which helped the different stakeholder groups to better understand the motivations and backgrounds of various other stakeholder groups. The interviewees also mentioned that the project led to a systemic picture of the whole longdistance freight transport system across modes.
 

Experimental Social Research

In experimental social research, the observer implements a lab-like environment trying to minimise influence on the observed object. The research setting is designed to generate quantitative data that claims to describe “the reality of the observed object” apart from the observing researcher. In our case, a social network analysis (SNA) approach was applied, reducing the observed part of the complex communication and learning process to different categories of ties established between participants of the foresight fora. Assuming that actors are embedded in a web of social interrelations, SNA provides a set of methodologies and tools to understand internal communication, organisation and aspects of their formation (Heimeriks, Hörlesberger, & Besselaar, 2003; Coromina, Guia, Coenders, & Ferligoj, 2008). A questionnaire was designed and distributed both at the beginning and end of every forum, listing names of participants and asking each participant to quantify the level of acquaintance with all remaining ones. The difference in levels of acquaintance before and after every forum served as a proxy for the number and quality of ties established during the fora (qualitative and quantitative statistical network analysis was applied in order to extract information from the questionnaires).
 
The team of researchers conducting the accompanying social research were external observers. The network analysis based on pre- and post-forum questionnaires showed that the network of participants had already reached a high density after Forum II and that there were no signs of emerging closed clusters of unconnected sub-groups. New participants were integrated quickly (approximately one quarter were new in every forum), and the network density remained stable until Forum 4. Figure 1 shows a network of personal ties (or relationships) between participants based on personalised questionnaires returned a) before Forum I (March 2009, n = 41/96 questionnaires) and b) after Forum III (October 2009, n = 35/79 questionnaires). Stakeholders are coloured in black, all other project partners in grey. Geometric positions and distances are determined by the combined strength of a participant’s ties (participants are positioned closer if ties are stronger). The shape of a node is determined by the number of inward vs. outward ties and its volume by the total number of ties. Network “connectors” have more outgoing vs. incoming ties (ellipses pointed upward) and “authorities” vice versa (ellipses pointed sideward). All computations were performed using the software PAJEK (Chen, 2003).
 
 

Learning Effects

The immediate learning effect of a large-scale foresight project was analysed based on three methods of accompanying social research. First, the practitioner model was applied in an analysis of the foresight process in moderated workshops. Learning in this context mainly referred to the creation of cultural islands and increased the participants’ identification with the foresight process. Secondly, a qualitative analysis was conducted in an ex-postfacto analysis where individual learning resulting from the
foresight process in focus was captured in different questions.
 
The main result here is that the major achievement of a large participative foresight process with respect to learning is probably that details out of the social contexts and rationalities of various stakeholders add up to a multidimensional picture at the system level. This results in perceiving oneself as being part of a system and gives a clearer view of one’s own role in the system. Interdependencies between the various actors become more apparent, which on the whole results in a more comprehensive big picture at the system level. Thirdly, we tried to empirically grasp the increase of personal ties between participants of a large foresight process by means of a social network analysis. We assumed that these ties reflect some extent of exchange of information and hence can be expected to enable learning processes. Overall, the number of newly formed acquaintances more than tripled during the fora; the network diameter settled at a low size of three ties. A higher density, an average degree of centrality and a lower diameter reflect a higher flow of information. It becomes clearer how participants perceive their position within the network of stakeholders and their influence and future agendas (Schartinger et al., 2011).

Effects at European and National Level

A clearly discernible effect is the continued collaboration of the project team in the following FP7 calls, which can be attributed to the well-designed collaboration in the project team as active participants in the fora. In addition, the project team held briefing and debriefing sessions before and after the fora to discuss and optimise the networking process. Less can be said about the direct effect of the foresight in terms of relevance to policy documents, as the accompanying research ended shortly after the Freightvision project.

In Austria, the results were presented up to the highest ranks of the ministry of transport, which led to the ministry funding a follow-up project (Freightvision Austria, see EFP Brief No. 231) at the national level through the Transport Research Program IV2plus. Media coverage both at the sectoral level (some was very offensive even criticising the scientific evidence) the national level gives some indication of the relevance of the Freightvision process. After the final dissemination conference, DG TREN (MOVE) ordered extra copies of the last management summary for distribution throughout the directorate, which can be seen as a sign of the project’s relevance to internal discussion. In 2012, we conducted some additional interviews to find out whether Freightvision had any direct influence on the White Paper on Transport published in 2011.

Although some affirmative statements were made, it is not possible to verify such an influence. The Commission Staff Working Document on the White Paper shows no reference to Freightvision or other parallel FP7 Support Actions. However, several of the 36 measures from the project are mentioned in this document (e.g., CO2 labelling and integration into standards, e-freight, ecodriving training, liberalisation of cabotage, IST, ERTMCS/ETCS etc.).

Further Need for Follow-up Research

A further step in research on the effects of foresight would be to analyse in depth how participants of a foresight process deal with what they have learnt during the foresight process once they return to their usual surroundings and home environments. In principle, large participatory foresight processes induce participants to carry new impulses to their home organisations. Strategic dialogues and mutual learning processes during the foresight exercise can provide guidance in situations with high degrees of unpredictability and become effective in the organisations the participants originate from.

However, it is a great challenge to methodologically grasp the different kinds of effects over time and to isolate the contribution of foresight processes to complex and continuous processes like strategy finding and policy formulation. Determining the contribution of foresight exercises will always be achieved only in part.

Highly Controversial Stakeholder Responses

Although the process was built on a well-founded evidence base, including several models that are also cited in the recent White Paper, it was foreseeable that controversial positions would emerge in the normative phase of the foresight. For reasons of transparency, an effort was made to make dissent explicit and to document minority positions in working groups. Although it was clear that the project, financed through a FP7 support action, was no formal stakeholder consultation process in preparation of the White Paper, lobbying occurred to the extent that some participants at the final conference were on the verge of boycotting the event because of unfavourable conclusions for a specific interest group. Due to the explicit backing by many of the forum participants who attended the dissemination conference, it became clear that the overall results were valid and that the foresight process had been transparent and sound.
 
 

Sources and References

Amanatidou, E. and Guy, K. (2008), “Interpreting foresight process impacts: Steps towards the development of a framework conceptualising the dynamics of ‘foresight systems’”, Technological Forecasting and Social Change, Vol. 75, No. 4, pp. 539-557.
 
Chen, C. (2003), Mapping Scientific Frontiers. The Quest for Knowledge Visualization, Berlin: Springer.
 
Coromina, L., Guia, J., Coenders, G. and Ferligoj, A. (2008), “Doucentered networks”, Social Networks, Vol. 30, No. 1, pp. 45-59.
 
European Commission (2011), “Commission Staff Working Document – Accompanying the White Paper – Roadmap to a Single European Transport Area – Towards a competitive and resource efficient transport system”, SEC(2011) 391 final.
 
Grossmann, R., Lobnig, H. and Scala, K (2007), Kooperationen im Public Management. Theorie und Praxis erfolgreicher Organisationsentwicklung in Leistungsverbünden, Netzwerken und Fusionen, Munich: Juventa Verlag.
 
Heimeriks, G., H., Hörlesberger, M. and Besselaar, P. van den (2003), “Mapping communication and collaboration in heterogeneous research networks“, Scientometrics, Vol. 58, No. 2, pp. 391-413.
 
Lewin, K. (1953), Die Lösung sozialer Konflikte. Ausgewählte Abhandlungen über Gruppendynamik, Bad Nauheim: Christian-Verlag.
 
Schartinger, D., D. Wilhelmer, D. Holste, K. Kubeczko (2011), Assessing immediate learning impacts of large foresight processes. Submitted to Foresight Journal.
 
Schein, E.H. (1995), “Kurt Lewin’s Change Theory in the Field and in the Classroom. Notes toward a Model of Managed Learning”, available at: http://www2.tech.purdue.edu/Ols/courses/ols582/SWP-3821-32871445.pdf (accessed 2 December 2012).