Research in context

Research in context
Dutch examples of multidimensional approaches – and the need for action research
2 | Leonie van Drooge
Contenidos
• Rathenau Instituut
• Ciencia en Holanda
• Relevancia o impacto social
• Proyecto ERiC - Evaluando la investigación dentro de
contexto
• Proyecto SIAMPI - Interacciones Productivas
• Proyecto Waardevol – Valuable – Valioso
• Action research
3 | Leonie van Drooge
Contenidos
• Rathenau Instituut
• Ciencia en Holanda
• Relevancia o impacto social:
• Proyecto ERiC - Evaluando la investigación dentro de
contexto
• Proyecto SIAMPI - Interacciones Productivas
• Proyecto Waardevol – Valuable – Valioso
• Action research
4 | Leonie van Drooge
Misión del Instituto Rathenau
El Instituto Rathenau promueve la opinión pública y política
sobre la ciencia y la tecnología. Para ello, el instituto hace
investigación sobre la organización y desarrollo del sistema de
la ciencia (Science System Assessment), realiza publicaciones
sobre el impacto social de las nuevas tecnologías (Technology
Assessment) y organiza debates sobre temas y dilemas en el
campo de la ciencia y la tecnología.
El Instituto Rathenau es financiado por el Ministerio de
Educación. Sin embargo, es independiente en la realización de
sus actividades.
5 | Leonie van Drooge
Science System Assessment – some projects
• Resilient Knowledge Infrastructure
• Future of Dutch universities
• Non-academic public research institutes
• Careers of scientists
• Social contract of science
• “Valorisation” or third mission, impact, relevance
• (Public) attitudes towards science
• (Research evaluation)
• Innovation:
• Globalisation
• Regulation
• Knowledge coproduction
6 | Leonie van Drooge
Contenidos
• Rathenau Instituut
• Ciencia en Holanda
• Relevancia o impacto social
• Proyecto ERiC - Evaluando la investigación dentro de
contexto
• Proyecto SIAMPI - Interacciones Productivas
• Proyecto Waardevol – Valuable – Valioso
• Action research
7 | Leonie van Drooge
Universities in the Netherlands
• 9 comprehensive universities
• 4 founded between 1575-1636; 5 founded1880-1976
• 8 with academic hospital
• 3 technical universities (founded in 1842, 1956, 1961)
• 1 agricultural university (founded in 1918)
• Total # students enrolled: 240,159
• Total # academic staff :
28,252 (24,595 fte)
of which # PhD students : 9,009 (8,706 fte)
• Total revenue:
€ 5.8 billion euro
(167 billion UYU)
8 | Leonie van Drooge
More education and research in the Netherlands
• Academic, non university, research institutes:
• 17 KNAW institutes (governed by the royal academy)
• 9 NWO institutes (geverned by the research council)
• 38 “universities of applied sciences”
• Teaching is main mission, research (relevant for
profession/practice) is new and relative small secondary
mission
• 421,000 students
• Total budget 2,5 billion euro
9 | Leonie van Drooge
Excellent performance
• # academic publications in 2010: 68,539
• Shanghai Jiao Tong Ranking:
• all universities in top-500
• 2 universities in top-100
• THE Ranking:
• all universities in top-300
• 4 universities in top-100
• ERC (European Research Council) – NL receives 9 %
• FP7 (European Framework Programme for Research) – 7 %
(NL contributes 5 % to European budget)
10 | Leonie van Drooge
How about societal relevance / valorisation?
• “Knowledge transfer for the benefit of – or towards – society”
is official third mission of universities
• Criterion in evaluation protocol for university research groups
• Required by research council and European Commission
• Efforts of Ministry of Economic Affairs (!) for > 30 years
• Plus: current economic situation
However:
• Academic excellence has been focus for decades
• What does relevance or valorisation mean and imply?
11 | Leonie van Drooge
Contenidos
• Rathenau Instituut
• Ciencia en Holanda
• Relevancia o impacto social
• Proyecto ERiC - Evaluando la investigación dentro de
contexto
• Proyecto SIAMPI - Interacciones Productivas
• Proyecto Waardevol – Valuable – Valioso
• Action research
12 | Leonie van Drooge
ERiC
• Evaluando la investigación dentro de contexto
• Colaboración entre la Asociación de Universidades, el
Consejo de Investigación, la Real Academia de Ciencias de
los Países Bajos y el Instituto Rathenau
13 | Leonie van Drooge
ERiC
• Describir la misión y los objetivos del grupo
• Describir la contribución social que el grupo ha hecho
1. Los resultados que son o pueden ser de importancia
para la sociedad
2. ¿Cómo se difunde el conocimiento entre las partes
interesadas
3. La evidencia de interés o apreciación de las partes
interesadas
4. Los efectos de los resultados tienen
• Definir indicadores de relevancia social
14 | Leonie van Drooge
ERiC
Indicators for societal relevance of law research:
• Papers in leading professional journals for fellow researchers
and law professionals (in particular Nederlands Juristenblad)
• Membership of national advisory committees on legal practice
and policy
• Work in legal practice
• Post-academic education (required for practioners)
Developed with researchers and administrators in law faculties
15 | Leonie van Drooge
SIAMPI
Métodos de Evaluación de Impacto Social de la investigación y
los instrumentos de financiación a través del estudio de las
interacciones productivas entre la ciencia y la sociedad
16 | Leonie van Drooge
SIAMPI
Interacciones productivas:
• Personales (conversaciones, correspondencia,
presentaciones, siendo miembros de comités)
• Indirectamente:
• A través de textos (artículos, protocolos, informes,
reportes de asesoría, cuestionarios)
• A través de artefactos (herramientas, sitios web, procesos)
• Financieras (a través de contribuciones monetarias, en
especie, o uso compartido de instalaciones)
17 | Leonie van Drooge
SIAMPI
• It is process-oriented, valuing the small but necessary steps in
an intricate course of interactions
• It is contextual, and takes into account the vital role of the
contributions of researchers and stakeholders alike (in
response to issues of time lag and attribution)
• It is oriented towards learning and improving rather than
judging and accounting
• Additional value:
• Concept of impact becomes more tangible
• Rise of awareness with researchers and stakeholders
concerning the value of their actions
18 | Leonie van Drooge
Waardevol – Valuable – Valioso
• Indicadores de valorización del conocimiento científico
(divulgación y participación en la sociedad o transferencia de
tecnología, transferencia del conocimiento, impacto social,
intercambio del conocimiento)
• Encargado por el Comité Nacional, junto con el Consejo de
Investigación
19 | Leonie van Drooge
Waardevol – Valuable – Valioso
Idea básica: la valorización tiene cuatro dimensiones:
1. Actor - diferentes grupos son requeridos para la
transferencia del conocimiento (proveedores de
conocimiento, usuarios, intermediarios)
2. Nivel de Agregación - la responsabilidad y la oportunidad se
distribuye entre los distintos niveles (institución,
departamento o programa, persona o proyecto)
20 | Leonie van Drooge
Waardevol – Valuable – Valioso
Idea básica: la valorización tiene cuatro dimensiones:
3. Disciplina - para cada disciplina u ocupación existen
diferentes formas de transferencia del conocimiento
4. Fase – la transferencia del conocimiento es un proceso que
ocurre en todas las fases de la investigación, a través de la
definición de la agenda, de la ejecución, hasta la difusión y
uso del conocimiento
21 | Leonie van Drooge
Waardevol – Valuable – Valioso
22 | Leonie van Drooge
Base común para la evaluación
(Base común para la evaluación de la transferencia del
conocimiento):
• Partiendo del objetivo de la unidad evaluada: La unidad
evaluada indica lo que ha hecho para lograr el objetivo
• Enfocándose en el proceso: queremos entender cómo se crea
la transferencia del conocimiento:
“It is only by analysing the processes that induce social impact
that we have a chance of recognizing potential research
impacts and the contributions made by research that might
otherwise not be evident” (Spaapen & van Drooge 2011)
• Yendo más allá de las coincidencias (de manera accidental)
23 | Leonie van Drooge
Contenidos
• Rathenau Instituut
• Ciencia en Holanda
• Relevancia o impacto social
• Proyecto ERiC - Evaluando la investigación dentro de
contexto
• Proyecto SIAMPI - Interacciones Productivas
• Proyecto Waardevol – Valuable – Valioso
• Action research
24 | Leonie van Drooge
The research questions
Former projects as well as contacts with academic researchers
and their managers have led to new questions:
• what does relevance mean at all for academic groups or
departments?
• why is relevance a problem? what is the problem?
• how to plan for relevance?
As well as:
• how can we develop approaches that, in the end, can be used
without our involvement or presence?
25 | Leonie van Drooge
Our involvement
We are asked to participate, help and intervene in various
settings, roles, forms:
• institutional awaydays of university departments
• development of impact strategies for projects, departments
and disciplines
• organisation of conferences / workshops
• writing of grant applications
• writing of self evaluation reports
• development of indicators
• expert role in discussions and interviews
26 | Leonie van Drooge
Participation as a form of action research
Our role is more than dissemination:
• we need to participate in order to answer our research
questions
Also, we learn from our involvement whether our research
questions are right – and find out some are not:
• we have changed our question of “development of new ways
to create an impact” into “what does relevance mean at all for
a specific academic group or department”
27 | Leonie van Drooge
A specific form of action research
Approach chosen is similar to Huxham & Vagen (2003)
• aim is to develop generic understanding – for use in other
contexts as well
• our participation should be of importance to all involved – and
happens to be our way of doing research
• our research questions are the reason to get involved and
help us in deciding whether to participate (or not) – our
involvement is driven by the other participants’ needs
• no ideological perspective to empower participants – we
accept the academic demands of relevance and excellence
28 | Leonie van Drooge
The trouble with action research
The process of research is difficult to manage
• the interventions are initiated by researchers and their
managers; we don’t plan interventions of a certain type, in a
certain setting or at a certain time
• each intervention is unique: each has a different context and a
specific form that fits the researchers’ needs
Also, we shouldn’t forget to focus on learning:
• the interventions are a satisfying way to disseminate
• we shouldn’t forget that our purpose is research
29 | Leonie van Drooge
Capturing data
We use a number of ways to gather information and to monitor
our progressive insights:
• we keep flipchart sheets, exercises, hand-outs, notes
• we use evaluation forms and ask specifically about new
insights and how they came about
• we discuss the event afterwards with participants
• we discuss events, developments, ideas and doubts
frequently amongst ourselves, whether face-to-face or
electronically
• we capture thoughts by means of Yammer
30 | Leonie van Drooge
Lessons learnt
So far we have learnt:
• we shouldn’t offer solutions straight away; in order to develop a
sustainable vision on relevance / impact, questions are much
more valuable than answers=>we focus on questions
• nevertheless, concrete examples are helpful to illustrate
possibilities; it helps getting from a more general and abstract
notion to conceivable and feasible ideas=>this fall we will
launch a website (in dutch) with a variety of examples
• we share our experiences with others and actively involve them
in our activities=>we train or coach people
31 | Leonie van Drooge
Thank you!
Dank u!
¡Gracias
Leonie van Drooge
senior researcher
Rathenau Instituut
l.vandrooge@rathenau.nl
@LeonievanDrooge
32 | Leonie van Drooge
Literature
• Leonie van Drooge, Rens Vandeberg et al Valuable: Indicators for
Valorisation. The Hague: Rathenau Institute, 2013
• Chris Huxham and Siv Vangen (2003): Researching Organizational
Practice through Action Research: Case Studies and Design
Choices. Organizational Research Methods 6 (3), 383-403
• De Jong, S.P.L., L.K. Hessels & B.J.R. van der Meulen: Societal
Impact Analysis Next Generation Infrastructures. The Hague:
Rathenau Instituut, SciSA report 1121
• Spaapen, J.B. and L. van Drooge (2011) Introducing 'productive
interactions' in social impact assessment. Research Evaluation, 20,
211-218
33 | Leonie van Drooge
Referencias
• ERiC - Evaluating the societal relevance of academic research
http://
www.rathenau.nl/en/publications/publication/evaluating-the-soc
ietal-relevance-of-academic-research-a-guide-1.html
• SIAMPI
http://www.siampi.eu/Pages/SIA/12/625.bGFuZz1FTkc.html
• Waardevol – Valuable – Valioso
http://
www.rathenau.nl/publicaties/publicatie/valuable-indicators-for-va
lorisation.html
• SEP - Protocolo estandarizado de evaluación
https://
www.knaw.nl/en/thematisch/kwaliteit/kwaliteitsbeoordeling-envalorisatie/overzicht?set_language=en
34 | Leonie van Drooge
Knowledge coproduction for societal challenges
Ensure that all stakeholders are represented in the project
Choose not only participants who are close together
Select participants with influence within their own organizations
Involve experts in knowledge coproduction
Create a "protected space"
Take enough time to clearly formulate what should be
investigated and resolved
• Evaluate while project is running: is it still on track?
• Work on small scale solutions first
• Evaluate the knowledge co-production process
•
•
•
•
•
•