Lessons from ESPA
Independent Evaluation of ESPA Projects
Evaluation company Jigsaw has analysed and reported on the outcomes and impact of five key ESPA projects, verifying that our research played "a key role in catalysing change". A summary of the report is available here in our News section.
Summary of Learning Review: The ESPA Approach to Interdisciplinary Review Processes
The Learning Review of the Ecosystem Services for Poverty Alleviation (ESPA) programme’s approach to interdisciplinary review processes was conducted at the request of the ESPA Directorate and Programme Executive Board.
The intention was to share lessons internally and externally, including with funders. The review focused on two key aspects:
- How the application and review process for ESPA-funded projects may have supported and shaped the type of interdisciplinary research funded
- How the process compares to other programmes and what lessons can be learnt that may support future interdisciplinary research programmes
Key Conclusions from the Review
- ESPA is an ambitious programme, creating challenges for commissioning projects which must simultaneously:
- exemplify our new conceptual niche of excellent research
- promote new ways of working, especially in terms of interdisciplinarity
- ESPA has served as an experiment in review processes for interdisciplinary research, developing methods through several phases of commissioning which can inform other interdisciplinary research programmes
- The experiences of the 2011 ESPA review process in particular, throw into sharp relief various problems that can stifle interdisciplinarity, for example:
- differing views among co-funders
- lack of clarity as to definition or weighting of interdisciplinarity in calls for proposals
- use of numerous reviewers not fully informed or comfortable with interdisciplinarity
- over-large review panels deliberating research excellence, interdisciplinarity and impact in the face of numerous lengthy proposals
- ESPA has evolved over time, learning lessons to improve review processes and offer good practices. During the 2012 call a number of innovations were introduced:
- a staged process which allowed expressions of interest to be sifted for a manageable number of realistic proposals
- workshops, discussions with the Director and other interaction with the programme gave applicants a better understanding of the programme’s criteria
- guidance materials, presentations and a mock panel session for peer reviewers and panel members
- There were some concerns expressed that the 2013 process was hurried, with an overly general call, underscoring the importance of taking time to ensure clarity in framing criteria.
- Future funders of interdisciplinary initiatives can learn lessons from ESPA as they design and implement review processes.
Recommendations for Funders of Initiatives Similar to the ESPA Programme
- Clarity of Expectations: Funders need to communicate clearly their expectations for interdisciplinarity to applicants, peer reviewers and panel reviewers.
- Management of the Review Process: Funders need to structure and manage the overall review process appropriately for the complexity of interdisciplinary research programmes.
- Composition and Preparation of Review Panels: Funders must pay particular attention to the review panel stage, where the crux of interdisciplinary proposal evaluation occurs.
- Awareness of the Challenges of Interdisciplinarity: Funders need to appreciate that interdisciplinary research is challenging.
- Learning and Adapting: Funders need to be willing to learn and adapt throughout the life of a programme promoting interdisciplinary research.
The Learning Review of ESPA’s approach to interdisciplinary review processes was conducted by Laura R. Meagher, Technology Development Group and Catherine Lyall, Information Browser Ltd. Methods included: document analysis, semi-structured interviews of individuals involved in ESPA and panel reviewers/chairs; rapid-fire focus groups; surveys of award-holders and reviewers.
Ethics in Collaborative Research in Developing Countries: Key Lessons from ESPA for Best Practice
Based on the below mentioned report, commissioned by the ESPA Directorate, of best practice approaches to collaborative social and natural science research in developing countries, this document outlines the key lessons from ESPA in assessing ethics in the research projects it supports.
Experience has shown that the application of ethical guidelines developed for research in developed countries to research in developing countries is impractical and raises a number of contentious issues. Various attempts have been made to provide guidelines more appropriate to the developing world context; however, to date these efforts have been dominated by the fields of bioscience, medical research and nutrition. There is very little advice available for those seeking to undertake collaborative social science or natural science research in developing countries and what is there tends to be held within disparate sources. Charting the development of a set of ethics documentation for future use by the ESPA research community, this report outlines past and present attitudes towards ethics procedures amongst this community and highlights the ways in which ethics procedures might be made more attractive to researchers working in this area.
Publications review
An ESPA publications review was carried out in October 2014. The breadth, interdisciplinarity and geographical diversity of the ESPA programme's research makes it a challenge to synthesise the findings from the many publications from the programme. This review is based broadly on the original research framework developed at the inception of the programme, and on a limited selection of papers available during summer 2014.
