CSIC signed the agreement to join CoARA in November 2022 and it is one of the three entities participating in the National Chapter Spain, created in July 2023 as the working framework for CoARA at national level. In this way, it assumes the joint commitment to develop specific tasks and concrete outcomes to implement the Agreement in the national context.
Furthermore, the mission of the CSIC CoARA Commission is to develop an in-depth institutional analysis that allows the design of a flexible and multidimensional evaluation proposal and its progressive adaptation to the evaluation practices of the CSIC's scientific activity, and to provide the institution with the methodology, culture and tools necessary for this purpose
There is evidence in the research community that scientific evaluation criteria applied in isolation and decontextualized from the research process (such as journal impact factor, h-index or institutional rankings) are detrimental to the quality, diversity and integrity of research. These bad practices create unsustainable pressures on researchers, distort their priorities or the choice of lower-risk work, undermining thematic, cognitive and methodological plurality and intellectual freedom. After a long list of declarations, manifestos and international initiatives reviewing research evaluation processes, the Coalition for the Advancement of Research Assessment (CoARA) was established in December 2022.
The starting point began in 2012 with the Declaration of San Francisco (DORA), which calls for the elimination of the impact factor in researcher-level evaluations. Other initiatives such as the Leiden Manifesto and The Metric Tide propose the use of responsible metrics in evaluation, the defense of multilingualism with the Helsinki declaration, the commitment to Open Science with the Hong Kong principles and the recognition of bibliodiversity, disciplinary differences and inclusiveness in the Paris Call on Research Assessment. In 2019, the EUA (European University Association) and Science Europe joint statement to improve research assessment methodologies is published. The European Commission included the mandate to reform the research assessment system in its 2021 European Research Area Policy Agenda, with the aim of improving the quality of research while respecting the principles of diversity, inclusiveness and openness. This collaboration culminates in the Agreement on Research Assessment Reform in July 2022 and the emergence of CoARA in December 2022, to which CSIC adheres in November 2022.
CoARA is an international coalition of scientific organizations working together to enable systemic reform of research evaluation based on the principles of diversity, inclusiveness and openness by facilitating mutual learning among all those committed to improving research evaluation practices. The coalition was initially driven by Science Europe, the European University Association (EUA) and the European Commission in January 2022, following the ERA Action 5 guidelines and the momentum for change reflected in initiatives such as the DORA statement.
This coalition represents a paradigm shift in the evaluation of science to maximize the quality of research and good scientific practices. It aims to be an inclusive and collaborative space, offering a platform for debate to develop new criteria, methods, tools, and a space for the exchange of good practices and mutual learning. To this end, a group of experts and institutions of the coalition drafted the Agreement on Reforming Research Assessment (July 2022) with an open process of signature and adhesion. More than 800 organizations worldwide have signed the agreement: public and private research funders, universities, research centers, institutes and infrastructures, their associations and alliances, national and regional authorities, accreditation and assessment agencies, scientific societies and researcher associations, and other relevant organizations. Since the first meeting of the General Assembly on December 1, 2022, the secretariat of the Coalition for the Advancement of Research Assessment (CoARA) is provided by the European Science Foundation - Science Connect (ESF-SC).
The Agreement on Reforming Research Assessment establishes a shared direction to guide needed changes in research assessment practices, with the overall goal of maximizing the quality and impacts of research. The Agreement includes principles, commitments, and timelines for organizations willing to work together to implement the reform changes. The signatories commit to recognize the diversity of research outputs, practices and activities while respecting disciplinary differences and career stages and maximizing openness and inclusiveness in science. This requires basing evaluation primarily on qualitative judgments, for which peer review is fundamental, supported by the responsible use of metrics. Supported by the work of the Coalition, organizations will decide on action plans and timelines for implementing commitments, which may vary depending on the context (e.g., national, disciplinary, or evaluation of individual investigators, research units, and research organizations or research projects) and the strategic goals and mission of each organization.
1. Recognize the diversity of research contributions and academic stages according to the needs and nature of the research.
2. Base the evaluation of research primarily on qualitative evaluation supported by the responsible use of quantitative indicators.
3. Abandon inappropriate uses in research evaluation of journal-based metrics, in particular inappropriate uses of the journal impact factor (JIF) and the h-index.
4. Avoid the use of institutional rankings in research evaluation.
5. Allocate resources to reform research evaluation to the extent necessary to achieve the organizational changes committed.
6. Review and develop research evaluation criteria, tools and processes.
7. Increase awareness of research assessment reform in the scientific ecosystem and provide transparent communication, guidance, and training on assessment criteria and processes, as well as their use.
8. Exchange practices and experiences to enable mutual learning within and outside the Coalition.
9. Communicate progress on adherence to the Principles and implementation of the Commitments.
10. Evaluate practices, criteria and tools based on sound evidence and the state of the art in research evaluation, and make the data collected openly available.
The development of the Agreement and the achievement of the deadlines is organized around two different work streams: the Working Groups and the National Chapters. The Working Groups are based on a bottom-up approach with voluntary participation of members, operating as “communities of practice”, providing mutual learning and collaboration in specific thematic areas. Participating members exchange knowledge, learn from each other's experience, discuss and develop products to advance research evaluation and support implementation of member commitments. National Chapters (NCs) facilitate knowledge sharing, mutual learning, and discussions on issues relevant to the different types and missions of scientific organizations in a given country. They are also committed to share lessons learned with other NCs and to foster synergies and mutual learning with relevant CoARA Working Groups.
In July 2023, the CoARA Standing Committee approved the proposal of the National Chapter Spain (Spanish Chapter) composed by CSIC, CRUE and ANECA, which was officially presented on February 9, 2024. The Spanish Chapter plans the objectives to be achieved with the reform of research in Spain by 2030, through the design and implementation of evaluation systems that respect the commitments of the Agreement and its application in pilot projects of the different recipient institutions. In order to implement the reform simultaneously in a gradual manner and minimizing risks in the development of research careers, it is essential to monitor these pilot studies at different levels (individuals, institutions and research), evaluating the impact on the scientific community and on the evaluating bodies themselves. Currently, a total of 84 Spanish institutions are participating in this national initiative: 12 quality agencies, 11 research centers, one research funding agency, two associations and 58 universities.
The mission of the CSIC CoARA Commission is to adapt to the CSIC scenario the evaluation commitments through an in-depth institutional analysis that allows the design of a flexible and multidimensional evaluation proposal and its progressive adaptation to the evaluation practices of the CSIC scientific activity. The CoARA-CSIC Commission is created in June 2023, with the following objectives:
1. To review and align the current system of evaluation of research activity with the paradigm of open science considering the diversity of activities and results of scientific activity, such as open access to research results, transfer to the business sector, public administrations and society, dissemination and scientific culture, among others. These are aspects that are currently considered part of the research activity and for which it is proposed to define objective and measurable indicators of relevance.
2. Consider the process and the areas in which research is carried out, taking into account inter- and multidisciplinarity, the gender component, social impact, training, mentoring, etc.
3. To act in coordination with other international (CoARA international, DORA, etc.) and national (ANECA and CRUE) agents involved in the application of the new parameters defined to evaluate research activity.
4. To provide the CSIC with the methodology, culture and tools necessary to carry out these objectives.
The CoARA CSIC Commission is composed of 18 people with a wide variety of expertise profiles and is organized around three working groups:
1. Analysis of CSIC evaluation practices group: criteria and methodologies
2. Awareness and communication group.
3. CSIC strategy to adopt CoARA
With all this, it is expected to develop during 2024 a proposal for CSIC evaluation practices and application methodology, the agenda of actions and resources needed to carry them out and a training and communication plan. In addition, it is expected to define and implement the future Current Research Information System (CRIS).
Coordinator: Isabel Varela Nieto (Advisory Member of the Presidency)
Vice presidencies: Fco. Javier Moreno Fuentes (VRI), Chema Martell (VICYT), Ana Castro Morera (VIT)
Presidency Units: Elena Corera (UNIN, presidency), Jaime Pérez del Val (VACC, presidency), Carmen Guerrero (VACC, presidency), Carmen Mayoral (Executive Vice-president CMyC CSIC, Presidency Cabinet)
VRI-VORI-VICYT: Elena Domínguez (CSIC Delegate in Brussels and CSIC Representative in the Working Group Research Assessment ERA 3 (VRI), Agnès Ponsatí Obiols (URICI Director, VORI), Elena Cartea González (VAACT, VICYT), Ada Ferrer Carbonell (Society Area Coordinator, VICYT), Jordi Pérez Tur (Life Area Coordinator, VICYT), Antonio Chica (Matter Area Coordinator, VICYT).
Invited experts: Zaida Chinchilla Rodríguez (Metrics and Innovation in Science and Technology Group, IPP), Isidro Aguillo Caño (Cybermetrics Laboratory, IPP), Jordi Molas Gallart (INGENIO Director), Alejandro Santos Botana (Library and Documentation Service, ICMAB).
Frequently Asked Questions
DORA is the Declaration on Research Assessment, which recognizes the need to improve the ways of assessing researchers and the results of academic research, and to which the different institutions adhere as a commitment to review and implement new methodologies for research assessment. CoARA is a global coalition, driven by European institutions in 2021, which aims to be a path with concrete objectives and deadlines for the systemic reform of research evaluation, being 2030 the year where it is intended to reach the culmination of the agreement.
The signatories commit to a common vision: “that the evaluation of research, researchers and research organizations recognizes the diverse outcomes, practices and activities that maximize the quality and impact of research”, but each organization will decide the steps to take to implement the commitments and the pace in its reform journey. For example, we find institutions that are developing small pilot projects on new evaluation methodologies on specific solicitations to study their impact, while other organizations are directing their efforts to internally evaluate and develop their own Current Research Information System (CRIS).
Spain is one of the fifteen National Chapters that have been constituted in CoARA as working units at national level. The NCS is led by three institutions: CSIC, ANECA and CRUE, each having its own working strategy, but sharing and facilitating knowledge exchange, mutual learning and discussions on relevant issues. CSIC is jointly committed to developing specific tasks and concrete results to help its members implement the Agreement in the national context.
While the agreement is being developed and finalized (remember for 2030), we can find examples at the international level, such as those carried out in Norway, Finland, Holland, among others. In Spain, the National Agency for Evaluation and Accreditation (ANECA) has already begun to apply evaluations aligned with CoARA in its call for six-year terms and in the ACADEMIA program for the state accreditation of civil servant teaching staff. Other institutional initiatives along these lines can be found at the Universidad Pablo de Olavide, the Universitat Oberta de Catalunya, the IIS Biogipuzkoa, among others.
CoARA News
El Consejo Superior de Investigaciones Científicas (CSIC), organismo…
Los equipos de dirección de los centros e institutos del …
El Ministerio de Ciencia, Innovación y Universidades, a través de la…