The assessment gap
Finding the right indicators to assess the implementation of RRI values in science education
Do we have the right indicators to assess the implementation of Responsible Research and Innovation (RRI) values in science learning? Given the recent emergence of the RRI paradigm and the complexity of learning processes assessment the answer might seem obvious: not yet. But reality is, as usual, more complex than its obvious appearance.
To better answer this question María Heras, researcher at the Institute of Environmental Science and Technology (ICTA) of the Autonomous University of Barcelona (UAB) and Isabel Ruiz-Mallén, coordinator of the PERFORM project, Ramón y Cajal senior Research Fellow at the IN3-Open University of Catalonia (UOC), and associated researcher at the ICTA recently developed a study aiming at identifying possible indicators of RRI values implementation in the everyday practice of science education.
They highlighted that “while indicators addressing the cognitive aspects of learning are widely implemented, indicators related to specific RRI values, transversal competences and emotional aspects of learning are missing in most of the assessments reviewed or are approached in rather narrow ways.” Moreover “some RRI process requirements that could be, in principle, easier to track and monitor, are amongst the most neglected criteria. This is the case of gender equality and ethics integration, which are, by far, the most absent RRI-related criteria in the assessments reviewed.”
How the research was developed
The authors conducted a literature review both about the RRI paradigm applied to the context of science education and science learning, and about the assessment framework for science education. This literature review was complemented by a set of exploratory workshops developed with secondary school students in Spain, France and UK as part of the PERFORM project, widely described in the article ‘The multi-skilled researcher’.
As described in Figure 1 the study process can be summarized into two main steps: during the first one a framework was developed based on the literature review to select assessment criteria; during the second one these criteria were combined with indicators. The first step helped to identify 4 RRI learning dimensions ranging from cognitive aspects to values such as inclusiveness, gender equality, engagement, creative and critical thinking, and integration of ethical issues within the learning processes. Corresponding to these dimensions 11 learning outcomes that can be related to 32 assessment criteria were listed and described. Once the assessment criteria are defined, how can we identify related assessment indicators? Researchers used the Scopus scientific database as the search engine to find out which are the most frequent indicators used to assess education activities and by using specific key words identified 72 articles from 68 peer-reviewed journals and 4 books focusing on assessment studies. The analysis of these publications led to a set of 86 indicators among which 41 are strongly related to RRI paradigm.
What the research found out
The research highlighted that students’ cognitive aspects are by far the most assessed, moreover among the most cited criteria related to RRI three were related to cognitive aspects. As the authors stress in their article “gender and ethical issues – core aspects od RRI – stand out for their absence in most of the assessment reviewed… only five assessments within the whole sample critically addressed gender aspects beyond the usual identification of participant numbers according to gender”. Criteria and indicators related to students’ emotions and feelings are rarely found and, when present, they mainly refer to the excitement and enthusiasm of students while doing specific education activities. Perceptions and attitudes towards science are commonly and frequently assessed together. It is probably not surprising that high-order thinking skills such as cognitive engagement, system thinking, questioning and reframing and reflective thinking were not explored while assessing basic cognitive aspects.
This is not a checklist
The authors of the study claim that the 86 indicators are not intended as checklist but a framework to be validated and adapted to different contexts. However, the research highlights how a multilevel RRI-based assessment framework allows to tackle complex learning process expanding the elements that can be taken into account when analysing the learning experience, including cognitive aspects as well a wide set of skills and values. Another key aspect stressed by the research is that this approach provides a tool both to students and teachers to enhance a consciousness of the learning processes they are leading. Although not a closed checklist, the set of indicators is a solid base to broaden the toolkits that teachers, students and education professionals can use to wisely observe the evolving panorama of science education. In front of such a complex landscape there are not, apparently, theoretical big obstacles for the implementation of new and more rich assessment frameworks but time and resources available to teachers and students. The assessment culture and its implementation within educators and students everyday life is a cultural challenge.