Best practice example
Title: DIANA application project
University: Open University of Catalonia (UOC), Spain (Cerro et al., 2020)
Section of the framework: E - Digital tools used to record, compare and synthesise data on learner progress
What competencies and learning outcomes of the DigicompEdu Framework are we addressing?
|
Competencies |
|
|
Assessment strategies Analysing evidence |
- To use digital technologies to allow learners to collect evidence and record progress, e.g. audio or video recordings, photos. - To critically reflect on the appropriateness digital assessment approaches and adapt strategies accordingly. - To critically value the evidence available to inform teaching and learning. |
Key issues:
This is a best practice based on a research study carried out at the Universitat Oberta de Catalunya to design and implement a learning analytics solution for teachers. The aim was to facilitate their access to information for monitoring and assessing asynchronous online discussions, a type of collaborative learning activity in which students interact with each other to jointly construct meanings using dialogue and reflection.
This practice had two main objectives, the first of which was to design a learning analytics tool to analyse students’ communicative interaction online. The second objective was to have teachers implement the tool to monitor and assess asynchronous online discussions and in turn measure the impact of the tool’s use on university students.
In order to accomplish these objectives, teachers received specific training on
learning analytics through a teaching guide on the academic use of DIANA 2.0 in
the classroom, which improved their understanding of the student learning
process. Some of the classrooms’ teachers used DIANA 2.0, allowing their
students to be influenced by the personalised feedback that the teacher gave
them based on the metrics that DIANA 2.0 reported on the online discussion
activity. These classrooms were considered experimental classrooms. The
classrooms whose teachers did not use DIANA 2.0 to monitor and evaluate the
online discussion activity were considered control classrooms in order to
compare the results with the rest of the online classrooms.
Figure 1. Research phases taken from Cerro et al. (2020, p. 5)
If the activity score is compared with the level of participation, a correlation of 68% is obtained, which indicates that those students who interacted more in the online discussion activity were more likely to perform better than those with a lesser degree of participation. The students who scored best in the online discussion were those who not only exchanged a higher number of messages, and with more extensive arguments, but also those who generated the most impact within the conversation through the number of responses received (level of popularity), in comparison with students with a lower grade.
Some of the information produced by learning analytics was used by teachers to understand the learning process carried out by their students. The teachers then used this information to provide and improve the feedback sent to students, complemented by other qualitative information.
As this best practice shows, if teachers are trained on the adequate use of learning analytics instruments, the interaction with students in asynchronous online discussions improves as well. As a consequence, an improvement in individual performance occurs and dropout rates are reduced.
Relevance for teachers:
One of the objectives was to have teachers implement the tool to monitor and assess asynchronous online discussions and in turn measure the impact of the tool’s use on university students. During the pilots’ development, teachers received specific training on learning analytics through a teaching guide on the academic use of DIANA 2.0 in the classroom, which improved their understanding of the student learning process.