Measuring student and educator digital competence beyond self-assessment: Developing and validating two rubric-based frameworks

Abstract

[Abstract] Over the past decade, self-assessment tools have garnered significant attention in the interest of measuring the skillset required by educators and students to function productively and ethically in digitally mediated environments, particularly in relation to education policy implementation. Since stated beliefs do not always align with actual practice, gaps have been shown to occur between self-reporting and performance in practice. Having an external assessor can counteract this imbalance; however, both perspectives should be taken into consideration as both are equally important. Against this background, this study develops and validates two rubric-based frameworks that supplement self perceived student and educator digital competence with classroom observation and task performance analysis. The DigComp and DigCompEdu self-assessment frameworks were used as a starting point to develop a student rubric and an educator rubric, respectively, underpinned by criteria validated in previous frameworks. The expert technique, which is the base for the Delphi Method, was used to validate each rubric, after which the rubrics were implemented at a Spanish university to test their reliability. The results indicated that the force of agreement across raters was consistent and both rubrics had a high degree of internal consistency, therefore both instruments are reliable.