Digital assessment systems - aka online evaluation systems / on-screen marking – are on the rise. Universities are increasingly implementing such systems not just for their distance-learning cohorts, but for mainline, on-campus, undergraduate assessments as well. As a component part of integrated systems management across university campuses this is all well and good – after all, universities need to manage the data of highly complex student populations as efficiently as possible – but in some cases academic staff may feel uneasy about using such systems. This article addresses some of these concerns, and also discusses some of the advantages of digital assessment.
What’s wrong with digital assessment?
The answer to this question, of course, depends on the person asking, and on the sort of digital assessment proposed. Systems that require direct input from the student (i.e. asking them to complete assessments directly online) may be costly in terms of student time, problematic in terms of accessibility, and have detrimental effects on the classroom environment by requiring students to work directly online. Similarly, systems that require students to submit work which is subsequently scanned in and uploaded place a considerable burden on administrative staff.
For examiners, too, there may be downsides. Some may object to reading large volumes of material on-screen; others complain of being unable to get a sense of the overall argument without holding the paper artefact (the assignment, or in some cases, the exam script) in their hands.
There are also concerns about data loss. Some worry that assignments may be incorrectly uploaded, and that missing material may not be immediately evident. Other forms of data loss may include the marginalia (rough workings out/ plans, etc) on the inside covers of exam scripts, which lecturers have traditionally encouraged students to write down as evidence of their process.
There is also considerable resistance around the issue of individualised feedback. While forms of assessment conducted by multiple choice or requiring binary answers may be well suited to digitised marking, there remain questions about how well a digital assessment environment can cater for nuanced and targeted narrative feedback. Academics also worry that in-text annotation (a key pedagogical tool for pinpointing specific areas that need attention) may prove to be more cumbersome, or even impossible, with such tools. Most problematic of all are attempts to restrict markers to pre-determined, reductive marking ‘templates’, which are sometimes directly linked to institutional marking guidelines.
So what’s right with digital assessment?
But let us consider the considerable benefits of integrated digital assessment systems. Such benefits include include harmonisation, standardisation, parity of marking across institutions and more accurate benchmarking.
Operational efficiencies are a further benefit. For example, there may be increased marking times available as a result. Two markers may be able to access the same material at the same time, thus avoiding the dreaded ‘bottleneck’ at marking time that can arise when a second marker has to wait for a first marker to finish his or her marking round and hand over the essays or scripts. This can lead to extremely stressful situations and tension between colleagues, and can have knock-on effects for student well-being and progress if there are delays in receiving feedback.
Digital assessment also offers improvements in plagiarism detection, since these can be integrated with such systems as ‘Turnitin’, as well as allowing markers to conduct quick and simple online searches for phrase matching or the use of unacknowledged material.
Lastly, digital assessment systems are designed to be integrated with other aspects of the student record system. In principle, this means that marks can be swiftly uploaded and collated, reducing the administrative pressures during exam boards, and also allowing both feedback and marks to appear more quickly in student-accessible records. Students therefore gain both access to data about their own progress, and the tools to manage their own progress successfully.
Not all of this is easily resolved, and institutions must weigh up the pros and cons of implementing costly digital environments against their considerable benefits. What’s clear is that assessment places a considerable burden on staff time, and that that is only going to continue with growing and ever diversifying student populations. Digital assessments could be at least part of the answer to this problem.