The problem with competence in medical education

Summative assessment plays a pivotal role in determining competence to enter the medical profession. Much effort has been deployed in order to develop psychometrically defensible assessment systems with the aim of determining which students are competent to graduate. Standard-setting procedures have been developed and refined so as to define and recognise the minimally competent student. These procedures usually allow a student to graduate if he has achieved an overall grade of competence. However, in practice, these methods are hard to implement and often feel unsatisfactory.

 High-stakes assessments typically create large amounts of data regarding a student’s strengths and weaknesses. Two students with the same overall mark will typically have very different profiles of strengths and weaknesses. Furthermore, this overall mark may well disguise significant areas in which the student is not yet competent. In the binary pass-fail nature of summative assessments, a student who is only minimally competent is treated the same as one who is maximally competent: both can progress to the next stage of learning or graduate as a doctor. 

There is, therefore, a risk of providing false reassurance about a candidate’s ability, both to the student but perhaps more importantly to patients; candidates who pass the assessment are simply labelled as competent. Although this label may be nuanced, perhaps by the provision of grades, the manner in which it is used is crucial: the learner is either good enough to progress or not. In addition, the implicit message given to learners is that those areas in which they did not excel must be unimportant as they have been labelled as competent overall. 

Reducing all the information obtained from an assessment down to a binary pass-fail verdict simplifies high-stakes decision-making. Binary decisions on individual assessments are often then combined to make decisions about overall progression through a course. While this is an understandable approach, it has the potential to mislead as much rich information about a learner has been lost in the process.

If students receive no feedback after summative assessment, but are aware that the primary focus of the assessment is to determine whether or not they have achieved minimal competence, there is an inevitable risk that they will prepare for the assessment by concentrating on the relatively low hurdle of minimally acceptable competence, rather than aspiring to attain the more challenging goal of optimal competence, as the latter is not normally rewarded.  

However, the closer a student's mark is to the pass mark the higher is the probability that there remain significant areas in which the student is incompetent. 

Please click below to explore other problems with summative assessment in medical education.