I do wonder however if this approach creates a problem. What is being measured could be described as:
“A person’s ability to be able to analyse, collate and respond to prescribed information in a structured and ordered way”.
If this were only the case, then if an answer has been reached in such a way it shouldn’t matter if it was ‘right’ or ‘wrong’. To counter this, an answer must be reached in the ‘correct’ way. This presents an interesting issue, where to be ‘correct’ is actually about conforming with educational and cultural expectations of what is the correct answer.
But if two people interpret a question in different ways, and both interpretations have been reached in a “structured and ordered way”, surely it follows that both individuals should receive a pass, albeit one may receive a higher grading than the other?
Some measurement tests, for example mathematics, are based on tried and tested formulae and are deemed to be valid because many people have interpreted the question in the same way.
Other measurements are not so easy to validate.
Law examinations deal in answers based on ‘the legal system’ but interpretation by students will be based on subjective analysis of the question. Consequently, answers to a question can reach the same conclusion but take different paths to reach that conclusion.
It is quite difficult to establish assessments, whatever form they may take, which provide people with the opportunity for only one interpretation of a question. Sufficiently difficult that there is a whole industry that has emerged around the development of assessments that generate consistently similar answers.
It is clearly important that any form of assessment is validated (measures what it is supposed to measure) before it is implemented. This reduces misunderstanding and subsequent ‘failure’. But we are all wired differently, we have all experienced things differently, we perceive things in different ways, and we can sometimes provide a creative solution to a problem which sits outside the expected norms. Maybe, sometimes, the established ways of measuring people don’t always take that into account.
Are you familiar with this issue? How have you addressed it?