The impact of poor assessments on your organisation
Training not making the business impact you predicted? John Kleeman thinks it could all be in the wording of your assessments.
If you’re an L&D professional, you will certainly understand the importance of training. Proper testing and assessments aren’t just about getting your employees to a certain level of competence, though that is certainly part of it – they’re about investing in your people on an individual and team level and achieving business value.
What you might not know is that the tests and assessments used to measure training and assess competence aren’t always as robust as they could be. In fact, they can sometimes produce misleading results – and these results can have unfortunate consequences.
Effective business decision making
Unreliable or invalid assessments usually lead to two major outcomes: the first is that employees who shouldn’t pass do; the second, that employees who should pass fail. Neither one is good for your decision making.
An unrealistic picture of employee data can have far-ranging consequences. For example, it may seem that a majority of your team have failed a test due to a lack of understanding, when in fact they have struggled to comprehend poorly-worded questions.
If employees don’t do the work to pass the tests, they may make mistakes and underperform, ultimately affecting a business’s bottom line.
When this happens, it is often left to the L&D team to solve – requiring more time and greater resources to train individuals who already have a firm grasp of the subject matter. Similarly, questions and tests which are too easy may not genuinely check understanding and competence.
In another example, unreliable assessments which allow employees to cheat make it difficult for organisations to determine who in their workforce needs more training. If employees don’t do the work to pass the tests, they may make mistakes and underperform, ultimately affecting a business’s bottom line.
Compliance to regulations is essential in many sectors, and any breach can have a massive financial impact. For example, recent data from Boston Consulting Group found that banks globally have paid around £262bn in fines since the 2008 financial crisis.
Many companies understand the importance of training employees to ensure they’re following regulations, and assessment is an excellent way of measuring and demonstrating understanding. However, if your assessment platform isn’t secure and allows cheating, regulators will be concerned about the integrity of your organisation, and you may ultimately face regulatory fines.
Secure assessment platforms can prevent cheating by allowing businesses to remotely 'lock down' a participant’s PC – disabling the keys and functions used to print, screen capture and task-switch.
They also keep a robust audit trail of the assessment in case of disputes, and allow you to shuffle the order of questions and choices to make it harder for people to copy from each other. If there are strict compliance requirements in your sector, it may be time to begin thinking about how you can make your testing more secure.
Regulators like robust assessments; in fact, they positively encourage them. The US Federal Deposit Insurance Corporation says in its compliance manual: “Once personnel have been trained on a particular subject, a compliance officer should periodically assess employees on their knowledge and comprehension of the subject matter”.
And the European Securities and Market Authority says in its guidelines for assessment of knowledge and competence: “Ongoing assessment will contain updated material and will test staff on their knowledge of, for example, regulatory changes, new products and services available on the market”.
The problem of forgetting
There is a wealth of research which shows individuals who sit assessments after learning retain more information. Answering questions doesn’t just track what employees know – it also gives them retrieval practice, which means they’re far more likely to be able to recall information when they need it.
If the assessments provided by your organisation are confusing or misleading, employees may not get the retrieval practice they need, rendering your learning and development programs ineffective.
To ensure employees remember important information, use open ended questions, as they don’t prompt for the answer. Multiple choice questions are also useful, but remember to provide feedback on incorrect answers, or else the learner may retain incorrect information.
There are other ways to practice retrieval, but the advantage of computerised assessments is that they are automated, and so encourage or require learners to retrieve information.
Assessment processes are often thought of in binary pass/fail terms, but they’re a multi-dimensional issue for all organisations. They have implications from a regulatory standpoint and can result in substantial fines; their data can positively influence your decision-making or lead you down the wrong path; and they can help employees retain information or ensure that they forget it.
Make the right decisions, and you’ll build a more engaged, more capable, and more knowledgeable workforce. The sooner you implement great assessments, the sooner you’ll see the benefits.
About the author
John Kleeman is executive director and founder at Questionmark
To err is human, so John Kleeman reveals how to do it less often!
Want your CV to shine? Aditya Sharma is on hand to help you revolutionise your L&D resume.
We come to you live (well, when we recorded it anyway) from Learning Technologies 2020 at London's Excel centre.
Trevor Wheatly discusses how 360° profiling can turn routine appraisals into practical assessments of performance based on the behaviours that matter in business.
L&D experts from LinkedIn, Coca-Cola and Capital One International are set to share their expertise at the renowned World of Learning Conference.
National Measurement Institute’s curriculum for apprentices has been licensed to training provider, EEF, giving industry access to world-leading measurement teaching for the first time....