Modern and evolving testing solutions have a number of essential attributes, David Grinham says
Training and certification requirements across all industry sectors including healthcare, financial services, technology and project management are evolving and now demand smarter testing solutions. In some instances, organisations are proficient in setting exams, but need help delivering them. Many organisations are seeking to expand their global reach through a workforce with the essential competencies. They may want to bring certification to their workforce, but do not know how to go about it. Others are working with testing partners who do not have the capability to deliver testing through use of the latest technology enabling a global reach, such as online proctoring.
Organisations face two major challenges. Firstly, to reach a candidate population that is increasingly geographically dispersed, providing testing fairly and equitably across cultures and languages. Secondly, organisations and governments are seeking to test individuals for a combination of four things – knowledge, skills, competency and experience – and determine whether these attributes can be applied in practice. Traditional exams were frequently knowledge based; however, modern certifications need to be relevant to the actual roles to be performed and to deliver confidence in individuals’ abilities to perform in the new job roles created in the modern knowledge economy. Knowledge testing is no longer adequate – smarter testing solutions are now required.
In many areas of the world there is an imbalance between supply and demand of skills. In Europe alone there is predicted shortfall of 500,000 people with essential IT skills. There is also a global shortage of skilled and qualified project managers. Organisations and employers need to access, assess and utilise talent pools across the globe to meet the requirements of the 21st century.
Equitable testing solutions are vital
However, it is not enough to equip employees moving into the marketplace with the knowledge and skills required; there needs to be certification and assessment programmes which are equitable, reliable and defensible. Psychometric rigour must be applied to ensure testing is fair and balanced across languages, cultures and continents; as well as across the different domain and sub-domain areas of the skillsets being assessed.
For example, many individuals undertaking certifications do not have English as their first language, so it is vital to make sure that the use of the English language enables people from different linguistic backgrounds to have a fair and valid testing experience. Organisations need to be able to bring opportunities for candidates to take certification examinations and to enhance their career and life opportunities to wherever they reside, right across the globe.
Modern and evolving testing solutions have a number of essential attributes. The test development process will begin with job task analysis. Then, subject matter experts, assisted by the development team, will be ready to write a blueprint for the exam, covering the competencies, experience or knowledge to be tested. The next task is to write the questions, edit them and equate them for difficulty level across the domain and sub-domain areas. Question types might include true or false, multiple choices, short answer questions, essay questions and innovative items such as simulation, video and audio, etc.
Testing solutions, development and delivery, must be secure, not least because there is significant intellectual property contained within a question bank for an examination. There may be 1000’s of questions within a question bank where an examination typically consists of less than 100 question, all of which are pre-equated in terms of difficulty levels across the primary skill areas.
A continuous monitoring and maintenance programme is important for all testing and certification programme. Detailed psychometric analysis of the performance of an examination and the questions contained therein can, for example, identify where questions may have been over exposed to candidates, or where questions are not performing as predicted. This enables a process of continuous improvement and refinement within the overall certification programme, ensuring that it remains ‘fit for purpose’. It also facilitates a process of continual update to the examination to refine and update content to reflect changes in job roles, technologies and regulatory requirements. Furthermore, testing that is treated as a continual improvement process will ensure the candidates and the organisation obtain the very best outcomes.
Top tips for test development
- Get the blueprint right. This dictates the overall content of the exam, the domain areas, the subdomain areas and the kind of skills that you need to test in each. On the basis of that you can start to write the questions to ensure you have got adequate coverage across all those domain and subdomain areas.
- Make sure you use a sizeable pool (10 – 20) of subject matter experts to write and edit the individual questions.
- Use ‘rapid item development workshops’. This is an excellent, high quality, fast track way of getting question development done quickly.
- Create a big enough question bank so that you can randomly select questions and avoid overexposing specific items. If specific questions appear within an examination too frequently, they become known and may appear in ‘cheating’ websites, seriously undermining the value of the certification programme.
- Implement effective and efficient testing delivery routes that are efficient and effective – and consistent delivery of your certifications.
- Safe and secure. Protection of your intellectual property and brand value together with the provision of a fair and equitable examination experience to each and every candidate, worldwide.
- Broad geographic coverage. Bring career and life changing opportunities to candidates wherever they are, across the world.