Milind Gurjar talks up the benefits of subscription learning.

For students starting a four-year technical degree today, half of what they learn in their first year of study will be outdated before they graduate. In today’s world of accelerating transformation, professional survival depends on education. To keep competitive, measurement of educational success is essential.

Historically, the focus of professional education has been on instructional training. This training is usually delivered as a transactional service, limited to occasions when individuals need to acquire new skills. But the pace of change today – especially in technical disciplines – has become so rapid that workers now need continuous education.

This need for ongoing education has driven the growth of subscription-based training. User needs for training align with three distinct educational phases:

The first phase is technology adoption. A company decides to deploy some exciting new technology. The IT team must have the right skills before they can start. So they enroll in a class to learn new skills. When the skills are acquired, the need for training is fulfilled.

It’s hard to follow up with a learner six months later to measure real-world training effects – especially in a subscription model where the timing of training may be different for each individual.

The second educational phase is ongoing management. Years ago, workers learned trades through apprenticeships. When their training was complete, they had all the knowledge needed to practice their trade for the rest of their professional life. Today it’s estimated that the amount of new technical information doubles every two years.

When it comes to IT, the change never stops. Software and hardware are updated, security patches and protocols are implemented, and systems are expanded and integrated.

It’s easy to imagine an IT professional needing to update their skills with information that has changed since their initial training. In the midst of doing their job, they need an occasional lifeline to help them stay productive.

The third phase of education – career development – is another product of our rapidly changing work environment. To stay competitive, companies need workers’ skills to be up-to-date. Individuals work to keep their skills current to reduce career risk. Others see new skill acquisition as a path to career advancement.

These three phases, technology adoption, ongoing management, and career development, align with the different models for education delivery. For the most part, technology adoption aligns with the transactional model. A defined set of skills is learned to complete a project.

The other two phases of education, ongoing management and career development, need continuous learning, with constant access to skills, information and knowledge. By nature, these are better fitted to a subscription model.

As more education becomes subscription-based it’s important to think about how best to measure its effectiveness and success. Educational theorists have suggested that training can be evaluated on five levels:  

• Reaction: how students feel about the training or learning experience.
• Learning: how the student’s knowledge has increased after training. 
• Behaviour: how much the student is applying what they learned once they’re back on the job.
• Results: the impact on the business environment.
• ROI: the financial return on the investment in training.

Reaction is probably the easiest level to measure. It’s simple to ask, “How effective did you think this training was? How useful was the course material?” And this provides a useful yardstick on training effectiveness.

But the other four levels are more difficult to measure. It’s hard to follow up with a learner six months later to measure real-world training effects – especially in a subscription model where the timing of training may be different for each individual.

Another way to measure is to focus on outcomes. In IT training, one outcome to measure is increased employee productivity. The second is improved uptime. It makes sense that there should be a correlation between training and productivity.

Increased network uptime might seem like a simple measure, but it’s not always easy to establish causality. If there is an improvement in uptime, was it because of training? Or was an expert hired? Or was it because a vendor fixed a critical bug?

RELATED CONTENT

• Five ways to get your employees to change​
• The keys to boosting performance​
• Millennials: Are they trained for work in the digital age?

Perhaps the best approach to measuring educational success is to go back to the three phases of education and evaluate training with the needs of the learner and of the company in mind. For technology adoption, two measurement approaches stand out.

The first is time to value. If a trained team can complete a project in four months, while an untrained team takes six to eight, this is a clear demonstration of value. Another approach is to compare error rates on new projects between trained and untrained individuals.

What does it cost to find errors and fix them later on? How much money can the company save by reducing errors? This may be difficult to measure with precision, but no one would argue that training doesn’t reduce mistakes.

For the other two educational phases, ongoing management and career development, it’s important to consider the specific needs of the individual receiving the training. Subscription-based education is typically much more focused on the individual than instructional learning.

Individuals frequently make their own decisions about what they want to learn. Often they proceed at their own pace. At the end, only the learner knows whether the training has met their need.

So effective evaluation of subscription-based learning must, by necessity, be more focused on the individual. It must answer the questions: What were the individual’s initial goals and expectations? Has the training helped them meet these goals?

What elements of the training were most effective and least effective in meeting these goals? Ultimately, how has the training helped the individual on their journey to success?

The genius of subscription-based learning lies in its ability to deliver education on demand in a way that meets the needs of the individual learner. The best way to measure the effectiveness of subscription-based training is to do it from the individual’s point of view.

About the author

Milind Gurjar is senior director and general manager for Learning@Cisco.