SMS / How to Measure Effectiveness of Aviation Safety Training

Recognizing Good Aviation Risk Management Training Is Crucial

Having good aviation risk management training is perhaps one of the most important ways to improve safety in your organization.

Good training is also not inexpensive, and it takes you and/or your employees away from regular duties. In other words, nobody wants to waste their time and money. Simply put, bad training is a risk that needs to be accounted for.

But you need to be able to evaluate the effectiveness of aviation safety training. This starts by looking at the syllabus and paying careful attention to three things:

  1. That the learning outcomes/goals;
  2. The methodology; and
  3. Structure of the course

When looking at these three things, here are 3 primary ways to evaluate the effectiveness of aviation safety training.

Know good aviation risk management training when you see it

1 – Pre and End of Training Assessments

If you really want to know how effective an aviation risk management training course is you need look no further than a pre-course assessment and end-of-course assessment. Without both assessments, you will have very little idea about how effective the training was in improving your abilities/knowledge.

Having end-of-course training is useful, but not good enough. With just end-of-course assessments, you have no reference point. For example:

  • Let’s say you receive a 90% score on a final assessment
  • That sounds like the training course did a great job; but
  • If you had also taken a pre-assessment and received an 85%, then the training actually only marginally improved your abilities.

Pre and end-of-course assessments also provide other quantifiable long term benefits for your company that are useful for safety performance monitoring;

  • Track average training assessment scores (for different training topics) for your organization over time;
  • Track individual employees scores over time; and
  • Monitor average pre to end assessment improvements to assess whether current training is effective or not.

Assessments before and after training are both crucial for understanding the quality of an aviation risk management training course. If a training course does not include these elements, you may want to look elsewhere.

2 – Practical Application vs Definitions or Concepts

Without question, this is the most problematic aspect of most training courses. It’s as simple as this – learning outcomes are mostly:

  • Being able to give definitions;
  • Understanding safety concepts; and
  • Understanding the meaning of ideas.

All three points are important aspect of any training course, and they most definitely need be integrated into the training curriculum. But they should NOT be the primary goals of any training. The primary goals of any training SHOULD be:

  • Application of concepts;
  • Practical use of ideas; and
  • Ability to identify “definitions” in the operational environment.

All three above points will be immensely more useful in the actual work environment. Training that focuses on these points takes the training a whole step further than merely remaining on an “academic” (i.e., definitions, concepts, ideas) level. Academic training will be forgotten right away.

3 – Safety Performance Before and After Training

The last thing you will want to do to monitor the effectiveness of aviation safety training is monitor relevant safety data in the months leading up to the safety training, and then compare that data to the same metrics in the months after the safety training. If the training was effective, you will see positive changes in your safety metrics.

For example, if you are monitoring the effectiveness of hazard identification training, you would want to gather several metrics that indicate the hazard reporting culture before the training, such as average issues reported per employee, average number of issues reported per day, average severity of issues, etc. from the months BEFORE training. Then evaluate those same metrics for the months AFTER training. If the training was a success, the average number of reported issues will increase, and the average severity will decrease.