Are you sure your FDM results truly reflect what’s happening in your operation? Many don’t – and here’s why that matters.
Amongst all the technical gotchas and data interpretation quirks I cover on Cranfield University’s FDM courses, the single most important thing I tell students is this:
“FDM software and services undergo no approval or certification process. It is solely down to the operator to assure themselves the system is fit for purpose.”
It’s at this point the FDM vendor delegates in the room usually start to shift in their seats. I go on:
“Any one of you here could write some code today (or get AI to do it) that appears to suck in some raw flight data and then churn out what looks like believable results. It doesn’t need to do anything fancy like actually look at the data, it just needs to present a vaguely believable “top 10 big safety issues” made-up list. Now you have your FDM software, you can go and sell it to airlines around the world – job done.”
Now I doubt anyone has, or would, actually do that maliciously, but it is possible that some well-meaning company or individuals could sell an FDM “solution” that is not, shall we say, reliable. But you’re not using a system like that are you? After all, your system has been around for years and years and it’s used by airlines and aircraft operators around the world – it must be good. Hold on there a sec, and FDM vendors look away now for a minute…
Many years ago, I had the good fortune to work for the UK airline bmi (remember them?) I had the opportunity to run two different FDM systems side-by-side for a while, so I set them up with the same event limits and fed them with the same data. Both systems came from popular and successful providers and were in use with many airlines at the time.
Guess what I found? The results from each system were different; very different.
If two systems, using the same event limits and analysing the same data, produce different results, which one do you trust to drive safety action plans or justify training changes?
We’re working in a safety-critical industry and it’s important how we assess risks and allocate resources. We need to do that as accurately as possible, because we cannot afford to waste time and effort. We rely on accurate and reliable data.
Imagine embarking on a major safety improvement campaign, perhaps involving changing simulator training or standard operating procedures, all because your FDM programme used algorithms that produced believable but fictitious events. That kind of stuff keeps safety managers awake at night.
These algorithms are at the heart of your FDM system – not the flashy 3D animations that are used to sell the systems. It is the quality of these algorithms that dictates the quality of the output you get, and unfortunately, developing these algorithms is hard. Tiny errors can produce significant numbers of spurious events which are hard for an inexperienced analyst to identify. For example, half a second late detecting the unstick point can generate stacks of false “pitch high at take-off” events which look believable unless you dig deeply.
And because individually they aren’t the most attention-grabbing kind of event (just a little bit over, not close to a tail strike etc), they don’t always get thoroughly inspected. Insidiously, they make their way into your “Top 10 big safety issues” through sheer numbers alone.
You might be surprised to hear that I’m not actually advocating for an approval process for FDM systems. I believe that such approvals could stifle innovation and be a barrier to entry into the market for new vendors. So how do we ensure that FDM systems are producing quality results?
As I tell my students, it is the responsibility of the users of those systems. They have to inspect and interrogate their FDM results. They need to dig deep into the data to satisfy themselves that the underlying algorithms work properly. This takes time and experience, but the time spent doing it is much less than the time you could waste by following red herrings produced by sub-standard algorithms.
Flight Informatics can help. If you don’t have the time or in-house expertise to dig into your FDM algorithms, we can do it for you. We work confidentially with operators to verify the accuracy and quality of their systems, protecting the sensitivity of their data while helping to refine and strengthen FDM performance.
Whether you handle it in-house or with our help, make sure your FDM system is telling you the truth about your operation.