Trustability in ML: What if you have no choice but to build a black box?
Oct 13, 2022 | Jagannath Rajagopal | 4 min read
A strategy to cope.
In the fields of science, computing, and engineering, a “black box” is a device, system or object which can be viewed in terms of its inputs and outputs, without any knowledge of its internal workings.
The trend in deep learning is to build deeper and deeper neural networks that contain several layers. The idea is to create one model for a large multi-stage problem, and train it. Many tech companies have huge teams dedicated to managing their large scale models.
The tech giants do it that way because it’s effective. Deep neural nets have consistently and significantly outperformed their peers at the tasks they are set out to resolve. Be it image recognition or cancer detection. Sometimes, there are too many ways to break up a problem. In earlier stages when extracting features, there are too many possible combinations. This can result in a scenario that does not scale well. The temptation to let a machine figure it all out is high!
There are downfalls to utilizing a deep net approach. The larger the architecture, the harder it is to decipher why the system produced the result it did. And as the builder of this system, that could lead to some discomfort.
Consider this: The perils of building a deep net/black box. You know what you put in. You just don’t know what it did with it, and you can’t always explain what it puts out.
I’ve created a LOT of resources on this topic. Here’s my course on Design Thinking for Hero Methods. Here’s my YouTube channel with previews of course videos. Here’s my website; navigate through the courses to find free previews & pdfs.
- - -
Let’s assume you’ve built a model for mission control operations for a spacecraft. In a traditional mission control scenario, you have several readouts that indicate whether your vessel is okay. But let’s assume that the model you built simply took in and consolidated all of that data with no outputs other than that the shuttle was operating properly and not in danger. Sure, maybe the shuttle would be getting where it’s going — but if you don’t have any data on what exactly is happening, or why — you could eventually run into some problems.
If there is a malfunction of some part of your model: in either interpreting the status of the craft, or in sharing the correct data about the craft, how will you answer as to why this issue is occurring? Or furthermore, how to fix it?
This is the Can’t look = Can’t defend scenario. And this ladies and gentlemen is how we end up lost in space.
But at Kado, we believe in what works! And there are indeed solutions to some of the perils of a black box model architecture.
If you end up with a model that does work like a black box, and it’s difficult to break up into further pieces — for example if the data you receive from each of the individual pieces of code still doesn’t yield anything easily understood — then you’ve got to tell a story.
We’re going science 101 here boys and girls-say it with me now, “You need the data from your conclusion to support the hypothesis, or your theory is NOT proven!”
You must build a solution so that it has information sourced from outside your model, that supports conclusions drawn from the model’s output so that they are more palatable to the end user.
On the other hand, in order to avoid the pratfalls of accidental black box architecture, it’s good practice to build this into your charter. The options range from producing the raw output of the model to telling a full story.
This is how we see the world
Machine Learning is King, but of narrow territory. Hero Methods do things that ML cannot. Taken together, not only do they help solve complex problems, they also lay the doorway to AI.
Get in touch
-
Milton, ON L9T 6T2, Canada
-
help@kado.ai
-
+1 416 619 0517
Copyright © 2024
Super!
An email is on it's way.
Super!
An email is on it's way.
Created with
