Decomposing your complex AI problem: Hierarchy
Oct 13, 2022 | Jagannath Rajagopal | 4 min read
Problem worlds often come with an innate hierarchy. Naturally, this may prompt the question: which level(s) of the hierarchy should be modelled? For example, the US Stock Market can be modelled as a whole or at the index level — think, the Dow Jones, or for individual stocks.
In a linear system, the way that the lower levels interact with the upper levels is “linear” or directly correlated. Take the example of an analytics system for business intelligence and reporting — sales, inventories, etc. The numbers of a product produced by the manufacturer and sent to the various distributors should all add up, as should those distributed to — and sold by — each individual retailer. The manufacturer, the distributors, and the retailers have a direct correlation to each other. In a non-linear system — if you were to look at your skin for example — the way that cells interact is not related linearly to the way that an organ functions; thus this is a non-linear system.
In addition, the objectives of your model may be different based on their hierarchy level. An aggregate model of a country’s economy relies on averages. By contrast, a model built for the economy of a small town would need to take into account the effect of local business networks and relationships, it cannot run based on averages alone and expect to be accurate.
So what happen if you can’t find a good pattern to analyze at your chosen hierarchy level? Try other ones. What if nothing works? Then go data driven with your analysis and identify groups based on how the data organizes itself.
What if that doesn’t work?
In this case, you don’t have any hierarchy-based patterns in the world data, and potentially no hierarchy levels to analyze. However, the chances are that you’ll find a pattern somewhere.
I’ve created a LOT of resources on this topic. Here’s my course on Design Thinking for Hero Methods. Here’s my YouTube channel with previews of course videos. Here’s my website; navigate through the courses to find free previews & pdfs.
In certain instances you may need to model more than one level at the same time. Simulation-based models can easily with deal with different complexity layers and tend to be multi-scale modelled.
Computational mechanics methods have the ability to model everything from materials to compound pieces and components.
Take the example of systems biology where there are models at many levels: from molecular dynamics, to brownian movement of molecule diffusion, to protein folding, to cell models, to organs, all the way up to individuals and ecosystems.
Hierarchical modelling may also require propagation of modelling results at every level of hierarchy. In an agent-based simulation of the stock market, agents may need to model a national stock-exchange behaviour and the behaviours of individual traders, but both levels are interdependent and propagable. From the bottom up, the model must aggregate the input from traders, and from the top-down the model must interpret context implications.
At Kado, our goal is to show you how to structure a problem. First, you define the problem itself; consider aspects that define the basics, let you trade-off/prioritize, and then operationalize it. Next, you define it’s world; we borrow from Russell & Norvig for this. You are then ready to break the problem into pieces. Hierarchy is a key part of the final step.
This is how we see the world
Machine Learning is King, but of narrow territory. Hero Methods do things that ML cannot. Taken together, not only do they help solve complex problems, they also lay the doorway to AI.
Get in touch
-
Milton, ON L9T 6T2, Canada
-
help@kado.ai
-
+1 416 619 0517
Copyright © 2024
Super!
An email is on it's way.
Super!
An email is on it's way.
Created with
