Normally in a discrete event simulation, you have to set rules or decision policies at points where flows bifurcate.
Since DELMIA does not include an optimization engine, the only way to find a good proposal is through "trial-and-error," via manual iterations. To do this, we can test standard policies (First Allowed, Cyclic-non-blocking, Priority, etc.) at the decision point to route flows, run the simulation for a long time under each configuration, and compare the output KPIs (Throughput, WIP, etc.).
If the approach also includes stochastic process variability or disruptions, the standard procedure would be to set up a Monte Carlo experiment to run each scenario many times and collect averages and standard deviations for a better comparison.
Finally, if standard policies are not enough, we can implement our own proposals using EKL code or Custom Logics, such as "distribute parts primarily to buffer A, but if buffer A has 3 or more parts, start distributing to B". The possibilities are endless, but in complex scenarios with a multitude of stations, flows, shared resources, etc., it is difficult to intuit a good decision rule that is also simple.
In short, this entire analysis process is very time-consuming, and we never really know if we are anywhere near a global optimum.
That is why I came up with the idea of connecting DELMIA with Artificial Intelligence through a Reinforcement Learning agent in Python, so that the algorithm itself "learns" to route parts autonomously by interacting with the simulation.
How did I build it without going crazy?
Using EKL logic, I pause the simulation at the decision point and put the system on hold (it really only takes a few milliseconds). Then, I hand over control to the RL algorithm in Python running in parallel. I notify it of the simulation's current state by writing it to a TXT file. The algorithm detects this file, reads it, and makes a decision based on what it has learned so far using a reward/penalty function, writing that decision to another TXT file. When DELMIA detects this file, it resumes control by reading the chosen decision and executing the indicated routing. Gradually, the algorithm learns what constitutes a good versus a bad decision.
From the Lab to Reality (Tests and Results)
To validate that the idea worked, I set up a very simple scenario where a stream of parts had to decide whether to feed route A or route B, purposely introducing asymmetries and stochastic breakdowns (MTBF/MTTR) in the machines on those routes.
- Traditional static rules (First Allowed or Cyclic) suffer performance drops because they keep blindly pushing parts toward bottlenecks.
- The RL agent, after receiving a few virtual "punishments" in its Q-table during the first few minutes, learned on its own to dynamically divert flow to weather the failures, outperforming fixed policies.
The most fun part is that, in the end, the analysis of the decision history showed that the AI had distilled an overwhelming logic of industrial common sense (basically, balancing buffers and using the fast machine as the main path). But the truly important thing is not the rule itself, but having proven that an external optimization engine can be coupled to DELMIA in real time and in a completely stable way.
Find attached the sample 3dxml file and the Python code. Based on this architecture, the code can be adapted to your needs.
Has anyone else here ever wrestled with trying to push DELMIA simulation beyond the limits of its standard rules? I'd love to read your thoughts or how you've solved complex routing challenges!
