Self-optimization in distributed manufacturing systems using Modular State-based Stackelberg games

In this study, we introduce Modular State-based Stackelberg Games (Mod-SbSG), a novel game structure
developed for distributed self-learning in modular manufacturing systems. Mod-SbSG enhances cooperative
decision-making among self-learning agents within production systems by integrating State-based Potential
Games (SbPG) with Stackelberg games. This hierarchical structure assigns more important modules of the
manufacturing system a first-mover advantage, while less important modules respond optimally to the leaders’
decisions. This decision-making process differs from typical multi-agent learning algorithms in manufacturing
systems, where decisions are made simultaneously. We provide convergence guarantees for the novel game
structure and design learning algorithms to account for the hierarchical game structure. We further analyse the effects of single-leader/multiple-follower and multiple-leader/multiple-follower scenarios within a Mod-SbSG. To assess its effectiveness, we implement and test Mod-SbSG in an industrial control setting using two laboratory-scale testbeds featuring sequential and serial–parallel processes. The proposed approach delivers promising results compared to the vanilla SbPG, which reduces overflow by 97.1%, and in some cases, prevents overflow entirely. Additionally, it decreases power consumption by 5%–13% while satisfying the production demand, which significantly improves potential (global objective) values.