Multi-Agent Reinforcement Learning-Based Automated Incident Response for Secure Digital Twin Environments

Abstract

The DT environment allows for real-time synchronization between the physical and virtual world, which makes it extremely vulnerable to advanced cyber-attacks. Most of the security techniques used so far emphasize the ability to detect attacks but are weak when it comes to responding to them in an automated and dynamic manner. To tackle this challenge, in this research, we propose an MARL-based automated response solution for Digital Twin security applications. The MARL model uses a decentralized agent architecture where the agents learn how to respond optimally under different circumstances. We formulate the problem as a multi-agent markov decision process, and use the Q-learning approach combined with the idea of experience replay. Performance of the developed solution will be measured with various criteria, such as accuracy, threat mitigation rate, attack success rate, response time, system downtime, cumulative rewards, and system resilience. The findings from the experiments clearly show that the suggested model is able to obtain an accuracy of 94.4%, threat mitigation capability of 93.1%, and lower response times than previous models. Moreover, the learning curve clearly illustrates that stable convergence and better optimization of policies are obtained over episodes. The suggested MARL-based framework is efficient in providing automated incident response for Digital Twin ecosystems.