State-Synchronization-Aware Deep Reinforcement Learning for Autoscaling Stateful Kubernetes Workloads

Abstract

Deep reinforcement learning (DRL) has been proposed repeatedly as a predictive alternative to threshold-based Kubernetes autoscaling, and recent DRL and recurrent-network approaches report strong results for stateless workloads. In practice, however, the majority of this literature treats horizontal scaling as a purely stateless compute problem. Modern enterprise microservices are frequently stateful: they maintain open WebSocket connections, manage distributed in-memory caches, and hold partitioned event-stream offsets. When an orchestration layer scales these stateful pods based solely on a generic reward signal, it can trigger split-brain conditions and corrupt state synchronization across distributed backing stores.

This paper presents an architecture that couples predictive, DRL-driven autoscaling with an explicit state-synchronization guarantee at the point of pod termination. A Proximal Policy Optimization (PPO) agent is trained to select replica-count adjustments from live cluster telemetry, and its reward function is extended with a synchronization-fault penalty term that is evaluated jointly with service-level and over-provisioning penalties. Scale-down actions are executed through a deterministic four-step handoff, comprising traffic drain, synchronous state flush, partition reassignment, and delayed termination, rather than an immediate pod kill.

The approach was evaluated on a fifty-node bare-metal Kubernetes testbed under a simulated flash-crowd traffic surge representing 5G telemetry ingestion. Relative to the native Horizontal Pod Autoscaler baseline, the proposed controller reduced P99 response latency from 4,200 milliseconds to 1.2 milliseconds during the surge, eliminated the measured 45-second cold-start penalty through pre-emptive provisioning, and recorded zero state-synchronization faults across the evaluation window compared to fourteen split-brain events for the baseline. An ablation that removes the synchronization-fault reward term confirms that this term, not the predictive scaling behavior alone, is responsible for the elimination of split-brain events. These results indicate that explicitly modeling state-synchronization cost inside the scaling policy, rather than treating it as an orthogonal operational concern, is a practical route toward predictive autoscaling for stateful cloud-native workloads.