Enhancing Throughput and Energy Efficiency in GSM Networks Through Watts–Strogatz Graph Modeling and Reinforcement Learning

Abstract

The problem of simultaneous optimization of both throughput and energy consumption in GSM network is considered as a multi-objective problem and studied in this paper. The cell topology is modeled as a “small world” graph based on the stochastic graph model of Watts and Strogatz, and properties of the graph such as degree, centrality, traffic load and QoS indices of each cell are extracted through exploration. With these features along with the channel information of each channel and the power level of the base stations, a deep reinforcement learning agent has the state space. The DRL agent learns the control actions (transmit power, ON/OFF of cells, routing traffic between adjacent cells) so that a reward function defined as normalized throughput, normalized energy consumption and penalty on QoS violation is maximized by using a neural network based Q-value function approximation. Simulation results in the scenario of one macro cell and 12 micro cells with 200 to 2000 users show that the proposed method has a significant advantage over the three reference methods (basic, heuristic and classical Q-learning); so that the average cumulative reward of the DRL algorithm reached 3.41, while for Q-learning and the heuristic method it was 2.28 and 0.7, respectively, and for the basic method it was reported to be negative 3.55. Moreover, the energy efficiency of the network, i.e., ratio of total throughput to total power at the base-station, for the proposed method is kept at a stable value of 2.5, whereas Q-learning has a value of 2.05, and the two non-intelligent methods have values of 1.56 and 1.21, respectively. The obtained results show that the small-world graph modeling and deep reinforcement learning can be an efficient framework for intelligent energy and capacity management in GSM networks.