Efficient Data Collection in UAV Sensors using Optimized Double Deep Q-learning Network (ODDQN)

Abstract

Introduction: In recent years, Unmanned Aerial Vehicles (UAVs) have developed as a highly promising solution for enhancing data-collection efficiency of Wireless Sensor Networks (WSNs). Most existing works on data collection by UAVs concentrate on clustering, trajectory planning, or resource allocation, ignoring congestion caused by massive simultaneous transmissions, which becomes critical in large-scale IoT deployments. Existing MAC and scheduling schemes fail to provide low latency, high throughput, and conflict-free access when the number of sensors grows.
Materials and Methods: In order to solve these issues, this paper outlines the design for a Double Deep Q-learning Network (DDQN) model to solve the problem of determining optimal data collection schedules for UAV sensors. The DDQN utilizes two identical deep neural networks to mitigate the overestimation bias inherent in standard DQN: the current Q-Network for action selection and the target Q-Network for value evaluation. The DDQN parameters are further optimized by applying backpropagation technique. The reward function is formed to maximize data freshness while minimizing operational costs.
Results: The performance of ODDQN is evaluated in terms Landing Success percentage, data collection ratio, data collection delay, packet drop rate and average residual energy.
Discussion: Experimental results show that the ODDQN model attains maximum data collection ratio with reduced data collection delay and packet drop rate.
Conclusion: Hence the ODDQN framework determines optimal data collection schedules for UAV sensors.