Framework for Intelligent navigation in Iraqi urban maps using deep reinforcement learning with LSTM

Abstract

Autonomous navigation is a complex mission in urban environments and represents an important challenge for intelligent transportation systems and autonomous vehicles, especially in dynamic traffic conditions, complex road topologies, sparse reward problems, and the need for long-term robust decision-making. The recent Deep Reinforcement Learning (DRL) gives promising performance in autonomous navigation in most existing studies that evaluated on a generic simulation environment and showed limited ability on an elastic road network, especially in developing regions. There is a great research gap in developing a navigation framework that is capable of working in complex real-world city environments.
This work aims to develop a navigation framework that improves navigation efficiency , convergence stability , and generation capability in dynamic Iraqi urban environments. The proposed framework integrates Proximal Policy Optimization (PPO) and Long Short-Term Memory (LSTM) with Curriculum learning to address the challenges of sparse rewards, dynamic obstacles, and decision making. To achieve these objectives, a custom dataset named IraqiUnav was constructed using a road network extracted from OpenStreetMap for six major Iraqi cities, such as Baghdad, Mosul,  Basra, Najaf, Diyala , and Erbil. The navigation environment was simulated using Carla and the SUMO platform. First, we modeled the traffic conditions , obstacle distribution , and road network . The navigation problem was formulated as a goal-conditioned Markov Decision Process, and the proposed Curriculum LSTM-PPO was trained and evaluated across multiple navigation scenarios. 
The experimental results showed that the designed framework achieved 91.4% success rate and reduced the collision rate to 3.5%. our work performs better than PPO, PPO-LSTM, SAC and classical A*. the integration of curriculum and LSTM improved temporal reasoning and navigation consistency in dynamic environments and cross city evaluation proven strong generalize ability across invisible cites maps.
The results indicate that combining oof deep learning and temporal mechanisms and curriculum training can significantly improve autonomous navigation performance in complex environments. The proposed work offers a practical and scalable solution for future intelligent transport systems and autonomous vehicles, especially in irregular road areas and dynamic conditions.