An Advanced Video Surveillance Framework with Artificial Intelligence Driven Human Activity Recognition and Behaviour Understanding

Abstract

It is extremely difficult for modern surveillance systems to accurately and efficiently interpret complex human behavior. Traditional monitoring is dependent on the human operator, and operators tend to get tired of continuous monitoring. This study offers an advanced framework of artificial intelligence-based human activity recognition. The system is composed of a combination of detection by YOLO, tracking by DeepSORT, and pose estimation by skeletal pose estimation. The behaviour is classified using a graph convolutional network with extracted skeletal sequences. The system differentiates between normal and potentially dangerous or suspicious activities. Experiments showed 94% of the images were correctly detected in a variety of surveillance applications. Tracking kept a 90% identity consistency, which allowed for a reliable, continuous behavioural analysis overall. The accuracy of the pose estimation was 91% of keypoint accuracy, and it allowed for a privacy-preserving action recognition. Activity classification was at 92% accuracy when walking, running, falling, and fighting were combined. Real-time anomaly detection was able to detect 89% of the anomalies with a speed of 2 seconds. Comparative analysis showed that there were obvious advantages over the conventional rule-based systems. These encouraging experimental results were backed up by global comparisons across India, China, Japan, and Europe. The suggested framework also took care of privacy issues by using skeleton-based representation methods. This is in line with the existing international data protection legislation and expectations. Results show that a scalable and real-time deployment of a surveillance system with respect to privacy is feasible. This work is valuable to the intelligent, automated public safety monitoring solutions community around the world.