Multi-Scale Spatio-Temporal Deformable Template Matching for Video Frame Extraction in Action Recognition

Abstract

Video frame extraction has become an important component in action recognition and video analysis; however, its performance is often degraded by non-rigid object deformations, illumination variations, occlusions, background clutter, and temporal inconsistencies. To address these challenges, this research proposes a Multi-Scale Spatio-Temporal Deformable Template Matching (MSST-DTM) framework for robust and accurate video frame extraction. The proposed framework initially converts video sequences into normalized grayscale frames and applies oriented multi-scale Gaussian derivative filter banks to model discriminative structural information across different scales and orientations. Subsequently, a dual-stage deformable template matching strategy is implemented, integrating multi-level similarity estimation and edge-based control-point alignment to enhance spatial matching accuracy. Furthermore, flow-guided deformable convolution is incorporated to maintain temporal continuity between consecutive frames and improve key-frame selection reliability. The effectiveness of the proposed MSST-DTM framework is validated using the HMDB-51 and UCF-101 benchmark datasets. Experimental results demonstrate that the proposed MSST-DTM approach achieves an accuracy of 81.38% and 98.37% on HMDB-51 and UCF-101 datasets, respectively. Experimental results demonstrate that MSST-DTM provides a reliable and computationally efficient solution for video frame extraction and action recognition applications.