Advanced Video Action Recognition using Hybrid models of ConvLSTM with 3DCNN Deep Learning Approach

Abstract

Video Activity Recognition (VAR) is important in people's daily lives due to their ability to learn an extensive amount of high-level information about human behavior. Video Activity Recognition is to identify a collection of human actions, a model of supervised learning must be trained, as well as the action or activity outcome must be displayed by the input action that was received. CNNs are extensively utilized in image analysis, while LSTM networks function as superior versions when it comes to sequence analysis and prediction; nevertheless, when we mix both, we obtain the best versions of both CNNs, and LSTM. Here, we use the UCF101 - Action Recognition Dataset, Sports1M Dataset, and YouTube Sports Dataset to test and train the models for deep learning as 3DCNN, ConvLSTM, and LRCN. ConvLSTM employs a methodology that is equivalent to the LSTM method, particularly the simultaneous use of computation and processing of results. CNN and LSTM are combined in LRCN; however, it is a distinct structure rather than a layered one. 3DCNN efficiently extracts spatiotemporal features directly from video frames, capturing appearance, motion, and depth information simultaneously. This research presents a novel approach based on a hybrid deep learning model using 3DCNN, CONVLSTM, and LRCN for human action recognition to determine comprehensive spatiotemporal features for video activity recognition. The model with the highest accuracy among the two previously mentioned models and the proposed model is the best model, and it is used to assess the accuracy model's ability to predict human actions on YouTube videos. Experimental results demonstrate the efficacy of the proposed hybrid model on benchmark datasets, showcasing its ability to attain the best possible results in tasks involving the recognition of video actions.