COALITION-INSPIRED ADAPTIVE WEIGHTING FRAMEWORK FOR MULTI-BRANCH IMAGE CLASSIFICATION USING DEEP LEARNING

Abstract

Multi branch deep learning architectures have demonstrated remarkable efficacy in learning complementing feature extraction over several routes for image classification applications. However, the majority of feature fusion techniques employ fixed aggregation techniques, which lack the flexibility to modify branch contributions according to the properties of the incoming data. This research presents an adaptive weighting system for multi-branch image classification based on parallel ResNet18 and ResNet34 architectures, inspired by coalitions. The proposed framework proposes a lightweight adaptive weighting module to adaptively estimate branch contribution coefficients and fuse features during inference based on the input data. The framework was trained with the PyTorch deep learning framework and tested on the CIFAR-10 benchmark dataset in a light-weight Google Colab computational environment. The experimental results showed that the proposed framework could reach competitive classification performance, with the accuracy of 82.99%, the precision of 0.8309, the recall of 0.8299 and the F1-score of 0.8300. To explore the representation learning ability of the proposed adaptive fusion strategy, additional analyses such as confusion matrix evaluation, adaptive weight visualization, and t-SNE feature embedding analysis were performed. Ablation studies also showed that multi-branch fusion outperformed stand-alone single-branch architectures and allowed estimating branch contribution adaptively and analyzing the data in an interpretable manner. Under the current lightweight experimental setting, static average fusion had a slightly higher accuracy, but the proposed framework proved the possibility of coalition-inspired adaptive feature fusion for interpretable multi-branch deep learning systems.