SafeSphere: A Framework for Disaster Detection and Response Recommendations in Social Media

Abstract

Disasters produce a huge amount of multimodal information on social media, such as updated texts and pictures that give up to the minute information regarding the situation. But this data is noisy, unstructured, and is extremely imbalanced by disaster category, and is therefore difficult to reliably classify. In this paper, we introduce a multimodal framework for detecting and recommending disaster-related responses, SafeSphere, that utilizes both text and visual data from social media. The proposed system consists of transformer-based text encoders and convolutional neural networks for image processing, and is connected with each other in a bidirectional cross-modal multihead attention network to improve the matching of semantic information between modalities. Focal loss is used in centralized training to tackle the problem of severe class imbalance, which enhances the detection capability of the minority disaster classes. In addition, SafeSphere implements a federated learning approach: decentralized model training on multiple clients, without compromising data privacy. The model is trained with an early stopping method to avoid overfitting, using macro-F1 as the indicator. The test results for MDI dataset show accuracy of 95.03% and macro-F1 score of 91.40 %, and balanced class-wise results. Moreover, the system also converts classification to actionable intelligence, providing context-aware disaster response recommendations. The proposed framework provides a solution that can grow with demand, protects privacy, and is easy to understand for use in disaster management.