Abstract:
The rapid spread of misinformation on online platforms, which is often presented through a combination of text and manipulative imagery, requires sophisticated multimodal detection methods. This project introduces a multimodal fake news detection system designed to effectively analyze and integrate features from both text and visual data. The core method uses a BERT (bert-base-uncased) Transformer model to extract deep contextual semantics from text and a custom convolutional neural network (CNN) to capture important visual features from associated images. These two different feature vectors are then combined using a concatenation technique and classified using a fully connected Fusion Classifier. The final, trained PyTorch model is seamlessly deployed as a real-time web application using the Flask framework, providing an accessible andpractical tool for users. Initial evaluation on an unbalanced social media dataset demonstrated the potential of the multimodal approach, although performance metricsincluding Accuracy approx. and F1 Score were limited by severe class imbalance (Recall was 1.00 for the Fake class), highlighting the significant bias of the model towards the majority class. This report describes in detail the architectural design, implementation steps, and critically analyzes the results, proposing concrete strategies such as class weighting and image encoder upgrades as necessary future work to increase the robustness of the system and generalize its predictive power.
