Abstract:
In an attempt to reach the end product of rich sentiment categorization, this research offers an end-to-end sentiment analysis system which incorporates the state-of-the-art in deep learning methodsin addition to traditional machine learning algorithms. Understanding of the public opinion has become critical in decision-making processes in areas such as politics, business and social observation due to the increased growth of user-generated information in places such as twitter. This study is based on Kaggle twitter sentiment dataset which was rigor legacy as data cleaning, normalization, tokenization, TF-IDF vectorization and class balancing using SMOTE to ensure reliability and reduces the biasness. This was caused by the demand for correctly and general-purpose systems of sentiment analysis. A variety modelssuch as Multi-Layer Perceptron (MLP), deep learning model, interpretable machine learning classifiers such as; KNN, Decision Tree, Random Forest, Extra Trees (ETC) and advanced techniques employing gradient boosting algorithm such as XGB, LGBM and CatBoost with the help of the processed data were trained. The major novelty of this study is the designed integration of the two paradigms presenting the synergistic workings of neural architectures and ensemble methodsin boosting for dealing with structured and capturing complex semantic relationships. While older models had offered efficiency and interpretable good models, the gradient boosting and ETC also regularly defeat other models with the ETC having the best accuracy of 99.12% as per a comparative assessment using the accuracy, precision, recall and F1, confusion matrices. The results obtained show that choosing the type of algorithm should be a tradeoff between processing resource, interpretability and complexity, and deep learning algorithms should be implemented in case of subtle identification of sentiment while simpler models are enough for getting quick insights. All thing said, this work allow to make repeatable pipeline with key focus on importance of preprocessing, feature engineering and balancing class for downstream success.
