Abstract:
The integration of Artificial Intelligence (AI) in education has transformed teaching and learning, offering new opportunities to enhance student experiences. However, understanding how students perceive this integration remains a critical concern. This study examines the prediction of students’ satisfaction with AI in education using machine learning classifiers applied to an imbalanced survey dataset, where responses were unevenly distributed across satisfaction levels. Initial evaluations revealed that while classifiers such as Random Forest, Logistic Regression, SVM, AdaBoost, Decision Tree, KNN, and KNN achieved high accuracy scores, their performance in capturing minority satisfaction responses was limited, as reflected in low F1-scores. Several classifiers on an imbalanced survey dataset using different balancing methods, including SMOTEENN, SMOTE oversampling, and class weight adjustment. Although accuracy appeared high across models (0.93–0.99), it was not reliable because it primarily reflected majority class performance and overlooked minority class predictions. F1-score, which accounts for both precision and recall, provided a more meaningful measure for imbalanced data. Based on F1- scores, SMOTEENN proved most effective, with Random Forest achieving the highest score of 0.99, followed by SVM, Decision Tree, and KNN (0.96–0.97). These findings suggest that advanced class balancing techniques are essential for reliable prediction of student satisfaction in AI-enabled education. Moreover, the study emphasizes that F1-score provides a more meaningful measure of model effectiveness than accuracy in imbalanced educational survey data. The study offers actionable insights for institutions seeking to optimize AI integration and enhance student satisfaction.
