Abstract:
Healthcare documentation management depends on medical specialty classification to achieve efficient organization and management of medical records. The research investigates the performance of transformer-based deep learning systems when used for medical transcription classification to achieve accurate medical domain identification. The dataset used consists of a large number of real clinical notes labeled with one of 29 medical specialties. The text-based data introduces an original approach to implement NLP technology for healthcare applications. The research creates automated medical specialty identification systems from unprocessed doctor-patient conversation recordings to enhance healthcare operational efficiency and patient referral accuracy. The research starts by reviewing existing studies about medical text classification and transformer models including DistilBERT. The research includes complete details about the dataset origin and volume and all preprocessing operations that were performed. Special attention is given to handling class imbalance, where underrepresented specialties are assigned appropriate class weights during training. The data preparation process is emphasized to ensure reliability and relevance of the content used in training. The model evaluation depends on four performance metrics which include top-1 accuracy and top-3 accuracy and macro F1-score and weighted F1-score. The proposed model reached a top-1 accuracy of 54.03% while delivering a top-3 accuracy of 95.61% which proves its capacity to generate multiple suitable predictions for each input. The evaluation process of these metrics takes place at various checkpoints to enable comparison. The model's output results undergo confusion matrix analysis to identify which specialties have the most misclassification errors. The research demonstrates that transformer-based NLP models achieve success in medical documentation automation and specialty classification work. The paper presents its current boundaries and future research paths which involve using domain- specific models together with ensemble methods. The research investigates actual clinical documentation problems to develop better AI healthcare solutions and build deep learning systems that understand medical texts.
