Abstract:
This work investigates the use of sophisticated pre-trained language models, such as
DistilBERT, DistilRoBERTa, and BETO, for heart disease analysis in the healthcare
domain, especially for the classification of heart illness using textual data extracted
from Colour Doppler Echocardiogram reports. The study used the "finite
automata/Beto- heart disease -analysis" tokenizer specifically designed for the Spanish
medical language to preprocess patient data for improved model training and
assessment. The experimental configuration used state-of-the-art GPUs, fine-tuned
hyperparameters, and the Hugging Face Transformers library. DistilRoBERTa
demonstrated superior performance compared to DistilBERT and BETO, with an
accuracy of 93.38% as opposed to 91.17%. These models exhibited substantial
improvements in accuracy, recall, and F1 score when compared to conventional
machine learning approaches. The research emphasizes the significance of model
design and training methodologies, emphasizing the enhanced contextual
comprehension of pre-trained models, resulting in improved accuracy in heart disease
categorization. The sustainability plan encompasses the environmental, economic,
social, ethical, and community dimensions to guarantee the responsible and enduring
utilisation of these models in healthcare. Some important tactics to consider include
optimizing resource allocation, implementing cost-efficient solutions, adhering to
ethical principles in AI, and fostering cooperation across different disciplines. This
study makes a substantial contribution by conducting a thorough examination of pre-
trained language models in heart disease analysis and proposing techniques for their
long-term implementation. The results demonstrate the potential of these models to
revolutionize healthcare by improving diagnostic precision and patient care via
sophisticated heart disease analysis. Future research should prioritize the optimization
of model training, the extension of applications to other medical fields, and the
maintenance of ongoing development in accordance with technology improvements
and regulatory needs.
