Comparative performance of machine learning and deep learning models for heart disease prediction in a small clinical dataset.

backgroundMachine learning (ML) and deep learning (DL) models are increasingly applied to cardiovascular risk prediction, yet their comparative performance in small tabular clinical datasets remains uncertain. This study compared conventional machine learning models and a deep neural network for predicting angiographically defined heart disease using the original UCI Heart Disease dataset.

methodsThe dataset was loaded directly from the UCI Machine Learning Repository and included 303 patient records. The original multiclass angiographic disease variable was converted into a binary outcome indicating absence or presence of heart disease. Continuous predictors included age, resting blood pressure, serum cholesterol, maximum heart rate, and exercise-induced ST depression. Categorical predictors were defined according to UCI attribute documentation and one-hot encoded. The dataset was divided using a stratified 70/15/15 train-validation-test split. Logistic regression, decision tree, random forest, XGBoost, and deep neural network models were trained. Machine learning hyperparameters were selected using 5-fold stratified cross-validation with AUC optimization. Classification thresholds were selected on the validation set by maximizing F1-score and then applied unchanged to the independent test set. Performance was assessed using accuracy, precision, recall, F1-score, and AUC with bootstrap 95% confidence intervals. Pairwise comparisons used McNemar's test and paired bootstrap AUC testing with Holm correction.

resultsThe deep neural network achieved the highest test-set accuracy, precision, and F1-score, each reaching 91.30%, 90.48%, and 90.48%, respectively, with an AUC of 0.958. Logistic regression showed the highest recall, 95.24%, and AUC, 0.968. XGBoost achieved 71.74% accuracy and 0.825 AUC.

conclusionDeep learning and regularized logistic regression showed the strongest performance, while XGBoost did not demonstrate superiority in this small clinical dataset.