Machine learning prediction of long-term sickness absence due to mental disorders using Brief Job Stress Questionnaire data.

Long-term sickness absence (LTSA) is a significant issue, causing productivity decline, financial difficulties, and increased mental health issues, with mental disorders being the most common cause. Occupational stressors are also linked to increased risk of LTSA due to mental disorders (LTSA-MD). This study uses occupational stressors data, assessed using the Brief Job Stress Questionnaire from 2011 to 2022, to predict LTSA-MD using machine learning and sampling methods, assessing their performance. This study analyzes data from 231,425 Japanese public servants from 2011 to 2022, focusing on LTSA-MD incidents. We compared five machine learning models and six sampling methods (random sampling, equal size sampling, SMOTE-synthetic minority oversampling technique, bootstrapping, borderline-SMOTE and ADASYN-adaptive synthetic sampling) to predict LTSA-MD incidents, addressing class imbalance. We prioritized average precision (AP) to identify the most promising model–sampling combinations to give the severe class imbalance. The gradient boosted trees model and bootstrap oversampling method demonstrated highest AP among all integrations of machine learning and sampling methods, with a AP of 0.040 and a ROC-AUC of 0.81. However, no significant difference in superiority was observed between the combinations of higher-level AP machine learning and sampling methods. The results demonstrate that machine learning models’ predictive ability for LTSA-MD is generally low, requiring further research.