Advanced topic modeling with large language models: analyzing social media content from dementia caregivers.

Background and Objectives: While traditional topic modeling methods have been applied to analyze social media content from dementia caregivers, they often struggle with semantic understanding and coherent topic generation. This study explores the direct application of large language models (LLMs) for topic modeling of caregiver tweets, aiming to leverage their advanced semantic comprehension capabilities. Research Design and Methods: We analyzed 231 870 tweets from dementia caregivers after preprocessing using ChatGPT as the primary topic modeling tool. To address context length limitations, we developed a 2-stage approach: first splitting the dataset into 226 batches of 1000 tweets each for initial topic extraction, then combining these results through a second-stage prompt for final topic synthesis. We compared our approach against 11 baseline methods, including Latent Dirichlet Allocation (LDA), Gibbs Sampling Dirichlet Multinomial Mixture Model (GSDMM), their term-weighted variants, and state-of-the-art BERTopic models. Topic quality was evaluated using Sentence-BERT-based coherence scores, and topic comprehensiveness was assessed through both ChatGPT and human expert evaluation. Results: Our LLM-based approach achieved a coherence score of 0.358, significantly outperforming all baseline methods. Traditional approaches like GSDMM (0.317) and LDA (0.320), their term-weighted variants (ranging from 0.264 to 0.302), and BERTopic variants (approximately 0.30) showed lower coherence scores. The 2-stage batching strategy effectively handled the large dataset while maintaining topic quality and representativeness. Expert evaluation confirmed the topics' relevance to caregiver experiences and their comprehensive coverage of key themes. Discussion and Implications: This study introduces a novel methodology for applying LLMs to large-scale topic modeling tasks, demonstrating superior performance over traditional and state-of-the-art approaches. The significant improvement in coherence scores suggests that LLMs can better capture the semantic relationships within topics. Our approach addresses key challenges in context length limitations and prompt engineering, while providing more coherent and interpretable insights into caregiver experiences that can inform targeted support strategies.