Human-AI co-design for clinical prediction models.

Developing safe, effective, and practically useful clinical prediction models (CPMs) traditionally requires extensive collaboration between clinical experts, data scientists, and informaticists to refine the many small but critical details of the model building process. When incorporating unstructured clinical notes, this challenge magnifies, as notes essentially contain an infinite number of concepts that can be used for modeling. We introduce HACHI, an iterative human-in-the-loop framework that uses AI agents to accelerate the development of fully interpretable CPMs from clinical notes, where CPMs are defined as linear models of yes-no questions. HACHI alternates between an AI agent that uses statistical tools and embedded knowledge to explore candidate concepts and domain experts who provide feedback to the AI agent. The framework optimizes for transparency, steerability, and reciprocal learning to ensure effective collaboration between the clinical AI team and the AI agent. In two real-world prediction tasks (acute kidney injury and traumatic brain injury), HACHI outperforms existing approaches, discovers clinically relevant concepts, and improves model generalizability across clinical sites and time periods. HACHI also highlights the critical role of human oversight, such as in directing the AI agent to explore new concept categories, adjusting concept granularity, and identifying data bias and leakage.