Human-AI Interaction in Kidney Transplant Decision Support Systems: Qualitative Study of Patient and Support Person Expectations.

Background: Artificial intelligence (AI) is increasingly applied in medicine, including clinical decision-making. AI-based decision support systems (DSS) can enhance early risk detection and treatment optimization. However, the perspectives of patients and their support persons on AI-assisted DSS in clinical care, particularly regarding shared decision-making (SDM), remain underexplored. Objective: This study investigates the expectations, informational needs, and perceptions of patients who underwent kidney transplantation and their support persons regarding AI-assisted DSS and its influence on SDM in posttransplant care. Methods: In a longitudinal qualitative study, 36 semistructured interviews were conducted with patients who underwent kidney transplantation and their support persons at a German kidney transplant center. Participants were asked about their views on AI's role in follow-up care, its impact on communication, trust, and decision-making, as well as their informational needs regarding AI-assisted DSS. Interviews were transcribed, pseudonymized, and analyzed using framework analysis. Results: Participants recognized AI's potential to support clinicians by identifying risks of transplant loss, rejection, and infection, and by providing data-driven treatment recommendations. However, they emphasized that final decisions should remain with physicians. A majority of participants (n=28, 78%) expressed concern that AI might depersonalize care and diminish physician-patient communication due to a lack of "human touch." Participants demonstrated limited understanding of AI-based DSS functionality and highlighted the need for simple, accessible educational materials (eg, leaflets) explaining AI operations. While most doubted AI could replicate human empathy, some acknowledged that AI might be perceived as more attentive than time-pressured physicians, offering consistent monitoring and support. Participants consistently stressed that AI should augment, not replace, clinical decision-making. Conclusions: Patients who underwent kidney transplantation and support persons endorse the integration of AI in follow-up care when it enhances clinical decision-making without supplanting the physician's role. Acceptance and trust depend on transparency, accountability, and preserving the "human touch" in care. The development of educational tools to communicate AI functions and limitations is crucial to empower patients and support persons in SDM processes and to ensure AI complements, rather than undermines, patient-centered care.