Examining Nonverbal Communication in Dyadic Interactions With Virtual Humans Using an Integrated Coding System: Mixed Methods Analysis.

Background: The patient-physician dyad involves both verbal and nonverbal communication. Traditional methods use quantitative or qualitative coding when analyzing dyadic data of nonverbal communication. Quantitative coding methods can capture the frequency of nonverbal communication, while qualitative coding methods can provide descriptive information on the context and nuance of the nonverbal communication expressed. Yet, limited research has examined the integration of quantitative and qualitative coding methods of nonverbal communication between a patient-physician dyad through video recordings. Objective: The objective of this formative study was to demonstrate how nonverbal communication data can be analyzed using a mixed methods analysis approach and propose an integrated coding system using a subset of the original dataset. Methods: A secondary analysis was conducted from the intervention study with a sample of 32 pairs of randomly selected video recordings based on first and second interactions after receiving feedback from a virtual human. A 2-minute segment was used to code nonverbal communication, and a codebook was developed, informed by the literature and inductive qualitative approaches. For the mixed methods analysis, we purposefully selected 2 participants from the sample of 32 who demonstrated high frequency in quantitative and qualitative coding of nonverbal behaviors. We developed a joint display to visually represent the integration of quantitative and qualitative coding methods and developed person-level meta-inferences. Results: This formative study demonstrates an approach to nonverbal communication analysis that mixes qualitative and quantitative methods. The mixed methods results indicated the frequency of participants' (n=32) nonverbal behaviors increased after repeated interactions, including eyebrow raise, nodding, and smiling, in addition to the increased average duration of nonverbal behaviors across interactions. Illustrated through an in-depth example of integrated mixed methods coding of 2 participants from the sample, the integration of quantitative and qualitative data provided insights into nonverbal communication. Quantitatively, we captured the frequency of nonverbal behaviors while qualitatively expanding on the context for nonverbal behaviors and generating person-level meta-inferences. The joint display informed our integrated coding system for mixed methods analysis of nonverbal communication. Conclusions: The resultant integrated coding system may be helpful to researchers engaging in nonverbal communication data of dyads by providing a step-by-step method using a mixed methods analysis approach. This approach can help us to advance methods for analyzing nonverbal communication to enhance the patient-physician dyad and education on nonverbal communication. We encourage applying the integrated coding system across several subdisciplines in health sciences research to identify how it can be further expanded and refined.