Heartbeat: a multimodal dataset of fetal echocardiography and clinical metadata for early detection of congenital heart disease.

Background: Congenital heart diseases (CHDs) remain the leading cause of infant mortality attributable to birth defects. Although artificial intelligence has demonstrated promise for automated CHD detection from prenatal ultrasound imaging, progress is constrained by the limited availability of large, high-quality datasets and the absence of multimodal resources integrating imaging with clinically relevant patient metadata. Methods: This study introduces Heartbeat, a multicenter, anonymized multimodal dataset comprising fetal cardiac ultrasound images and associated clinical metadata collected between 2019 and 2023. Standard echocardiographic views, including the three-vessel trachea, four-chamber, left ventricular outflow tract, and right ventricular outflow tract planes, were acquired according to established protocols. Multiple deep learning architectures, including convolutional neural networks and a transformer-based baseline, were trained and evaluated. Heart-ViT, a multimodal transformer-based model, was developed to integrate imaging features with patient-specific clinical metadata via adaptive layer normalization. Results: Heartbeat includes 1,475 patients, with CHD prevalences of 6.50% and 7.25% in the second and third trimesters, respectively. On the second-trimester test set ( Conclusion: Heartbeat provides a clinically relevant multimodal resource for prenatal CHD research that reflects real-world imaging variability and class imbalance. Integrating ultrasound imaging with patient-specific clinical metadata significantly enhances CHD detection compared with image-only approaches. These findings establish a reproducible benchmark and support the development of AI-assisted tools for early prenatal screening. The dataset and analytical framework will be made publicly available to support further research in prenatal cardiology and the early detection of CHDs.