Wheat growth parameters prediction based on dual output Bayesian neural network using multi-modal information.

Introduction: eaf area index (LAI) and leaf nitrogen accumulation (LNA) are key indicators of wheat growth and nitrogen nutritional status. However, existing prediction methods predominantly rely on single-modal information and single-output models, limiting their ability to characterize the complex structural and physiological traits of crops. This study aimed to develop a multimodal learning framework for the simultaneous and accurate prediction of wheat LAI and LNA. Methods: Spectral, image, and canopy structural features were extracted from wheat canopies across different cultivars, nitrogen treatments, and growth stages. A canopy height correction-based preprocessing method was developed to improve the extraction of structural features. A Dual-Output Bayesian Neural Network (DO-BNN) was then constructed to simultaneously predict LAI and LNA. In addition, an Extreme Sample Mining (ESM) strategy and a joint loss function were introduced to strengthen the learning of complementary information across modalities and the intrinsic correlation between the two target variables. Results: The DO-BNN achieved its best predictive performance when all feature modalities were fused. The coefficients of determination (R²) for LAI and LNA were 0.89 and 0.77, respectively, while the corresponding relative root mean square errors (RRMSEs) were 0.15 and 0.35. Compared with single-modal and conventional single-output approaches, the proposed method provided more accurate and robust predictions of both wheat growth parameters. Discussion: The results demonstrate that integrating spectral, image, and structural information can improve the characterization of wheat canopy traits. By jointly modeling LAI and LNA, the DO-BNN effectively exploited the physiological relationship between crop growth and nitrogen accumulation. The proposed framework provides a promising approach for the high-accuracy, collaborative monitoring of wheat growth and nitrogen nutritional status.