backgroundAlthough many prognostic single-gene (SG) lists have been identified in cancer research, application of these features is hampered due to poor robustness and performance on independent datasets. Pathway-based approaches have thus emerged which embed biological knowledge to yield reproducible features.

methodsPathifier estimates pathways deregulation score (PDS) to represent the extent of pathway deregulation based on expression data, and most of its applications treat pathways as independent without addressing the effect of gene overlap between pathway pairs which we refer to as crosstalk. Here, we propose a novel procedure based on Pathifier methodology, which for the first time has been utilized with crosstalk accommodated to identify disease-specific features to predict prognosis in patients with hepatocellular carcinoma (HCC).

findingsWith the cohort (N = 355) of HCC patients from The Cancer Genome Atlas (TCGA), cross validation (CV) revealed that PDSs identified were more robust and accurate than the SG features by deep learning (DL)-based approach. When validated on external HCC datasets, these features outperformed the SGs consistently.

interpretationOn average, we provide 10.2% improvement of prediction accuracy. Importantly, governing genes in these features provide valuable insight into the cancer hallmarks of HCC. We develop an R package PATHcrosstalk (available from GitHub https://github.com/fabotao/PATHcrosstalk) with which users can discover pathways of interest with crosstalk effect considered.