Effects of Model Choice, Corpus Context, and Post Hoc Correction on Layer-Level Embedding Degradation in Clinical Document Retrieval: Experimental Study.

backgroundClinical retrieval-augmented generation depends on embedding models. A companion study found that non-retrieval-trained encoders underperformed retrieval-trained general-purpose embeddings and produced near-degenerate embedding geometry, but did not localize the architectural origin, separate training-domain from training-objective effects, or test whether the degradation can be corrected without retraining.

objectiveThis study aimed to (1) characterize layer-wise retrieval and geometric trajectories across 13 transformer configurations on clinical documents, (2) separate training-objective from training-domain effects through matched architectural comparisons, (3) reanalyze the panel under a per-query layer-wise linear mixed-effects (LME) framework, and (4) evaluate deployment-relevant post hoc geometric correction.

methodsLayer-wise embeddings were extracted from 13 transformer configurations on 3 clinical corpora (n=100 documents each: MTSamples, PMC-Patients, and Mistral-7B-Instruct-generated synthetic notes) under 2 query formats (keyword and natural-language via GPT-4o). Retrieval performance (mean reciprocal rank at cutoff 10 [MRR@10] and recall at cutoff 10 [recall@10]) and geometric properties (participation ratio, average pairwise cosine, and anisotropy) were measured at every layer. A per-query layer-wise LME model was fit independently per configuration. Corpus-only zero-phase component analysis (ZCA) whitening was evaluated as the primary deployment-relevant intervention, with 5-fold cross-validation, an epsilon sweep, a lexical-overlap audit, and a chunking sensitivity analysis.

resultsDocument embeddings clustered into 3 anisotropy tiers: extreme (average pairwise cosine >0.92) for non-retrieval-trained encoders and large language models (LLMs), moderate (0.65-0.92) for general retrievers and most LLMs, and reduced (<0.65) for BioLORD-2023, instruction-tuned E5-Mistral-7B, and Nomic-embed-text-nopfx. The per-query random-slope LME identified 2 layer-depth patterns: classical degradation with depth in 3 non-retrieval-trained encoders (all P<.001), vs net improvement with depth in the remaining 10 models (all P<.001), with the strongest negative coefficients in decoder LLMs. Matched-contrast tests confirmed significant training-objective × layer-depth interactions in all 3 matched pairs (all P<.001). Corpus-only ZCA whitening produced a 2-tier pattern under 5-fold cross-validation: tier 2 non-retrieval-trained models showed positive ΔMRR@10 (+0.066 to +0.304), while tier 1 retrieval-trained models showed negative ΔMRR@10 (-0.021 to -0.051). Best Match 25-vs-embedding Spearman rank correlations spanned -0.02 to 0.37, indicating substantial nonlexical contribution to retrieval. Same-source ranking stability replicated at 4-5× corpus scale (ρ=0.952 for PMC-500, ρ=0.929 for MTSamples-400) for the BERT-scale subset.

conclusionsAnisotropy in transformer embeddings is widespread across architectural classes and is lower in configurations with retrieval-specific training. Corpus-only ZCA whitening is a deployment-compatible, retraining-free post hoc correction candidate that improved retrieval for non-retrieval-trained models on this controlled benchmark but requires target-corpus validation before clinical deployment. The matched-comparison evidence supports training objective rather than training domain as the stronger explanatory axis, though residual confounding is not eliminated. The principal contribution is mechanistic: layer-level localization of embedding degradation and the geometric basis for the 2-tier intervention response.