ICLR 2026

Uncertainty-driven Embedding Convolution

Sungjun Lim, Kangjun Noh, Youngjun Choi, Heeyoung Lee, Kyungwoo Song

Yonsei University

Abstract

Text embeddings are essential components in modern NLP pipelines. Although numerous embedding models have been proposed, no single model consistently dominates across domains and tasks. This variability motivates the use of ensemble techniques to combine complementary strengths. However, most existing ensemble methods operate on deterministic embeddings and fail to account for model-specific uncertainty, limiting their robustness and reliability in downstream applications. To address these limitations, we propose Uncertainty-driven Embedding Convolution (UEC). UEC first transforms deterministic embeddings into probabilistic ones in a post-hoc manner. It then computes adaptive ensemble coefficients based on embedding uncertainty, derived from a principled surrogate-loss formulation. Additionally, UEC employs an uncertainty-aware similarity function that directly incorporates uncertainty into the similarity scoring, providing a theoretically grounded and efficient surrogate to distributional distances. Extensive experiments on diverse benchmarks demonstrate that UEC consistently improves both performance and robustness by leveraging principled uncertainty modeling.

Slides

Loading slides…

– / – Download

Method

Overview of the UEC framework showing three steps: probabilistic embedding generation via Laplace approximation, uncertainty-driven ensemble coefficient computation, and uncertainty-aware similarity measurement. — Overview of the UEC framework: UEC first transforms deterministic embeddings from multiple encoder models into probabilistic representations using Laplace approximation. These probabilistic embeddings are then adaptively combined by computing uncertainty-driven ensemble coefficients based on per-dimension variances. Finally, similarity is measured using an uncertainty-aware metric that accounts for both the mean and uncertainty of the ensembled embedding.

Experimental Results

MIRACL retrieval benchmark results showing UEC achieving performance comparable to the oracle model, with strong gains in AUC@10. — **MIRACL (Retrieval)** — UEC achieves performance comparable to the oracle and even surpasses in some cases, with particularly strong gains in AUC@10.

MMTEB benchmark results showing UEC achieving the highest average performance across retrieval, classification, and uncertainty metrics. — **MMTEB Benchmark** — UEC achieves the highest average performance across all retrieval, classification, and uncertainty metrics.

Ablation study results showing both uncertainty convolution and uncertainty-aware similarity are essential components. — **Ablation Study** — Both uncertainty convolution and uncertainty-aware similarity are essential components of the framework.

Robustness analysis showing UEC corrects individual model failures by leveraging cross-model uncertainty. — **Robustness** — UEC corrects individual model failures by leveraging cross-model uncertainty.

1 / 4

Citation

@inproceedings{lim2026uncertainty,
  title={Uncertainty-driven Embedding Convolution},
  author={Lim, Sungjun and Noh, Kangjun and Choi, Youngjun and Lee, Heeyoung and Song, Kyungwoo},
  booktitle={International Conference on Learning Representations},
  year={2026},
  url={https://arxiv.org/abs/2507.20718}
}