Title : Learning the genetic language: Transformer models for next-generation genomic selection
Abstract:
Genomic Selection (GS) is widely used in plant breeding to accelerate genetic gain by predicting breeding values from genome-wide markers. However, most GS approaches rely on linear models that primarily capture additive genetic effects, limiting their ability to represent the complex regulatory networks and gene–gene interactions underlying quantitative traits. Bridging predictive performance with biological insight remains a central challenge in plant biology.
Here, we evaluate transformer-based models as an alternative framework for genomic prediction that explicitly leverages the sequential and interdependent structure of the genome. By encoding genotypic marker data as ordered sequences, attention mechanisms can learn context-dependent relationships among loci, analogous to regulatory interactions within genetic networks. Model performance was benchmarked against ridge regression using cross-validation schemes that simulate prediction on new genotypes and environments, with accuracy assessed as the correlation between observed and predicted phenotypes.
Transformer models improved prediction accuracy relative to conventional approaches, with the largest gains observed in cross-environment predictions for traits with lower heritability. These results suggest that attention-based architectures capture components of genetic architecture not represented in linear models, including higher-order and environment-dependent interactions. Predictions were also more stable for unobserved genotypes, indicating improved robustness under extrapolation.
Importantly, analysis of attention weights revealed putative digenic interactions associated with seed yield in common bean, highlighting the potential of these models to uncover biologically meaningful relationships. By linking predictive modeling with the structure of genetic regulation, this work positions transformerbased GS as both a tool for selection and a framework for interrogating the molecular architecture of complex traits.
