ProtFinder: An efficient machine learning framework for protein model selection on real data

Model selection is a critical step in phylogenetics, determining the best-fit model of sequence evolution for a multiple sequence alignment. Traditional methods like ModelFinder rely on statistical information criteria such as BIC or AIC, which are computationally expensive and have been debated. Machine learning offers a faster alternative, but the only existing ML method, ModelDetector, is trained on simulated data and performs poorly on real datasets, and it does not support rate heterogeneity across sites (RHAS) models.

ProtFinder addresses these gaps with a transfer learning strategy in three stages: initial training on large-scale simulated data, joint training on both simulated and real data, and final fine-tuning on real data only. This enables robust performance on real datasets. The framework predicts amino acid substitution models, RHAS models, and amino acid frequency models.

In experiments, ProtFinder outperformed ModelDetector in substitution model selection. It achieved accuracy comparable to ModelFinder on medium and large MSAs, slightly better than ModelFinder for RHAS model selection, and substantially better for amino acid frequency model determination. Crucially, ProtFinder is up to 1,400 times faster than ModelFinder in inference time, making it particularly suitable for medium and large datasets that are otherwise computationally demanding.