Search
Close
Search
Advanced Search
Search Menu
AI Discovery Assistant

Abstract

Genetic and phenotypic variation among populations is one of the fundamental subjects of evolutionary genetics. One question that arises often in data on natural populations is whether differentiation among populations on a particular trait might be caused in part by natural selection. For the past several decades, researchers have used QST–FST approaches to compare the amount of trait differentiation among populations on one or more traits (measured by the statistic QST⁠) with differentiation on genome-wide genetic variants (measured by FST⁠). Theory says that under neutrality, FST and QST should be approximately equal in expectation, so QST values much larger than FST are consistent with local adaptation driving subpopulations’ trait values apart, and QST values much smaller than FST are consistent with stabilizing selection on similar optima. At the same time, investigators have differed in their definitions of genome-wide FST (such as “ratio of averages” vs. “average of ratios” versions of FST⁠) and in their definitions of the variance components in QST⁠. Here, we show that these details matter. Different versions of FST and QST have different interpretations in terms of coalescence time, and comparing incompatible statistics can lead to elevated type I error rates, with some choices leading to type I error rates near one when the nominal rate is 5%⁠. We conduct simulations under varying genetic architectures and forms of population structure and show how they affect the distribution of QST⁠. When many loci influence the trait, our simulations support procedures grounded in a coalescent-based framework for neutral phenotypic differentiation.

natural selection, FST, QST, coalescent
Issue Section:
Investigation > Population and Evolutionary Genetics
Editor: Y Brandvain
Y Brandvain
Editor
Search for other works by this author on:
Oxford Academic
Google Scholar

© The Author(s) 2025. Published by Oxford University Press on behalf of The Genetics Society of America. All rights reserved. For commercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com.
This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://dbpia.nl.go.kr/pages/standard-publication-reuse-rights)
You do not currently have access to this article.
Download all slides