When pluripotent cells are exposed to a uniform culture environment they routinely display heterogeneous gene expression. Aspects of this heterogeneity, such as Nanog expression, are linked to differences in the propensity of individual cells to either self-renew or commit towards differentiation. Recent findings have provided new insight into the underlying causes of this heterogeneity, which we summarise here using Nanog, a key regulator of pluripotency, as a model gene. We discuss the role of transcription factor heterogeneity in facilitating the intrinsically dynamic and stochastic nature of the pluripotency network, which in turn provides a potential benefit to a population of cells that needs to balance cell fate decisions.