There is increasing evidence that breast cancers contain tumor-initiating cells with stem cell properties. The importance of estrogen in the development of the mammary gland and in breast cancer is well known, but the influence of estrogen on the stem cell population has not been assessed. We show that estrogen reduces the proportion of stem cells in the normal human mammary gland and in breast cancer cells. The embryonic stem cell genes NANOG, OCT4, and SOX2 are expressed in normal breast stem cells and at higher levels in breast tumor cells and their expression decreases upon differentiation. Overexpression of each stem cell gene reduces estrogen receptor (ER) expression, and increases the number of stem cells and their capacity for invasion, properties associated with tumorigenesis and poor prognosis. These results indicate that estrogen reduces the size of the human breast stem cell pool and may provide an explanation for the better prognosis of ER-positive tumors.