Intellectual disability (ID) occurs in almost 3% of newborns. Despite substantial research, a fundamental question about its origin and links to intelligence (IQ) still remains. ID has been shown to be inherited and has been accepted as the extreme low of the normal IQ distribution. However, ID displays a complex pattern of inheritance. Previously, noninherited rare mutations were shown to contribute to severe ID risk in individual families, but in the majority of cases causes remain unknown. Common variants associated with ID risk in the population have not been systematically established. Here we evaluate the hypothesis, originally proposed almost 1 century ago, that most ID is caused by the same genetic and environmental influences responsible for the normal distribution of IQ, but that severe ID is not. We studied more than 1,000,000 sibling pairs and 9,000 twin pairs assessed for IQ and for the presence of ID. We evaluated whether genetic and environmental influences at the extremes of the distribution are different from those operating in the normal range. Here we show that factors influencing mild ID (lowest 3% of IQ distribution) were similar to those influencing IQ in the normal range. In contrast, the factors influencing severe ID (lowest 0.5% of IQ distribution) differ from those influencing mild ID or IQ scores in the normal range. Taken together, our results suggest that most severe ID is a distinct condition, qualitatively different from the preponderance of ID, which, in turn, represents the low extreme of the normal distribution of intelligence.