The basal ganglia play a central role in regulating the response selection abilities that are critical for mental flexibility. In neocortical areas, higher cardiorespiratory fitness levels are associated with increased gray matter volume, and these volumetric differences mediate enhanced cognitive performance in a variety of tasks. Here we examine whether cardiorespiratory fitness correlates with the volume of the subcortical nuclei that make up the basal ganglia and whether this relationship predicts cognitive flexibility in older adults. Structural MRI was used to determine the volume of the basal ganglia nuclei in a group of older, neurologically healthy individuals (mean age 66 years, N = 179). Measures of cardiorespiratory fitness (VO_2max), cognitive flexibility (task switching), and attentional control (flanker task) were also collected. Higher fitness levels were correlated with higher accuracy rates in the Task Switching paradigm. In addition, the volume of the caudate nucleus, putamen, and globus pallidus positively correlated with Task Switching accuracy. Nested regression modeling revealed that caudate nucleus volume was a significant mediator of the relationship between cardiorespiratory fitness, and task switching performance. These findings indicate that higher cardiorespiratory fitness predicts better cognitive flexibility in older adults through greater grey matter volume in the dorsal striatum.