Prior studies have shown that performance on standardized measures of memory in children with autism spectrum disorder (ASD) is substantially reduced in comparison to matched typically developing controls (TDC). Given reported deficits in face processing in autism, the current study compared performance on an immediate and delayed facial memory task for individuals with ASD and TDC. In addition, we examined volumetric differences in classic facial memory regions of interest (ROI) between the two groups, including the fusiform, amygdala, and hippocampus. We then explored the relationship between ROI volume and facial memory performance. We found larger volumes in the autism group in the left amygdala and left hippocampus compared to TDC. In contrast, TDC had larger left fusiform gyrus volumes when compared with ASD. Interestingly, we also found significant negative correlations between delayed facial memory performance and volume of the left and right fusiform and the left hippocampus for the ASD group but not for TDC. The possibility of larger fusiform volume as a marker of abnormal connectivity and decreased facial memory is discussed.