The prefrontal cortex (PFC) is involved in working memory and self-regulatory and goal-directed behaviors and displays remarkable structural and functional plasticity over the life course. Neural circuitry, molecular profiles, and neurochemistry can be changed by experiences, which influence behavior as well as neuroendocrine and autonomic function. Such effects have a particular impact during infancy and in adolescence. Behavioral stress affects both the structure and function of PFC, though such effects are not necessarily permanent, as young animals show remarkable neuronal resilience if the stress is discontinued. During aging, neurons within the PFC become less resilient to stress. There are also sex differences in the PFC response to stressors. While such stress and sex hormone-related alterations occur in regions mediating the highest levels of cognitive function and self-regulatory control, the fact that they are not necessarily permanent has implications for future behavior-based therapies that harness neural plasticity for recovery.