In recent decades, there has been substantial growth in our understanding of the immune system and its role in tumor growth and overall survival. A central finding has been the cross-talk between tumor cells and the surrounding environment or stroma. This tumor stroma, comprised of various cells, and extracellular matrix (ECM), has been shown to aid in suppressing host immune responses against tumor cells. Through immunosuppressive cytokine secretion, metabolic alterations, and other mechanisms, the tumor stroma provides a complex network of safeguards for tumor proliferation. With recent advances in more effective, localized treatment, radiation therapy (XRT) has allowed for strategies that can effectively alter and ablate tumor stromal tissue. This includes promoting immunogenic cell death through tumor antigen release to increasing immune cell trafficking, XRT has a unique advantage against the tumoral immune evasion mechanisms that are orchestrated by stromal cells. Current studies are underway to elucidate pathways within the tumor stroma as potential targets for immunotherapy and chemoradiation. This review summarizes the effects of tumor stroma in tumor immune evasion, explains how XRT may help overcome these effects, with potential combinatorial approaches for future treatment modalities.