The redshift of a galaxy uniquely determines its distance in our expanding universe. The colors of a galaxy measured in different bands constrain the approximate “photometric” redshift, although the prediction models are imprecise and improving these estimates is a hot topic in cosmology. We use machine learning models to refine the photometric redshifts of galaxies using their spatial structure organized into clusters, filaments, and walls. In particular, we test the hypothesis that additional information from the" neighborhood" of a galaxy sharpens our photometric redshift estimate. We demonstrate that a graph convolutional neural network—trained on a data set of high-resolution redshift observations on a small region of the sky—captures this information by learning to predict the redshift of all the galaxies in a viewing region simultaneously, improving the performance over single-galaxy redshift prediction by 10% median absolute deviation on a held-out region of the sky.