Focal brain damage caused by stroke can result in aphasia and advances in cognitive neuroscience suggest that impairment may be associated with network-level disorder rather than just circumscribed cortical damage. Several studies have shown meaningful relationships between brain-behaviour using lesions; however, only a handful of studies have incorporated in vivo structural and functional connectivity. Patients with chronic post-stroke aphasia were assessed with structural (n = 68) and functional (n = 39) MRI to assess whether predicting performance can be improved with multiple modalities and if additional variance can be explained compared to lesion models alone. These neural measurements were used to construct models to predict four key language-cognitive factors: (i) phonology; (ii) semantics; (iii) executive function; and (iv) fluency. Our results showed that each factor (except executive ability) could be significantly related to each neural measurement alone; however, structural and functional connectivity models did not explain additional variance above the lesion models. We did find evidence that the structural and functional predictors may be linked to the core lesion sites. First, the predictive functional connectivity features were found to be located within functional resting-state networks identified in healthy controls, suggesting that the result might reflect functionally specific reorganization (damage to a node within a network can result in disruption to the entire network). Second, predictive structural connectivity features were located within core lesion sites, suggesting that multimodal information may be redundant in prediction modelling. In addition, we observed that the optimum sparsity within the regularized regression models differed for each behavioural component and across different imaging features, suggesting that future studies should consider optimizing hyperparameters related to sparsity per target. Together, the results indicate that the observed network-level disruption was predicted by the lesion alone and does not significantly improve model performance in predicting the profile of language impairment.