The COVID-19 pandemic substantially impacted the mental health of the general population and particularly vulnerable individuals and groups. A wealth of research allows for estimating this impact and identifying relevant factors contributing to or mitigating it. The current paper presents and synthesizes this evidence into a multiaxial model of COVID-19 mental health impacts. Based on existing research, we propose four axes: (1) Exposure to COVID-related events; (2) Personal and social vulnerability, such as previous mental health problems or belonging to a vulnerable group; (3) Time, which accounts for the differential impacts throughout the development of the pandemic; and (4) Context, including healthcare and public policies, and social representations of the illness influencing individual emotional reactions and relevant behaviors. These axes help acknowledge the complexity of communities’ reactions and are pragmatic in identifying and prioritizing factors. The axes can provide individual information (i.e., more exposure is harmful) and account for interactions (e.g., exposure in an early phase of the pandemic differs from a later stage). This model contributes to the reflections of the evidence and informs the mental health response to the next pandemic.