Despite SARS-CoV-2 being a "novel" virus, early detection of anti-spike IgG in severe COVID-19 patients may be caused by the amplification of humoral memory responses against seasonal coronaviruses. Here, we examine this phenomenon by characterizing anti-spike IgG responses in non-hospitalized convalescent individuals across a spectrum of COVID-19 severity. We observe that disease severity positively correlates with anti-spike IgG levels, IgG cross-reactivity against other betacoronaviruses (β-CoVs), and FcγR activation. Analysis of IgG targeting β-CoV-conserved and non-conserved immunodominant epitopes within the SARS-CoV-2 spike protein revealed epitope-specific relationships: IgG targeting the conserved heptad repeat (HR) 2 region significantly correlates with milder disease, while targeting the conserved S2'FP region correlates with more severe disease. Furthermore, a lower HR2-to-S2'FP IgG-binding ratio correlates with greater disease severity, with ICU-hospitalized COVID-19 patients showing the lowest HR2/S2'FP ratios. These findings suggest that HR2/S2'FP IgG profiles may predict disease severity and offer insight into protective versus deleterious humoral recall responses.