Neutralizing antibodies (nAbs) to SARS-CoV-2 hold powerful potentials for clinical interventions against COVID-19 disease. However, their common genetic and biologic features remain elusive. Here we interrogate a total of 165 antibodies from eight COVID-19 patients, and find that potent nAbs from different patients have disproportionally high representation of IGHV3-53/3-66 usage, and therefore termed as public antibodies. Crystal structural comparison of these antibodies reveals they share similar angle of approach to RBD, overlap in buried surface and binding residues on RBD, and have substantial spatial clash with receptor angiotensin-converting enzyme-2 (ACE2) in binding to RBD. Site-directed mutagenesis confirms these common binding features although some minor differences are found. One representative antibody, P5A-3C8, demonstrates extraordinarily protective efficacy in a golden Syrian hamster model against SARS-CoV-2 infection. However, virus escape analysis identifies a single natural mutation in RBD, namely K417N found in B.1.351 variant from South Africa, abolished the neutralizing activity of these public antibodies. The discovery of public antibodies and shared escape mutation highlight the intricate relationship between antibody response and SARS-CoV-2, and provide critical reference for the development of antibody and vaccine strategies to overcome the antigenic variation of SARS-CoV-2.