Zika virus (ZIKV), a mosquito-borne pathogen, is associated with neurological complications in adults and congenital abnormalities in newborns. There are no vaccines or treatments for ZIKV infection. Understanding the specificity of natural antibody responses to ZIKV could help inform vaccine efforts. Here, we used a technology called Deep Sequence-Coupled Biopanning to map the targets of the human antibody responses to ZIKV infection. A bacteriophage virus-like particle (VLP) library displaying overlapping linear peptides derived from the ZIKV polyprotein was generated. The library was panned using IgG from 23 ZIKV-infected patients from Panama and deep sequencing identified common targets of anti-ZIKV antibodies within the ZIKV envelope glycoprotein. These included epitopes within the fusion loop within domain II and four epitopes within domain III. Additionally, we showed that VLPs displaying selected epitopes elicited antibodies that bound to native ZIKV envelope protein but failed to prevent infection in a mouse challenge model.