Neratinib +/- Fulvestrant in Metastatic HER2 Non-amplified But HER2 Mutant Breast Cancer

Inclusion Criteria for Pre-registration (for patients with unknown HER2 mutation status to have tumor tissue screened centrally by Washington University GPS laboratory): * Histologically or cytologically confirmed HER2-negative (0 or 1+ by IHC or non-amplified by FISH) breast cancer that is stage IV. * Agree to provide archival tumor material for research * There is no limitation on the number of prior lines of systemic therapy. * Presence of measurable or non-measurable disease by RECIST 1.1 is acceptable, except to be eligible for the Part II fulvestrant-naive ER+ cohort, at least one measurable disease by RECIST 1.1 is required. * At least 18 years of age. * ECOG performance status ≤ 2 * Adequate organ function as defined below within 8 weeks of pre-registration: * Serum creatinine ≤1.5 x ULN * Chil-Pugh class A if with liver disease * Able to understand and willing to sign an IRB approved written informed consent document. Note: HER2 mutation testing may be performed while the patient is receiving active systemic therapy for metastatic breast cancer so that the result can be used to determine eligibility for study drug therapy in the future. Exclusion Criteria for Pre-registration: * Testing for LVEF is not required for pre-registration, but patient must not have a recent LVEF \< LLN or have symptoms of congestive heart failure. * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements. * Acute or currently active hepatic or biliary disease requiring antiviral therapy (with the exception of Gilbert's syndrome, asymptomatic gallstones, liver metastases, or stable chronic liver disease per investigator assessment). * History of significant cardiac disease, cardiac risk factors, or uncontrolled arrhythmias. * Symptomatic intrinsic lung disease or extensive tumor involvement of the lungs resulting in dyspnea at rest. Inclusion Criteria for Registration (for patients initially pre-registered and with HER2 mutation identified by Washington University GPS laboratory) * Tumor tissue tested positive for HER2 mutation. HER2 mutations detected by Guardant360 are also eligible. * Agree to provide archival tumor material for research * ECOG performance status ≤2 * Adequate organ function as defined below within 2 weeks of registration: * ANC ≥1.5 x 10\^9/L * Platelet count ≥100 x 10\^9/L * Serum creatinine ≤1.5 x ULN * Child-Pugh class A if with liver disease * The patient must have completed radiation therapy and be at least 1 week from the last systemic chemotherapy administration, with adequate recovery of bone marrow and organ functions, before starting neratinib. * Presence of disease progression on the most recent disease evaluation. * Patients with known brain metastasis are eligible, but must have received radiation and be off steroids and stable (without evidence of disease progression by imaging or exam) for 3 months. * QTc interval ≤450 msec for men or \< or ≤ 470 msec for women within 2 weeks of registration. * LVEF \> or = institutional ILLN within 4 weeks of registration. * Women of childbearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control, abstinence) prior to study entry and for the duration of study participation. Should a woman become pregnant or suspect she is pregnant while participating in this study, she must inform her treating physician immediately. Men must agree and commit to use a barrier method of contraception while on treatment and for 3 months after the last dose of the investigational product. * Able to understand and willing to sign an IRB approved written informed consent document. * There is no limitation on the number of prior lines of systemic therapy. * To be eligible for the Part II fulvestrant-naive ER+ cohort, prior treatment with fulvestrant is not allowed. In addition, ER and/or PR positivity by institutional standard is required on pathology from the most recent tumor specimen if biopsy was done unless the tissue source (for example, pleural effusion or ascites or bone biopsy) may yield false negative ER and/or PR result, in which case the pathology from an earlier time point could be used and a discussion with the study chair is required. * To be eligible for the Part II fulvestrant-treated ER+ cohort, prior disease progression on fulvestrant is required. In addition, ER and/or PR positivity by institutional standard is required on pathology from the most recent tumor specimen unless the tissue source (for example, pleural effusion or ascites or bone biopsy) may yield false negative ER and/or PR result, in which case the pathology form an earlier time point could be used and a discussion with the study chair is required. Inclusion Criteria for Registration (for patients with HER2 mutation identified at an outside CLIA certified location): * Histologically or cytologically confirmed HER2-negative (0 or 1+ by IHC or non-amplified by FISH) breast cancer that is stage IV. * Tumor tissue or circulating tumor DNA tested positive for HER2 mutation. Mutations outside the list will be assessed on a case-by-case basis by the study team to determine eligibility. * Presence of measurable or non-measurable disease by RECIST 1.1 is acceptable, except to be eligible for the Part II fulvestrant-naïve ER+ cohort, at least one measurable disease by RECIST 1.1 is required. * At least 18 years of age. * ECOG performance status \< 2 (see Appendix A). * Adequate organ function as defined below within 2 weeks of registration: * Absolute neutrophil count: ≥1.5 × 109/L (1500/mm3) * Platelet count: ≥100 × 109/L (100,000/mm3) * Serum creatinine: ≤1.5 x ULN * Child-Pugh class A if with liver disease * The patient must have completed radiation therapy and be at least 1 week from the last systemic therapy administration, with adequate recovery of bone marrow and organ functions, before starting neratinib. * Presence of disease progression on the most recent disease evaluation. * Patients with known treated brain metastasis are eligible, but must have received radiation and be off steroids and stable (without evidence of disease progression by imaging or exam) for 3 months. * QTc interval ≤ 450 msec for men or ≤ 470 msec for women within 2 weeks of registration. * LVEF \> institutional LLN within 4 weeks of registration. * Women of childbearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control, abstinence) prior to study entry and for the duration of study participation. Should a woman become pregnant or suspect she is pregnant while participating in this study, she must inform her treating physician immediately. Men must agree and commit to use a barrier method of contraception while on treatment and for 3 months after the last dose of the investigational product * Able to understand and willing to sign an IRB approved written informed consent document. * There is no limitation on the number of prior lines of systemic therapy. * To be eligible for the Part II fulvestrant-naïve ER+ cohort, prior treatment with fulvestrant is not allowed. In addition, ER and/or PR positivity by institutional standard is required on pathology from the most recent tumor specimen if biopsy was done unless the tissue source (for example, pleural effusion or ascites or bone biopsy) may yield false negative ER and/or PR result, in which case the pathology from an earlier time point could be used and a discussion with the study chair is required. * To be eligible for the Part II fulvestrant-treated ER+ cohort, prior disease progression on fulvestrant is required. In addition, ER and/or PR positivity by institutional standard is required on pathology from the most recent tumor specimen unless the tissue source (for example, pleural effusion or ascites or bone biopsy) may yield false negative ER and/or PR result, in which case the pathology from an earlier time point could be used and a discussion with the study chair is required. Exclusion Criteria for Registration: * Currently receiving any other investigational agents or systemic cancer therapy. * Currently taking medications and herbal or dietary supplements that are strong cytochrome P450 (CYP) 3A4 inducers or inhibitors. The washout period must have been completed prior to the start of neratinib if the patient was taking any of these agents. If unavoidable, patients taking CYP3A4 inhibitors should be monitored closely. * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements. * Acute or currently active/requiring antiviral therapy hepatic or biliary disease (with the exception of patients with Gilbert's syndrome, asymptomatic gallstones, liver metastases, or stable chronic liver disease per investigator assessment). * Pregnant and/or breastfeeding. * History of significant cardiac disease, cardiac risk factors, or uncontrolled arrhythmias. * Symptomatic intrinsic lung disease or extensive tumor involvement of the lungs resulting in dyspnea at rest. * Experiencing grade 2 or greater diarrhea. * Prior treatment with neratinib * Child-Pugh class B or C liver dysfunction