BN-Brachyury, Entinostat, Adotrastuzumab Emtansine and M7824 in Advanced Stage Breast Cancer (BrEAsT)

* INCLUSION CRITERIA: * Patients must have histologically or cytologically confirmed metastatic human epidermal growth factor receptor 2 (HER2+) breast cancer. HER2 positive or amplified breast cancer is defined as Immunohistochemistry (IHC) 3+ or fluorescence in situ hybridization (FISH) average HER2 copy number greater than or equal to 6 signals per cell or HER2/chromosome enumeration probe 17 (CEP17) greater than or equal to 2.0. HER2 testing must have been performed in a laboratory accredited by the College of American Pathology (CAP) or another accrediting entity. * Patients must have hormone receptor negative, HER2+ breast cancer. Hormone receptor negative is defined as estrogen receptor \< 10% by immunohistochemistry (IHC) and progesterone receptor \< 10% by IHC. * Patients must have measurable disease, per Response Evaluation Criteria in Solid Tumors (RECIST) 1.1. * Patients must have at least one lesion deemed safe to biopsy and be willing to undergo up to three biopsies while on trial. * Patients must have received front-line treatment for metastatic disease with a taxane, trastuzumab and pertuzumab (THP; docetaxel or paclitaxel allowed) and progressed on treatment or were intolerant to treatment. Patients must have received at least one prior therapy in the metastatic setting. Prior ado-trastuzumab (T-DM1) therapy is allowed. * Female or male greater than or equal to 18 years. * Eastern Cooperative Oncology Group (ECOG) performance status 0 or 1. * Patients must have adequate bone marrow function as defined below: * absolute neutrophil count greater than or equal to 1,500/mcL (\> 1.5X 10(6)/L) * platelets greater than or equal to 100,000/mcL * hemoglobin greater than or equal to 9 mg/dL (transfusion to obtain hemoglobin greater than or equal to 9 mg/dL within 24 hours prior to dosing is allowed) * Patients must have adequate renal function, defined as: * serum creatinine less than or equal to 1.5 X upper limit of normal (ULN) OR * measured or calculated creatinine clearance greater than or equal to 60 mL/min for participant with creatinine levels \> 1.5 X institutional ULN (Glomerular filtration rate (GFR) can also be used in place of creatinine or creatinine clearance (CrCl). * Patients must have adequate hepatic function, defined as aspartate aminotransferase (AST) and alanine aminotransferase (ALT) levels less than or equal to 3 X ULN and total bilirubin \< 1.5 X ULN, unless known diagnosis of Gilbert's syndrome, where bilirubin less than or equal to 5 mg/dL will be permitted. Gilbert's syndrome will be defined as elevated unconjugated bilirubin, with conjugated (direct) bilirubin within the normal range and less than 20% of the total. Total bilirubin will be permitted up to 5 mg/dL, if patients have historical readings consistent with the definition of Gilbert's syndrome prior to entering study. Adequate hepatic function for patients with known liver metastases is defined as AST and ALT levels less than or equal to 5 X ULN. * Patients must have adequate cardiac function as defined by an ejection fraction greater than or equal to 50%. * The effects of BN-Brachyury, entinostat, M7824 and T-DM1 on the developing human fetus are unknown. However, the two components of T-DM1 are known to have negative fetal effects including oligohydramnios and oligohydramnios sequence (pulmonary hypoplasia, skeletal malformations and neonatal death). For this reason: * Women of child-bearing potential must agree to use adequate contraception study entry, for the duration of study participation and for 7 months after the last dose of study medication. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. * Men should refrain from fathering a child or donating sperm during the study and for 4 months after the last dose of study medications. * Patients with well-controlled human immunodeficiency virus (HIV) infection are eligible for trial as long as: * On an effective anti-retroviral therapy (ART) 4 weeks and with evidence of viral suppression as defined as HIV viral load 400 copies/mL within the last 3 months; * Cluster of differentiation 4 (CD4) greater than or equal to 200 cells/microL within the last 3 months; and * No reported opportunistic infections within 6 months prior to enrollment, except for the following which will be allowed: i. Esophageal candidiasis treated within last 6 months or currently improving with antifungal treatment. ii. Oral and/or genital herpes simplex virus (HSV) treated within last 6 months or currently improving with antiviral treatment. iii. Mycobacterium avium infection in last 6 months or that has been treated for at least 1 month. * Patients with evidence of chronic hepatitis B virus (HBV) infection are eligible for trial as long as the HBV viral load is undetectable on suppressive therapy, if indicated. * Patients with history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable or unquantifiable HCV ribonucleic acid (RNA) 12 weeks or longer after definitive treatment completion. * Patients must be able to understand and willing to sign a written informed consent document. EXCLUSION CRITERIA: * Patients who have received chemotherapy, including herceptin and/or pertuzumab in the prior 3 weeks (6 weeks for nitrosoureas or mitomycin); other investigational agents or a programmed cell death protein 1/programmed death-ligand 1 (PD-1/L1) agent within 4 weeks prior to study enrollment. * Patients who have received radiotherapy within 4 weeks prior to study entry. * Patients with active brain metastases or leptomeningeal metastases should be excluded from this clinical trial because of their poor prognosis and because they often develop progressive neurologic dysfunction that would confound the evaluation of neurologic and other adverse events. However, patients with treated brain metastases are eligible if there is no magnetic resonance imaging (MRI) evidence of progression for 6 weeks after treatment is complete (no radiotherapy within 6 weeks) and within 28 days prior to the first dose of trial drug or asymptomatic brain metastasis. Patients requiring immunosuppressive doses of systemic corticosteroids (\> 10mg/day prednisone equivalent) for palliation are excluded. * Patients with a history of another invasive malignancy less than or equal to 3 years prior to enrollment (patients with non-melanoma skin cancers, carcinoma in situ of the breast or cervix are eligible). * History of allergic reactions attributed to compounds of similar chemical or biologic composition to agents used in study. For example, reaction to prior vaccination with vaccinia virus or known hypersensitivity reaction to fully humanized monoclonal antibodies (Grade greater than or equal to 3 National Cancer Institute (NCI)-Common Terminology Criteria for Adverse Events (CTCAEv5). * Uncontrolled intercurrent illness including, but not limited to, ongoing active infection that requires systemic treatment with ongoing antibiotics (eligible if can stop antibiotics on day of enrollment), unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situation that in the opinion of the primary investigator would prohibit the patient from complying with study requirements. * Known history of a gastrointestinal illness that in the investigator's opinion would prevent the absorption of entinostat, which is an oral agent. (Arm 3 ONLY) * Patients with bone metastases who have initiated denosumab or a bisphosphonate therapy within 28 days prior to or after Cycle 1 Day 1. Continuation of prior therapy is allowed. * Patients with inherited bleeding disorders, a history of bleeding diathesis such as von Willebrand Factor (VWF) deficiency or recent (within 3 months prior to enrollment) clinically significant bleeding events that, in the judgment of the investigator, would interfere with patient's ability to carry out the treatment program. * Patients should have no evidence of being immunocompromised as listed below: * Active, known or suspected autoimmune disease. Patients are permitted to enroll if they have vitiligo, type I diabetes mellitus, residual hypothyroidism due to an autoimmune condition only requiring hormone replacement, psoriasis not requiring systemic treatment or conditions not expected to recur in the absence of an external trigger in the opinion of the primary investigator. * Altered immune function, that in the judgement of the principal investigator (PI) that may affect a patient's ability to adequately engage the immune system and respond to the immunotherapy agents being administered, including but not limited to: inflammatory bowel disease; active infectious enteritis; eosinophilic enteritis; lupus erythematous; ankylosing spondylitis; scleroderma; multiple sclerosis. These criteria do not include all disease with an immune-related component but are not autoimmune in nature or have a primary alteration in the general immune function that may interfere with the vaccine mechanism of action, for example celiac disease. * Immunosuppressive therapy post-organ transplant. * Concurrent use of chronic use of systemic steroids, except for physiologic doses of systemic steroids for replacement, defined as 10mg of prednisone per day or equivalent, or local (topical, nasal, ophthalmic or inhaled) steroid use or prior concomitant use with chemotherapy. Systemic steroids must have been discontinued \>2 weeks prior to trial start. Prior use of corticosteroids in short-term schemes (duration shorter than 3 days) for indications such as prophylaxis of reactions to intravenous contrast for imaging studies or chemotherapy-related adverse events (AEs) are not considered part of this exclusion. Prior use of corticosteroids for brain metastasis ending at least 14 days prior to enrollment is not considered part of this exclusion criteria. * Pregnant and breastfeeding women are excluded from this study because of the potential for teratogenic or abortifacient effects with all of the agents involved in this trial. * Clinically significant cardiomyopathy, coronary disease, chronic heart failure (CHF; New York Heart Association class III or IV or hospitalization for CHF), or cerebrovascular accident within 6 months prior to enrollment. * Patients with a history of myocarditis are excluded due to the potential of myocarditis with anti-PD-L1 antibodies. * Patients with pulse oximetry \< 92% on room air will be excluded due to the potential of pneumonitis with anti-PD-L1 antibodies. * Any other condition, which would, in the opinion of the Principal Investigator or Medical Monitor, indicated the subject is a poor candidate for the clinical trial or would jeopardize the subject or the integrity of the data obtained. ELIGIBILITY CRITERIA FOR ARMS 2 AND 3: M7824, BN-BRACHYURY, T-DM1 +/- ENTINOSTAT IN HER2+BC INCLUSION CRITERIA: * Patients must have histologically or cytologically confirmed metastatic HER2+ breast cancer. HER2 positive or amplified breast cancer is defined as IHC 3+ or FISH average HER2 copy number greater than or equal to 6 signals per cell or HER2/CEP17 greater than or equal to 2.0. (117) HER2 testing must have been performed in a laboratory accredited by the College of American Pathology (CAP) or another accrediting entity. * Patients must have hormone receptor negative, HER2+ breast cancer. Hormone receptor negative is defined as estrogen receptor \< 10% by IHC and progesterone receptor \<10% by IHC. * Patients must have measurable disease, per RECIST 1.1. * Patients in Cohort 3 must have at least one lesion deemed safe to biopsy and be willing to undergo up to three biopsies while on trial. * Patients must have received front-line treatment for metastatic disease with a taxane, trastuzumab and pertuzumab (THP; docetaxel or paclitaxel allowed) and progressed on treatment or were intolerant to treatment. Patients must have received at least one prior therapy in the metastatic setting. Prior T-DM1 therapy is allowed. * Female or male greater than or equal to 18 years. * ECOG performance status 0 or 1 * Patients must have adequate bone marrow function as defined below: * absolute neutrophil count greater than or equal to1,500/mcL (greater than or equal to 1.5X 106/L) * platelets greater than or equal to 100,000/mcL * hemoglobin greater than or equal to 9 mg/dL (transfusion to obtain hemoglobin greater than or equal to 9 mg/dL within 24 hours prior to dosing is allowed) * Patients must have adequate renal function, defined as: * serum creatinine less than or equal to 1.5 X upper limit of normal (ULN) OR * measured or calculated creatinine clearance greater than or equal to 60 mL/min for participant with creatinine levels \> 1.5 X institutional ULN (GFR can also be used in place of creatinine or CrCl). * Patients must have adequate hepatic function, defined as AST and ALT levels less than or equal to 3 X ULN and total bilirubin \< 1.5 X ULN, unless known diagnosis of Gilbert's syndrome, where bilirubin less than or equal to 5 mg/dL will be permitted. Gilbert's syndrome will be defined as elevated unconjugated bilirubin, with conjugated (direct) bilirubin within the normal range and less than 20% of the total. Total bilirubin will be permitted up to 5 mg/dL, if patients have historical readings consistent with the definition of Gilbert's syndrome prior to entering study. Adequate hepatic function for patients with known liver metastases is defined as AST and ALT levels less than or equal to 5 X ULN. * Patients must have adequate cardiac function as defined by an ejection fraction greater than or equal to 50%. * The effects of BN-Brachyury, entinostat, M7824 and T-DM1 on the developing human fetus are unknown. However, the two components of T-DM1 are known to have negative fetal effects including oligohydramnios and oligohydramnios sequence (pulmonary hypoplasia, skeletal malformations and neonatal death). For this reason, * Women of child-bearing potential must agree to use adequate contraception study entry, for the duration of study participation and for 7 months after the last dose of study medication. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study.