Reversing Therapy Resistance With Epigenetic-Immune Modification

Inclusion Criteria: * Pre and postmenopausal women or men with stage IV ER+ breast cancer histological or cytological confirmation ER-positive tumors * Progressed after at least one line of hormonal therapy * Any number of prior chemotherapy in the metastatic setting * Any number of prior hormonal therapies. * human epidermal growth factor receptor 2 (HER2) positive or negative ER-Negative tumors * PD-L1 low, high or unknown * Progression after prior PD-1 or PD-L1 inhibitors allowed * HER2 positive or negative * 18 years or older * Eastern Cooperative Oncology Group (ECOG) Performance Status of ≤ 2. * Understand and voluntarily sign an informed consent prior to any study-related assessments or procedures are conducted and are able adhere to the study visit schedule and other protocol requirements. * Consent to paired tumor biopsy, for accessible tumors * Measureable tumor by RECIST criteria v.1.1 * Archived tumor tissue (minimum of 8 slides for paraffin-embedded tumor tissue) for assessment of tumor-based biomarkers and immune score is required for eligibility. * Per Good Clinical Practice, any toxicity related to prior therapies that, in the opinion of the investigator, would potentially be worsened with anti-PD1 therapy should be resolved to less than Grade 1 * Adequate organ function within 14 days of study start: * Absolute neutrophil count (ANC) ≥ 1.5 X 109/L * Hemoglobin (Hgb) ≥9g/dL (may transfuse if clinically indicated) * Platelets (plt) ≥ 100 x 109/L * Potassium within normal range, or correctable with supplements; * Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) ≤2.5 x Upper Limit Normal (ULN) or ≤5.0 x ULN if liver tumor is present; * Serum total bilirubin ≤ 1.5 x ULN * Serum creatinine ≤ 1.5 x ULN, or 24-hr clearance ≥ 60ml/min; and * Females of child-bearing potential (defined as a sexually mature women who): * Has not undergone a hysterectomy (the surgical removal of the uterus) or bilateral oophorectomy (the surgical removal of both ovaries) or, * Has not been naturally postmenopausal for at least 24 consecutive months (i.e., has had menses at any time during the preceding 24 consecutive months). * Must have negative serum pregnancy test within 7 days before starting study treatment in females of childbearing potential (FCBP) and willingness to adhere to acceptable forms or birth control (a physician- approved contraceptive method (oral, injectable, or implantable hormonal contraceptive; tubal ligation; intra-uterine device; barrier contraceptive with spermicide; or vasectomized partner). * Male subjects with female partner of childbearing potential must agree to the use of a physician-approved contraceptive method throughout the course of the study Exclusion Criteria: * Any significant medical condition, laboratory abnormalities, which places the subject at unacceptable risk if he/she were to participate in the study. * Has had prior chemotherapy, targeted small molecule therapy, or radiation therapy within 2 weeks prior to study Day 1 or who has not recovered (i.e., ≤ Grade 1 or at baseline) from adverse events due to a previously administered agent. * Note: Subjects with ≤ Grade 2 neuropathy are an exception to this criterion and may qualify for the study. * Note: If subject received major surgery, they must have recovered adequately from the toxicity and/or complications from the intervention prior to starting therapy. * Patients may continue on ovarian suppression * Has a known additional malignancy that is progressing or requires active treatment. Exceptions include basal cell carcinoma of the skin, squamous cell carcinoma of the skin, or in situ cervical cancer that has undergone potentially curative therapy. * Any condition that confounds the ability to interpret data from the study. * Symptomatic central nervous system metastases. Subjects with brain metastases that have been previously treated and are stable for 6 weeks are allowed. * Persistent diarrhea or malabsorption ≥ NCI CTCAE grade 2, despite medical management. * Unstable angina, significant cardiac arrhythmia, or New York Heart Association (NYHA) class 3 or 4 congestive heart failure. * Prior systemic cancer-directed treatments or investigational modalities ≤ 5 half-lives or 4 weeks, whichever is shorter, prior to starting study drug or who have not recovered from side effects of such therapy (except alopecia). * Has an active auto-immune disease requiring systemic treatment within the past 3 months or a documented history of clinically severe autoimmune disease, or a syndrome that requires systemic steroids or immunosuppressive agents. Subjects with vitiligo or resolved childhood asthma/atopy would be an exception to this rule. Subjects that require intermittent use of bronchodilators or local steroid injections would not be excluded from the study. Subjects with hypothyroidism stable on hormone replacement or Sjorgen's syndrome will not be excluded from the study. * Has evidence of interstitial lung disease or active, non-infectious pneumonitis. * Has an active infection requiring systemic therapy. * Has a history or current evidence of any condition, therapy, or laboratory abnormality that might confound the results of the trial, interfere with the subject's participation for the full duration of the trial, or is not in the best interest of the subject to participate, in the opinion of the treating investigator. * Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial. * Is pregnant or breastfeeding, or expecting to conceive or father children within the projected duration of the trial, starting with the pre-screening or screening visit through 120 days after the last dose of trial treatment. * Active and ongoing steroid use, except for non-systemically absorbed treatments (such as inhaled or topical steroid therapy for asthma, Chronic Obstructive Pulmonary Disease (COPD), allergic rhinitis). * Major surgery ≤ 2 weeks prior to starting a study drug or who have not recovered from side effects of such therapy. * Pregnant or breast feeding. * Known Human Immunodeficiency Virus (HIV) infection and/or Hepatitis B or C positive. * Known hypersensitivity to pembrolizumab or any of its excipients. * Has received a live vaccine within 30 days prior to the first dose of trial treatment. * Patients receiving medications or substances that are strong inhibitors or inducers of CYP450 enzyme(s) are ineligible. * Pregnant women are excluded from this study because vorinostat, tamoxifen and PD-1 are drug classes with the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother, breastfeeding should be discontinued if the mother is treated with vorinostat, tamoxifen and PD-1 inhibitors.