Abexinostat, Palbociclib, and Fulvestrant for the Treatment of Breast or Gynecologic Cancer

Inclusion Criteria: * BREAST CANCER: Participants must have histologically confirmed hormone receptor (HR)+, HER2- locally advanced or metastatic stage IV breast cancer. HER2- should be defined as 0 or 1 by immunohistochemistry, or HER2 gene amplification by fluorescence in situ hybridization (FISH), chromogenic in situ hybridization (CISH), or in situ hybridization (ISH) performed upon the primary tumor or metastatic lesion (ration \< 2 and HER2 copy \< 4). Estrogen receptor (ER) and progesterone receptor (PR) expression positivity is defined as more than 5% of tumor cells nuclei positive by immunohistochemistry in the sample on testing * BREAST CANCER: Patients must have had disease progression after treatment with anti-estrogen therapy combined with Cyclin-dependent kinase 4/6 (CDK4/6) inhibitor after a minimum of at least 3 months therapy in the metastatic setting and no more than 3 prior lines of systemic therapy for metastatic breast cancer (MBC), unrestricted prior therapy in the dose escalation * Note: Patients with breast cancer who were not previously treated with CDK4/6 inhibitors or have not tolerated full doses of prior ribociclib or palbociclib or abemaciclib are not eligible * ENDOMETRIAL CANCER: Patients must have histologically confirmed metastatic endometrial cancer of endometrioid type * ENDOMETRIAL CANCER: Tumors must have ER expression * ENDOMETRIAL CANCER: Patients must have received a maximum of one line of hormonal therapy for the treatment of endometrial cancer and may have received any lines of chemotherapy treatment * Note: Mixed tumor histology is allowed if the non-endometrioid component is less than 1%. Tumor must be estrogen receptor positive * OVARIAN, FALLOPIAN TUBE, OR PERITONEAL EPITHELIAL CANCER: Patients must have histologically confirmed recurrent or metastatic ovarian, fallopian, or peritoneal epithelial carcinoma * OVARIAN, FALLOPIAN TUBE, OR PERITONEAL EPITHELIAL CANCER: Tumors must have ER expression * OVARIAN, FALLOPIAN TUBE, OR PERITONEAL EPITHELIAL CANCER: Patients must have received a maximum of one line of hormonal therapy for the treatment of ovarian cancer and may have received any lines of chemotherapy treatment * Note: pure clear cell and pure mucinous ovarian carcinomas are not eligible * ALL PATIENTS * Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0-1 * Patients must have measurable disease, defined as at least one lesion that can be accurately measured in at least one dimension in accordance with RECIST criteria v. 1.1 * Leukocytes \>= 2,500/microliter (mcL) * Absolute neutrophil count \>= 1,500/mcL * Hemoglobin \> 9 g/dl (transfusions are allowed if more than 7 days prior to enrollment) * Platelets \>= 100,000/mcL * Total bilirubin \< upper limit of normal (ULN) except for patients with Gilbert's syndrome, who may only be included if the total bilirubin is =\< 3.0 x ULN or direct bilirubin =\< 1.5 x ULN * International normalized ratio (INR) =\< 1.5 (unless the patient is receiving anticoagulants and the INR is within the therapeutic range of intended use for that anticoagulant within 7 days prior to the first dose of study drug) * Aspartate transaminase (AST) \< 2.5 x ULN, except for patients with liver metastasis, who are only included if the AST is \< 5 x ULN * Alanine transaminase (ALT) \< 2.5 x ULN, except for patients with liver metastasis, who are only included if the ALT is \< 5 x ULN * Alkaline phosphate =\< 2.5 x ULN (unless bone metastasis is present in the absence of liver metastasis, in which 3.0 x ULN would be acceptable) * Serum creatinine =\< 1.5 mg/dl * Patients must be recovered from the effects of any prior surgery, radiotherapy, or other antineoplastic therapy * Patients may have received fulvestrant if the duration of response was more than 3 months * Ability to understand a written informed consent document, and the willingness to sign it * All female patients must be post-menopausal or rendered postmenopausal during the therapy * All male patients must be surgically sterile or agree to abstain from sperm donation and use both, a highly effective contraception with child bearing potential female partners (implants, injectables, combined oral contraceptives, some intrauterine device (IUD)s, sexual abstinence) and a barrier method (e.g., condoms, cervical rings, cervical condoms, sponge) during participation in the study and for 90 days after the last dose of study drug * Must be able to swallow pills Exclusion Criteria: * Patient with symptomatic visceral disease or any disease burden that renders the patient ineligible for endocrine therapy per the investigator's best judgment * Patients who have had chemotherapy or radiotherapy within 2 weeks prior to entering the study or those who have not recovered from adverse event to grade 1 or less from agents administered more than 2 weeks earlier * Patient has received treatment with any investigational drug within 21 days prior to study treatment administration. For classes of investigational agents that are not known to have prolonged toxicities, the washout time may be decreased to 14 days at the discretion of the principal investigator * Patients may not have any known intolerability to any of the involved agents or established cytopenias to CDK4/6 inhibitors that require dose modifications or dose delays of greater than 2 weeks * Patient with a known hypersensitivity to any of the excipients of palbociclib, abexinostat, or fulvestrant, including to peanut and soy * Patient has a concurrent malignancy or malignancy within 3 years of study entry, with the exception of adequately treated, basal or squamous cell skin carcinoma, nonmelanomatous skin cancer or curatively resected cervical cancer * Patient who has a history of untreated brain, or leptomeningeal, metastases (central nervous system (CNS) imaging is not required before study entry unless there is a clinical suspicion of CNS involvement) * Participants with previously treated brain metastases may participate, provided: * They are stable (without evidence of progression by imaging for at least four weeks and any neurologic symptoms have returned to baseline) * They have no evidence of new or enlarging brain metastases (confirmed by imaging within 28 days of the first dose of study drug) * They are not using steroids for at least 7 days before the first dose of study drug * Have isolated lesions that were treated with localized radiation therapy * This exception does not include leptomeningeal metastases, which is excluded regardless of clinical stability * Patient must not have been previously treated with histone deacetylase inhibitor (HDACi), with the exception of low dose of divalproex sodium (Depakote) or valproic acid * Patient has any medical, psychiatric or social condition, which in the opinion of the investigator, places the patient at an unacceptably high risk for toxicities or affects compliance to study procedures * Patient has impairment of gastrointestinal (GI) function or GI disease that may significantly alter the absorption of the study drugs (e.g., uncontrolled ulcerative diseases, uncontrolled nausea, vomiting, diarrhea, malabsorption syndrome, or small bowel resection) * Patient is currently receiving or has received systemic therapeutic doses of corticosteroids =\< 2 weeks prior to starting study drug, or has not fully recovered from side effects of such treatment * Note: The following uses of corticosteroids are permitted: single doses, topical applications (e.g., for rash), inhaled sprays (e.g., for obstructive airways diseases), eye drops or local injections (e.g., intra-articular) * Patient has known clinically significant, uncontrolled heart disease and/or cardiac repolarization abnormality * Patient is currently receiving any of the following substances and cannot be discontinued 30 days prior to cycle 1 day 1: * Concomitant medications, herbal supplements, and/or fruits (e.g. grapefruit, pummelos, star fruit, Seville oranges) and their juices that are strong inducers or inhibitors of CYP3A4/5, * Medications that have a narrow therapeutic window and are predominantly metabolized through CYP3A4/5 and CYP2D6 substrates * Has uncontrolled intercurrent illness including, but not limited to: * Uncontrolled infection * Disseminated intravascular coagulation * Psychiatric illness/social situations that would limit compliance with study requirements