Vorinostat in Treating Patients With Stage IV Breast Cancer Receiving Hormone Therapy

Inclusion Criteria: * Histologically or cytologically proven diagnosis of breast cancer * Stage IV disease * Patient has previously derived clinical benefit from endocrine therapy, but is no longer deriving benefit to endocrine therapy in the opinion of the treating investigator * At least one site of measurable disease, as defined by the modified Response Evaluation Criteria in Solid Tumors (RECIST) criteria * Eastern Cooperative Oncology Group (ECOG) performance status 0-2 * Female patient is post menopausal as defined by one of the following; free from menses for \>= 2 years, surgically sterilized, FSH and estradiol in post-menopausal range AND surgical absence of uterus OR chemotherapy induced amenorrhea lasting \> 1 year OR currently on ovarian suppression * Female patient of childbearing potential has a negative urine or serum (beta human chorionic gonadotropin \[B-hCG\]) pregnancy test within 14 days prior to receiving the first dose of vorinostat * Male patient agrees to use two barrier methods of contraception or abstain from intercourse for the duration of the study * Absolute neutrophil count (ANC) \>= 1,500/mcL * Platelets \>= 50,000/mcL * Hemoglobin \>= 9 g/dL * Prothrombin time or international normalized ratio (INR) =\< 1.5 x upper limit of normal (ULN) unless receiving therapeutic anticoagulation * Partial thromboplastin time (PTT) =\< 1.2 times the ULN unless the patient is receiving therapeutic anticoagulation * Potassium (K) levels normal limits * Magnesium (Mg) levels normal limits * Calculated creatinine clearance \>= 30 mL/min * Creatinine clearance should be calculated per institutional standard * Serum total bilirubin =\< 1.5 x ULN * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\]) and alanine aminotransferase (ALT) (serum glutamic pyruvate transaminase \[SGPT\]) =\< 2.5 x ULN * Alkaline phosphatase =\< 2.5 x ULN * Patient, or the patient's legal representative, has voluntarily agreed to participate by giving written informed consent * Patient has a life expectancy of at least 12 weeks in the opinion of the treating investigator * Patient is willing to continue on same AI therapy * Patient agrees to participate in imaging protocol 7184 and is separately consented Exclusion Criteria: * Patient has not derived clinical benefit from prior endocrine therapy * Patient is currently participating or has participated in a study with an investigational compound or device within 30 days of initial dosing with study drug(s) other than the imaging protocol 7184 * Patient has received an ER blocking therapy (selective estrogen receptor modulating or downregulating selective estrogen receptor modulator \[SERM\] or selective estrogen receptor degrader \[SERD\] i.e. tamoxifen or fulvestrant) within the past 6 weeks * Patient had prior treatment with an histone deacetylase (HDAC) inhibitor (e.g., romidepsin \[Depsipeptide\], NSC-630176, MS 275, LAQ-824, belinostat (PXD-101), LBH589, MGCD0103, CRA024781, etc); patients who have received compounds with HDAC inhibitor-like activity, such as valproic acid, as anti-tumor therapy should not enroll in this study; patients who have received such compounds for other indications, e.g. valproic acid for epilepsy, may enroll after a 30-day washout period * Patient is on any systemic steroids that have not been stabilized to the equivalent of =\< 10 mg/day prednisone during the 30 days prior to the start of the study drugs * Patient has known hypersensitivity to the components of study drug or its analogs * Patients with uncontrolled brain metastases * New York Heart Association (NYHA) class III or IV congestive heart failure, myocardial infarction within the previous 6 months, QTc \> 0.47 seconds, or uncontrolled arrhythmia * Type I diabetes mellitus; patients with type II diabetes mellitus will be included as long as their glucose can be controlled to under 200 mg/dL * Patient is pregnant or breast feeding, or expecting to conceive or father children within the projected duration of the study * Patient with a "currently active" second malignancy, other than non-melanoma skin cancer and carcinoma in situ of the cervix, should not be enrolled; patients are not considered to have a "currently active" malignancy if they have completed therapy for a prior malignancy, are disease free from prior malignancies for \> 5 years or are considered by their physician to be at less than 30% risk of relapse * Patients with known active viral hepatitis * Patient has a history or current evidence of any condition, therapy, or laboratory (lab) abnormality that might confound the results of the study, interfere with the patient's participation for the full duration of the study or is not in the best interest of the patient to participate