DS3201 and Ipilimumab for the Treatment of Metastatic Prostate, Urothelial and Renal Cell Cancers

Inclusion Criteria: * Eastern Cooperative Oncology Group (ECOG) performance status 0-1 * Histologically or cytologically confirmed prostate carcinomas, urothelial carcinomas and clear cell renal carcinomas. For patients with prostate carcinomas, variant histologies, such as small cell or neuroendocrine carcinomas are permitted * Evidence of metastatic disease by conventional imaging studies (computed tomography \[CT\], magnetic resonance imaging \[MRI\] and/or bone scan). Patients with locally advanced disease that is not amenable to locoregional therapies such as surgery or radiation, are considered metastatic and eligible to participate * Patients with prostate carcinomas must have castration resistant disease, i.e. evidence of disease progression while having castrate levels of testosterone (=\< 50 ng/dL or =\< 2.0 nM) or an unsatisfactory response to \>= 1 month of castration, as defined by lack of symptom control and/or serum tumor marker response of \< 20% (confirmed by a second value drawn on a different day). Exception: patients with small cell carcinoma histology are not required to have progressed during prior androgen deprivation therapy. However, all patients with prostate cancer (including those with small cell carcinomas) are required to maintain castrate levels of testosterone throughout the duration of the study * Patients with prostate carcinomas must also display the AVPC molecular signature (i.e. known loss or mutation \[by Clinical Laboratory Improvement Act (CLIA) certified molecular testing by immunohistochemistry (IHC) and/or deoxyribonucleic acid (DNA) sequencing in solid tumor samples, and/or in circulating tumor DNA\]) in at least 2 of the following: Tp53, RB1 and PTEN * Patients with renal cell carcinomas (RCC) must have had progressive disease during or after treatment with at least one anti-angiogenic agent and one PD-1 or PD-L1 inhibitor * Patients with urothelial carcinomas (UC) must have had progressive disease during or after treatment with at least one anti-PD1 or PD-L1 inhibitor and must have previously been treated with platinum-based chemotherapy or not be eligible for platinum based chemotherapy * Patients with AVPC are allowed to have received prior treatment with a PD-1 or PD-L1 inhibitor, but are not required to have had it * Evidence of disease progression as defined by new or increasing measurable and/or non-measurable disease as per RECIST * For patients with AVPC, rising PSA values (a minimum of 2 rising values over 3 measurements obtained a minimum of 7 days apart with the last result being \>= 1.0 ng/mL, as per Prostate Cancer Working Group 3 \[PCWG3\]) can also be considered evidence of progressive disease for eligibility as long as the molecular AVPC criteria are also met * If patient has known brain metastases, must have stable neurologic status following local therapy for at least 4 weeks without the use of steroids or on stable or decreasing dose of =\< 10 mg daily prednisone (or equivalent), and must be without neurologic dysfunction that would confound the evaluation of neurologic and other adverse events (AEs) * Recovery from recent surgery, radiotherapy, chemotherapy or any other anti-cancer therapy to baseline or =\< grade 1 (other than alopecia). Other low grade toxicities (e.g. =\< grade 2 lymphopenia or hypomagnesemia) may be allowed at the discretion of the investigator if considered clinically insignificant * Absolute neutrophil count (ANC) \>= 1,500/uL (obtained within 28 days prior to day 1 of treatment) * Platelet count \>= 100,000/uL (obtained within 28 days prior to day 1 of treatment) * Hemoglobin (Hgb) \>= 9 g/dL (obtained within 28 days prior to day 1 of treatment) * Serum creatinine =\< 2 x upper limit of normal (ULN) OR creatinine clearance (CrCl) \>= 40 mL/min as estimated by the Cockcroft and Gault formula in subjects with creatinine \> 2 x ULN (obtained within 28 days prior to day 1 of treatment) * Bilirubin =\< 1.5 x ULN unless evidence of Gilbert's disease in which case \< 5 x ULN (obtained within 28 days prior to day 1 of treatment) * Aspartate aminotransferase (AST) =\< 3.0 x ULN without liver metastases; must be =\< 5 x ULN with liver metastases (obtained within 28 days prior to day 1 of treatment) * Alanine aminotransferase (ALT) =\< 3.0 x ULN without liver metastases; must be =\< 5 x ULN with liver metastases (obtained within 28 days prior to day 1 of treatment) * Serum albumin \>= 3 g/dL (obtained within 28 days prior to day 1 of treatment) * Females of childbearing potential (FCBP) must have a negative serum pregnancy test within 24 hours prior to start of study drug first dose of ipilimumab. NOTE: Females are considered of child bearing potential unless they are surgically sterile (have undergone a hysterectomy, bilateral tubal ligation, or bilateral oophorectomy) or are postmenopausal (at least 12 consecutive months with no menses without an alternative medical cause) * FCBP must be willing to use a highly effective contraceptive method (i.e., achieves a failure rate of \< 1% per year when used consistently and correctly) from the time of informed consent until 3 months after treatment discontinuation. Male subjects must be willing to use condoms from the time of informed consent until 3 months after treatment discontinuation. For a non-pregnant FCBP partner, contraception recommendations should also be considered * Presence of neoplastic disease documented on imaging studies (bone scan, CT and/or MRI scans) deemed accessible for serial biopsies (including bone and soft tissues) * Patients must agree to tissue collection for correlative studies (including participation in PA13-0291 and PA13-0247 laboratory protocols, and archival tissue from a prior biopsy or surgery for prostate cancer where available) * Ability to swallow and retain oral medications * Ability to understand and willingness to sign an Institutional Review Board (IRB) approved written informed consent form (ICF) and authorization permitting release of personal health information including genetic testing relevant to cancer * Ability to comply with study visit schedule and assessments Exclusion Criteria: * Pregnant or lactating * Carcinomatous meningitis * Treatment with any of the following as anti-cancer agents (see Inclusion Criteria #8 regarding required resolution of treatment-related toxicities): * Prior treatment with an EZH2 inhibitor * PD-1, PD-L1, PD-L2 or CTLA-4 inhibitor (whether commercially available or investigational) within 4 weeks prior to day 1 of treatment * Monoclonal antibody (whether commercially available or investigational) within 4 weeks prior to day 1 of treatment * Investigational drug within 2 weeks prior to day 1 of study treatment * Chemotherapy within 2 weeks prior to day 1 of study treatment * Radiation therapy or radionuclide therapy within 2 weeks prior to day 1 of study treatment * Receipt of live virus vaccination within 30 days prior to day 1 of study treatment * Untreated symptomatic spinal cord compressions or brain metastases * Experienced an immune-related adverse event (irAE) that led to permanent discontinuation of prior immunotherapy * Experienced a \>= grade 3 irAE within the past 16 weeks, any grade 4 life- threatening irAE (regardless of duration) or neurologic or ocular AE of any grade while receiving prior immunotherapy (NOTE: Patients with endocrine AEs of any grade are permitted to enroll if they are stably maintained on appropriate replacement therapy, but must have no history of adrenal crisis and be asymptomatic) * Active, known or suspected autoimmune disease. Subjects with vitiligo, type I diabetes mellitus, residual hypothyroidism due to autoimmune condition only requiring hormone replacement, psoriasis or eczema not requiring systemic treatment, or conditions not expected to recur in the absence of an external trigger are permitted to enroll * Requires chronic systemic treatment with either corticosteroids (\> 10 mg daily prednisone equivalents) or other immunosuppressive medications. Inhaled, intranasal, intra-articular and topical (including ocular) steroids are allowed. Adrenal replacement (i.e., physiologic replacement) doses \> 10 mg daily prednisone equivalents are permitted in the absence of active autoimmune disease * History of interstitial lung disease, idiopathic pulmonary fibrosis or evidence of active pneumonitis on screening chest computed tomography (CT) scan * Clinically significant cardiovascular disease including: * Myocardial infarction (MI)/stroke within 6 months prior to day 1 of study treatment * Unstable angina within 3 months prior to day 1 of study treatment * Congestive heart failure (CHF) with New York Heart Association (NYHA) class 3 or 4 * History of clinically significant ventricular arrhythmias (e.g., ventricular tachycardia, ventricular fibrillation, torsades de pointes) * Uncontrolled hypertension (systolic blood pressure \[BP\] \>= 140 mmHg or diastolic BP \>= 90 mmHg). Patients with a history of hypertension are allowed provided blood pressure is controlled by anti-hypertensive treatment * Baseline 12-lead electrocardiogram (ECG) that demonstrates clinically relevant abnormalities that may affect participant safety or interpretation of study results (eg, baseline corrected QT \[QTc\] interval \> 470 msec, complete left bundle branch block \[LBBB\], signs of an acute myocardial infarction, ST-T interval changes suggestive of active myocardial ischemia, second- or third-degree atrioventricular \[AV\] block, or serious bradyarrhythmias or tachyarrhythmias). If the baseline uncorrected QT interval is \> 470 msec, this interval should be rate-corrected using the Fridericia method and the resulting Fridericia's correction formula (QTcF) should be used for decision making and reporting. If QTc exceeds 470 msec, or QRS exceeds 120 msec, the ECG should be repeated 2 more times and the average of the 3 QTc or QRS values should be used to determine the participant's eligibility. Computer-interpreted ECGs should be over read by a physician experienced in reading ECGs before excluding participants. Cases must be discussed in detail with the principal investigator (PI) to judge eligibility * Known active infection with hepatitis B virus (HBV) or hepatitis C virus (HCV) screening will include hepatitis B surface antigen (HepBsAg) and HCV antibody (Ab). In the presence of a positive HCV Ab, HCV ribonucleic acid (RNA) levels will be requested. A positive HepBsAg and/or detectable levels of HCV RNA will make patient ineligible * Known history of human immunodeficiency virus (HIV) (HIV1/2 antibodies) * Gastrointestinal (GI) disorder that negatively affects absorption (e.g., significant intestinal resection resulting in short intestinal syndrome, severe diarrhea, requirement for total parenteral nutrition \[TPN\]) * Known additional malignancy that is progressing, requires active treatment or has a \>= 30% probability of recurrence within 24 months. Exceptions include non-melanoma skin cancer that has undergone potentially curative therapy, Ta urothelial carcinoma or stage 0 chronic lymphocytic leukemia * Any other concurrent severe and/or uncontrolled concomitant medical condition that could compromise participation in the study (e.g., known psychiatric disorder, clinically significant neurological disorder, active or uncontrolled infection) * Patient unwilling or unable to comply with this study protocol