ZEN-3694, Enzalutamide, and Pembrolizumab for the Treatment of Metastatic Castration-Resistant Prostate Cancer

Inclusion Criteria: 1. Participants must have histologically confirmed prostate adenocarcinoma at the time of diagnosis, with subsequent development of metastatic castration-resistant prostate cancer. Patients with de novo small cell prostate cancer at the time of diagnosis are excluded from study participation 2. Evidence of disease progression by PSA and/or radiographic progression by Prostate Cancer Working Group 3 (PCWG3) criteria at the time of study entry 3. Patients must be evaluable for the primary endpoint of composite response, and must have either serum PSA \> 2 ng/mL during Screening and/or measurable disease by Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria 4. Safety lead-in only: 1. Metastatic castration resistant prostate cancer with evidence of disease progression by PCWG3 criteria at study entry 2. Progression on at least one prior androgen signaling inhibitor (e.g. abiraterone/prednisone, enzalutamide, apalutamide, darolutamide) 3. No prior chemotherapy for the treatment of metastatic castration-resistant prostate cancer (mCRPC). Prior chemotherapy administered in the castration-sensitive setting is allowed provided last dose of chemotherapy is \> 6 months prior to cycle 1 day 1 (C1D1) 5. Phase 2 Cohort A (transdifferentiated mCRPC) only: a. Participants must have clinicogenomic evidence of treatment emergent small cell neuroendocrine prostate cancer as defined by one or more of the following. i. Histologic evidence of small cell neuroendocrine prostate cancer on metastatic tumor biopsy and/or ii. Presence of loss-of-function mutation or deletion of RB1 on a Clinical Laboratory Improvement Act (CLIA)-approved genomic-sequencing platform and/or iii. Low PSA secretors as defined by meeting all of the following criteria during screening period - * Serum PSA \>=2 ng/mL, * Radiographic progression by PCWG3 criteria, * Presence of \> 5 metastases on conventional imaging and/or iv. Presence of at least one soft tissue lesion measuring \> 1 cm that is negative on PSMA PET. A negative lesion on PSMA PET is defined as uptake below the background uptake in the liver. b. No more than two prior lines of chemotherapy administered in the mCRPC setting. Chemotherapy administered in the castration-sensitive setting does not count towards this limit. c. Measurable disease by RECIST 1.1 criteria. 6. Phase 2 Cohort B (mCRPC without transdifferentiation) only: 1. Patients must not meet any criteria of transdifferentiation as outlined above 2. Progression on at least one prior androgen signaling inhibitor (e.g. abiraterone/prednisone, enzalutamide, apalutamide, darolutamide) 3. No prior chemotherapy for the treatment of mCRPC. Prior chemotherapy administered in the castration-sensitive setting is allowed provided last dose of chemotherapy is \> 6 months prior to C1D1 7. Castrate level of serum testosterone at study entry (\< 50 ng/dL). Patients without prior bilateral orchiectomy are required to remain on luteinizing hormone-releasing hormone (LHRH) analogue treatment for duration of study 8. No other systemic anti-cancer therapies administered other than LHRH analogue within 14 days or, 5 half-lives, whichever is shorter, prior to initiation of study treatment. Adverse events related to prior anti-cancer treatment must have recovered to grade =\< 1 with the exception of any grade alopecia and grade =\< 2 neuropathy a. Patients receiving enzalutamide prior to study entry may continue treatment at their current enzalutamide dose level without requirement for wash-out period 9. Age \>= 18 years 10. Eastern Cooperative Oncology Group (ECOG) performance status =\< 1 (Karnofsky performance status \>= 70%) 11. Demonstrates adequate organ function as defined below: * Absolute neutrophil count \>= 1,500/ per microliter (mcL) * Platelets \>= 100,000/mcL * Hemoglobin \>= 9.0 g/dL * Total bilirubin =\< 1.5 x institutional upper limit of normal, unless elevated due to Gilbert's syndrome and direct bilirubin is within normal limits * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase (SGOT) =\< 3 x institutional upper limit of normal (=\< 5 x upper limit of normal (ULN) in presence of liver metastases) * Alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase (SGPT)) =\< 3 x institutional upper limit of normal (=\< 5 x ULN in presence of liver metastases) * Serum creatinine =\< 1.5 x institutional upper limit of normal OR calculated creatinine clearance glomerular filtration rate (GFR) \>= 50 mL/min/1.73 m\^2 , calculated using the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation 12. Ability to understand a written informed consent document, and the willingness to sign it 13. Patients must agree to use adequate contraception prior to the study, for the duration of study participation, and 60 days after last administration of study treatment. Adequate contraception includes: 1. Patients who are sexually active should consider their female partner to be of childbearing potential if she has experienced menarche and is not postmenopausal (defined as amenorrhea \> 24 consecutive months) or has not undergone successful surgical sterilization. Even women who use contraceptive hormones (oral, implanted, or injected), an intrauterine device, or barrier methods (diaphragms, condoms, spermicide) should be considered to be of childbearing potential 2. Patients who have undergone vasectomy themselves should also be considered to be of childbearing potential 3. Acceptable methods of contraception include continuous total abstinence, or double-barrier method of birth control (e.g. condoms used with spermicide, or condoms used with oral contraceptives). Periodic abstinence and withdrawal are not acceptable methods of contraception 14. Patients must be willing to undergo metastatic tumor biopsy during screening. If no metastatic lesion is safely accessible to tumor biopsy, this requirement will be waived. Bone or soft tissue lesion is allowed, but soft tissue will be prioritized. If a patient has archival tissue obtained within 90 days of C1D1 the requirement for fresh tumor biopsy will be waived Exclusion Criteria: 1. Has participated in a study of an investigational product and received study treatment or used an investigational device other than those specified in the protocol within 2 weeks of C1D1 2. Hypersensitivity to ZEN-3694, pembrolizumab, enzalutamide, or any of its excipients 3. Has received prior radiotherapy within 2 weeks of C1D1. Participants must have recovered from all radiation-related toxicities, not require corticosteroids, and not have had radiation pneumonitis. A 1-week washout is permitted for palliative radiation (=\< 2 weeks of radiotherapy) to non-central nervous system (CNS) disease 4. Receipt of prior pembrolizumab or another immune checkpoint inhibitor (e.g. nivolumab, ipilimumab). Prior treatment with sipuleucel-T is allowed 5. Receipt of a radiopharmaceutical (e.g. radium-223, 177Lu-prostate-specific membrane antigen (PSMA) within 6 weeks prior to C1D1 6. Prior treatment with a bromodomain inhibitor (BETi) 7. Individuals with concurrent second malignancy requiring active treatment at study entry. Non-melanoma skin cancer, non-muscle invasive bladder cancer, and other carcinomas-in-situ are allowable exceptions 8. Has an active autoimmune disease that has required systemic treatment in the past 2 years (i.e., with use of disease modifying agents, corticosteroids or immunosuppressive drugs). Patients on low dose oral weekly methotrexate are allowed. Replacement therapy (e.g., thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency) or treatment with drugs (e.g. methimazole, neomercazol, carbamazole, etc.) that function to decrease the generation of thyroid hormone by a hyper-functioning thyroid gland (e.g., in Graves' disease) is not considered a form of systemic treatment of an autoimmune disease 9. Has received a live vaccine or live-attenuated vaccine within 30 days prior to the first dose of study drug. Administration of killed vaccines is allowed. 10. Cardiac condition as defined as one or more of the following: 1. QT interval by Fridericia (QTcF) \> 480 ms (machine or manual read allowed) 2. Uncontrolled supraventricular arrhythmia or ventricular arrhythmia requiring treatment 3. New York Heart Association (NYHA) congestive heart failure class III or IV 4. History of unstable angina, myocardial infarction, or cerebrovascular accident within 6 months prior to C1D1 11. History of seizure or pre-disposing condition (e.g. brain metastases) 12. Has a diagnosis of immunodeficiency or is receiving systemic steroid therapy at a prednisone equivalent dose of \> 10 mg daily or other form of immunosuppressive therapy within 7 days prior to first dose of study drug 13. Human immunodeficiency virus (HIV)-infected individuals on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial (screening not required in the absence of risk factors) 14. For participants with evidence of chronic hepatitis B virus (HBV) infection (positive hepatitis B surface antigen (HBsAg) and/or hepatitis B core antibody (HBcAb)), the HBV viral load must be undetectable at the time of study enrollment (screening not required in the absence of risk factors) 15. Chronic active hepatitis C virus (HCV) infection defined as positive viral load (screening not required in the absence of risk factors) 16. Has a history of (non-infectious) pneumonitis/interstitial lung disease that required steroids or has current pneumonitis/ interstitial lung disease 17. Has an active infection requiring intravenous antibiotics within 7 days prior to C1D1 18. Use of a prohibited concomitant medication within 7 days of C1D1 19. Major surgery within 28 days prior to C1D1. Minor procedures including biopsies, dental surgery, cataract surgery, or outpatient procedure are allowed 20. Has a history or current evidence of any condition, therapy, or laboratory abnormality that might confound the results of the study, interfere with the subject's participation for the full duration of the study, or is not in the best interest of the subject to participate, in the opinion of the treating investigator 21. Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial.