Antiandrogen Therapy and SBRT in Treating Patients With Recurrent, Metastatic Prostate Cancer

Inclusion Criteria: * Confirmed diagnosis of prostate adenocarcinoma after radical prostatectomy (primary small cell carcinoma of the prostate is not allowed, however adenocarcinoma with neuroendocrine differentiation is allowed) * Presence of 1-5 visible metastases (by PSMA PET-CT) * At least one metastasis must be M1a-b * Visceral metastases are not allowed * Patients may have any number of pelvic nodal metastases (but largest must be \< 2 cm) * Metastases must be amenable to treatment with SBRT * Biopsy of one metastasis must be attempted, unless unsafe to perform * Patient must be fit to undergo SBRT to all visible sites of metastases, ADT * Total testosterone \> 150 ng/dL prior to ADT (optimal time to measure total testosterone is between 8 and 9 am) * Adequate performance status (Eastern Cooperative Oncology Group \[ECOG\] 0-1) * Hemoglobin \>= 9.0 g/dL, independent of transfusion and/or growth factors within 3 months prior to randomization * Platelet count \>= 100,000 x 10\^9/uL independent of transfusion and/or growth factors within 3 months prior to randomization * Serum albumin \>= 3.0 g/dL * Glomerular filtration rate (GFR) \>= 45 mL/min * Serum potassium \>= 3.5 mmol/L * Serum total bilirubin =\< 1.5 x upper limits of normal (ULN) * Note: In subjects with Gilbert?s syndrome, if total bilirubin is \> 1.5 x ULN, measure direct and indirect bilirubin and if direct bilirubin is =\< 1.5 x ULN, subject may be eligible * Aspartate aminotransferase (AST) or alanine aminotransferase (ALT) \< 2.5 x ULN * Medications known to lower the seizure threshold must be discontinued or substituted at least 4 weeks prior to study entry Exclusion Criteria: * Any evidence of spinal cord compression (radiological or clinical) * Prior pelvic malignancy * Prior pelvic radiation aside from salvage prostate radiation * Concurrent malignancy aside from superficial skin cancers or superficial bladder tumors * Inability to undergo radiotherapy, or ADT * Primary small cell carcinoma of the prostate (prostate adenocarcinoma with neuroendocrine differentiation is allowed) * Inflammatory bowel disease or active collagen vascular disease * History of any of the following: * Seizure or known condition that may pre-dispose to seizure (e.g. prior stroke within 1 year to randomization, brain arteriovenous malformation, Schwannoma, meningioma, or other benign central nervous system \[CNS\] or meningeal disease which may require treatment with surgery or radiation therapy) * Severe or unstable angina, myocardial infarction, symptomatic congestive heart failure, arterial or venous thromboembolic events (eg, pulmonary embolism, cerebrovascular accident including transient ischemic attacks), or clinically significant ventricular arrhythmias within 6 months prior to randomization * Current evidence of any of the following: * Uncontrolled hypertension * Gastrointestinal disorder affecting absorption * Active infection (eg, human immunodeficiency virus \[HIV\] or viral hepatitis) * Any chronic medical condition requiring a higher dose of corticosteroid than 10 mg prednisone/prednisolone once daily * Any condition that in the opinion of the investigator would preclude participation in this study * Concomitant strong CYP3A4 inducers. (If a strong CYP3A4 inducer must be co-administered, abiraterone acetate dose frequency will be adjusted). \[SAFETY: Warning against use with CYP2C8 inhibitors with narrow therapeutic index is also pertinent to be included as it is also part of United States Prescribing Information (USPI): In a CYP2C8 drug-drug interaction trial in healthy subjects, the area under curve (AUC) of pioglitazone (CYP2C8 substrate) was increased by 46% when pioglitazone was given together with a single dose of 1,000 mg abiraterone acetate. Therefore, patients should be monitored closely for signs of toxicity related to a CYP2C8 substrate with a narrow therapeutic index if used concomitantly with ZYTIGA\] * Treatment with CYP2D6 substrates that have a narrow therapeutic index. If an alternative treatment cannot be used, a dose reduction of the CYP2D6 substrate may be considered * Baseline moderate and severe hepatic impairment (ChildPugh Class B \& C) * Presence of visceral metastases (i.e., stage M1c)