Focal Radiation With Pulsed Systemic Therapy of Abiraterone, Androgen Deprivation Therapy (ADT), Lynparza Towards Castration Sensitive Oligometastatic Prostate Cancer (FAALCON)

Inclusion Criteria: * Histologic or cytologic diagnosis of prostate adenocarcinoma (pure small cell or pure neuroendocrine prostate cancer are not allowed). * Prior radical prostatectomy OR external beam radiation with curative intent (e.g. HiFU or partial gland therapies are not acceptable) delivered to prostate. Patients with prior radical prostatectomy with positive margins must have undergone salvage or adjuvant radiation. * Have newly diagnosed oligometastatic prostate cancer based on molecular imaging (e.g. 68Ga-PSMA PET/CT or Axumin, excludes FDG-PET). Oligometastatic prostate cancer is between 1 and ≤ 5 radiation treatment sites. A site can be up to 5 cm and contain multiple lesions. * Newly diagnosed oligometastatic disease requires that no prior image guided radiation was given to sites outside of the prostate bed or pelvic lymph nodes that are typically treated in the salvage or adjuvant radiation setting. * Patients must have a PSA \>0.2 ng/mL (confirmed ≥4 weeks later with subsequent rise) for those who underwent radical prostatectomy. For those with prior curative radiotherapy, they must meet the Phoenix criteria for progression (nadir of PSA + 2 ng/mL) * Medically fit for radiotherapy * All molecular positive disease is within an anatomic distribution that (in the view of the radiation oncologist) can be treated safely per standard radiation oncology principles * Candidate for androgen deprivation therapy (e.g. leuprolide, goserelin, degarelix) abiraterone therapy (financial and medical) in view of medical oncology using package insert for guidance * Patient must have normal organ and bone marrow function measured within 28 days prior to administration of study treatment as defined per protocol * Androgen deprivation therapy with or without second generation androgen receptor inhibitors or abiraterone (when given to optimize focal therapies like surgery or radiation) in the curative setting are allowed as long as testosterone has recovered to above 150 ng/dL. * Androgen deprivation therapy (with or without second generation androgen receptor inhibitors or abiraterone) for metastatic disease is allowed up to 4 weeks prior to registration. If previous ADT was used in curative setting, testosterone recovery must be documented (testosterone \>150 ng/dL) OR \>1 year elapsed from last administration of curative attempt ADT before recent ADT was resumed. * ECOG ≤1 * Patients must use a condom during treatment and for 6 months after the last dose of olaparib or abiraterone when having sexual intercourse with a pregnant woman or with a woman of childbearing potential. Female partners of childbearing potential of patients on study should also use a highly effective form of contraception. * Capable of giving signed informed consent Exclusion Criteria: * Prior orchiectomy * Prior exposure to PARP inhibitors, docetaxel or cabazitaxel. * Has a known additional malignancy within the past 3 years that has required treatment excluding superficial squamous skin cancer or carcinoma in situ of bladder or head and neck (those are permissible). * Life expectancy ≤3 years in view of treating provider * Presence of known parenchymal brain metastasis (imaging not required in absence of symptoms) * Symptoms of cord compression requiring immediate radiation. * Patients with myelodysplastic syndrome (MDS)/ acute myeloid leukemia (AML) or with features suggestive of MDS/AML per primary provider * Severe hepatic impairment (Child-Pugh Class C) * Patients with known active hepatitis infection (e.g. hepatitis B, or C) * Concurrent use of strong CYP3A inducers (e.g. phenytoin, carbamazepine, rifampin, rifabutin, rifapentine, phenobarbital, nevirapine, St. John's Wort) or moderate inducers (e.g bosentan, efavirenz or modafinil). The required washout period prior to starting olaparib is 5 weeks for phenobarbital or enzlautamide and 3 weeks for other agents. The washout requirement is measured from anticipated start of Olaparib, NOT from start of study. * Concomitant use of known strong CYP3A inhibitors (e.g. intraconazole, telithromycin, clarithromycin, protease inhibitors boosted with ritonavir or cobicistat, indinavir, saquinavir, nelfinavir, boveprevir, telaprevir) or moderate CYP3A inhibitors (e.g. ciprofloxacin, erythromycin, diltiazem, fluconazole, verapamil). The required washout period prior to starting olaparib is 2 weeks. The washout requirement is measured from anticipated start of olaparib, NOT from start of study. * Major surgery within 2 weeks of starting study treatment and patient must have recovered from any effects of any major surgery * Clinically significant cardiovascular disease as evidenced by: * myocardial infarction or arterial thrombotic events (e.g. stroke) in the past 6 months * resting EKG indicating uncontrolled, potentially reversible cardiac candiation, as judged by the investigator (e.g. unstable ischemia, uncontrolled symptomatic arrhythmia, QTc Fridericia prolongation \>500 ms) or patients with congenital long QT syndrome * severe or unstable angina, uncontrolled atrial fibrillation (controlled atrial fibrillation is allowed) or other (non-atrial fibrillation) cardiac arrhythmia requiring therapy * Active New York Heart Association Class II-IV heart failure * if any prior history of CHF (regardless of New York Heart Association assignment), a cardiac ejection fraction measurement (by echocardiography or multigated acquisition scan) is required within 6 months and mush not be \<50%. * Planned or scheduled cardiac surgery or percutaneous coronary intervention procedure * Prior revascularization procedure (coronary, carotid or peripheral artery stenosis) within the past 12 months * History of uncontrolled pituitary or adrenal dysfunction * Active infection or other medical condition that would make prednisone use contraindicated * Any chronic medical condition requiring a systemic dose of corticosteroid \>10 mg prednisone daily * Prior allogeneic bone marrow transplant or double umbilical cord blood transplantation * Participation in another clinical study with an investigational product or investigational medical devices within 1 month of registration * Involvement in the planning and/or conduct of the study (applies to both AstraZeneca and Merck staff and/or staff at the University of Michigan). * Judgment by the investigator that the patient should not participate in the study if the patient is unlikely to comply with study procedures, restrictions and requirements. * Uncontrolled hypertension (systolic blood pressure ≥160 mmHg) of diastolic blood pressure ≥95 mmHg. Patients with documented white coat syndrome (with home blood pressure machine compared to office for calibration) are allowed if home blood pressure measured daily for a week meet eligibility * Known hypersensitivity to olaparib, abiraterone, planned ADT agent (e.g. leuprolide, goserelin, degarelix), any of the excipients of any of these agents (olaparib, abiraterone, planned ADT agent) or drugs with a similar chemical structure to them or class to agents (olaparib, abiraterone or planned ADT) * Immunocompromised patients * Patients who are considered a poor medical risk due to a serious uncontrolled medical disorder, non-malignant systemic disease or active, uncontrolled infection. Examples not discussed elsewhere include, but are not limited to uncontrolled seizure disorder, superior vena cava syndrome or any psychiatric disorder that prohibits informed consent * Persistent toxicities (CTCAE Grade ≥2) caused by previous cancer therapy, excluding alopecia or sensory peripheral neuropathy * Patients who are unable to swallow orally administered medication and patients with gastrointestinal disorders likely to interfere with absorption of the study medication