A Study of Stereotactic Body Radiotherapy and 177Lu-PSMA-617 for the Treatment of Prostate Cancer

Inclusion Criteria: * The patient must have a biopsy proven adenocarcinoma of the prostate (biopsy confirmation of the primary tumor or oligometastatic tumor is acceptable) * The patient's primary tumor must have been previously treated with surgery and/or definitive radiation. Prior salvage treatments (radiation or surgery) to the prostate bed or pelvis is allowed. * Patients must have a negative multiparametric MRI and/or negative biopsy of the prostate (or prostate bed) even if other imaging modality (including PSMA) was negative for disease in the prostate (or prostate bed) within 2 months of enrollment on study °If patient is post-prostatectomy, the MRI Prostate can be excluded at clinician's discretion * Patients must have had a PSMA scan within 2 months of enrollment on study * Patient has not received any form of prostate-cancer directed therapy since undergoing screening PSMA scan * Patient must have 1-5 oligometastatic tumors or lesions of the bone or soft tissue that are detectable on a PSMA PET scan. * Outside PSMA scans not performed at MSKCC are acceptable but will require official read by MSKCC nuclear medicine for confirmation of metastasis * Patients with sclerotic, non-PSMA avid osseous lesions which are not felt to reflect active metastatic disease by a radiologist are eligible for the protocol assuming they also have 1-5sites of PSMA avid disease as well as no non-PSMA avid sites which are felt to reflect active metastatic prostate cancer * All oligometastatic lesions must be amenable to SBRT to a dose of 9 Gy x 3 without exceeding nationally recognized dose limits to adjacent organs at risk as deemed by the treating radiation oncologist * Patient's insurance is willing to cover SBRT treatment or the patient agrees to cover the costs of this therapy * Patient must have a prostate specific antigen (PSA) ≥ 0.2 ng/mL but ≤ 50 ng/mL * Patient may have had prior systemic therapy and/or ADT associated with treatment of their primary prostate cancer. Patient may have had ADT associated with salvage radiation therapy. * Patients who have been on hormone therapy within 12 months of consent must have testosterone within normal range (221-716 ng/dL at MSKCC) in order to be eligible for treatment on study. * Baseline testosterone below the normal range for patients who have not had hormone therapy within 12 months of study entry is acceptable * Patient must have an Eastern Cooperative Oncology Group (ECOG) performance status ≤ 2 or the equivalent KPS conversion value of ≥ 60 . * Patient must have a serum creatinine level \< 1.5 x ULN or EGFR \> 60 mL/min * Patient must have adequate liver laboratory values: * ALT and AST ≤ 2.5 x ULN * Albumin \> 2 g/ dL * Bilirubin \< 3 X ULN * Patient must have normal organ and marrow function as defined as: * Total white blood count \> 3.0 K/mcL * Absolute Neutrophil Count ≥ 1.5 K/mcL * Platelets ≥ 100 K/mcL * Hemoglobin ≥ 9 g/dL * The effects of 177Lu-PSMA-617 and SBRT on the developing human fetus at the recommended therapeutic dose are unknown. Men (including men with vasectomies) must agree to use adequate contraception (a condom and another effective method of birth control) prior to registration, for the duration of study participation, and for at least 3 months thereafter. Men must also agree not to donate sperm for the duration of study participation, and for at least 3 months thereafter. * Patient must be ≥ 18 years of age * Ability to understand, and willingness to sign the informed consent Exclusion Criteria: * Pathological findings consistent with small cell and/or neuroendocrine carcinoma of the prostate or any other histology not consistent with prostate adenocarcinoma * Patients with documented castration resistant prostate cancer (CRPC) * Patients with a PSMA scan within 2 months of enrollment that do not demonstrate metastatic lesions or demonstrate more than 5 discrete metastatic lesions * Patients with PSMA negative metastatic lesions or with a mixture of PSMA avid and PSMA negative disease * Patients with metastatic lesions not amenable to SBRT or treatment of which using a 9 Gy x 3 regimen would result in exceeding nationally accepted or institutional dose limits for nearby organs at risk * Patients with previous radiation therapy for oligometastatic disease are still eligible * Patients with prior radiotherapy to \> 25% of the skeleton or prior exposure to prior 223Radium, 89Strontium or 153Samarium containing compounds * Patients with spinal cord compression, impending spinal cord compression, or parenchymal brain metastases (patients with epidural disease without cord compression are eligible) * History of a prior or concurrent malignancy whose natural history or treatment has potential to interfere with the safety or efficacy assessments * Patients with known or suspected history of grade II or higher chronic kidney disease (CKD) * Severe or unstable angina, myocardial infarction, symptomatic congestive heart failure, arterial or venous thromboembolic events (e.g., pulmonary embolism, cerebrovascular accident including transient ischemic attacks), or clinically significant ventricular arrhythmias within 6 months prior to registration. * Seizure or known condition that may predispose to seizure (including but not limited to prior stroke, transient ischemic attack, loss of consciousness within 1 year prior to enrollment, brain arteriovenous malformation; or intracranial masses such as schwannomas and meningiomas that are causing edema or mass effect) * Gastrointestinal disorder affecting absorption * History of viral hepatitis or chronic liver disease with active symptoms * History of pituitary or adrenal dysfunction * Previously diagnosed active infection (e.g., human immunodeficiency virus \[HIV\] or viral hepatitis) * Any condition that in the opinion of the investigator, would preclude participation in this study * Concurrent corticosteroids and/or adrenal hormone inhibitors, PC-SPES, finasteride, or dutasteride is not allowed. * Receipt of any other investigational agents or participation in a concurrent treatment protocol * Inability to life flat during or tolerate PET CT/MR or SBRT * Patients must not have claustrophobia that would preclude PET/CT imaging or other contraindications to CT imaging * Known allergies, hypersensitivities, or intolerance to Lu-PSMA-617 or its inactive compounding components