Combining Ipilimumab, Degarelix, and Radical Prostatectomy in Men With Newly Diagnosed Metastatic Castration Sensitive Prostate Cancer or Ipilimumab and Degarelix in Men With Biochemically Recurrent Castration Sensitive Prostate Cancer After Radical Prostatectomy

Inclusion Criteria: Target Population: Cohort A Patients with castration-sensitive oligometastatic prostate cancer who have not received primary local therapy (radiation or surgery), and no more than 5 months of prior androgen deprivation therapy. * The subject must be age 18 or older, and be willing and able to provide informed consent. * The subject must have histologically confirmed adenocarcinoma of the prostate with tissue confirmation at selected study site. * The subject must have newly diagnosed prostate cancer with a metastatic site(s). * The subject must have a history or presence of ≤ 10 bony metastatic lesions * Note: bone mets that are not clearly identified on bone imaging, but are biopsy proven are allowed * History or presence of distant metastatic lymph node(s) (e.g., retroperitoneal or non-regional pelvic lymph nodes) are allowed * History or presence of regional pelvic lymph nodes (as per AJCC Cancer Staging \[7th edition\]) will be considered a metastatic site if greater than 1.5cm in shortest dimension. * The subject must have Karnofsky performance status of 80-100. * Normal organ function with acceptable initial laboratory values: * WBC ≥ 2000/μL * ANC ≥ 1000/ μL * Platelets ≥ 75 x 103/μL * Creatinine ≤ 2.0 x ULN * AST/ALT ≤ 2.5 x ULN * Bilirubin ≤ 1.5 x ULN (except patients with Gilbert's Syndrome, who must have a total bilirubin less than 3.0 mg/dL) * No active or chronic infection with HIV, Hepatitis B or Hepatitis C (negative screening tests required). * The subject must be deemed medically fit for radical prostatectomy by the attending urologic surgeon at the selected study site. * Men of fathering potential must be using an adequate method of contraception to avoid conception throughout the study \[and for up to 26 weeks after the last dose of ipilimumab\] in such a manner that the risk of pregnancy is minimized. Exclusion Criteria: * Patients that meet any of the criteria listed below will not be eligible for Cohort A entry: * Any other malignancy from which the patient has been disease-free for less than 5 years, with the exception of adequately treated and cured basal or squamous cell skin cancer or superficial bladder cancer * Major surgery within 4 weeks of enrollment (Week 1 Visit). * Current or prior radiation therapy to the prostate Prior radiation to a metastatic site (e.g., palliative radiation) is allowed.. * More than 5 months of prior hormonal therapy (e.g., gonadotropin hormone releasing analogs, megestrol acetate, or casodex) . There is no washout period required for GnRH analogs. A two week washout is required for megestrol or anti-androgen. * Prior use of an 5 alpha reductase inhibitor is allowed (no limit on duration of use), however a two week washout is required. * Prior ketoconazole, abiraterone acetate, or enzalutamide for the treatment of prostate cancer. * Current or prior investigational therapies for prostate cancer, or chemotherapy administered with the intent to treat prostate cancer. * Concomitant therapy with any other experimental drug. * Autoimmune disease: Patients with a history of inflammatory bowel disease, including ulcerative colitis and Crohn's Disease, are excluded from this study, as are patients with a history of symptomatic disease (e.g., rheumatoid arthritis, systemic progressive sclerosis \[scleroderma\], systemic lupus erythematosus, autoimmune vasculitis \[e.g., Wegener's granulomatosis\]); motor neuropathy considered of autoimmune origin (e.g., Guillain-Barre syndrome and myasthenia gravis). * Any underlying medical or psychiatric condition, which in the opinion of the investigator will make the administration of ipilimumab hazardous or obscure the interpretation of AEs, such as a condition associated with frequent diarrhea. * Patients with underlying heart conditions who are deemed ineligible for surgery. * Any non-oncology vaccine therapy used for prevention of infectious diseases (for up to 1 month before or after any dose of ipilimumab). * Note: Inactivated vaccines are allowed at any time on study. * A history of prior treatment with ipilimumab or prior CD137 agonist or CTLA-4 inhibitor or agonist. * Concomitant or prior therapy with any of the following: IL-2, interferon, or other nonstudy immunotherapy regimens; immunosuppressive agents; ; or chronic use of systemic corticosteroids within 6 weeks of study entry. * Persons of reproductive potential unwilling to use an adequate method of contraception throughout treatment and for at least 26 weeks after ipilimumab is stopped. * Patients who have had a history of acute diverticulitis, intra-abdominal abscess, GI obstruction and abdominal carcinomatosis which are known risk factors for bowel perforation. Inclusion Criteria: Target Population: Cohort B * The subject must be age 18 or older, and be willing and able to provide informed consent. * Histologically confirmed prostate cancer with progressive disease, defined as: * Rising PSA (50% or more increase to a level of 1 ng/mL or more, based on at least 3 PSA determinations obtained at least 1 week apart). The 50% rise in PSA is across the 3 determinations, and these determinations do not need to be sequential. * PSA doubling time of ≤ 12 months as calculated according to the Memorial Sloan-Kettering Cancer Center nomogram (http://www.mskcc.org/mskcc/html/10088.cfm) * The subject must have Karnofsky performance status of 80-100. * Normal organ function with acceptable initial laboratory values: * WBC ≥ 2000/μL * ANC ≥ 1000/ μL * Platelets ≥ 75 x 103/μL * Creatinine ≤ 2.0 x ULN * AST/ALT ≤ 2.5 x ULN * Bilirubin ≤ 1.5 x ULN (except patients with Gilbert's Syndrome, who must have a total bilirubin less than 3.0 mg/dL) * No active or chronic infection with HIV, Hepatitis B or Hepatitis C (negative screening tests required). * Prior radiotherapy to the prostate (adjuvant or salvage radiotherapy) is allowed * Men of fathering potential must be using an adequate method of contraception to avoid conception throughout the study \[and for up to 26 weeks after the last dose of ipilimumab\] in such a manner that the risk of pregnancy is minimized. Exclusion Criteria: Target Population: Cohort B Patients that meet any of the criteria listed below will not be eligible for Cohort B entry: * Any other malignancy from which the patient has been disease-free for less than 5 years, with the exception of adequately treated and cured basal or squamous cell skin cancer or superficial bladder cancer * Major surgery within 4 weeks of enrollment (Week 1 Visit). * Visceral metastatic disease * More than 3 cycles of intermittent hormones (for the treatment of biochemical recurrence or castration sensitive metastatic disease), with a cycle defined as a period of consistent androgen deprivation therapy (generally 3-12 months) followed by intentional cessation of ADT without reinitiation of ADT until the PSA rises. * Prior use of an 5 alpha reductase inhibitor is allowed (no limit on duration of use), however a two week washout is required. * Current or prior investigational therapies for prostate cancer, or chemotherapy administered with the intent to treat prostate cancer. * Concomitant therapy with any other experimental drug. * Autoimmune disease: Patients with a history of inflammatory bowel disease, including ulcerative colitis and Crohn's Disease, are excluded from this study, as are patients with a history of symptomatic disease (e.g., rheumatoid arthritis, systemic progressive sclerosis \[scleroderma\], systemic lupus erythematosus, autoimmune vasculitis \[e.g., Wegener's granulomatosis\]); motor neuropathy considered of autoimmune origin (e.g., Guillain-Barre syndrome and myasthenia gravis). * Any underlying medical or psychiatric condition, which in the opinion of the investigator will make the administration of ipilimumab hazardous or obscure the interpretation of AEs, such as a condition associated with frequent diarrhea. * Any non-oncology vaccine therapy used for prevention of infectious diseases (for up to 1 month before or after any dose of ipilimumab). * Note: Inactivated vaccines are allowed at any time on study. * A history of prior treatment with ipilimumab or prior CD137 agonist or CTLA-4 inhibitor or agonist. * Concomitant or prior therapy with any of the following: IL-2, interferon, or other non-study immunotherapy regimens; immunosuppressive agents; ; or chronic use of systemic corticosteroids within 6 weeks of study entry. * Persons of reproductive potential unwilling to use an adequate method of contraception throughout treatment and for at least 26 weeks after ipilimumab is stopped. * Patients who have had a history of acute diverticulitis, intra-abdominal abscess, GI obstruction and abdominal carcinomatosis which are known risk factors for bowel perforation.