Nivolumab+Ipilimumab+RT in MSS mCRC

Inclusion Criteria: * Participants must have histologically or cytologically confirmed adenocarcinoma of colorectal origin * Age \>18 years. * ECOG performance status \<1 * Life expectancy of greater than 3 months * Participants must have normal organ and marrow function as defined in Table 1, all screening labs should be performed within 14 days of protocol registration. * Table 1 Adequate Organ Function Laboratory Values: * System Laboratory Value * Hematological * Absolute neutrophil count (ANC) ≥1000 /mcL * White blood count (WBC) ≥2000 /mcL * Platelets ≥75,000 / mcL * Hemoglobin ≥7.5 g/dL * Renal * Serum creatinine OR Measured or calculated creatinine clearance (GFR can also be used in place of creatinine or CrCl)≤ Serum creatinine ≤ 1.5 x ULN or creatinine clearance (CrCl) * 40 mL/min (if using the Cockcroft-Gault formula below): Female CrCl = (140 - age in years) x weight in kg x 0.85 72 x serum creatinine in mg/dL Male CrCl = (140 - age in years) x weight in kg x 1.00 72 x serum creatinine in mg/dL * Hepatic * Serum total bilirubin ≤ 1.5 X ULN (subjects with Gilbert Syndrome can have a total bilirubin \<3 mg/dL * AST (SGOT) and ALT (SGPT) ≤ 3 X ULN OR ≤ 5 X ULN for subjects with liver metastases * Coagulation * International Normalized Ratio (INR) or Prothrombin Time (PT) ≤1.5 X ULN unless subject is receiving anticoagulant therapy as long as PT is within therapeutic range of intended use of anticoagulants * Activated Partial Thromboplastin Time (aPTT) ≤2.5 X ULN unless subject is receiving anticoagulant therapy as long as PTT is within therapeutic range of intended use of anticoagulants * Creatinine clearance should be calculated per institutional standard. * Women of childbearing potential (WOCBP) must use appropriate method(s) of contraception. WOCBP should use an adequate method to avoid pregnancy for 5 months (30 days plus the time required for nivolumab to undergo five half-lives) after the last dose of investigational drug. * Women of childbearing potential must have a negative serum or urine pregnancy test (minimum sensitivity 25 IU/L or equivalent units of HCG). * Women must not be breastfeeding. * Men who are sexually active with WOCBP must use any contraceptive method with a failure rate of less than 1% per year. Men receiving nivolumab and who are sexually active with WOCBP will be instructed to adhere to contraception for a period of 7 months after the last dose of investigational product. Women who are not of childbearing potential, ie, who are postmenopausal or surgically sterile as well as azoospermic men do not require contraception. * Ability to understand and the willingness to sign a written informed consent document. * If applicable, stable dose of dexamethasone 2 mg or less (or equivalent dose of another steroid) for 7 days prior to initiation of treatment. * One previously unirradiated measurable lesion amenable to radiotherapy 8 Gy x 3 and can meet dose constraints, and another unirradiated measurable lesion \> 1 cm in size (\>15 mm for nodal disease) outside the radiation field that can be used as measurable disease. * Colorectal patients must have documentation of microsatellite status: MSS or pMMR. Immunohistochemistry (IHC) and/or PCR is acceptable. * Colorectal patients must have received any prior combination of Fluorouracil (5FU), Irinotecan and Oxaliplatin, or have a contraindication/intolerance to receiving these agents. Patients may have received these agents all together or sequentially. Exclusion Criteria: * Participants who have had chemotherapy, targeted small molecule therapy or study therapy within 14 days of protocol treatment, or those who have not recovered (i.e., ≤ Grade 1 or at baseline) from adverse events due to agents administered more than 2 weeks earlier. Subjects with ≤ Grade 2 neuropathy are an exception to this criterion and may qualify for the study. If subject received major surgery, they must have recovered adequately from the toxicity and/or complications from the intervention prior to starting therapy. * Patients who have had radiation within 8 weeks prior to protocol registration. * Participants who are receiving any other investigational agents. * Patients are excluded if they have an active, known or suspected autoimmune disease other than those listed below. Subjects are permitted to enroll if they have vitiligo, type I diabetes mellitus, residual hypothyroidism due to autoimmune condition only requiring hormone replacement, psoriasis not requiring systemic treatment, or conditions not expected to recur in the absence of an external trigger. * Patients are excluded if they have a condition requiring systemic treatment with either corticosteroids (\> 12 mg daily prednisone equivalents) or other immunosuppressive medications within 14 days of study drug administration. Inhaled or topical steroids and adrenal replacement doses \> 12 mg daily prednisone equivalents are permitted in the absence of active autoimmune disease. Subjects are permitted to use topical, ocular, intraarticular, intranasal, and inhalational corticosteroids (with minimal systemic absorption). Physiologic replacement doses of systemic corticosteroids are permitted, even if \> 12 mg/day prednisone equivalents. A brief course of corticosteroids for prophylaxis (e.g., contrast dye allergy) or for treatment of non-autoimmune conditions (e.g., delayed-type hypersensitivity reaction caused by contact allergen) is permitted. * Patients are excluded if they have previously received anti-CTLA-4 therapy. Prior anti- PD-1 or anti-PD-L1 therapy is permitted with 6-month washout period unless the following treatment-related toxicities are present: * Any toxicity NCI CTCAE grade \>/= 3 from previous immunotherapy that did not resolve to grade 1 with or without immunosuppressive therapy. Patients must be off all immunosuppressive therapy prior to enrollment. * Myocarditis, any grade. * Pneumonitis, any grade. * Patients with grade 2 hepatitis or colitis will be evaluated on a case-by-case basis and may be included only after consultation with a Gastroenterologist and the study physician. * Has a known history of active TB (Bacillus Tuberculosis). * No active or chronic HBV or HCV. Patients are excluded if they have a positive test for hepatitis B virus surface antigen (HBV sAg) or hepatitis C virus ribonucleic acid (HCV antibody) indicating acute or chronic infection. Subjects with a history of HBV or HCV require documentation of treatment completion, further testing is not required. * Patients are excluded if they have known history of testing positive for human immunodeficiency virus (HIV) or known acquired immunodeficiency syndrome (AIDS). These participants are at increased risk of lethal infections when treated with marrow suppressive therapy. Appropriate studies will be undertaken in participants receiving combination antiretroviral therapy when indicated. * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements. * Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial. * Is pregnant or breastfeeding, or expecting to conceive or father children within the projected duration of the trial, starting with the pre-screening or screening visit through 5 months for woman and 7 months for men, after the last dose of trial treatment. * Has a known additional malignancy that is progressing or requires active treatment. Exceptions include basal cell carcinoma of the skin and squamous cell carcinoma of the skin that has undergone potentially curative therapy or in situ cervical cancer. * Has known history of, or any evidence of active, non-infectious pneumonitis. * Has an active infection requiring systemic therapy. * Has received a live/attenuated vaccine within 30 days of planned start of study therapy. Note: Seasonal influenza vaccines for injection are generally inactivated flu vaccines and are allowed; however intranasal influenza vaccines (e.g., Flu-Mist®) are live attenuated vaccines, and are not allowed. * History of allergy to study drug components. * History of severe hypersensitivity reaction to any monoclonal antibody. * Uncontrolled brain metastases. Patients treated with radiation \> 4 weeks prior with follow up imaging showing control are eligible