Study of Durvalumab and Tremelimumab After Radiation for Microsatellite Stable Metastatic Colorectal Cancer Progressing on Chemotherapy

Inclusion Criteria: * The ECOG performance status must be 0 or 1. * There must be histologic confirmation of a diagnosis of colorectal adenocarcinoma. * The tumor must have been determined to be microsatellite stable (MSS). * There must be documentation by positron emission tomography (PET)/computed tomography (CT) scan, CT scan, or magnetic resonance imaging (MRI), that the patient has evidence of measurable metastatic disease per RECIST 1.1. * Patients must have an accessible metastatic lesion for pretreatment core biopsy. * Unless either drug is medically contraindicated, patients must have received oxaliplatin and irinotecan as part of standard metastatic chemotherapy regimens. * The patient must have multiple sites of metastatic disease with at least one lesion amenable to treatment with stereotactic radiation therapy (SBRT) in the lung or liver and at least one lesion not being irradiated and meeting RECIST 1.1. * At the time of study entry, blood counts performed within 2 weeks prior to study entry must meet the following criteria: * ANC (absolute neutrophil count) must be greater than or equal to 1500/mm3, * Platelet count must be greater than or equal to 100,000/mm3; and * Hemoglobin must be greater than or equal to 9 g/dL. * The following criteria for evidence of adequate hepatic function performed within 2 weeks prior to study entry must be met: * Total bilirubin must be less than or equal to 1.5 x ULN (upper limit of normal) for the lab unless the patient has a bilirubin elevation greater than 1.5 x ULN to 3 x ULN due to Gilbert's disease or similar syndrome involving slow conjugation of bilirubin; and * AST (aspartate aminotransferase)and ALT (alanine transaminase) must be less than or equal to 2.5 x ULN for the lab with the following exception: for patients with documented liver metastases, AST and ALT must be less than or equal to 5 x ULN. * Adequate renal function within 4 weeks prior to study entry, defined as serum creatinine less than or equal to 1.5 x ULN for the lab or measured or calculated creatinine clearance greater than 40 mL/min by Cockcroft-Gault formula. * All hematologic, gastrointestinal, and genitourinary chemotherapy toxicities must be less than Grade 2 at the time study therapy is to begin. (Note: Transfusions may be used to correct hemoglobin for patients experiencing anemia from therapy who otherwise would be eligible for the study. * Patients with reproductive potential (male/female) must agree to use accepted and highly effective methods of contraception while receiving study therapy and for at least 6 months after the completion of study therapy. The definition of effective method of contraception will be based on the investigator's discretion. * Female patients must either be of non-reproductive potential (i.e., post-menopausal by history: greater than or equal to 60 years old and no menses for greater than or equal to 1 year without an alternative medical cause; OR history of hysterectomy, OR history of bilateral tubal ligation, OR history of bilateral oophorectomy) or must have a negative serum pregnancy test upon study entry. Exclusion Criteria: * Diagnosis of anal or small bowel carcinoma. * Colorectal cancer other than adenocarcinoma, e.g., sarcoma, lymphoma, carcinoid. * Previous therapy with any PD-1 or PD-L1 (programmed cell death-1 (PD-1) protein, programmed death ligand-1 (PDL-1) protein inhibitor including durvalumab or anti-CTLA4 (including tremelimumab) for any malignancy. * Receipt of live attenuated vaccination within 30 days prior to study entry or within 30 days of receiving study therapy. * Active or chronic hepatitis B or hepatitis C. * Symptomatic or uncontrolled brain metastases requiring concurrent treatments, uncontrolled spinal cord compression, carcinomatous meningitis, or new evidence of brain or leptomeningeal disease; uncontrolled seizures. * Active infection or chronic infection requiring chronic suppressive antibiotics. * Active or documented inflammatory disease. * Known history of human immunodeficiency virus (HIV) or acquired immunodeficiency-related (AIDS) illnesses. * Current or prior use of immunosuppressive medication within 28 days before the first dose of study therapy with the exceptions of intranasal corticosteroids or systemic corticosteroids at physiological doses that do not exceed 10mg/day of prednisone or an equivalent corticosteroid. * History of allogeneic organ transplantation. * Any of the following cardiac conditions: * Documented NYHA (New York Heart Association) Class III or IV congestive heart failure, * Myocardial infarction within 6 months prior to study entry, * Unstable angina within 6 months prior to study entry, * Symptomatic arrhythmia. If QTc (corrected QT interval) greater than or equal to 470ms, confirmation of eligible QTc requires mean calculation from 2 additional electrocardiograms (ECGs) 2-5 minutes apart using Fridericia's Correction Formula (mean less than 470 ms). * Uncontrolled high blood pressure defined as systolic blood pressure (BP) greater than or equal to 150 mmHg or diastolic BP greater than or equal to 100 mmHg with or without anti-hypertensive medication. Patients with initial BP elevations are eligible if initiation or adjustment of BP medication lowers pressure to meet entry criteria. * Active or prior documented inflammatory bowel disease (e.g., Crohn's disease, ulcerative colitis). * Ongoing or active gastritis or peptic ulcer disease. * Active bleeding diatheses. * Known history of previous diagnosis of tuberculosis. * History of hypersensitivity to durvalumab or tremelimumab or any excipients of these drugs. * Known history or confirmation of active pneumonia, pneumonitis, symptomatic interstitial lung disease, or definitive evidence of interstitial lung disease described on CT scan, MRI, or chest x-ray in asymptomatic patients; dyspnea at rest requiring current continuous oxygen therapy. * Active or prior documented autoimmune disease within the past 2 years. (Note: Patients with vitiligo, Grave disease, or psoriasis not requiring systemic treatment within the past 2 years are eligible.) * Other malignancies unless the patient is considered to be disease-free and has completed therapy for the malignancy greater than or equal to 12 months prior to study entry. Patients with the following cancers are eligible if diagnosed and treated within the past 12 months: carcinoma in situ of the cervix, and basal cell and squamous cell carcinoma of the skin. * Psychiatric or addictive disorders or other conditions that, in the opinion of the investigator, would preclude the patient from meeting the study requirements or interfere with interpretation of study results. * Pregnancy or lactation at the time of study entry. (Note: Pregnancy testing should be performed within 14 days prior to study entry according to institutional standards for women of childbearing potential.) * Use of any investigational agent. Use and/or receipt of the last dose of anti-cancer therapy (chemotherapy, immunotherapy, endocrine therapy, targeted therapy, biologic therapy, tumor embolization, monoclonal anti-bodies) within 14 days prior to the first dose of study therapy. * Use of any investigational agent within 4 weeks of starting study therapy.