Durvalumab, an Anti-PDLI Antibody, and Tremelimumab, an Anti-CTLA4 Antibody, and Chemoradiation Before Surgery for Esophageal Cancer

Inclusion Criteria: * Histologically or cytologically confirmed adenocarcinoma of the esophagus or gastroesophageal junction (GEJ). Pathology must be confirmed at Memorial Sloan Kettering Cancer Center * Tumors that are Her2 positive are eligible * Availability of archived tumor tissue for banking * TanyN+M0 or T3-4NanyM0 tumors * Disease must be clinically limited to the esophagus or GEJ. GEJ tumors must be Siewert Type I-III * No prior chemotherapy * Prior radiation is permitted, provided it does not limit the ability to deliver per-protocol radiation in the opinion of the treating radiation oncologist * Patients must have surgically resectable disease treatable by esophagectomy, as assessed by a thoracic surgeon * mSUV in the primary tumor must be ≥5.0 * Patients must be ≥18 years of age * Eastern Cooperative Oncology Group performance status of 0-1 * Female subjects must either be of non-reproductive potential (i.e. post-menopausal by history: ≥60 years old and no menses for ≥1 year without an alternative medical cause; * OR history of hysterectomy, OR history of bilateral tubal ligation, OR history of bilateral oophorectomy) or must have a negative serum pregnancy test upon study entry * Adequate organ function defined at baseline as: * WBC ≥3,000/ L * ANC ≥1,500/ L * Platelets ≥100,000/ L * Hb ≥9 g/dl * Calculated creatinine clearance \>40 ml/min using Cockcroft-Gault method: Males: Creatinine CL = Weight (kg) x (140 - Age) . (mL/min) 72 x serum creatinine (mg/dL) Females: Creatinine CL (mL/min) = Weight (kg) x (140 - Age) x 0.85 72 x serum creatinine (mg/dL) * Total serum bilirubin ≤1.5 mg/dL * AST/ALT ≤2.5× upper limit of normal * Mean QT interval corrected for heart rate (QTc) \<470 ms calculated from 3 ECGs using Frediricia's Correction * Able to provide written informed consent * Subject willing to provide informed consent for MSKCC IRB#12-245 for IMPACT testing * Subject is willing and able to comply with the protocol for the duration of the study including undergoing treatment and scheduled visits and examinations including follow up Exclusion Criteria: * Carcinoma in-situ and tumors determined to be T1-2N0 * Tumors with significant involvement of the proximal stomach which, in the opinion of the treating thoracic surgeon, would require an esophagogastrectomy * Patients with evidence of metastatic disease, including: * Positive malignant cytology of the pleural, pericardium or peritoneum * Radiographic evidence of distant organ involvement * Non-regional lymph nodes that cannot be contained within a radiation field * Biopsy-proven tumor invasion of the tracheobronchial tree or presence of tracheoesophageal fistula. Recurrent laryngeal or phrenic nerve paralysis * Grade 2 ≥ peripheral neuropathy * Active or prior documented autoimmune or inflammatory disorders (including inflammatory bowel disease; systemic lupus erythematosus; Wegener syndrome \[granulomatosis with polyangiitis\]; myasthenia gravis; Graves' disease; rheumatoid arthritis, hypophysitis, uveitis) within the past 3 years prior to the start of treatment. The following are exceptions to this criterion: * Subjects with vitiligo or alopecia * Subjects with hypothyroidism (e.g., following Hashimoto syndrome) stable on hormone replacement or psoriasis not requiring systemic treatment * History of pneumonitis * The use of immunosuppressive medication within 28 days prior to the first dose of durvalumab-/tremelimumab. The following are exceptions to this criterion: * Intranasal, topical, inhaled corticosteroids or local steroid injections (e.g. intraarticular injection) * Systemic corticosteroids at physiologic doses ≤10 mg/day of prednisone or equivalent * Steroids as premedication for hypersensitivity reactions (e.g. CT scan premedication) * Known HIV positivity * Chronic Hepatitis B or known Hepatitis C infection (e.g. Hepatitis B surface Ag positive or detectable viral load for Hepatitis B). Patients with prior evidence of Hepatitis B or C without active infection are eligible * Known history of previous clinical diagnosis of tuberculosis * Uncontrolled seizures * Pregnant or breast-feeding women. Women of childbearing potential (WOCBP) must undergo a negative pregnancy test (either serum or urine) prior to study entry. Male and female patients of reproductive potential need to employ two highly effective and acceptable forms of contraception throughout their participation in the study and for 90 days after last dose of study drug. WOCBP include: * Any woman who has experienced menarche and who has not undergone surgical sterilization (hysterectomy, bilateral tubal ligation or oophorectomy) or who is not post-menopausal (defined as amenorrheic ≥12 consecutive months) * Women on hormone replacement therapy with documented serum follicle stimulating hormone level \> 35 mIU/ml * Women who are using oral, implanted or injectable contraceptive hormones or mechanical products such as intrauterine device or barrier methods to prevent pregnancy or are practicing abstinence of where the partner is sterile * Prior malignancy (other than basal cell/squamous cell carcinoma of the skin, in-situ cervical carcinoma or superficial transitional cell bladder carcinoma) diagnosed and/or treated within three years of study entry * Connective tissue disorders, e.g. scleroderma, that in the opinion of the treating physicians is a contraindication to radiation therapy * History of primary immunodeficiency * History of allogenic organ transplant * Receipt of live attenuated vaccination within 30 days prior to study entry or within 30 days of receiving durvalumab. For example, the intramuscular influenza vaccine can be administered but the intranasal vaccine is a live attenuated virus that cannot be given * Any previous treatment with a PD-1 or PD-L1 inhibitor, including durvalumab, or a CTLA-4 inhibitor, including tremelimumab. * History of hypersensitivity to durvalumab or tremelimumab or any excipient Uncontrolled intercurrent illness including but not limited to, ongoing or active infection, symptomatic congestive heart failure, uncontrolled hypertension, unstable angina pectoris, cardiac arrhythmia, active peptic ulcer disease, active bleeding diatheses or psychiatric illness/social situations that would limit compliance with study requirements or compromise the ability of the subject to give written informed consent