Chemotherapy With or Without Radiation or Surgery in Treating Participants With Oligometastatic Esophageal or Gastric Cancer

Inclusion Criteria: * The patient has a pathologic diagnosis of tumor biopsy or FNA of esophageal or gastric cancer of adenocarcinoma histology * The patient is staged with EGD and PET/CT scan. * The patient has three or less observable metastatic lesions. Patients may have three or less radiographically visible metastatic lesions at diagnosis or if have regressed to three or less metastatic lesions after induction chemotherapy at time of randomization. The patient must have pathologic confirmation and or radiologically visible disease. For esophageal tumors, the maximal dimension of the primary tumor may not provide reproducible measurements for RECIST and may not be visible on CT or PET/CT at diagnosis or after induction chemotherapy. Accordingly, patients are eligible regardless of the imaging measurements of the primary tumor. Additionally, in patients with non-measurable metastases, patients are eligible if there is pathology confirming metastases from a distant site. However, biopsy of a metastatic site is not required if there are visible metastases on imaging (such as ultrasound, diagnostic CT , EUS, PET/CT). * The patient has three or less observable metastatic lesions by diagnostic scans (CT scan, PET/CT, eEndoscopic ultrasound, MRI, or bone scan). Metastatic lesions include distant M1 lymph node group; which will be counted as one site (M1 metastatic lymph nodes to include cervical, mediastinal, gastric, retroperitoneal lymph nodes will be counted as one lesion). * Osseous metastases or visceral metastases will each count as one metastatic site. * Each CNS metastases will count as one metastatic site. * Satellite lesions in the primary esophageal malignancy such as skipped esophageal primaries are not considered metastatic sites. Symptomatic metastatic sites can be treated locally prior to randomization or by palliative radiation. * Symptomatic metastatic sites may be treated with radiation or surgery prior to enrollment. * Patient ECOG of 0-2, with life expectancy of at least 6 months * Patients age \>18 yrs old but \<80 yrs old and signed informed consent * Women of child-bearing age must have pregnancy test at time of enrollment, agree to use of adequate contraception (birth control hormone or barrier method) for the duration of the study and for six months after discontinuation of systemic agents. Exclusion Criteria: * Patients with prior chemotherapy or radiation therapy for their diagnosis of esophageal or gastric cancer. Patients with prior radiation therapy to same site for another diagnosis of cancer. Note: Patients may receive palliative radiation to their symptomatic sites of metastases but not definitive local therapy to esophageal or gastric primary prior to randomization. All patients may be enrolled on protocol then start systemic therapy; if they do not have evidence of disease progression at re-staging following initial therapy, they may be randomized. * Patients with fistula documented radiographically or by EDG/EUS, EBUS. * Patients with life expectancy less than 6 months, ECOG \>3 * Female patients who are pregnant confirmed by bHCG lab test. * Patient has history of uncontrolled angina, congestive heart failure or recent MI within 6 months. * Nursing females * Patients in poor nutritional state * Patients with: * Severely depressed bone marrow function * Potentially serious infections * Known hypersensitivity to 5-fluorouracil * Known or suspected to have a dihydropyrimidine dehydrogenase deficiency (as these patients are at a greater risk of experiencing symptoms of toxicity)