Oxaliplatin, Fluorouracil, Erlotinib Hydrochloride, and Radiation Therapy Before Surgery and Erlotinib Hydrochloride After Surgery in Treating Patients With Locally Advanced Cancer of the Esophagus or Gastroesophageal Junction

Inclusion Criteria: * Newly diagnosed patients with locally advanced esophageal cancer with either squamous or adenocarcinoma histology; patients should have evidence of extension of disease into or through the wall of the esophagus (T2-4) and/or regional nodal metastasis (N1) * Eastern Cooperative Oncology Group (ECOG) performance status 0-2 * Non-pregnant; patients of childbearing potential and their partners must agree to use an effective form of contraception during the study and for 90 days following the last dose of study medication (an effective form of contraception is an oral contraceptive or a double barrier method); nursing mothers are also ineligible * Prior treatment: Greater than one week shall have elapsed since any major surgery; no prior chemotherapy or radiotherapy is allowed * Adequate whole blood cell (WBC) and platelets (Plt) as determined by medical oncology * Serum creatinine =\< 1.5 mg/dl * Creatinine clearance \>= 60 ml/min * Hemoglobin (Hgb) \>= 9.0 gm/dl * Absolute neutrophil count \>= 1,500/uL * Serum total bilirubin =\< 1.5 mg/dL * Alkaline phosphatase =\< 3X the upper limit of normal (ULN) for the reference lab * Alanine aminotransferase (ALT)/aspartate aminotransferase (AST) less than 2X ULN for the reference laboratory * Patients must be told of the investigational nature of the study and must sign a written informed consent * No serious medical or psychiatric illnesses which would prevent informed consent or otherwise limit survival to less than two years; no history of refractory congestive heart failure or cardiomyopathy * Patients should be evaluated by medical oncology, radiation oncology, and surgery, and felt to by all to be suitable for trimodality therapy Exclusion Criteria: Patients with an active infection or with a fever \>= 38.5 degrees Celsius (C) within 3 days of the first scheduled day of protocol treatment * History of prior malignancy within the past 5 years except for curatively treated basal cell carcinoma of the skin, cervical intra-epithelial neoplasia, or localized prostate cancer with a current prostate surface antigen (PSA) of \< 1.0 mg/dL on 2 successive evaluations, at least 3 months apart, with the most recent evaluation no more than 4 weeks prior to entry * Patients with known hypersensitivity to any of the components of oxaliplatin * Patients who are receiving concurrent investigational therapy or who have received investigational therapy within 30 days of the first scheduled day of protocol treatment (investigational therapy is defined as treatment for which there is currently no regulatory authority approved indication) * Peripheral neuropathy \>= Grade 2 * History of allogeneic transplant * Known human immunodeficiency virus (HIV) or Hepatitis B or C (active, previously treated or both) * Pregnancy