Gemcitabine Hydrochloride, Oxaliplatin, and Erlotinib Hydrochloride in Treating Patients With Advanced Biliary Tract Cancer, Pancreatic Cancer, Duodenal Cancer, or Ampullary Cancer

Inclusion Criteria: * Only advanced carcinomas defined as unresectable or metastatic that are histologically or cytologically confirmed to be biliary tract, pancreas, duodenal, or ampullary carcinomas will be included. * Dose-escalation: Patients \> 18 years of age with biopsy-confirmed advanced biliary tract adenocarcinoma, pancreas cancer, duodenal cancer, or ampullary cancer * MTD expansion cohort: Patients \> 18 years of age with biopsy-confirmed advanced biliary tract adenocarcinoma only. * No prior chemotherapy or prior EGF receptor inhibitor therapy * Measurable tumor by imaging examination * Performance status (PS) 0-2 on the ECOG performance scale * Have pretreatment bilirubin\<2.5x upper limit of normal (ULN), serum creatinine\<1.5x ULN, AST and ALT \<2.5xULN or in the presence of liver metastasis \<5xULN, neutrophils\>1500, platelets\>100K, hemoglobin \>9 g/dL * Patients - both males and females - with reproductive potential (ie, menopausal for less than 1 year and not surgically sterilized) must practice effective contraceptive measures throughout the study. Women of childbearing potential must provide a negative pregnancy test (serum or urine) within 14 days prior to registration. * Have the ability to understand the requirements of the study and provide informed consent Exclusion Criteria: * CNS metastases * Uncontrolled infection * Pregnant or nursing women may not participate. * No other prior malignancy is allowed except for the following: adequately treated basal cell or squamous cell skin cancer, in situ cervical cancer, adequately treated Stage I or II cancer from which the patient is currently in complete remission, or any other cancer from which the patient has been disease-free for 5 years. * Psychiatric illness that would prevent understanding the nature of the investigational therapy and complying with protocol requirements * Patients with \> grade 2 neuropathy * Patients with \> grade 2 uncontrolled nausea and vomiting despite antiemetics * Any concurrent medical condition that, in the judgment of the investigator, would make the patient an inappropriate candidate for study enrollment * Prior chemotherapy or EGFR inhibitor