Erlotinib and Combination Chemotherapy in Treating Patients With Metastatic Colorectal Cancer

DISEASE CHARACTERISTICS: * Histologically confirmed colon or rectal cancer * Metastatic or unresectable disease * Unidimensionally measurable disease required for phase II only * At least 20 mm by x-ray, CT scan, MRI, or photography * The following are not considered measurable: * Pleural effusion or ascites * Osteoblastic lesions * Evidence of disease on bone scan alone * Progressive irradiated lesions alone * Bone marrow involvement * Brain metastases * Malignant hepatomegaly by physical exam alone * Chemical markers (e.g., carcinoembryonic antigen) * Recurrent disease after surgery or radiotherapy is considered measurable as long as the following criteria are met: * At least 4 weeks since prior surgery or radiotherapy * Measurable disease exists outside the radiation port or clear progression exists within the radiation port * Tissue accessible for immunohistochemical evidence of epidermal growth factor receptor expression from a metastatic site (phase II only) * No known brain metastases PATIENT CHARACTERISTICS: Age * 18 and over Performance status * ECOG 0-2 OR * Karnofsky 60-100% Life expectancy * More than 3 months Hematopoietic * WBC at least 3,000/mm\^3 * Absolute neutrophil count at least 1,500/mm\^3 * Platelet count at least 100,000/mm\^3 Hepatic * Bilirubin normal * AST/ALT no greater than 2.5 times upper limit of normal Renal * Creatinine normal OR * Creatinine clearance at least 60 mL/min Cardiovascular * No unstable angina pectoris * No symptomatic congestive heart failure * No cardiac arrhythmia * No uncontrolled hypertension (systolic blood pressure greater than 150 mm Hg) Opthalmic * No abnormalities of the cornea (e.g., severe dry eye syndrome or Sjogren's syndrome) * No congenital abnormality (e.g., Fuch's dystrophy) * No abnormal slit-lamp examination using vital dye (e.g., fluorescein or Bengal-Rose) * No abnormal corneal sensitivity test (e.g., Schirmer test or similar tear-production test) * Mild dry eye syndrome allowed if patient can use artificial tears and ophthalmologist concurs Gastrointestinal * No gastrointestinal tract disease resulting in an inability to take oral medication or a requirement for IV alimentation * No active peptic ulcer disease Other * Must be able and willing to undergo a mediport insertion * Not pregnant or nursing * Negative pregnancy test * Fertile patients must use effective contraception * No other malignancy within the past 5 years except previously excised and inactive basal cell or squamous cell skin cancer * No prior allergic reactions attributed to compounds of similar chemical or biological composition to erlotinib or other study drugs (e.g., epidermal growth factor inhibitors like cetuximab) * No significant traumatic injury within the past 3 weeks * No peripheral neuropathy grade 2 or greater * No ongoing or active infection * No other uncontrolled concurrent illness that would preclude study entry * No psychiatric illness or social situation that would preclude study compliance PRIOR CONCURRENT THERAPY: Biologic therapy * Not specified Chemotherapy * More than 4 weeks since prior chemotherapy (6 weeks for nitrosoureas or mitomycin) and recovered * Phase I: * Prior chemotherapy allowed * Phase II: * No prior chemotherapy for metastatic disease * Prior adjuvant therapy allowed if disease progresses during adjuvant therapy * No prior oxaliplatin Endocrine therapy * Not specified Radiotherapy * See Disease Characteristics * More than 4 weeks since prior radiotherapy and recovered Surgery * See Disease Characteristics * More than 3 weeks since prior major surgery and recovered * No prior surgical procedures affecting absorption Other * No other concurrent investigational agents * No other concurrent anticancer agents or therapies (commercial or investigational) * No concurrent combination antiretroviral therapy for HIV-positive patients