Panitumumab With or Without Trametinib in Treating Patients With Stage IV Colorectal Cancer

Inclusion Criteria: * Histologically or cytologically confirmed colorectal adenocarcinoma, with metastatic disease documented on diagnostic imaging studies * Progression during or within 6 months after fluoropyrimidine, irinotecan, and oxaliplatin; for oxaliplatin-based therapy, failure of therapy will also include patients who progressed within 12 months of adjuvant therapy and patients who had oxaliplatin discontinued secondary to toxicity or allergic reaction; patients with a known history of Gilbert's disease who cannot receive irinotecan or patients who are intolerant of irinotecan or fluoropyrimidine are eligible * Confirmed wild-type status in KRAS exons 2, 3, and 4; NRAS exons 2, 3, and 4; and BRAF, by standard of care testing of tumor specimen; tissue used for testing may have been collected prior to treatment with anti-EGFR therapy * Patient must have been already tested and have available results of the mutations status of KRAS/NRAS/BRAF/MEK (MAP2K1) and EGFR from the circulating tumor DNA within 10 weeks prior to starting study therapy * Previous treatment with anti-EGFR therapy with evidence of clinical benefit, as defined by complete response, partial response, or prolonged stable disease with 16 or more weeks of treatment without radiographic progression, as assessed by the treating physician and documented in the medical record; this treatment may have occurred at any point in the patient's clinical course for treatment of metastatic colorectal cancer * Ultimate progression through previous treatment with anti-EGFR therapy, with documented clinical progression; patients who discontinued anti-EGFR therapy for any other reason, such as decline in performance status, hypersensitivity, or other adverse effects of therapy, are not eligible * All prior treatment-related toxicities must be Common Terminology Criteria for Adverse Events (CTCAE) (version 4.0) =\< grade 1 (except =\< grade 2 for alopecia peripheral neuropathy) * Radiographically measurable disease present per Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 * Eastern Cooperative Oncology Group (ECOG) performance status 0 or 1 * Blood counts performed within 3 weeks prior to starting study therapy must have absolute neutrophil count \>= 1,500/mm\^3 * Blood counts performed within 3 weeks prior to starting study therapy must have platelets \>= 100,000/mm\^3 * Blood counts performed within 3 weeks prior to starting study therapy must have hemoglobin \>= 9 g/dL * Liver function tests performed within 3 weeks prior to starting study therapy must have total bilirubin =\< 1.5 x upper limit of normal (ULN) * Liver function tests performed within 3 weeks prior to starting study therapy must have alanine aminotransferase and aspartate aminotransferase =\< 2.5 x ULN (or =\< 5 x ULN if liver metastases are present) * Liver function tests performed within 3 weeks prior to starting study therapy must have albumin \>= 2.5 g/dL * Serum creatinine performed within 3 weeks prior to starting study therapy must be =\< 1.5 x ULN, or have calculated creatinine clearance (using Cockcroft-Gault formula) of \>= 50 mL/minute * Prothrombin time (PT)/international normalized ratio (INR) and partial thromboplastin time (PTT) performed within 3 weeks prior to starting study therapy must be =\< 1.5 x ULN * Left ventricular ejection fraction (LVEF) \>= lower limit of normal (LLN) by echocardiogram (ECHO) or multiple gated acquisition scan (MUGA) within 3 weeks prior to starting study therapy * Women of childbearing potential must have a negative serum pregnancy test within 14 days prior to randomization and must agree to use effective contraception throughout the treatment period and for 4 months after the last dose of study treatment * Ability to sign informed consent form; informed consent form for this study must be signed prior to the performance of any study-specific procedures and initiation of any study therapy * Ability to swallow and retain oral medication, with no clinically significant gastrointestinal abnormalities that may alter absorption such as malabsorption syndrome or major resection of the stomach or bowels * In cohort 1, must have EGFR S492R or other ectodomain mutation detected from circulating tumor DNA from plasma collected after progression on prior anti-EGFR therapy; may have a concomitant mutation in KRAS, NRAS, or BRAF, if there is at least a 5-fold higher allele frequency of the most prevalent EGFR mutation than the most prevalent KRAS/NRAS/BRAF mutation * In cohort 2, must have one or more mutations found in KRAS exon 2, 3, or 4; NRAS exon 2, 3, or 4; BRAF codon 600; or in MEK (MAP2K1); may have a concomitant EGFR ectodomain mutation, if the most prevalent EGFR ectodomain mutation does not have over a 5-fold higher allele frequency than the most prevalent KRAS/NRAS/BRAF mutation * In cohort 3, must not have EGFR ectodomain mutation or any mutations in KRAS, NRAS, or BRAF Exclusion Criteria: * Past treatment with any MEK or ERK inhibitor * Previous retreatment with anti-EGFR therapy following progression on initial course of anti-EGFR therapy * Known untreated brain metastasis or brain metastasis treated within 3 months prior to enrollment in this trial * Symptomatic or untreated leptomeningeal or brain metastases or spinal cord compression * History of interstitial lung disease or pneumonitis * History of any other malignancy within 3 years, except for adequately treated carcinoma in situ of the cervix or non-melanoma skin cancer and/or subjects with indolent second malignancies are eligible * Prior treatment within 21 days of the first dose of study drug with any other chemotherapy, immunotherapy, biologic therapy, vaccine therapy, or investigational treatment, or failure to recover from adverse effects of prior therapies administered over 4 weeks prior to study day 1; all toxicities from prior therapies must be =\< grade 1 (or =\< grade 2 for alopecia or peripheral neuropathy); prior systemic treatment in the adjuvant setting is allowed * Any major surgery, extensive radiotherapy, chemotherapy with delayed toxicity, biologic therapy, or immunotherapy within 21 days prior to randomization and/or daily or weekly chemotherapy without the potential for delayed toxicity within 14 days prior to randomization * Impaired cardiac function or clinically significant cardiac disease, as defined: a) left ventricular ejection fraction \< lower limit of normal (LLN) on multiple gated acquisition scan (MUGA) or echocardiogram; b) congenital long QT syndrome or family history of unexpected sudden cardiac death; c) corrected QT (QTc) corrected with Bazett's formula (QTcB) \>= 480 ms.; d) history or evidence of current clinically significant uncontrolled arrhythmias; note subjects with atrial fibrillation controlled for \> 30 days prior to dosing are eligible; e) history of acute coronary syndromes (including myocardial infarction and unstable angina), coronary angioplasty, or stenting within 6 months prior to randomization; f) history or evidence of current \>= class II congestive heart failure as defined by New York Heart Association (NYHA); g) treatment refractory hypertension defined as a blood pressure of systolic \> 140 mmHg and/or diastolic \> 90 mm Hg which cannot be controlled by anti-hypertensive therapy; h) patients with intra-cardiac defibrillators * Any serious and/or unstable pre-existing medical disorder (aside from malignancy exception above), psychiatric disorder, or other conditions that could interfere with subject's safety, obtaining informed consent, or compliance to the study procedures * History of retinal vein occlusion (RVO) * Pregnant or breastfeeding, or planning to become pregnant within 6 months after the end of treatment * History of organ allograft or other history of immunodeficiency * Inability or unwillingness to comply with study and/or follow-up requirements * Known immediate or delayed hypersensitivity reaction or idiosyncrasy to drugs chemically related to study drug, or excipients or to dimethyl sulfoxide (DMSO) * Known human immunodeficiency virus (HIV), hepatitis B virus (HBV), or hepatitis C virus (HCV) infection; subjects with laboratory evidence of cleared HBV and HCV infection will be permitted * Current use of a prohibited medication