Testing the Addition of an Anti-Cancer Drug, Irinotecan, to the Standard Chemotherapy Treatment (FOLFOX) After Long-Course Radiation Therapy for Advanced-Stage Rectal Cancers to Improve the Rate of Complete Response and Long-Term Rates of Organ Preservation and Continuing Response

Inclusion Criteria: * Histologic Documentation: rectal adenocarcinoma, mismatch repair proficient (pMMR) * Stage: Clinical stage II or III rectal adenocarcinoma defined as T4N0 or any T with node positive disease (any T, N+); also T3N0 requiring abdominal perineal resection (APR) or coloanal anastomosis * Tumor site: Rectum; distal edge of the tumor =\< 12cm from the anal verge (as determined by surgeon's endoscopic assessment; MRI can be used to compliment this information, but endoscopy should be the primary means of assessment) * No prior systemic chemotherapy, targeted therapy, or immunotherapy; or radiation therapy administered as treatment for colorectal cancer within the past 5 years is allowed. No local approaches to excising the rectal cancer (even if done for diagnostic purposes) are allowed (e.g., transanal excision \[open or minimally invasive\], local excision, endoscopic submucosal dissection or endoscopic submucosal resection). * Not pregnant and not nursing, because this study involves an agent that has known genotoxic, mutagenic and teratogenic effects \* Therefore, for women of childbearing potential only, a negative pregnancy test (urine or serum according to institutional guidelines) done =\< 14 days prior to registration is required. Female subjects agree to use highly effective contraception combined with an additional barrier method (e.g, diaphragm, with a spermicide) while on study and for \>= 9 months after last dose of study drug, and the same criteria are applicable to male subjects if they have a partner of childbirth potential. Male subject agrees to use a condom and not donate sperm while in this study and for \>= 6 months after the last treatment * Age \>= 18 years * Eastern Cooperative Oncology Group (ECOG) performance status 0-1 (or Karnofsky \>= 60%) * Absolute neutrophil count (ANC) \>= 1,500/mm\^3 * Platelet count \>= 100,000/mm * Creatinine =\< 1.5 x upper limit of normal (ULN) OR calculated (calc.) creatinine clearance \>= 50 mL/min \^3 * Total bilirubin =\< 1.5 x upper limit of normal (ULN) * Aspartate aminotransferase (AST)/alanine aminotransferase (ALT) =\< 3 x upper limit of normal (ULN) * No upper rectal tumors (i.e., distal portion of tumor must be =\> 12 cm from the anal verge) * No recurrent rectal cancer; prior transanal excision, prior distal sigmoid cancer with a low anastomosis or prior endoscopic submucosal dissection * No known mismatch repair deficient rectal adenocarcinoma * HIV-infected patients on effective anti-retro viral therapy with undetectable viral load within 6 months are eligible for this trial * Patients with known history or current symptoms of cardiac disease, or history of treatment with cardio toxic agents, should have a clinical risk assessment of cardiac function using the New York Heart Association Functional Classification1. To be eligible for this trial, patients should be class 2B or better * Testing for dihydropyrimidine dehydrogenase (DPD) deficiency is not required. However, when available, patients with complete lack of DPD should not be treated with fluoropyrimidines (such patients must not be enrolled or if initiated on therapy and noted to have fluoropyrimidine related toxicities be taken off protocol; dose reductions for patients with partial deficiency may be done per local guidelines * Chronic concomitant treatment with strong inhibitors of CYP3A4 is not allowed on this study. Patients on strong CYP3A4 inhibitors must discontinue the drug for 14 days prior to registration on the study * Chronic concomitant treatment with strong CYP3A4 inducers is not allowed. Patients must discontinue the drug 14 days prior to the start of study treatment * Once systemic chemotherapy has been completed and the patient is either in surveillance or being considered for surgery then medically necessary CYP3A4 medications can be resume