Trifluridine/Tipiracil and Talazoparib for the Treatment of Patients With Locally Advanced or Metastatic Colorectal or Gastroesophageal Cancer

Inclusion Criteria: * Histologically or cytologically confirmed CRC or EGC adenocarcinoma that is locally advanced or metastatic (Cohort A); histologically or cytologically confirmed p53mt/RASonc (Cohort B1) or p53mt/RASwt CRC (Cohort B2) that is locally advanced or metastatic. Patients with adenocarcinoma histology only are allowed to participate. * Has received at least one prior line of therapy with progression or intolerance * Have measurable disease per Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria present * Have an Eastern Cooperative Oncology Group (ECOG) performance status of 0-1 * Life expectancy \>= 3 months by investigator assessment * Hemoglobin \>= 9 g/dL * Absolute neutrophil count \>= 1500/mm\^3 * Platelet count \>= 100,000/mm\^3 without transfusion or growth factor support * Creatinine \< 1.5 upper limit of normal (ULN) or creatinine clearance \> 60 mL/min * Total bilirubin \< 1.5 x ULN * Aspartate aminotransferase (AST)/alanine aminotransferase (ALT) =\< 2.5 x ULN or \< x 5 ULN in the presence of liver metastasis * Albumin \> 3 g/dL * Ability to swallow oral medications * Participants of child-bearing potential must agree to use adequate contraceptive methods (e.g., hormonal or barrier method of birth control; abstinence) prior to study entry. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately * Participant must understand the investigational nature of this study and sign an Independent Ethics Committee/Institutional Review Board approved written informed consent form prior to receiving any study related procedure Exclusion Criteria: * Systemic antineoplastic therapy within 2 weeks prior to day -14 (Dose Escalation, Cohort A) or Cycle 1 day 1 (Dose Expansion, Cohorts B1 and B2) or within the past 6 weeks if this treatment is mitomycin C or nitrosourea * Radiotherapy within the past 2 weeks excluding palliative radiotherapy to painful bone lesions * Prior treatment with PARP inhibitor, FUDR or FTD/TPI * Any condition that in the investigator's opinion can limit absorption of FTD/TPI or talazoparib from the gastrointestinal (GI) tract * Gastrointestinal obstruction (without diversion) or perforation within 4 weeks from initiation of day -14 (Dose Escalation, Cohort A) or Cycle 1 Day 1 (Dose Expansion, Cohorts B1 and B2. * Refractory ascites (requiring weekly or more frequent paracentesis or permanent indwelling peritoneal catheter) * Untreated central nervous system (CNS) disease. Patients with leptomeningeal disease are ineligible but patients with treated, stable CNS metastasis for at least 4 weeks are allowed to participate * Significant cardiac disease defined as congestive heart failure stage III or IV (New York Heart Association \[NYHA\]), acute coronary event, cerebrovascular event, peripheral arterial embolic event, venous thromboembolic event (pulmonary embolism or lower extremity deep vein thrombosis), or ventricular arrhythmia within the past 3 months * Other malignancy requiring active therapy * Presence of toxicities from prior therapy of grade 2 or higher * Active infection requiring antibiotic therapy * Known human immunodeficiency virus (HIV) or hepatitis B infection or untreated hepatitis C infection. Patients with treated hepatitis C infection and undetectable viral load are allowed to participate * Any history of myelodysplastic syndrome, acute leukemia, or bone marrow transplant * Pregnant or nursing female participants * Unwilling or unable to follow protocol requirements * Any condition which in the investigator's opinion deems the participant an unsuitable candidate to receive study drug