Liposomal Irinotecan, Fluorouracil and Leucovorin in Treating Patients With Refractory Advanced High Grade Neuroendocrine Cancer of Gastrointestinal, Unknown, or Pancreatic Origin

Inclusion Criteria: * Participant must have a locally advanced and unresectable or metastatic gastroenteropancreatic neuroendocrine carcinoma of the gastrointestinal (GI) tract, pancreas, or of other known or unknown primary site where 5FU based therapy would be considered reasonable by the treated MD, lung primary is excluded. Patients may have either progressed on therapy or within 6 months of completing therapy, or be intolerant of therapy to be considered eligible. * Participant must have tissue available for central pathology review and, must have pathologically/histologically confirmed high grade neuro endocrine defined as Ki-67 proliferative index of 20-100% or, must have evidence of at least 10 mitotic figures per 10 high powered fields. * Comprehensive Genomic Profiling will be performed on archival tissue available prior to enrollment. If no archival tissue is available, then patient must have fresh biopsy prior to treatment administration if clinically indicated. * Have an Eastern Cooperative Oncology Group (ECOG) Performance Status of 0 - 2. * Leukocytes \>= 3,000/mm\^3 . * Absolute neutrophil count \>= 1,500/mm\^3. * Hemoglobin \>= 9 g/dL. * Platelets \>= 100,000/mm\^3. * Total bilirubin =\< institutional upper limit of normal (ULN) or =\< 1.5 x institutional ULN (if the patient has liver metastases. * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional ULN or (=\< 5 x institutional ULN if the patient has liver metastases). * Serum creatinine =\< 1.5 x institutional ULN or measured or calculated creatinine clearance by Cockcroft Gault Equation \>= 50ml/min for subjects with creatinine levels \> 1.5 x ULN. * Have measurable disease per Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria present. * Participant must have a life expectancy of \>= 12 weeks as determined clinically by the treating physician. * Participants of child-bearing potential must agree to use adequate contraceptive methods (e.g., hormonal or barrier method of birth control; abstinence) prior to study entry. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. * A female of childbearing potential is any woman, regardless of sexual orientation or whether they have undergone tubal ligation, who meets the following criteria: 1) has not undergone a hysterectomy or bilateral oophorectomy; or 2) has not been naturally postmenopausal for at least 24 consecutive months (i.e., has had menses at any time in the preceding 24 consecutive months). * Women of childbearing potential and sexually active males must be strongly advised to use an accepted and effective method of contraception or to abstain from sexual intercourse for the duration of their participation in the study. * Participant must understand the investigational nature of this study and sign an Independent Ethics Committee/Institutional Review Board approved written informed consent form prior to receiving any study related procedure. Exclusion Criteria: * Known active central nervous system (CNS) metastases and/or carcinomatous meningitis. Subjects with previously treated brain metastases may participate provided they are stable (without evidence of progression by imaging for at least four weeks prior to the first dose of trial treatment and any neurologic symptoms have returned to baseline), have no evidence of new or enlarging brain metastases, and are not using steroids for at least 7 days prior to trial treatment. This exception does not include carcinomatous meningitis which is excluded regardless of clinical stability. * Participants with known dihydropyrimidine dehydrogenase (DPD) deficiency. * Known history of human immunodeficiency virus (HIV) (HIV 1/2 antibodies). * Known hypersensitivity to any of the components of Nal-IRI, other liposomal products, fluoropyrimidines or leucovorin. * Investigational therapy administered within 4 weeks, or within a time interval less than at least 5 half-lives of the investigational agent, whichever is longer, prior to the first scheduled day of dosing in this study. * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements. * Pregnant or nursing female participants. * Unwilling or unable to follow protocol requirements. * Any condition which in the Investigator?s opinion deems the participant an unsuitable candidate to receive study drug.