APN401 in Treating Patients With Recurrent or Metastatic Pancreatic Cancer, Colorectal Cancer, or Other Solid Tumors That Cannot Be Removed by Surgery

Inclusion Criteria: * Patients with histologically confirmed inoperable, recurrent or metastatic malignant solid tumors, deemed incurable, and who have either: * Failed to respond to standard therapy or * For whom no standard therapy is available or * Refuse to receive standard therapies * The study is intended to enroll patients with pancreatic and colorectal cancer; patients with other types of solid tumors will require approval by the principal investigator * Measurable disease as defined by Response Evaluation Criteria in Solid Tumors (RECIST) * Patients with treated, stable, and asymptomatic brain metastases are eligible * Patients on every 3 or every 4 week systemic therapy programs must be at least 4 weeks since treatment and recovered from any clinically significant toxicity experienced; patients on weekly or daily systemic therapy programs and patients receiving radiation must be at least 1 week since treatment and recovered from any clinically significant toxicity experienced; must be at least 4 weeks and have recovered from major surgery * Eastern Cooperative Oncology Group (ECOG) performance status 0 or 1 * White blood cells \>= 3000/uL * Platelets \>= 100,000/uL * Hematocrit \>= 28% * Creatinine =\< 1.6 mg/dL * Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) \< 2.5 x upper limit of normal * Bilirubin =\< 1.6 mg/dL (except patients with Gilbert's syndrome, who must have a total bilirubin less than 3.0 mg/dL) * Albumin \>= 3.0 g/dL * International normalized ratio (INR) =\< 1.5 Exclusion Criteria: * Women must not be pregnant or breastfeeding; all women of childbearing potential must have a blood test within 72 hours to rule out pregnancy; women of childbearing potential and sexually active males must be strongly advised to use an accepted and effective method of contraception; women of childbearing potential (WOCBP) must be using an adequate method of contraception to avoid pregnancy throughout the study and for 26 weeks after the last dose of investigational product, in such a manner that the risk of pregnancy is minimized; sexually mature females who have not undergone a hysterectomy or who have not been postmenopausal naturally for at least 24 consecutive months (i.e., who have had menses at some time in the preceding 24 consecutive months) are considered to be of childbearing potential; women who are using oral contraceptives, other hormonal contraceptives (vaginal products, skin patches, or implanted or injectable products), or mechanical products such as an intrauterine device or barrier methods (diaphragm, condoms, spermicides) to prevent pregnancy, or are practicing abstinence or where their partner is sterile (e.g., vasectomy) should be considered to be of childbearing potential * Untreated, progressing, or symptomatic brain metastases * Autoimmune disease, as follows: patients with a history of inflammatory bowel disease are excluded as are patients with a history of symptomatic disease (e.g., rheumatoid arthritis, systemic progressive sclerosis \[scleroderma\], systemic lupus erythematosus, autoimmune vasculitis \[e.g., Wegener's granulomatosis\]); patients with motor neuropathy considered of autoimmune origin (e.g., Guillain-Barre syndrome and myasthenia gravis) are excluded; patients with a history of autoimmune thyroiditis are eligible if their current thyroid disorder is treated and stable with replacement or other medical therapy * Any other malignancy from which the patient has been disease-free for less than 2 years, with the exception of adequately treated and cured basal or squamous cell skin cancer, superficial bladder cancer or carcinoma in situ of the cervix * Other ongoing systemic therapy for cancer, including any other experimental treatment; these include concomitant therapy with any of the following: IL-2, interferon, ipilimumab, pembrolizumab, nivolumab, or other immunotherapy; cytotoxic chemotherapy; and targeted therapies * Ongoing requirement for an immunosuppressive treatment, including the use of glucocorticoids or cyclosporine, or with a history of chronic use of any such medication within the last 4 weeks before enrollment; patients are excluded if they have any concurrent medical condition that requires the use of systemic steroids (the use of inhaled or topical steroids is permitted) * Infection with human immunodeficiency virus (HIV) * Active infection with hepatitis B; active or chronic infection with hepatitis C * Clinically significant pulmonary dysfunction, as determined by medical history and physical examination; patients with a history of pulmonary dysfunction must have pulmonary function tests with a forced expiratory volume in 1 second (FEV1) \>= 60% of predicted and a diffusing capacity of the lung for carbon monoxide (DLCO) \>= 55% (corrected for hemoglobin) * Clinically significant cardiovascular abnormalities (e.g., congestive heart failure or symptoms of coronary artery disease), as determined by medical history and physical examination; patients with a history of cardiac disease must have a normal cardiac stress test (treadmill, echocardiogram, or myocardial perfusion scan) within the past 6 months of study entry * Active infections or oral temperature \> 38.2 degrees Celsius (C) within 48 hours of study entry * Systemic infection requiring chronic maintenance or suppressive therapy * Patients are excluded for any underlying medical or psychiatric condition, which in the opinion of the investigator, will make treatment hazardous or obscure the interpretation of adverse events, such as a condition associated with frequent rashes or diarrhea