A Phase I/II Study of Combination Immunotherapy for Advanced Cancers Including HPV-Associated Malignancies, Small Bowel, and Colon Cancers

* INCLUSION CRITERIA: * Phase I: Subjects with cytologically or histologically confirmed locally advanced or metastatic HPV associated malignancies or MSS small bowel or colorectal cancer (Cohort 1). * Phase II: Subjects with cytologically or histologically confirmed locally advanced or metastatic HPV associated malignancies (Cohort 2), or MSS small bowel or colorectal cancer (Cohort 3). * Cohort 2 includes: * Cervical cancers; * P16+ Oropharyngeal cancers; * Anal cancers; * Vulvar, vaginal, penile, and squamous cell rectal cancers; * Other locally advanced or metastatic solid tumors (e.g., lung, esophagus) that are known HPV+. * Subjects with MSS colorectal or small bowel cancer must have received two prior lines of systemic chemotherapy. Subjects with HPV associated malignancies must have received one prior line of systemic chemotherapy as well as checkpoint therapy if checkpoint therapy is FDA approved for that specific tumor type (e.g., HNSCC and PDL1+ cervical cancer). Prior checkpoint therapy is not needed where checkpoint therapy has not been FDA approved for that specific tumor type (e.g., anal, vaginal, vulvar, penile, PDL1 negative cervical). Exceptions to the above include participant who are not eligible to receive the above therapies or who decline these standard treatment options after appropriate counseling has been provided. * Subjects must have measurable disease, per RECIST 1.1. * Age \>=18 years. * ECOG performance status \<=2 * Adequate hematologic function at screening, as follows: * Absolute neutrophil count (ANC) \>=1.5 x 10\^9/L; * Hemoglobin \>= 9 g/dL; * Platelets \>= 100,000/microliter. * Adequate renal and hepatic function at screening, as follows: * Measured or calculated creatinine clearance (using the Cockcroft-Gault equation) \> 50 mL/min * Bilirubin \<= 1.5 x ULN OR in subjects with Gilbert s syndrome, a total bilirubin \<= 3.0 x ULN * Alanine aminotransferase (ALT) and aspartate aminotransferase (AST) \<= 2.5 x ULN, unless liver metastases are present, then values must be \<= 3 x ULN). * The effects of the immunotherapies on the developing human fetus are unknown. For this reason and because immunotherapeutic agents as well as other therapeutic agents used in this trial are known to be teratogenic, individuals of child-bearing potential and those who can father children must agree to use highly effective contraception (hormonal or barrier method of birth control; abstinence) at the study entry and for 4 months after the last bintrafusp alfa dose, 3 months after the last entinostat dose and 2 months after the last PDS01ADC dose, whichever occurs later. * Participants serologically positive for HIV, Hep B, Hep C are eligible as long as the viral loads are undetectable by quantitative PCR. HIV positive participants must have CD4 count \>= 200 cells per cubic millimeter at enrollment, be on stable antiretroviral therapy for at least 4 weeks and have no reported opportunistic infections or Castleman's disease within 12 months prior to enrollment. * Subjects must be able to understand and be willing to sign a written informed consent document. EXCLUSION CRITERIA: * Participants with prior investigational drug, chemotherapy, immunotherapy or any prior radiotherapy (except for palliative bone directed therapy) within the past 28 days prior to enrollment except if the investigator has assessed that all residual treatment-related toxicities have resolved or are at grade 1 severity and feel the participant is otherwise suitable for enrollment. * Administration of live vaccine within 30 days prior to enrollment * Major surgery within 28 days prior to enrollment (minimally invasive procedures such as diagnostic biopsies are permitted). * Known active brain or central nervous system metastasis (less than a month out from definitive radiotherapy or surgery), seizures requiring anticonvulsant treatment (\<3 months) or clinically significant cerebrovascular accident (\<3 months). In order to be eligible participants must have repeat CNS imaging at least a month after definitive treatment showing CNS disease that has not progressed. Participants with CNS disease that has not progressed at least a month after definitive treatment and continued on \<=10 mg of prednisone (or equivalent) for treatment of brain or central nervous system metastasis are eligible to enroll in this study. Participants with evidence of intratumoral or peritumoral hemorrhage on baseline imaging are also excluded unless the hemorrhage is grade \<= 1 and has been shown to be stable on two consecutive imaging scans. * Active autoimmune disease that might deteriorate when receiving an immunostimulatory agent with exception of: * Diabetes type I, eczema, vitiligo, alopecia, psoriasis, hypo- or hyperthyroid disease or other mild autoimmune disorders not requiring immunosuppressive treatment; * Administration of steroids for other conditions through a route known to result in a minimal systemic exposure (topical, intranasal, intro-ocular, or inhalation) is acceptable; * Subjects on systemic intravenous or oral corticosteroid therapy with the exception of hormone replacement or physiologic doses of corticosteroids (\<= the equivalent of prednisone 10 mg/day) or other immunosuppressive drugs such as azathioprine or cyclosporin A are excluded on the basis of potential immune suppression. For these subjects these excluded treatments must be discontinued at least 1 weeks prior to enrollment for recent short course use (\<= 14 days) or discontinued at least 4 weeks prior to enrollment for long term use (\> 14 days). In addition, the use of corticosteroids as premedication for contrast-enhanced studies is allowed prior to enrollment and on study. * Subjects with a history of serious intercurrent chronic or acute illness, such as cardiac or pulmonary disease, hepatic disease, bleeding diathesis or recent (within 3 months before enrollment) clinically significant bleeding events, or other illness or medical condition considered by the Investigator as high risk for investigational drug treatment. Note: For participants otherwise eligible to be assigned to receive just PDS01ADC and entinostat (without bintrafusp alfa, Arm 3), bleeding diathesis or recent bleeding will not be an exclusion. * Subjects unwilling to accept blood products as medically indicated. * Subjects with conditions associated with significant necrosis of nontumor-bearing tissues: esophageal or gastroduodenal ulcers \< 3 months prior to enrollment, organ infarction \< 3 months prior to enrollment, or active ischemic bowel disease. * Presence of medically significant third space fluid (symptomatic pericardial effusion, ascites or pleural effusion requiring repetitive paracentesis). * History of second malignancy within 3 years of enrollment except for the following: adequately treated localized skin cancer, cervical carcinoma in situ, superficial bladder cancer, or other localized malignancy which has been adequately treated or malignancy which does not require active systemic treatment (e.g. low risk chronic lymphocytic leukemia (CLL)). * Subjects with a known severe hypersensitivity reaction to compounds of similar chemical or biologic composition to any of study drugs (grade \>/= 3 NCI-CTCAE v5) will be evaluated by the allergy/immunology team prior to enrollment. * Pregnant individuals are excluded from this study because these drugs have not been tested in pregnant individuals and there is potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with these immunotherapies, nursing should be discontinued if the mother is treated on this protocol.