Vidutolimod (CMP-001) in Combination With Nivolumab for the Treatment of Metastatic Castration Resistant Prostate Cancer

Inclusion Criteria: * Male \>= 18 years of age * Histologically confirmed adenocarcinoma of the prostate with metastatic disease * Subjects who are refractory to a novel antiandrogen therapy (abiraterone, darolutamide, enzalutamide and/or apalutamide) and have failed at least 1 taxane based chemotherapy regimen (or who are deemed medically unsuitable to be treated with a taxane regimen or have actively refused treatment with a taxane regimen) * Prior orchiectomy or serum testosterone levels \< 50 ng/dL determined within 4 weeks prior to start of study drug * Having measurable disease per RECIST 1.1 (at least one additional lesion \>= 0.5 cm amenable to repeated IT injection per investigator) * Eastern Cooperative Oncology Group performance status of 0 or 1 * Neutrophil count \>= 1,000/mm\^3 (within 4 weeks prior to the first dose of CMP-001) * Platelet count \>= 100,000/mm\^3 (within 4 weeks prior to the first dose of CMP-001) * Hemoglobin concentration \>= 9 g/dL (within 4 weeks prior to the first dose of CMP-001) * Total bilirubin =\< 1.5 times the upper limit of normal (ULN) with the following exception: subjects with Gilbert Disease serum bilirubin \>= 3 times ULN (within 4 weeks prior to the first dose of CMP-001) * Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) =\< 3 times the ULN or =\< 5 times the ULN for patients with active liver metastases (within 4 weeks prior to the first dose of CMP-001) * Serum creatinine =\< 1.5 times the ULN or calculated creatinine clearance \>= 40 mL/min (\>= 0.67 mL/sec) using the Cockcroft-Gault equation (within 4 weeks prior to the first dose of CMP-001) * Subjects should have an international normalized ratio (INR) or a partial thromboplastin time (PTT) =\< 1.5 x ULN unless the subject is receiving anticoagulant therapy (within 4 weeks prior to the first dose of CMP-001). Subjects on anticoagulant therapy should have a prothrombin time (PT) or PTT within therapeutic range of intended use and no history of severe hemorrhage. Patients who require therapeutic anticoagulation and cannot discontinue anticoagulation safely during the day prior, day of, and day after injection are excluded. Patients requiring anticoagulation with warfarin are excluded unless they can be transitioned to an alternative anticoagulant (e.g., low molecular weight heparin or direct oral anticoagulants) prior to enrollment. Antiplatelet agents (e.g., aspirin, clopidogrel, etc.) are not considered anticoagulants for the purposes of this study (i.e., they are allowed) * Willingness to provide pre- and post-treatment fresh tumor biopsies, if safe and medically feasible * Male subjects must be surgically sterile or must agree to use adequate method of contraception from the time of consent until at least 120 days after the last dose of CMP-001 * Willingness and ability of the subject to comply with scheduled visits, drug administration plan, protocol-specified laboratory tests, other study procedures, and study restrictions * Completion of all previous therapy (including surgery, radiotherapy, chemotherapy, immunotherapy, or investigational therapy) for the treatment of cancer \>= 2 weeks before the start of study therapy. (No radiotherapy to CMP-001 injection site within 4 weeks) * Evidence of a personally signed informed consent indicating that the subject is aware of the neoplastic nature of the disease and has been informed of the procedures to be followed, the experimental nature of the therapy, alternatives, potential risks and discomforts, potential benefits, and other pertinent aspects of study participation Exclusion Criteria: * Pathological finding consistent with pure small cell, neuroendocrine carcinoma of prostate or any other histology different from adenocarcinoma * Requires systemic pharmacologic doses of corticosteroids \> 10 mg/day prednisone within 7 days prior to the first dose of CMP-001 on C1D1 * Subjects who are currently receiving steroids at a prednisone-equivalent dose of =\< 10 mg/day do not need to discontinue steroids prior to enrollment * Replacement doses, topical, ophthalmologic and inhalational steroids are permitted * History of immune-related adverse event (AE) that required permanent discontinuation of anti-PD1/PDL1 antibody * History of (non-infectious) pneumonitis that required steroids or current pneumonitis * Patients with active autoimmune disease * Known history of immunodeficiency * Known additional malignancy that is progressing or requires active treatment within the past 3 years. Exceptions include basal cell carcinoma of the skin, squamous cell carcinoma of the skin that has undergone potentially curative therapy, ductal carcinoma in situ, non-invasive bladder cancer and thyroid cancer (except anaplastic) * Untreated, symptomatic, or growing central nervous system (CNS) metastases * Patients with treated and stable (defined as non-progression on a restaging contrast enhanced magnetic resonance imaging (MRI) or computed tomography (CT) at least 30 days after CNS directed therapy) CNS disease are eligible to enroll * Patients with treated and stable CNS disease enrolled on study must be willing to undergo restaging contrast-enhanced CT or MRI every 12 weeks * Prior allogenic tissue/solid organ transplant * Known infection with human immunodeficiency virus, hepatitis B virus or hepatitis C virus; testing is not required unless suspected * Received a live virus vaccination within 30 days prior to the first dose of CMP-001 on D1. Seasonal flu vaccines that do not contain live virus or COVID vaccines that administered more than 1 week prior to first dose of CMP-001 on D1 the are permitted * Active infection requiring systemic therapy * Severe uncontrolled cardiac disease within 6 months prior to consent, including but not limited to uncontrolled hypertension; unstable angina; myocardial infarction or cerebrovascular accident. Implanted or continuous use of a pacemaker or defibrillator * Any concurrent uncontrolled illness, including mental illness or substance abuse, which in the opinion of the Investigator, would make the subject unable to cooperate or participate in the trial * Adverse event related to previously administered anti-cancer therapy that has not resolved to \< grade 2 prior to the first dose of CMP-001 on day 1 (D1) * Participation in another clinical study of an investigational anti-cancer therapy or device within 28 days prior to the first dose of CMP-001 on D1 * Received chemotherapy, radiation therapy, or biological anti-cancer therapy within 14 days prior to the first dose of CMP-001 on W1D1 * Received previous CMP-001 treatment or anti-PD1/PDL1 * Expecting to conceive or father children within the projected duration of the study, from the time of consent until at least 120 days after the last dose of CMP-001