Vudalimab (XmAb20717) in Combination With Standard of Care Treatment in Patients With Metastatic Castration Sensitive Prostate Cancer

Inclusion Criteria: * Age \>= 18 years * Histologically confirmed adenocarcinoma of the prostate with metastatic disease * Castration-sensitive status: either not have been treated with androgen deprivation therapy (ADT) (hormone therapy) or not on ADT at the time of progression * Participants can have received up to 3 months of ADT with luteinizing hormone-releasing hormone (LHRH) agonists or antagonists or orchiectomy with or without concurrent first-generation antiandrogens prior to enrollment, with no radiographic evidence of disease progression or rising prostate-specific antigen (PSA) prior to enrollment * Eastern Cooperative Oncology Group (ECOG) performance status =\< 2 (Karnofsky \>= 60%) * Life expectancy \> 12 weeks as determined by the investigator * Hemoglobin \>= 9.0 g/dl (within 28 days of cycle 1 day 1) (no transfusions allowed within 7 days of Cycle 1 Day 1 to meet entry criteria) * White blood cell (WBC) \>= 2000/uL (within 28 days of cycle 1 day 1) (after at least 7 days without growth factor support or transfusion) * Absolute neutrophil count (ANC) \>= 1,500/mcL (within 28 days of cycle 1 day 1) (after at least 7 days without growth factor support or transfusion) * Platelets \>= 100,000/mcL (within 28 days of cycle 1 day 1) (no transfusions allowed within 7 days of cycle 1 day 1 to meet entry criteria) * Prothrombin time (PT)/ partial thromboplastin time (PTT) =\< 1.5 x upper limit of normal (ULN) (within 28 days of cycle 1 day 1) * Total bilirubin =\< 1.5 institutional upper limit of normal (ULN) (within 28 days of cycle 1 day 1) * Aspartate aminotransferase (AST) / alanine aminotransferase (ALT) =\< 3 institutional upper limit of normal (ULN) (within 28 days of cycle 1 day 1) * Serum creatinine =\< 2 mg/dL (or glomerular filtration rate \>= 40 mL/min) (within 28 days of cycle 1 day 1) * Willingness to provide pre- and post-treatment fresh tumor biopsies, if safe and medically feasible * Male subjects must be surgically sterile or must agree to use adequate method of contraception from the time of consent until at least 120 days after the last dose of Xmab27017 * Willingness and ability of the subject to comply with scheduled visits, drug administration plan, protocol specified laboratory tests, other study procedures, and study restrictions * Completion of all previous surgery, radiotherapy, chemotherapy, immunotherapy, or investigational therapy for the treatment of cancer \>= 2 weeks before the start of study therapy. (No radiotherapy to Xmab27017 injection site within 4 weeks) * Evidence of a personally signed informed consent indicating that the subject is aware of the neoplastic nature of the disease and has been informed of the procedures to be followed, the experimental nature of the therapy, alternatives, potential risks and discomforts, potential benefits, and other pertinent aspects of study participation Exclusion Criteria: * Patients who have had chemotherapy or radiotherapy within 4 weeks prior to entering the study or those who have not recovered from adverse events due to agents administered more than 4 weeks earlier (i.e., have residual toxicities \> grade 1) * Patients who are receiving any other investigational agents or an investigational device within 21 days before administration of first dose of study drugs * Prior treatment with any CTLA4, PD1, or PDL1, or directed immunotherapy * History of allergic reactions attributed to compounds of similar chemical or biologic composition to (investigational new drug \[IND\] agent\[s\]) or other agents used in study * Have known active central nervous system metastases and/or carcinomatous meningitis. Subjects with previously treated brain metastases may participate provided they are radiologically stable, ie, are without evidence of progression for at least 4 weeks by repeat imaging (note that the repeat imaging should be performed during study screening), are clinically stable, and are without requirement of steroid treatment for at least 14 days prior to first dose of study treatment * Active known or suspected autoimmune disease (except that subjects are permitted to enroll if they have vitiligo; type 1 diabetes mellitus or residual hypothyroidism due to an autoimmune condition that is treatable with hormone replacement therapy only; psoriasis, atopic dermatitis, or another autoimmune skin condition that is managed without systemic therapy; or arthritis that is managed without systemic therapy beyond oral acetaminophen and nonsteroidal anti-inflammatory drugs) * Has any condition requiring systemic treatment with corticosteroids, prednisone equivalents, or other immunosuppressive medications within 14 days prior to first dose of study drug (except that inhaled or topical corticosteroids or brief courses of corticosteroids given for prophylaxis of contrast dye allergic response are permitted.) * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * Receipt of an organ allograft * Known history of left ventricular ejection fraction =\< 40% * Receipt of a live-virus vaccine within 30 days prior to first dose of study drug (seasonal flu vaccines that do not contain live virus are permitted. COVID-19 vaccines are permitted) * Known human immunodeficiency virus (HIV) positive subject with CD4+ T-cell (CD4+) counts \< 350 cells/uL, or an HIV viral load greater than 400 copies/mL, or a history of an AIDS (acquired immunodeficiency syndrome)-defining opportunistic infection within the past 12 months, or who has not been on established antiretroviral therapy (ART) for at least 4 weeks prior to initiation of study drug dosing. (Effective ART is defined as a drug, dosage, and schedule associated with reduction and control of the viral load.) * Known positive test for hepatitis C ribonucleic acid (RNA) (a subject who is hepatitis C virus \[HCV\] antibody positive but HCV RNA negative due to documented, curative prior antiviral treatment or natural resolution is eligible). * Known positive test for hepatitis B surface antigen (HBsAg) or hepatitis B core antibody (HBcAb; a subject whose HBsAg is negative and HBcAb is positive may be enrolled if a hepatitis B virus \[HBV\] deoxyribonucleic acid (DNA) test is negative, and the subject is retested for HBsAg and HBV DNA every 2 months)