Enfortumab Vedotin and Pembrolizumab Combined With Radiotherapy in Muscle Invasive Bladder Cancer

Inclusion Criteria: * Biopsy-confirmed muscle-invasive bladder cancer (cT2,T3,T4a). (Note: Tissue samples are required.) (Participants with cT3/T4a staged disease will be capped at 25% of patients treated at RP2D). * Urothelial-predominant histology. Mixed histologies other than small cell/neuroendocrine are allowed as long as some urothelial histology is present. Neuroendocrine histology of any component and pure variant (non-urothelial) histology tumors will be excluded. (Patients with \< 50% urothelial histology will be capped at 25% of patients treated at RP2D). * Must be judged by the investigator to be ineligible for radical cystectomy or electing not to undergo radical cystectomy. * Must be eligible for and agree to receive bladder irradiation as determined by the treating investigator. * Must have a TURBT within 8 weeks of combination treatment start with viable tumor content. If no viable tumor content is present on TURBT, the patient will be replaced in the study. * Patients who have autoimmune disease will be evaluated on a case-by-case basis and can only enroll so long as participants are not on active immunosuppression with a corticoid steroid allowance exceeding 10mg of prednisone or equivalent per day. * Age \>= 18 years. * Eastern Cooperative Oncology Group (ECOG) performance status \<= 1. * Absolute neutrophil count ≥ 1,500/microliter (mcL). * Platelets \>= 100,000/mcL. * Hemoglobin \>= 9.0 g/dL or ≥ 5.6 mmol/L. \* Criteria must be met without erythropoietin dependency and without packed red blood cell (pRBC) transfusion within last 2 weeks * Total bilirubin \<= 1.5 × upper limit of normal, unless elevated due to Gilbert's syndrome and direct bilirubin is within normal limits. * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase (SGOT)) \<= 2.5 X institutional upper limit of normal. * Alanine aminotransferase (ALT) (serum glutamic pyruvic transaminase (SGPT)) \<= 2.5 X institutional upper limit of normal. * Creatinine clearance glomerular filtration rate (GFR) \>= 30 mL/min/1.73 m\^2, calculated by Cockcroft-Gault or measured using 24-hour creatinine clearance. * International normalized ratio (INR) OR prothrombin time (PT) \<= 1.5 × upper limit of normal (ULN). \* If participant is receiving anticoagulant therapy, as long as PT or activated partial thromboplastin time (aPTT) is within therapeutic range of intended use of anticoagulants, participant is eligible. * Activated partial thromboplastin time (aPTT) \<= 1.5 × ULN. \* If participant is receiving anticoagulant therapy, as long as PT or aPTT is within therapeutic range of intended use of anticoagulants, participant is eligible. * Ability to understand and the willingness to sign a written informed consent document. * Human immunodeficiency virus (HIV)-infected individuals on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial. * Participants who are hepatitis B virus surface antigen (HBsAg) positive are eligible if they have received hepatitis B virus (HBV) anti-viral therapy for at least 4-weeks, and have undetectable HBV viral load prior to randomization. Participants should remain on anti-viral therapy throughout study intervention and follow local guidelines for HBV anti-viral therapy post completion of study intervention. \* Note: Hepatitis B screening tests are not required unless patients have a known history of HBV infection. * Participants with a history of hepatitis C virus (HCV) infection are eligible if HCV viral load is undetectable at screening. \* Note: Participants must have completed curative anti-viral therapy at least 4 weeks prior to cycle 1 day 1. * Women of child-bearing potential and men with sexual partners of childbearing potential must agree to use adequate contraception for the duration of study participation. Enfortumab vedotin (EV) may cause fetal harm. Women of child-bearing potential must use contraception during treatment with EV and for 120 days after the last dose. Men with female partners who are women of child-bearing potential must use contraception during treatment with EV and for 120 days after the last dose. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Acceptable methods include barrier method, hormonal method, as well as intrauterine devices * Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 8 weeks after last administration of study treatment. Exclusion Criteria: * Presence of distant metastases on imaging (M1 disease). * Presence of lymph nodes concerning for regional disease spread (lymph node (LN) \> 1.0 cm on short axis present on cross sectional imaging) that is attributable to cancer spread (≥ N1 disease). * Presence of small cell / neuroendocrine histology in tumor sample (any content). * Absence of urothelial histology in TURBT tumor sample (pure variant histology). * Presence of untreated upper tract urothelial cancer. * Presence of moderate/severe hydronephrosis. * Presence of extensive carcinoma in situ (CIS). * Baseline neuropathy grade 2 (G2) or greater. * Baseline uncontrolled diabetes mellitus.Uncontrolled diabetes is defined as hemoglobin A1c (HbA1c) ≥ 8% or HbA1c 7 to \< 8% with associated diabetes symptoms (polyuria or polydipsia) that are not otherwise explained. \* Note: Patients with prior diagnosis but with disease under control are eligible * Prior treatment with systemic immunotherapy or chemotherapy for urothelial cancer.\* Note: Prior bacillus calmette-guerin (BCG) and intravesical treatments are allowed * Has received prior therapy with an anti-PD-1, anti-PD-L1, or anti PD L2 agent or with an agent directed to another stimulatory or co-inhibitory T-cell receptor (e.g., cytotoxic T-lymphocyte-associated protein 4 (CTLA-4), OX40, CD137). * Has received prior systemic anti-cancer therapy including investigational agents within 4 weeks prior to cycle 1 day 1. * Has received an investigational agent or has used an investigational device within 4 weeks prior to study intervention administration. * Has received prior radiotherapy within 2 weeks of cycle 1 day 1 or radiation-related toxicities requiring corticosteroids. \* Note: Two weeks or fewer of palliative radiotherapy for non-central nervous system (CNS) disease is permitted. The last radiotherapy treatment must have been performed at least 7 days before the first dose of study intervention. * Has received a live vaccine or live-attenuated vaccine within 30 days before the first dose of study intervention. \* Note: Administration of killed vaccines is allowed. Any licensed coronavirus 2019 (COVID-19) vaccine (including for emergency use) is allowed in the study as long as they are messenger ribonucleic acid (mRNA) vaccines, replication-incompetent adenoviral vaccines, or inactivated vaccines. * Major surgery within 2 weeks prior to first dose of EV. \* Note: Cataract surgery, standard tissue biopsies, and standard of care cardiac devices, such as a pacemaker or stent placed on an elective basis, are allowable procedures. * Has a diagnosis of immunodeficiency or is receiving chronic systemic steroid therapy (in dosing exceeding 10 mg daily of prednisone equivalent) or any other form of immunosuppressive therapy within 7 days prior to the first dose of study drug. \* Note: Inhaled or topical steroids are permitted in the absence of active autoimmune disease. * HIV-infected participants with a history of Kaposi's sarcoma and/or Multicentric Castleman's Disease. * Known additional malignancy that is progressing or has required active treatment within the past 3 years. \* Note: Participants with basal cell carcinoma of the skin, squamous cell carcinoma of the skin, or carcinoma in situ, excluding carcinoma in situ of the bladder, that have undergone potentially curative therapy are not excluded. Participants with low-risk early-stage prostate cancer (T1-T2a, Gleason score ≤ 6, and prostate specific antigen (PSA) \< 10 ng/mL) either treated with definitive intent or untreated in active surveillance with stable disease are not excluded. * Has known active CNS metastases and/or carcinomatous meningitis. \* Note: Participants with previously treated brain metastases may participate provided they are radiologically stable, i.e. without evidence of progression for at least 4 weeks by repeat imaging (note that the repeat imaging should be performed during study screening), clinically stable and without requirement of steroid treatment for at least 14 days prior to first dose of study intervention. * History of another significant life-limiting malignancy within 2 years prior to the first dose of study drug, or any evidence of residual disease from a previously diagnosed malignancy. \* Note: Patients with nonmelanoma skin cancer, curatively treated localized prostate cancer, or carcinoma in situ of any type (if complete resection was performed) are allowed. * Hypersensitivity to pembrolizumab or enfortumab vedotin, or any of their excipients. * Prior allogeneic stem cell or solid organ transplant. * Has a history of (non-infectious) pneumonitis/interstitial lung disease that required steroids or has current pneumonitis/interstitial lung disease. * Has an active infection requiring systemic therapy. * Has a history or current evidence of any condition, therapy, or laboratory abnormality or other circumstance that might confound the results of the study, interfere with the participant's participation for the full duration of the study, such that it is not in the best interest of the participant to participate, in the opinion of the treating investigator. * History of hepatitis B with detectable HBV viral load (participants who are HBsAg positive are eligible if they have received HBV anti-viral therapy for at least 4 weeks, and have undetectable HBV viral load prior to randomization) or known active hepatitis C virus (defined as detectable HCV RNA .\[qualitative\]) infection. \* Note: Testing for hepatitis B or C is not required unless clinically indicated or if there is a known history of hepatitis infection * Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial. * Has had an allogenic tissue/solid organ transplant. * Pregnant and chest feeding participants are excluded from this study because targeted chemotherapy and radiation have the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with EV+pembrolizumab, breastfeeding should be discontinued if the mother is treated with these investigational products.