Immunotherapy With or Without Radiation Therapy for Metastatic Urothelial Cancer

Inclusion Criteria: * Histologically confirmed metastatic urothelial carcinoma * Patients must be either ineligible for platinum treatment or platinum refractory as defined below: * Platinum-ineligible: If patients meet any one of the following criteria: * Impaired renal function (creatinine clearance \[CrCl\] of \< 30 mL/min) * Eastern Cooperative Oncology Group (ECOG) performance status (PS) of \> 2 * Grade \> 2 peripheral neuropathy * New York Heart Association (NYHA) Heart Failure of \> 3 * Platinum-refractory: If patients meet any one of the following criteria: * Prior platinum-based perioperative chemotherapy within 12 months of relapse * Prior platinum-based chemotherapy for metastatic disease * Patients must have at least one measurable site \>= 1 cm in diameter per RECIST 1.1 and a site targetable for radiotherapy. Measurable site must not overlap with radiated site such that measurable site cannot receive \> 2 Gy per fraction * Patients must have measurable disease, defined as at least one lesion that can be accurately measured in at least one dimension (longest diameter to be recorded for non-nodal lesions and short axis for nodal lesions) * Men and women, ages \>= 18 years of age * ECOG performance status =\< 2 * Leukocytes \>= 2,500/mm\^3 * Absolute neutrophil count \>= 1,500/mm\^3 * Platelets \>= 100,000/mm\^3 * Hemoglobin \>= 8 g/dL * Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) (however, patients with known Gilbert disease who have serum bilirubin level =\< 3 x ULN may be enrolled) * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamic pyruvic transaminase \[SGPT\]) =\< 3 x upper limit of normal (ULN) * AST and/or ALT =\< 5 x ULN for patients with liver involvement * Alkaline phosphatase =\< 2.5 x ULN * =\< 5 x ULN for patients with documented liver involvement or if due to bone metastases primarily in absence of liver disease, no limitation * No prior allogeneic bone marrow transplantation or prior solid organ transplantation * No prior radiotherapy to targetable site or measurable site * No chemotherapy or radiotherapy within 4 weeks prior to entering the study or those who have not recovered from adverse events (other than alopecia) due to agents administered more than 4 weeks earlier. However, the following therapies are allowed: * Hormone-replacement therapy or oral contraceptives * Palliative radiotherapy for bone metastases \> 2 weeks prior to registration * No prior treatment with CD137 agonists or immune checkpoint blockade therapies, including anti-CTLA-4, anti-PD-1, and anti-PD-L1 therapeutic antibodies * No treatment with systemic immunostimulatory agents (including, but not limited to, interferon and interleukin 2 \[IL-2\]) within 4 weeks or 5 half-lives of the drug (whichever is longer) prior to initiation of study treatment * No prior treatment with any other investigational agent within 4 weeks prior to registration * No prior treatment with systemic immunostimulatory agents (including, but not limited to, interferon \[IFN\]-alpha or interleukin \[IL\]-2) within 6 weeks prior to registration * Any prior systemic therapy is permitted except therapy with PD1/PDL1 inhibitor * Patients who have received acute, low dose, systemic immunosuppressant medications (e.g., a one-time dose of dexamethasone for nausea) may be enrolled * The use of inhaled corticosteroids and mineralocorticoids (e.g., fludrocortisone) for patients with orthostatic hypotension or adrenocortical insufficiency is allowed * No active tuberculosis (TB) * No known additional malignancy that is progressing or requires active treatment. Exceptions include basal cell carcinoma of the skin or squamous cell carcinoma of the skin that has undergone potentially curative therapy or in situ cervical cancer * Patients with known primary central nervous system (CNS) malignancy or symptomatic CNS metastases are excluded, with the following exceptions: * Patients with asymptomatic untreated CNS disease may be enrolled, provided all of the following criteria are met: * Evaluable or measurable disease outside the CNS * No metastases to brain stem, midbrain, pons, medulla, cerebellum, or within 10 mm of the optic apparatus (optic nerves and chiasm) * No history of intracranial hemorrhage or spinal cord hemorrhage * No ongoing requirement for dexamethasone for CNS disease; patients on a stable dose of anticonvulsants are permitted * No neurosurgical resection or brain biopsy within 28 days prior to registration * Patients with asymptomatic treated CNS metastases may be enrolled, provided all the criteria listed above are met as well as the following: * Radiographic demonstration of improvement upon the completion of CNS directed therapy and no evidence of interim progression between the completion of CNS directed therapy and the screening radiographic study * No stereotactic radiation or whole-brain radiation within 28 days prior to registration * Screening CNS radiographic study \>= 4 weeks from completion of radiotherapy and \>= 2 weeks from discontinuation of corticosteroids * No active autoimmune disease that has required systemic treatment in the past 2 years (i.e. with use of disease modifying agents, corticosteroids or immunosuppressive drugs). Replacement therapy (e.g. thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment * Patients with a history of autoimmune hypothyroidism on a stable dose of thyroid replacement hormone are eligible * No known history of, or any evidence of active, non-infectious pneumonitis or colitis * No known hypersensitivity to Chinese hamster ovary cell products or other recombinant human antibodies * No history of severe allergic, anaphylactic, or other hypersensitivity reactions to chimeric or humanized antibodies or fusion proteins * No known clinically significant liver disease, including active viral, alcoholic, or other hepatitis; or inherited liver disease causing decompensated cirrhosis * No history of idiopathic pulmonary fibrosis, pneumonitis (including drug induced), organizing pneumonia (i.e., bronchiolitis obliterans, cryptogenic organizing pneumonia, etc.), or evidence of active pneumonitis on screening chest computed tomography (CT) scan. History of radiation pneumonitis in the radiation field (fibrosis) is permitted * No significant cardiovascular disease (such as New York Heart Association class II or greater cardiac disease, myocardial infarction, or cerebrovascular accident) within 3 months prior to initiation of study treatment, unstable arrhythmia, or unstable angina * No other disease, metabolic dysfunction, physical examination finding, or clinical laboratory finding that contraindicates the use of an investigational drug, may affect the interpretation of the results, or may render the patient at high risk from treatment complications * No history of leptomeningeal disease * No uncontrolled tumor-related pain * Patients requiring pain medication must be on a stable regimen at study entry * Symptomatic lesions (e.g., bone metastases or metastases causing nerve impingement) amenable to palliative radiotherapy should be treated prior to enrollment. Patients should be recovered from the effects of radiation. There is no required minimum recovery period * Asymptomatic metastatic lesions that would likely cause functional deficits or intractable pain with further growth (e.g., epidural metastasis that is not currently associated with spinal cord compression) should be considered for loco-regional therapy if appropriate prior to enrollment * No uncontrolled pleural effusion, pericardial effusion, or ascites requiring recurrent drainage procedures (once monthly or more frequently) * Patients with indwelling catheters (e.g., PleurX \[registered trademark\]) are allowed * Patients with controlled type 1 diabetes mellitus on a stable insulin regimen are eligible * Patients with eczema, psoriasis, lichen simplex chronicus of vitiligo with dermatologic manifestations only (e.g., patients with psoriatic arthritis would be excluded) are permitted provided that they meet the following conditions: * Patients with psoriasis must have a baseline ophthalmologic exam to rule out ocular manifestations * Rash must cover less than 10% of body surface area (BSA) * Disease is well controlled at baseline and only requiring low potency topical steroids (e.g., hydrocortisone 2.5%, hydrocortisone butyrate 0.1%, flucinolone 0.01%, desonide 0.05%, aclometasone dipropionate 0.05%) * No acute exacerbations of underlying condition within the last 12 months (not requiring psoralen plus ultraviolet A radiation \[PUVA\], methotrexate, retinoids, biologic agents, oral calcineurin inhibitors; high potency or oral steroids) * No uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * No active infections requiring systemic antibiotics within 2 weeks prior to registration. Patients receiving prophylactic antibiotics (e.g., for prevention of a urinary tract infection or chronic obstructive pulmonary disease) are eligible * No major surgical procedure within 28 days prior to registration or anticipation of need for a major surgical procedure during the course of the study * No administration of a live, attenuated vaccine within 30 days before registration or anticipation that such a live, attenuated vaccine will be required during the study and up to 5 months after the last dose of immunotherapy * Patients who have received live attenuated vaccines within 30 days of the first dose of trial treatment are eligible at the discretion of the investigator. All seasonal influenza vaccines and vaccines intended to prevent severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease 2019 (COVID-19) are allowed * Physicians should consider whether any of the following may render the patient inappropriate for this protocol: * Patients with life expectancy of less than 6 months * Psychiatric illness which would prevent the patient from giving informed consent * Medical conditions such as uncontrolled infection, uncontrolled diabetes mellitus or cardiac disease which, in the opinion of the treating physician, would make this protocol unreasonably hazardous for the patient * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial * For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated * Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load * Patients with a "currently active" second malignancy other than non-melanoma skin cancers or cervical carcinoma in situ. Patients are not considered to have a "currently active" malignancy if they have completed therapy and are free of disease for \>= 3 years * Women and men of reproductive potential should agree to use an appropriate method of birth control throughout their participation in this study and for 4 months (120 days) after the last dose of study agent due to the teratogenic potential of the therapy utilized in this trial. Appropriate methods of birth control include abstinence, oral contraceptives, implantable hormonal contraceptives or double barrier method (diaphragm plus condom). Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately * A female of childbearing potential is a sexually mature female who: 1) has not undergone a hysterectomy or bilateral oophorectomy; or 2) has not been naturally postmenopausal for at least 12 consecutive months (i.e., has had menses at any time in the preceding 12 consecutive months)