A Study of Chemotherapy and Radiation Therapy Compared to Chemotherapy and Radiation Therapy Plus MEDI4736 (Durvalumab) Immunotherapy for Bladder Cancer Which Has Spread to the Lymph Nodes, INSPIRE Trial

Inclusion Criteria: * STEP 1 (RANDOMIZATION) ELIGIBILITY CRITERIA: * Patient must be \>= 18 years of age. * Patient must have an Eastern Cooperative Oncology Group (ECOG) performance status of 0-2 at the time of step 1 randomization. * Patient must have histologically proven pure or mixed urothelial cancer of the bladder. * NOTE: Small cell carcinoma is excluded, however other variant histologies are permitted provided a component of urothelial carcinoma is present. * Patient must have documented node-positive and non-metastatic disease (any T, any N, M0). Node positivity must have been defined prior to receiving any systemic chemotherapy or induction chemotherapy. Node positivity can fall into either of the following categories and will be defined by imaging and/or biopsy: * A lymph node \>= 1.0 cm in short axis on imaging (i.e., CT or MRI or positron emission tomography \[PET\]/CT). * A lymph node that is \< 1 cm on imaging with either a biopsy confirming involvement with cancer or high suspicion of cancer. * For patients who have received induction chemotherapy (any type of systemic chemotherapy) for node positive bladder cancer prior to enrollment, there must be no signs of disease progression (CR/partial response \[PR\] or stable disease \[SD\]) based on restaging imaging and cystoscopy, which consists of: * CT chest, abdomen, and pelvis obtained after completion of induction chemotherapy and within 8 weeks prior to step 1 randomization. * NOTE: MRI can be used instead of CT per treating physician discretion. * Cystoscopic evaluation and attempt to perform maximal transurethral resection of bladder tumor (TURBT) performed by the participating urologist after completion of induction chemotherapy and within 12 weeks prior to step 1 randomization. If maximal TURBT is not possible for medical reasons, the enrollment must be discussed and approved with the study chair. Documentation of correspondences with the study chair must be kept on file. * Patients who achieve CR upon cystoscopy per urologist with no visible tumor (i.e., no need for additional TURBT), are allowed to proceed in the study as adequate resection with no residual disease in bladder. * For patients who have did not receive induction chemotherapy (any type of systemic chemotherapy) for node positive bladder cancer prior to enrollment, the following must be obtained: * CT chest, abdomen, and pelvis completed within 8 weeks prior to step 1 randomization. * NOTE: MRI can be used instead of CT per treating physician discretion. * Cystoscopic evaluation and attempt to perform maximal TURBT performed by the participating urologist within 12 weeks prior to step 1 randomization. If maximal TURBT is not possible for medical reasons, the enrollment must be discussed and approved with the study chair. Documentation of correspondences with the study chair must be kept on file. * For patients who may need repeat TURBT if their old TURBT has fallen out of window: If urologist determine no visible tumor (i.e., no need for additional resection) upon cystoscopy, they are allowed to proceed in the study as complete resection. * Patient must not have received any previous radiation therapy to the pelvic area. * Patient must agree to undergo CT simulation and treatment planning. If this is the first case registered at the site, then a pre-treatment radiation therapy (RT) review will be required and will take up to 3 business days. The patient cannot start radiation treatment prior to successful completion of this pre-treatment review. Therefore, careful planning is necessary to meet the deadline of starting radiation within 20 business days of step 1 randomization. * Patient must not have presence of concomitant active upper tract tumors or urethra tumors. History of previously adequately treated non-muscle invasive bladder cancer (NMIBC) are eligible. Previously treated urothelial cancer or histological variant at any site outside of the urinary bladder are allowed, provided they have been Ta/T1/carcinoma in situ (CIS) and post treatment follow up imaging and endoscopic evaluation shows no evidence of disease. * Patients with previous exposure to immune checkpoint inhibitor for non-muscle invasive disease are eligible. If given for NMIBC, the last dose must have been completed \> 12 months prior to step 1 randomization. * Patient must not be pregnant or breast-feeding due to the potential harm to an unborn fetus and possible risk for adverse events in nursing infants with the treatment regimens being used. All patients of childbearing potential must have a blood test or urine study within 14 days prior to step 1 randomization to rule out pregnancy. A patient of childbearing potential is defined as any patient, regardless of sexual orientation or whether they have undergone tubal ligation, who meets the following criteria: 1) has achieved menarche at some point, 2) has not undergone a hysterectomy or bilateral oophorectomy; or 3) has not been naturally postmenopausal (amenorrhea following cancer therapy does not rule out childbearing potential) for at least 24 consecutive months (i.e., has had menses at any time in the preceding 24 consecutive months). * Patients must not expect to conceive or father children by using accepted and effected method(s) of contraception or by abstaining from sexual intercourse from the time of step 1 randomization for the duration of their participation in the study and continue for at least 3 months after the last dose of protocol treatment. * Leukocytes \>= 3,000/mcL (obtained \< 14 days prior to step 1 randomization). * Absolute neutrophil count (ANC) \>= 1,500/mcL (obtained \< 14 days prior to step 1 randomization). * Hemoglobin \>= 9 g/dL (obtained \< 14 days prior to step 1 randomization). * Platelets \>= 100,000/mcL (obtained \< 14 days prior to step 1 randomization). * Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) (obtained \< 14 days prior to step 1 randomization). * Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional ULN (obtained \< 14 days prior to step 1 randomization). * Adequate renal function as evidenced by calculated (Cockcroft's formula) creatinine clearance or 24 hours actual creatinine clearance \>= 30mL/min. The creatinine used to calculate the clearance result must have been obtained within 14 days prior to step 1 randomization. Actual body weight, not ideal body weight, must be used in the calculation. * Patients with human immunodeficiency virus (HIV) on effective anti-retroviral therapy with undetectable viral load within 6 months of step 1 randomization are eligible for this trial. * For patients with autoimmune conditions, patient must not have history of prior documented autoimmune disease within 2 years prior to step 1 randomization. * NOTE: Patient with vitiligo, Grave's disease, eczema or psoriasis (not requiring systemic treatment within 2 years prior to step 1 randomization) are not excluded. Patients with history of completely resolved childhood asthma or atopy are not excluded. Patients with asthma not requiring more than 10 mg/d or equivalent of prednisone are not excluded. Patients with well-controlled hypothyroidism on thyroxine replacement will be eligible as well. Patients with known history of hypoadrenalism on maintenance steroids will be eligible. Patients with type I diabetes mellitus will be eligible, provided their disease is well controlled. History of autoimmune related alopecia is also not an exclusion criteria. * Patient with active or prior documented inflammatory bowel disease (e.g., Crohn's disease, ulcerative colitis) are not eligible. * Patient with a history of and/or confirmed pneumonitis are not eligible. * Patient with a history of primary immunodeficiency are not eligible. * Patient with history of allogeneic organ transplant are not eligible. * Patient must not have an active infection, including: * Tuberculosis (based on clinical evaluation that includes clinical history, physical examination, and radiographic findings, and tuberculosis testing in line with local practice). * Hepatitis B (known positive hepatitis B virus \[HBV\] surface antigen \[HBsAg\] result). Past or resolved HBV infection (defined as the presence of hepatitis b core antibody \[anti-HBc\] and absence of HBsAg) are eligible. * Hepatitis C. Patients positive for hepatitis C (HCV) antibody are eligible only if polymerase chain reaction test is negative for HCV ribonucleic acid (RNA). * Patient must not have clinically significant liver disease that precludes patient from treatment regimens prescribed on the study (including, but not limited to, active viral, alcoholic or other autoimmune hepatitis, cirrhosis or inherited liver disease). * Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial. Site is encouraged to discuss with the study chair if needed prior to enrollment. * Patients with known history or current symptoms of cardiac disease, or history of treatment with cardiotoxic agents, should have a clinical risk assessment of cardiac function using the New York Heart Association Functional Classification. To be eligible for this trial, patients should be class IIB or better. * Patient must not have received live attenuated vaccine within 30 days prior to the first dose of durvalumab * NOTE: Patient, if enrolled, must not receive live vaccine whilst receiving durvalumab and up to 30 days after the last dose of durvalumab. * NOTE: Patient is permitted to receive inactivated vaccines and any non-live vaccines including those for the seasonal influenza and coronavirus disease 2019 (COVID-19) (Note: intranasal influenza vaccines, such as Flu-Mist (registered trademark) are live attenuated vaccines and are not allowed). If possible, it is recommended to separate study drug administration from vaccine administration by about a week (primarily in order to minimize an overlap of adverse events). * Patient must not have current or prior use of immunosuppressive medication within 14 days before the first dose of durvalumab. The following are exceptions to this criterion: * Intranasal, inhaled, topical steroids, or local steroid injections (e.g., intra articular injection). * Systemic corticosteroids at physiologic doses not to exceed 10 mg/day of prednisone or its equivalent. * Steroids as premedication for hypersensitivity reactions (e.g., CT scan premedication). * Patient must not have any unresolved toxicity (National Cancer Institute \[NCI\] CTCAE grade \>= 2) from previous anti-cancer therapy with the exception of alopecia, vitiligo, and the laboratory values. * NOTE: Patients with grade \>= 2 neuropathy will be evaluated on a case-by-case basis after consultation with the study chair. * NOTE: Patients with irreversible toxicity not reasonably expected to be exacerbated by treatment with durvalumab may be included only after consultation with the study chair. Documentation of correspondences with the study chair must be kept on file. * STEP 2 (REGISTRACTION: ADJUVANT DURVALUMAB VERSUS \[VS\] OBSERVATION) ELIGIBILITY CRITERIA: * Patient must have evaluation to determine clinical outcome post step 1 treatment (chemoRT+/- durvalumab) with imaging (CT chest, abdomen, and pelvis) (preferably contrast with urogram, if not contraindicated) and cystoscopy with biopsy confirmation to ensure no progression and absence of \>= T2 disease in the bladder. Patient should be registered to step 2 within 28 days from the determination of primary response to step 1 treatment. However, for patients previously on Arm C, an additional 4 week delay to step 2 registration is allowed. * Patient on the chemoRT+ durvalumab (Arm C) must meet the following: * Patient must have achieved either complete clinical response OR have demonstrated clinical benefit prior to continuing onto adjuvant durvalumab. * Patients who are to go on the adjuvant durvalumab (Arm E) must have recovered to at least grade 2 or less immune related adverse event (AE) prior to starting treatment except for immune related alopecia, clinically asymptomatic endocrinopathies. For patients who may have gotten immune related AEs during chemoRT+ durvalumab (Arm C), step 2 registration could be delayed up to additional 4 weeks to ensure recovery to at least grade 2 or lower prior to starting adjuvant therapy. However patients with durvalumab related AEs that require permanent discontinuation of durvalumab will not continue on the adjuvant treatment regardless of the response. * Patient must not have experienced immune related neurological disorder described as Guillain-Barré syndrome, myasthenic syndrome or myasthenia gravis, or meningoencephalitis during chemoRT+ durvalumab treatment. * Patient must not have experienced immune related myocarditis or immune related pericarditis during chemoRT+ durvalumab treatment. * ANC \>= 1,000 mcL (must be obtained \< 28 days prior to step 2 registration). * Hemoglobin \>= 8g/dL (must be obtained \< 28 days prior to step 2 registration). * Platelets \>= 70,000 mcL (must be obtained \< 28 days prior to step 2 registration). * NOTE: If recovery is not achieved, blood counts could be repeated weekly and step 2 registration could be delayed up to additional 4 weeks. * Patient on the chemoRT arm (Arm D) must have achieved either complete clinical response OR have demonstrated clinical benefit prior to be placed on the observation alone arm (Arm F).